2026-07-06 14:33:27 +07:00
# S3-Compatible TeleUploader Implementation Plan
2026-08-02 16:19:30 +07:00
> ⚠️ **LEGACY** — Dokumen historis (2026-07-06). Port & infrastruktur sudah berubah:
> produksi kini berjalan di port `4000` (Nix + systemd + Caddy, domain
> `upload.asepharyana.my.id`), database via PgBouncer pool `100.121.180.82:6432`
> (bukan 5432/localhost). Contoh kode di bawah memakai `localhost:4000` untuk dev.
2026-07-06 14:33:27 +07:00
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Transform TeleUploader into an S3-compatible storage server (Telegram-backed) with a web file manager UI.
**Architecture:** An S3 protocol dispatcher at the catch-all route (`$` ) intercepts SigV4-authenticated requests and routes them to ~20 S3-compatible endpoints. A JSON v1 API layer wraps the same operations for use by a single-page web file manager served at `/` . Storage remains Telegram (single channel); S3 buckets are virtual entities in PostgreSQL.
**Tech Stack:** Bun, TypeScript, PostgreSQL (Drizzle ORM), raw AWS SigV4 (no library), zero external XML deps
## Global Constraints
- All new code follows existing codebase style (Bun, TypeScript, Drizzle ORM)
- Existing routes (`/api/upload` , `/f/:public_id` , `/file/:public_id/info` , `/health` , `/docs` ) must remain unchanged and functional
- No external libraries for S3 protocol, auth, or XML handling
- All S3 error responses must use correct XML format with AWS error codes
- Only one S3 credential pair (`S3_ACCESS_KEY` + `S3_SECRET_KEY` ) in env config
- Hard-code S3 region as `us-east-1`
- Chunked transfer encoding (`aws-chunked` ) not supported — return 501
- Presigned GET URLs supported; presigned PUT is optional for v1
- Tests use Bun's built-in test runner and mock patterns (`mock.module()` )
- Files in `schema.sql` are raw SQL (not Drizzle migrations); new tables go there
- All `/api/v1/*` routes are unprotected (no SigV4); rely on network-level security
- Avoid ORM features that don't work with raw `schema.sql` and `postgres` driver — use raw SQL queries or Drizzle's `sql` template tag for complex queries
---
### File Structure
#### New files to create:
| File | Responsibility |
|------|----------------|
| `src/utils/s3/auth.ts` | AWS SigV4 signature verification, signing key derivation, canonical request construction |
| `src/utils/s3/xml.ts` | S3 XML response builders (all 20+ endpoint templates), XML parser for DeleteObjects body |
| `src/db/buckets.ts` | CRUD for `buckets` table: create, list, findByName, delete |
| `src/db/multipart.ts` | CRUD for `multipart_uploads` and `multipart_parts` tables: create, list parts, complete, abort, insert part |
| `src/db/files-ext.ts` | Extended file queries: findByBucketAndKey, listByPrefix, findActiveByBucket, softDelete, copyObject, listByBucketForUICount |
| `src/routes/s3.ts` | S3 protocol dispatcher and all ~20 S3 operation handlers |
| `src/routes/web-api.ts` | JSON v1 API endpoints for web UI (buckets, objects, upload, download, copy, delete) |
| `src/routes/home.ts` | Serves the file manager HTML at `/` |
| `src/home.html` | Web file manager SPA with embedded CSS/JS |
| `test/s3-auth.test.ts` | Tests for SigV4 signature verification |
| `test/s3-operations.test.ts` | Tests for S3 bucket/object/multipart operations |
| `test/web-api.test.ts` | Tests for JSON v1 API endpoints |
#### Existing files to modify:
| File | Changes |
|------|---------|
| `schema.sql` | Add `buckets` , `multipart_uploads` , `multipart_parts` tables; add columns to `files` |
| `src/env.ts` | Add `S3_ACCESS_KEY` , `S3_SECRET_KEY` , `S3_DEFAULT_REGION` config fields |
| `src/index.ts` | Import and register home, web-api, and S3 catch-all routes |
| `src/db/files.ts` | No changes needed (existing queries remain); new queries go in `files-ext.ts` |
| `.env.example` | Add commented S3 env vars |
| `tsconfig.json` | No changes needed |
---
### Task 1: Environment Config + DB Schema
**Files:**
- Modify: `src/env.ts`
- Modify: `schema.sql`
- Modify: `.env.example`
**Interfaces:**
- Consumes: existing `src/env.ts` pattern
- Produces: `config.s3AccessKey` , `config.s3SecretKey` , `config.s3DefaultRegion` env fields; new DB tables and columns
- [ ] **Step 1: Add S3 env vars to `src/env.ts`**
Add to the `AppConfig` interface:
```typescript
interface AppConfig {
// ... existing fields ...
s3AccessKey : string ;
s3SecretKey : string ;
s3DefaultRegion : string ;
}
```
Add to the config object after the `maxRequestBodyBytes` line:
```typescript
s3AccessKey : process.env.S3_ACCESS_KEY || 'teleuploader-admin' ,
s3SecretKey : process.env.S3_SECRET_KEY || '' ,
s3DefaultRegion : process.env.S3_DEFAULT_REGION || 'us-east-1' ,
```
Add S3 vars to the logger.info config block.
- [ ] **Step 2: Update `schema.sql` — add new tables and columns**
Append after the existing files table:
```sql
-- S3-compatible buckets
CREATE TABLE IF NOT EXISTS buckets (
id UUID PRIMARY KEY DEFAULT gen_random_uuid (),
name VARCHAR ( 63 ) UNIQUE NOT NULL ,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);
-- Extend files table for S3
ALTER TABLE files ADD COLUMN IF NOT EXISTS bucket_id UUID REFERENCES buckets ( id );
ALTER TABLE files ADD COLUMN IF NOT EXISTS s3_key TEXT ;
ALTER TABLE files ADD COLUMN IF NOT EXISTS storage_backend VARCHAR DEFAULT 'telegram' ;
ALTER TABLE files ADD COLUMN IF NOT EXISTS is_deleted BOOLEAN DEFAULT false ;
ALTER TABLE files ADD COLUMN IF NOT EXISTS multipart_upload_id TEXT ;
CREATE UNIQUE INDEX IF NOT EXISTS idx_files_bucket_key ON files ( bucket_id , s3_key ) WHERE is_deleted = false ;
CREATE INDEX IF NOT EXISTS idx_files_bucket_prefix ON files ( bucket_id , s3_key text_pattern_ops );
CREATE INDEX IF NOT EXISTS idx_files_s3_key ON files ( s3_key );
CREATE INDEX IF NOT EXISTS idx_files_bucket_id ON files ( bucket_id );
-- Multipart upload tracking
CREATE TABLE IF NOT EXISTS multipart_uploads (
upload_id VARCHAR PRIMARY KEY ,
bucket_id UUID NOT NULL REFERENCES buckets ( id ),
s3_key TEXT NOT NULL ,
initiated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ,
status VARCHAR DEFAULT 'in_progress' ,
initiated_by TEXT
);
CREATE TABLE IF NOT EXISTS multipart_parts (
id SERIAL PRIMARY KEY ,
upload_id VARCHAR NOT NULL REFERENCES multipart_uploads ( upload_id ) ON DELETE CASCADE ,
part_number INT NOT NULL ,
telegram_file_id VARCHAR NOT NULL ,
telegram_file_unique_id VARCHAR NOT NULL ,
storage_message_id BIGINT NOT NULL ,
size_bytes BIGINT NOT NULL ,
etag VARCHAR NOT NULL ,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ,
UNIQUE ( upload_id , part_number )
);
CREATE INDEX IF NOT EXISTS idx_multipart_parts_upload ON multipart_parts ( upload_id , part_number );
CREATE INDEX IF NOT EXISTS idx_multipart_uploads_status ON multipart_uploads ( status );
```
- [ ] **Step 3: Update `.env.example`**
Append to .env.example:
```env
# S3-compatible API credentials
# S3_ACCESS_KEY=teleuploader-admin
# S3_SECRET_KEY=your-secret-key-here
# S3_DEFAULT_REGION=us-east-1
```
- [ ] **Step 4: Run migration to verify**
```bash
cd /mnt/code/TeleUploader && bun run db:migrate
```
Expected: `Database migration completed` logged. No errors.
- [ ] **Step 5: Commit**
```bash
git add src/env.ts schema.sql .env.example
git commit -m "feat: add S3 env config, bucket and multipart DB schema"
```
---
### Task 2: Database CRUD Layer
**Files:**
- Create: `src/db/buckets.ts`
- Create: `src/db/multipart.ts`
- Create: `src/db/files-ext.ts`
**Interfaces:**
- Consumes: `src/db/index.ts` (existing `db` and `files` ), `config` from `src/env.ts`
- Produces: Bucket CRUD functions, Multipart CRUD functions, Extended file queries by bucket+key
- [ ] **Step 1: Create `src/db/buckets.ts`**
```typescript
import { sql } from 'drizzle-orm' ;
import { db } from './index' ;
import logger from '../utils/logger' ;
export interface Bucket {
id : string ;
name : string ;
createdAt : Date ;
updatedAt : Date ;
}
export const createBucket = async ( name : string ) : Promise < Bucket > => {
const result = await db . execute (
sql `INSERT INTO buckets (name) VALUES ( ${ name } ) RETURNING id, name, created_at, updated_at` ,
);
const row = result . rows [ 0 ] as Record < string , unknown >;
return {
id : row.id as string ,
name : row.name as string ,
createdAt : new Date ( row . created_at as string ),
updatedAt : new Date ( row . updated_at as string ),
};
};
export const findBucketByName = async ( name : string ) : Promise < Bucket | null > => {
const result = await db . execute ( sql `SELECT id, name, created_at, updated_at FROM buckets WHERE name = ${ name } ` );
if ( result . rows . length === 0 ) return null ;
const row = result . rows [ 0 ] as Record < string , unknown >;
return {
id : row.id as string ,
name : row.name as string ,
createdAt : new Date ( row . created_at as string ),
updatedAt : new Date ( row . updated_at as string ),
};
};
export const listBuckets = async () : Promise < Bucket [] > => {
const result = await db . execute ( sql `SELECT id, name, created_at, updated_at FROM buckets ORDER BY name` );
return result . rows . map (( row ) => {
const r = row as Record < string , unknown >;
return {
id : r.id as string ,
name : r.name as string ,
createdAt : new Date ( r . created_at as string ),
updatedAt : new Date ( r . updated_at as string ),
};
});
};
export const deleteBucket = async ( name : string ) : Promise < boolean > => {
const result = await db . execute ( sql `DELETE FROM buckets WHERE name = ${ name } ` );
return ( result as unknown as { rowCount : number }). rowCount > 0 ;
};
export const bucketExists = async ( name : string ) : Promise < boolean > => {
const result = await db . execute ( sql `SELECT 1 FROM buckets WHERE name = ${ name } ` );
return result . rows . length > 0 ;
};
```
- [ ] **Step 2: Create `src/db/multipart.ts`**
```typescript
import { sql } from 'drizzle-orm' ;
import { db } from './index' ;
import logger from '../utils/logger' ;
import { nanoid } from 'nanoid' ;
export interface MultipartUpload {
uploadId : string ;
bucketId : string ;
s3Key : string ;
initiatedAt : Date ;
status : string ;
initiatedBy : string ;
}
export interface MultipartPart {
id : number ;
uploadId : string ;
partNumber : number ;
telegramFileId : string ;
telegramFileUniqueId : string ;
storageMessageId : number ;
sizeBytes : number ;
etag : string ;
createdAt : Date ;
}
export const createMultipartUpload = async (
bucketId : string ,
s3Key : string ,
initiatedBy : string ,
) : Promise < string > => {
const uploadId = nanoid ( 32 );
await db . execute (
sql `INSERT INTO multipart_uploads (upload_id, bucket_id, s3_key, initiated_by) VALUES ( ${ uploadId } , ${ bucketId } , ${ s3Key } , ${ initiatedBy } )` ,
);
return uploadId ;
};
export const findMultipartUpload = async ( uploadId : string ) : Promise < MultipartUpload | null > => {
const result = await db . execute (
sql `SELECT upload_id, bucket_id, s3_key, initiated_at, status FROM multipart_uploads WHERE upload_id = ${ uploadId } AND status = 'in_progress'` ,
);
if ( result . rows . length === 0 ) return null ;
const r = result . rows [ 0 ] as Record < string , unknown >;
return {
uploadId : r.upload_id as string ,
bucketId : r.bucket_id as string ,
s3Key : r.s3_key as string ,
initiatedAt : new Date ( r . initiated_at as string ),
status : r.status as string ,
initiatedBy : '' ,
};
};
export const completeMultipartUpload = async ( uploadId : string ) : Promise < void > => {
await db . execute ( sql `UPDATE multipart_uploads SET status = 'completed' WHERE upload_id = ${ uploadId } ` );
};
export const abortMultipartUpload = async ( uploadId : string ) : Promise < void > => {
await db . execute ( sql `UPDATE multipart_uploads SET status = 'aborted' WHERE upload_id = ${ uploadId } ` );
// Parts are cascade-deleted by FK
};
export const insertMultipartPart = async ( part : Omit < MultipartPart , 'id' | 'createdAt' >) : Promise < void > => {
await db . execute (
sql `INSERT INTO multipart_parts (upload_id, part_number, telegram_file_id, telegram_file_unique_id, storage_message_id, size_bytes, etag)
VALUES ( ${ part . uploadId } , ${ part . partNumber } , ${ part . telegramFileId } , ${ part . telegramFileUniqueId } , ${ part . storageMessageId } , ${ part . sizeBytes } , ${ part . etag } )` ,
);
};
export const listMultipartParts = async ( uploadId : string ) : Promise < MultipartPart [] > => {
const result = await db . execute (
sql `SELECT id, upload_id, part_number, telegram_file_id, telegram_file_unique_id, storage_message_id, size_bytes, etag, created_at
FROM multipart_parts WHERE upload_id = ${ uploadId } ORDER BY part_number` ,
);
return result . rows . map (( row ) => {
const r = row as Record < string , unknown >;
return {
id : r.id as number ,
uploadId : r.upload_id as string ,
partNumber : r.part_number as number ,
telegramFileId : r.telegram_file_id as string ,
telegramFileUniqueId : r.telegram_file_unique_id as string ,
storageMessageId : r.storage_message_id as number ,
sizeBytes : r.size_bytes as number ,
etag : r.etag as string ,
createdAt : new Date ( r . created_at as string ),
};
});
};
```
- [ ] **Step 3: Create `src/db/files-ext.ts`**
```typescript
import { eq , and , isNull , sql } from 'drizzle-orm' ;
import { db , files as fileSchema } from './index' ;
import type { File } from './schema' ;
import logger from '../utils/logger' ;
export interface S3FileRecord extends File {
bucketId : string ;
s3Key : string ;
}
export const findFileByBucketAndKey = async ( bucketId : string , s3Key : string ) : Promise < File | null > => {
const result = await db
. select ()
. from ( fileSchema )
. where ( and ( eq ( fileSchema . bucketId , bucketId ), eq ( fileSchema . s3Key , s3Key ), eq ( fileSchema . isDeleted , false )))
. limit ( 1 );
return result [ 0 ] || null ;
};
export interface ListObjectsRow {
key : string ;
fileName : string ;
sizeBytes : number ;
mimeType : string ;
etag : string ;
lastModified : Date ;
isDeleted : boolean ;
}
export const listObjectsByPrefix = async (
bucketId : string ,
prefix : string ,
delimiter : string | null ,
maxKeys : number ,
startAfter : string | null ,
) : Promise < { objects : S3FileRecord []; prefixes : string [] } > => {
const keys = [];
// Base query: active files in this bucket with key starting with prefix
const conditions = [ sql `bucket_id = ${ bucketId } ::uuid` , sql `is_deleted = false` , sql `s3_key LIKE ${ prefix + '%' } ` ];
const orderClause = sql `ORDER BY s3_key` ;
if ( startAfter ) {
conditions . push ( sql `s3_key > ${ startAfter } ` );
}
const whereClause = conditions . map (( c ) => c ? . text ? . replace ( /\$(\d+)/g , '' ));
const result = await db
. select ()
. from ( fileSchema )
. where ( and (... conditions . map (( c ) => sql ` ${ c } ` )) as unknown as ReturnType < typeof and >)
. orderBy ( fileSchema . s3Key as unknown as 'asc' | 'desc' )
. limit ( maxKeys + 1 );
if ( delimiter === '/' ) {
const prefixSet = new Set < string >();
const objects : S3FileRecord [] = [];
for ( const file of result ) {
const relativeKey = file . s3Key ! . substring ( prefix . length );
const slashIndex = relativeKey . indexOf ( '/' );
if ( slashIndex >= 0 ) {
// It's under a subfolder — extract the folder prefix
const folderPrefix = prefix + relativeKey . substring ( 0 , slashIndex + 1 );
if ( folderPrefix !== prefix ) {
prefixSet . add ( folderPrefix );
}
} else {
// It's a direct child object
objects . push ( file as unknown as S3FileRecord );
}
}
return {
objects : objects.slice ( 0 , maxKeys ),
prefixes : Array.from ( prefixSet ). sort (),
};
}
return {
objects : result as unknown as S3FileRecord [],
prefixes : [],
};
};
export const softDeleteFile = async ( bucketId : string , s3Key : string ) : Promise < boolean > => {
const result = await db
. update ( fileSchema )
. set ({ isDeleted : true })
. where ( and ( eq ( fileSchema . bucketId , bucketId ), eq ( fileSchema . s3Key , s3Key )))
. returning ();
return result . length > 0 ;
};
export const softDeleteFilesBatch = async ( bucketId : string , keys : string []) : Promise < number > => {
let deleted = 0 ;
for ( const key of keys ) {
const ok = await softDeleteFile ( bucketId , key );
if ( ok ) deleted ++ ;
}
return deleted ;
};
export const countBucketObjects = async ( bucketId : string ) : Promise < number > => {
const result = await db
. select ({ count : sql < number > `count(*)` })
. from ( fileSchema )
. where ( and ( eq ( fileSchema . bucketId , bucketId ), eq ( fileSchema . isDeleted , false )));
return Number ( result [ 0 ] ? . count || 0 );
};
export const findOrphanFilesByBucket = async ( bucketId : string ) : Promise < File [] > => {
return await db
. select ()
. from ( fileSchema )
. where ( and ( eq ( fileSchema . bucketId , bucketId ), eq ( fileSchema . isDeleted , true )))
. limit ( 100 );
};
```
- [ ] **Step 4: Commit**
```bash
git add src/db/buckets.ts src/db/multipart.ts src/db/files-ext.ts
git commit -m "feat: add DB CRUD layer for buckets, multipart, and S3 file extensions"
```
---
### Task 3: S3 Auth (SigV4) + XML Utilities
**Files:**
- Create: `src/utils/s3/auth.ts`
- Create: `src/utils/s3/xml.ts`
**Interfaces:**
- Produces: `verifySignature(req, s3AccessKey, s3SecretKey, region, bucket, key) → { isValid, credential }`
- Produces: `XML builder functions` for all S3 responses, `parseDeleteObjectsXml(body) → string[]`
- [ ] **Step 1: Create `src/utils/s3/auth.ts`**
The `crypto` module in Bun uses Web Crypto API. For HMAC-SHA256, use `crypto.subtle` :
```typescript
import { config } from '../../env' ;
export interface SigV4Result {
isValid : boolean ;
credential : {
accessKey : string ;
date : string ;
region : string ;
service : string ;
} | null ;
errorCode? : string ;
}
const SERVICE = 's3' ;
const TERMINATION = 'aws4_request' ;
// Create SHA-256 hash
const sha256 = async ( data : string | BufferSource ) : Promise < string > => {
const encoder = new TextEncoder ();
const dataBuffer = typeof data === 'string' ? encoder . encode ( data ) : data ;
const hashBuffer = await crypto . subtle . digest ( 'SHA-256' , dataBuffer );
const hashArray = Array . from ( new Uint8Array ( hashBuffer ));
return hashArray . map (( b ) => b . toString ( 16 ). padStart ( 2 , '0' )). join ( '' );
};
// HMAC-SHA256
const hmacSha256 = async ( key : BufferSource , message : string ) : Promise < ArrayBuffer > => {
const cryptoKey = await crypto . subtle . importKey (
'raw' ,
key ,
{ name : 'HMAC' , hash : 'SHA-256' },
false ,
[ 'sign' ],
);
const encoder = new TextEncoder ();
return await crypto . subtle . sign ( 'HMAC' , cryptoKey , encoder . encode ( message ));
};
// Derive signing key
const getSigningKey = async ( secretKey : string , dateStamp : string , region : string ) : Promise < ArrayBuffer > => {
const encoder = new TextEncoder ();
let key = await hmacSha256 ( encoder . encode ( `AWS4 ${ secretKey } ` ), dateStamp );
key = await hmacSha256 ( key , region );
key = await hmacSha256 ( key , SERVICE );
return await hmacSha256 ( key , TERMINATION );
};
// Hex-encode HMAC result
const hmacHex = async ( key : BufferSource , message : string ) : Promise < string > => {
const result = await hmacSha256 ( key , message );
const hashArray = Array . from ( new Uint8Array ( result ));
return hashArray . map (( b ) => b . toString ( 16 ). padStart ( 2 , '0' )). join ( '' );
};
// Parse AWS4-HMAC-SHA256 Authorization header
const parseAuthorizationHeader = ( authHeader : string ) => {
// "AWS4-HMAC-SHA256 Credential=AKID/20260706/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=..."
const credentialMatch = authHeader . match ( /Credential=([^,]+)/ );
const signedHeadersMatch = authHeader . match ( /SignedHeaders=([^,]+)/ );
const signatureMatch = authHeader . match ( /Signature=([^,]+)/ );
if ( ! credentialMatch || ! signedHeadersMatch || ! signatureMatch ) return null ;
const credentialParts = credentialMatch [ 1 ]. split ( '/' );
if ( credentialParts . length !== 5 ) return null ;
return {
accessKey : credentialParts [ 0 ],
date : credentialParts [ 1 ],
region : credentialParts [ 2 ],
service : credentialParts [ 3 ],
termination : credentialParts [ 4 ],
signedHeaders : signedHeadersMatch [ 1 ],
signature : signatureMatch [ 1 ],
};
};
// Build canonical request
const buildCanonicalRequest = (
method : string ,
canonicalUri : string ,
canonicalQueryString : string ,
signedHeaders : string ,
headers : Record < string , string >,
hashedPayload : string ,
) : string => {
const canonicalHeaders = signedHeaders
. split ( ';' )
. map (( h ) => {
const value = headers [ h . toLowerCase ()] || '' ;
return ` ${ h . toLowerCase () } : ${ value . trim () } \ n` ;
})
. join ( '' );
return ` ${ method } \ n ${ canonicalUri } \ n ${ canonicalQueryString } \ n ${ canonicalHeaders } \ n ${ signedHeaders } \ n ${ hashedPayload } ` ;
};
// Normalize URI (S3 requires URI-encoded paths but decoded for canonical request)
const normalizeUri = ( uri : string ) : string => {
if ( ! uri || uri === '' ) return '/' ;
return uri ;
};
// Build canonical query string from URLSearchParams
const buildCanonicalQueryString = ( searchParams : URLSearchParams ) : string => {
const params : string [] = [];
// Sort by key, then by value
const keys = Array . from ( searchParams . keys ()). sort ();
for ( const key of keys ) {
const values = searchParams . getAll ( key ). sort ();
for ( const value of values ) {
params . push ( ` ${ encodeURIComponent ( key ) } = ${ encodeURIComponent ( value ) } ` );
}
}
return params . join ( '&' );
};
// Get hashed payload from x-amz-content-sha256 header or body
const getHashedPayload = async ( body : string | null , contentSha256 : string | null ) : Promise < string > => {
if ( contentSha256 ) return contentSha256 ;
if ( ! body || body . length === 0 ) return await sha256 ( '' );
return await sha256 ( body );
};
export const verifySignature = async (
method : string ,
url : string ,
headers : Record < string , string >,
body : string | null ,
s3AccessKey : string ,
s3SecretKey : string ,
region : string ,
) : Promise < SigV4Result > => {
const authHeader = headers [ 'authorization' ];
if ( ! authHeader || ! authHeader . startsWith ( 'AWS4-HMAC-SHA256' )) {
return { isValid : false , credential : null , errorCode : 'AccessDenied' };
}
const parsed = parseAuthorizationHeader ( authHeader );
if ( ! parsed ) {
return { isValid : false , credential : null , errorCode : 'AccessDenied' };
}
// Reject if access key doesn't match
if ( parsed . accessKey !== s3AccessKey ) {
return { isValid : false , credential : null , errorCode : 'SignatureDoesNotMatch' };
}
const parsedUrl = new URL ( url , 'http://localhost' );
const canonicalUri = normalizeUri ( parsedUrl . pathname );
const canonicalQueryString = buildCanonicalQueryString ( parsedUrl . searchParams );
const contentSha256 = headers [ 'x-amz-content-sha256' ] || null ;
const hashedPayload = await getHashedPayload ( body , contentSha256 );
const canonicalRequest = buildCanonicalRequest (
method ,
canonicalUri ,
canonicalQueryString ,
parsed . signedHeaders ,
headers ,
hashedPayload ,
);
const hashedCanonicalRequest = await sha256 ( canonicalRequest );
const amzDate = headers [ 'x-amz-date' ] || '' ;
const dateStamp = parsed . date ; // YYYYMMDD from credential
const credentialScope = ` ${ dateStamp } / ${ parsed . region } / ${ parsed . service } / ${ parsed . termination } ` ;
const stringToSign = `AWS4-HMAC-SHA256 \ n ${ amzDate } \ n ${ credentialScope } \ n ${ hashedCanonicalRequest } ` ;
const signingKey = await getSigningKey ( s3SecretKey , dateStamp , region );
const expectedSignature = await hmacHex ( signingKey , stringToSign );
if ( expectedSignature !== parsed . signature ) {
return { isValid : false , credential : null , errorCode : 'SignatureDoesNotMatch' };
}
return {
isValid : true ,
credential : {
accessKey : parsed.accessKey ,
date : parsed.date ,
region : parsed.region ,
service : parsed.service ,
},
};
};
// Simplified verify for presigned URLs
export const verifyPresignedUrl = async (
url : string ,
s3AccessKey : string ,
s3SecretKey : string ,
region : string ,
) : Promise < SigV4Result > => {
const parsedUrl = new URL ( url );
const queryParams = Object . fromEntries ( parsedUrl . searchParams . entries ());
const algorithm = queryParams [ 'X-Amz-Algorithm' ];
const credential = queryParams [ 'X-Amz-Credential' ];
const signedHeaders = queryParams [ 'X-Amz-SignedHeaders' ];
const signature = queryParams [ 'X-Amz-Signature' ];
const expires = parseInt ( queryParams [ 'X-Amz-Expires' ] || '0' , 10 );
const amzDate = queryParams [ 'X-Amz-Date' ];
if ( ! algorithm || algorithm !== 'AWS4-HMAC-SHA256' || ! credential || ! signature || ! expires || ! amzDate ) {
return { isValid : false , credential : null , errorCode : 'AccessDenied' };
}
// Check expiration
const dateObj = new Date (
parseInt ( amzDate . substring ( 0 , 4 ), 10 ),
parseInt ( amzDate . substring ( 4 , 6 ), 10 ) - 1 ,
parseInt ( amzDate . substring ( 6 , 8 ), 10 ),
parseInt ( amzDate . substring ( 9 , 11 ), 10 ),
parseInt ( amzDate . substring ( 11 , 13 ), 10 ),
parseInt ( amzDate . substring ( 13 , 15 ), 10 ),
);
const expiresMs = expires * 1000 ;
if ( Date . now () > dateObj . getTime () + expiresMs ) {
return { isValid : false , credential : null , errorCode : 'AccessDenied' };
}
const credParts = credential . split ( '/' );
const dateStamp = credParts [ 1 ] || amzDate . substring ( 0 , 8 );
// Build canonical request for presigned URL (no body hash — unsigned-payload)
const canonicalUri = normalizeUri ( parsedUrl . pathname );
// Sort query params (excluding signature)
const sortedParams = new URLSearchParams ();
const paramKeys = Object . keys ( queryParams ). sort ();
for ( const key of paramKeys ) {
if ( key !== 'X-Amz-Signature' ) {
sortedParams . append ( key , queryParams [ key ]);
}
}
const canonicalQueryString = buildCanonicalQueryString ( sortedParams );
const canonicalHeaders = ` ${ signedHeaders . split ( ';' ). map (( h ) => ` ${ h } :host \ n` ). join ( '' ) } ` ;
const signedHeadersStr = signedHeaders ;
const hashedPayload = 'UNSIGNED-PAYLOAD' ;
const canonicalRequest = ` ${ canonicalUri } \ n ${ canonicalQueryString } \ n ${ canonicalHeaders } \ n ${ signedHeadersStr } \ n ${ hashedPayload } ` ;
const hashedCanonicalRequest = await sha256 ( canonicalRequest );
const credentialScope = ` ${ dateStamp } / ${ region } /s3/aws4_request` ;
const stringToSign = `AWS4-HMAC-SHA256 \ n ${ amzDate } \ n ${ credentialScope } \ n ${ hashedCanonicalRequest } ` ;
const signingKey = await getSigningKey ( s3SecretKey , dateStamp , region );
const expectedSignature = await hmacHex ( signingKey , stringToSign );
if ( expectedSignature !== signature ) {
return { isValid : false , credential : null , errorCode : 'SignatureDoesNotMatch' };
}
return { isValid : true , credential : null };
};
export const isS3Request = ( headers : Record < string , string >) : boolean => {
const auth = headers [ 'authorization' ] || '' ;
return auth . startsWith ( 'AWS4-HMAC-SHA256' );
};
```
- [ ] **Step 2: Create `src/utils/s3/xml.ts`**
```typescript
const escapeXml = ( str : string ) : string =>
str
. replace ( /&/g , '&' )
. replace ( /</g , '<' )
. replace ( />/g , '>' )
. replace ( /"/g , '"' )
. replace ( /'/g , ''' );
const isoDate = ( d : Date ) : string => d . toISOString (). replace ( /\.\d{3}Z$/ , 'Z' );
// ─────── Bucket operations ───────
export const listBucketsXml = (
buckets : { name : string ; createdAt : Date }[],
requestId : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Buckets>
${ buckets . map (( b ) => `<Bucket>
<Name> ${ escapeXml ( b . name ) } </Name>
<CreationDate> ${ isoDate ( b . createdAt ) } </CreationDate>
</Bucket>` ). join ( '' ) }
</Buckets>
</ListAllMyBucketsResult>` ;
// ─────── Object listing ───────
export const listBucketResultXml = (
bucketName : string ,
objects : { key : string ; sizeBytes : number ; etag : string ; lastModified : Date ; mimeType : string }[],
prefixes : string [],
isTruncated : boolean ,
marker : string | null ,
maxKeys : number ,
prefix : string ,
delimiter : string | null ,
nextMarker : string | null ,
requestId : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name> ${ escapeXml ( bucketName ) } </Name>
<Prefix> ${ escapeXml ( prefix ) } </Prefix>
<Marker> ${ escapeXml ( marker || '' ) } </Marker>
<MaxKeys> ${ maxKeys } </MaxKeys>
<Delimiter> ${ escapeXml ( delimiter || '' ) } </Delimiter>
<IsTruncated> ${ isTruncated } </IsTruncated>
${ objects . map (( o ) => `<Contents>
<Key> ${ escapeXml ( o . key ) } </Key>
<LastModified> ${ isoDate ( o . lastModified ) } </LastModified>
<ETag>" ${ o . etag } "</ETag>
<Size> ${ o . sizeBytes } </Size>
<StorageClass>STANDARD</StorageClass>
</Contents>` ). join ( '' ) }
${ prefixes . map (( p ) => `<CommonPrefixes>
<Prefix> ${ escapeXml ( p ) } </Prefix>
</CommonPrefixes>` ). join ( '' ) }
${ nextMarker ? `<NextMarker> ${ escapeXml ( nextMarker ) } </NextMarker>` : '' }
</ListBucketResult>` ;
export const listBucketV2ResultXml = (
bucketName : string ,
objects : { key : string ; sizeBytes : number ; etag : string ; lastModified : Date ; mimeType : string }[],
prefixes : string [],
isTruncated : boolean ,
maxKeys : number ,
prefix : string ,
delimiter : string | null ,
continuationToken : string | null ,
nextContinuationToken : string | null ,
keyCount : number ,
requestId : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<ListBucketResultV2 xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Name> ${ escapeXml ( bucketName ) } </Name>
<Prefix> ${ escapeXml ( prefix ) } </Prefix>
<MaxKeys> ${ maxKeys } </MaxKeys>
<KeyCount> ${ keyCount } </KeyCount>
${ delimiter ? `<Delimiter> ${ escapeXml ( delimiter ) } </Delimiter>` : '' }
${ continuationToken ? `<ContinuationToken> ${ escapeXml ( continuationToken ) } </ContinuationToken>` : '' }
<IsTruncated> ${ isTruncated } </IsTruncated>
${ objects . map (( o ) => `<Contents>
<Key> ${ escapeXml ( o . key ) } </Key>
<LastModified> ${ isoDate ( o . lastModified ) } </LastModified>
<ETag>" ${ o . etag } "</ETag>
<Size> ${ o . sizeBytes } </Size>
<StorageClass>STANDARD</StorageClass>
</Contents>` ). join ( '' ) }
${ prefixes . map (( p ) => `<CommonPrefixes>
<Prefix> ${ escapeXml ( p ) } </Prefix>
</CommonPrefixes>` ). join ( '' ) }
${ nextContinuationToken ? `<NextContinuationToken> ${ escapeXml ( nextContinuationToken ) } </NextContinuationToken>` : '' }
</ListBucketResultV2>` ;
// ─────── Multipart ───────
export const initiateMultipartUploadXml = (
bucketName : string ,
key : string ,
uploadId : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket> ${ escapeXml ( bucketName ) } </Bucket>
<Key> ${ escapeXml ( key ) } </Key>
<UploadId> ${ uploadId } </UploadId>
</InitiateMultipartUploadResult>` ;
export const listPartsXml = (
bucketName : string ,
key : string ,
uploadId : string ,
parts : { partNumber : number ; etag : string ; sizeBytes : number ; createdAt : Date }[],
maxParts : number ,
isTruncated : boolean ,
requestId : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<ListPartsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Bucket> ${ escapeXml ( bucketName ) } </Bucket>
<Key> ${ escapeXml ( key ) } </Key>
<UploadId> ${ uploadId } </UploadId>
<MaxParts> ${ maxParts } </MaxParts>
<IsTruncated> ${ isTruncated } </IsTruncated>
${ parts . map (( p ) => `<Part>
<PartNumber> ${ p . partNumber } </PartNumber>
<LastModified> ${ isoDate ( p . createdAt ) } </LastModified>
<ETag>" ${ p . etag } "</ETag>
<Size> ${ p . sizeBytes } </Size>
</Part>` ). join ( '' ) }
</ListPartsResult>` ;
export const completeMultipartUploadXml = (
bucketName : string ,
key : string ,
etag : string ,
location : string ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<CompleteMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<Location> ${ escapeXml ( location ) } </Location>
<Bucket> ${ escapeXml ( bucketName ) } </Bucket>
<Key> ${ escapeXml ( key ) } </Key>
<ETag>" ${ etag } "</ETag>
</CompleteMultipartUploadResult>` ;
// ─────── Delete result ───────
export const deleteResultXml = (
deleted : string [],
errors : { key : string ; code : string ; message : string }[],
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<DeleteResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
${ deleted . map (( key ) => `<Deleted>
<Key> ${ escapeXml ( key ) } </Key>
</Deleted>` ). join ( '' ) }
${ errors . map (( e ) => `<Error>
<Key> ${ escapeXml ( e . key ) } </Key>
<Code> ${ e . code } </Code>
<Message> ${ escapeXml ( e . message ) } </Message>
</Error>` ). join ( '' ) }
</DeleteResult>` ;
// ─────── Copy ───────
export const copyObjectResultXml = (
etag : string ,
lastModified : Date ,
) : string => `<?xml version="1.0" encoding="UTF-8"?>
<CopyObjectResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<ETag>" ${ etag } "</ETag>
<LastModified> ${ isoDate ( lastModified ) } </LastModified>
</CopyObjectResult>` ;
// ─────── Error ───────
export const s3ErrorXml = ( code : string , message : string , resource : string , requestId : string ) : string =>
`<?xml version="1.0" encoding="UTF-8"?>
<Error>
<Code> ${ code } </Code>
<Message> ${ escapeXml ( message ) } </Message>
<Resource> ${ escapeXml ( resource ) } </Resource>
<RequestId> ${ requestId } </RequestId>
</Error>` ;
export const s3ErrorResponse = ( code : string , message : string , resource : string , status : number ) : Response =>
new Response ( s3ErrorXml ( code , message , resource , '' ), {
status ,
headers : { 'content-type' : 'application/xml' },
});
// ─────── DeleteObjects XML parser ───────
export const parseDeleteObjectsBody = ( body : string ) : { keys : string []; quiet : boolean } => {
const keys : string [] = [];
const keyRegex = /<Key>([^<]+)<\/Key>/g ;
let match ;
while (( match = keyRegex . exec ( body )) !== null ) {
keys . push ( match [ 1 ]);
}
const quiet = body . includes ( '<Quiet>true</Quiet>' ) || body . includes ( '<Quiet>true ' );
return { keys , quiet };
};
// ─────── CompleteMultipartUpload XML parser ───────
export interface CompletePart {
partNumber : number ;
etag : string ;
}
export const parseCompleteMultipartBody = ( body : string ) : CompletePart [] => {
const parts : CompletePart [] = [];
const partRegex = /<Part>[\s\S]*?<\/Part>/g ;
const partMatch = body . match ( partRegex ) || [];
for ( const partXml of partMatch ) {
const numMatch = partXml . match ( /<PartNumber>(\d+)<\/PartNumber>/ );
const etagMatch = partXml . match ( /<ETag>"?([^"<\s]+)"?<\/ETag>/ );
if ( numMatch && etagMatch ) {
parts . push ({
partNumber : parseInt ( numMatch [ 1 ], 10 ),
etag : etagMatch [ 1 ]. replace ( /^"/ , '' ). replace ( /"$/ , '' ),
});
}
}
return parts ;
};
```
- [ ] **Step 3: Commit auth + xml utilities**
```bash
mkdir -p src/utils/s3
git add src/utils/s3/auth.ts src/utils/s3/xml.ts
git commit -m "feat: add S3 SigV4 auth verification and XML builders"
```
---
### Task 4: S3 Dispatcher and Bucket Operations
**Files:**
- Create: `src/routes/s3.ts`
**Interfaces:**
- Consumes: `verifySignature` from `auth.ts` , `createBucket` /`findBucketByName` /`listBuckets` /`deleteBucket` from `db/buckets.ts` , XML builders from `xml.ts` , `config` from `env.ts`
- Produces: S3 operation handlers for ListBuckets, CreateBucket, HeadBucket, DeleteBucket
- Exports: `handleS3Request(req) → Response`
- [ ] **Step 1: Create S3 dispatcher and bucket operations skeleton**
The S3 route handler parses method + path + query params, detects the S3 operation, verifies auth, and delegates:
```typescript
import { verifySignature , isS3Request , verifyPresignedUrl } from '../utils/s3/auth' ;
import {
listBucketsXml , s3ErrorXml , s3ErrorResponse , listBucketResultXml , listBucketV2ResultXml ,
initiateMultipartUploadXml , listPartsXml , completeMultipartUploadXml , deleteResultXml ,
copyObjectResultXml , parseDeleteObjectsBody , parseCompleteMultipartBody ,
} from '../utils/s3/xml' ;
import { createBucket , findBucketByName , listBuckets , deleteBucket , bucketExists } from '../db/buckets' ;
import {
createMultipartUpload , findMultipartUpload , completeMultipartUpload , abortMultipartUpload ,
insertMultipartPart , listMultipartParts ,
} from '../db/multipart' ;
import {
findFileByBucketAndKey , listObjectsByPrefix , softDeleteFile , softDeleteFilesBatch , countBucketObjects ,
} from '../db/files-ext' ;
import { findFileByHash } from '../db/files' ;
import { config } from '../env' ;
import { forwardToStorage } from '../utils/telegram' ;
import { getFileInfo } from '../utils/telegram' ;
import { buildUploadResponse , computeHash , ensureExtension , extractMimeType , getErrorMessage , cleanupTempFile } from '../utils/file' ;
import { enqueuePreparedUpload , type PreparedUpload } from '../utils/uploadBatcher' ;
import { nanoid } from 'nanoid' ;
import { createReadStream } from 'node:fs' ;
import { createWriteStream } from 'node:fs' ;
import logger from '../utils/logger' ;
const REGION = config . s3DefaultRegion || 'us-east-1' ;
const REQUEST_ID = () => nanoid ( 16 );
// Extract bucket name from path (path-style: /bucket/key or /bucket)
const parseS3Path = ( pathname : string ) : { bucket : string | null ; key : string | null } => {
const parts = pathname . split ( '/' ). filter ( Boolean );
if ( parts . length === 0 ) return { bucket : null , key : null };
if ( parts . length === 1 ) return { bucket : parts [ 0 ], key : null };
return { bucket : parts [ 0 ], key : parts.slice ( 1 ). join ( '/' ) };
};
// Get headers as record
const headersToRecord = ( req : Request ) : Record < string , string > => {
const record : Record < string , string > = {};
for ( const [ key , value ] of req . headers . entries ()) {
record [ key . toLowerCase ()] = value ;
}
return record ;
};
// Check rate limit for S3
const s3RateLimit = ( req : Request ) : boolean => {
// Reuse existing rate limiter pattern
return true ;
};
export const handleS3Request = async ( req : Request ) : Promise < Response > => {
const method = req . method ;
const url = new URL ( req . url );
const pathname = url . pathname ;
const { bucket , key } = parseS3Path ( pathname );
const headers = headersToRecord ( req );
const searchParams = url . searchParams ;
const reqId = REQUEST_ID ();
// Verify auth
const authResult = await verifySignature ( method , req . url , headers , null , config . s3AccessKey , config . s3SecretKey , REGION );
if ( ! authResult . isValid ) {
return s3ErrorResponse ( authResult . errorCode || 'AccessDenied' , 'Authentication required' , pathname , 403 );
}
try {
// ──── Route by (method, path, query) ────
// Root: ListBuckets
if ( ! bucket ) {
if ( method === 'GET' ) {
return handleListBuckets ( reqId );
}
return s3ErrorResponse ( 'MethodNotAllowed' , 'The specified method is not allowed against this resource.' , '/' , 405 );
}
// Bucket-level operations
if ( ! key ) {
if ( method === 'GET' ) {
// Check for list-type parameter
const listType = searchParams . get ( 'list-type' );
if ( listType === '2' ) {
return handleListObjectsV2 ( bucket , searchParams , reqId );
}
return handleListObjectsV1 ( bucket , searchParams , reqId );
}
if ( method === 'PUT' ) return handleCreateBucket ( bucket , reqId );
if ( method === 'HEAD' ) return handleHeadBucket ( bucket , reqId );
if ( method === 'DELETE' ) return handleDeleteBucket ( bucket , reqId );
if ( method === 'POST' ) {
// Check for ?delete
if ( searchParams . has ( 'delete' )) {
const body = await req . text ();
return handleDeleteObjects ( bucket , body , reqId );
}
// ?tagging
if ( searchParams . has ( 'tagging' )) {
return new Response ( null , { status : 204 });
}
}
return s3ErrorResponse ( 'MethodNotAllowed' , '...' , `/ ${ bucket } ` , 405 );
}
// Object-level operations
// Check for multipart query params
if ( searchParams . has ( 'uploads' ) && method === 'POST' ) {
return handleCreateMultipartUpload ( bucket , key , searchParams , reqId );
}
if ( searchParams . has ( 'uploadId' ) && searchParams . has ( 'partNumber' ) && method === 'PUT' ) {
return handleUploadPart ( bucket , key , searchParams , req , reqId );
}
if ( searchParams . has ( 'uploadId' ) && method === 'POST' ) {
const body = await req . text ();
return handleCompleteMultipartUpload ( bucket , key , searchParams , body , reqId );
}
if ( searchParams . has ( 'uploadId' ) && method === 'DELETE' ) {
return handleAbortMultipartUpload ( bucket , key , searchParams , reqId );
}
if ( searchParams . has ( 'uploadId' ) && method === 'GET' ) {
return handleListParts ( bucket , key , searchParams , reqId );
}
// Standard object operations
if ( method === 'GET' ) return handleGetObject ( bucket , key , searchParams , headers , reqId );
if ( method === 'HEAD' ) return handleHeadObject ( bucket , key , reqId );
if ( method === 'PUT' ) return handlePutObject ( bucket , key , searchParams , headers , req , reqId );
if ( method === 'DELETE' ) return handleDeleteObject ( bucket , key , reqId );
return s3ErrorResponse ( 'MethodNotAllowed' , '...' , `/ ${ bucket } / ${ key } ` , 405 );
} catch ( error : unknown ) {
logger . error ( 'S3 operation error' , { bucket , key , error : getErrorMessage ( error ) });
return s3ErrorResponse ( 'InternalError' , 'We encountered an internal error.' , pathname , 500 );
}
};
```
- [ ] **Step 2: Implement bucket operation handlers**
Add these functions in `src/routes/s3.ts` after the dispatcher:
```typescript
// ─────── Bucket Operations ───────
const handleListBuckets = async ( reqId : string ) : Promise < Response > => {
const buckets = await listBuckets ();
const xml = listBucketsXml ( buckets , reqId );
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
const handleCreateBucket = async ( bucketName : string , reqId : string ) : Promise < Response > => {
// Validate bucket name
if ( ! /^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/ . test ( bucketName )) {
return s3ErrorResponse ( 'InvalidBucketName' , 'The specified bucket is not valid.' , `/ ${ bucketName } ` , 400 );
}
const existing = await findBucketByName ( bucketName );
if ( existing ) {
return s3ErrorResponse ( 'BucketAlreadyExists' , 'The requested bucket name is not available.' , `/ ${ bucketName } ` , 409 );
}
await createBucket ( bucketName );
return new Response ( null , { status : 200 , headers : { 'x-amz-request-id' : reqId } });
};
const handleHeadBucket = async ( bucketName : string , reqId : string ) : Promise < Response > => {
const bucket = await findBucketByName ( bucketName );
if ( ! bucket ) {
return s3ErrorResponse ( 'NoSuchBucket' , 'The specified bucket does not exist.' , `/ ${ bucketName } ` , 404 );
}
return new Response ( null , { status : 200 , headers : { 'x-amz-request-id' : reqId } });
};
const handleDeleteBucket = async ( bucketName : string , reqId : string ) : Promise < Response > => {
const bucket = await findBucketByName ( bucketName );
if ( ! bucket ) {
return s3ErrorResponse ( 'NoSuchBucket' , 'The specified bucket does not exist.' , `/ ${ bucketName } ` , 404 );
}
const objCount = await countBucketObjects ( bucket . id );
if ( objCount > 0 ) {
return s3ErrorResponse ( 'BucketNotEmpty' , 'The bucket you tried to delete is not empty.' , `/ ${ bucketName } ` , 409 );
}
await deleteBucket ( bucketName );
return new Response ( null , { status : 204 , headers : { 'x-amz-request-id' : reqId } });
};
```
- [ ] **Step 3: Implement object operation handlers (stubs with minimal implementation)**
Add these after bucket operations:
```typescript
// ─────── Object Operations ───────
const handleGetObject = async ( bucket : string , key : string , searchParams : URLSearchParams , headers : Record < string , string >, reqId : string ) : Promise < Response > => {
// Check presigned URL
if ( searchParams . has ( 'X-Amz-Signature' )) {
const fullUrl = `http://localhost/ ${ bucket } / ${ key } ? ${ searchParams . toString () } ` ;
const presignedResult = await verifyPresignedUrl ( fullUrl , config . s3AccessKey , config . s3SecretKey , REGION );
if ( ! presignedResult . isValid ) {
return s3ErrorResponse ( 'AccessDenied' , 'Request has expired' , `/ ${ bucket } / ${ key } ` , 403 );
}
}
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
const file = await findFileByBucketAndKey ( bucketRecord . id , key );
if ( ! file ) return s3ErrorResponse ( 'NoSuchKey' , 'The specified key does not exist.' , `/ ${ bucket } / ${ key } ` , 404 );
// For multipart objects, stream parts sequentially
if ( file . multipartUploadId ) {
return handleGetMultipartObject ( file , bucket , key , reqId );
}
// Standard GetObject: redirect to Telegram CDN
const fileInfo = await getFileInfo ( file . telegramFileId );
const redirectUrl = `https://api.telegram.org/file/bot ${ fileInfo . bot_token } / ${ fileInfo . file_path } ` ;
return new Response ( null , {
status : 302 ,
headers : {
Location : redirectUrl ,
'x-amz-request-id' : reqId ,
},
});
};
const handleGetMultipartObject = async ( file : Record < string , unknown >, bucket : string , key : string , reqId : string ) : Promise < Response > => {
const uploadId = file . multipartUploadId as string ;
const parts = await listMultipartParts ( uploadId );
if ( parts . length === 0 ) {
return s3ErrorResponse ( 'InternalError' , 'Multipart object has no parts.' , `/ ${ bucket } / ${ key } ` , 500 );
}
// For multipart: redirect to the first part's Telegram URL (simplest approach)
const fileInfo = await getFileInfo ( parts [ 0 ]. telegramFileId );
const redirectUrl = `https://api.telegram.org/file/bot ${ fileInfo . bot_token } / ${ fileInfo . file_path } ` ;
return new Response ( null , {
status : 302 ,
headers : {
Location : redirectUrl ,
'x-amz-request-id' : reqId ,
},
});
};
const handleHeadObject = async ( bucket : string , key : string , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
const file = await findFileByBucketAndKey ( bucketRecord . id , key );
if ( ! file ) return s3ErrorResponse ( 'NoSuchKey' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
return new Response ( null , {
status : 200 ,
headers : {
'content-type' : file . mimeType ,
'content-length' : String ( file . sizeBytes ),
'etag' : `" ${ file . fileHash || nanoid ( 16 ) } "` ,
'last-modified' : file . createdAt instanceof Date ? file . createdAt . toUTCString () : new Date (). toUTCString (),
'x-amz-request-id' : reqId ,
},
});
};
const handlePutObject = async ( bucket : string , key : string , searchParams : URLSearchParams , headers : Record < string , string >, req : Request , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
// Check for ?tagging
if ( searchParams . has ( 'tagging' )) {
return new Response ( null , { status : 204 });
}
// Check if it's a CopyObject
const copySource = headers [ 'x-amz-copy-source' ];
if ( copySource ) {
return handleCopyObject ( bucket , key , copySource , bucketRecord . id , reqId );
}
// Standard PutObject
const formData = await req . formData ();
const fileField = formData . get ( 'file' );
const file = fileField instanceof File ? fileField : null ;
if ( ! file ) {
// Direct body upload (stream)
const body = await req . arrayBuffer ();
const fileBuffer = Buffer . from ( body );
const hash = computeHash ( fileBuffer );
const existing = await findFileByBucketAndKey ( bucketRecord . id , key );
if ( existing ) return new Response ( null , { status : 200 , headers : { 'etag' : `" ${ hash } "` , 'x-amz-request-id' : reqId } });
return await storeFileToTelegram ( fileBuffer , hash , key , bucketRecord , reqId );
}
const buffer = Buffer . from ( await file . arrayBuffer ());
const hash = computeHash ( buffer );
const existing = await findFileByBucketAndKey ( bucketRecord . id , key );
if ( existing ) {
// Update existing
return new Response ( null , { status : 200 , headers : { 'etag' : `" ${ hash } "` , 'x-amz-request-id' : reqId } });
}
return await storeFileToTelegram ( buffer , hash , key , bucketRecord , reqId );
};
const storeFileToTelegram = async ( buffer : Buffer , hash : string , key : string , bucketRecord : { id : string ; name : string }, reqId : string ) : Promise < Response > => {
const tempPath = `/tmp/teleuploader-s3- ${ nanoid () } ` ;
await Bun . write ( tempPath , buffer );
const signatureBuffer = buffer . subarray ( 0 , 16 );
const fileName = key . split ( '/' ). pop () || 'file' ;
const { fileName : finalFileName , mimeType } = ensureExtension ( fileName , signatureBuffer , 'application/octet-stream' );
const fileType = 'document' ;
const forwardResult = await forwardToStorage (
createReadStream ( tempPath ),
`s3- ${ bucketRecord . name } - ${ key . replace ( /\//g , '_' ) } ` ,
fileType ,
);
const publicId = nanoid ();
const { db , files : fileSchema } = await import ( '../db/index' );
await db . insert ( fileSchema ). values ({
publicId ,
telegramFileId : forwardResult.telegramFileId ,
telegramFileUniqueId : forwardResult.telegramFileUniqueId ,
storageChatId : config.storageChatId ,
storageMessageId : forwardResult.storageMessageId ,
fileName : finalFileName ,
mimeType ,
sizeBytes : buffer.byteLength ,
fileType ,
uploaderId : 0 ,
fileHash : hash ,
bucketId : bucketRecord.id ,
s3Key : key ,
storageBackend : 'telegram' ,
isDeleted : false ,
createdAt : new Date (),
updatedAt : new Date (),
});
await cleanupTempFile ( tempPath );
return new Response ( null , {
status : 200 ,
headers : { 'etag' : `" ${ hash } "` , 'x-amz-request-id' : reqId },
});
};
const handleCopyObject = async ( destBucket : string , destKey : string , copySource : string , destBucketId : string , reqId : string ) : Promise < Response > => {
// Copy source format: /bucket/key or bucket/key
const sourcePath = copySource . startsWith ( '/' ) ? copySource . slice ( 1 ) : copySource ;
const parts = sourcePath . split ( '/' );
const sourceBucket = parts [ 0 ];
const sourceKey = parts . slice ( 1 ). join ( '/' );
const sourceBucketRecord = await findBucketByName ( sourceBucket );
if ( ! sourceBucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , copySource , 404 );
const sourceFile = await findFileByBucketAndKey ( sourceBucketRecord . id , sourceKey );
if ( ! sourceFile ) return s3ErrorResponse ( 'NoSuchKey' , '...' , copySource , 404 );
// Create new file record pointing to same telegram file
const publicId = nanoid ();
const { db , files : fileSchema } = await import ( '../db/index' );
await db . insert ( fileSchema ). values ({
publicId ,
telegramFileId : sourceFile.telegramFileId ,
telegramFileUniqueId : sourceFile.telegramFileUniqueId ,
storageChatId : sourceFile.storageChatId ,
storageMessageId : sourceFile.storageMessageId ,
fileName : sourceFile.fileName ,
mimeType : sourceFile.mimeType ,
sizeBytes : sourceFile.sizeBytes ,
fileType : sourceFile.fileType ,
uploaderId : 0 ,
fileHash : sourceFile.fileHash ,
bucketId : destBucketId ,
s3Key : destKey ,
storageBackend : 'telegram' ,
isDeleted : false ,
createdAt : new Date (),
updatedAt : new Date (),
});
const xml = copyObjectResultXml ( sourceFile . fileHash || nanoid ( 16 ), new Date ());
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
const handleDeleteObject = async ( bucket : string , key : string , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
await softDeleteFile ( bucketRecord . id , key );
return new Response ( null , { status : 204 , headers : { 'x-amz-request-id' : reqId } });
};
const handleDeleteObjects = async ( bucket : string , body : string , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } ` , 404 );
const { keys } = parseDeleteObjectsBody ( body );
const deleted = await softDeleteFilesBatch ( bucketRecord . id , keys );
const xml = deleteResultXml ( keys . slice ( 0 , deleted ), []);
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
```
- [ ] **Step 4: Implement object listing handlers**
```typescript
// ─────── Object Listing ───────
const handleListObjectsV1 = async ( bucket : string , searchParams : URLSearchParams , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } ` , 404 );
const prefix = searchParams . get ( 'prefix' ) || '' ;
const delimiter = searchParams . get ( 'delimiter' ) || null ;
const maxKeys = parseInt ( searchParams . get ( 'max-keys' ) || '1000' , 10 );
const marker = searchParams . get ( 'marker' ) || null ;
const { objects , prefixes : commonPrefixes } = await listObjectsByPrefix (
bucketRecord . id , prefix , delimiter , maxKeys , marker ,
);
const isTruncated = objects . length > maxKeys ;
const displayObjects = objects . slice ( 0 , maxKeys );
const nextMarker = isTruncated ? displayObjects [ displayObjects . length - 1 ] ? . s3Key || null : null ;
const xml = listBucketResultXml (
bucket ,
displayObjects . map (( o ) => ({
key : o.s3Key ! ,
sizeBytes : o.sizeBytes ,
etag : o.fileHash || nanoid ( 16 ),
lastModified : o.createdAt instanceof Date ? o.createdAt : new Date (),
mimeType : o.mimeType ,
})),
commonPrefixes ,
isTruncated ,
marker ,
maxKeys ,
prefix ,
delimiter ,
nextMarker ,
reqId ,
);
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
const handleListObjectsV2 = async ( bucket : string , searchParams : URLSearchParams , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } ` , 404 );
const prefix = searchParams . get ( 'prefix' ) || '' ;
const delimiter = searchParams . get ( 'delimiter' ) || null ;
const maxKeys = parseInt ( searchParams . get ( 'max-keys' ) || '1000' , 10 );
const continuationToken = searchParams . get ( 'continuation-token' ) || null ;
const startAfter = searchParams . get ( 'start-after' ) || null ;
const { objects , prefixes : commonPrefixes } = await listObjectsByPrefix (
bucketRecord . id , prefix , delimiter , maxKeys , continuationToken || startAfter ,
);
const isTruncated = objects . length > maxKeys ;
const displayObjects = objects . slice ( 0 , maxKeys );
const nextContinuationToken = isTruncated ? displayObjects [ displayObjects . length - 1 ] ? . s3Key || null : null ;
const xml = listBucketV2ResultXml (
bucket ,
displayObjects . map (( o ) => ({
key : o.s3Key ! ,
sizeBytes : o.sizeBytes ,
etag : o.fileHash || nanoid ( 16 ),
lastModified : o.createdAt instanceof Date ? o.createdAt : new Date (),
mimeType : o.mimeType ,
})),
commonPrefixes ,
isTruncated ,
maxKeys ,
prefix ,
delimiter ,
continuationToken ,
nextContinuationToken ,
displayObjects . length ,
reqId ,
);
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
```
- [ ] **Step 5: Implement multipart upload handlers**
```typescript
// ─────── Multipart Upload ───────
const handleCreateMultipartUpload = async ( bucket : string , key : string , searchParams : URLSearchParams , reqId : string ) : Promise < Response > => {
const bucketRecord = await findBucketByName ( bucket );
if ( ! bucketRecord ) return s3ErrorResponse ( 'NoSuchBucket' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
const uploadId = await createMultipartUpload ( bucketRecord . id , key , 's3' );
const xml = initiateMultipartUploadXml ( bucket , key , uploadId );
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
const handleUploadPart = async ( bucket : string , key : string , searchParams : URLSearchParams , req : Request , reqId : string ) : Promise < Response > => {
const uploadId = searchParams . get ( 'uploadId' ) ! ;
const partNumber = parseInt ( searchParams . get ( 'partNumber' ) ! , 10 );
const multipart = await findMultipartUpload ( uploadId );
if ( ! multipart || multipart . s3Key !== key ) {
return s3ErrorResponse ( 'NoSuchUpload' , 'The specified upload does not exist.' , `/ ${ bucket } / ${ key } ` , 404 );
}
const body = await req . arrayBuffer ();
const buffer = Buffer . from ( body );
// Forward part to Telegram as a separate document
const tempPath = `/tmp/teleuploader-mp- ${ nanoid () } ` ;
await Bun . write ( tempPath , buffer );
const forwardResult = await forwardToStorage (
createReadStream ( tempPath ),
`mp- ${ uploadId } -part- ${ partNumber } ` ,
'document' ,
);
await cleanupTempFile ( tempPath );
// Store part metadata
const etag = computeHash ( buffer );
await insertMultipartPart ({
uploadId ,
partNumber ,
telegramFileId : forwardResult.telegramFileId ,
telegramFileUniqueId : forwardResult.telegramFileUniqueId ,
storageMessageId : forwardResult.storageMessageId ,
sizeBytes : buffer.byteLength ,
etag ,
});
return new Response ( null , {
status : 200 ,
headers : { 'etag' : `" ${ etag } "` , 'x-amz-request-id' : reqId },
});
};
const handleCompleteMultipartUpload = async ( bucket : string , key : string , searchParams : URLSearchParams , body : string , reqId : string ) : Promise < Response > => {
const uploadId = searchParams . get ( 'uploadId' ) ! ;
const multipart = await findMultipartUpload ( uploadId );
if ( ! multipart ) {
return s3ErrorResponse ( 'NoSuchUpload' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
}
const parts = parseCompleteMultipartBody ( body );
const storedParts = await listMultipartParts ( uploadId );
// Verify part order matches
if ( parts . length !== storedParts . length ) {
return s3ErrorResponse ( 'InvalidPart' , 'One or more specified parts could not be found.' , `/ ${ bucket } / ${ key } ` , 400 );
}
// Calculate total size
const totalSize = storedParts . reduce (( sum , p ) => sum + p . sizeBytes , 0 );
// Create the file record
const publicId = nanoid ();
const { db , files : fileSchema } = await import ( '../db/index' );
await db . insert ( fileSchema ). values ({
publicId ,
telegramFileId : storedParts [ 0 ]. telegramFileId ,
telegramFileUniqueId : storedParts [ 0 ]. telegramFileUniqueId ,
storageChatId : config.storageChatId ,
storageMessageId : storedParts [ 0 ]. storageMessageId ,
fileName : key.split ( '/' ). pop () || 'file' ,
mimeType : 'application/octet-stream' ,
sizeBytes : totalSize ,
fileType : 'document' ,
uploaderId : 0 ,
bucketId : multipart.bucketId ,
s3Key : key ,
storageBackend : 'telegram' ,
isDeleted : false ,
multipartUploadId : uploadId ,
createdAt : new Date (),
updatedAt : new Date (),
});
await completeMultipartUpload ( uploadId );
const location = ` ${ config . baseUrl } / ${ bucket } / ${ key } ` ;
const combinedEtag = storedParts . map (( p ) => p . etag ). join ( '-' );
const xml = completeMultipartUploadXml ( bucket , key , combinedEtag , location );
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
const handleAbortMultipartUpload = async ( bucket : string , key : string , searchParams : URLSearchParams , reqId : string ) : Promise < Response > => {
const uploadId = searchParams . get ( 'uploadId' ) ! ;
const multipart = await findMultipartUpload ( uploadId );
if ( ! multipart ) {
return s3ErrorResponse ( 'NoSuchUpload' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
}
await abortMultipartUpload ( uploadId );
return new Response ( null , { status : 204 , headers : { 'x-amz-request-id' : reqId } });
};
const handleListParts = async ( bucket : string , key : string , searchParams : URLSearchParams , reqId : string ) : Promise < Response > => {
const uploadId = searchParams . get ( 'uploadId' ) ! ;
const multipart = await findMultipartUpload ( uploadId );
if ( ! multipart ) {
return s3ErrorResponse ( 'NoSuchUpload' , '...' , `/ ${ bucket } / ${ key } ` , 404 );
}
const parts = await listMultipartParts ( uploadId );
const maxParts = parseInt ( searchParams . get ( 'max-parts' ) || '1000' , 10 );
const xml = listPartsXml (
bucket ,
key ,
uploadId ,
parts . map (( p ) => ({
partNumber : p.partNumber ,
etag : p.etag ,
sizeBytes : p.sizeBytes ,
createdAt : p.createdAt ,
})),
maxParts ,
false ,
reqId ,
);
return new Response ( xml , {
status : 200 ,
headers : { 'content-type' : 'application/xml' , 'x-amz-request-id' : reqId },
});
};
```
- [ ] **Step 6: Commit S3 route handlers**
```bash
git add src/routes/s3.ts
git commit -m "feat: implement S3 protocol dispatcher with bucket, object, listing, and multipart operations"
```
---
### Task 5: JSON Web API (v1)
**Files:**
- Create: `src/routes/web-api.ts`
**Interfaces:**
- Consumes: DB layer (buckets, files-ext, multipart), `forwardToStorage` from telegram utils
- Produces: JSON endpoints at `/api/v1/*` for the web UI
- [ ] **Step 1: Create `src/routes/web-api.ts`**
```typescript
import { createBucket , findBucketByName , listBuckets , deleteBucket } from '../db/buckets' ;
import {
findFileByBucketAndKey , listObjectsByPrefix , softDeleteFile , softDeleteFilesBatch , countBucketObjects ,
} from '../db/files-ext' ;
import { createReadStream } from 'node:fs' ;
import { config } from '../env' ;
import { forwardToStorage , getFileInfo } from '../utils/telegram' ;
import { computeHash , ensureExtension , getErrorMessage , cleanupTempFile , buildUploadResponse , formatCreatedAt } from '../utils/file' ;
import { enqueuePreparedUpload } from '../utils/uploadBatcher' ;
import { nanoid } from 'nanoid' ;
import logger from '../utils/logger' ;
type RouteParams = { bucket? : string ; key? : string };
const json = ( data : unknown , status = 200 ) =>
Response . json ( data , { status });
const jsonError = ( error : string , status : number ) =>
Response . json ({ error }, { status });
// ─────── Bucket endpoints ───────
export const handleListBucketsV1 = async () : Promise < Response > => {
const buckets = await listBuckets ();
const result = await Promise . all (
buckets . map ( async ( b ) => ({
id : b.id ,
name : b.name ,
createdAt : b.createdAt.toISOString (),
objectCount : await countBucketObjects ( b . id ),
})),
);
return json ({ buckets : result });
};
export const handleCreateBucketV1 = async ( req : Request ) : Promise < Response > => {
const body = ( await req . json ()) as { name? : string };
if ( ! body . name || ! /^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/ . test ( body . name )) {
return jsonError ( 'Invalid bucket name. Use lowercase, 3-63 chars, no underscore' , 400 );
}
const existing = await findBucketByName ( body . name );
if ( existing ) return jsonError ( 'Bucket already exists' , 409 );
const bucket = await createBucket ( body . name );
return json ({ id : bucket.id , name : bucket.name }, 201 );
};
export const handleDeleteBucketV1 = async ( _req : Request , params : RouteParams ) : Promise < Response > => {
const bucket = await findBucketByName ( params . bucket ! );
if ( ! bucket ) return jsonError ( 'Bucket not found' , 404 );
const count = await countBucketObjects ( bucket . id );
if ( count > 0 ) return jsonError ( 'Bucket is not empty' , 409 );
await deleteBucket ( params . bucket ! );
return json ({ success : true });
};
// ─────── Object endpoints ───────
export const handleListObjectsV1 = async ( req : Request , params : RouteParams ) : Promise < Response > => {
const bucket = await findBucketByName ( params . bucket ! );
if ( ! bucket ) return jsonError ( 'Bucket not found' , 404 );
const url = new URL ( req . url );
const prefix = url . searchParams . get ( 'prefix' ) || '' ;
const delimiter = url . searchParams . get ( 'delimiter' ) || '/' ;
const maxKeys = parseInt ( url . searchParams . get ( 'max-keys' ) || '1000' , 10 );
const continuationToken = url . searchParams . get ( 'continuation-token' ) || null ;
const { objects , prefixes } = await listObjectsByPrefix ( bucket . id , prefix , delimiter , maxKeys , continuationToken );
const isTruncated = objects . length > maxKeys ;
const displayObjects = objects . slice ( 0 , maxKeys );
return json ({
objects : displayObjects.map (( o ) => ({
key : o.s3Key ,
fileName : o.fileName ,
mimeType : o.mimeType ,
sizeBytes : o.sizeBytes ,
fileType : o.fileType ,
etag : o.fileHash ,
lastModified : o.createdAt instanceof Date ? o . createdAt . toISOString () : new Date ( o . createdAt ). toISOString (),
downloadUrl : ` ${ config . baseUrl } /f/ ${ o . publicId } ` ,
})),
prefixes ,
isTruncated ,
nextContinuationToken : isTruncated ? displayObjects [ displayObjects . length - 1 ] ? . s3Key : null ,
});
};
export const handleUploadObjectV1 = async ( req : Request , params : RouteParams ) : Promise < Response > => {
const bucket = await findBucketByName ( params . bucket ! );
if ( ! bucket ) return jsonError ( 'Bucket not found' , 404 );
const formData = await req . formData ();
const file = formData . get ( 'file' );
if ( ! file || ! ( file instanceof File )) {
return jsonError ( 'No file provided' , 400 );
}
const key = ( formData . get ( 'key' ) as string ) || file . name ;
const buffer = Buffer . from ( await file . arrayBuffer ());
const hash = computeHash ( buffer );
// Upload to Telegram
const tempPath = `/tmp/teleuploader-web- ${ nanoid () } ` ;
await Bun . write ( tempPath , buffer );
const signatureBuffer = buffer . subarray ( 0 , 16 );
const { fileName : finalFileName , mimeType } = ensureExtension ( key . split ( '/' ). pop () || 'file' , signatureBuffer , file . type || 'application/octet-stream' );
const fileType = 'document' ;
const forwardResult = await forwardToStorage (
createReadStream ( tempPath ),
`s3- ${ bucket . name } - ${ key . replace ( /\//g , '_' ) } ` ,
fileType ,
);
const publicId = nanoid ();
const { db , files : fileSchema } = await import ( '../db/index' );
await db . insert ( fileSchema ). values ({
publicId ,
telegramFileId : forwardResult.telegramFileId ,
telegramFileUniqueId : forwardResult.telegramFileUniqueId ,
storageChatId : config.storageChatId ,
storageMessageId : forwardResult.storageMessageId ,
fileName : finalFileName ,
mimeType ,
sizeBytes : buffer.byteLength ,
fileType ,
uploaderId : 0 ,
fileHash : hash ,
bucketId : bucket.id ,
s3Key : key ,
storageBackend : 'telegram' ,
isDeleted : false ,
createdAt : new Date (),
updatedAt : new Date (),
});
await cleanupTempFile ( tempPath );
return json ({
key ,
size : buffer.byteLength ,
etag : hash ,
downloadUrl : ` ${ config . baseUrl } /f/ ${ publicId } ` ,
}, 201 );
};
export const handleDeleteObjectV1 = async ( _req : Request , params : RouteParams ) : Promise < Response > => {
const bucket = await findBucketByName ( params . bucket ! );
if ( ! bucket ) return jsonError ( 'Bucket not found' , 404 );
await softDeleteFile ( bucket . id , params . key ! );
return json ({ success : true });
};
export const handleDownloadObjectV1 = async ( _req : Request , params : RouteParams ) : Promise < Response > => {
const bucket = await findBucketByName ( params . bucket ! );
if ( ! bucket ) return jsonError ( 'Bucket not found' , 404 );
const file = await findFileByBucketAndKey ( bucket . id , params . key ! );
if ( ! file ) return jsonError ( 'Object not found' , 404 );
const fileInfo = await getFileInfo ( file . telegramFileId );
const redirectUrl = `https://api.telegram.org/file/bot ${ fileInfo . bot_token } / ${ fileInfo . file_path } ` ;
return new Response ( null , {
status : 302 ,
headers : { Location : redirectUrl },
});
};
export const handleCopyObjectV1 = async ( req : Request , params : RouteParams ) : Promise < Response > => {
const body = ( await req . json ()) as { sourceKey? : string ; destBucket? : string ; destKey? : string };
if ( ! body . sourceKey || ! body . destKey ) {
return jsonError ( 'sourceKey and destKey are required' , 400 );
}
const destBucketName = body . destBucket || params . bucket ! ;
const sourceBucket = await findBucketByName ( params . bucket ! );
const destBucket = await findBucketByName ( destBucketName );
if ( ! sourceBucket || ! destBucket ) return jsonError ( 'Bucket not found' , 404 );
const sourceFile = await findFileByBucketAndKey ( sourceBucket . id , body . sourceKey );
if ( ! sourceFile ) return jsonError ( 'Source object not found' , 404 );
const publicId = nanoid ();
const { db , files : fileSchema } = await import ( '../db/index' );
await db . insert ( fileSchema ). values ({
publicId ,
telegramFileId : sourceFile.telegramFileId ,
telegramFileUniqueId : sourceFile.telegramFileUniqueId ,
storageChatId : sourceFile.storageChatId ,
storageMessageId : sourceFile.storageMessageId ,
fileName : sourceFile.fileName ,
mimeType : sourceFile.mimeType ,
sizeBytes : sourceFile.sizeBytes ,
fileType : sourceFile.fileType ,
uploaderId : 0 ,
fileHash : sourceFile.fileHash ,
bucketId : destBucket.id ,
s3Key : body.destKey ,
storageBackend : 'telegram' ,
isDeleted : false ,
createdAt : new Date (),
updatedAt : new Date (),
});
return json ({ sourceKey : body.sourceKey , destKey : body.destKey , destBucket : destBucketName });
};
// ─────── Router ───────
export const handleWebApiV1 = async ( req : Request ) : Promise < Response > => {
const url = new URL ( req . url );
const pathname = url . pathname . replace ( /^\/api\/v1/ , '' );
const parts = pathname . split ( '/' ). filter ( Boolean );
const method = req . method ;
try {
// GET /api/v1/buckets
if ( parts . length === 1 && parts [ 0 ] === 'buckets' && method === 'GET' ) {
return await handleListBucketsV1 ();
}
// POST /api/v1/buckets
if ( parts . length === 1 && parts [ 0 ] === 'buckets' && method === 'POST' ) {
return await handleCreateBucketV1 ( req );
}
// DELETE /api/v1/buckets/{name}
if ( parts . length === 2 && parts [ 0 ] === 'buckets' && method === 'DELETE' ) {
return await handleDeleteBucketV1 ( req , { bucket : parts [ 1 ] });
}
// GET /api/v1/buckets/{name}/objects
if ( parts . length === 3 && parts [ 0 ] === 'buckets' && parts [ 2 ] === 'objects' && method === 'GET' ) {
return await handleListObjectsV1 ( req , { bucket : parts [ 1 ] });
}
// POST /api/v1/buckets/{name}/upload
if ( parts . length === 3 && parts [ 0 ] === 'buckets' && parts [ 2 ] === 'upload' && method === 'POST' ) {
return await handleUploadObjectV1 ( req , { bucket : parts [ 1 ] });
}
// POST /api/v1/buckets/{name}/copy
if ( parts . length === 3 && parts [ 0 ] === 'buckets' && parts [ 2 ] === 'copy' && method === 'POST' ) {
return await handleCopyObjectV1 ( req , { bucket : parts [ 1 ] });
}
// DELETE /api/v1/buckets/{name}/{key+}
if ( parts . length >= 3 && parts [ 0 ] === 'buckets' && method === 'DELETE' ) {
const bucket = parts [ 1 ];
const key = parts . slice ( 2 ). join ( '/' );
return await handleDeleteObjectV1 ( req , { bucket , key });
}
// GET /api/v1/buckets/{name}/download/{key+}
if ( parts . length >= 4 && parts [ 0 ] === 'buckets' && parts [ 2 ] === 'download' && method === 'GET' ) {
const bucket = parts [ 1 ];
const key = parts . slice ( 3 ). join ( '/' );
return await handleDownloadObjectV1 ( req , { bucket , key });
}
return jsonError ( 'Not found' , 404 );
} catch ( error : unknown ) {
logger . error ( 'Web API error' , { path : pathname , error : getErrorMessage ( error ) });
return jsonError ( 'Internal server error' , 500 );
}
};
```
- [ ] **Step 2: Commit web API**
```bash
git add src/routes/web-api.ts
git commit -m "feat: add JSON v1 web API for S3 management UI"
```
---
### Task 6: Web File Manager UI
**Files:**
- Create: `src/home.html`
- Create: `src/routes/home.ts`
- Modify: `src/index.ts` (register new routes)
- [ ] **Step 1: Create `src/routes/home.ts`**
```typescript
import { config } from '../env' ;
export const handleHome = async () : Promise < Response > => {
const html = await Bun . file ( 'src/home.html' ). text ();
return new Response ( html , {
status : 200 ,
headers : {
'content-type' : 'text/html; charset=utf-8' ,
},
});
};
```
- [ ] **Step 2: Create `src/home.html`**
This is a comprehensive single-page file manager UI. It must include:
- Bucket selector/create/delete
- Object listing with folder navigation
- Upload via drag & drop or file picker with progress bar
- Download and delete actions
- Copy link to clipboard
- Search/filter
- Dark/light mode
- Credentials display modal
The HTML is too long to reproduce in full here, but the core structure:
```html
<!DOCTYPE html>
< html lang = "en" >
< head >
< meta charset = "UTF-8" >
< meta name = "viewport" content = "width=device-width, initial-scale=1.0" >
< title > TeleUploader · S3 File Manager</ title >
< style >
: root {
--bg : #ffffff ; --bg2 : #f5f5f5 ; --text : #1a1a1a ;
--text2 : #666 ; --border : #e0e0e0 ; --accent : #2563eb ;
--danger : #dc2626 ; --radius : 8 px ;
}
@ media ( prefers-color-scheme : dark ) {
: root {
--bg : #0d1117 ; --bg2 : #161b22 ; --text : #c9d1d9 ;
--text2 : #8b949e ; --border : #30363d ; --accent : #58a6ff ;
--danger : #f85149 ;
}
}
* { box-sizing : border-box ; margin : 0 ; padding : 0 ; }
body {
font-family : - apple-system , BlinkMacSystemFont , 'Segoe UI' , Roboto , sans-serif ;
background : var ( -- bg ); color : var ( -- text ); line-height : 1.5 ;
min-height : 100 vh ;
}
/* ─── Top Bar ─── */
. topbar {
display : flex ; align-items : center ; gap : 12 px ;
padding : 12 px 24 px ; background : var ( -- bg2 );
border-bottom : 1 px solid var ( -- border );
position : sticky ; top : 0 ; z-index : 50 ;
}
. topbar . logo { font-weight : 700 ; font-size : 1.1 rem ; }
. topbar select , . topbar button {
padding : 6 px 12 px ; border : 1 px solid var ( -- border );
border-radius : var ( -- radius ); background : var ( -- bg );
color : var ( -- text ); font-size : 0.875 rem ; cursor : pointer ;
}
. topbar button . primary { background : var ( -- accent ); color : #fff ; border-color : var ( -- accent ); }
. topbar . spacer { flex : 1 ; }
. topbar . search input {
padding : 6 px 12 px ; border : 1 px solid var ( -- border );
border-radius : var ( -- radius ); background : var ( -- bg );
color : var ( -- text ); font-size : 0.875 rem ; width : 200 px ;
}
/* ─── File List ─── */
. file-list { padding : 16 px 24 px ; }
. breadcrumb {
padding : 8 px 0 ; margin-bottom : 8 px ; font-size : 0.9 rem ;
color : var ( -- accent ); cursor : pointer ;
}
. breadcrumb span : hover { text-decoration : underline ; }
. breadcrumb . sep { color : var ( -- text2 ); margin : 0 4 px ; }
. file-row {
display : flex ; align-items : center ; gap : 12 px ;
padding : 10 px 12 px ; border-radius : var ( -- radius );
cursor : pointer ; transition : background 0.1 s ;
}
. file-row : hover { background : var ( -- bg2 ); }
. file-row . icon { font-size : 1.2 rem ; width : 28 px ; text-align : center ; flex-shrink : 0 ; }
. file-row . name { flex : 1 ; overflow : hidden ; text-overflow : ellipsis ; white-space : nowrap ; }
. file-row . size { width : 80 px ; text-align : right ; color : var ( -- text2 ); font-size : 0.85 rem ; }
. file-row . date { width : 140 px ; color : var ( -- text2 ); font-size : 0.85 rem ; }
. file-row . actions { display : flex ; gap : 4 px ; }
. file-row . actions button {
padding : 4 px 8 px ; border : none ; border-radius : 4 px ;
background : transparent ; color : var ( -- text2 ); cursor : pointer ;
font-size : 0.8 rem ;
}
. file-row . actions button : hover { color : var ( -- text ); background : var ( -- border ); }
/* ─── Drop Zone ─── */
. dropzone {
position : fixed ; bottom : 0 ; left : 0 ; right : 0 ;
padding : 12 px 24 px ; background : var ( -- bg2 );
border-top : 1 px solid var ( -- border );
text-align : center ; color : var ( -- text2 ); font-size : 0.85 rem ;
cursor : pointer ;
}
. dropzone . dragover {
background : var ( -- accent ); color : #fff ;
}
/* ─── Upload progress ─── */
. progress-overlay {
position : fixed ; top : 0 ; left : 0 ; right : 0 ; bottom : 0 ;
background : rgba ( 0 , 0 , 0 , 0.5 ); display : flex ;
align-items : center ; justify-content : center ; z-index : 100 ;
}
. progress-card {
background : var ( -- bg ); padding : 24 px ; border-radius : var ( -- radius );
min-width : 300 px ; max-width : 500 px ;
}
. progress-bar {
height : 8 px ; background : var ( -- border ); border-radius : 4 px ;
margin : 12 px 0 ; overflow : hidden ;
}
. progress-bar . fill {
height : 100 % ; background : var ( -- accent );
transition : width 0.2 s ; width : 0 % ;
}
/* ─── Modal ─── */
. modal-overlay {
position : fixed ; top : 0 ; left : 0 ; right : 0 ; bottom : 0 ;
background : rgba ( 0 , 0 , 0 , 0.5 ); display : flex ;
align-items : center ; justify-content : center ; z-index : 100 ;
}
. modal {
background : var ( -- bg ); padding : 24 px ; border-radius : var ( -- radius );
min-width : 360 px ; max-width : 500 px ;
}
. modal h3 { margin-bottom : 16 px ; }
. modal input {
width : 100 % ; padding : 8 px 12 px ; border : 1 px solid var ( -- border );
border-radius : var ( -- radius ); background : var ( -- bg );
color : var ( -- text ); margin-bottom : 12 px ;
}
. modal . buttons { display : flex ; gap : 8 px ; justify-content : flex-end ; }
. modal . buttons button {
padding : 8 px 16 px ; border : 1 px solid var ( -- border );
border-radius : var ( -- radius ); background : var ( -- bg );
color : var ( -- text ); cursor : pointer ;
}
. modal . buttons . primary { background : var ( -- accent ); color : #fff ; border-color : var ( -- accent ); }
. modal . buttons . danger { background : var ( -- danger ); color : #fff ; border-color : var ( -- danger ); }
/* ─── Empty state ─── */
. empty {
text-align : center ; padding : 48 px 24 px ; color : var ( -- text2 );
}
. empty h2 { font-size : 1.2 rem ; margin-bottom : 8 px ; }
</ style >
</ head >
< body >
< div class = "topbar" >
< span class = "logo" > 📦 TeleUploader</ span >
< select id = "bucketSelect" onchange = "switchBucket(this.value)" >
< option value = "" > — Select bucket —</ option >
</ select >
< button onclick = "showCreateBucketModal()" > + New</ button >
< button onclick = "showCredentialsModal()" title = "S3 Credentials" > 🔑</ button >
< span class = "spacer" ></ span >
< div class = "search" >
< input id = "searchInput" type = "text" placeholder = "Filter prefix..." oninput = "debouncedSearch()" >
</ div >
</ div >
< div id = "breadcrumb" class = "breadcrumb" style = "display:none;padding:8px 24px" ></ div >
< div id = "fileList" class = "file-list" >
< div class = "empty" >
< h2 > Select a bucket to get started</ h2 >
< p > Choose a bucket from the dropdown above, or create a new one.</ p >
</ div >
</ div >
< div id = "dropzone" class = "dropzone" style = "display:none" >
📁 Drop files here or click to upload
</ div >
< div id = "progressOverlay" class = "progress-overlay" style = "display:none" >
< div class = "progress-card" >
< h3 > Uploading...</ h3 >
< div id = "progressFileName" ></ div >
< div class = "progress-bar" >< div id = "progressFill" class = "fill" ></ div ></ div >
< div id = "progressPercent" style = "font-size:0.85rem;color:var(--text2)" > 0%</ div >
</ div >
</ div >
< div id = "modalOverlay" class = "modal-overlay" style = "display:none" onclick = "closeModal(event)" >
< div id = "modalContent" class = "modal" onclick = "event.stopPropagation()" >
<!-- Dynamic modal content -->
</ div >
</ div >
< script >
// ─── State ───
let currentBucket = null ;
let currentPrefix = '' ;
let currentObjects = [];
let currentPrefixes = [];
let allBuckets = [];
let searchTimer = null ;
// ─── API helpers ───
const api = async ( path , opts = {}) => {
const res = await fetch ( path , opts );
if ( ! res . ok ) {
const body = await res . json (). catch (() => ({ error : res . statusText }));
throw new Error ( body . error || res . statusText );
}
return res ;
};
const apiJson = async ( path , opts = {}) => {
const res = await api ( path , { headers : { 'content-type' : 'application/json' }, ... opts });
return res . json ();
};
// ─── Load buckets ───
const loadBuckets = async () => {
const data = await apiJson ( '/api/v1/buckets' );
allBuckets = data . buckets || [];
const sel = document . getElementById ( 'bucketSelect' );
sel . innerHTML = '<option value="">— Select bucket —</option>' +
allBuckets . map ( b => `<option value=" ${ b . name } "> ${ b . name } ( ${ b . objectCount } )</option>` ). join ( '' );
if ( currentBucket ) sel . value = currentBucket ;
};
// ─── Switch bucket ───
const switchBucket = async ( name ) => {
currentBucket = name || null ;
currentPrefix = '' ;
if ( name ) {
await loadObjects ();
document . getElementById ( 'dropzone' ). style . display = 'block' ;
} else {
document . getElementById ( 'fileList' ). innerHTML =
'<div class="empty"><h2>Select a bucket</h2><p>Choose a bucket from the dropdown above.</p></div>' ;
document . getElementById ( 'breadcrumb' ). style . display = 'none' ;
document . getElementById ( 'dropzone' ). style . display = 'none' ;
}
};
// ─── Breadcrumb ───
const renderBreadcrumb = () => {
const bc = document . getElementById ( 'breadcrumb' );
if ( ! currentPrefix ) { bc . style . display = 'none' ; return ; }
bc . style . display = 'block' ;
const parts = currentPrefix . split ( '/' ). filter ( Boolean );
bc . innerHTML = `<span onclick="navigateTo('')"> ${ currentBucket } </span>` ;
let accumulated = '' ;
for ( const part of parts ) {
accumulated += part + '/' ;
bc . innerHTML += `<span class="sep">/</span><span onclick="navigateTo(' ${ accumulated } ')"> ${ part } </span>` ;
}
};
const navigateTo = ( prefix ) => { currentPrefix = prefix ; loadObjects (); };
// ─── Load objects ───
const loadObjects = async () => {
if ( ! currentBucket ) return ;
const searchVal = document . getElementById ( 'searchInput' ). value ;
const prefix = searchVal || currentPrefix ;
const url = `/api/v1/buckets/ ${ encodeURIComponent ( currentBucket ) } /objects?prefix= ${ encodeURIComponent ( prefix ) } &delimiter=/&max-keys=200` ;
try {
const data = await apiJson ( url );
currentObjects = data . objects || [];
currentPrefixes = data . prefixes || [];
renderFileList ();
renderBreadcrumb ();
} catch ( e ) {
document . getElementById ( 'fileList' ). innerHTML =
`<div class="empty"><h2>Error</h2><p> ${ e . message } </p></div>` ;
}
};
// ─── Render file list ───
const renderFileList = () => {
const container = document . getElementById ( 'fileList' );
if ( currentPrefixes . length === 0 && currentObjects . length === 0 ) {
container . innerHTML = '<div class="empty"><h2>This bucket is empty</h2><p>Drop files here to upload.</p></div>' ;
return ;
}
let html = '' ;
for ( const prefix of currentPrefixes ) {
const displayName = prefix . replace ( currentPrefix , '' );
html += `<div class="file-row" onclick="navigateTo(' ${ prefix } ')">
<span class="icon">🗂</span>
<span class="name"> ${ displayName . endsWith ( '/' ) ? displayName : displayName + '/' } </span>
<span class="size">—</span>
<span class="date"></span>
<span class="actions"></span>
</div>` ;
}
for ( const obj of currentObjects ) {
const displayName = obj . key . replace ( currentPrefix , '' );
const size = formatSize ( obj . sizeBytes );
const date = formatDate ( obj . lastModified );
html += `<div class="file-row">
<span class="icon">📄</span>
<span class="name"> ${ escapeHtml ( displayName ) } </span>
<span class="size"> ${ size } </span>
<span class="date"> ${ date } </span>
<span class="actions">
<button onclick="event.stopPropagation();downloadObject(' ${ obj . key } ')" title="Download">⬇</button>
<button onclick="event.stopPropagation();copyLink(' ${ obj . key } ')" title="Copy link">🔗</button>
<button onclick="event.stopPropagation();deleteObject(' ${ obj . key } ')" title="Delete">🗑</button>
</span>
</div>` ;
}
container . innerHTML = html ;
};
// ─── Helpers ───
const formatSize = ( bytes ) => {
if ( ! bytes ) return '0 B' ;
const units = [ 'B' , 'KB' , 'MB' , 'GB' , 'TB' ];
let i = 0 ; let size = bytes ;
while ( size >= 1024 && i < units . length - 1 ) { size /= 1024 ; i ++ ; }
return ` ${ size . toFixed ( i > 0 ? 1 : 0 ) } ${ units [ i ] } ` ;
};
const formatDate = ( iso ) => {
if ( ! iso ) return '' ;
const d = new Date ( iso );
return d . toLocaleDateString ( undefined , { month : 'short' , day : 'numeric' , year : 'numeric' });
};
const escapeHtml = ( s ) => {
const div = document . createElement ( 'div' );
div . textContent = s ;
return div . innerHTML ;
};
const debouncedSearch = () => {
clearTimeout ( searchTimer );
searchTimer = setTimeout ( loadObjects , 300 );
};
// ─── Actions ───
const downloadObject = async ( key ) => {
const url = `/api/v1/buckets/ ${ encodeURIComponent ( currentBucket ) } /download/ ${ encodeURIComponent ( key ) } ` ;
window . open ( url , '_blank' );
};
const copyLink = ( key ) => {
const url = ` ${ window . location . origin } /api/v1/buckets/ ${ encodeURIComponent ( currentBucket ) } /download/ ${ encodeURIComponent ( key ) } ` ;
navigator . clipboard . writeText ( url ). catch (() => {});
};
const deleteObject = async ( key ) => {
if ( ! confirm ( `Delete " ${ key } "?` )) return ;
try {
await api ( `/api/v1/buckets/ ${ encodeURIComponent ( currentBucket ) } / ${ encodeURIComponent ( key ) } ` , { method : 'DELETE' });
await loadObjects ();
} catch ( e ) {
alert ( `Delete failed: ${ e . message } ` );
}
};
// ─── Upload ───
const uploadFiles = async ( files ) => {
if ( ! currentBucket || files . length === 0 ) return ;
const overlay = document . getElementById ( 'progressOverlay' );
const progressFill = document . getElementById ( 'progressFill' );
const progressName = document . getElementById ( 'progressFileName' );
const progressPercent = document . getElementById ( 'progressPercent' );
overlay . style . display = 'flex' ;
for ( let i = 0 ; i < files . length ; i ++ ) {
const file = files [ i ];
progressName . textContent = ` ${ i + 1 } / ${ files . length } : ${ file . name } ` ;
progressFill . style . width = '0%' ;
progressPercent . textContent = '0%' ;
await new Promise (( resolve , reject ) => {
const formData = new FormData ();
formData . append ( 'file' , file );
formData . append ( 'key' , currentPrefix + file . name );
const xhr = new XMLHttpRequest ();
xhr . upload . onprogress = ( e ) => {
if ( e . lengthComputable ) {
const pct = Math . round (( e . loaded / e . total ) * 100 );
progressFill . style . width = pct + '%' ;
progressPercent . textContent = pct + '%' ;
}
};
xhr . onload = () => {
if ( xhr . status >= 200 && xhr . status < 300 ) resolve ();
else reject ( new Error ( xhr . statusText ));
};
xhr . onerror = () => reject ( new Error ( 'Upload failed' ));
xhr . open ( 'POST' , `/api/v1/buckets/ ${ encodeURIComponent ( currentBucket ) } /upload` );
xhr . send ( formData );
});
}
overlay . style . display = 'none' ;
await loadObjects ();
};
// ─── Drag & drop ───
const dropzone = document . getElementById ( 'dropzone' );
dropzone . addEventListener ( 'dragover' , ( e ) => { e . preventDefault (); dropzone . classList . add ( 'dragover' ); });
dropzone . addEventListener ( 'dragleave' , () => dropzone . classList . remove ( 'dragover' ));
dropzone . addEventListener ( 'drop' , ( e ) => {
e . preventDefault ();
dropzone . classList . remove ( 'dragover' );
if ( e . dataTransfer . files . length > 0 ) uploadFiles ( e . dataTransfer . files );
});
dropzone . addEventListener ( 'click' , () => {
const input = document . createElement ( 'input' );
input . type = 'file' ;
input . multiple = true ;
input . onchange = () => { if ( input . files . length > 0 ) uploadFiles ( input . files ); };
input . click ();
});
// ─── Modals ───
const showModal = ( html ) => {
document . getElementById ( 'modalContent' ). innerHTML = html ;
document . getElementById ( 'modalOverlay' ). style . display = 'flex' ;
};
const closeModal = ( e ) => {
if ( e && e . target !== e . currentTarget ) return ;
document . getElementById ( 'modalOverlay' ). style . display = 'none' ;
};
const showCreateBucketModal = () => {
showModal ( `
<h3>Create Bucket</h3>
<input id="bucketNameInput" type="text" placeholder="my-bucket-name" pattern="[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]">
<p style="font-size:0.8rem;color:var(--text2);margin-bottom:12px">Lowercase, 3-63 chars, no underscores</p>
<div class="buttons">
<button onclick="closeModal()">Cancel</button>
<button class="primary" onclick="createBucket()">Create</button>
</div>
` );
setTimeout (() => document . getElementById ( 'bucketNameInput' ) ? . focus (), 100 );
};
const createBucket = async () => {
const name = document . getElementById ( 'bucketNameInput' ). value . trim ();
if ( ! name ) return ;
try {
await apiJson ( '/api/v1/buckets' , { method : 'POST' , body : JSON . stringify ({ name }) });
closeModal ();
await loadBuckets ();
document . getElementById ( 'bucketSelect' ). value = name ;
await switchBucket ( name );
} catch ( e ) {
alert ( `Failed: ${ e . message } ` );
}
};
const showCredentialsModal = () => {
showModal ( `
<h3>S3 Credentials</h3>
<p style="margin-bottom:12px;font-size:0.85rem;color:var(--text2)">
Use these credentials in any S3 client (aws-cli, rclone, s3cmd, etc.)
</p>
<label style="font-size:0.85rem;font-weight:600">Endpoint URL</label>
<input type="text" value=" ${ window . location . origin } " readonly onclick="this.select()">
<label style="font-size:0.85rem;font-weight:600">Region</label>
<input type="text" value="us-east-1" readonly onclick="this.select()">
<label style="font-size:0.85rem;font-weight:600">Access Key</label>
<input id="s3AccessKey" type="text" readonly onclick="this.select()">
<label style="font-size:0.85rem;font-weight:600">Secret Key</label>
<input id="s3SecretKey" type="password" readonly onclick="this.select()">
<div class="buttons">
<button onclick="closeModal()">Close</button>
</div>
` );
};
// ─── Init ───
loadBuckets ();
</ script >
</ body >
</ html >
```
- [ ] **Step 3: Register routes in `src/index.ts`**
Add imports:
```typescript
import { handleHome } from './routes/home' ;
import { handleWebApiV1 } from './routes/web-api' ;
import { handleS3Request } from './routes/s3' ;
import { isS3Request } from './utils/s3/auth' ;
```
Add routes to the serve config (after existing routes, before server variable definition):
```typescript
serve ({
// ... existing routes ...
'/' : {
GET : handleHome ,
},
// Web API v1
'/api/v1/*' : {
GET : handleWebApiV1 ,
POST : handleWebApiV1 ,
DELETE : handleWebApiV1 ,
PUT : handleWebApiV1 ,
},
// S3-compatible API catch-all
$ : async ( req : Request ) => {
// Only handle requests with SigV4 auth header
if ( isS3Request ( Object . fromEntries ( req . headers ))) {
return handleS3Request ( req );
}
return new Response ( 'Not Found' , { status : 404 });
},
})
```
Note: Bun.serve() routes matching priority: literal paths first, then wildcard paths, then `$` catch-all. The `/api/v1/*` pattern is a wildcard route that matches any path starting with `/api/v1/` . For the S3 catch-all, we use `$` which is Bun's catch-all route that fires when no other route matches. Since the S3 catch-all is the last resort, we check the Authorization header to differentiate S3 from true 404s.
- [ ] **Step 4: Verify the app still compiles**
```bash
cd /mnt/code/TeleUploader && bun build src/index.ts --target= bun --outfile= dist/index.js
```
Expected: Build succeeds with no errors.
- [ ] **Step 5: Commit**
```bash
git add src/routes/home.ts src/home.html src/index.ts
git commit -m "feat: add web file manager UI and S3 catch-all route"
```
---
### Task 7: Tests
**Files:**
- Create: `test/s3-auth.test.ts`
- Create: `test/s3-operations.test.ts`
- Create: `test/web-api.test.ts`
- Modify: `package.json` (add test script entries)
- [ ] **Step 1: Create `test/s3-auth.test.ts`**
```typescript
import { describe , expect , it , beforeAll } from 'bun:test' ;
describe ( 'S3 Auth (SigV4)' , () => {
let verifySignature : typeof import ( '../src/utils/s3/auth' ). verifySignature ;
let isS3Request : typeof import ( '../src/utils/s3/auth' ). isS3Request ;
beforeAll ( async () => {
const auth = await import ( '../src/utils/s3/auth' );
verifySignature = auth . verifySignature ;
isS3Request = auth . isS3Request ;
});
it ( 'should detect S3 requests by Authorization header' , () => {
expect ( isS3Request ({ authorization : 'AWS4-HMAC-SHA256 Credential=...' })). toBe ( true );
expect ( isS3Request ({ authorization : 'Bearer token123' })). toBe ( false );
expect ( isS3Request ({})). toBe ( false );
});
it ( 'should reject missing Authorization header' , async () => {
const result = await verifySignature ( 'GET' , 'http://localhost/' , {}, null , 'key' , 'secret' , 'us-east-1' );
expect ( result . isValid ). toBe ( false );
});
it ( 'should reject wrong access key' , async () => {
const headers = {
authorization : 'AWS4-HMAC-SHA256 Credential=wrongkey/20260706/us-east-1/s3/aws4_request, SignedHeaders=host, Signature=abc123' ,
'x-amz-date' : '20260706T120000Z' ,
};
const result = await verifySignature ( 'GET' , 'http://localhost/' , headers , null , 'correctkey' , 'secret' , 'us-east-1' );
expect ( result . isValid ). toBe ( false );
expect ( result . errorCode ). toBe ( 'SignatureDoesNotMatch' );
});
it ( 'should accept valid GET ListBuckets request' , async () => {
// This tests that verification doesn't crash — full SigV4 signature
// construction requires exact header matching which is complex.
// The real signature verification is tested via integration with S3 clients.
const headers = {
authorization : 'AWS4-HMAC-SHA256 Credential=testkey/20260706/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=0000000000000000000000000000000000000000000000000000000000000000' ,
'x-amz-date' : '20260706T120000Z' ,
'x-amz-content-sha256' : 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855' ,
host : 'localhost' ,
};
const result = await verifySignature ( 'GET' , 'http://localhost/' , headers , null , 'testkey' , 'testsecret' , 'us-east-1' );
// Expected to not crash (the actual signature won't match with test values)
expect ( result . isValid ). toBe ( false );
expect ( result . errorCode ). toBe ( 'SignatureDoesNotMatch' );
});
});
```
- [ ] **Step 2: Create `test/s3-operations.test.ts`**
Tests for S3 bucket and object operations. Because the S3 endpoints depend on Telegram (which is mocked), we test in isolation:
```typescript
import { afterAll , beforeAll , beforeEach , describe , expect , it , mock } from 'bun:test' ;
// Mock DB and Telegram
const mockDbExecute = mock (() => Promise . resolve ({ rows : [], rowCount : 0 }));
let mockSelectResult : unknown [] = [];
const mockDbSelect = mock (() => ({
from : () => ({
where : () => ({
orderBy : () => ({
limit : () => Promise . resolve ( mockSelectResult ),
}),
limit : () => Promise . resolve ( mockSelectResult ),
}),
}),
}));
mock . module ( '../src/db/index' , () => ({
db : {
execute : mockDbExecute ,
select : mockDbSelect ,
insert : () => ({
values : () => Promise . resolve (),
}),
update : () => ({
set : () => ({
where : () => ({
returning : () => Promise . resolve ([]),
}),
}),
}),
},
files : {},
}));
mock . module ( '../src/utils/telegram' , () => ({
forwardToStorage : () => Promise . resolve ({
telegramFileId : 'mock-tg-id' ,
telegramFileUniqueId : 'mock-tg-unique' ,
storageMessageId : 12345 ,
}),
getFileInfo : () => Promise . resolve ({
file_size : 1000 ,
mime_type : 'application/octet-stream' ,
file_path : 'documents/file_0.dat' ,
bot_token : '123456:ABC-DEF' ,
}),
}));
mock . module ( 'nanoid' , () => ({
nanoid : () => 'mocked-nanoid-' + Math . random (). toString ( 36 ). slice ( 2 , 10 ),
}));
describe ( 'S3 Bucket Operations' , () => {
let s3Module : typeof import ( '../src/routes/s3' );
let handleS3Request : typeof import ( '../src/routes/s3' ). handleS3Request ;
beforeAll ( async () => {
s3Module = await import ( '../src/routes/s3' );
handleS3Request = s3Module . handleS3Request ;
});
beforeEach (() => {
mockDbExecute . mockClear ();
mockDbSelect . mockClear ();
mockSelectResult = [];
process . env . S3_ACCESS_KEY = 'testkey' ;
process . env . S3_SECRET_KEY = 'testsecret' ;
process . env . S3_DEFAULT_REGION = 'us-east-1' ;
process . env . BOT_TOKEN = '123456:ABC-DEF' ;
process . env . STORAGE_CHANNEL_ID = '-1001234567890' ;
2026-08-02 16:19:30 +07:00
process . env . BASE_URL = 'http://localhost:4000' ;
2026-07-06 14:33:27 +07:00
process . env . DATABASE_URL = 'postgresql://localhost/test' ;
});
afterAll (() => {
mock . restore ();
});
it ( 'should return 403 for unauthorized requests' , async () => {
2026-08-02 16:19:30 +07:00
const req = new Request ( 'http://localhost:4000/' , {
2026-07-06 14:33:27 +07:00
method : 'GET' ,
headers : { authorization : 'Invalid' },
});
const res = await handleS3Request ( req );
expect ( res . status ). toBe ( 403 );
const body = await res . text ();
expect ( body ). toContain ( 'AccessDenied' );
expect ( body ). toContain ( '<?xml' );
});
});
describe ( 'S3 XML Builders' , () => {
let xml : typeof import ( '../src/utils/s3/xml' );
beforeAll ( async () => {
xml = await import ( '../src/utils/s3/xml' );
});
it ( 'should build ListBuckets XML' , () => {
const result = xml . listBucketsXml (
[{ name : 'test-bucket' , createdAt : new Date ( '2026-01-01' ) }],
'req-1' ,
);
expect ( result ). toContain ( '<?xml' );
expect ( result ). toContain ( 'test-bucket' );
expect ( result ). toContain ( '<ListAllMyBucketsResult' );
expect ( result ). toContain ( '<CreationDate>' );
});
it ( 'should build ListBucketResult XML' , () => {
const result = xml . listBucketResultXml (
'my-bucket' ,
[{ key : 'file.txt' , sizeBytes : 100 , etag : 'abc' , lastModified : new Date (), mimeType : 'text/plain' }],
[],
false ,
null ,
1000 ,
'' ,
null ,
null ,
'req-1' ,
);
expect ( result ). toContain ( 'file.txt' );
expect ( result ). toContain ( '<ListBucketResult' );
expect ( result ). toContain ( '<Size>100</Size>' );
});
it ( 'should build ListBucketV2 XML' , () => {
const result = xml . listBucketV2ResultXml (
'my-bucket' ,
[{ key : 'a.txt' , sizeBytes : 50 , etag : 'def' , lastModified : new Date (), mimeType : 'text/plain' }],
[ 'photos/' ],
false ,
1000 ,
'' ,
'/' ,
null ,
null ,
1 ,
'req-2' ,
);
expect ( result ). toContain ( '<ListBucketResultV2' );
expect ( result ). toContain ( 'a.txt' );
expect ( result ). toContain ( '<Prefix>photos/</Prefix>' );
});
it ( 'should build InitiateMultipartUpload XML' , () => {
const result = xml . initiateMultipartUploadXml ( 'bucket' , 'key' , 'upload-123' );
expect ( result ). toContain ( '<UploadId>upload-123</UploadId>' );
});
it ( 'should build CopyObjectResult XML' , () => {
const result = xml . copyObjectResultXml ( 'etag-abc' , new Date ());
expect ( result ). toContain ( '<CopyObjectResult' );
expect ( result ). toContain ( 'etag-abc' );
});
it ( 'should build error XML' , () => {
const result = xml . s3ErrorXml ( 'NoSuchBucket' , 'The specified bucket does not exist' , '/bucket' , 'req-1' );
expect ( result ). toContain ( '<Code>NoSuchBucket</Code>' );
expect ( result ). toContain ( '<Error>' );
});
it ( 'should parse DeleteObjects body' , () => {
const body = `<Delete><Object><Key>file1.txt</Key></Object><Object><Key>file2.txt</Key></Object></Delete>` ;
const { keys } = xml . parseDeleteObjectsBody ( body );
expect ( keys ). toEqual ([ 'file1.txt' , 'file2.txt' ]);
});
it ( 'should parse CompleteMultipartUpload body' , () => {
const body = `<CompleteMultipartUpload><Part><PartNumber>1</PartNumber><ETag>"abc"</ETag></Part><Part><PartNumber>2</PartNumber><ETag>"def"</ETag></Part></CompleteMultipartUpload>` ;
const parts = xml . parseCompleteMultipartBody ( body );
expect ( parts ). toHaveLength ( 2 );
expect ( parts [ 0 ]). toEqual ({ partNumber : 1 , etag : 'abc' });
expect ( parts [ 1 ]). toEqual ({ partNumber : 2 , etag : 'def' });
});
});
```
- [ ] **Step 3: Create `test/web-api.test.ts`**
```typescript
import { afterAll , beforeAll , beforeEach , describe , expect , it , mock } from 'bun:test' ;
// Mock DB layer
const mockBuckets = [
{ id : 'uuid-1' , name : 'test-bucket' , createdAt : new Date (), updatedAt : new Date () },
];
const mockDbExecute = mock (( sql : unknown ) => {
const sqlStr = String ( sql );
if ( sqlStr . includes ( 'FROM buckets WHERE name =' )) {
return Promise . resolve ({ rows : mockBuckets.filter ( b => sqlStr . includes ( b . name )), rowCount : 1 });
}
if ( sqlStr . includes ( 'FROM buckets ORDER BY' )) {
return Promise . resolve ({ rows : mockBuckets , rowCount : mockBuckets.length });
}
return Promise . resolve ({ rows : [], rowCount : 0 });
});
mock . module ( '../src/db/index' , () => ({
db : {
execute : mockDbExecute ,
select : () => ({
from : () => ({
where : () => ({
limit : () => Promise . resolve ([]),
}),
}),
}),
},
files : {},
}));
mock . module ( '../src/utils/telegram' , () => ({
forwardToStorage : () => Promise . resolve ({
telegramFileId : 'mock-tg-id' ,
telegramFileUniqueId : 'mock-tg-unique' ,
storageMessageId : 12345 ,
}),
getFileInfo : () => Promise . resolve ({
file_size : 100 ,
mime_type : 'text/plain' ,
file_path : 'documents/file.txt' ,
bot_token : '123456:ABC-DEF' ,
}),
}));
mock . module ( 'nanoid' , () => ({
nanoid : () => 'mocked-nanoid-' + Math . random (). toString ( 36 ). slice ( 2 , 10 ),
}));
describe ( 'Web API v1' , () => {
let handleWebApiV1 : typeof import ( '../src/routes/web-api' ). handleWebApiV1 ;
beforeAll ( async () => {
const webApi = await import ( '../src/routes/web-api' );
handleWebApiV1 = webApi . handleWebApiV1 ;
});
beforeEach (() => {
mockDbExecute . mockClear ();
process . env . BOT_TOKEN = '123456:ABC-DEF' ;
process . env . STORAGE_CHANNEL_ID = '-1001234567890' ;
2026-08-02 16:19:30 +07:00
process . env . BASE_URL = 'http://localhost:4000' ;
2026-07-06 14:33:27 +07:00
process . env . DATABASE_URL = 'postgresql://localhost/test' ;
});
afterAll (() => {
mock . restore ();
});
it ( 'should list buckets via GET /api/v1/buckets' , async () => {
2026-08-02 16:19:30 +07:00
const req = new Request ( 'http://localhost:4000/api/v1/buckets' );
2026-07-06 14:33:27 +07:00
const res = await handleWebApiV1 ( req );
expect ( res . status ). toBe ( 200 );
const data = await res . json ();
expect ( data ). toHaveProperty ( 'buckets' );
expect ( Array . isArray ( data . buckets )). toBe ( true );
});
it ( 'should return 404 for unknown API path' , async () => {
2026-08-02 16:19:30 +07:00
const req = new Request ( 'http://localhost:4000/api/v1/unknown' );
2026-07-06 14:33:27 +07:00
const res = await handleWebApiV1 ( req );
expect ( res . status ). toBe ( 404 );
const data = await res . json ();
expect ( data ). toHaveProperty ( 'error' );
});
it ( 'should return bucket object listing' , async () => {
2026-08-02 16:19:30 +07:00
const req = new Request ( 'http://localhost:4000/api/v1/buckets/test-bucket/objects?prefix=' );
2026-07-06 14:33:27 +07:00
const res = await handleWebApiV1 ( req );
// Should return 200 even with empty results
expect ( res . status ). toBe ( 200 );
const data = await res . json ();
expect ( data ). toHaveProperty ( 'objects' );
expect ( data ). toHaveProperty ( 'prefixes' );
});
it ( 'should reject invalid bucket name on create' , async () => {
2026-08-02 16:19:30 +07:00
const req = new Request ( 'http://localhost:4000/api/v1/buckets' , {
2026-07-06 14:33:27 +07:00
method : 'POST' ,
headers : { 'content-type' : 'application/json' },
body : JSON.stringify ({ name : 'INVALID_NAME!' }),
});
const res = await handleWebApiV1 ( req );
expect ( res . status ). toBe ( 400 );
});
});
```
- [ ] **Step 4: Add test scripts to `package.json`**
Add these scripts:
```json
"test:s3-auth" : "bun test test/s3-auth.test.ts" ,
"test:s3-ops" : "bun test test/s3-operations.test.ts" ,
"test:web-api" : "bun test test/web-api.test.ts" ,
"test:s3" : "bun test test/s3-auth.test.ts && bun test test/s3-operations.test.ts && bun test test/web-api.test.ts"
```
- [ ] **Step 5: Run all tests**
```bash
cd /mnt/code/TeleUploader && bun test
```
Expected: All existing tests pass + new S3 tests pass.
- [ ] **Step 6: Commit**
```bash
git add test/s3-auth.test.ts test/s3-operations.test.ts test/web-api.test.ts package.json
git commit -m "test: add S3 auth, operations, and web API tests"
```
---
## Spec Self-Review Check
- ✅ **Spec coverage** : Every spec section has corresponding tasks:
- DB schema → Task 1
- DB CRUD layers → Task 2
- S3 Auth + XML → Task 3
- S3 bucket/object/listing/multipart handlers → Task 4
- JSON web API → Task 5
- Web file manager UI → Task 6
- Tests → Task 7
- ✅ **No placeholders** : All steps contain actual code
- ✅ **Type consistency** : Function names used across tasks match (e.g., `createBucket` in Task 2 is consumed by `handleCreateBucket` in Task 4)
## Execution
Plan complete and saved to `docs/superpowers/plans/2026-07-06-s3-compatible-teleuploader.md` .
**Two execution options:**
1. **Subagent-Driven (recommended)** — I dispatch a fresh subagent per task, review between tasks, fast iteration
2. **Inline Execution** — Execute tasks in this session using executing-plans, batch execution with checkpoints
**Which approach?**