Commit Graph
128 Commits
Author SHA1 Message Date
Claude 002492626b fix: secure SigV4, temp leaks, OOM risk, duplicate migration, and cache issues
Deploy FileDrop / deploy (push) Failing after 15s
Security fixes:
- SigV4 signature comparison now uses crypto.timingSafeEqual (timing attack fix)
  - AccessKey, region, and HMAC signature all timing-safe
- Presigned URL expiry capped at 7 days (AWS spec compliance)
- Removed duplicate migration import (dead code)

Memory & leak fixes:
- Temp file leak in createZip(): cleanup temp file on error in both utils/ and shared/utils/
- OOM risk in web-api/v1 upload: stream File to temp instead of arrayBuffer()
- Removed duplicate migration import at startup

Performance fixes:
- Removed file.arrayBuffer() -> Bun.write() pattern in web-api-controller (stream + hash)

Test improvements:
- All fixes verified: 74/75 tests pass (1 pre-existing env config test)
- S3 auth tests: 7/7 pass after timing-safe fix
2026-07-28 19:33:05 +07:00
Claude 82c7f81ffa fix: streaming uploads, timeouts, and rate limiting for Docker registry safety
Critical fixes for S3 Docker registry backend:
- Stream PutObject body to temp file instead of req.arrayBuffer()
  - O(1) memory usage regardless of file size
  - SHA-256 hash computed while streaming
- Stream UploadPart body similarly
  - Also fixes: size check after streaming, not before
- Add 30s timeout to Telegram CDN chunk fetches (object-stream.ts)
  - Prevents hanging on stalled CDN connections
- Add rate limiting to S3 API routes (100 req/60s window)
  - Prevents resource exhaustion from concurrent layer pushes
- Add comprehensive test suite (10 tests):
  - Streaming verification (no arrayBuffer in PUT path)
  - Multi-MB body streaming safety
  - Empty body edge case
  - Concurrent upload isolation
  - Timeout signal presence
  - Rate limit route coverage

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:47:16 +07:00
Claude da7d7c2396 chore: clean up leftover directories from agent portability
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:39:27 +07:00
Claude a9a1938ffe Merge branch 'worktree-ddd-clean-architecture-restructure'
# Conflicts:
#	src/infrastructure/cache/index.ts
#	src/interfaces/http/middleware/auth.ts
#	src/interfaces/http/middleware/rate-limit.ts
#	src/shared/logger/index.ts
2026-07-28 18:38:49 +07:00
Claude ee10cb494e fix: resolve code review issues - import paths and structure cleanup
- Fix infrastructure imports: chunked-storage uses new path for shared/utils and interfaces/s3
- Fix health-controller: imports from infrastructure/persistence/drizzle instead of old db/
- Fix routes/index.ts: imports from new interfaces/s3 and middleware paths
- Marked Telegram-specific types in shared/utils/file.ts as future extraction

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:21:57 +07:00
Claude 4568644922 fix: correct import paths and add missing protocol files for DDD structure
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:19:45 +07:00
Claude 234ca7b14c feat: rewire entry point to new architecture
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:12:48 +07:00
Claude e40dfd8084 feat: create HTTP middleware layer
Create auth.ts and rate-limit.ts middleware files in the interfaces layer
as part of the DDD/clean architecture restructure. Also add a config
re-export at src/interfaces/config/index.ts so the middleware can access
configuration through the interfaces layer boundary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:06:38 +07:00
Claude ea48e8fc03 feat: create application use cases (bucket, s3-object, multipart)
- manage-bucket.ts: extract bucket CRUD logic with validation
  (createListBuckets, createGetBucket, createCreateBucket,
   createDeleteBucket, createBucketExists)
- s3-object.ts: extract S3 object operations
  (createGetObject, createHeadObject, createPutObject,
   createCopyObject, createDeleteObject, createDeleteObjects,
   createListObjects, createFindObject)
- multipart-upload.ts: extract S3 multipart upload logic
  (createInitiateMultipartUpload, createUploadPart,
   createCompleteMultipartUpload, createAbortMultipartUpload,
   createListMultipartUploads, createListParts)

All use cases follow the existing factory function pattern with
dependency injection via repository/telegram service interfaces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:04:01 +07:00
Claude e2de8c4245 feat: create application use cases (upload, get-file, auth)
Create upload-file.ts use case with factory pattern supporting dedup, file type detection, size validation, and chunked/single storage strategies.
Create get-file.ts use case supporting redirect, chunked, and archive-entry retrieval strategies.
Create authenticate.ts use case with login, logout, and me operations.
All use cases use dependency injection and return typed DTOs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:59:49 +07:00
Claude 5e29589f1a feat: create application DTOs
Add data-transfer-object interfaces for the application layer:
upload, file, bucket, S3, and auth domains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:56:49 +07:00
Claude b15219335d feat: create infrastructure cache layer
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:49:17 +07:00
Claude ee3167fbfb feat: create domain port interfaces with JSDoc
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:48:24 +07:00
Claude e7657453d5 feat: create shared utilities layer with JSDoc
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:43:18 +07:00
Claude 5b5d4b0ba8 feat: create shared errors and logger layer
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:42:06 +07:00
Claude af8949de02 docs: add DDD/clean architecture restructure implementation plan
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:31:11 +07:00
Claude d8f02b79b1 docs: add DDD/clean architecture restructure design spec
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:29:20 +07:00
asepharyana 44e887c7cd fix: keep public upload API unauthenticated
Deploy FileDrop / deploy (push) Successful in 37s
2026-07-08 01:59:41 +07:00
asepharyana e8226bba64 ci: migrate deployment to gitea actions
Deploy FileDrop / deploy (push) Successful in 3m21s
2026-07-07 20:39:41 +07:00
asepharyana 340c12d671 feat: implement authentication routes with login, logout, and user info retrieval
feat: add S3 bucket versioning support and related XML response handling

refactor: rename temporary file paths from 'teleuploader' to 'filedrop' for consistency

fix: update Swagger documentation to reflect new API name and descriptions

test: add unit tests for authentication routes and utilities

test: implement end-to-end tests for S3 bucket configuration and versioning

chore: update environment variable defaults for new service name
2026-07-07 19:58:15 +07:00
asepharyana 144ebe6dd3 feat: implement chunked storage for Telegram file uploads and retrieval 2026-07-07 08:08:22 +07:00
asepharyana fbfff3d9ec fix: normalize web object sizes 2026-07-07 06:27:18 +07:00
asepharyana 1ef73fefa9 fix: restore multipart part order validation (without strict ETag check) 2026-07-07 06:11:49 +07:00
asepharyana bc806313b1 fix: use safe LIKE prefix matching instead of non-character range 2026-07-07 06:04:17 +07:00
asepharyana d1c8cc9d23 fix: ensure S3_DEFAULT_REGION and S3_VHOST_DOMAINS env vars are properly set in docker-compose 2026-07-07 06:02:16 +07:00
asepharyana 34962eb08a chore: update S3 vhost domains to include asepharana.web.id root/wildcard 2026-07-07 05:47:44 +07:00
asepharyana 8e6ccf2110 feat: S3 client compatibility — virtual-hosted style, CORS, presigned multi-method, ListMultipartUploads, edge case fixes
- Virtual-hosted bucket detection from Host header (extractS3BucketFromHost)
- S3 CORS headers + OPTIONS preflight + x-amz-id-2/HostId everywhere
- Presigned GET/PUT/HEAD/DELETE via centralized auth (no GET-only restriction)
- ListMultipartUploads with DB/xml helpers
- UploadPart partNumber range validation (1-10000)
- CompleteMultipartUpload ETag matching and ascending order validation
- DeleteObjects quiet mode support
- CopyObject URL-decode and conditional if-match/if-none-match
- encoding-type=url support in ListObjects V1/V2 XML
- Safe range-based prefix matching (replaces SQL LIKE)
- STREAMING-AWS4-HMAC-SHA256-PAYLOAD → 501 NotImplemented
- Traefik wildcard HostRegex for virtual-hosted style
- S3_VHOST_DOMAINS config env var
2026-07-07 05:39:23 +07:00
asepharyana 6aee3723fb fix: remove home page CSS lint warning 2026-07-07 04:48:44 +07:00
asepharyana ad3f4ce75e test: enable strict AWS SDK multipart coverage 2026-07-07 04:47:07 +07:00
asepharyana 339f788dc9 fix: ensure multipart part sizeBytes are Numbers, not DB strings 2026-07-07 04:42:07 +07:00
asepharyana 2e447ee58a fix: stream complete multipart S3 objects 2026-07-07 04:37:20 +07:00
asepharyana ad45404132 feat: support ranged S3 GetObject responses
Wire createGetObjectResponse into single-part GetObject handler for
proper Range/Content-Range support (200, 206, 416). Update presigned
E2E test to require 200. Add SDK Range request test.
2026-07-07 04:30:04 +07:00
asepharyana 378a084fd3 feat: stream S3 object bodies from Telegram parts 2026-07-07 04:25:05 +07:00
asepharyana 5253fb384d fix: verify presigned S3 URLs against public host 2026-07-07 04:21:01 +07:00
asepharyana 85aa54592a feat: add S3 byte range parser 2026-07-07 04:14:10 +07:00
asepharyana 2bf0fa5af5 docs: plan S3 compatibility completion 2026-07-07 04:03:11 +07:00
asepharyana dbdc5ed1ee docs: design remaining S3 compatibility fixes 2026-07-07 03:56:40 +07:00
asepharyana 9a48fbf227 fix: satisfy deploy lint gate for S3 compatibility work
- Apply Biome organize-import/formatting fixes across changed S3 files
- Replace remaining string concatenations with template literals for lint
- Make home page inline handlers explicit via window.* and add button types
- Clean S3 auth lint issues with dot-property access and optional chaining
- Keep GetObject proxy and production/S3 SDK tests passing

Verification:
- bun run lint (0 errors, 1 CSS specificity warning)
- S3_SECRET_KEY=<env> bun test test/production-e2e.test.ts (29 pass)
- S3_SECRET_KEY=<env> bun test test/s3-sdk.test.ts (20 pass)
- bun test test/s3-auth.test.ts (5 pass)
2026-07-07 03:07:51 +07:00
asepharyana 06f93e30f6 feat: proxy GetObject from Telegram CDN for real S3 client compatibility
- Add proxyS3Get config (PROXY_S3_GET env, default true) to env.ts
- Proxy handleGetObject and handleGetMultipartObject: fetch from Telegram
  CDN and return 200 with streaming body instead of 302 redirect
- Real S3 clients (AWS SDK v3) expect 200+body on GetObject, not redirect
- Legacy 302 redirect path preserved when proxyS3Get=false
- Updated production-e2e: GetObject asserts 200 with body content
- Updated s3-sdk.test.ts: GetObject asserts 200 with body (removed try/catch)
- Cleaned up unused multipart imports in s3-sdk.test.ts

All 49 tests pass (29 production-e2e + 20 s3-sdk).
2026-07-07 02:54:51 +07:00
asepharyana f88ac1a124 test: add real AWS SDK v3 S3 compatibility tests (20 tests)
Tests use @aws-sdk/client-s3 against https://upload.asepharyana.my.id
with forcePathStyle=true. Covers all standard S3 operations:
ListBuckets, CreateBucket, HeadBucket, DeleteBucket, PutObject,
GetObject, HeadObject, ListObjectsV1/V2 (prefix, delimiter),
CopyObject, DeleteObject, DeleteObjects (batch), error handling.

Multipart is excluded from SDK tests — the SDK adds amz-sdk-* /
x-amz-user-agent to signed headers, which can differ between
signing time and the actual request through Cloudflare, causing
403 SignatureDoesNotMatch. Multipart is verified via manual
SigV4 signing in test/production-e2e.test.ts.
2026-07-07 01:59:17 +07:00
asepharyana 57f2740df9 fix: production bugs + comprehensive production e2e tests
Fixes:
- RowList bug: postgres.js returns array directly, not {rows}. Fix in
  buckets.ts, files-ext.ts, multipart.ts (3 files, 8 functions)
- S3 ListBuckets routing: GET / was intercepted by handleHome route
- Presigned URL detection: isS3Request() only checked Authorization header
- FK constraint on bucket delete: cascade-delete files & multipart rows first
- docker-compose.yml: pass S3_ACCESS_KEY / S3_SECRET_KEY to container
- Dockerfile: copy home.html to runner stage for handleHome

Tests:
- test/production-e2e.test.ts: 29 tests (11 Web API + 18 S3 SigV4)
  All pass against https://upload.asepharyana.my.id
- Creates and cleans up real buckets/objects on production
2026-07-07 00:49:45 +07:00
asepharyana b731ddd7e1 fix: sequential test execution to avoid mock pollution between files 2026-07-06 22:50:59 +07:00
asepharyana 245138e23a Refactor code for improved readability and consistency
- Updated formatting in web-api.ts for better alignment and readability.
- Enhanced XML builders in xml.ts for clearer structure and maintainability.
- Improved test cases in s3-auth.test.ts and s3-operations.test.ts for better clarity and consistency.
- Refactored mock data in web-api.test.ts for improved readability.
2026-07-06 22:36:15 +07:00
asepharyana 2d3d7891a0 refactor: auto-run DB migration at startup, robust schema.sql path resolution 2026-07-06 22:32:06 +07:00
asepharyana 0ad783b794 fix: PutObject raw body, presigned URL auth guard, DeleteObjects tracking, home.ts path 2026-07-06 21:38:17 +07:00
asepharyana e175a27512 test: add S3 auth, operations, and web API tests 2026-07-06 21:15:26 +07:00
asepharyana 5e38f80267 feat: add web file manager UI and S3 catch-all route 2026-07-06 20:56:55 +07:00
asepharyana a4f902a37b feat: add JSON v1 web API for S3 management UI 2026-07-06 16:10:17 +07:00
asepharyana dd61e7e0fc feat: implement S3 protocol dispatcher with bucket, object, listing, and multipart operations 2026-07-06 15:52:42 +07:00
asepharyana 084ff3fb9d fix: add region validation, fix presigned URL canonical request (add method, fix headers), normalizeUri decode 2026-07-06 15:31:06 +07:00