Commit Graph
17 Commits
Author SHA1 Message Date
Claude adbf9b5efa refactor: remove global upload queue
Per-bot queues now handle concurrency internally.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 13:28:38 +07:00
Claude d8da2044b2 fix: S3→Telegram upload pipeline — OOM, queue limits, shutdown drain, timeouts
Deploy FileDrop / deploy (push) Successful in 46s
CRITICAL:
- Content-MD5 no longer loads entire file via arrayBuffer() — MD5 computed
  incrementally in streamBodyToTemp alongside SHA-256 (fixes OOM for GB files)

HIGH:
- Add 120s timeout to Telegraf API calls via Promise.race in executeWithBotRetry
  (prevents queue slot exhaustion from hung Telegram connections)
- Add queue size limit (1000 pending max) — reject new tasks when full
- Add graceful shutdown drain — waitForQueue with 30s timeout before exit
- Fix temp file leak when findFileByBucketAndKey throws (wrap in try-catch)
- Fix createReadStream fd leak — destroy stream on forwardToStorage error
- writer.end() wrapped in silent try-catch to prevent error swallowing
- writer.end() result ignored, writerFailed flag prevents double-end

MEDIUM:
- Remove 'retry after' from isTransientError patterns to stop double-retry
  layering (was causing up to 96 bot attempts per chunk)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:40:50 +07:00
Claude 667921b100 chore: fix lint errors — duplicate import, unused imports, formatting
Deploy FileDrop / deploy (push) Successful in 45s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:09:42 +07:00
Claude 002492626b fix: secure SigV4, temp leaks, OOM risk, duplicate migration, and cache issues
Deploy FileDrop / deploy (push) Failing after 15s
Security fixes:
- SigV4 signature comparison now uses crypto.timingSafeEqual (timing attack fix)
  - AccessKey, region, and HMAC signature all timing-safe
- Presigned URL expiry capped at 7 days (AWS spec compliance)
- Removed duplicate migration import (dead code)

Memory & leak fixes:
- Temp file leak in createZip(): cleanup temp file on error in both utils/ and shared/utils/
- OOM risk in web-api/v1 upload: stream File to temp instead of arrayBuffer()
- Removed duplicate migration import at startup

Performance fixes:
- Removed file.arrayBuffer() -> Bun.write() pattern in web-api-controller (stream + hash)

Test improvements:
- All fixes verified: 74/75 tests pass (1 pre-existing env config test)
- S3 auth tests: 7/7 pass after timing-safe fix
2026-07-28 19:33:05 +07:00
Claude 234ca7b14c feat: rewire entry point to new architecture
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 18:12:48 +07:00
asepharyana 44e887c7cd fix: keep public upload API unauthenticated
Deploy FileDrop / deploy (push) Successful in 37s
2026-07-08 01:59:41 +07:00
asepharyana 340c12d671 feat: implement authentication routes with login, logout, and user info retrieval
feat: add S3 bucket versioning support and related XML response handling

refactor: rename temporary file paths from 'teleuploader' to 'filedrop' for consistency

fix: update Swagger documentation to reflect new API name and descriptions

test: add unit tests for authentication routes and utilities

test: implement end-to-end tests for S3 bucket configuration and versioning

chore: update environment variable defaults for new service name
2026-07-07 19:58:15 +07:00
asepharyana 8e6ccf2110 feat: S3 client compatibility — virtual-hosted style, CORS, presigned multi-method, ListMultipartUploads, edge case fixes
- Virtual-hosted bucket detection from Host header (extractS3BucketFromHost)
- S3 CORS headers + OPTIONS preflight + x-amz-id-2/HostId everywhere
- Presigned GET/PUT/HEAD/DELETE via centralized auth (no GET-only restriction)
- ListMultipartUploads with DB/xml helpers
- UploadPart partNumber range validation (1-10000)
- CompleteMultipartUpload ETag matching and ascending order validation
- DeleteObjects quiet mode support
- CopyObject URL-decode and conditional if-match/if-none-match
- encoding-type=url support in ListObjects V1/V2 XML
- Safe range-based prefix matching (replaces SQL LIKE)
- STREAMING-AWS4-HMAC-SHA256-PAYLOAD → 501 NotImplemented
- Traefik wildcard HostRegex for virtual-hosted style
- S3_VHOST_DOMAINS config env var
2026-07-07 05:39:23 +07:00
asepharyana 9a48fbf227 fix: satisfy deploy lint gate for S3 compatibility work
- Apply Biome organize-import/formatting fixes across changed S3 files
- Replace remaining string concatenations with template literals for lint
- Make home page inline handlers explicit via window.* and add button types
- Clean S3 auth lint issues with dot-property access and optional chaining
- Keep GetObject proxy and production/S3 SDK tests passing

Verification:
- bun run lint (0 errors, 1 CSS specificity warning)
- S3_SECRET_KEY=<env> bun test test/production-e2e.test.ts (29 pass)
- S3_SECRET_KEY=<env> bun test test/s3-sdk.test.ts (20 pass)
- bun test test/s3-auth.test.ts (5 pass)
2026-07-07 03:07:51 +07:00
asepharyana 57f2740df9 fix: production bugs + comprehensive production e2e tests
Fixes:
- RowList bug: postgres.js returns array directly, not {rows}. Fix in
  buckets.ts, files-ext.ts, multipart.ts (3 files, 8 functions)
- S3 ListBuckets routing: GET / was intercepted by handleHome route
- Presigned URL detection: isS3Request() only checked Authorization header
- FK constraint on bucket delete: cascade-delete files & multipart rows first
- docker-compose.yml: pass S3_ACCESS_KEY / S3_SECRET_KEY to container
- Dockerfile: copy home.html to runner stage for handleHome

Tests:
- test/production-e2e.test.ts: 29 tests (11 Web API + 18 S3 SigV4)
  All pass against https://upload.asepharyana.my.id
- Creates and cleans up real buckets/objects on production
2026-07-07 00:49:45 +07:00
asepharyana 2d3d7891a0 refactor: auto-run DB migration at startup, robust schema.sql path resolution 2026-07-06 22:32:06 +07:00
asepharyana 5e38f80267 feat: add web file manager UI and S3 catch-all route 2026-07-06 20:56:55 +07:00
asepharyana 52bd704d21 refactor: enhance file handling and metrics tracking; remove unused bot health tracker 2026-07-06 01:25:18 +07:00
MythEclipse 5425f6d33d feat: enhance configuration and rate limiting
- Added new configuration options: trustProxy, uploadConcurrency, batchMaxItems, batchMaxSizeBytes, and maxRequestBodyBytes to AppConfig.
- Implemented utility functions for parsing environment variables and masking sensitive data.
- Updated rate limiting logic to use configurable window size and maximum requests per window.
- Introduced a middleware for rate limiting on specific routes.
- Refactored file handling routes to support streaming downloads instead of redirects.
- Improved error handling and response formatting in file routes.
- Added support for oversized request rejection based on Content-Length header.
- Updated Swagger documentation to reflect changes in API behavior and responses.
- Enhanced tests to cover new features and ensure proper functionality.
2026-05-29 03:33:39 +07:00
MythEclipse 6feca3564a feat: add Swagger OpenAPI documentation endpoints 2026-05-18 07:39:27 +07:00
MythEclipse cf79a1f195 style: format and lint codebase using Biome v2 2026-05-18 07:29:01 +07:00
MythEclipse 3f4e697733 feat: migrate entire codebase to TypeScript 2026-05-18 07:27:06 +07:00