Commit Graph
19 Commits
Author SHA1 Message Date
asepharyana 24dfb1c6b1 fix: fail-fast guard TELEGRAM_CHUNK_SIZE_BYTES <= 19MB (Telegram getFile limit 20MB)
Build & Deploy (Nix) / build-and-deploy (push) Successful in 56s
Chunk parts > 19MB are stored to Telegram but getFile cannot resolve files over 20MB ('Bad Request: file is too big'), making every part undownloadable (prod bug 2026-08-01: 48MB chunk -> download 500).

- src/env.ts: reject TELEGRAM_CHUNK_SIZE_BYTES > 19922944 at startup (log error + throw), default changed 20MB -> 19MB
- src/shared/utils/validation.ts: TELEGRAM_CHUNK_SIZE_MAX_BYTES constant; asSafeChunkSize now enforces the max at runtime (covers S3 multipart parts too)
- test/env.test.ts: unit tests + subprocess fail-fast tests (48MB rejected, 19MB accepted)
- test/helpers/setup-env.ts: pin safe chunk size so a stale .env can't break the suite
- .env.example + CLAUDE.md: document the 20MB getFile limit
2026-08-01 13:06:42 +07:00
Claude 9a4853a484 fix: remove UploadBatcher crash window, make bot concurrency configurable, fix all test import paths
Deploy FileDrop / deploy (push) Successful in 48s
- Removed UploadBatcher (src/infrastructure/telegram/upload-batcher.ts + DI):
  pending uploads no longer lost on crash, files sent directly to Telegram
- Changed upload-controller to use Bun.file().stream() instead of createReadStream
- Made PER_BOT_CONCURRENCY configurable via TELEGRAM_BOT_CONCURRENCY env
- Fixed 18 test files with updated import paths and mock shapes
- Updated package.json test script: telegramQueue.test.ts → bot-pool.test.ts
- Build, lint, and test suite all pass
2026-07-29 17:06:35 +07:00
Claude 4c216b1d9f refactor: full DDD + Clean Architecture refactor
Deploy FileDrop / deploy (push) Successful in 48s
- Hapus src/utils/ (17 files) + src/db/ (8 files) dead code
- Absorb 7 re-export stubs → real impl di lokasi DDD
- Buat src/infrastructure/di.ts (DI container)
- Rewrite 5 controllers pakai repository/DI
- Fix shared/utils imports, env.ts, routes, index.ts
- Update package.json build path migrate
- Lint clean, build clean
2026-07-29 15:41:50 +07:00
Claude 1484d5265d refactor: merge BOT_TOKEN + ADDITIONAL_BOT_TOKENS into single BOT_TOKENS env + speed audit
Deploy FileDrop / deploy (push) Failing after 19s
BOT_TOKENS env:
- Single BOT_TOKENS env var (comma-separated) replaces BOT_TOKEN + ADDITIONAL_BOT_TOKENS
- Backward compat: falls back to BOT_TOKEN + ADDITIONAL_BOT_TOKENS if BOT_TOKENS unset
- Config exposes botTokens: string[] instead of botToken + additionalBotTokens
- Updated env.ts, bot-pool.ts, docker-compose.yml, .env.example, CLAUDE.md, all tests

Speed audit (S3 -> Telegram upload flow):
- Hoisted 5 dynamic await import('../../../db/index') to top-level static imports
  in s3-controller.ts (3x) and web-api-controller.ts (2x)
  -> saves module resolution + async overhead on every upload
- Removed stale UPLOAD_CONCURRENCY env from docker-compose.yml
  (already removed from env.ts in prior refactor)

Upload flow is already concurrent:
- streamBodyToTemp() uses Bun.file(path).writer() — O(1) memory, safe for multi-GB blobs
- utils/chunked-storage.ts reads chunks serially but uploads concurrently with
  inFlight backpressure at effectiveConcurrency * 2 (= 16 with 8 bots)
- bot-pool.ts: per-bot PQueue(concurrency=1), 8 bots = 8 concurrent uploads per file,
  selectBot() picks least-loaded, 429 detection + inner+outer retry loops
- TELEGRAM_API_TIMEOUT_MS=120s — ample for 48MB chunks
- Infrastructure chunked-storage.ts (DI-based, dead code) has serial upload trap —
  noted for future cleanup
2026-07-29 15:03:02 +07:00
Claude d0de4de2d5 refactor: remove uploadConcurrency from config
Effective concurrency derived from bot pool size. Chunked-storage
backpressure now uses botPool.getEffectiveConcurrency().

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 13:30:49 +07:00
Claude ea87397801 refactor: remove upload and web-api routes, migrate to new controller structure
Deploy FileDrop / deploy (push) Failing after 12s
- Deleted `upload.ts` and `web-api.ts` routes, consolidating logic into dedicated controllers.
- Updated import paths in tests to reflect new controller structure.
- Refactored Telegram API utilities to utilize a bot pool for improved bot management and error handling.
- Enhanced environment variable tests to ensure additional bot tokens are correctly populated.
- Adjusted S3 bucket configuration tests to align with new controller imports.
- Updated Telegram queue implementation to reflect new infrastructure organization.
2026-07-29 07:28:30 +07:00
asepharyana 340c12d671 feat: implement authentication routes with login, logout, and user info retrieval
feat: add S3 bucket versioning support and related XML response handling

refactor: rename temporary file paths from 'teleuploader' to 'filedrop' for consistency

fix: update Swagger documentation to reflect new API name and descriptions

test: add unit tests for authentication routes and utilities

test: implement end-to-end tests for S3 bucket configuration and versioning

chore: update environment variable defaults for new service name
2026-07-07 19:58:15 +07:00
asepharyana 144ebe6dd3 feat: implement chunked storage for Telegram file uploads and retrieval 2026-07-07 08:08:22 +07:00
asepharyana fbfff3d9ec fix: normalize web object sizes 2026-07-07 06:27:18 +07:00
asepharyana 34962eb08a chore: update S3 vhost domains to include asepharana.web.id root/wildcard 2026-07-07 05:47:44 +07:00
asepharyana 8e6ccf2110 feat: S3 client compatibility — virtual-hosted style, CORS, presigned multi-method, ListMultipartUploads, edge case fixes
- Virtual-hosted bucket detection from Host header (extractS3BucketFromHost)
- S3 CORS headers + OPTIONS preflight + x-amz-id-2/HostId everywhere
- Presigned GET/PUT/HEAD/DELETE via centralized auth (no GET-only restriction)
- ListMultipartUploads with DB/xml helpers
- UploadPart partNumber range validation (1-10000)
- CompleteMultipartUpload ETag matching and ascending order validation
- DeleteObjects quiet mode support
- CopyObject URL-decode and conditional if-match/if-none-match
- encoding-type=url support in ListObjects V1/V2 XML
- Safe range-based prefix matching (replaces SQL LIKE)
- STREAMING-AWS4-HMAC-SHA256-PAYLOAD → 501 NotImplemented
- Traefik wildcard HostRegex for virtual-hosted style
- S3_VHOST_DOMAINS config env var
2026-07-07 05:39:23 +07:00
asepharyana 06f93e30f6 feat: proxy GetObject from Telegram CDN for real S3 client compatibility
- Add proxyS3Get config (PROXY_S3_GET env, default true) to env.ts
- Proxy handleGetObject and handleGetMultipartObject: fetch from Telegram
  CDN and return 200 with streaming body instead of 302 redirect
- Real S3 clients (AWS SDK v3) expect 200+body on GetObject, not redirect
- Legacy 302 redirect path preserved when proxyS3Get=false
- Updated production-e2e: GetObject asserts 200 with body content
- Updated s3-sdk.test.ts: GetObject asserts 200 with body (removed try/catch)
- Cleaned up unused multipart imports in s3-sdk.test.ts

All 49 tests pass (29 production-e2e + 20 s3-sdk).
2026-07-07 02:54:51 +07:00
asepharyana e37fb00295 feat: add S3 env config, bucket and multipart DB schema 2026-07-06 13:59:03 +07:00
asepharyana bbf420759b chore: update Dockerfile and docker-compose for improved deployment process; add deploy script 2026-07-05 22:06:52 +07:00
MythEclipse 5425f6d33d feat: enhance configuration and rate limiting
- Added new configuration options: trustProxy, uploadConcurrency, batchMaxItems, batchMaxSizeBytes, and maxRequestBodyBytes to AppConfig.
- Implemented utility functions for parsing environment variables and masking sensitive data.
- Updated rate limiting logic to use configurable window size and maximum requests per window.
- Introduced a middleware for rate limiting on specific routes.
- Refactored file handling routes to support streaming downloads instead of redirects.
- Improved error handling and response formatting in file routes.
- Added support for oversized request rejection based on Content-Length header.
- Updated Swagger documentation to reflect changes in API behavior and responses.
- Enhanced tests to cover new features and ensure proper functionality.
2026-05-29 03:33:39 +07:00
MythEclipse 0bc12651db fix: clean up logger info message formatting and remove unused import in rate limit tests 2026-05-18 21:19:38 +07:00
MythEclipse 825e969569 feat: implement multi-bot support for Telegram API with rate limit handling and add task queue for uploads 2026-05-18 21:18:16 +07:00
MythEclipse cf79a1f195 style: format and lint codebase using Biome v2 2026-05-18 07:29:01 +07:00
MythEclipse 3f4e697733 feat: migrate entire codebase to TypeScript 2026-05-18 07:27:06 +07:00