Commit Graph
41 Commits
Author SHA1 Message Date
Claude af160e0f33 fix: audit S3 protocol — 15+ security & correctness fixes
Deploy FileDrop / deploy (push) Successful in 43s
HIGH severity fixes:
- H1: Bot token leak via 302 redirect — always proxy S3 GETs
- H2: PUT TOCTOU race — add unique partial index (bucket_id, s3_key) WHERE NOT deleted
- H3: GET/HEAD ignore conditional headers (If-Match, If-None-Match, etc.)
- H4: Body payload hash not verified — add verifyBodyHash() post-stream check
- H5: Header-based auth has no expiry check — add 15-min clock skew window
- H7: Multipart abort does not delete parts — DELETE before UPDATE status
- H8: CompleteMultipartUpload skips part number & etag verification
- H9: XML regex fails on keys containing < — use non-greedy [\s\S]*?
- H10: Path-style vs virtual-hosted key decode mismatch

MEDIUM severity fixes:
- M1: Add Date header fallback for x-amz-date
- M2/M3: Validate service/termination in credential scope
- M4: Temp file leak when forwardToStorage throws in handleUploadPart
- M5: Multipart key consistency check (s3Key matches URL)
- M7: Use stored content-type from multipart initiate
- M9: Copy conditional headers skip when fileHash is null
- M11: Add 1000-key limit on DeleteObjects
- M13: Stricter bucket name validation (no .., no IP format)
- M14: NaN partNumber bypasses validation

LOW fixes:
- normalizeUri: dot-segment removal per RFC 3986
- localeCompare -> byte-order comparison in canonical query string
- Validate host in signed headers
- Server: AmazonS3 header on all responses
- x-amz-id-2 separate from x-amz-request-id
- IPv6 handling in stripPort
- Quiet element whitespace tolerance in XML parser
- content-type: application/xml on empty 2xx responses
- Duplicate interfaces/s3/ -> re-exports from utils/s3/

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 08:02:28 +07:00
Claude f5d56f52d4 chore: fix lint errors — noBannedTypes, import ordering, formatting
Deploy FileDrop / deploy (push) Successful in 42s
- Replace unsafe 'Function' type in test with ITelegramService interface
- Biome auto-fix formatting and import sorting across 8 files

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-29 07:30:31 +07:00
Claude ea87397801 refactor: remove upload and web-api routes, migrate to new controller structure
Deploy FileDrop / deploy (push) Failing after 12s
- Deleted `upload.ts` and `web-api.ts` routes, consolidating logic into dedicated controllers.
- Updated import paths in tests to reflect new controller structure.
- Refactored Telegram API utilities to utilize a bot pool for improved bot management and error handling.
- Enhanced environment variable tests to ensure additional bot tokens are correctly populated.
- Adjusted S3 bucket configuration tests to align with new controller imports.
- Updated Telegram queue implementation to reflect new infrastructure organization.
2026-07-29 07:28:30 +07:00
Claude 667921b100 chore: fix lint errors — duplicate import, unused imports, formatting
Deploy FileDrop / deploy (push) Successful in 45s
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 20:09:42 +07:00
Claude 002492626b fix: secure SigV4, temp leaks, OOM risk, duplicate migration, and cache issues
Deploy FileDrop / deploy (push) Failing after 15s
Security fixes:
- SigV4 signature comparison now uses crypto.timingSafeEqual (timing attack fix)
  - AccessKey, region, and HMAC signature all timing-safe
- Presigned URL expiry capped at 7 days (AWS spec compliance)
- Removed duplicate migration import (dead code)

Memory & leak fixes:
- Temp file leak in createZip(): cleanup temp file on error in both utils/ and shared/utils/
- OOM risk in web-api/v1 upload: stream File to temp instead of arrayBuffer()
- Removed duplicate migration import at startup

Performance fixes:
- Removed file.arrayBuffer() -> Bun.write() pattern in web-api-controller (stream + hash)

Test improvements:
- All fixes verified: 74/75 tests pass (1 pre-existing env config test)
- S3 auth tests: 7/7 pass after timing-safe fix
2026-07-28 19:33:05 +07:00
asepharyana 340c12d671 feat: implement authentication routes with login, logout, and user info retrieval
feat: add S3 bucket versioning support and related XML response handling

refactor: rename temporary file paths from 'teleuploader' to 'filedrop' for consistency

fix: update Swagger documentation to reflect new API name and descriptions

test: add unit tests for authentication routes and utilities

test: implement end-to-end tests for S3 bucket configuration and versioning

chore: update environment variable defaults for new service name
2026-07-07 19:58:15 +07:00
asepharyana 144ebe6dd3 feat: implement chunked storage for Telegram file uploads and retrieval 2026-07-07 08:08:22 +07:00
asepharyana 8e6ccf2110 feat: S3 client compatibility — virtual-hosted style, CORS, presigned multi-method, ListMultipartUploads, edge case fixes
- Virtual-hosted bucket detection from Host header (extractS3BucketFromHost)
- S3 CORS headers + OPTIONS preflight + x-amz-id-2/HostId everywhere
- Presigned GET/PUT/HEAD/DELETE via centralized auth (no GET-only restriction)
- ListMultipartUploads with DB/xml helpers
- UploadPart partNumber range validation (1-10000)
- CompleteMultipartUpload ETag matching and ascending order validation
- DeleteObjects quiet mode support
- CopyObject URL-decode and conditional if-match/if-none-match
- encoding-type=url support in ListObjects V1/V2 XML
- Safe range-based prefix matching (replaces SQL LIKE)
- STREAMING-AWS4-HMAC-SHA256-PAYLOAD → 501 NotImplemented
- Traefik wildcard HostRegex for virtual-hosted style
- S3_VHOST_DOMAINS config env var
2026-07-07 05:39:23 +07:00
asepharyana ad45404132 feat: support ranged S3 GetObject responses
Wire createGetObjectResponse into single-part GetObject handler for
proper Range/Content-Range support (200, 206, 416). Update presigned
E2E test to require 200. Add SDK Range request test.
2026-07-07 04:30:04 +07:00
asepharyana 378a084fd3 feat: stream S3 object bodies from Telegram parts 2026-07-07 04:25:05 +07:00
asepharyana 5253fb384d fix: verify presigned S3 URLs against public host 2026-07-07 04:21:01 +07:00
asepharyana 85aa54592a feat: add S3 byte range parser 2026-07-07 04:14:10 +07:00
asepharyana 9a48fbf227 fix: satisfy deploy lint gate for S3 compatibility work
- Apply Biome organize-import/formatting fixes across changed S3 files
- Replace remaining string concatenations with template literals for lint
- Make home page inline handlers explicit via window.* and add button types
- Clean S3 auth lint issues with dot-property access and optional chaining
- Keep GetObject proxy and production/S3 SDK tests passing

Verification:
- bun run lint (0 errors, 1 CSS specificity warning)
- S3_SECRET_KEY=<env> bun test test/production-e2e.test.ts (29 pass)
- S3_SECRET_KEY=<env> bun test test/s3-sdk.test.ts (20 pass)
- bun test test/s3-auth.test.ts (5 pass)
2026-07-07 03:07:51 +07:00
asepharyana 245138e23a Refactor code for improved readability and consistency
- Updated formatting in web-api.ts for better alignment and readability.
- Enhanced XML builders in xml.ts for clearer structure and maintainability.
- Improved test cases in s3-auth.test.ts and s3-operations.test.ts for better clarity and consistency.
- Refactored mock data in web-api.test.ts for improved readability.
2026-07-06 22:36:15 +07:00
asepharyana 084ff3fb9d fix: add region validation, fix presigned URL canonical request (add method, fix headers), normalizeUri decode 2026-07-06 15:31:06 +07:00
asepharyana ee758aa939 feat: add S3 SigV4 auth verification and XML builders 2026-07-06 14:56:39 +07:00
asepharyana 52bd704d21 refactor: enhance file handling and metrics tracking; remove unused bot health tracker 2026-07-06 01:25:18 +07:00
asepharyana bbf420759b chore: update Dockerfile and docker-compose for improved deployment process; add deploy script 2026-07-05 22:06:52 +07:00
MythEclipse 20b622c4c2 fix(core): resolve wrong file_id error and zip stream memory leaks
- Refactored Telegram bot pool iteration to handle 'wrong file_id' properly by verifying against all active bots
- Upgraded TelegramFileInfo interface to track owner bot_token
- Patched MaxListenersExceededWarning memory leak in zip.ts stream writes with events.once and async iteration
2026-06-20 22:24:23 +07:00
MythEclipse 3ca5ab7983 refactor: update Telegram file handling to use new getFileInfo function and include bot token in cache 2026-05-29 16:53:55 +07:00
MythEclipse 5425f6d33d feat: enhance configuration and rate limiting
- Added new configuration options: trustProxy, uploadConcurrency, batchMaxItems, batchMaxSizeBytes, and maxRequestBodyBytes to AppConfig.
- Implemented utility functions for parsing environment variables and masking sensitive data.
- Updated rate limiting logic to use configurable window size and maximum requests per window.
- Introduced a middleware for rate limiting on specific routes.
- Refactored file handling routes to support streaming downloads instead of redirects.
- Improved error handling and response formatting in file routes.
- Added support for oversized request rejection based on Content-Length header.
- Updated Swagger documentation to reflect changes in API behavior and responses.
- Enhanced tests to cover new features and ensure proper functionality.
2026-05-29 03:33:39 +07:00
MythEclipse db5de1367d feat: extend file schema with archive metadata, implement batch upload processing, and add zip utilities for file handling 2026-05-22 00:51:49 +07:00
MythEclipse 56b929ffb1 feat: enhance file upload process with temporary file handling, improved error management, and metrics logging 2026-05-22 00:29:54 +07:00
MythEclipse fd5eb98586 feat: implement caching for file info, add rate limiting, and enhance upload handling with retry logic 2026-05-21 23:31:49 +07:00
MythEclipse 10c968cf01 feat: add file_hash column and related index to files table; refactor Telegram API utilities for improved file handling 2026-05-21 22:55:42 +07:00
MythEclipse 52340ee77d feat: enhance file upload handling with improved file detection, size limits, and response formatting 2026-05-21 22:31:55 +07:00
MythEclipse d58b2390ef refactor: remove concurrency limit from Telegram upload queue and enhance media group handling 2026-05-18 21:54:39 +07:00
MythEclipse b91b276214 feat: add support for batch uploads of media groups to Telegram with debounce handling 2026-05-18 21:31:20 +07:00
MythEclipse 0bc12651db fix: clean up logger info message formatting and remove unused import in rate limit tests 2026-05-18 21:19:38 +07:00
MythEclipse 825e969569 feat: implement multi-bot support for Telegram API with rate limit handling and add task queue for uploads 2026-05-18 21:18:16 +07:00
MythEclipse 721fa3db7d feat: implement auto-retry for Telegram API requests on error 429 and update documentation 2026-05-18 21:02:38 +07:00
MythEclipseandClaude Opus 4.7 525348b1f3 fix: make forwardToStorage fully dynamic based on fileType
- Replace forceDocument boolean with fileType string in forwardToStorage
- Dynamically call matching sendPhoto, sendAudio, sendVideo, sendVoice, sendAnimation, sendSticker, or sendDocument Telegraf API method
- Correctly extract uploaded file details based on what Telegram returned
- Fixes IMAGE_PROCESS_FAILED 500 error when uploading ogg audio, video, sticker, or voice notes via HTTP
- Update upload.ts, bot.ts, and test suites to match the new dynamic signature

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 19:52:46 +07:00
MythEclipseandClaude Opus 4.7 4e805689c1 feat: extend file type classification and implement computeHash
- Add sticker (webp) and video_note to classification mapping and file limits
- Handle unknown mime or caption fallback to document
- Implement computeHash utility using Bun.CryptoHasher for sha256 checksums
- Update and add comprehensive tests in file.test.ts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 19:38:12 +07:00
MythEclipse 3d2500ed7f fix: auto-append file extension based on magic bytes or mime type if missing 2026-05-18 09:20:03 +07:00
MythEclipse 91986d1d51 fix: upload binary files to Telegram with source payload 2026-05-18 08:18:55 +07:00
MythEclipse cf79a1f195 style: format and lint codebase using Biome v2 2026-05-18 07:29:01 +07:00
MythEclipse 3f4e697733 feat: migrate entire codebase to TypeScript 2026-05-18 07:27:06 +07:00
MythEclipseandClaude Opus 4.7 1e29433d5d feat: add Telegram API utilities
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 07:06:21 +07:00
MythEclipseandClaude Opus 4.7 b621a73ca9 feat: add file validation utilities
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 07:02:14 +07:00
MythEclipseandClaude Opus 4.7 db71f29288 feat: add in-memory rate limiter
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 07:01:32 +07:00
MythEclipseandClaude Opus 4.7 77032538e8 feat: add Winston logger setup
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-18 06:53:33 +07:00