- Now 6 total bots: 1 main + 5 additional
- Removed UPLOAD_CONCURRENCY from PRODUCTION_ENV
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Each bot has its own PQueue with concurrency=1
- selectBot() assigns uploads to least-loaded available bot
- 429 rate limits are tracked per-bot with cooldown timers
- Failed uploads retry on next available bot
- Removed global upload-queue.ts and uploadConcurrency config
- Updated ITelegramService interface
Effective concurrency derived from bot pool size. Chunked-storage
backpressure now uses botPool.getEffectiveConcurrency().
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add MAX_OUTER_RETRIES constant and transientAttempts counter for outer-loop retry
- Restore getFileInfo transient retry logging with bot identity and fileId
- Create test/bot-pool.test.ts with 4 tests for core BotPool behavior
- Add empty-bots guard in selectBot() returning null
- Add safety net comment and improved logging for outer 429 catch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each bot has its own PQueue (concurrency=1). Uploads are assigned to
the least-loaded available bot. On 429, the bot is marked rate-limited
and the upload retries on the next available bot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a setTimeout(0) microtask yield after Promise.race to ensure
the .finally() handler that removes promises from the inFlight
set has executed before the next backpressure check.
Also ensure parts array is sorted by partNumber after concurrent
uploads complete, since promises resolve in arbitrary order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Previously uploadFileInTelegramChunks awaited each chunk's upload
before reading the next, making all chunks sequential within a file.
Now chunks are uploaded concurrently using a managed Set of in-flight
promises with backpressure limiting (2x uploadConcurrency).
This means a single 1GB Docker layer split into 48MB chunks will
have up to 32 chunks uploading simultaneously, not one at a time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tested actual Telegram Bot API limit:
- 49MB ✅
- 50MB ❌ (413 Request Entity Too Large)
Set TELEGRAM_CHUNK_SIZE_BYTES=50331648 (48MB) for safety
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The createMultipartUpload function inserts content_type but the
database column was missing, causing 500 errors on every Gitea
Docker registry push (which uses multipart uploads for blob storage).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploy FileDrop / deploy (push) Failing after 14m18s
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The empty-segment skip in normalizeUri (introduced in round 1 fix)
was stripping trailing slashes from canonical URIs, e.g. /bucket/
became /bucket. The AWS SDK signs with the trailing slash intact, so
signatures never matched for any S3 operation with a body.
The fix: only skip '.' segments (dot-segment removal per RFC 3986),
preserve all other segments including empty ones from trailing
slashes and double slashes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SDK includes trailing slash in the canonical URI for bucket
operations (e.g. PUT /bucket-name/). My earlier 'fix' that stripped
trailing slashes broke SigV4 signature verification. The trailing
slash is intentional per AWS SigV4 — only dot-segments are removed,
not trailing slashes.
Re-verified with @smithy/signature-v4: path /bucket-name/ produces
the client signature, while /bucket-name does not match.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SigV4 canonical URI must not have trailing slash (except root '/').
Bun can receive paths with trailing slash from SDK, causing signature
mismatch for all bucket operations (CreateBucket, HeadBucket, etc.)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bun's '/' route only matches root path '/'. S3 SDK clients using
forcePathStyle:true send ALL requests to /{bucket}/{key} which never
matched any route → 404. Added '/*' catch-all that checks for S3
auth headers before dispatching.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- Content-MD5 no longer loads entire file via arrayBuffer() — MD5 computed
incrementally in streamBodyToTemp alongside SHA-256 (fixes OOM for GB files)
HIGH:
- Add 120s timeout to Telegraf API calls via Promise.race in executeWithBotRetry
(prevents queue slot exhaustion from hung Telegram connections)
- Add queue size limit (1000 pending max) — reject new tasks when full
- Add graceful shutdown drain — waitForQueue with 30s timeout before exit
- Fix temp file leak when findFileByBucketAndKey throws (wrap in try-catch)
- Fix createReadStream fd leak — destroy stream on forwardToStorage error
- writer.end() wrapped in silent try-catch to prevent error swallowing
- writer.end() result ignored, writerFailed flag prevents double-end
MEDIUM:
- Remove 'retry after' from isTransientError patterns to stop double-retry
layering (was causing up to 96 bot attempts per chunk)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- SigV4 canonical request used sha256Hex('') instead of x-amz-content-sha256
header value — every PUT/POST with body would fail 403. Now uses the
signed header value for canonical request, verifyBodyHash after streaming
for integrity.
HIGH:
- Add 30s AbortSignal.timeout to all Telegram CDN fetches in object-stream.ts
(previously could hang indefinitely, exhausting connection pool)
MEDIUM:
- Content-MD5 validation: compute and compare when header is present
- Content-Length validation: reject if actual body size != header
- max-keys=0 clamping: enforce minimum of 1 per S3 spec
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
HIGH severity fixes:
- H1: Bot token leak via 302 redirect — always proxy S3 GETs
- H2: PUT TOCTOU race — add unique partial index (bucket_id, s3_key) WHERE NOT deleted
- H3: GET/HEAD ignore conditional headers (If-Match, If-None-Match, etc.)
- H4: Body payload hash not verified — add verifyBodyHash() post-stream check
- H5: Header-based auth has no expiry check — add 15-min clock skew window
- H7: Multipart abort does not delete parts — DELETE before UPDATE status
- H8: CompleteMultipartUpload skips part number & etag verification
- H9: XML regex fails on keys containing < — use non-greedy [\s\S]*?
- H10: Path-style vs virtual-hosted key decode mismatch
MEDIUM severity fixes:
- M1: Add Date header fallback for x-amz-date
- M2/M3: Validate service/termination in credential scope
- M4: Temp file leak when forwardToStorage throws in handleUploadPart
- M5: Multipart key consistency check (s3Key matches URL)
- M7: Use stored content-type from multipart initiate
- M9: Copy conditional headers skip when fileHash is null
- M11: Add 1000-key limit on DeleteObjects
- M13: Stricter bucket name validation (no .., no IP format)
- M14: NaN partNumber bypasses validation
LOW fixes:
- normalizeUri: dot-segment removal per RFC 3986
- localeCompare -> byte-order comparison in canonical query string
- Validate host in signed headers
- Server: AmazonS3 header on all responses
- x-amz-id-2 separate from x-amz-request-id
- IPv6 handling in stripPort
- Quiet element whitespace tolerance in XML parser
- content-type: application/xml on empty 2xx responses
- Duplicate interfaces/s3/ -> re-exports from utils/s3/
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Replace unsafe 'Function' type in test with ITelegramService interface
- Biome auto-fix formatting and import sorting across 8 files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Deleted `upload.ts` and `web-api.ts` routes, consolidating logic into dedicated controllers.
- Updated import paths in tests to reflect new controller structure.
- Refactored Telegram API utilities to utilize a bot pool for improved bot management and error handling.
- Enhanced environment variable tests to ensure additional bot tokens are correctly populated.
- Adjusted S3 bucket configuration tests to align with new controller imports.
- Updated Telegram queue implementation to reflect new infrastructure organization.
- Remove rate limiting from all S3 endpoints (used by Docker registry
for concurrent blob pushes — 429 would abort the entire push).
- Add retry with exponential backoff in botPool.forwardToStorage for
transient Telegram errors (network timeouts, 5xx, socket issues).
- Add retry with exponential backoff in botPool.getFileInfo per bot.
- Introduce isTransientError() pattern matcher covering ~20 transient
error signatures.
- Fix temp file leak in handlePutObject when storeFileFromTemp throws.
- Fix pre-existing missing botPool namespace on getFileInfo call in
handleGetMultipartObject.
- Fix route handler return type in PUT handler.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Critical fixes for S3 Docker registry backend:
- Stream PutObject body to temp file instead of req.arrayBuffer()
- O(1) memory usage regardless of file size
- SHA-256 hash computed while streaming
- Stream UploadPart body similarly
- Also fixes: size check after streaming, not before
- Add 30s timeout to Telegram CDN chunk fetches (object-stream.ts)
- Prevents hanging on stalled CDN connections
- Add rate limiting to S3 API routes (100 req/60s window)
- Prevents resource exhaustion from concurrent layer pushes
- Add comprehensive test suite (10 tests):
- Streaming verification (no arrayBuffer in PUT path)
- Multi-MB body streaming safety
- Empty body edge case
- Concurrent upload isolation
- Timeout signal presence
- Rate limit route coverage
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Fix infrastructure imports: chunked-storage uses new path for shared/utils and interfaces/s3
- Fix health-controller: imports from infrastructure/persistence/drizzle instead of old db/
- Fix routes/index.ts: imports from new interfaces/s3 and middleware paths
- Marked Telegram-specific types in shared/utils/file.ts as future extraction
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Create auth.ts and rate-limit.ts middleware files in the interfaces layer
as part of the DDD/clean architecture restructure. Also add a config
re-export at src/interfaces/config/index.ts so the middleware can access
configuration through the interfaces layer boundary.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Create upload-file.ts use case with factory pattern supporting dedup, file type detection, size validation, and chunked/single storage strategies.
Create get-file.ts use case supporting redirect, chunked, and archive-entry retrieval strategies.
Create authenticate.ts use case with login, logout, and me operations.
All use cases use dependency injection and return typed DTOs.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add data-transfer-object interfaces for the application layer:
upload, file, bucket, S3, and auth domains.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude