Deploy FileDrop / deploy (push) Failing after 14m18s
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The empty-segment skip in normalizeUri (introduced in round 1 fix)
was stripping trailing slashes from canonical URIs, e.g. /bucket/
became /bucket. The AWS SDK signs with the trailing slash intact, so
signatures never matched for any S3 operation with a body.
The fix: only skip '.' segments (dot-segment removal per RFC 3986),
preserve all other segments including empty ones from trailing
slashes and double slashes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- Content-MD5 no longer loads entire file via arrayBuffer() — MD5 computed
incrementally in streamBodyToTemp alongside SHA-256 (fixes OOM for GB files)
HIGH:
- Add 120s timeout to Telegraf API calls via Promise.race in executeWithBotRetry
(prevents queue slot exhaustion from hung Telegram connections)
- Add queue size limit (1000 pending max) — reject new tasks when full
- Add graceful shutdown drain — waitForQueue with 30s timeout before exit
- Fix temp file leak when findFileByBucketAndKey throws (wrap in try-catch)
- Fix createReadStream fd leak — destroy stream on forwardToStorage error
- writer.end() wrapped in silent try-catch to prevent error swallowing
- writer.end() result ignored, writerFailed flag prevents double-end
MEDIUM:
- Remove 'retry after' from isTransientError patterns to stop double-retry
layering (was causing up to 96 bot attempts per chunk)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- SigV4 canonical request used sha256Hex('') instead of x-amz-content-sha256
header value — every PUT/POST with body would fail 403. Now uses the
signed header value for canonical request, verifyBodyHash after streaming
for integrity.
HIGH:
- Add 30s AbortSignal.timeout to all Telegram CDN fetches in object-stream.ts
(previously could hang indefinitely, exhausting connection pool)
MEDIUM:
- Content-MD5 validation: compute and compare when header is present
- Content-Length validation: reject if actual body size != header
- max-keys=0 clamping: enforce minimum of 1 per S3 spec
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
HIGH severity fixes:
- H1: Bot token leak via 302 redirect — always proxy S3 GETs
- H2: PUT TOCTOU race — add unique partial index (bucket_id, s3_key) WHERE NOT deleted
- H3: GET/HEAD ignore conditional headers (If-Match, If-None-Match, etc.)
- H4: Body payload hash not verified — add verifyBodyHash() post-stream check
- H5: Header-based auth has no expiry check — add 15-min clock skew window
- H7: Multipart abort does not delete parts — DELETE before UPDATE status
- H8: CompleteMultipartUpload skips part number & etag verification
- H9: XML regex fails on keys containing < — use non-greedy [\s\S]*?
- H10: Path-style vs virtual-hosted key decode mismatch
MEDIUM severity fixes:
- M1: Add Date header fallback for x-amz-date
- M2/M3: Validate service/termination in credential scope
- M4: Temp file leak when forwardToStorage throws in handleUploadPart
- M5: Multipart key consistency check (s3Key matches URL)
- M7: Use stored content-type from multipart initiate
- M9: Copy conditional headers skip when fileHash is null
- M11: Add 1000-key limit on DeleteObjects
- M13: Stricter bucket name validation (no .., no IP format)
- M14: NaN partNumber bypasses validation
LOW fixes:
- normalizeUri: dot-segment removal per RFC 3986
- localeCompare -> byte-order comparison in canonical query string
- Validate host in signed headers
- Server: AmazonS3 header on all responses
- x-amz-id-2 separate from x-amz-request-id
- IPv6 handling in stripPort
- Quiet element whitespace tolerance in XML parser
- content-type: application/xml on empty 2xx responses
- Duplicate interfaces/s3/ -> re-exports from utils/s3/
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Replace unsafe 'Function' type in test with ITelegramService interface
- Biome auto-fix formatting and import sorting across 8 files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Deleted `upload.ts` and `web-api.ts` routes, consolidating logic into dedicated controllers.
- Updated import paths in tests to reflect new controller structure.
- Refactored Telegram API utilities to utilize a bot pool for improved bot management and error handling.
- Enhanced environment variable tests to ensure additional bot tokens are correctly populated.
- Adjusted S3 bucket configuration tests to align with new controller imports.
- Updated Telegram queue implementation to reflect new infrastructure organization.
- Remove rate limiting from all S3 endpoints (used by Docker registry
for concurrent blob pushes — 429 would abort the entire push).
- Add retry with exponential backoff in botPool.forwardToStorage for
transient Telegram errors (network timeouts, 5xx, socket issues).
- Add retry with exponential backoff in botPool.getFileInfo per bot.
- Introduce isTransientError() pattern matcher covering ~20 transient
error signatures.
- Fix temp file leak in handlePutObject when storeFileFromTemp throws.
- Fix pre-existing missing botPool namespace on getFileInfo call in
handleGetMultipartObject.
- Fix route handler return type in PUT handler.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Critical fixes for S3 Docker registry backend:
- Stream PutObject body to temp file instead of req.arrayBuffer()
- O(1) memory usage regardless of file size
- SHA-256 hash computed while streaming
- Stream UploadPart body similarly
- Also fixes: size check after streaming, not before
- Add 30s timeout to Telegram CDN chunk fetches (object-stream.ts)
- Prevents hanging on stalled CDN connections
- Add rate limiting to S3 API routes (100 req/60s window)
- Prevents resource exhaustion from concurrent layer pushes
- Add comprehensive test suite (10 tests):
- Streaming verification (no arrayBuffer in PUT path)
- Multi-MB body streaming safety
- Empty body edge case
- Concurrent upload isolation
- Timeout signal presence
- Rate limit route coverage
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Fix infrastructure imports: chunked-storage uses new path for shared/utils and interfaces/s3
- Fix health-controller: imports from infrastructure/persistence/drizzle instead of old db/
- Fix routes/index.ts: imports from new interfaces/s3 and middleware paths
- Marked Telegram-specific types in shared/utils/file.ts as future extraction
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude