Build & Deploy (Nix) / build-and-deploy (push) Failing after 46s
Diagnosing deploy failure: systemctl unavailable inside runner container.
Adding inspect step to understand available mounts and access mechanisms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build & Deploy (Nix) / build-and-deploy (push) Failing after 47s
The deploy step runs with 'sudo' which resets PATH, so nix-env is
not found. Use explicit path to the nix-env binary.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build & Deploy (Nix) / build-and-deploy (push) Failing after 52s
Re-instate Determinate Systems installer with correct flags:
- install --no-confirm (not --no-daemon which it doesn't support)
- Nix installs to /nix/var/nix/profiles/default/bin
- Source daemon profile in build step
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build & Deploy (Nix) / build-and-deploy (push) Failing after 11s
The Determinate Systems installer doesn't support --no-daemon.
Switch to the official Nix installer which has a well-documented
--no-daemon flag suitable for container/CI use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Build & Deploy (Nix) / build-and-deploy (push) Failing after 7s
The Gitea Actions runner image (docker.gitea.com/runner-images:ubuntu-latest)
does not have Nix pre-installed, causing 'nix: command not found' on the
build step.
Add an 'Install Nix' step using the Determinate Systems installer
(--no-daemon mode since the runner runs in a container).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- flake.nix with Bun build from nixpkgs (1.3.x)
- flake.lock pinned inputs
- .gitignore result symlink from nix build
Build: nix build .#teleuploader --impure --option sandbox false
Until CI/CD with binary cache is set up
- Replaces createReadStream / bare ReadableStream with
Readable.from(Bun.file(path).stream()) — works in both
test (Bun.write + Bun.file) and production (Node Readable)
- Reverts writeBufferToTemp back to Bun.write
- getFileType: only classify image/jpeg as 'photo'; png/gif/webp → 'document'
(Telegram Bot API rejects non-JPEG for sendPhoto)
- Dockerfile: copy home.html to dist/ instead of root (import.meta.dir = dist/)
- Update test assertion for getFileType(image/png) → 'document'
- Removed UploadBatcher (src/infrastructure/telegram/upload-batcher.ts + DI):
pending uploads no longer lost on crash, files sent directly to Telegram
- Changed upload-controller to use Bun.file().stream() instead of createReadStream
- Made PER_BOT_CONCURRENCY configurable via TELEGRAM_BOT_CONCURRENCY env
- Fixed 18 test files with updated import paths and mock shapes
- Updated package.json test script: telegramQueue.test.ts → bot-pool.test.ts
- Build, lint, and test suite all pass
- Now 6 total bots: 1 main + 5 additional
- Removed UPLOAD_CONCURRENCY from PRODUCTION_ENV
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Each bot has its own PQueue with concurrency=1
- selectBot() assigns uploads to least-loaded available bot
- 429 rate limits are tracked per-bot with cooldown timers
- Failed uploads retry on next available bot
- Removed global upload-queue.ts and uploadConcurrency config
- Updated ITelegramService interface
Effective concurrency derived from bot pool size. Chunked-storage
backpressure now uses botPool.getEffectiveConcurrency().
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Add MAX_OUTER_RETRIES constant and transientAttempts counter for outer-loop retry
- Restore getFileInfo transient retry logging with bot identity and fileId
- Create test/bot-pool.test.ts with 4 tests for core BotPool behavior
- Add empty-bots guard in selectBot() returning null
- Add safety net comment and improved logging for outer 429 catch
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each bot has its own PQueue (concurrency=1). Uploads are assigned to
the least-loaded available bot. On 429, the bot is marked rate-limited
and the upload retries on the next available bot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adds a setTimeout(0) microtask yield after Promise.race to ensure
the .finally() handler that removes promises from the inFlight
set has executed before the next backpressure check.
Also ensure parts array is sorted by partNumber after concurrent
uploads complete, since promises resolve in arbitrary order.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Previously uploadFileInTelegramChunks awaited each chunk's upload
before reading the next, making all chunks sequential within a file.
Now chunks are uploaded concurrently using a managed Set of in-flight
promises with backpressure limiting (2x uploadConcurrency).
This means a single 1GB Docker layer split into 48MB chunks will
have up to 32 chunks uploading simultaneously, not one at a time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tested actual Telegram Bot API limit:
- 49MB ✅
- 50MB ❌ (413 Request Entity Too Large)
Set TELEGRAM_CHUNK_SIZE_BYTES=50331648 (48MB) for safety
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The createMultipartUpload function inserts content_type but the
database column was missing, causing 500 errors on every Gitea
Docker registry push (which uses multipart uploads for blob storage).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deploy FileDrop / deploy (push) Failing after 14m18s
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SDK requires ETag header in 304 responses. Without it, the SDK
throws UnknownError despite receiving a valid 304 status code.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The empty-segment skip in normalizeUri (introduced in round 1 fix)
was stripping trailing slashes from canonical URIs, e.g. /bucket/
became /bucket. The AWS SDK signs with the trailing slash intact, so
signatures never matched for any S3 operation with a body.
The fix: only skip '.' segments (dot-segment removal per RFC 3986),
preserve all other segments including empty ones from trailing
slashes and double slashes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SDK includes trailing slash in the canonical URI for bucket
operations (e.g. PUT /bucket-name/). My earlier 'fix' that stripped
trailing slashes broke SigV4 signature verification. The trailing
slash is intentional per AWS SigV4 — only dot-segments are removed,
not trailing slashes.
Re-verified with @smithy/signature-v4: path /bucket-name/ produces
the client signature, while /bucket-name does not match.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
AWS SigV4 canonical URI must not have trailing slash (except root '/').
Bun can receive paths with trailing slash from SDK, causing signature
mismatch for all bucket operations (CreateBucket, HeadBucket, etc.)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bun's '/' route only matches root path '/'. S3 SDK clients using
forcePathStyle:true send ALL requests to /{bucket}/{key} which never
matched any route → 404. Added '/*' catch-all that checks for S3
auth headers before dispatching.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- Content-MD5 no longer loads entire file via arrayBuffer() — MD5 computed
incrementally in streamBodyToTemp alongside SHA-256 (fixes OOM for GB files)
HIGH:
- Add 120s timeout to Telegraf API calls via Promise.race in executeWithBotRetry
(prevents queue slot exhaustion from hung Telegram connections)
- Add queue size limit (1000 pending max) — reject new tasks when full
- Add graceful shutdown drain — waitForQueue with 30s timeout before exit
- Fix temp file leak when findFileByBucketAndKey throws (wrap in try-catch)
- Fix createReadStream fd leak — destroy stream on forwardToStorage error
- writer.end() wrapped in silent try-catch to prevent error swallowing
- writer.end() result ignored, writerFailed flag prevents double-end
MEDIUM:
- Remove 'retry after' from isTransientError patterns to stop double-retry
layering (was causing up to 96 bot attempts per chunk)
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
CRITICAL:
- SigV4 canonical request used sha256Hex('') instead of x-amz-content-sha256
header value — every PUT/POST with body would fail 403. Now uses the
signed header value for canonical request, verifyBodyHash after streaming
for integrity.
HIGH:
- Add 30s AbortSignal.timeout to all Telegram CDN fetches in object-stream.ts
(previously could hang indefinitely, exhausting connection pool)
MEDIUM:
- Content-MD5 validation: compute and compare when header is present
- Content-Length validation: reject if actual body size != header
- max-keys=0 clamping: enforce minimum of 1 per S3 spec
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
HIGH severity fixes:
- H1: Bot token leak via 302 redirect — always proxy S3 GETs
- H2: PUT TOCTOU race — add unique partial index (bucket_id, s3_key) WHERE NOT deleted
- H3: GET/HEAD ignore conditional headers (If-Match, If-None-Match, etc.)
- H4: Body payload hash not verified — add verifyBodyHash() post-stream check
- H5: Header-based auth has no expiry check — add 15-min clock skew window
- H7: Multipart abort does not delete parts — DELETE before UPDATE status
- H8: CompleteMultipartUpload skips part number & etag verification
- H9: XML regex fails on keys containing < — use non-greedy [\s\S]*?
- H10: Path-style vs virtual-hosted key decode mismatch
MEDIUM severity fixes:
- M1: Add Date header fallback for x-amz-date
- M2/M3: Validate service/termination in credential scope
- M4: Temp file leak when forwardToStorage throws in handleUploadPart
- M5: Multipart key consistency check (s3Key matches URL)
- M7: Use stored content-type from multipart initiate
- M9: Copy conditional headers skip when fileHash is null
- M11: Add 1000-key limit on DeleteObjects
- M13: Stricter bucket name validation (no .., no IP format)
- M14: NaN partNumber bypasses validation
LOW fixes:
- normalizeUri: dot-segment removal per RFC 3986
- localeCompare -> byte-order comparison in canonical query string
- Validate host in signed headers
- Server: AmazonS3 header on all responses
- x-amz-id-2 separate from x-amz-request-id
- IPv6 handling in stripPort
- Quiet element whitespace tolerance in XML parser
- content-type: application/xml on empty 2xx responses
- Duplicate interfaces/s3/ -> re-exports from utils/s3/
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Replace unsafe 'Function' type in test with ITelegramService interface
- Biome auto-fix formatting and import sorting across 8 files
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Deleted `upload.ts` and `web-api.ts` routes, consolidating logic into dedicated controllers.
- Updated import paths in tests to reflect new controller structure.
- Refactored Telegram API utilities to utilize a bot pool for improved bot management and error handling.
- Enhanced environment variable tests to ensure additional bot tokens are correctly populated.
- Adjusted S3 bucket configuration tests to align with new controller imports.
- Updated Telegram queue implementation to reflect new infrastructure organization.
Claude