fix: S3→Telegram upload pipeline — OOM, queue limits, shutdown drain, timeouts
Deploy FileDrop / deploy (push) Successful in 46s

CRITICAL:
- Content-MD5 no longer loads entire file via arrayBuffer() — MD5 computed
  incrementally in streamBodyToTemp alongside SHA-256 (fixes OOM for GB files)

HIGH:
- Add 120s timeout to Telegraf API calls via Promise.race in executeWithBotRetry
  (prevents queue slot exhaustion from hung Telegram connections)
- Add queue size limit (1000 pending max) — reject new tasks when full
- Add graceful shutdown drain — waitForQueue with 30s timeout before exit
- Fix temp file leak when findFileByBucketAndKey throws (wrap in try-catch)
- Fix createReadStream fd leak — destroy stream on forwardToStorage error
- writer.end() wrapped in silent try-catch to prevent error swallowing
- writer.end() result ignored, writerFailed flag prevents double-end

MEDIUM:
- Remove 'retry after' from isTransientError patterns to stop double-retry
  layering (was causing up to 96 bot attempts per chunk)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude
2026-07-29 08:40:50 +07:00
parent de7d276245
commit d8da2044b2
4 changed files with 110 additions and 42 deletions
+15 -1
View File
@@ -1,6 +1,7 @@
import { serve } from 'bun';
import { config } from './config/index';
import { fileInfoCache } from './infrastructure/cache/index';
import { clearQueue, getQueueStats, waitForQueue } from './infrastructure/telegram/upload-queue';
import { startBot } from './interfaces/bot/handler';
import { handleS3Request } from './interfaces/http/controllers/s3-controller';
import { cleanupRateLimitCache } from './interfaces/http/middleware/rate-limit';
@@ -63,9 +64,22 @@ logger.info('Server started', { port: config.port, url: config.baseUrl });
const gracefulShutdown = async (signal: string): Promise<void> => {
logger.info('Graceful shutdown signal received', { signal });
logger.info('Closing HTTP server');
logger.info('Closing HTTP server — no new requests accepted');
server.stop();
// Drain pending upload queue with a timeout
const { pending, size } = getQueueStats();
if (pending > 0 || size > 0) {
logger.info('Draining upload queue', { pending, size });
const drainTimeout = setTimeout(() => {
logger.warn('Upload queue drain timeout — clearing remaining tasks');
clearQueue();
}, 30_000);
await waitForQueue();
clearTimeout(drainTimeout);
logger.info('Upload queue drained');
}
logger.info('Stopping Telegram bot');
bot.stop(signal);