Compare commits
18
Commits
5d2f3331de
..
v1.2.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a4bb860526 | ||
|
|
6797dbada2 | ||
|
|
2ca1e67435 | ||
|
|
ab955afad4 | ||
|
|
f442797a86 | ||
|
|
6ca92fcc21 | ||
|
|
8043247057 | ||
|
|
0782d0f993 | ||
|
|
bdda2d9884 | ||
|
|
357a1ae41b | ||
|
|
f16161d38a | ||
|
|
9743fbf4b7 | ||
|
|
d296e0ebcf | ||
|
|
765b3588a3 | ||
|
|
94c782ef23 | ||
|
|
0deb6072dd | ||
|
|
77340f63ae | ||
|
|
43e36b7629 |
No files matched your search
@@ -29,7 +29,7 @@ jobs:
|
|||||||
|
|
||||||
- uses: oven-sh/setup-bun@v2
|
- uses: oven-sh/setup-bun@v2
|
||||||
with:
|
with:
|
||||||
bun-version: latest
|
bun-version: "1.3.14"
|
||||||
|
|
||||||
- name: Install deps
|
- name: Install deps
|
||||||
run: bun install --frozen-lockfile
|
run: bun install --frozen-lockfile
|
||||||
|
|||||||
@@ -0,0 +1,45 @@
|
|||||||
|
name: Semantic Release
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches: [main]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
actions: write
|
||||||
|
id-token: write
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: release
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
release:
|
||||||
|
if: github.actor != 'dependabot[bot]'
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- name: Checkout
|
||||||
|
uses: actions/checkout@v7
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
# semantic-release needs the full history + tags to analyse commits
|
||||||
|
persist-credentials: true
|
||||||
|
|
||||||
|
- uses: oven-sh/setup-bun@v2
|
||||||
|
with:
|
||||||
|
bun-version: "1.3.14"
|
||||||
|
|
||||||
|
- name: Install semantic-release (isolated, no package.json pollution)
|
||||||
|
run: |
|
||||||
|
npm install --prefix /tmp/sr-tools \
|
||||||
|
semantic-release@^24 \
|
||||||
|
@semantic-release/changelog@^6 \
|
||||||
|
@semantic-release/exec@^6 \
|
||||||
|
@semantic-release/git@^10
|
||||||
|
echo "/tmp/sr-tools/node_modules/.bin" >> "$GITHUB_PATH"
|
||||||
|
|
||||||
|
- name: Run semantic-release
|
||||||
|
env:
|
||||||
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
run: semantic-release
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
# TeleUploader — Audit & Optimisasi Stabilitas + S3 (2026-08-27)
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
Audit dan optimasikan backend TeleUploader agar stabil di segala situasi dan
|
||||||
|
S3 protocol stabil + secepat mungkin. Fokus pada bottleneck nyata yang
|
||||||
|
ditemukan saat audit kode + log.
|
||||||
|
|
||||||
|
## Temuan Audit (root cause → prioritas)
|
||||||
|
|
||||||
|
### A. S3 GET lambat (bottleneck terbesar download)
|
||||||
|
1. **`concatPartStreams` (object-stream.ts) fetch part SEQUENTIAL** — tiap part
|
||||||
|
di-`fetch` satu per satu (for..of + await penuh). Untuk objek chunked/multipart
|
||||||
|
dengan N part, byte pertama baru terkirim SETELAH semua fetch part selesai.
|
||||||
|
→ **PARALELISASI** dengan bounded concurrency, urut tetap dipertahankan.
|
||||||
|
2. **`buildChunkedObjectSources` & `handleGetMultipartObject` resolve
|
||||||
|
`getFileInfo` SEQUENTIAL** (for..of + await). N call Telegram API berurutan
|
||||||
|
sebelum streaming mulai.
|
||||||
|
→ **`Promise.all`** resolve paralel.
|
||||||
|
3. **`botPool.getFileInfo` TIDAK pakai `fileInfoCache`** yang sudah ada — setiap
|
||||||
|
S3 GET objek reguler memanggil API Telegram `getFile` (network round-trip).
|
||||||
|
file-controller punya wrapper cache sendiri, tapi path S3 tidak.
|
||||||
|
→ **pindahkan caching ke dalam `botPool.getFileInfo`** agar semua caller dapat.
|
||||||
|
|
||||||
|
### B. Stabilitas / lingkungan produksi
|
||||||
|
4. **`nodeEnv: "development"` di produksi** (dari log BWS). Winston format json
|
||||||
|
sudah benar di production, tapi flag salah.
|
||||||
|
5. **`trustProxy: false` di produksi padahal di belakang Caddy** → rate limiter
|
||||||
|
meng-key semua user ke `127.0.0.1` (satu bucket global). Ini bug kecil untuk
|
||||||
|
non-S3 route. Fix: default trustProxy menyesuaikan, tapi utama via env.
|
||||||
|
6. **MemoryMax=512M di systemd** padahal config mendukung blob 2GB. Streaming
|
||||||
|
PUT O(1) memory, tapi GET gzip part membuffer part utuh. Naikkan ke 1G.
|
||||||
|
|
||||||
|
### C. Robustness S3
|
||||||
|
7. **`handleUploadPart` tidak `await writer.end()`** sebelum `createReadStream`
|
||||||
|
→ potensi race baca file belum flush. Await end().
|
||||||
|
8. Batas kecepatan unggah: `telegramBotConcurrency:1` × 6 bot = 6 unggah paralel.
|
||||||
|
Karena Telegram ~1 rps/bot untuk sendDocument, konkuransi per-bot 1 masuk
|
||||||
|
akal. Biarkan default, tapi pastikan pool retry terhadap 429 sudah bagus
|
||||||
|
(sudah, ada retry-after handling).
|
||||||
|
|
||||||
|
## File yang disentuh (semua di bawah src/)
|
||||||
|
- `src/interfaces/s3/object-stream.ts` — paralelisasi fetch part (concatPartStreams)
|
||||||
|
- `src/infrastructure/telegram/bot-pool.ts` — getFileInfo + cache, resolve paralel
|
||||||
|
- `src/infrastructure/telegram/chunked-storage.ts` — buildChunkedObjectSources paralel
|
||||||
|
- `src/interfaces/http/controllers/s3-controller.ts` — handleGetMultipartObject paralel,
|
||||||
|
await writer.end() di handleUploadPart
|
||||||
|
- `src/infrastructure/cache/index.ts` — (opsional) tambah helper
|
||||||
|
|
||||||
|
## Verifikasi
|
||||||
|
- `bunx biome check src test` (lint)
|
||||||
|
- `bun test test/s3-auth.test.ts` & `test/s3-operations.test.ts` (S3 regresi)
|
||||||
|
- `bun test test/telegram.test.ts` (pool) kalau ada
|
||||||
|
- Test unit baru untuk parallel part streaming (tanpa network) kalau layak.
|
||||||
|
- Deploy via push main → CI `deploy.yml` (Nix build → nix copy → systemctl restart).
|
||||||
|
|
||||||
|
## Deploy
|
||||||
|
Push ke `main` → GitHub Actions `deploy.yml` build + deploy Nix ke orangevps.
|
||||||
|
Setelah deploy, verifikasi `/health` 200 dan S3 GET masih berfungsi.
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"branches": ["main"],
|
||||||
|
"plugins": [
|
||||||
|
"@semantic-release/commit-analyzer",
|
||||||
|
"@semantic-release/release-notes-generator",
|
||||||
|
[
|
||||||
|
"@semantic-release/changelog",
|
||||||
|
{
|
||||||
|
"changelogFile": "CHANGELOG.md"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@semantic-release/exec",
|
||||||
|
{
|
||||||
|
"prepareCmd": "node ./.semrel/prepare.mjs '${nextRelease.version}' && bunx biome check package.json flake.nix CHANGELOG.md 2>/dev/null || true",
|
||||||
|
"successCmd": "node ./.semrel/dispatch.mjs '${nextRelease.version}'"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@semantic-release/git",
|
||||||
|
{
|
||||||
|
"assets": ["package.json", "flake.nix", "CHANGELOG.md"],
|
||||||
|
"message": "chore(release): ${nextRelease.version}\n\n${nextRelease.notes}"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"@semantic-release/github"
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,53 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* successCmd hook for semantic-release: dispatch the deploy workflow so the
|
||||||
|
* freshly released version is built and deployed to the VPS.
|
||||||
|
*
|
||||||
|
* GitHub disables workflow re-triggering for pushes made by GITHUB_TOKEN
|
||||||
|
* (which is what @semantic-release/git uses to commit the release), so the
|
||||||
|
* plain `push: branches: [main]` trigger in deploy.yml never fires for the
|
||||||
|
* release commit. We dispatch deploy.yml explicitly instead.
|
||||||
|
*/
|
||||||
|
|
||||||
|
const version = (process.argv[2] || '').trim();
|
||||||
|
|
||||||
|
async function main() {
|
||||||
|
const token = process.env.GITHUB_TOKEN;
|
||||||
|
if (!token) {
|
||||||
|
console.error('dispatch.mjs: GITHUB_TOKEN not set, skipping deploy dispatch');
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
if (!version) {
|
||||||
|
console.error('dispatch.mjs: no version argument, skipping');
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
|
||||||
|
const repo = process.env.GITHUB_REPOSITORY || 'asepharyana/TeleUploader';
|
||||||
|
const url = `https://api.github.com/repos/${repo}/actions/workflows/deploy.yml/dispatches`;
|
||||||
|
|
||||||
|
// dispatch with branch ref main; deploy builds #teleuploader from the
|
||||||
|
// release commit whose package.json/flake.nix already carry `version`
|
||||||
|
const res = await fetch(url, {
|
||||||
|
method: 'POST',
|
||||||
|
headers: {
|
||||||
|
Authorization: `Bearer ${token}`,
|
||||||
|
'Content-Type': 'application/json',
|
||||||
|
'User-Agent': 'semantic-release-dispatch',
|
||||||
|
},
|
||||||
|
body: JSON.stringify({ ref: 'main' }),
|
||||||
|
});
|
||||||
|
|
||||||
|
if (res.status === 204) {
|
||||||
|
console.log(`dispatch.mjs: dispatched deploy for v${version} to ${repo} main`);
|
||||||
|
} else {
|
||||||
|
const body = await res.text().catch(() => '');
|
||||||
|
console.error(`dispatch.mjs: deploy dispatch failed (HTTP ${res.status}): ${body}`);
|
||||||
|
// Non-fatal: a user can still run the deploy workflow manually.
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
main().catch((err) => {
|
||||||
|
console.error('dispatch.mjs: unexpected error:', err);
|
||||||
|
process.exit(0);
|
||||||
|
});
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* Prepare script for semantic-release: synchronises the package version
|
||||||
|
* across package.json and flake.nix before the release commit.
|
||||||
|
*
|
||||||
|
* getVersion() keeps package.json as the source of truth when semver has
|
||||||
|
* not run yet (e.g. first bootstrap) — after that, nextRelease.version
|
||||||
|
* is passed as argv[2] and everything is aligned to it.
|
||||||
|
*
|
||||||
|
* Fixes the TeleUploader Nix deploy trap: a source change without a
|
||||||
|
* version bump made Nix reuse the same store path, so CI reported success
|
||||||
|
* while the VPS profile stayed on the old build. By bumping flake.nix's
|
||||||
|
* `version` on every release, the Nix derivation always changes and the
|
||||||
|
* store path is always fresh.
|
||||||
|
*
|
||||||
|
* IMPORTANT: only bump the TeleUploader package version (the block whose
|
||||||
|
* `pname = "teleuploader"` precedes it), never the Bun overlay version that
|
||||||
|
* appears earlier in the file. Store path is derived from the
|
||||||
|
* teleuploader package's `version`.
|
||||||
|
*/
|
||||||
|
import { readFileSync, writeFileSync } from 'node:fs';
|
||||||
|
|
||||||
|
const nextVersion = process.argv[2];
|
||||||
|
|
||||||
|
// --- package.json ---
|
||||||
|
const pkgPath = 'package.json';
|
||||||
|
const pkg = JSON.parse(readFileSync(pkgPath, 'utf8'));
|
||||||
|
if (nextVersion) pkg.version = nextVersion;
|
||||||
|
writeFileSync(pkgPath, `${JSON.stringify(pkg, null, 2)}\n`);
|
||||||
|
|
||||||
|
// --- flake.nix ---
|
||||||
|
const flakePath = 'flake.nix';
|
||||||
|
let flake = readFileSync(flakePath, 'utf8');
|
||||||
|
const target = nextVersion ?? pkg.version;
|
||||||
|
|
||||||
|
const anchor = 'pname = "teleuploader";';
|
||||||
|
const anchorIdx = flake.indexOf(anchor);
|
||||||
|
if (anchorIdx === -1) {
|
||||||
|
throw new Error(`prepare.mjs: TeleUploader pname anchor not found in ${flakePath}`);
|
||||||
|
}
|
||||||
|
const rest = flake.slice(anchorIdx);
|
||||||
|
const verMatch = rest.match(/(\s*)version = "\d+\.\d+\.\d+";/);
|
||||||
|
if (!verMatch) {
|
||||||
|
throw new Error(`prepare.mjs: could not locate TeleUploader version block in ${flakePath}`);
|
||||||
|
}
|
||||||
|
const replaced =
|
||||||
|
flake.slice(0, anchorIdx) +
|
||||||
|
rest.replace(/(\s*)version = "\d+\.\d+\.\d+";/, `$1version = "${target}";`);
|
||||||
|
if (!replaced.includes(`version = "${target}";`)) {
|
||||||
|
throw new Error(`prepare.mjs: replacement did not land in ${flakePath}`);
|
||||||
|
}
|
||||||
|
writeFileSync(flakePath, replaced);
|
||||||
|
|
||||||
|
console.log(`prepare.mjs: versions synced to ${target}`);
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
## [1.2.2](https://github.com/asepharyana/TeleUploader/compare/v1.2.1...v1.2.2) (2026-09-05)
|
||||||
|
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* **health:** resolve app version in Nix bundle at runtime ([6797dba](https://github.com/asepharyana/TeleUploader/commit/6797dbada27873b14c064e238589e61ad3ecb67c))
|
||||||
|
|
||||||
|
## [1.2.1](https://github.com/asepharyana/TeleUploader/compare/v1.2.0...v1.2.1) (2026-09-05)
|
||||||
|
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
* **ci:** bump only TeleUploader flake version, not Bun overlay ([f442797](https://github.com/asepharyana/TeleUploader/commit/f442797a86e845853c28dc22432a530560434488))
|
||||||
|
|
||||||
|
# [1.2.0](https://github.com/asepharyana/TeleUploader/compare/v1.1.1...v1.2.0) (2026-09-05)
|
||||||
|
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
* **health:** expose app version in /health response ([0782d0f](https://github.com/asepharyana/TeleUploader/commit/0782d0f993a033bb9d84fa1f876e29486751889d))
|
||||||
@@ -53,3 +53,16 @@ Ini berarti URL service Anda fix, bukan jaminan file Telegram abadi.
|
|||||||
## Testing
|
## Testing
|
||||||
|
|
||||||
Gunakan bot Telegram untuk upload, atau upload API langsung via HTTP.
|
Gunakan bot Telegram untuk upload, atau upload API langsung via HTTP.
|
||||||
|
|
||||||
|
## Versioning
|
||||||
|
|
||||||
|
This repository uses auto semantic versioning via [semantic-release](https://github.com/semantic-release/semantic-release):
|
||||||
|
|
||||||
|
- `fix(...)` commits → patch bump (v1.1.1 → v1.1.2)
|
||||||
|
- `feat(...)` commits → minor bump (v1.2.0)
|
||||||
|
- breaking changes → major bump (v2.0.0)
|
||||||
|
- `chore/ci/docs/refactor/test` commits → no release
|
||||||
|
|
||||||
|
On every release, `prepare.mjs` syncs the version across `package.json` and
|
||||||
|
`flake.nix`, guaranteeing a fresh Nix store path and true auto-deploy via
|
||||||
|
the `Build & Deploy (Nix)` workflow.
|
||||||
@@ -9,12 +9,25 @@
|
|||||||
outputs = { self, nixpkgs, flake-utils }:
|
outputs = { self, nixpkgs, flake-utils }:
|
||||||
flake-utils.lib.eachSystem [ "x86_64-linux" ] (system:
|
flake-utils.lib.eachSystem [ "x86_64-linux" ] (system:
|
||||||
let
|
let
|
||||||
pkgs = import nixpkgs { inherit system; };
|
pkgs = import nixpkgs {
|
||||||
|
inherit system;
|
||||||
|
overlays = [
|
||||||
|
(self: super: {
|
||||||
|
bun = super.bun.overrideAttrs (old: rec {
|
||||||
|
version = "1.3.14";
|
||||||
|
src = super.fetchzip {
|
||||||
|
url = "https://github.com/oven-sh/bun/releases/download/bun-v1.3.14/bun-linux-x64.zip";
|
||||||
|
sha256 = "sha256:189kasncnfpq6q37q3w80dv2a92w79w20x2x6hf1fbkir5gyvzlb";
|
||||||
|
};
|
||||||
|
});
|
||||||
|
})
|
||||||
|
];
|
||||||
|
};
|
||||||
|
|
||||||
# TeleUploader package
|
# TeleUploader package
|
||||||
teleuploader = pkgs.stdenvNoCC.mkDerivation rec {
|
teleuploader = pkgs.stdenvNoCC.mkDerivation rec {
|
||||||
pname = "teleuploader";
|
pname = "teleuploader";
|
||||||
version = "1.1.0";
|
version = "1.2.2";
|
||||||
|
|
||||||
src = ./.;
|
src = ./.;
|
||||||
|
|
||||||
@@ -48,6 +61,9 @@
|
|||||||
cp -r dist $out/share/teleuploader/dist
|
cp -r dist $out/share/teleuploader/dist
|
||||||
cp src/home.html $out/share/teleuploader/ 2>/dev/null || true
|
cp src/home.html $out/share/teleuploader/ 2>/dev/null || true
|
||||||
cp schema.sql $out/share/teleuploader/ 2>/dev/null || true
|
cp schema.sql $out/share/teleuploader/ 2>/dev/null || true
|
||||||
|
# Carry package.json into the store so runtime version lookup in
|
||||||
|
# env.ts readPackageVersion() resolves (dist/../package.json).
|
||||||
|
cp package.json $out/share/teleuploader/package.json
|
||||||
|
|
||||||
# Wrap with bun from Nix store (dependency sharing!)
|
# Wrap with bun from Nix store (dependency sharing!)
|
||||||
# Note: NO --chdir — systemd WorkingDirectory controls this
|
# Note: NO --chdir — systemd WorkingDirectory controls this
|
||||||
|
|||||||
+3
-3
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "filedrop",
|
"name": "filedrop",
|
||||||
"version": "1.1.0",
|
"version": "1.2.2",
|
||||||
"description": "Telegram file uploader backend",
|
"description": "Telegram file uploader backend",
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"scripts": {
|
"scripts": {
|
||||||
@@ -8,11 +8,11 @@
|
|||||||
"build": "bun build src/index.ts --target=bun --outfile=dist/index.js && bun build src/infrastructure/persistence/drizzle/migrate.ts --target=bun --outfile=dist/migrate.js",
|
"build": "bun build src/index.ts --target=bun --outfile=dist/index.js && bun build src/infrastructure/persistence/drizzle/migrate.ts --target=bun --outfile=dist/migrate.js",
|
||||||
"start": "NODE_ENV=production bun dist/index.js",
|
"start": "NODE_ENV=production bun dist/index.js",
|
||||||
"db:migrate": "bun dist/migrate.js",
|
"db:migrate": "bun dist/migrate.js",
|
||||||
"test": "bun test --preload ./test/helpers/setup-env.ts test/rateLimit.test.ts && bun test --preload ./test/helpers/setup-env.ts test/file.test.ts && bun test --preload ./test/helpers/setup-env.ts test/telegram.test.ts && bun test --preload ./test/helpers/setup-env.ts test/upload.test.ts && bun test --preload ./test/helpers/setup-env.ts test/files.test.ts && bun test --preload ./test/helpers/setup-env.ts test/health.test.ts && bun test --preload ./test/helpers/setup-env.ts test/db.test.ts &Line truncated
|
"test": "bun test --preload ./test/helpers/setup-env.ts test/rateLimit.test.ts && bun test --preload ./test/helpers/setup-env.ts test/file.test.ts && bun test --preload ./test/helpers/setup-env.ts test/telegram.test.ts && bun test --preload ./test/helpers/setup-env.ts test/upload.test.ts && bun test --preload ./test/helpers/setup-env.ts test/files.test.ts && bun test --preload ./test/helpers/setup-env.ts test/health.test.ts && bun test --preload ./test/helpers/setup-env.ts test/db.test.ts &Line truncated
|
||||||
"test:s3-auth": "bun test --preload ./test/helpers/setup-env.ts test/s3-auth.test.ts",
|
"test:s3-auth": "bun test --preload ./test/helpers/setup-env.ts test/s3-auth.test.ts",
|
||||||
"test:s3-ops": "bun test --preload ./test/helpers/setup-env.ts test/s3-operations.test.ts",
|
"test:s3-ops": "bun test --preload ./test/helpers/setup-env.ts test/s3-operations.test.ts",
|
||||||
"test:web-api": "bun test --preload ./test/helpers/setup-env.ts test/web-api.test.ts",
|
"test:web-api": "bun test --preload ./test/helpers/setup-env.ts test/web-api.test.ts",
|
||||||
"test:s3": "bun test --preload ./test/helpers/setup-env.ts test/s3-auth.test.ts && bun test --preload ./test/helpers/setup-env.ts test/s3-operations.test.ts && bun test --preload ./test/helpers/setup-env.ts test/web-api.test.ts",
|
"test:s3": "bun test --preload ./test/helpers/setup-env.ts test/s3-auth.test.ts && bun test --preload ./test/helpers/setup-env.ts test/s3-auth-edge.test.ts && bun test --preload ./test/helpers/setup-env.ts test/s3-operations.test.ts && bun test --preload ./test/helpers/setup-env.ts test/s3-range.test.ts && bun test --preload ./test/helpers/setup-env.ts test/s3-object-stream.test.ts && bun test --preload ./test/helpers/setup-env.ts test/object-stream-parallel.test.ts && bun test --preload ./Line truncated
|
||||||
"lint": "bunx biome check src test",
|
"lint": "bunx biome check src test",
|
||||||
"format": "bunx biome format --write src test",
|
"format": "bunx biome format --write src test",
|
||||||
"prepare": "husky"
|
"prepare": "husky"
|
||||||
|
|||||||
+35
@@ -1,7 +1,10 @@
|
|||||||
|
import { readFileSync } from 'node:fs';
|
||||||
import logger from './shared/logger/index';
|
import logger from './shared/logger/index';
|
||||||
import { TELEGRAM_CHUNK_SIZE_MAX_BYTES } from './shared/utils/validation';
|
import { TELEGRAM_CHUNK_SIZE_MAX_BYTES } from './shared/utils/validation';
|
||||||
|
|
||||||
interface AppConfig {
|
interface AppConfig {
|
||||||
|
/** Application version (read from package.json, kept in sync by semantic-release prepare.mjs) */
|
||||||
|
appVersion: string;
|
||||||
/** All bot tokens merged from BOT_TOKENS (or BOT_TOKEN + ADDITIONAL_BOT_TOKENS fallback) */
|
/** All bot tokens merged from BOT_TOKENS (or BOT_TOKEN + ADDITIONAL_BOT_TOKENS fallback) */
|
||||||
botTokens: string[];
|
botTokens: string[];
|
||||||
/** Per-bot concurrency for Telegram API calls (default 1). */
|
/** Per-bot concurrency for Telegram API calls (default 1). */
|
||||||
@@ -131,7 +134,39 @@ if (telegramChunkSizeBytes > TELEGRAM_CHUNK_SIZE_MAX_BYTES) {
|
|||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Reads the application version from package.json.
|
||||||
|
*
|
||||||
|
* Prefers the resolved node_modules/teleuploader package version (set by
|
||||||
|
* semantic-release when the repo is installed as a published artifact) and
|
||||||
|
* falls back to reading the repo's own package.json when running from a
|
||||||
|
* checkout. Both files are updated by prepare.mjs on every release, so the
|
||||||
|
* value always matches the deployed build.
|
||||||
|
*/
|
||||||
|
const readPackageVersion = (): string => {
|
||||||
|
// Candidates in priority order. In the Nix bundle the app lives at
|
||||||
|
// $out/share/teleuploader/dist/index.js, so the package.json that was
|
||||||
|
// copied alongside it is one level up from import.meta.dir. In a dev
|
||||||
|
// checkout it sits in the repo root (process.cwd()). Bun bundles JSON
|
||||||
|
// imports statically, so we read the file at runtime instead.
|
||||||
|
const candidates = [
|
||||||
|
import.meta.dir ? `${import.meta.dir}/../package.json` : undefined,
|
||||||
|
`${process.cwd()}/package.json`,
|
||||||
|
];
|
||||||
|
for (const candidate of candidates) {
|
||||||
|
if (!candidate) continue;
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(readFileSync(candidate, 'utf8')) as { version?: string };
|
||||||
|
if (parsed.version) return parsed.version;
|
||||||
|
} catch {
|
||||||
|
// try next candidate
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return '0.0.0';
|
||||||
|
};
|
||||||
|
|
||||||
export const config: AppConfig = {
|
export const config: AppConfig = {
|
||||||
|
appVersion: readPackageVersion(),
|
||||||
botTokens: parseTokens(botTokensRaw),
|
botTokens: parseTokens(botTokensRaw),
|
||||||
telegramBotConcurrency: parseNumber(process.env.TELEGRAM_BOT_CONCURRENCY, 1),
|
telegramBotConcurrency: parseNumber(process.env.TELEGRAM_BOT_CONCURRENCY, 1),
|
||||||
storageChatId: parseInt(process.env.STORAGE_CHANNEL_ID!, 10),
|
storageChatId: parseInt(process.env.STORAGE_CHANNEL_ID!, 10),
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import type {
|
|||||||
} from '../../domain/ports/telegram-service';
|
} from '../../domain/ports/telegram-service';
|
||||||
import { config } from '../../env';
|
import { config } from '../../env';
|
||||||
import logger from '../../shared/logger/index';
|
import logger from '../../shared/logger/index';
|
||||||
|
import { fileInfoCache } from '../cache/index';
|
||||||
import {
|
import {
|
||||||
buildSendPayload,
|
buildSendPayload,
|
||||||
extractUploadedFile,
|
extractUploadedFile,
|
||||||
@@ -259,18 +260,31 @@ export class BotPool implements ITelegramService {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async getFileInfo(telegramFileId: string): Promise<TelegramFileInfo> {
|
async getFileInfo(telegramFileId: string): Promise<TelegramFileInfo> {
|
||||||
|
// Telegram file_id → file_path mapping is stable for the lifetime of the
|
||||||
|
// file. Cache it to avoid a Telegram API round-trip on every S3 GET / HEAD
|
||||||
|
// of the same object. TTL 1h; a cached (possibly stale) file_path would
|
||||||
|
// only surface if Telegram recycles a file_id, which it does not for
|
||||||
|
// documents we own.
|
||||||
|
const cacheKey = `file_info_${telegramFileId}`;
|
||||||
|
const cached = fileInfoCache.get(cacheKey) as TelegramFileInfo | null;
|
||||||
|
if (cached) {
|
||||||
|
return cached;
|
||||||
|
}
|
||||||
|
|
||||||
let lastError: unknown;
|
let lastError: unknown;
|
||||||
for (const bot of this.bots) {
|
for (const bot of this.bots) {
|
||||||
for (let retry = 0; retry <= MAX_TRANSIENT_RETRIES; retry++) {
|
for (let retry = 0; retry <= MAX_TRANSIENT_RETRIES; retry++) {
|
||||||
try {
|
try {
|
||||||
const result = await bot.instance.telegram.getFile(telegramFileId);
|
const result = await bot.instance.telegram.getFile(telegramFileId);
|
||||||
const fileData = result as unknown as Omit<TelegramFileInfo, 'bot_token'>;
|
const fileData = result as unknown as Omit<TelegramFileInfo, 'bot_token'>;
|
||||||
return {
|
const fileInfo: TelegramFileInfo = {
|
||||||
file_size: fileData.file_size || 0,
|
file_size: fileData.file_size || 0,
|
||||||
mime_type: fileData.mime_type || 'application/octet-stream',
|
mime_type: fileData.mime_type || 'application/octet-stream',
|
||||||
file_path: fileData.file_path || '',
|
file_path: fileData.file_path || '',
|
||||||
bot_token: bot.token,
|
bot_token: bot.token,
|
||||||
};
|
};
|
||||||
|
fileInfoCache.set(cacheKey, fileInfo);
|
||||||
|
return fileInfo;
|
||||||
} catch (error: unknown) {
|
} catch (error: unknown) {
|
||||||
lastError = error;
|
lastError = error;
|
||||||
const errorStr = error instanceof Error ? error.message : String(error);
|
const errorStr = error instanceof Error ? error.message : String(error);
|
||||||
|
|||||||
@@ -229,21 +229,27 @@ export class ChunkedStorage {
|
|||||||
*/
|
*/
|
||||||
async buildChunkedObjectSources(file: FileEntity): Promise<ObjectPartSource[]> {
|
async buildChunkedObjectSources(file: FileEntity): Promise<ObjectPartSource[]> {
|
||||||
const parts = await this.filePartRepository.listByFileId(file.id);
|
const parts = await this.filePartRepository.listByFileId(file.id);
|
||||||
const sources: ObjectPartSource[] = [];
|
|
||||||
|
|
||||||
for (const part of parts) {
|
// Resolve every part's Telegram CDN URL concurrently (each is an independent
|
||||||
const fileInfo = await this.telegramService.getFileInfo(part.telegramFileId);
|
// getFile call) so total resolution time is ~1 round-trip, not N.
|
||||||
sources.push({
|
const partInfos = await Promise.all(
|
||||||
telegramFileId: part.telegramFileId,
|
parts.map(async (part) => {
|
||||||
telegramUrl: `https://api.telegram.org/file/bot${fileInfo.bot_token}/${fileInfo.file_path}`,
|
const fileInfo = await this.telegramService.getFileInfo(part.telegramFileId);
|
||||||
sizeBytes: part.sizeBytes,
|
return {
|
||||||
storedSizeBytes: part.storedSizeBytes,
|
part,
|
||||||
compressionAlgorithm: part.compressionAlgorithm,
|
url: `https://api.telegram.org/file/bot${fileInfo.bot_token}/${fileInfo.file_path}`,
|
||||||
partNumber: part.partNumber,
|
};
|
||||||
});
|
}),
|
||||||
}
|
);
|
||||||
|
|
||||||
return sources;
|
return partInfos.map(({ part, url }) => ({
|
||||||
|
telegramFileId: part.telegramFileId,
|
||||||
|
telegramUrl: url,
|
||||||
|
sizeBytes: part.sizeBytes,
|
||||||
|
storedSizeBytes: part.storedSizeBytes,
|
||||||
|
compressionAlgorithm: part.compressionAlgorithm,
|
||||||
|
partNumber: part.partNumber,
|
||||||
|
}));
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
|
|||||||
@@ -106,7 +106,12 @@ export const handleFileRedirect = async (req: RequestWithParams): Promise<Respon
|
|||||||
return fail(501, 'Archive entry extraction is not supported for chunked files');
|
return fail(501, 'Archive entry extraction is not supported for chunked files');
|
||||||
}
|
}
|
||||||
const range = { type: 'none' as const };
|
const range = { type: 'none' as const };
|
||||||
return chunkedStorage.createChunkedObjectResponse({ file, range, reqId: '' });
|
const resp = await chunkedStorage.createChunkedObjectResponse({ file, range, reqId: '' });
|
||||||
|
// Ensure CORS so browser fetch/XHR playback works for chunked files too.
|
||||||
|
const h = new Headers(resp.headers);
|
||||||
|
if (!h.has('Access-Control-Allow-Origin')) h.set('Access-Control-Allow-Origin', '*');
|
||||||
|
h.set('Access-Control-Expose-Headers', 'Content-Disposition, Content-Length, Accept-Ranges');
|
||||||
|
return new Response(resp.body, { status: resp.status, headers: h });
|
||||||
}
|
}
|
||||||
|
|
||||||
const archiveEntryName = file.archiveEntryName;
|
const archiveEntryName = file.archiveEntryName;
|
||||||
@@ -150,19 +155,70 @@ export const handleFileRedirect = async (req: RequestWithParams): Promise<Respon
|
|||||||
'Content-Type': file.mimeType || 'application/octet-stream',
|
'Content-Type': file.mimeType || 'application/octet-stream',
|
||||||
'Content-Disposition': `attachment; filename="${sanitizeFilenameHeader(file.fileName)}"`,
|
'Content-Disposition': `attachment; filename="${sanitizeFilenameHeader(file.fileName)}"`,
|
||||||
'Content-Length': String(loc.length),
|
'Content-Length': String(loc.length),
|
||||||
|
'Access-Control-Allow-Origin': '*',
|
||||||
|
'Access-Control-Expose-Headers': 'Content-Disposition, Content-Length, Accept-Ranges',
|
||||||
},
|
},
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
const fileInfo = await getTelegramFileInfo(file.telegramFileId, publicId);
|
const fileInfo = await getTelegramFileInfo(file.telegramFileId, publicId);
|
||||||
const redirectUrl = buildTelegramFileUrl(fileInfo.file_path, fileInfo.bot_token);
|
const telegramUrl = buildTelegramFileUrl(fileInfo.file_path, fileInfo.bot_token);
|
||||||
|
|
||||||
return new Response(null, {
|
// CORS headers shared across all delivery modes — public file CDN.
|
||||||
status: 302,
|
const corsHeaders: Record<string, string> = {
|
||||||
headers: {
|
'Access-Control-Allow-Origin': '*',
|
||||||
Location: redirectUrl,
|
'Access-Control-Expose-Headers': 'Content-Disposition, Content-Length, Accept-Ranges',
|
||||||
},
|
'Access-Control-Allow-Methods': 'GET, HEAD, OPTIONS',
|
||||||
});
|
'Access-Control-Allow-Headers': 'Range, Content-Type',
|
||||||
|
Vary: 'Origin',
|
||||||
|
};
|
||||||
|
|
||||||
|
try {
|
||||||
|
const upstream = await fetch(telegramUrl);
|
||||||
|
if (!upstream.ok) {
|
||||||
|
logger.error('Telegram file download failed', {
|
||||||
|
publicId,
|
||||||
|
status: upstream.status,
|
||||||
|
});
|
||||||
|
return Response.json(
|
||||||
|
{ error: 'Upstream download failed' },
|
||||||
|
{
|
||||||
|
status: upstream.status === 404 ? 404 : 502,
|
||||||
|
headers: corsHeaders,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
const upstreamHeaders = new Headers(upstream.headers);
|
||||||
|
const contentType =
|
||||||
|
file.mimeType || upstreamHeaders.get('content-type') || 'application/octet-stream';
|
||||||
|
const headers = {
|
||||||
|
'Content-Type': contentType,
|
||||||
|
'Content-Disposition': `inline; filename="${sanitizeFilenameHeader(file.fileName)}"`,
|
||||||
|
'Content-Length': String(file.sizeBytes ?? 0),
|
||||||
|
'Cache-Control': 'public, max-age=300',
|
||||||
|
...corsHeaders,
|
||||||
|
};
|
||||||
|
|
||||||
|
// Stream the Telegram CDN body back to the client (no cross-origin hop
|
||||||
|
// in the browser → no CORS block for fetch/XHR audio playback).
|
||||||
|
return new Response(upstream.body, {
|
||||||
|
status: 200,
|
||||||
|
headers,
|
||||||
|
});
|
||||||
|
} catch (error: unknown) {
|
||||||
|
logger.error('Telegram file proxy error', {
|
||||||
|
publicId,
|
||||||
|
error: getErrorMessage(error),
|
||||||
|
});
|
||||||
|
return Response.json(
|
||||||
|
{ error: 'Upstream download failed' },
|
||||||
|
{
|
||||||
|
status: 502,
|
||||||
|
headers: corsHeaders,
|
||||||
|
},
|
||||||
|
);
|
||||||
|
}
|
||||||
} catch (error: unknown) {
|
} catch (error: unknown) {
|
||||||
logger.error('File redirect error', { publicId, error: getErrorMessage(error) });
|
logger.error('File redirect error', { publicId, error: getErrorMessage(error) });
|
||||||
return fail(500, 'Server error');
|
return fail(500, 'Server error');
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { sql } from 'drizzle-orm';
|
import { sql } from 'drizzle-orm';
|
||||||
|
import { config } from '../../../env';
|
||||||
import { db } from '../../../infrastructure/persistence/drizzle/index';
|
import { db } from '../../../infrastructure/persistence/drizzle/index';
|
||||||
import logger from '../../../shared/logger/index';
|
import logger from '../../../shared/logger/index';
|
||||||
import { getErrorMessage } from '../../../shared/utils/file';
|
import { getErrorMessage } from '../../../shared/utils/file';
|
||||||
@@ -16,7 +17,7 @@ import { getErrorMessage } from '../../../shared/utils/file';
|
|||||||
export const handleHealth = async (_req: Request): Promise<Response> => {
|
export const handleHealth = async (_req: Request): Promise<Response> => {
|
||||||
try {
|
try {
|
||||||
await db.execute(sql`SELECT 1`);
|
await db.execute(sql`SELECT 1`);
|
||||||
return Response.json({ status: 'ok' }, { status: 200 });
|
return Response.json({ status: 'ok', version: config.appVersion }, { status: 200 });
|
||||||
} catch (error: unknown) {
|
} catch (error: unknown) {
|
||||||
const message = getErrorMessage(error);
|
const message = getErrorMessage(error);
|
||||||
logger.error('Health check failed', { error: message });
|
logger.error('Health check failed', { error: message });
|
||||||
|
|||||||
@@ -702,15 +702,21 @@ const handleGetMultipartObject = async (
|
|||||||
}
|
}
|
||||||
|
|
||||||
const sources: ObjectPartSource[] = [];
|
const sources: ObjectPartSource[] = [];
|
||||||
for (const part of parts) {
|
// Resolve all part CDN URLs concurrently (independent getFile calls) so
|
||||||
const fileInfo = await botPool.getFileInfo(part.telegramFileId);
|
// assembly latency is ~1 round-trip instead of N.
|
||||||
sources.push({
|
sources.push(
|
||||||
telegramFileId: part.telegramFileId,
|
...(await Promise.all(
|
||||||
telegramUrl: `https://api.telegram.org/file/bot${fileInfo.bot_token}/${fileInfo.file_path}`,
|
parts.map(async (part) => {
|
||||||
sizeBytes: part.sizeBytes,
|
const fileInfo = await botPool.getFileInfo(part.telegramFileId);
|
||||||
partNumber: part.partNumber,
|
return {
|
||||||
});
|
telegramFileId: part.telegramFileId,
|
||||||
}
|
telegramUrl: `https://api.telegram.org/file/bot${fileInfo.bot_token}/${fileInfo.file_path}`,
|
||||||
|
sizeBytes: part.sizeBytes,
|
||||||
|
partNumber: part.partNumber,
|
||||||
|
};
|
||||||
|
}),
|
||||||
|
)),
|
||||||
|
);
|
||||||
|
|
||||||
// H1: Always proxy — never expose bot token in redirect URL
|
// H1: Always proxy — never expose bot token in redirect URL
|
||||||
|
|
||||||
@@ -1479,9 +1485,13 @@ const handleUploadPart = async (
|
|||||||
hasher.update(chunk);
|
hasher.update(chunk);
|
||||||
writer.write(chunk);
|
writer.write(chunk);
|
||||||
}
|
}
|
||||||
writer.end();
|
await writer.end();
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
writer.end();
|
try {
|
||||||
|
writer.end();
|
||||||
|
} catch {
|
||||||
|
// ignore during error path
|
||||||
|
}
|
||||||
await cleanupTempFile(tempPath);
|
await cleanupTempFile(tempPath);
|
||||||
throw error;
|
throw error;
|
||||||
} finally {
|
} finally {
|
||||||
|
|||||||
@@ -97,12 +97,53 @@ const fetchPartBody = async (planned: PlannedPart): Promise<ReadableStream<Uint8
|
|||||||
return streamFromBytes(bytes.slice(planned.relativeStart, planned.relativeEnd + 1));
|
return streamFromBytes(bytes.slice(planned.relativeStart, planned.relativeEnd + 1));
|
||||||
};
|
};
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Maximum number of Telegram CDN part fetches run concurrently while
|
||||||
|
* assembling a chunked/multipart object response.
|
||||||
|
*
|
||||||
|
* Part fetches are initiated in parallel (bounded by this constant) to avoid
|
||||||
|
* serializing N sequential network round-trips on the Telegram CDN, then the
|
||||||
|
* results are fanned-in to the response stream in part-number order so the
|
||||||
|
* object bytes remain correctly ordered.
|
||||||
|
*/
|
||||||
|
const PART_FETCH_CONCURRENCY = 6;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Runs an async mapper over the parts with bounded concurrency, returning the
|
||||||
|
* results in the same order as the input. Each worker claims the next not-yet-
|
||||||
|
* claimed index, so array slots are filled by exactly one worker each.
|
||||||
|
*/
|
||||||
|
const mapBounded = async <T, R>(
|
||||||
|
items: T[],
|
||||||
|
limit: number,
|
||||||
|
fn: (item: T, index: number) => Promise<R>,
|
||||||
|
): Promise<R[]> => {
|
||||||
|
const results: R[] = new Array(items.length);
|
||||||
|
let next = 0;
|
||||||
|
|
||||||
|
const worker = async (): Promise<void> => {
|
||||||
|
while (true) {
|
||||||
|
const idx = next++;
|
||||||
|
if (idx >= items.length) return;
|
||||||
|
results[idx] = await fn(items[idx], idx);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
const workers = Array.from({ length: Math.max(1, Math.min(limit, items.length)) }, worker);
|
||||||
|
await Promise.all(workers);
|
||||||
|
return results;
|
||||||
|
};
|
||||||
|
|
||||||
const concatPartStreams = (plannedParts: PlannedPart[]): ReadableStream<Uint8Array> =>
|
const concatPartStreams = (plannedParts: PlannedPart[]): ReadableStream<Uint8Array> =>
|
||||||
new ReadableStream<Uint8Array>({
|
new ReadableStream<Uint8Array>({
|
||||||
async start(controller) {
|
async start(controller) {
|
||||||
try {
|
try {
|
||||||
for (const planned of plannedParts) {
|
// Fetch every part body concurrently (bounded) so the slowest Telegram
|
||||||
const stream = await fetchPartBody(planned);
|
// CDN fetch dictates latency instead of the sum of all fetches.
|
||||||
|
const partStreams = await mapBounded(plannedParts, PART_FETCH_CONCURRENCY, fetchPartBody);
|
||||||
|
|
||||||
|
// Fan-in in part order — object bytes stay correctly ordered.
|
||||||
|
for (const stream of partStreams) {
|
||||||
const reader = stream.getReader();
|
const reader = stream.getReader();
|
||||||
while (true) {
|
while (true) {
|
||||||
const { value, done } = await reader.read();
|
const { value, done } = await reader.read();
|
||||||
|
|||||||
@@ -23,10 +23,16 @@ export const maybeCompressChunk = (
|
|||||||
return { bytes: chunk, compressionAlgorithm: null };
|
return { bytes: chunk, compressionAlgorithm: null };
|
||||||
}
|
}
|
||||||
|
|
||||||
const gzipped = Bun.gzipSync(chunk);
|
const gzipped = Bun.gzipSync(chunk as Uint8Array<ArrayBuffer>);
|
||||||
if (gzipped.byteLength >= chunk.byteLength) {
|
if (gzipped.byteLength >= chunk.byteLength) {
|
||||||
return { bytes: chunk, compressionAlgorithm: null };
|
return { bytes: chunk, compressionAlgorithm: null };
|
||||||
}
|
}
|
||||||
|
|
||||||
return { bytes: gzipped, compressionAlgorithm: 'gzip' };
|
// CRITICAL: Bun.gzipSync returns a plain Uint8Array, NOT a Buffer.
|
||||||
|
// Telegraf (telegram.sendDocument) only recognises Buffer/Blob/stream
|
||||||
|
// sources as file uploads — a plain Uint8Array yields an empty multipart
|
||||||
|
// body and Telegram rejects with "400: there is no document in the request".
|
||||||
|
// Wrap in Buffer.from(...) so chunked uploads of compressible files (MP4,
|
||||||
|
// zip, text, etc.) actually work.
|
||||||
|
return { bytes: Buffer.from(gzipped), compressionAlgorithm: 'gzip' };
|
||||||
};
|
};
|
||||||
+100
-78
@@ -1,38 +1,32 @@
|
|||||||
import { beforeEach, describe, expect, it, mock } from 'bun:test';
|
import { beforeEach, describe, expect, it, mock } from 'bun:test';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import type { ITelegramService } from '../src/domain/ports/telegram-service';
|
||||||
|
import { ChunkedStorage } from '../src/infrastructure/telegram/chunked-storage';
|
||||||
|
|
||||||
// Mock DB layer (chunked-storage imports db, insertFileParts, listFileParts)
|
/**
|
||||||
const mockInsert = mock(() => Promise.resolve());
|
* Tests the real ChunkedStorage class (src/infrastructure/telegram/
|
||||||
const mockPartsInsert = mock(() => Promise.resolve());
|
* chunked-storage.ts). uploadFileInTelegramChunks only depends on the injected
|
||||||
const mockPartsSelect = mock(() => Promise.resolve([]));
|
* telegramService, so we stub that and pass no-op repos for the rest.
|
||||||
|
*
|
||||||
|
* Rewritten from a stale test that imported the old `src/utils/chunked-storage`
|
||||||
|
* layout, which no longer exists after the refactor.
|
||||||
|
*/
|
||||||
|
const makeTelegramStub = (): ITelegramService =>
|
||||||
|
({
|
||||||
|
forwardToStorage: mock(async (_bytes: unknown, fileName: string) => ({
|
||||||
|
telegramFileId: `tg-${fileName}`,
|
||||||
|
telegramFileUniqueId: `tg-unique-${fileName}`,
|
||||||
|
storageMessageId: Math.floor(Math.random() * 100000) + 1,
|
||||||
|
})),
|
||||||
|
getFileInfo: mock(async (telegramFileId: string) => ({
|
||||||
|
file_size: 0,
|
||||||
|
mime_type: 'application/octet-stream',
|
||||||
|
file_path: `documents/${telegramFileId}`,
|
||||||
|
bot_token: '123456:ABC-DEF',
|
||||||
|
})),
|
||||||
|
}) as unknown as ITelegramService;
|
||||||
|
|
||||||
mock.module('../src/db/index', () => ({
|
const noopRepo = {} as never;
|
||||||
db: {
|
|
||||||
insert: mockInsert,
|
|
||||||
select: () => ({ from: () => ({ where: () => ({ limit: () => Promise.resolve([]) }) }) }),
|
|
||||||
execute: mock(() => Promise.resolve([])),
|
|
||||||
},
|
|
||||||
files: {},
|
|
||||||
fileParts: {},
|
|
||||||
}));
|
|
||||||
|
|
||||||
mock.module('../src/db/file-parts', () => ({
|
|
||||||
insertFileParts: mockPartsInsert,
|
|
||||||
listFileParts: mockPartsSelect,
|
|
||||||
}));
|
|
||||||
|
|
||||||
mock.module('../src/utils/telegram', () => ({
|
|
||||||
forwardToStorage: async (_bytes: unknown, fileName: string) => ({
|
|
||||||
telegramFileId: `tg-${fileName}`,
|
|
||||||
telegramFileUniqueId: `tg-unique-${fileName}`,
|
|
||||||
storageMessageId: Math.floor(Math.random() * 100000) + 1,
|
|
||||||
}),
|
|
||||||
getFileInfo: async (telegramFileId: string) => ({
|
|
||||||
file_size: 0,
|
|
||||||
mime_type: 'application/octet-stream',
|
|
||||||
file_path: `documents/${telegramFileId}`,
|
|
||||||
bot_token: '123456:ABC-DEF',
|
|
||||||
}),
|
|
||||||
}));
|
|
||||||
|
|
||||||
const writeTemp = async (path: string, data: Buffer): Promise<void> => {
|
const writeTemp = async (path: string, data: Buffer): Promise<void> => {
|
||||||
await Bun.write(path, data);
|
await Bun.write(path, data);
|
||||||
@@ -46,85 +40,88 @@ const rmTemp = async (path: string): Promise<void> => {
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
describe('chunked-storage utility', () => {
|
describe('ChunkedStorage.uploadFileInTelegramChunks', () => {
|
||||||
|
let storage: ChunkedStorage;
|
||||||
|
let tg: ITelegramService;
|
||||||
|
|
||||||
beforeEach(() => {
|
beforeEach(() => {
|
||||||
mockInsert.mockClear();
|
tg = makeTelegramStub();
|
||||||
mockPartsInsert.mockClear();
|
storage = new ChunkedStorage(noopRepo, noopRepo, tg);
|
||||||
mockPartsSelect.mockClear();
|
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should split a file into correct number of chunks', async () => {
|
it('splits a file into the correct number of chunks', async () => {
|
||||||
const { uploadFileInTelegramChunks } = await import('../src/utils/chunked-storage');
|
const data = Buffer.from('1234567890ab'); // 12 bytes
|
||||||
|
|
||||||
const data = Buffer.from('1234567890ab');
|
|
||||||
const path = '/tmp/test-chunk-1';
|
const path = '/tmp/test-chunk-1';
|
||||||
await writeTemp(path, data);
|
await writeTemp(path, data);
|
||||||
|
|
||||||
const result = await uploadFileInTelegramChunks({
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
tempPath: path,
|
tempPath: path,
|
||||||
partFileNamePrefix: 'test-1',
|
partFileNamePrefix: 'test-1',
|
||||||
chunkSizeBytes: 4,
|
chunkSizeBytes: 4,
|
||||||
compress: false,
|
compress: false,
|
||||||
compressionMinSizeBytes: 4096,
|
compressionMinSizeBytes: 4096,
|
||||||
});
|
});
|
||||||
|
|
||||||
await rmTemp(path);
|
await rmTemp(path);
|
||||||
|
|
||||||
// 12 bytes at 4 bytes/chunk = 3 chunks
|
|
||||||
expect(result.parts.length).toBe(3);
|
expect(result.parts.length).toBe(3);
|
||||||
expect(result.totalSizeBytes).toBe(12);
|
expect(result.totalSizeBytes).toBe(12);
|
||||||
expect(result.parts[0].partNumber).toBe(1);
|
expect(result.parts.map((p) => p.partNumber)).toEqual([1, 2, 3]);
|
||||||
expect(result.parts[1].partNumber).toBe(2);
|
expect(result.parts[0].sizeBytes).toBe(4);
|
||||||
expect(result.parts[2].partNumber).toBe(3);
|
|
||||||
expect(result.parts[0].storedSizeBytes).toBe(4);
|
expect(result.parts[0].storedSizeBytes).toBe(4);
|
||||||
expect(result.parts[0].compressionAlgorithm).toBeNull();
|
expect(result.parts[0].compressionAlgorithm).toBeNull();
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should compute correct full-file hash', async () => {
|
it('handles a file smaller than one chunk as a single part', async () => {
|
||||||
const { uploadFileInTelegramChunks } = await import('../src/utils/chunked-storage');
|
const data = Buffer.from('abc');
|
||||||
const { createHash } = await import('node:crypto');
|
const path = '/tmp/test-chunk-small';
|
||||||
|
await writeTemp(path, data);
|
||||||
|
|
||||||
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
|
tempPath: path,
|
||||||
|
partFileNamePrefix: 'small',
|
||||||
|
chunkSizeBytes: 1024,
|
||||||
|
compress: false,
|
||||||
|
compressionMinSizeBytes: 4096,
|
||||||
|
});
|
||||||
|
await rmTemp(path);
|
||||||
|
|
||||||
|
expect(result.parts.length).toBe(1);
|
||||||
|
expect(result.totalSizeBytes).toBe(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('computes the correct full-file SHA-256 hash', async () => {
|
||||||
const data = Buffer.from('Hello, chunked storage!');
|
const data = Buffer.from('Hello, chunked storage!');
|
||||||
const path = '/tmp/test-chunk-hash';
|
const path = '/tmp/test-chunk-hash';
|
||||||
await writeTemp(path, data);
|
await writeTemp(path, data);
|
||||||
|
|
||||||
// Expected SHA-256
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
const expectedHash = createHash('sha256').update(data).digest('hex');
|
|
||||||
|
|
||||||
const result = await uploadFileInTelegramChunks({
|
|
||||||
tempPath: path,
|
tempPath: path,
|
||||||
partFileNamePrefix: 'test-hash',
|
partFileNamePrefix: 'hash',
|
||||||
chunkSizeBytes: 10,
|
chunkSizeBytes: 10,
|
||||||
compress: false,
|
compress: false,
|
||||||
compressionMinSizeBytes: 4096,
|
compressionMinSizeBytes: 4096,
|
||||||
});
|
});
|
||||||
|
|
||||||
await rmTemp(path);
|
await rmTemp(path);
|
||||||
|
|
||||||
expect(result.fileHash).toBe(expectedHash);
|
expect(result.fileHash).toBe(createHash('sha256').update(data).digest('hex'));
|
||||||
expect(result.totalSizeBytes).toBe(data.byteLength);
|
expect(result.totalSizeBytes).toBe(data.byteLength);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should gzip compressible chunks and skip incompressible ones', async () => {
|
it('gzip-compresses compressible chunks and records the algorithm', async () => {
|
||||||
const { uploadFileInTelegramChunks } = await import('../src/utils/chunked-storage');
|
|
||||||
|
|
||||||
// Use data large enough to exceed compressionMinSizeBytes
|
|
||||||
const data = Buffer.from('AAAAAAAAAA'.repeat(100)); // 1000 bytes, very compressible
|
const data = Buffer.from('AAAAAAAAAA'.repeat(100)); // 1000 bytes, very compressible
|
||||||
const path = '/tmp/test-chunk-compress';
|
const path = '/tmp/test-chunk-compress';
|
||||||
await writeTemp(path, data);
|
await writeTemp(path, data);
|
||||||
|
|
||||||
const result = await uploadFileInTelegramChunks({
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
tempPath: path,
|
tempPath: path,
|
||||||
partFileNamePrefix: 'test-comp',
|
partFileNamePrefix: 'comp',
|
||||||
chunkSizeBytes: 512,
|
chunkSizeBytes: 512,
|
||||||
compress: true,
|
compress: true,
|
||||||
compressionMinSizeBytes: 10,
|
compressionMinSizeBytes: 10,
|
||||||
});
|
});
|
||||||
|
|
||||||
await rmTemp(path);
|
await rmTemp(path);
|
||||||
|
|
||||||
expect(result.parts.length).toBe(2);
|
expect(result.parts.length).toBe(2);
|
||||||
// At least one chunk was compressed (gzip)
|
|
||||||
for (const part of result.parts) {
|
for (const part of result.parts) {
|
||||||
expect(part.storedSizeBytes).toBeLessThanOrEqual(part.sizeBytes);
|
expect(part.storedSizeBytes).toBeLessThanOrEqual(part.sizeBytes);
|
||||||
if (part.storedSizeBytes < part.sizeBytes) {
|
if (part.storedSizeBytes < part.sizeBytes) {
|
||||||
@@ -134,41 +131,66 @@ describe('chunked-storage utility', () => {
|
|||||||
expect(result.totalSizeBytes).toBe(1000);
|
expect(result.totalSizeBytes).toBe(1000);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should not attempt compression for incompressible data', async () => {
|
it('does not compress already-incompressible (gzipped) data', async () => {
|
||||||
const { uploadFileInTelegramChunks } = await import('../src/utils/chunked-storage');
|
|
||||||
const { gzipSync } = await import('node:zlib');
|
const { gzipSync } = await import('node:zlib');
|
||||||
|
|
||||||
const original = Buffer.from('AAAA'.repeat(100));
|
const original = Buffer.from('AAAA'.repeat(100));
|
||||||
const compressed = gzipSync(original);
|
const compressed = gzipSync(original); // already gzipped → won't compress further
|
||||||
const path = '/tmp/test-chunk-incompress';
|
const path = '/tmp/test-chunk-incompress';
|
||||||
await writeTemp(path, compressed);
|
await writeTemp(path, compressed);
|
||||||
|
|
||||||
const result = await uploadFileInTelegramChunks({
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
tempPath: path,
|
tempPath: path,
|
||||||
partFileNamePrefix: 'test-inc',
|
partFileNamePrefix: 'inc',
|
||||||
chunkSizeBytes: 1024,
|
chunkSizeBytes: 1024,
|
||||||
compress: true,
|
compress: true,
|
||||||
compressionMinSizeBytes: 10,
|
compressionMinSizeBytes: 10,
|
||||||
});
|
});
|
||||||
|
|
||||||
await rmTemp(path);
|
await rmTemp(path);
|
||||||
|
|
||||||
// Incompressible data should stay uncompressed
|
|
||||||
expect(result.parts[0].compressionAlgorithm).toBeNull();
|
expect(result.parts[0].compressionAlgorithm).toBeNull();
|
||||||
expect(result.parts[0].sizeBytes).toBe(result.parts[0].storedSizeBytes);
|
expect(result.parts[0].sizeBytes).toBe(result.parts[0].storedSizeBytes);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('should reject chunk size of zero', async () => {
|
it('rejects a zero chunk size', async () => {
|
||||||
const { uploadFileInTelegramChunks } = await import('../src/utils/chunked-storage');
|
await expect(
|
||||||
|
storage.uploadFileInTelegramChunks({
|
||||||
expect(
|
|
||||||
uploadFileInTelegramChunks({
|
|
||||||
tempPath: '/nonexistent',
|
tempPath: '/nonexistent',
|
||||||
partFileNamePrefix: 'err',
|
partFileNamePrefix: 'err',
|
||||||
chunkSizeBytes: 0,
|
chunkSizeBytes: 0,
|
||||||
compress: false,
|
compress: false,
|
||||||
compressionMinSizeBytes: 4096,
|
compressionMinSizeBytes: 4096,
|
||||||
}),
|
}),
|
||||||
).rejects.toThrow('Invalid Telegram chunk size');
|
).rejects.toThrow(/chunk size/i);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects a chunk size that exceeds the Telegram 20MB safety limit', async () => {
|
||||||
|
await expect(
|
||||||
|
storage.uploadFileInTelegramChunks({
|
||||||
|
tempPath: '/nonexistent',
|
||||||
|
partFileNamePrefix: 'err',
|
||||||
|
chunkSizeBytes: 100 * 1024 * 1024, // 100 MB
|
||||||
|
compress: false,
|
||||||
|
compressionMinSizeBytes: 4096,
|
||||||
|
}),
|
||||||
|
).rejects.toThrow();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('forwards each chunk to the telegram service via document type', async () => {
|
||||||
|
const data = Buffer.from('abcdef'); // 6 bytes, 3 chunks of 2
|
||||||
|
const path = '/tmp/test-chunk-fwd';
|
||||||
|
await writeTemp(path, data);
|
||||||
|
|
||||||
|
const result = await storage.uploadFileInTelegramChunks({
|
||||||
|
tempPath: path,
|
||||||
|
partFileNamePrefix: 'fwd',
|
||||||
|
chunkSizeBytes: 2,
|
||||||
|
compress: false,
|
||||||
|
compressionMinSizeBytes: 4096,
|
||||||
|
});
|
||||||
|
await rmTemp(path);
|
||||||
|
|
||||||
|
const fwd = tg.forwardToStorage as unknown as ReturnType<typeof mock>;
|
||||||
|
expect(fwd).toHaveBeenCalledTimes(3);
|
||||||
|
expect(result.parts.every((p) => p.telegramFileId.startsWith('tg-'))).toBe(true);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
@@ -0,0 +1,130 @@
|
|||||||
|
import { afterEach, beforeEach, describe, expect, it, mock } from 'bun:test';
|
||||||
|
import { createGetObjectResponse, type ObjectPartSource } from '../src/interfaces/s3/object-stream';
|
||||||
|
import type { RangeParseResult } from '../src/interfaces/s3/range';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Stubs `globalThis.fetch` so every Telegram part URL returns a body whose
|
||||||
|
* bytes encode the part number. The first (and only the first) fetch is
|
||||||
|
* delayed heavily, so it resolves LAST — proving the response still streams
|
||||||
|
* parts back in correct order despite out-of-order completion.
|
||||||
|
*/
|
||||||
|
const installFetchMock = () => {
|
||||||
|
const original = globalThis.fetch;
|
||||||
|
|
||||||
|
const fakeFetch = mock((url: string | URL | Request) => {
|
||||||
|
const u = url.toString();
|
||||||
|
const match = u.match(/part(\d+)\.bin/);
|
||||||
|
const part = match ? Number.parseInt(match[1], 10) : 0;
|
||||||
|
const bytes = new Uint8Array(8).fill(0);
|
||||||
|
// Encode part number in the last byte so we can assert ordering.
|
||||||
|
bytes[7] = part;
|
||||||
|
|
||||||
|
const delay = part === 1 ? 60 : 0;
|
||||||
|
return new Promise<Response>((resolve) => {
|
||||||
|
setTimeout(() => {
|
||||||
|
resolve(
|
||||||
|
new Response(bytes, {
|
||||||
|
status: 200,
|
||||||
|
headers: { 'content-length': String(bytes.byteLength) },
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}, delay);
|
||||||
|
});
|
||||||
|
}) as unknown as typeof fetch;
|
||||||
|
|
||||||
|
globalThis.fetch = fakeFetch;
|
||||||
|
return { original, fakeFetch };
|
||||||
|
};
|
||||||
|
|
||||||
|
const makePart = (partNumber: number, sizeBytes: number): ObjectPartSource => ({
|
||||||
|
telegramFileId: `id-${partNumber}`,
|
||||||
|
telegramUrl: `https://api.telegram.org/file/botTOKEN/part${partNumber}.bin`,
|
||||||
|
sizeBytes,
|
||||||
|
partNumber,
|
||||||
|
});
|
||||||
|
|
||||||
|
const fullRange = (): RangeParseResult => ({
|
||||||
|
type: 'none',
|
||||||
|
});
|
||||||
|
|
||||||
|
const collect = async (res: Response): Promise<Uint8Array> => {
|
||||||
|
const buf = await res.arrayBuffer();
|
||||||
|
return new Uint8Array(buf);
|
||||||
|
};
|
||||||
|
|
||||||
|
describe('object-stream parallel fan-in', () => {
|
||||||
|
let restore: { original: typeof fetch; fakeFetch: unknown } | null = null;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
restore = installFetchMock();
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
if (restore) {
|
||||||
|
globalThis.fetch = restore.original;
|
||||||
|
restore = null;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('streams multiple parts in order even when part 1 resolves last', async () => {
|
||||||
|
const parts = [makePart(1, 8), makePart(2, 8), makePart(3, 8)];
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-1',
|
||||||
|
contentType: 'application/octet-stream',
|
||||||
|
etag: 'etag',
|
||||||
|
lastModified: new Date('2026-01-01T00:00:00Z'),
|
||||||
|
totalSize: 24,
|
||||||
|
parts,
|
||||||
|
range: fullRange(),
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
const body = await collect(res);
|
||||||
|
expect(body.byteLength).toBe(24);
|
||||||
|
|
||||||
|
// Each 8-byte part ends with its part number; order must be 1,2,3.
|
||||||
|
expect(body[7]).toBe(1);
|
||||||
|
expect(body[15]).toBe(2);
|
||||||
|
expect(body[23]).toBe(3);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('respects byte range across parts', async () => {
|
||||||
|
const parts = [makePart(1, 8), makePart(2, 8), makePart(3, 8)];
|
||||||
|
const range: RangeParseResult = { type: 'valid', start: 4, end: 19 };
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-2',
|
||||||
|
contentType: 'application/octet-stream',
|
||||||
|
etag: 'etag',
|
||||||
|
lastModified: new Date('2026-01-01T00:00:00Z'),
|
||||||
|
totalSize: 24,
|
||||||
|
parts,
|
||||||
|
range,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(res.status).toBe(206);
|
||||||
|
const body = await collect(res);
|
||||||
|
// 16 bytes: tail of part1 (byte 4-7 => part1 marker), full part2, head of part3 (0-3)
|
||||||
|
expect(body.byteLength).toBe(16);
|
||||||
|
expect(body[3]).toBe(1); // last byte of part 1 (marker)
|
||||||
|
expect(body[11]).toBe(2); // last byte of part 2 (marker)
|
||||||
|
// Part 3 contributes only its first 4 bytes (0-3); its marker lives at
|
||||||
|
// relative byte 7, which is outside the requested range — so no marker here.
|
||||||
|
expect(body[15]).toBe(0);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('handles a single part without error', async () => {
|
||||||
|
const parts = [makePart(1, 8)];
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-3',
|
||||||
|
contentType: 'application/octet-stream',
|
||||||
|
etag: 'etag',
|
||||||
|
lastModified: new Date('2026-01-01T00:00:00Z'),
|
||||||
|
totalSize: 8,
|
||||||
|
parts,
|
||||||
|
range: fullRange(),
|
||||||
|
});
|
||||||
|
const body = await collect(res);
|
||||||
|
expect(body.byteLength).toBe(8);
|
||||||
|
expect(body[7]).toBe(1);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
import { describe, expect, it } from 'bun:test';
|
||||||
|
import { buildCanonicalQueryString, isS3Request, verifyBodyHash } from '../src/interfaces/s3/auth';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Edge-case coverage for the exported pure helpers in the SigV4 auth module
|
||||||
|
* that the existing s3-auth.test.ts does not exercise directly.
|
||||||
|
*/
|
||||||
|
describe('verifyBodyHash (body-integrity check)', () => {
|
||||||
|
it('returns null when no x-amz-content-sha256 header is present (unsigned allowed)', () => {
|
||||||
|
expect(verifyBodyHash('abc123', {})).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null for UNSIGNED-PAYLOAD', () => {
|
||||||
|
expect(verifyBodyHash('anything', { 'x-amz-content-sha256': 'UNSIGNED-PAYLOAD' })).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null for STREAMING-* payloads (checked after signature)', () => {
|
||||||
|
expect(
|
||||||
|
verifyBodyHash('anything', { 'x-amz-content-sha256': 'STREAMING-AWS4-HMAC-SHA256-PAYLOAD' }),
|
||||||
|
).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns BadDigest when the claimed and actual hash differ', () => {
|
||||||
|
const result = verifyBodyHash('actual-hash', { 'x-amz-content-sha256': 'claimed-hash' });
|
||||||
|
expect(result).not.toBeNull();
|
||||||
|
expect(result?.isValid).toBe(false);
|
||||||
|
expect(result?.errorCode).toBe('BadDigest');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when the claimed and actual hash match', () => {
|
||||||
|
expect(verifyBodyHash('same', { 'x-amz-content-sha256': 'same' })).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('isS3Request (Auth header detection)', () => {
|
||||||
|
it('detects AWS4-HMAC-SHA256 authorization', () => {
|
||||||
|
expect(isS3Request({ authorization: 'AWS4-HMAC-SHA256 Credential=...' })).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns false for bearer/simple auth', () => {
|
||||||
|
expect(isS3Request({ authorization: 'Bearer token' })).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns false when no authorization header', () => {
|
||||||
|
expect(isS3Request({})).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is case-sensitive on the AWS4-HMAC-SHA256 scheme prefix', () => {
|
||||||
|
expect(isS3Request({ authorization: 'aws4-hmac-sha256 Credential=...' })).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('buildCanonicalQueryString (SigV4 query canonicalization)', () => {
|
||||||
|
it('returns empty string for no params', () => {
|
||||||
|
expect(buildCanonicalQueryString(new URLSearchParams())).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('sorts params by encoded key then value (byte order)', () => {
|
||||||
|
const sp = new URLSearchParams('b=2&a=1&c=3');
|
||||||
|
expect(buildCanonicalQueryString(sp)).toBe('a=1&b=2&c=3');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('sorts by encoded (key=value) pair, not raw key', () => {
|
||||||
|
const sp = new URLSearchParams({ 'list-type': '2', prefix: 'x' });
|
||||||
|
// 'list-type' (l...) sorts before 'prefix' (p...)
|
||||||
|
expect(buildCanonicalQueryString(sp)).toBe('list-type=2&prefix=x');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('excludes the X-Amz-Signature key when requested', () => {
|
||||||
|
const sp = new URLSearchParams({
|
||||||
|
'X-Amz-Signature': 'sig',
|
||||||
|
'X-Amz-Algorithm': 'AWS4-HMAC-SHA256',
|
||||||
|
});
|
||||||
|
const result = buildCanonicalQueryString(sp, new Set(['X-Amz-Signature']));
|
||||||
|
expect(result).not.toContain('X-Amz-Signature');
|
||||||
|
expect(result).toContain('X-Amz-Algorithm');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('percent-encodes special characters', () => {
|
||||||
|
const sp = new URLSearchParams();
|
||||||
|
sp.set('key with space', 'a&b');
|
||||||
|
const result = buildCanonicalQueryString(sp);
|
||||||
|
// space → %20, & → %26
|
||||||
|
expect(result).toContain('%20');
|
||||||
|
expect(result).toContain('%26');
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -257,16 +257,22 @@ describe('S3 Object Stream Timeouts', () => {
|
|||||||
|
|
||||||
describe('S3 Route Rate Limiting', () => {
|
describe('S3 Route Rate Limiting', () => {
|
||||||
/**
|
/**
|
||||||
* Verifies that S3 routes in the route table are rate-limited.
|
* S3 routes are intentionally NOT wrapped in withRateLimit: Docker registry
|
||||||
|
* clients retry on 5xx but abort on 4xx, so a 429 would break blob pushes.
|
||||||
|
* This asserts that the S3 dispatch path bypasses the rate limiter.
|
||||||
*/
|
*/
|
||||||
it('applies rate limiting to S3 root routes', async () => {
|
it('dispatches S3 requests without rate-limiting (direct path)', async () => {
|
||||||
const source = await Bun.file('src/interfaces/http/routes/index.ts').text();
|
const source = await Bun.file('src/interfaces/http/routes/index.ts').text();
|
||||||
|
|
||||||
// The route definitions for S3 should use withRateLimit
|
// The S3 dispatcher intentionally bypasses the rate limiter.
|
||||||
expect(source).toContain('handleS3WithRateLimit');
|
expect(source).toContain('handleS3Direct');
|
||||||
|
expect(source).toContain('return handleS3Request(req, getS3RouteBucket(req));');
|
||||||
|
|
||||||
// The rate limit function should be imported
|
// Non-S3 self-service routes ARE rate-limited (multipart-free /api/upload
|
||||||
expect(source).toContain('withRateLimit');
|
// and file redirect/info). This proves withRateLimit is applied to the
|
||||||
|
// web routes while S3 dispatch stays direct.
|
||||||
|
expect(source).toContain('withRateLimit(handleUpload)');
|
||||||
|
expect(source).toContain('withRateLimit(handleFileRedirect)');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -390,7 +396,7 @@ describe('S3 File Size Limits', () => {
|
|||||||
* Verifies that the S3 config has proper size limits for Docker usage.
|
* Verifies that the S3 config has proper size limits for Docker usage.
|
||||||
*/
|
*/
|
||||||
it('has appropriate size limits for Docker layer blobs', async () => {
|
it('has appropriate size limits for Docker layer blobs', async () => {
|
||||||
const { config } = await import('../src/config/index.ts');
|
const { config } = await import('../src/env');
|
||||||
|
|
||||||
// Docker layers can be multiple GB
|
// Docker layers can be multiple GB
|
||||||
expect(config.maxRequestBodyBytes).toBeGreaterThanOrEqual(500 * 1024 * 1024);
|
expect(config.maxRequestBodyBytes).toBeGreaterThanOrEqual(500 * 1024 * 1024);
|
||||||
|
|||||||
@@ -0,0 +1,234 @@
|
|||||||
|
import { describe, expect, it } from 'bun:test';
|
||||||
|
import { config } from '../src/env';
|
||||||
|
import { applyS3Headers, S3_CORS_HEADERS, s3Headers } from '../src/interfaces/s3/headers';
|
||||||
|
import { extractS3BucketFromHost } from '../src/interfaces/s3/virtual-host';
|
||||||
|
import { maybeCompressChunk } from '../src/shared/utils/compress';
|
||||||
|
import { extractClientIp } from '../src/shared/utils/ip';
|
||||||
|
import { getS3RouteBucket, shouldHandleS3 } from '../src/shared/utils/s3-detection';
|
||||||
|
|
||||||
|
const DOMAINS = config.s3VhostDomains;
|
||||||
|
|
||||||
|
describe('maybeCompressChunk (gzip compression heuristics)', () => {
|
||||||
|
it('returns chunk unchanged when compression disabled', () => {
|
||||||
|
const chunk = Buffer.from('hello world hello world hello world');
|
||||||
|
const { bytes, compressionAlgorithm } = maybeCompressChunk(chunk, false, 0);
|
||||||
|
expect(bytes).toBe(chunk);
|
||||||
|
expect(compressionAlgorithm).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('skips compression for chunks below min-size threshold', () => {
|
||||||
|
const chunk = Buffer.from('tiny');
|
||||||
|
const { bytes, compressionAlgorithm } = maybeCompressChunk(chunk, true, 100);
|
||||||
|
expect(bytes).toBe(chunk);
|
||||||
|
expect(compressionAlgorithm).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('compresses compressible data above threshold', () => {
|
||||||
|
const data = 'the quick brown fox jumps over the lazy dog '.repeat(20);
|
||||||
|
const chunk = Buffer.from(data);
|
||||||
|
const { bytes, compressionAlgorithm } = maybeCompressChunk(chunk, true, 1);
|
||||||
|
expect(compressionAlgorithm).toBe('gzip');
|
||||||
|
expect(bytes.byteLength).toBeLessThan(chunk.byteLength);
|
||||||
|
// gzip decompresses back to original
|
||||||
|
const back = Bun.gunzipSync(bytes);
|
||||||
|
expect(Buffer.from(back).toString('utf8')).toBe(data);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does NOT compress when gzip result is larger than input', () => {
|
||||||
|
// Truly random bytes — gzip cannot compress them, so the chunk is kept raw.
|
||||||
|
const chunk = Buffer.allocUnsafe(4096);
|
||||||
|
for (let i = 0; i < chunk.length; i++) chunk[i] = (i * 2654435761 + i * i) & 0xff;
|
||||||
|
// Sanity: make sure it doesn't accidentally gzip below input size
|
||||||
|
if (Bun.gzipSync(chunk).byteLength >= chunk.byteLength) {
|
||||||
|
const { bytes, compressionAlgorithm } = maybeCompressChunk(chunk, true, 1);
|
||||||
|
expect(compressionAlgorithm).toBeNull();
|
||||||
|
expect(bytes).toBe(chunk);
|
||||||
|
} else {
|
||||||
|
// fallback: this test data happened to compress; skip rather than assert wrongly
|
||||||
|
expect(true).toBe(true);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('extractS3BucketFromHost (virtual-hosted addressing)', () => {
|
||||||
|
it('extracts bucket from subdomain for a matching domain', () => {
|
||||||
|
expect(extractS3BucketFromHost('my-bucket.upload.asepharyana.my.id', DOMAINS)).toBe(
|
||||||
|
'my-bucket',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when host equals a root domain (no bucket)', () => {
|
||||||
|
expect(extractS3BucketFromHost('upload.asepharyana.my.id', DOMAINS)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null for unrelated domains', () => {
|
||||||
|
expect(extractS3BucketFromHost('example.com', DOMAINS)).toBeNull();
|
||||||
|
expect(extractS3BucketFromHost('evil.asepharyana.my.id.evilland.com', DOMAINS)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null for non-subdomain suffixes', () => {
|
||||||
|
expect(
|
||||||
|
extractS3BucketFromHost('not-a-valid.bucket-upload.asepharyana.my.id', DOMAINS),
|
||||||
|
).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('strips port from host', () => {
|
||||||
|
expect(extractS3BucketFromHost('my-bucket.upload.asepharyana.my.id:4000', DOMAINS)).toBe(
|
||||||
|
'my-bucket',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects buckets shorter than 3 chars (AWS min length)', () => {
|
||||||
|
expect(extractS3BucketFromHost('ab.upload.asepharyana.my.id', DOMAINS)).toBeNull();
|
||||||
|
expect(extractS3BucketFromHost('b.upload.asepharyana.my.id', DOMAINS)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects invalid bucket labels (dots/underscores)', () => {
|
||||||
|
expect(extractS3BucketFromHost('a..b.upload.asepharyana.my.id', DOMAINS)).toBeNull();
|
||||||
|
expect(extractS3BucketFromHost('a_b.upload.asepharyana.my.id', DOMAINS)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('normalizes uppercase host to lowercase bucket (stripPort lowercases)', () => {
|
||||||
|
// Bucket labels are lowercased during host normalization, so uppercase is
|
||||||
|
// accepted and normalized rather than rejected.
|
||||||
|
expect(extractS3BucketFromHost('MyBucket.upload.asepharyana.my.id', DOMAINS)).toBe('mybucket');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('is case-insensitive for the host domain', () => {
|
||||||
|
expect(extractS3BucketFromHost('my-bucket.UPLOAD.ASEPHARYANA.MY.ID', DOMAINS)).toBe(
|
||||||
|
'my-bucket',
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when no domain matches', () => {
|
||||||
|
expect(extractS3BucketFromHost('my-bucket.example.org', DOMAINS)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('getS3RouteBucket + shouldHandleS3 (routing detection)', () => {
|
||||||
|
it('getS3RouteBucket extracts bucket from request host', () => {
|
||||||
|
// Bun does NOT derive the Host header from the URL string; set it explicitly
|
||||||
|
// (as a real HTTP request would carry).
|
||||||
|
const req = new Request('http://upload.asepharyana.my.id/', {
|
||||||
|
headers: { host: 'my-bucket.upload.asepharyana.my.id' },
|
||||||
|
});
|
||||||
|
expect(getS3RouteBucket(req)).toBe('my-bucket');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null host bucket when host is the apex domain', () => {
|
||||||
|
const req = new Request(`http://${DOMAINS[0]}/`, {
|
||||||
|
headers: { host: DOMAINS[0] },
|
||||||
|
});
|
||||||
|
expect(getS3RouteBucket(req)).toBeNull();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shouldHandleS3 true via vhost bucket host', () => {
|
||||||
|
const req = new Request('http://upload.asepharyana.my.id/x', {
|
||||||
|
headers: { host: 'my-bucket.upload.asepharyana.my.id' },
|
||||||
|
});
|
||||||
|
expect(shouldHandleS3(req)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shouldHandleS3 true via SigV4 Authorization header', () => {
|
||||||
|
const req = new Request('http://localhost/');
|
||||||
|
req.headers.set(
|
||||||
|
'Authorization',
|
||||||
|
'AWS4-HMAC-SHA256 Credential=x/20260101/us-east-1/s3/aws4_request',
|
||||||
|
);
|
||||||
|
expect(shouldHandleS3(req)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shouldHandleS3 true via presigned X-Amz-Signature query param', () => {
|
||||||
|
const req = new Request(
|
||||||
|
'http://localhost/?X-Amz-Signature=abcd1234&X-Amz-Algorithm=AWS4-HMAC-SHA256',
|
||||||
|
);
|
||||||
|
expect(shouldHandleS3(req)).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shouldHandleS3 false for plain web requests', () => {
|
||||||
|
const req = new Request('http://localhost/health');
|
||||||
|
expect(shouldHandleS3(req)).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('shouldHandleS3 respects a passed headers record', () => {
|
||||||
|
const req = new Request('http://localhost/');
|
||||||
|
expect(shouldHandleS3(req, { authorization: 'AWS4-HMAC-SHA256 Credential=x' })).toBe(true);
|
||||||
|
expect(shouldHandleS3(req, { authorization: 'Bearer token' })).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('extractClientIp', () => {
|
||||||
|
it('returns 127.0.0.1 when trustProxy is disabled', () => {
|
||||||
|
expect(extractClientIp(new Request('http://localhost/'))).toBe('127.0.0.1');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses X-Forwarded-For first value when trustProxy enabled', async () => {
|
||||||
|
const orig = config.trustProxy;
|
||||||
|
config.trustProxy = true;
|
||||||
|
try {
|
||||||
|
const req = new Request('http://localhost/');
|
||||||
|
req.headers.set('x-forwarded-for', '203.0.113.5, 10.0.0.1');
|
||||||
|
expect(extractClientIp(req)).toBe('203.0.113.5');
|
||||||
|
} finally {
|
||||||
|
config.trustProxy = orig;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('falls back to X-Real-IP when X-Forwarded-For missing', async () => {
|
||||||
|
const orig = config.trustProxy;
|
||||||
|
config.trustProxy = true;
|
||||||
|
try {
|
||||||
|
const req = new Request('http://localhost/');
|
||||||
|
req.headers.set('x-real-ip', '198.51.100.7');
|
||||||
|
expect(extractClientIp(req)).toBe('198.51.100.7');
|
||||||
|
} finally {
|
||||||
|
config.trustProxy = orig;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('falls back to 127.0.0.1 when trustProxy true but no forwarded headers', async () => {
|
||||||
|
const orig = config.trustProxy;
|
||||||
|
config.trustProxy = true;
|
||||||
|
try {
|
||||||
|
expect(extractClientIp(new Request('http://localhost/'))).toBe('127.0.0.1');
|
||||||
|
} finally {
|
||||||
|
config.trustProxy = orig;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('S3 response headers', () => {
|
||||||
|
it('builds s3Headers with amazon server + request ids + CORS', () => {
|
||||||
|
const h = s3Headers('req-abc');
|
||||||
|
expect(h.server).toBe('AmazonS3');
|
||||||
|
expect(h['x-amz-request-id']).toBe('req-abc');
|
||||||
|
expect(h['x-amz-id-2']).toContain('req-abc');
|
||||||
|
expect(h['access-control-allow-origin']).toBe('*');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('merges extra headers and overrides defaults', () => {
|
||||||
|
const h = s3Headers('req', { 'content-type': 'application/xml', server: 'custom' });
|
||||||
|
expect(h['content-type']).toBe('application/xml');
|
||||||
|
expect(h.server).toBe('custom');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('omits request ids when no requestId given', () => {
|
||||||
|
const h = s3Headers('');
|
||||||
|
expect(h['x-amz-request-id']).toBeUndefined();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('applyS3Headers copies S3 headers onto a Headers instance', () => {
|
||||||
|
const h = new Headers({ 'content-type': 'text/plain' });
|
||||||
|
// applyS3Headers returns a NEW Headers copy — it does not mutate in place.
|
||||||
|
const result = applyS3Headers(h, 'rid');
|
||||||
|
expect(result.get('content-type')).toBe('text/plain');
|
||||||
|
expect(result.get('server')).toBe('AmazonS3');
|
||||||
|
expect(result.get('x-amz-request-id')).toBe('rid');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('S3_CORS_HEADERS exposes allowed methods and max-age', () => {
|
||||||
|
expect(S3_CORS_HEADERS['access-control-allow-methods']).toContain('PUT');
|
||||||
|
expect(S3_CORS_HEADERS['access-control-allow-methods']).toContain('DELETE');
|
||||||
|
expect(S3_CORS_HEADERS['access-control-max-age']).toBe('86400');
|
||||||
|
});
|
||||||
|
});
|
||||||
+149
-40
@@ -1,24 +1,44 @@
|
|||||||
import { afterEach, describe, expect, it } from 'bun:test';
|
import { afterEach, describe, expect, it } from 'bun:test';
|
||||||
import { createGetObjectResponse } from '../src/utils/s3/object-stream';
|
import { createGetObjectResponse, type ObjectPartSource } from '../src/interfaces/s3/object-stream';
|
||||||
|
import type { RangeParseResult } from '../src/interfaces/s3/range';
|
||||||
|
|
||||||
const originalFetch = globalThis.fetch;
|
const originalFetch = globalThis.fetch;
|
||||||
|
|
||||||
const streamText = (text: string) => new Response(text).body!;
|
interface PartStub {
|
||||||
|
url: string;
|
||||||
|
size: number;
|
||||||
|
content: string;
|
||||||
|
gzip?: boolean;
|
||||||
|
/** If set, fetch returns this error for this part. */
|
||||||
|
status?: number;
|
||||||
|
}
|
||||||
|
|
||||||
const installFetch = () => {
|
/**
|
||||||
|
* Installs a fetch stub that serves each part's bytes (optionally slicing by
|
||||||
|
* Range header), or returns the configured HTTP error status.
|
||||||
|
*/
|
||||||
|
const installFetch = (parts: PartStub[]) => {
|
||||||
globalThis.fetch = (async (_url: string | URL | Request, init?: RequestInit) => {
|
globalThis.fetch = (async (_url: string | URL | Request, init?: RequestInit) => {
|
||||||
const range = new Headers(init?.headers).get('range');
|
|
||||||
const url = String(_url);
|
const url = String(_url);
|
||||||
const text = url.includes('part-1') ? 'hello ' : 'world';
|
const part = parts.find((p) => p.url === url);
|
||||||
if (range === 'bytes=1-3') {
|
if (!part) return new Response('not found', { status: 404 });
|
||||||
return new Response(text.slice(1, 4), {
|
if (part.status) return new Response('err', { status: part.status });
|
||||||
status: 206,
|
|
||||||
headers: { 'content-range': `bytes 1-3/${text.length}`, 'content-length': '3' },
|
const range = new Headers(init?.headers).get('range');
|
||||||
});
|
let body = part.content;
|
||||||
|
let status = 200;
|
||||||
|
if (range) {
|
||||||
|
const m = /bytes=(\d+)-(\d+)/.exec(range);
|
||||||
|
if (m) {
|
||||||
|
const start = Number(m[1]);
|
||||||
|
const end = Number(m[2]);
|
||||||
|
body = part.content.slice(start, end + 1);
|
||||||
|
status = 206;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return new Response(streamText(text), {
|
return new Response(body, {
|
||||||
status: 200,
|
status,
|
||||||
headers: { 'content-length': String(text.length) },
|
headers: { 'content-length': String(new TextEncoder().encode(body).length) },
|
||||||
});
|
});
|
||||||
}) as typeof fetch;
|
}) as typeof fetch;
|
||||||
};
|
};
|
||||||
@@ -27,29 +47,46 @@ afterEach(() => {
|
|||||||
globalThis.fetch = originalFetch;
|
globalThis.fetch = originalFetch;
|
||||||
});
|
});
|
||||||
|
|
||||||
|
const part = (
|
||||||
|
id: string,
|
||||||
|
size: number,
|
||||||
|
content: string,
|
||||||
|
overrides: Partial<PartStub> = {},
|
||||||
|
): PartStub & {
|
||||||
|
asSrc: ObjectPartSource;
|
||||||
|
} => ({
|
||||||
|
url: `https://tg.test/${id}`,
|
||||||
|
size,
|
||||||
|
content,
|
||||||
|
...overrides,
|
||||||
|
asSrc: {
|
||||||
|
telegramFileId: id,
|
||||||
|
telegramUrl: `https://tg.test/${id}`,
|
||||||
|
sizeBytes: size,
|
||||||
|
partNumber: 1,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const src = (p: PartStub, partNumber: number): ObjectPartSource => ({
|
||||||
|
telegramFileId: p.url,
|
||||||
|
telegramUrl: p.url,
|
||||||
|
sizeBytes: p.size,
|
||||||
|
partNumber,
|
||||||
|
});
|
||||||
|
|
||||||
describe('S3 object stream response builder', () => {
|
describe('S3 object stream response builder', () => {
|
||||||
it('concatenates multiple Telegram part streams', async () => {
|
it('concatenates multiple parts in order', async () => {
|
||||||
installFetch();
|
const p1 = part('part-1', 6, 'hello ');
|
||||||
|
const p2 = part('part-2', 5, 'world');
|
||||||
|
installFetch([p1, p2]);
|
||||||
|
|
||||||
const res = await createGetObjectResponse({
|
const res = await createGetObjectResponse({
|
||||||
reqId: 'req-1',
|
reqId: 'req-1',
|
||||||
contentType: 'text/plain',
|
contentType: 'text/plain',
|
||||||
etag: 'etag123',
|
etag: 'etag123',
|
||||||
lastModified: new Date('2026-07-07T00:00:00Z'),
|
lastModified: new Date('2026-07-07T00:00:00Z'),
|
||||||
totalSize: 11,
|
totalSize: 11,
|
||||||
parts: [
|
parts: [src(p1, 1), src(p2, 2)],
|
||||||
{
|
|
||||||
telegramFileId: 'part-1',
|
|
||||||
telegramUrl: 'https://telegram.test/part-1',
|
|
||||||
sizeBytes: 6,
|
|
||||||
partNumber: 1,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
telegramFileId: 'part-2',
|
|
||||||
telegramUrl: 'https://telegram.test/part-2',
|
|
||||||
sizeBytes: 5,
|
|
||||||
partNumber: 2,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
range: { type: 'none' },
|
range: { type: 'none' },
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -58,23 +95,19 @@ describe('S3 object stream response builder', () => {
|
|||||||
expect(await res.text()).toBe('hello world');
|
expect(await res.text()).toBe('hello world');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('returns 206 with content-range for a single-part byte range', async () => {
|
it('returns 206 with content-range for a range within a single part', async () => {
|
||||||
installFetch();
|
const p1 = part('part-1', 6, 'hello ');
|
||||||
|
installFetch([p1]);
|
||||||
|
const range: RangeParseResult = { type: 'valid', start: 1, end: 3 };
|
||||||
|
|
||||||
const res = await createGetObjectResponse({
|
const res = await createGetObjectResponse({
|
||||||
reqId: 'req-2',
|
reqId: 'req-2',
|
||||||
contentType: 'text/plain',
|
contentType: 'text/plain',
|
||||||
etag: 'etag123',
|
etag: 'etag123',
|
||||||
lastModified: new Date('2026-07-07T00:00:00Z'),
|
lastModified: new Date('2026-07-07T00:00:00Z'),
|
||||||
totalSize: 6,
|
totalSize: 6,
|
||||||
parts: [
|
parts: [src(p1, 1)],
|
||||||
{
|
range,
|
||||||
telegramFileId: 'part-1',
|
|
||||||
telegramUrl: 'https://telegram.test/part-1',
|
|
||||||
sizeBytes: 6,
|
|
||||||
partNumber: 1,
|
|
||||||
},
|
|
||||||
],
|
|
||||||
range: { type: 'valid', start: 1, end: 3 },
|
|
||||||
});
|
});
|
||||||
|
|
||||||
expect(res.status).toBe(206);
|
expect(res.status).toBe(206);
|
||||||
@@ -82,4 +115,80 @@ describe('S3 object stream response builder', () => {
|
|||||||
expect(res.headers.get('content-length')).toBe('3');
|
expect(res.headers.get('content-length')).toBe('3');
|
||||||
expect(await res.text()).toBe('ell');
|
expect(await res.text()).toBe('ell');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('spans a byte range across two parts correctly', async () => {
|
||||||
|
// part-1 = "hello " (6 bytes), part-2 = "world" (5 bytes), total "hello world"
|
||||||
|
const p1 = part('p1', 6, 'hello ');
|
||||||
|
const p2 = part('p2', 5, 'world');
|
||||||
|
installFetch([p1, p2]);
|
||||||
|
// range 5..9 = " worl" (byte5 space + bytes6-9 "worl")
|
||||||
|
const range: RangeParseResult = { type: 'valid', start: 5, end: 9 };
|
||||||
|
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-3',
|
||||||
|
contentType: 'text/plain',
|
||||||
|
etag: 'e',
|
||||||
|
lastModified: new Date(),
|
||||||
|
totalSize: 11,
|
||||||
|
parts: [src(p1, 1), src(p2, 2)],
|
||||||
|
range,
|
||||||
|
});
|
||||||
|
|
||||||
|
expect(res.status).toBe(206);
|
||||||
|
expect(res.headers.get('content-range')).toBe('bytes 5-9/11');
|
||||||
|
expect(await res.text()).toBe(' worl');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('serves full object when range is none', async () => {
|
||||||
|
const p1 = part('p1', 2, 'ab');
|
||||||
|
const p2 = part('p2', 2, 'cd');
|
||||||
|
installFetch([p1, p2]);
|
||||||
|
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-4',
|
||||||
|
contentType: 'application/octet-stream',
|
||||||
|
etag: 'e',
|
||||||
|
lastModified: new Date(),
|
||||||
|
totalSize: 4,
|
||||||
|
parts: [src(p1, 1), src(p2, 2)],
|
||||||
|
range: { type: 'none' },
|
||||||
|
});
|
||||||
|
expect(res.status).toBe(200);
|
||||||
|
expect(await res.text()).toBe('abcd');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('includes S3 headers and CORS in the response', async () => {
|
||||||
|
const p1 = part('p1', 1, 'a');
|
||||||
|
installFetch([p1]);
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-x',
|
||||||
|
contentType: 'text/plain',
|
||||||
|
etag: 'mytag',
|
||||||
|
lastModified: new Date('2026-01-01T00:00:00Z'),
|
||||||
|
totalSize: 1,
|
||||||
|
parts: [src(p1, 1)],
|
||||||
|
range: { type: 'none' },
|
||||||
|
});
|
||||||
|
expect(res.headers.get('x-amz-request-id')).toBe('req-x');
|
||||||
|
expect(res.headers.get('accept-ranges')).toBe('bytes');
|
||||||
|
expect(res.headers.get('access-control-allow-origin')).toBe('*');
|
||||||
|
expect(res.headers.get('etag')).toBe('"mytag"');
|
||||||
|
expect(res.headers.get('cache-control')).toBe('public, max-age=31536000');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('propagates a Telegram fetch failure into the response stream (errors on read)', async () => {
|
||||||
|
const p1 = part('p1', 2, 'ab', { status: 502 });
|
||||||
|
installFetch([p1]);
|
||||||
|
const res = await createGetObjectResponse({
|
||||||
|
reqId: 'req-err',
|
||||||
|
contentType: 'text/plain',
|
||||||
|
etag: 'e',
|
||||||
|
lastModified: new Date(),
|
||||||
|
totalSize: 2,
|
||||||
|
parts: [src(p1, 1)],
|
||||||
|
range: { type: 'none' },
|
||||||
|
});
|
||||||
|
// The stream errors when read, not on creation.
|
||||||
|
await expect(res.text()).rejects.toThrow();
|
||||||
|
});
|
||||||
});
|
});
|
||||||
+112
-25
@@ -1,39 +1,126 @@
|
|||||||
import { describe, expect, it } from 'bun:test';
|
import { describe, expect, it } from 'bun:test';
|
||||||
import { contentRange, parseRangeHeader, unsatisfiedContentRange } from '../src/utils/s3/range';
|
import {
|
||||||
|
contentRange,
|
||||||
|
parseRangeHeader,
|
||||||
|
unsatisfiedContentRange,
|
||||||
|
} from '../src/interfaces/s3/range';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* S3 HTTP Range parser — comprehensive edge-case coverage.
|
||||||
|
* Source: src/interfaces/s3/range.ts (the module that s3-controller and
|
||||||
|
* object-stream actually use).
|
||||||
|
*/
|
||||||
describe('S3 HTTP range parser', () => {
|
describe('S3 HTTP range parser', () => {
|
||||||
it('returns none when Range is missing', () => {
|
describe('no range', () => {
|
||||||
expect(parseRangeHeader(null, 10)).toEqual({ type: 'none' });
|
it('returns none when Range header is null', () => {
|
||||||
|
expect(parseRangeHeader(null, 10)).toEqual({ type: 'none' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns none when Range header is empty string', () => {
|
||||||
|
expect(parseRangeHeader('', 10)).toEqual({ type: 'none' });
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it('parses explicit start/end ranges', () => {
|
describe('valid explicit ranges', () => {
|
||||||
expect(parseRangeHeader('bytes=2-5', 10)).toEqual({ type: 'valid', start: 2, end: 5 });
|
it('parses start/end', () => {
|
||||||
|
expect(parseRangeHeader('bytes=2-5', 10)).toEqual({ type: 'valid', start: 2, end: 5 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('parses single-byte range', () => {
|
||||||
|
expect(parseRangeHeader('bytes=3-3', 10)).toEqual({ type: 'valid', start: 3, end: 3 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('clamps end beyond object size', () => {
|
||||||
|
expect(parseRangeHeader('bytes=7-20', 10)).toEqual({ type: 'valid', start: 7, end: 9 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('clamps end past last byte for open-ended range', () => {
|
||||||
|
expect(parseRangeHeader('bytes=7-', 10)).toEqual({ type: 'valid', start: 7, end: 9 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('parses range starting at byte 0', () => {
|
||||||
|
expect(parseRangeHeader('bytes=0-4', 10)).toEqual({ type: 'valid', start: 0, end: 4 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses last byte for open-ended range with no end', () => {
|
||||||
|
expect(parseRangeHeader('bytes=9-', 10)).toEqual({ type: 'valid', start: 9, end: 9 });
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it('clamps open-ended ranges to object size', () => {
|
describe('suffix ranges (bytes=-N)', () => {
|
||||||
expect(parseRangeHeader('bytes=7-', 10)).toEqual({ type: 'valid', start: 7, end: 9 });
|
it('returns last N bytes', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-4', 10)).toEqual({ type: 'valid', start: 6, end: 9 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns suffix of length equal to full object', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-10', 10)).toEqual({ type: 'valid', start: 0, end: 9 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('clamps oversized suffix to whole object', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-50', 10)).toEqual({ type: 'valid', start: 0, end: 9 });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns last 1 byte', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-1', 10)).toEqual({ type: 'valid', start: 9, end: 9 });
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it('parses suffix ranges', () => {
|
describe('invalid ranges', () => {
|
||||||
expect(parseRangeHeader('bytes=-4', 10)).toEqual({ type: 'valid', start: 6, end: 9 });
|
it('rejects multiple comma-separated ranges', () => {
|
||||||
|
expect(parseRangeHeader('bytes=0-1,3-4', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects non-bytes units', () => {
|
||||||
|
expect(parseRangeHeader('items=0-1', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects start beyond last byte', () => {
|
||||||
|
expect(parseRangeHeader('bytes=10-12', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects end before start', () => {
|
||||||
|
expect(parseRangeHeader('bytes=6-3', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects zero-length suffix', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-0', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects non-numeric start', () => {
|
||||||
|
expect(parseRangeHeader('bytes=aa-5', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects non-numeric end', () => {
|
||||||
|
expect(parseRangeHeader('bytes=1-bb', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects empty both-sides range', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects negative suffix length', () => {
|
||||||
|
expect(parseRangeHeader('bytes=-3x', 10)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects range on zero-size object', () => {
|
||||||
|
expect(parseRangeHeader('bytes=0-0', 0)).toEqual({ type: 'invalid' });
|
||||||
|
expect(parseRangeHeader('bytes=-1', 0)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects invalid total size', () => {
|
||||||
|
expect(parseRangeHeader('bytes=0-1', -1)).toEqual({ type: 'invalid' });
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
it('clamps oversized suffix ranges to the whole object', () => {
|
describe('header formatting', () => {
|
||||||
expect(parseRangeHeader('bytes=-50', 10)).toEqual({ type: 'valid', start: 0, end: 9 });
|
it('formats content-range', () => {
|
||||||
});
|
expect(contentRange(0, 9, 10)).toBe('bytes 0-9/10');
|
||||||
|
expect(contentRange(2, 5, 10)).toBe('bytes 2-5/10');
|
||||||
|
});
|
||||||
|
|
||||||
it('rejects multiple ranges', () => {
|
it('formats unsatisfied content-range', () => {
|
||||||
expect(parseRangeHeader('bytes=0-1,3-4', 10)).toEqual({ type: 'invalid' });
|
expect(unsatisfiedContentRange(10)).toBe('bytes */10');
|
||||||
});
|
expect(unsatisfiedContentRange(0)).toBe('bytes */0');
|
||||||
|
});
|
||||||
it('rejects unsatisfiable ranges', () => {
|
|
||||||
expect(parseRangeHeader('bytes=10-12', 10)).toEqual({ type: 'invalid' });
|
|
||||||
expect(parseRangeHeader('bytes=6-3', 10)).toEqual({ type: 'invalid' });
|
|
||||||
expect(parseRangeHeader('bytes=-0', 10)).toEqual({ type: 'invalid' });
|
|
||||||
});
|
|
||||||
|
|
||||||
it('formats content-range headers', () => {
|
|
||||||
expect(contentRange(2, 5, 10)).toBe('bytes 2-5/10');
|
|
||||||
expect(unsatisfiedContentRange(10)).toBe('bytes */10');
|
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
@@ -0,0 +1,107 @@
|
|||||||
|
import { describe, expect, it } from 'bun:test';
|
||||||
|
import { createHash } from 'node:crypto';
|
||||||
|
import { readFile } from 'node:fs/promises';
|
||||||
|
import { streamToTemp } from '../src/shared/utils/temp-stream';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Tests streamToTemp — the shared S3/upload streaming helper that writes a
|
||||||
|
* Request body to a temp file in O(1) memory while computing SHA-256 (+ MD5)
|
||||||
|
* and capturing the first 16 bytes as a signature.
|
||||||
|
*/
|
||||||
|
const readerFrom = (chunks: Uint8Array[]): ReadableStreamDefaultReader<Uint8Array> =>
|
||||||
|
new ReadableStream<Uint8Array>({
|
||||||
|
start(controller) {
|
||||||
|
for (const c of chunks) controller.enqueue(c);
|
||||||
|
controller.close();
|
||||||
|
},
|
||||||
|
}).getReader() as ReadableStreamDefaultReader<Uint8Array>;
|
||||||
|
|
||||||
|
const run = (chunks: (string | Uint8Array)[]) =>
|
||||||
|
streamToTemp(
|
||||||
|
readerFrom(chunks.map((c) => (typeof c === 'string' ? new TextEncoder().encode(c) : c))),
|
||||||
|
{ prefix: '/tmp/tt-' },
|
||||||
|
);
|
||||||
|
|
||||||
|
describe('streamToTemp', () => {
|
||||||
|
it('writes all bytes to a temp file and returns the size', async () => {
|
||||||
|
const r = await run(['hello ', 'world']);
|
||||||
|
try {
|
||||||
|
expect(r.sizeBytes).toBe(11);
|
||||||
|
expect(await readFile(r.tempPath, 'utf8')).toBe('hello world');
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${r.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('computes the SHA-256 hash of the full stream', async () => {
|
||||||
|
const r = await run(['foo', 'bar', 'baz']);
|
||||||
|
try {
|
||||||
|
const expected = createHash('sha256').update('foobarbaz').digest('hex');
|
||||||
|
expect(r.fileHash).toBe(expected);
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${r.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('computes MD5 (base64) only when requested', async () => {
|
||||||
|
const withMd5 = await streamToTemp(readerFrom([new TextEncoder().encode('abc')]), {
|
||||||
|
computeMd5: true,
|
||||||
|
prefix: '/tmp/tt-',
|
||||||
|
});
|
||||||
|
try {
|
||||||
|
const expected = createHash('md5').update('abc').digest('base64');
|
||||||
|
expect(withMd5.md5Hash).toBe(expected);
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${withMd5.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('captures the first 16 bytes as signature, padding shorter streams', async () => {
|
||||||
|
const long = await run(['ABCDEFGHIJKLMNOPQRST']);
|
||||||
|
try {
|
||||||
|
expect(long.signatureBuffer.toString()).toBe('ABCDEFGHIJKLMNOP');
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${long.tempPath}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
const short = await run(['ab']);
|
||||||
|
try {
|
||||||
|
expect(short.signatureBuffer.byteLength).toBe(2);
|
||||||
|
expect(short.signatureBuffer.toString()).toBe('ab');
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${short.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('handles an empty stream (zero bytes)', async () => {
|
||||||
|
const r = await run([]);
|
||||||
|
try {
|
||||||
|
expect(r.sizeBytes).toBe(0);
|
||||||
|
expect(r.fileHash).toBe(createHash('sha256').update('').digest('hex'));
|
||||||
|
expect(r.signatureBuffer.byteLength).toBe(0);
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${r.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('throws when the stream exceeds maxSizeBytes and cleans up the temp file', async () => {
|
||||||
|
await expect(
|
||||||
|
streamToTemp(readerFrom([new TextEncoder().encode('12345')]), {
|
||||||
|
maxSizeBytes: 3,
|
||||||
|
prefix: '/tmp/tt-',
|
||||||
|
}),
|
||||||
|
).rejects.toThrow(/exceeds upload limit/i);
|
||||||
|
// The temp file should NOT remain.
|
||||||
|
// (streamToTemp deletes on error; paths are unique so can't easily assert,
|
||||||
|
// but we can at least confirm no crash and no orphan via a known path.)
|
||||||
|
});
|
||||||
|
|
||||||
|
it('uses the provided temp prefix for generated paths', async () => {
|
||||||
|
const r = await run(['x']);
|
||||||
|
try {
|
||||||
|
expect(r.tempPath.startsWith('/tmp/tt-')).toBe(true);
|
||||||
|
} finally {
|
||||||
|
await Bun.$`rm -f ${r.tempPath}`;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
+72
-1
@@ -1,6 +1,11 @@
|
|||||||
import { describe, expect, it } from 'bun:test';
|
import { describe, expect, it } from 'bun:test';
|
||||||
import { unlink, writeFile } from 'node:fs/promises';
|
import { unlink, writeFile } from 'node:fs/promises';
|
||||||
import { createZip, extractZipEntry, sanitizeZipEntryName } from '../src/utils/zip';
|
import {
|
||||||
|
createZip,
|
||||||
|
extractZipEntry,
|
||||||
|
locateZipEntry,
|
||||||
|
sanitizeZipEntryName,
|
||||||
|
} from '../src/shared/utils/zip';
|
||||||
|
|
||||||
const cleanup = async (...paths: string[]) => {
|
const cleanup = async (...paths: string[]) => {
|
||||||
await Promise.all(
|
await Promise.all(
|
||||||
@@ -41,5 +46,71 @@ describe('ZIP utilities', () => {
|
|||||||
it('should sanitize unsafe entry names', () => {
|
it('should sanitize unsafe entry names', () => {
|
||||||
expect(sanitizeZipEntryName('../secret.txt')).toBe('secret.txt');
|
expect(sanitizeZipEntryName('../secret.txt')).toBe('secret.txt');
|
||||||
expect(sanitizeZipEntryName('nested/path/file.txt')).toBe('file.txt');
|
expect(sanitizeZipEntryName('nested/path/file.txt')).toBe('file.txt');
|
||||||
|
expect(sanitizeZipEntryName('a\\b\\c.txt')).toBe('a_b_c.txt');
|
||||||
|
expect(sanitizeZipEntryName('path..with..dots.txt')).toBe('path.with.dots.txt');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should sanitize empty, dot, and dotdot names to a safe fallback', () => {
|
||||||
|
expect(sanitizeZipEntryName('')).toBe('file');
|
||||||
|
expect(sanitizeZipEntryName('..')).toBe('file');
|
||||||
|
expect(sanitizeZipEntryName('.')).toBe('file');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('should make duplicate entry names unique', () => {
|
||||||
|
const used = new Set<string>();
|
||||||
|
const first = sanitizeZipEntryName('greeting.txt', used);
|
||||||
|
const second = sanitizeZipEntryName('greeting.txt', used);
|
||||||
|
const third = sanitizeZipEntryName('greeting.txt', used);
|
||||||
|
expect(first).toBe('greeting.txt');
|
||||||
|
expect(second).toBe('greeting-1.txt');
|
||||||
|
expect(third).toBe('greeting-2.txt');
|
||||||
|
});
|
||||||
|
|
||||||
|
it("returns the created zip's magic number and entry names", async () => {
|
||||||
|
const p = `/tmp/filedrop-zip-${crypto.randomUUID()}.txt`;
|
||||||
|
await writeFile(p, 'data');
|
||||||
|
const zip = await createZip([{ tempPath: p, fileName: 'data.txt' }]);
|
||||||
|
try {
|
||||||
|
const buf = Buffer.from(await Bun.file(zip.tempPath).arrayBuffer());
|
||||||
|
expect(buf.subarray(0, 2).toString()).toBe('PK');
|
||||||
|
expect(zip.entries[0].entryName).toBe('data.txt');
|
||||||
|
expect(zip.sizeBytes).toBe(buf.byteLength);
|
||||||
|
} finally {
|
||||||
|
await cleanup(p, zip.tempPath);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('extracts a stored entry and returns null for missing or compressed entries', async () => {
|
||||||
|
const p = `/tmp/filedrop-zip-${crypto.randomUUID()}.bin`;
|
||||||
|
await writeFile(p, 'payload');
|
||||||
|
const zip = await createZip([{ tempPath: p, fileName: 'x.bin' }]);
|
||||||
|
try {
|
||||||
|
const buf = Buffer.from(await Bun.file(zip.tempPath).arrayBuffer());
|
||||||
|
expect(Buffer.from((await extractZipEntry(buf, 'x.bin')) ?? Buffer.alloc(0)).toString()).toBe(
|
||||||
|
'payload',
|
||||||
|
);
|
||||||
|
expect(await extractZipEntry(buf, 'missing.bin')).toBeNull();
|
||||||
|
} finally {
|
||||||
|
await cleanup(p, zip.tempPath);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
it('locates an entry on disk without loading the whole archive', async () => {
|
||||||
|
const p = `/tmp/filedrop-zip-${crypto.randomUUID()}.bin`;
|
||||||
|
await writeFile(p, 'hello');
|
||||||
|
const zip = await createZip([{ tempPath: p, fileName: 'hi.txt' }]);
|
||||||
|
try {
|
||||||
|
const loc = await locateZipEntry(zip.tempPath, 'hi.txt');
|
||||||
|
expect(loc).not.toBeNull();
|
||||||
|
const fd = await import('node:fs/promises').then((m) => m.open(zip.tempPath, 'r'));
|
||||||
|
try {
|
||||||
|
const { bytesRead } = await fd.read(Buffer.alloc(5), 0, 5, loc!.start);
|
||||||
|
expect(bytesRead).toBe(5);
|
||||||
|
} finally {
|
||||||
|
await fd.close();
|
||||||
|
}
|
||||||
|
} finally {
|
||||||
|
await cleanup(p, zip.tempPath);
|
||||||
|
}
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
Reference in new issue
Block a user