From dbdc5ed1ee8138d08c4fb094be95633a39db25bf Mon Sep 17 00:00:00 2001
From: asepharyana <superaseph@gmail.com>
Date: Tue, 7 Jul 2026 03:56:40 +0700
Subject: [PATCH] docs: design remaining S3 compatibility fixes

---
 ...7-07-s3-compatibility-completion-design.md | 175 ++++++++++++++++++
 1 file changed, 175 insertions(+)
 create mode 100644 docs/superpowers/specs/2026-07-07-s3-compatibility-completion-design.md

diff --git a/docs/superpowers/specs/2026-07-07-s3-compatibility-completion-design.md b/docs/superpowers/specs/2026-07-07-s3-compatibility-completion-design.md
new file mode 100644
index 0000000..1562bb5
--- /dev/null
+++ b/docs/superpowers/specs/2026-07-07-s3-compatibility-completion-design.md
@@ -0,0 +1,175 @@
+# S3 Compatibility Completion Design
+
+Date: 2026-07-07
+
+## Goal
+
+Close the remaining known S3 compatibility gaps in TeleUploader while preserving strict SigV4 security. The target is real-client compatibility for common S3 flows against the production endpoint, especially AWS SDK v3, without accepting malformed or unverifiable signatures.
+
+## Scope
+
+Implement and verify:
+
+1. Presigned `GET` URLs return object content instead of `403 SignatureDoesNotMatch`.
+2. `GetObject` supports `Range: bytes=...` for single-part objects.
+3. Multipart `GetObject` returns the complete logical object, not just the first Telegram part.
+4. Multipart `GetObject` supports byte ranges across part boundaries.
+5. AWS SDK multipart upload is investigated and fixed under strict SigV4 if the mismatch is in TeleUploader canonicalization/routing.
+6. The remaining CSS lint warning in `src/home.html` is removed.
+
+Out of scope:
+
+- Weakening SigV4 verification to accept invalid signatures.
+- Full AWS S3 feature parity for ACLs, bucket policy, object tags, virtual-hosted-style routing, or checksum-specific APIs.
+- Replacing Telegram storage with a different backend.
+
+## Design Choice
+
+Use a **strict compatibility, shared object streaming layer**.
+
+Rather than adding one-off fixes inside `handleGetObject`, introduce small helpers with clear responsibilities:
+
+- request authentication remains strict and spec-based;
+- object lookup returns object metadata and an ordered list of physical Telegram parts;
+- range parsing converts a `Range` header into byte offsets;
+- response streaming emits either a full body (`200`) or a partial body (`206`) with correct S3/HTTP headers.
+
+This keeps S3 auth, object resolution, byte planning, and streaming separately testable.
+
+## Presigned URL Verification
+
+Current behavior builds a URL with `http://localhost/...`, which can produce a host mismatch for presigned signatures generated against the public production host.
+
+New behavior:
+
+- pass the original request URL and headers into presigned verification;
+- compute canonical URI from the actual URL path;
+- compute canonical query string by excluding `X-Amz-Signature`, sorting encoded key/value pairs, and preserving AWS SigV4 encoding rules;
+- validate credential scope parts: access key, date, region, service `s3`, termination `aws4_request`;
+- use the signed `host` header value when present, otherwise the request URL host;
+- parse `X-Amz-Date` as UTC and enforce `X-Amz-Expires`.
+
+Production E2E must change from accepting `[200, 302, 403]` to requiring `200` and body content for presigned `GET` when `PROXY_S3_GET` is enabled.
+
+## Object Body Model
+
+Represent the physical body as ordered parts:
+
+```ts
+interface ObjectPartSource {
+  telegramFileId: string;
+  sizeBytes: number;
+  partNumber: number;
+}
+```
+
+Single-part objects produce one source from the `files` row.
+Multipart objects produce sources from `multipart_parts` ordered by `part_number`.
+
+The logical object size is:
+
+- `file.sizeBytes` for single-part;
+- sum of part sizes for multipart, cross-checked against `file.sizeBytes` when available.
+
+## Range Handling
+
+Support single-range requests only:
+
+- `bytes=0-4`
+- `bytes=5-`
+- `bytes=-500`
+
+Invalid or unsatisfiable ranges return `416` with `Content-Range: bytes */<size>`.
+Multiple ranges such as `bytes=0-1,3-4` are not supported and return `416`.
+
+For a valid range:
+
+- return status `206`;
+- set `Content-Range: bytes <start>-<end>/<size>`;
+- set `Content-Length` to `end - start + 1`;
+- keep `Accept-Ranges: bytes`.
+
+## Telegram Streaming
+
+For each needed part:
+
+1. Call `getFileInfo(telegramFileId)`.
+2. Build the Telegram CDN URL.
+3. Fetch only the needed range when possible using a `Range` request to Telegram.
+4. If Telegram returns `206`, stream that response body.
+5. If Telegram returns `200` for a partial request, slice the response in TeleUploader for correctness.
+6. If Telegram fetch fails, return S3 XML `InternalError` with HTTP `502`.
+
+Multipart full-object responses concatenate part streams in order with a `ReadableStream`.
+Multipart range responses calculate part overlap using cumulative byte offsets and only stream overlapped segments.
+
+## AWS SDK Multipart Upload
+
+Add a real AWS SDK multipart test path using:
+
+- `CreateMultipartUploadCommand`
+- `UploadPartCommand`
+- `CompleteMultipartUploadCommand`
+- `GetObjectCommand`
+- cleanup via `AbortMultipartUploadCommand` on failure
+
+Keep SigV4 strict. If the SDK multipart signature still fails:
+
+1. capture server-side canonical request components in a temporary diagnostic path or test-only log;
+2. compare against the SDK request inputs;
+3. fix TeleUploader canonicalization if wrong;
+4. if the mismatch is caused by upstream proxy/header mutation outside the app, leave strict behavior intact and document the blocker with evidence.
+
+No compatibility fallback is allowed in this pass.
+
+## CSS Lint Warning
+
+Remove `noDescendingSpecificity` by reordering the `.modal input` rule before the more specific `.topbar .search input` rule, or by making selector ordering explicitly non-conflicting. No visual redesign is required.
+
+## Tests and Verification
+
+Unit tests:
+
+- presigned URL canonicalization and expiration edge cases;
+- range parser valid/invalid forms;
+- multipart range planning across part boundaries.
+
+Production E2E:
+
+- presigned `GET` returns `200` and expected body;
+- single-part range returns `206` and expected substring;
+- multipart upload via manual signer returns complete body;
+- multipart range returns expected cross-part substring;
+- invalid range returns `416`.
+
+AWS SDK E2E:
+
+- existing SDK suite remains passing;
+- `GetObjectCommand({ Range: 'bytes=...' })` verifies partial body;
+- real SDK multipart is enabled if strict SigV4 succeeds after canonicalization fixes.
+
+Deploy verification:
+
+- `bun run lint` has no errors and no warnings;
+- targeted unit tests pass;
+- production E2E passes;
+- AWS SDK E2E passes;
+- `./deploy.sh` completes and container is healthy;
+- post-deploy production E2E and SDK E2E pass.
+
+## Error Handling
+
+- Authentication failures return S3 XML errors with `403`.
+- Missing bucket/key behavior is unchanged.
+- Telegram fetch failures return S3 XML `InternalError` with `502`.
+- Invalid ranges return `InvalidRange` XML with `416`.
+- Multipart objects with no parts return `InternalError`.
+
+## Rollout
+
+The existing `PROXY_S3_GET` flag remains:
+
+- default `true`: S3-compatible streaming responses;
+- `false`: legacy redirect path for single-part objects only.
+
+Multipart complete-body support requires proxy mode. If proxy mode is disabled and a multipart object is requested, the response may fall back to the first-part redirect only as legacy behavior; production compatibility should keep proxy mode enabled.