- Export handler as default function for Vercel Serverless Function runtime.
- Attach fetch method to default export object for Bun and Cloudflare Workers compatibility.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Make generateDeviceFingerprint resilient to os.cpus() and os.hostname() returning empty or throwing in lambda sandboxes.
- Await async handleChatCompletion and handleAnthropicMessages calls in route handlers to prevent unhandled promise rejections.
- Catch runtime exceptions from completions and return clean HTTP 500 JSON error responses.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Fix type mappings in OpenAIRequest and BackendConfig to handle structured message parts (cache-control).
- Stop adaptRequest crashing when message contents are objects or arrays.
- Forward stream, top_k, and stop_sequences to adaptRequest.
- Forward anthropic-version header to backend endpoints during adaptation.
- Parse structured JSON error payloads from upstream before returning generic "Upstream rejected request" error.
- Create .github/workflows/cloudflare.yml to auto-deploy on master branch pushes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- public/test-api.html: value di baseUrl input kosong, apiBase() fallback ke window.location.origin,
plus DOMContentLoaded listener utk auto-fill
- api/test-api-content.ts: sama (legacy Next.js route)
- Router.ts handleDocs() sudah pakai pendekatan ini sebelumnya
- Replace router.ts handleDocs static placeholder with full interactive test page
- Replace worker.ts handleDocs with interactive test page
- Remove dead handleDocs() from index.ts (route already serves public/test-api.html)
- Update public/test-api.html with proper model defaults (deepseek-v4-flash-free)
- Add cache_control system prompt field to Anthropic test card
- Update handleIndex() to link to Interactive Test Page
- Remove handleDocs tests from index.test.ts
- Auto-default anthropic-version to 2023-06-01 for prompt caching
- Forward cache headers (x-cache, cf-cache-status, age, etc.) from backend
- Add features: ["prompt_caching"] to /v1/models response
- Extract and report actual usage tokens in streaming message_delta
- Track output char count for token estimation when backend omits it
- Cache-aware usage reporting: cache_creation_input_tokens, cache_read_input_tokens
- Comprehensive tests for cache_control preservation in content blocks & system
- Add anthropicPassthrough flag to BackendConfig for native Anthropic backends
- Preserve cache_control on content blocks and system prompt (keep structured)
- Extract and forward anthropic-version header from client to backend
- Report actual token usage from backend response (input_tokens, output_tokens)
- Support native Anthropic passthrough (no translation) for compatible backends
- Wire anthropic-version through all entry points: index.ts, router.ts, worker.ts
- Fix streaming timeout: use AbortController for connection-only timeout
instead of AbortSignal.timeout() that kills active SSE streams
- Fix fetchViaCurl: stream body via ReadableStream instead of buffering
entire response in memory
- Fix JWT/aichat race condition: add Promise dedup to prevent concurrent
bootstrap calls (10 requests = 1 bootstrap, not 10)
- Fix ACTIVE_READERS memory leak: auto-remove readers on stream completion
- Fix WebSocket backpressure: log warning when client buffer exceeds 1MB
- Add SSE heartbeat/keepalive: send ': keepalive' every 15s to prevent
LB/proxy timeout during AI thinking
- Fix SSELineBuffer: graceful overflow handling (warn + discard instead
of throwing error that crashes stream)
- Fix transformStream tight loop: yield to event loop after each chunk
to prevent starvation
- Fix fetchViaCurl process cleanup: use SIGKILL + proper timeout cleanup
- Add retry on 502/504: retry transient server errors before returning
to caller (both fetchWithRetry and fetchWithSessionRetry)
Ubah strategi retry di fetchWithRetry dan fetchWithSessionRetry:
- Langsung (direct) sebagai percobaan pertama sebelum proxy pool
- Proxy sebagai fallback jika direct gagal
Co-Authored-By: Claude <noreply@anthropic.com>
- Replace broken deep-seek.ai endpoint with aichat.org relay
- Add AichatSession manager with cookie/CSRF bootstrap and auto-refresh
- Add exponential backoff retry (3 attempts) for session bootstrap
- Add all 9 aichat.org model routes via shared config (zero duplication)
- Session cookies auto-refreshed from Set-Cookie on every API response
- Auto-heal: 401 triggers session re-bootstrap + retry transparently
Co-Authored-By: Claude <noreply@anthropic.com>
Bun import of .html files returns HTMLBundle (not a string) in Vercel's
serverless environment, and Bun.file() has no filesystem access there.
Copy the HTML content into api/test-api-content.ts as an escaped string
export, then import it in api/relay.ts — Bun bundler inlines it.
Co-Authored-By: Claude <noreply@anthropic.com>
Bun HTML import returns HTMLBundle object on Vercel serverless, which
renders as '[object HTMLBundle]'. Using Bun.file().text() at top-level
module scope ensures Bun bundler inlines the raw string at build time.
Co-Authored-By: Claude <noreply@anthropic.com>
Bun HTML import returns HTMLBundle object, not a string — Vercel
renders it as '[object HTMLBundle]'. Use 'with { type: "text" }'
to import as raw string so it renders properly.
Co-Authored-By: Claude <noreply@anthropic.com>
Update api/relay.ts (Vercel entry point) and src/worker.ts (Cloudflare
Workers entry point) to serve the interactive API test console at /docs
and /test, replacing the old inline docs page.
api/relay.ts uses Bun's HTML import (bundled inline at build time)
since filesystem access is unavailable in Vercel serverless runtime.
Co-Authored-By: Claude <noreply@anthropic.com>
Both /docs and /test now serve the API test console (public/test-api.html),
replacing the old inline docs page.
Co-Authored-By: Claude <noreply@anthropic.com>
- Add mimo-auth.ts: JWT bootstrap with device fingerprint, cached auto-refresh, 401/403 invalidation
- Add mimo-auto entry to MODEL_ROUTES with anti-abuse system message injection
- Inject JWT auth + x-session-affinity header before Mimo API fetch
- Retry once on 401/403 (invalidate JWT, re-bootstrap)
- Extend SSE passthrough for mimo-free provider
- Strip 'data:' prefix from Mimo non-streaming responses
Co-Authored-By: Claude <noreply@anthropic.com>
ProxyPool:
- Add cooldowns map (host:port::model -> expiry) and cooldownDuration (60s default)
- Add markRateLimited(model) — puts current proxy in cooldown for a model
- Add isProxyInCooldown, isIndexInCooldown, isCurrentInCooldown checks
- Modify rotate(model?) — skip proxies in cooldown for the given model
- Add setCooldownDuration(ms) for configuration
SessionProxyPool:
- Add markRateLimited(sessionId, model) — delegates to underlying pool
- Modify rotateNow(sessionId, model?) — skip cooldown proxies
- Modify acquire(sessionId, model?) — skip cooldown when picking least-loaded
- Modify pickLeastUsedIndex(model?) — skip cooldown proxies in scan
fetch-utils.ts:
- Add extractModel helper — extracts model from context string
- On HTTP 429: call markRateLimited(model) before markFailed
- Pass model to rotate() / rotateNow() / acquire() throughout
Co-Authored-By: Claude <noreply@anthropic.com>
fetchWithRetry:
- Dynamically calculate maxAttempts = pool.size + 1 (direct fallback)
- Try every proxy in pool via rotation, then direct as last resort
- Only classify as error when even direct produces no response
fetchWithSessionRetry:
- Default maxRetries = sessionPool.size + 1 instead of hardcoded 3
- Try pool.size proxy attempts (each on a different proxy via rotateNow),
then 1 direct attempt (no proxy) before giving up
- Only classify as error if direct also returns no response
ProxyPool.rotate():
- Skip proxies that have exceeded the failure threshold (isFailed)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add rotateNow() to SessionProxyPool — force-rotate session to a
different proxy immediately (excludes current index to ensure real
rotation). Uses round-robin scan from oldIndex+1 so all proxies
get used, not just bouncing between two.
- fetchWithSessionRetry: call rotateNow() on every failure instead of
markFailed() which only rotated after threshold. Return last HTTP
response (e.g. 429) instead of classifying as 502 when we have one.
- fetchWithRetry: rotate pool on every failure for consistency.
Co-Authored-By: Claude <noreply@anthropic.com>
- Log session ID, model, pool size, and active session count on every
POST /v1/chat/completions and POST /v1/messages
Co-Authored-By: Claude <noreply@anthropic.com>
- fetch-utils.ts: logProxy() — logs every attempt, proxy used, success/failure
with sessionId prefix for traceability
- proxy-pool.ts: logPool() — logs acquire/release/rotate/markFailed/markSuccess
with active session count and proxy host info
- Both use consistent [prefix] HH:MM:SS.mmm key=value format
Co-Authored-By: Claude <noreply@anthropic.com>
- fetchWithRetry: use proxy on every attempt (not just fallback after direct)
- fetchWithSessionRetry: call acquire() on attempt 0 so new sessions get
a least-loaded proxy assigned, enabling IP spread across concurrent users
- Update docstrings to reflect new strategy
Co-Authored-By: Claude <noreply@anthropic.com>
Add SessionProxyPool for per-session sticky proxy allocation with
load-balanced least-used selection and auto-rotation on failure.
Introduce fetchWithSessionRetry for transparent retry with proxy
rotation. Wire into AI proxy handlers (OpenAI + Anthropic) with
stream lifecycle cleanup.
Co-Authored-By: Claude <noreply@anthropic.com>
- New public/test-api.html — browser-based test UI for all proxy endpoints
- Health check, model listing, Chat Completions (stream+non-stream),
Anthropic Messages (stream+non-stream), and generic HTTP relay
- Server serves /test route from public/test-api.html
Co-Authored-By: Claude <noreply@anthropic.com>
Introduce a centralized `fetch-utils.ts` to handle retry logic with proxy fallback, SSE line buffering to prevent chunk-boundary corruption, and graceful shutdown via active reader tracking.
Key changes:
- Add `fetchWithRetry` for automatic direct-to-proxy failover.
- Implement `SSELineBuffer` to ensure reliable parsing of split SSE chunks.
- Add `createStreamBodyLimiter` to enforce payload limits on streaming requests.
- Refactor `ProxyPool` to decouple failure marking from rotation.
- Standardize CORS handling and environment variable configuration.
- Clean up documentation and remove obsolete skill files.
Standardize comment separators and arrow usage in documentation headers.
Additionally, introduce `accumulateSSEText` and `extractTextFromSSE` to
provide robust parsing for various SSE stream formats, including
Claude Code and OpenAI-compatible deltas.
Refactor the backend-to-Anthropic SSE transformation to use a more robust
state machine approach. This replaces manual string building with a
structured phase-based system (`init`, `block`, `done`) to better
manage the Anthropic streaming protocol, including `message_start` and
`content_block_delta` events.
- Extract `formatContentBlockDelta` helper for consistent event formatting
- Implement phase-based state machine in `transformAnthropicStream`
- Delegate `[DONE]` handling to the stream transformer instead of
manually emitting `message_stop`
- Improve reliability of message ID generation and event sequencing
- Replace env-based API_KEY with sk-dummy-key hardcoded
- All entry points use same key: Authorization: Bearer sk-dummy-key
- Simplifies usage for library/client consumption
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Remove the ANTHROPIC_MODEL_MAP layer — /v1/messages now uses the
same model names as /v1/chat/completions (deepseek-v4-flash-free,
gpt-5.4-mini-no-login, deepseek/deepseek-v4-flash). This way users
send the original model name and it routes to the correct backend.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Use anthReq.model instead of backendModel so the backend receives
the user-sent model name (e.g. claude-sonnet-4) rather than the
internal mapped name.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Response handler already calls markFailed() on non-2xx and network
errors (which rotates the proxy). The preamble at attempt >= 2 was
calling markFailed() again, double-rotating and skipping a proxy.
Changed preamble to use rotate() instead of markFailed().
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Previously only retried on network errors (fetch exceptions). Now also
rotates to next proxy when upstream returns non-2xx (429 rate limit,
5xx, etc). Applies to both OpenAI and Anthropic endpoints.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Add /v1/chat/completions, /v1/messages, /v1/models routes to
src/worker.ts (Cloudflare Workers) and api/relay.ts (Vercel)
- Import handleChatCompletion, handleAnthropicMessages handlers
- Using the same backend routing as the standalone server
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Create src/lib/anthropic-proxy.ts: accepts Anthropic Messages API
format (POST /v1/messages) and routes to the same backend providers
- Anthropic model names (claude-sonnet-4, claude-3-haiku, claude-opus-4)
map to backend models with full request/response translation
- Streaming (SSE) via Anthropic protocol: message_start,
content_block_delta, message_stop events
- Add /v1/messages route to server with CORS and proxy pool fallback
- Reuses MODEL_ROUTES from ai-proxy.ts for consistent backend routing
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Create src/lib/ai-proxy.ts: routes OpenAI chat completions requests
to backend providers (opencode.ai, surfsense.com, deep-seek.ai)
based on model name with request/response translation
- Add POST /v1/chat/completions route to server with streaming (SSE)
support, CORS, and proxy pool fallback on failure
- Add GET /v1/models route returning available model list
- Direct-first strategy: try backend directly, fall back to proxy pool
on network failure (consistent with relay behavior)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
