feat(headroom): make the compression request timeout configurable

The 3000 ms timeout on /v1/compress was fixed, so busy or slow machines
timed out often and sent the LLM an inconsistently compressed body,
hurting prompt caching. Add a headroomTimeoutMs setting, thread it from
the chat handler down to compressWithHeadroom, expose it in the Token
Saver dashboard, and normalize invalid values back to the 3000 ms default.
This commit is contained in:
snower committed 2026-08-28 16:34:33 +07:00
1 parent 28d005772a
commit 993c6eb469
6 files changed
+128 -2

No files matched your search

+1
View File
@@ -274,6 +274,7 @@ async function handleSingleModelChat(body, modelStr, clientRawRequest = null, re
headroomEnabled: !!chatSettings.headroomEnabled,
headroomUrl: chatSettings.headroomUrl || DEFAULT_HEADROOM_URL,
headroomCompressUserMessages: !!chatSettings.headroomCompressUserMessages,
headroomTimeoutMs: chatSettings.headroomTimeoutMs,
cavemanEnabled: !!chatSettings.cavemanEnabled,
cavemanLevel: chatSettings.cavemanLevel || "full",
ponytailEnabled: !!chatSettings.ponytailEnabled,