feat(ai): make standalone image/vision analysis timeout explicit (1 min)
The standalone image analysis path (analyzeSingleMediaImage → llmVision → llmChat) previously had no request-level timeout of its own — it silently inherited the shared OpenAI client default (60s), and AI_LLM_MEDIA_ANALYSIS_ TIMEOUT_MS only governed the text+media *batch*, not a single vision call. - Add AI_LLM_VISION_ANALYSIS_TIMEOUT_MS (default 60000) to config. - llmChat now accepts an optional per-request `timeout` in LlmCallOpts, forwarded to the OpenAI request options (falls back to the 60s client default when omitted). - llmVision passes config.AI_LLM_VISION_ANALYSIS_TIMEOUT_MS, so a single image/sticker/emoji analysis gets a guaranteed 1-minute budget and is independently tunable from the text path. Verified: tsc + biome green, 129 gateway tests pass. Co-Authored-By: Claude Opus 5 (Nous Research)
This commit is contained in:
co-authored by
Claude Opus 5 (Nous Research)
parent
a4abe3abea
commit
d8552a9fb8
@@ -158,6 +158,12 @@ export interface LlmCallOpts {
|
|||||||
stream?: boolean;
|
stream?: boolean;
|
||||||
/** Optional AbortSignal to cancel the API request */
|
/** Optional AbortSignal to cancel the API request */
|
||||||
signal?: AbortSignal;
|
signal?: AbortSignal;
|
||||||
|
/**
|
||||||
|
* Per-request timeout in ms. Falls back to the client-level default
|
||||||
|
* (60s) when omitted. Vision/image analysis passes a longer budget here
|
||||||
|
* so a single large-image call isn't killed early by the shared default.
|
||||||
|
*/
|
||||||
|
timeout?: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -242,6 +248,7 @@ export async function llmChat(
|
|||||||
) => {
|
) => {
|
||||||
const response = await client.chat.completions.create(currentParams, {
|
const response = await client.chat.completions.create(currentParams, {
|
||||||
signal,
|
signal,
|
||||||
|
...(opts.timeout ? { timeout: opts.timeout } : {}),
|
||||||
});
|
});
|
||||||
if (currentParams.stream) {
|
if (currentParams.stream) {
|
||||||
let content = "";
|
let content = "";
|
||||||
@@ -352,6 +359,7 @@ export async function llmVision(
|
|||||||
top_p: 0.9,
|
top_p: 0.9,
|
||||||
retries: 0,
|
retries: 0,
|
||||||
stream: true, // router always streams SSE; non-stream waits for full body and times out
|
stream: true, // router always streams SSE; non-stream waits for full body and times out
|
||||||
|
timeout: config.AI_LLM_VISION_ANALYSIS_TIMEOUT_MS ?? 60_000,
|
||||||
});
|
});
|
||||||
|
|
||||||
if (!completion) return null;
|
if (!completion) return null;
|
||||||
|
|||||||
@@ -189,6 +189,15 @@ export const configSchema = z
|
|||||||
.int()
|
.int()
|
||||||
.positive()
|
.positive()
|
||||||
.default(60000),
|
.default(60000),
|
||||||
|
// Standalone image/sticker/emoji vision analysis (analyzeSingleMediaImage
|
||||||
|
// → llmVision → llmChat). Decoupled from the media *batch* timeout above so
|
||||||
|
// a single vision call can be tuned independently. 1 minute by default —
|
||||||
|
// vision models (especially behind a router) need headroom for large images.
|
||||||
|
AI_LLM_VISION_ANALYSIS_TIMEOUT_MS: z.coerce
|
||||||
|
.number()
|
||||||
|
.int()
|
||||||
|
.positive()
|
||||||
|
.default(60000),
|
||||||
// Text-only moderation batches are cheaper than media (no downloads /
|
// Text-only moderation batches are cheaper than media (no downloads /
|
||||||
// vision pre-pass), so they get their own (shorter) timeout instead of
|
// vision pre-pass), so they get their own (shorter) timeout instead of
|
||||||
// being tied to the media budget.
|
// being tied to the media budget.
|
||||||
|
|||||||
Reference in New Issue
Block a user