Files
GMW/services
asepharyana 4cf5b87f2b perf(ai-moderation): pack more messages per LLM request (fewer API calls when busy)
User insight: rather than many small per-batch API requests, pack many
messages into ONE request so a burst is analyzed with far fewer calls.

- AI_LLM_TEXT_BATCH_SIZE 20 -> 60 (one request now carries ~3x more messages).
- AI_ANALYSIS_MAX_TARGET_TOKENS 4000 -> 14000 (the scheduler's token-budget
  gate was trimming pending messages to ~20 before they reached the sub-batch
  splitter; raising it lets ~60 messages through to a single LLM call).
- AI_LLM_TEXT_ANALYSIS_TIMEOUT_MS 30000 -> 45000 (one larger call needs more
  headroom; gemini-flash-lite has a 1M-token context so 14k+8k is trivial).

Net effect when ramai: a 60-message burst = 1-2 API calls instead of 3+,
less semaphore contention, faster throughput.
2026-08-16 19:00:29 +07:00
..