feat(ai-moderation): implement auto-fallback for streaming and refine prompt rules

- Add automatic fallback to streaming mode if the provider rejects non-streaming requests with a 400 error
- Refactor `llmChat` to use an internal execution function to support retry logic with modified parameters
- Update moderation prompt to explicitly allow Japanese pop culture terms (e.g., "moe", "waifu", "wibu") to prevent false positive sexual deviation flags
This commit is contained in:
MythEclipse
2026-06-05 19:44:06 +07:00
parent bdf80b081c
commit 49ada183b2
2 changed files with 22 additions and 13 deletions
@@ -78,7 +78,7 @@ export async function llmChat(
top_p,
jsonResponse,
retries = DEFAULT_RETRIES,
stream = false,
stream,
} = opts;
const params: any = {
@@ -100,15 +100,13 @@ export async function llmChat(
return retryWithBackoff(
async () => {
return withLlmConcurrency(async () => {
try {
const response = await client.chat.completions.create(params);
if (stream) {
const execute = async (currentParams: any) => {
const response = await client.chat.completions.create(currentParams);
if (currentParams.stream) {
let content = "";
let finishReason = "stop";
for await (const chunk of response as any) {
const choice = chunk?.choices?.[0];
// Dynamic parsing to support multiple providers (OpenAI, Ollama, Groq, Anthropic via proxy, etc.)
const textChunk =
choice?.delta?.content ||
choice?.message?.content ||
@@ -117,13 +115,9 @@ export async function llmChat(
chunk?.response ||
chunk?.content ||
"";
content += textChunk;
const fr = choice?.finish_reason || chunk?.finish_reason;
if (fr) {
finishReason = fr;
}
if (fr) finishReason = fr;
}
return {
id: 'stream-aggregated',
@@ -136,17 +130,31 @@ export async function llmChat(
},
],
created: Math.floor(Date.now() / 1000),
model: model,
model: currentParams.model,
object: 'chat.completion',
} as OpenAI.Chat.Completions.ChatCompletion;
}
return response as OpenAI.Chat.Completions.ChatCompletion;
};
try {
return await execute(params);
} catch (error: any) {
const rawResponse = error.error || error.body || error.response?.data || "N/A";
const errorStr = (JSON.stringify(rawResponse) + String(error.message)).toLowerCase();
// Auto-fallback: If provider strictly demands streaming (400 Bad Request on stream params)
if (error.status === 400 && errorStr.includes("stream") && !params.stream) {
log.warn({ model }, "Provider rejected non-streaming request. Fallback to stream: true initiated.");
params.stream = true;
return await execute(params);
}
log.error(
{
error: error.message,
status: error.status,
rawResponse: error.error || error.body || error.response?.data || "N/A",
rawResponse,
model
},
"LLM API request failed"
@@ -20,6 +20,7 @@ Bahasa utama komunitas ini adalah BAHASA INDONESIA. Bahasa Inggris adalah bahasa
## Aturan Umum
- Bahasa gaul/slang Indonesia: "anjay", "wkwk", "gws", "gaskeun", "santuy", "njir", "baka", "woy", "woi", "hadeh", dll adalah AMAN.
- Istilah kultur pop/anime Jepang: "moe", "waifu", "husbando", "tsundere", "wibu", "otaku" adalah ekspresi normal/AMAN dan BUKAN "sexual_deviation". JANGAN flag kata-kata ini kecuali diiringi deskripsi/ajakan seksual eksplisit.
- Singkatan umum: "gw", "lo", "emg", "kyk", "tdk", "krn", "jgn", dll adalah AMAN.
- Makian/kata kasar umum (emosi marah seperti "anjing", "asu", "bangsat", "ngehe") BUKAN pelanggaran SARA. Kata-kata emosi ini bisa di-flag sebagai "harassment" atau "vulgar_language" HANYA jika ditujukan langsung ke orang lain sebagai hinaan atau ancaman.
- **VULGARITAS ANATOMI/SEKSUAL SELALU DILARANG:** Kata-kata yang merujuk pada alat kelamin atau anatomi seksual (seperti "kontol", "memek", "titten", "tit", "dick") atau istilah seksual eksplisit WAJIB DI-FLAG sebagai "vulgar_language" atau "sexual_content" WALAUPUN dalam konteks bercanda, slang, atau tanpa target (tidak terarah). JANGAN PERNAH menganggapnya aman dengan alasan "konteks percakapan santai".