MythEclipse/GMW
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix(ai-moderation): remove user history from prompt to eliminate confirmation bias loop

Browse Source 
user_history (riwayat flag sebelumnya) dan clean_streak/total_infractions
dikirim ke LLM setiap kali menganalisis pesan — ini bikin self-fulfilling
prophecy: user yg pernah kena false positive jadi makin gampang dituduh
lagi, dan link Instagram pun dianggap sexual_deviation cuma karena
riwayat user.

Changes:
- Hapus getUserRecentInfractions dari text batch path
- Hapus getUserRecentInfractions dari media analysis path
- Hapus import getUserRecentInfractions yg gak dipakai
- Ubah instruksi prompt dari 'jadilah lebih tegas jika riwayat jelek'
  jadi 'setiap pesan dinilai berdasarkan isinya sendiri'

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
MythEclipse
2026-06-06 13:55:24 +07:00
co-authored by Claude Opus 4.8
parent edb55fb5bf
commit da885339f9
2 changed files with 5 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/discord-gateway/src/modules/ai-moderation/llmModerationClient.ts
+4 -19
View File
@@ -14,10 +14,7 @@ import type {
import { llmChat, llmVision } from "./llmClient.js";
import { buildSystemPrompt as buildSystemPromptModular } from "./moderationPrompt.js";
import {
initializeUserReputation,
getUserRecentInfractions,
} from "./userReputationStore.js";
import { initializeUserReputation } from "./userReputationStore.js";
import { getChannelCulture } from "./channelCultureStore.js";
import { logModerationAnalysis, logModerationError } from "./responseLogger.js";
import {
@@ -1108,19 +1105,12 @@ async function runTextOnlyBatch(
const batch = subBatches[i];
const targetIds = batch.map((t) => t.id);
// Fetch user context for this batch
// Abstract user reputation (no history — prevents confirmation bias)
const userContexts = new Map<string, string>();
for (const msg of batch) {
if (!userContexts.has(msg.user_id)) {
const rep = await initializeUserReputation(msg.user_id, msg.guild_id);
const history = await getUserRecentInfractions(msg.user_id);
let historyStr = "";
if (history.length > 0) {
historyStr = `\n <user_history>\n${history.map(h => ` - Flagged for ${h.flags} (Severity: ${h.severity}) pada pesan: "${h.content}"`).join("\n")}\n </user_history>`;
}
const contextStr = `<user_reputation trust_score="${rep.trust_score}" clean_streak="${rep.clean_message_streak}" total_infractions="${rep.total_infractions}" />${historyStr}`;
const contextStr = `<user_reputation trust_score="${rep.trust_score}" />`;
userContexts.set(msg.user_id, contextStr);
}
}
@@ -1670,12 +1660,7 @@ async function _runSingleMediaAnalysis(
const channelCulture = channelCultureObj ? channelCultureObj.culture_summary : undefined;
const rep = await initializeUserReputation(target.user_id, target.guild_id);
const history = await getUserRecentInfractions(target.user_id);
let historyStr = "";
if (history.length > 0) {
historyStr = `\n <user_history>\n${history.map(h => ` - Flagged for ${h.flags} (Severity: ${h.severity}) pada pesan: "${h.content}"`).join("\n")}\n </user_history>`;
}
const userCtx = `<user_reputation trust_score="${rep.trust_score}" clean_streak="${rep.clean_message_streak}" total_infractions="${rep.total_infractions}" />${historyStr}`;
const userCtx = `<user_reputation trust_score="${rep.trust_score}" />`;
// XML delimiters wrap the message content (R1)
const messageBlock = `<message id="${target.id}" user="${target.username}">\n ${userCtx}\n <content>${content}</content>${mediaContext ? ` ${mediaContext}` : ""}${webContext}${mediaAnalysisContext}\n</message>`;
services/discord-gateway/src/modules/ai-moderation/moderationPrompt.ts
+1 -1
View File
@@ -455,7 +455,7 @@ export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
parts.push(`## Kultur Channel (Pembelajaran AI)\n${channelCulture}`);
}
parts.push(`## Konteks Pengguna (Ingatan & Kebijaksanaan)\nSetiap pesan mungkin memiliki tag <user_reputation> dan <user_history>. *Gunakan Kebijaksanaan: Jika trust_score tinggi, beri benefit of the doubt pada ambiguitas. Jika trust_score rendah dan memiliki riwayat pelanggaran serupa, jadilah lebih tegas.*`);
parts.push(`## Konteks Pengguna\nSetiap pesan mungkin memiliki tag <user_reputation>. Tag ini hanya indikator **referensi**, bukan bukti pelanggaran. Nilai trust_score yang rendah bukan alasan untuk memflag pesan yang bersih. Nilai trust_score yang tinggi bukan alasan untuk mengabaikan pelanggaran nyata. **Setiap pesan harus dinilai berdasarkan isinya sendiri.**`);
parts.push(OUTPUT_INSTRUCTIONS);