fix(automod): flow real LLM analysis + descriptive fallback
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m2s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m25s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m40s

Root cause: ai-analysis-worker read llmResult.explanation and
llmResult.toxicityScore — fields the LLM pipeline never produces
(canonical AnalysisResult uses analysis/score). Every message fell back
to the bare template "Tidak ada indikasi pelanggaran." and the stored
score was always 0.

- Map analysis/score correctly; fallback now quotes the message content
- Prompt: ban generic analysis phrasing, require reply context
- LLM context: include replied-to message content (metadata.reference)
  so the model can explain what the user is replying to
- Frontend: show thread/channel names from metadata instead of raw IDs
  (message card, detail views, search overlay); detail panel now
  displays the ai_analysis text
- Auto-delete log/DM include the descriptive analysis as the reason
This commit is contained in:
Developer
2026-07-31 19:11:13 +07:00
parent 0bd4369ae9
commit 60084b3cc3
11 changed files with 142 additions and 34 deletions
@@ -20,8 +20,14 @@ export async function sendDeletionNotification(
try {
const targetUser = await client.users.fetch(message.user_id);
if (targetUser) {
// Prefer the descriptive LLM analysis so the user understands WHY;
// fall back to category/flag labels when it is unavailable.
const analysis = (message.ai_analysis ?? "").trim();
const reason: string =
message.ai_categories ?? message.ai_moderation_flags ?? "(unknown)";
(analysis.length > 240 ? `${analysis.slice(0, 240)}` : analysis) ||
message.ai_categories ??
message.ai_moderation_flags ??
"(unknown)";
await targetUser.send(
`Pesan Anda di **${guildName}** telah dihapus oleh sistem moderasi otomatis.\n` +
`Alasan: ${reason}\n` +