refactor(chatbot): drop static server-stats context, go fully tool-based

The chatbot already had an agentic tool loop (get_server_stats,
get_top_channels, get_recent_activity, get_top_flagged), but processMessage
still baked a serverInsights snapshot into the system prompt and told the
model to "answer from that data". That defeats the tools: the model answered
from a stale snapshot instead of living numbers, and the guild/channel scope
the frontend sends was never forwarded to the tools.

Changes (services/backend/src/modules/chatbot):
- Remove getServerInsights() + ServerInsights (dead after this change).
- buildSystemPrompt(): drop the hardcoded stats block; instruct the model it
  has NO memorized server numbers and MUST call a tool for any server-data
  question, answering only from tool results.
- processMessage(): stop fetching insights; pass the request guildId/channelId
  scope through to callLLM.
- callLLM(): accept scope; auto-fill empty guildId/channelId on tool calls from
  the request scope so the model never has to guess IDs and tools always query
  the right server.

Behavior: answers now come from live DB data via tools, scoped to the server
the user is chatting in. tsc + biome + 36 backend tests green.

Co-Authored-By: Claude Opus 5 (Nous Research)
This commit is contained in:
asepharyana
2026-08-16 09:15:39 +07:00
co-authored by Claude Opus 5 (Nous Research)
parent a3e5a8c1b9
commit 30828a5534
2 changed files with 39 additions and 79 deletions
@@ -31,13 +31,6 @@ export interface ChatbotHistoryRow {
created_at: string; created_at: string;
} }
export interface ServerInsights {
total_messages: number;
active_users: number;
flagged: number;
warned: number;
}
export class ChatbotRepository { export class ChatbotRepository {
async saveConversation(input: SaveConversationInput): Promise<void> { async saveConversation(input: SaveConversationInput): Promise<void> {
const db = getDatabase(); const db = getDatabase();
@@ -83,56 +76,6 @@ export class ChatbotRepository {
"Chat history cleared", "Chat history cleared",
); );
} }
async getServerInsights(
guildId?: string,
channelId?: string,
): Promise<ServerInsights> {
try {
const db = getDatabase();
const conditions: SQL[] = [];
if (guildId) {
conditions.push(eq(pgMessagesTable.guild_id, guildId));
}
if (channelId) {
conditions.push(eq(pgMessagesTable.channel_id, channelId));
}
const where = conditions.length > 0 ? and(...conditions) : undefined;
const [result] = await db
.select({
total_messages: sql<number>`COUNT(*)::int`,
active_users: sql<number>`COUNT(DISTINCT ${pgMessagesTable.user_id})::int`,
flagged: sql<number>`COUNT(*) FILTER (WHERE ${pgMessagesTable.ai_status} = 'flagged')::int`,
warned: sql<number>`COUNT(*) FILTER (WHERE ${pgMessagesTable.ai_status} = 'warn')::int`,
})
.from(pgMessagesTable)
.where(where);
const insights = result ?? {
total_messages: 0,
active_users: 0,
flagged: 0,
warned: 0,
};
logger.debug({ guildId, channelId, insights }, "Server insights fetched");
return insights;
} catch (error) {
logger.warn(
{ error, guildId, channelId },
"Failed to load server insights",
);
return {
total_messages: 0,
active_users: 0,
flagged: 0,
warned: 0,
};
}
}
} }
export const chatbotRepository = new ChatbotRepository(); export const chatbotRepository = new ChatbotRepository();
@@ -17,22 +17,26 @@ class ChatbotService {
userId: string, userId: string,
): Promise<string> { ): Promise<string> {
logger.info( logger.info(
{ userId, messageLength: message.length }, { userId, messageLength: message.length, context },
"processMessage called", "processMessage called",
); );
const recentContext = await this.getRecentConversationContext(userId); const recentContext = await this.getRecentConversationContext(userId);
const serverInsights = await chatbotRepository.getServerInsights( // Scope the agent to the server/channel the user is chatting in. We no
context?.guildId, // longer bake server stats into the prompt — the model must pull current
context?.channelId, // data via tools (see buildSystemPrompt), so it always answers from live
); // numbers instead of a stale snapshot.
const scope = {
guildId: context?.guildId,
channelId: context?.channelId,
};
// Build LLM messages const systemPrompt = this.buildSystemPrompt(scope);
const systemPrompt = this.buildSystemPrompt(serverInsights);
const conversationHistory = this.buildHistoryMessages(recentContext); const conversationHistory = this.buildHistoryMessages(recentContext);
const llmResponse = await this.callLLM( const llmResponse = await this.callLLM(
systemPrompt, systemPrompt,
conversationHistory, conversationHistory,
message, message,
scope,
); );
return llmResponse; return llmResponse;
@@ -66,27 +70,29 @@ class ChatbotService {
]); ]);
} }
private buildSystemPrompt(insights: { private buildSystemPrompt(scope: {
total_messages: number; guildId?: string;
active_users: number; channelId?: string;
flagged: number;
warned: number;
}): string { }): string {
return `Kamu lagi ngobrol sama chatbot Discord Watcher — temen ngobrol yang tau keadaan server. const scopeLine = scope.guildId
? `- Scope: kamu menjawab soal server/guild id="${scope.guildId}"${scope.channelId ? `, channel id="${scope.channelId}"` : ""}.`
: "- Scope: tidak ada guild spesifik — jawab umum soal server ini.";
return `Kamu adalah chatbot Discord Watcher — temen ngobrol yang tau keadaan server, dan kamu PUNYA AKSES ke data server lewat tools.
Data server saat ini: ${scopeLine}
- Pesan: ${insights.total_messages}
- User aktif: ${insights.active_users} ATURAN PENTING — JANGAN PAKAI KONTEKS STATIS:
- Flagged: ${insights.flagged} - Kamu TIDAK punya hafalan soal angka server (jumlah pesan, user aktif, flagged, dll). JANGAN tebak atau karang angka.
- Warning: ${insights.warned} - Untuk SEMUA pertanyaan soal data server (jumlah pesan, user aktif, channel ramai, aktivitas terbaru, pesan di-flag), WAJIB panggil tool yang sesuai (get_server_stats, get_top_channels, get_recent_activity, get_top_flagged). Jawab HANYA dari hasil tool.
- Tool otomatis di-scope ke guild/channel di atas — kalau argumen guildId/channelId kosong, biarkan kosong (sudah otomatis ter-isi). Jangan isi ID yang kamu tebak.
- Kalau tool balas error atau kosong, bilang aja data lagi ga ketemu, jangan karang.
Gaya ngobrol: Gaya ngobrol:
- Santai, hangat, kayak ngobrol sama temen - Santai, hangat, kayak ngobrol sama temen
- Pake Bahasa Indonesia sehari-hari, ga perlu kaku - Pake Bahasa Indonesia sehari-hari, ga perlu kaku
- Sesekali pake emoji wajar aja, ga berlebihan - Sesekali pake emoji wajar aja, ga berlebihan
- Kalo ditanya sesuatu yang kamu tau dari data server, jawab pake data itu - Kalo ditanya di luar data server dan kamu ga tau, bilang aja terus tanya balik biar ngobrolnya jalan
- Kalo ga tau atau ga nyambung, bilang aja terus tanya balik biar ngobrolnya jalan - Jangan sebut "rule", "instruksi", "prompt", "tool", atau apapun soal cara kamu berpikir
- Jangan sebut "rule", "instruksi", "prompt" atau apapun soal cara kamu berpikir
- Biasa aja, ga usaha lucu-lucu amat — natural`; - Biasa aja, ga usaha lucu-lucu amat — natural`;
} }
@@ -106,6 +112,7 @@ Gaya ngobrol:
systemPrompt: string, systemPrompt: string,
history: Array<{ role: "user" | "assistant"; content: string }>, history: Array<{ role: "user" | "assistant"; content: string }>,
userMessage: string, userMessage: string,
scope: { guildId?: string; channelId?: string },
): Promise<string> { ): Promise<string> {
const apiKey = config.AI_LLM_API_KEY; const apiKey = config.AI_LLM_API_KEY;
const baseUrl = config.AI_LLM_BASE_URL; const baseUrl = config.AI_LLM_BASE_URL;
@@ -190,9 +197,19 @@ Gaya ngobrol:
}, },
], ],
}); });
// Auto-scope: if the model omitted guildId/channelId, fill them
// from the request scope so tools query the right server without
// the model having to guess IDs.
const scopedArgs = { ...tc.args };
if (scope.guildId && scopedArgs.guildId == null) {
scopedArgs.guildId = scope.guildId;
}
if (scope.channelId && scopedArgs.channelId == null) {
scopedArgs.channelId = scope.channelId;
}
let result = ""; let result = "";
try { try {
result = await executeTool(tc.name, tc.args); result = await executeTool(tc.name, scopedArgs);
} catch (e) { } catch (e) {
result = `Tool error: ${(e as Error).message}`; result = `Tool error: ${(e as Error).message}`;
} }