- Remove shouldSearchContent() trigger gate — search runs on all messages
- extractSearchQueries() now extracts from ANY message, not just trigger-matched
- Redis cache (24h TTL) prevents redundant searches for same query
- initSearxngCache() lazy-connects via config.REDIS_URL
- Cache miss→API, hit→skip — fire-and-forget writes
- Both text batch + media path simplified
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
- userProfileLearner.ts: filter query to only clean messages (eq ai_status='clean')
to prevent profile contamination from flagged content. Also select channel_id
to group messages by channel in prompt, enabling channel-aware personality
summarization (user may behave differently across channels).
- llmModerationClient.ts (runSimpleTextFallback): inject user profile into
both the classify prompt and the reason prompt, so even the last-resort
fallback path has personality/memory context instead of being blind.
- Expand IMPHNEN domain rule to cover wildcard (*.imphnen.*)
- Trim redundant SARA examples from TEXT_ONLY_MODE (save ~950 tokens)
- Add debugging logs for channel culture injection into prompt
- Sync flag validation set with missing flags: potential_evasion, unclear_context
Co-Authored-By: Claude <noreply@anthropic.com>
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.
- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table
Co-Authored-By: Claude <noreply@anthropic.com>
- hasMediaContent now also checks evidence.attachments from metadata
(not just DB attachment records), catching the race where attachment
DB rows aren't inserted yet when analysis runs.
- Cache-hit guard: treat cached entries as miss when the message has
media evidence in metadata, so stale 24h-freezes are avoided.
- Cache-write guard: skip storing text-only analysis results for
messages whose metadata shows attachments/stickers/embeds. This
prevents a text-only 'clean' result (from failed vision) being
frozen for 24h, blocking future re-analysis with full media context.
Co-Authored-By: Claude <noreply@anthropic.com>
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
user_history (riwayat flag sebelumnya) dan clean_streak/total_infractions
dikirim ke LLM setiap kali menganalisis pesan — ini bikin self-fulfilling
prophecy: user yg pernah kena false positive jadi makin gampang dituduh
lagi, dan link Instagram pun dianggap sexual_deviation cuma karena
riwayat user.
Changes:
- Hapus getUserRecentInfractions dari text batch path
- Hapus getUserRecentInfractions dari media analysis path
- Hapus import getUserRecentInfractions yg gak dipakai
- Ubah instruksi prompt dari 'jadilah lebih tegas jika riwayat jelek'
jadi 'setiap pesan dinilai berdasarkan isinya sendiri'
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The rule-based badword detector was injecting [normalized_text] and
[normalization_notes] tags into the LLM prompt that caused false
positive hallucinations - the LLM started associating innocent words
('sapik', 'furina') with furry/sexual_deviation due to misleading
context injected by the normalizer.
Removed:
- indonesianTextNormalizer.ts (full file deletion)
- formatModerationTextEvidenceForPrompt import/usage in llmModerationClient
- formatModerationTextEvidenceForPrompt import/usage in conversationContext
- stale re-exports in index.ts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Nama karakter game/anime populer seperti 'Furina' dari Genshin Impact
sering kena false positive sebagai 'sexual_deviation' karena kemiripan
fonetik dengan kata 'furry'. Menambahkan aturan eksplisit bahwa nama
karakter fiksi normal bukan referensi furry fetish, dgn pengecualian
jika konteks pesan secara eksplisit membahas aspek fetish/seksual.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements a context-aware moderation system by tracking user behavior
and channel-specific norms to improve AI decision-making accuracy.
- Adds `user_reputations` table to track trust scores, clean streaks,
and infraction history.
- Adds `channel_cultures` table to store AI-generated summaries of
channel-specific norms and slang.
- Implements `userReputationStore` to autonomously update user scores
based on moderation outcomes (clean vs. flagged).
- Implements `cultureLearner` and `channelCultureStore` to manage
evolving channel contexts.
- Enhances LLM prompts to inject user reputation (trust scores,
history) and channel culture summaries, enabling "wisdom-based"
moderation (e.g., giving benefit of the doubt to high-trust users).
- Integrates reputation and culture updates into the existing
`aiAnalyzer` pipeline.
Refactors the AI moderation pipeline to improve concurrency control and
cache efficiency by moving from user-centric to content-centric caching.
- Implements a distributed locking mechanism for media analysis using
`acquireMediaAnalysisLock` to prevent redundant LLM vision calls across
multiple pods.
- Transitions text moderation caching from `user_mod:userId:hash` to a
purely content-based `text_mod:hash` approach to increase hit rates.
- Enhances `getPendingMessagesByConversation` with atomic transactions
and `FOR UPDATE SKIP LOCKED` to safely transition messages from
`pending` to `processing` state.
- Adds `processing` status to the `AIStatus` type and database schema to
track active analysis lifecycles.
- Implements polling logic in `llmModerationClient.ts` to wait for
in-progress media analyses.
MythEclipse