Two-layer fix for forwarded messages showing as empty/clean:
Layer 1 (messageMetadata.ts): getReferencedMessageContent() now falls
back to message.messageSnapshots Collection when channel.messages.cache
lookup fails. Discord stores forward content in message_snapshots API
field, not in message.content.
Layer 2 (moderationBuilders.ts): buildReferenceXml() now parses msg.
metadata JSON to extract reference.content when DB getMessageById()
fails (cross-server forwards not in local DB).
Previously: forward messages captured with empty parentContent →
LLM saw no reference text → '99% confidence, pesan kosong'.
Now: forward content flows through capture → metadata → analysis.
- Previously: text+media message went ONLY to media array → text waited for vision
- Now: text part goes to text batch (immediate LLM analysis), media parallel
- DB update is idempotent — second write to same message_id overwrites
- User sees text moderation results instantly, media follows when ready
- text-only and media analysis now run concurrently via Promise.all
- text no longer blocks on media download + vision analysis
- each path independently saves to DB when its own results are ready
- same batch still uses single context fetch + attachment lookup
- Remove shouldSearchContent() trigger gate — search runs on all messages
- extractSearchQueries() now extracts from ANY message, not just trigger-matched
- Redis cache (24h TTL) prevents redundant searches for same query
- initSearxngCache() lazy-connects via config.REDIS_URL
- Cache miss→API, hit→skip — fire-and-forget writes
- Both text batch + media path simplified
- Remove hardcoded hentai title lists (Boku no Pico, Euphoria, etc.) from prompt
- Remove hardcoded SARA examples from prompt (Kitabonia, etc.)
- Prompt now tells LLM to use <web_searches> as evidence instead of hardcoded knowledge
- Evidence priority: searxng > web_content > media_analysis > internal model
- Code-side triggers in searxngSearch.ts still filter which messages to search
- LLM makes final decision based on search results, not static lists
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
- userProfileLearner.ts: filter query to only clean messages (eq ai_status='clean')
to prevent profile contamination from flagged content. Also select channel_id
to group messages by channel in prompt, enabling channel-aware personality
summarization (user may behave differently across channels).
- llmModerationClient.ts (runSimpleTextFallback): inject user profile into
both the classify prompt and the reason prompt, so even the last-resort
fallback path has personality/memory context instead of being blind.
Redis root cause: VoiceHandler passed redisPub to transmitter.start(),
which called .subscribe() on it — permanently converting the publish
connection to subscriber mode. Every subsequent command reply and status
update failed.
Fixes:
1. Transmitter now creates its own Redis client via new IORedis()
2. Mutual exclusion gate serialises start/stop to prevent null-deref races
3. PassThrough drain listeners cleaned up to stop MaxListenersExceeded
4. FFmpeg SIGTERM flagged as expected exit (no more false level-50 errors)
5. Voice recording repo captures PG error code/detail for diagnostics
Co-Authored-By: Claude <noreply@anthropic.com>
- Expand IMPHNEN domain rule to cover wildcard (*.imphnen.*)
- Trim redundant SARA examples from TEXT_ONLY_MODE (save ~950 tokens)
- Add debugging logs for channel culture injection into prompt
- Sync flag validation set with missing flags: potential_evasion, unclear_context
Co-Authored-By: Claude <noreply@anthropic.com>
- Add explicit system rule that IMPHNEN is the project's own name, not religion
- Rename 'Imphnemia 11:17' example to 'Kitabonia 11:17' to avoid name collision
- Ensures mentioning/promoting the project URL is not flagged as SARA
Co-Authored-By: Claude <noreply@anthropic.com>
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.
- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table
Co-Authored-By: Claude <noreply@anthropic.com>
- hasMediaContent now also checks evidence.attachments from metadata
(not just DB attachment records), catching the race where attachment
DB rows aren't inserted yet when analysis runs.
- Cache-hit guard: treat cached entries as miss when the message has
media evidence in metadata, so stale 24h-freezes are avoided.
- Cache-write guard: skip storing text-only analysis results for
messages whose metadata shows attachments/stickers/embeds. This
prevents a text-only 'clean' result (from failed vision) being
frozen for 24h, blocking future re-analysis with full media context.
Co-Authored-By: Claude <noreply@anthropic.com>
- Resolve parent channel ID for threads before checking
EXCLUDED_CHANNEL_IDS set
- Thread messages now blocked if their parent channel is excluded
Co-Authored-By: Claude <noreply@anthropic.com>
- Move isAgeRestrictedMessage to messageMetadata.ts alongside
isAgeRestrictedMetadata — single source of truth for NSFW logic
- Replace || chain of channel IDs with a Set for O(1) lookup
- Import isAgeRestrictedMessage in capture layer instead of inline
Co-Authored-By: Claude <noreply@anthropic.com>
Auto-filter all age-restricted and NSFW channels at capture layer
so messages never enter the database or reach frontend.
Co-Authored-By: Claude <noreply@anthropic.com>
Add hardcoded channel ID to shouldCaptureMessageLocation so messages
from this channel are not stored, analyzed by AI, or sent to frontend.
Co-Authored-By: Claude <noreply@anthropic.com>
Update the moderation prompt to include high-priority detection categories for:
- Fake scripture/verse parodies
- Claims of divinity or false religious movements
- Misuse of theological terms as internet slang/memes
- Mockery of religious figures and rituals
This change ensures stricter enforcement of SARA (Suku, Agama, Ras, Antargolongan) policies by explicitly defining religious blasphemy and parody as high-severity violations.
This commit introduces several significant improvements across the backend and gateway services:
- **Data Retention**: Added an automated cleanup scheduler in `discord-gateway` to prune expired messages, attachments, and voice recordings based on configurable retention policies.
- **Observability**: Integrated `prom-client` in the `backend` service to expose Prometheus metrics via `/api/metrics` and added default Node.js runtime metrics.
- **Media Handling**: Enhanced `MediaHandler` in `discord-gateway` to support media URL resolution and improved playback status tracking.
- **API & Config**: Expanded the configuration endpoint to expose more system settings and reorganized `.env.example` for better readability.
- **Refactoring & Cleanup**:
- Removed unused `better-sqlite3` dependency.
- Refactored voice channel routing.
- Improved error handling and testing coverage with comprehensive unit tests for shared utilities and error classes.
- **Documentation**: Added `MEMORY.md` for project context.
asepharyana