- Count only flagged messages in the active users Flag column
- Coerce PostgreSQL count results to numbers before calculating violation_score
- Prevent string concatenation in flagged*3 + warned scoring
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
API provider returns 400 error for these params. Removed from all LLM calls
in llmModerationClient.ts and indonesianTextNormalizer.ts.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Remove LOCAL_BADWORDS array (25 hardcoded words) and FALSE_POSITIVE_WHITELISTS
- Remove detectLocalBadwords function — all detection now goes through API pipeline
- Fix DEFERRAL_ANALYSIS_PATTERN: remove overly broad patterns (admin perlu, bisa berpotensi, maaf/sorry, saya tidak yakin)
- Expand DEFERRAL_EXCEPTION_PATTERN to catch more decisive-deferral variations
- Update tests to reflect API-only detection (local fallback removed)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add XML delimiters to prevent prompt injection (R1)
- Use JSON Schema response format instead of json_object (R2)
- Add concurrency limiter via p-limit (R3)
- Add timeout per media analysis call (R4)
- Resize images with sharp before vision API (R5)
- Split text batches when exceeding batch size limit (R6)
- Add few-shot examples to system prompt (R7)
- Modularize system prompt builder (R8)
- Enhance deferral detection regex with exception patterns (R9)
- Sanitize error messages to avoid leaking internals (R10)
New files: concurrencyLimiter.ts, imageResizer.ts, moderationPrompt.ts
Updated: llmModerationClient.ts, config.ts, package.json, tests
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Extract custom emoji metadata with Discord CDN URLs
- Download and send emoji images to vision model for moderation analysis
- Render custom emoji as inline images in dashboard instead of raw <:name:id> text
- Add emoji vision cache with deterministic keying by emoji ID
- Add custom emoji vision prompt for context-aware moderation
- Updated .env.example to include Groq API configuration.
- Enhanced config schema in config.ts to support Groq API keys and settings.
- Implemented Groq moderation API call in indonesianTextNormalizer.ts as a fallback for badword detection.
- Removed the Indonesian slang normalization function and related tests to streamline moderation logic.
- Updated tests to reflect changes in moderation strategy, focusing on emoji normalization and badword detection.
fix(redis): enhance Redis connection handling with fallback to in-memory storage
chore(sql): create missing messages and attachments tables with necessary constraints and indexes
- Updated test files to use a separate test database configuration.
- Introduced a new helper module for managing test database operations.
- Added a setup file to configure the environment for tests.
- Created new database migration scripts to optimize message indexing.
- Added a sample environment file for test database configuration.
- Removed SQLite support from the configuration and database initialization logic.
- Updated database migration scripts to focus solely on PostgreSQL migrations.
- Simplified logging messages to reflect PostgreSQL usage.
- Adjusted database schema definitions to remove SQLite-specific types and structures.
- Modified tests to ensure compatibility with PostgreSQL, including changes to table creation and data types.
- Cleaned up unused imports and code related to SQLite.
- Added new API endpoints for daily trend data and activity heatmap in analyticsRoutes.ts.
- Created new frontend components: ActivityChart, ControlBar, Heatmap, SummaryCards, TopicList, TrendChart, UserTable, and ViolatorTable for displaying analytics data.
- Implemented loading and empty states in the new components.
- Enhanced the existing moderation tests with remote fallback handling for Indonesian text normalization.
Drop 3 columns that were written but never read:
- ai_moderation_raw: raw LLM JSON response (~KB per message, never consumed)
- ai_policy_version: hardcoded 'default-2026-05-30', never used for decisions
- ai_evidence: JSON evidence array, never read after write
Changes:
- schema.ts: remove columns from both Postgres and SQLite table definitions
- messageStore.ts: remove from AIAnalysisUpdate interface and SET clauses
- aiAnalyzer.ts: remove from individual fallback update calls
- aiAnalysisWorker.ts: remove raw write, add missing fields (categories, severity, etc.)
- types.ts: remove from MessageRecord interface
- analysisRoutes.ts: remove from reset analysis call
- New migration: src/database/migrations/001_drop_unused_ai_columns.sql
- Migration applied to live DB: 27 columns → 24 columns
Kept ai_error (useful for future debugging, currently 0 non-null)
Kept metadata (1.9MB total, used for AI sticker/embed evidence analysis)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Adjusted formatting in conversationContext.ts for better token estimation readability.
- Enhanced readability in indonesianTextNormalizer.ts by formatting multiline replacements.
- Reformatted badword lists and whitelists in indonesianTextNormalizer.ts for consistency.
- Improved function signatures in messageStore.ts for clarity.
- Reformatted messageCapture.ts to enhance readability of channel ID checks.
- Cleaned up error logging in messageStore.ts and retentionManager.ts for better clarity.
- Reformatted indonesianSlangLexicon.ts for consistent object formatting.
- Enhanced URL fetching regex patterns in urlFetcher.ts for better readability.
- Simplified query parameter destructuring in analyticsRoutes.ts for cleaner code.
- Improved test readability in autoDeleteManager.test.ts and indonesianTextNormalizer.test.ts by formatting expectations.
- Cleaned up whitespace in messageCaptureFilter.test.ts for consistency.
- New sticker-specific vision prompt that tells LLM stickers are cartoon/meme art, not real photos
- New text-only warning for stickers that fail to download — prevents flagging based on name alone
- Updated system prompt with dedicated sticker guidance section (looser standards for cartoon content)
- Filesystem-backed sticker cache (keyed by name, 7-day TTL, 100MB max with LRU eviction)
- Config: STICKER_CACHE_DIR and STICKER_CACHE_MAX_SIZE_MB with defaults
- Updated .env.example with auto-delete + sticker config docs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Added configuration options for NVIDIA Nemotron API key, model, and base URL.
- Refactored badword detection to utilize NVIDIA API, with a fallback to a local badword list.
- Updated moderation functions to handle asynchronous operations for text evidence generation.
- Removed dependency on the `indonesian-badwords` package and implemented custom detection logic.
- Enhanced tests to accommodate asynchronous behavior and validate new detection methods.
- Derive severity from status/score when ai_severity missing
- Derive recommended_action from severity/status when field missing
- Upgrade skip logs from debug to info so skip reasons are visible
- Allow both 'flagged' and 'warn' status for eligibility
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Implemented `useAnalytics` hook for fetching and managing analytics data.
- Created `analyticsStore.ts` to handle database queries for hourly stats, topic trends, user leaderboard, and moderation stats.
- Added Express routes for analytics endpoints including overview, hourly stats, topic trends, user leaderboard, moderation stats, and top violators.
- Introduced a utility function `filterHits` for filtering specific terms in text.
- Add indonesian-badwords dependency for local lexical signal
- Add Indonesian slang lexicon with woy/woi/hadeh as safe casual terms
- Normalize Discord custom emoji <:name:id> to [emoji:name] in prompts
- Wire normalization evidence into both conversationContext and llmModerationClient prompts
- Harden system prompt: woy/woi are casual greetings, not SARA/hate
- Add tests for emoji normalization, slang mapping, badword detection
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
