Commit Graph
212 Commits
Author SHA1 Message Date
MythEclipse 18ee8a35d2 feat: SearXNG search enrichment for moderation
- extractSearchQueries(): extract specific terms from trigger patterns
  instead of searching entire message (e.g. 'bokuno pico' from 'nonton bokuno pico anime')
- Add hentai studio/publisher triggers (Pink Pineapple, Maryjane, etc.)
- Fix media path SearXNG: use extractSearchQueries instead of quoted-only
- Prompt: add <web_searches> usage instructions so LLM actually uses search data
- Priority: searxng evidence > web_content > internal model knowledge
2026-06-22 10:58:52 +07:00
MythEclipse 641fb47d10 fix: hentai reference detection + severity high auto-delete
- Add explicit hentai title detection rule (Boku no Pico, Euphoria, Bible Black,
  Kuroinu, loli/shota) -> sexual_deviation severity high, recommended_action delete
- LLM flagged correctly but severity 'medium' caused auto-delete skip in fallback path
2026-06-22 10:28:16 +07:00
MythEclipse f502918b27 feat: multimodal video detection + reply/forward/crosspost + batch optimization
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
2026-06-22 09:46:06 +07:00
MythEclipse dc62b283c0 refactor(ai-moderation): fix all 8 issues per review
- #2: Duplikasi contoh → single-source ALL_EXAMPLES array + formatExamples()
- #3: Hierarki evasion → Level 1 (Wajib Flag) vs Level 2 (Pilih Clean)
- #4: Dead code → remove duplicate PromptMode, cleanup
- #5: Sanitasi AI content → sanitizeAiContent() + CDATA wrapping for
      channel culture & user profile (prevents prompt injection)
- #6: Contoh personality → match actual XML format (<message>,
      <user_profile>, <content> tags instead of plain text)
- #7: Contoh invite link → example #29 added
- #8: 'Deskripsi selesai.' removed from stickerPrompt.ts
- Image analysis balance: teks = gambar (equal weight), no more bias
- Removed LGBT exclusion from SYSTEM_RULES
- Removed all template fallback language, enforce specific descriptions
2026-06-21 22:14:59 +07:00
MythEclipse 04f2862abc feat(ai-moderation): profile learner ai_status filter + channel-aware + simple fallback memory
- userProfileLearner.ts: filter query to only clean messages (eq ai_status='clean')
  to prevent profile contamination from flagged content. Also select channel_id
  to group messages by channel in prompt, enabling channel-aware personality
  summarization (user may behave differently across channels).
- llmModerationClient.ts (runSimpleTextFallback): inject user profile into
  both the classify prompt and the reason prompt, so even the last-resort
  fallback path has personality/memory context instead of being blind.
2026-06-21 20:07:51 +07:00
MythEclipse e5fa2b3f30 feat(ai-moderation): personality-rich user/channel profiling & memory-aware analysis
- moderationPrompt.ts: Add PERSONALITY & MEMORY section instructing LLM to
  use user_profile and channel_culture in analysis, with 3 new few-shot
  examples (normal profile use, deviation detection, don't overfit)
- userProfileLearner.ts: Richer personality summary prompt — asks for
  communication style, passion topics, unique habits/signature traits,
  social dynamics, and writes in warm observer prose
- cultureLearner.ts: Richer channel culture summary — asks for 'vibe',
  inside jokes, social norms, dynamics. Writes in anthropologist style
2026-06-21 19:11:11 +07:00
MythEclipse 21b818a344 feat: add forwarding & reply detection with frontend indicators
- Enhanced messageMetadata to capture referenced message content
- Added reply indicator (↩️ Replying to @username + content snippet)
- Added forward badge (🔁 Forwarded) and crosspost badge (📢 Crossposted)
- Frontend auto-fetches referenced message content when metadata is empty
- Added getMessageById API helper
2026-06-21 18:22:27 +07:00
MythEclipse 2b084b326f a 2026-06-14 05:10:58 +07:00
MythEclipseandClaude 6733ea8f01 fix(db): add 0011_add_voice_transcription to drizzle journal
Migration SQL file existed but was never registered in _journal.json,
so Drizzle skipped it. Every voice recording insert failed with:
  PG code 42703: column "transcription" does not exist

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-14 04:59:44 +07:00
MythEclipseandClaude a2beda77c4 fix(db): surface PG error code/detail/cause in voice recording insert failures
Drizzle wraps PG errors — the previous log only showed the formatted query
text. Now extracts code, detail, schema, table, constraint, severity from
the error envelope and from .cause to make the actual Postgres error visible.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-14 04:40:22 +07:00
MythEclipseandClaude dbc2879e16 fix(voice): resolve Redis subscriber corruption, transmitter races, and DB error logging
Redis root cause: VoiceHandler passed redisPub to transmitter.start(),
which called .subscribe() on it — permanently converting the publish
connection to subscriber mode. Every subsequent command reply and status
update failed.

Fixes:
1. Transmitter now creates its own Redis client via new IORedis()
2. Mutual exclusion gate serialises start/stop to prevent null-deref races
3. PassThrough drain listeners cleaned up to stop MaxListenersExceeded
4. FFmpeg SIGTERM flagged as expected exit (no more false level-50 errors)
5. Voice recording repo captures PG error code/detail for diagnostics

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-14 04:23:04 +07:00
MythEclipse 32b0b20ca7 chore(services): update components based on recent changes
Changes:
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
2026-06-14 03:31:40 +07:00
MythEclipse 93634d129a chore(services): update components based on recent changes
Changes:
 services/discord-gateway/src/modules/command-handler/voice.handler.ts | 10 ++++----
 services/discord-gateway/src/modules/voice-recording/transmitter.ts   | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------
 2 files changed, 72 insertions(+), 29 deletions(-)
2026-06-14 03:28:24 +07:00
MythEclipseandClaude c317cfb420 fix(ci): resolve TS build errors across all services
- recordings.routes.ts: cast req.params.id to string (express 5 typed params)
- recordings.service.ts: cast Record<string,unknown>[] via unknown first
- guilds.routes.ts: cast req.params.guildId to string (2 sites)
- transmitter.ts: capture pcmStream ref locally + re-acquire in drain callback

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-14 01:17:18 +07:00
MythEclipse 92d73fe12d chore(services): update components based on recent changes
Changes:
 .env.example                                                     | 5 +++++
 services/discord-gateway/src/modules/voice-recording/recorder.ts | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)
2026-06-14 00:57:08 +07:00
MythEclipse 7789aca620 chore(services): update components based on recent changes
Changes:
 services/discord-gateway/src/app/shutdown.ts | 8 ++++++++
 1 file changed, 8 insertions(+)
2026-06-14 00:45:59 +07:00
MythEclipse ac661b64f4 chore(services): update components based on recent changes
Changes:
 services/discord-gateway/src/app/bootstrap.ts                                    | 24 ++++++++++++++++++++++++
 services/discord-gateway/src/modules/voice-recording/recorder.ts                 | 10 ++++++++++
 services/discord-gateway/src/modules/voice-recording/recorder/speakingHandler.ts |  9 ++++++++-
 3 files changed, 42 insertions(+), 1 deletion(-)
2026-06-14 00:45:31 +07:00
MythEclipse c886dfb184 chore(services): update components based on recent changes
Changes:
 services/discord-gateway/src/modules/voice-pcm-ws/index.ts            |   1 +
 services/discord-gateway/src/modules/voice-pcm-ws/voicePcmWsClient.ts | 151 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 152 insertions(+)
2026-06-14 00:43:50 +07:00
MythEclipse 28baeda5ad chore(services): update components based on recent changes
Changes:
 packages/shared/src/config/index.ts                                          |   7 +++
 packages/shared/src/moderation-types.ts                                      |   1 +
 services/backend/src/modules/recordings/recordings.service.ts                |   3 +-
 services/discord-gateway/drizzle/migrations/0011_add_voice_transcription.sql |   1 +
 services/discord-gateway/src/modules/voice-recording/recorder/segment.ts     |  10 ++++
 services/discord-gateway/src/modules/voice-recording/recorder/uploader.ts    |  15 ++++++
 services/discord-gateway/src/modules/voice-recording/voiceTranscriber.ts     |  51 ++++++++++++++++++
 services/discord-gateway/src/shared/database/schema.ts                       |   1 +
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts           |  17 ++++++
 services/frontend/src/features/live/components/RecordingsSubPanel.tsx        | 108 +++++++++++++++++++-------------------
2026-06-13 17:08:54 +07:00
MythEclipse d0d2c316cf chore(auto): task completed - unknown 2026-06-13 15:11:06 +07:00
MythEclipse 265251233a chore(auto): task completed - unknown 2026-06-13 15:03:21 +07:00
MythEclipse 9e9ce463cb chore(auto): task completed - unknown 2026-06-13 14:59:00 +07:00
MythEclipseandClaude 3965ea79bc fix(gateway): voice speaking handler, metadata cache, transmitter backpressure
- speakingHandler: check subscriptions.has() BEFORE voiceActiveUser broadcast
- segment.ts: add LRU metadata cache (200 entries, avoids Discord API in hotpath)
- transmitter.ts: add backpressure handling with drain queue/flush
  (PassThrough.write() return value was ignored — potential OOM under load)
- cleanup backpressure queue on transmitter stop

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 14:38:06 +07:00
MythEclipseandClaude 29d4cb87b6 fix(gateway): voice module critical bugs
- shutdown ordering: DB pool closed AFTER voice disconnect (was before)
- muxer: amix (mix) -> concat (sequential) filter for OGG segments
- upload retries: 0->3 with proper backoff 1-10s
- teleUpload retry backoff: minTimeout 0->1000, maxTimeout 0->10000
- mediaSource: 64KB buffer bound on readFirstTwoLines, 1MB on extractMediaInfo
- ffmpegProcess: pipe stderr to structured logger instead of inherit

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 14:35:35 +07:00
MythEclipseandClaude 14c0081f01 feat(discord-gateway): implement voice & push improvements
- Voice disconnect broadcast on stopRecording
- Multi-guild voice support (VoiceController Map<guildId>)
- Session finalization + auto-enqueue muxer job
- Recordings API: duration field, channelId/userId filters
- Transmitter Redis connection reuse (shared conn)
- FFmpeg stderr memory cap (4KB limit)
- 10 new Redis event channels + Redis bridge subscriptions
- New DB tables: message_reactions, message_edits
- Webhook notification module
- Gateway metrics / Prometheus endpoint
- Multi-guild message capture (MONITOR_GUILD_IDS array)
- Thread tracking, presence, channel topic, guild member events
- Edit history snapshot on message update
- Muxer audio post-processing worker

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 13:51:21 +07:00
MythEclipse 111ba0a139 chore(auto): task completed - unknown 2026-06-13 13:36:13 +07:00
MythEclipse 942c33db7e chore(auto): task completed - unknown 2026-06-13 13:35:56 +07:00
MythEclipse eab8505889 chore(auto): task completed - unknown 2026-06-13 13:35:41 +07:00
MythEclipse 2e14b5e4ed chore(auto): task completed - unknown 2026-06-13 13:35:11 +07:00
MythEclipse d569ffa2db chore(auto): task completed - unknown 2026-06-13 13:12:49 +07:00
MythEclipse 4097b23dc3 chore(auto): task completed - unknown 2026-06-13 13:12:41 +07:00
MythEclipse e413cf9fca chore(auto): task completed - unknown 2026-06-13 13:12:29 +07:00
MythEclipse ff006980f8 chore(auto): task completed - unknown 2026-06-13 13:12:01 +07:00
MythEclipse 140a0ba46a chore(auto): task completed - unknown 2026-06-13 13:11:49 +07:00
MythEclipseandClaude 30f8d7cce3 feat: replace corrections/tuner with dashboard module
Replace the corrections/adaptive-prompt-tuner feature with a new
dashboard module providing server stats and user profile overview.

Backend:
- Add dashboard module (routes, service, repository) with stats + user list + user detail endpoints
- Remove corrections module entirely
- Wire dashboard router in app.ts

Frontend:
- Add dashboard feature (DashboardStats, UserSummaryList, UserProfileDetail components + useDashboard hook)
- Remove tuner feature (CorrectionStats, CorrectionHistory, SubmitCorrection, useCorrections)
- Update API client from corrections → dashboard types/fns
- Rename tab 'tuner' → 'dashboard'
- Update MobileTabBar, Header, Sidebar links

Tests:
- Expand backend placeholder test with dashboard assertions
- Expand discord-gateway placeholder test with config/channel assertions

AI moderation:
- llmModerationClient: improve status/reply detection, expand safety categories, fix timer reset
- userProfileLearner: fix isReply refinement
- userProfileStore: add pending cache check
- messageMetadata: add crosspost type mapping
- migrate.ts: improve partial-index safety in schema push

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 11:24:42 +07:00
MythEclipseandClaude e3249edb6c feat: detect reply/forward/crosspost in message capture + inject into AI moderation
- Add is_reply, is_forward, is_crosspost, reference_message_id,
  reference_channel_id, reference_guild_id columns to messages table
- Update MessageRecord type with reference fields
- Extract reply/forward/crosspost from Discord message type/flags
- Track reference.type (DEFAULT=reply, FORWARD) and CROSSPOSTED flag
- Inject <reference> XML with parent content into LLM moderation prompt
- Add reply/forward/crosspost rules to moderation prompt rules
- Format context messages with [reply_to], [forward_from], [crosspost]
- Add migration 0009

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 00:55:42 +07:00
MythEclipseandClaude 0ac056dc8a fix(ai-moderation): audit prompts to reduce false positives
- ENTROPY rule: replace 'PILIH WARN' default with 'PILIH CLEAN'
- Add safe-list for code/log/stack traces/project names
- Narrow sexual_deviation: only flag explicit sexual solicitation
- Remove overbroad ontological graph (kostum hewan → furry) detection
- Add clean few-shot examples: error logs, project names, orientation disclosure
- Update simple fallback prompt with FP prevention rules
- Remove LGBT/furry from offensive username criteria

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 00:01:09 +07:00
MythEclipseandClaude 12f9b55000 fix(ai-moderation): implement prompt analysis recommendations
- Expand IMPHNEN domain rule to cover wildcard (*.imphnen.*)
- Trim redundant SARA examples from TEXT_ONLY_MODE (save ~950 tokens)
- Add debugging logs for channel culture injection into prompt
- Sync flag validation set with missing flags: potential_evasion, unclear_context

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 23:40:25 +07:00
MythEclipseandClaude b508a39ff7 fix(ai-moderation): prevent false positive SARA flag for IMPHNEN project name
- Add explicit system rule that IMPHNEN is the project's own name, not religion
- Rename 'Imphnemia 11:17' example to 'Kitabonia 11:17' to avoid name collision
- Ensures mentioning/promoting the project URL is not flagged as SARA

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 23:28:09 +07:00
MythEclipseandClaude fbc2184c6e feat(ai-moderation): add user profile self-learning system
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.

- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
  calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 20:11:34 +07:00
MythEclipseandClaude 6effb51b5d fix(ai-moderation): don't cache text-only results for messages with media
- hasMediaContent now also checks evidence.attachments from metadata
  (not just DB attachment records), catching the race where attachment
  DB rows aren't inserted yet when analysis runs.
- Cache-hit guard: treat cached entries as miss when the message has
  media evidence in metadata, so stale 24h-freezes are avoided.
- Cache-write guard: skip storing text-only analysis results for
  messages whose metadata shows attachments/stickers/embeds. This
  prevents a text-only 'clean' result (from failed vision) being
  frozen for 24h, blocking future re-analysis with full media context.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 18:45:33 +07:00
MythEclipseandClaude 33b787dbd6 fix(migration): make all DDL idempotent with IF NOT EXISTS
- 0000: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS,
  FK constraint wrapped in DO 539994 EXCEPTION WHEN duplicate_object
- 0001: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION WHEN duplicate_column
- 0002: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0003: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0006: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION,
  DROP COLUMN IF EXISTS

Allows migrations to run cleanly regardless of DB state —
handles fresh DB, partial migration, or wiped __drizzle_migrations.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 18:05:09 +07:00
MythEclipseandClaude fcb199a6ae fix(capture): exclude threads inside excluded channels
- Resolve parent channel ID for threads before checking
  EXCLUDED_CHANNEL_IDS set
- Thread messages now blocked if their parent channel is excluded

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 17:07:34 +07:00
MythEclipseandClaude c91f9f18cc refactor(capture): DRY age-restricted check, use Set for excluded IDs
- Move isAgeRestrictedMessage to messageMetadata.ts alongside
  isAgeRestrictedMetadata — single source of truth for NSFW logic
- Replace || chain of channel IDs with a Set for O(1) lookup
- Import isAgeRestrictedMessage in capture layer instead of inline

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 17:03:01 +07:00
MythEclipseandClaude ebfe942db9 fix(capture): skip age-restricted/NSFW channels entirely
Auto-filter all age-restricted and NSFW channels at capture layer
so messages never enter the database or reach frontend.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 17:00:02 +07:00
MythEclipseandClaude 1efa7b6505 feat(ignore): exclude channel 1323365288447574128 from capture
Add hardcoded channel ID to shouldCaptureMessageLocation so messages
from this channel are not stored, analyzed by AI, or sent to frontend.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 16:49:14 +07:00
MythEclipseandClaude Fable 5 906aef1807 fix(database): handle PG15+ public schema CREATE privilege error in seedDrizzleHistory
In PG15+, the CREATE privilege on the public schema is revoked from
non-owner roles by default. The seedDrizzleHistory function's
CREATE TABLE IF NOT EXISTS for __drizzle_migrations fails with 42501,
causing the gateway to crash-loop on startup.

Wrap the CREATE in a try/catch for 42501 — if the table already exists
(created by a prior run), we continue gracefully; otherwise re-throw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 04:06:11 +07:00
MythEclipse 2e5371f3ee fix(discord-gateway): bypass Drizzle PG15 schema creation restrictions 2026-06-11 03:53:48 +07:00
MythEclipseandClaude Fable 5 378f1b6bdc fix(database): use public schema for drizzle migrations to avoid permission error
PostgreSQL error 42501 when Drizzle tries CREATE SCHEMA IF NOT EXISTS
"drizzle" — the DB user lacks schema creation privileges. Configuring
migrationsSchema: "public" keeps __drizzle_migrations in the existing
public schema, matching what seedDrizzleHistory already expects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 03:34:16 +07:00
MythEclipse 8b114ca278 refactor(ai-moderation): expand SARA and religious blasphemy detection rules
Update the moderation prompt to include high-priority detection categories for:
- Fake scripture/verse parodies
- Claims of divinity or false religious movements
- Misuse of theological terms as internet slang/memes
- Mockery of religious figures and rituals

This change ensures stricter enforcement of SARA (Suku, Agama, Ras, Antargolongan) policies by explicitly defining religious blasphemy and parody as high-severity violations.
2026-06-11 02:40:11 +07:00