MythEclipse/GMW
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
837 Commits 1 Branch 0 Tags
bf36bf55fec5f5575dff0e9a27ebc36ce3fa1773
Commit Graph
223 Commits
Author SHA1 Message Date
asepharyanaandworkflow agents ade5d6a7c3 fix: backend and discord-gateway improvements 
- Update shared database schema
- Add shared utils
- Refactor backend middleware, auth routes, and dashboard repository
- Improve media analysis client with better error handling
- Fix searxng search URL construction
- Update URL fetcher for robustness

Co-authored-by: workflow agents
 2026-07-02 06:02:07 +07:00
asepharyana bf9d018f7a fix: skip AI analysis for channel 1508059937031589949 2026-07-02 04:28:33 +07:00
asepharyana fa82593a38 fix: switch to node:22-slim, install deps from npm, remove vendor 2026-07-02 04:07:18 +07:00
asepharyana 7efaf00c93 Revert "feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config" 
This reverts commit d59b59a7a7.
 2026-07-02 03:54:44 +07:00
asepharyana d59b59a7a7 feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config 
Frontend:
- migrate from Vite to Astro (astro.config.mjs, pages/, layouts/)
- add admin panel, settings page, command palette, error boundary
- refactor App.tsx, MascotChatbot, Sidebar, Header, DashboardLayout
- update API client, WebSocket, auth, dashboard features

Backend:
- add admin module and config routes
- refactor middlewares, Redis connection, WebSocket server/bridge
- add runtime config loader

Discord Gateway:
- refactor AI moderation: circuit breaker, concurrency limiter, fallback processor
- add media analysis client, Seaxng search, user profile learner
- add new drizzle migration

Shared:
- extend database schema, add new config fields
 2026-07-02 00:02:41 +07:00
MythEclipse d5c22a3959 fix(discord-gateway): handle forwarded message content via messageSnapshots 
Two-layer fix for forwarded messages showing as empty/clean:

Layer 1 (messageMetadata.ts): getReferencedMessageContent() now falls
back to message.messageSnapshots Collection when channel.messages.cache
lookup fails. Discord stores forward content in message_snapshots API
field, not in message.content.

Layer 2 (moderationBuilders.ts): buildReferenceXml() now parses msg.
metadata JSON to extract reference.content when DB getMessageById()
fails (cross-server forwards not in local DB).

Previously: forward messages captured with empty parentContent →
LLM saw no reference text → '99% confidence, pesan kosong'.
Now: forward content flows through capture → metadata → analysis.
 2026-06-23 23:27:23 +07:00
MythEclipse ed4a506ca7 refactor: split llmModerationClient.ts + add tests + metrics 
## Split llmModerationClient.ts (2103 → 3 files)
- **moderationBuilders.ts** (67 lines) — shared: escapeXml, getAnalysisContent, buildReferenceXml
- **mediaAnalysisClient.ts** (656 lines) — vision analysis with multi-layer LRU/DB/phash caching, image/video download, ffmpeg frame extraction, prepareMediaMessage
- **moderationOrchestrator.ts** (998 lines) — callModerationLLM, runTextOnlyBatch, runMediaBatch, runModerationAnalysis, runSimpleTextFallback
- **llmModerationClient.ts** (30 lines) — re-export bridge (backward compat)

No import changes needed — aiAnalysisWorker.ts still imports from llmModerationClient.js.

## Unit tests (backend)
- vitest.config.ts + e2e.test.ts with 9 tests against production:
  - health, metrics, dashboard/stats, recordings, config, auth, guilds, negative (404/400)

## Monitoring metrics
- moderationMetrics.ts in backend health module:
  - LLM call count/duration/tokens
  - Cache hit/miss per layer
  - Media analysis count/download duration
  - Batch size distribution, errors, SearXNG, auto-delete
 2026-06-22 20:38:34 +07:00
MythEclipse ad84d0a1bb fix: messages with text+media also go into text batch immediately 
- Previously: text+media message went ONLY to media array → text waited for vision
- Now: text part goes to text batch (immediate LLM analysis), media parallel
- DB update is idempotent — second write to same message_id overwrites
- User sees text moderation results instantly, media follows when ready
 2026-06-22 17:45:32 +07:00
MythEclipse e8286247d6 fix: parallelize text + media LLM analysis instead of sequential 
- text-only and media analysis now run concurrently via Promise.all
- text no longer blocks on media download + vision analysis
- each path independently saves to DB when its own results are ready
- same batch still uses single context fetch + attachment lookup
 2026-06-22 17:42:46 +07:00
MythEclipse dfabdc85cd feat: search ALL messages via SearXNG + Redis cache dedup 
- Remove shouldSearchContent() trigger gate — search runs on all messages
- extractSearchQueries() now extracts from ANY message, not just trigger-matched
- Redis cache (24h TTL) prevents redundant searches for same query
- initSearxngCache() lazy-connects via config.REDIS_URL
- Cache miss→API, hit→skip — fire-and-forget writes
- Both text batch + media path simplified
 2026-06-22 12:41:48 +07:00
MythEclipse 84f87104e7 refactor: remove hardcoded lists from prompt, use SearXNG as single source of truth 
- Remove hardcoded hentai title lists (Boku no Pico, Euphoria, etc.) from prompt
- Remove hardcoded SARA examples from prompt (Kitabonia, etc.)
- Prompt now tells LLM to use <web_searches> as evidence instead of hardcoded knowledge
- Evidence priority: searxng > web_content > media_analysis > internal model
- Code-side triggers in searxngSearch.ts still filter which messages to search
- LLM makes final decision based on search results, not static lists
 2026-06-22 12:01:55 +07:00
MythEclipse 18ee8a35d2 feat: SearXNG search enrichment for moderation 
- extractSearchQueries(): extract specific terms from trigger patterns
  instead of searching entire message (e.g. 'bokuno pico' from 'nonton bokuno pico anime')
- Add hentai studio/publisher triggers (Pink Pineapple, Maryjane, etc.)
- Fix media path SearXNG: use extractSearchQueries instead of quoted-only
- Prompt: add <web_searches> usage instructions so LLM actually uses search data
- Priority: searxng evidence > web_content > internal model knowledge
 2026-06-22 10:58:52 +07:00
MythEclipse 641fb47d10 fix: hentai reference detection + severity high auto-delete 
- Add explicit hentai title detection rule (Boku no Pico, Euphoria, Bible Black,
  Kuroinu, loli/shota) -> sexual_deviation severity high, recommended_action delete
- LLM flagged correctly but severity 'medium' caused auto-delete skip in fallback path
 2026-06-22 10:28:16 +07:00
MythEclipse f502918b27 feat: multimodal video detection + reply/forward/crosspost + batch optimization 
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
 2026-06-22 09:46:06 +07:00
MythEclipse dc62b283c0 refactor(ai-moderation): fix all 8 issues per review 
- #2: Duplikasi contoh → single-source ALL_EXAMPLES array + formatExamples()
- #3: Hierarki evasion → Level 1 (Wajib Flag) vs Level 2 (Pilih Clean)
- #4: Dead code → remove duplicate PromptMode, cleanup
- #5: Sanitasi AI content → sanitizeAiContent() + CDATA wrapping for
      channel culture & user profile (prevents prompt injection)
- #6: Contoh personality → match actual XML format (<message>,
      <user_profile>, <content> tags instead of plain text)
- #7: Contoh invite link → example #29 added
- #8: 'Deskripsi selesai.' removed from stickerPrompt.ts
- Image analysis balance: teks = gambar (equal weight), no more bias
- Removed LGBT exclusion from SYSTEM_RULES
- Removed all template fallback language, enforce specific descriptions
 2026-06-21 22:14:59 +07:00
MythEclipse 04f2862abc feat(ai-moderation): profile learner ai_status filter + channel-aware + simple fallback memory 
- userProfileLearner.ts: filter query to only clean messages (eq ai_status='clean')
  to prevent profile contamination from flagged content. Also select channel_id
  to group messages by channel in prompt, enabling channel-aware personality
  summarization (user may behave differently across channels).
- llmModerationClient.ts (runSimpleTextFallback): inject user profile into
  both the classify prompt and the reason prompt, so even the last-resort
  fallback path has personality/memory context instead of being blind.
 2026-06-21 20:07:51 +07:00
MythEclipse e5fa2b3f30 feat(ai-moderation): personality-rich user/channel profiling & memory-aware analysis 
- moderationPrompt.ts: Add PERSONALITY & MEMORY section instructing LLM to
  use user_profile and channel_culture in analysis, with 3 new few-shot
  examples (normal profile use, deviation detection, don't overfit)
- userProfileLearner.ts: Richer personality summary prompt — asks for
  communication style, passion topics, unique habits/signature traits,
  social dynamics, and writes in warm observer prose
- cultureLearner.ts: Richer channel culture summary — asks for 'vibe',
  inside jokes, social norms, dynamics. Writes in anthropologist style
 2026-06-21 19:11:11 +07:00
MythEclipse 21b818a344 feat: add forwarding & reply detection with frontend indicators 
- Enhanced messageMetadata to capture referenced message content
- Added reply indicator (↩️ Replying to @username + content snippet)
- Added forward badge (🔁 Forwarded) and crosspost badge (📢 Crossposted)
- Frontend auto-fetches referenced message content when metadata is empty
- Added getMessageById API helper
 2026-06-21 18:22:27 +07:00
MythEclipse 2b084b326f a 2026-06-14 05:10:58 +07:00
MythEclipseandClaude 6733ea8f01 fix(db): add 0011_add_voice_transcription to drizzle journal 
Migration SQL file existed but was never registered in _journal.json,
so Drizzle skipped it. Every voice recording insert failed with:
  PG code 42703: column "transcription" does not exist

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-14 04:59:44 +07:00
MythEclipseandClaude a2beda77c4 fix(db): surface PG error code/detail/cause in voice recording insert failures 
Drizzle wraps PG errors — the previous log only showed the formatted query
text. Now extracts code, detail, schema, table, constraint, severity from
the error envelope and from .cause to make the actual Postgres error visible.

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-14 04:40:22 +07:00
MythEclipseandClaude dbc2879e16 fix(voice): resolve Redis subscriber corruption, transmitter races, and DB error logging 
Redis root cause: VoiceHandler passed redisPub to transmitter.start(),
which called .subscribe() on it — permanently converting the publish
connection to subscriber mode. Every subsequent command reply and status
update failed.

Fixes:
1. Transmitter now creates its own Redis client via new IORedis()
2. Mutual exclusion gate serialises start/stop to prevent null-deref races
3. PassThrough drain listeners cleaned up to stop MaxListenersExceeded
4. FFmpeg SIGTERM flagged as expected exit (no more false level-50 errors)
5. Voice recording repo captures PG error code/detail for diagnostics

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-14 04:23:04 +07:00
MythEclipse 32b0b20ca7 chore(services): update components based on recent changes 
Changes:
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)
 2026-06-14 03:31:40 +07:00
MythEclipse 93634d129a chore(services): update components based on recent changes 
Changes:
 services/discord-gateway/src/modules/command-handler/voice.handler.ts | 10 ++++----
 services/discord-gateway/src/modules/voice-recording/transmitter.ts   | 91 ++++++++++++++++++++++++++++++++++++++++++++++++++++--------------------
 2 files changed, 72 insertions(+), 29 deletions(-)
 2026-06-14 03:28:24 +07:00
MythEclipseandClaude c317cfb420 fix(ci): resolve TS build errors across all services 
- recordings.routes.ts: cast req.params.id to string (express 5 typed params)
- recordings.service.ts: cast Record<string,unknown>[] via unknown first
- guilds.routes.ts: cast req.params.guildId to string (2 sites)
- transmitter.ts: capture pcmStream ref locally + re-acquire in drain callback

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-14 01:17:18 +07:00
MythEclipse 92d73fe12d chore(services): update components based on recent changes 
Changes:
 .env.example                                                     | 5 +++++
 services/discord-gateway/src/modules/voice-recording/recorder.ts | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)
 2026-06-14 00:57:08 +07:00
MythEclipse 7789aca620 chore(services): update components based on recent changes 
Changes:
 services/discord-gateway/src/app/shutdown.ts | 8 ++++++++
 1 file changed, 8 insertions(+)
 2026-06-14 00:45:59 +07:00
MythEclipse ac661b64f4 chore(services): update components based on recent changes 
Changes:
 services/discord-gateway/src/app/bootstrap.ts                                    | 24 ++++++++++++++++++++++++
 services/discord-gateway/src/modules/voice-recording/recorder.ts                 | 10 ++++++++++
 services/discord-gateway/src/modules/voice-recording/recorder/speakingHandler.ts |  9 ++++++++-
 3 files changed, 42 insertions(+), 1 deletion(-)
 2026-06-14 00:45:31 +07:00
MythEclipse c886dfb184 chore(services): update components based on recent changes 
Changes:
 services/discord-gateway/src/modules/voice-pcm-ws/index.ts            |   1 +
 services/discord-gateway/src/modules/voice-pcm-ws/voicePcmWsClient.ts | 151 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 152 insertions(+)
 2026-06-14 00:43:50 +07:00
MythEclipse 28baeda5ad chore(services): update components based on recent changes 
Changes:
 packages/shared/src/config/index.ts                                          |   7 +++
 packages/shared/src/moderation-types.ts                                      |   1 +
 services/backend/src/modules/recordings/recordings.service.ts                |   3 +-
 services/discord-gateway/drizzle/migrations/0011_add_voice_transcription.sql |   1 +
 services/discord-gateway/src/modules/voice-recording/recorder/segment.ts     |  10 ++++
 services/discord-gateway/src/modules/voice-recording/recorder/uploader.ts    |  15 ++++++
 services/discord-gateway/src/modules/voice-recording/voiceTranscriber.ts     |  51 ++++++++++++++++++
 services/discord-gateway/src/shared/database/schema.ts                       |   1 +
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts           |  17 ++++++
 services/frontend/src/features/live/components/RecordingsSubPanel.tsx        | 108 +++++++++++++++++++-------------------
 2026-06-13 17:08:54 +07:00
MythEclipse d0d2c316cf chore(auto): task completed - unknown 2026-06-13 15:11:06 +07:00
MythEclipse 265251233a chore(auto): task completed - unknown 2026-06-13 15:03:21 +07:00
MythEclipse 9e9ce463cb chore(auto): task completed - unknown 2026-06-13 14:59:00 +07:00
MythEclipseandClaude 3965ea79bc fix(gateway): voice speaking handler, metadata cache, transmitter backpressure 
- speakingHandler: check subscriptions.has() BEFORE voiceActiveUser broadcast
- segment.ts: add LRU metadata cache (200 entries, avoids Discord API in hotpath)
- transmitter.ts: add backpressure handling with drain queue/flush
  (PassThrough.write() return value was ignored — potential OOM under load)
- cleanup backpressure queue on transmitter stop

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 14:38:06 +07:00
MythEclipseandClaude 29d4cb87b6 fix(gateway): voice module critical bugs 
- shutdown ordering: DB pool closed AFTER voice disconnect (was before)
- muxer: amix (mix) -> concat (sequential) filter for OGG segments
- upload retries: 0->3 with proper backoff 1-10s
- teleUpload retry backoff: minTimeout 0->1000, maxTimeout 0->10000
- mediaSource: 64KB buffer bound on readFirstTwoLines, 1MB on extractMediaInfo
- ffmpegProcess: pipe stderr to structured logger instead of inherit

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 14:35:35 +07:00
MythEclipseandClaude 14c0081f01 feat(discord-gateway): implement voice & push improvements 
- Voice disconnect broadcast on stopRecording
- Multi-guild voice support (VoiceController Map<guildId>)
- Session finalization + auto-enqueue muxer job
- Recordings API: duration field, channelId/userId filters
- Transmitter Redis connection reuse (shared conn)
- FFmpeg stderr memory cap (4KB limit)
- 10 new Redis event channels + Redis bridge subscriptions
- New DB tables: message_reactions, message_edits
- Webhook notification module
- Gateway metrics / Prometheus endpoint
- Multi-guild message capture (MONITOR_GUILD_IDS array)
- Thread tracking, presence, channel topic, guild member events
- Edit history snapshot on message update
- Muxer audio post-processing worker

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 13:51:21 +07:00
MythEclipse 111ba0a139 chore(auto): task completed - unknown 2026-06-13 13:36:13 +07:00
MythEclipse 942c33db7e chore(auto): task completed - unknown 2026-06-13 13:35:56 +07:00
MythEclipse eab8505889 chore(auto): task completed - unknown 2026-06-13 13:35:41 +07:00
MythEclipse 2e14b5e4ed chore(auto): task completed - unknown 2026-06-13 13:35:11 +07:00
MythEclipse d569ffa2db chore(auto): task completed - unknown 2026-06-13 13:12:49 +07:00
MythEclipse 4097b23dc3 chore(auto): task completed - unknown 2026-06-13 13:12:41 +07:00
MythEclipse e413cf9fca chore(auto): task completed - unknown 2026-06-13 13:12:29 +07:00
MythEclipse ff006980f8 chore(auto): task completed - unknown 2026-06-13 13:12:01 +07:00
MythEclipse 140a0ba46a chore(auto): task completed - unknown 2026-06-13 13:11:49 +07:00
MythEclipseandClaude 30f8d7cce3 feat: replace corrections/tuner with dashboard module 
Replace the corrections/adaptive-prompt-tuner feature with a new
dashboard module providing server stats and user profile overview.

Backend:
- Add dashboard module (routes, service, repository) with stats + user list + user detail endpoints
- Remove corrections module entirely
- Wire dashboard router in app.ts

Frontend:
- Add dashboard feature (DashboardStats, UserSummaryList, UserProfileDetail components + useDashboard hook)
- Remove tuner feature (CorrectionStats, CorrectionHistory, SubmitCorrection, useCorrections)
- Update API client from corrections → dashboard types/fns
- Rename tab 'tuner' → 'dashboard'
- Update MobileTabBar, Header, Sidebar links

Tests:
- Expand backend placeholder test with dashboard assertions
- Expand discord-gateway placeholder test with config/channel assertions

AI moderation:
- llmModerationClient: improve status/reply detection, expand safety categories, fix timer reset
- userProfileLearner: fix isReply refinement
- userProfileStore: add pending cache check
- messageMetadata: add crosspost type mapping
- migrate.ts: improve partial-index safety in schema push

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 11:24:42 +07:00
MythEclipseandClaude e3249edb6c feat: detect reply/forward/crosspost in message capture + inject into AI moderation 
- Add is_reply, is_forward, is_crosspost, reference_message_id,
  reference_channel_id, reference_guild_id columns to messages table
- Update MessageRecord type with reference fields
- Extract reply/forward/crosspost from Discord message type/flags
- Track reference.type (DEFAULT=reply, FORWARD) and CROSSPOSTED flag
- Inject <reference> XML with parent content into LLM moderation prompt
- Add reply/forward/crosspost rules to moderation prompt rules
- Format context messages with [reply_to], [forward_from], [crosspost]
- Add migration 0009

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 00:55:42 +07:00
MythEclipseandClaude 0ac056dc8a fix(ai-moderation): audit prompts to reduce false positives 
- ENTROPY rule: replace 'PILIH WARN' default with 'PILIH CLEAN'
- Add safe-list for code/log/stack traces/project names
- Narrow sexual_deviation: only flag explicit sexual solicitation
- Remove overbroad ontological graph (kostum hewan → furry) detection
- Add clean few-shot examples: error logs, project names, orientation disclosure
- Update simple fallback prompt with FP prevention rules
- Remove LGBT/furry from offensive username criteria

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-13 00:01:09 +07:00
MythEclipseandClaude 12f9b55000 fix(ai-moderation): implement prompt analysis recommendations 
- Expand IMPHNEN domain rule to cover wildcard (*.imphnen.*)
- Trim redundant SARA examples from TEXT_ONLY_MODE (save ~950 tokens)
- Add debugging logs for channel culture injection into prompt
- Sync flag validation set with missing flags: potential_evasion, unclear_context

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-12 23:40:25 +07:00
MythEclipseandClaude b508a39ff7 fix(ai-moderation): prevent false positive SARA flag for IMPHNEN project name 
- Add explicit system rule that IMPHNEN is the project's own name, not religion
- Rename 'Imphnemia 11:17' example to 'Kitabonia 11:17' to avoid name collision
- Ensures mentioning/promoting the project URL is not flagged as SARA

Co-Authored-By: Claude <noreply@anthropic.com>
 2026-06-12 23:28:09 +07:00