Commit Graph
38 Commits
Author SHA1 Message Date
MythEclipse 28baeda5ad chore(services): update components based on recent changes
Changes:
 packages/shared/src/config/index.ts                                          |   7 +++
 packages/shared/src/moderation-types.ts                                      |   1 +
 services/backend/src/modules/recordings/recordings.service.ts                |   3 +-
 services/discord-gateway/drizzle/migrations/0011_add_voice_transcription.sql |   1 +
 services/discord-gateway/src/modules/voice-recording/recorder/segment.ts     |  10 ++++
 services/discord-gateway/src/modules/voice-recording/recorder/uploader.ts    |  15 ++++++
 services/discord-gateway/src/modules/voice-recording/voiceTranscriber.ts     |  51 ++++++++++++++++++
 services/discord-gateway/src/shared/database/schema.ts                       |   1 +
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts           |  17 ++++++
 services/frontend/src/features/live/components/RecordingsSubPanel.tsx        | 108 +++++++++++++++++++-------------------
2026-06-13 17:08:54 +07:00
MythEclipse 2e14b5e4ed chore(auto): task completed - unknown 2026-06-13 13:35:11 +07:00
MythEclipseandClaude 30f8d7cce3 feat: replace corrections/tuner with dashboard module
Replace the corrections/adaptive-prompt-tuner feature with a new
dashboard module providing server stats and user profile overview.

Backend:
- Add dashboard module (routes, service, repository) with stats + user list + user detail endpoints
- Remove corrections module entirely
- Wire dashboard router in app.ts

Frontend:
- Add dashboard feature (DashboardStats, UserSummaryList, UserProfileDetail components + useDashboard hook)
- Remove tuner feature (CorrectionStats, CorrectionHistory, SubmitCorrection, useCorrections)
- Update API client from corrections → dashboard types/fns
- Rename tab 'tuner' → 'dashboard'
- Update MobileTabBar, Header, Sidebar links

Tests:
- Expand backend placeholder test with dashboard assertions
- Expand discord-gateway placeholder test with config/channel assertions

AI moderation:
- llmModerationClient: improve status/reply detection, expand safety categories, fix timer reset
- userProfileLearner: fix isReply refinement
- userProfileStore: add pending cache check
- messageMetadata: add crosspost type mapping
- migrate.ts: improve partial-index safety in schema push

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 11:24:42 +07:00
MythEclipseandClaude fbc2184c6e feat(ai-moderation): add user profile self-learning system
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.

- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
  calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 20:11:34 +07:00
MythEclipseandClaude Fable 5 906aef1807 fix(database): handle PG15+ public schema CREATE privilege error in seedDrizzleHistory
In PG15+, the CREATE privilege on the public schema is revoked from
non-owner roles by default. The seedDrizzleHistory function's
CREATE TABLE IF NOT EXISTS for __drizzle_migrations fails with 42501,
causing the gateway to crash-loop on startup.

Wrap the CREATE in a try/catch for 42501 — if the table already exists
(created by a prior run), we continue gracefully; otherwise re-throw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 04:06:11 +07:00
MythEclipse 2e5371f3ee fix(discord-gateway): bypass Drizzle PG15 schema creation restrictions 2026-06-11 03:53:48 +07:00
MythEclipseandClaude Fable 5 378f1b6bdc fix(database): use public schema for drizzle migrations to avoid permission error
PostgreSQL error 42501 when Drizzle tries CREATE SCHEMA IF NOT EXISTS
"drizzle" — the DB user lacks schema creation privileges. Configuring
migrationsSchema: "public" keeps __drizzle_migrations in the existing
public schema, matching what seedDrizzleHistory already expects.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 03:34:16 +07:00
MythEclipseandClaude Opus 4.8 07032ab521 refactor: atomic, DRY, and logging improvements across codebase
- Split llmModerationClient.ts (2170 lines) into 5 focused sub-modules
- Split aiAnalyzer.ts (1282 lines) into 4 modular pipelines
- Split messages.db.ts (826 lines) into 5 domain-specific modules
- Moved shared schema to @bete/shared, eliminated backend duplication
- Added createChildLogger to all voice-recording and AI moderation modules
- Extracted tryCommandThenFallback, normalizeMediaState, DEFAULT_VOICE_STATUS
- Created shared pagination.ts utility, eliminated 5+ cursor-pagination duplications
- Created shared messageMapper.ts for row mapping
- Standardized backend error handling with asyncHandler
- Added frontend createLogger utility and useAsyncAction hook
- Added structured logging to frontend hooks, socket, and API client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 19:46:08 +07:00
MythEclipseandClaude Opus 4.8 b68789fffc refactor: atomic, DRY, and logging improvements
- Shared Redis channel constants as single source of truth (redis-channels.ts)
- commandHandler.ts split into VoiceHandler, MediaHandler, GuildHandler,
  ModerationHandler with handler-registry.ts dispatch
- messageStore.ts (1322 lines) split into domain-specific DB files:
  messages.db.ts, attachments.db.ts, reviews.db.ts,
  moderation-actions.db.ts, retention.db.ts
- recorder.ts startSpeaking callback extracted into speakingHandler.ts,
  streamSetup.ts, segmentFinalizer.ts
- autoDeleteManager.ts split into autoDeleteEligibility.ts,
  autoDeleteNotify.ts, autoDeleteLogger.ts
- Added createChildLogger() logging across 8 service files
- Backend messages.repository.ts migrated from raw SQL to Drizzle ORM
- Fixed biome.json to exclude packages/**/dist/* from lint
- Fixed config.ts GUILD_ID pre-existing type error

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:34:18 +07:00
MythEclipseandClaude Opus 4.8 3a7b005d95 fix: resolve architecture disconnects and codebase weaknesses
- Add TEXT_CHANNEL_ID and TEXT_GUILD_ID to config schema (fix silent channel monitoring)
- Remove dead files: message-capture/broadcaster.ts, voice-recording/index.ts
- Fix WebSocket voice_command payload to forward from frontend
- Implement moderation:action handler in commandHandler
- Fix useMascotChat to use canonical request() wrapper
- Fix useAudioPlayback userId hash collision (use string not parseInt)
- Add catch blocks to useMediaControl.skip/stop
- Add typed broadcast functions (messageAnalyzed, voicePcmData, voiceActiveUser)
- Apply Biome formatting and lint fixes

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 16:36:23 +07:00
MythEclipseandClaude Opus 4.8 3614d32701 fix: resolve architecture disconnects and codebase weaknesses
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 13:07:01 +07:00
MythEclipse 67d66bb5dd refactor(shared): centralize configuration and types
Migrate configuration validation and core moderation types from individual services to the `@bete/shared` package to ensure consistency across the monorepo.

- Move `AppConfig` and moderation-related interfaces to `packages/shared`.
- Replace service-specific Zod schemas with the centralized shared configuration.
- Refactor `services/backend` and `services/discord-gateway` to consume shared config and types.
- Remove redundant type definitions and local configuration logic in services.
- Update `packages/shared` exports to include new `config` and `moderation-types` modules.
- Clean up unused files and deprecated utility functions in `packages/shared`.
2026-06-09 11:56:03 +07:00
MythEclipseandClaude Opus 4.8 4becf0d6f1 refactor: comprehensive codebase cleanup and architecture hardening
- Sprint 1 (Quick Wins): Remove dead analytics modules, fix 4 unresolved
  imports, replace 3 console.warn with logger, remove mock-crc import
- Sprint 2 (Architecture): Create MascotChatRepository, AnalysisRepository,
  3 Zod schemas (mascot-chat, analysis, voice), deduplicate error classes,
  move 3 SQL queries from routes to repository
- Sprint 3 (Complexity): Replace 7 any types with proper interfaces,
  extract 6 helpers from prepareMediaMessage (CC 85 -> ~15)
- Sprint 4 (Config): Remove 22 dead env vars from .env, add 30 missing
  vars to .env.example, standardize naming

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 10:16:04 +07:00
MythEclipseandClaude Opus 4.8 13a75a5101 fix(voice): restore voice features and apply critical optimizations
Voice Feature Restoration:
- Implemented full Redis pub/sub pipeline for real-time voice data
- Added VOICE_PCM and VOICE_ACTIVE_USER Redis channels
- Implemented EventBroadcaster.voicePcmData() and voiceActiveUser() methods
- Extended backend redis-bridge to subscribe to voice channels
- Updated backend WebSocket server for binary PCM broadcast
- Replaced globalThis PcmBroadcaster pattern with proper EventBroadcaster DI
- Fixed root cause: PcmBroadcaster functions were never initialized

Bug Fixes:
- Fixed prism-media version conflict (2.0.0-alpha.0 → 1.3.5)
- Fixed type inconsistency in commandHandler.ts (AudioPlayerStatus → string)

Critical Optimizations:
- P1.1: Fixed unbounded memory growth in aiAnalyzer (added LRU caching, max 10K entries)
- P1.2: Converted sync file I/O to async in audio hot paths (recorder, sessionRecording)
- P1.3: Replaced process.exit(1) with proper error handling (bootstrap, aiAnalysisWorker)

Code Quality:
- Removed unused logger field in EventBroadcaster
- Replaced console.* with structured logger.* calls (player, decoder)
- Fixed typos and removed commented debug code
- Added DatabaseError class for better error handling

Files Modified: 18 (discord-gateway: 14, backend: 3, root: 1)
Architecture: Discord → EventBroadcaster → Redis → Backend WebSocket → Frontend

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 19:14:34 +07:00
MythEclipse 2643a3c278 refactor(discord-gateway): optimize redis usage and audio stream handling
- refactor(command-handler): replace per-call Redis connection creation with a persistent publisher connection to reduce overhead
- refactor(recorder): switch from manual audio stream subscription to direct event listeners on the existing stream
- feat(recorder): implement exponential backoff for voice connection retries
- chore(config): update default DECODER_COOLDOWN_MS to 30000ms
2026-06-08 18:09:17 +07:00
MythEclipse 71a8a9e6e1 fix(ai-moderation): patch structural bypass vulnerabilities in NLP pipeline
- Implement Pre-computation Normalization for Polyglot Obfuscation.

- Inject Ontological Graph for literal translation evasion (e.g., 'kostum hewan').

- Enforce Entropy-Triggered Routing to deny softmax fallback exploitation.

- Format discord-gateway codebase.
2026-06-06 15:38:36 +07:00
MythEclipse 6a3d0eb17c feat(discord-gateway): replace base64 sticker cache column with image URL storage 2026-06-06 14:10:55 +07:00
MythEclipse 25c86dcc30 fix(ai-moderation): raise retry backoffs and cooldowns, make 429 retryable, reduce prompt false positives 2026-06-05 21:47:02 +07:00
MythEclipse 2f3d7e1d61 feat(ai-moderation): introduce user reputation and channel culture context
Implements a context-aware moderation system by tracking user behavior
and channel-specific norms to improve AI decision-making accuracy.

- Adds `user_reputations` table to track trust scores, clean streaks,
  and infraction history.
- Adds `channel_cultures` table to store AI-generated summaries of
  channel-specific norms and slang.
- Implements `userReputationStore` to autonomously update user scores
  based on moderation outcomes (clean vs. flagged).
- Implements `cultureLearner` and `channelCultureStore` to manage
  evolving channel contexts.
- Enhances LLM prompts to inject user reputation (trust scores,
  history) and channel culture summaries, enabling "wisdom-based"
  moderation (e.g., giving benefit of the doubt to high-trust users).
- Integrates reputation and culture updates into the existing
  `aiAnalyzer` pipeline.
2026-06-05 18:04:57 +07:00
MythEclipse 09f6e80ddd fix(ai-moderation): resolve race conditions and implement processing state
Introduces a `processing` state to the AI analysis lifecycle to prevent
duplicate processing of the same messages.

- Implements row-level locking using `FOR UPDATE SKIP LOCKED` in
  `messageStore.ts` to ensure atomic message acquisition.
- Adds a `processing` status to the `AIStatus` type and database schema.
- Fixes a TOCTOU race condition in `aiAnalyzer.ts` by synchronizing
  the conversation processing lock before async database operations.
- Implements `revertStuckProcessingMessages` to recover messages stuck
  in the `processing` state due to worker crashes or timeouts.
- Updates `processBatch` and scheduling logic to correctly manage and
  release conversation-level locks.
2026-06-05 16:36:14 +07:00
MythEclipse d1d4510ce8 chore(format): auto-format files with biome 2026-06-04 18:52:56 +07:00
MythEclipse ce8a42f5fd feat(ai-moderation): eliminate LLM-based badword detection in favor of pure rule-based matching 2026-06-04 17:33:08 +07:00
MythEclipse 5d9713b162 chore(config): update TELE_UPLOAD_URL domain across all services 2026-06-04 17:12:02 +07:00
MythEclipse d81aaf2b4a feat(db): add migration to drop unused AI columns and dynamically resolve initial migration tag 2026-06-02 23:55:12 +07:00
MythEclipse f10fa9aa88 feat(discord-gateway): export correctedModerationsTable from database schema 2026-06-02 23:05:37 +07:00
MythEclipse eb16c1f504 ``` 2026-06-02 23:05:33 +07:00
MythEclipse 0cd4f197e9 No staged changes found — the staging area is empty. Stage some files first with git add, then I can help craft the commit message. 2026-06-02 23:05:06 +07:00
MythEclipse f163ead3cf feat(discord-gateway): add local badword pre-filter, accurate token estimation, Piscina maxThreads config, and PromptMode support 2026-06-02 22:50:23 +07:00
MythEclipse 1e1fbb4583 fix(discord-gateway): require LLM API key only when AI analysis is enabled 2026-06-02 21:46:15 +07:00
MythEclipse 19028cf244 refactor: migrate discord-gateway from winston to @bete/shared/logger and utils
- Replace all local logger imports (../../shared/logger/logger.js) with @bete/shared/logger across 26 files
- Remove winston dependency, add pino to discord-gateway package.json
- Delete shared/logger/logger.ts (winston-based, 132 lines) and serialization.ts (109 lines)
- Replace local retryWithBackoff imports with @bete/shared/utils across 6 files
- Delete shared/utils/retry.ts (42 lines)
- Add CustomLogger type alias to @bete/shared/logger for backwards compatibility
- Remove logger param from all retryWithBackoff calls and uploadToTele interfaces
- Frontend: convert entity type files to re-exports from shared/api/client.ts
- Full monorepo typecheck clean (4/4 packages)

35 files changed, 42 insertions(+), 488 deletions(-)
2026-06-02 21:23:12 +07:00
MythEclipse b23ccf0837 chore: session auto-commit 2026-06-02 17:12:42 +07:00
MythEclipse 498ab73d62 chore: auto-commit task - 2026-06-02 17:08:42 +07:00
MythEclipse 406a6cbf79 refactor: migrate sticker cache to PostgreSQL, remove versioning
- Replace file-based sticker cache (.dat + index.json) with PostgreSQL sticker_cache table
- Remove model_version column and all versioning logic (git branch detection, GitHub API, CACHE_MODEL_VERSION)
- Strip VISION_MODEL_VERSION, logCacheEvent calls, and version filtering from SQL queries
- Remove STICKER_CACHE_DIR and STICKER_CACHE_MAX_SIZE_MB config variables
- Delete dead migration add_model_version_to_cache.sql

textCacheStore.ts reduced 482→204 lines (-57%)
stickerCache.ts reduced 210→144 lines (-31%)
Total: 11 files changed, 233 insertions, 591 deletions
2026-06-02 14:05:51 +07:00
MythEclipseandClaude Opus 4.8 e584a7941e fix(text-cache): apply model version migration on existing dbs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 13:05:05 +07:00
MythEclipse 385fc0bbbf feat: add response logger for moderation/vision audit trail + cache model versioning
- Add responseLogger.ts with comprehensive logging: vision analysis, moderation
  analysis batches, cache hit/miss events, errors, retries, false positives,
  model version changes, cache invalidation
- Integrate responseLogger calls into llmModerationClient.ts (cache events,
  moderation results, errors, retries)
- Add model_version column to text_analysis_cache schema with composite
  index for efficient filtering
- Add migration SQL for model_version column
- Document CACHE_MODEL_VERSION in .env.example
2026-06-02 12:23:52 +07:00
MythEclipseandClaude Opus 4.8 64643bb066 perf: hapus semua cooldown/backoff delay — retry tanpa bottleneck
Semua retry sekarang instant (retries=0, minTimeout=0, maxTimeout=0):
- retry.ts (kedua) — default minTimeout/maxTimeout 0, factor 1
- config.ts (kedua) — DECODER_COOLDOWN_MS=0, AI_ANALYSIS_ERROR_COOLDOWN_MS=0
- llmModerationClient.ts (kedua) — retries 0 tanpa delay
- aiAnalyzer.ts (kedua) — retries 0, AI_ANALYSIS_ERROR_COOLDOWN jadi 0
- indonesianTextNormalizer.ts (kedua) — semua rate limit cooldown 0
- recorder.ts (kedua), teleUpload (3 files), uploader (2 files) — retry instant
- attachmentUploader (kedua) — retries 0
- syncRoutes.ts — BACKLOG_SYNC_COOLDOWN 0

20 files changed. Semua cooldown bottleneck dihapus.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 11:29:52 +07:00
MythEclipseandClaude Opus 4.6 58afa98132 fix(discord-gateway): skip migration if schema tables already exist
Instead of failing on CREATE TABLE when tables already exist, check
information_schema for all 10 schema tables. If all exist, skip
migration gracefully. This handles deployments where DB was created
by a previous deployment with different migration files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-01 23:09:04 +07:00
MythEclipseandClaude Opus 4.8 c48a0c5e3b refactor: split monolith into 3 microservices (frontend, backend, discord-gateway)
- Extract services into services/{frontend,backend,discord-gateway}
- Create packages/shared/ for shared logger, errors, utils, types
- Setup Modular MVC pattern in backend (controller→service→repository)
- Setup event-driven architecture in discord-gateway with Redis pub/sub
- Move Docker files to infra/docker/ with per-service Dockerfiles
- Update docker-compose.yml to use Traefik-only routing (no port exposes)
- Update GitHub Actions deploy workflow for multi-service matrix build
- Fix all import paths and resolve type errors across all services
- All 3 services pass tsc --noEmit clean

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:29 +07:00