24 Commits
Author SHA1 Message Date
asepharyana ada7a768f8 fix(gateway): bump 0011 voice_transcription journal when above applied max
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m15s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 3m23s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 10m59s
Migration 0011 (add voice_recordings.transcription) when=1781388000000
lebih kecil dari 0010 (1781390000000) yang sudah ter-apply — drizzle
skip diam-diam (folderMillis <= max(created_at)), kolom transcription
tidak pernah dibuat. Recording OGG sukses tapi INSERT voice_recordings
gagal 42703 di produksi.

Fix: when=1785600000000 (> max applied 1785551832190) + apply manual
ALTER TABLE + insert row __drizzle_migrations dengan hash file yang
sama (c368acb0...) supaya gateway restart berikutnya skip (idempotent).
2026-08-01 13:15:03 +07:00
asepharyana 6f41c22cb5 fix(db): migration 0013 rename mascot_chat_messages to chatbot_messages
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 2m17s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m59s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 3m7s
/api/chat/history 500 (relation "chatbot_messages" does not exist):
codebase di-rename mascot->chatbot (977a6f9) tapi tabel DB tidak
pernah di-migrate — backend SELECT dari chatbot_messages, DB masih
mascot_chat_messages dengan kolom mascot_response.

- Migration 0013 (idempotent): ALTER TABLE mascot_chat_messages RENAME
  TO chatbot_messages, RENAME COLUMN mascot_response -> bot_response,
  RENAME INDEX -> idx_chatbot_messages_user_created; journal when >
  max(created_at) di __drizzle_migrations (0012)
- Diterapkan live via psql; riwayat chat lama tetap ada
- Verifikasi: GET /api/chat/history 200 + data, POST /api/chat 200
  + tersimpan (total history 1 -> 2)
2026-08-01 09:38:27 +07:00
Developer 2f298cfe22 fix(db): correct migration 0012 timestamp — drizzle skips when < last applied
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 1m36s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m30s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m33s
Drizzle 0.45 migrate() runs only migrations with folderMillis > the
latest created_at in __drizzle_migrations. Hand-written when was
1781580000000 (June 16) < stored 1781672400000 (June 17), so 0012 was
silently skipped and the embedding column never created. Bumped when to
now; migrator will apply it on next deploy.
2026-07-31 20:22:21 +07:00
Developer 1249ae81d8 perf(automod): compress prompts ~40% + semantic cache via AI_LLM_EMBEDDING_MODEL
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m4s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m29s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m33s
Prompt overhaul (token-frugal, same quality):
- rules.ts 28KB -> 10.3KB: every normative rule kept (safe lists, SARA
  6 kategori, LGBT/Israel zero tolerance, anti-evasion, decision tree,
  evasi hierarchy, image rules) with duplicated phrasing removed
- examples.ts 24.7KB -> 20KB: all 31 teaching examples kept; analysis
  strings shortened, redundant categories/policy_version dropped from
  example outputs (both optional in the response schema)
- output.ts 13.8KB -> 6.8KB: compressed schema + personality + format
  rules; CRITICAL bans on generic analysis and reply-context requirement
  retained
- system.ts: MEDIA_INSTRUCTIONS compressed, key rules kept

Semantic moderation cache (AI_LLM_EMBEDDING_MODEL):
- New embeddingClient.ts: OpenAI-compatible embeddings + cosine
  similarity; degrades gracefully when model/key unset
- textCacheStore: stores embedding JSON per verdict, findSimilarTextModeration
  reuses near-duplicate verdicts (min 0.97 cosine, processing locks skipped)
- moderationOrchestrator: after exact-hash miss, embed text-only targets
  and reuse stored verdict for near-duplicates -> skips expensive chat
  completion for spam variants; fresh verdicts written back with embedding
- Config: AI_LLM_EMBEDDING_MODEL / MIN_SIMILARITY (0.97) / MAX_CANDIDATES (30)
- Migration 0012: ADD COLUMN embedding to text_analysis_cache (idempotent)
- .env.example documents the new vars
2026-07-31 19:37:53 +07:00
asepharyana 7efaf00c93 Revert "feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config"
This reverts commit d59b59a7a7.
2026-07-02 03:54:44 +07:00
asepharyana d59b59a7a7 feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config
Frontend:
- migrate from Vite to Astro (astro.config.mjs, pages/, layouts/)
- add admin panel, settings page, command palette, error boundary
- refactor App.tsx, MascotChatbot, Sidebar, Header, DashboardLayout
- update API client, WebSocket, auth, dashboard features

Backend:
- add admin module and config routes
- refactor middlewares, Redis connection, WebSocket server/bridge
- add runtime config loader

Discord Gateway:
- refactor AI moderation: circuit breaker, concurrency limiter, fallback processor
- add media analysis client, Seaxng search, user profile learner
- add new drizzle migration

Shared:
- extend database schema, add new config fields
2026-07-02 00:02:41 +07:00
MythEclipseandClaude 6733ea8f01 fix(db): add 0011_add_voice_transcription to drizzle journal
Migration SQL file existed but was never registered in _journal.json,
so Drizzle skipped it. Every voice recording insert failed with:
  PG code 42703: column "transcription" does not exist

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-14 04:59:44 +07:00
MythEclipse 28baeda5ad chore(services): update components based on recent changes
Changes:
 packages/shared/src/config/index.ts                                          |   7 +++
 packages/shared/src/moderation-types.ts                                      |   1 +
 services/backend/src/modules/recordings/recordings.service.ts                |   3 +-
 services/discord-gateway/drizzle/migrations/0011_add_voice_transcription.sql |   1 +
 services/discord-gateway/src/modules/voice-recording/recorder/segment.ts     |  10 ++++
 services/discord-gateway/src/modules/voice-recording/recorder/uploader.ts    |  15 ++++++
 services/discord-gateway/src/modules/voice-recording/voiceTranscriber.ts     |  51 ++++++++++++++++++
 services/discord-gateway/src/shared/database/schema.ts                       |   1 +
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts           |  17 ++++++
 services/frontend/src/features/live/components/RecordingsSubPanel.tsx        | 108 +++++++++++++++++++-------------------
2026-06-13 17:08:54 +07:00
MythEclipse 2e14b5e4ed chore(auto): task completed - unknown 2026-06-13 13:35:11 +07:00
MythEclipseandClaude e3249edb6c feat: detect reply/forward/crosspost in message capture + inject into AI moderation
- Add is_reply, is_forward, is_crosspost, reference_message_id,
  reference_channel_id, reference_guild_id columns to messages table
- Update MessageRecord type with reference fields
- Extract reply/forward/crosspost from Discord message type/flags
- Track reference.type (DEFAULT=reply, FORWARD) and CROSSPOSTED flag
- Inject <reference> XML with parent content into LLM moderation prompt
- Add reply/forward/crosspost rules to moderation prompt rules
- Format context messages with [reply_to], [forward_from], [crosspost]
- Add migration 0009

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-13 00:55:42 +07:00
MythEclipseandClaude fbc2184c6e feat(ai-moderation): add user profile self-learning system
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.

- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
  calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 20:11:34 +07:00
MythEclipseandClaude 33b787dbd6 fix(migration): make all DDL idempotent with IF NOT EXISTS
- 0000: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS,
  FK constraint wrapped in DO 539994 EXCEPTION WHEN duplicate_object
- 0001: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION WHEN duplicate_column
- 0002: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0003: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0006: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION,
  DROP COLUMN IF EXISTS

Allows migrations to run cleanly regardless of DB state —
handles fresh DB, partial migration, or wiped __drizzle_migrations.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 18:05:09 +07:00
MythEclipseandClaude Opus 4.8 3614d32701 fix: resolve architecture disconnects and codebase weaknesses
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 13:07:01 +07:00
MythEclipse 6a3d0eb17c feat(discord-gateway): replace base64 sticker cache column with image URL storage 2026-06-06 14:10:55 +07:00
MythEclipse c0a067322b refactor(ai-moderation): enhance llmClient flexibility and idempotency
Update the LLM client to support streaming responses and improve
database migration idempotency.

- Add `stream` support to `llmChat` with dynamic chunk parsing for
  compatibility across different LLM providers.
- Remove hardcoded default temperature and top_p to allow for more
  flexible parameter passing.
- Update database migrations to use `IF NOT EXISTS` for tables and
  indexes to prevent errors during re-runs.
2026-06-05 18:45:58 +07:00
MythEclipse 42ea2924ac feat(db): add migrations for user reputation and channel culture tables
Adds new database schema migrations to support the implementation of
user reputation tracking and channel-specific culture context.

- Creates migration `0005_large_squadron_sinister.sql` containing
  `user_reputations` and `channel_cultures` tables.
- Updates Drizzle migration journal to include the new migration step.
- Adds `fix_prompt.py` utility for managing prompt adjustments.
2026-06-05 18:21:52 +07:00
MythEclipse d81aaf2b4a feat(db): add migration to drop unused AI columns and dynamically resolve initial migration tag 2026-06-02 23:55:12 +07:00
MythEclipse 88acda4703 feat(db): add migration snapshot for corrected_moderations table 2026-06-02 23:44:29 +07:00
MythEclipseandClaude Opus 4.8 8ff547af43 fix(db): add migration for corrected_moderations table
Table was defined in Drizzle schema at schema.ts:425 but no migration
existed. Runtime queries from text-cache-store fail with:
  relation "corrected_moderations" does not exist

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 23:43:40 +07:00
MythEclipse e0d54b7770 chore: auto-commit task - 2026-06-02 17:33:19 +07:00
MythEclipse 498ab73d62 chore: auto-commit task - 2026-06-02 17:08:42 +07:00
MythEclipseandClaude Opus 4.8 e584a7941e fix(text-cache): apply model version migration on existing dbs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 13:05:05 +07:00
MythEclipseandClaude Opus 4.8 682fb31deb fix(text-cache): add missing model_version column to migration + strict git detection
- Create new migration 0001_add_model_version_to_cache.sql to add model_version column to text_analysis_cache table (default 'v1')
- Add composite index on source + model_version for efficient cache lookups
- Improve git branch detection in textCacheStore.ts:
  * Check CACHE_MODEL_VERSION env var first (required in Docker)
  * Suppress stderr with stdio pipes to eliminate spurious warnings
  * Log ERROR (not warning) when git unavailable AND no env var set
  * Only fall back to 'v1' as last resort, preventing silent dummy usage
- Remove unused imports (logModelVersionChange, logCacheInvalidation) and previousVersion variable
- Fixes repeated 'column model_version does not exist' errors on bot startup

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 12:46:37 +07:00
MythEclipseandClaude Opus 4.6 e2c1197caf fix(discord-gateway): generate Drizzle migrations and copy to Docker image
Container crashed with 'Can't find meta/_journal.json file' because
Drizzle ORM migration files were never generated. Generated migrations
for all 10 schema tables and updated Dockerfile to copy drizzle/ to
/app/drizzle/ where the migrator expects them.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-01 22:52:03 +07:00