Drizzle wraps PG errors — the previous log only showed the formatted query
text. Now extracts code, detail, schema, table, constraint, severity from
the error envelope and from .cause to make the actual Postgres error visible.
Co-Authored-By: Claude <noreply@anthropic.com>
Redis root cause: VoiceHandler passed redisPub to transmitter.start(),
which called .subscribe() on it — permanently converting the publish
connection to subscriber mode. Every subsequent command reply and status
update failed.
Fixes:
1. Transmitter now creates its own Redis client via new IORedis()
2. Mutual exclusion gate serialises start/stop to prevent null-deref races
3. PassThrough drain listeners cleaned up to stop MaxListenersExceeded
4. FFmpeg SIGTERM flagged as expected exit (no more false level-50 errors)
5. Voice recording repo captures PG error code/detail for diagnostics
Co-Authored-By: Claude <noreply@anthropic.com>
Default tab is now 'messages'; Voice & Media moved below
Messages in both desktop sidebar and mobile tab bar.
Updated Header titles/subtitles to match new order.
Co-Authored-By: Claude <noreply@anthropic.com>
- Expand IMPHNEN domain rule to cover wildcard (*.imphnen.*)
- Trim redundant SARA examples from TEXT_ONLY_MODE (save ~950 tokens)
- Add debugging logs for channel culture injection into prompt
- Sync flag validation set with missing flags: potential_evasion, unclear_context
Co-Authored-By: Claude <noreply@anthropic.com>
- Add explicit system rule that IMPHNEN is the project's own name, not religion
- Rename 'Imphnemia 11:17' example to 'Kitabonia 11:17' to avoid name collision
- Ensures mentioning/promoting the project URL is not flagged as SARA
Co-Authored-By: Claude <noreply@anthropic.com>
- Add pgCorrectedModerationsTable to shared schema
- Create backend corrections module (stats, list, create endpoints)
- Add TunerPanel with Stats, History, and Submit sub-tabs
- Add AuthOverlay gate for Tuner (admin-only, same as Live)
- Set messages as default tab
- Wire Tuner into sidebar, header, and mobile tab bar
Co-Authored-By: Claude <noreply@anthropic.com>
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.
- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table
Co-Authored-By: Claude <noreply@anthropic.com>
- hasMediaContent now also checks evidence.attachments from metadata
(not just DB attachment records), catching the race where attachment
DB rows aren't inserted yet when analysis runs.
- Cache-hit guard: treat cached entries as miss when the message has
media evidence in metadata, so stale 24h-freezes are avoided.
- Cache-write guard: skip storing text-only analysis results for
messages whose metadata shows attachments/stickers/embeds. This
prevents a text-only 'clean' result (from failed vision) being
frozen for 24h, blocking future re-analysis with full media context.
Co-Authored-By: Claude <noreply@anthropic.com>
- 0000: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS,
FK constraint wrapped in DO 539994 EXCEPTION WHEN duplicate_object
- 0001: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION WHEN duplicate_column
- 0002: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0003: CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS
- 0006: ALTER ADD COLUMN wrapped in DO 539994 EXCEPTION,
DROP COLUMN IF EXISTS
Allows migrations to run cleanly regardless of DB state —
handles fresh DB, partial migration, or wiped __drizzle_migrations.
Co-Authored-By: Claude <noreply@anthropic.com>
- Resolve parent channel ID for threads before checking
EXCLUDED_CHANNEL_IDS set
- Thread messages now blocked if their parent channel is excluded
Co-Authored-By: Claude <noreply@anthropic.com>
- Move isAgeRestrictedMessage to messageMetadata.ts alongside
isAgeRestrictedMetadata — single source of truth for NSFW logic
- Replace || chain of channel IDs with a Set for O(1) lookup
- Import isAgeRestrictedMessage in capture layer instead of inline
Co-Authored-By: Claude <noreply@anthropic.com>
Auto-filter all age-restricted and NSFW channels at capture layer
so messages never enter the database or reach frontend.
Co-Authored-By: Claude <noreply@anthropic.com>
Add hardcoded channel ID to shouldCaptureMessageLocation so messages
from this channel are not stored, analyzed by AI, or sent to frontend.
Co-Authored-By: Claude <noreply@anthropic.com>
In PG15+, the CREATE privilege on the public schema is revoked from
non-owner roles by default. The seedDrizzleHistory function's
CREATE TABLE IF NOT EXISTS for __drizzle_migrations fails with 42501,
causing the gateway to crash-loop on startup.
Wrap the CREATE in a try/catch for 42501 — if the table already exists
(created by a prior run), we continue gracefully; otherwise re-throw.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@discordjs/opus requires Python and C++ build tools for native
compilation. The discord-gateway Dockerfile already had these but
the backend Dockerfile was missing them, causing CI build failure.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PostgreSQL error 42501 when Drizzle tries CREATE SCHEMA IF NOT EXISTS
"drizzle" — the DB user lacks schema creation privileges. Configuring
migrationsSchema: "public" keeps __drizzle_migrations in the existing
public schema, matching what seedDrizzleHistory already expects.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Update the moderation prompt to include high-priority detection categories for:
- Fake scripture/verse parodies
- Claims of divinity or false religious movements
- Misuse of theological terms as internet slang/memes
- Mockery of religious figures and rituals
This change ensures stricter enforcement of SARA (Suku, Agama, Ras, Antargolongan) policies by explicitly defining religious blasphemy and parody as high-severity violations.
This commit introduces several significant improvements across the backend and gateway services:
- **Data Retention**: Added an automated cleanup scheduler in `discord-gateway` to prune expired messages, attachments, and voice recordings based on configurable retention policies.
- **Observability**: Integrated `prom-client` in the `backend` service to expose Prometheus metrics via `/api/metrics` and added default Node.js runtime metrics.
- **Media Handling**: Enhanced `MediaHandler` in `discord-gateway` to support media URL resolution and improved playback status tracking.
- **API & Config**: Expanded the configuration endpoint to expose more system settings and reorganized `.env.example` for better readability.
- **Refactoring & Cleanup**:
- Removed unused `better-sqlite3` dependency.
- Refactored voice channel routing.
- Improved error handling and testing coverage with comprehensive unit tests for shared utilities and error classes.
- **Documentation**: Added `MEMORY.md` for project context.
Refactor the voice recording and playback systems to improve efficiency, reduce latency, and enhance Docker build performance.
- **Infrastructure**: Optimize Dockerfiles using build mounts for pnpm cache and reorder layers for better caching of dependencies and build tools.
- **Backend/Gateway**:
- Refactor `broadcast` module to use a generic event-based system instead of hardcoded functions.
- Simplify voice recording logic by merging metadata and segment management into a unified `segment.ts`.
- Optimize audio downsampling in `streamSetup.ts` using `Int16Array` views for better performance.
- Implement `withFallback` utility for more robust Redis/Database command execution.
- **Frontend**:
- Optimize audio playback visualization using pre-computed level shapes and efficient RMS calculation.
- Reduce latency in voice commands by prioritizing WebSocket communication over HTTP.
- Improve base64 encoding efficiency in audio transmission.
- **General**:
- Add default value for `ADMIN_PASSWORD` in shared config.
- Fix Docker healthcheck to use `127.0.0.1` instead of `localhost`.
The --wait flag causes the deploy to abort when a container exits
immediately (e.g., due to transient DB connection issue on startup).
With restart: unless-stopped, containers will auto-recover without
blocking the deployment pipeline.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GitHub Container Registry pulls occasionally fail with 'connection reset
by peer'. Adding a 3-attempt retry loop with 5s backoff between pulls
to handle transient network errors gracefully.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
TypeScript compilation in the frontend Docker build fails with TS2307:
Cannot find module '@bete/shared' because the shared package's .d.ts
files in dist/ were never generated. The backend and discord-gateway
Dockerfiles already have this build step — frontend was missing it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Shared Redis channel constants as single source of truth (redis-channels.ts)
- commandHandler.ts split into VoiceHandler, MediaHandler, GuildHandler,
ModerationHandler with handler-registry.ts dispatch
- messageStore.ts (1322 lines) split into domain-specific DB files:
messages.db.ts, attachments.db.ts, reviews.db.ts,
moderation-actions.db.ts, retention.db.ts
- recorder.ts startSpeaking callback extracted into speakingHandler.ts,
streamSetup.ts, segmentFinalizer.ts
- autoDeleteManager.ts split into autoDeleteEligibility.ts,
autoDeleteNotify.ts, autoDeleteLogger.ts
- Added createChildLogger() logging across 8 service files
- Backend messages.repository.ts migrated from raw SQL to Drizzle ORM
- Fixed biome.json to exclude packages/**/dist/* from lint
- Fixed config.ts GUILD_ID pre-existing type error
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Improve system reliability and real-time capabilities by implementing a robust lifecycle management system and adding new broadcast events for attachments and voice recordings.
- Implement asynchronous graceful shutdown in backend to close HTTP, WebSocket, Redis, and database connections.
- Add new WebSocket broadcast events: `attachment_created`, `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`, and `analysis_queue_status`.
- Refactor media status handling to use boolean `playing` state instead of string-based status.
- Centralize `PageResult` and `VoiceRecording` types to improve consistency between frontend and backend.
- Update frontend API client to include `listRecordings` and handle new WebSocket event types.
- Fix type mismatches in voice command handling and media status reporting.
Prevents vitest from exiting with code 1 when no test files exist.
Adds minimal test files to backend and discord-gateway services.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Consolidate eventTypes.ts as single source of truth for Redis channels:
- Remove duplicate DiscordGatewayEvent interface from eventBroadcaster.ts
- Replace all hardcoded channel strings with EventChannels constants
- eventTypes.ts is no longer an orphan file
- Remove dangerous moderation action feature (selfbot safety):
- Remove /messages/:id/moderate endpoint from backend
- Remove moderation:action handler from commandHandler.ts
- Remove publishFireAndForgetCommand (wrong envelope format)
- Verified no remaining references to moderation:action in code
- Remove unused getCommandPublisher import from redis-bridge.ts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Migrate configuration validation and core moderation types from individual services to the `@bete/shared` package to ensure consistency across the monorepo.
- Move `AppConfig` and moderation-related interfaces to `packages/shared`.
- Replace service-specific Zod schemas with the centralized shared configuration.
- Refactor `services/backend` and `services/discord-gateway` to consume shared config and types.
- Remove redundant type definitions and local configuration logic in services.
- Update `packages/shared` exports to include new `config` and `moderation-types` modules.
- Clean up unused files and deprecated utility functions in `packages/shared`.
- fix(backend): replace raw .parse() with proper loadConfig() + ConfigError
- fix(gateway): connect voice recording uploader to EventBroadcaster
- fix(gateway): remove dead globalThis.moderationBroadcaster path in AI analyzer
- fix(gateway): eliminate audioStream race condition by attaching handlers before pipe
- fix(gateway): enable inlineVolume by default for setMusicVolume to work
- fix(gateway): reuse persistent redisPub for command replies (no new connection per cmd)
- fix(frontend): add missing voice_active_user/voice_pcm_data to WsEventMap
- fix(frontend): correct onAttachmentUploaded handler signature to accept data
- chore: move @types/pg from dependencies to devDependencies
- chore: translate remaining Indonesian comments to English
- chore: remove stale P3 TODO comment
- Rollback frontend latency changes (keep processor.connect(destination))
- Rollback FFmpeg aggressive optimizations (keep 20ms frames)
- Transmit was working before latency optimization; restore stable state
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route was POST /api/command (mounted at /api + /command)
Fixed to POST /api/voice/command (mounted at /api + /voice/command)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Send voice:transmit:start/stop via POST /api/voice/command
instead of WebSocket. The WebSocket approach failed because:
1. Separate WS connections weren't handled by backend
2. HTTP fallback wasn't working correctly
Now uses HTTP API directly which is more reliable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add POST /api/voice/command endpoint in backend
- Frontend transmit now sends commands via dedicated WebSocket connection
with HTTP API fallback if WebSocket fails
- Fixes issue where transmit start command was never received by discord-gateway
Previously commands were sent via the existing dashboard WebSocket,
which may not be connected when the Transmit button is pressed.
Now each command opens a fresh WebSocket connection with HTTP fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Voice transmitter requires FFmpeg to encode PCM to OggOpus.
Previously missing from Alpine image, causing:
spawn ffmpeg ENOENT
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Replace prism Opus.Encoder + OggLogicalBitstream with FFmpeg
- FFmpeg handles upsampling, encoding, and OGG container in one process
- Input: raw PCM 24kHz mono s16le via stdin
- Output: OggOpus via stdout (StreamType.OggOpus)
- FFmpeg arguments optimized for real-time low-delay streaming
Previous approach failed because:
1. Raw Opus packets without OGG wrapper don't work with @discordjs/voice
2. prism's OggLogicalBitstream has CRC bug with node-crc native bindings
3. Manual upsampling was error-prone
Pipeline:
Browser Mic → base64 PCM → Redis → FFmpeg → OggOpus → Discord
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Opus.Encoder in prism-media outputs raw Opus packets directly,
no OGG wrapping. OggDemuxer was breaking the stream by
trying to unwrap a non-existent OGG container.
Pipeline was: PCM → Encoder → OggDemuxer → Discord (broken)
Pipeline now: PCM → Encoder → Discord (correct)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add onVoicePcmData and onVoiceActiveUser handlers to WebSocket interface
- Add voice_pcm_data and voice_active_user event cases in socket message handler
- Update useAudioPlayback to decode base64 PCM from JSON format
- Connect onVoicePcmData to audio.handleIncomingPcm in App.tsx
Format change:
- Old: Binary (4 bytes userId + PCM buffer)
- New: JSON {userId: string, pcm: base64string}
This matches the format sent by discord-gateway via Redis.
Now PCM audio will flow correctly: Discord → Gateway → Redis → Backend → WebSocket → Browser!
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add discord:voice:pcm to SUBSCRIPTIONS as regular JSON channel
- Remove BINARY_CHANNELS and handleBinaryMessage function
- Discord-gateway sends PCM as JSON with base64, not raw binary
- This fixes PCM data not reaching browser via WebSocket
The issue was backend expected binary format but gateway sends:
{"type":"voice_pcm_data","data":{"userId":"...","pcm":"base64..."}}
Now backend correctly subscribes and broadcasts this to WebSocket clients.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add OggDemuxer to unwrap OGG container from Opus encoder output
- Enable inlineVolume for better audio control
- Discord expects raw Opus packets, not OGG-wrapped stream
This should fix the no-audio issue in voice transmit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add VoiceTransmitter class to handle PCM audio from backend/browser to Discord
- Implement voice:transmit:start and voice:transmit:stop command handlers
- Upsample 24kHz mono PCM to 48kHz stereo for Discord compatibility
- Encode PCM to Opus and stream via browser-bridge player owner
- Subscribe to Redis channel backend:voice:transmit for real-time PCM data
Features:
- Backend can send PCM audio (24kHz mono s16le base64) via Redis
- Automatic upsampling and encoding to Discord-compatible format
- Clean start/stop lifecycle with resource cleanup
This completes the bidirectional voice streaming:
- Listen: Discord → Backend (already working via voicePcmData broadcast)
- Transmit: Backend → Discord (now implemented)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add recordings directory structure with .gitkeep, .gitignore, and README
- Update deployment script to set chmod 777 on recordings directory
- Fixes ENOENT/EACCES errors when discord-gateway tries to create user subdirectories
- Ensures container app user (UID 100) can write recording files
This resolves the crash on voice channel connection where the service
failed to create user-specific recording directories like:
- recordings/<user-id>/
- recordings/sessions/
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- prism-media@2.0.0-alpha.0 OggLogicalBitstream requires node-crc
- node-crc is Rust native addon that fails to build (MSRV compat)
- Setting crc: false makes prism-media skip require('node-crc')
- OGG streams work correctly without CRC checksums
- node-crc kept in package.json
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
