- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
Default tab is now 'messages'; Voice & Media moved below
Messages in both desktop sidebar and mobile tab bar.
Updated Header titles/subtitles to match new order.
Co-Authored-By: Claude <noreply@anthropic.com>
- Add pgCorrectedModerationsTable to shared schema
- Create backend corrections module (stats, list, create endpoints)
- Add TunerPanel with Stats, History, and Submit sub-tabs
- Add AuthOverlay gate for Tuner (admin-only, same as Live)
- Set messages as default tab
- Wire Tuner into sidebar, header, and mobile tab bar
Co-Authored-By: Claude <noreply@anthropic.com>
This commit introduces several significant improvements across the backend and gateway services:
- **Data Retention**: Added an automated cleanup scheduler in `discord-gateway` to prune expired messages, attachments, and voice recordings based on configurable retention policies.
- **Observability**: Integrated `prom-client` in the `backend` service to expose Prometheus metrics via `/api/metrics` and added default Node.js runtime metrics.
- **Media Handling**: Enhanced `MediaHandler` in `discord-gateway` to support media URL resolution and improved playback status tracking.
- **API & Config**: Expanded the configuration endpoint to expose more system settings and reorganized `.env.example` for better readability.
- **Refactoring & Cleanup**:
- Removed unused `better-sqlite3` dependency.
- Refactored voice channel routing.
- Improved error handling and testing coverage with comprehensive unit tests for shared utilities and error classes.
- **Documentation**: Added `MEMORY.md` for project context.
Refactor the voice recording and playback systems to improve efficiency, reduce latency, and enhance Docker build performance.
- **Infrastructure**: Optimize Dockerfiles using build mounts for pnpm cache and reorder layers for better caching of dependencies and build tools.
- **Backend/Gateway**:
- Refactor `broadcast` module to use a generic event-based system instead of hardcoded functions.
- Simplify voice recording logic by merging metadata and segment management into a unified `segment.ts`.
- Optimize audio downsampling in `streamSetup.ts` using `Int16Array` views for better performance.
- Implement `withFallback` utility for more robust Redis/Database command execution.
- **Frontend**:
- Optimize audio playback visualization using pre-computed level shapes and efficient RMS calculation.
- Reduce latency in voice commands by prioritizing WebSocket communication over HTTP.
- Improve base64 encoding efficiency in audio transmission.
- **General**:
- Add default value for `ADMIN_PASSWORD` in shared config.
- Fix Docker healthcheck to use `127.0.0.1` instead of `localhost`.
Improve system reliability and real-time capabilities by implementing a robust lifecycle management system and adding new broadcast events for attachments and voice recordings.
- Implement asynchronous graceful shutdown in backend to close HTTP, WebSocket, Redis, and database connections.
- Add new WebSocket broadcast events: `attachment_created`, `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`, and `analysis_queue_status`.
- Refactor media status handling to use boolean `playing` state instead of string-based status.
- Centralize `PageResult` and `VoiceRecording` types to improve consistency between frontend and backend.
- Update frontend API client to include `listRecordings` and handle new WebSocket event types.
- Fix type mismatches in voice command handling and media status reporting.
- Consolidate eventTypes.ts as single source of truth for Redis channels:
- Remove duplicate DiscordGatewayEvent interface from eventBroadcaster.ts
- Replace all hardcoded channel strings with EventChannels constants
- eventTypes.ts is no longer an orphan file
- Remove dangerous moderation action feature (selfbot safety):
- Remove /messages/:id/moderate endpoint from backend
- Remove moderation:action handler from commandHandler.ts
- Remove publishFireAndForgetCommand (wrong envelope format)
- Verified no remaining references to moderation:action in code
- Remove unused getCommandPublisher import from redis-bridge.ts
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- fix(backend): replace raw .parse() with proper loadConfig() + ConfigError
- fix(gateway): connect voice recording uploader to EventBroadcaster
- fix(gateway): remove dead globalThis.moderationBroadcaster path in AI analyzer
- fix(gateway): eliminate audioStream race condition by attaching handlers before pipe
- fix(gateway): enable inlineVolume by default for setMusicVolume to work
- fix(gateway): reuse persistent redisPub for command replies (no new connection per cmd)
- fix(frontend): add missing voice_active_user/voice_pcm_data to WsEventMap
- fix(frontend): correct onAttachmentUploaded handler signature to accept data
- chore: move @types/pg from dependencies to devDependencies
- chore: translate remaining Indonesian comments to English
- chore: remove stale P3 TODO comment
- Rollback frontend latency changes (keep processor.connect(destination))
- Rollback FFmpeg aggressive optimizations (keep 20ms frames)
- Transmit was working before latency optimization; restore stable state
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Send voice:transmit:start/stop via POST /api/voice/command
instead of WebSocket. The WebSocket approach failed because:
1. Separate WS connections weren't handled by backend
2. HTTP fallback wasn't working correctly
Now uses HTTP API directly which is more reliable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add POST /api/voice/command endpoint in backend
- Frontend transmit now sends commands via dedicated WebSocket connection
with HTTP API fallback if WebSocket fails
- Fixes issue where transmit start command was never received by discord-gateway
Previously commands were sent via the existing dashboard WebSocket,
which may not be connected when the Transmit button is pressed.
Now each command opens a fresh WebSocket connection with HTTP fallback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Add onVoicePcmData and onVoiceActiveUser handlers to WebSocket interface
- Add voice_pcm_data and voice_active_user event cases in socket message handler
- Update useAudioPlayback to decode base64 PCM from JSON format
- Connect onVoicePcmData to audio.handleIncomingPcm in App.tsx
Format change:
- Old: Binary (4 bytes userId + PCM buffer)
- New: JSON {userId: string, pcm: base64string}
This matches the format sent by discord-gateway via Redis.
Now PCM audio will flow correctly: Discord → Gateway → Redis → Backend → WebSocket → Browser!
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
asepharyana