Commit Graph

107 Commits

Author SHA1 Message Date
MythEclipse
3c918692cb refactor: update messageStore to use database adapter 2026-05-14 14:56:14 +07:00
MythEclipse
94a3acf12e refactor: update muxer-queue to use database adapter
- Replace direct better-sqlite3 imports with DatabaseAdapter pattern
- Make all muxer-queue functions async to support both SQLite and PostgreSQL
- Update database initialization to use adapter's getDatabase()
- Export DatabaseAdapter as SqliteDatabase for backward compatibility
- Update index.ts to handle async database initialization
- Update webserver.ts to await async database operations
- All functions now work with both SQLite and PostgreSQL backends
- Tests pass, no TypeScript errors
2026-05-14 14:55:21 +07:00
MythEclipse
84e20ae373 feat: add database adapter layer for SQLite/PostgreSQL abstraction 2026-05-14 14:46:35 +07:00
MythEclipse
caf90ea9e6 feat: add PostgreSQL client and migration runner 2026-05-14 14:45:21 +07:00
MythEclipse
818a059121 feat: add PostgreSQL config and dependencies 2026-05-14 14:44:01 +07:00
MythEclipse
0eee7b9390 fix: cap AI batch size and split failed batches
Reduce effective AI batch size so streaming requests finish before timeout. Keep token-based batching but cap each request to 80 messages or about 9k content tokens, and recursively split failed batches instead of marking the whole batch failed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:48:20 +07:00
MythEclipse
81bb9cc6ab perf: maximize AI batches by token budget
Batch AI moderation by estimated token budget instead of fixed message count. Send as many messages as fit within an 80k token request budget while keeping one concurrent API request. Include message metadata and chronological conversation context so the model can judge provocation and replies from surrounding discussion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:42:28 +07:00
MythEclipse
4ff79bea73 chore: relax moderation prompt for casual chat
Remove unclear-message and low-quality-message warning criteria because this is a casual group. Keep short, ambiguous, informal, and light profanity messages clean unless they target someone or provoke conflict.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:36:59 +07:00
MythEclipse
bb7e3885ac chore: remove channel topic rule from moderation prompt
Remove the channel topic/OOT rule from AI moderation criteria and renumber the remaining rules. WARN criteria no longer includes OOT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:28:21 +07:00
MythEclipse
c31c4df15e docs: add detailed community rules to AI moderation prompt
- Expand system prompt with complete community rules (9 sections)
- Add specific examples for each rule category
- Clarify WARN vs FLAGGED decision criteria
- Include all prohibited content types and behaviors
- Provide clear guidance for AI analyzer on rule enforcement

Community rules now cover:
1. Jaga Sikap dan Hormati Sesama
2. Hindari Konflik
3. Gunakan Channel Sesuai Topik
4. Konten Eksplisit Dilarang
5. Jaga Privasi
6. Profil yang Sopan
7. Dilarang Spam dan Penipuan
8. Langsung ke Inti Pertanyaan
9. Diskusi Berkualitas

This ensures AI analyzer makes consistent moderation decisions based on actual community rules.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:24:19 +07:00
MythEclipse
93eb2303c7 feat: add warn category for minor rule violations
- Add "warn" status between "clean" and "flagged" for minor violations
- Update AI analyzer system prompt with community rules and warn category
- Warn: profanity, OOT, tone issues - requires warning but not deletion
- Flagged: NSFW, illegal, hacking, scam, harassment, violence, SARA - requires review/deletion
- Update types to support warn status in MessageRecord and AIAnalysisUpdate
- Update client UI to show three panels: All Messages, Warned, Flagged
- Warned messages show in right-top panel for quick review
- Flagged messages show in right-bottom panel for moderation action

This resolves:
- Need to distinguish between minor and severe violations
- Moderators can now warn users before taking action
- Better moderation workflow with three-tier system

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:23:11 +07:00
MythEclipse
0060c4a097 feat: batch AI analysis messages for faster processing
- Change runLLMAnalysis to accept array of texts instead of single text
- Batch up to 5 messages per AI request instead of 1 message per request
- drainQueue now collects batch before sending to AI API
- Reduces API calls by 5x and speeds up analysis significantly
- System prompt updated to handle batch JSON array responses

This resolves:
- Slow AI analysis (3 messages every 15 seconds)
- Too many API calls (one per message)
- Long queue backlog

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:08:41 +07:00
MythEclipse
5aa57f884f feat: add API endpoint for syncing selected channel backlog 2026-05-14 04:02:25 +07:00
MythEclipse
d5977c8845 fix: handle streaming JSON response from AI LLM API
- Fix fetchJson to extract JSON from streaming response text
- API returns text/event-stream with complete JSON object embedded
- Extract JSON by finding first { and last } in response
- Prevents "Unexpected non-whitespace character after JSON" parse errors
- Streaming response now properly parsed and analyzed

This resolves:
- AI analysis stuck on "[Streaming in progress...]"
- JSON parse failures on streaming responses
- AI analysis now completes successfully

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:00:31 +07:00
MythEclipse
6dc6a31ea7 fix: enforce max 1 concurrent AI LLM request
- Add activeRequests counter to track in-flight AI requests
- Limit concurrent requests to 1 (MAX_CONCURRENT_REQUESTS)
- drainQueue now waits if at max concurrency before processing next message
- Prevents overwhelming streaming LLM API with multiple concurrent requests

This resolves:
- AI LLM API overload from concurrent requests
- Streaming response conflicts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:54:12 +07:00
MythEclipse
54534fe84c perf: parallelize backlog sync with concurrency limit
- Sync channels in parallel with concurrency limit of 3 instead of sequentially
- Reduces backlog sync time from O(n) to O(n/3) for n channels
- Prevents overwhelming Discord API with too many concurrent requests
- Maintains per-channel message batching for memory efficiency

This resolves:
- Slow backlog sync performance
- Sequential channel processing bottleneck

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:52:21 +07:00
MythEclipse
eb27d36cce fix: remove Picser upload, use Discord URLs directly
- Skip attachment download/upload to Picser (was failing with 400 errors)
- Store Discord's original attachment URLs directly as uploaded_url
- Mark attachments as immediately uploaded with Discord URL
- Remove processAttachmentUpload call and unused attachmentUploader import
- Eliminates slow upload cycle and API failures

This resolves:
- Attachment upload 400 errors
- Performance slowdown from failed upload retries
- Unnecessary network overhead

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:51:48 +07:00
MythEclipse
0a5cedfed1 fix: resolve UI state reset and backlog sync hang
- Fix client startup order: fetch/apply server UI state BEFORE loadGuilds() to prevent overwriting persisted state with default guild
- Remove auto-post of first guild in loadGuilds() — let server state drive selection
- Refactor collectWatchableChannels() to collect text channels fast first, then discover threads in parallel with 5s per-channel timeout and 30s overall timeout to prevent blocking message sync
- Ignore /favicon.ico 404 in error logging to reduce noise

Fixes:
- UI state now persists across restart (was being overwritten by client startup race)
- Backlog sync no longer hangs on thread discovery (was blocking before message sync)
- Cleaner logs without favicon 404 errors

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:45:51 +07:00
MythEclipse
d4a4f737a8 feat: enhance backlog sync logging and implement UI state persistence 2026-05-14 03:17:07 +07:00
MythEclipse
6e203604ec feat: remove OpenAI moderation configuration and update AI analysis logic 2026-05-14 02:44:26 +07:00
MythEclipse
be6c9f8132 feat: add AI analysis integration with moderation and LLM processing 2026-05-14 02:31:16 +07:00
MythEclipse
b36d038eba feat: implement shared UI state management with API endpoints for state retrieval and updates 2026-05-14 01:45:27 +07:00
MythEclipse
a02d1fb7c0 chore: add pnpm workspace configuration with only built dependencies 2026-05-14 01:07:50 +07:00
MythEclipse
0bdab3b446 Refactor dashboard page: replace React component with static HTML, remove unused dashboardPage.tsx, and update webserver to serve new index.html 2026-05-14 00:14:25 +07:00
MythEclipse
65dc73e903 fix: use Tone.js global object from CDN instead of ES module import
- Remove ES module import statement from dashboard.js
- Use global Tone object loaded via CDN
- Remove type=module from script tag
- Fixes module resolution error
2026-05-13 22:48:03 +07:00
MythEclipse
bc212333d8 feat: implement Tone.js for professional audio transmit and listen
- Replace AudioWorkletNode with Tone.UserMedia for microphone capture
- Use Tone.Analyser for waveform analysis and Tone.Meter for level detection
- Implement proper stereo audio capture with real-time PCM transmission
- Use Tone.context for audio playback with proper buffer handling
- Add Tone.js CDN to HTML template
- Convert dashboard.js to ES module for Tone.js import
- Improve audio quality and reliability with battle-tested library
2026-05-13 22:46:24 +07:00
MythEclipse
0f30a4aa67 revert: simplify listen to PCM-only, remove WebCodecs complexity
- Remove broadcastOpusToWeb from types and webserver
- Revert recorder to PCM-only broadcast path
- Remove WebCodecs AudioDecoder from dashboard.js
- Simplify WebSocket handler to just play PCM frames
- Remove Howler.js and Tone.js dependencies
- Keep simple AudioContext playback that works reliably
2026-05-13 22:26:59 +07:00
MythEclipse
25dbd8413b feat: implement WebCodecs Opus decode for browser listen
- Add broadcastOpusToWeb to PcmBroadcaster interface for raw Opus packets
- Server broadcasts Opus frames with mode byte (1) + user hash + packet data
- Browser detects packet mode: mode=1 for Opus, mode=0 for legacy PCM
- Implement WebCodecs AudioDecoder for Opus decoding in browser
- Keep existing PCM playback as fallback for compatibility
- Show error if WebCodecs unsupported
- Fixes listen feature under Bun where native Opus decode unavailable
2026-05-13 22:13:03 +07:00
MythEclipse
95cb8b837a feat(thread-discovery): implement separate endpoint for fetching threads and update channel loading logic 2026-05-13 21:22:05 +07:00
MythEclipse
0b8111de81 feat(moderation): implement backlog message synchronization and enhance message metadata handling 2026-05-13 21:04:45 +07:00
MythEclipse
d55b56c897 feat(moderation): enhance message capture and storage with thread support
- Added functions to retrieve message location, sticker metadata, and display content in messageCapture.ts.
- Updated captureMessage function to store thread information and sticker metadata in the database.
- Modified messageStore.ts to support querying messages and attachments by thread ID.
- Updated types.ts to include thread_id in AttachmentRecord.
- Altered database schema in muxer-queue.ts to add thread_id column to attachments.
- Introduced ChannelSummary interface and listWatchableChannels method in voiceController.ts to fetch watchable channels.
- Added API endpoint in webserver.ts to retrieve channels for a given guild.
2026-05-13 20:52:37 +07:00
MythEclipse
9f09f5ef28 refactor: replace Database type with SqliteDatabase in moderation modules 2026-05-13 19:47:44 +07:00
MythEclipse
692962408f feat: add REST API and WebSocket events for moderation
- Add /api/messages endpoint for querying text and image data
- Add WebSocket broadcast functions for real-time updates
- Support message_created, message_updated, message_deleted events
- Support attachment_uploaded event for real-time image updates
2026-05-13 19:34:15 +07:00
MythEclipse
738f5cfbd6 feat: integrate message capture into bot startup
- Register message capture handlers on bot ready
- Export getDatabase function from muxer-queue
- Initialize database schema for messages and attachments
2026-05-13 19:34:15 +07:00
MythEclipse
b13dfb2ece config: add moderation watcher configuration variables
- Add MONITOR_GUILD_ID for target server monitoring
- Add PICSER_UPLOAD_URL for attachment upload endpoint
- Add attachment upload timeout and size limit settings
- Add retry attempts configuration for upload failures
2026-05-13 19:34:14 +07:00
MythEclipse
017efb0b86 feat: implement moderation message capture system
- Add message store with database operations (insert, update, query)
- Implement attachment uploader with picser integration
- Add Discord event listeners for message create/update/delete
- Support attachment upload with retry logic and error handling
- Add comprehensive unit tests for message store and uploader
2026-05-13 19:34:14 +07:00
MythEclipse
220c3b93d2 feat: implement Opus decoder runtime checks and add tests for decoder functionality 2026-05-13 18:56:44 +07:00
MythEclipse
4dadcf3871 feat: implement web-driven voice connection with guild/channel selection and API integration 2026-05-13 18:23:20 +07:00
MythEclipse
0e056732bc feat: implement one-port WebSocket server with updated connection handling and configuration 2026-05-13 17:49:33 +07:00
MythEclipse
f9a4b4a92d feat: add muxer job queue with SQLite integration and metrics tracking 2026-05-13 17:06:22 +07:00
MythEclipse
978c2c468d feat: enhance configuration management and add error handling 2026-05-13 16:57:07 +07:00
MythEclipse
673a06376c feat: implement graceful shutdown process for bot
- Added graceful shutdown functionality to handle SIGINT, SIGTERM, uncaught exceptions, and unhandled promise rejections.
- Integrated stopRecording and pause functionality during shutdown.
- Enhanced logging for shutdown steps and error handling.
- Updated package.json to include pino-pretty for improved logging output.
2026-05-13 16:32:14 +07:00
MythEclipse
3ae28157a3 feat: implement logging and retry mechanism with pino and p-retry 2026-05-13 16:25:01 +07:00
MythEclipse
7a5ac2e34a refactor: make hardcoded values configurable via env vars 2026-05-13 16:05:19 +07:00
MythEclipse
80081a6f62 refactor: modularize recorder orchestration 2026-05-13 16:00:21 +07:00
MythEclipse
f655daa0c7 style: format recorder modules 2026-05-13 15:56:56 +07:00
MythEclipse
dbe593c6fd refactor: extract recorder domain types and modules 2026-05-13 15:56:42 +07:00
MythEclipse
da108c5d84 refactor: type application config 2026-05-13 15:56:00 +07:00
MythEclipse
d470321321 style: format packetFilter 2026-05-13 15:54:54 +07:00
MythEclipse
138aa397e2 chore: add code quality tooling 2026-05-13 15:28:25 +07:00