Commit Graph
100 Commits
Author SHA1 Message Date
MythEclipse 235c1120c2 feat: enhance moderation functionality with type improvements and global broadcaster integration 2026-05-15 07:13:37 +07:00
MythEclipse 930c399484 feat: enhance message capture and processing with backlog support 2026-05-15 06:52:20 +07:00
MythEclipse 958a6d7236 feat: implement vendor selfbot dependency modernization plan
- Update submodule `discord.js-selfbot-v13` to latest commit.
- Create implementation plan for modernizing the vendored dependency, focusing on toolchain replacement with Biome and runtime dependency auditing.
- Establish a design document outlining goals, scope, approach, toolchain design, runtime dependency design, and validation steps.
- Ensure public API compatibility and maintain existing functionality during modernization efforts.
2026-05-15 06:26:22 +07:00
MythEclipse bbd65d369e feat: implement selfbot workspace submodule
- Add `discord.js-selfbot-v13` as a git submodule in `vendor/`
- Update `pnpm-workspace.yaml` to include the new submodule path
- Modify `.gitmodules` to track the submodule repository
- Change `discord.js-selfbot-v13` dependency in `package.json` to use `workspace:*`
- Create implementation and design documents for the submodule integration
2026-05-15 05:45:19 +07:00
MythEclipse bbd3a88471 Merge branch 'worktree-library-modernization' 2026-05-15 04:34:41 +07:00
MythEclipse ed10f45e8c fix: adapt retry logging to p-retry v8 2026-05-15 04:28:11 +07:00
MythEclipseandClaude Opus 4.7 29fcde69e4 fix: adapt code to modernized libraries
- Migrate validation.ts from class-transformer/class-validator to Zod
- Apply Biome auto-fixes (import sorting, nodejs protocol)
- Fix duplicate type identifier in validation.ts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 04:25:06 +07:00
MythEclipseandClaude Opus 4.7 0060d1ba95 chore: modernize dependency set
- Remove deprecated packages: class-transformer, class-validator, fluent-ffmpeg, @types/fluent-ffmpeg
- Update remaining packages to latest versions via pnpm update --latest
- @discordjs/voice: ^0.19.1 → ^0.19.2
- @snazzah/davey: ^0.1.10 → ^0.1.11
- libsodium-wrappers: ^0.8.2 → ^0.8.4
- p-retry: ^6.2.0 → ^8.0.0
- pino: ^9.4.0 → ^10.3.1
- sodium-native: ^4.3.2 → ^5.1.0
- @types/node: ^24.10.1 → ^25.8.0
- pino-pretty: ^10.3.1 → ^13.1.3
- tsx: ^4.20.6 → ^4.22.0
- vitest: latest (preserved)
- @biomejs/biome: latest (preserved)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 04:18:33 +07:00
MythEclipse 895b47890c refactor: invoke ffmpeg without deprecated wrapper 2026-05-14 23:42:11 +07:00
MythEclipseandClaude Opus 4.7 e5aa398e5c refactor: use esm-safe database migrations
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 23:23:07 +07:00
MythEclipse 954faf6c5a refactor: validate user state with zod 2026-05-14 23:11:25 +07:00
MythEclipse 5010a4d1f1 docs: record dependency modernization audit 2026-05-14 23:01:18 +07:00
MythEclipse 203aa9a589 style: organize imports after dashboard rebuild 2026-05-14 21:19:43 +07:00
MythEclipseandClaude Opus 4.7 196ca1b784 feat: serve React dashboard from public/app when available
Check for public/app/index.html before falling back to the old
public/index.html so the React moderation dashboard is served at
the root in production while preserving the legacy frontend during
the transition period.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 21:16:03 +07:00
MythEclipse a812182218 fix: harden dashboard message state 2026-05-14 21:10:24 +07:00
MythEclipse cb2cfc76f2 fix: import dashboard event type from websocket client 2026-05-14 21:03:38 +07:00
MythEclipseandClaude Opus 4.7 ae02556f75 feat: add moderation review dashboard
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 21:01:57 +07:00
MythEclipse fbf0f11db1 fix: align dashboard message api shape 2026-05-14 20:55:05 +07:00
MythEclipse 0a7965e1d4 fix: remove unused dashboard type import 2026-05-14 20:47:51 +07:00
MythEclipse 2d56012bf5 feat: add dashboard api clients 2026-05-14 20:46:21 +07:00
MythEclipse c3f2d01e9b a 2026-05-14 20:27:35 +07:00
MythEclipse 298048aaaf fix: isolate frontend typescript config 2026-05-14 20:26:38 +07:00
MythEclipse 71a28085d2 feat: scaffold react dashboard 2026-05-14 20:24:41 +07:00
MythEclipse da5e191b0f fix: make moderation index migration portable 2026-05-14 20:19:47 +07:00
MythEclipse 558d65342b feat: add moderation query indexes 2026-05-14 20:14:42 +07:00
MythEclipse 44368e646f fix: reanalyze edited messages 2026-05-14 20:10:16 +07:00
MythEclipse dbfc6866f9 refactor: keep message capture on fast path 2026-05-14 20:03:02 +07:00
MythEclipse d8aeefb739 fix: preserve ui state broadcasts in routes 2026-05-14 19:58:37 +07:00
MythEclipse 08cd515d8a fix: preserve ui state broadcasts in routes 2026-05-14 19:55:26 +07:00
MythEclipse c81a499535 refactor: split api routes by concern 2026-05-14 19:46:47 +07:00
MythEclipse 3fb1fcb72c fix: remove unused analysis import 2026-05-14 19:41:18 +07:00
MythEclipse 243a18ecad fix: harden analysis queue scheduling 2026-05-14 19:39:25 +07:00
MythEclipse f14e893cb7 feat: debounce ai analysis by conversation 2026-05-14 19:32:44 +07:00
MythEclipse 54cd4e0386 fix: type llm moderation test fixtures 2026-05-14 19:28:56 +07:00
MythEclipse 65ab5ecb32 fix: harden llm moderation parsing 2026-05-14 19:26:54 +07:00
MythEclipse 81253e4ffe feat: add strict llm moderation client 2026-05-14 19:21:15 +07:00
MythEclipse ce35b335d0 test: cover conversation context edge cases 2026-05-14 19:16:46 +07:00
MythEclipse 2b4e2a7ab7 feat: add conversation context builder 2026-05-14 19:10:13 +07:00
MythEclipse 2d511e08db style: format message query changes 2026-05-14 19:06:17 +07:00
MythEclipse 13078e7c3c fix: resolve message query test diagnostics 2026-05-14 19:02:19 +07:00
MythEclipse 298fc968cf test: cover message query pagination 2026-05-14 19:01:01 +07:00
MythEclipse 7a8883f623 test: cover message query pagination 2026-05-14 19:00:03 +07:00
MythEclipse 0c54ac4ba8 test: cover message query pagination 2026-05-14 18:54:13 +07:00
MythEclipse 9f5f8a3090 feat: add cursor-based message queries 2026-05-14 18:48:02 +07:00
MythEclipse 8ab7aaa32d fix: harden moderation broadcaster 2026-05-14 18:44:47 +07:00
MythEclipseandClaude Opus 4.7 01dc9b1836 feat: add typed moderation broadcaster
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 18:40:34 +07:00
MythEclipse 49d4bbf781 chore: remove obsolete migration documentation files 2026-05-14 16:40:43 +07:00
MythEclipse 4fbbc056bb docs: add postgresql setup completion documentation 2026-05-14 16:26:12 +07:00
MythEclipse 47ae7f8650 chore: remove temporary test files 2026-05-14 16:25:45 +07:00
MythEclipse 35269b5bef feat: configure postgresql as primary database with neon connection
- Updated .env to use PostgreSQL (neondb) instead of SQLite
- Updated drizzle.ts to support DATABASE_URL connection string
- Regenerated migrations for PostgreSQL syntax
- Bot successfully connects and operates with PostgreSQL
- All database operations working correctly
2026-05-14 16:25:39 +07:00
MythEclipse c63a61460c docs: add comprehensive drizzle orm migration final summary 2026-05-14 16:21:35 +07:00
MythEclipse 9889d20edd feat: add programmatic migration runner for better PostgreSQL support 2026-05-14 16:21:02 +07:00
MythEclipse b580430eb6 docs: add drizzle orm migration completion summary 2026-05-14 16:17:25 +07:00
MythEclipse b9d0a06d01 fix: update drizzle config to read env vars directly for CLI compatibility 2026-05-14 16:14:48 +07:00
MythEclipse b600dad011 fix: correct import ordering and update tests for drizzle-orm migration 2026-05-14 15:47:03 +07:00
MythEclipse 50d4517079 refactor: remove old database adapter files 2026-05-14 15:43:52 +07:00
MythEclipse 9ff0f0bede feat: update application initialization for drizzle 2026-05-14 15:43:16 +07:00
MythEclipse 1c4b0afbce refactor: migrate messageStore to drizzle-orm
- Replace all raw SQL queries in messageStore.ts with Drizzle ORM queries
- Remove DatabaseAdapter dependency from messageStore functions
- Update all function signatures to be async and remove db parameter
- Functions now use getDatabase() internally for database access
- Update all call sites in messageCapture.ts, attachmentUploader.ts, aiAnalyzer.ts, webserver.ts, and index.ts
- All functions remain backward compatible in behavior
- TypeScript typecheck passes with no errors
- All tests pass (11 passed)
2026-05-14 15:41:11 +07:00
MythEclipse dfe3444018 refactor: migrate muxer-queue to drizzle-orm 2026-05-14 15:35:55 +07:00
MythEclipse 7e528a473b feat: create drizzle database client 2026-05-14 15:33:45 +07:00
MythEclipse 4e28cf9671 feat: add drizzle configuration and initial migrations 2026-05-14 15:33:10 +07:00
MythEclipse 52b36c963f feat: create drizzle schema definitions 2026-05-14 15:32:20 +07:00
MythEclipse b833b6d978 feat: add drizzle-orm and drizzle-kit dependencies 2026-05-14 15:31:08 +07:00
MythEclipse d1282f2f57 fix: organize imports and apply linting fixes 2026-05-14 15:02:23 +07:00
MythEclipse 1623c612c3 docs: add PostgreSQL migration guide 2026-05-14 15:00:35 +07:00
MythEclipse 8c9e8aa64d test: add PostgreSQL connection tests 2026-05-14 14:59:37 +07:00
MythEclipse c5297da795 feat: initialize database adapter on startup 2026-05-14 14:58:28 +07:00
MythEclipse dbc11bbd16 feat: add data migration script from SQLite to PostgreSQL 2026-05-14 14:57:08 +07:00
MythEclipse 3c918692cb refactor: update messageStore to use database adapter 2026-05-14 14:56:14 +07:00
MythEclipse 94a3acf12e refactor: update muxer-queue to use database adapter
- Replace direct better-sqlite3 imports with DatabaseAdapter pattern
- Make all muxer-queue functions async to support both SQLite and PostgreSQL
- Update database initialization to use adapter's getDatabase()
- Export DatabaseAdapter as SqliteDatabase for backward compatibility
- Update index.ts to handle async database initialization
- Update webserver.ts to await async database operations
- All functions now work with both SQLite and PostgreSQL backends
- Tests pass, no TypeScript errors
2026-05-14 14:55:21 +07:00
MythEclipse 84e20ae373 feat: add database adapter layer for SQLite/PostgreSQL abstraction 2026-05-14 14:46:35 +07:00
MythEclipse caf90ea9e6 feat: add PostgreSQL client and migration runner 2026-05-14 14:45:21 +07:00
MythEclipse 818a059121 feat: add PostgreSQL config and dependencies 2026-05-14 14:44:01 +07:00
MythEclipseandClaude Opus 4.7 0eee7b9390 fix: cap AI batch size and split failed batches
Reduce effective AI batch size so streaming requests finish before timeout. Keep token-based batching but cap each request to 80 messages or about 9k content tokens, and recursively split failed batches instead of marking the whole batch failed.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:48:20 +07:00
MythEclipseandClaude Opus 4.7 81bb9cc6ab perf: maximize AI batches by token budget
Batch AI moderation by estimated token budget instead of fixed message count. Send as many messages as fit within an 80k token request budget while keeping one concurrent API request. Include message metadata and chronological conversation context so the model can judge provocation and replies from surrounding discussion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:42:28 +07:00
MythEclipseandClaude Opus 4.7 4ff79bea73 chore: relax moderation prompt for casual chat
Remove unclear-message and low-quality-message warning criteria because this is a casual group. Keep short, ambiguous, informal, and light profanity messages clean unless they target someone or provoke conflict.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:36:59 +07:00
MythEclipseandClaude Opus 4.7 9ff1261239 feat: merge warn and flagged messages into one review panel
Combine the right-side Warned and Flagged panels into a single Needs Review panel. Keep ordering by the existing newest-first message list.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:33:48 +07:00
MythEclipseandClaude Opus 4.7 bb7e3885ac chore: remove channel topic rule from moderation prompt
Remove the channel topic/OOT rule from AI moderation criteria and renumber the remaining rules. WARN criteria no longer includes OOT.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:28:21 +07:00
MythEclipseandClaude Opus 4.7 c31c4df15e docs: add detailed community rules to AI moderation prompt
- Expand system prompt with complete community rules (9 sections)
- Add specific examples for each rule category
- Clarify WARN vs FLAGGED decision criteria
- Include all prohibited content types and behaviors
- Provide clear guidance for AI analyzer on rule enforcement

Community rules now cover:
1. Jaga Sikap dan Hormati Sesama
2. Hindari Konflik
3. Gunakan Channel Sesuai Topik
4. Konten Eksplisit Dilarang
5. Jaga Privasi
6. Profil yang Sopan
7. Dilarang Spam dan Penipuan
8. Langsung ke Inti Pertanyaan
9. Diskusi Berkualitas

This ensures AI analyzer makes consistent moderation decisions based on actual community rules.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:24:19 +07:00
MythEclipseandClaude Opus 4.7 93eb2303c7 feat: add warn category for minor rule violations
- Add "warn" status between "clean" and "flagged" for minor violations
- Update AI analyzer system prompt with community rules and warn category
- Warn: profanity, OOT, tone issues - requires warning but not deletion
- Flagged: NSFW, illegal, hacking, scam, harassment, violence, SARA - requires review/deletion
- Update types to support warn status in MessageRecord and AIAnalysisUpdate
- Update client UI to show three panels: All Messages, Warned, Flagged
- Warned messages show in right-top panel for quick review
- Flagged messages show in right-bottom panel for moderation action

This resolves:
- Need to distinguish between minor and severe violations
- Moderators can now warn users before taking action
- Better moderation workflow with three-tier system

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:23:11 +07:00
MythEclipseandClaude Opus 4.7 54a4096323 feat: add two-column text tab layout with flagged messages panel
- Split text tab into two columns: All Messages (left) and Flagged (right)
- Left panel shows all captured messages
- Right panel shows only AI-flagged messages for quick review
- Flagged panel auto-populates when messages are analyzed
- Improves moderation workflow by separating flagged content

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:16:53 +07:00
MythEclipseandClaude Opus 4.7 1c945b9cac fix: trigger backlog sync when text channel is selected
- Call POST /api/backlog-sync when user selects a text channel
- Backlog sync now runs automatically on channel selection
- Fetches messages from last 24 hours for selected channel only
- Prevents empty message list on first channel selection

This resolves:
- Empty message list when selecting channel
- Backlog sync not being triggered
- Messages not loading until manual refresh

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:12:33 +07:00
MythEclipseandClaude Opus 4.7 0060c4a097 feat: batch AI analysis messages for faster processing
- Change runLLMAnalysis to accept array of texts instead of single text
- Batch up to 5 messages per AI request instead of 1 message per request
- drainQueue now collects batch before sending to AI API
- Reduces API calls by 5x and speeds up analysis significantly
- System prompt updated to handle batch JSON array responses

This resolves:
- Slow AI analysis (3 messages every 15 seconds)
- Too many API calls (one per message)
- Long queue backlog

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:08:41 +07:00
MythEclipse 5aa57f884f feat: add API endpoint for syncing selected channel backlog 2026-05-14 04:02:25 +07:00
MythEclipseandClaude Opus 4.7 d5977c8845 fix: handle streaming JSON response from AI LLM API
- Fix fetchJson to extract JSON from streaming response text
- API returns text/event-stream with complete JSON object embedded
- Extract JSON by finding first { and last } in response
- Prevents "Unexpected non-whitespace character after JSON" parse errors
- Streaming response now properly parsed and analyzed

This resolves:
- AI analysis stuck on "[Streaming in progress...]"
- JSON parse failures on streaming responses
- AI analysis now completes successfully

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 04:00:31 +07:00
MythEclipseandClaude Opus 4.7 6dc6a31ea7 fix: enforce max 1 concurrent AI LLM request
- Add activeRequests counter to track in-flight AI requests
- Limit concurrent requests to 1 (MAX_CONCURRENT_REQUESTS)
- drainQueue now waits if at max concurrency before processing next message
- Prevents overwhelming streaming LLM API with multiple concurrent requests

This resolves:
- AI LLM API overload from concurrent requests
- Streaming response conflicts

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:54:12 +07:00
MythEclipseandClaude Opus 4.7 54534fe84c perf: parallelize backlog sync with concurrency limit
- Sync channels in parallel with concurrency limit of 3 instead of sequentially
- Reduces backlog sync time from O(n) to O(n/3) for n channels
- Prevents overwhelming Discord API with too many concurrent requests
- Maintains per-channel message batching for memory efficiency

This resolves:
- Slow backlog sync performance
- Sequential channel processing bottleneck

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:52:21 +07:00
MythEclipseandClaude Opus 4.7 eb27d36cce fix: remove Picser upload, use Discord URLs directly
- Skip attachment download/upload to Picser (was failing with 400 errors)
- Store Discord's original attachment URLs directly as uploaded_url
- Mark attachments as immediately uploaded with Discord URL
- Remove processAttachmentUpload call and unused attachmentUploader import
- Eliminates slow upload cycle and API failures

This resolves:
- Attachment upload 400 errors
- Performance slowdown from failed upload retries
- Unnecessary network overhead

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:51:48 +07:00
MythEclipseandClaude Opus 4.7 0a5cedfed1 fix: resolve UI state reset and backlog sync hang
- Fix client startup order: fetch/apply server UI state BEFORE loadGuilds() to prevent overwriting persisted state with default guild
- Remove auto-post of first guild in loadGuilds() — let server state drive selection
- Refactor collectWatchableChannels() to collect text channels fast first, then discover threads in parallel with 5s per-channel timeout and 30s overall timeout to prevent blocking message sync
- Ignore /favicon.ico 404 in error logging to reduce noise

Fixes:
- UI state now persists across restart (was being overwritten by client startup race)
- Backlog sync no longer hangs on thread discovery (was blocking before message sync)
- Cleaner logs without favicon 404 errors

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 03:45:51 +07:00
MythEclipse d4a4f737a8 feat: enhance backlog sync logging and implement UI state persistence 2026-05-14 03:17:07 +07:00
MythEclipse 6e203604ec feat: remove OpenAI moderation configuration and update AI analysis logic 2026-05-14 02:44:26 +07:00
MythEclipse be6c9f8132 feat: add AI analysis integration with moderation and LLM processing 2026-05-14 02:31:16 +07:00
MythEclipse b36d038eba feat: implement shared UI state management with API endpoints for state retrieval and updates 2026-05-14 01:45:27 +07:00
MythEclipse a02d1fb7c0 chore: add pnpm workspace configuration with only built dependencies 2026-05-14 01:07:50 +07:00
MythEclipse 0bdab3b446 Refactor dashboard page: replace React component with static HTML, remove unused dashboardPage.tsx, and update webserver to serve new index.html 2026-05-14 00:14:25 +07:00
MythEclipse cbfb99f755 debug: add listen path logging
- Log binary packet receipt and listen state
- Log listen toggle and AudioContext creation
- Log playPcm calls, packet size, user hash, and playback timing
- Helps trace why listen produces no audio
2026-05-13 23:04:56 +07:00
MythEclipse bac42b1d53 fix: match listen logic exactly from aa85dd9
- Add CHANNELS constant (1 for mono)
- Use audioBuffer.length for loop instead of float32Array.length
- Use getChannelData(0) assignment pattern from aa85dd9
- Proper buffer frame calculation with CHANNELS divisor
2026-05-13 23:01:42 +07:00
MythEclipse ff83e34b77 fix: restore working transmit and listen logic from aa85dd9
- Use createScriptProcessor instead of Tone.UserMedia for transmit
- Use analyser.getByteFrequencyData() for proper visualizer
- Keep per-user timing with userTimelines Map for listen
- Remove Tone.js dependencies from transmit
- Restore proper AudioContext usage
2026-05-13 22:58:59 +07:00
MythEclipse 94dc460fc7 fix: implement proper audio timing for listen with per-user timelines
- Add audioContextListen and userTimelines to state
- Create AudioContext on listen toggle
- Parse user ID hash from PCM packet header
- Track playback timing per user to prevent audio gaps
- Use proper AudioContext timing instead of Tone.now()
- Clear timelines when toggling listen off
2026-05-13 22:53:17 +07:00
MythEclipse 65dc73e903 fix: use Tone.js global object from CDN instead of ES module import
- Remove ES module import statement from dashboard.js
- Use global Tone object loaded via CDN
- Remove type=module from script tag
- Fixes module resolution error
2026-05-13 22:48:03 +07:00