Compare commits
13 Commits
4dadcf3871
...
95cb8b837a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
95cb8b837a | ||
|
|
0b8111de81 | ||
|
|
d55b56c897 | ||
|
|
c7d8353403 | ||
|
|
9f09f5ef28 | ||
|
|
471e3bac82 | ||
|
|
6d353c1753 | ||
|
|
692962408f | ||
|
|
738f5cfbd6 | ||
|
|
b13dfb2ece | ||
|
|
017efb0b86 | ||
|
|
579fcb4684 | ||
|
|
220c3b93d2 |
11
.env.example
11
.env.example
@@ -1,7 +1,5 @@
|
||||
# Discord Bot Configuration
|
||||
DISCORD_TOKEN=your_bot_token_here
|
||||
VOICE_CHANNEL_ID=your_voice_channel_id_here
|
||||
GUILD_ID=your_guild_id_here
|
||||
|
||||
# Recording Configuration
|
||||
RECORDINGS_DIR=./recordings
|
||||
@@ -31,5 +29,12 @@ RECONNECT_TIMEOUT_MS=5000
|
||||
LOG_LEVEL=info
|
||||
NODE_ENV=development
|
||||
|
||||
|
||||
# Moderation Configuration
|
||||
MONITOR_GUILD_ID=your_guild_id_here
|
||||
PICSER_UPLOAD_URL=https://picser.asepharyana.tech/api/upload
|
||||
ATTACHMENT_UPLOAD_TIMEOUT_MS=30000
|
||||
ATTACHMENT_MAX_SIZE_MB=100
|
||||
ATTACHMENT_RETRY_ATTEMPTS=3
|
||||
BACKLOG_SYNC_HOURS=24
|
||||
BACKLOG_SYNC_BATCH_SIZE=100
|
||||
|
||||
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -2,3 +2,4 @@ node_modules
|
||||
recordings
|
||||
.env
|
||||
dist/
|
||||
.muxer-queue.**
|
||||
513
CLAUDE.md
Normal file
513
CLAUDE.md
Normal file
@@ -0,0 +1,513 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
|
||||
## Project Overview
|
||||
|
||||
**Discord Moderation Watcher Bot** — A comprehensive monitoring bot that captures voice, text messages, and images from Discord servers. Records audio from voice channels, captures all text messages (new/edited/deleted) from channels and threads, and uploads attachments to external storage. All data stored in SQLite with real-time dashboard.
|
||||
|
||||
Built with **Bun** + **discord.js-selfbot-v13** + **@discordjs/voice** + **Express** + **WebSocket**.
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level Flow
|
||||
|
||||
1. **Bot Entry** (`src/index.ts`) — Initializes Discord client, registers event listeners, starts webserver
|
||||
2. **Message Capture** (`src/moderation/messageCapture.ts`) — Listens to Discord events (messageCreate, messageUpdate, messageDelete)
|
||||
3. **Message Store** (`src/moderation/messageStore.ts`) — Database operations for messages and attachments
|
||||
4. **Attachment Uploader** (`src/moderation/attachmentUploader.ts`) — Downloads from Discord, uploads to picser, stores URLs
|
||||
5. **Voice Controller** (`src/voiceController.ts`) — Manages voice channel connections
|
||||
6. **Recorder** (`src/recorder.ts`) — Records voice audio to OGG segments
|
||||
7. **Web Server** (`src/webserver.ts`) — Express + WebSocket for REST API and real-time updates
|
||||
8. **Dashboard** (`public/dashboard.html`) — Web UI with three tabs (Text, Images, Voice)
|
||||
|
||||
### Key Modules
|
||||
|
||||
**Moderation Subsystem** (`src/moderation/`):
|
||||
- `types.ts` — TypeScript types for messages, attachments, voice segments
|
||||
- `messageCapture.ts` — Discord event listeners (messageCreate, messageUpdate, messageDelete)
|
||||
- `messageStore.ts` — Database CRUD operations (insert, update, query)
|
||||
- `attachmentUploader.ts` — Picser integration with retry logic and error handling
|
||||
|
||||
**Database Schema** (SQLite):
|
||||
- `messages` table — text messages with edit/delete tracking, user metadata, timestamps
|
||||
- `attachments` table — attachment metadata, Discord URLs, picser URLs, upload status
|
||||
- Indexes on channel_id, user_id, created_at for fast queries
|
||||
|
||||
**Voice Recording** (existing, unchanged):
|
||||
- `recorder.ts` — Joins voice channel, subscribes to user audio streams
|
||||
- `recorder/audioStream.ts` — Opus packet subscription
|
||||
- `recorder/decoder.ts` — Opus decoder with runtime checks
|
||||
- `recorder/segment.ts` — OGG file rotation (5s segments)
|
||||
|
||||
**Web Interface**:
|
||||
- REST API: `/api/messages?channel=<id>&type=text|image`
|
||||
- WebSocket: real-time events (message_created, message_updated, message_deleted, attachment_uploaded)
|
||||
- Dashboard: three tabs (Text Messages, Images, Voice) with channel filtering
|
||||
|
||||
### Recording Structure
|
||||
|
||||
```
|
||||
recordings/
|
||||
├── <user-id>/
|
||||
│ ├── <user-id>-<session-start>-0.ogg
|
||||
│ ├── <user-id>-<session-start>-0.json
|
||||
│ └── ...
|
||||
|
||||
messages (SQLite):
|
||||
├── id, guild_id, channel_id, thread_id
|
||||
├── user_id, username, avatar_url
|
||||
├── content, edited_content
|
||||
├── created_at, edited_at, deleted_at
|
||||
└── type (text|edited|deleted)
|
||||
|
||||
attachments (SQLite):
|
||||
├── id, message_id, guild_id, channel_id, user_id
|
||||
├── filename, size, type (MIME)
|
||||
├── discord_url, uploaded_url (picser raw_commit)
|
||||
├── upload_status (pending|uploaded|failed)
|
||||
└── created_at, uploaded_at
|
||||
```
|
||||
|
||||
## Development Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
bun install
|
||||
|
||||
# Development (auto-restart on file changes)
|
||||
bun run dev
|
||||
|
||||
# Production
|
||||
bun run start
|
||||
|
||||
# Type checking
|
||||
bun run typecheck
|
||||
|
||||
# Linting (Biome)
|
||||
bun run lint
|
||||
|
||||
# Format code (Biome)
|
||||
bun run format
|
||||
|
||||
# Run tests
|
||||
bun run test
|
||||
|
||||
# Build TypeScript
|
||||
bun run build
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All config via `.env` (see `.env.example`). Key variables:
|
||||
|
||||
**Discord & Monitoring:**
|
||||
- `DISCORD_TOKEN` — Bot token (required)
|
||||
- `MONITOR_GUILD_ID` — Target server to monitor (required for moderation)
|
||||
- `GUILD_ID` — Legacy voice channel guild (optional)
|
||||
- `VOICE_CHANNEL_ID` — Legacy voice channel ID (optional)
|
||||
|
||||
**Recording:**
|
||||
- `RECORDINGS_DIR` — Where to save audio files (default: `./recordings`)
|
||||
- `RECORDING_SEGMENT_MS` — OGG segment duration (default: 5000ms)
|
||||
|
||||
**Decoder:**
|
||||
- `DECODER_ROTATE_MS` — Opus decoder rotation interval (default: 5000ms)
|
||||
- `DECODER_COOLDOWN_MS` — Cooldown after decoder error (default: 30000ms)
|
||||
|
||||
**Attachments:**
|
||||
- `PICSER_UPLOAD_URL` — Picser upload endpoint (default: https://picser.asepharyana.tech/api/upload)
|
||||
- `ATTACHMENT_UPLOAD_TIMEOUT_MS` — Upload timeout (default: 30000ms)
|
||||
- `ATTACHMENT_MAX_SIZE_MB` — Max file size (default: 100MB)
|
||||
- `ATTACHMENT_RETRY_ATTEMPTS` — Retry count (default: 3)
|
||||
|
||||
**Web Server:**
|
||||
- `WEBSERVER_PORT` — HTTP/WebSocket port (default: 3000)
|
||||
|
||||
**Connection:**
|
||||
- `VOICE_CONNECTION_TIMEOUT_MS` — Voice join timeout (default: 15000ms)
|
||||
- `RECONNECT_TIMEOUT_MS` — Reconnect timeout (default: 5000ms)
|
||||
- `AUDIO_STREAM_SILENCE_DURATION_MS` — Silence threshold (default: 3000ms)
|
||||
|
||||
**Logging:**
|
||||
- `LOG_LEVEL` — Pino log level (default: info)
|
||||
- `VERBOSE` — Enable debug logging (default: false)
|
||||
- `NODE_ENV` — Environment (development|production|test)
|
||||
|
||||
## Testing
|
||||
|
||||
Tests use **Vitest** in `tests/` directory. Run with `bun run test`.
|
||||
|
||||
**Test Coverage:**
|
||||
- `tests/moderation/messageStore.test.ts` — Message store CRUD operations
|
||||
- `tests/moderation/attachmentUploader.test.ts` — Picser response parsing
|
||||
- `tests/config.test.ts` — Configuration validation
|
||||
- `tests/decoder.test.ts` — Opus decoder runtime detection
|
||||
|
||||
## Code Style
|
||||
|
||||
- **Formatter**: Biome (2-space indent)
|
||||
- **Linter**: Biome with custom rules (warn on non-null assertions, noExplicitAny)
|
||||
- **Language**: TypeScript with strict mode
|
||||
- **Logging**: Use `createChildLogger(context)` for scoped logs
|
||||
- **Errors**: Throw custom AppError subclasses with code + statusCode
|
||||
- **Database**: Use prepared statements, never string interpolation
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Message Capture Lifecycle
|
||||
|
||||
1. Discord event fires (messageCreate, messageUpdate, messageDelete)
|
||||
2. Check if guild matches MONITOR_GUILD_ID
|
||||
3. Extract message metadata (user, channel, content, timestamp)
|
||||
4. Insert into messages table
|
||||
5. Broadcast WebSocket event to connected clients
|
||||
6. If attachments exist:
|
||||
- Insert into attachments table with status='pending'
|
||||
- Start async upload to picser (non-blocking)
|
||||
- On success: update uploaded_url, status='uploaded'
|
||||
- On failure: store error, status='failed'
|
||||
|
||||
### Attachment Upload Flow
|
||||
|
||||
1. Download from Discord URL (with timeout)
|
||||
2. Validate file size against ATTACHMENT_MAX_SIZE_MB
|
||||
3. Upload to picser with retry logic (exponential backoff)
|
||||
4. Parse response, extract raw_commit URL
|
||||
5. Update database with uploaded_url and status
|
||||
6. Broadcast attachment_uploaded event
|
||||
|
||||
### WebSocket Protocol
|
||||
|
||||
**Inbound (browser → bot):**
|
||||
- Binary: Raw PCM buffers (24kHz mono s16le) for voice transmission
|
||||
|
||||
**Outbound (bot → browser):**
|
||||
- Binary: 4-byte user ID hash + PCM chunk (voice)
|
||||
- JSON: `{ type: "user_state", users: [...] }` (active speakers)
|
||||
- JSON: `{ type: "message_created", data: {...} }` (new text message)
|
||||
- JSON: `{ type: "message_updated", data: {...} }` (edited message)
|
||||
- JSON: `{ type: "message_deleted", data: {...} }` (deleted message)
|
||||
- JSON: `{ type: "attachment_uploaded", data: {...} }` (image uploaded)
|
||||
|
||||
### Graceful Shutdown
|
||||
|
||||
Handles SIGINT/SIGTERM/uncaughtException/unhandledRejection:
|
||||
1. Stop voice connection
|
||||
2. Pause player
|
||||
3. Destroy Discord client
|
||||
4. Exit process
|
||||
|
||||
## Dashboard Usage
|
||||
|
||||
**Access:** `http://localhost:3000/dashboard.html`
|
||||
|
||||
**Features:**
|
||||
- Three tabs: Text Messages | Images | Voice
|
||||
- Channel/thread filter dropdown
|
||||
- Real-time WebSocket updates
|
||||
- Polling fallback if WebSocket disconnects
|
||||
- Message display with metadata (author, timestamp, edits, deletions)
|
||||
- Image grid with previews and upload status
|
||||
- Voice segment list (future enhancement)
|
||||
|
||||
**Keyboard/UI:**
|
||||
- Click tab to switch content type
|
||||
- Select channel to filter
|
||||
- Click image to view full size
|
||||
- WebSocket status indicator (green = connected)
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Add a new config variable
|
||||
|
||||
1. Add to `configSchema` in `src/config.ts` with Zod validation
|
||||
2. Add to `.env.example` with description
|
||||
3. Use via `config.VARIABLE_NAME`
|
||||
|
||||
### Add a new REST endpoint
|
||||
|
||||
1. Add route in `src/webserver.ts` (Express)
|
||||
2. Use database functions from `src/moderation/messageStore.ts`
|
||||
3. Wrap in try-catch, pass errors to Express error handler
|
||||
4. Return JSON response
|
||||
|
||||
### Add a new WebSocket event
|
||||
|
||||
1. Define broadcast function in `src/webserver.ts` (attach to globalThis)
|
||||
2. Call from event handler (e.g., messageCapture.ts)
|
||||
3. Send JSON with `{ type, data, timestamp }`
|
||||
4. Handle in dashboard JavaScript
|
||||
|
||||
### Debug message capture
|
||||
|
||||
- Set `VERBOSE=true` in `.env` for detailed logging
|
||||
- Check `/health` endpoint for active users/connections
|
||||
- Monitor `/metrics` endpoint (Prometheus format)
|
||||
- Check `recordings/<user-id>/` for voice segments
|
||||
- Query SQLite directly: `sqlite3 .muxer-queue.db "SELECT * FROM messages LIMIT 10;"`
|
||||
|
||||
### Debug attachment uploads
|
||||
|
||||
- Check `upload_status` in attachments table
|
||||
- View `upload_error` field for failure reasons
|
||||
- Monitor logs for "Attachment upload" messages
|
||||
- Verify picser endpoint is accessible
|
||||
- Check file size against ATTACHMENT_MAX_SIZE_MB
|
||||
|
||||
## Dependencies
|
||||
|
||||
**Core:**
|
||||
- **discord.js-selfbot-v13** — Discord client (selfbot variant)
|
||||
- **@discordjs/voice** — Voice connection management
|
||||
- **@discordjs/opus** — Native Opus codec (optional, required for web PCM)
|
||||
- **prism-media** — Audio encoding/decoding (Opus, OGG)
|
||||
|
||||
**Web:**
|
||||
- **express** — HTTP server
|
||||
- **ws** — WebSocket server
|
||||
- **helmet** — Security headers
|
||||
|
||||
**Data:**
|
||||
- **better-sqlite3** — SQLite database
|
||||
- **zod** — Config validation
|
||||
|
||||
**Logging & Monitoring:**
|
||||
- **pino** — Structured logging
|
||||
- **pino-http** — HTTP request logging
|
||||
- **prom-client** — Prometheus metrics
|
||||
|
||||
**Utilities:**
|
||||
- **p-retry** — Retry logic with backoff
|
||||
- **class-transformer** — Object transformation
|
||||
- **class-validator** — Data validation
|
||||
|
||||
**Dev:**
|
||||
- **Biome** — Linting/formatting
|
||||
- **Vitest** — Testing framework
|
||||
- **TypeScript** — Type checking
|
||||
|
||||
## Notes
|
||||
|
||||
- Bot uses selfbot variant (user account) rather than standard bot token — check Discord ToS
|
||||
- Opus decoding disabled on Bun without native opus to avoid crashes
|
||||
- OGG segments include metadata JSON for each segment (user info, timestamps, duration)
|
||||
- WebSocket broadcasts PCM in real-time; browser can transmit audio back to Discord
|
||||
- Graceful shutdown ensures clean disconnection and resource cleanup
|
||||
- All database operations use prepared statements to prevent SQL injection
|
||||
- Attachment uploads are non-blocking (async) to avoid blocking message capture
|
||||
- Message capture continues even if attachment upload fails
|
||||
- Dashboard uses textContent for XSS prevention (not innerHTML)
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Reaction tracking
|
||||
- Message search/full-text search
|
||||
- Moderation actions (flag, delete, mute)
|
||||
- Export/archive functionality
|
||||
- Retention policies (auto-delete old data)
|
||||
- Voice segment metadata in dashboard
|
||||
- User activity analytics
|
||||
- Audit log export
|
||||
|
||||
|
||||
## Architecture
|
||||
|
||||
### High-Level Flow
|
||||
|
||||
1. **Bot Entry** (`src/index.ts`) — Initializes Discord client, sets up graceful shutdown, starts webserver
|
||||
2. **Voice Controller** (`src/voiceController.ts`) — Manages guild/channel selection and connection lifecycle
|
||||
3. **Recorder** (`src/recorder.ts`) — Joins voice channel, subscribes to user audio streams, handles Opus decoding and segment rotation
|
||||
4. **Web Server** (`src/webserver.ts`) — Express + WebSocket server for:
|
||||
- REST API: guild/channel listing, connect/disconnect
|
||||
- WebSocket: real-time PCM broadcast to browser, browser-to-Discord audio transmission
|
||||
5. **Muxer Queue** (`src/muxer-queue.ts`) — SQLite-backed job queue for post-processing audio segments (future use)
|
||||
|
||||
### Key Modules
|
||||
|
||||
- **Recorder subsystem** (`src/recorder/`):
|
||||
- `audioStream.ts` — Subscribes to Discord audio receiver, emits Opus packets
|
||||
- `decoder.ts` — Opus decoder with runtime checks (Bun vs Node), cooldown/rotation logic for web PCM broadcast
|
||||
- `segment.ts` — Manages OGG file rotation (5s default segments per user)
|
||||
- `metadata.ts` — Collects user/role info, creates segment metadata JSON
|
||||
|
||||
- **Voice Connection** — Uses `@discordjs/voice` receiver to subscribe to speaking users; each user gets their own stream
|
||||
- **Audio Pipeline**:
|
||||
- Discord → Opus packets → PacketFilter → OGG segments (disk) + OpusDecoder → PCM (web broadcast)
|
||||
- Browser → 24kHz mono PCM → upsample to 48kHz stereo → Opus encoder → OGG → Discord player
|
||||
|
||||
- **Metrics** (`src/metrics.ts`) — Prometheus metrics for audio levels, recordings, connections, WebSocket clients
|
||||
- **Logging** (`src/logger.ts`) — Pino logger with pretty-print in dev, JSON in prod
|
||||
- **Config** (`src/config.ts`) — Zod-validated environment variables with sensible defaults
|
||||
- **Error Handling** (`src/errors.ts`) — Custom error classes (AppError, ConfigError, AudioError, VoiceConnectionError, ValidationError)
|
||||
|
||||
### Recording Structure
|
||||
|
||||
```
|
||||
recordings/
|
||||
├── <user-id>/
|
||||
│ ├── <user-id>-<session-start>-0.ogg
|
||||
│ ├── <user-id>-<session-start>-0.json
|
||||
│ ├── <user-id>-<session-start>-1.ogg
|
||||
│ ├── <user-id>-<session-start>-1.json
|
||||
│ └── ...
|
||||
```
|
||||
|
||||
Each segment is 5s (configurable). Metadata JSON includes user info, roles, timestamps, duration.
|
||||
|
||||
### Database
|
||||
|
||||
- **Muxer Queue** (`.muxer-queue.db`) — SQLite with WAL mode, tracks pending/processing/completed/failed jobs for audio post-processing
|
||||
|
||||
## Development Commands
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
bun install
|
||||
|
||||
# Development (auto-restart on file changes)
|
||||
bun run dev
|
||||
|
||||
# Production
|
||||
bun run start
|
||||
|
||||
# Type checking
|
||||
bun run typecheck
|
||||
|
||||
# Linting (Biome)
|
||||
bun run lint
|
||||
|
||||
# Format code (Biome)
|
||||
bun run format
|
||||
|
||||
# Run tests
|
||||
bun run test
|
||||
|
||||
# Build TypeScript
|
||||
bun run build
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
All config via `.env` (see `.env.example`). Key variables:
|
||||
|
||||
- `DISCORD_TOKEN` — Bot token (required)
|
||||
- `RECORDINGS_DIR` — Where to save audio files (default: `./recordings`)
|
||||
- `RECORDING_SEGMENT_MS` — OGG segment duration (default: 5000ms)
|
||||
- `DECODER_ROTATE_MS` — Opus decoder rotation interval (default: 5000ms)
|
||||
- `DECODER_COOLDOWN_MS` — Cooldown after decoder error (default: 30000ms)
|
||||
- `WEBSERVER_PORT` — HTTP/WebSocket port (default: 3000)
|
||||
- `VOICE_CONNECTION_TIMEOUT_MS` — Voice join timeout (default: 15000ms)
|
||||
- `AUDIO_STREAM_SILENCE_DURATION_MS` — Silence threshold before ending stream (default: 3000ms)
|
||||
- `LOG_LEVEL` — Pino log level (default: info)
|
||||
- `VERBOSE` — Enable debug logging (default: false)
|
||||
|
||||
## Testing
|
||||
|
||||
Tests use **Vitest** in `tests/` directory. Run with `bun run test`.
|
||||
|
||||
Example: `tests/decoder.test.ts` tests Opus decoder runtime detection (Bun vs Node, native opus availability).
|
||||
|
||||
## Code Style
|
||||
|
||||
- **Formatter**: Biome (2-space indent)
|
||||
- **Linter**: Biome with custom rules (warn on non-null assertions, noExplicitAny)
|
||||
- **Language**: TypeScript with strict mode
|
||||
- **Logging**: Use `createChildLogger(context)` for scoped logs
|
||||
- **Errors**: Throw custom AppError subclasses with code + statusCode
|
||||
|
||||
## Key Patterns
|
||||
|
||||
### Voice Connection Lifecycle
|
||||
|
||||
1. `VoiceController.connect(guildId, channelId)` → calls `startRecording()`
|
||||
2. `startRecording()` joins channel, sets up receiver, subscribes to speaking users
|
||||
3. On user speak: create stream, segment manager, decoder; pipe to OGG + web broadcast
|
||||
4. On silence (3s): close stream, save metadata JSON
|
||||
5. `VoiceController.disconnect()` → calls `stopRecording()` → destroys connection
|
||||
|
||||
### Audio Decoding (Web Broadcast)
|
||||
|
||||
- OpusDecoder wraps prism decoder with error recovery
|
||||
- Rotates decoder every 5s to prevent memory leaks
|
||||
- Cools down for 30s after error before retrying
|
||||
- Downsamples 48kHz stereo → 24kHz mono for web transmission
|
||||
|
||||
### WebSocket Protocol
|
||||
|
||||
- **Inbound** (browser → bot): Raw PCM buffers (24kHz mono s16le)
|
||||
- **Outbound** (bot → browser):
|
||||
- Binary: 4-byte user ID hash + PCM chunk
|
||||
- JSON: `{ type: "user_state", users: [...] }` on connect/user activity change
|
||||
|
||||
### Graceful Shutdown
|
||||
|
||||
Handles SIGINT/SIGTERM/uncaughtException/unhandledRejection:
|
||||
1. Stop voice connection
|
||||
2. Pause player
|
||||
3. Destroy Discord client
|
||||
4. Exit process
|
||||
|
||||
## Future Expansion (Text/Image Monitoring)
|
||||
|
||||
Current scope: voice only. Planned additions:
|
||||
- Text channel message capture
|
||||
- Image/attachment logging
|
||||
- Per-channel/per-user filtering
|
||||
- Moderation action triggers
|
||||
|
||||
These will likely require:
|
||||
- Additional event listeners in recorder
|
||||
- Extended metadata schema
|
||||
- New storage/indexing strategy
|
||||
- Webhook/alert system
|
||||
|
||||
## Common Tasks
|
||||
|
||||
### Add a new config variable
|
||||
|
||||
1. Add to `configSchema` in `src/config.ts` with Zod validation
|
||||
2. Add to `.env.example`
|
||||
3. Use via `config.VARIABLE_NAME`
|
||||
|
||||
### Add a new REST endpoint
|
||||
|
||||
1. Add route in `src/webserver.ts` (Express)
|
||||
2. Use `VoiceController` methods or create new ones
|
||||
3. Wrap in try-catch, pass errors to Express error handler
|
||||
|
||||
### Add metrics
|
||||
|
||||
1. Define gauge/counter/histogram in `src/metrics.ts`
|
||||
2. Update in relevant code paths
|
||||
3. Metrics exposed at `/metrics` endpoint (Prometheus format)
|
||||
|
||||
### Debug audio issues
|
||||
|
||||
- Set `VERBOSE=true` in `.env` for detailed logging
|
||||
- Check `/health` endpoint for active users/connections
|
||||
- Monitor audio levels via `/metrics` (audio_level_db gauge)
|
||||
- Check segment files in `recordings/<user-id>/` directory
|
||||
|
||||
## Dependencies
|
||||
|
||||
- **discord.js-selfbot-v13** — Discord client (selfbot variant for user account access)
|
||||
- **@discordjs/voice** — Voice connection management
|
||||
- **@discordjs/opus** — Native Opus codec (optional, required for web PCM decode)
|
||||
- **prism-media** — Audio encoding/decoding (Opus, OGG)
|
||||
- **express** — HTTP server
|
||||
- **ws** — WebSocket server
|
||||
- **better-sqlite3** — SQLite database (muxer queue)
|
||||
- **pino** — Structured logging
|
||||
- **prom-client** — Prometheus metrics
|
||||
- **zod** — Config validation
|
||||
- **Biome** — Linting/formatting
|
||||
- **Vitest** — Testing framework
|
||||
|
||||
## Notes
|
||||
|
||||
- Bot uses selfbot variant (user account) rather than standard bot token — check Discord ToS
|
||||
- Opus decoding disabled on Bun without native opus to avoid crashes
|
||||
- OGG segments include metadata JSON for each segment (user info, timestamps, duration)
|
||||
- WebSocket broadcasts PCM in real-time; browser can transmit audio back to Discord
|
||||
- Graceful shutdown ensures clean disconnection and resource cleanup
|
||||
@@ -0,0 +1,64 @@
|
||||
# Backlog Sync Rich Metadata Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Fetch prior Discord messages up to 24 hours on startup, persist rich Discord-client-like metadata, and render rich message content in homepage tabs.
|
||||
|
||||
**Architecture:** Add `messageMetadata.ts` for reusable extraction, `backlogSync.ts` for bounded startup history fetch, reuse existing store/uploader. UI reads metadata JSON and renders stickers, embeds, attachments/replies/thread badges.
|
||||
|
||||
**Tech Stack:** Bun, TypeScript, discord.js-selfbot-v13, bun:sqlite, Express/WebSocket, vanilla HTML/CSS/JS.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Extract rich message metadata
|
||||
|
||||
**Files:**
|
||||
- Create: `src/moderation/messageMetadata.ts`
|
||||
- Modify: `src/moderation/messageCapture.ts`
|
||||
|
||||
- [ ] Create helper functions: `getMessageLocation`, `getStickerMetadata`, `getEmbedMetadata`, `getAttachmentMetadata`, `getMessageMetadata`, `getDisplayContent`.
|
||||
- [ ] Replace duplicate capture helper logic with imports from `messageMetadata.ts`.
|
||||
- [ ] Verify: `bun run typecheck`.
|
||||
|
||||
### Task 2: Make message inserts idempotent
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/moderation/messageStore.ts`
|
||||
|
||||
- [ ] Change message insert to `INSERT OR IGNORE` so backlog sync and live events do not conflict.
|
||||
- [ ] Change attachment insert to `INSERT OR IGNORE`.
|
||||
- [ ] Verify: `bun run typecheck && bun run test`.
|
||||
|
||||
### Task 3: Add backlog sync
|
||||
|
||||
**Files:**
|
||||
- Create: `src/moderation/backlogSync.ts`
|
||||
- Modify: `src/index.ts`
|
||||
- Modify: `src/config.ts`
|
||||
- Modify: `.env.example`
|
||||
|
||||
- [ ] Add config: `BACKLOG_SYNC_HOURS=24`, `BACKLOG_SYNC_BATCH_SIZE=100`.
|
||||
- [ ] Fetch text/thread channels from monitored guild on ready.
|
||||
- [ ] For each channel/thread, page `channel.messages.fetch({ limit, before })` until older than cutoff.
|
||||
- [ ] Store messages with rich metadata and attachments.
|
||||
- [ ] Start sync after registering live capture; run async and log progress.
|
||||
- [ ] Verify: `bun run typecheck && bun run test`.
|
||||
|
||||
### Task 4: Render richer UI
|
||||
|
||||
**Files:**
|
||||
- Modify: `public/index.html`
|
||||
|
||||
- [ ] Render metadata embeds as embed cards.
|
||||
- [ ] Render attachments as inline previews/links in Text tab.
|
||||
- [ ] Render reply and thread badges.
|
||||
- [ ] Keep sticker rendering.
|
||||
- [ ] Verify static JS syntax by typecheck/tests where applicable.
|
||||
|
||||
### Task 5: Final verification
|
||||
|
||||
**Files:** all touched files
|
||||
|
||||
- [ ] Run `bun run typecheck`.
|
||||
- [ ] Run `bun run test`.
|
||||
- [ ] Verify short DB init with `bun -e 'import("./src/muxer-queue.ts").then((m)=>{const db=m.getDatabase(); db.close(); console.log("sqlite ok")})'`.
|
||||
@@ -0,0 +1,38 @@
|
||||
# Separate Thread Discovery Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Keep channel listing fast and move expensive active/archived thread discovery to a separate endpoint loaded asynchronously by the homepage.
|
||||
|
||||
**Architecture:** `VoiceController` exposes cache-only channels and network-backed threads separately. `webserver.ts` adds `/api/guilds/:guildId/threads`. `public/index.html` loads channels first, then appends thread options after thread endpoint returns.
|
||||
|
||||
**Tech Stack:** TypeScript, discord.js-selfbot-v13, Express, vanilla JS.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add thread discovery method
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/voiceController.ts`
|
||||
|
||||
- [ ] Add `listThreads(guildId)` that fetches active and archived threads per parent text channel.
|
||||
- [ ] Keep `listWatchableChannels` cache-only.
|
||||
- [ ] Verify typecheck.
|
||||
|
||||
### Task 2: Add thread API endpoint
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/webserver.ts`
|
||||
|
||||
- [ ] Add `GET /api/guilds/:guildId/threads`.
|
||||
- [ ] Return thread summaries.
|
||||
- [ ] Verify typecheck.
|
||||
|
||||
### Task 3: Update homepage dropdown loading
|
||||
|
||||
**Files:**
|
||||
- Modify: `public/index.html`
|
||||
|
||||
- [ ] `loadChannels` fetches `/channels` first and renders immediately.
|
||||
- [ ] Then fetches `/threads` async and appends thread options.
|
||||
- [ ] Verify typecheck/tests.
|
||||
240
docs/superpowers/specs/2026-05-13-moderation-watcher-design.md
Normal file
240
docs/superpowers/specs/2026-05-13-moderation-watcher-design.md
Normal file
@@ -0,0 +1,240 @@
|
||||
# Moderation Watcher Expansion Design
|
||||
|
||||
**Date:** 2026-05-13
|
||||
**Status:** Design Phase
|
||||
**Scope:** Expand Discord bot from voice-only recorder to full moderation watcher capturing text, images, and voice
|
||||
|
||||
## Overview
|
||||
|
||||
Transform the existing voice recorder bot into a unified moderation watcher that captures:
|
||||
- **Voice:** Audio from voice channels (existing)
|
||||
- **Text:** Messages (new/edited/deleted) from all channels and threads
|
||||
- **Images:** Attachments uploaded to all channels and threads
|
||||
|
||||
All data stored in SQLite database. Attachments uploaded to external picser service. Unified dashboard with separate tabs for each content type, filterable by channel/thread.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Functional
|
||||
|
||||
1. **Text Message Capture**
|
||||
- Capture new messages: content, author, channel, timestamp
|
||||
- Capture edited messages: original + edited content, edit timestamp
|
||||
- Capture deleted messages: content, author, deletion timestamp
|
||||
- Store in database with full metadata
|
||||
|
||||
2. **Image/Attachment Capture**
|
||||
- Detect attachments in messages
|
||||
- Upload to `https://picser.asepharyana.tech/api/upload`
|
||||
- Store `raw_commit` URL in database
|
||||
- Store attachment metadata: filename, size, type, upload timestamp
|
||||
|
||||
3. **Voice Recording** (existing, no changes)
|
||||
- Continue recording voice segments as-is
|
||||
- Segments already stored in database via muxer queue
|
||||
|
||||
4. **Dashboard API**
|
||||
- `/api/messages?channel=<id>&type=text|image|voice` — Query messages by type and channel
|
||||
- `/api/channels` — List all monitored channels
|
||||
- Real-time WebSocket updates: `message_created`, `message_updated`, `message_deleted`, `attachment_uploaded`
|
||||
|
||||
5. **Dashboard UI**
|
||||
- Three tabs: Voice | Text | Images
|
||||
- Channel/thread filter dropdown
|
||||
- Display messages/attachments with metadata (author, timestamp, content)
|
||||
- Real-time updates via WebSocket, polling fallback
|
||||
|
||||
### Non-Functional
|
||||
|
||||
- **Target Server:** Configured via `MONITOR_GUILD_ID` environment variable
|
||||
- **Database:** Single SQLite (`.muxer-queue.db`), extended schema
|
||||
- **Attachment Upload:** Async, non-blocking; store URL when ready
|
||||
- **Real-time:** WebSocket for live updates, REST polling as fallback
|
||||
- **Performance:** Index on channel_id, user_id, created_at for fast queries
|
||||
|
||||
## Architecture
|
||||
|
||||
### Database Schema
|
||||
|
||||
**New Tables:**
|
||||
|
||||
```sql
|
||||
-- Text messages
|
||||
CREATE TABLE messages (
|
||||
id TEXT PRIMARY KEY,
|
||||
guild_id TEXT NOT NULL,
|
||||
channel_id TEXT NOT NULL,
|
||||
thread_id TEXT,
|
||||
user_id TEXT NOT NULL,
|
||||
username TEXT NOT NULL,
|
||||
avatar_url TEXT,
|
||||
content TEXT NOT NULL,
|
||||
edited_content TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
edited_at INTEGER,
|
||||
deleted_at INTEGER,
|
||||
type TEXT NOT NULL DEFAULT 'text', -- 'text', 'edited', 'deleted'
|
||||
metadata TEXT -- JSON: roles, etc.
|
||||
);
|
||||
|
||||
CREATE INDEX idx_messages_channel ON messages(channel_id);
|
||||
CREATE INDEX idx_messages_user ON messages(user_id);
|
||||
CREATE INDEX idx_messages_created ON messages(created_at DESC);
|
||||
CREATE INDEX idx_messages_thread ON messages(thread_id);
|
||||
|
||||
-- Attachments
|
||||
CREATE TABLE attachments (
|
||||
id TEXT PRIMARY KEY,
|
||||
message_id TEXT NOT NULL,
|
||||
guild_id TEXT NOT NULL,
|
||||
channel_id TEXT NOT NULL,
|
||||
user_id TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
size INTEGER NOT NULL,
|
||||
type TEXT NOT NULL, -- MIME type
|
||||
discord_url TEXT NOT NULL,
|
||||
uploaded_url TEXT, -- picser raw_commit URL
|
||||
upload_status TEXT NOT NULL DEFAULT 'pending', -- 'pending', 'uploaded', 'failed'
|
||||
upload_error TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
uploaded_at INTEGER,
|
||||
FOREIGN KEY (message_id) REFERENCES messages(id)
|
||||
);
|
||||
|
||||
CREATE INDEX idx_attachments_channel ON attachments(channel_id);
|
||||
CREATE INDEX idx_attachments_message ON attachments(message_id);
|
||||
CREATE INDEX idx_attachments_status ON attachments(upload_status);
|
||||
```
|
||||
|
||||
### Event Handlers
|
||||
|
||||
**New Discord Event Listeners:**
|
||||
|
||||
1. `messageCreate` — Insert into `messages` table
|
||||
2. `messageUpdate` — Update `messages` table with edited content + timestamp
|
||||
3. `messageDelete` — Mark message as deleted (soft delete with `deleted_at`)
|
||||
4. `messageReactionAdd` — (Optional: track reactions)
|
||||
|
||||
**Attachment Processing:**
|
||||
|
||||
- On `messageCreate`: Extract attachments, insert into `attachments` table with `upload_status='pending'`
|
||||
- Async job: Download from Discord URL, upload to picser, update `uploaded_url` and `upload_status`
|
||||
- If upload fails: Set `upload_status='failed'`, store error message
|
||||
|
||||
### API Endpoints
|
||||
|
||||
**REST:**
|
||||
|
||||
```
|
||||
GET /api/messages?channel=<id>&type=text|image|voice&limit=50&offset=0
|
||||
→ Returns paginated messages/attachments
|
||||
|
||||
GET /api/channels
|
||||
→ Returns list of all channels in monitored guild
|
||||
|
||||
GET /api/attachments?channel=<id>&limit=50
|
||||
→ Returns attachments with upload status
|
||||
```
|
||||
|
||||
**WebSocket Events (outbound):**
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "message_created",
|
||||
"data": { "id", "channel_id", "user_id", "username", "content", "created_at" }
|
||||
}
|
||||
|
||||
{
|
||||
"type": "message_updated",
|
||||
"data": { "id", "edited_content", "edited_at" }
|
||||
}
|
||||
|
||||
{
|
||||
"type": "message_deleted",
|
||||
"data": { "id", "deleted_at" }
|
||||
}
|
||||
|
||||
{
|
||||
"type": "attachment_uploaded",
|
||||
"data": { "id", "message_id", "filename", "uploaded_url", "created_at" }
|
||||
}
|
||||
```
|
||||
|
||||
### File Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── moderation/
|
||||
│ ├── messageCapture.ts -- Discord event listeners
|
||||
│ ├── attachmentUploader.ts -- Upload to picser, manage queue
|
||||
│ ├── messageStore.ts -- Database operations
|
||||
│ └── types.ts -- Message/Attachment types
|
||||
├── webserver.ts -- Add /api/messages, /api/channels endpoints
|
||||
├── index.ts -- Register message event listeners
|
||||
└── config.ts -- Add MONITOR_GUILD_ID
|
||||
```
|
||||
|
||||
### Configuration
|
||||
|
||||
**New Environment Variables:**
|
||||
|
||||
```env
|
||||
MONITOR_GUILD_ID=<guild-id> # Target server to monitor
|
||||
PICSER_UPLOAD_URL=https://picser.asepharyana.tech/api/upload
|
||||
ATTACHMENT_UPLOAD_TIMEOUT_MS=30000 # Upload timeout
|
||||
ATTACHMENT_MAX_SIZE_MB=100 # Max file size to upload
|
||||
```
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Database & Core Capture
|
||||
- Extend SQLite schema (messages, attachments tables)
|
||||
- Implement message capture handlers (create/edit/delete)
|
||||
- Add message store functions (insert, update, query)
|
||||
|
||||
### Phase 2: Attachment Upload
|
||||
- Implement picser uploader with retry logic
|
||||
- Add attachment processing queue
|
||||
- Store URLs in database
|
||||
|
||||
### Phase 3: API & WebSocket
|
||||
- Add REST endpoints for querying messages/attachments
|
||||
- Add WebSocket events for real-time updates
|
||||
- Implement channel listing
|
||||
|
||||
### Phase 4: Dashboard UI
|
||||
- Build frontend with Voice | Text | Images tabs
|
||||
- Implement channel filter
|
||||
- Add real-time WebSocket listener + polling fallback
|
||||
|
||||
## Error Handling
|
||||
|
||||
- **Upload failures:** Retry with exponential backoff, store error in `upload_error` field
|
||||
- **Database errors:** Log and continue (don't crash bot)
|
||||
- **Missing attachments:** Handle Discord URL expiry gracefully
|
||||
- **WebSocket disconnects:** Clients reconnect and poll for missed messages
|
||||
|
||||
## Testing
|
||||
|
||||
- Unit tests for message store functions (insert, update, query)
|
||||
- Integration tests for attachment uploader (mock picser API)
|
||||
- E2E tests for Discord event capture (mock Discord client)
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- ✅ All text messages captured (new/edited/deleted)
|
||||
- ✅ All attachments uploaded to picser with URLs stored
|
||||
- ✅ Dashboard displays all three content types in separate tabs
|
||||
- ✅ Channel filter works correctly
|
||||
- ✅ Real-time WebSocket updates working
|
||||
- ✅ Polling fallback works if WebSocket disconnects
|
||||
- ✅ No data loss on bot restart
|
||||
- ✅ Graceful handling of upload failures
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
- Reaction tracking
|
||||
- Message search/full-text search
|
||||
- Moderation actions (flag, delete, mute)
|
||||
- Export/archive functionality
|
||||
- Retention policies (auto-delete old data)
|
||||
@@ -37,10 +37,10 @@
|
||||
"devDependencies": {
|
||||
"@biomejs/biome": "latest",
|
||||
"@types/better-sqlite3": "^7.6.13",
|
||||
"@types/bun": "latest",
|
||||
"@types/express": "^5.0.6",
|
||||
"@types/fluent-ffmpeg": "^2.1.28",
|
||||
"@types/ws": "^8.18.1",
|
||||
"bun-types": "latest",
|
||||
"pino-pretty": "^10.3.1",
|
||||
"vitest": "latest"
|
||||
}
|
||||
|
||||
528
public/dashboard.html
Normal file
528
public/dashboard.html
Normal file
@@ -0,0 +1,528 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Moderation Dashboard</title>
|
||||
<style>
|
||||
* {
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, sans-serif;
|
||||
background: #0f0f0f;
|
||||
color: #e0e0e0;
|
||||
padding: 20px;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
header {
|
||||
margin-bottom: 30px;
|
||||
border-bottom: 2px solid #333;
|
||||
padding-bottom: 20px;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 28px;
|
||||
margin-bottom: 10px;
|
||||
color: #fff;
|
||||
}
|
||||
|
||||
.status {
|
||||
display: flex;
|
||||
gap: 20px;
|
||||
font-size: 14px;
|
||||
color: #999;
|
||||
}
|
||||
|
||||
.status-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
}
|
||||
|
||||
.status-dot {
|
||||
width: 8px;
|
||||
height: 8px;
|
||||
border-radius: 50%;
|
||||
background: #666;
|
||||
}
|
||||
|
||||
.status-dot.connected {
|
||||
background: #4ade80;
|
||||
}
|
||||
|
||||
.controls {
|
||||
display: flex;
|
||||
gap: 15px;
|
||||
margin-bottom: 20px;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
select {
|
||||
padding: 8px 12px;
|
||||
background: #1a1a1a;
|
||||
border: 1px solid #333;
|
||||
color: #e0e0e0;
|
||||
border-radius: 4px;
|
||||
font-size: 14px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
select:hover {
|
||||
border-color: #555;
|
||||
}
|
||||
|
||||
.tabs {
|
||||
display: flex;
|
||||
gap: 10px;
|
||||
margin-bottom: 20px;
|
||||
border-bottom: 1px solid #333;
|
||||
}
|
||||
|
||||
.tab {
|
||||
padding: 12px 20px;
|
||||
background: none;
|
||||
border: none;
|
||||
color: #999;
|
||||
cursor: pointer;
|
||||
font-size: 14px;
|
||||
font-weight: 500;
|
||||
border-bottom: 2px solid transparent;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.tab:hover {
|
||||
color: #e0e0e0;
|
||||
}
|
||||
|
||||
.tab.active {
|
||||
color: #fff;
|
||||
border-bottom-color: #4ade80;
|
||||
}
|
||||
|
||||
.content {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.content.active {
|
||||
display: block;
|
||||
}
|
||||
|
||||
.message-list {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 12px;
|
||||
}
|
||||
|
||||
.message-item {
|
||||
background: #1a1a1a;
|
||||
border: 1px solid #333;
|
||||
border-radius: 6px;
|
||||
padding: 15px;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.message-item:hover {
|
||||
border-color: #555;
|
||||
background: #222;
|
||||
}
|
||||
|
||||
.message-header {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 12px;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
|
||||
.avatar {
|
||||
width: 32px;
|
||||
height: 32px;
|
||||
border-radius: 50%;
|
||||
background: #333;
|
||||
flex-shrink: 0;
|
||||
}
|
||||
|
||||
.avatar img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
border-radius: 50%;
|
||||
}
|
||||
|
||||
.user-info {
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
.username {
|
||||
font-weight: 600;
|
||||
color: #fff;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.timestamp {
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.message-content {
|
||||
color: #e0e0e0;
|
||||
font-size: 14px;
|
||||
line-height: 1.5;
|
||||
margin-bottom: 10px;
|
||||
word-break: break-word;
|
||||
}
|
||||
|
||||
.message-meta {
|
||||
display: flex;
|
||||
gap: 15px;
|
||||
font-size: 12px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.badge {
|
||||
display: inline-block;
|
||||
padding: 2px 8px;
|
||||
background: #333;
|
||||
border-radius: 3px;
|
||||
font-size: 11px;
|
||||
color: #999;
|
||||
}
|
||||
|
||||
.badge.edited {
|
||||
background: #4a3a00;
|
||||
color: #ffd700;
|
||||
}
|
||||
|
||||
.badge.deleted {
|
||||
background: #3a0000;
|
||||
color: #ff6b6b;
|
||||
}
|
||||
|
||||
.image-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(200px, 1fr));
|
||||
gap: 15px;
|
||||
}
|
||||
|
||||
.image-item {
|
||||
background: #1a1a1a;
|
||||
border: 1px solid #333;
|
||||
border-radius: 6px;
|
||||
overflow: hidden;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
|
||||
.image-item:hover {
|
||||
border-color: #555;
|
||||
}
|
||||
|
||||
.image-preview {
|
||||
width: 100%;
|
||||
height: 150px;
|
||||
background: #0a0a0a;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
overflow: hidden;
|
||||
}
|
||||
|
||||
.image-preview img {
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
object-fit: cover;
|
||||
}
|
||||
|
||||
.image-info {
|
||||
padding: 12px;
|
||||
border-top: 1px solid #333;
|
||||
}
|
||||
|
||||
.image-filename {
|
||||
font-size: 12px;
|
||||
color: #e0e0e0;
|
||||
margin-bottom: 8px;
|
||||
word-break: break-all;
|
||||
}
|
||||
|
||||
.image-meta {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
font-size: 11px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.image-url {
|
||||
color: #4ade80;
|
||||
cursor: pointer;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
.image-url:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
.empty {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
.error {
|
||||
background: #3a0000;
|
||||
border: 1px solid #660000;
|
||||
color: #ff6b6b;
|
||||
padding: 12px;
|
||||
border-radius: 4px;
|
||||
margin-bottom: 20px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<header>
|
||||
<h1>🛡️ Moderation Dashboard</h1>
|
||||
<div class="status">
|
||||
<div class="status-item">
|
||||
<div class="status-dot" id="wsStatus"></div>
|
||||
<span id="wsStatusText">Connecting...</span>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<div class="controls">
|
||||
<label for="channelFilter">Filter by Channel:</label>
|
||||
<select id="channelFilter">
|
||||
<option value="">All Channels</option>
|
||||
</select>
|
||||
</div>
|
||||
|
||||
<div class="tabs">
|
||||
<button class="tab active" data-tab="text">💬 Text Messages</button>
|
||||
<button class="tab" data-tab="image">🖼️ Images</button>
|
||||
<button class="tab" data-tab="voice">🎙️ Voice</button>
|
||||
</div>
|
||||
|
||||
<div id="error" class="error" style="display: none;"></div>
|
||||
|
||||
<div id="text" class="content active">
|
||||
<div class="message-list" id="textList"></div>
|
||||
</div>
|
||||
|
||||
<div id="image" class="content">
|
||||
<div class="image-grid" id="imageGrid"></div>
|
||||
</div>
|
||||
|
||||
<div id="voice" class="content">
|
||||
<div class="message-list" id="voiceList"></div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const API_BASE = window.location.origin;
|
||||
let ws = null;
|
||||
let selectedChannel = '';
|
||||
let messageCache = { text: [], image: [], voice: [] };
|
||||
|
||||
function connectWebSocket() {
|
||||
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
|
||||
ws = new WebSocket(`${protocol}//${window.location.host}/ws`);
|
||||
|
||||
ws.onopen = () => {
|
||||
updateWSStatus(true);
|
||||
console.log('WebSocket connected');
|
||||
};
|
||||
|
||||
ws.onmessage = (event) => {
|
||||
try {
|
||||
if (typeof event.data === 'string') {
|
||||
const message = JSON.parse(event.data);
|
||||
handleWebSocketMessage(message);
|
||||
}
|
||||
} catch (e) {
|
||||
console.error('Failed to parse WebSocket message:', e);
|
||||
}
|
||||
};
|
||||
|
||||
ws.onerror = (error) => {
|
||||
console.error('WebSocket error:', error);
|
||||
updateWSStatus(false);
|
||||
};
|
||||
|
||||
ws.onclose = () => {
|
||||
updateWSStatus(false);
|
||||
setTimeout(connectWebSocket, 3000);
|
||||
};
|
||||
}
|
||||
|
||||
function updateWSStatus(connected) {
|
||||
const dot = document.getElementById('wsStatus');
|
||||
const text = document.getElementById('wsStatusText');
|
||||
if (connected) {
|
||||
dot.classList.add('connected');
|
||||
text.textContent = 'Connected';
|
||||
} else {
|
||||
dot.classList.remove('connected');
|
||||
text.textContent = 'Disconnected';
|
||||
}
|
||||
}
|
||||
|
||||
function handleWebSocketMessage(message) {
|
||||
const { type, data } = message;
|
||||
|
||||
if (type === 'message_created') {
|
||||
messageCache.text.unshift(data);
|
||||
renderMessages();
|
||||
} else if (type === 'message_updated') {
|
||||
const msg = messageCache.text.find(m => m.id === data.id);
|
||||
if (msg) {
|
||||
msg.edited_content = data.edited_content;
|
||||
msg.edited_at = data.edited_at;
|
||||
}
|
||||
renderMessages();
|
||||
} else if (type === 'message_deleted') {
|
||||
messageCache.text = messageCache.text.filter(m => m.id !== data.id);
|
||||
renderMessages();
|
||||
} else if (type === 'attachment_uploaded') {
|
||||
messageCache.image.unshift(data);
|
||||
renderImages();
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchMessages() {
|
||||
try {
|
||||
const params = new URLSearchParams();
|
||||
if (selectedChannel) params.append('channel', selectedChannel);
|
||||
params.append('type', 'text');
|
||||
params.append('limit', '50');
|
||||
|
||||
const response = await fetch(`${API_BASE}/api/messages?${params}`);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
|
||||
const result = await response.json();
|
||||
messageCache.text = result.data || [];
|
||||
renderMessages();
|
||||
} catch (error) {
|
||||
showError(`Failed to fetch messages: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchImages() {
|
||||
try {
|
||||
const params = new URLSearchParams();
|
||||
if (selectedChannel) params.append('channel', selectedChannel);
|
||||
params.append('type', 'image');
|
||||
params.append('limit', '50');
|
||||
|
||||
const response = await fetch(`${API_BASE}/api/messages?${params}`);
|
||||
if (!response.ok) throw new Error(`HTTP ${response.status}`);
|
||||
|
||||
const result = await response.json();
|
||||
messageCache.image = result.data || [];
|
||||
renderImages();
|
||||
} catch (error) {
|
||||
showError(`Failed to fetch images: ${error.message}`);
|
||||
}
|
||||
}
|
||||
|
||||
function renderMessages() {
|
||||
const list = document.getElementById('textList');
|
||||
if (messageCache.text.length === 0) {
|
||||
list.innerHTML = '<div class="empty">No messages</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
list.innerHTML = messageCache.text
|
||||
.map(msg => `
|
||||
<div class="message-item">
|
||||
<div class="message-header">
|
||||
<div class="avatar">
|
||||
${msg.avatar_url ? `<img src="${msg.avatar_url}" alt="${msg.username}">` : ''}
|
||||
</div>
|
||||
<div class="user-info">
|
||||
<div class="username">${escapeHtml(msg.username)}</div>
|
||||
<div class="timestamp">${new Date(msg.created_at).toLocaleString()}</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="message-content">${escapeHtml(msg.content)}</div>
|
||||
<div class="message-meta">
|
||||
${msg.type === 'edited' ? '<span class="badge edited">Edited</span>' : ''}
|
||||
${msg.type === 'deleted' ? '<span class="badge deleted">Deleted</span>' : ''}
|
||||
${msg.edited_at ? `<span class="badge">Edited at ${new Date(msg.edited_at).toLocaleString()}</span>` : ''}
|
||||
</div>
|
||||
</div>
|
||||
`)
|
||||
.join('');
|
||||
}
|
||||
|
||||
function renderImages() {
|
||||
const grid = document.getElementById('imageGrid');
|
||||
if (messageCache.image.length === 0) {
|
||||
grid.innerHTML = '<div class="empty">No images</div>';
|
||||
return;
|
||||
}
|
||||
|
||||
grid.innerHTML = messageCache.image
|
||||
.map(img => `
|
||||
<div class="image-item">
|
||||
<div class="image-preview">
|
||||
${img.uploaded_url ? `<img src="${img.uploaded_url}" alt="${img.filename}">` : '<span>Uploading...</span>'}
|
||||
</div>
|
||||
<div class="image-info">
|
||||
<div class="image-filename">${escapeHtml(img.filename)}</div>
|
||||
<div class="image-meta">
|
||||
<span>${(img.size / 1024).toFixed(1)}KB</span>
|
||||
${img.uploaded_url ? `<a href="${img.uploaded_url}" target="_blank" class="image-url">View</a>` : '<span>Pending</span>'}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
`)
|
||||
.join('');
|
||||
}
|
||||
|
||||
function showError(message) {
|
||||
const errorDiv = document.getElementById('error');
|
||||
errorDiv.textContent = message;
|
||||
errorDiv.style.display = 'block';
|
||||
setTimeout(() => {
|
||||
errorDiv.style.display = 'none';
|
||||
}, 5000);
|
||||
}
|
||||
|
||||
function escapeHtml(text) {
|
||||
const div = document.createElement('div');
|
||||
div.textContent = text;
|
||||
return div.innerHTML;
|
||||
}
|
||||
|
||||
document.querySelectorAll('.tab').forEach(tab => {
|
||||
tab.addEventListener('click', () => {
|
||||
document.querySelectorAll('.tab').forEach(t => t.classList.remove('active'));
|
||||
document.querySelectorAll('.content').forEach(c => c.classList.remove('active'));
|
||||
|
||||
tab.classList.add('active');
|
||||
const tabName = tab.dataset.tab;
|
||||
document.getElementById(tabName).classList.add('active');
|
||||
|
||||
if (tabName === 'text') fetchMessages();
|
||||
else if (tabName === 'image') fetchImages();
|
||||
});
|
||||
});
|
||||
|
||||
document.getElementById('channelFilter').addEventListener('change', (e) => {
|
||||
selectedChannel = e.target.value;
|
||||
fetchMessages();
|
||||
fetchImages();
|
||||
});
|
||||
|
||||
connectWebSocket();
|
||||
fetchMessages();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
1430
public/index.html
1430
public/index.html
File diff suppressed because it is too large
Load Diff
@@ -27,6 +27,13 @@ const configSchema = z.object({
|
||||
NODE_ENV: z
|
||||
.enum(["development", "production", "test"])
|
||||
.default("development"),
|
||||
MONITOR_GUILD_ID: z.string().min(1).optional(),
|
||||
PICSER_UPLOAD_URL: z.string().url().default("https://picser.asepharyana.tech/api/upload"),
|
||||
ATTACHMENT_UPLOAD_TIMEOUT_MS: z.coerce.number().positive().default(30000),
|
||||
ATTACHMENT_MAX_SIZE_MB: z.coerce.number().positive().default(100),
|
||||
ATTACHMENT_RETRY_ATTEMPTS: z.coerce.number().positive().default(3),
|
||||
BACKLOG_SYNC_HOURS: z.coerce.number().positive().default(24),
|
||||
BACKLOG_SYNC_BATCH_SIZE: z.coerce.number().int().positive().max(100).default(100),
|
||||
});
|
||||
|
||||
export type AppConfig = z.infer<typeof configSchema>;
|
||||
|
||||
17
src/index.ts
17
src/index.ts
@@ -8,16 +8,18 @@ import { createChildLogger } from "./logger";
|
||||
import { discordPlayer } from "./player";
|
||||
import { VoiceController } from "./voiceController";
|
||||
import { startWebserver } from "./webserver";
|
||||
import { registerMessageCapture } from "./moderation/messageCapture";
|
||||
import { syncBacklogMessages } from "./moderation/backlogSync";
|
||||
import { getDatabase } from "./muxer-queue";
|
||||
|
||||
const logger = createChildLogger("bot");
|
||||
|
||||
const token = config.DISCORD_TOKEN;
|
||||
|
||||
// Inisialisasi selfbot client
|
||||
const client = new Client();
|
||||
const voiceController = new VoiceController(client);
|
||||
const db = getDatabase();
|
||||
|
||||
// Track shutdown state
|
||||
let isShuttingDown = false;
|
||||
|
||||
async function gracefulShutdown(signal: string) {
|
||||
@@ -30,15 +32,12 @@ async function gracefulShutdown(signal: string) {
|
||||
logger.info({ signal }, "Graceful shutdown initiated");
|
||||
|
||||
try {
|
||||
// Step 1: Stop voice connection
|
||||
logger.info("Stopping voice connection...");
|
||||
await voiceController.disconnect();
|
||||
|
||||
// Step 2: Pause player
|
||||
logger.info("Pausing player...");
|
||||
discordPlayer.pause();
|
||||
|
||||
// Step 3: Destroy client
|
||||
logger.info("Destroying Discord client...");
|
||||
try {
|
||||
client.destroy();
|
||||
@@ -56,6 +55,10 @@ async function gracefulShutdown(signal: string) {
|
||||
|
||||
client.on("ready", async () => {
|
||||
logger.info({ user: client.user?.tag }, "Bot logged in");
|
||||
registerMessageCapture(client, db);
|
||||
syncBacklogMessages(client, db).catch((error) => {
|
||||
logger.warn({ error }, "Backlog sync failed");
|
||||
});
|
||||
startWebserver(config.WEBSERVER_PORT, client, voiceController);
|
||||
});
|
||||
|
||||
@@ -63,7 +66,6 @@ client.on("error", (err) => {
|
||||
logger.error({ error: err }, "Client error");
|
||||
});
|
||||
|
||||
// Graceful shutdown handlers
|
||||
process.on("SIGINT", () => {
|
||||
gracefulShutdown("SIGINT");
|
||||
});
|
||||
@@ -72,16 +74,15 @@ process.on("SIGTERM", () => {
|
||||
gracefulShutdown("SIGTERM");
|
||||
});
|
||||
|
||||
// Handle uncaught exceptions
|
||||
process.on("uncaughtException", (err) => {
|
||||
logger.error({ error: err }, "Uncaught exception");
|
||||
gracefulShutdown("uncaughtException");
|
||||
});
|
||||
|
||||
// Handle unhandled promise rejections
|
||||
process.on("unhandledRejection", (reason, promise) => {
|
||||
logger.error({ reason, promise }, "Unhandled rejection");
|
||||
gracefulShutdown("unhandledRejection");
|
||||
});
|
||||
|
||||
client.login(token);
|
||||
|
||||
|
||||
136
src/moderation/attachmentUploader.ts
Normal file
136
src/moderation/attachmentUploader.ts
Normal file
@@ -0,0 +1,136 @@
|
||||
import { createChildLogger } from "../logger";
|
||||
import { config } from "../config";
|
||||
import { retryWithBackoff } from "../retry";
|
||||
import type { SqliteDatabase } from "../muxer-queue";
|
||||
import { updateAttachmentAsUploaded, updateAttachmentAsFailedUpload } from "./messageStore";
|
||||
|
||||
const logger = createChildLogger("attachment-uploader");
|
||||
|
||||
export interface PicserUploadResponse {
|
||||
success: boolean;
|
||||
filename: string;
|
||||
urls: {
|
||||
raw_commit?: string;
|
||||
[key: string]: string | undefined;
|
||||
};
|
||||
size: number;
|
||||
type: string;
|
||||
}
|
||||
|
||||
export interface ParsedUploadResponse {
|
||||
success: boolean;
|
||||
url: string;
|
||||
filename: string;
|
||||
size: number;
|
||||
type: string;
|
||||
}
|
||||
|
||||
export function parseUploadResponse(response: PicserUploadResponse): ParsedUploadResponse {
|
||||
if (!response.success) {
|
||||
throw new Error("Upload failed: success=false");
|
||||
}
|
||||
|
||||
const rawCommitUrl = response.urls.raw_commit;
|
||||
if (!rawCommitUrl) {
|
||||
throw new Error("Upload response missing raw_commit URL");
|
||||
}
|
||||
|
||||
return {
|
||||
success: true,
|
||||
url: rawCommitUrl,
|
||||
filename: response.filename,
|
||||
size: response.size,
|
||||
type: response.type,
|
||||
};
|
||||
}
|
||||
|
||||
export async function uploadAttachmentToPicser(
|
||||
fileBuffer: Buffer,
|
||||
filename: string,
|
||||
): Promise<ParsedUploadResponse> {
|
||||
const formData = new FormData();
|
||||
const blob = new Blob([new Uint8Array(fileBuffer)], { type: "application/octet-stream" });
|
||||
formData.append("file", blob, filename);
|
||||
|
||||
try {
|
||||
const response = await retryWithBackoff(
|
||||
async () => {
|
||||
const res = await fetch(config.PICSER_UPLOAD_URL, {
|
||||
method: "POST",
|
||||
body: formData,
|
||||
signal: AbortSignal.timeout(config.ATTACHMENT_UPLOAD_TIMEOUT_MS),
|
||||
});
|
||||
|
||||
if (!res.ok) {
|
||||
throw new Error(`Upload failed with status ${res.status}`);
|
||||
}
|
||||
|
||||
return res.json() as Promise<PicserUploadResponse>;
|
||||
},
|
||||
{
|
||||
retries: config.ATTACHMENT_RETRY_ATTEMPTS,
|
||||
minTimeout: 1000,
|
||||
maxTimeout: 5000,
|
||||
logger,
|
||||
},
|
||||
);
|
||||
|
||||
const parsed = parseUploadResponse(response);
|
||||
logger.info({ filename, url: parsed.url }, "Attachment uploaded successfully");
|
||||
return parsed;
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ filename, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to upload attachment",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function downloadDiscordAttachment(url: string): Promise<Buffer> {
|
||||
try {
|
||||
const response = await fetch(url, {
|
||||
signal: AbortSignal.timeout(config.ATTACHMENT_UPLOAD_TIMEOUT_MS),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`Download failed with status ${response.status}`);
|
||||
}
|
||||
|
||||
const buffer = await response.arrayBuffer();
|
||||
return Buffer.from(buffer);
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ url, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to download Discord attachment",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export async function processAttachmentUpload(
|
||||
db: SqliteDatabase,
|
||||
attachmentId: string,
|
||||
discordUrl: string,
|
||||
filename: string,
|
||||
): Promise<void> {
|
||||
try {
|
||||
logger.info({ attachmentId, filename }, "Starting attachment upload");
|
||||
|
||||
const buffer = await downloadDiscordAttachment(discordUrl);
|
||||
|
||||
const sizeMb = buffer.length / (1024 * 1024);
|
||||
if (sizeMb > config.ATTACHMENT_MAX_SIZE_MB) {
|
||||
throw new Error(`File size ${sizeMb.toFixed(2)}MB exceeds limit of ${config.ATTACHMENT_MAX_SIZE_MB}MB`);
|
||||
}
|
||||
|
||||
const result = await uploadAttachmentToPicser(buffer, filename);
|
||||
|
||||
updateAttachmentAsUploaded(db, attachmentId, result.url, Date.now());
|
||||
logger.info({ attachmentId, uploadedUrl: result.url }, "Attachment upload completed");
|
||||
} catch (error) {
|
||||
const errorMsg = error instanceof Error ? error.message : String(error);
|
||||
updateAttachmentAsFailedUpload(db, attachmentId, errorMsg);
|
||||
logger.error({ attachmentId, error: errorMsg }, "Attachment upload failed");
|
||||
}
|
||||
}
|
||||
120
src/moderation/backlogSync.ts
Normal file
120
src/moderation/backlogSync.ts
Normal file
@@ -0,0 +1,120 @@
|
||||
import type { Client, Message } from "discord.js-selfbot-v13";
|
||||
import { config } from "../config";
|
||||
import { createChildLogger } from "../logger";
|
||||
import type { SqliteDatabase } from "../muxer-queue";
|
||||
import { captureMessage } from "./messageCapture";
|
||||
|
||||
const logger = createChildLogger("backlog-sync");
|
||||
|
||||
function isWatchableChannel(channel: { type?: string; messages?: unknown }): boolean {
|
||||
return Boolean(
|
||||
channel.messages &&
|
||||
["GUILD_TEXT", "GUILD_PUBLIC_THREAD", "GUILD_PRIVATE_THREAD"].includes(
|
||||
channel.type ?? "",
|
||||
),
|
||||
);
|
||||
}
|
||||
|
||||
async function collectWatchableChannels(guild: any): Promise<any[]> {
|
||||
const channels: any[] = [];
|
||||
for (const channel of guild.channels.cache.values()) {
|
||||
if (isWatchableChannel(channel)) {
|
||||
channels.push(channel);
|
||||
}
|
||||
|
||||
if (channel.threads?.fetch) {
|
||||
for (const archived of [false, true]) {
|
||||
const fetched = await channel.threads
|
||||
.fetch({ archived, limit: 100 })
|
||||
.catch(() => null);
|
||||
if (!fetched?.threads) continue;
|
||||
for (const thread of fetched.threads.values()) {
|
||||
if (isWatchableChannel(thread)) channels.push(thread);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(new Map(channels.map((channel) => [channel.id, channel])).values());
|
||||
}
|
||||
|
||||
async function syncChannelMessages(
|
||||
db: SqliteDatabase,
|
||||
channel: any,
|
||||
cutoffTime: number,
|
||||
): Promise<number> {
|
||||
let before: string | undefined;
|
||||
let synced = 0;
|
||||
let shouldContinue = true;
|
||||
|
||||
while (shouldContinue) {
|
||||
const batch = await channel.messages.fetch({
|
||||
limit: config.BACKLOG_SYNC_BATCH_SIZE,
|
||||
...(before ? { before } : {}),
|
||||
});
|
||||
|
||||
if (batch.size === 0) break;
|
||||
|
||||
const messages = Array.from(batch.values()) as Message[];
|
||||
for (const message of messages) {
|
||||
if (message.author?.bot) continue;
|
||||
if (message.createdTimestamp < cutoffTime) {
|
||||
shouldContinue = false;
|
||||
continue;
|
||||
}
|
||||
|
||||
await captureMessage(db, message, "text");
|
||||
synced++;
|
||||
}
|
||||
|
||||
before = messages[messages.length - 1]?.id;
|
||||
if (!before || batch.size < config.BACKLOG_SYNC_BATCH_SIZE) break;
|
||||
}
|
||||
|
||||
return synced;
|
||||
}
|
||||
|
||||
export async function syncBacklogMessages(
|
||||
client: Client,
|
||||
db: SqliteDatabase,
|
||||
): Promise<void> {
|
||||
if (!config.MONITOR_GUILD_ID) {
|
||||
logger.warn("MONITOR_GUILD_ID not configured, skipping backlog sync");
|
||||
return;
|
||||
}
|
||||
|
||||
const guild = client.guilds.cache.get(config.MONITOR_GUILD_ID);
|
||||
if (!guild) {
|
||||
logger.warn({ guildId: config.MONITOR_GUILD_ID }, "Monitor guild not found, skipping backlog sync");
|
||||
return;
|
||||
}
|
||||
|
||||
const cutoffTime = Date.now() - config.BACKLOG_SYNC_HOURS * 60 * 60 * 1000;
|
||||
await guild.channels.fetch().catch(() => null);
|
||||
|
||||
const channels = await collectWatchableChannels(guild);
|
||||
|
||||
let total = 0;
|
||||
logger.info(
|
||||
{ guildId: guild.id, channels: channels.length, hours: config.BACKLOG_SYNC_HOURS },
|
||||
"Starting message backlog sync",
|
||||
);
|
||||
|
||||
for (const channel of channels) {
|
||||
try {
|
||||
const count = await syncChannelMessages(db, channel as any, cutoffTime);
|
||||
total += count;
|
||||
logger.info({ channelId: channel.id, count }, "Backlog channel sync completed");
|
||||
} catch (error) {
|
||||
logger.warn(
|
||||
{
|
||||
channelId: channel.id,
|
||||
error: error instanceof Error ? error.message : String(error),
|
||||
},
|
||||
"Backlog channel sync failed",
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
logger.info({ total }, "Message backlog sync completed");
|
||||
}
|
||||
176
src/moderation/messageCapture.ts
Normal file
176
src/moderation/messageCapture.ts
Normal file
@@ -0,0 +1,176 @@
|
||||
import type { Client, Message } from "discord.js-selfbot-v13";
|
||||
import { createChildLogger } from "../logger";
|
||||
import type { SqliteDatabase } from "../muxer-queue";
|
||||
import { config } from "../config";
|
||||
import { insertMessage, insertAttachment } from "./messageStore";
|
||||
import { processAttachmentUpload } from "./attachmentUploader";
|
||||
import { getDisplayContent, getMessageLocation, getMessageMetadata } from "./messageMetadata";
|
||||
import type { MessageRecord, AttachmentRecord } from "./types";
|
||||
|
||||
const logger = createChildLogger("message-capture");
|
||||
|
||||
export async function captureMessage(
|
||||
db: SqliteDatabase,
|
||||
message: Message,
|
||||
type: "text" | "edited" | "deleted",
|
||||
): Promise<void> {
|
||||
const location = getMessageLocation(message);
|
||||
const metadata = getMessageMetadata(message);
|
||||
|
||||
const messageRecord: MessageRecord = {
|
||||
id: message.id,
|
||||
guild_id: message.guildId!,
|
||||
channel_id: location.channelId,
|
||||
thread_id: location.threadId,
|
||||
user_id: message.author!.id,
|
||||
username: message.author!.username,
|
||||
avatar_url: message.author!.avatarURL() || null,
|
||||
content: getDisplayContent(message),
|
||||
edited_content: null,
|
||||
created_at: message.createdTimestamp,
|
||||
edited_at: null,
|
||||
deleted_at: null,
|
||||
type,
|
||||
metadata: JSON.stringify(metadata),
|
||||
};
|
||||
|
||||
insertMessage(db, messageRecord);
|
||||
|
||||
const broadcaster = globalThis as any;
|
||||
if (broadcaster.broadcastMessageCreated) {
|
||||
broadcaster.broadcastMessageCreated({
|
||||
...messageRecord,
|
||||
type: "text",
|
||||
});
|
||||
}
|
||||
|
||||
if (message.attachments.size > 0) {
|
||||
for (const [, attachment] of message.attachments) {
|
||||
const attachmentRecord: AttachmentRecord = {
|
||||
id: attachment.id,
|
||||
message_id: message.id,
|
||||
guild_id: message.guildId!,
|
||||
channel_id: location.channelId,
|
||||
thread_id: location.threadId,
|
||||
user_id: message.author!.id,
|
||||
filename: attachment.name || "unknown",
|
||||
size: attachment.size,
|
||||
type: attachment.contentType || "application/octet-stream",
|
||||
discord_url: attachment.url,
|
||||
uploaded_url: null,
|
||||
upload_status: "pending",
|
||||
upload_error: null,
|
||||
created_at: Date.now(),
|
||||
uploaded_at: null,
|
||||
};
|
||||
|
||||
insertAttachment(db, attachmentRecord);
|
||||
|
||||
processAttachmentUpload(db, attachment.id, attachment.url, attachment.name || "unknown")
|
||||
.then(() => {
|
||||
if (broadcaster.broadcastAttachmentUploaded) {
|
||||
broadcaster.broadcastAttachmentUploaded({
|
||||
id: attachment.id,
|
||||
message_id: message.id,
|
||||
filename: attachment.name || "unknown",
|
||||
channel_id: location.channelId,
|
||||
created_at: Date.now(),
|
||||
});
|
||||
}
|
||||
})
|
||||
.catch((error) => {
|
||||
logger.error(
|
||||
{ attachmentId: attachment.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Background attachment upload failed",
|
||||
);
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
logger.info(
|
||||
{
|
||||
messageId: message.id,
|
||||
channelId: message.channelId,
|
||||
attachmentCount: message.attachments.size,
|
||||
},
|
||||
"Message captured",
|
||||
);
|
||||
}
|
||||
|
||||
export function registerMessageCapture(client: Client, db: SqliteDatabase): void {
|
||||
client.on("messageCreate", async (message) => {
|
||||
if (!message.guildId || message.guildId !== config.MONITOR_GUILD_ID) return;
|
||||
if (message.author?.bot) return;
|
||||
|
||||
try {
|
||||
await captureMessage(db, message, "text");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId: message.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to capture message",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
client.on("messageUpdate", async (_oldMessage, newMessage) => {
|
||||
if (!newMessage.guildId || newMessage.guildId !== config.MONITOR_GUILD_ID) return;
|
||||
if (newMessage.author?.bot) return;
|
||||
|
||||
try {
|
||||
const { updateMessageAsEdited } = await import("./messageStore");
|
||||
|
||||
const existing = db
|
||||
.prepare("SELECT id FROM messages WHERE id = ?")
|
||||
.get(newMessage.id) as { id: string } | undefined;
|
||||
|
||||
if (existing) {
|
||||
const editedAt = Date.now();
|
||||
updateMessageAsEdited(db, newMessage.id, getDisplayContent(newMessage as Message), editedAt);
|
||||
|
||||
const broadcaster = globalThis as any;
|
||||
if (broadcaster.broadcastMessageUpdated) {
|
||||
broadcaster.broadcastMessageUpdated({
|
||||
id: newMessage.id,
|
||||
edited_content: getDisplayContent(newMessage as Message),
|
||||
edited_at: editedAt,
|
||||
});
|
||||
}
|
||||
} else if (newMessage.author) {
|
||||
await captureMessage(db, newMessage as Message, "text");
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId: newMessage.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to capture message update",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
client.on("messageDelete", async (message) => {
|
||||
if (!message.guildId || message.guildId !== config.MONITOR_GUILD_ID) return;
|
||||
if (!message.author) return;
|
||||
|
||||
try {
|
||||
const { updateMessageAsDeleted } = await import("./messageStore");
|
||||
const deletedAt = Date.now();
|
||||
updateMessageAsDeleted(db, message.id, deletedAt);
|
||||
|
||||
const broadcaster = globalThis as any;
|
||||
if (broadcaster.broadcastMessageDeleted) {
|
||||
broadcaster.broadcastMessageDeleted({
|
||||
id: message.id,
|
||||
deleted_at: deletedAt,
|
||||
});
|
||||
}
|
||||
|
||||
logger.info({ messageId: message.id }, "Message deletion captured");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId: message.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to capture message deletion",
|
||||
);
|
||||
}
|
||||
});
|
||||
|
||||
logger.info("Message capture handlers registered");
|
||||
}
|
||||
167
src/moderation/messageMetadata.ts
Normal file
167
src/moderation/messageMetadata.ts
Normal file
@@ -0,0 +1,167 @@
|
||||
import type { Message, TextChannel, ThreadChannel } from "discord.js-selfbot-v13";
|
||||
|
||||
export interface MessageLocation {
|
||||
channelId: string;
|
||||
threadId: string | null;
|
||||
threadName: string | null;
|
||||
channelName: string | null;
|
||||
}
|
||||
|
||||
export interface RichMessageMetadata {
|
||||
stickers: Array<{ id: string; name: string; url: string; format: string | null }>;
|
||||
embeds: Array<{
|
||||
title: string | null;
|
||||
description: string | null;
|
||||
url: string | null;
|
||||
color: number | null;
|
||||
image: string | null;
|
||||
thumbnail: string | null;
|
||||
author: { name: string | null; url: string | null; iconURL: string | null } | null;
|
||||
footer: { text: string | null; iconURL: string | null } | null;
|
||||
fields: Array<{ name: string; value: string; inline: boolean }>;
|
||||
}>;
|
||||
attachments: Array<{
|
||||
id: string;
|
||||
name: string;
|
||||
url: string;
|
||||
contentType: string | null;
|
||||
size: number;
|
||||
}>;
|
||||
author: {
|
||||
id: string;
|
||||
username: string;
|
||||
tag: string | null;
|
||||
avatarURL: string | null;
|
||||
bot: boolean;
|
||||
};
|
||||
member: {
|
||||
displayName: string | null;
|
||||
roles: Array<{ id: string; name: string }>;
|
||||
joinedTimestamp: number | null;
|
||||
} | null;
|
||||
channel: MessageLocation;
|
||||
reference: {
|
||||
messageId: string | null;
|
||||
channelId: string | null;
|
||||
guildId: string | null;
|
||||
} | null;
|
||||
}
|
||||
|
||||
export function getMessageLocation(message: Message): MessageLocation {
|
||||
const channel = message.channel as TextChannel | ThreadChannel;
|
||||
if (!channel.isThread?.()) {
|
||||
return {
|
||||
channelId: message.channelId,
|
||||
threadId: null,
|
||||
threadName: null,
|
||||
channelName: "name" in channel ? channel.name : null,
|
||||
};
|
||||
}
|
||||
|
||||
return {
|
||||
channelId: channel.parentId ?? message.channelId,
|
||||
threadId: channel.id,
|
||||
threadName: channel.name,
|
||||
channelName: channel.parent?.name ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
export function getStickerMetadata(message: Message): RichMessageMetadata["stickers"] {
|
||||
return Array.from(message.stickers.values()).map((sticker) => ({
|
||||
id: sticker.id,
|
||||
name: sticker.name,
|
||||
url: sticker.url,
|
||||
format: sticker.format ?? null,
|
||||
}));
|
||||
}
|
||||
|
||||
export function getAttachmentMetadata(message: Message): RichMessageMetadata["attachments"] {
|
||||
return Array.from(message.attachments.values()).map((attachment) => ({
|
||||
id: attachment.id,
|
||||
name: attachment.name || "unknown",
|
||||
url: attachment.url,
|
||||
contentType: attachment.contentType ?? null,
|
||||
size: attachment.size,
|
||||
}));
|
||||
}
|
||||
|
||||
export function getEmbedMetadata(message: Message): RichMessageMetadata["embeds"] {
|
||||
return message.embeds.map((embed) => ({
|
||||
title: embed.title ?? null,
|
||||
description: embed.description ?? null,
|
||||
url: embed.url ?? null,
|
||||
color: embed.color ?? null,
|
||||
image: embed.image?.url ?? null,
|
||||
thumbnail: embed.thumbnail?.url ?? null,
|
||||
author: embed.author
|
||||
? {
|
||||
name: embed.author.name ?? null,
|
||||
url: embed.author.url ?? null,
|
||||
iconURL: embed.author.iconURL ?? null,
|
||||
}
|
||||
: null,
|
||||
footer: embed.footer
|
||||
? {
|
||||
text: embed.footer.text ?? null,
|
||||
iconURL: embed.footer.iconURL ?? null,
|
||||
}
|
||||
: null,
|
||||
fields: embed.fields.map((field) => ({
|
||||
name: field.name,
|
||||
value: field.value,
|
||||
inline: Boolean(field.inline),
|
||||
})),
|
||||
}));
|
||||
}
|
||||
|
||||
export function getMessageMetadata(message: Message): RichMessageMetadata {
|
||||
const member = message.member;
|
||||
return {
|
||||
stickers: getStickerMetadata(message),
|
||||
embeds: getEmbedMetadata(message),
|
||||
attachments: getAttachmentMetadata(message),
|
||||
author: {
|
||||
id: message.author.id,
|
||||
username: message.author.username,
|
||||
tag: "tag" in message.author ? message.author.tag : null,
|
||||
avatarURL: message.author.avatarURL() ?? null,
|
||||
bot: Boolean(message.author.bot),
|
||||
},
|
||||
member: member
|
||||
? {
|
||||
displayName: member.displayName ?? null,
|
||||
roles: member.roles.cache.map((role) => ({ id: role.id, name: role.name })),
|
||||
joinedTimestamp: member.joinedTimestamp ?? null,
|
||||
}
|
||||
: null,
|
||||
channel: getMessageLocation(message),
|
||||
reference: message.reference
|
||||
? {
|
||||
messageId: message.reference.messageId ?? null,
|
||||
channelId: message.reference.channelId ?? null,
|
||||
guildId: message.reference.guildId ?? null,
|
||||
}
|
||||
: null,
|
||||
};
|
||||
}
|
||||
|
||||
export function getDisplayContent(message: Message): string {
|
||||
if (message.content.trim().length > 0) return message.content;
|
||||
|
||||
const stickers = getStickerMetadata(message);
|
||||
if (stickers.length > 0) {
|
||||
return stickers.map((sticker) => `[Sticker: ${sticker.name}]`).join(" ");
|
||||
}
|
||||
|
||||
const attachments = getAttachmentMetadata(message);
|
||||
if (attachments.length > 0) {
|
||||
return attachments.map((attachment) => `[Attachment: ${attachment.name}]`).join(" ");
|
||||
}
|
||||
|
||||
const embeds = getEmbedMetadata(message);
|
||||
if (embeds.length > 0) {
|
||||
return embeds.map((embed) => embed.title || embed.description || "[Embed]").join(" ");
|
||||
}
|
||||
|
||||
return "";
|
||||
}
|
||||
222
src/moderation/messageStore.ts
Normal file
222
src/moderation/messageStore.ts
Normal file
@@ -0,0 +1,222 @@
|
||||
import { createChildLogger } from "../logger";
|
||||
import type { SqliteDatabase } from "../muxer-queue";
|
||||
import type { MessageRecord, AttachmentRecord } from "./types";
|
||||
|
||||
const logger = createChildLogger("message-store");
|
||||
|
||||
export function insertMessage(db: SqliteDatabase, message: MessageRecord): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
INSERT OR IGNORE INTO messages (
|
||||
id, guild_id, channel_id, thread_id, user_id, username, avatar_url,
|
||||
content, edited_content, created_at, edited_at, deleted_at, type, metadata
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
stmt.run(
|
||||
message.id,
|
||||
message.guild_id,
|
||||
message.channel_id,
|
||||
message.thread_id,
|
||||
message.user_id,
|
||||
message.username,
|
||||
message.avatar_url,
|
||||
message.content,
|
||||
message.edited_content,
|
||||
message.created_at,
|
||||
message.edited_at,
|
||||
message.deleted_at,
|
||||
message.type,
|
||||
message.metadata,
|
||||
);
|
||||
|
||||
logger.debug({ messageId: message.id, channelId: message.channel_id }, "Message inserted");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId: message.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to insert message",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function updateMessageAsEdited(
|
||||
db: SqliteDatabase,
|
||||
messageId: string,
|
||||
editedContent: string,
|
||||
editedAt: number,
|
||||
): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
UPDATE messages
|
||||
SET edited_content = ?, edited_at = ?, type = 'edited'
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(editedContent, editedAt, messageId);
|
||||
logger.debug({ messageId }, "Message marked as edited");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to update message as edited",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function updateMessageAsDeleted(
|
||||
db: SqliteDatabase,
|
||||
messageId: string,
|
||||
deletedAt: number,
|
||||
): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
UPDATE messages
|
||||
SET deleted_at = ?, type = 'deleted'
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(deletedAt, messageId);
|
||||
logger.debug({ messageId }, "Message marked as deleted");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ messageId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to update message as deleted",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function getMessagesByChannel(
|
||||
db: SqliteDatabase,
|
||||
channelId: string,
|
||||
limit: number = 50,
|
||||
offset: number = 0,
|
||||
): MessageRecord[] {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
SELECT * FROM messages
|
||||
WHERE channel_id = ? OR thread_id = ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`);
|
||||
|
||||
const rows = stmt.all(channelId, channelId, limit, offset) as MessageRecord[];
|
||||
return rows;
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ channelId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to get messages by channel",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function insertAttachment(db: SqliteDatabase, attachment: AttachmentRecord): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
INSERT OR IGNORE INTO attachments (
|
||||
id, message_id, guild_id, channel_id, thread_id, user_id, filename, size, type,
|
||||
discord_url, uploaded_url, upload_status, upload_error, created_at, uploaded_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
`);
|
||||
|
||||
stmt.run(
|
||||
attachment.id,
|
||||
attachment.message_id,
|
||||
attachment.guild_id,
|
||||
attachment.channel_id,
|
||||
attachment.thread_id,
|
||||
attachment.user_id,
|
||||
attachment.filename,
|
||||
attachment.size,
|
||||
attachment.type,
|
||||
attachment.discord_url,
|
||||
attachment.uploaded_url,
|
||||
attachment.upload_status,
|
||||
attachment.upload_error,
|
||||
attachment.created_at,
|
||||
attachment.uploaded_at,
|
||||
);
|
||||
|
||||
logger.debug({ attachmentId: attachment.id, messageId: attachment.message_id }, "Attachment inserted");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ attachmentId: attachment.id, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to insert attachment",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function getAttachmentsByChannel(
|
||||
db: SqliteDatabase,
|
||||
channelId: string,
|
||||
limit: number = 50,
|
||||
offset: number = 0,
|
||||
): AttachmentRecord[] {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
SELECT * FROM attachments
|
||||
WHERE channel_id = ? OR thread_id = ?
|
||||
ORDER BY created_at DESC
|
||||
LIMIT ? OFFSET ?
|
||||
`);
|
||||
|
||||
const rows = stmt.all(channelId, channelId, limit, offset) as AttachmentRecord[];
|
||||
return rows;
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ channelId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to get attachments by channel",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function updateAttachmentAsUploaded(
|
||||
db: SqliteDatabase,
|
||||
attachmentId: string,
|
||||
uploadedUrl: string,
|
||||
uploadedAt: number,
|
||||
): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
UPDATE attachments
|
||||
SET uploaded_url = ?, upload_status = 'uploaded', uploaded_at = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(uploadedUrl, uploadedAt, attachmentId);
|
||||
logger.debug({ attachmentId, uploadedUrl }, "Attachment marked as uploaded");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ attachmentId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to update attachment as uploaded",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export function updateAttachmentAsFailedUpload(
|
||||
db: SqliteDatabase,
|
||||
attachmentId: string,
|
||||
error: string,
|
||||
): void {
|
||||
try {
|
||||
const stmt = db.prepare(`
|
||||
UPDATE attachments
|
||||
SET upload_status = 'failed', upload_error = ?
|
||||
WHERE id = ?
|
||||
`);
|
||||
|
||||
stmt.run(error, attachmentId);
|
||||
logger.debug({ attachmentId, error }, "Attachment marked as failed upload");
|
||||
} catch (error) {
|
||||
logger.error(
|
||||
{ attachmentId, error: error instanceof Error ? error.message : String(error) },
|
||||
"Failed to update attachment as failed",
|
||||
);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
56
src/moderation/types.ts
Normal file
56
src/moderation/types.ts
Normal file
@@ -0,0 +1,56 @@
|
||||
export interface MessageRecord {
|
||||
id: string;
|
||||
guild_id: string;
|
||||
channel_id: string;
|
||||
thread_id: string | null;
|
||||
user_id: string;
|
||||
username: string;
|
||||
avatar_url: string | null;
|
||||
content: string;
|
||||
edited_content: string | null;
|
||||
created_at: number;
|
||||
edited_at: number | null;
|
||||
deleted_at: number | null;
|
||||
type: "text" | "edited" | "deleted";
|
||||
metadata: string | null;
|
||||
}
|
||||
|
||||
export interface AttachmentRecord {
|
||||
id: string;
|
||||
message_id: string;
|
||||
guild_id: string;
|
||||
channel_id: string;
|
||||
thread_id: string | null;
|
||||
user_id: string;
|
||||
filename: string;
|
||||
size: number;
|
||||
type: string;
|
||||
discord_url: string;
|
||||
uploaded_url: string | null;
|
||||
upload_status: "pending" | "uploaded" | "failed";
|
||||
upload_error: string | null;
|
||||
created_at: number;
|
||||
uploaded_at: number | null;
|
||||
}
|
||||
|
||||
export interface VoiceSegmentRecord {
|
||||
id: string;
|
||||
user_id: string;
|
||||
session_id: string;
|
||||
guild_id: string;
|
||||
channel_id: string;
|
||||
filename: string;
|
||||
duration_ms: number;
|
||||
created_at: number;
|
||||
}
|
||||
|
||||
export interface DashboardMessage {
|
||||
id: string;
|
||||
channel_id: string;
|
||||
user_id: string;
|
||||
username: string;
|
||||
avatar_url: string | null;
|
||||
content: string;
|
||||
created_at: number;
|
||||
type: "text" | "image" | "voice";
|
||||
}
|
||||
@@ -1,9 +1,21 @@
|
||||
import path from "node:path";
|
||||
import Database from "better-sqlite3";
|
||||
import { Database } from "bun:sqlite";
|
||||
import { createChildLogger } from "./logger";
|
||||
|
||||
const logger = createChildLogger("muxer-queue");
|
||||
|
||||
export interface SqliteStatement {
|
||||
run: (...params: unknown[]) => { changes: number };
|
||||
all: (...params: unknown[]) => unknown[];
|
||||
get: (...params: unknown[]) => unknown;
|
||||
}
|
||||
|
||||
export interface SqliteDatabase {
|
||||
prepare: (sql: string) => SqliteStatement;
|
||||
exec: (sql: string) => void;
|
||||
close: () => void;
|
||||
}
|
||||
|
||||
export interface MuxerJobData {
|
||||
userId: string;
|
||||
sessionId: string;
|
||||
@@ -23,13 +35,14 @@ interface StoredJob {
|
||||
}
|
||||
|
||||
const dbPath = path.join(process.cwd(), ".muxer-queue.db");
|
||||
let db: Database.Database | null = null;
|
||||
let db: SqliteDatabase | null = null;
|
||||
|
||||
function initializeDatabase(): Database.Database {
|
||||
const database = new Database(dbPath);
|
||||
database.pragma("journal_mode = WAL");
|
||||
function initializeDatabase(): SqliteDatabase {
|
||||
const database = new Database(dbPath) as SqliteDatabase;
|
||||
|
||||
database.exec(`
|
||||
PRAGMA journal_mode = WAL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS muxer_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
data TEXT NOT NULL,
|
||||
@@ -43,18 +56,71 @@ function initializeDatabase(): Database.Database {
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_status ON muxer_jobs(status);
|
||||
CREATE INDEX IF NOT EXISTS idx_createdAt ON muxer_jobs(createdAt);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS messages (
|
||||
id TEXT PRIMARY KEY,
|
||||
guild_id TEXT NOT NULL,
|
||||
channel_id TEXT NOT NULL,
|
||||
thread_id TEXT,
|
||||
user_id TEXT NOT NULL,
|
||||
username TEXT NOT NULL,
|
||||
avatar_url TEXT,
|
||||
content TEXT NOT NULL,
|
||||
edited_content TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
edited_at INTEGER,
|
||||
deleted_at INTEGER,
|
||||
type TEXT NOT NULL DEFAULT 'text',
|
||||
metadata TEXT
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_channel ON messages(channel_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_user ON messages(user_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_created ON messages(created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_messages_thread ON messages(thread_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS attachments (
|
||||
id TEXT PRIMARY KEY,
|
||||
message_id TEXT NOT NULL,
|
||||
guild_id TEXT NOT NULL,
|
||||
channel_id TEXT NOT NULL,
|
||||
thread_id TEXT,
|
||||
user_id TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
size INTEGER NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
discord_url TEXT NOT NULL,
|
||||
uploaded_url TEXT,
|
||||
upload_status TEXT NOT NULL DEFAULT 'pending',
|
||||
upload_error TEXT,
|
||||
created_at INTEGER NOT NULL,
|
||||
uploaded_at INTEGER,
|
||||
FOREIGN KEY (message_id) REFERENCES messages(id)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_attachments_channel ON attachments(channel_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_attachments_message ON attachments(message_id);
|
||||
CREATE INDEX IF NOT EXISTS idx_attachments_status ON attachments(upload_status);
|
||||
`);
|
||||
|
||||
try {
|
||||
database.exec("ALTER TABLE attachments ADD COLUMN thread_id TEXT");
|
||||
} catch {
|
||||
// Column already exists on databases initialized after the moderation schema was added.
|
||||
}
|
||||
|
||||
return database;
|
||||
}
|
||||
|
||||
function getDatabase(): Database.Database {
|
||||
function getDatabase(): SqliteDatabase {
|
||||
if (!db) {
|
||||
db = initializeDatabase();
|
||||
}
|
||||
return db;
|
||||
}
|
||||
|
||||
export { getDatabase };
|
||||
|
||||
export async function enqueueMuxerJob(data: MuxerJobData): Promise<string> {
|
||||
try {
|
||||
const database = getDatabase();
|
||||
|
||||
@@ -1,6 +1,34 @@
|
||||
import { createRequire } from "node:module";
|
||||
import prism from "prism-media";
|
||||
import { config } from "../config";
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
|
||||
interface OpusDecoderRuntime {
|
||||
isBun: boolean;
|
||||
canLoadNativeOpus: boolean;
|
||||
}
|
||||
|
||||
export function shouldEnableDefaultOpusDecoder(
|
||||
runtime: OpusDecoderRuntime,
|
||||
): boolean {
|
||||
return !runtime.isBun || runtime.canLoadNativeOpus;
|
||||
}
|
||||
|
||||
function canLoadNativeOpus(): boolean {
|
||||
try {
|
||||
require("@discordjs/opus");
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
const defaultDecoderEnabled = shouldEnableDefaultOpusDecoder({
|
||||
isBun: Boolean(process.versions.bun),
|
||||
canLoadNativeOpus: canLoadNativeOpus(),
|
||||
});
|
||||
|
||||
export interface OpusDecoderOptions {
|
||||
cooldownMs: number;
|
||||
rotateMs: number;
|
||||
@@ -23,8 +51,14 @@ export class OpusDecoder {
|
||||
this.onData = options.onData;
|
||||
this.createDecoderFn =
|
||||
options.createDecoder ??
|
||||
(() =>
|
||||
new prism.opus.Decoder({
|
||||
(() => {
|
||||
if (!defaultDecoderEnabled) {
|
||||
throw new Error(
|
||||
"Native @discordjs/opus is unavailable under Bun; web PCM decode disabled to avoid opusscript aborts",
|
||||
);
|
||||
}
|
||||
|
||||
return new prism.opus.Decoder({
|
||||
frameSize: config.OPUS_FRAME_SIZE,
|
||||
channels: config.AUDIO_CHANNELS as 1 | 2,
|
||||
rate: config.AUDIO_SAMPLE_RATE as
|
||||
@@ -33,7 +67,8 @@ export class OpusDecoder {
|
||||
| 16000
|
||||
| 24000
|
||||
| 48000,
|
||||
}));
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
rotateIfNeeded(): void {
|
||||
|
||||
@@ -25,6 +25,12 @@ export interface VoiceChannelSummary {
|
||||
name: string;
|
||||
}
|
||||
|
||||
export interface ChannelSummary {
|
||||
id: string;
|
||||
name: string;
|
||||
type: string;
|
||||
}
|
||||
|
||||
export class VoiceController {
|
||||
private activeGuildId: string | null = null;
|
||||
private activeChannelId: string | null = null;
|
||||
@@ -63,6 +69,49 @@ export class VoiceController {
|
||||
.sort((a, b) => a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
async listWatchableChannels(guildId: string): Promise<ChannelSummary[]> {
|
||||
const guild = this.getGuild(guildId);
|
||||
await guild.channels.fetch().catch(() => null);
|
||||
|
||||
return guild.channels.cache
|
||||
.filter((channel) => channel.type === "GUILD_TEXT")
|
||||
.map((channel) => ({
|
||||
id: channel.id,
|
||||
name: channel.name,
|
||||
type: channel.type,
|
||||
}))
|
||||
.sort((a, b) => a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
async listThreads(guildId: string): Promise<ChannelSummary[]> {
|
||||
const guild = this.getGuild(guildId);
|
||||
await guild.channels.fetch().catch(() => null);
|
||||
|
||||
const threads: ChannelSummary[] = [];
|
||||
for (const channel of guild.channels.cache.values()) {
|
||||
const threadParent = channel as typeof channel & {
|
||||
threads?: { fetch: (options: { archived: boolean; limit: number }) => Promise<any> };
|
||||
};
|
||||
if (!threadParent.threads?.fetch) continue;
|
||||
|
||||
for (const archived of [false, true]) {
|
||||
const fetched = await threadParent.threads.fetch({ archived, limit: 100 }).catch(() => null);
|
||||
if (!fetched?.threads) continue;
|
||||
|
||||
for (const thread of fetched.threads.values()) {
|
||||
threads.push({
|
||||
id: thread.id,
|
||||
name: `${channel.name} / ${thread.name}`,
|
||||
type: thread.type,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return Array.from(new Map(threads.map((thread) => [thread.id, thread])).values())
|
||||
.sort((a, b) => a.name.localeCompare(b.name));
|
||||
}
|
||||
|
||||
async connect(guildId: string, channelId: string): Promise<VoiceStatus> {
|
||||
if (!this.client.isReady()) {
|
||||
throw new AppError(
|
||||
|
||||
@@ -11,6 +11,8 @@ import { createChildLogger, logger } from "./logger";
|
||||
import { getMetrics, uptimeGauge } from "./metrics";
|
||||
import { discordPlayer } from "./player";
|
||||
import type { VoiceController } from "./voiceController";
|
||||
import { getDatabase } from "./muxer-queue";
|
||||
import { getMessagesByChannel, getAttachmentsByChannel } from "./moderation/messageStore";
|
||||
|
||||
const wsLogger = createChildLogger("webserver");
|
||||
|
||||
@@ -57,8 +59,12 @@ export function startWebserver(
|
||||
const wss = new WebSocketServer({ server, path: wsPath });
|
||||
wsLogger.info({ port, wsPath }, "WebSocket server listening");
|
||||
|
||||
// Security headers
|
||||
app.use(helmet());
|
||||
// Security headers. CSP disabled because the current static UI uses inline scripts/styles.
|
||||
app.use(
|
||||
helmet({
|
||||
contentSecurityPolicy: false,
|
||||
}),
|
||||
);
|
||||
|
||||
// HTTP request logging
|
||||
app.use(pinoHttp({ logger }));
|
||||
@@ -100,6 +106,22 @@ export function startWebserver(
|
||||
}
|
||||
});
|
||||
|
||||
app.get("/api/guilds/:guildId/channels", async (req, res, next) => {
|
||||
try {
|
||||
res.json(await voiceController.listWatchableChannels(req.params.guildId));
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
});
|
||||
|
||||
app.get("/api/guilds/:guildId/threads", async (req, res, next) => {
|
||||
try {
|
||||
res.json(await voiceController.listThreads(req.params.guildId));
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
});
|
||||
|
||||
app.post("/api/connect", async (req, res, next) => {
|
||||
try {
|
||||
const { guildId, channelId } = req.body as {
|
||||
@@ -129,6 +151,44 @@ export function startWebserver(
|
||||
}
|
||||
});
|
||||
|
||||
// Moderation API endpoints
|
||||
app.get("/api/messages", async (req, res, next) => {
|
||||
try {
|
||||
const db = getDatabase();
|
||||
const { channel, type, limit = "50", offset = "0" } = req.query as {
|
||||
channel?: string;
|
||||
type?: string;
|
||||
limit?: string;
|
||||
offset?: string;
|
||||
};
|
||||
|
||||
if (!channel) {
|
||||
throw new AppError("channel query parameter is required", "MISSING_CHANNEL", 400);
|
||||
}
|
||||
|
||||
const limitNum = Math.min(parseInt(limit) || 50, 100);
|
||||
const offsetNum = parseInt(offset) || 0;
|
||||
|
||||
if (type === "image") {
|
||||
const attachments = getAttachmentsByChannel(db, channel, limitNum, offsetNum);
|
||||
res.json({
|
||||
type: "image",
|
||||
data: attachments,
|
||||
count: attachments.length,
|
||||
});
|
||||
} else {
|
||||
const messages = getMessagesByChannel(db, channel, limitNum, offsetNum);
|
||||
res.json({
|
||||
type: "text",
|
||||
data: messages,
|
||||
count: messages.length,
|
||||
});
|
||||
}
|
||||
} catch (error) {
|
||||
next(error);
|
||||
}
|
||||
});
|
||||
|
||||
// Inbound: Discord PCM → tagged chunks → browser
|
||||
(global as any).broadcastPcmToWeb = (chunk: Buffer, userId: string) => {
|
||||
let hash = 0;
|
||||
@@ -165,6 +225,33 @@ export function startWebserver(
|
||||
});
|
||||
}
|
||||
|
||||
function broadcastMessageEvent(type: string, data: any) {
|
||||
const payload = JSON.stringify({
|
||||
type,
|
||||
data,
|
||||
timestamp: Date.now(),
|
||||
});
|
||||
wsClients.forEach((client) => {
|
||||
if (client.readyState === 1) client.send(payload);
|
||||
});
|
||||
}
|
||||
|
||||
(global as any).broadcastMessageCreated = (data: any) => {
|
||||
broadcastMessageEvent("message_created", data);
|
||||
};
|
||||
|
||||
(global as any).broadcastMessageUpdated = (data: any) => {
|
||||
broadcastMessageEvent("message_updated", data);
|
||||
};
|
||||
|
||||
(global as any).broadcastMessageDeleted = (data: any) => {
|
||||
broadcastMessageEvent("message_deleted", data);
|
||||
};
|
||||
|
||||
(global as any).broadcastAttachmentUploaded = (data: any) => {
|
||||
broadcastMessageEvent("attachment_uploaded", data);
|
||||
};
|
||||
|
||||
// --- Outbound: browser PCM (24kHz mono) → Opus → Discord ---
|
||||
const RATE = 48000;
|
||||
const CHANNELS = 2;
|
||||
|
||||
33
tests/decoder.test.ts
Normal file
33
tests/decoder.test.ts
Normal file
@@ -0,0 +1,33 @@
|
||||
import process from "node:process";
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
beforeEach(() => {
|
||||
process.env = {
|
||||
...process.env,
|
||||
DISCORD_TOKEN: "token",
|
||||
NODE_ENV: "test",
|
||||
};
|
||||
vi.resetModules();
|
||||
});
|
||||
|
||||
describe("shouldEnableDefaultOpusDecoder", () => {
|
||||
it("disables default decoder on Bun when native opus is unavailable", async () => {
|
||||
const { shouldEnableDefaultOpusDecoder } = await import(
|
||||
"../src/recorder/decoder"
|
||||
);
|
||||
|
||||
expect(
|
||||
shouldEnableDefaultOpusDecoder({ isBun: true, canLoadNativeOpus: false }),
|
||||
).toBe(false);
|
||||
});
|
||||
|
||||
it("enables default decoder when native opus is available", async () => {
|
||||
const { shouldEnableDefaultOpusDecoder } = await import(
|
||||
"../src/recorder/decoder"
|
||||
);
|
||||
|
||||
expect(
|
||||
shouldEnableDefaultOpusDecoder({ isBun: true, canLoadNativeOpus: true }),
|
||||
).toBe(true);
|
||||
});
|
||||
});
|
||||
46
tests/moderation/attachmentUploader.test.ts
Normal file
46
tests/moderation/attachmentUploader.test.ts
Normal file
@@ -0,0 +1,46 @@
|
||||
import { describe, it, expect, beforeEach } from "vitest";
|
||||
|
||||
beforeEach(() => {
|
||||
process.env = {
|
||||
...process.env,
|
||||
DISCORD_TOKEN: "test-token",
|
||||
MONITOR_GUILD_ID: "test-guild",
|
||||
NODE_ENV: "test",
|
||||
};
|
||||
});
|
||||
|
||||
describe("attachmentUploader", () => {
|
||||
it("parses picser upload response correctly", async () => {
|
||||
const { parseUploadResponse } = await import("../../src/moderation/attachmentUploader");
|
||||
|
||||
const response = {
|
||||
success: true,
|
||||
filename: "uploads/abc123.jpg",
|
||||
urls: {
|
||||
raw_commit: "https://raw.githubusercontent.com/user/repo/commit/uploads/abc123.jpg",
|
||||
},
|
||||
size: 102400,
|
||||
type: "image/jpeg",
|
||||
};
|
||||
|
||||
const result = parseUploadResponse(response);
|
||||
|
||||
expect(result.success).toBe(true);
|
||||
expect(result.url).toBe("https://raw.githubusercontent.com/user/repo/commit/uploads/abc123.jpg");
|
||||
expect(result.filename).toBe("uploads/abc123.jpg");
|
||||
});
|
||||
|
||||
it("handles upload response with missing raw_commit", async () => {
|
||||
const { parseUploadResponse } = await import("../../src/moderation/attachmentUploader");
|
||||
|
||||
const response = {
|
||||
success: true,
|
||||
filename: "uploads/abc123.jpg",
|
||||
urls: {},
|
||||
size: 102400,
|
||||
type: "image/jpeg",
|
||||
};
|
||||
|
||||
expect(() => parseUploadResponse(response)).toThrow();
|
||||
});
|
||||
});
|
||||
@@ -6,6 +6,7 @@
|
||||
"strict": true,
|
||||
"esModuleInterop": true,
|
||||
"skipLibCheck": true,
|
||||
"types": ["bun-types"],
|
||||
"outDir": "dist",
|
||||
"rootDir": "src",
|
||||
"experimentalDecorators": true,
|
||||
|
||||
Reference in New Issue
Block a user