GMW/services/discord-gateway

Discord Gateway Service - Extraction Complete

Overview

Successfully extracted Discord Gateway service with Modular MVC + Event-Driven Architecture using Redis pub/sub for inter-service communication.

Directory Structure

services/discord-gateway/
├── src/
│   ├── app/
│   │   ├── bootstrap.ts          # Service initialization (Discord client, DB, Redis)
│   │   └── shutdown.ts           # Graceful shutdown handler
│   ├── shared/                   # Shared infrastructure layer
│   │   ├── config/
│   │   │   └── config.ts         # Zod-validated environment config
│   │   ├── database/
│   │   │   ├── schema.ts         # Drizzle ORM schema
│   │   │   ├── drizzle.ts        # PostgreSQL connection
│   │   │   ├── migrate.ts        # Migration runner
│   │   │   └── voiceRecordingRepo.ts
│   │   ├── errors/
│   │   │   └── errors.ts         # Custom error classes
│   │   ├── logger/
│   │   │   ├── logger.ts         # Winston logger wrapper
│   │   │   └── serialization.ts  # Log serialization
│   │   ├── utils/
│   │   │   └── retry.ts          # Retry with exponential backoff
│   │   └── discord/
│   │       └── clientOptions.ts  # Discord.js client config
│   ├── modules/                  # Feature modules (Modular MVC)
│   │   ├── message-capture/      # Controller-Service-Repository
│   │   │   ├── messageCapture.ts # Controller: Discord event listeners
│   │   │   ├── messageStore.ts   # Repository: DB operations
│   │   │   ├── messageMetadata.ts # Service: Metadata extraction
│   │   │   ├── types.ts          # Domain types
│   │   │   └── index.ts          # Module exports
│   │   ├── ai-moderation/        # Controller-Service-Repository
│   │   │   ├── aiAnalyzer.ts     # Controller: Analysis orchestration
│   │   │   ├── llmModerationClient.ts # Service: LLM API client
│   │   │   ├── aiAnalysisWorker.ts # Service: Worker pool
│   │   │   ├── indonesianTextNormalizer.ts # Service: Text normalization
│   │   │   ├── moderationPrompt.ts # Service: Prompt generation
│   │   │   └── index.ts          # Module exports
│   │   ├── voice-recording/      # Controller-Service-Repository
│   │   │   ├── voiceController.ts # Controller: Voice connection mgmt
│   │   │   ├── recorder.ts       # Service: Recording orchestration
│   │   │   ├── recorder/         # Sub-services
│   │   │   │   ├── audioStream.ts # Audio stream subscription
│   │   │   │   ├── decoder.ts    # Opus decoding
│   │   │   │   ├── segment.ts    # OGG segment rotation
│   │   │   │   ├── metadata.ts   # Segment metadata
│   │   │   │   ├── sessionRecording.ts # Session management
│   │   │   │   └── uploader.ts   # Segment upload
│   │   │   └── index.ts          # Module exports
│   │   ├── attachment-upload/    # Controller-Service-Repository
│   │   │   ├── attachmentUploader.ts # Service: Upload orchestration
│   │   │   ├── imageResizer.ts   # Service: Image resizing
│   │   │   └── index.ts          # Module exports
│   │   └── event-broadcaster/    # Event-driven layer
│   │       ├── eventBroadcaster.ts # Service: Redis pub/sub publisher
│   │       ├── eventTypes.ts     # Domain: Event type definitions
│   │       └── index.ts          # Module exports
│   ├── mock-crc.ts               # CRC polyfill for discord.js
│   └── index.ts                  # Service entry point
├── ARCHITECTURE.md               # Detailed architecture documentation
├── package.json                  # Service dependencies
└── tsconfig.json                 # TypeScript configuration (inherited)

Architecture Patterns

1. Modular MVC Structure

Each feature module follows Controller-Service-Repository pattern:

Message Capture Module:

• Controller (messageCapture.ts): Listens to Discord events (messageCreate, messageUpdate, messageDelete)
• Service (messageMetadata.ts): Extracts and normalizes message metadata
• Repository (messageStore.ts): Database CRUD operations

AI Moderation Module:

• Controller (aiAnalyzer.ts): Orchestrates analysis workflow
• Service (llmModerationClient.ts): LLM API integration
• Service (aiAnalysisWorker.ts): Worker pool management
• Service (indonesianTextNormalizer.ts): Text preprocessing

Voice Recording Module:

• Controller (voiceController.ts): Voice channel connection management
• Service (recorder.ts): Recording orchestration
• Sub-services (recorder/*): Audio stream, decoding, segmentation, upload

Attachment Upload Module:

• Service (attachmentUploader.ts): Upload orchestration
• Service (imageResizer.ts): Image processing

2. Event-Driven Architecture

Redis Pub/Sub replaces WebSocket broadcaster:

Discord Events → Discord Gateway Service → Redis Pub/Sub → Backend Service
                                         ↓
                                    Event Channels:
                                    - discord:message:created
                                    - discord:message:updated
                                    - discord:message:deleted
                                    - discord:message:analyzed
                                    - discord:attachment:created
                                    - discord:attachment:uploaded
                                    - discord:voice:started
                                    - discord:voice:stopped
                                    - discord:voice:uploaded
                                    - discord:analysis:queue_status

3. Shared Infrastructure Layer

Centralized, reusable components:

• Config: Zod-validated environment variables
• Logger: Winston logger with context support
• Database: Drizzle ORM with PostgreSQL
• Errors: Custom error classes with codes and HTTP status codes
• Utils: Retry logic with exponential backoff
• Discord: Client configuration and options

4. No HTTP Server

• Event-driven only: No Express, WebSocket, or HTTP routes
• Redis pub/sub: All inter-service communication via Redis
• Backend service: Consumes events and serves HTTP API
• Frontend: Continues to use Backend HTTP API

Key Features

Message Capture

1. Discord emits messageCreate, messageUpdate, messageDelete events
2. messageCapture.ts listener receives and validates event
3. Extract metadata: user, channel, content, timestamp, attachments
4. messageStore.ts inserts into PostgreSQL
5. eventBroadcaster.messageCreated() publishes to Redis
6. Backend service subscribes and processes

AI Moderation

1. aiAnalyzer.ts queues messages for analysis
2. llmModerationClient.ts calls LLM API with context
3. indonesianTextNormalizer.ts preprocesses text
4. Results stored in database
5. eventBroadcaster.messageAnalyzed() publishes results
6. Backend service receives and updates UI

Voice Recording

1. voiceController.connect() joins voice channel
2. recorder.ts subscribes to user audio streams
3. For each speaking user:
   • audioStream.ts subscribes to Opus packets
   • decoder.ts decodes Opus to PCM
   • segment.ts rotates OGG files (5s default)
   • metadata.ts collects user info
4. On silence (3s):
   • sessionRecording.ts finalizes segment
   • uploader.ts uploads to storage
   • eventBroadcaster.voiceRecordingUploaded() publishes
5. Backend service indexes recording

Attachment Upload

1. messageCapture.ts detects attachments
2. attachmentUploader.ts downloads from Discord
3. imageResizer.ts resizes images if needed
4. Upload to external storage with retry logic
5. eventBroadcaster.attachmentUploaded() publishes
6. Backend service stores metadata

Initialization Flow

1. Load environment config (Zod validation)
   ↓
2. Initialize PostgreSQL connection
   ↓
3. Run pending database migrations
   ↓
4. Create Discord client with optimized cache
   ↓
5. Initialize Redis event broadcaster
   ↓
6. Register Discord event listeners
   - messageCapture (message events)
   - aiAnalyzer (analysis worker)
   ↓
7. Login to Discord
   ↓
8. Listen for graceful shutdown signals

Graceful Shutdown

On SIGINT/SIGTERM/uncaughtException/unhandledRejection:

1. Close PostgreSQL connection
2. Disconnect from voice channels
3. Close Redis connection
4. Destroy Discord client
5. Exit process (code 0 for clean, 1 for error)

Dependencies

Core Discord:

• discord.js-selfbot-v13 — Discord client (selfbot variant)
• @discordjs/voice — Voice connection management
• @discordjs/opus — Native Opus codec

Audio Processing:

• prism-media — Opus encoding/decoding
• opusscript — Opus fallback for Node v26+
• sharp — Image resizing

Data & Config:

• drizzle-orm — Type-safe ORM
• pg — PostgreSQL driver
• zod — Config validation
• ioredis — Redis client

Logging & Utilities:

• winston — Structured logging
• p-retry — Retry with backoff
• p-limit — Concurrency limiting
• piscina — Worker pool

No Breaking Changes

• Original src/ remains untouched
• Discord Gateway is a new service in services/discord-gateway/
• Can run alongside existing monolith during transition
• Backend service will consume Redis events
• Frontend continues to use Backend HTTP API

Next Steps

1. Create Backend service (services/backend/)

   • HTTP API endpoints
   • Redis event subscribers
   • Database models
   • WebSocket broadcaster

2. Update Frontend (frontend/)

   • Connect to Backend HTTP API
   • Subscribe to WebSocket events

3. Nix & CI/CD

   • flake.nix package for Discord Gateway
   • systemd services (gmw-backend, gmw-discord-gateway)
   • GitHub Actions for build/deploy (nix copy → systemctl restart)

4. Documentation

   • API documentation
   • Event schema documentation
   • Deployment guide

Files Created

Total: 43 files

Shared Infrastructure (9 files)

• src/shared/config/config.ts
• src/shared/database/ (5 files)
• @bete/shared/errors (shared package)
• src/shared/logger/logger.ts
• src/shared/logger/serialization.ts
• src/shared/utils/retry.ts
• src/shared/discord/clientOptions.ts

Modules (28 files)

• src/modules/message-capture/ (5 files)
• src/modules/ai-moderation/ (6 files)
• src/modules/voice-recording/ (9 files)
• src/modules/attachment-upload/ (3 files)
• src/modules/event-broadcaster/ (3 files)

App & Entry (4 files)

• src/app/bootstrap.ts
• src/app/shutdown.ts
• src/index.ts
• src/mock-crc.ts

Configuration (2 files)

• package.json
• ARCHITECTURE.md

Verification Checklist

✅ Directory structure created ✅ Shared infrastructure migrated ✅ Message capture module migrated ✅ AI moderation module migrated ✅ Voice recording module migrated ✅ Attachment upload module migrated ✅ Event broadcaster module created (Redis pub/sub) ✅ Bootstrap and entry point created ✅ Package.json with dependencies ✅ No HTTP server code (Express, WebSocket removed) ✅ Event-driven architecture implemented ✅ Graceful shutdown handler ✅ Module index files for clean exports ✅ Architecture documentation

Event Flow Diagram

┌─────────────────────────────────────────────────────────────────┐
│                    Discord Gateway Service                      │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────┐ │
│  │ Message Capture  │  │  AI Moderation   │  │ Voice Record │ │
│  │   (Controller)   │  │   (Controller)   │  │ (Controller) │ │
│  └────────┬─────────┘  └────────┬─────────┘  └──────┬───────┘ │
│           │                     │                   │          │
│           ├─────────────────────┼───────────────────┤          │
│           │                     │                   │          │
│           ▼                     ▼                   ▼          │
│  ┌─────────────────────────────────────────────────────────┐  │
│  │         Event Broadcaster (Redis Pub/Sub)              │  │
│  │  - discord:message:created                             │  │
│  │  - discord:message:updated                             │  │
│  │  - discord:message:deleted                             │  │
│  │  - discord:message:analyzed                            │  │
│  │  - discord:attachment:created                          │  │
│  │  - discord:attachment:uploaded                         │  │
│  │  - discord:voice:started                               │  │
│  │  - discord:voice:stopped                               │  │
│  │  - discord:voice:uploaded                              │  │
│  │  - discord:analysis:queue_status                       │  │
│  └─────────────────────────────────────────────────────────┘  │
│           │                                                    │
└───────────┼────────────────────────────────────────────────────┘
            │
            │ Redis Pub/Sub
            │
            ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Backend Service                              │
│  (Subscribes to events, serves HTTP API, manages WebSocket)    │
└─────────────────────────────────────────────────────────────────┘
            │
            │ HTTP API
            │
            ▼
┌─────────────────────────────────────────────────────────────────┐
│                    Frontend Application                         │
│  (React SPA, real-time updates via WebSocket)                  │
└─────────────────────────────────────────────────────────────────┘

Summary

The Discord Gateway service has been successfully extracted with:

• Modular MVC architecture for clean separation of concerns
• Event-driven design using Redis pub/sub for inter-service communication
• Shared infrastructure layer for reusable components
• No HTTP server — pure event-driven service
• Graceful shutdown handling
• Type-safe configuration with Zod validation
• Structured logging with Winston
• PostgreSQL integration with Drizzle ORM

The service is ready for integration with the Backend service, which will consume Redis events and serve the HTTP API to the Frontend.