feat(docs): Add comprehensive agent guides and templates for spec-driven development

This commit is contained in:
asepharyana
2026-09-11 18:56:28 +07:00
parent b3418ad799
commit 43f2f8449d
14 changed files with 727 additions and 5 deletions
+179
View File
@@ -0,0 +1,179 @@
# GMW — Agent Development Guide
> **Rule #1: No code without a spec.**
> Setiap perubahan signifikan dimulai dari spec di `docs/specs/`.
> Baca spec → verifikasi facts → implement → verify → commit.
> Gunakan `todo.md` (TODO list terdokumentasi) untuk melacak progress.
## Project overview
GMW (Go Mod Watch) adalah Discord bot + dashboard untuk AI-powered moderation.
Monorepo berisi 3 service utama:
| Service | Path | Port | Tech |
|---|---|---|---|
| **discord-gateway** | `services/discord-gateway/` | 4016 (metrics) | discord.js-selfbot-v13, Piscina, Drizzle ORM, pino |
| **backend** | `services/backend/` | 4001 | Express, oRPC, Drizzle ORM, Redis pub/sub, Vitest |
| **frontend** | `services/frontend/` | 4017 (standalone) | Next.js 16 App Router, React 19, Tailwind v4, SWR |
## Data flow
```
Discord → discord-gateway → Redis pub/sub → backend (:4001) ←→ frontend Next.js SSR
↑ REST /api/*
└ WS /ws
```
- Gateway = event-driven (no HTTP, except Prometheus :4016/metrics)
- Backend = HTTP + WebSocket server, serves the frontend
- Frontend = SSR (RSC) + client hydration, proxied by nginx :4009
## Konvensi dokumentasi: `docs/`
Semua dokumen kerja ada di `docs/`, bukan di `.hermes/`:
```
docs/
├── README.md # Panduan workflow (spec-driven + todo)
├── spec-template.md # Template spec standar
├── todo-template.md # Template todo list
└── specs/ # Semua spec & implementation plan
├── YYYY-MM-DD_<slug>-spec.md # Spec (apa & mengapa)
└── YYYY-MM-DD_<slug>.md # Plan/fix (spec + plan dalam satu file)
```
Service-specific docs: `services/<service>/docs/specs/`.
## Workflow: Spec-Driven Development
### Checklist wajib untuk setiap perubahan signifikan
1. **Tulis spec** → `docs/specs/YYYY-MM-DD_<slug>-spec.md`
2. **Tulis `todo.md`** → daftar task konkret yang bisa diceklis
3. **Verifikasi facts** → baca kode aktual, konfirmasi referensi file:line
4. **Keputusan desain** → pilih approach, dokumentasikan alternatif yang ditolak
5. **Implementasi** → ikuti spec + todo step-by-step
6. **Verifikasi** → jalankan semua verification steps dari spec
7. **Commit** → reference spec di commit message
### Todo list (`todo.md`)
Setiap tugas berjalan WAJIB punya `todo.md`. Format:
```markdown
# Todo — <judul tugas>
## Task
- [ ] Tulis spec
- [ ] Verifikasi facts (baca kode: file:line)
- [ ] Implementasi tahap 1: ...
- [ ] Implementasi tahap 2: ...
- [ ] Verifikasi: pnpm typecheck / lint / build / test
- [ ] Commit + push
```
Aturan:
- Task harus **konkret & verifiable** — bukan "fix bug", tapi "ubah X di file Y".
- Ceklis `[x]` saat selesai, jangan menunggu batch di akhir.
- Gunakan `docs/todo-template.md` sebagai template.
- Simpan `todo.md` di `docs/` untuk tugas lintas-service, atau di
`services/<service>/docs/` untuk tugas satu service.
### Spec template
Salin `docs/spec-template.md` untuk setiap spec baru. Sections:
1. **Problem** — apa yang rusak/missing, dengan evidence
2. **Root cause** — analisis teknis (bukan symptom)
3. **Behavior target** — perilaku setelah fix, daftar verifiable
4. **Verified facts** — fakta dari pembacaan kode, citation ke file:line
5. **Keputusan desain** — pilihan + rationale + alternatif ditolak
6. **Perubahan file** — semua file yang disentuh, per service
7. **Schema/type changes** — perubahan tipe/DB
8. **Verification** — command executable + expected outcome
### Kapan perlu spec + todo.md
Perlu: fitur baru, bug fix non-trivial, refactor behavior-changing, perubahan
DB schema, perubahan API contract, perubahan arsitektur.
Tidak perlu: typo fix, dep bump, format/lint auto-fix, test-only, README update.
Baca selengkapnya di `docs/README.md`.
## Coding conventions
### General
- **TypeScript strict mode** — semua service
- **Biome** — formatting + linting (`pnpm format`, `pnpm lint`)
- **Bun** — package manager (`bun install`, `bun run`)
- **Bisa bilingual** — code comments & specs boleh Indonesia/English
### Per-service conventions
#### discord-gateway
- **Event-driven** — no HTTP server (except metrics). Listeners → Redis pub/sub.
- **Module pattern**: `src/modules/<module>/` — each module encapsulates own logic.
- **Piscina pools**: text pool (4 threads) + media pool (2 threads), each with own pg Pool.
- **Logger**: `createChildLogger('module-name')` — never raw `console`.
- **Config**: Zod-validated env in `shared/config/index.ts` — single source of truth.
- **DB**: Drizzle ORM. Migrations in `drizzle/migrations/`.
- **Invariant**: LLM is the only judge. Never reintroduce regex content classification.
- **Invariant**: Discord tokens sanitized before reaching LLM.
- Lihat `services/discord-gateway/AGENTS.md` untuk detail.
#### backend
- **Modular MVC**: `modules/<module>/` — schema → repository → service → controller → routes.
- **No cross-module repo imports** — each module owns its data.
- **Data flows up only**: Repository → Service → Controller.
- **Error hierarchy**: `AppError` subclasses with code + statusCode.
- **Config**: Zod-validated env in `shared/config/index.ts`.
- **API**: oRPC for type-safe procedures + standard Express routes.
- Lihat `services/backend/AGENTS.md` untuk detail.
#### frontend
- **SSR-first**: `page.tsx` (server component) → fetch via `src/lib/api/server.ts` → pass to `view.tsx` (client).
- **No auth**: all endpoints public.
- **Never hardcode host**: same-origin or `GMW_BACKEND_URL` only.
- **WebSocket**: `src/lib/ws/` — auto-reconnecting, typed events.
- **Local dev**: `NEXT_PUBLIC_API_URL`, `NEXT_PUBLIC_WS_URL`, `GMW_BACKEND_URL`.
- Lihat `services/frontend/AGENTS.md` untuk detail Next.js rules.
## Build & Verify
```bash
# Per service (run from service root)
pnpm typecheck # TypeScript strict
pnpm lint # Biome check
pnpm build # Compile
pnpm test # Vitest (gateway & backend only)
pnpm format # Biome auto-format
```
## Commit conventions
```
<type>(<scope>): <subject>
<optional body>
Ref: docs/specs/<spec-file>.md
```
Types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`, `build`, `ci`
## Deployment
CI/CD: GitHub Actions → build → deploy to production server via Nix flakes.
- Gateway: `nixos-rebuild` or `systemctl restart gmw-discord-gateway`
- Backend: `nixos-rebuild` or `systemctl restart gmw-backend`
- Frontend: Next.js standalone, proxied by nginx :4009
## Remember
- Spec dulu, code belakangan.
- Todo list (`todo.md`) wajib untuk tugas yang berjalan — ceklis tiap selesai.
- Verified facts harus dari pembacaan kode aktual, bukan asumsi.
- Setiap change harus verifiable — tulis command di spec.
- One spec = one focused change. Don't mix unrelated features.
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
+9
View File
@@ -0,0 +1,9 @@
# Roadmap
What is built, what is next, and what has been deliberately declined.
---
## Next
_Empty._
+23
View File
@@ -0,0 +1,23 @@
# TODO
Next up. One item, one outcome, verifiable when done.
Longer-term direction lives in [ROADMAP.md](ROADMAP.md).
---
## Now
_Empty._
---
## Next
_Empty._
---
## Maintenance
_Empty._
+131
View File
@@ -0,0 +1,131 @@
# docs/ — Spec-Driven Development Hub
Direktori ini adalah pusat dari workflow **spec-driven development** di GMW.
Semua perubahan signifikan dimulai dari satu spec, **sebelum kode ditulis**,
dan setiap tugas yang berjalan dilacak lewat `todo.md`.
## Struktur
```
docs/
├── README.md # Dokumen ini
├── spec-template.md # Template standar untuk semua spec
├── todo-template.md # Template todo list
└── specs/ # Semua spec dan implementation plan
├── YYYY-MM-DD_<slug>-spec.md # Spec (apa & mengapa)
└── YYYY-MM-DD_<slug>.md # Plan/fix (bisa langsung spec+plan)
```
### Naming convention
```
YYYY-MM-DD_<slug>-spec.md # Spec baru untuk fitur/fix
YYYY-MM-DD_<slug>.md # Plan yang sudah include spec di dalamnya
```
Contoh:
- `2026-08-30_recordings-v2-features-spec.md` — spec untuk Recordings v2
- `2026-08-24-attachment-delay-fix.md` — spec + plan untuk attachment delay fix
### Service-specific docs
```
services/<service>/docs/specs/ # Spec yang spesifik untuk 1 service
```
## Workflow: Spec-Driven Development
### Prinsip utama
> **No code without a spec.** Semua perubahan signifikan harus punya spec
> terlebih dahulu. Spec adalah kontrak: apa yang akan dibangun, mengapa,
> dan bagaimana memverifikasinya.
### Todo list (`todo.md`) — WAJIB
Setiap tugas berjalan (implementasi fitur/fix) harus punya `todo.md`:
```markdown
# Todo — <judul tugas>
## Task
- [ ] Tulis spec
- [ ] Verifikasi facts (baca kode: file:line)
- [ ] Implementasi tahap 1: ...
- [ ] Implementasi tahap 2: ...
- [ ] Verifikasi: pnpm typecheck / lint / build / test
- [ ] Commit + push
```
Aturan `todo.md`:
- Task **konkret & verifiable** — "ubah X di file Y", bukan "fix bug".
- Satu task `[in_progress]` pada satu waktu; ceklis `[x]` segera setelah selesai.
- Gunakan `docs/todo-template.md` sebagai template.
- Simpan `todo.md` di `docs/` (tugas lintas-service) atau di
`services/<service>/docs/` (tugas satu service).
### Kapan perlu spec + todo
| Perlu spec + todo | Tidak perlu |
|---|---|
| Fitur baru | Typo fix |
| Bug fix non-trivial | Dependency bump (dependabot) |
| Refactor yang mengubah behavior | Format/lint auto-fix |
| Perubahan DB schema | Test-only change |
| Perubahan API contract | README/doc update |
| Perubahan arsitektur | Variable rename |
### Workflow step-by-step
1. **Tulis spec** — Salin `docs/spec-template.md`, isi semua section yang relevan.
- **Verified facts**: baca kode yang terpengaruh, catat temuan dengan file:line.
- **Root cause**: analisis sebab (jangan hanya describe symptom).
- **Decisions**: pilih pendekatan, jelaskan alternatif yang ditolak.
- **Verification**: command executable, bukan "should work".
- Simpan sebagai `docs/specs/YYYY-MM-DD_<slug>-spec.md`.
2. **Tulis `todo.md`** — Pecah spec jadi task konkret yang bisa diceklis
(gunakan `docs/todo-template.md`), dengan urutan implementasi yang logis.
3. **Review spec** — Baca ulang spec sendiri. Cek:
- Apakah verified facts benar-benar verified (bukan asumsi)?
- Apakah file changes lengkap (tidak ada yang terlewat)?
- Apakah verification steps executable?
- Jika spec untuk user request: pastikan user setuju dengan approach.
4. **Implement** — Ikuti spec + todo step-by-step. Ceklis `[x]` setiap task
yang selesai. Jika menemukan sesuatu yang berubah dari asumsi spec,
**update spec dulu**, baru implement.
5. **Verify** — Jalankan semua verification steps di spec. Catat hasilnya.
6. **Commit** — Reference spec di commit message:
```
feat(module): deskripsi singkat
Ref: docs/specs/YYYY-MM-DD_<slug>-spec.md
```
### Tips menulis spec yang baik
- **Evidence-based**: setiap klaim harus ada sumbernya (file:line, log, error).
- **Actionable**: orang lain (atau AI agent) harus bisa implementasi dari spec saja.
- **Verifiable**: setiap requirement harus bisa di-test secara eksplisit.
- **Scoped**: satu spec = satu perubahan terfokus. Jangan campur 3 fitur dalam 1 spec.
- **Bahasa campuran**: narrative boleh Indonesia/English, istilah teknis pakai English.
### Role of AI agents
AI coding agents (Shiro Neko / Hermes) **harus**:
- Membaca spec sebelum menulis kode
- Membuat & meng-update `todo.md` untuk setiap tugas berjalan
- Verifikasi fakta dari kode aktual (bukan dari asumsi training data)
- Update spec jika temuan baru mengubah approach
- Jalankan verification steps setelah implementasi
- Reference spec di commit message
AI agents **tidak boleh**:
- Langsung implement tanpa membaca/menulis spec
- Mengabaikan verified facts yang bertentangan dengan asumsi
- Skip verification steps
- Mengerjakan tugas tanpa todo list yang jelas
+80
View File
@@ -0,0 +1,80 @@
# Spec: <Judul singkat, aktif, deskriptif>
Status: **DRAFT** | **APPROVED** | **IN PROGRESS** | **DONE**
Date: YYYY-MM-DD
Author: <nama/agent>
Related: <link ke spec/plan terkait jika ada>
Todo: <link ke todo.md untuk tugas ini, jika ada>
## Problem
<Deskripsi masalah atau keinginan. Apa yang rusak? Apa yang belum ada?
Sebutkan siapa yang melaporkan (user/bot/audit) dan konteksnya.
Gunakan **evidence**: log, error, metrik, atau observasi langsung.>
## Root cause
<Analisis teknis mengapa masalah ini terjadi. Jika bug: trace dari symptom ke cause.
Jika fitur baru: jelaskan gap saat ini. Referensikan file + line number yang relevan.>
## Behavior target
<Deskripsi eksplisit perilaku yang diharapkan setelah fix/fitur.
Buat list bernomor. Setiap item harus bisa di-verify.>
## Verified facts
>Fakta-fakta teknis yang **sudah diverifikasi** dari pembacaan kode, log produksi,
>atau eksperimen langsung. Setiap fakta harus citation ke file:line.
>Jika belum diverifikasi, tulis `UNVERIFIED` dan rencana verifikasi.
- `path/to/file.ts:42` — <apa yang terjadi di sini>
- `path/to/file.ts:88` — <apa yang terjadi di sini>
- Kolom DB `table.column` — <tipe, constraint, index>
- Config `ENV_VAR` default <value> — <dibaca di mana>
## Keputusan desain
<Pilihan desain yang dibuat, beserta rationale (mengapa bukan alternatif lain).
Format: nomor, judul singkat, penjelasan.>
1. **<Judul keputusan>**: <penjelasan + rationale>
- Alternatif yang ditolak: <apa + mengapa ditolak>
## Perubahan file
>Daftar **semua file** yang perlu diubah/ditambah/dihapus, dikelompokkan per service.
>Untuk setiap file: sebutkan **apa yang berubah** (bukan copy-paste kode).
### Gateway (`services/discord-gateway/`)
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
### Backend (`services/backend/`)
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
### Frontend (`services/frontend/`)
- `src/<path>/<file>.ts|tsx` — <ringkasan perubahan>
### Database / Config
- <migration file atau config change jika ada>
## Schema/type changes
<Perubahan tipe, interface, atau DB schema. Jika tidak ada, tulis "TIDAK ada perubahan.">
## Verification
>**Harus spesifik dan executable.** Jangan tulis "works correctly".
>Tulis command yang bisa dijalankan dan expected outcome.
- **Typecheck**: `<service> pnpm typecheck` — clean
- **Lint**: `<service> pnpm lint` — no errors
- **Build**: `<service> pnpm build` — compiles
- **Test**: `<service> pnpm test` — all pass (+ test baru jika ada)
- **Smoke test**: <langkah manual untuk verifikasi visual/behavioral>
- **DB**: <query untuk cek data jika applicable>
- **Deploy**: CI green → deploy → cek <specific observable>
## Notes (opsional)
<Catatan tambahan: risiko, fallback, future work, atau hal yang sengaja di-scope-out.>
@@ -3,8 +3,8 @@
Status: **P1–P3 DONE + 4th CRITICAL FIX deployed (f1a7b0c2); DAVE Ready + MLS handshake CONFIRMED live; P4 = waiting on active streamer to confirm video-burst→mp4**
Date: 2026-08-31
Author: Hermes
Related: `.hermes/plans/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
`.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
Related: `docs/specs/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
`docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
## Problem / Ground truth (established from live logs 2026-08-31)
GMW must record OTHER members' screen-share + camera video in a voice channel it
@@ -3,7 +3,7 @@
Status: PLANNED
Date: 2026-08-31
Author: Hermes
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
## Symptom (from live logs, 2026-08-31 ~12:34)
A user was actively screen-sharing + on camera in the recorded voice channel.
@@ -3,7 +3,7 @@
Status: PLANNED (not built)
Date: 2026-08-31
Author: Hermes
Related: `.hermes/plans/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
Related: `docs/specs/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
## TL;DR — what changed vs Phase A/B
@@ -3,7 +3,7 @@
Status: PLANNED (not yet built)
Date: 2026-09-02
Author: Hermes
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
for selfbot), `gmw-ops/references/selfbot-presence-detection-limits.md`,
`gmw-ops/references/discord-voice-fork-video-receive.md`
+27
View File
@@ -0,0 +1,27 @@
# Todo — <judul tugas>
Status: **IN PROGRESS** | **BLOCKED** | **DONE**
Spec: <link ke spec: `docs/specs/YYYY-MM-DD_<slug>-spec.md`>
Date: YYYY-MM-DD
## Task
> Urutan implementasi sesuai spec. Task harus konkret & verifiable
> ("ubah X di file Y"), bukan "fix bug". Satu task dikerjakan pada satu
> waktu; ceklis `[x]` segera setelah selesai.
- [ ] Tulis spec
- [ ] Verifikasi facts (baca kode: `file:line`)
- [ ] Implementasi tahap 1: <apa + file mana>
- [ ] Implementasi tahap 2: <apa + file mana>
- [ ] Implementasi tahap 3: <apa + file mana>
- [ ] Typecheck: `pnpm typecheck` — clean
- [ ] Lint: `pnpm lint` — no errors
- [ ] Test: `pnpm test` — all pass
- [ ] Build: `pnpm build` — compiles
- [ ] Daftar ulang hasil verifikasi di spec
- [ ] Commit + push (reference spec di commit message)
## Notes
<Blocking issue, temuan yang mengubah approach, atau hal yang perlu dicatat.>
+117
View File
@@ -0,0 +1,117 @@
# Backend Service — Agent Guide
> **Read `../../AGENTS.md` first.** This file adds backend-specific conventions.
Backend service: Express HTTP server + WebSocket, serves the GMW dashboard API.
## Quick reference
```bash
pnpm typecheck # tsc --noEmit
pnpm lint # biome check --diagnostic-level=error .
pnpm build # tsc
pnpm test # vitest run
pnpm format # biome format --write .
```
## Architecture (Modular MVC)
```
src/
├── shared/ # Infrastructure (no business logic)
│ ├── config/index.ts # Zod-validated env
│ ├── database/ # Drizzle ORM + pg Pool
│ ├── errors/index.ts # AppError hierarchy
│ ├── logger/index.ts # pino + createChildLogger()
│ ├── middlewares/index.ts # errorHandler, asyncHandler, rateLimit
│ └── utils/ # Pagination, messageMapper
├── modules/ # Feature modules
│ └── <module>/
│ ├── <module>.schema.ts # Zod validation schemas
│ ├── <module>.repository.ts # DB operations only
│ ├── <module>.service.ts # Business logic
│ ├── <module>.controller.ts # HTTP handlers
│ └── routes/index.ts # Express router
├── http/ # app.ts (factory) + server.ts (startup)
├── ws/ # WebSocket server + Redis bridge
└── index.ts # Entry point
```
## Dependency rules
- Controller → Service → Repository → Database
- No cross-module repo imports (each module owns its data)
- No HTTP in Service layer (no req/res)
- No DB in Controller layer
- Any layer → Config, Logger, Errors
## API: oRPC + Express
- **oRPC** (`src/orpc/router.ts` + `src/orpc/ws.ts`): type-safe procedures for frontend
- **Express routes** (`src/modules/*/routes/`): REST endpoints under `/api/`
- **WebSocket** (`src/ws/`): Redis bridge → broadcast to connected browsers
- Never invent endpoints. Match what the frontend calls (`src/lib/api/`).
## Key modules
| Module | Purpose | DB tables |
|---|---|---|
| messages | Store & query Discord messages | `messages`, `ai_moderations`, `ai_moderation_flags` |
| moderation | Moderation actions & metrics | `ai_moderations`, `moderation_actions` |
| media | Media file management | `media_attachments` |
| voice | Live speakers + recordings | `voice_recordings` |
| recordings | Recordings API | `voice_recordings` |
| dashboard | Stats aggregation | Various (read-only) |
| knowledge | Semantic search | Qdrant vector DB |
| chatbot | AI chatbot with tools | `chatbot_history` |
| health | Health checks + metrics | Various |
| analysis | Text analysis cache | `text_analysis_cache` |
| ui-state | Persist UI preferences | `ui_state` |
## Config (env vars)
All validated via Zod in `shared/config/index.ts`. Key vars:
- `WEBSERVER_PORT` (default 4001)
- `DATABASE_URL` or individual `DATABASE_HOST/PORT/NAME/USER/PASSWORD`
- `REDIS_URL` — for pub/sub with gateway
- `MONITOR_GUILD_ID` — primary Discord guild
- `ADMIN_PASSWORD` — admin endpoints
## Data contract with frontend
The frontend fetches via `src/lib/api/server.ts` (SSR, server-side) and
`src/lib/api/client.ts` (browser, same-origin through proxy).
**Do not change response shapes without updating both sides.** Check
`services/frontend/src/lib/types/` for frontend type definitions.
## Redis channels (inbound from gateway)
```
discord:message:{created,updated,deleted,analyzed}
discord:attachment:{created,uploaded}
discord:voice:{started,stopped,uploaded,active_user,pcm,analyzed}
discord:analysis:queue_status
discord:reaction:{added,removed}
discord:thread:{created,deleted,updated}
discord:channel_topic:updated
discord:presence:updated
discord:guild_member:{added,removed}
```
Canonical names: `src/shared/redis-channels.ts`.
## Testing
- Vitest for unit tests
- Mock database and external services
- Test files: `src/modules/<module>/*.test.ts` or `tests/*.test.ts`
- Run: `pnpm test`
## Common pitfalls
- **oRPC vs REST**: check both routers when adding an endpoint
- **Redis channel mismatch**: gateway publishes → backend subscribes. Channel
names must match exactly (see `redis-channels.ts` in BOTH services)
- **DB pool**: backend uses one pool (main thread). Gateway has per-piscina-thread pools.
+153
View File
@@ -0,0 +1,153 @@
# Discord Gateway — Agent Guide
> **Read `../../AGENTS.md` first.** This file adds gateway-specific conventions.
Event-driven microservice: captures Discord events, runs AI moderation, publishes to Redis.
## Quick reference
```bash
pnpm typecheck # tsc --noEmit
pnpm lint # biome check --diagnostic-level=error .
pnpm build # tsc
pnpm test # vitest run
pnpm format # biome format --write .
```
## Architecture (Event-driven)
```
src/
├── index.ts # Entry point → initializeDiscordGateway()
├── app/
│ ├── bootstrap.ts # Wires client, DB, Redis, workers, schedulers
│ ├── shutdown.ts # Graceful shutdown
│ └── retention.ts # Expired-record cleanup
├── shared/
│ ├── config/index.ts # Zod-validated env (SINGLE source of truth)
│ ├── database/ # Drizzle ORM + pg Pool + migrations
│ ├── logger/index.ts # pino + createChildLogger()
│ ├── errors/index.ts # AppError hierarchy
│ ├── utils/ # retry, pagination
│ ├── discord/clientOptions.ts # discord.js-selfbot-v13 options
│ ├── uploader.ts # Attachment upload helper
│ ├── redis-channels.ts # Redis channel-name constants
│ └── moderation-types.ts # Shared AI analysis types
├── modules/
│ ├── ai-moderation/ # LLM moderation pipeline (largest module)
│ ├── message-capture/ # Discord event listeners + DB store
│ ├── voice-recording/ # Voice connect + Opus→OGG recording
│ │ └── recorder/ # decoder, segment, session, uploader, oggCrc
│ ├── voice-pcm-ws/ # Real-time PCM → backend WebSocket
│ ├── attachment-upload/ # Download + sharp resize + upload
│ ├── event-broadcaster/ # Redis pub/sub publisher
│ ├── command-handler/ # Backend→gateway Redis commands
│ ├── reaction-tracking/ # Reaction events
│ ├── thread-tracking/ # Thread events
│ ├── user-presence/ # Presence/status events
│ ├── channel-topic/ # Channel topic events
│ ├── guild-member-events/ # Member join/leave
│ └── gateway-metrics/ # Prometheus /metrics (port 4016)
└── tests/ # Vitest suites
```
## Key invariants (DO NOT BREAK)
1. **LLM is the only judge.** Failed LLM → `status:"error"` + recovery retry.
**Never** reintroduce regex/heuristic content classification.
2. **Discord tokens sanitized** before reaching LLM (`discordTokens.ts`).
3. **Semantic cache is batched** — one embed call + one Qdrant batch search.
4. **Streaming is mandatory** against the omniroute base URL.
## AI moderation pipeline
```
aiAnalyzer.ts → batchScheduler.ts → batchProcessor.ts → individualFallbackProcessor.ts
↓ ↓ ↓ ↓
moderationOrchestrator.ts → (hash cache → Qdrant → LLM)
↓ ↓ ↓
textBatchProcessor.ts mediaBatchProcessor.ts llmClient.ts
embeddingClient.ts
qdrantClient.ts
```
- Entry: `aiAnalyzer.ts` (`queueMessageAnalysis`, `startPendingAIAnalysisWorker`)
- Concurrency: LLM semaphore (`AI_LLM_MAX_CONCURRENT`, default 5)
- Piscina: text pool (4 threads) + media pool (2 threads)
- **Each worker thread has its own pg Pool** (min 0, grows to `POSTGRES_POOL_MAX`)
## Voice recording pipeline
```
receiver.speaking "start" → speakingHandler(userId)
→ collectUserMetadata → receiver.subscribe → PacketFilter → oggPacketStream
→ SegmentManager.open → OggLogicalBitstream → .ogg file
→ data: rotateIfNeeded + decoder.write
→ end: finalizeSegment → upload + transcribe
```
Key files:
- `voiceController.ts` — connect/disconnect/list
- `recorder.ts` — orchestration
- `recorder/segment.ts` — segment rotation
- `recorder/sessionRecording.ts` — session management
- `recorder/uploader.ts` — upload to storage
- `voiceTranscriber.ts` — Whisper transcription (if enabled)
## Module: message-capture
- `messageCapture.ts` — Discord event listeners (messageCreate/Update/Delete)
- `messageStore.ts` — DB operations
- `messageMetadata.ts` — metadata extraction
- `messagesDb.ts` / `messagesCrud.ts` — DB schema operations
- `archiveEmbedder.ts` — Qdrant embedding (respect age-restricted guard)
- `retentionDb.ts` / `reviewsDb.ts` / `attachmentsDb.ts` — auxiliary tables
## Redis channels (outbound to backend)
See `src/shared/redis-channels.ts` for canonical names. Examples:
```
discord:message:created, discord:voice:active_user, discord:attachment:uploaded
```
## Config (env vars)
All validated via Zod in `shared/config/index.ts`. Critical:
- `DISCORD_TOKEN` — selfbot token
- `MONITOR_GUILD_ID` — primary guild
- `DATABASE_URL` — PostgreSQL
- `REDIS_URL` — pub/sub to backend
- `AI_ANALYSIS_ENABLED` — master toggle for AI moderation
- `AI_LLM_BASE_URL` / `AI_LLM_API_KEY` — LLM router
- `AI_VOICE_TRANSCRIPTION_ENABLED` — toggle Whisper transcription
- `PISCINA_MAX_THREADS` / `PISCINA_MEDIA_MAX_THREADS` — worker pool sizing
## Concurrency model
- Main thread: event loop + LLM semaphore
- Text Piscina pool: `PISCINA_MAX_THREADS` (default 4)
- Media Piscina pool: `PISCINA_MEDIA_MAX_THREADS` (default 2)
- Batch routed to media pool if ANY message has attachment/sticker/embed
- Each worker thread initializes own pg Pool (min 0, grows to `POSTGRES_POOL_MAX`)
- MemoryMax: 1G (service systemd limit)
## Testing
- Vitest. Test files: `tests/<name>.test.ts`
- Run: `pnpm test`
- Key test areas: batch operations, cache guards, image/video handling, context enrichment
- Unit tests for pure helpers (classifier, normalize, parse), integration for pipeline stages
## Common pitfalls
- **Piscina pool isolation**: worker threads are NOT the main thread. Cannot
share state via module-level variables. Use DB or Redis for cross-thread state.
- **AfterSilence race**: `@discordjs/voice` AfterSilence can fail to emit "end"
on disconnect. Always have a watchdog/timeout.
- **Cache eviction**: LRU caches (user metadata, term glossary) evict at max size.
Don't assume cache hit after eviction.
- **Circuit breaker**: per-conversation CB opens after repeated failures.
Check `circuitBreaker.ts` state when debugging "missing analysis".
- **Gateway≠Backend schema**: both have `redis-channels.ts` and `moderation-types.ts`.
Keep them in sync manually.
+2
View File
@@ -1,3 +1,5 @@
@../../AGENTS.md
# Bete Frontend — Project Overview
Next.js 16 (App Router), React 19, TypeScript strict, Tailwind v4, shadcn/ui, base-ui.