feat(docs): Add comprehensive agent guides and templates for spec-driven development
This commit is contained in:
@@ -0,0 +1,179 @@
|
||||
# GMW — Agent Development Guide
|
||||
|
||||
> **Rule #1: No code without a spec.**
|
||||
> Setiap perubahan signifikan dimulai dari spec di `docs/specs/`.
|
||||
> Baca spec → verifikasi facts → implement → verify → commit.
|
||||
> Gunakan `todo.md` (TODO list terdokumentasi) untuk melacak progress.
|
||||
|
||||
## Project overview
|
||||
|
||||
GMW (Go Mod Watch) adalah Discord bot + dashboard untuk AI-powered moderation.
|
||||
Monorepo berisi 3 service utama:
|
||||
|
||||
| Service | Path | Port | Tech |
|
||||
|---|---|---|---|
|
||||
| **discord-gateway** | `services/discord-gateway/` | 4016 (metrics) | discord.js-selfbot-v13, Piscina, Drizzle ORM, pino |
|
||||
| **backend** | `services/backend/` | 4001 | Express, oRPC, Drizzle ORM, Redis pub/sub, Vitest |
|
||||
| **frontend** | `services/frontend/` | 4017 (standalone) | Next.js 16 App Router, React 19, Tailwind v4, SWR |
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
Discord → discord-gateway → Redis pub/sub → backend (:4001) ←→ frontend Next.js SSR
|
||||
↑ REST /api/*
|
||||
└ WS /ws
|
||||
```
|
||||
|
||||
- Gateway = event-driven (no HTTP, except Prometheus :4016/metrics)
|
||||
- Backend = HTTP + WebSocket server, serves the frontend
|
||||
- Frontend = SSR (RSC) + client hydration, proxied by nginx :4009
|
||||
|
||||
## Konvensi dokumentasi: `docs/`
|
||||
|
||||
Semua dokumen kerja ada di `docs/`, bukan di `.hermes/`:
|
||||
|
||||
```
|
||||
docs/
|
||||
├── README.md # Panduan workflow (spec-driven + todo)
|
||||
├── spec-template.md # Template spec standar
|
||||
├── todo-template.md # Template todo list
|
||||
└── specs/ # Semua spec & implementation plan
|
||||
├── YYYY-MM-DD_<slug>-spec.md # Spec (apa & mengapa)
|
||||
└── YYYY-MM-DD_<slug>.md # Plan/fix (spec + plan dalam satu file)
|
||||
```
|
||||
|
||||
Service-specific docs: `services/<service>/docs/specs/`.
|
||||
|
||||
## Workflow: Spec-Driven Development
|
||||
|
||||
### Checklist wajib untuk setiap perubahan signifikan
|
||||
|
||||
1. **Tulis spec** → `docs/specs/YYYY-MM-DD_<slug>-spec.md`
|
||||
2. **Tulis `todo.md`** → daftar task konkret yang bisa diceklis
|
||||
3. **Verifikasi facts** → baca kode aktual, konfirmasi referensi file:line
|
||||
4. **Keputusan desain** → pilih approach, dokumentasikan alternatif yang ditolak
|
||||
5. **Implementasi** → ikuti spec + todo step-by-step
|
||||
6. **Verifikasi** → jalankan semua verification steps dari spec
|
||||
7. **Commit** → reference spec di commit message
|
||||
|
||||
### Todo list (`todo.md`)
|
||||
|
||||
Setiap tugas berjalan WAJIB punya `todo.md`. Format:
|
||||
|
||||
```markdown
|
||||
# Todo — <judul tugas>
|
||||
|
||||
## Task
|
||||
- [ ] Tulis spec
|
||||
- [ ] Verifikasi facts (baca kode: file:line)
|
||||
- [ ] Implementasi tahap 1: ...
|
||||
- [ ] Implementasi tahap 2: ...
|
||||
- [ ] Verifikasi: pnpm typecheck / lint / build / test
|
||||
- [ ] Commit + push
|
||||
```
|
||||
|
||||
Aturan:
|
||||
- Task harus **konkret & verifiable** — bukan "fix bug", tapi "ubah X di file Y".
|
||||
- Ceklis `[x]` saat selesai, jangan menunggu batch di akhir.
|
||||
- Gunakan `docs/todo-template.md` sebagai template.
|
||||
- Simpan `todo.md` di `docs/` untuk tugas lintas-service, atau di
|
||||
`services/<service>/docs/` untuk tugas satu service.
|
||||
|
||||
### Spec template
|
||||
|
||||
Salin `docs/spec-template.md` untuk setiap spec baru. Sections:
|
||||
|
||||
1. **Problem** — apa yang rusak/missing, dengan evidence
|
||||
2. **Root cause** — analisis teknis (bukan symptom)
|
||||
3. **Behavior target** — perilaku setelah fix, daftar verifiable
|
||||
4. **Verified facts** — fakta dari pembacaan kode, citation ke file:line
|
||||
5. **Keputusan desain** — pilihan + rationale + alternatif ditolak
|
||||
6. **Perubahan file** — semua file yang disentuh, per service
|
||||
7. **Schema/type changes** — perubahan tipe/DB
|
||||
8. **Verification** — command executable + expected outcome
|
||||
|
||||
### Kapan perlu spec + todo.md
|
||||
|
||||
Perlu: fitur baru, bug fix non-trivial, refactor behavior-changing, perubahan
|
||||
DB schema, perubahan API contract, perubahan arsitektur.
|
||||
|
||||
Tidak perlu: typo fix, dep bump, format/lint auto-fix, test-only, README update.
|
||||
|
||||
Baca selengkapnya di `docs/README.md`.
|
||||
|
||||
## Coding conventions
|
||||
|
||||
### General
|
||||
|
||||
- **TypeScript strict mode** — semua service
|
||||
- **Biome** — formatting + linting (`pnpm format`, `pnpm lint`)
|
||||
- **Bun** — package manager (`bun install`, `bun run`)
|
||||
- **Bisa bilingual** — code comments & specs boleh Indonesia/English
|
||||
|
||||
### Per-service conventions
|
||||
|
||||
#### discord-gateway
|
||||
- **Event-driven** — no HTTP server (except metrics). Listeners → Redis pub/sub.
|
||||
- **Module pattern**: `src/modules/<module>/` — each module encapsulates own logic.
|
||||
- **Piscina pools**: text pool (4 threads) + media pool (2 threads), each with own pg Pool.
|
||||
- **Logger**: `createChildLogger('module-name')` — never raw `console`.
|
||||
- **Config**: Zod-validated env in `shared/config/index.ts` — single source of truth.
|
||||
- **DB**: Drizzle ORM. Migrations in `drizzle/migrations/`.
|
||||
- **Invariant**: LLM is the only judge. Never reintroduce regex content classification.
|
||||
- **Invariant**: Discord tokens sanitized before reaching LLM.
|
||||
- Lihat `services/discord-gateway/AGENTS.md` untuk detail.
|
||||
|
||||
#### backend
|
||||
- **Modular MVC**: `modules/<module>/` — schema → repository → service → controller → routes.
|
||||
- **No cross-module repo imports** — each module owns its data.
|
||||
- **Data flows up only**: Repository → Service → Controller.
|
||||
- **Error hierarchy**: `AppError` subclasses with code + statusCode.
|
||||
- **Config**: Zod-validated env in `shared/config/index.ts`.
|
||||
- **API**: oRPC for type-safe procedures + standard Express routes.
|
||||
- Lihat `services/backend/AGENTS.md` untuk detail.
|
||||
|
||||
#### frontend
|
||||
- **SSR-first**: `page.tsx` (server component) → fetch via `src/lib/api/server.ts` → pass to `view.tsx` (client).
|
||||
- **No auth**: all endpoints public.
|
||||
- **Never hardcode host**: same-origin or `GMW_BACKEND_URL` only.
|
||||
- **WebSocket**: `src/lib/ws/` — auto-reconnecting, typed events.
|
||||
- **Local dev**: `NEXT_PUBLIC_API_URL`, `NEXT_PUBLIC_WS_URL`, `GMW_BACKEND_URL`.
|
||||
- Lihat `services/frontend/AGENTS.md` untuk detail Next.js rules.
|
||||
|
||||
## Build & Verify
|
||||
|
||||
```bash
|
||||
# Per service (run from service root)
|
||||
pnpm typecheck # TypeScript strict
|
||||
pnpm lint # Biome check
|
||||
pnpm build # Compile
|
||||
pnpm test # Vitest (gateway & backend only)
|
||||
pnpm format # Biome auto-format
|
||||
```
|
||||
|
||||
## Commit conventions
|
||||
|
||||
```
|
||||
<type>(<scope>): <subject>
|
||||
|
||||
<optional body>
|
||||
|
||||
Ref: docs/specs/<spec-file>.md
|
||||
```
|
||||
|
||||
Types: `feat`, `fix`, `refactor`, `test`, `docs`, `chore`, `build`, `ci`
|
||||
|
||||
## Deployment
|
||||
|
||||
CI/CD: GitHub Actions → build → deploy to production server via Nix flakes.
|
||||
- Gateway: `nixos-rebuild` or `systemctl restart gmw-discord-gateway`
|
||||
- Backend: `nixos-rebuild` or `systemctl restart gmw-backend`
|
||||
- Frontend: Next.js standalone, proxied by nginx :4009
|
||||
|
||||
## Remember
|
||||
|
||||
- Spec dulu, code belakangan.
|
||||
- Todo list (`todo.md`) wajib untuk tugas yang berjalan — ceklis tiap selesai.
|
||||
- Verified facts harus dari pembacaan kode aktual, bukan asumsi.
|
||||
- Setiap change harus verifiable — tulis command di spec.
|
||||
- One spec = one focused change. Don't mix unrelated features.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Roadmap
|
||||
|
||||
What is built, what is next, and what has been deliberately declined.
|
||||
|
||||
---
|
||||
|
||||
## Next
|
||||
|
||||
_Empty._
|
||||
@@ -0,0 +1,23 @@
|
||||
# TODO
|
||||
|
||||
Next up. One item, one outcome, verifiable when done.
|
||||
|
||||
Longer-term direction lives in [ROADMAP.md](ROADMAP.md).
|
||||
|
||||
---
|
||||
|
||||
## Now
|
||||
|
||||
_Empty._
|
||||
|
||||
---
|
||||
|
||||
## Next
|
||||
|
||||
_Empty._
|
||||
|
||||
---
|
||||
|
||||
## Maintenance
|
||||
|
||||
_Empty._
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
# docs/ — Spec-Driven Development Hub
|
||||
|
||||
Direktori ini adalah pusat dari workflow **spec-driven development** di GMW.
|
||||
Semua perubahan signifikan dimulai dari satu spec, **sebelum kode ditulis**,
|
||||
dan setiap tugas yang berjalan dilacak lewat `todo.md`.
|
||||
|
||||
## Struktur
|
||||
|
||||
```
|
||||
docs/
|
||||
├── README.md # Dokumen ini
|
||||
├── spec-template.md # Template standar untuk semua spec
|
||||
├── todo-template.md # Template todo list
|
||||
└── specs/ # Semua spec dan implementation plan
|
||||
├── YYYY-MM-DD_<slug>-spec.md # Spec (apa & mengapa)
|
||||
└── YYYY-MM-DD_<slug>.md # Plan/fix (bisa langsung spec+plan)
|
||||
```
|
||||
|
||||
### Naming convention
|
||||
|
||||
```
|
||||
YYYY-MM-DD_<slug>-spec.md # Spec baru untuk fitur/fix
|
||||
YYYY-MM-DD_<slug>.md # Plan yang sudah include spec di dalamnya
|
||||
```
|
||||
|
||||
Contoh:
|
||||
- `2026-08-30_recordings-v2-features-spec.md` — spec untuk Recordings v2
|
||||
- `2026-08-24-attachment-delay-fix.md` — spec + plan untuk attachment delay fix
|
||||
|
||||
### Service-specific docs
|
||||
|
||||
```
|
||||
services/<service>/docs/specs/ # Spec yang spesifik untuk 1 service
|
||||
```
|
||||
|
||||
## Workflow: Spec-Driven Development
|
||||
|
||||
### Prinsip utama
|
||||
|
||||
> **No code without a spec.** Semua perubahan signifikan harus punya spec
|
||||
> terlebih dahulu. Spec adalah kontrak: apa yang akan dibangun, mengapa,
|
||||
> dan bagaimana memverifikasinya.
|
||||
|
||||
### Todo list (`todo.md`) — WAJIB
|
||||
|
||||
Setiap tugas berjalan (implementasi fitur/fix) harus punya `todo.md`:
|
||||
|
||||
```markdown
|
||||
# Todo — <judul tugas>
|
||||
|
||||
## Task
|
||||
- [ ] Tulis spec
|
||||
- [ ] Verifikasi facts (baca kode: file:line)
|
||||
- [ ] Implementasi tahap 1: ...
|
||||
- [ ] Implementasi tahap 2: ...
|
||||
- [ ] Verifikasi: pnpm typecheck / lint / build / test
|
||||
- [ ] Commit + push
|
||||
```
|
||||
|
||||
Aturan `todo.md`:
|
||||
- Task **konkret & verifiable** — "ubah X di file Y", bukan "fix bug".
|
||||
- Satu task `[in_progress]` pada satu waktu; ceklis `[x]` segera setelah selesai.
|
||||
- Gunakan `docs/todo-template.md` sebagai template.
|
||||
- Simpan `todo.md` di `docs/` (tugas lintas-service) atau di
|
||||
`services/<service>/docs/` (tugas satu service).
|
||||
|
||||
### Kapan perlu spec + todo
|
||||
|
||||
| Perlu spec + todo | Tidak perlu |
|
||||
|---|---|
|
||||
| Fitur baru | Typo fix |
|
||||
| Bug fix non-trivial | Dependency bump (dependabot) |
|
||||
| Refactor yang mengubah behavior | Format/lint auto-fix |
|
||||
| Perubahan DB schema | Test-only change |
|
||||
| Perubahan API contract | README/doc update |
|
||||
| Perubahan arsitektur | Variable rename |
|
||||
|
||||
### Workflow step-by-step
|
||||
|
||||
1. **Tulis spec** — Salin `docs/spec-template.md`, isi semua section yang relevan.
|
||||
- **Verified facts**: baca kode yang terpengaruh, catat temuan dengan file:line.
|
||||
- **Root cause**: analisis sebab (jangan hanya describe symptom).
|
||||
- **Decisions**: pilih pendekatan, jelaskan alternatif yang ditolak.
|
||||
- **Verification**: command executable, bukan "should work".
|
||||
- Simpan sebagai `docs/specs/YYYY-MM-DD_<slug>-spec.md`.
|
||||
|
||||
2. **Tulis `todo.md`** — Pecah spec jadi task konkret yang bisa diceklis
|
||||
(gunakan `docs/todo-template.md`), dengan urutan implementasi yang logis.
|
||||
|
||||
3. **Review spec** — Baca ulang spec sendiri. Cek:
|
||||
- Apakah verified facts benar-benar verified (bukan asumsi)?
|
||||
- Apakah file changes lengkap (tidak ada yang terlewat)?
|
||||
- Apakah verification steps executable?
|
||||
- Jika spec untuk user request: pastikan user setuju dengan approach.
|
||||
|
||||
4. **Implement** — Ikuti spec + todo step-by-step. Ceklis `[x]` setiap task
|
||||
yang selesai. Jika menemukan sesuatu yang berubah dari asumsi spec,
|
||||
**update spec dulu**, baru implement.
|
||||
|
||||
5. **Verify** — Jalankan semua verification steps di spec. Catat hasilnya.
|
||||
|
||||
6. **Commit** — Reference spec di commit message:
|
||||
```
|
||||
feat(module): deskripsi singkat
|
||||
|
||||
Ref: docs/specs/YYYY-MM-DD_<slug>-spec.md
|
||||
```
|
||||
|
||||
### Tips menulis spec yang baik
|
||||
|
||||
- **Evidence-based**: setiap klaim harus ada sumbernya (file:line, log, error).
|
||||
- **Actionable**: orang lain (atau AI agent) harus bisa implementasi dari spec saja.
|
||||
- **Verifiable**: setiap requirement harus bisa di-test secara eksplisit.
|
||||
- **Scoped**: satu spec = satu perubahan terfokus. Jangan campur 3 fitur dalam 1 spec.
|
||||
- **Bahasa campuran**: narrative boleh Indonesia/English, istilah teknis pakai English.
|
||||
|
||||
### Role of AI agents
|
||||
|
||||
AI coding agents (Shiro Neko / Hermes) **harus**:
|
||||
- Membaca spec sebelum menulis kode
|
||||
- Membuat & meng-update `todo.md` untuk setiap tugas berjalan
|
||||
- Verifikasi fakta dari kode aktual (bukan dari asumsi training data)
|
||||
- Update spec jika temuan baru mengubah approach
|
||||
- Jalankan verification steps setelah implementasi
|
||||
- Reference spec di commit message
|
||||
|
||||
AI agents **tidak boleh**:
|
||||
- Langsung implement tanpa membaca/menulis spec
|
||||
- Mengabaikan verified facts yang bertentangan dengan asumsi
|
||||
- Skip verification steps
|
||||
- Mengerjakan tugas tanpa todo list yang jelas
|
||||
@@ -0,0 +1,80 @@
|
||||
# Spec: <Judul singkat, aktif, deskriptif>
|
||||
|
||||
Status: **DRAFT** | **APPROVED** | **IN PROGRESS** | **DONE**
|
||||
Date: YYYY-MM-DD
|
||||
Author: <nama/agent>
|
||||
Related: <link ke spec/plan terkait jika ada>
|
||||
Todo: <link ke todo.md untuk tugas ini, jika ada>
|
||||
|
||||
## Problem
|
||||
|
||||
<Deskripsi masalah atau keinginan. Apa yang rusak? Apa yang belum ada?
|
||||
Sebutkan siapa yang melaporkan (user/bot/audit) dan konteksnya.
|
||||
Gunakan **evidence**: log, error, metrik, atau observasi langsung.>
|
||||
|
||||
## Root cause
|
||||
|
||||
<Analisis teknis mengapa masalah ini terjadi. Jika bug: trace dari symptom ke cause.
|
||||
Jika fitur baru: jelaskan gap saat ini. Referensikan file + line number yang relevan.>
|
||||
|
||||
## Behavior target
|
||||
|
||||
<Deskripsi eksplisit perilaku yang diharapkan setelah fix/fitur.
|
||||
Buat list bernomor. Setiap item harus bisa di-verify.>
|
||||
|
||||
## Verified facts
|
||||
|
||||
>Fakta-fakta teknis yang **sudah diverifikasi** dari pembacaan kode, log produksi,
|
||||
>atau eksperimen langsung. Setiap fakta harus citation ke file:line.
|
||||
>Jika belum diverifikasi, tulis `UNVERIFIED` dan rencana verifikasi.
|
||||
|
||||
- `path/to/file.ts:42` — <apa yang terjadi di sini>
|
||||
- `path/to/file.ts:88` — <apa yang terjadi di sini>
|
||||
- Kolom DB `table.column` — <tipe, constraint, index>
|
||||
- Config `ENV_VAR` default <value> — <dibaca di mana>
|
||||
|
||||
## Keputusan desain
|
||||
|
||||
<Pilihan desain yang dibuat, beserta rationale (mengapa bukan alternatif lain).
|
||||
Format: nomor, judul singkat, penjelasan.>
|
||||
|
||||
1. **<Judul keputusan>**: <penjelasan + rationale>
|
||||
- Alternatif yang ditolak: <apa + mengapa ditolak>
|
||||
|
||||
## Perubahan file
|
||||
|
||||
>Daftar **semua file** yang perlu diubah/ditambah/dihapus, dikelompokkan per service.
|
||||
>Untuk setiap file: sebutkan **apa yang berubah** (bukan copy-paste kode).
|
||||
|
||||
### Gateway (`services/discord-gateway/`)
|
||||
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
|
||||
|
||||
### Backend (`services/backend/`)
|
||||
- `src/modules/<module>/<file>.ts` — <ringkasan perubahan>
|
||||
|
||||
### Frontend (`services/frontend/`)
|
||||
- `src/<path>/<file>.ts|tsx` — <ringkasan perubahan>
|
||||
|
||||
### Database / Config
|
||||
- <migration file atau config change jika ada>
|
||||
|
||||
## Schema/type changes
|
||||
|
||||
<Perubahan tipe, interface, atau DB schema. Jika tidak ada, tulis "TIDAK ada perubahan.">
|
||||
|
||||
## Verification
|
||||
|
||||
>**Harus spesifik dan executable.** Jangan tulis "works correctly".
|
||||
>Tulis command yang bisa dijalankan dan expected outcome.
|
||||
|
||||
- **Typecheck**: `<service> pnpm typecheck` — clean
|
||||
- **Lint**: `<service> pnpm lint` — no errors
|
||||
- **Build**: `<service> pnpm build` — compiles
|
||||
- **Test**: `<service> pnpm test` — all pass (+ test baru jika ada)
|
||||
- **Smoke test**: <langkah manual untuk verifikasi visual/behavioral>
|
||||
- **DB**: <query untuk cek data jika applicable>
|
||||
- **Deploy**: CI green → deploy → cek <specific observable>
|
||||
|
||||
## Notes (opsional)
|
||||
|
||||
<Catatan tambahan: risiko, fallback, future work, atau hal yang sengaja di-scope-out.>
|
||||
@@ -3,8 +3,8 @@
|
||||
Status: **P1–P3 DONE + 4th CRITICAL FIX deployed (f1a7b0c2); DAVE Ready + MLS handshake CONFIRMED live; P4 = waiting on active streamer to confirm video-burst→mp4**
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
|
||||
`.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
|
||||
Related: `docs/specs/2026-08-31_video-receive-eager-selfbot-connection-spec.md` (superseded by this)
|
||||
`docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, selfbot path — dead)
|
||||
|
||||
## Problem / Ground truth (established from live logs 2026-08-31)
|
||||
GMW must record OTHER members' screen-share + camera video in a voice channel it
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
|
||||
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (Phase C build, made Option A this fix)
|
||||
|
||||
## Symptom (from live logs, 2026-08-31 ~12:34)
|
||||
A user was actively screen-sharing + on camera in the recorded voice channel.
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED (not built)
|
||||
Date: 2026-08-31
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
|
||||
Related: `docs/specs/2026-08-30_video-record-receive-spec.md` (Phase A/B — raw UDP hook, superseded for receive)
|
||||
|
||||
## TL;DR — what changed vs Phase A/B
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
Status: PLANNED (not yet built)
|
||||
Date: 2026-09-02
|
||||
Author: Hermes
|
||||
Related: `.hermes/plans/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
|
||||
Related: `docs/specs/2026-08-31_video-receive-phaseC-spec.md` (auto-receive, superseded
|
||||
for selfbot), `gmw-ops/references/selfbot-presence-detection-limits.md`,
|
||||
`gmw-ops/references/discord-voice-fork-video-receive.md`
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
# Todo — <judul tugas>
|
||||
|
||||
Status: **IN PROGRESS** | **BLOCKED** | **DONE**
|
||||
Spec: <link ke spec: `docs/specs/YYYY-MM-DD_<slug>-spec.md`>
|
||||
Date: YYYY-MM-DD
|
||||
|
||||
## Task
|
||||
|
||||
> Urutan implementasi sesuai spec. Task harus konkret & verifiable
|
||||
> ("ubah X di file Y"), bukan "fix bug". Satu task dikerjakan pada satu
|
||||
> waktu; ceklis `[x]` segera setelah selesai.
|
||||
|
||||
- [ ] Tulis spec
|
||||
- [ ] Verifikasi facts (baca kode: `file:line`)
|
||||
- [ ] Implementasi tahap 1: <apa + file mana>
|
||||
- [ ] Implementasi tahap 2: <apa + file mana>
|
||||
- [ ] Implementasi tahap 3: <apa + file mana>
|
||||
- [ ] Typecheck: `pnpm typecheck` — clean
|
||||
- [ ] Lint: `pnpm lint` — no errors
|
||||
- [ ] Test: `pnpm test` — all pass
|
||||
- [ ] Build: `pnpm build` — compiles
|
||||
- [ ] Daftar ulang hasil verifikasi di spec
|
||||
- [ ] Commit + push (reference spec di commit message)
|
||||
|
||||
## Notes
|
||||
|
||||
<Blocking issue, temuan yang mengubah approach, atau hal yang perlu dicatat.>
|
||||
@@ -0,0 +1,117 @@
|
||||
# Backend Service — Agent Guide
|
||||
|
||||
> **Read `../../AGENTS.md` first.** This file adds backend-specific conventions.
|
||||
|
||||
Backend service: Express HTTP server + WebSocket, serves the GMW dashboard API.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
pnpm typecheck # tsc --noEmit
|
||||
pnpm lint # biome check --diagnostic-level=error .
|
||||
pnpm build # tsc
|
||||
pnpm test # vitest run
|
||||
pnpm format # biome format --write .
|
||||
```
|
||||
|
||||
## Architecture (Modular MVC)
|
||||
|
||||
```
|
||||
src/
|
||||
├── shared/ # Infrastructure (no business logic)
|
||||
│ ├── config/index.ts # Zod-validated env
|
||||
│ ├── database/ # Drizzle ORM + pg Pool
|
||||
│ ├── errors/index.ts # AppError hierarchy
|
||||
│ ├── logger/index.ts # pino + createChildLogger()
|
||||
│ ├── middlewares/index.ts # errorHandler, asyncHandler, rateLimit
|
||||
│ └── utils/ # Pagination, messageMapper
|
||||
├── modules/ # Feature modules
|
||||
│ └── <module>/
|
||||
│ ├── <module>.schema.ts # Zod validation schemas
|
||||
│ ├── <module>.repository.ts # DB operations only
|
||||
│ ├── <module>.service.ts # Business logic
|
||||
│ ├── <module>.controller.ts # HTTP handlers
|
||||
│ └── routes/index.ts # Express router
|
||||
├── http/ # app.ts (factory) + server.ts (startup)
|
||||
├── ws/ # WebSocket server + Redis bridge
|
||||
└── index.ts # Entry point
|
||||
```
|
||||
|
||||
## Dependency rules
|
||||
|
||||
- Controller → Service → Repository → Database
|
||||
- No cross-module repo imports (each module owns its data)
|
||||
- No HTTP in Service layer (no req/res)
|
||||
- No DB in Controller layer
|
||||
- Any layer → Config, Logger, Errors
|
||||
|
||||
## API: oRPC + Express
|
||||
|
||||
- **oRPC** (`src/orpc/router.ts` + `src/orpc/ws.ts`): type-safe procedures for frontend
|
||||
- **Express routes** (`src/modules/*/routes/`): REST endpoints under `/api/`
|
||||
- **WebSocket** (`src/ws/`): Redis bridge → broadcast to connected browsers
|
||||
- Never invent endpoints. Match what the frontend calls (`src/lib/api/`).
|
||||
|
||||
## Key modules
|
||||
|
||||
| Module | Purpose | DB tables |
|
||||
|---|---|---|
|
||||
| messages | Store & query Discord messages | `messages`, `ai_moderations`, `ai_moderation_flags` |
|
||||
| moderation | Moderation actions & metrics | `ai_moderations`, `moderation_actions` |
|
||||
| media | Media file management | `media_attachments` |
|
||||
| voice | Live speakers + recordings | `voice_recordings` |
|
||||
| recordings | Recordings API | `voice_recordings` |
|
||||
| dashboard | Stats aggregation | Various (read-only) |
|
||||
| knowledge | Semantic search | Qdrant vector DB |
|
||||
| chatbot | AI chatbot with tools | `chatbot_history` |
|
||||
| health | Health checks + metrics | Various |
|
||||
| analysis | Text analysis cache | `text_analysis_cache` |
|
||||
| ui-state | Persist UI preferences | `ui_state` |
|
||||
|
||||
## Config (env vars)
|
||||
|
||||
All validated via Zod in `shared/config/index.ts`. Key vars:
|
||||
|
||||
- `WEBSERVER_PORT` (default 4001)
|
||||
- `DATABASE_URL` or individual `DATABASE_HOST/PORT/NAME/USER/PASSWORD`
|
||||
- `REDIS_URL` — for pub/sub with gateway
|
||||
- `MONITOR_GUILD_ID` — primary Discord guild
|
||||
- `ADMIN_PASSWORD` — admin endpoints
|
||||
|
||||
## Data contract with frontend
|
||||
|
||||
The frontend fetches via `src/lib/api/server.ts` (SSR, server-side) and
|
||||
`src/lib/api/client.ts` (browser, same-origin through proxy).
|
||||
|
||||
**Do not change response shapes without updating both sides.** Check
|
||||
`services/frontend/src/lib/types/` for frontend type definitions.
|
||||
|
||||
## Redis channels (inbound from gateway)
|
||||
|
||||
```
|
||||
discord:message:{created,updated,deleted,analyzed}
|
||||
discord:attachment:{created,uploaded}
|
||||
discord:voice:{started,stopped,uploaded,active_user,pcm,analyzed}
|
||||
discord:analysis:queue_status
|
||||
discord:reaction:{added,removed}
|
||||
discord:thread:{created,deleted,updated}
|
||||
discord:channel_topic:updated
|
||||
discord:presence:updated
|
||||
discord:guild_member:{added,removed}
|
||||
```
|
||||
|
||||
Canonical names: `src/shared/redis-channels.ts`.
|
||||
|
||||
## Testing
|
||||
|
||||
- Vitest for unit tests
|
||||
- Mock database and external services
|
||||
- Test files: `src/modules/<module>/*.test.ts` or `tests/*.test.ts`
|
||||
- Run: `pnpm test`
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
- **oRPC vs REST**: check both routers when adding an endpoint
|
||||
- **Redis channel mismatch**: gateway publishes → backend subscribes. Channel
|
||||
names must match exactly (see `redis-channels.ts` in BOTH services)
|
||||
- **DB pool**: backend uses one pool (main thread). Gateway has per-piscina-thread pools.
|
||||
@@ -0,0 +1,153 @@
|
||||
# Discord Gateway — Agent Guide
|
||||
|
||||
> **Read `../../AGENTS.md` first.** This file adds gateway-specific conventions.
|
||||
|
||||
Event-driven microservice: captures Discord events, runs AI moderation, publishes to Redis.
|
||||
|
||||
## Quick reference
|
||||
|
||||
```bash
|
||||
pnpm typecheck # tsc --noEmit
|
||||
pnpm lint # biome check --diagnostic-level=error .
|
||||
pnpm build # tsc
|
||||
pnpm test # vitest run
|
||||
pnpm format # biome format --write .
|
||||
```
|
||||
|
||||
## Architecture (Event-driven)
|
||||
|
||||
```
|
||||
src/
|
||||
├── index.ts # Entry point → initializeDiscordGateway()
|
||||
├── app/
|
||||
│ ├── bootstrap.ts # Wires client, DB, Redis, workers, schedulers
|
||||
│ ├── shutdown.ts # Graceful shutdown
|
||||
│ └── retention.ts # Expired-record cleanup
|
||||
├── shared/
|
||||
│ ├── config/index.ts # Zod-validated env (SINGLE source of truth)
|
||||
│ ├── database/ # Drizzle ORM + pg Pool + migrations
|
||||
│ ├── logger/index.ts # pino + createChildLogger()
|
||||
│ ├── errors/index.ts # AppError hierarchy
|
||||
│ ├── utils/ # retry, pagination
|
||||
│ ├── discord/clientOptions.ts # discord.js-selfbot-v13 options
|
||||
│ ├── uploader.ts # Attachment upload helper
|
||||
│ ├── redis-channels.ts # Redis channel-name constants
|
||||
│ └── moderation-types.ts # Shared AI analysis types
|
||||
├── modules/
|
||||
│ ├── ai-moderation/ # LLM moderation pipeline (largest module)
|
||||
│ ├── message-capture/ # Discord event listeners + DB store
|
||||
│ ├── voice-recording/ # Voice connect + Opus→OGG recording
|
||||
│ │ └── recorder/ # decoder, segment, session, uploader, oggCrc
|
||||
│ ├── voice-pcm-ws/ # Real-time PCM → backend WebSocket
|
||||
│ ├── attachment-upload/ # Download + sharp resize + upload
|
||||
│ ├── event-broadcaster/ # Redis pub/sub publisher
|
||||
│ ├── command-handler/ # Backend→gateway Redis commands
|
||||
│ ├── reaction-tracking/ # Reaction events
|
||||
│ ├── thread-tracking/ # Thread events
|
||||
│ ├── user-presence/ # Presence/status events
|
||||
│ ├── channel-topic/ # Channel topic events
|
||||
│ ├── guild-member-events/ # Member join/leave
|
||||
│ └── gateway-metrics/ # Prometheus /metrics (port 4016)
|
||||
└── tests/ # Vitest suites
|
||||
```
|
||||
|
||||
## Key invariants (DO NOT BREAK)
|
||||
|
||||
1. **LLM is the only judge.** Failed LLM → `status:"error"` + recovery retry.
|
||||
**Never** reintroduce regex/heuristic content classification.
|
||||
2. **Discord tokens sanitized** before reaching LLM (`discordTokens.ts`).
|
||||
3. **Semantic cache is batched** — one embed call + one Qdrant batch search.
|
||||
4. **Streaming is mandatory** against the omniroute base URL.
|
||||
|
||||
## AI moderation pipeline
|
||||
|
||||
```
|
||||
aiAnalyzer.ts → batchScheduler.ts → batchProcessor.ts → individualFallbackProcessor.ts
|
||||
↓ ↓ ↓ ↓
|
||||
moderationOrchestrator.ts → (hash cache → Qdrant → LLM)
|
||||
↓ ↓ ↓
|
||||
textBatchProcessor.ts mediaBatchProcessor.ts llmClient.ts
|
||||
embeddingClient.ts
|
||||
qdrantClient.ts
|
||||
```
|
||||
|
||||
- Entry: `aiAnalyzer.ts` (`queueMessageAnalysis`, `startPendingAIAnalysisWorker`)
|
||||
- Concurrency: LLM semaphore (`AI_LLM_MAX_CONCURRENT`, default 5)
|
||||
- Piscina: text pool (4 threads) + media pool (2 threads)
|
||||
- **Each worker thread has its own pg Pool** (min 0, grows to `POSTGRES_POOL_MAX`)
|
||||
|
||||
## Voice recording pipeline
|
||||
|
||||
```
|
||||
receiver.speaking "start" → speakingHandler(userId)
|
||||
→ collectUserMetadata → receiver.subscribe → PacketFilter → oggPacketStream
|
||||
→ SegmentManager.open → OggLogicalBitstream → .ogg file
|
||||
→ data: rotateIfNeeded + decoder.write
|
||||
→ end: finalizeSegment → upload + transcribe
|
||||
```
|
||||
|
||||
Key files:
|
||||
- `voiceController.ts` — connect/disconnect/list
|
||||
- `recorder.ts` — orchestration
|
||||
- `recorder/segment.ts` — segment rotation
|
||||
- `recorder/sessionRecording.ts` — session management
|
||||
- `recorder/uploader.ts` — upload to storage
|
||||
- `voiceTranscriber.ts` — Whisper transcription (if enabled)
|
||||
|
||||
## Module: message-capture
|
||||
|
||||
- `messageCapture.ts` — Discord event listeners (messageCreate/Update/Delete)
|
||||
- `messageStore.ts` — DB operations
|
||||
- `messageMetadata.ts` — metadata extraction
|
||||
- `messagesDb.ts` / `messagesCrud.ts` — DB schema operations
|
||||
- `archiveEmbedder.ts` — Qdrant embedding (respect age-restricted guard)
|
||||
- `retentionDb.ts` / `reviewsDb.ts` / `attachmentsDb.ts` — auxiliary tables
|
||||
|
||||
## Redis channels (outbound to backend)
|
||||
|
||||
See `src/shared/redis-channels.ts` for canonical names. Examples:
|
||||
```
|
||||
discord:message:created, discord:voice:active_user, discord:attachment:uploaded
|
||||
```
|
||||
|
||||
## Config (env vars)
|
||||
|
||||
All validated via Zod in `shared/config/index.ts`. Critical:
|
||||
|
||||
- `DISCORD_TOKEN` — selfbot token
|
||||
- `MONITOR_GUILD_ID` — primary guild
|
||||
- `DATABASE_URL` — PostgreSQL
|
||||
- `REDIS_URL` — pub/sub to backend
|
||||
- `AI_ANALYSIS_ENABLED` — master toggle for AI moderation
|
||||
- `AI_LLM_BASE_URL` / `AI_LLM_API_KEY` — LLM router
|
||||
- `AI_VOICE_TRANSCRIPTION_ENABLED` — toggle Whisper transcription
|
||||
- `PISCINA_MAX_THREADS` / `PISCINA_MEDIA_MAX_THREADS` — worker pool sizing
|
||||
|
||||
## Concurrency model
|
||||
|
||||
- Main thread: event loop + LLM semaphore
|
||||
- Text Piscina pool: `PISCINA_MAX_THREADS` (default 4)
|
||||
- Media Piscina pool: `PISCINA_MEDIA_MAX_THREADS` (default 2)
|
||||
- Batch routed to media pool if ANY message has attachment/sticker/embed
|
||||
- Each worker thread initializes own pg Pool (min 0, grows to `POSTGRES_POOL_MAX`)
|
||||
- MemoryMax: 1G (service systemd limit)
|
||||
|
||||
## Testing
|
||||
|
||||
- Vitest. Test files: `tests/<name>.test.ts`
|
||||
- Run: `pnpm test`
|
||||
- Key test areas: batch operations, cache guards, image/video handling, context enrichment
|
||||
- Unit tests for pure helpers (classifier, normalize, parse), integration for pipeline stages
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
- **Piscina pool isolation**: worker threads are NOT the main thread. Cannot
|
||||
share state via module-level variables. Use DB or Redis for cross-thread state.
|
||||
- **AfterSilence race**: `@discordjs/voice` AfterSilence can fail to emit "end"
|
||||
on disconnect. Always have a watchdog/timeout.
|
||||
- **Cache eviction**: LRU caches (user metadata, term glossary) evict at max size.
|
||||
Don't assume cache hit after eviction.
|
||||
- **Circuit breaker**: per-conversation CB opens after repeated failures.
|
||||
Check `circuitBreaker.ts` state when debugging "missing analysis".
|
||||
- **Gateway≠Backend schema**: both have `redis-channels.ts` and `moderation-types.ts`.
|
||||
Keep them in sync manually.
|
||||
@@ -1,3 +1,5 @@
|
||||
@../../AGENTS.md
|
||||
|
||||
# Bete Frontend — Project Overview
|
||||
|
||||
Next.js 16 (App Router), React 19, TypeScript strict, Tailwind v4, shadcn/ui, base-ui.
|
||||
|
||||
Reference in New Issue
Block a user