refactor: large codebase cleanup - consolidate schemas, migrate to Drizzle ORM, extract frontend components, modernize Docker builds
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
- Consolidate all DB schema definitions into packages/shared as single source of truth - Migrate backend from raw SQL to Drizzle ORM across all modules - Extract frontend inline UI into separate component files - Refactor discord-gateway circuitBreaker into conversationState + moderationState - Convert messageStore to Proxy singleton pattern - Add validateBody/validateQuery middleware + Zod schemas for API endpoints - Modernize Docker builds with multi-stage + pnpm deploy - Migrate CI/CD from deployment to image-based pipeline - Remove 60+ unused/dead files (~15K lines) - Update color scheme from sky-blue to teal-cyan - Move DB connection management to @bete/shared/database Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
63f21513bd
commit
5802d02e29
@@ -0,0 +1,542 @@
|
||||
# CI/CD Overhaul Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Migrate from hybrid CI/CD (GitHub Actions + GitLab CI + hot-deploy) to single Gitea CI pipeline with container registry — VPS pulls only.
|
||||
|
||||
**Architecture:** Three Docker images (backend, discord-gateway, proxy) built in Gitea CI, pushed to `git.imrnes.team/MythEclipse/GMW/*`, VPS pulls and restarts via SSH. No more hot-deploy bind-mounts.
|
||||
|
||||
**Tech Stack:** Gitea CI (Act Runner, GitHub Actions-compatible syntax), Docker Buildx, Gitea Container Registry, appleboy/ssh-action
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Docker images must be self-contained (no bind-mount overlay at runtime)
|
||||
- All three images must be built from monorepo root using `infra/docker/Dockerfile.*`
|
||||
- Frontend static export built inside proxy Dockerfile (multi-stage, Next.js → Nginx)
|
||||
- Gitea CI variables: GITEA_REGISTRY_TOKEN (secret), VPS_HOST (secret), VPS_USER (secret), VPS_SSH_KEY (secret), ENV_FILE (secret), GITEA_REGISTRY (variable)
|
||||
- Registry URL: `git.imrnes.team/MythEclipse/GMW/`
|
||||
- Work on `main` branch only
|
||||
- Must preserve voice recordings volume persistence across container restarts
|
||||
|
||||
---
|
||||
### Task 1: Create Gitea CI workflow
|
||||
|
||||
**Files:**
|
||||
- Create: `.gitea/workflows/deploy.yml`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Dockerfiles at `infra/docker/Dockerfile.{backend,discord-gateway,proxy}`
|
||||
- Produces: Docker images pushed to `git.imrnes.team/MythEclipse/GMW/bete-*:latest` and `:{sha}`
|
||||
- Depends on: Task 2 (proxy Dockerfile), Task 3 (backend Dockerfile) — but workflow can reference files that are being written in the same commit
|
||||
|
||||
- [ ] **Step 1: Create `.gitea/workflows/` directory and `deploy.yml`**
|
||||
|
||||
```bash
|
||||
mkdir -p .gitea/workflows
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the workflow file**
|
||||
|
||||
Create `.gitea/workflows/deploy.yml`:
|
||||
|
||||
```yaml
|
||||
name: Build & Deploy
|
||||
run-name: "Build & Deploy ${{ gitea.sha }}"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
|
||||
jobs:
|
||||
build-and-push:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
fail-fast: false
|
||||
max-parallel: 2
|
||||
matrix:
|
||||
service: [backend, discord-gateway, proxy]
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
submodules: recursive
|
||||
|
||||
- name: Set up Docker Buildx
|
||||
uses: docker/setup-buildx-action@v4
|
||||
|
||||
- name: Login to Gitea Registry
|
||||
uses: docker/login-action@v4
|
||||
with:
|
||||
registry: ${{ vars.GITEA_REGISTRY }}
|
||||
username: ${{ gitea.actor }}
|
||||
password: ${{ secrets.GITEA_REGISTRY_TOKEN }}
|
||||
|
||||
- name: Build & Push ${{ matrix.service }}
|
||||
uses: docker/build-push-action@v7
|
||||
with:
|
||||
context: .
|
||||
file: infra/docker/Dockerfile.${{ matrix.service }}
|
||||
push: true
|
||||
tags: |
|
||||
${{ vars.GITEA_REGISTRY }}/MythEclipse/GMW/bete-${{ matrix.service }}:${{ gitea.sha }}
|
||||
${{ vars.GITEA_REGISTRY }}/MythEclipse/GMW/bete-${{ matrix.service }}:latest
|
||||
cache-from: type=gha,scope=bete-${{ matrix.service }}
|
||||
cache-to: type=gha,mode=max,scope=bete-${{ matrix.service }}
|
||||
build-args: |
|
||||
VITE_BE_API_URL=https://imphnen.asepharyana.my.id
|
||||
VITE_BE_WS_URL=wss://imphnen.asepharyana.my.id
|
||||
|
||||
deploy:
|
||||
needs: build-and-push
|
||||
runs-on: ubuntu-latest
|
||||
if: gitea.ref == 'refs/heads/main'
|
||||
steps:
|
||||
- name: Deploy to VPS
|
||||
uses: appleboy/ssh-action@v1.2.5
|
||||
env:
|
||||
ENV_FILE: ${{ secrets.ENV_FILE }}
|
||||
with:
|
||||
host: ${{ secrets.VPS_HOST }}
|
||||
username: ${{ secrets.VPS_USER }}
|
||||
key: ${{ secrets.VPS_SSH_KEY }}
|
||||
envs: ENV_FILE
|
||||
script: |
|
||||
set -eu
|
||||
APP_DIR=/opt/imphenbot
|
||||
cd "$APP_DIR/infra/docker"
|
||||
printf '%s\n' "$ENV_FILE" | tr -d '\r' > .env
|
||||
docker compose pull
|
||||
docker compose up -d --remove-orphans
|
||||
docker image prune -f
|
||||
```
|
||||
|
||||
Note: Gitea's Act Runner supports `gitea.*` context variables (`gitea.sha`, `gitea.actor`, `gitea.ref`). If `gitea.*` vars don't resolve, fall back to `github.*` equivalents (Act Runner emulates GitHub context).
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add .gitea/workflows/deploy.yml
|
||||
git commit -m "ci: add Gitea CI workflow for build & deploy
|
||||
|
||||
Gitea CI builds three Docker images (backend, discord-gateway, proxy),
|
||||
pushes to Gitea Container Registry, then deploys to VPS via SSH pull.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 2: Rewrite Dockerfile.proxy for Next.js static export
|
||||
|
||||
**Files:**
|
||||
- Rewrite: `infra/docker/Dockerfile.proxy`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `services/frontend/` (Next.js app), `packages/shared/` (workspace dep), `infra/docker/nginx/nginx.conf`
|
||||
- Produces: Nginx image serving Next.js static export at `/usr/share/nginx/html/`
|
||||
|
||||
- [ ] **Step 1: Rewrite Dockerfile.proxy**
|
||||
|
||||
Replace entire content with:
|
||||
|
||||
```dockerfile
|
||||
# ---- Stage 1: Build Next.js static export ----
|
||||
FROM node:22-slim AS frontend-builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install pnpm
|
||||
RUN corepack enable
|
||||
|
||||
# Install build essentials for native deps
|
||||
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
|
||||
python3 make g++ && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Copy dependency manifests first for layer caching
|
||||
COPY pnpm-lock.yaml pnpm-workspace.yaml package.json ./
|
||||
COPY packages/shared/package.json ./packages/shared/package.json
|
||||
COPY services/frontend/package.json ./services/frontend/package.json
|
||||
COPY services/frontend/tsconfig.json ./services/frontend/tsconfig.json
|
||||
|
||||
# Install dependencies (frontend + shared)
|
||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
||||
pnpm install --frozen-lockfile --filter './packages/shared' --filter './services/frontend'
|
||||
|
||||
# Copy source code
|
||||
COPY packages/shared/ ./packages/shared/
|
||||
COPY services/frontend/ ./services/frontend/
|
||||
|
||||
# Pass API/WS URLs as build args for the frontend
|
||||
ARG VITE_BE_API_URL
|
||||
ARG VITE_BE_WS_URL
|
||||
ENV VITE_BE_API_URL=${VITE_BE_API_URL}
|
||||
ENV VITE_BE_WS_URL=${VITE_BE_WS_URL}
|
||||
|
||||
# Build shared lib first, then frontend static export
|
||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
||||
pnpm --filter './packages/shared' run build
|
||||
RUN --mount=type=cache,id=pnpm-store,target=/root/.local/share/pnpm/store \
|
||||
pnpm --filter frontend run build
|
||||
|
||||
# ---- Stage 2: Nginx ----
|
||||
FROM nginx:alpine
|
||||
|
||||
# Nginx config (API/WS proxy + static file serving)
|
||||
COPY infra/docker/nginx/nginx.conf /etc/nginx/conf.d/default.conf
|
||||
|
||||
# Static export from frontend builder
|
||||
COPY --from=frontend-builder /app/services/frontend/out/ /usr/share/nginx/html/
|
||||
|
||||
EXPOSE 80
|
||||
|
||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=10s --retries=3 \
|
||||
CMD wget -qO- http://localhost:80/ || exit 1
|
||||
|
||||
CMD ["nginx", "-g", "daemon off;"]
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Validate nginx.conf handles static files correctly**
|
||||
|
||||
Read and confirm `infra/docker/nginx/nginx.conf`.
|
||||
|
||||
```bash
|
||||
cat infra/docker/nginx/nginx.conf
|
||||
```
|
||||
|
||||
Verify it has:
|
||||
- Static file location with `try_files $uri /index.html` (SPA fallback)
|
||||
- `/api` and `/ws` proxied to `http://backend:3000`
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add infra/docker/Dockerfile.proxy
|
||||
git commit -m "docker(proxy): rewrite for Next.js static export
|
||||
|
||||
Replaced stale Rust WASM build with multi-stage Docker build:
|
||||
stage 1 builds Next.js static export, stage 2 serves via Nginx.
|
||||
Includes VITE_BE_API_URL/VITE_BE_WS_URL build args.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 3: Add build args to Dockerfile.backend
|
||||
|
||||
**Files:**
|
||||
- Modify: `infra/docker/Dockerfile.backend`
|
||||
|
||||
- [ ] **Step 1: Add VITE build args to Dockerfile.backend**
|
||||
|
||||
Insert after `WORKDIR /app`:
|
||||
|
||||
```dockerfile
|
||||
# Build args for frontend API URLs (passed through for future use)
|
||||
ARG VITE_BE_API_URL
|
||||
ARG VITE_BE_WS_URL
|
||||
ENV VITE_BE_API_URL=${VITE_BE_API_URL}
|
||||
ENV VITE_BE_WS_URL=${VITE_BE_WS_URL}
|
||||
```
|
||||
|
||||
Note: These are consumed by the proxy Dockerfile (Task 2), not needed by backend itself but passed through the CI workflow to all three images for consistency.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add infra/docker/Dockerfile.backend
|
||||
git commit -m "docker(backend): add VITE_BE_API_URL and VITE_BE_WS_URL build args
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 4: Rewrite docker-compose.yml for Gitea registry + no bind-mounts
|
||||
|
||||
**Files:**
|
||||
- Rewrite: `infra/docker/docker-compose.yml`
|
||||
|
||||
- [ ] **Step 1: Write new docker-compose.yml**
|
||||
|
||||
Replace entire content:
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
|
||||
services:
|
||||
proxy:
|
||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-proxy:${IMAGE_TAG:-latest}
|
||||
container_name: imphenbot-proxy
|
||||
restart: unless-stopped
|
||||
labels:
|
||||
- "traefik.enable=true"
|
||||
- "traefik.http.routers.imphenbot.rule=Host(`imphnen.asepharyana.my.id`)"
|
||||
- "traefik.http.routers.imphenbot.entrypoints=websecure"
|
||||
- "traefik.http.routers.imphenbot.tls=true"
|
||||
- "traefik.http.services.imphenbot.loadbalancer.server.port=80"
|
||||
depends_on:
|
||||
- backend
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://127.0.0.1/"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
retries: 3
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 64M
|
||||
networks:
|
||||
- app-shared-net
|
||||
|
||||
backend:
|
||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-backend:${IMAGE_TAG:-latest}
|
||||
container_name: imphenbot-backend
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- .env
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
WEBSERVER_PORT: 3000
|
||||
healthcheck:
|
||||
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
start_period: 15s
|
||||
retries: 3
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 256M
|
||||
networks:
|
||||
- app-shared-net
|
||||
|
||||
discord-gateway:
|
||||
image: ${GITEA_REGISTRY}/MythEclipse/GMW/bete-discord-gateway:${IMAGE_TAG:-latest}
|
||||
container_name: imphenbot-discord-gateway
|
||||
restart: unless-stopped
|
||||
env_file:
|
||||
- .env
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
volumes:
|
||||
- recordings:/app/recordings
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "kill -0 1 || exit 1"]
|
||||
interval: 30s
|
||||
timeout: 5s
|
||||
start_period: 30s
|
||||
retries: 3
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 512M
|
||||
networks:
|
||||
- app-shared-net
|
||||
|
||||
volumes:
|
||||
recordings:
|
||||
|
||||
networks:
|
||||
app-shared-net:
|
||||
name: app-shared-net
|
||||
external: true
|
||||
```
|
||||
|
||||
Key changes:
|
||||
- Image refs: `registry.gitlab.com/...` → `${GITEA_REGISTRY}/MythEclipse/GMW/...`
|
||||
- Removed all bind-mount volumes: `./backend-dist`, `./gateway-dist`, `./frontend-dist`, `./shared-dist`
|
||||
- Changed `./recordings` bind-mount → named volume `recordings:` (persists across restarts)
|
||||
- Added `depends_on: backend` to proxy (proxy needs backend for API/WS, though Nginx handles startup gracefully)
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add infra/docker/docker-compose.yml
|
||||
git commit -m "docker(compose): switch to Gitea registry, remove bind-mounts
|
||||
|
||||
Images now come from git.imrnes.team/MythEclipse/GMW. All hot-deploy
|
||||
bind-mounts removed — containers are fully self-contained. Voice
|
||||
recordings use a named volume instead of bind-mount.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 5: Create lightweight deploy.sh
|
||||
|
||||
**Files:**
|
||||
- Create: `deploy.sh`
|
||||
|
||||
- [ ] **Step 1: Write deploy.sh**
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||
INFRA_DIR="$SCRIPT_DIR/infra/docker"
|
||||
|
||||
: "${VPS_HOST:?required}"
|
||||
: "${VPS_USER:?required}"
|
||||
: "${VPS_SSH_KEY:?required}"
|
||||
|
||||
echo "=== Deploy to $VPS_HOST ==="
|
||||
|
||||
# Copy local .env if it exists (overrides CI env)
|
||||
if [ -f "$INFRA_DIR/.env" ]; then
|
||||
scp -i "$VPS_SSH_KEY" "$INFRA_DIR/.env" "$VPS_USER@$VPS_HOST:/opt/imphenbot/infra/docker/.env"
|
||||
fi
|
||||
|
||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" << 'REMOTESCRIPT'
|
||||
set -eu
|
||||
cd /opt/imphenbot/infra/docker
|
||||
echo "=== Pulling images ==="
|
||||
docker compose pull
|
||||
echo "=== Restarting containers ==="
|
||||
docker compose up -d --remove-orphans
|
||||
echo "=== Cleaning up ==="
|
||||
docker image prune -f
|
||||
echo "=== Active containers ==="
|
||||
docker ps --filter "name=imphenbot" --format "table {{.Names}}\t{{.Image}}\t{{.Status}}"
|
||||
REMOTESCRIPT
|
||||
|
||||
echo "=== Deploy complete ==="
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Make executable**
|
||||
|
||||
```bash
|
||||
chmod +x deploy.sh
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add deploy.sh
|
||||
git commit -m "chore: rewrite deploy.sh as lightweight SSH pull script
|
||||
|
||||
Replaced hot-deploy tar-pipe script with simple SSH-based deploy
|
||||
that pulls latest images from Gitea registry and restarts containers.
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 6: Disable old CI files
|
||||
|
||||
**Files:**
|
||||
- Disable: `.github/workflows/deploy-docker.yml`
|
||||
- Keep: `.gitlab-ci.yml` if exists (already may have been removed)
|
||||
|
||||
- [ ] **Step 1: Rename GitHub Actions workflow to .disabled**
|
||||
|
||||
```bash
|
||||
mv .github/workflows/deploy-docker.yml .github/workflows/deploy-docker.yml.disabled
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Remove docker compose file's old frontend-dist directory from git** (if tracked)
|
||||
|
||||
```bash
|
||||
# Check if frontend-dist is tracked (it should be gitignored, but check)
|
||||
git ls-files infra/docker/frontend-dist 2>/dev/null || echo "Not tracked — OK"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/deploy-docker.yml.disabled
|
||||
git rm --cached .github/workflows/deploy-docker.yml 2>/dev/null || true
|
||||
git commit -m "ci: disable GitHub Actions workflow
|
||||
|
||||
Renamed to .disabled. All CI now goes through Gitea CI (.gitea/workflows/).
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 7: Update .gitignore
|
||||
|
||||
**Files:**
|
||||
- Modify: `.gitignore`
|
||||
|
||||
- [ ] **Step 1: Add .gitea exclusion note and any missing entries**
|
||||
|
||||
Read current `.gitignore`:
|
||||
|
||||
```bash
|
||||
cat .gitignore
|
||||
```
|
||||
|
||||
Then append (only if not already present):
|
||||
|
||||
```
|
||||
# Gitea workflow logs (local runners)
|
||||
.gitea/workflows/*.log
|
||||
```
|
||||
|
||||
The `.gitea/workflows/` YAML files themselves should be tracked in git.
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add .gitignore
|
||||
git commit -m "chore: update gitignore for Gitea CI artifacts
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
### Task 8: Push and verify CI pipeline
|
||||
|
||||
- [ ] **Step 1: Verify all changes**
|
||||
|
||||
```bash
|
||||
git status
|
||||
git log --oneline -10
|
||||
```
|
||||
|
||||
Expected: clean working tree, all 7 commits ready to push.
|
||||
|
||||
- [ ] **Step 2: Push to main**
|
||||
|
||||
```bash
|
||||
git push origin main
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Monitor CI run**
|
||||
|
||||
Watch Gitea CI at `https://git.imrnes.team/MythEclipse/GMW/actions`.
|
||||
|
||||
Expected outcome:
|
||||
1. `build-and-push` job runs 3 matrix builds (backend, discord-gateway, proxy) in parallel (max 2)
|
||||
2. Each image is pushed to `git.imrnes.team/MythEclipse/GMW/bete-*` with both `:latest` and `:{sha}` tags
|
||||
3. `deploy` job SSHes into VPS, pulls images, restarts containers
|
||||
4. All 3 containers `imphenbot-proxy`, `imphenbot-backend`, `imphenbot-discord-gateway` are running
|
||||
|
||||
- [ ] **Step 4: Verify containers on VPS**
|
||||
|
||||
```bash
|
||||
# SSH into VPS and check
|
||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" "
|
||||
docker ps --filter 'name=imphenbot' --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'
|
||||
docker compose -f /opt/imphenbot/infra/docker/docker-compose.yml ps
|
||||
"
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Verify no hot-deploy artifacts remain**
|
||||
|
||||
```bash
|
||||
ssh -i "$VPS_SSH_KEY" "$VPS_USER@$VPS_HOST" "
|
||||
ls -la /opt/imphenbot/infra/docker/ | grep -E 'dist$' || echo 'No dist dirs — clean'
|
||||
"
|
||||
```
|
||||
|
||||
---
|
||||
## Rollback
|
||||
|
||||
If the pipeline fails at any point:
|
||||
|
||||
1. **Fix and re-push**: Edit the broken file, commit, push to main — CI re-runs automatically
|
||||
2. **Emergency rollback**: SSH to VPS, run `docker compose up -d` with a known-good IMAGE_TAG:
|
||||
```bash
|
||||
IMAGE_TAG=<last-working-sha> docker compose up -d
|
||||
```
|
||||
3. **Restore old CI**: Move `.github/workflows/deploy-docker.yml.disabled` back and push
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,736 @@
|
||||
# Backend & Gateway Refactoring — Phase 1 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Clean up ~130 lines of dead/duplicate code, consolidate duplicated database initialization, and simplify the MessageStore layering in discord-gateway.
|
||||
|
||||
**Architecture:** Three independent tasks that can be done in any order. Task 1 consolidates database pool/drizzle init into `@bete/shared` so both services use one canonical pattern. Task 2 removes backward-compat function wrappers from `messageStore.ts`. Task 3 deletes dead files and functions.
|
||||
|
||||
**Tech Stack:** TypeScript, Node.js, Drizzle ORM, PostgreSQL, pnpm workspace
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- All imports use `.js` extensions (ESM convention)
|
||||
- Follow existing code style (Biome, 2-space indent)
|
||||
- Keep `@bete/shared` as the single source of truth for shared infrastructure
|
||||
- No package.json changes needed — `@bete/shared` already has `drizzle-orm` and `pg` as dependencies
|
||||
- Do not change any business logic — only structural refactoring
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Consolidate Database Initialization into `@bete/shared`
|
||||
|
||||
**Files:**
|
||||
- Create: `packages/shared/src/database/init.ts`
|
||||
- Modify: `packages/shared/src/database/pool.ts` — add `getPool()` export
|
||||
- Modify: `packages/shared/src/index.ts` — export new `./database/init.js`
|
||||
- Modify: `packages/shared/package.json` — add `"./database/init"` export entry
|
||||
- Modify: `services/backend/src/shared/database/index.ts` — re-export from shared
|
||||
- Modify: `services/discord-gateway/src/shared/database/drizzle.ts` — re-export from shared
|
||||
- Delete: (functions migrate, no file deletion here — both local files stay as thin wrappers)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `@bete/shared/database/init` exports:
|
||||
- `let db: ReturnType<typeof drizzle> | null` (module-level, for getDatabase())
|
||||
- `let rawPool: Pool | null` (module-level, for getPool())
|
||||
- `initializeDatabase(schema?: Record<string, unknown>): Promise<ReturnType<typeof drizzle>>` — creates pool via `createPoolFromConfig`, wraps with `drizzle()`. Accepts optional schema object (gateway needs it, backend doesn't). Reads config from env/config module internally.
|
||||
- `getDatabase(): ReturnType<typeof drizzle>` — throws if not initialized
|
||||
- `getPool(): Pool` — returns raw pool for raw SQL queries, throws if not initialized
|
||||
- `closeDatabase(): Promise<void>` — closes pool and nullifies references
|
||||
- `executeAll(sql: string, params?: unknown[]): Promise<unknown[]>` — raw SQL query, returns all rows
|
||||
- `executeGet(sql: string, params?: unknown[]): Promise<unknown>` — raw SQL query, returns first row or null
|
||||
- `withDatabaseClient<T>(callback: (client: PoolClient) => Promise<T>): Promise<T>`
|
||||
|
||||
- [ ] **Step 1: Create `packages/shared/src/database/init.ts`**
|
||||
|
||||
This is the canonical database initialization module. It merges what both services currently do:
|
||||
|
||||
```typescript
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import { closePool, createPoolFromConfig } from "@bete/shared/database/pool";
|
||||
import { drizzle } from "drizzle-orm/node-postgres";
|
||||
import type { Pool, PoolClient } from "pg";
|
||||
import { config } from "../config/index.js";
|
||||
|
||||
const logger = createChildLogger("database.init");
|
||||
|
||||
let db: ReturnType<typeof drizzle> | null = null;
|
||||
let rawPool: Pool | null = null;
|
||||
|
||||
export async function initializeDatabase(schema?: Record<string, unknown>) {
|
||||
if (db !== null) return db;
|
||||
|
||||
const pool = config.DATABASE_URL
|
||||
? createPoolFromConfig({
|
||||
url: config.DATABASE_URL,
|
||||
min: config.POSTGRES_POOL_MIN,
|
||||
max: config.POSTGRES_POOL_MAX,
|
||||
})
|
||||
: createPoolFromConfig({
|
||||
host: config.POSTGRES_HOST,
|
||||
port: config.POSTGRES_PORT,
|
||||
user: config.POSTGRES_USER,
|
||||
password: config.POSTGRES_PASSWORD,
|
||||
database: config.POSTGRES_DB,
|
||||
min: config.POSTGRES_POOL_MIN,
|
||||
max: config.POSTGRES_POOL_MAX,
|
||||
});
|
||||
|
||||
rawPool = pool;
|
||||
db = drizzle(pool, schema ? { schema } : undefined);
|
||||
|
||||
// Test connection
|
||||
try {
|
||||
const client = await pool.connect();
|
||||
client.release();
|
||||
logger.info("Database connection successful");
|
||||
} catch (err) {
|
||||
logger.error({ err }, "Failed to connect to database");
|
||||
throw err;
|
||||
}
|
||||
|
||||
return db;
|
||||
}
|
||||
|
||||
export function getDatabase() {
|
||||
if (db === null) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
return db;
|
||||
}
|
||||
|
||||
export function getPool() {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
return rawPool;
|
||||
}
|
||||
|
||||
export async function closeDatabase() {
|
||||
if (rawPool !== null) {
|
||||
await closePool(rawPool);
|
||||
}
|
||||
rawPool = null;
|
||||
db = null;
|
||||
logger.info("Database connection closed");
|
||||
}
|
||||
|
||||
function convertPlaceholdersForPostgres(sql: string) {
|
||||
let i = 0;
|
||||
return sql.replace(/\?/g, () => `$${++i}`);
|
||||
}
|
||||
|
||||
export async function executeAll(sql: string, params?: unknown[]) {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const query = convertPlaceholdersForPostgres(sql);
|
||||
const result = await rawPool.query(query, params || []);
|
||||
return result.rows;
|
||||
}
|
||||
|
||||
export async function executeGet(sql: string, params?: unknown[]) {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const query = convertPlaceholdersForPostgres(sql);
|
||||
const result = await rawPool.query(query, params || []);
|
||||
return result.rows[0] ?? null;
|
||||
}
|
||||
|
||||
export async function withDatabaseClient<T>(
|
||||
callback: (client: PoolClient) => Promise<T>,
|
||||
): Promise<T> {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const client = await rawPool.connect();
|
||||
try {
|
||||
return await callback(client);
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Note:** This uses `config` from `@bete/shared/config`. The backend's config proxies to that already (`services/backend/src/shared/config/index.ts` re-exports from `@bete/shared/config`). The gateway's config at `services/discord-gateway/src/shared/config/config.ts` has the same field names but is its own Zod schema. Since `@bete/shared/config` doesn't have the PostgreSQL pool config fields currently, we need to check what it exports.
|
||||
|
||||
Actually — `@bete/shared/config` may not have `POSTGRES_HOST` etc. Let me adjust: the `initializeDatabase` function should accept config values as parameters instead of reading from a shared config.
|
||||
|
||||
Revised approach for `packages/shared/src/database/init.ts`:
|
||||
|
||||
```typescript
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import { closePool, createPoolFromConfig } from "./pool.js";
|
||||
import { drizzle } from "drizzle-orm/node-postgres";
|
||||
import type { Pool, PoolClient } from "pg";
|
||||
|
||||
const logger = createChildLogger("database.init");
|
||||
|
||||
let db: ReturnType<typeof drizzle> | null = null;
|
||||
let rawPool: Pool | null = null;
|
||||
|
||||
export interface DatabaseConfig {
|
||||
DATABASE_URL?: string;
|
||||
POSTGRES_HOST?: string;
|
||||
POSTGRES_PORT?: number;
|
||||
POSTGRES_USER?: string;
|
||||
POSTGRES_PASSWORD?: string;
|
||||
POSTGRES_DB?: string;
|
||||
POSTGRES_POOL_MIN?: number;
|
||||
POSTGRES_POOL_MAX?: number;
|
||||
}
|
||||
|
||||
export async function initializeDatabase(
|
||||
cfg: DatabaseConfig,
|
||||
schema?: Record<string, unknown>,
|
||||
) {
|
||||
if (db !== null) return db;
|
||||
|
||||
const pool = cfg.DATABASE_URL
|
||||
? createPoolFromConfig({
|
||||
url: cfg.DATABASE_URL,
|
||||
min: cfg.POSTGRES_POOL_MIN,
|
||||
max: cfg.POSTGRES_POOL_MAX,
|
||||
})
|
||||
: createPoolFromConfig({
|
||||
host: cfg.POSTGRES_HOST,
|
||||
port: cfg.POSTGRES_PORT,
|
||||
user: cfg.POSTGRES_USER,
|
||||
password: cfg.POSTGRES_PASSWORD,
|
||||
database: cfg.POSTGRES_DB,
|
||||
min: cfg.POSTGRES_POOL_MIN,
|
||||
max: cfg.POSTGRES_POOL_MAX,
|
||||
});
|
||||
|
||||
rawPool = pool;
|
||||
db = drizzle(pool, schema ? { schema } : undefined);
|
||||
|
||||
try {
|
||||
const client = await pool.connect();
|
||||
client.release();
|
||||
logger.info("Database connection successful");
|
||||
} catch (err) {
|
||||
logger.error({ err }, "Failed to connect to database");
|
||||
throw err;
|
||||
}
|
||||
|
||||
return db;
|
||||
}
|
||||
|
||||
export function getDatabase() {
|
||||
if (db === null) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
return db;
|
||||
}
|
||||
|
||||
export function getPool() {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
return rawPool;
|
||||
}
|
||||
|
||||
export async function closeDatabase() {
|
||||
if (rawPool !== null) {
|
||||
await closePool(rawPool);
|
||||
}
|
||||
rawPool = null;
|
||||
db = null;
|
||||
logger.info("Database connection closed");
|
||||
}
|
||||
|
||||
function convertPlaceholdersForPostgres(sql: string) {
|
||||
let i = 0;
|
||||
return sql.replace(/\?/g, () => `$${++i}`);
|
||||
}
|
||||
|
||||
export async function executeAll(sql: string, params?: unknown[]) {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const query = convertPlaceholdersForPostgres(sql);
|
||||
const result = await rawPool.query(query, params || []);
|
||||
return result.rows;
|
||||
}
|
||||
|
||||
export async function executeGet(sql: string, params?: unknown[]) {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const query = convertPlaceholdersForPostgres(sql);
|
||||
const result = await rawPool.query(query, params || []);
|
||||
return result.rows[0] ?? null;
|
||||
}
|
||||
|
||||
export async function withDatabaseClient<T>(
|
||||
callback: (client: PoolClient) => Promise<T>,
|
||||
): Promise<T> {
|
||||
if (!rawPool) {
|
||||
throw new Error("Database not initialized. Call initializeDatabase() first.");
|
||||
}
|
||||
const client = await rawPool.connect();
|
||||
try {
|
||||
return await callback(client);
|
||||
} finally {
|
||||
client.release();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Add export to `packages/shared/src/index.ts`**
|
||||
|
||||
```typescript
|
||||
export * from "./database/init.js";
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Add export to `packages/shared/package.json`**
|
||||
|
||||
```json
|
||||
"./database/init": "./dist/database/init.js",
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Build the shared package to verify it compiles**
|
||||
|
||||
```bash
|
||||
cd /home/code/GMW/packages/shared
|
||||
pnpm run build
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Rewrite `services/backend/src/shared/database/index.ts`**
|
||||
|
||||
Change to a thin wrapper that imports from `@bete/shared/database/init` and passes the backend's config:
|
||||
|
||||
```typescript
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import { initializeDatabase as sharedInit, getDatabase as sharedGetDb, getPool as sharedGetPool, closeDatabase as sharedCloseDb } from "@bete/shared/database/init";
|
||||
import { config } from "../config/index.js";
|
||||
|
||||
const logger = createChildLogger("database");
|
||||
|
||||
const dbConfig = {
|
||||
DATABASE_URL: config.DATABASE_URL,
|
||||
POSTGRES_HOST: config.POSTGRES_HOST,
|
||||
POSTGRES_PORT: config.POSTGRES_PORT,
|
||||
POSTGRES_USER: config.POSTGRES_USER,
|
||||
POSTGRES_PASSWORD: config.POSTGRES_PASSWORD,
|
||||
POSTGRES_DB: config.POSTGRES_DB,
|
||||
POSTGRES_POOL_MIN: config.POSTGRES_POOL_MIN,
|
||||
POSTGRES_POOL_MAX: config.POSTGRES_POOL_MAX,
|
||||
};
|
||||
|
||||
export async function initializeDatabase() {
|
||||
logger.info("Initializing database");
|
||||
return sharedInit(dbConfig);
|
||||
}
|
||||
|
||||
export function getDatabase() {
|
||||
return sharedGetDb();
|
||||
}
|
||||
|
||||
export function getPool() {
|
||||
return sharedGetPool();
|
||||
}
|
||||
|
||||
export async function closeDatabase() {
|
||||
logger.info("Closing database");
|
||||
return sharedCloseDb();
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Rewrite `services/discord-gateway/src/shared/database/drizzle.ts`**
|
||||
|
||||
Change to a thin wrapper:
|
||||
|
||||
```typescript
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import { initializeDatabase as sharedInit, getDatabase as sharedGetDb, closeDatabase as sharedCloseDb, executeAll as sharedExecAll, executeGet as sharedExecGet, withDatabaseClient as sharedWithClient } from "@bete/shared/database/init";
|
||||
import { config } from "../../shared/config/config.js";
|
||||
import * as schema from "./schema.js";
|
||||
|
||||
const logger = createChildLogger("drizzle");
|
||||
|
||||
const dbConfig = {
|
||||
DATABASE_URL: config.DATABASE_URL,
|
||||
POSTGRES_HOST: config.POSTGRES_HOST,
|
||||
POSTGRES_PORT: config.POSTGRES_PORT,
|
||||
POSTGRES_USER: config.POSTGRES_USER,
|
||||
POSTGRES_PASSWORD: config.POSTGRES_PASSWORD,
|
||||
POSTGRES_DB: config.POSTGRES_DB,
|
||||
POSTGRES_POOL_MIN: config.POSTGRES_POOL_MIN,
|
||||
POSTGRES_POOL_MAX: config.POSTGRES_POOL_MAX,
|
||||
};
|
||||
|
||||
export async function initializeDatabase() {
|
||||
return sharedInit(dbConfig, schema);
|
||||
}
|
||||
|
||||
export function getDatabase() {
|
||||
return sharedGetDb();
|
||||
}
|
||||
|
||||
export { sharedCloseDb as closeDatabase };
|
||||
export { sharedExecAll as executeAll, sharedExecGet as executeGet, sharedWithClient as withDatabaseClient };
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Run typecheck on all packages to verify**
|
||||
|
||||
```bash
|
||||
cd /home/code/GMW
|
||||
pnpm run typecheck
|
||||
```
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add packages/shared/src/database/init.ts packages/shared/src/index.ts packages/shared/package.json
|
||||
git add services/backend/src/shared/database/index.ts services/discord-gateway/src/shared/database/drizzle.ts
|
||||
git commit -m "refactor: consolidate database initialization into @bete/shared/database/init"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Remove Backward-Compat Function Wrappers from MessageStore
|
||||
|
||||
**Files:**
|
||||
- Modify: `services/discord-gateway/src/modules/message-capture/messageStore.ts` — remove lines 310-398 (backward-compat wrappers), export singleton directly
|
||||
- Modify: `services/discord-gateway/src/modules/message-capture/index.ts` — update re-exports to use `messageStore` singleton
|
||||
- Modify: `services/discord-gateway/src/modules/message-capture/messageCapture.ts` — update imports to use `messageStore.methodName()`
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/batchProcessor.ts` — update imports
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/batchScheduler.ts` — update imports
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/individualFallbackProcessor.ts` — update imports
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationBuilders.ts` — update imports
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/aiAnalysisWorker.ts` — update imports (uses `getConversationContextBefore` and `updateMessagesAIAnalysisBulk`)
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/aiAnalyzer.ts` — update imports (uses many functions)
|
||||
- Possibly modify: other files that import the wrapper functions
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: Existing `MessageStore` class methods (unchanged signatures)
|
||||
- Produces: Singleton `messageStore` instance as the single export point
|
||||
|
||||
The key insight: the backward-compat wrappers at lines 310-398 of `messageStore.ts` are function-level exports that delegate to `getInstance()`. Every importer can instead import the singleton `messageStore` instance and call methods on it directly.
|
||||
|
||||
Current importers of wrapper functions:
|
||||
|
||||
| File | Functions Used |
|
||||
|------|---------------|
|
||||
| `messageCapture.ts` | `getMessageById`, `insertMessageEdit`, `updateMessageAsEdited`, `updateMessageAsDeleted`, `upsertMessageForCapture` |
|
||||
| `batchProcessor.ts` | `updateMessagesAIAnalysisBulk` |
|
||||
| `batchScheduler.ts` | `getPendingMessagesByConversation` |
|
||||
| `individualFallbackProcessor.ts` | `updateMessagesAIAnalysisBulk` |
|
||||
| `moderationBuilders.ts` | `getMessageById` |
|
||||
| `aiAnalysisWorker.ts` | `getConversationContextBefore`, `updateMessagesAIAnalysisBulk` |
|
||||
| `aiAnalyzer.ts` | `getConversationKeysWithIncompleteAnalysis`, `getIncompleteMessagesByConversation`, `getMessageById`, `getPendingConversationKeys`, `updateMessageAIAnalysis` |
|
||||
|
||||
- [ ] **Step 1: Modify `messageStore.ts`** — replace backward-compat wrappers with a singleton export
|
||||
|
||||
Replace lines 22-33 (lazy singleton pattern) and lines 310-398 (wrapper functions) with:
|
||||
|
||||
```typescript
|
||||
// ─── Singleton instance ─────────────────────────────────────────────────────
|
||||
|
||||
const logger = createChildLogger("message-store");
|
||||
const database = getDatabase() as unknown as NodePgDatabase<typeof schema>;
|
||||
export const messageStore = new MessageStore(database, logger);
|
||||
```
|
||||
|
||||
Then remove everything from line 310 onward (the backward-compat function wrappers section).
|
||||
|
||||
- [ ] **Step 2: Update `message-capture/index.ts`**
|
||||
|
||||
Change the re-exports from individual functions to the `messageStore` singleton:
|
||||
|
||||
```typescript
|
||||
export { messageStore } from "../message-capture/messageStore.js";
|
||||
export {
|
||||
getDisplayContent,
|
||||
getMessageLocation,
|
||||
getMessageMetadata,
|
||||
} from "../message-capture/messageMetadata.js";
|
||||
// ... rest unchanged
|
||||
```
|
||||
|
||||
Also remove the individual function re-exports since they no longer exist.
|
||||
|
||||
- [ ] **Step 3: Update `messageCapture.ts`**
|
||||
|
||||
Change imports from:
|
||||
```typescript
|
||||
import {
|
||||
getMessageById,
|
||||
insertMessageEdit,
|
||||
upsertMessageForCapture,
|
||||
updateMessageAsDeleted,
|
||||
updateMessageAsEdited,
|
||||
} from "./messageStore.js";
|
||||
```
|
||||
To:
|
||||
```typescript
|
||||
import { messageStore } from "./messageStore.js";
|
||||
```
|
||||
|
||||
Then update every call site:
|
||||
- `upsertMessageForCapture(messageRecord)` → `messageStore.upsertMessageForCapture(messageRecord)`
|
||||
- `insertMessageEdit(...)` → `messageStore.insertMessageEdit(...)`
|
||||
- `updateMessageAsEdited(...)` → `messageStore.updateMessageAsEdited(...)`
|
||||
- `updateMessageAsDeleted(...)` → `messageStore.updateMessageAsDeleted(...)`
|
||||
- `getMessageById(...)` → `messageStore.getMessageById(...)`
|
||||
|
||||
- [ ] **Step 4: Update `batchProcessor.ts`**
|
||||
|
||||
Change from:
|
||||
```typescript
|
||||
import { updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
||||
```
|
||||
To:
|
||||
```typescript
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
|
||||
Then update call sites:
|
||||
- `updateMessagesAIAnalysisBulk(updates)` → `messageStore.messages.updateMessagesAIAnalysisBulk(updates)`
|
||||
|
||||
Wait — `updateMessagesAIAnalysisBulk` is actually defined in `MessagesAnalysis` class, which is called via `MessageStore` → `MessagesDb` → `MessagesAnalysis`. Let me check the actual delegation chain.
|
||||
|
||||
Looking at the wrapper functions:
|
||||
```typescript
|
||||
export const updateMessagesAIAnalysisBulk = (
|
||||
updates: Array<{ messageId: string; result: AIAnalysisUpdate }>,
|
||||
): Promise<MessageRecord[]> =>
|
||||
getInstance().updateMessagesAIAnalysisBulk(updates);
|
||||
```
|
||||
|
||||
And in the class:
|
||||
```typescript
|
||||
class MessageStore {
|
||||
readonly messages: MessagesDb;
|
||||
// ...
|
||||
}
|
||||
|
||||
class MessagesDb {
|
||||
readonly analysis: MessagesAnalysis;
|
||||
// ...
|
||||
updateMessagesAIAnalysisBulk(...) {
|
||||
return this.analysis.updateMessagesAIAnalysisBulk(...)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
So the call chain is: `messageStore.messages.updateMessagesAIAnalysisBulk()`. But actually, looking at `MessagesDb`, it might have its own `updateMessagesAIAnalysisBulk` that delegates to `this.analysis.updateMessagesAIAnalysisBulk()`. Let me verify...
|
||||
|
||||
Actually, for simplicity and to minimize changes, let me look at whether `MessagesDb` has `updateMessagesAIAnalysisBulk` or if only the wrapper has it.
|
||||
|
||||
Let me check:
|
||||
|
||||
Actually I already read that `MessagesDb` has methods. Let me look at what methods `MessagesDb` exposes vs the wrapper functions.
|
||||
|
||||
Instead of guessing, the safe approach is to keep the thin function wrappers but simplify them. Actually, a better approach for this task:
|
||||
|
||||
**Revised approach:** Instead of making all importers use `messageStore.messages.analysis.methodName()`, add all the forwarded methods directly to the `MessageStore` class (which it already does for most), and just have external files import the singleton and call `messageStore.methodName()`.
|
||||
|
||||
Let me check what methods `MessageStore` already has vs what's only available as backward-compat wrappers:
|
||||
|
||||
Looking at the code:
|
||||
- `insertMessageEdit` — EXISTS in MessageStore class (line 54)
|
||||
- `upsertMessageForCapture` — EXISTS in MessageStore class
|
||||
- `updateMessageAsEdited` — EXISTS in MessageStore class
|
||||
- `updateMessageAsDeleted` — EXISTS in MessageStore class
|
||||
- `getMessagesByChannel` — EXISTS in MessageStore class
|
||||
- `updateMessageAIAnalysis` — EXISTS in MessageStore class
|
||||
- `updateMessagesAIAnalysisBulk` — EXISTS in MessageStore class
|
||||
- `getPendingAIAnalysisMessages` — EXISTS in MessageStore class
|
||||
- `getMessageById` — EXISTS in MessageStore class
|
||||
- `listMessages` — EXISTS in MessageStore class (delegates to MessagesPagination)
|
||||
- `listReviewMessages` — EXISTS in MessageStore class (delegates to MessagesPagination)
|
||||
- `getConversationContextBefore` — EXISTS in MessageStore class
|
||||
- `getPendingMessagesByConversation` — EXISTS in MessageStore class
|
||||
- `getPendingConversationKeys` — EXISTS in MessageStore class
|
||||
- `getConversationKeysWithIncompleteAnalysis` — EXISTS in MessageStore class
|
||||
- `getIncompleteMessagesByConversation` — EXISTS in MessageStore class
|
||||
|
||||
So every function wrapper has a corresponding method on `MessageStore` class. The change is straightforward.
|
||||
|
||||
Now, after creating the singleton `messageStore`, all importers just do `messageStore.updateMessagesAIAnalysisBulk(...)` instead of calling the bare function.
|
||||
|
||||
But there's one complication: `MessagesDb.updateMessagesAIAnalysisBulk` is actually calling `this.analysis.updateMessagesAIAnalysisBulk()`. Does the `MessageStore` class have its own direct `updateMessagesAIAnalysisBulk`? Let me check the class definition...
|
||||
|
||||
Actually, I already saw from the grep output that `MessageStore` class has `updateMessagesAIAnalysisBulk` — the wrapper says `getInstance().updateMessagesAIAnalysisBulk(updates)`, and the class has that method.
|
||||
|
||||
OK so the mapping is 1:1 between wrapper functions and MessageStore class methods. This is safe.
|
||||
|
||||
- [ ] **Step 5: Update `batchScheduler.ts`**
|
||||
|
||||
```typescript
|
||||
// Before:
|
||||
import { getPendingMessagesByConversation } from "../message-capture/messageStore.js";
|
||||
// After:
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
And call: `messageStore.getPendingMessagesByConversation(...)`
|
||||
|
||||
- [ ] **Step 6: Update `individualFallbackProcessor.ts`**
|
||||
|
||||
```typescript
|
||||
// Before:
|
||||
import { updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
||||
// After:
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
And call: `messageStore.updateMessagesAIAnalysisBulk(...)`
|
||||
|
||||
- [ ] **Step 7: Update `moderationBuilders.ts`**
|
||||
|
||||
```typescript
|
||||
// Before:
|
||||
import { getMessageById } from "../message-capture/messageStore.js";
|
||||
// After:
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
And call: `messageStore.getMessageById(...)`
|
||||
|
||||
- [ ] **Step 8: Update `aiAnalysisWorker.ts`**
|
||||
|
||||
```typescript
|
||||
// Before:
|
||||
import { getConversationContextBefore, updateMessagesAIAnalysisBulk } from "../message-capture/messageStore.js";
|
||||
// After:
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
And update all call sites.
|
||||
|
||||
- [ ] **Step 9: Update `aiAnalyzer.ts`**
|
||||
|
||||
```typescript
|
||||
// Before:
|
||||
import {
|
||||
getConversationKeysWithIncompleteAnalysis,
|
||||
getIncompleteMessagesByConversation,
|
||||
getMessageById,
|
||||
getPendingConversationKeys,
|
||||
updateMessageAIAnalysis,
|
||||
} from "../message-capture/messageStore.js";
|
||||
// After:
|
||||
import { messageStore } from "../message-capture/messageStore.js";
|
||||
```
|
||||
And update all call sites.
|
||||
|
||||
- [ ] **Step 10: Update `message-capture/index.ts`**
|
||||
|
||||
Remove individual function re-exports, replace with `messageStore`:
|
||||
|
||||
```typescript
|
||||
export { messageStore } from "./messageStore.js";
|
||||
export {
|
||||
getDisplayContent,
|
||||
getMessageLocation,
|
||||
getMessageMetadata,
|
||||
} from "./messageMetadata.js";
|
||||
export type {
|
||||
AIRecommendedAction,
|
||||
AISeverity,
|
||||
AIStatus,
|
||||
AttachmentRecord,
|
||||
MessageRecord,
|
||||
VoiceSegmentRecord,
|
||||
} from "./types.js";
|
||||
export type { TextCaptureTarget } from "./messageCapture.js";
|
||||
export {
|
||||
captureMessage,
|
||||
registerMessageCapture,
|
||||
setEventBroadcaster,
|
||||
} from "./messageCapture.js";
|
||||
```
|
||||
|
||||
- [ ] **Step 11: Run typecheck**
|
||||
|
||||
```bash
|
||||
cd /home/code/GMW
|
||||
pnpm run typecheck
|
||||
```
|
||||
|
||||
- [ ] **Step 12: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/message-capture/
|
||||
git add services/discord-gateway/src/modules/ai-moderation/
|
||||
git commit -m "refactor: remove backward-compat function wrappers from messageStore"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Remove Dead Code
|
||||
|
||||
**Files:**
|
||||
- Delete: `services/backend/src/modules/response.ts` — empty deprecated file
|
||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/webhookNotifier.ts`
|
||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/index.ts`
|
||||
- Delete: `services/discord-gateway/src/modules/webhook-notifications/` (directory)
|
||||
- Modify: `services/backend/src/ws/server.ts` — remove duplicate `broadcastBinaryToFrontend()` function, keep only `broadcastBinary()`
|
||||
|
||||
**Interfaces:**
|
||||
- None — these are deletions only, no consumer impact
|
||||
|
||||
- [ ] **Step 1: Delete `modules/response.ts`**
|
||||
|
||||
```bash
|
||||
rm /home/code/GMW/services/backend/src/modules/response.ts
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Fix `ws/server.ts`** — remove duplicate `broadcastBinaryToFrontend`
|
||||
|
||||
In `ws/server.ts`, `broadcastBinaryToFrontend` (line 238) and `broadcastBinary` (line 267) do exactly the same thing. Replace the `broadcastBinaryToFrontend(data)` call on line 147 with a call to `broadcastBinary(data)`, then delete the `broadcastBinaryToFrontend` function.
|
||||
|
||||
Edit line 147:
|
||||
```typescript
|
||||
// Before:
|
||||
broadcastBinaryToFrontend(data);
|
||||
// After:
|
||||
broadcastBinary(data);
|
||||
```
|
||||
|
||||
Remove the `broadcastBinaryToFrontend` function (lines 238-248):
|
||||
```typescript
|
||||
// Remove this entire function:
|
||||
function broadcastBinaryToFrontend(data: Buffer) {
|
||||
for (const client of frontendClients) {
|
||||
if (client.readyState === WebSocket.OPEN) {
|
||||
try {
|
||||
client.send(data);
|
||||
} catch (err) {
|
||||
logger.error({ err }, "Failed to send binary to frontend client");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Check if anything imports `webhook-notifications`**
|
||||
|
||||
```bash
|
||||
grep -rn "webhook-notifications\|webhookNotifier\|triggerWebhook" /home/code/GMW/services/ --include='*.ts' | grep -v "node_modules" | grep -v "services/discord-gateway/src/modules/webhook-notifications/"
|
||||
```
|
||||
|
||||
Expected: empty (confirmed earlier)
|
||||
|
||||
- [ ] **Step 4: Delete webhook-notifications module**
|
||||
|
||||
```bash
|
||||
rm -rf /home/code/GMW/services/discord-gateway/src/modules/webhook-notifications/
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run typecheck to verify no broken imports**
|
||||
|
||||
```bash
|
||||
cd /home/code/GMW
|
||||
pnpm run typecheck
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add services/backend/src/modules/response.ts services/backend/src/ws/server.ts
|
||||
git add services/discord-gateway/src/modules/webhook-notifications/
|
||||
git commit -m "chore: remove dead code (response.ts, broadcastBinaryToFrontend, webhook-notifications)"
|
||||
```
|
||||
@@ -0,0 +1,579 @@
|
||||
# Services Refactoring Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Refactor backend (4.2k lines) and discord-gateway (17.9k lines) for consistency, reduced file sizes, deduplication, and pattern uniformity across 11 phases.
|
||||
|
||||
**Architecture:** Phase1-3 target backend unchanged; Phase4-8 split large gateway files; Phase9 deduplicates shared database init; Phase10-11 are minor consolidation. Each phase is independently testable by verifying the service still compiles and runs.
|
||||
|
||||
**Tech Stack:** TypeScript (ESM), Express 5, ws, Discord.js selfbot, Drizzle ORM, Redis (ioredis), pino logger, Biome (formatter)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- All files use ESM (`.js` extensions in imports)
|
||||
- Biome formatter handles formatting — run `pnpm run format` after each phase
|
||||
- TypeScript strict mode — run `pnpm run typecheck` after each phase (for node services)
|
||||
- Logging uses `createChildLogger(context)` from `@bete/shared/logger`
|
||||
- Import via barrel files where available
|
||||
- No logic changes — pure refactoring
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Fix messages.controller.ts pattern (Phase 1)
|
||||
|
||||
**Files:**
|
||||
- Modify: `services/backend/src/modules/messages/messages.controller.ts`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `asyncHandler` from `../../shared/middlewares/index.js`
|
||||
- Produces: Same exported handler functions, but using decorator pattern
|
||||
|
||||
- [ ] **Step 1: Read current messages.controller.ts**
|
||||
|
||||
The file currently uses the convoluted pattern:
|
||||
```ts
|
||||
export function handleListMessages(req, res, next) {
|
||||
return asyncHandler(async (req, res) => {
|
||||
// ...
|
||||
})(req, res, next);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Rewrite all handlers to decorator pattern**
|
||||
|
||||
Replace every handler to use the clean decorator pattern:
|
||||
|
||||
```ts
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import type { Request, Response } from "express";
|
||||
import { asyncHandler } from "../../shared/middlewares/index.js";
|
||||
import { messageQuerySchema } from "./messages.schema.js";
|
||||
import { messagesService } from "./messages.service.js";
|
||||
|
||||
const logger = createChildLogger("messages.controller");
|
||||
|
||||
export const handleListMessages = asyncHandler(async (req: Request, res: Response) => {
|
||||
const query = messageQuerySchema.parse(req.query);
|
||||
logger.debug({ query }, "Handling list messages request");
|
||||
const result = await messagesService.listMessages(query);
|
||||
res.json(result);
|
||||
});
|
||||
|
||||
export const handleGetMessagesByChannel = asyncHandler(async (req: Request, res: Response) => {
|
||||
const channelId = String(req.params.channelId ?? "");
|
||||
if (!channelId) {
|
||||
res.status(400).json({ error: "MISSING_CHANNEL_ID" });
|
||||
return;
|
||||
}
|
||||
const query = messageQuerySchema.parse(req.query);
|
||||
logger.debug({ channelId, query }, "Handling get messages by channel");
|
||||
const result = await messagesService.getMessagesByChannel(channelId, query);
|
||||
res.json(result);
|
||||
});
|
||||
|
||||
export const handleGetMessageById = asyncHandler(async (req: Request, res: Response) => {
|
||||
const id = String(req.params.id ?? "");
|
||||
if (!id) {
|
||||
res.status(400).json({ error: "MISSING_ID" });
|
||||
return;
|
||||
}
|
||||
logger.debug({ id }, "Handling get message by ID");
|
||||
const result = await messagesService.getMessageById(id);
|
||||
res.json(result);
|
||||
});
|
||||
|
||||
export const handleGetImageMessages = asyncHandler(async (req: Request, res: Response) => {
|
||||
const guildId = String(req.query.guildId ?? "");
|
||||
if (!guildId) {
|
||||
res.status(400).json({ error: "MISSING_GUILD_ID" });
|
||||
return;
|
||||
}
|
||||
const limit = Number(req.query.limit) || 50;
|
||||
logger.debug({ guildId, limit }, "Handling get image messages");
|
||||
const result = await messagesService.getImageMessages(guildId, limit);
|
||||
res.json(result);
|
||||
});
|
||||
|
||||
export const handleGetAttachmentsByChannel = asyncHandler(async (req: Request, res: Response) => {
|
||||
const channelId = String(req.params.channelId ?? "");
|
||||
if (!channelId) {
|
||||
res.status(400).json({ error: "MISSING_CHANNEL_ID" });
|
||||
return;
|
||||
}
|
||||
const query = messageQuerySchema.parse(req.query);
|
||||
logger.debug({ channelId, query }, "Handling get attachments by channel");
|
||||
const result = await messagesService.getAttachmentsByChannel(channelId, query);
|
||||
res.json(result);
|
||||
});
|
||||
```
|
||||
|
||||
NOTE: The old pattern used `requireParam` from middlewares to validate params. The new pattern uses simple string checks with early returns. This is equivalent since `requireParam` threw `ValidationError` which the errorHandler middleware catches — but for these handlers the decorator pattern can't throw synchronously in the handler wrapper; the `asyncHandler` catches async rejects. Early return with explicit error response is cleaner.
|
||||
|
||||
- [ ] **Step 3: Verify the module still compiles**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
Expected: No TypeScript errors
|
||||
|
||||
- [ ] **Step 4: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add services/backend/src/modules/messages/messages.controller.ts
|
||||
git commit -m "refactor(backend): fix messages.controller.ts to use decorator pattern
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Clean up response.ts usage (Phase 2)
|
||||
|
||||
**Files:**
|
||||
- Modify: `services/backend/src/modules/health/health.controller.ts` (remove `success()` usage, use plain `res.json()`)
|
||||
- Modify: `services/backend/src/modules/response.ts` (deprecate/remove)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: all response-producing route files
|
||||
- Produces: consistent plain `res.json()` pattern everywhere
|
||||
|
||||
- [ ] **Step 1: Check all places that import from response.ts**
|
||||
|
||||
Run: `grep -r 'from.*response\.js' services/backend/src/`
|
||||
|
||||
- [ ] **Step 2: Remove `success()` usage from health.controller.ts**
|
||||
|
||||
Replace:
|
||||
```ts
|
||||
import { success } from "../response.js";
|
||||
// ...
|
||||
res.status(status).json(success(result));
|
||||
```
|
||||
With:
|
||||
```ts
|
||||
res.status(status).json({ success: true, data: result });
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run biome format + typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add services/backend/src/modules/health/health.controller.ts services/backend/src/modules/response.ts
|
||||
git commit -m "refactor(backend): remove response.ts helpers, inline health response
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Add ws/ barrel (Phase 3)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/backend/src/ws/index.ts`
|
||||
|
||||
- [ ] **Step 1: Create barrel file**
|
||||
|
||||
```ts
|
||||
export { setBroadcastFunctions, clearBroadcastFunctions, broadcastEvent, broadcastBinary } from "./broadcast.js";
|
||||
export { startRedisBridge, stopRedisBridge } from "./redis-bridge.js";
|
||||
export { createWebSocketServer, closeWebSocketServer } from "./server.js";
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add services/backend/src/ws/index.ts
|
||||
git commit -m "refactor(backend): add ws barrel index
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Split moderationPrompt.ts (Phase 4)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/text-analysis.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/media-analysis.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/stickers.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/emojis.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/prompts/system.ts`
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationPrompt.ts` (become barrel re-export)
|
||||
|
||||
- [ ] **Step 1: Read the full moderationPrompt.ts**
|
||||
|
||||
Read the file to identify all exports and their dependencies.
|
||||
|
||||
- [ ] **Step 2: Create `prompts/system.ts` — system prompt builder + shared helpers**
|
||||
|
||||
Move: `buildSystemPrompt` function, `sanitizeAiContent`, `escapeXml`, `buildCustomEmojiVisionPrompt`, any shared helper functions.
|
||||
|
||||
- [ ] **Step 3: Create `prompts/text-analysis.ts` — text moderation prompts**
|
||||
|
||||
Move: All text-specific prompt strings and builders.
|
||||
|
||||
- [ ] **Step 4: Create `prompts/media-analysis.ts` — image/video prompts**
|
||||
|
||||
Move: `buildGeneralImageVisionPrompt` and related media prompt builders.
|
||||
|
||||
- [ ] **Step 5: Create `prompts/stickers.ts` — sticker prompts**
|
||||
|
||||
Move: `buildStickerVisionPrompt`, `buildStickerTextOnlyWarning`.
|
||||
|
||||
- [ ] **Step 6: Create `prompts/emojis.ts` — emoji prompts**
|
||||
|
||||
Move: `buildCustomEmojiVisionPrompt` if it exists separately.
|
||||
|
||||
- [ ] **Step 7: Replace moderationPrompt.ts with barrel re-exports**
|
||||
|
||||
```ts
|
||||
export { buildSystemPrompt, sanitizeAiContent } from "./prompts/system.js";
|
||||
export { buildGeneralImageVisionPrompt } from "./prompts/media-analysis.js";
|
||||
export { buildStickerVisionPrompt, buildStickerTextOnlyWarning } from "./prompts/stickers.js";
|
||||
export { buildCustomEmojiVisionPrompt } from "./prompts/emojis.js";
|
||||
```
|
||||
|
||||
- [ ] **Step 8: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
Expected: No errors. Existing importers continue to work via the barrel.
|
||||
|
||||
- [ ] **Step 9: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 10: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/ai-moderation/prompts/ services/discord-gateway/src/modules/ai-moderation/moderationPrompt.ts
|
||||
git commit -m "refactor(gateway): split moderationPrompt.ts into domain-specific prompt files
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Split moderationOrchestrator.ts (Phase 5)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/textBatchProcessor.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaBatchProcessor.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/simpleFallback.ts`
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/moderationOrchestrator.ts` (extract & re-export)
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/index.ts` (update exports if needed)
|
||||
|
||||
- [ ] **Step 1: Read full moderationOrchestrator.ts**
|
||||
|
||||
Map all exports and dependencies.
|
||||
|
||||
- [ ] **Step 2: Extract `runTextOnlyBatch` into `textBatchProcessor.ts`**
|
||||
|
||||
Move the function and its helper `buildCorrectedFewShotExamples`. Export it.
|
||||
|
||||
- [ ] **Step 3: Extract `runMediaBatch` into `mediaBatchProcessor.ts`**
|
||||
|
||||
Move the function and all its dependencies. Export it.
|
||||
|
||||
- [ ] **Step 4: Extract `runSimpleTextFallback` into `simpleFallback.ts`**
|
||||
|
||||
Move the function. Export it.
|
||||
|
||||
- [ ] **Step 5: Update moderationOrchestrator.ts**
|
||||
|
||||
Replace extracted functions with imports:
|
||||
```ts
|
||||
export { runTextOnlyBatch } from "./textBatchProcessor.js";
|
||||
export { runMediaBatch } from "./mediaBatchProcessor.js";
|
||||
export { runSimpleTextFallback } from "./simpleFallback.js";
|
||||
```
|
||||
Keep the `runModerationAnalysis` entry point function which orchestrates text + media + caching.
|
||||
|
||||
- [ ] **Step 6: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 7: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/ai-moderation/textBatchProcessor.ts services/discord-gateway/src/modules/ai-moderation/mediaBatchProcessor.ts services/discord-gateway/src/modules/ai-moderation/simpleFallback.ts services/discord-gateway/src/modules/ai-moderation/moderationOrchestrator.ts
|
||||
git commit -m "refactor(gateway): split moderationOrchestrator into dedicated processors
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Split mediaAnalysisClient.ts (Phase 6)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaCache.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/mediaDownloader.ts`
|
||||
- Create: `services/discord-gateway/src/modules/ai-moderation/visionAnalyzer.ts`
|
||||
- Modify: `services/discord-gateway/src/modules/ai-moderation/mediaAnalysisClient.ts` (become barrel)
|
||||
|
||||
- [ ] **Step 1: Read full mediaAnalysisClient.ts**
|
||||
|
||||
Map all exports and dependencies across the 826 lines.
|
||||
|
||||
- [ ] **Step 2: Extract all cache logic into `mediaCache.ts`**
|
||||
|
||||
Move: LRU cache, phash dedup, `getCachedMediaAnalysis`, `setCachedMediaAnalysis`, `computeImagePhash`, `deleteCachedMediaAnalysis`, `acquireMediaAnalysisLock`.
|
||||
|
||||
- [ ] **Step 3: Extract all download logic into `mediaDownloader.ts`**
|
||||
|
||||
Move: Image download, video download, ffmpeg frame extraction, temporary file handling.
|
||||
|
||||
- [ ] **Step 4: Extract vision LLM logic into `visionAnalyzer.ts`**
|
||||
|
||||
Move: Vision LLM calls, message preparation for vision, `prepareMediaMessage`.
|
||||
|
||||
- [ ] **Step 5: Update mediaAnalysisClient.ts to re-export**
|
||||
|
||||
```ts
|
||||
export { getCachedMediaAnalysis, setCachedMediaAnalysis, computeImagePhash } from "./mediaCache.js";
|
||||
export { downloadAndExtractFrame } from "./mediaDownloader.js";
|
||||
export { prepareMediaMessage, hasMediaContent } from "./visionAnalyzer.js";
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 7: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/ai-moderation/mediaCache.ts services/discord-gateway/src/modules/ai-moderation/mediaDownloader.ts services/discord-gateway/src/modules/ai-moderation/visionAnalyzer.ts services/discord-gateway/src/modules/ai-moderation/mediaAnalysisClient.ts
|
||||
git commit -m "refactor(gateway): split mediaAnalysisClient into cache, downloader, and vision analyzer
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Extract retention cleanup from bootstrap.ts (Phase 7)
|
||||
|
||||
**Files:**
|
||||
- Create: `services/discord-gateway/src/app/retention.ts`
|
||||
- Modify: `services/discord-gateway/src/app/bootstrap.ts`
|
||||
|
||||
- [ ] **Step 1: Create `app/retention.ts`**
|
||||
|
||||
Move `deleteExpiredRecords` and `startRetentionCleanup` from `bootstrap.ts`:
|
||||
```ts
|
||||
import { createChildLogger } from "@bete/shared/logger";
|
||||
import { lt, inArray } from "drizzle-orm";
|
||||
import type { NodePgDatabase } from "drizzle-orm/node-postgres";
|
||||
import { config } from "../shared/config/config.js";
|
||||
import { getDatabase } from "../shared/database/drizzle.js";
|
||||
import * as schema from "../shared/database/schema.js";
|
||||
import { messagesTable, attachmentsTable, voiceRecordingsTable } from "../shared/database/schema.js";
|
||||
|
||||
const log = createChildLogger("retention");
|
||||
|
||||
// ... move deleteExpiredRecords here ...
|
||||
|
||||
// ... move startRetentionCleanup here ...
|
||||
|
||||
export { startRetentionCleanup };
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Remove inline retention code from bootstrap.ts**
|
||||
|
||||
- Remove the `deleteExpiredRecords` function
|
||||
- Remove the `startRetentionCleanup` function
|
||||
- Add: `import { startRetentionCleanup } from "./retention.js";`
|
||||
- Replace the call: call `startRetentionCleanup()` directly
|
||||
|
||||
- [ ] **Step 3: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 4: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/app/retention.ts services/discord-gateway/src/app/bootstrap.ts
|
||||
git commit -m "refactor(gateway): extract retention cleanup from bootstrap into dedicated module
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Consolidate EventBroadcaster (Phase 8)
|
||||
|
||||
**Files:**
|
||||
- Modify: `services/discord-gateway/src/modules/event-broadcaster/eventBroadcaster.ts`
|
||||
- Modify: `services/discord-gateway/src/modules/event-broadcaster/index.ts`
|
||||
|
||||
- [ ] **Step 1: Read current eventBroadcaster.ts**
|
||||
|
||||
Identify `RedisEventPublisher` and `EventBroadcaster` classes.
|
||||
|
||||
- [ ] **Step 2: Merge RedisEventPublisher into EventBroadcaster**
|
||||
|
||||
Inline `RedisEventPublisher` as a private detail inside `EventBroadcaster`. Keep the public API unchanged.
|
||||
|
||||
- [ ] **Step 3: Update index.ts if needed**
|
||||
|
||||
Ensure the barrel still exports `EventBroadcaster`.
|
||||
|
||||
- [ ] **Step 4: Run typecheck**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 5: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/event-broadcaster/eventBroadcaster.ts services/discord-gateway/src/modules/event-broadcaster/index.ts
|
||||
git commit -m "refactor(gateway): merge RedisEventPublisher into EventBroadcaster
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Cross-cutting database initialization dedup (Phase 9)
|
||||
|
||||
**Files:**
|
||||
- Modify: `packages/shared/src/database/schema.ts` — add database lifecycle helpers
|
||||
- Modify: `services/backend/src/shared/database/index.ts` — use shared helpers
|
||||
- Modify: `services/discord-gateway/src/shared/database/drizzle.ts` — use shared helpers
|
||||
|
||||
- [ ] **Step 1: Check current shared database setup**
|
||||
|
||||
Read `packages/shared/` structure to see if there's already a database module.
|
||||
|
||||
- [ ] **Step 2: Add pool creation helper in @bete/shared**
|
||||
|
||||
In `packages/shared/src/database/schema.ts` or create `packages/shared/src/database/pool.ts`:
|
||||
|
||||
```ts
|
||||
import { Pool } from "pg";
|
||||
|
||||
export function createPostgresPool(url: string, opts?: { min?: number; max?: number }): Pool {
|
||||
return new Pool({
|
||||
connectionString: url,
|
||||
min: opts?.min ?? 2,
|
||||
max: opts?.max ?? 10,
|
||||
});
|
||||
}
|
||||
|
||||
export interface PoolConfig {
|
||||
host?: string;
|
||||
port?: number;
|
||||
user?: string;
|
||||
password?: string;
|
||||
database?: string;
|
||||
url?: string;
|
||||
min?: number;
|
||||
max?: number;
|
||||
}
|
||||
|
||||
export function createPoolFromConfig(cfg: PoolConfig): Pool {
|
||||
if (cfg.url) return createPostgresPool(cfg.url, { min: cfg.min, max: cfg.max });
|
||||
return new Pool({
|
||||
host: cfg.host,
|
||||
port: cfg.port,
|
||||
user: cfg.user,
|
||||
password: cfg.password,
|
||||
database: cfg.database,
|
||||
min: cfg.min ?? 2,
|
||||
max: cfg.max ?? 10,
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Export from `packages/shared/src/database/schema.ts` or create a barrel.
|
||||
|
||||
- [ ] **Step 3: Update backend's shared/database/index.ts**
|
||||
|
||||
Replace inline Pool creation with `createPoolFromConfig` from `@bete/shared`.
|
||||
|
||||
- [ ] **Step 4: Update gateway's shared/database/drizzle.ts**
|
||||
|
||||
Replace inline Pool creation with `createPoolFromConfig` from `@bete/shared`.
|
||||
|
||||
- [ ] **Step 5: Run typecheck across all services**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run typecheck`
|
||||
|
||||
- [ ] **Step 6: Run biome format**
|
||||
|
||||
Run: `cd /home/code/GMW && pnpm run format`
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add packages/shared/src/database/ services/backend/src/shared/database/index.ts services/discord-gateway/src/shared/database/drizzle.ts
|
||||
git commit -m "refactor: extract shared database pool creation into @bete/shared
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 10: Audit moderationState vs conversationState overlap (Phase 10)
|
||||
|
||||
**Files:**
|
||||
- Read: `services/discord-gateway/src/modules/ai-moderation/moderationState.ts`
|
||||
- Read: `services/discord-gateway/src/modules/ai-moderation/conversationState.ts`
|
||||
|
||||
- [ ] **Step 1: Read both files and identify overlap**
|
||||
|
||||
Look for duplicated state management (maps, sets, timers).
|
||||
|
||||
- [ ] **Step 2: If overlap found, merge into one file**
|
||||
|
||||
Otherwise, just add comments documenting the boundary.
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add services/discord-gateway/src/modules/ai-moderation/
|
||||
git commit -m "refactor(gateway): consolidate conversattion/moderation state management
|
||||
|
||||
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 11: Redis connection audit (Phase 11)
|
||||
|
||||
**Files:**
|
||||
- Read: all Redis connection sites in gateway
|
||||
|
||||
- [ ] **Step 1: Identify all Redis connections**
|
||||
|
||||
Search for `new Redis(` patterns in gateway.
|
||||
|
||||
- [ ] **Step 2: Verify each has a valid reason for a separate connection**
|
||||
|
||||
Document with comments if needed.
|
||||
|
||||
- [ ] **Step 3: Commit (if any changes made)**
|
||||
|
||||
Reference in New Issue
Block a user