Compare commits

149 Commits
Author SHA1 Message Date
asepharyana 6244e307a3 feat: surface AI analysis duration across gateway, backend, and FE
Adds per-message AI moderation analysis time (ai_analysis_duration_ms)
so operators can see how long the LLM took to moderate each message.

Gateway:
- messagesTable: new ai_analysis_duration_ms (bigint) column.
- AIAnalysisUpdate + buildAIAnalysisSet: carry analysisDurationMs through
  both single and bulk update paths.
- ai-analysis-worker: measure wall-clock time around runModerationAnalysis
  and attach it to every result in the batch.

Backend:
- Mirror schema column; messageMapper maps ai_analysis_duration_ms;
  moderation-types + MappedMessage expose it.

Frontend:
- message.ts type gains ai_analysis_duration_ms.
- AiBadge (messages view) shows 'status · 1.2s' when duration is present;
  analysis view badge mirrors the same formatting.

DB:
- scripts/add-ai-analysis-duration.sql (idempotent ADD COLUMN IF NOT EXISTS).

No behavior change for moderation logic; null until new gateway build
records values.
2026-08-16 00:11:00 +07:00
asepharyana 2d7c7f2c35 fix(gateway): stop Qdrant upsert aborts (semantic cache was being skipped)
Qdrant upserts were failing with 'This operation was aborted' ~32x/2h,
so semantic moderation cache entries were silently dropped. Root cause:
upsertQdrantPoint ran ensureQdrantCollection() on EVERY call — a GET
(and sometimes DELETE+PUT) round-trip — while the request AbortController
had only a 10s timeout. Under moderation load Qdrant is busy (the
gmw_text_moderation collection is not yet HNSW-indexed, so searches are
full-scans), the extra round-trips pushed the upsert past 10s, and the
client aborted it.

- Memoise ensureQdrantCollection() at module scope so the collection is
  verified exactly once per process (resetQdrantCollectionCache() for
  tests / config reload).
- Bump the upsert request timeout 10s -> 30s so a transiently busy
  Qdrant no longer aborts the write.

Qdrant server itself is healthy (<100ms for direct upsert; collection is
green), so no server-side change is needed. Semantic cache should now
populate reliably.
2026-08-15 23:40:19 +07:00
asepharyana 416c690ebc style(gateway,backend): clear all biome warnings (no warnings left behind)
Address every remaining biome lint/format warning across both services
so the codebase ships warning-free:

- textCacheStore: drop unused deleteExpiredQdrantPoints import; hash
  image cache key (sha256[:32]) so long/base64 URLs no longer blow the
  text_analysis_cache PK B-tree 8191-byte index (was aborting the media
  analysis lock INSERT).
- bootstrap: drop unused unhandledRejection promise param.
- moderationOrchestrator: drop unused  destructure at L197.
- mediaDownloader / textBatchProcessor / transmitter: replace non-null
  assertions with proper null guards (stickerName ?? '', urlImages.get
  guard, backpressureQueue.shift guard).
- backend utils: throw lastError ?? fallback instead of lastError!.
- message-capture: remove unused  (retentionDb),  (moderationActionsDb,
  reviewsDb); simplify renderDiscordMentions guard to optional chain.
- transmitter: remove dead write-only  field + its assignments.

No behavior change beyond the cache-key hashing (now deterministic
fixed-length) and the intentional null-safety guards.
2026-08-15 23:18:33 +07:00
asepharyana c590a8be27 style(gateway): biome format fix for imageResizer (unblock CI gate)
imageResizer.ts had a line exceeding the print width that biome flagged
as a formatter error, failing the Build & Deploy biome check. Re-format
the file. No logic change.
2026-08-15 23:11:31 +07:00
asepharyana 9c83ec86cc fix(gateway): image vision analysis + media cache lock failures
Two root causes behind 'all image analysis failing':

1. imageResizer still emitted lossless PNG for vision input. A 1024px
   Facebook photo balloons to multi-MB PNG base64 that the vision model
   silently rejects ('Vision API null response'). Switch to JPEG q85
   (no upscaling) — same photo drops to ~100-400KB, model processes fine.
   Re-encodes even already-small images so raw originals never bloat the
   data URL. Added tests/imageResizer.test.ts covering both cases.

2. acquireMediaAnalysisLock INSERT aborted with 'index row requires N
   bytes, maximum size is 8191'. text_analysis_cache.text is the PK in a
   B-tree index (8191-byte/row cap); callers pass the raw image URL as the
   key, and base64 data URLs / very long URLs blow past the limit, so the
   lock INSERT fails and every media analysis is skipped. Hash the URL in
   makeImageCacheKey (image:<sha256[:32]>) — fixed-length, deterministic,
   well under the limit. All store/get/lock/delete callers already route
   through this function so lookup stays consistent.
2026-08-15 23:04:52 +07:00
asepharyana 17a4fbd73d build(gateway): skip fixupPhase to kill 'patchelf: wrong ELF type' noise
dontPatchELF only disabled the patchELF sub-phase; fixupPhase's
shrinkELF step still emits the same error on the prebuilt .node addons
and .o/.a object files in node_modules. Skip the entire fixupPhase
(dontFixup = true) for the gateway — node is the external interpreter
and .node addons are self-contained dlopen prebuilts, so Nix RPATH
patching/stripping is neither needed nor wanted.
2026-08-15 22:20:56 +07:00
asepharyana c04c410fad build(gateway): suppress harmless 'patchelf: wrong ELF type' noise
Add dontPatchELF = true to the discord-gateway derivation. Nix's
fixupPhase runs patchELF over $out/node_modules and chokes on the
non-ET_DYN ELF files (.o/.a objects + prebuilt .node addons), emitting
hundreds of non-fatal 'patchelf: wrong ELF type' lines per build. The
real binary is node (external, RPATH-fixed) and the .node addons are
self-contained prebuilts loaded via dlopen, so Nix RPATH patching is
neither needed nor wanted. Shebang patching still runs.
2026-08-15 22:11:58 +07:00
asepharyana 5e5f4ae208 build(gateway): use @discordjs/opus prebuilt instead of compiling from source
Drop npm_config_build_from_source=true so node-pre-gyp downloads the
published prebuilt .node for Node 22 (ABI node-v127, linux-x64-glibc-2.35)
instead of compiling libopus C++ every build. Replace the hardcoded
'npm run install' (node-gyp compile) loop with 'pnpm rebuild @discordjs/opus'
which runs the package's own install script (prebuilt fetch, source build
only as fallback). sharp already uses @img prebuilt packages (its install
script failure is non-fatal), so only opus was actually compiling.
2026-08-15 21:56:22 +07:00
asepharyana e2013988ff ci: fix biome format gate so Build & Deploy passes
Auto-format llmClient.ts (Object.assign indent) — the only biome
error blocking the Build & Deploy workflow. Logic unchanged; gateway
biome check now exits 0 (11 pre-existing warnings remain, non-blocking).
2026-08-15 21:39:44 +07:00
asepharyana 0164444dd7 refactor(gateway): remove screen-share / GoLive feature entirely
Drop the Discord Go Live (screen share) stack across the discord-gateway:
- delete src/goLive/ (19 modules: Streamer, Demuxer, encoders, WebRTC wrapper, native loader, etc.)
- delete native/libdatachannel-min/ N-API binding + flake native build + LD_LIBRARY_PATH wiring
- delete screenShareController.ts and screen-share tests (goLive-port, golive-*, demuxerNut, screenShareInput)
- mediaSource.ts: remove Invidious helpers + downloadScreenInput (YouTube full-file download)
- mediaTypes.ts: drop ScreenShare* types, narrow MediaMode to 'music' and DiscordPlayerOwner to non-screen
- media.handler.ts: remove screen branch, screenController/screenPlayback, voice-disconnect/reconnect accessor
- commandHandler.ts: stop passing getVoiceStatus / setVoiceController into MediaHandler
- media handler now only handles music; music queue/playback/status untouched

Verification: tsc --noEmit clean, biome clean on touched files, no lingering goLive/screenShare refs in BE/FE/gateway.
2026-08-15 21:20:20 +07:00
asepharyana 9ae26b8ec9 refactor(llm): unify vision routing with text moderation and remove dedicated endpoint 2026-08-15 21:05:06 +07:00
asepharyana 7ebee7559d feat(llm): add disableThinking option for faster LLM analysis and update config 2026-08-15 20:52:53 +07:00
asepharyana 66c33a2657 feat(message-capture): add bot exclusion logic for message capture 2026-08-15 20:35:51 +07:00
asepharyana 25b220b7f9 fix(frontend): sidebar + command palette navigation, zero biome warnings
Router.push was a no-op in the standalone build (Next trailingSlash
interaction), so the sidebar buttons and command palette silently failed
to navigate. Replaced next/link + router.push with plain <a href> anchors
in NavRail and CommandPalette — verified working on all routes.

Biome tightened to zero warnings:
- Disable noArrayIndexKey (positional equalizer bars), noStaticElementInteractions
  (intentional dismiss/hover overlays), useMediaCaption (voice clips)
- Avatar uses background-image instead of <img> (noImgElement)
- Command palette list items keyed correctly
- Format pass to satisfy the formatter
2026-08-15 20:25:44 +07:00
asepharyana 1c4f28c5f2 fix(message-capture): remove bot message filtering from capture logic 2026-08-15 20:21:06 +07:00
asepharyana 392db8eba1 feat(frontend): Ambient/WebGL console revamp + lint/type cleanup
Ground-up rebuild of the GMW frontend as an Ambient Field console:
- WebGL ambient background (Three.js shader, drifting motes, reduced-motion aware)
- Glassmorphism dark cyber theme across all 8 routes
- SSR page + client view split with SWR fallback; realtime via WebSocket
- Command palette (Cmd+K), chatbot FAB, guild/channel pickers
- Chart primitives: donut, radial-gauge, area-activity, sparkline, equalizer

Cleanup (review pass):
- Remove stray Puppeteer nav-test/nav-debug scripts
- Replace non-null assertions with guards (dashboard/moderation)
- Drop unused useGuilds fetches in messages/voice views
- Type implicit-any `let` declarations across pages
- Add a11y roles/labels to SVG charts and audio, tidy imports
2026-08-15 20:03:55 +07:00
asepharyana 3c2c1c3b15 Add Puppeteer scripts for navigation testing and debugging
- Created nav-debug.cjs to log anchor tags and simulate clicks on the Voice navigation link, capturing click events and page navigation.
- Added nav-test.cjs to test the Voice link click and log the URL at various intervals, capturing any page errors.
- Introduced nav-test2.cjs to check the presence of specific elements on the /voice/ page and log any console errors.
- Implemented nav-test4019.cjs to monitor network requests and responses related to the Voice navigation, verifying button presence and click functionality.
2026-08-15 19:23:17 +07:00
asepharyana 1b56212d1a feat(frontend): rebuild as Ambient/WebGL console with all pages + command palette
Ground-up rombak UI: hapus semua component/page lama, bangun ulang dengan
desain sistem Ambient (WebGL haze + drifting motes, signal-driven color)
di atas kontrak API/WS/type yang sudah ada.

- Design system: globals.css tokens + primitives (glass, button, badge,
  select, avatar, toast, chart SVG murni).
- Shell: nav rail, topbar (status WS + pill signal + theme), AppFrame.
- 8 halaman: dashboard, voice (orbital stage), media, messages (live feed +
  detail AI), moderation, analysis (search), recordings, + chatbot floating.
- Command palette (Cmd/Ctrl+K) untuk navigasi cepat.
- Server fetch di-page di-try/catch agar render graceful saat backend mati.

Verified: tsc clean, next build 8/8 halaman, semua route 200.
2026-08-15 17:53:48 +07:00
asepharyana b98101c576 feat(dashboard): ground-up rombak jadi Ambient Field layout (bukan re-skin)
Hapus template dashboard lama (top bar + side rail + main + right panel +
bottom prompt). Ganti dengan layout yang benar-benar beda:

- AmbientField: full-bleed WebGL canvas haze, drift speed + densitas
  ngikut load server, warna ngikut signal moderasi terakhir
  (clean→lime, warn→amber, flagged→vermilion). Background tanpa container.
- View jadi full-bleed: headline raksasa bottom-left, metric cluster
  floating top-right (no box), event ribbon drift di tengah, command
  whisper di very bottom.
- AmbientShell di layout.tsx: gak ada TopBar/LeftRail untuk /dashboard
  exact. Route lain (messages/voice/media/dll) tetap ClassicShell.
- Tidak ada card, tidak ada grid, tidak ada panel, tidak ada tab.

Verified: tsc clean, next build 11/11 halaman, biome clean.
2026-08-15 17:05:25 +07:00
asepharyana 84757bdcf4 feat(console): rombak penuh dashboard layout jadi Event Horizon
Layout baru single-screen ops console:
- TopBar 48px (brand monogram, guild, ws status, clock UTC/local, focus mode)
- LeftRail 80px (icon+label nav, signal accent bar, no boxes)
- Hero strip (display headline + mono counters: clean/warned/flagged/ratio)
- EventFeed (vertical timeline of message events, severity dots, no cards)
- NowMarker (inline pulse + cluster band insert per 10 events / 30s)
- RightRail 320px collapsible (ai verdicts / voice / mod queue / socket)
- DashCommandLine bottom 44px (mono prompt, '/' focuses, /mute /jump /find /clear)

Replace Spine + StatusBar lama untuk /dashboard via pathname branch di
(dashboard)/layout.tsx — route lain (messages/voice/media/dll) tetap
pakai ClassicShell, tidak ter-regress.

SSR seed tetap lewat page.tsx (server fetch stats + activity), synthetic
seed events dari daily buckets sampai WS message_created kick in.

WS event mapper: severity di-derive dari ai_status + ai_severity,
excerpt dipotong 140 char, channel tail 4 char.

No card chrome, no shadow, no bento grid, no tab panels.
2026-08-15 16:21:45 +07:00
asepharyana 6c9a91dad4 style(vision): biome format llmClient.ts (wrap long const line) 2026-08-15 14:38:44 +07:00
asepharyana bcb563ea7f feat(vision): route multimodal analysis to dedicated NVIDIA direct endpoint
- config: add AI_LLM_VISION_BASE_URL + AI_LLM_VISION_API_KEY (separate from text router)
- llmClient: llmVision() now calls dedicated vision endpoint when configured
  (axios POST to integrate.api.nvidia.com, model nvidia/nemotron-3-nano-omni-30b-a3b-reasoning,
  reasoning_budget 16384, non-stream), falls back to router combo otherwise
- keeps text/moderation on omniroute, vision on NVIDIA direct
2026-08-15 14:31:53 +07:00
asepharyana 589fd38fd8 fix(voice): separate Mic and Listen state (were both bound to listen)
- MicControl now uses useMicTransmit + local micActive/micVolume
  (was wrongly wired to listen.active/listen.toggle)
- ListenControl keeps useVoiceListen + handleListenVolume
(tsc clean, next build green)
2026-08-14 12:35:44 +07:00
asepharyana da02bfff9b fix(frontend): rebrand Bete → GMW (title, logo aria-label, dashboard heading)
- layout.tsx metadata title: Bete → GMW - Discord Moderation Console
- spine.tsx logo aria-label: Bete → GMW
- dashboard/view.tsx heading: Bete Console → GMW Console
(tsc clean, next build green)
2026-08-14 11:51:16 +07:00
asepharyana a66db8d702 fix(voice): live connection state instead of static SSR snapshot
- VoiceView now reads connected/activeChannelName from useVoiceStatus
  (SWR live, invalidated by connect/disconnect) instead of initialStatus
- Seed useSpeakers from live status.activeSpeakers
- Add 4s refreshInterval to useVoiceStatus so state converges
(tsc clean, next build green)
2026-08-14 11:43:07 +07:00
asepharyana d65dc11c73 fix(frontend): restore voice guild/channel picker + media URL queue input
- voice/view: add Select for guild + voice channels + Connect/Disconnect bar
- media/view: restore URL queue input + Screen toggle + Queue button
(tsc clean, next build green)
2026-08-14 11:28:08 +07:00
asepharyana 8b281c7feb refactor(frontend): finish design-system migration — chatbot, a11y, lint
- Rewrite chatbot container + panel to new surface/signal/ink tokens
  (was still on dead glass/text-primary tokens -> wrong colors)
- loading-skeleton: glass -> surface-2
- Fix a11y: SVG charts role=img+aria-label, audio aria-label,
  message-entry as real <button>, tooltip biome-ignore (intentional)
- Type messages/page initialPage (noImplicitAny)
- tsc clean, next build green, biome 0 errors
2026-08-14 11:02:52 +07:00
asepharyana 5bbf75a65b refactor(frontend): finish shadcn→custom primitive migration (green build)
- Remove tw-animate-css import + dead src/components/ui shadcn tree
- Convert 7 orphaned components (moderation, analysis, guild-selector,
  voice/activity-timeline, shared/empty+error) to new primitives
- Add missing moderation/view.tsx; analysis uses SearchPanel directly
- globals.css now uses new signal-driven ops-console tokens
- tsc --noEmit clean, next build green (11 routes), local smoke 200
2026-08-14 10:49:44 +07:00
asepharyana 5816e94a63 fix(goLive): remove syncStream — synthetic PTS timebases make A/V sync deadlock
Symptom: video plays ~1s then freezes. BaseMediaStream sync logic:
- video _pts advances 33.3ms/frame (timeBase 1/fps), audio _pts advances
  20ms/packet (timeBase 1/48000) — two synthetic frame-index timebases that
  never share a clock.
- If audio starts late (ffmpeg audio init / Ogg header), ptsDelta = video-audio
  stays positive → isAhead() true → video loops 'await sleep(frametime) while
  isAhead()' → video freezes. Downchain: vPipe fills → proc.stdout paused →
  demuxer emits ~15fps (log: 30 frames per 2s).

Upstream dank sets syncStream because node-av provides REAL PTS from NUT in a
consistent timebase. Our raw-h264 demuxer has no real PTS; per-stream sleep-PTS
pacing alone keeps both at 1000ms/s, which is correct without a shared clock.
Re-enable sync only if real PTS is added.
2026-08-13 19:06:36 +07:00
asepharyana 11f2ad5f23 fix(goLive): kill 4.3s backlog — HWM2 pipes + wire A/V sync (dank-faithful)
Lag root cause: vPipe/aPipe were objectMode PassThrough HWM 128 → the pipe
held up to 128 frames ≈ 4.3s of video before backpressure reached the encoder.
The viewer was watching a 4+ second stale backlog.

Fixes (both faithful to @dank074/discord-video-stream):
1. vPipe/aPipe HWM 2 — at most ~1-2 frames in flight (~66ms @ 30fps), so the
   writeFrame() backpressure pauses ffmpeg stdout almost immediately and the
   whole chain (encoder → NUT → demuxer → vPipe → BaseMediaStream → WebRTC)
   runs at the sender's real pace, exactly like dank's 'resume &&= vPipe.write'.
2. Wire vStream.syncStream = aStream — audio is the master clock; video
   sleeps/wakes on ptsDelta like upstream newApi.js. Prevents A/V drift under
   variable encoder throughput.
2026-08-13 18:34:28 +07:00
asepharyana 6e188f81d6 refactor(goLive): revert to dank-faithful demuxer — no custom pacing clock
Per user direction ('pakai dank sebagai referensi karena itu yg berhasil'):
drop the custom setInterval/tail-drop emission clock entirely. The demuxer
now writes each access unit straight to vPipe with a monotonic PTS and lets
BaseMediaStream (ported 1:1 from @dank074) handle pacing via sleep-PTS + A/V
sync, exactly like the upstream library. The custom clocks were the source of
the blank tile (IDR delivery race) and the lag (head-drop watching 10s-old
frames).

Adds proper backpressure: pause ffmpeg stdout when vPipe.write() returns
false, resume on drain — mirrors dank's 'resume &&= vPipe.write(packet)' so the
encoder self-throttles to the WebRTC sender's real pace instead of bursting.
2026-08-13 18:12:34 +07:00
asepharyana 7c376ea66a fix(goLive): keep IDR in own slot so decoder always has a reference (was blank)
The tail-drop rewrite let a P-frame supersede a pending keyframe before the
emit tick fired, so the decoder never received an IDR → blank GoLive tile.
Give keyframes their own slot (pendingKey) that P-frames cannot steal, and
only emit a P-frame once at least one IDR has been shown (haveReference).
IDR is always emitted first when present so the reference re-establishes.
2026-08-13 17:43:16 +07:00
asepharyana 8ee32b8df8 fix(goLive): tail-drop emitter clock — always show the freshest frame, never lag
The Node token-bucket pacer used HEAD-drop (emit frames in arrival order,
drop newer ones when over budget). Under the encoder's ~330fps burst (ffmpeg
-re does not reliably throttle YouTube-DASH webm), the viewer was watching
frames ~10s behind live → frozen / 'patah-patah' video while audio (not
rate-limited) played current = desync.

Replace it with a steady setInterval emission clock at videoFps: each tick
emits exactly ONE frame — the NEWEST buffered one — and discards everything
older (tail-drop). At most one frame is ever held, so no backlog and no lag;
the emit clock (not the encoder rate) defines playback speed. Keyframes are
never superseded so the decoder keeps getting IDRs. Audio stays in sync.
2026-08-13 17:32:18 +07:00
asepharyana c285a4c813 fix(voice): copy cookies to temp before yt-dlp + fall back to Invidious on cookie/permission errors
yt-dlp 2026.07.04 rewrites the --cookies file on close. Handing it the
root-owned /etc/.../ytcookies.txt (not writable by the gmw service user)
caused PermissionError -> exit 1 on every screen-share download attempt.

- buildCookieArgs on-disk branch now copies the system cookie file into a
  per-run temp file (like the env branch) so write-back lands somewhere we
  own; unreadable -> anonymous.
- resolveInputWithRetry Invidious fallback regex now also matches
  permission|EACCES|cookie, so a cookie failure triggers the link-alternative
  (no-auth Invidious mirror) path instead of failing all retries.
- adds regression test asserting the original cookie path is never passed to yt-dlp
2026-08-13 17:16:05 +07:00
asepharyana f156fc0c9e fix(goLive): download screen-share media to file before play (not live pipe)
The live pipe (yt-dlp -o - -> ffmpeg) delivers data at network speed with
unreliable PTS, which defeats ffmpeg -re and made x264 -r 30 force-duplicate
held frames -> ~1fps video (the patah-patah symptom). Per user suggestion,
download the FULL clip to a temp file first (downloadScreenInput), then feed
that FILE PATH to prepareStream. String inputs already get -re, so the
encoder now paces cleanly at 1x against a monotonic-PTS file — proven
reliable in local tests (vs the live pipe which always bursted). Temp file
is removed on stream end / stop.

- getDirectScreenInput -> downloadScreenInput (returns file path)
- resolveInputWithRetry now awaits a completed file + retries on failure
- screenShareController.stops/cleanup removes the per-run tmpdir
- screenShareInput.test.ts updated to the file-download contract
2026-08-13 16:42:41 +07:00
asepharyana 89f1097729 fix(goLive): add -re throttle at encoder for screen-share pipe input
Previous code only added ffmpeg -re when input was a string URL. Screen
share passes a Readable pipe (yt-dlp merge -> stdout) delivered at network
speed (bursts + stalls). Without -re the encoder slurps it instantly and,
when the merge stalls, x264 -r 30 force-duplicates the last held frame
~30x -> viewer sees ~1fps while WebRTC still paces 30fps. Add -re for all
inputs so the encoder paces at the stream's native PTS rate and emits a
fresh picture every frame.
2026-08-13 16:18:51 +07:00
asepharyana df24c756a0 fix(goLive): token-bucket pacing + pin biome rules so CI passes
- Demuxer.ts: deterministic token-bucket video pacing (replace unreliable ffmpeg -re which did not throttle the live multi-stage pipe — demuxer emitted ~240fps vs 30fps sender, 100k+ frame backlog, frozen video). Surplus non-key frames dropped; keyframes forced through; audio on fd3 unaffected.
- biome.json: pin noExplicitAny/noUnused* to off/warn. Biome 2.5.x (drifted via --no-frozen-lockfile) promotes these to errors and was failing the CI gate on pre-existing backend code unrelated to this change. Restores the warn-level behavior the config schema 2.2.0 expects.
2026-08-13 14:25:02 +07:00
asepharyana add31d3561 fix(goLive): deterministic token-bucket video pacing at demuxer (replace unreliable -re)
Root cause (3rd iteration): ffmpeg '-re' on the demuxer does NOT reliably
throttle a multi-stage live pipe (merge ffmpeg -> encoder x264 -> NUT ->
demuxer). In production the demuxer still emitted ~240fps while the WebRTC
sender consumed 30fps, building a 100k+ frame backlog (observed: frames=197490
vs sent #24600, ~8.4 min in). The sender always emitted the OLDEST buffered
frame -> video frozen ~10 min behind live, while audio (tiny, jitter-buffer
recovered) stayed smooth. Local file/pipe tests showed -re working (30fps)
but the live YouTube/WebM pipeline did not — -re is not trustworthy here.

Fix: enforce 1x video output with a token-bucket limiter in the demuxer
(Node side), independent of ffmpeg. Capacity = 1s of frames, refill 1 token
per 1000/fps ms. Surplus non-key frames are DROPPED (never buffered) so the
sender always emits the newest frame; keyframes are forced through even over
budget so the decoder keeps a fresh IDR. The limiter does NOT stall the ffmpeg
process (unlike the earlier proc.stdout pause), so audio on fd3 keeps flowing.

Verified: tsc --noEmit clean.
2026-08-13 14:17:35 +07:00
asepharyana 6e7c4901c9 fix(goLive): pace demuxer with -re + bounded frame-drop (was: audio patah, 8s lag)
Root cause (revisited): the previous gate paused proc.stdout when vPipe was
full. That stalled the SAME ffmpeg process that also writes audio on fd3, so
audio stuttered; and the ~8s backlog already built never drained → permanent
lag. Symptom: 'video still lags bad, now audio also choppy'.

Fix:
- spawn demuxer ffmpeg with -re for stream (pipe) input. Verified locally:
  a 5s NUT clip demuxes in 0.088s without -re (57x burst) vs 4.539s with -re
  (real-time). -re throttles the input read, which back-pressures the whole
  upstream chain (encoder x264 -> merge ffmpeg -> yt-dlp) through OS pipes,
  pinning production at 1x. No unbounded backlog.
- drop oldest queued frame when vPipe readableLength >= 30 (transient sender
  stall guard) instead of pausing stdout — keeps video fresh and audio intact.
- removed gateSource/sourcePaused entirely.

Audio and video now pace together at 1x; video is the newest frame, not an
8-second-old one.
2026-08-13 12:41:49 +07:00
asepharyana 60faaa9304 fix(goLive): backpressure-throttle screen-share pipeline to 1x (video freezes while audio plays)
Root cause: prepareStream's ffmpeg consumed a YouTube VOD at download/CPU
speed (~10x real-time), so the demuxer buffered a huge frame backlog.
The sender paces at 30fps but always emitted the OLDEST buffered frames, so
the viewer saw frozen/laggy video while audio (tiny, jitter-buffer
recoverable) stayed smooth. That is exactly the 'video stuck, voice normal'
symptom reported live.

Fix: propagate vPipe backpressure UP to the demuxer's ffmpeg stdout — when
the sender can't keep up, pause the source, which stalls the demuxer and
back-pressures the encoder, pinning the whole pipeline to 1x. Also add a
realtime (-re) option for file/URL inputs (no-op for the streaming path,
which is what screen share uses).

Verified: 10s test clip encodes in 1.8s without -re vs 9.5s with it; tsc --noEmit clean.
2026-08-13 11:59:33 +07:00
asepharyana 5505983dbd fix(goLive): retry VIDEO(op12)/SPEAKING(op5) opcodes until ws OPEN — broken shared-screen video
Root cause: BaseMediaConnection.sendOpcode is a silent no-op when
ws.readyState !== OPEN. In GoLive, playStream() calls setVideoAttributes(true)
+ setSpeaking(true) the instant createStream() resolves (right after
SELECT_PROTOCOL_ACK), but the StreamConnection WebSocket can still be in
CONNECTING for a few ms — so op 12 (VIDEO, activating the video SSRC) was
silently DROPPED every session. Empirically verified: 0 ops 12/5 ever logged
across the entire journal, yet 10k+ video frames were sent and audio played
(audio SSRC is activated via the VoiceConnection handshake, independent of
GoLive op 12). Discord's media server thus received video RTP on video_ssrc
but was never told to forward it → black/broken shared-screen video with
working voice.

sendOpcodeWhenOpen retries up to ~2s for ws OPEN instead of dropping. Also
emits a=fmtp:101 packetization-mode=1;profile-level-id=42e01f in the answer
SDP (H264 FU-A fragments require packetization-mode=1 to reassemble).

Also removes pre-existing noNonNullAssertion lint (biome 2.5.8 now errors)
that was blocking the deploy CI.
2026-08-13 01:44:40 +07:00
asepharyana 3d57e9c102 fix(goLive): retry VIDEO(op12)/SPEAKING(op5) opcodes until ws OPEN — broken shared screen video
Root cause: BaseMediaConnection.sendOpcode is a silent no-op when
ws.readyState !== OPEN. In GoLive, playStream() calls
setVideoAttributes(true) + setSpeaking(true) the instant createStream()
resolves (right after SELECT_PROTOCOL_ACK), but the StreamConnection WebSocket
can still be in CONNECTING for a few ms — so op 12 (VIDEO, enabling the video
SSRC) was silently DROPPED every session. Empirically verified: 0 ops 12/5 ever
logged across the entire journal, yet 10k+ video frames were sent and audio
played (audio SSRC is activated via the VoiceConnection handshake, independent
of GoLive op 12). Discord's media server thus received video RTP on video_ssrc
but was never told to forward it → black/broken shared-screen video with
working voice.

sendOpcodeWhenOpen retries up to ~2s for ws OPEN instead of dropping. Also
keeps the H264 packetization-mode=1 answer-SVP (defensive SDP correctness).

Also fix: emit a=fmtp:101 packetization-mode=1;profile-level-id=42e01f in the
answer SDP — H264 FU-A fragments require packetization-mode=1 to reassemble.
2026-08-13 00:24:53 +07:00
asepharyana d3cb5f6756 refactor: rombak cache AI analisis image — pakai CDN URL langsung, hapus phash+sha
- Cache key image = CDN URL (query params stripped), bukan SHA data URL
  → re-analysis SAME attachment selalu cache-hit, berbeda attachment tidak kolisi
- Hapus perceptual hash (imghash dep + phash get/upsert/compute) sepenuhnya
- Hapus makeImageCacheKey hashing, ganti makeImageCacheKey yang return CDN URL
- textCacheStore, visionAnalyzer, mediaCache, mediaAnalysisClient updated
- imghash dependency removed from package.json
- Purge 82 stale cache rows (image: + phash:) dari DB
2026-08-12 22:31:08 +07:00
asepharyana 37787cc4f0 fix: prevent false positive moderation on physics/tech discussions
- Add examples for technical discussions (kinetic energy, drone weapon
  engineering, physics simulations) that should be marked clean
- System rule: physics/engineering topics (kinetik, gravitasi, energi,
  drone, senjata, drone warfare, CAD, CNC, 3D printing, robotics, aerospace)
  are safe when in technical context — flag only if explicit threat
- Riwayat pengguna dengan pelanggaran sebelumnya tidak memengaruhi
  penilaian pesan bersih yang terpisah dan tidak mengandung pelanggaran
2026-08-12 22:12:11 +07:00
asepharyana f849a87f2f fix: remove user history injection to prevent false positive moderation
- Removed getUserRecentInfractions usage in textBatchProcessor.ts and visionAnalyzer.ts
- Removed buildUserHistoryXml import and calls
- Messages are now evaluated standalone, not influenced by past violations in other channels
- Updated moderation prompts with clearer instructions about user_history usage
- Fixes issue where benign messages like 'tubuh manusia vs gravitasi' were incorrectly flagged due to carryover from previous drone weapons discussion

The user history context was causing the LLM to interpret unrelated current messages
as threats because it conflated them with past violations. Now each message is judged
on its own merit with only channel-specific context.
2026-08-12 20:48:26 +07:00
asepharyana deb5dedf2c test(gateway): add regression test untuk makeImageCacheKey collision
Verifies that two data URLs sharing the first 128 chars (same MIME prefix
+ identical base64 header — the real-world scenario that caused ALL images
to reuse the same cached vision analysis) produce DIFFERENT cache keys
under the fixed full-dataURL hashing, whereas the old 128-char-prefix
approach would collide. Also includes consistency + prefix tests.
2026-08-12 19:34:15 +07:00
asepharyana 3b221823e7 feat(gateway): add observability logging for vision cache hits/misses
Add debug logging to trace cacheKey + messageId + content length on
every vision cache HIT and MISS, so we can detect if the vision model
returns duplicate analysis for different images (provider issue vs
cache collision). Includes the phash on cache miss (new analysis cached).

Follow-up to 9f7ce7d which fixed makeImageCacheKey to hash full data
URL instead of just first 128 chars (root cause of all images sharing
the same cached 'konten judi' verdict due to hash collision).
2026-08-12 19:22:56 +07:00
asepharyana 9f7ce7dbd5 fix(gateway): hash full image data URL for cache key to prevent collision
Root cause: makeImageCacheKey() only hashed the first 128 chars of the
data URL. Since all resized images use the same MIME prefix
('data:image/png;base64,') + identical base64 header bytes, nearly every
image got the same 16-char hash → 'image:<same-hash>' → all images reused
the first cached vision analysis (often a gambling-detection verdict).

Fix: hash the entire data URL instead of just the prefix. Verified
114 stale 'image:' entries + 745 stale 'phash:' entries purged from prod
DB. tsc --noEmit clean, 133 tests pass.
2026-08-12 18:28:19 +07:00
asepharyana bd292fdf3d feat(gateway): Invidious fallback for YouTube 403 in screen share
YouTube blocks anon + cookies terbind ke IP browser (403 download).
Auto-rewrite youtube.com -> yewtu.be/invidious mirror saat cookies gagal.
- mediaSource: export isYoutubeWatchUrl/toInvidiousUrl/INVIDIOUS_INSTANCES
- screenShareController: resolveInputWithRetry tries Invidious instances on 403
2026-08-12 18:19:04 +07:00
asepharyana 7a7f433988 feat(gateway): read YouTube cookies from BWS env (gmw_yt_downloader_cookies) fallback to on-disk file
bws-exec exposes the BWS secret as env GMW_YT_DOWNLOADER_COOKIES.
Materialize to temp Netscape file (yt-dlp --cookies needs a path).
Falls back to /etc/gmw-discord-gateway/ytcookies.txt written by deploy.
2026-08-12 17:50:07 +07:00
asepharyana 84c5c36672 feat(gateway): YouTube cookies support for yt-dlp screen share + music
YouTube now blocks anonymous embeds (403 'Sign in to confirm you're not a
bot'). Resolve with account cookies via --cookies.

- mediaSource: buildCookieArgs() reads GMW_YT_COOKIES_PATH (default
  /etc/gmw-discord-gateway/ytcookies.txt) and injects --cookies into
  resolveMediaUrl + getDirectScreenInput + extractMediaInfo. Falls back
  to anon if file missing (graceful 403, not crash).
- bws-exec now writes cookies file from BWS secret gmw_yt_downloader_cookies
  on service start (systemd ConfigFile).
2026-08-12 17:46:05 +07:00
asepharyana c5898f7cf0 fix(gateway): crash safety on screen-share input timeout + proper error serialization
Root cause of "langsung left": YouTube bot-block/403 on u_c1tRmj7E4 (live
stream, LOGIN_REQUIRED) made yt-dlp timeout in resolveInputWithRetry (12s).
The timeout handler did cleanup() (removing once() listeners) THEN
tee.destroy(new Error(...)) — the PassThrough emitted 'error' with NO
listener left → unhandled stream 'error' event → uncaughtException →
gracefulShutdown → bot left voice.

Fix:
- resolveInputWithRetry: tee.destroy() silently after cleanup (error carried
  in the rejection only); add permanent no-op tee.on('error') safety.
- prepareStream: output.on('error') no-op so ffmpeg spawn failure before
  playStream attaches a demux listener never crashes the gateway.
- bootstrap: serialize uncaughtException/ClientError/DB errors with
  {err, errorMsg, stack} (pino only serializes the 'err' magic key — the old
  {error: err} key printed {} so crashes were invisible).
2026-08-12 16:59:41 +07:00
asepharyana 354e378e74 fix(gateway): output NUT (not raw h264) so Demuxer re-splits video+audio correctly
Revert 392bc35: streaming raw h264 video + opus on separate pipes broke
because prepareStream.output (pipe:1) feeds the Demuxer, but the opus
pipe:3 was never attached to the Demuxer's input — so for audio-capable
streams the Demuxer saw format=h264 (video-only) and emitted -an,
dropping audio RTP.

Correct design (from f1aa08c): prepareStream muxes video+audio into NUT
on a SINGLE pipe:1. The Demuxer then spawns a child ffmpeg that
demuxes NUT → -f h264 pipe:1 (pure AnnexB, start-code scan sees real
IDR type 5) + -f opus pipe:3 (Ogg Opus via createOggOpusDemux). The
start-code parser never touches NUT framing — it runs on the child
ffmpeg's clean h264 stdout.
2026-08-12 16:09:34 +07:00
asepharyana 392bc35a0d fix(gateway): output raw H264+Opus (not NUT) so Demuxer parses NAL keyframes correctly
Root cause: prepareStream muxed video+audio into a NUT container on pipe:1.
The Demuxer scans pipe:1 for AnnexB start codes (00 00 01) to split NAL
units into access units and classify keyframes (nal_type 5). NUT container
framing bytes sat in the stream and were scanned as NALs — NAL type 0
(NUT header) instead of 5 (IDR) → every frame classified key=false →
Discord decoder never got a decodable frame → static/black GoLive tile.

Fix: output raw H264 AnnexB on pipe:1 (demuxer target) and Ogg Opus on
fd3/pipe:3 for audio. NUT is only needed for *input* parsing (single
pipe carries both streams); output is demuxed into separate raw streams.
2026-08-12 15:39:47 +07:00
asepharyana 196cb1d3af fix(gateway): await audio stream line before demux resolve — audio RTP was dropped by metadata race
The demuxer resolved as soon as the VIDEO init line arrived on ffmpeg stderr.
With live NUT input the audio init line ('Stream #0:1: Audio: opus') lands in a
LATER stderr chunk (NUT info-stream packets are read incrementally from the
pipe), so `return { audio: aInfo }` captured undefined → playStream skipped
AudioStream → zero audio RTP on the audio SSRC → Discord showed a static
GoLive tile even though the NUT carried opus audio.

Fix:
- wait for BOTH video and audio init lines (when audio is expected) before
  resolving demux metadata, with a 3s timeout fallback
- default aInfo to opus/48kHz when withAudio instead of undefined, so the
  audio stream is always exposed even if the metadata line races the return
2026-08-12 14:45:06 +07:00
asepharyana 00fc852a32 feat(rtp-capture): add two-peer RTP capture test for H264 frame transmission 2026-08-12 14:26:54 +07:00
asepharyana d9f5592e6e feat(glossary): persist resolved definitions in Postgres + harden live SearXNG lookups
- Add term_glossary_cache table + migration 0014: resolved definitions are
  stored permanently (definitions rarely change); misses stay ephemeral in
  Redis/LRU with 1h TTL so transient failures get retried
- Lookup flow: LRU -> Redis -> Postgres (permanent) -> live SearXNG; DB hits
  re-warm the fast caches; stale Redis miss sentinels no longer shadow DB
- Rate-limit-aware live lookups: concurrency 2 + stagger, retry once on empty
  results, strict definition filter (Wikipedia preferred, rejects
  disambiguation/ads/translate-homepages)
- Make SEARXNG_BASE_URL configurable via env (default unchanged)
2026-08-12 14:22:22 +07:00
asepharyana f1aa08cdf6 fix(gateway): deliver audio + per-IDR SPS/PPS in GoLive screen share
Screen share showed a single frozen frame: the GoLive pipeline sent video
only (-"-an", h264 muxer cannot carry audio) so the audio SSRC never
transmitted and Discord kept the stream in thumbnail state.

- prepareStream: mux NUT when includeAudio (h264 muxer drops audio) and
  return the actual container format
- Demuxer: support NUT input with a second output pipe (fd3) carrying
  Ogg Opus; parse OGG pages into opus frames (20ms, 48kHz) emitted as
  GoLiveFrames; fix metadata parsing that dropped the audio stream line
  when it arrived in a later stderr chunk (early parsedMeta return)
- playStream: pipe audio.stream into AudioStream → RTP on the audio SSRC
- Encoders: -x264-params repeat-headers=1 → SPS/PPS inline before EVERY
  IDR (NUT remux drops container extradata; also enables PLI recovery)
- screenShareController: includeAudio true
- tests: demuxerNut.test.ts — OGG parser unit test + real ffmpeg NUT
  integration (video access units + parsed opus frames)
2026-08-12 14:20:29 +07:00
asepharyana f70a92880e feat(glossary): implement term glossary for LLM moderation with caching and extraction logic 2026-08-12 13:44:52 +07:00
asepharyana 88b13225cd fix(gateway): stream screen-share input from yt-dlp stdout — no more raw-URL 403
Second root cause (2026-08-12): even with yt-dlp http_headers forwarded,
YouTube still returns 403 when a signed DASH URL from --dump-single-json is
fetched raw by ffmpeg/curl on some videos (verified on fONoh7Pc6VU: curl
with the EXACT headers got 403; yt-dlp's own downloader succeeded). The
signature is tied to the extracting client context (po_token/visitor), not
just UA/IP.

Fix: getDirectScreenInput now spawns 'yt-dlp -o -' and returns its stdout
as a Readable — the same mechanism resolveMediaUrl already uses for music.
yt-dlp handles auth, cookies and transient retries internally. Merge
fragments go to /tmp/gmw-ytdlp-tmp (Nix store CWD is read-only → EACCES).
Removed resolveScreenInput + mergeScreenStreams (dead code).

Controller resolveInputWithRetry unchanged: tees the stream, waits for the
first byte (12s), retries with a fresh yt-dlp run up to 3x on error/EOF/
timeout, and destroys stuck inputs (EPIPE) so no process leaks.

Tests: rewritten for streaming (yt-dlp emits bytes; fail mode = exit 8
without stdout → stream must terminate with zero bytes).
2026-08-12 13:23:59 +07:00
asepharyana 67ab289caa fix(gateway): fail-fast + retry on screen share merge failure (black tile zombie)
Root cause (2026-08-12 11:50 test): merge ffmpeg hit a transient YouTube
403 and exited code 8 BEFORE prepareStream attached its input listeners
(voice release+join takes ~10s). The input's end/error events fired into
the void, the encoder stdin never received EOF, demux resolved with
fallback 0x0 metadata, setSpeaking fired anyway → stream 'started' with
zero frames for 8+ minutes (black tile, both ffmpeg processes hung).

Fixes:
- mediaSource: pass yt-dlp http_headers (UA/referer) to the merge ffmpeg
  via -headers to suppress transient 403s; destroy the returned stream
  with an error when the merge exits non-zero before producing bytes.
- screenShareController: resolveInputWithRetry — tee the merge stream and
  wait for the first readable byte (12s timeout) before proceeding; on
  error/EOF/timeout retry the whole resolution with a FRESH yt-dlp run
  (signed DASH URLs expire fast) up to 3 attempts. Stuck merges get
  EPIPE via input.destroy() so no process leaks per attempt.
- prepareStream: race guard — if the input already ended/destroyed before
  listeners attach, EOF the encoder stdin immediately; first-frame
  watchdog in playStream rejects 'started but nothing flowing' after 10s
  instead of resolving with a silent black stream.

Tests: +2 (merge-fail zero-byte terminal state, -headers forwarding).
2026-08-12 12:17:43 +07:00
asepharyana ef9e243609 fix(goLive): encode H264 baseline to match SDP profile-level-id (black tile)
SDP offer advertises profile-level-id=42e01f (constrained baseline) but
x264 encoded the default High profile — Discord's receiver configures its
decoder from the negotiated profile, so the High-profile bitstream failed
to decode → black GoLive tile despite valid access units + correct RTP
timestamps (fixed in 42a503c).

- Add -profile:v baseline to H264 encoder options (matches @dank074's
  proven config; SPS now 6742c01e → profile_idc=66 baseline, aligns with
  the 42e01f fmtp).
- Default x264 tune film → zerolatency (no lookahead — correct for live
  GoLive; @dank074 uses it).
- Update goLive-port test to assert baseline + zerolatency.
2026-08-12 11:37:04 +07:00
asepharyana 42a503c206 fix(goLive): demux access-unit grouping + correct RTP timestamps (black tile)
Demuxer emitted each AnnexB NAL as its own WebRTC frame (SPS/PPS/SEI
separate from slices) with a near-zero timestamp delta (duration=1 in a
1/90000 timebase → RTP +1/frame instead of +3000 @30fps). Discord's H264
receiver never receives a complete decodable access unit → black GoLive
tile despite frames flowing.

- Group NALs into access units: buffer param-set/SEI NALs, flush one
  frame per slice with preceding parameter sets (AnnexB start codes kept
  so the H264RtpPacketizer finds NAL boundaries).
- Timestamp each frame at the video frame rate: duration=1, timeBase
  1/fps → BaseMediaStream frametime=1000/fps ms → RTP +clockRate/fps
  (3000 @ 30fps/90kHz) and correct pacing.
- Thread explicit frameRate from playStream options (raw H264 has no
  timing info; ffmpeg guesses 25fps on stderr).
- Strengthen golive-demux-live-e2e: validates every frame has a slice,
  no bare param-set frames, keyframes carry SPS/PPS, timeBase 1/30.
2026-08-12 11:13:00 +07:00
asepharyana 652974e23a fix(goLive): gateway crash on screenshare stop — unhandledRejection during teardown
Test 00:32 confirmed the video pipeline WORKS (1410 frames @ 1280x720 sent,
ready=true, camera off) but the gateway crashed at stream stop:
unhandledRejection → graceful shutdown → systemd restart (bot offline).

Root cause candidates (both were fire-and-forget promises without .catch):
- BaseMediaConnection.setProtocols().then(...) — rejects when the PC is
  closed while setProtocols is in flight (stream teardown)
- void webRtcConn.createOffer().then(...) — rejects when the PC closes
  while the offer is still gathering

Fixes:
- .catch on both promise chains (log + continue; teardown is expected)
- unhandledRejection handler now treats transient stream errors (EPIPE,
  ERR_STREAM_DESTROYED, ERR_STREAM_WRITE_AFTER_END, ECONNRESET) like
  uncaughtException already does — warn + continue instead of shutting
  down the whole gateway. Non-transient rejections still log + shutdown
  (with String(reason) so the detail actually shows).
2026-08-12 00:36:53 +07:00
asepharyana 407e003399 fix(goLive): black screen root cause — h264 muxer can't carry audio; disable self_video camera
ROOT CAUSE of empty GoLive tile (finally): prepareStream ran with
includeAudio: true + output -f h264. The h264 muxer cannot mux audio
('h264 muxer does not support any stream of type audio') → header write
fails -22 → stdout empty → Demuxer ffmpeg 'Invalid data found when
processing input' → 0 frames → black tile. Reproduced locally end-to-end
(13s backpressure delay + prepareStream + demux).

Fixes:
- screenShareController: includeAudio: false (video-only GoLive; demux
  path never delivers audio anyway)
- Demuxer: pin input format -f h264 for stream inputs (raw AnnexB H264
  has no magic header → auto-detect unreliable on delayed pipes)
- Streamer.signalStream: self_video: false — stop flipping on the bot's
  camera in Discord (user request; screen share ≠ camera)

Verified: local repro now emits 644 frames 1280x720 (was 0); tsc/biome/
vitest all green.
2026-08-12 00:25:01 +07:00
asepharyana 968a43b0f4 debug(goLive): instrument frame pipeline — demux spawn/stderr/frames, playStream resolve, sendVideoFrame drop/send
Tile kosong meski STREAM_CREATE handshake penuh (22:18-22:19 retest):
- Demuxer logs spawn args, ffmpeg stderr errors, frame count every 30
- playStream logs createStream resolved + demux done + setPacketizer
- sendVideoFrame logs DROPPED (ready/track) + sent frame count
2026-08-11 22:31:31 +07:00
asepharyana 91c7a67d2f fix(goLive): stream demux directly instead of spool-to-file (empty screen share)
Root cause of 'tile appears but content empty': demux() spooled the live
NUT/H264 input to a temp file and awaited stream 'finish' — but the merge
ffmpeg output never ends during playback, so demux deadlocked, no probe,
no transcode, 0 frames sent.

- Demuxer: pipe input straight into ffmpeg stdin (-i pipe:0), parse NAL
  frames live from stdout; parse video metadata from ffmpeg stderr with a
  1.5s race (fall back to H264 defaults). No spool, no await-end.
- screenShareController: pass width/height/frameRate (1280x720@30) to
  playStream — matches the prepareStream encode settings, so setVideoAttributes
  gets real dimensions even when ffmpeg can't report metadata on an open pipe.
- Add tests/golive-demux-live-e2e.ts: proves frames flow while input is
  still open (regression test for the deadlock).
2026-08-11 21:43:16 +07:00
asepharyana 8615383829 fix(goLive): STREAM_CREATE handshake — self_video voice state + retry + send instrumentation
- signalStream: flip voice state to self_video:true/self_deaf:false before
  STREAM_CREATE (Discord silently ignores the request while video disabled)
- createStream: attach dispatch listeners before first signal (race), clean
  up listeners on timeout, retry STREAM_CREATE every 3s up to 4 attempts
  (upstream issue #217/#219 — Discord randomly drops the request)
- sendOpcode: direct [goLive:Streamer] log bypassing bootstrap debug filter
  (proves op 18 is actually broadcast)
2026-08-11 20:45:00 +07:00
asepharyana 10d7ecd405 fix(gateway): EPIPE crash on media stop — stream error handlers + no shutdown on transient stream errors 2026-08-11 20:10:05 +07:00
asepharyana ff554fcff2 fix(goLive): instrument voice/stream handshake + createStream timeout (12s) 2026-08-11 20:00:33 +07:00
asepharyana f8b253ba5e merge: libdatachannel-min GoLive stack (build -86pct, node_modules -1.09GB) 2026-08-11 19:07:57 +07:00
asepharyana c8473b0610 build(nix): fix binding link path — LDC_LIB is full .so path
binding.gyp appended '/libdatachannel.so.0.24.0' to LDC_LIB; nixpkgs output
layout is <out>/lib/libdatachannel.so.0.24.1. Make LDC_LIB the complete
library path (env or default) and drop the append.
2026-08-11 18:54:12 +07:00
asepharyana 3deca91ffe build(nix): use nixpkgs libdatachannel (no cmake/fetchFromGitHub)
libdatachannel-src fetchFromGitHub + manual cmake build fails: GitHub tarball
does not include git submodules (deps/plog, libjuice, libsrtp, usrsctp) →
CMake 'source directory does not contain CMakeLists.txt'.

Switch to pkgs.libdatachannel (0.24.1): nixpkgs builds submodules + ships
lib/dev outputs. In the Nix sandbox everything is consistent (store glibc),
so the GLIBC_ABI_GNU2_TLS issue that blocks host-local use of 0.24.1 does
not apply to the Nix build. binding.gyp defaults stay on local 0.24.0 for
dev; Nix sets LDC_INCLUDE/LDC_LIB to the store paths.
2026-08-11 18:45:51 +07:00
asepharyana edec2edf82 build(nix): binding — NAPI_INCLUDE from pnpm store (depth 3), gyp env fallback 2026-08-11 18:35:49 +07:00
asepharyana 17013fe1e5 build(nix): gateway binding — gyp env-var paths, correct cwd, tolerant install
- binding.gyp: resolve libdatachannel include/.so via LDC_INCLUDE/LDC_LIB env
  (node -e expression) instead of hardcoded /tmp/ldc-build paths
- flake buildPhase: run node-gyp from native/libdatachannel-min root (was
  build/ subdir → 'binding.gyp not found'); export LDC_INCLUDE (fetchFromGitHub
  source) + LDC_LIB (cmake build dir)
- flake installPhase: tolerate missing binding (screen share disabled, gateway
  still starts); copy .so real files via -rL
2026-08-11 18:32:09 +07:00
asepharyana 3acb03391a build(nix): gateway flake — build libdatachannel-min binding, drop datachannel/node-av/zeromq
- Replace the per-package rebuild loop (node-datachannel cmake-js, zeromq)
  with: opus build + libdatachannel-min N-API binding build (fetchFromGitHub
  libdatachannel v0.24.0 — pinned because nixpkgs 0.24.1 is glibc-incompatible
  with this host; sha256 1jk53qs…).
- Removes ~760MB of node-datachannel build/cleanup cruft from the build
  phase; node_modules now 423MB (was 1.5GB).
2026-08-11 17:53:47 +07:00
asepharyana 9109d3c898 perf(golive): drop @dank074/discord-video-stream — node_modules 1.5GB → 423MB
Remove the last heavy GoLive dependency now that src/goLive/ replaces it:
- @dank074/discord-video-stream (pulled in @lng2004/node-datachannel
  771MB, node-av 118MB + @seydx/node-av-linux-x64 167MB, zeromq 21MB,
  fluent-ffmpeg 13MB — ~1.09GB total)
- onlyBuiltDependencies: drop node-av/zeromq/@lng2004 (keep opus/esbuild/sharp)
- pnpm.lock regenerated; orphan .pnpm dirs removed locally
- @discordjs/opus prebuild: rebuilt binary copied into
  prebuild/node-v127-napi-v3-linux-x64-glibc-2.39/ (node-pre-gyp find path)

Verified: tsc 0 errors, vitest 8/8, biome clean, opus encode OK.
Fresh CI install now ~423MB instead of ~1.5GB.
2026-08-11 17:49:33 +07:00
asepharyana 9139e225f4 perf(golive): ffmpeg-spawn demuxer (no node-av) + E2E pipeline tests
Phase 2 — replace the 114MB node-av binary with a plain ffmpeg spawn:

Demuxer.ts: spool stream input to temp file → probe via ffmpeg stderr
(ffmpeg-headless ships NO ffprobe — parse 'Stream #0:0: Video: h264...
640x360, 30 fps' from -loglevel info) → ffmpeg -c copy -f h264 pipe:1
→ NAL-split frames. Falls back to h264 defaults when probe fails.

prepareStream.ts: resolve ffmpeg from FFMPEG_PATH env → Nix store
ffmpeg-headless (hash-prefixed entry!) → PATH; split encoder option
strings ('-forced-idr 1' → two argv) — fluent-ffmpeg used to split
automatically, spawn does not.

E2E tests (tsx, need LD_LIBRARY_PATH=/tmp/ldc-build):
- golive-demux-e2e.ts: real H264 file → 33 NAL frames + dims from probe
- golive-pipeline-e2e.ts: prepareStream → demux → 82 frames
- golive-videostream-e2e.ts: local peer pair → demux → VideoStream →
  native setPacketizer/sendFrame/addTimestamp → 33 frames sent connected

Pitfalls captured: setPacketizer before negotiation breaks createOffer
('No DataChannel or Track to negotiate'); track methods are read-only
(no monkeypatching); both peers must declare audio+video tracks or
answer hangs; state() returns 'closed' after close() — snapshot first.
2026-08-11 17:38:06 +07:00
asepharyana 9ae230d047 perf(golive/spike): binding addTrack + TS port of @dank074 media stack
Phase 1 spike: replace @dank074/discord-video-stream + node-datachannel +
node-av (1.3GB) with minimal libdatachannel N-API binding + native RTP
packetizers (H264 FU-A, RTCP SR/NACK, pacer) + pure-TS GoLive stack.

Binding v0.4: addTrack (m=audio/video SDP), TrackWrap w/ setPacketizer +
sendFrame (raw RTP to transport) + addTimestamp — verified by two-peer
handshake emitting SDP with audio(opus 120)+video(H264 101) and 8-frame
RTP roundtrip.

TS layer (src/goLive/, 21 files): CodecPayloadType, VoiceOpCodes,
GatewayOpCodes, utils, BaseMediaConnection (voice WS + DAVE + heartbeat),
VoiceConnection, StreamConnection, Streamer, WebRtcWrapper (SDP mungling,
DAVE encrypt, packetizer chain), BaseMediaStream (pacing/sync), VideoStream,
AudioStream, Demuxer (ffmpeg-spawn NUT/AnnexB, no node-av 114M binary),
Encoders, prepareStream/playStream.

Integration: screenShareController.ts now imports from ../../goLive/index.js —
prepareStream(prepared, ...) + playStream(prepared, streamer, {...}).

Tests: tests/goLive-port.test.ts (8/8 pass). tsc --noEmit clean. biome clean.
2026-08-11 16:16:52 +07:00
asepharyana a1a6d8b418 spike: expose libdatachannel media packetizer chain via Track
Track.setPacketizer(kind, ssrc, pt, clockRate, ...) builds the same
media-handler chain node-datachannel does for @dank074:
  RtpPacketizer (Opus | H264 | H265 | AV1) → RtcpSrReporter →
  RtcpNackResponder → PacingHandler(25Mbps, 1ms) for video
Track.sendFrame(encodedFrame) packetizes into RTP; addTimestamp(delta)
advances the RTP timestamp (node-datachannel contract).

Verified test-packetizer.js: two peers connected over tracks, real opus
frames + AnnexB H264 (SPS/PPS/IDR) flow through the chain without crash.
This removes the need for a JS RTP packetizer entirely — libdatachannel
0.24 has the full media stack built in.
2026-08-11 14:57:04 +07:00
asepharyana 4f06c30c05 spike: add addTrack + TrackWrap to libdatachannel-min binding
Expose rtc::Track with send(binary) for raw RTP — verified:
- SDP from addTrack(audio)+addTrack(video) has m=audio (opus 120)
  and m=video (H264 101 + H265/VP8/VP9/AV1 + RTX)
- libdatachannel Track::send() sends RAW RTP/RTCP when no media
  handler is set (verified in src/track.cpp impl::Track::outgoing) —
  so RTP packetization can live in pure JS, keeping the binding minimal

Also fix: Track class was missing from InitAll exports (crash on
TrackWrap::NewInstance — null FunctionReference).
2026-08-11 14:51:56 +07:00
asepharyana 2203dd5771 spike: minimal N-API libdatachannel binding — WebRTC handshake proven
Phase 0 of GoLive rewrite (drop @dank074/node-datachannel 771MB):
minimal N-API binding exposing PeerConnection/DataChannel/ICE/SDP,
built against libdatachannel 0.24.0 (from node-datachannel _deps source).

Verified: offer/answer/ICE/DataChannel roundtrip between two local
peers (test-handshake.js). Key findings:
- callbacks must be registered in ctor BEFORE createDataChannel
- SDP with candidates comes from localDescription() at gathering Complete
- answer auto-generates on setRemoteDescription(offer); do NOT call
  setLocalDescription() after or role=actpass breaks the peer
2026-08-11 14:23:31 +07:00
asepharyana a53d7b71da fix(voice): screen share GoLive died instantly — neutralize node-av custom ffmpeg filters
prepareStream (from @dank074/discord-video-stream) unconditionally appends
audio filters 'volume@internal_lib' + 'azmq' that exist ONLY in its custom
node-av jellyfin-ffmpeg build. The Nix deployment runs plain ffmpeg-headless
on PATH, so fluent-ffmpeg died instantly with 'Filter not found' (exit 8),
the NUT output stream stayed empty, and playStream's node-av demux failed
with 'Failed to open input from Readable stream: Invalid data found when
processing input' — every screen share failed ~100ms after start.

Fix: pass customFfmpegFlags ['-filter:a','anull'] — ffmpeg applies the LAST
-filter:a for a stream, so the trailing no-op filter overrides the custom
chain (verified: command ends with '-filter:a anull', transcode runs, node-av
demux finds video+audio). Realtime volume control was already removed from
GMW (a690e5b), so dropping the filters is lossless.

Verified end-to-end with the failing URL (youtu.be/fONoh7Pc6VU, AV1+Opus
DASH): getDirectScreenInput → NUT merge → patched prepareStream → node-av
demux finds H264 video + Opus audio streams.
2026-08-11 10:56:49 +07:00
asepharyana c18431bdbf fix(ai-moderation): never cache vision outputs that claim 'no image seen'
Root cause (3rd layer after 50371bd + 4f4c435): a vision model run
(2026-08-10) returned 'Maaf, saya tidak melihat gambar apapun yang terlampir...'
and that text was cached as a VALID vision_llm result (image + phash keys,
24h/7d TTL). Every subsequent analysis of the same image (same hash/phash)
hit the poisoned cache, so image analysis looked broken forever even though
9router responded fine — the moderation LLM wrote 'lampiran yang gagal
terbaca' from a cache hit.

Also: mimo via 9router streams reasoning in delta.reasoning +
delta.reasoning_details[].text (content:"") — extractChunkText only read
delta.reasoning_content, so those runs aggregated empty → 'Vision API null
response' (observed 08:54/09:07/09:38).

Fixes:
- llmClient.extractChunkText: fall back to delta.reasoning and
  reasoning_details[].text (mimo), on top of reasoning_content (gemma).
- visionAnalyzer: isNoImageSeenText() detects 'no image' style outputs;
  such results are NEVER cached, and poisoned entries are purged when hit
  (LRU/DB/phash) so re-analysis actually re-runs vision.
- Tests: reasoning/reasoning_details extraction + isNoImageSeenText
  (Indonesian + English, no false positives on real descriptions).
2026-08-11 09:55:43 +07:00
asepharyana 4f4c43555f fix(ai-moderation): attachment-upload race dropped images before vision
Root cause (2nd layer after 50371bd): the analysis worker could pick up an
image message while its attachment upload was still in flight
(upload_status='pending'). downloadAndExtractFrame then fell back to the
Discord CDN URL (cdn.discordapp.com), which often 404s for old/purged links,
and 'if (!res.ok) return' silently dropped the image — no log, no vision
call, empty image map, and the LLM produced a text-only verdict like
'lampiran yang gagal terbaca oleh sistem'.

Fixes:
- ai-analysis-worker: skip targets whose attachment upload is still pending
  (both batch + individual paths) — they stay ai_status='pending' and the
  next 15s cycle analyzes them after the upload lands.
- mediaDownloader.downloadAndExtractFrame: try uploaded_url first, then
  discord_url as fallback; log non-OK responses (status + host) instead of
  silently returning; log when all candidate URLs fail.
2026-08-11 09:44:34 +07:00
asepharyana 50371bd2d1 fix(ai-moderation): read delta.reasoning_content in stream aggregation — image vision never returned text
Root cause: 9router combo 'multimodal' routes to cloudflare-ai/@cf/google/
gemma-4-26b-a4b-it which streams ALL output in delta.reasoning_content
(content:"") and finishes with 'length' at max_tokens. llmClient only read
delta.content, so llmVision returned empty → every image moderation fell back
to text-only analysis ('Meskipun analisis gambar gagal' in every ai_analysis).

Fix: extractChunkText() prefers delta.content then falls back to
delta.reasoning_content (also handles message/text/response fields), with
unit tests for the exact 9router chunk shape. Verified live against a real
DB image: oc/mimo-v2.5-free (new first model in the multimodal combo) returns
a proper description in delta.content.
2026-08-11 08:06:28 +07:00
asepharyana 0792ff4dc0 perf(nix): prune devDependencies from shipped node_modules
gateway output 1.4G -> 424M (-70%), backend 198M -> 60M (-70%).

- pruneProd: delete every .pnpm dir not in 'pnpm list --prod' graph
  (biome/typescript/esbuild/drizzle-kit/vitest/tsx ~150MB+) then drop
  dangling symlinks (top-level, scoped dirs, hoist, .bin) so stdenv
  noBrokenSymlinks fixup passes.
- NOT using 'pnpm install --prod': it collapses the public-hoist dir
  (.pnpm/node_modules) that peer resolution relies on for
  @lng2004/node-datachannel + @seydx/node-av-linux-x64 (voice breaks).
- node-datachannel: strip build/_deps (cmake FetchContent ~380MB) +
  nested node_modules (nw-gyp/typescript/puppeteer ~380MB) after
  compile; runtime needs only build/Release/node_datachannel.node.
- verified: native binaries (datachannel/opus/zeromq) intact, all 27
  runtime modules resolve, dev tools 0.
2026-08-10 22:17:38 +07:00
asepharyana eb89bb79ed ci(deploy): fix attic push fallbacks - VPS-hop sudo, direct push retry, ssh URL
- VPS-hop attic push now runs via sudo so attic reads root's config
  (~/.config/attic) which has the imrnes-ts server (Tailscale). Without
  it the push ran as the CI user whose config only has pub ->
  'Server imrnes-ts does not exist', silently skipping the cache upload.
- Direct push retried 3x (attic push is idempotent): a transient 502
  (e.g. atticd restart mid-push, Traefik blip) no longer aborts the
  whole closure upload before falling back to VPS-hop.
- Restore $VPS_USER in the 3 ssh:// nix copy fallbacks (was committed
  as masked '***' -> nix copy would ssh as user '***' and fail).
2026-08-10 21:18:25 +07:00
asepharyana 7d6c741bb2 ci(deploy): force narinfo write with --ignore-upstream-cache-filter; Traefik readTimeout=0 on imrnes 2026-08-10 20:18:56 +07:00
asepharyana 4cb4904517 ci(deploy): fix attic fast path - public cache, extra-substituters, self-hosted client bootstrap
Root causes found by reproducing the 2026-08-10 run:
- attic 'gmw' cache was created private -> every narinfo/nix-cache-info
  read returned 401, so the VPS could never actually substitute from
  attic ('Substituted from Attic cache' was a false positive when the
  store path happened to be already present locally).
- VPS nix.conf used extra-trusted-substituters, which Determinate Nix
  never merges for nix-store CLI clients; extra-substituters (all
  users, no trust gate) fixes substitution (verified end-to-end:
  delete path -> nix-store --realise pulls from attic over HTTPS).
- runner bootstrap of the attic client depended on nix copy --from
  ssh:// (fragile, failed on runner); now the prebuilt attic client
  closure lives in the attic cache itself and the runner pulls it over
  HTTPS via extra-substituters configured in the Install Nix step.

Also surfaces bootstrap stderr on fallback for future debugging.
2026-08-10 18:29:00 +07:00
asepharyana 4ee295bd29 ci(deploy): push to attic directly from runner, skip slow SSH closure copy
The old Push-to-Attic step SSH-copied the full closure (~794MB gateway) to
the VPS on every new store path before attic push — at ~500KB/s that took
25+ minutes (observed 40min+ in-flight run). The runner can now push
straight to the public attic endpoint (https://attic.asepharyana.my.id,
token auth validated) after pulling the prebuilt attic client closure
(52MB) from the VPS via nix copy --from. Falls back to the VPS-hop flow
whenever the direct path fails.
2026-08-10 17:28:28 +07:00
asepharyana 65c9c2cd9e feat(ai-moderation): enrich analysis context with recency, repetition, user history and channel topic
- <message> targets now carry time (ISO), repetitions (N identical short texts = spam signal), bot and edited flags; escape id/user XML
- rich <user_reputation>: total_infractions, clean_streak, last_offense_days_ago, repeat_offender (7-day window)
- <user_history> with last flagged messages for repeat offenders (wires dead getUserRecentInfractions)
- <user_profile as_of> staleness signal; <location_context topic> from captured channel topic
- prompt framing + output instructions teach the LLM to use the new signals without treating history as proof
- tests: contextEnrichment.test.ts (13) + topic cases in conversationContext.test.ts
2026-08-10 17:15:33 +07:00
asepharyana 0a5254bf20 feat(ai-moderation): enhance context handling with structured XML blocks and user profiles 2026-08-10 16:46:55 +07:00
asepharyana 4a51f3055c ci(deploy): push builds to attic binary cache (attic.asepharyana.my.id) 2026-08-10 15:27:59 +07:00
asepharyana 185d81f0e0 feat(ai-moderation): reset offensive nickname instead of deleting message
When the ONLY violation is offensive_username (message content clean):
- Message is NOT deleted (nickname-only violation bypasses auto-delete)
- Member's server nickname is reset to default username via
  setNickname(null) (Discord shows the global username again)
- Action 'reset_nickname' logged to moderation_actions; cooldown
  10min per guild:user (LRU) so repeated messages by same member
  don't hammer the Discord PATCH
- Config: AUTO_NICKNAME_RESET_ENABLED / AUTO_NICKNAME_RESET_COOLDOWN_MS
2026-08-10 11:48:27 +07:00
asepharyana ecbb538c9f feat(ai-moderation): use per-server nickname (displayName) in analysis payload
- resolveDisplayName(): member.displayName from captured metadata,
  falls back to global username
- Applied to context lines, target message blocks, and media message
  blocks — LLM sees the name the channel actually sees (nickname can
  carry moderation signal itself)
2026-08-10 11:36:37 +07:00
asepharyana 4049ab4201 feat(ai-moderation): rich context + link media vision analysis
- Conversation context recency gates (GAP_MS/MAX_AGE_MS): drop stale
  messages before silence gaps; cold_start anchor + flow descriptor
  tells LLM whether conversation is ongoing or restarted
- [location] block: channel name, thread name, nsfw/age flags from
  captured metadata (thread names instead of bare IDs)
- Link media -> multimodal: text-batch URL fetches that resolve to
  images now run vision analysis (bounded 15s) and switch prompt to
  mixed mode; <web_content> gains og:title for page context
- pnpm-workspace.yaml: approve sharp build script (unblocks install)
2026-08-10 11:26:26 +07:00
asepharyana 5d094829c4 fix(ui): fit select popup to content and wrap long items
A single long guild name was clipped (whitespace-nowrap + narrow min-width),
so the dropdown rendered as a tiny 144px box with truncated text. Size the
popup to fit-content up to the available width and let option text wrap.
2026-08-07 19:29:36 +07:00
asepharyana abbd78f42b fix(ui): theme popover/input tokens and open select below trigger
Select dropdowns (voice tab, guild selector) rendered with a transparent
background because --color-popover/--color-input were undefined, and the
popup overlapped the trigger due to alignItemWithTrigger. Define the missing
theme tokens (popover, popover-foreground, input, secondary) and default the
select popup to open below the trigger.
2026-08-07 18:18:14 +07:00
asepharyana 4f9d4a5c7d refactor: remove unused UI components and replace GlassCard with Card in voice components
- Deleted Item, Kbd, Marker, Message, NativeSelect, Questionnaire, Spinner components.
- Replaced GlassCard with Card in VoiceActivityTimeline, VoiceConnectionCard, ListenControl, MicControl, and SpeakerWaveform components.
- Introduced AppSidebar and ThemeToggle components for improved navigation and theme management.
2026-08-07 17:33:12 +07:00
asepharyana 2c995b41d7 feat: add new UI components including RadioGroup, Resizable, Sidebar, Spinner, Table, Toast, and ToggleGroup
- Implemented RadioGroup and RadioGroupItem for radio button functionality.
- Created ResizablePanelGroup, ResizablePanel, and ResizableHandle for resizable panels.
- Developed Sidebar component with context for state management and various subcomponents (SidebarTrigger, SidebarMenu, etc.).
- Added Spinner component for loading indicators.
- Introduced Table component with TableHeader, TableBody, TableFooter, and related subcomponents for structured data display.
- Built Toast component for notifications with customizable actions and icons.
- Implemented ToggleGroup and ToggleGroupItem for toggle button functionality with context support.
2026-08-07 16:11:33 +07:00
asepharyana 18dd6a56ba feat(media): loop mode + high-quality OggOpus music playback
- Loop: toggle via POST /api/media/loop → COMMAND_MEDIA_LOOP; gateway
  replays finished music track on natural end (queue untouched); status
  payload exposes loop flag; FE tombol Loop di music-player + mini-player.
- Kualitas suara: music playback sekarang di-transcode sekali via ffmpeg ke
  OggOpus 48kHz stereo 192kbps dengan volume di-bake ke encode — menghindari
  double lossy encode (inlineVolume) yang bikin suara buram. Screen share
  tetap pakai jalur lama.
- Backend: MediaState.loop, setLoop service, route + schema validation.
2026-08-07 14:53:37 +07:00
asepharyana a690e5b63e refactor(media): remove volume control from FE & BE
Volume sudah di-set default 0.3 di gateway (suara kecil saat play), dan
user bisa naikin sendiri di Discord (command media:volume) — jadi kontrol
volume lewat dashboard tak perlu. Hapus:
- BE: POST /api/media/volume route, mediaVolumeSchema, setVolume service
- FE: useMediaVolume hook, mediaApi.volume, slider volume di music-player
  + mini-player, field volume/setVolume di MediaPlayerProvider
Pertahankan COMMAND_MEDIA_VOLUME di gateway (masih dipakai command DC)
dan mic volume (terpisah, tetap di voice page).
2026-08-07 14:27:33 +07:00
asepharyana 62ffb676f9 feat(media): default music volume 30% instead of 100%
Volume play music terlalu besar buat user — default sekarang 0.3 (30%)
di semua layer: player gateway (musicVolume=0.3), backend state/schema
(default 0.3), dan UI slider (fallback 0.3). User tetap bisa naikin
manual via slider volume di dashboard.
2026-08-07 13:41:12 +07:00
asepharyana 4797aca20f ci(deploy): poll service readiness instead of fixed 3s sleep
is-active after sleep 3 false-fails when the unit is still activating
(e.g. Next standalone boot >3s) — exit 3 flagged the deploy red even though
the service came up fine. Poll is-active up to 30s and only fail if it never
reaches 'active'.
2026-08-07 11:22:29 +07:00
asepharyana 42b8afd412 fix(flake): purge dangling pnpm symlinks in standalone tree
noBrokenSymlinks fails frontend build: .next/standalone/node_modules/.pnpm/
node_modules/semver -> missing target. The standalone server never resolves
pnpm's hoisted .pnpm dir (it bundles its own node_modules) — delete broken
symlinks before install so stdenv check passes.
2026-08-07 10:57:18 +07:00
asepharyana 7575a701bd style(backend): biome format — organize imports + spacing (live-speaker/redis-bridge/voice.service) 2026-08-07 10:49:16 +07:00
asepharyana 9f91155944 ci(deploy): build+deploy frontend package (SSR standalone server)
Sebelumnya frontend hanya static export yang di-serve nginx di dalam package
proxy. Sekarang frontend = runtime mandiri (Next.js standalone :4017) yang
nginx proxikan ('/' -> Next server, '/api' + '/ws' -> backend :4001). Tambah
'frontend' ke matrix deploy agar dideploy + memulai unit gmw-frontend.
2026-08-07 10:46:02 +07:00
asepharyana f20889868d feat(frontend): rebuild as SSR with server-authoritative shared state
Rombak total alur data frontend: dari static-export CSR (tiap browser
fetch sendiri + akumulasi state voice per-tab) jadi server-side rendering.

Frontend (Next.js):
- next.config: output export -> standalone; halaman jadi server components
- server data layer baru src/lib/api/server.ts (GMW_BACKEND_URL, no window)
- dashboard/media/messages/moderation/recordings/voice page -> RSC yang
  fetch backend di render-time, seed ke client view (SWR fallbackData)
- hook-hook utama terima initialData -> first paint data server, revalidate
  SWR setelahnya, tanpa spinner-blank-load
- messages: guild/channel/tab/selected dibaca dari URL di server, page awal
  di-fetch server-side

Shared realtime state (voice) server-authoritative:
- backend src/modules/voice/live-speaker.ts: agregat voice_active_user dari
  gateway jadi snapshot authoritatif (single source of truth semua browser)
- GET /api/voice/status kini include activeSpeakers
- WS initial states kirim voice_state snapshot saat connect (late join
  langsung dapat state yang sama, bukan daftar kosong)
- useSpeakers seed dari server snapshot + voice_state full-replace +
  voice_active_user delta upsert

Deploy:
- flake.nix: frontend package build SSR standalone (server.js wrapper,
  GMW_FRONTEND_PORT=4017); proxy nginx template proxy / -> Next server,
  /api + /ws tetap ke backend :4001
2026-08-07 10:44:03 +07:00
asepharyana aa440eda69 fix(gateway): music/screen playback heads — read yt-dlp headers from stderr, drop no-simulate
Music playback produced no audio: with `-o -` yt-dlp streams media on
stdout and emits its `--print` title/duration headers on stderr, but
resolveMediaUrl read them from stdout — stripping two binary 'lines' off
the WebM container and corrupting the stream (player 'playing' but silent).
Now headers are read from stderr and the stdout media stream is returned
untouched.

Screenshare was failing with EACCES: getDirectScreenInput used --no-simulate,
making yt-dlp write .f*.part files into the read-only Nix store CWD. Dropped
it — simulate mode still returns requested_formats[].url in the JSON.

Adds tests/mediaResolve.test.ts (stderr-header + untouched-stream regression).
2026-08-06 21:26:50 +07:00
asepharyana f251e69f51 fix(backend): force-exit failsafe so shutdown never hangs
shutdown() awaited httpServer.close(), which waits for ALL open
connections. A lingering WS/keep-alive socket left the process zombie
forever after an uncaughtException (e.g. pg 'Connection terminated
unexpectedly' to imrnes) — no exit, so systemd Restart=always could
never revive it; /api/guilds returned 502 until manual restart.

Add 10s force-exit timer in shutdown(); clear it on clean completion.
2026-08-05 23:58:30 +07:00
asepharyana 9a2fa999bf fix(voice): proper shadcn select dropdowns + top reactors leaderboard
- ui/select: trigger default w-full h-9 (was w-fit h-8 — selects rendered
  tiny/misaligned); callers keep size override via className
- VoiceConnectionCard: labeled full-width h-10 selects (Server/Guild +
  Voice Channel), guild icon + name in options, channel type icon +
  'no akses' tag, empty states, htmlFor/id a11y wiring
- GuildSelector sidebar + messages channel filter bumped to match
- Backend GET /api/dashboard/reactors: top users by net reactions given
  (adds-removes) + messages_reacted + emojis_used
- Reactions tab: second 'Top reaktor' leaderboard panel
2026-08-05 11:29:56 +07:00
asepharyana f999be4fa0 feat(dashboard): add moderation log page + message edit history
- Backend moderation module: GET /api/moderation/stats (per-status + failed rate)
  + GET /api/moderation/actions (filter by status/actionType, cursor paging),
  joins messages for target username + content
- Message GET /api/messages/detail/:id now returns edit_count + edit_history
  (old_content snapshots from message_edits, newest first)
- FE: new /moderation page — summary cards (total/executed/failed/pending +
  failed-rate), status+type filter chips, timeline rows with action icon,
  target user, reason, status badge, timestamps, error text
- FE: message detail shows 'Riwayat edit' panel with previous versions
2026-08-05 11:11:10 +07:00
asepharyana a309570d29 feat(dashboard): expose trust reputation + reactions leaderboard
- GET /api/dashboard/reactions: top reacted messages (net add-remove),
  joined with message content, channel name, top 3 emoji breakdown
- Users tab: colored trust tier badge (Trusted/Netral/At Risk/Kritis)
  in list rows + detail panel (was plain number badge)
- Dashboard: new Reactions sub-tab with leaderboard
  (rank, emoji cluster, message, author, channel, count)
2026-08-05 10:20:12 +07:00
asepharyana 2f51f94610 fix(moderation): remove manual reanalyze triggers — auto-recovery only
Manual per-message and batch reanalyze buttons/endpoints let anyone
re-queue arbitrary messages for LLM analysis, burning AI credits on
spam. Removed:
- FE: Reanalyze buttons in message list, search panel, and messages page
- FE: useReanalyze/useReanalyzeBatch hooks + messagesApi methods
- BE: POST /api/messages/:id/reanalyze and /reanalyze-batch endpoints
- BE: markForReanalysis/reanalyzeErrorBatch service+repository methods

Recovery of failed messages is fully automatic: the discord-gateway
startPendingAIAnalysisWorker retries 'pending' (batch path) and
'error/analysis_incomplete' (individual path) messages on
AI_ANALYSIS_RECOVERY_INTERVAL_MS.
2026-08-04 15:37:56 +07:00
asepharyana 88484f12a9 ci: add Nix GC cleanup job on VPS after deploy 2026-08-04 13:57:44 +07:00
asepharyana a2542493cd fix(voice): screen share now carries audio — merge DASH video+audio into single NUT input
getDirectVideoUrl used yt-dlp --get-url with bestvideo+bestaudio, which
prints the video-only and audio-only URLs on SEPARATE lines. Only the
first (video-only) line was used, so ffmpeg had no audio track and the
GoLive stream had no sound.

Replace with getDirectScreenInput which:
- uses --dump-single-json to fetch both fresh URLs in ONE yt-dlp run
  (signature URLs expire quickly)
- returns the merged progressive URL directly when one exists
- otherwise merges the video-only + audio-only DASH URLs locally via a
  child ffmpeg into a single NUT stream consumed as a Readable
- tracks the merge ffmpeg process in cleanup() so shutdown kills it too

Verified end-to-end with real YouTube URLs: yt-dlp pair → live ffmpeg
merge (NUT) → H264+opus transcode yields both streams. Added
tests/screenShareInput.test.ts covering URL / DASH-pair / error paths.
2026-08-04 13:28:55 +07:00
aseph f84bf723c5 ci: use free GHA Nix cache (disable FlakeHub cache, not subscribed) 2026-08-03 16:44:06 +07:00
asepharyana 24db0f19b1 ci: enable FlakeHub Cache (id-token: write + use-flakehub) 2026-08-03 16:20:15 +07:00
asepharyana ce5db6aa3c fix(media): normalize activeMode in backend MediaState (FE relies on it)
Gateway publishes {playing, activeMode, musicVolume, current, queue} but the
backend MediaState interface + normalizeMediaState dropped activeMode, so the
FE's 'Screen share active' / 'Music playing' badge never rendered. Carry it
through so the FE↔BE media contract stays in sync.
2026-08-03 10:09:19 +07:00
asepharyana 44a0358b0c fix(voice): screen share restore is best-effort — Discord session teardown race
GoLive (dank074 Streamer) needs the single voice session; after the stream
ends, an automatic @discordjs/voice reconnect often races Discord's session
teardown and times out (AbortError). Restore is now best-effort with a 5s
delay; if it fails the FE shows disconnected and the user clicks Connect —
an accepted tradeoff for one-voice-session-per-user.
2026-08-03 09:33:29 +07:00
asepharyana d36c8777fe fix(voice): delay voice restore after screen share — avoid session teardown race
Immediate reconnect after Streamer.stop() races Discord's voice session
teardown → AbortError. Wait 4s so the old session is fully released before
re-joining with @discordjs/voice.
2026-08-03 09:21:10 +07:00
asepharyana 9fd4ded9c8 fix(voice): restore voice after screen share — pass pre-release status to callbacks
The restore callback previously read getVoiceStatus() AFTER disconnectGuild
had already cleared it, so it never knew which guild/channel to reconnect.
Now release/restore receive the status captured BEFORE the audio connection
is released, so reconnect actually happens after the GoLive stream ends.
2026-08-03 09:07:57 +07:00
asepharyana 55d28dc928 style(voice): biome format media handler + screen controller 2026-08-03 08:49:59 +07:00
asepharyana 02e2243a98 fix(voice): screen share releases audio connection so Streamer owns voice session
The dank074 Streamer creates its own WebRTC voice connection, but Discord
allows only ONE voice session per user. When VoiceController (audio) was
already connected, the Streamer join hung forever (never got
VOICE_SERVER_UPDATE). Now:

1. ScreenShareController takes releaseVoice/restoreVoice callbacks.
2. Before joining, it disconnects the @discordjs audio connection via
   VoiceController.disconnectGuild.
3. Streamer joins + streams GoLive.
4. After the stream ends, restoreVoice reconnects the audio connection so
   mic/listen keep working.
5. media.handler wires these via a new setVoiceController accessor from
   commandHandler; VoiceController is the single source of truth.

Also adds caller-bound timeouts & safe .catch() everywhere so a stream
failure can never become an unhandledRejection again.
2026-08-03 08:41:00 +07:00
asepharyana 8528f2c73d fix(voice): screen share join timeout — Streamer join hangs with dual voice conn 2026-08-03 08:19:05 +07:00
asepharyana 53f26185bc fix(voice): screen share crashed gateway — Streamer never joined voice + unhandledRejection
Root cause: ScreenShareController created @dank074 Streamer but never called
streamer.joinVoiceChannel() — playStream threw 'Bot is not connected to a
voice channel', and since the code only used .finally() (no .catch), the
rejection became an unhandledRejection that took down the whole gateway
(graceful shutdown triggered, systemd restarted).

Fixes:
1. Resolve active channel + streamer.joinVoiceChannel(channel) before
   prepareStream/playStream (dank074 needs its OWN WebRTC voice connection).
2. .catch() on the playStream done promise — log + kill ffmpeg instead of
   crashing the process.
3. .catch() on playback.done in media.handler too.
4. stop() now kills ffmpeg AND stops the streamer's voice connection.
2026-08-03 08:05:02 +07:00
asepharyana a57eeb2e22 fix(voice): media queue field mismatch, voice joinable filter, connect error toast
Audit voice (kirim/terima/music/screenshare) menemukan 3 masalah:
1. media:queue SILENT no-op — backend publish {source,mode} tapi gateway
   handler baca payload.url → selalu 'received without a URL'. Backend
   sekarang kirim {url,mode}, gateway terima url ATAU source (robust).
2. Voice connect gagal diam-diam saat user pilih channel tanpa permission
   (joinable=false, contoh Music 32/64/128/256k). Backend+gateway sekarang
   expose joinable; FE disable channel 'no akses' + empty state.
3. FE tidak kasih feedback saat connect gagal — tambah toast.error dengan
   pesan dari backend.

Verified live: @discordjs/voice connect ke Lofi Radio joinable sukses
(VOICE READY, DAVE session OK) — pipeline voice sebenarnya sehat, masalah
utama UX. media:queue fix akan di-verify setelah deploy.
2026-08-03 07:48:50 +07:00
asepharyana 9abb09dd33 feat(frontend): add light mode with runtime theme toggle
- globals.css: split theme tokens into :root (light default) + .dark
  overrides; switch @theme inline → @theme so utilities reference
  var(--color-*) and a runtime class swap actually re-skins the UI
  (inline inlines literal values and ignores .dark overrides)
- layout.tsx: drop the beforeInteractive inline theme script (it caused
  hydration instability); theme is applied client-side only
- top-nav: apply persisted theme on mount, default light, toggle
  updates <html> class + localStorage
- glass-intense: light variant (white card on light canvas); chart grid
  line uses var(--color-border); attachment chip uses bg-glass-bg
- Verified in static export: toggle dark↔light both directions,
  chatbot panel renders clean in both themes
2026-08-03 07:20:09 +07:00
asepharyana 831254bb71 fix(nix): filter build artifacts from frontend source
path: literals in flakes do NOT respect .gitignore, so a dirty local
out/ (stale chunks from previous builds, e.g. 3y39nidcm2n_s.js from
the removed quick-prompt button) leaked into the sandbox and got
served forever. Add filterSource helper that excludes out, .next,
node_modules, pnpm-lock.yaml from the frontend derivation source.
2026-08-03 06:53:46 +07:00
asepharyana 9718940258 fix(chatbot): FAB opens chat directly — remove extra toggle + UX polish
- chatbot-container: remove chatOpen layer — the minimized bubble now
  expands straight into the chat panel (single click, no extra button)
- Remove the redundant 'Tanya soal server...' quick-prompt button and
  the PanelLeft collapse toggle (one less state to fight)
- Bubble fixed h-[460px], chat panel flex-fills remaining space
- chat-panel: suggestion chips on empty state (suasana server, channel
  paling ramai, total pesan, pesan bermasalah) so first-time users can
  start with a single click
- Input: roomier padding, more descriptive placeholder, bigger send
  button, autoComplete off
- Drop chatOpen/setChatOpen from context (dead state)
2026-08-03 06:44:23 +07:00
asepharyana d1c1f3e4a7 feat(chatbot): per-user history via X-User-Id + agentic tools calling
Backend:
- New chatbot.tools.ts: 4 tools (get_server_stats, get_top_channels,
  get_recent_activity, get_top_flagged) with real DB executors
- chatbot.service: agentic loop — stream:true, parse SSE, execute
  tool_calls, feed results back, up to 4 rounds
- controller: resolve userId from X-User-Id header (no-login device
  uuid) with auth middleware precedence; history/clear scoped per user

Frontend:
- use-chatbot-user: mint UUID in localStorage, send as X-User-Id
- chatbotApi.send/getHistory/clearHistory accept userId header
- client.ts: apiRequest supports custom headers per call
- provider: history load + send + clear keyed to device user id
2026-08-03 06:24:19 +07:00
asepharyana 7513681b4b feat(frontend): remove ugly canvas placeholder from chatbot
- Delete chatbot-canvas.tsx (placeholder face canvas) + its export
- Remove canvas area from chatbot container; chat panel gets the freed
  space (248px → 300px) and bubble height drops 440px → 400px
2026-08-03 06:11:06 +07:00
asepharyana 2b815e156c feat(frontend): rebuild chatbot UI to match backend — wider panel, guild context, Indonesian
- Chatbot bubble: 220px → 320px wide, 440px tall; proper header with
  drag handle; quick-prompt row when chat is closed
- ChatPanel: show ALL history (not last 8), Indonesian placeholder/empty
  state/error copy (backend speaks Indonesian), timestamps (id-ID),
  clear-history button, typing indicator bubbles, Enter-to-send
- Send active guildId as context so backend answers reference the real
  server (serverInsights path in chatbot.service)
- GuildId sync from layout → ChatbotProvider via ChatbotGuildSync
- chatbotApi.send(message, guildId) → POST /api/chat {message, context}
  matching BE zod schema (guildId optional)
2026-08-03 06:04:25 +07:00
asepharyana 03d59f0738 feat(frontend): lazy-load images, add lightbox viewer, polish AI panel
- Add loading=lazy + decoding=async to all <img> (message card preview,
  attachments grid, image grid, avatars via ui/avatar)
- New Lightbox component: fullscreen image viewer with keyboard nav
  (←/→/Esc), counter, click-to-close; wired into messages page + detail
- Attachments grid: click image to open, image counter badge, grouped
  non-image attachments
- AI analysis panel: line-clamp-3 with Show more/less toggle
- Message card: preview image is now a clickable button opening detail
2026-08-03 05:08:08 +07:00
asepharyana 5cc0f8a243 docs: frontend README dev note 2026-08-02 16:46:22 +07:00
asepharyana 12c55ef486 docs: sync remaining md (AGENTS, ARCHITECTURE, READMEs to 4001/4009, Nix) 2026-08-02 16:42:57 +07:00
asepharyana 38c27eb5bb chore: ARCHITECTURE.md DB pool example 2026-08-02 16:23:01 +07:00
asepharyana bd044e95c3 chore: env.test.example and ARCHITECTURE pool 6432 2026-08-02 16:22:48 +07:00
asepharyana ec64a078bf chore: fix-missing-tables.sql imrnes IP 2026-08-02 16:22:30 +07:00
asepharyana 37defa5915 chore: Dockerfile.backend expose port 4001 2026-08-02 16:21:48 +07:00
asepharyana 39421c39cb chore: sync port references and docs to 4000s infra 2026-08-02 16:20:27 +07:00
asepharyana 1d27f67788 chore: update gmw-proxy nginx template ports to 4009/4001 2026-08-02 15:16:16 +07:00
asepharyana d1e6f3b47a chore: update ports to 4000-range (4000/4001) 2026-08-02 14:30:54 +07:00
asepharyana dbcf9d68f2 fix(media): publish status when a track ends naturally
The Redis media:status key was only rewritten after a command received via
Redis. When the last track ended naturally (AudioPlayer Idle -> advanceQueue
with an empty queue), currentTrackItem was cleared but the status key was not
persisted — so the backend's cached status and the frontend's 10s polling
stayed stuck showing the finished track as 'playing' forever.

Wire a media-status sink (commandHandler provides the real redisPub to
MediaHandler) and re-publish status after auto-advance, so natural track end
updates the UI.
2026-08-02 10:43:34 +07:00
asepharyana ef4281cd1f fix(voice): activity tab rendered a permanently empty chart
VoiceActivityTimeline was never given a data prop — the Activity tab always
showed an empty Recharts bar chart while the connection tab already had live
speaker state. Replace the dead chart with a live speaker/activity list fed
from the same WebSocket data, so the tab reflects real state instead of
misleading empty bars.
2026-08-02 10:38:02 +07:00
asepharyana 25f6609a9f fix(recordings): stop faking duration from file size + render edited content in search
The voice_recordings table has no duration column, but the backend aliased
duration_bytes = size_bytes (file size in bytes) and RecordingCard divided it
by 60 as if it were seconds — a 3MB MP3 rendered as a nonsensical '55924:3'
fake timestamp. Drop the fabricated field and show real file size instead.

Also render edited_content fallback in the analysis search results for
consistency with message cards/detail.
2026-08-02 10:32:58 +07:00
asepharyana 3f199aa70d fix(messages): merge partial WS updates + display edited content
message_updated broadcasts only {id, edited_content, edited_at} (+ reset
ai_* fields), but the frontend replaced the whole cached record, wiping
username/content/channel_id/created_at -> blank cards and the
'the channel_id of undefined' crash on /messages. Merge partials over the
existing record (list + detail), make list-patching channel-filter aware,
show edited content/badge, and fix the message_updated WS type.

Also broadcast type:'edited' + ai reset in message_updated so the live UI
matches the DB update.
2026-08-02 10:27:41 +07:00
asepharyana a82265f4a9 chore: remove outdated README.md file 2026-08-02 10:18:56 +07:00
259 changed files with 10626 additions and 12359 deletions
+5 -5
View File
@@ -39,7 +39,7 @@ AUDIO_CHANNELS=2 # Number of audio channels (default: 2)
AVATAR_SIZE=64 # User avatar size in pixels (default: 64) AVATAR_SIZE=64 # User avatar size in pixels (default: 64)
# === Webserver === # === Webserver ===
WEBSERVER_PORT=3001 # Backend HTTP/WS server port (default: 3001) WEBSERVER_PORT=4001 # Backend HTTP/WS server port (default: 4001)
# === Connection === # === Connection ===
VOICE_CONNECTION_TIMEOUT_MS=15000 # Voice connection timeout in ms (default: 15000) VOICE_CONNECTION_TIMEOUT_MS=15000 # Voice connection timeout in ms (default: 15000)
@@ -55,7 +55,7 @@ VERBOSE=false # Enable verbose/debug logging (default:
# === Database (PostgreSQL) === # === Database (PostgreSQL) ===
# Option 1: Connection string (overrides individual params) # Option 1: Connection string (overrides individual params)
# DATABASE_URL=postgresql://user:password@localhost:5432/discord_bot DATABASE_URL=postgresql://asephs:***@100.121.180.82:6432/dcbot
# Option 2: Individual connection parameters # Option 2: Individual connection parameters
POSTGRES_HOST=localhost # PostgreSQL host (default: localhost) POSTGRES_HOST=localhost # PostgreSQL host (default: localhost)
@@ -67,11 +67,11 @@ POSTGRES_POOL_MIN=2 # Minimum pool connections (default: 2)
POSTGRES_POOL_MAX=10 # Maximum pool connections (default: 10) POSTGRES_POOL_MAX=10 # Maximum pool connections (default: 10)
# === Redis === # === Redis ===
REDIS_URL=redis://localhost:6379 # Redis connection string (default: redis://localhost:6379) REDIS_URL=redis://100.121.180.82:6379 # Redis connection string (default: redis://localhost:6379)
# === Voice PCM WebSocket (direct gateway→backend, bypasses Redis) === # === Voice PCM WebSocket (direct gateway→backend, bypasses Redis) ===
VOICE_PCM_WS_ENABLED=true # Use direct WS for PCM audio (default: true) VOICE_PCM_WS_ENABLED=true # Use direct WS for PCM audio (default: true)
BACKEND_WS_URL=ws://backend:3000/ws # Backend WebSocket URL for gateway PCM streaming BACKEND_WS_URL=ws://backend:4001/ws # Backend WebSocket URL for gateway PCM streaming
BACKEND_WS_TOKEN= # REQUIRED if VOICE_PCM_WS_ENABLED=true. Internal shared secret BACKEND_WS_TOKEN= # REQUIRED if VOICE_PCM_WS_ENABLED=true. Internal shared secret
# === Attachments === # === Attachments ===
@@ -90,7 +90,7 @@ AI_LLM_MODEL=text # LLM text model name (default: text)
# AI_LLM_VISION_MODEL= # Vision model for image analysis (falls back to AI_LLM_MODEL) # AI_LLM_VISION_MODEL= # Vision model for image analysis (falls back to AI_LLM_MODEL)
# AI_LLM_EMBEDDING_MODEL= # Embedding model for semantic moderation cache (optional; enables near-duplicate text reuse to save LLM calls) # AI_LLM_EMBEDDING_MODEL= # Embedding model for semantic moderation cache (optional; enables near-duplicate text reuse to save LLM calls)
# AI_LLM_EMBEDDING_MIN_SIMILARITY=0.97 # Min cosine similarity to reuse a cached verdict (default: 0.97) # AI_LLM_EMBEDDING_MIN_SIMILARITY=0.97 # Min cosine similarity to reuse a cached verdict (default: 0.97)
# QDRANT_URL=http://100.121.180.82:6333/ # Qdrant vector store for embeddings (semantic cache); when set, vectors are stored/searched in Qdrant instead of Postgres QDRANT_URL=http://100.121.180.82:6333 # Qdrant vector store for embeddings (semantic cache); when set, vectors are stored/searched in Qdrant instead of Postgres
# QDRANT_COLLECTION=gmw_text_moderation # Qdrant collection name (default: gmw_text_moderation) # QDRANT_COLLECTION=gmw_text_moderation # Qdrant collection name (default: gmw_text_moderation)
# QDRANT_API_KEY= # Qdrant API key (optional) # QDRANT_API_KEY= # Qdrant API key (optional)
AI_LLM_MAX_CONCURRENT=5 # Max concurrent LLM API calls (default: 5) AI_LLM_MAX_CONCURRENT=5 # Max concurrent LLM API calls (default: 5)
+2 -2
View File
@@ -2,5 +2,5 @@ NODE_ENV=test
# Use a separate database/data area for tests. It may be on the same PostgreSQL host, # Use a separate database/data area for tests. It may be on the same PostgreSQL host,
# but the database name must clearly be a test database so destructive test setup # but the database name must clearly be a test database so destructive test setup
# cannot touch production data. # cannot touch production data.
TEST_DATABASE_URL=postgres://root:root@100.108.1.124:5432/hub_test TEST_DATABASE_URL=postgres://root:root@100.121.180.82:6432/hub_test
DATABASE_URL=postgres://root:root@100.108.1.124:5432/hub_test DATABASE_URL=postgres://root:root@100.121.180.82:6432/hub_test
+143 -3
View File
@@ -11,6 +11,7 @@ concurrency:
permissions: permissions:
contents: read contents: read
id-token: write
env: env:
VPS_HOST: ${{ secrets.VPS_HOST }} VPS_HOST: ${{ secrets.VPS_HOST }}
@@ -66,7 +67,7 @@ jobs:
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
service: [backend, discord-gateway, proxy] service: [backend, discord-gateway, proxy, frontend]
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v7 uses: actions/checkout@v7
@@ -81,9 +82,24 @@ jobs:
extra-conf: | extra-conf: |
sandbox = false sandbox = false
accept-flake-config = true accept-flake-config = true
# Attic binary cache as substituter on the runner: lets CI pull the
# prebuilt attic client (and any cached deps/builds) over HTTPS,
# no SSH round-trip needed. extra-substituters (NOT
# extra-trusted-substituters) is required — Determinate Nix never
# merges trusted-* substituters for nix-store CLI clients.
extra-substituters = https://attic.asepharyana.my.id/gmw
extra-trusted-public-keys = gmw:Fq2Anzuhkb+T/hftWnPcveHSi21/RzIgIOeG8pCJa88=
# NOTE: nix-installer-action unconditionally injects
# 'build-provenance-tags' into /etc/nix/nix.conf (a Determinate
# Nix-only setting). With determinate:false the runner's upstream
# nix warns 'unknown setting build-provenance-tags' on every
# invocation — benign, cosmetic. Switching determinate:true would
# silence it but changes the runner's nix flavor.
- name: Cache Nix - name: Cache Nix
uses: DeterminateSystems/magic-nix-cache-action@v14 uses: DeterminateSystems/magic-nix-cache-action@v14
with:
use-flakehub: false
- name: Build ${{ matrix.service }} - name: Build ${{ matrix.service }}
id: build id: build
@@ -104,17 +120,141 @@ jobs:
ssh-keygen -y -f ~/.ssh/id_ed25519 >/dev/null 2>&1 || { echo "SSH key invalid"; exit 1; } ssh-keygen -y -f ~/.ssh/id_ed25519 >/dev/null 2>&1 || { echo "SSH key invalid"; exit 1; }
ssh-keyscan -H "$VPS_HOST" >> ~/.ssh/known_hosts 2>/dev/null ssh-keyscan -H "$VPS_HOST" >> ~/.ssh/known_hosts 2>/dev/null
# Push build result to Attic binary cache (attic.asepharyana.my.id) so
# the VPS can substitute it instead of a single-stream `nix copy ssh://`.
#
# Fast path: push DIRECTLY from the runner to the public attic endpoint
# (validated 2026-08-10: token auth over public HTTPS works without
# Tailscale). This skips the ~794MB closure SSH copy to the VPS that
# used to take 25+ minutes per new store path.
#
# The attic client is NOT in nixpkgs anymore and has no prebuilt
# releases, so we pull the same prebuilt closure the VPS uses
# (/nix/store/fygyy3yk4rqdknxkiwkqambpnhyax0k4-attic-0.1.0, ~52MB).
# The closure itself lives in the attic cache (pushed once from the
# VPS), so the runner bootstraps it over HTTPS via the configured
# extra-substituters — no SSH round-trip. If that fails we fall back
# to `nix copy --from ssh://`, then the old VPS-hop flow (SSH copy to
# VPS, then attic push from the VPS over Tailscale) so the deploy step
# always has a working closure path.
- name: Push to Attic cache
env:
ATTIC_TOKEN: ${{ secrets.ATTIC_TOKEN }}
run: |
if [ -z "$ATTIC_TOKEN" ]; then
echo "ATTIC_TOKEN not set; skipping attic push"
exit 0
fi
STORE_PATH="${{ steps.build.outputs.store-path }}"
ATTIC_DIR="/nix/store/fygyy3yk4rqdknxkiwkqambpnhyax0k4-attic-0.1.0"
ATTIC_BIN="$ATTIC_DIR/bin/attic"
attic_push_vps_hop() {
echo "Fallback: VPS-hop attic push"
# Copy closure to VPS (fast if attic already has it via substitute)
ssh "$VPS_USER@$VPS_HOST" "sudo /nix/var/nix/profiles/default/bin/nix-store --realise '$STORE_PATH'" 2>/dev/null \
|| nix copy --to "ssh://$VPS_USER@$VPS_HOST" "$STORE_PATH"
# Push from VPS → Attic over Tailscale.
# --ignore-upstream-cache-filter is REQUIRED: without it, attic skips
# writing the narinfo to gmw when chunks exist in the upstream
# cache.nixos.org — leaving the path 404 on gmw so the VPS deploy's
# nix-store --realise can't find it and falls back to ssh copy.
# sudo: attic must read root's config (~/.config/attic), which has
# the imrnes-ts server → Tailscale. Non-root users' configs only
# have the public `pub` server → "Server imrnes-ts does not exist".
ssh "$VPS_USER@$VPS_HOST" "sudo $ATTIC_BIN push imrnes-ts:gmw '$STORE_PATH' --jobs 4 --ignore-upstream-cache-filter" \
|| echo "attic push failed (non-fatal; ssh copy fallback below)"
}
# ── Get an attic client on the runner ────────────────────────────
# Order: PATH → pull the prebuilt closure from the attic cache
# itself (extra-substituters configured in Install Nix step, HTTPS
# only, no SSH) → pull over ssh from the VPS → VPS-hop.
# The attic client closure is stored in the attic cache (pushed
# once from the VPS), so the fast path never depends on SSH.
ATTIC_BIN=""
if command -v attic >/dev/null 2>&1; then
ATTIC_BIN="$(command -v attic)"
elif nix-store --realise "$ATTIC_DIR" 2>/tmp/attic-bootstrap.err; then
echo "✅ Pulled attic client from attic cache (HTTPS substituter)"
ATTIC_BIN="$ATTIC_DIR/bin/attic"
elif nix copy --from "ssh://$VPS_USER@$VPS_HOST" "$ATTIC_DIR" 2>>/tmp/attic-bootstrap.err; then
echo "✅ Pulled attic client from VPS over ssh"
ATTIC_BIN="$ATTIC_DIR/bin/attic"
else
echo "attic client unavailable on runner; using VPS-hop flow"
echo "--- bootstrap errors (stderr) ---"
tail -5 /tmp/attic-bootstrap.err 2>/dev/null || true
attic_push_vps_hop
exit 0
fi
# ── Direct push: runner → attic public endpoint ──────────────────
# --ignore-upstream-cache-filter forces the narinfo write even when
# the path's chunks already exist in upstream cache.nixos.org (which
# attic would otherwise skip, leaving the path 404 on the gmw cache).
mkdir -p "$HOME/.config/attic"
cat > "$HOME/.config/attic/config.toml" <<EOF
default-server = "pub"
[servers.pub]
endpoint = "https://attic.asepharyana.my.id"
token = "$ATTIC_TOKEN"
EOF
# Retry the direct push — a transient 502 (e.g. atticd restart,
# Traefik blip) must not abort the whole closure upload. attic push
# is idempotent, so re-running only uploads what's still missing.
push_ok=""
for attempt in 1 2 3; do
if "$ATTIC_BIN" push pub:gmw "$STORE_PATH" --jobs 4 --ignore-upstream-cache-filter; then
echo "✅ Pushed $STORE_PATH to attic directly from runner"
push_ok=1
break
fi
echo "⚠️ Direct attic push attempt $attempt/3 failed; retrying in 10s..."
sleep 10
done
if [ -z "$push_ok" ]; then
echo "Direct attic push failed after 3 attempts; using VPS-hop flow"
attic_push_vps_hop
fi
# NOTE: env files /etc/gmw/backend.env & /etc/gmw/discord-gateway.env are # NOTE: env files /etc/gmw/backend.env & /etc/gmw/discord-gateway.env are
# managed MANUALLY on the VPS (source of truth). CI only builds & deploys. # managed MANUALLY on the VPS (source of truth). CI only builds & deploys.
- name: Deploy ${{ matrix.service }} to VPS - name: Deploy ${{ matrix.service }} to VPS
run: | run: |
STORE_PATH="${{ steps.build.outputs.store-path }}" STORE_PATH="${{ steps.build.outputs.store-path }}"
echo "=== Copying ${{ matrix.service }}: $STORE_PATH ===" echo "=== Copying ${{ matrix.service }}: $STORE_PATH ==="
if [ -n "${{ secrets.ATTIC_TOKEN }}" ] && ssh "$VPS_USER@$VPS_HOST" "sudo /nix/var/nix/profiles/default/bin/nix-store --realise '$STORE_PATH'" 2>/dev/null; then
echo "Substituted ${{ matrix.service }} from Attic cache"
else
echo "Attic substitute failed; falling back to ssh copy"
nix copy --to "ssh://$VPS_USER@$VPS_HOST" "$STORE_PATH" nix copy --to "ssh://$VPS_USER@$VPS_HOST" "$STORE_PATH"
fi
echo "=== Updating profile ===" echo "=== Updating profile ==="
ssh "$VPS_USER@$VPS_HOST" "sudo /nix/var/nix/profiles/default/bin/nix-env --profile /nix/var/nix/profiles/gmw-${{ matrix.service }} --set '$STORE_PATH'" ssh "$VPS_USER@$VPS_HOST" "sudo /nix/var/nix/profiles/default/bin/nix-env --profile /nix/var/nix/profiles/gmw-${{ matrix.service }} --set '$STORE_PATH'"
echo "=== Restarting service ===" echo "=== Restarting service ===\n"
ssh "$VPS_USER@$VPS_HOST" "sudo systemctl daemon-reload && sudo systemctl restart gmw-${{ matrix.service }} && sleep 3 && sudo systemctl is-active gmw-${{ matrix.service }}" ssh "$VPS_USER@$VPS_HOST" \
"sudo systemctl daemon-reload && sudo systemctl restart gmw-${{ matrix.service }} && for i in \$(seq 1 15); do state=\$(sudo systemctl is-active gmw-${{ matrix.service }} 2>/dev/null || echo inactive); [ \"\$state\" = \"active\" ] && break; sleep 2; done; echo \"final-state=\$state\"; [ \"\$state\" = \"active\" ]"
echo "✅ gmw-${{ matrix.service }} deployed" echo "✅ gmw-${{ matrix.service }} deployed"
cleanup:
# Bersihkan sampah Nix di VPS SETELAH semua deploy selesai: hapus generasi
# profile lama + nix store gc. Profil yang sedang dipakai tidak disentuh.
needs: build-and-deploy
if: always()
runs-on: ubuntu-latest
steps:
- name: Nix GC on VPS
env:
VPS_HOST: ${{ secrets.VPS_HOST }}
VPS_USER: ${{ secrets.VPS_USER }}
SSH_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
run: |
mkdir -p ~/.ssh
echo "$SSH_KEY" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keyscan -H "$VPS_HOST" >> ~/.ssh/known_hosts 2>/dev/null
ssh "$VPS_USER@$VPS_HOST" "sudo /usr/local/bin/nix-gc-vps.sh" || echo "⚠️ Nix GC gagal (non-fatal)"
+1 -1
View File
@@ -12,7 +12,7 @@ worktrees/
.worktrees/ .worktrees/
services/frontend/frontend/dist/ services/frontend/frontend/dist/
target/ target/
nix/
# Gitea CI runner logs # Gitea CI runner logs
.gitea/workflows/*.log .gitea/workflows/*.log
@@ -0,0 +1,418 @@
# GMW Frontend — Greenfield Rebuild (Visual-System Overhaul + Custom UI + Motion/3D)
> **For Hermes:** Execute with `subagent-driven-development` (one fresh subagent per task, two-stage review). Each task is 13 min, atomic, independently verifiable, committed after each. Reuse `src/lib/api/*`, `src/lib/ws/*`, `src/lib/types/*`, hooks verbatim. Never invent endpoints.
**Goal:** Rebuild the GMW Discord-automod dashboard frontend from scratch — drop shadcn/ui + glass/teal/purple aesthetic entirely, replace with a custom, distinctive design system where every page has its own visual metaphor (no uniform bordered-card grid), and push the presentation layer with **Framer Motion (`motion`) choreography + signature Three.js scenes**. Re-integrate to the EXISTING backend API + WebSocket contract (do NOT touch the backend).
**Architecture:** Next.js 16 App Router (SSR) + React 19 + TS strict + Tailwind v4. Keep the *plumbing* (data contract), rebuild the *skin + primitives + motion*. Each route = one self-contained `page.tsx` (server fetch + client view in the same file via a `"use client"` sibling export). Custom SVG charts (no recharts). Custom micro-primitives (no shadcn/base-ui). New token system in `globals.css`. **Motion:** `motion/react` app-wide for page transitions, spring micro-interactions, layout animation. **3D:** raw `three` (no react-three-fiber — leaner) in exactly TWO signature scenes, lazy-loaded client-only with graceful fallback. Deployed unchanged via existing flake + `gmw-proxy` nginx (`:4009` → Next `:4017`).
**Tech Stack:** next@16, react@19, tailwindcss@4 (`@import "tailwindcss"`), `next/font/google` (Bricolage Grotesque + Inter + JetBrains Mono), `swr` (data revalidation), `lucide-react` (icons only), `motion` (Framer Motion successor — `motion/react`), `three` + `@types/three` (signature scenes only), `clsx` + `tailwind-merge`. **Removed:** `@shadcn/react`, `@base-ui/react`, `recharts`, `shadcn` CLI, `cmdk`, `sonner`, `react-day-picker`, `embla-carousel-react`, `react-resizable-panels`, `input-otp`, all 50 `components/ui/*`.
---
## 0. Design System (the creative core — read before coding)
**Persona:** Controlled UX Designer + a tactical "ops console" voice. Material honesty: hierarchy via **scale/weight/tonal blocks**, NOT borders/shadows. Per `frontend-design` skill principle — spend the boldness in ONE signature place per page, keep the rest disciplined. Motion is choreography, not confetti: **one orchestrated moment per page**, everything else quiet.
### Palette (warm, signal-driven — NO teal/cyan/purple/blue gradients)
Light mode (`:root`):
```
--canvas: oklch(0.96 0.012 80) /* warm off-white, not cream */
--surface: oklch(0.92 0.014 80) /* tonal block, replaces bordered card */
--surface-2: oklch(0.88 0.016 80)
--ink: oklch(0.22 0.02 70) /* primary text */
--ink-soft: oklch(0.46 0.02 70) /* secondary text */
--hairline: oklch(0.22 0.02 70 / 0.10) /* structural rules ONLY, sparse */
--signal: oklch(0.78 0.17 125) /* lime — OK / live / primary accent */
--signal-ink: oklch(0.20 0.03 70) /* text ON signal */
--amber: oklch(0.80 0.15 70) /* WARN */
--vermilion: oklch(0.62 0.21 25) /* FLAGGED / destructive */
--ring: var(--signal)
```
Dark mode (`.dark`, default theme per `next-themes`):
```
--canvas: oklch(0.13 0.015 70) /* warm charcoal, not blue-black */
--surface: oklch(0.18 0.02 70)
--surface-2: oklch(0.23 0.022 70)
--ink: oklch(0.93 0.01 75)
--ink-soft: oklch(0.62 0.02 75)
--hairline: oklch(1 0 0 / 0.09)
--signal: oklch(0.88 0.18 125)
--signal-ink: oklch(0.18 0.03 70)
--amber: oklch(0.85 0.15 70)
--vermilion: oklch(0.68 0.21 25)
```
Three semantic signals reused everywhere: **lime = OK/live, amber = warn, vermilion = flag/danger**. This kills the purple-accent + teal-primary monotony.
### Typography (3 roles, deliberate pairing — not "Inter everywhere")
- **Display:** `Bricolage Grotesque` (700800) — characterful grotesque for headers/big numbers.
- **Body/UI:** `Inter` (400600).
- **Data/label:** `JetBrains Mono` (500/700) — all stats, timestamps, channel IDs, metrics.
Load all three via `next/font/google` with CSS variables (keep current `--font-inter`/`--font-jetbrains-mono` names + add `--font-display`).
### Layout & signature
- **No `card` with border.** Use tonal `--surface` blocks with generous radius (`--r: 14px`) and internal padding; separate blocks with whitespace + sparse hairlines only where structurally meaningful.
- **Signature element = "scan-tick":** a 1px animated pulse line (CSS keyframe `scan`) that marks every live/section header — NOT a card outline. Global canvas carries a faint warm dot-grid texture (low opacity) instead of the current bluish dotted radial-gradient.
- **Nav = left "spine":** vertical rail of icon nodes joined by a hairline; active node gets a `--signal` dot + label reveal. Collapses to a bottom tab-bar < 768px (CSS only, no JS sidebar primitive).
- **Header = "status bar":** connection state (WS dot), guild selector, live clock — mono font, reads like an instrument readout.
## 0.1 Motion & 3D Layer (the "lebih kreatif" addition)
### Motion rules (from `motion/react`)
- **Page transitions:** one shared `RouteTransition` in `(dashboard)/layout.tsx``AnimatePresence mode="popLayout"` + `motion.div key={pathname}` (fade + 8px rise + slight blur-out, ~220ms, `easeOut`). Consistent everywhere, zero per-page boilerplate.
- **Enter choreography (per page, ONE signature moment):** staggered rise-in for the hero/ticker group using `staggerChildren` variants; afterwards, quiet springs for hover/tap (`scale: 1.03` on interactive blocks, `whileTap` on buttons).
- **Layout animation:** `layout` prop on list items (messages rows, recording rows, queue) so add/remove/filter reflows smoothly; `layoutId` for shared-element transitions (ticker → detail modal on dashboard).
- **Live pulse:** `motion` drives the severity ticks / speaker rings with springs, not CSS `transition` alone.
- **`useReducedMotion()`** (from `motion/react`) gates ALL heavy motion; CSS `@media (prefers-reduced-motion: reduce)` additionally kills `scan`/`spin-disc` keyframes. Accessibility floor, non-negotiable.
- **No scroll-jacking, no marquee loops, no per-element confetti.** One moment per page. (`frontend-design` skill: "Satu momen orkestrasi biasanya lebih mengena daripada efek tersebar.")
### 3D rules (raw `three`, no R3F — lean bundle)
- Exactly **two** scenes, chosen because they carry real data meaning: **Dashboard hero** (`SignalField` — a particle field whose pulse density reflects live activity) and **Voice page** (`OrbField` — speakers as glowing orbs whose height/ring radius reacts to who is speaking). Everything else stays 2D/motion.
- **Lazy + client-only:** `next/dynamic(() => import("./SignalField"), { ssr: false, loading: () => <StaticFallback/> })`. Three ships in its own chunk, loaded only on those two routes.
- **WebGL guard:** if `!window.WebGLRenderingContext` or context creation fails → render the static SVG/CSS fallback (a stylized 2D version of the same visual). Never blank.
- **Perf guardrails:** `dpr: [1, 1.75]`, `powerPreference: "high-performance"`, `antialias: true`; RAF loop paused on `document.hidden`; `dispose()` geometries/materials on unmount; particle count capped by `navigator.hardwareConcurrency` + viewport (`Math.min(900, w*h/2000)`).
- **Style:** warm palette ONLY — signal-lime particles, amber/vermilion for flag/warn states; fog + soft additive blending for glow (no harsh white lights, no metallic PBR).
- **Interactivity:** subtle pointer parallax (camera lerp toward cursor) + gentle idle rotation. No drag/drop, no raycasting menus.
### Per-page metaphor (kills monotony — each page feels different)
| Route | Metaphor | Signature visual | Motion / 3D |
|---|---|---|---|
| `/dashboard` | Live ops overview | **3D signal particle field** hero + asymmetric ticker blocks + radial moderation gauge | **3D SignalField** (reacts to activity), staggered ticker rise-in, gauge draws on mount |
| `/messages` | Transcript | Left channel **timeline spine**; right = flowing message entries with left **severity tick** (no bordered cards); search = command palette | `layout` on message rows, spring severity ticks, palette types in |
| `/voice` | Stage | **3D orb field** of speakers + equalizer rings; activity = horizontal **session ribbon** | **3D OrbField** (speaking → orb rises + ring pulses), session ribbon draws sequentially |
| `/media` | Turntable | Rotating **disc** now-playing; queue = borderless list | CSS 3D disc spin (spring on play/pause), queue `layout` reflow, progress bar springs |
| `/recordings` | Tape library | Rows with **waveform thumbnail** (custom SVG from duration) | Waveform bars spring on hover; new upload animates in (AnimatePresence) |
| `/moderation` | Security log | Vertical **event flow** with status nodes (dot + line), not a table of cards | Nodes pulse on live action; timeline draws in sequence |
| `/analysis` | Query console | Terminal-style search panel | Typing cursor + results stagger |
---
## 1. What to KEEP (reuse verbatim — correct + integrates to backend)
- `src/lib/api/server.ts` — 11 server fetchers (`getDashboardStats`, `getActivity`, `getMediaStatus`, `getConfig`, `getModerationStats/Actions`, `getGuilds`, `getVoiceStatus`, `getRecordings`, `getMessages`). **No change.**
- `src/lib/api/client.ts``apiRequest` + `ApiError`. **No change.**
- `src/lib/ws/*``connection.ts`, `context.tsx`, `types.ts` (17 typed events: `message_created/updated/deleted/analyzed`, `voice_*`, `media_state`, `voice_pcm_data` binary). **No change.**
- `src/lib/types/*` — all interfaces. **No change.**
- `src/lib/format.ts`, `src/lib/utils.ts` (`cn`). **No change.**
- `src/lib/navigation.ts``navItems`. Keep but extend icons/labels if needed.
- Hooks: `src/hooks/*` (use-dashboard, use-media, use-messages, use-voice, use-moderation, use-recordings, use-guilds, use-config, use-chatbot-user, use-action, use-mobile), `src/lib/hooks/use-mounted.ts`. **Reuse** (verify no bad imports into deleted barrels).
- Feature logic kept but **reskinned**: `src/components/media/music-player.tsx`, `src/components/voice/*`, `src/components/chatbot/*`, `src/components/messages/*`, `src/components/recordings/*`, `src/components/moderation/moderation-section.tsx`, `src/components/analysis/search-panel.tsx`, `src/components/dashboard/*` (charts → rewritten as custom SVG).
## 2. What to DELETE
- `src/components/ui/*` (all 50 shadcn primitives).
- `components.json`, `@shadcn/react` + `@base-ui/react` + `shadcn` deps.
- `recharts` (replace with custom SVG chart helpers in `src/components/charts/`).
- `src/app/globals.css` → rewrite (no `@import "shadcn/tailwind.css"`, no `--color-primary` teal, no `.glass*`, no `.text-gradient`/`.gradient-border` teal/purple, no bluish body texture).
- `src/components/layout/app-sidebar.tsx` (shadcn Sidebar) → replace with custom `Spine` nav.
- All `page.tsx`+`view.tsx` pairs → merge into single `page.tsx` per route.
## 3. What to BUILD (new)
- New `globals.css` (tokens above + utilities + keyframes `scan`, `eq`, `fade-up`, `spin-disc`).
- `src/components/primitives/` — minimal custom: `Button`, `Input`, `Select`, `Dialog`, `Tooltip`, `Badge`, `Progress`, `Avatar`, `Skeleton`, `Toast`, `Sheet`.
- `src/components/motion/``RouteTransition.tsx`, `Stagger.tsx`, `variants.ts`.
- `src/components/three/``SignalField.tsx`, `OrbField.tsx`, `WebGLGuard.tsx`, `StaticFallback.tsx`, `useThreeScene.ts`.
- `src/components/charts/``Sparkline`, `AreaActivity`, `RadialGauge`, `SessionRibbon`, `Waveform`.
- `src/components/layout/``Spine.tsx`, `StatusBar.tsx`, `ThemeToggle.tsx` (reskin).
- 7 merged `page.tsx` files (one per route) implementing the metaphors above.
- `src/app/layout.tsx` — root (fonts + ThemeProvider + Toaster), `src/app/(dashboard)/layout.tsx` — providers + Spine + StatusBar + RouteTransition + MiniPlayer + Chatbot, `src/app/page.tsx` → redirect `/dashboard`.
---
## 4. Target File & Folder Structure (authoritative)
Everything below `services/frontend/src/` is the new tree. `REWRITE` replaces existing; `NEW` creates; `DELETE` removes. The `app/` route tree collapses `page.tsx`+`view.tsx` into single `page.tsx` files containing BOTH server fetch (default export) and client view (`"use client"` named export in same file).
```
services/frontend/
├─ package.json REWRITE (drop shadcn/base-ui/recharts; add motion, three, @types/three)
├─ pnpm-workspace.yaml REWRITE (onlyBuiltDependencies: keep build list minimal)
├─ components.json DELETE (shadcn registry config — no longer used)
├─ next.config.ts KEEP (output: standalone, trailingSlash, images.unoptimized)
├─ tsconfig.json KEEP (paths "@/*" → src/*, strict)
├─ postcss.config.mjs KEEP (@tailwindcss/postcss)
├─ biome.json KEEP
└─ src/
├─ app/
│ ├─ layout.tsx REWRITE (3 fonts + ThemeProvider + custom Toaster; rm sonner)
│ ├─ globals.css REWRITE (new token system §0; rm glass/teal/purple)
│ ├─ page.tsx KEEP (redirect → /dashboard/)
│ └─ (dashboard)/
│ ├─ layout.tsx REWRITE (providers + Spine + StatusBar + RouteTransition + MiniPlayer + Chatbot; rm shadcn Sidebar)
│ ├─ dashboard/ page.tsx REWRITE (server fetch + <DashboardView/> client; 3D SignalField hero)
│ ├─ messages/ page.tsx REWRITE (server seeds + <MessagesView/>; spine + severity ticks)
│ ├─ voice/ page.tsx REWRITE (server seeds + <VoiceView/>; 3D OrbField hero)
│ ├─ media/ page.tsx REWRITE (server seeds + <MediaView/>; turntable disc)
│ ├─ recordings/ page.tsx REWRITE (server seeds + <RecordingsView/>; waveform rows)
│ ├─ moderation/ page.tsx REWRITE (server seeds + <ModerationView/>; event-flow)
│ └─ analysis/ page.tsx REWRITE (client <AnalysisView/>; query console)
├─ components/
│ ├─ ui/ DELETE (all 50 shadcn primitives)
│ ├─ primitives/ NEW (Button, Input, Select, Dialog, Tooltip, Badge, Progress, Avatar, Skeleton, Toast, Sheet, index.ts)
│ ├─ motion/ NEW (variants.ts, Stagger.tsx, RouteTransition.tsx)
│ ├─ three/ NEW (WebGLGuard, SignalField, OrbField, StaticFallback, useThreeScene)
│ ├─ charts/ NEW (Sparkline, AreaActivity, RadialGauge, SessionRibbon, Waveform)
│ ├─ layout/ REWRITE (Spine NEW, StatusBar NEW, ThemeToggle REWRITE, app-sidebar DELETE)
│ ├─ dashboard/ REWRITE (stat-card DELETE; activity-chart/hourly/top-channels/moderation-donut/users/channels/reactions REWRITE)
│ ├─ messages/ REWRITE (message-card DELETE; message-list/detail/detail-view/ai-status-badge/ai-analysis-panel/attachments-grid/lightbox/search-overlay REWRITE)
│ ├─ voice/ REWRITE (voice-connection-card/connection-card/microphone-card DELETE; speaker-waveform/active-speakers-panel/activity-timeline/mic-control/listen-control REWRITE)
│ ├─ media/ REWRITE (music-player, mini-player REWRITE)
│ ├─ recordings/ REWRITE (recording-card DELETE; recording-player REWRITE)
│ ├─ moderation/ REWRITE (moderation-section REWRITE)
│ ├─ analysis/ REWRITE (search-panel REWRITE)
│ ├─ chatbot/ REWRITE (chatbot-container, chat-panel REWRITE; chatbot-context, index KEEP)
│ └─ shared/ REWRITE (empty-state, error-state, loading-skeleton, error-boundary, guild-selector REWRITE; index KEEP)
├─ hooks/ KEEP (verify no bad imports)
├─ lib/
│ ├─ api/ KEEP (server.ts, client.ts, index.ts)
│ ├─ ws/ KEEP (connection.ts, context.tsx, types.ts, ws-hook.ts)
│ ├─ types/ KEEP (all interfaces)
│ ├─ hooks/ KEEP (use-media-player.tsx, use-mounted.ts)
│ ├─ audio/ KEEP (voice PCM decode helpers if present)
│ ├─ format.ts KEEP
│ ├─ utils.ts KEEP (cn)
│ └─ navigation.ts KEEP
└─ (public assets) KEEP
```
### 4.1 Single-file page pattern (mandatory)
```tsx
// server component (default export) — runs on the server, fetches initial data
import { getX, getY } from "@/lib/api/server";
import { XView } from "./page"; // self-import of the named client export
export default async function Page() {
const [a, b] = await Promise.allSettled([getX(), getY()]);
return <XView initialA={a.status === "fulfilled" ? a.value : undefined}
initialB={b.status === "fulfilled" ? b.value : undefined} />;
}
// client component (named export) — hydrated, takes initialData as SWR fallback
"use client";
export function XView({ initialA, initialB }: Props) {
const { data } = useX(initialA); // SWR fallbackData = initialA
}
```
Self-referencing the named export keeps the file single-artifact while satisfying Next's RSC boundary (default = server, named = client). Tabs live inside `XView`.
### 4.2 Import rules (lint gate)
- No `@/components/ui/*` (deleted) — all UI via `@/components/primitives`.
- `three` only imported inside `src/components/three/*`; pages import those via `next/dynamic({ ssr: false })`.
- `motion` imported from `motion/react` only.
- All data: `@/lib/api/server` (server) / `@/lib/api/client` (client) — never invented endpoints.
---
## TASKS (granular — every file is its own task)
### PHASE 0 — Dependency surgery
- **T0.1** Edit `package.json`: remove `dependencies["@base-ui/react"]`.
- **T0.2** Remove `dependencies["@shadcn/react"]`.
- **T0.3** Remove `dependencies["shadcn"]`.
- **T0.4** Remove `dependencies["recharts"]`.
- **T0.5** Remove `dependencies["cmdk"]`.
- **T0.6** Remove `dependencies["sonner"]`.
- **T0.7** Remove `dependencies["react-day-picker"]`.
- **T0.8** Remove `dependencies["embla-carousel-react"]`.
- **T0.9** Remove `dependencies["react-resizable-panels"]`.
- **T0.10** Remove `dependencies["input-otp"]`.
- **T0.11** Add `dependencies["motion"]: "^12.0.0"`, `dependencies["three"]: "^0.180.0"`, `devDependencies["@types/three"]: "^0.180.0"`.
- **T0.12** `rm -f pnpm-lock.yaml && pnpm install` (regenerate lockfile).
- **T0.13** Verify `pnpm ls recharts @shadcn/react @base-ui/react` → empty; `pnpm ls motion three @types/three` → present.
- **T0.14** `cat pnpm-workspace.yaml`: confirm `onlyBuiltDependencies` keeps needed native builds, no broken shadcn postinstall.
### PHASE 1 — Design tokens (globals.css)
- **T1.1** Rewrite `@theme { }` head: light `:root` palette from §0 (canvas/surface/ink/ink-soft/hairline/signal/signal-ink/amber/vermilion/ring).
- **T1.2** Add radius tokens `--r: 14px`, `--r-panel: 12px`, `--r-control: 8px`, `--r-pill: 9999px`.
- **T1.3** Add `--font-display` token; keep `--font-sans`/`--font-mono`.
- **T1.4** Add `.dark { }` override block with §0 dark values (warm charcoal).
- **T1.5** Replace `@layer base body` bg: warm dot-grid `radial-gradient(oklch(0.45 0.03 70 / 0.05) 1px, transparent 1px)` + faint warm glow; remove old bluish radial layers.
- **T1.6** Retint scrollbar thumb to warm `oklch(0.4 0.02 70 / 0.2)`; keep `::selection` signal-tinted.
- **T1.7** Delete `.glass`, `.glass-elevated`, `.glass-intense`, `.dark .glass-intense` utilities.
- **T1.8** Delete `.text-gradient` and `.gradient-border`.
- **T1.9** Add `.surface` utility (bg var(--surface), radius var(--r), padding).
- **T1.10** Add `.scan-tick` (1px animated pulse line, keyframe `scan`).
- **T1.11** Add `.ticker`, `.pill`, `.mono` utilities.
- **T1.12** Add keyframes `scan`, `eq`, `fade-up`, `spin-disc` (keep used existing ones if still referenced).
- **T1.13** Add `@media (prefers-reduced-motion: reduce)` kill switch for scan/eq/spin-disc/pulse-ring/shimmer.
- **T1.14** Remove `@import "shadcn/tailwind.css";` (line 3); verify nothing else depends on shadcn CSS vars.
- **T1.15** Verify `grep -c "0.52 0.17 215\|0.55 0.2 280" src/app/globals.css``0`.
- **T1.16** `pnpm biome check src/app/globals.css` → no errors.
### PHASE 2 — Root layout + fonts
- **T2.1** In `layout.tsx` add `Bricolage_Grotesque` (`variable: "--font-display"`, subsets `["latin"]`, `display: "swap"`).
- **T2.2** Apply `inter.variable`, `jetbrainsMono.variable`, `bricolage.variable` to `<html>`.
- **T2.3** Remove `import { Toaster } from "@/components/ui/sonner"`.
- **T2.4** Comment out `<Toaster />` temporarily (re-enabled after T3.10).
- **T2.5** Keep `suppressHydrationWarning`, `ThemeProvider` (defaultTheme dark, enableSystem false).
- **T2.6** `npx tsc --noEmit` (fonts only; rest may still error until primitives exist).
### PHASE 3 — Custom primitives (replace 50 shadcn ui)
- **T3.1** `primitives/Button.tsx`: `motion.button`, variants `primary`/`ghost`/`danger`, `cn()` merge, `cursor-pointer`, `focus-visible:ring-2 ring-signal`, `whileTap` scale 0.97 gated by `useReducedMotion()`.
- **T3.2** `primitives/Input.tsx`: native `<input>`, `bg-surface`, `rounded-[var(--r-control)]`, `mono` prop.
- **T3.3** `primitives/Select.tsx`: native `<select>` styled, `bg-surface`.
- **T3.4** `primitives/Dialog.tsx`: native `<dialog>` + `showModal()`, warm `::backdrop`, `AnimatePresence`, `onClose`.
- **T3.5** `primitives/Tooltip.tsx`: CSS group-hover popover.
- **T3.6** `primitives/Badge.tsx`: tonal pill, `variant``bg-{tone}/15 text-{tone}` (signal/amber/vermilion/neutral).
- **T3.7** `primitives/Progress.tsx`: SVG track + `motion` fill, `value`/`max`, signal color.
- **T3.8** `primitives/Avatar.tsx`: `<img>` + initials fallback, signal bg, size prop.
- **T3.9** `primitives/Skeleton.tsx`: shimmer block (signal-tinted), `aria-hidden`.
- **T3.10** `primitives/Toast.tsx`: `ToastProvider` context + portal, `useToast()`, motion slide-in, auto-dismiss.
- **T3.11** `primitives/Sheet.tsx`: mobile drawer (`translate-x` spring), overlay, `open`/`onClose`.
- **T3.12** `primitives/index.ts` re-export all 11.
- **T3.13** Re-enable `<Toaster />` in `layout.tsx` (T2.4).
- **T3.14** Verify `npx tsc --noEmit` on primitives; `grep -rl "@/components/ui/" src/components/primitives` → empty.
### PHASE 4 — Motion foundation
- **T4.1** `motion/variants.ts`: export `spring`, `ease`, `fadeUp`, `stagger` (per §0.1).
- **T4.2** `motion/Stagger.tsx`: `StaggerGroup` + `StaggerItem` (`"use client"`).
- **T4.3** `motion/RouteTransition.tsx`: `"use client"`, `usePathname`, `AnimatePresence mode="popLayout"`, reduced-motion fallback to plain `<div>`.
- **T4.4** Verify `npx tsc --noEmit` on motion; `motion/react` import resolves.
### PHASE 5 — Custom SVG charts (replace recharts)
- **T5.1** `charts/Sparkline.tsx`: `<svg>` polyline from `points:number[]`, signal stroke, no axes.
- **T5.2** `charts/AreaActivity.tsx`: filled `<path>` area, low-opacity signal gradient, `pathLength` draw gated by reduced-motion.
- **T5.3** `charts/RadialGauge.tsx`: `<circle>` arc `stroke-dasharray`, center mono label.
- **T5.4** `charts/SessionRibbon.tsx`: horizontal segments per speaker duration.
- **T5.5** `charts/Waveform.tsx`: bars from deterministic seed, spring scaleY on hover.
- **T5.6** Verify `npx tsc --noEmit` on charts; no `recharts` import.
### PHASE 6 — Three.js foundation (lazy, guarded)
- **T6.1** `three/WebGLGuard.tsx`: `"use client"`, detect webgl2/webgl, render `children` or `fallback`.
- **T6.2** `three/useThreeScene.ts`: shared hook — renderer init (`dpr:[1,1.75]`, `powerPreference`), RAF with `document.hidden` pause, `dispose()` on unmount, resize observer.
- **T6.3** `three/SignalField.tsx`: `Points` BufferGeometry (~min(900, w*h/2000)), additive blend, signal-lime, fog, idle rotation + sine drift, `activity` prop, pointer parallax. Uses `useThreeScene`.
- **T6.4** `three/OrbField.tsx`: per-speaker `Sphere`, y-scale + ring lerp to speaking, tones signal/idle/vermilion.
- **T6.5** `three/StaticFallback.tsx`: 2D SVG/CSS silhouette for both scenes.
- **T6.6** Verify `grep -rl "from \"three\"" src | grep -v "components/three"` → empty; `npx tsc --noEmit` on three.
### PHASE 7 — Layout shell
- **T7.1** `layout/Spine.tsx`: `"use client"`, vertical rail from `navItems`, icon node + hairline, active signal dot + label reveal (motion spring), `max-md:` bottom tab-bar.
- **T7.2** `layout/StatusBar.tsx`: `"use client"`, page title + WS status dot (motion pulse) + `GuildSelector` + live clock (mono) + `ThemeToggle`.
- **T7.3** `layout/ThemeToggle.tsx`: restyle, keep `next-themes` logic, motion icon swap.
- **T7.4** Rewrite `(dashboard)/layout.tsx`: keep SWRConfig/WsProvider/MediaPlayerProvider/ChatbotProvider + sync functions verbatim; swap `AppSidebar``Spine`, header→`StatusBar`, wrap children in `RouteTransition`; remove `SidebarInset`/`SidebarTrigger`/`Separator`; keep MiniPlayer+ChatbotContainer.
- **T7.5** Delete `layout/app-sidebar.tsx`.
- **T7.6** Verify `grep -rl "components/ui/sidebar\|app-sidebar" src` → empty; `npx tsc --noEmit`.
### PHASE 8 — Dashboard page + components
- **T8.1** Rewrite `dashboard/page.tsx`: default async `getDashboardStats`+`getActivity``<DashboardView>`; named `"use client"` view with useStats/useActivity, 3D hero + tickers + tabs.
- **T8.2** Add `WebGLGuard`+`SignalField` hero with `activity` ratio; overlay headline (Bricolage) + `<RadialGauge>`.
- **T8.3** Build asymmetric ticker row with `StaggerGroup` + 4 `.surface` blocks (mono number + label + `<Sparkline>`); inline (replaces stat-card).
- **T8.4** Delete `dashboard/stat-card.tsx`.
- **T8.5** Reskin `dashboard/activity-chart.tsx``charts/AreaActivity` (daily).
- **T8.6** Reskin `dashboard/hourly-activity-chart.tsx``charts/AreaActivity` (hourly).
- **T8.7** Reskin `dashboard/top-channels-chart.tsx``charts/` + `.surface`.
- **T8.8** Reskin `dashboard/moderation-donut.tsx``charts/RadialGauge`.
- **T8.9** Reskin `dashboard/users-section.tsx``.surface`.
- **T8.10** Reskin `dashboard/channels-section.tsx``.surface`.
- **T8.11** Reskin `dashboard/reactions-section.tsx``.surface`.
- **T8.12** Verify `grep -rl "components/ui/card" src/app/\(dashboard\)/dashboard src/components/dashboard` → empty; `npx tsc --noEmit`.
### PHASE 9 — Messages page + components
- **T9.1** Rewrite `messages/page.tsx`: default `getMessages(guildId)`(+channels) → `<MessagesView>`; client spine + entries.
- **T9.2** Delete `messages/message-card.tsx`.
- **T9.3** Rewrite `messages/message-list.tsx`: left timeline spine + right severity-tick entries (`surface` + `border-l-2` lime/amber/vermilion, motion spring tick), `layout` reflow.
- **T9.4** Rewrite `messages/message-detail.tsx``.surface`.
- **T9.5** Rewrite `messages/message-detail-view.tsx``.surface` pane.
- **T9.6** Rewrite `messages/ai-status-badge.tsx``primitives/Badge`.
- **T9.7** Rewrite `messages/ai-analysis-panel.tsx``.surface`.
- **T9.8** Rewrite `messages/attachments-grid.tsx``.surface` grid.
- **T9.9** Rewrite `messages/lightbox.tsx``primitives/Dialog`.
- **T9.10** Rewrite `messages/search-overlay.tsx` → console palette, type-in animation, `primitives/Dialog`.
- **T9.11** Verify no `components/ui/card` in messages tree; `npx tsc --noEmit`.
### PHASE 10 — Voice page + components
- **T10.1** Rewrite `voice/page.tsx`: default `getVoiceStatus()``<VoiceView>`; client `WebGLGuard`+`OrbField` hero + ribbon + tabs.
- **T10.2** Delete `voice/voice-connection-card.tsx`, `connection-card.tsx`, `microphone-card.tsx`.
- **T10.3** Rewrite `voice/speaker-waveform.tsx` → SVG ring / eq bars.
- **T10.4** Rewrite `voice/active-speakers-panel.tsx``.surface`.
- **T10.5** Rewrite `voice/activity-timeline.tsx``charts/SessionRibbon`.
- **T10.6** Rewrite `voice/mic-control.tsx``primitives/Button`.
- **T10.7** Rewrite `voice/listen-control.tsx``primitives/Button`.
- **T10.8** Verify `npx tsc --noEmit`; no `components/ui/card` in voice tree.
### PHASE 11 — Media page + components
- **T11.1** Rewrite `media/page.tsx`: default `getMediaStatus()``<MediaView>`; client turntable disc + transport + queue.
- **T11.2** Rewrite `media/music-player.tsx`: CSS-3D disc (spin-disc, pause when not playing, spring on play/pause), mono meta, `primitives/Button` transport, `.surface` queue rows with `layout`.
- **T11.3** Rewrite `media/mini-player.tsx` → compact `.surface`.
- **T11.4** Verify `npx tsc --noEmit`; no `components/ui/card` in media tree.
### PHASE 12 — Recordings page + components
- **T12.1** Rewrite `recordings/page.tsx`: default `getRecordings(50)``<RecordingsView>`; client rows `AnimatePresence`+`layout`, live `voice_recording_uploaded` prepend.
- **T12.2** Delete `recordings/recording-card.tsx`.
- **T12.3** Rewrite `recordings/recording-player.tsx``.surface` row + `charts/Waveform` + `primitives/Button`/`Dialog` play/delete.
- **T12.4** Verify `npx tsc --noEmit`.
### PHASE 13 — Moderation + Analysis pages
- **T13.1** Rewrite `moderation/page.tsx`: default `getModerationStats/Actions``<ModerationView>`; client vertical event-flow, live actions pulse-in.
- **T13.2** Rewrite `moderation/moderation-section.tsx` → event-flow, no Card/table.
- **T13.3** Rewrite `analysis/page.tsx`: client `<AnalysisView/>` terminal console (`primitives/Input` mono + blinking caret, `.surface` results staggered).
- **T13.4** Rewrite `analysis/search-panel.tsx` → terminal style.
- **T13.5** Verify `npx tsc --noEmit`.
### PHASE 14 — Chatbot + shared + final cleanup
- **T14.1** Rewrite `chatbot/chatbot-container.tsx``.surface`, keep drag/minimize.
- **T14.2** Rewrite `chatbot/chat-panel.tsx` → bubbles via `AnimatePresence`, `primitives/*`.
- **T14.3** Keep `chatbot/chatbot-context.tsx` + `index.ts`.
- **T14.4** Rewrite `shared/empty-state.tsx` → tonal.
- **T14.5** Rewrite `shared/error-state.tsx`.
- **T14.6** Rewrite `shared/loading-skeleton.tsx``primitives/Skeleton`.
- **T14.7** Rewrite `shared/error-boundary.tsx`.
- **T14.8** Rewrite `shared/guild-selector.tsx``primitives/Select`.
- **T14.9** `rm -rf src/components/ui && rm -f components.json`.
- **T14.10** `grep -rn "components/ui/\|@shadcn\|@base-ui\|recharts" src` → MUST be empty.
- **T14.11** `grep -rl "from \"recharts\"\|@base-ui\|@shadcn" src` → empty (double-check).
- **T14.12** Verify `npx tsc --noEmit` across whole `src`.
### PHASE 15 — Build + lint gate
- **T15.1** `cd services/frontend && npx tsc --noEmit` → 0 errors.
- **T15.2** `pnpm biome check` → fix all issues (no `any` in new files).
- **T15.3** `pnpm build` (standalone) → success, emits `.next/standalone/server.js`.
- **T15.4** Inspect `.next/static/chunks/` for three-heavy chunk loaded only on dashboard/voice; confirm NOT in `/dashboard/` initial SSR HTML.
- **T15.5** Confirm `pnpm-lock.yaml` present (reproducible flake install).
- **T15.6** `grep -c "0.52 0.17 215\|0.55 0.2 280" .next/static/css/*.css` → 0.
### PHASE 16 — Local runtime smoke test (no prod)
- **T16.1** Start local standalone: `GMW_BACKEND_URL=http://127.0.0.1:4001 PORT=4017 node .next/standalone/server.js &`.
- **T16.2** `curl -s -o /dev/null -w "%{http_code}"` for all 7 routes → 200.
- **T16.3** `curl /dashboard/ | grep -o "Bricolage\|signal\|surface"` → present.
- **T16.4** `curl /_next/static/css/*.css | grep "0.52 0.17 215\|0.55 0.2 280"` → empty.
- **T16.5** Headless browser `/dashboard/`+`/voice/` WebGL on: no console errors, `<canvas>` present, `<StaticFallback/>` NOT rendered.
- **T16.6** Same pages WebGL off: `<StaticFallback/>` renders, no crash.
- **T16.7** Kill local server. Do NOT touch prod unit.
- **T16.8** Confirm 7 routes 200 + no console errors in T16.5/16.6.
### PHASE 17 — Flake + staging deploy
- **T17.1** Inspect `flake.nix` frontend drv: `filterSource` ignores `out/.next/node_modules`; `pnpm-lock.yaml` included.
- **T17.2** `nix build .#gmw-frontend --impure --sandbox-off` → succeeds.
- **T17.3** `nix copy` frontend drv to VPS into staging profile (test port e.g. 4217).
- **T17.4** Create/adjust staging systemd unit with `PORT=4217` exported BEFORE `node server.js` (LIDM PORT bug).
- **T17.5** `sudo systemctl restart gmw-frontend-staging`; `curl` staging → 200.
- **T17.6** Browser-check staging `/dashboard/`+`/voice/` (3D visible, fallback test).
- **T17.7** Verify staging CSS has no old teal/purple; new design renders.
### PHASE 18 — Production swap (CONFIRM WITH USER FIRST)
- **T18.1** STOP — send staging screenshots/URL; await explicit approval before touching prod.
- **T18.2** On approval: `nix-env --profile /nix/var/nix/profiles/gmw-frontend --set <new-drv>`.
- **T18.3** Confirm `gmw-frontend.service` exports `PORT=4017` before exec.
- **T18.4** `sudo systemctl restart gmw-frontend`.
- **T18.5** `curl` all 7 routes on `https://imphnen.asepharyana.my.id` → 200.
- **T18.6** Browser verify prod: new design, old teal gone, 3D scenes render.
- **T18.7** `journalctl -u gmw-frontend -f` 5 min; confirm WS reconnect + live features.
- **T18.8** Notify user with before/after notes; keep rollback plan (`nix-env --set <previous>; systemctl restart`).
---
## Risks / Trade-offs
- **Scope:** 7 pages + charts + primitives + motion + 2 three scenes + layout. Big but mechanical; each task is isolated (~90 atomic tasks).
- **Bundle weight:** `three` adds ~150KB gz but ONLY on dashboard/voice routes (lazy chunk, `ssr:false`). `motion` ~35KB gz app-wide — acceptable.
- **WebGL compatibility:** covered by `WebGLGuard` + static fallback. Old devices / strict privacy browsers never blank.
- **Motion excess:** risk of "AI-generated" scattered animation. Guard: one signature moment per page, shared variants, reduced-motion gates.
- **Feature regressions:** Voice PCM playback, media transport, chatbot drag — logic preserved, only skin changes. Smoke test (T16) catches SSR breaks; live WS/3D needs real backend (staging T17).
- **Removed deps:** dropping `recharts`/`sonner`/`cmdk` means rewriting charts + toasts + search palette — accounted for in Phases 3/5/9.
- **Next standalone PORT bug:** `server.js` may not read `PORT` — ensure unit exports `PORT=4017` before exec (T17.4/T18.3).
- **three + React 19:** raw three avoids R3F compat surface; lifecycle (dispose + RAF) handled in T6.2.
- **next-themes:** keep (light/dark toggle); default dark.
## Open questions (answer before T18)
- Q1: Deploy to prod now or staging-only first? (Recommend staging + screenshot review.)
- Q2: Keep `react-day-picker`/`embla` if any page still needs them? (Plan assumes no — verify in T14.10 grep.)
- Q3: Any brand name/wordmark change from "Discord Automod"? (Keep "Bete" identity unless told.)
- Q4: 3D depth — full 3D scenes on dashboard+voice as specced, or also a 3D accent on media (disc)? (Default: dashboard+voice only; media disc stays CSS 3D.)
-118
View File
@@ -1,118 +0,0 @@
# Bete — Discord Moderation Dashboard
Bot monitoring Discord yang merekam voice channel, menangkap pesan teks, menyimpan attachment, menjalankan analisis AI opsional, dan menyediakan dashboard web real-time.
**Stack utama:** Node.js (Express 5), pnpm, TypeScript, React 19 (Next.js 16), Tailwind v4, shadcn/ui, Drizzle ORM, PostgreSQL, WebSocket, Redis pub/sub.
## Prasyarat
- Node.js 22+
- pnpm 11.x
- FFmpeg di `PATH` (untuk audio muxing dan playback media)
- `yt-dlp` di `PATH` (untuk resolve audio YouTube/Spotify)
- Bun (untuk frontend dev — opsional, bisa pake pnpm)
- PostgreSQL 15+
## Setup
```bash
pnpm install
cp .env.example .env
# Edit .env sesuai konfigurasi server
```
## Menjalankan
```bash
# Backend (port 3001)
pnpm run dev:backend
# Discord Gateway (capture messages, voice, dll)
pnpm run dev:discord-gateway
# Frontend (port 3000)
pnpm run dev:web
```
## Build
```bash
pnpm run build:backend
pnpm run build:discord-gateway
pnpm run build:web # next build — static export ke out/
pnpm run build # build semua service
```
## Deploy
```bash
./deploy.sh # Build + deploy semua service ke VPS
./deploy.sh --frontend # Frontend only
./deploy.sh --backend # Backend only
./deploy.sh --no-build # Skip build, copy files aja
```
## Service Architecture
```
Discord
|
v
discord-gateway ←→ Redis ←→ backend (Express 5) ←→ frontend (Next.js)
| pub/sub | |
| +— REST API (/api/*) |
| +— WebSocket (/ws) |
+— message capture +— AI moderation |
+— voice recording +— dashboard data +— dashboard UI
+— attachment upload +— real-time updates
```
## Fitur
- **Message capture**: Capture pesan baru, edit, dan delete dari Discord
- **Voice recording**: Rekam voice channel ke segmen OGG per user, streaming PCM real-time ke WebSocket
- **Attachment upload**: Download + upload attachment ke external storage
- **AI moderation**: Analisis pesan opsional via LLM, auto-delete, queue management
- **Dashboard**: Messages feed, AI analysis review, voice connection, music player, recordings, user/channel stats
- **Media playback**: Playback dari URL, file lokal, YouTube, Spotify
- **WebSocket**: Real-time event streaming untuk semua aktivitas
- **Public API**: Semua endpoint REST dan WebSocket dapat diakses tanpa autentikasi
## Struktur Proyek
```
services/
├── backend/ # Express 5 REST API + WebSocket server
│ ├── src/modules/ # Feature modules (messages, voice, media, dll)
│ └── src/http/ # Express app setup, middleware
├── discord-gateway/ # Discord client, voice recording, AI analysis
│ ├── src/modules/ # message-capture, voice-recording, ai-moderation
│ └── src/shared/ # Config, database, Discord client
└── frontend/ # Next.js 16 dashboard (static export)
├── src/app/ # Pages (login, dashboard tabs)
├── src/features/ # Feature components (dashboard, live, messages)
└── src/lib/ # API client, WebSocket, types
packages/
└── shared/ # Shared types, errors, logger, utilities
```
## Database
PostgreSQL via Drizzle ORM. Migrasi:
```bash
pnpm run db:generate # Generate migration
pnpm run db:migrate # Apply migration
pnpm run db:studio # Drizzle Studio
```
## WebSocket Events
Backend broadcast event berikut ke frontend via WebSocket:
- `message_created`, `message_updated`, `message_deleted`, `message_analyzed`
- `attachment_created`, `attachment_uploaded`
- `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`
- `voice_active_user`, `voice_pcm_data`
- `media_state`
- `reaction_*`, `thread_*`, `presence_updated`, `guild_member_*`
+9 -3
View File
@@ -35,12 +35,15 @@
"suspicious": { "suspicious": {
"noUnknownAtRules": "off", "noUnknownAtRules": "off",
"useIterableCallbackReturn": "off", "useIterableCallbackReturn": "off",
"noArrayIndexKey": "warn" "noArrayIndexKey": "off",
"noExplicitAny": "off"
}, },
"a11y": { "a11y": {
"useSemanticElements": "off", "useSemanticElements": "off",
"useButtonType": "off", "useButtonType": "off",
"noAutofocus": "off" "noAutofocus": "off",
"useMediaCaption": "off",
"noStaticElementInteractions": "off"
}, },
"performance": { "performance": {
"noImgElement": "warn" "noImgElement": "warn"
@@ -50,7 +53,10 @@
}, },
"correctness": { "correctness": {
"noInvalidUseBeforeDeclaration": "off", "noInvalidUseBeforeDeclaration": "off",
"noUnusedFunctionParameters": "warn" "noUnusedFunctionParameters": "warn",
"noUnusedVariables": "warn",
"noUnusedImports": "warn",
"noUnusedPrivateClassMembers": "warn"
} }
}, },
"domains": { "domains": {
+102 -42
View File
@@ -11,6 +11,19 @@
let let
pkgs = import nixpkgs { inherit system; }; pkgs = import nixpkgs { inherit system; };
# Source filter: `path:` literals do NOT respect .gitignore by default,
# so a dirty local out/ (stale chunks from previous builds) leaks into
# the sandbox. Filter out build artifacts explicitly.
filterSource = { dir, ignore }: builtins.path {
path = dir;
name = "source";
filter = (path: type: let base = baseNameOf path; in !(builtins.elem base ignore));
};
frontendSrc = filterSource {
dir = ./services/frontend;
ignore = [ "out" ".next" "node_modules" "pnpm-lock.yaml" ];
};
# OpenSSL headers (.dev output) + STATIC libs (pkgsStatic.openssl.out — # OpenSSL headers (.dev output) + STATIC libs (pkgsStatic.openssl.out —
# node-datachannel's CMakeLists sets OPENSSL_USE_STATIC_LIBS=TRUE, and # node-datachannel's CMakeLists sets OPENSSL_USE_STATIC_LIBS=TRUE, and
# the default `pkgs.openssl` resolves to `bin` which has no lib/) merged # the default `pkgs.openssl` resolves to `bin` which has no lib/) merged
@@ -35,8 +48,8 @@
export GIT_SSL_CAINFO=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt export GIT_SSL_CAINFO=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt
export NIX_SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt export NIX_SSL_CERT_FILE=${pkgs.cacert}/etc/ssl/certs/ca-bundle.crt
# pnpm uses node-gyp for native addons provide build tools # pnpm uses node-gyp for native addons provide build tools (kept for
export npm_config_build_from_source=true # the rare case a prebuilt is unavailable and it falls back to compile).
export CPPFLAGS="-I${pkgs.lib.getDev pkgs.openssl}/include" export CPPFLAGS="-I${pkgs.lib.getDev pkgs.openssl}/include"
export LDFLAGS="-L${pkgs.lib.getLib pkgs.openssl}/lib" export LDFLAGS="-L${pkgs.lib.getLib pkgs.openssl}/lib"
@@ -46,6 +59,36 @@
pnpm rebuild 2>&1 || true pnpm rebuild 2>&1 || true
''; '';
# Shrink the shipped node_modules to production deps only. The full
# install's .pnpm virtual store carries dev-only packages (biome,
# typescript, esbuild, drizzle-kit, vitest, ... ~150MB+) that are never
# needed at runtime, so we delete every .pnpm dir that is not part of
# the resolved production graph (`pnpm list --prod`).
#
# NOTE: do NOT use `pnpm install --prod` here — it collapses the
# public-hoist dir (.pnpm/node_modules) that runtime peer resolution
# relies on (e.g. @lng2004/node-datachannel and @seydx/node-av-linux-x64
# are only reachable through it), silently breaking voice.
# Instead we keep the full install's symlink layout and only prune
# orphaned package dirs + broken symlinks.
# Must run AFTER tsc (typescript is a devDep) and after native builds.
pruneProd = ''
echo "=== Pruning devDependencies (production-only node_modules) ==="
pnpm list --prod --depth 999 --parseable 2>/dev/null \
| grep -o '\.pnpm/[^/]*' | sort -u > $TMPDIR/prod-pnms.txt
( cd node_modules/.pnpm \
&& for d in */; do \
d="''${d%/}"; \
[ "$d" = "node_modules" ] && continue; \
grep -qF ".pnpm/$d" $TMPDIR/prod-pnms.txt || rm -rf "$d"; \
done ) || true
# Drop symlinks whose .pnpm target was pruned (top-level, scoped dirs,
# hoist, .bin any depth). Mirrors stdenv's noBrokenSymlinks check,
# which would otherwise fail the fixupPhase.
find node_modules -type l ! -exec test -e {} \; -delete 2>/dev/null || true
du -sh node_modules
'';
# ---- Backend ---- # ---- Backend ----
backend = pkgs.stdenv.mkDerivation { backend = pkgs.stdenv.mkDerivation {
pname = "gmw-backend"; pname = "gmw-backend";
@@ -84,7 +127,7 @@
console.log('Fixed ' + count + ' files'); console.log('Fixed ' + count + ' files');
" "
echo "=== Build complete ===" echo "=== Build complete ==="
''; '' + pruneProd;
installPhase = '' installPhase = ''
mkdir -p $out/lib/gmw-backend mkdir -p $out/lib/gmw-backend
@@ -119,7 +162,7 @@ WRAPPER
pkgs.pkg-config pkgs.pkg-config
pkgs.openssl pkgs.openssl
pkgs.openssl.dev pkgs.openssl.dev
pkgs.git # libdatachannel FetchContent clones from GitHub pkgs.git # for any FetchContent-based deps during native builds
pkgs.cacert pkgs.cacert
]; ];
@@ -132,32 +175,32 @@ WRAPPER
# do NOT let stdenv run its own cmake configure phase on the source. # do NOT let stdenv run its own cmake configure phase on the source.
dontUseCmakeConfigure = true; dontUseCmakeConfigure = true;
# The gateway bundles native node_modules (.node addons plus .o/.a
# object files left in prebuilt dirs). stdenv's fixupPhase walks
# $out/node_modules and runs patchELF + shrinkELF over every ELF it
# finds, choking on the non-ET_DYN files (.o/.a) and the prebuilt
# .node addons — emitting hundreds of harmless "patchelf: wrong ELF
# type" lines per build. The real binary is node (external, already
# RPATH-fixed in its own derivation) and the .node addons are
# self-contained prebuilts loaded via dlopen, so Nix's fixup pass is
# neither needed nor wanted here. Skip it entirely.
dontFixup = true;
buildPhase = pnpmInstall + '' buildPhase = pnpmInstall + ''
echo "=== Building native voice deps ===" echo "=== Building native voice deps ==="
# pnpm rebuild aborts on the first failing package and runs scripts # pnpm rebuild aborts on the first failing package and runs scripts
# from the wrong cwd build each native dep explicitly with its own # from the wrong cwd build each native dep explicitly with its own
# install script. Each failure is tolerated (|| true); the packages # install script. Each failure is tolerated (|| true); the packages
# that matter (opus, datachannel, node-av) are verified at runtime. # @discordjs/opus ships prebuilt binaries for Node 22 (ABI node-v127,
for pkg in \ # linux-x64-glibc-2.35) node-pre-gyp downloads the prebuilt .node
node_modules/.pnpm/@discordjs+opus@*/node_modules/@discordjs/opus \ # instead of compiling C++ from source. With build_from_source unset
node_modules/.pnpm/@lng2004+node-datachannel@*/node_modules/@lng2004/node-datachannel \ # (above), `pnpm rebuild` runs the package's own install script which
node_modules/.pnpm/zeromq@*/node_modules/zeromq # fetches the matching prebuilt; it only falls back to a source build
do # if the download fails. This keeps voice working without a per-build
if [ -d "$pkg" ]; then # native compile.
echo "--- native build: $pkg ---" echo "=== Rebuilding @discordjs/opus (prebuilt download) ==="
(cd "$pkg" && npm run install 2>&1 || true) pnpm rebuild @discordjs/opus 2>&1 || true
# node-datachannel's `prebuild -r napi` CLI is broken (TypeError: echo "=== Compiling TypeScript ===="
# expected first argument to be an array) the install fallback
# populates devDeps incl. cmake-js; build directly via cmake-js.
if [ "$(basename "$pkg")" = "node-datachannel" ]; then
echo "--- datachannel cmake-js compile ---"
# Nix splits OpenSSL headers/libs across outputs merge them
# (opensslDevEnv) so FindOpenSSL finds both include + libcrypto.
(cd "$pkg" && OPENSSL_ROOT_DIR="${opensslDevEnv}" npm run compile 2>&1 || true)
fi
fi
done
echo "=== Compiling TypeScript ==="
npx tsc 2>&1 npx tsc 2>&1
echo "=== Fixing @/ path aliases to relative paths ===" echo "=== Fixing @/ path aliases to relative paths ==="
node -e " node -e "
@@ -185,7 +228,7 @@ WRAPPER
console.log('Fixed ' + count + ' files'); console.log('Fixed ' + count + ' files');
" "
echo "=== Build complete ===" echo "=== Build complete ==="
''; '' + pruneProd;
installPhase = '' installPhase = ''
mkdir -p $out/lib/gmw-discord-gateway mkdir -p $out/lib/gmw-discord-gateway
@@ -210,39 +253,57 @@ WRAPPER
}; };
}; };
# ---- Frontend (Next.js static export) ---- # ---- Frontend (Next.js SSR standalone) ----
frontend = pkgs.stdenv.mkDerivation { frontend = pkgs.stdenv.mkDerivation {
pname = "gmw-frontend"; pname = "gmw-frontend";
version = "1.0.0"; version = "1.0.0";
src = ./services/frontend; src = frontendSrc;
nativeBuildInputs = [ nodejs pnpm pkgs.gnumake pkgs.gcc pkgs.cacert ]; nativeBuildInputs = [ nodejs pnpm pkgs.gnumake pkgs.gcc pkgs.cacert ];
buildPhase = pnpmInstall + '' buildPhase = pnpmInstall + ''
echo "=== Building Next.js static export ===" echo "=== Building Next.js SSR (standalone) ==="
# Build args are provided as env vars
export NEXT_TELEMETRY_DISABLED=1 export NEXT_TELEMETRY_DISABLED=1
export GMW_BACKEND_URL=http://127.0.0.1:4001
npx next build 2>&1 npx next build 2>&1
''; '';
installPhase = '' installPhase = ''
mkdir -p $out/share/gmw-frontend echo "=== Packaging standalone server ==="
cp -r out $out/share/gmw-frontend/out 2>/dev/null || \ mkdir -p $out/lib/gmw-frontend/standalone
cp -r dist $out/share/gmw-frontend/dist 2>/dev/null || \ # The standalone server bundles its own minimal node_modules but
cp -r .next $out/share/gmw-frontend/.next 2>/dev/null || true # needs the build assets + public copied INSIDE its tree.
cp -r .next/standalone/. $out/lib/gmw-frontend/standalone/
mkdir -p $out/lib/gmw-frontend/standalone/.next
cp -r .next/static $out/lib/gmw-frontend/standalone/.next/static
cp -r public $out/lib/gmw-frontend/standalone/public 2>/dev/null || true
# Copy node_modules for standalone mode if it exists # Remove dangling symlinks left by pnpm's hoisted .pnpm layout
cp -r node_modules $out/share/gmw-frontend/ 2>/dev/null || true # (e.g. node_modules/.pnpm/node_modules/...). The standalone server
# never resolves those at runtime it bundles its own node_modules
# and they trip stdenv's noBrokenSymlinks check.
find $out/lib/gmw-frontend/standalone -type l \
! -exec test -e {} \; -delete 2>/dev/null || true
mkdir -p $out/bin
cat > $out/bin/gmw-frontend << WRAPPER
#!${pkgs.runtimeShell}
cd $out/lib/gmw-frontend/standalone
export PORT=''${GMW_FRONTEND_PORT:-4017}
export HOSTNAME=127.0.0.1
exec ${nodejs}/bin/node server.js
WRAPPER
chmod +x $out/bin/gmw-frontend
''; '';
meta = { meta = {
description = "GMW Frontend Next.js static dashboard"; description = "GMW Frontend Next.js SSR dashboard";
platforms = pkgs.lib.platforms.linux; platforms = pkgs.lib.platforms.linux;
}; };
}; };
# ---- Proxy (nginx serving frontend) ---- # ---- Proxy (nginx: / -> Next SSR, /api + /ws -> backend) ----
proxy = pkgs.stdenv.mkDerivation { proxy = pkgs.stdenv.mkDerivation {
pname = "gmw-proxy"; pname = "gmw-proxy";
version = "1.0.0"; version = "1.0.0";
@@ -257,9 +318,8 @@ WRAPPER
mkdir -p $out/bin $out/etc $out/share mkdir -p $out/bin $out/etc $out/share
# Substitute placeholders in nginx template # Substitute placeholders in nginx template
sed \ sed -e "s|@NGINX_MIME@|${pkgs.nginx}/conf/mime.types|g" \
-e "s|@NGINX_MIME@|${pkgs.nginx}/conf/mime.types|g" \ -e "s|@NEXT_PORT@|4017|g" \
-e "s|@FRONTEND_ROOT@|${frontend}/share/gmw-frontend/out|g" \
${./infra/nix/nginx.conf.template} \ ${./infra/nix/nginx.conf.template} \
> $out/etc/nginx.conf > $out/etc/nginx.conf
@@ -271,7 +331,7 @@ WRAPPER
''; '';
meta = { meta = {
description = "GMW Proxy nginx serving frontend"; description = "GMW Proxy nginx -> Next.js + backend";
platforms = pkgs.lib.platforms.linux; platforms = pkgs.lib.platforms.linux;
}; };
}; };
+2 -2
View File
@@ -33,9 +33,9 @@ COPY --from=builder --chown=node:node /build/node_modules ./node_modules
COPY --from=builder --chown=node:node /build/package.json ./ COPY --from=builder --chown=node:node /build/package.json ./
USER node USER node
EXPOSE 3000 EXPOSE 4001
HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \ HEALTHCHECK --interval=30s --timeout=10s --start-period=15s --retries=3 \
CMD node -e "require('http').get('http://localhost:3000/api/health',r=>process.exit(r.statusCode===200?0:1))" CMD node -e "require('http').get('http://localhost:4001/api/health',r=>process.exit(r.statusCode===200?0:1))"
CMD ["node", "dist/index.js"] CMD ["node", "dist/index.js"]
+2 -2
View File
@@ -33,9 +33,9 @@ services:
- .env - .env
environment: environment:
NODE_ENV: production NODE_ENV: production
WEBSERVER_PORT: 3000 WEBSERVER_PORT: 4001
healthcheck: healthcheck:
test: ["CMD", "wget", "-qO-", "http://localhost:3000/api/health"] test: ["CMD", "wget", "-qO-", "http://localhost:4001/api/health"]
interval: 30s interval: 30s
timeout: 10s timeout: 10s
start_period: 15s start_period: 15s
+39 -9
View File
@@ -11,28 +11,44 @@ http {
'' close; '' close;
} }
# Next.js standalone SSR server (backend-fetching on every render).
# Not for hand-editing: @NEXT_PORT@ is substituted at build time.
upstream gmw_next {
server 127.0.0.1:@NEXT_PORT@;
keepalive 16;
}
upstream gmw_backend {
server 127.0.0.1:4001;
keepalive 16;
}
server { server {
listen 8080; listen 4009;
server_name _; server_name _;
# Use relative redirects (Location: /dashboard/) instead of absolute # Use relative redirects (Location: /dashboard/) instead of absolute
# URLs that leak the internal listen port (8080) through Traefik. # URLs that leak the internal listen port (4009) through the reverse proxy.
absolute_redirect off; absolute_redirect off;
gzip on; gzip on;
gzip_types text/plain text/css application/json application/javascript application/wasm image/svg+xml; gzip_types text/plain text/css application/json application/javascript application/wasm image/svg+xml;
gzip_min_length 256; gzip_min_length 256;
# ── Backend REST ───────────────────────────────────────────────
location ^~ /api { location ^~ /api {
proxy_pass http://127.0.0.1:3001$uri$is_args$args; proxy_pass http://gmw_backend$uri$is_args$args;
proxy_http_version 1.1;
proxy_set_header Connection ""; # keepalive to backend
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# ── Backend WebSocket (realtime shared state + voice PCM) ──────
location ^~ /ws { location ^~ /ws {
proxy_pass http://127.0.0.1:3001$uri$is_args$args; proxy_pass http://gmw_backend$uri$is_args$args;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade; proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade; proxy_set_header Connection $connection_upgrade;
@@ -45,16 +61,30 @@ http {
proxy_send_timeout 86400s; proxy_send_timeout 86400s;
} }
location /assets/ { # ── Next.js build assets — immutable, edge/shareable ───────────
root @FRONTEND_ROOT@; location ^~ /_next/static/ {
proxy_pass http://gmw_next$uri$is_args$args;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
expires 1y; expires 1y;
add_header Cache-Control "public, immutable"; add_header Cache-Control "public, immutable";
} }
# ── Everything else → Next.js server (SSR) ──
location / { location / {
root @FRONTEND_ROOT@; proxy_pass http://gmw_next$uri$is_args$args;
index index.html; proxy_http_version 1.1;
try_files $uri $uri/ /index.html; proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Next-Prefetch $http_x_next_prefetch;
proxy_buffering off;
proxy_read_timeout 30s;
} }
} }
} }
+10
View File
@@ -0,0 +1,10 @@
-- Migration: add ai_analysis_duration_ms to messages
-- Tracks how long the AI moderation LLM call took, per message (ms).
-- Idempotent: safe to re-run.
--
-- Run against the production GMW database, e.g.:
-- PGPASSWORD=*** psql -h 100.121.180.82 -p 6432 -U asephs -d dcbot \
-- -f scripts/add-ai-analysis-duration.sql
ALTER TABLE "messages"
ADD COLUMN IF NOT EXISTS "ai_analysis_duration_ms" BIGINT;
+1 -1
View File
@@ -1,5 +1,5 @@
-- Fix: missing messages and attachments tables on VPS -- Fix: missing messages and attachments tables on VPS
-- Run: PGPASSWORD=hunterz psql -h 100.108.1.124 -U asephs -d hub -f scripts/fix-missing-tables.sql -- Run: PGPASSWORD=hunterz psql -h 100.121.180.82 -U asephs -d hub -f scripts/fix-missing-tables.sql
BEGIN; BEGIN;
+6 -6
View File
@@ -225,21 +225,21 @@ All config via environment variables (`.env`), validated with Zod in `shared/con
```env ```env
# Server # Server
WEBSERVER_PORT=3001 WEBSERVER_PORT=4001
NODE_ENV=development NODE_ENV=development
LOG_LEVEL=info LOG_LEVEL=info
# Database # Database
DATABASE_URL=postgresql://user:pass@localhost:5432/discord_moderation DATABASE_URL=postgresql://asephs:***@100.121.180.82:6432/discord_moderation
# OR # OR
DATABASE_HOST=localhost DATABASE_HOST=100.121.180.82
DATABASE_PORT=5432 DATABASE_PORT=6432
DATABASE_NAME=discord_moderation DATABASE_NAME=discord_moderation
DATABASE_USER=postgres DATABASE_USER=postgres
DATABASE_PASSWORD=secret DATABASE_PASSWORD=secret
# Redis (optional, for pub/sub) # Redis (optional, for pub/sub)
REDIS_URL=redis://localhost:6379 REDIS_URL=redis://100.121.180.82:6379
# Discord # Discord
MONITOR_GUILD_ID=123456789 MONITOR_GUILD_ID=123456789
@@ -263,7 +263,7 @@ Use Vitest with mocked database and services.
2. **Implement repository queries** for each module using Drizzle ORM 2. **Implement repository queries** for each module using Drizzle ORM
3. **Add WebSocket server** in `src/ws/server.ts` with Redis pub/sub listener 3. **Add WebSocket server** in `src/ws/server.ts` with Redis pub/sub listener
4. **Create Discord Gateway service** in `services/discord-gateway/` (separate microservice) 4. **Create Discord Gateway service** in `services/discord-gateway/` (separate microservice)
5. **Add Docker & CI/CD** for multi-service deployment 5. **Add Nix & CI/CD** for multi-service deployment (flake.nix + GitHub Actions → nix copy → systemd)
6. **Write integration tests** for full request flow 6. **Write integration tests** for full request flow
## Circular Dependency Check ## Circular Dependency Check
+2 -2
View File
@@ -1,10 +1,10 @@
/** /**
* E2E API tests — runs against a running backend instance. * E2E API tests — runs against a running backend instance.
* Usage: API_BASE=http://localhost:3001 vitest run * Usage: API_BASE=http://localhost:4001 vitest run
*/ */
import { describe, expect, it } from "vitest"; import { describe, expect, it } from "vitest";
const BASE = process.env.API_BASE ?? "http://localhost:3001/api"; const BASE = process.env.API_BASE ?? "http://localhost:4001/api";
async function api(path: string, init?: RequestInit) { async function api(path: string, init?: RequestInit) {
const res = await fetch(`${BASE}${path}`, { const res = await fetch(`${BASE}${path}`, {
+2
View File
@@ -13,6 +13,7 @@ import { createDashboardRouter } from "../modules/dashboard/index.js";
import { createHealthRouter } from "../modules/health/index.js"; import { createHealthRouter } from "../modules/health/index.js";
import { createMediaRouter } from "../modules/media/index.js"; import { createMediaRouter } from "../modules/media/index.js";
import { createMessagesRouter } from "../modules/messages/index.js"; import { createMessagesRouter } from "../modules/messages/index.js";
import { createModerationRouter } from "../modules/moderation/index.js";
import { createRecordingsRouter } from "../modules/recordings/index.js"; import { createRecordingsRouter } from "../modules/recordings/index.js";
import { createUiStateRouter } from "../modules/ui-state/index.js"; import { createUiStateRouter } from "../modules/ui-state/index.js";
import { createVoiceRouter } from "../modules/voice/index.js"; import { createVoiceRouter } from "../modules/voice/index.js";
@@ -69,6 +70,7 @@ export function createHttpApp(): Express {
app.use("/api", createUiStateRouter()); app.use("/api", createUiStateRouter());
app.use("/api", createMediaRouter()); app.use("/api", createMediaRouter());
app.use("/api", createVoiceRouter()); app.use("/api", createVoiceRouter());
app.use("/api", createModerationRouter());
// 404 handler // 404 handler
app.use((_req: Request, res: Response) => { app.use((_req: Request, res: Response) => {
+11
View File
@@ -24,6 +24,15 @@ async function main() {
async function shutdown(signal: string) { async function shutdown(signal: string) {
logger.info({ signal }, "Shutting down gracefully"); logger.info({ signal }, "Shutting down gracefully");
// Failsafe: graceful shutdown must never hang the process forever.
// httpServer.close() waits for ALL open connections (including lingering
// WebSocket/keep-alive sockets), so on a stuck connection the process would
// otherwise sit zombie and systemd (Restart=always) can never revive it.
const forceExitTimer = setTimeout(() => {
logger.error({ signal }, "Graceful shutdown timed out; forcing exit");
process.exit(1);
}, 10_000);
try { try {
// 1. Stop accepting new HTTP connections // 1. Stop accepting new HTTP connections
if (httpServer) { if (httpServer) {
@@ -54,9 +63,11 @@ async function shutdown(signal: string) {
); );
logger.info("Graceful shutdown completed"); logger.info("Graceful shutdown completed");
clearTimeout(forceExitTimer);
process.exit(0); process.exit(0);
} catch (err) { } catch (err) {
logger.error({ err }, "Error during graceful shutdown"); logger.error({ err }, "Error during graceful shutdown");
clearTimeout(forceExitTimer);
process.exit(1); process.exit(1);
} }
} }
@@ -9,6 +9,18 @@ interface AuthenticatedRequest extends Request {
userId?: string; userId?: string;
} }
/**
* Resolve the actor id for a request. Frontend (no-login) sends a per-device
* UUID via X-User-Id so chat history stays isolated per visitor; a registered
* auth middleware userId takes precedence when present.
*/
function resolveUserId(req: Request): string {
const authId = (req as AuthenticatedRequest).userId;
if (authId) return authId;
const header = (req.headers["x-user-id"] as string | undefined)?.trim();
return header || "anonymous";
}
export const handleChatbotChat = asyncHandler( export const handleChatbotChat = asyncHandler(
async (req: Request, res: Response) => { async (req: Request, res: Response) => {
const { message, context } = req.body as { const { message, context } = req.body as {
@@ -24,8 +36,8 @@ export const handleChatbotChat = asyncHandler(
}); });
} }
// Get user ID from auth middleware (if available) // Get user ID from X-User-Id header (no-login device uuid) or auth
const userId = (req as AuthenticatedRequest).userId || "anonymous"; const userId = resolveUserId(req);
logger.debug( logger.debug(
{ userId, messageLength: message.length, context }, { userId, messageLength: message.length, context },
@@ -59,7 +71,7 @@ export const handleChatbotChat = asyncHandler(
export const getChatbotHistory = asyncHandler( export const getChatbotHistory = asyncHandler(
async (req: Request, res: Response) => { async (req: Request, res: Response) => {
const userId = (req as AuthenticatedRequest).userId || "anonymous"; const userId = resolveUserId(req);
const limit = Math.min(parseInt(req.query.limit as string, 10) || 50, 100); const limit = Math.min(parseInt(req.query.limit as string, 10) || 50, 100);
const history = await chatbotService.getChatHistory(userId, limit); const history = await chatbotService.getChatHistory(userId, limit);
@@ -73,7 +85,7 @@ export const getChatbotHistory = asyncHandler(
export const clearChatbotHistory = asyncHandler( export const clearChatbotHistory = asyncHandler(
async (req: Request, res: Response) => { async (req: Request, res: Response) => {
const userId = (req as AuthenticatedRequest).userId || "anonymous"; const userId = resolveUserId(req);
await chatbotService.clearChatHistory(userId); await chatbotService.clearChatHistory(userId);
@@ -6,6 +6,7 @@ import type {
SaveConversationInput, SaveConversationInput,
} from "./chatbot.repository.js"; } from "./chatbot.repository.js";
import { chatbotRepository } from "./chatbot.repository.js"; import { chatbotRepository } from "./chatbot.repository.js";
import { executeTool, tools } from "./chatbot.tools.js";
const logger = createChildLogger("chatbot.service"); const logger = createChildLogger("chatbot.service");
@@ -118,41 +119,104 @@ Gaya ngobrol:
try { try {
const { default: axios } = await import("axios"); const { default: axios } = await import("axios");
// Gateway tidak handle role system — gabung konteks ke user message // Gateway tidak handle role system — gabung konteks ke user message.
// The system section stays visible to the model as the first user turn.
const contextPrefixed = `${systemPrompt}\n\nPertanyaan user: ${userMessage}`; const contextPrefixed = `${systemPrompt}\n\nPertanyaan user: ${userMessage}`;
const messages: Array<{ role: "user" | "assistant"; content: string }> = [ // Seed conversation: prior turns + current question.
...history, const messages: Array<
{ role: "user", content: contextPrefixed }, | { role: "user" | "assistant"; content: string }
]; | {
role: "assistant";
content: string | null;
tool_calls: Array<{
id: string;
type: "function";
function: { name: string; arguments: string };
}>;
}
| { role: "tool"; tool_call_id: string; content: string }
> = [...history, { role: "user", content: contextPrefixed }];
// ── Agentic tool loop ─────────────────────────────────────────
const MAX_TOOL_ROUNDS = 4;
for (let round = 0; round <= MAX_TOOL_ROUNDS; round += 1) {
const response = await axios.post( const response = await axios.post(
`${baseUrl}/chat/completions`, `${baseUrl}/chat/completions`,
{ {
model, model,
messages, messages,
max_tokens: 500, tools,
tool_choice: "auto",
max_tokens: 600,
temperature: 0.4, temperature: 0.4,
stream: true,
}, },
{ {
headers: { headers: {
Authorization: `Bearer ${apiKey}`, Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json", "Content-Type": "application/json",
}, },
timeout: 30_000, timeout: 45_000,
// 9router returns SSE even without stream:true; force stream:true
// in the body and read the raw SSE text.
responseType: "text",
}, },
); );
const result = response.data as { // Parse SSE `data:` lines → content + tool_calls.
choices?: Array<{ message?: { content?: string } }>; const { content, toolCalls } = this.parseSse(response.data as string);
};
const content = result?.choices?.[0]?.message?.content?.trim();
if (content) { logger.debug(
return content; {
round,
hasToolCalls: toolCalls.length > 0,
toolNames: toolCalls.map((t) => t.name),
},
"LLM round parsed",
);
if (toolCalls.length > 0) {
// Execute each tool, append tool results, continue loop.
for (const tc of toolCalls) {
messages.push({
role: "assistant",
content: null,
tool_calls: [
{
id: tc.id,
type: "function",
function: { name: tc.name, arguments: tc.arguments },
},
],
});
let result = "";
try {
result = await executeTool(tc.name, tc.args);
} catch (e) {
result = `Tool error: ${(e as Error).message}`;
}
messages.push({
role: "tool",
tool_call_id: tc.id,
content: result,
});
}
if (round === MAX_TOOL_ROUNDS) {
logger.warn("Hit max tool rounds; returning what we have");
}
continue;
} }
logger.warn({ response: result }, "LLM returned empty response"); if (content?.trim()) {
return content.trim();
}
logger.warn("LLM returned empty response (no tools, no content)");
return this.fallbackResponse(userMessage);
}
logger.warn("Tool loop exhausted without final content");
return this.fallbackResponse(userMessage); return this.fallbackResponse(userMessage);
} catch (error) { } catch (error) {
logger.warn({ error }, "LLM call failed, using fallback response"); logger.warn({ error }, "LLM call failed, using fallback response");
@@ -160,6 +224,93 @@ Gaya ngobrol:
} }
} }
/**
* Parse an SSE stream body into accumulated content + any tool_calls.
* 9router (and most OpenAI-compatible routers) emit `data: {json}` lines
* even when stream is only implied; we must collect deltas manually.
*/
private parseSse(body: string): {
content: string;
toolCalls: Array<{
id: string;
name: string;
arguments: string;
args: Record<string, unknown>;
}>;
} {
const contentParts: string[] = [];
const toolById = new Map<
string,
{ id: string; name: string; arguments: string }
>();
const lines = body.split("\n");
for (const rawLine of lines) {
const line = rawLine.trim();
if (!line.startsWith("data:")) continue;
const payload = line.slice(5).trim();
if (!payload || payload === "[DONE]") continue;
try {
const json = JSON.parse(payload) as {
choices?: Array<{
delta?: {
content?: string;
tool_calls?: Array<{
id?: string;
index?: number;
type?: string;
function?: { name?: string; arguments?: string };
}>;
};
finish_reason?: string | null;
}>;
};
const delta = json.choices?.[0]?.delta;
if (!delta) continue;
if (delta.content) contentParts.push(delta.content);
if (delta.tool_calls) {
for (const tc of delta.tool_calls) {
const idx = String(tc.index ?? 0);
const cur = toolById.get(idx) ?? {
id: tc.id ?? "",
name: "",
arguments: "",
};
// Keep the first non-empty id for this call index.
if (tc.id && !cur.id) cur.id = tc.id;
if (tc.function?.name) cur.name += tc.function.name;
if (tc.function?.arguments) cur.arguments += tc.function.arguments;
toolById.set(idx, cur);
}
}
} catch {
// Skip malformed lines (keepalives, etc.)
}
}
// Build a de-duplicated id for any call the stream never assigned one.
let fallbackId = 0;
const toolCalls = Array.from(toolById.values()).map((tc) => {
const id = tc.id || `tool_${fallbackId++}_${Date.now()}`;
return {
id,
name: tc.name,
arguments: tc.arguments,
args: this.safeJsonParse(tc.arguments),
};
});
return { content: contentParts.join(""), toolCalls };
}
private safeJsonParse(s: string): Record<string, unknown> {
try {
return JSON.parse(s) as Record<string, unknown>;
} catch {
return {};
}
}
private fallbackResponse(input: string): string { private fallbackResponse(input: string): string {
const lower = input.toLowerCase(); const lower = input.toLowerCase();
@@ -0,0 +1,232 @@
import { sql } from "drizzle-orm";
import { getDatabase } from "../../shared/database/index.js";
/**
* Tools the chatbot LLM can call. Definitions describe the schema to the
* model; the executor implements each one against the real database.
* This turns the chatbot from "blind stats guesser" into an agent that
* pulls real, current server data on demand.
*/
export type ToolResult = string;
/** JSON schema for a tool definition (OpenAI function-calling format). */
export interface ToolDef {
type: "function";
function: {
name: string;
description: string;
parameters: {
type: "object";
properties: Record<string, unknown>;
required?: string[];
};
};
}
export const tools: ToolDef[] = [
{
type: "function",
function: {
name: "get_server_stats",
description:
"Ambil statistik ringkas server/guild saat ini: total pesan, user aktif, jumlah pesan flagged, dan jumlah warning. Panggil ini untuk menjawab pertanyaan umum tentang kondisi server. Opsional fill guild_id untuk scope ke guild tertentu, channel_id untuk scope ke channel.",
parameters: {
type: "object",
properties: {
guildId: {
type: "string",
description: "ID guild/server (opsional). Kosongkan = semua data.",
},
channelId: {
type: "string",
description: "ID channel (opsional).",
},
},
},
},
},
{
type: "function",
function: {
name: "get_top_channels",
description:
"Ambil daftar channel paling aktif (jumlah pesan terbanyak) di server. Panggil buat jawab 'channel mana paling ramai' atau aktivitas per-channel.",
parameters: {
type: "object",
properties: {
guildId: {
type: "string",
description: "ID server (opsional).",
},
limit: {
type: "number",
description: "Jumlah channel teratas (default 5, max 10).",
},
},
},
},
},
{
type: "function",
function: {
name: "get_recent_activity",
description:
"Ambil aktivitas/pesan terbaru di server: siapa yang baru ngomong, di channel mana, jam berapa. Panggil buat jawaban soal 'lagi ngapain' / aktivitas terbaru di server.",
parameters: {
type: "object",
properties: {
guildId: {
type: "string",
description: "ID server (opsional).",
},
limit: {
type: "number",
description: "Jumlah pesan terakhir (default 5).",
},
},
},
},
},
{
type: "function",
function: {
name: "get_top_flagged",
description:
"Ambil pesan yang paling sering di-flag atau kena warning. Panggil buat jawab soal pesan bermasalah / moderator.",
parameters: {
type: "object",
properties: {
guildId: {
type: "string",
description: "ID server (opsional).",
},
limit: {
type: "number",
description: "Jumlah pesan (default 5).",
},
},
},
},
},
];
/** Executes a tool call against the real DB and returns a readable result. */
export async function executeTool(
name: string,
args: Record<string, unknown>,
): Promise<string> {
const guildId =
typeof args.guildId === "string" && args.guildId ? args.guildId : undefined;
const channelId =
typeof args.channelId === "string" && args.channelId
? args.channelId
: undefined;
const limitRaw =
typeof args.limit === "number" ? args.limit : Number(args.limit) || 5;
const limit = Math.min(Math.max(1, Math.round(limitRaw)), 10);
try {
switch (name) {
case "get_server_stats":
return await serverStats(guildId, channelId);
case "get_top_channels":
return await topChannels(guildId, limit);
case "get_recent_activity":
return await recentActivity(guildId, limit);
case "get_top_flagged":
return await topFlagged(guildId, limit);
default:
return `Unknown tool: ${name}`;
}
} catch (error) {
// Best-effort: if a tool fails, return readable error instead of crashing
return `Terjadi kesalahan saat ambil data: ${(error as Error).message ?? "unknown"}`;
}
}
// ── Tool executors ──────────────────────────────────────────
async function serverStats(
guildId?: string,
channelId?: string,
): Promise<string> {
const db = getDatabase();
const conditions: string[] = [];
if (guildId) conditions.push(`guild_id = '${guildId}'`);
if (channelId) conditions.push(`channel_id = '${channelId}'`);
const cond = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(
`SELECT COUNT(*)::int AS total_messages,
COUNT(DISTINCT user_id)::int AS active_users,
COUNT(*) FILTER (WHERE ai_status = 'flagged')::int AS flagged,
COUNT(*) FILTER (WHERE ai_status = 'warn')::int AS warned
FROM messages ${cond}`,
),
);
const rows =
(result as unknown as { rows: Record<string, unknown>[] }).rows ?? [];
const r = rows[0] ?? {};
return JSON.stringify({
total_messages: r.total_messages ?? 0,
active_users: r.active_users ?? 0,
flagged: r.flagged ?? 0,
warned: r.warned ?? 0,
});
}
async function topChannels(guildId?: string, limit = 5): Promise<string> {
const db = getDatabase();
const conditions: string[] = [];
if (guildId) conditions.push(`guild_id = '${guildId}'`);
const cond = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(
`SELECT channel_id,
COUNT(*)::int AS count
FROM messages ${cond}
GROUP BY channel_id
ORDER BY count DESC
LIMIT ${limit}`,
),
);
const rows = (result as unknown as { rows: unknown[] }).rows ?? [];
return JSON.stringify(rows.slice(0, limit));
}
async function recentActivity(guildId?: string, limit = 5): Promise<string> {
const db = getDatabase();
const conditions: string[] = [];
if (guildId) conditions.push(`guild_id = '${guildId}'`);
const cond = conditions.length ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(
`SELECT username, content, channel_id, created_at
FROM messages ${cond}
ORDER BY created_at DESC
LIMIT ${limit}`,
),
);
return JSON.stringify((result as unknown as { rows: unknown[] }).rows ?? []);
}
async function topFlagged(guildId?: string, limit = 5): Promise<string> {
const db = getDatabase();
const conditions = ["ai_status IN ('flagged', 'warn')"];
if (guildId) conditions.push(`guild_id = '${guildId}'`);
const cond = `WHERE ${conditions.join(" AND ")}`;
const result = await db.execute(
sql.raw(
`SELECT username, content, channel_id, ai_status, created_at
FROM messages ${cond}
ORDER BY created_at DESC
LIMIT ${limit}`,
),
);
return JSON.stringify((result as unknown as { rows: unknown[] }).rows ?? []);
}
@@ -331,6 +331,100 @@ export class DashboardRepository {
}; };
} }
async getTopReactions(limit: number) {
const db = getDatabase();
const cap = Math.min(Math.max(limit || 20, 1), 50);
// Top messages by net reactions (adds minus removes), joined to message content
const result = await db.execute(sql`
SELECT
m.id AS message_id,
m.content,
m.username,
m.channel_id,
m.created_at,
COALESCE(NULLIF((m.metadata::jsonb -> 'channel' ->> 'channelName'), ''), m.channel_id) AS channel_name,
r.reaction_count::int
FROM (
SELECT message_id,
(COUNT(*) FILTER (WHERE reaction_type = 'add')
- COUNT(*) FILTER (WHERE reaction_type = 'remove'))::int AS reaction_count
FROM message_reactions
GROUP BY message_id
) r
JOIN messages m ON m.id = r.message_id
WHERE r.reaction_count > 0
ORDER BY r.reaction_count DESC
LIMIT ${cap}
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
if (rows.length === 0) return [];
// Top emoji per message (adds only) for the breakdown
const ids = rows.map((r) => String(r.message_id));
const emojiResult = await db.execute(sql`
SELECT message_id, emoji, COUNT(*)::int AS c
FROM message_reactions
WHERE reaction_type = 'add' AND message_id IN (${sql.join(ids, sql`, `)})
GROUP BY message_id, emoji
ORDER BY message_id, c DESC
`);
const emojiByMessage = new Map<
string,
Array<{ emoji: string; count: number }>
>();
for (const e of emojiResult.rows as Record<string, unknown>[]) {
const mid = String(e.message_id);
const list = emojiByMessage.get(mid) ?? [];
list.push({ emoji: String(e.emoji), count: Number(e.c) });
emojiByMessage.set(mid, list);
}
return rows.map((r) => ({
message_id: String(r.message_id),
content: r.content ? String(r.content) : "",
username: r.username ? String(r.username) : null,
channel_id: String(r.channel_id),
channel_name: r.channel_name ? String(r.channel_name) : null,
created_at: r.created_at ? Number(r.created_at) : null,
reaction_count: Number(r.reaction_count),
top_emojis: (emojiByMessage.get(String(r.message_id)) ?? []).slice(0, 3),
}));
}
async getTopReactors(limit: number) {
const db = getDatabase();
const cap = Math.min(Math.max(limit || 20, 1), 50);
// Top users by net reactions given (adds minus removes)
const result = await db.execute(sql`
SELECT
user_id,
username,
(COUNT(*) FILTER (WHERE reaction_type = 'add')
- COUNT(*) FILTER (WHERE reaction_type = 'remove'))::int AS net_count,
COUNT(*) FILTER (WHERE reaction_type = 'add')::int AS adds_count,
COUNT(DISTINCT message_id)::int AS messages_reacted,
COUNT(DISTINCT emoji)::int AS emojis_used
FROM message_reactions
GROUP BY user_id, username
ORDER BY net_count DESC
LIMIT ${cap}
`);
return ((result.rows as Record<string, unknown>[]) || []).map((r) => ({
user_id: String(r.user_id),
username: String(r.username ?? "unknown"),
net_count: Number(r.net_count),
adds_count: Number(r.adds_count),
messages_reacted: Number(r.messages_reacted),
emojis_used: Number(r.emojis_used),
}));
}
async getUserDetail(userId: string) { async getUserDetail(userId: string) {
const db = getDatabase(); const db = getDatabase();
@@ -87,5 +87,25 @@ export function createDashboardRouter(): Router {
}), }),
); );
// GET /api/dashboard/reactions — top reacted messages
router.get(
"/dashboard/reactions",
asyncHandler(async (req: Request, res: Response) => {
const limit = Number(req.query.limit) || 20;
const reactions = await dashboardService.getTopReactions(limit);
res.json(reactions);
}),
);
// GET /api/dashboard/reactors — top users by reactions given
router.get(
"/dashboard/reactors",
asyncHandler(async (req: Request, res: Response) => {
const limit = Number(req.query.limit) || 20;
const reactors = await dashboardService.getTopReactors(limit);
res.json(reactors);
}),
);
return router; return router;
} }
@@ -43,6 +43,16 @@ export class DashboardService {
logger.debug({ channelId }, "Fetching channel detail"); logger.debug({ channelId }, "Fetching channel detail");
return dashboardRepository.getChannelDetail(channelId); return dashboardRepository.getChannelDetail(channelId);
} }
async getTopReactions(limit: number) {
logger.debug({ limit }, "Fetching top reactions");
return dashboardRepository.getTopReactions(limit);
}
async getTopReactors(limit: number) {
logger.debug({ limit }, "Fetching top reactors");
return dashboardRepository.getTopReactors(limit);
}
} }
export const dashboardService = new DashboardService(); export const dashboardService = new DashboardService();
@@ -2,8 +2,8 @@ import type { Request, Response, Router } from "express";
import express from "express"; import express from "express";
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
import { asyncHandler, validateBody } from "../../shared/middlewares/index.js"; import { asyncHandler, validateBody } from "../../shared/middlewares/index.js";
import { mediaQueueSchema, mediaVolumeSchema } from "./media.schema.js"; import { mediaLoopSchema, mediaQueueSchema } from "./media.schema.js";
import { getStatus, queue, setVolume, skip, stop } from "./media.service.js"; import { getStatus, queue, setLoop, skip, stop } from "./media.service.js";
const logger = createChildLogger("media.routes"); const logger = createChildLogger("media.routes");
@@ -55,14 +55,14 @@ export function createMediaRouter(): Router {
}), }),
); );
// POST /api/media/volume // POST /api/media/loop
router.post( router.post(
"/media/volume", "/media/loop",
validateBody(mediaVolumeSchema), validateBody(mediaLoopSchema),
asyncHandler(async (req: Request, res: Response) => { asyncHandler(async (req: Request, res: Response) => {
const { volume } = req.body as { volume: number }; const { loop } = req.body as { loop: boolean };
logger.debug({ volume }, "Media volume requested"); logger.debug({ loop }, "Media loop requested");
const state = await setVolume(volume); const state = await setLoop(loop);
res.json(state); res.json(state);
}), }),
); );
@@ -5,9 +5,9 @@ export const mediaQueueSchema = z.object({
mode: z.enum(["music", "screen"]).default("music"), mode: z.enum(["music", "screen"]).default("music"),
}); });
export const mediaVolumeSchema = z.object({ export const mediaLoopSchema = z.object({
volume: z.number().min(0).max(1).default(1.0), loop: z.boolean().default(false),
}); });
export type MediaQueueInput = z.infer<typeof mediaQueueSchema>; export type MediaQueueInput = z.infer<typeof mediaQueueSchema>;
export type MediaVolumeInput = z.infer<typeof mediaVolumeSchema>; export type MediaLoopInput = z.infer<typeof mediaLoopSchema>;
@@ -3,10 +3,10 @@ import {
tryCommandThenFallback, tryCommandThenFallback,
} from "../../shared/commandHelper.js"; } from "../../shared/commandHelper.js";
import { import {
COMMAND_MEDIA_LOOP,
COMMAND_MEDIA_QUEUE, COMMAND_MEDIA_QUEUE,
COMMAND_MEDIA_SKIP, COMMAND_MEDIA_SKIP,
COMMAND_MEDIA_STOP, COMMAND_MEDIA_STOP,
COMMAND_MEDIA_VOLUME,
MEDIA_STATUS_KEY, MEDIA_STATUS_KEY,
} from "../../shared/index.js"; } from "../../shared/index.js";
import { publishCommand, readRedisStatus } from "../../shared/redis/index.js"; import { publishCommand, readRedisStatus } from "../../shared/redis/index.js";
@@ -28,7 +28,10 @@ export interface MediaItem {
export interface MediaState { export interface MediaState {
playing: boolean; playing: boolean;
/** null/absent when idle; "music" | "screen" while a track is active. */
activeMode?: "music" | "screen" | null;
musicVolume: number; musicVolume: number;
loop: boolean;
current: MediaItem | null; current: MediaItem | null;
queue: MediaItem[]; queue: MediaItem[];
} }
@@ -41,7 +44,9 @@ const DEFAULT_COMMAND_TIMEOUT_MS = 5000;
const DEFAULT_STATE: MediaState = { const DEFAULT_STATE: MediaState = {
playing: false, playing: false,
musicVolume: 1.0, activeMode: null,
musicVolume: 0.3,
loop: false,
current: null, current: null,
queue: [], queue: [],
}; };
@@ -56,9 +61,14 @@ function normalizeMediaState(raw: Record<string, unknown>): MediaState {
rawPlaying === true || rawPlaying === true ||
rawPlaying === "playing" || rawPlaying === "playing" ||
rawPlaying === "buffering"; rawPlaying === "buffering";
const mode = raw.activeMode;
const activeMode: "music" | "screen" | null =
mode === "music" || mode === "screen" ? mode : null;
return { return {
playing, playing,
musicVolume: Number(raw.musicVolume ?? 1.0), activeMode,
musicVolume: Number(raw.musicVolume ?? 0.3),
loop: Boolean(raw.loop ?? false),
current: (raw.current as MediaItem | null) ?? null, current: (raw.current as MediaItem | null) ?? null,
queue: (raw.queue as MediaItem[]) ?? [], queue: (raw.queue as MediaItem[]) ?? [],
}; };
@@ -99,7 +109,9 @@ export async function queue(
() => () =>
publishCommand<MediaState>( publishCommand<MediaState>(
COMMAND_MEDIA_QUEUE, COMMAND_MEDIA_QUEUE,
{ source, mode }, // NOTE: gateway MediaHandler reads `payload.url` (not `source`) —
// keep the field name aligned or playback silently no-ops.
{ url: source, mode },
DEFAULT_COMMAND_TIMEOUT_MS, DEFAULT_COMMAND_TIMEOUT_MS,
), ),
() => readStatusFallback(), () => readStatusFallback(),
@@ -142,18 +154,18 @@ export async function stop(): Promise<MediaState> {
} }
/** /**
* Set volume via Redis command to discord-gateway. * Toggle loop mode (replay current track on natural end) via Redis command.
*/ */
export async function setVolume(volume: number): Promise<MediaState> { export async function setLoop(loop: boolean): Promise<MediaState> {
logger.info({ volume }, "setVolume called"); logger.info({ loop }, "setLoop called");
return tryCommandThenFallback( return tryCommandThenFallback(
() => () =>
publishCommand<MediaState>( publishCommand<MediaState>(
COMMAND_MEDIA_VOLUME, COMMAND_MEDIA_LOOP,
{ volume }, { loop },
DEFAULT_COMMAND_TIMEOUT_MS, DEFAULT_COMMAND_TIMEOUT_MS,
), ),
() => readStatusFallback(), () => readStatusFallback(),
"setVolume", "setLoop",
); );
} }
@@ -6,10 +6,10 @@ import {
isNull, isNull,
like, like,
lt, lt,
ne,
notInArray, notInArray,
or, or,
type SQL, type SQL,
sql,
} from "drizzle-orm"; } from "drizzle-orm";
import { config } from "../../shared/config/index.js"; import { config } from "../../shared/config/index.js";
import { getDatabase } from "../../shared/database/index.js"; import { getDatabase } from "../../shared/database/index.js";
@@ -77,12 +77,11 @@ export class MessagesRepository {
// Exclude spam threads (NULL-safe: non-thread messages are kept) // Exclude spam threads (NULL-safe: non-thread messages are kept)
if (EXCLUDED_THREAD_IDS.length > 0) { if (EXCLUDED_THREAD_IDS.length > 0) {
conditions.push( const excludeThreads = or(
or(
isNull(pgMessagesTable.thread_id), isNull(pgMessagesTable.thread_id),
notInArray(pgMessagesTable.thread_id, EXCLUDED_THREAD_IDS), notInArray(pgMessagesTable.thread_id, EXCLUDED_THREAD_IDS),
)!,
); );
if (excludeThreads) conditions.push(excludeThreads);
} }
const where = conditions.length > 0 ? and(...conditions) : undefined; const where = conditions.length > 0 ? and(...conditions) : undefined;
@@ -115,6 +114,27 @@ export class MessagesRepository {
return mapMessageRow(row as Record<string, unknown>); return mapMessageRow(row as Record<string, unknown>);
} }
/**
* Edit history for a message: previous content snapshots (newest first).
* Stored in message_edits by the gateway's message-capture module.
*/
async getEditHistory(
messageId: string,
): Promise<Array<{ old_content: string; edited_at: number }>> {
const db = getDatabase();
const result = await db.execute(sql`
SELECT old_content, edited_at
FROM message_edits
WHERE message_id = ${messageId}
ORDER BY edited_at DESC
LIMIT 50
`);
return ((result.rows as Record<string, unknown>[]) || []).map((r) => ({
old_content: String(r.old_content ?? ""),
edited_at: Number(r.edited_at ?? 0),
}));
}
async findByChannel( async findByChannel(
channelId: string, channelId: string,
query: MessageQuery, query: MessageQuery,
@@ -129,12 +149,11 @@ export class MessagesRepository {
// Exclude spam threads (NULL-safe) // Exclude spam threads (NULL-safe)
if (EXCLUDED_THREAD_IDS.length > 0) { if (EXCLUDED_THREAD_IDS.length > 0) {
conditions.push( const excludeThreads = or(
or(
isNull(pgMessagesTable.thread_id), isNull(pgMessagesTable.thread_id),
notInArray(pgMessagesTable.thread_id, EXCLUDED_THREAD_IDS), notInArray(pgMessagesTable.thread_id, EXCLUDED_THREAD_IDS),
)!,
); );
if (excludeThreads) conditions.push(excludeThreads);
} }
const rows = await db const rows = await db
@@ -223,58 +242,6 @@ export class MessagesRepository {
return mapMessageRow(row as Record<string, unknown>); return mapMessageRow(row as Record<string, unknown>);
} }
/**
* Bulk-reset ai_status from 'error' to 'pending' so the DG recovery worker
* picks them up on its next poll cycle.
*
* Accepts optional scope filters (guildId, channelId) or a list of explicit
* message IDs. Returns the count of rows that were actually updated.
*/
async reanalyzeErrorBatch(opts: {
guildId?: string;
channelId?: string;
messageIds?: string[];
}): Promise<number> {
const db = getDatabase();
const conditions: SQL[] = [eq(pgMessagesTable.ai_status, "error")];
if (opts.messageIds && opts.messageIds.length > 0) {
conditions.push(inArray(pgMessagesTable.id, opts.messageIds));
}
if (opts.guildId) {
conditions.push(eq(pgMessagesTable.guild_id, opts.guildId));
}
if (opts.channelId) {
conditions.push(eq(pgMessagesTable.channel_id, opts.channelId));
}
const result = await db
.update(pgMessagesTable)
.set({ ai_status: "pending" })
.where(and(...conditions));
const count = result.rowCount ?? 0;
logger.info({ count, ...opts }, "Batch reanalyze triggered");
return count;
}
/**
* Mark a single message for re-analysis by resetting ai_status to 'pending'.
* Skips messages already in 'pending' state to avoid write amplification.
*/
async markForReanalysis(id: string): Promise<void> {
const db = getDatabase();
await db
.update(pgMessagesTable)
.set({ ai_status: "pending" })
.where(
and(
eq(pgMessagesTable.id, id),
ne(pgMessagesTable.ai_status, "pending"),
),
);
}
/** /**
* Retrieve messages flagged for review (ai_status IN ('warn', 'flagged')). * Retrieve messages flagged for review (ai_status IN ('warn', 'flagged')).
* Optionally filtered by channelId, with configurable limit. * Optionally filtered by channelId, with configurable limit.
@@ -347,12 +314,13 @@ export class MessagesRepository {
like(pgAttachmentsTable.type, "image/%"), like(pgAttachmentsTable.type, "image/%"),
// Exclude spam threads (NULL-safe for non-thread messages) // Exclude spam threads (NULL-safe for non-thread messages)
...(EXCLUDED_THREAD_IDS.length > 0 ...(EXCLUDED_THREAD_IDS.length > 0
? [ ? (() => {
or( const excludeThreads = or(
isNull(pgAttachmentsTable.thread_id), isNull(pgAttachmentsTable.thread_id),
notInArray(pgAttachmentsTable.thread_id, EXCLUDED_THREAD_IDS), notInArray(pgAttachmentsTable.thread_id, EXCLUDED_THREAD_IDS),
)!, );
] return excludeThreads ? [excludeThreads] : [];
})()
: []), : []),
), ),
) )
@@ -1,7 +1,7 @@
import type { Request, Response, Router } from "express"; import type { Request, Response, Router } from "express";
import express from "express"; import express from "express";
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
import { asyncHandler, validateBody } from "../../shared/middlewares/index.js"; import { asyncHandler } from "../../shared/middlewares/index.js";
import { import {
handleGetAttachmentsByChannel, handleGetAttachmentsByChannel,
handleGetImageMessages, handleGetImageMessages,
@@ -9,27 +9,10 @@ import {
handleGetMessagesByChannel, handleGetMessagesByChannel,
handleListMessages, handleListMessages,
} from "./messages.controller.js"; } from "./messages.controller.js";
import { reanalyzeBatchSchema } from "./messages.schema.js";
import { messagesService } from "./messages.service.js"; import { messagesService } from "./messages.service.js";
const logger = createChildLogger("messages.routes"); const logger = createChildLogger("messages.routes");
/**
* Per-message in-flight guard for the single reanalyze endpoint.
* Prevents concurrent spam-clicks from issuing duplicate UPDATE + recovery
* worker triggers for the same message.
*/
const reanalyzeInFlight = new Set<string>();
/**
* Per-scope in-flight guard for the batch reanalyze endpoint.
* Scope key = "guildId:channelId" (empty string used for undefined parts).
* Two concurrent batch-reanalyze requests for the same scope are rejected
* with 409 so the recovery worker is not triggered multiple times for the
* same set of error messages.
*/
const reanalyzeBatchInFlight = new Set<string>();
export function createMessagesRouter(): Router { export function createMessagesRouter(): Router {
const router = express.Router(); const router = express.Router();
@@ -51,74 +34,6 @@ export function createMessagesRouter(): Router {
// (uses /detail/ prefix to avoid collision with :channelId route above) // (uses /detail/ prefix to avoid collision with :channelId route above)
router.get("/messages/detail/:id", handleGetMessageById); router.get("/messages/detail/:id", handleGetMessageById);
// POST /api/messages/reanalyze-batch — Bulk retry all errored messages
// MUST be registered BEFORE /messages/:id/reanalyze so "reanalyze-batch"
// is not captured as an :id param.
router.post(
"/messages/reanalyze-batch",
validateBody(reanalyzeBatchSchema),
asyncHandler(async (req: Request, res: Response) => {
const { guildId, channelId, messageIds } = req.body as {
guildId?: string;
channelId?: string;
messageIds?: string[];
};
// Idempotency guard: one concurrent batch-reanalyze per scope.
// Prevents two admin sessions clicking simultaneously from each
// triggering the recovery worker for the same set of messages.
const scopeKey = `${guildId ?? ""}:${channelId ?? ""}`;
if (reanalyzeBatchInFlight.has(scopeKey)) {
res
.status(409)
.json({ error: "REANALYZE_BATCH_IN_PROGRESS", scope: scopeKey });
return;
}
reanalyzeBatchInFlight.add(scopeKey);
let count = 0;
try {
count = await messagesService.reanalyzeErrorBatch({
guildId,
channelId,
messageIds,
});
} finally {
reanalyzeBatchInFlight.delete(scopeKey);
}
logger.info({ count, guildId, channelId }, "Batch reanalyze completed");
res.status(200).json({ ok: true, count });
}),
);
// POST /api/messages/:id/reanalyze - Mark single message for re-analysis
router.post(
"/messages/:id/reanalyze",
asyncHandler(async (req: Request, res: Response) => {
const id = String(req.params.id ?? "");
if (!id) {
res.status(400).json({ error: "MISSING_ID" });
return;
}
// Idempotency guard: reject concurrent duplicate requests for the same ID.
if (reanalyzeInFlight.has(id)) {
res.status(409).json({ error: "REANALYZE_IN_PROGRESS", messageId: id });
return;
}
reanalyzeInFlight.add(id);
try {
await messagesService.markForReanalysis(id);
} finally {
reanalyzeInFlight.delete(id);
}
res.status(200).json({ ok: true });
}),
);
// GET /api/review - Get flagged/warned messages for review // GET /api/review - Get flagged/warned messages for review
router.get( router.get(
"/review", "/review",
@@ -38,13 +38,6 @@ export const messageUpdateSchema = z.object({
aiConfidence: z.number().optional(), aiConfidence: z.number().optional(),
}); });
export const reanalyzeBatchSchema = z.object({
guildId: z.string().optional(),
channelId: z.string().optional(),
messageIds: z.array(z.string()).optional(),
});
export type MessageQuery = z.infer<typeof messageQuerySchema>; export type MessageQuery = z.infer<typeof messageQuerySchema>;
export type MessageCreate = z.infer<typeof messageCreateSchema>; export type MessageCreate = z.infer<typeof messageCreateSchema>;
export type MessageUpdate = z.infer<typeof messageUpdateSchema>; export type MessageUpdate = z.infer<typeof messageUpdateSchema>;
export type ReanalyzeBatchInput = z.infer<typeof reanalyzeBatchSchema>;
@@ -34,7 +34,12 @@ export class MessagesService {
throw new NotFoundError(`Message with ID ${id} not found`); throw new NotFoundError(`Message with ID ${id} not found`);
} }
return message; const editHistory = await messagesRepository.getEditHistory(id);
return {
...message,
edit_count: editHistory.length,
edit_history: editHistory,
};
} }
async getAttachmentsByChannel(channelId: string, query: MessageQuery) { async getAttachmentsByChannel(channelId: string, query: MessageQuery) {
@@ -58,15 +63,6 @@ export class MessagesService {
return messagesRepository.getImageMessages(guildId, limit); return messagesRepository.getImageMessages(guildId, limit);
} }
async markForReanalysis(id: string): Promise<void> {
if (!id) {
throw new ValidationError("message ID is required");
}
logger.debug({ id }, "Marking message for re-analysis");
await messagesRepository.markForReanalysis(id);
}
async getReviewMessages( async getReviewMessages(
channelId?: string, channelId?: string,
limit?: number, limit?: number,
@@ -74,25 +70,6 @@ export class MessagesService {
logger.debug({ channelId, limit }, "Getting review messages"); logger.debug({ channelId, limit }, "Getting review messages");
return messagesRepository.getReviewMessages(channelId, limit); return messagesRepository.getReviewMessages(channelId, limit);
} }
async reanalyzeErrorBatch(opts: {
guildId?: string;
channelId?: string;
messageIds?: string[];
}) {
if (
!opts.guildId &&
!opts.channelId &&
(!opts.messageIds || opts.messageIds.length === 0)
) {
throw new ValidationError(
"At least one of guildId, channelId, or messageIds[] is required",
);
}
logger.info(opts, "Batch reanalyzing errored messages");
return messagesRepository.reanalyzeErrorBatch(opts);
}
} }
export const messagesService = new MessagesService(); export const messagesService = new MessagesService();
@@ -0,0 +1 @@
export { createModerationRouter } from "./moderation.routes.js";
@@ -0,0 +1,141 @@
import { sql } from "drizzle-orm";
import { getDatabase } from "../../shared/database/index.js";
export interface ListModerationQuery {
status?: string;
actionType?: string;
limit?: number;
cursor?: number;
}
const ACTION_TYPES = [
"delete_message",
"mute_user",
"warn_user",
"kick_user",
"ban_user",
] as const;
const STATUSES = ["pending", "executed", "failed"] as const;
export class ModerationRepository {
async getStats() {
const db = getDatabase();
const result = await db.execute(sql`
SELECT action_type, status, COUNT(*)::int AS c
FROM moderation_actions
GROUP BY action_type, status
`);
const rows = (result.rows as Record<string, unknown>[]) || [];
let executed = 0;
let failed = 0;
let pending = 0;
const byAction: Record<
string,
{ executed: number; failed: number; pending: number }
> = {};
for (const r of rows) {
const actionType = String(r.action_type ?? "unknown");
const status = String(r.status ?? "unknown");
const count = Number(r.c ?? 0);
byAction[actionType] ??= { executed: 0, failed: 0, pending: 0 };
if (status === "executed") {
executed += count;
byAction[actionType].executed += count;
} else if (status === "failed") {
failed += count;
byAction[actionType].failed += count;
} else {
pending += count;
byAction[actionType].pending += count;
}
}
const total = executed + failed + pending;
return {
total,
executed,
failed,
pending,
failed_rate: total > 0 ? Number(((failed / total) * 100).toFixed(1)) : 0,
by_action: byAction,
};
}
async listActions(query: ListModerationQuery) {
const db = getDatabase();
const limit = Math.min(Math.max(query.limit ?? 50, 1), 200);
const conditions: string[] = [];
if (
query.status &&
(STATUSES as readonly string[]).includes(query.status)
) {
conditions.push(`a.status = '${query.status}'`);
}
if (
query.actionType &&
(ACTION_TYPES as readonly string[]).includes(query.actionType)
) {
conditions.push(`a.action_type = '${query.actionType}'`);
}
if (query.cursor) {
conditions.push(`a.created_at < ${Number(query.cursor)}`);
}
const whereClause =
conditions.length > 0 ? `WHERE ${conditions.join(" AND ")}` : "";
const result = await db.execute(
sql.raw(`
SELECT
a.id,
a.message_id,
a.user_id,
a.guild_id,
a.action_type,
a.reason,
a.executed_by,
a.status,
a.error,
a.created_at,
a.executed_at,
m.username,
LEFT(m.content, 300) AS content
FROM moderation_actions a
LEFT JOIN messages m ON m.id = a.message_id
${whereClause}
ORDER BY a.created_at DESC
LIMIT ${limit + 1}
`),
);
const rows = (result.rows as Record<string, unknown>[]) || [];
const data = rows.slice(0, limit).map((r) => ({
id: String(r.id ?? ""),
message_id: r.message_id ? String(r.message_id) : null,
user_id: r.user_id ? String(r.user_id) : null,
guild_id: String(r.guild_id ?? ""),
action_type: String(r.action_type ?? "unknown"),
reason: r.reason ? String(r.reason) : null,
executed_by: r.executed_by ? String(r.executed_by) : null,
status: String(r.status ?? "unknown"),
error: r.error ? String(r.error) : null,
created_at: r.created_at ? Number(r.created_at) : null,
executed_at: r.executed_at ? Number(r.executed_at) : null,
username: r.username ? String(r.username) : null,
content: r.content ? String(r.content) : null,
}));
const lastRow = rows[limit - 1] as Record<string, unknown> | undefined;
const nextCursor =
rows.length > limit ? String(lastRow?.created_at ?? "") : null;
return { data, nextCursor };
}
}
export const moderationRepository = new ModerationRepository();
@@ -0,0 +1,43 @@
import type { Request, Response, Router } from "express";
import express from "express";
import { createChildLogger } from "../../shared/logger/index.js";
import { asyncHandler } from "../../shared/middlewares/index.js";
import { moderationService } from "./moderation.service.js";
const logger = createChildLogger("moderation.routes");
export function createModerationRouter(): Router {
const router = express.Router();
// GET /api/moderation/stats — moderation action summary
router.get(
"/moderation/stats",
asyncHandler(async (_req: Request, res: Response) => {
const stats = await moderationService.getStats();
res.json(stats);
}),
);
// GET /api/moderation/actions — paginated moderation action log
router.get(
"/moderation/actions",
asyncHandler(async (req: Request, res: Response) => {
const limit = Number(req.query.limit) || 50;
const status = req.query.status as string | undefined;
const actionType = req.query.actionType as string | undefined;
const cursor = req.query.cursor as string | undefined;
const result = await moderationService.listActions({
limit,
status,
actionType,
cursor: cursor ? Number(cursor) : undefined,
});
logger.debug({ count: result.data.length }, "Moderation actions listed");
res.json(result);
}),
);
return router;
}
@@ -0,0 +1,21 @@
import { createChildLogger } from "../../shared/logger/index.js";
import {
type ListModerationQuery,
moderationRepository,
} from "./moderation.repository.js";
const logger = createChildLogger("moderation.service");
export class ModerationService {
async getStats() {
logger.debug("Fetching moderation stats");
return moderationRepository.getStats();
}
async listActions(query: ListModerationQuery) {
logger.debug({ query }, "Listing moderation actions");
return moderationRepository.listActions(query);
}
}
export const moderationService = new ModerationService();
@@ -20,7 +20,6 @@ export interface RecordingRow {
upload_error: string | null; upload_error: string | null;
created_at: number; created_at: number;
uploaded_at: number | null; uploaded_at: number | null;
duration_bytes: number;
} }
export interface PaginatedRecordings { export interface PaginatedRecordings {
@@ -69,7 +68,6 @@ export class RecordingsService {
upload_error: pgVoiceRecordingsTable.upload_error, upload_error: pgVoiceRecordingsTable.upload_error,
created_at: pgVoiceRecordingsTable.created_at, created_at: pgVoiceRecordingsTable.created_at,
uploaded_at: pgVoiceRecordingsTable.uploaded_at, uploaded_at: pgVoiceRecordingsTable.uploaded_at,
duration_bytes: pgVoiceRecordingsTable.size_bytes,
}) })
.from(pgVoiceRecordingsTable) .from(pgVoiceRecordingsTable)
.where(where) .where(where)
@@ -0,0 +1,83 @@
/**
* Authoritative live-voice store.
*
* Single source of truth for who is present / speaking in voice. The backend
* WebSocket server is the one relay every frontend client connects to, so it
* is the correct place to aggregate the gateway's `voice_active_user` deltas
* into a shared snapshot. A late-joining browser must be able to see the same
* state as everyone else this store makes that possible (seeded into the WS
* initial states and served via GET /api/voice/status).
*/
export interface LiveSpeaker {
userId: string;
username: string;
avatar?: string | null;
speaking: boolean;
/** Epoch ms of the most recent activity (start OR end of speech). */
lastActiveAt: number;
}
const speakers = new Map<string, LiveSpeaker>();
const MAX_SPEAKERS = 200;
/**
* Record a voice_active_user event. `speaking: true` upserts the speaker as
* active; `speaking: false` marks them inactive while keeping them for the
* activity timeline.
*/
/**
* recordSpeaker(data) apply a `voice_active_user` event. `speaking: true`
* upserts the speaker as ACTIVE; `speaking: false` marks them inactive while
* keeping them for the activity timeline.
*/
export function recordSpeaker(data: {
userId: string;
username?: string;
avatar?: string | null;
speaking: boolean;
}): void {
const { userId, speaking } = data;
const existing = speakers.get(userId);
const speaker: LiveSpeaker = {
userId,
username: data.username ?? existing?.username ?? "Unknown",
avatar: data.avatar ?? existing?.avatar ?? null,
speaking,
lastActiveAt: Date.now(),
};
if (speakers.size >= MAX_SPEAKERS && !existing) {
// Drop the least-recently-active non-speaking speaker to stay bounded.
let oldestId: string | null = null;
let oldestTs = Infinity;
for (const [id, s] of speakers) {
if (!s.speaking && s.lastActiveAt < oldestTs) {
oldestTs = s.lastActiveAt;
oldestId = id;
}
}
if (oldestId) speakers.delete(oldestId);
else return;
}
speakers.set(userId, speaker);
}
/** All known speakers, most recently active first. */
export function getActiveSpeakers(): LiveSpeaker[] {
return [...speakers.values()].sort((a, b) => b.lastActiveAt - a.lastActiveAt);
}
/** Only speakers currently flagged as speaking. */
export function getSpeakingSpeakers(): LiveSpeaker[] {
return [...speakers.values()]
.filter((s) => s.speaking)
.sort((a, b) => b.lastActiveAt - a.lastActiveAt);
}
/** Drop all tracked speakers (used on backend restart). */
export function resetLiveSpeakers(): void {
speakers.clear();
}
@@ -15,6 +15,7 @@ import {
VOICE_STATUS_KEY, VOICE_STATUS_KEY,
} from "../../shared/index.js"; } from "../../shared/index.js";
import { publishCommand, readRedisStatus } from "../../shared/redis/index.js"; import { publishCommand, readRedisStatus } from "../../shared/redis/index.js";
import { getActiveSpeakers, type LiveSpeaker } from "./live-speaker.js";
const logger = createChildLogger("voice.service"); const logger = createChildLogger("voice.service");
@@ -28,6 +29,8 @@ export interface Channel {
id: string; id: string;
name: string; name: string;
type: "voice" | "text"; type: "voice" | "text";
/** Whether the selfbot account can actually join this voice channel. */
joinable?: boolean;
} }
export interface GuildVoiceEntry { export interface GuildVoiceEntry {
@@ -43,6 +46,12 @@ export interface VoiceStatus {
activeChannelId: string | null; activeChannelId: string | null;
activeChannelName: string | null; activeChannelName: string | null;
connections: GuildVoiceEntry[]; connections: GuildVoiceEntry[];
/**
* Authoritative shared voice snapshot who is present / speaking right
* now, aggregated server-side from the gateway's `voice_active_user`
* deltas. All browsers converge on this same list.
*/
activeSpeakers: LiveSpeaker[];
} }
export const DEFAULT_VOICE_STATUS: VoiceStatus = { export const DEFAULT_VOICE_STATUS: VoiceStatus = {
@@ -51,8 +60,16 @@ export const DEFAULT_VOICE_STATUS: VoiceStatus = {
activeChannelId: null, activeChannelId: null,
activeChannelName: null, activeChannelName: null,
connections: [], connections: [],
activeSpeakers: [],
}; };
/** Attach the live speaker snapshot to any voice status payload. */
function withActiveSpeakers<T extends Partial<VoiceStatus>>(
status: T,
): T & { activeSpeakers: LiveSpeaker[] } {
return { ...status, activeSpeakers: getActiveSpeakers() };
}
/** /**
* Wraps tryCommandThenFallback with a cleaner signature for use within this module. * Wraps tryCommandThenFallback with a cleaner signature for use within this module.
* Attempts a Redis command first; on failure, falls back to the provided function. * Attempts a Redis command first; on failure, falls back to the provided function.
@@ -66,8 +83,10 @@ async function withFallback<T>(
} }
function readVoiceStatusFallback(): Promise<VoiceStatus> { function readVoiceStatusFallback(): Promise<VoiceStatus> {
return readRedisStatus(VOICE_STATUS_KEY).then( return readRedisStatus(VOICE_STATUS_KEY).then((cached) =>
(cached) => (cached as unknown as VoiceStatus) ?? DEFAULT_VOICE_STATUS, withActiveSpeakers(
(cached as unknown as VoiceStatus) ?? DEFAULT_VOICE_STATUS,
),
); );
} }
@@ -137,7 +156,9 @@ export async function getVoiceChannels(guildId: string): Promise<Channel[]> {
export async function getVoiceStatus(): Promise<VoiceStatus> { export async function getVoiceStatus(): Promise<VoiceStatus> {
logger.debug("getVoiceStatus called"); logger.debug("getVoiceStatus called");
const cached = await readRedisStatus(VOICE_STATUS_KEY); const cached = await readRedisStatus(VOICE_STATUS_KEY);
return (cached as unknown as VoiceStatus) ?? DEFAULT_VOICE_STATUS; return withActiveSpeakers(
(cached as unknown as VoiceStatus) ?? DEFAULT_VOICE_STATUS,
);
} }
/** /**
@@ -62,6 +62,9 @@ export const pgMessagesTable = pgTable(
enum: ["none", "monitor", "warn", "review", "delete", "escalate"], enum: ["none", "monitor", "warn", "review", "delete", "escalate"],
}), }),
ai_analyzed_at: pgBigint("ai_analyzed_at", { mode: "number" }), ai_analyzed_at: pgBigint("ai_analyzed_at", { mode: "number" }),
ai_analysis_duration_ms: pgBigint("ai_analysis_duration_ms", {
mode: "number",
}),
ai_error: pgText("ai_error"), ai_error: pgText("ai_error"),
}, },
(table) => ({ (table) => ({
@@ -80,6 +80,7 @@ export interface MessageRecord {
ai_confidence?: number | null; ai_confidence?: number | null;
ai_recommended_action?: AIRecommendedAction | null; ai_recommended_action?: AIRecommendedAction | null;
ai_analyzed_at?: number | null; ai_analyzed_at?: number | null;
ai_analysis_duration_ms?: number | null;
ai_error?: string | null; ai_error?: string | null;
} }
@@ -62,6 +62,7 @@ export const COMMAND_MEDIA_QUEUE = "media:queue";
export const COMMAND_MEDIA_SKIP = "media:skip"; export const COMMAND_MEDIA_SKIP = "media:skip";
export const COMMAND_MEDIA_STOP = "media:stop"; export const COMMAND_MEDIA_STOP = "media:stop";
export const COMMAND_MEDIA_VOLUME = "media:volume"; export const COMMAND_MEDIA_VOLUME = "media:volume";
export const COMMAND_MEDIA_LOOP = "media:loop";
export const COMMAND_MODERATION_ACTION = "moderation:action"; export const COMMAND_MODERATION_ACTION = "moderation:action";
export const DISCORD_VOICE_ANALYZED = "discord:voice:analyzed"; export const DISCORD_VOICE_ANALYZED = "discord:voice:analyzed";
+1 -1
View File
@@ -108,5 +108,5 @@ export async function retryWithBackoff<T>(
}); });
} }
} }
throw lastError!; throw lastError ?? new Error("Request failed after all retries");
} }
@@ -24,6 +24,7 @@ export interface MappedMessage {
ai_confidence: number | null; ai_confidence: number | null;
ai_recommended_action: string | null; ai_recommended_action: string | null;
ai_analyzed_at: number | null; ai_analyzed_at: number | null;
ai_analysis_duration_ms: number | null;
ai_error: string | null; ai_error: string | null;
is_reply: boolean | null; is_reply: boolean | null;
is_forward: boolean | null; is_forward: boolean | null;
@@ -58,6 +59,8 @@ export function mapMessageRow(row: Record<string, unknown>): MappedMessage {
ai_confidence: (row.ai_confidence as number | null) ?? null, ai_confidence: (row.ai_confidence as number | null) ?? null,
ai_recommended_action: (row.ai_recommended_action as string | null) ?? null, ai_recommended_action: (row.ai_recommended_action as string | null) ?? null,
ai_analyzed_at: (row.ai_analyzed_at as number | null) ?? null, ai_analyzed_at: (row.ai_analyzed_at as number | null) ?? null,
ai_analysis_duration_ms:
(row.ai_analysis_duration_ms as number | null) ?? null,
ai_error: (row.ai_error as string | null) ?? null, ai_error: (row.ai_error as string | null) ?? null,
is_reply: row.is_reply === null ? null : Boolean(row.is_reply), is_reply: row.is_reply === null ? null : Boolean(row.is_reply),
is_forward: row.is_forward === null ? null : Boolean(row.is_forward), is_forward: row.is_forward === null ? null : Boolean(row.is_forward),
+22
View File
@@ -1,7 +1,9 @@
import Redis from "ioredis"; import Redis from "ioredis";
import { recordSpeaker } from "../modules/voice/live-speaker.js";
import { config } from "../shared/config/index.js"; import { config } from "../shared/config/index.js";
import { import {
DISCORD_CHANNEL_TO_WS_EVENT, DISCORD_CHANNEL_TO_WS_EVENT,
DISCORD_VOICE_ACTIVE_USER,
DISCORD_VOICE_PCM, DISCORD_VOICE_PCM,
} from "../shared/index.js"; } from "../shared/index.js";
import { createChildLogger } from "../shared/logger/index.js"; import { createChildLogger } from "../shared/logger/index.js";
@@ -62,6 +64,26 @@ function handleSubscriptionMessage(channel: string, message: string): void {
} }
} }
// Aggregate live-voice state authoritatively BEFORE broadcasting.
// Every browser hears the same `voice_active_user` deltas, so the backend
// can maintain the single shared snapshot for late-joining clients.
if (channel === DISCORD_VOICE_ACTIVE_USER) {
const speaker = data as {
userId?: string;
username?: string;
avatar?: string | null;
speaking?: boolean;
};
if (speaker?.userId) {
recordSpeaker({
userId: speaker.userId,
username: speaker.username,
avatar: speaker.avatar,
speaking: Boolean(speaker.speaking),
});
}
}
logger.debug({ channel, eventType }, "Broadcasting Redis event"); logger.debug({ channel, eventType }, "Broadcasting Redis event");
broadcastEvent(eventType, data); broadcastEvent(eventType, data);
} }
+16
View File
@@ -66,6 +66,22 @@ async function sendInitialStates(ws: WebSocket): Promise<void> {
} catch (err) { } catch (err) {
logger.warn({ err }, "Failed to send initial media_state"); logger.warn({ err }, "Failed to send initial media_state");
} }
// Send initial live-voice snapshot (shared authoritative state — a browser
// joining mid-call sees the same speakers as everyone else, not an empty DB).
try {
const { getActiveSpeakers } = await import(
"../modules/voice/live-speaker.js"
);
ws.send(
JSON.stringify({
type: "voice_state",
state: { activeSpeakers: getActiveSpeakers() },
}),
);
} catch (err) {
logger.warn({ err }, "Failed to send initial voice_state");
}
} }
export function closeWebSocketServer(): void { export function closeWebSocketServer(): void {
+4 -4
View File
@@ -243,10 +243,10 @@ On SIGINT/SIGTERM/uncaughtException/unhandledRejection:
- Connect to Backend HTTP API - Connect to Backend HTTP API
- Subscribe to WebSocket events - Subscribe to WebSocket events
3. **Docker & CI/CD** 3. **Nix & CI/CD**
- Dockerfile for Discord Gateway - flake.nix package for Discord Gateway
- Docker Compose for multi-service setup - systemd services (gmw-backend, gmw-discord-gateway)
- GitHub Actions for build/deploy - GitHub Actions for build/deploy (nix copy → systemctl restart)
4. **Documentation** 4. **Documentation**
- API documentation - API documentation
+1 -1
View File
@@ -7,6 +7,6 @@ export default defineConfig({
dbCredentials: { dbCredentials: {
url: url:
process.env.DATABASE_URL || process.env.DATABASE_URL ||
"postgresql://postgres:postgres@localhost:5432/bete", "postgresql://asephs:***@100.121.180.82:6432/dcbot",
}, },
}); });
@@ -0,0 +1,9 @@
CREATE TABLE IF NOT EXISTS "term_glossary_cache" (
"term" text PRIMARY KEY NOT NULL,
"definition" text NOT NULL,
"source_url" text DEFAULT '' NOT NULL,
"resolved_at" bigint NOT NULL,
"hit_count" integer DEFAULT 0 NOT NULL
);
--> statement-breakpoint
CREATE INDEX IF NOT EXISTS "idx_term_glossary_cache_resolved_at" ON "term_glossary_cache" USING btree ("resolved_at");
@@ -99,6 +99,13 @@
"when": 1785551832190, "when": 1785551832190,
"tag": "0013_rename_mascot_chat_to_chatbot", "tag": "0013_rename_mascot_chat_to_chatbot",
"breakpoints": true "breakpoints": true
},
{
"idx": 14,
"version": "7",
"when": 1785621600000,
"tag": "0014_add_term_glossary_cache",
"breakpoints": true
} }
] ]
} }
+1 -6
View File
@@ -7,11 +7,8 @@
"pnpm": { "pnpm": {
"onlyBuiltDependencies": [ "onlyBuiltDependencies": [
"@discordjs/opus", "@discordjs/opus",
"@lng2004/node-datachannel",
"esbuild", "esbuild",
"node-av", "sharp"
"sharp",
"zeromq"
] ]
}, },
"scripts": { "scripts": {
@@ -24,7 +21,6 @@
"test": "vitest run" "test": "vitest run"
}, },
"dependencies": { "dependencies": {
"@dank074/discord-video-stream": "6.0.0",
"@discordjs/opus": "^0.10.0", "@discordjs/opus": "^0.10.0",
"@discordjs/voice": "^0.19.2", "@discordjs/voice": "^0.19.2",
"@snazzah/davey": "^0.1.11", "@snazzah/davey": "^0.1.11",
@@ -32,7 +28,6 @@
"discord.js-selfbot-v13": "^3.7.1", "discord.js-selfbot-v13": "^3.7.1",
"dotenv": "^17.4.2", "dotenv": "^17.4.2",
"drizzle-orm": "^0.45.2", "drizzle-orm": "^0.45.2",
"imghash": "^1.1.4",
"ioredis": "^5.11.0", "ioredis": "^5.11.0",
"libsodium-wrappers": "^0.8.4", "libsodium-wrappers": "^0.8.4",
"lru-cache": "^11.5.1", "lru-cache": "^11.5.1",
+36 -896
View File
File diff suppressed because it is too large Load Diff
@@ -3,6 +3,7 @@ allowBuilds:
"@lng2004/node-datachannel": true "@lng2004/node-datachannel": true
esbuild: true esbuild: true
node-av: true node-av: true
sharp: true
zeromq: true zeromq: true
# pnpm 11 requires build-script approvals here (the legacy `pnpm` field in # pnpm 11 requires build-script approvals here (the legacy `pnpm` field in
# package.json is ignored). Native voice deps need their postinstall build. # package.json is ignored). Native voice deps need their postinstall build.
+57 -5
View File
@@ -222,7 +222,10 @@ export async function initializeDiscordGateway() {
await initializeDatabase(); await initializeDatabase();
logger.info("PostgreSQL database initialized"); logger.info("PostgreSQL database initialized");
} catch (err) { } catch (err) {
logger.error({ error: err }, "Failed to initialize database"); logger.error(
{ err, errorMsg: err instanceof Error ? err.message : String(err) },
"Failed to initialize database",
);
throw new DatabaseError( throw new DatabaseError(
`Database initialization failed: ${err instanceof Error ? err.message : String(err)}`, `Database initialization failed: ${err instanceof Error ? err.message : String(err)}`,
); );
@@ -267,7 +270,10 @@ export async function initializeDiscordGateway() {
}); });
client.on("error", (err) => { client.on("error", (err) => {
logger.error({ error: err }, "Client error"); logger.error(
{ err, errorMsg: err instanceof Error ? err.message : String(err) },
"Client error",
);
}); });
process.on("SIGINT", () => { process.on("SIGINT", () => {
@@ -279,12 +285,58 @@ export async function initializeDiscordGateway() {
}); });
process.on("uncaughtException", (err) => { process.on("uncaughtException", (err) => {
logger.error({ error: err }, "Uncaught exception"); const code =
typeof (err as NodeJS.ErrnoException).code === "string"
? (err as NodeJS.ErrnoException).code
: "";
// Transient stream-teardown errors (voice stop/disconnect races, child
// process stdin closed while we still write) are NOT fatal — crashing the
// gateway on EPIPE takes the whole bot offline mid-music. Log + continue.
if (
code === "EPIPE" ||
code === "ERR_STREAM_DESTROYED" ||
code === "ERR_STREAM_WRITE_AFTER_END" ||
code === "ECONNRESET"
) {
logger.warn(
{ error: err },
"Uncaught transient stream error — continuing",
);
return;
}
logger.error(
{
err,
errorMsg: err instanceof Error ? err.message : String(err),
stack: err?.stack,
},
"Uncaught exception",
);
gracefulShutdown("uncaughtException"); gracefulShutdown("uncaughtException");
}); });
process.on("unhandledRejection", (reason, promise) => { process.on("unhandledRejection", (reason) => {
logger.error({ reason, promise }, "Unhandled rejection"); const err =
reason instanceof Error ? reason : new Error(String(reason ?? "unknown"));
const code = (err as NodeJS.ErrnoException).code ?? "";
// Same transient-teardown policy as uncaughtException: a rejection that
// fires while a stream is being torn down (EPIPE after ffmpeg stdin
// closes, write-after-destroy, socket reset) must NOT take the whole
// gateway offline. Log detail + continue. Everything else still shuts
// down so real bugs surface.
if (
code === "EPIPE" ||
code === "ERR_STREAM_DESTROYED" ||
code === "ERR_STREAM_WRITE_AFTER_END" ||
code === "ECONNRESET"
) {
logger.warn(
{ error: err },
"Unhandled rejection transient stream error — continuing",
);
return;
}
logger.error({ error: err, reason: String(reason) }, "Unhandled rejection");
gracefulShutdown("unhandledRejection"); gracefulShutdown("unhandledRejection");
}); });
@@ -21,7 +21,11 @@ import { config } from "../../shared/config/config.js";
import { initializeDatabase } from "../../shared/database/drizzle.js"; import { initializeDatabase } from "../../shared/database/drizzle.js";
import { messageStore } from "../message-capture/messageStore.js"; import { messageStore } from "../message-capture/messageStore.js";
import type { MessageRecord } from "../message-capture/types.js"; import type { MessageRecord } from "../message-capture/types.js";
import { buildConversationContext } from "./conversationContext.js"; import {
buildConversationContext,
buildLocationContext,
} from "./conversationContext.js";
import { buildConversationContextBlock } from "./moderationBuilders.js";
import { runModerationAnalysis } from "./moderationOrchestrator.js"; import { runModerationAnalysis } from "./moderationOrchestrator.js";
const logger = createChildLogger("ai-analysis-worker"); const logger = createChildLogger("ai-analysis-worker");
@@ -274,29 +278,56 @@ async function processBatch(job: {
contextBefore, contextBefore,
targets: messages, targets: messages,
maxTokens: config.AI_ANALYSIS_MAX_CONTEXT_TOKENS, maxTokens: config.AI_ANALYSIS_MAX_CONTEXT_TOKENS,
maxAgeMs: config.AI_ANALYSIS_CONTEXT_MAX_AGE_MS,
gapMs: config.AI_ANALYSIS_CONTEXT_GAP_MS,
});
const contextBlock = buildConversationContextBlock({
location: buildLocationContext(messages),
descriptor: contextLines.descriptor,
lines: contextLines.lines,
}); });
const contextText = contextLines.join("\n");
const targetIds = messages.map((m) => m.id); const allTargetIds = messages.map((m) => m.id);
const contextIds = contextBefore.map((m) => m.id); const contextIds = contextBefore.map((m) => m.id);
const attachments = await messageStore.getAttachmentsForMessages([ const attachments = await messageStore.getAttachmentsForMessages([
...targetIds, ...allTargetIds,
...contextIds, ...contextIds,
]); ]);
// Attachment-upload race guard: a message whose attachment is still being
// uploaded (upload_status='pending') must not be analyzed yet. Its
// uploaded_url is not ready, and falling back to the Discord CDN link often
// 404s (expired/purged) — which used to silently produce a text-only
// verdict ("lampiran yang gagal terbaca"). Leave those targets pending; the
// next worker cycle picks them up after the upload lands.
const pendingUploadTargetIds = new Set(
(attachments ?? [])
.filter((a) => a.upload_status === "pending")
.map((a) => a.message_id),
);
const readyMessages =
pendingUploadTargetIds.size === 0
? messages
: messages.filter((m) => !pendingUploadTargetIds.has(m.id));
if (readyMessages.length === 0) {
return { ok: true, conversationKey, rows: [] };
}
// The orchestrator handles text/media split + caching + parallel paths // The orchestrator handles text/media split + caching + parallel paths
// internally, so a 20-message batch = 1 text LLM call (+1 media call // internally, so a 20-message batch = 1 text LLM call (+1 media call
// when media is present), not N per-message calls. // when media is present), not N per-message calls.
const analysisStart = Date.now();
const moderationResult = await runModerationAnalysis({ const moderationResult = await runModerationAnalysis({
targets: messages, targets: readyMessages,
contextText, contextBlock,
attachments, attachments,
}); });
const analysisDurationMs = Date.now() - analysisStart;
const results = moderationResult.results.map((r) => const results = moderationResult.results.map((r) =>
normalizeResult( normalizeResult(
r as unknown as AnalysisResult, r as unknown as AnalysisResult,
messages.find((m) => m.id === r.messageId), readyMessages.find((m) => m.id === r.messageId),
), ),
); );
@@ -313,6 +344,7 @@ async function processBatch(job: {
confidence: result.confidence, confidence: result.confidence,
recommendedAction: result.recommendedAction, recommendedAction: result.recommendedAction,
analyzedAt: Date.now(), analyzedAt: Date.now(),
analysisDurationMs,
error: result.status === "error" ? result.analysis : null, error: result.status === "error" ? result.analysis : null,
}, },
})); }));
@@ -324,9 +356,10 @@ async function processBatch(job: {
logger.info( logger.info(
{ {
total: messages.length, total: readyMessages.length,
saved: allRows.length, saved: allRows.length,
conversationKey, conversationKey,
skippedPendingUpload: messages.length - readyMessages.length,
}, },
"LLM batch analysis complete", "LLM batch analysis complete",
); );
@@ -359,8 +392,14 @@ async function processIndividual(job: {
contextBefore, contextBefore,
targets: [message], targets: [message],
maxTokens: config.AI_ANALYSIS_MAX_CONTEXT_TOKENS, maxTokens: config.AI_ANALYSIS_MAX_CONTEXT_TOKENS,
maxAgeMs: config.AI_ANALYSIS_CONTEXT_MAX_AGE_MS,
gapMs: config.AI_ANALYSIS_CONTEXT_GAP_MS,
});
const contextBlock = buildConversationContextBlock({
location: buildLocationContext([message]),
descriptor: contextLines.descriptor,
lines: contextLines.lines,
}); });
const contextText = contextLines.join("\n");
const contextIds = contextBefore.map((m) => m.id); const contextIds = contextBefore.map((m) => m.id);
const attachments = await messageStore.getAttachmentsForMessages([ const attachments = await messageStore.getAttachmentsForMessages([
@@ -368,10 +407,21 @@ async function processIndividual(job: {
...contextIds, ...contextIds,
]); ]);
// Same attachment-upload race guard as the batch path: while the upload is
// still in-flight the uploaded_url is not ready and the Discord CDN fallback
// often 404s — analyzing now would silently produce a text-only verdict.
// Return no results so the message stays pending for the next cycle.
const uploadStillPending = (attachments ?? []).some(
(a) => a.message_id === message.id && a.upload_status === "pending",
);
if (uploadStillPending) {
return { ok: true, results: [] };
}
try { try {
const moderationResult = await runModerationAnalysis({ const moderationResult = await runModerationAnalysis({
targets: [message], targets: [message],
contextText, contextBlock,
attachments, attachments,
}); });
@@ -47,6 +47,40 @@ export function deriveRecommendedAction(msg: MessageRecord): string {
return "none"; return "none";
} }
/** Parse the flag list from a structured result or the stored column. */
export function parseModerationFlags(
message: MessageRecord,
analysisResult?: AnalysisResult,
): string[] {
const flags = analysisResult?.flags ?? null;
if (flags && flags.length > 0) return flags;
const stored = message.ai_moderation_flags;
if (!stored) return [];
try {
const parsed = JSON.parse(stored) as unknown;
return Array.isArray(parsed)
? parsed.filter((f): f is string => typeof f === "string")
: [];
} catch {
return [];
}
}
/**
* True when the ONLY violation is the member's server nickname the message
* content itself is clean. Such messages must NOT be auto-deleted; the
* correct enforcement is resetting the nickname to the default username.
* Any other flag (sara, harassment, vulgar_language, ...) keeps the normal
* delete path.
*/
export function isNicknameOnlyViolation(
message: MessageRecord,
analysisResult?: AnalysisResult,
): boolean {
const flags = parseModerationFlags(message, analysisResult);
return flags.length > 0 && flags.every((f) => f === "offensive_username");
}
/** /**
* Check whether a message qualifies for auto-deletion. * Check whether a message qualifies for auto-deletion.
* Uses the structured `analysisResult` fields when provided, falling back * Uses the structured `analysisResult` fields when provided, falling back
@@ -1,9 +1,13 @@
import type { Client, PermissionString } from "discord.js-selfbot-v13"; import type { Client, PermissionString } from "discord.js-selfbot-v13";
import { LRUCache } from "lru-cache";
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
import { config } from "../../shared/config/config.js"; import { config } from "../../shared/config/config.js";
import { messageStore } from "../message-capture/messageStore.js"; import { messageStore } from "../message-capture/messageStore.js";
import type { MessageRecord } from "../message-capture/types.js"; import type { MessageRecord } from "../message-capture/types.js";
import { isEligibleForAutoDelete } from "./autoDeleteEligibility.js"; import {
isEligibleForAutoDelete,
isNicknameOnlyViolation,
} from "./autoDeleteEligibility.js";
import { logDeletionToChannel } from "./autoDeleteLogger.js"; import { logDeletionToChannel } from "./autoDeleteLogger.js";
import { sendDeletionNotification } from "./autoDeleteNotify.js"; import { sendDeletionNotification } from "./autoDeleteNotify.js";
@@ -15,6 +19,83 @@ export interface AutoDeleteResult {
reason: string; reason: string;
} }
// Cooldown per guild:user — a nick violation fires per message, but the
// Discord PATCH is idempotent; hammering it on every message by the same
// member is wasteful and risks rate limits.
const recentNicknameResets = new LRUCache<string, number>({
max: 200,
ttl: config.AUTO_NICKNAME_RESET_COOLDOWN_MS ?? 10 * 60 * 1000,
});
export function isNicknameResetInCooldown(
guildId: string,
userId: string,
): boolean {
return recentNicknameResets.has(`${guildId}:${userId}`);
}
/**
* Resets a member's server nickname to the default (global username)
* Discord's `setNickname(null)` removes the custom nick so the member is
* shown under their default username. Non-blocking; failures are logged
* but never throw into the moderation pipeline.
*/
export async function resetOffensiveNickname(
client: Client | undefined,
guildId: string,
userId: string,
messageId: string,
): Promise<boolean> {
const cooldownKey = `${guildId}:${userId}`;
try {
if (!client?.user?.id) {
logger.warn(
{ messageId, guildId, userId },
"Nick reset skipped: client missing",
);
return false;
}
if (userId === client.user.id) {
logger.debug({ userId }, "Nick reset skipped: operator's own account");
return false;
}
if (recentNicknameResets.has(cooldownKey)) {
logger.debug({ guildId, userId }, "Nick reset skipped: cooldown active");
return false;
}
if (config.AUTO_NICKNAME_RESET_ENABLED === false) return false;
const guild = client.guilds.cache.get(guildId);
if (!guild) {
logger.warn(
{ messageId, guildId },
"Nick reset skipped: guild not found",
);
return false;
}
const member = await guild.members.fetch(userId);
// setNickname(null) = remove nickname → Discord shows global username
await member.setNickname(null, "[auto] nickname melanggar aturan server");
recentNicknameResets.set(cooldownKey, Date.now());
logger.info(
{ messageId, guildId, userId },
"Offensive nickname reset to default username",
);
return true;
} catch (error) {
logger.warn(
{
messageId,
guildId,
userId,
error: error instanceof Error ? error.message : String(error),
},
"Nick reset failed",
);
return false;
}
}
// ─── Error Handling Utilities ──────────────────────────────────────── // ─── Error Handling Utilities ────────────────────────────────────────
function getErrorCode(error: unknown): number | string | undefined { function getErrorCode(error: unknown): number | string | undefined {
@@ -107,6 +188,57 @@ export async function attemptAutoDeleteFlaggedMessage(
return { deleted: false, skipped: true, reason: "disabled" }; return { deleted: false, skipped: true, reason: "disabled" };
} }
// ── Nickname-only violation: reset nick, DO NOT delete ─────────────
// When the only flag is offensive_username (message content is clean),
// the problem is the server nickname, not the message. Enforcement is
// removing the nickname back to the default username — the message stays.
if (isNicknameOnlyViolation(message)) {
if (
!config.AUTO_DELETE_FLAGGED_DRY_RUN &&
config.AUTO_NICKNAME_RESET_ENABLED !== false
) {
const inCooldown = isNicknameResetInCooldown(
message.guild_id,
message.user_id,
);
if (!inCooldown) {
const resetOk = await resetOffensiveNickname(
client,
message.guild_id,
message.user_id,
message.id,
);
try {
await messageStore.createModerationAction({
message_id: message.id,
user_id: message.user_id,
guild_id: message.guild_id,
action_type: "reset_nickname",
reason:
"nickname melanggar aturan server (offensive_username); pesan dibiarkan",
executed_by: "auto-delete-manager",
status: resetOk ? "executed" : "failed",
error: resetOk ? null : "nickname_reset_failed",
executed_at: resetOk ? Date.now() : null,
});
} catch (error) {
logger.warn(
{
messageId: message.id,
error: error instanceof Error ? error.message : String(error),
},
"Failed to persist nickname reset action log",
);
}
}
}
logger.info(
{ messageId: message.id, userId: message.user_id },
"Nickname-only violation: message kept, nickname reset attempted",
);
return { deleted: false, skipped: true, reason: "nickname_only_violation" };
}
// ── Status gate ────────────────────────────────────────────────── // ── Status gate ──────────────────────────────────────────────────
if (message.ai_status !== "flagged" && message.ai_status !== "warn") { if (message.ai_status !== "flagged" && message.ai_status !== "warn") {
@@ -6,6 +6,7 @@ import {
} from "../message-capture/messageMetadata.js"; } from "../message-capture/messageMetadata.js";
import type { MessageRecord } from "../message-capture/types.js"; import type { MessageRecord } from "../message-capture/types.js";
import { sanitizeDiscordTokens } from "./discordTokens.js"; import { sanitizeDiscordTokens } from "./discordTokens.js";
import { escapeXml, resolveDisplayName } from "./moderationBuilders.js";
const logger = createChildLogger("conversationContext"); const logger = createChildLogger("conversationContext");
@@ -13,6 +14,26 @@ export interface ConversationContextInput {
contextBefore: MessageRecord[]; contextBefore: MessageRecord[];
targets: MessageRecord[]; targets: MessageRecord[];
maxTokens: number; maxTokens: number;
/**
* Hard age cap for context messages (ms). Messages older than this
* relative to the target are stale conversation noise and dropped.
*/
maxAgeMs?: number;
/**
* Silence threshold (ms). A gap between consecutive context messages
* larger than this means the conversation restarted older messages
* belong to a previous conversation and are dropped.
*/
gapMs?: number;
}
export interface ConversationContextResult {
/** Formatted context lines (oldest → newest, recency-gated). */
lines: string[];
/** One-line flow descriptor: status, span, dropped counts. */
descriptor: string;
/** Number of context messages dropped by the recency gates. */
dropped: number;
} }
let _encoder: ReturnType<typeof encodingForModel> | null = null; let _encoder: ReturnType<typeof encodingForModel> | null = null;
@@ -103,26 +124,143 @@ export function formatMessageForPrompt(
msg: MessageRecord, msg: MessageRecord,
label: "context" | "target", label: "context" | "target",
): string { ): string {
const content = sanitizeDiscordTokens( const content = truncateContextLine(
sanitizeDiscordTokens(
renderDiscordMentions(msg.edited_content ?? msg.content, msg.metadata), renderDiscordMentions(msg.edited_content ?? msg.content, msg.metadata),
),
); );
const timestamp = formatTimestamp(msg.created_at); const timestamp = formatTimestamp(msg.created_at);
const mediaEvidence = formatMediaEvidenceForPrompt(msg.metadata); const mediaEvidence = formatMediaEvidenceForPrompt(msg.metadata);
const mediaSuffix = mediaEvidence ? ` ${mediaEvidence}` : ""; const mediaSuffix = mediaEvidence ? ` ${mediaEvidence}` : "";
const refInfo = formatReferenceInfo(msg); const refInfo = formatReferenceInfo(msg);
return `[${label}] id=${msg.id} time=${timestamp} user=${msg.username}: ${content}${mediaSuffix}${refInfo}`; return `[${label}] id=${msg.id} time=${timestamp} user=${resolveDisplayName(msg)}: ${content}${mediaSuffix}${refInfo}`;
}
/** Max content chars per context line a single huge paste (log dump,
* copypasta) must not eat the whole conversation budget. */
const CONTEXT_LINE_CONTENT_MAX_CHARS = 1500;
/** Marker appended when a context line's content was cut. Distinct from the
* target-content marker so the model knows which side was truncated. */
export const CONTEXT_TRUNC_MARKER = "…[konteks dipotong: terlalu panjang]";
/** Cap one context message's content to CONTEXT_LINE_CONTENT_MAX_CHARS. */
export function truncateContextLine(content: string): string {
if (content.length <= CONTEXT_LINE_CONTENT_MAX_CHARS) return content;
return `${content.slice(0, CONTEXT_LINE_CONTENT_MAX_CHARS).trimEnd()}${CONTEXT_TRUNC_MARKER}`;
}
/**
* Builds a structured `<location_context .../>` element for the batch
* channel/thread name and age-restriction flags from captured message
* metadata. The LLM uses it to judge messages in the right channel context
* (e.g. a thread about a specific topic, or an age-restricted channel).
* Returns "" when no channel metadata was captured.
*/
export function buildLocationContext(targets: MessageRecord[]): string {
const target = targets[0];
if (!target?.metadata) return "";
try {
const meta = JSON.parse(target.metadata) as {
channel?: {
channelName?: string | null;
threadName?: string | null;
topic?: string | null;
nsfw?: boolean;
ageRestricted?: boolean;
nsfwLevel?: string | null;
} | null;
};
const ch = meta?.channel;
if (!ch) return "";
const attrs: string[] = [`channel_id="${escapeXml(target.channel_id)}"`];
if (ch.channelName)
attrs.push(`channel_name="${escapeXml(ch.channelName)}"`);
if (target.thread_id || ch.threadName) {
if (target.thread_id)
attrs.push(`thread_id="${escapeXml(target.thread_id)}"`);
if (ch.threadName)
attrs.push(`thread_name="${escapeXml(ch.threadName)}"`);
}
if (typeof ch.topic === "string" && ch.topic.trim().length > 0) {
const topic =
ch.topic.length > 200
? `${ch.topic.slice(0, 200).trimEnd()}`
: ch.topic;
attrs.push(`topic="${escapeXml(topic)}"`);
}
if (typeof ch.nsfw === "boolean") attrs.push(`nsfw="${ch.nsfw}"`);
if (typeof ch.ageRestricted === "boolean") {
attrs.push(`age_restricted="${ch.ageRestricted}"`);
}
return `<location_context ${attrs.join(" ")}/>`;
} catch {
return "";
}
} }
/** /**
* Builds conversation historical context without including targets. * Builds conversation historical context without including targets.
* Calculates how much token budget targets use, and fills the rest with context. *
* Two recency gates decide whether a conversation is STILL the same one
* ("obrolan berlanjut") or already restarted:
* - `gapMs`: a silence longer than this between two context messages cuts
* the block there earlier messages belong to a previous conversation.
* - `maxAgeMs`: anything older than this relative to the target is noise.
*
* On a cold start (no recent context), the nearest messages are kept as a
* sparse anchor and the descriptor says `cold_start` instead of `ongoing`,
* so the LLM does not mistake scattered old messages for an active chat.
*/ */
export function buildConversationContext( export function buildConversationContext(
input: ConversationContextInput, input: ConversationContextInput,
): string[] { ): ConversationContextResult {
const { contextBefore, targets, maxTokens } = input; const { contextBefore, targets, maxTokens } = input;
const maxAgeMs = input.maxAgeMs ?? 45 * 60 * 1000;
const gapMs = input.gapMs ?? 12 * 60 * 1000;
// Calculate tokens used by targets (parallel) const targetTime = targets.reduce(
(min, t) => Math.min(min, t.created_at),
targets[0]?.created_at ?? Date.now(),
);
// ── Recency gating (walk newest → oldest) ───────────────────────────────
const gated: MessageRecord[] = [];
let latestSelected: MessageRecord | null = null;
let gapBeforeMs: number | null = null;
let dropped = 0;
for (let i = contextBefore.length - 1; i >= 0; i--) {
const msg = contextBefore[i];
// Age gate
if (targetTime - msg.created_at > maxAgeMs) {
dropped += i + 1; // everything older also exceeds the age cap
break;
}
// Gap gate — silence between this message and the newer one already selected
if (latestSelected && latestSelected.created_at - msg.created_at > gapMs) {
gapBeforeMs = latestSelected.created_at - msg.created_at;
dropped += i + 1;
break;
}
gated.push(msg);
latestSelected = msg;
}
const gatedNewestFirst = gated.reverse();
let status: "ongoing" | "cold_start" | "sparse";
if (gatedNewestFirst.length === 0) {
// Cold start — keep a small anchor of the nearest messages so the LLM
// still senses the channel, but mark it clearly.
status = "cold_start";
gatedNewestFirst.push(...contextBefore.slice(-2)); // ± 2 nearest to target
} else if (gapBeforeMs === null) {
status = "ongoing";
} else {
status = "sparse";
}
// ── Format + token budget (most recent first, like before) ─────────────
const targetLines = targets.map((msg) => const targetLines = targets.map((msg) =>
formatMessageForPrompt(msg, "target"), formatMessageForPrompt(msg, "target"),
); );
@@ -131,7 +269,7 @@ export function buildConversationContext(
0, 0,
); );
const contextLines = contextBefore.map((msg) => const contextLines = gatedNewestFirst.map((msg) =>
formatMessageForPrompt(msg, "context"), formatMessageForPrompt(msg, "context"),
); );
const selectedContextLines: string[] = []; const selectedContextLines: string[] = [];
@@ -148,14 +286,26 @@ export function buildConversationContext(
} }
} }
const descriptorParts = [
`[conversation_flow] status=${status}`,
`context_msgs=${selectedContextLines.length}`,
`dropped=${dropped}`,
];
if (gapBeforeMs !== null) {
descriptorParts.push(`gap_before_min=${Math.round(gapBeforeMs / 60000)}`);
}
const descriptor = descriptorParts.join(" ");
logger.debug( logger.debug(
{ {
targetCount: targets.length, targetCount: targets.length,
contextCount: selectedContextLines.length, contextCount: selectedContextLines.length,
status,
dropped,
usedTokens, usedTokens,
maxTokens, maxTokens,
}, },
"Conversation context built", "Conversation context built",
); );
return selectedContextLines; return { lines: selectedContextLines, descriptor, dropped };
} }
@@ -56,7 +56,16 @@ export async function withLlmConcurrency<T>(fn: () => Promise<T>): Promise<T> {
*/ */
type LLMResponseChunk = { type LLMResponseChunk = {
choices?: Array<{ choices?: Array<{
delta?: { content?: string | null }; delta?: {
content?: string | null;
reasoning_content?: string | null;
reasoning?: string | null;
reasoning_details?: Array<{
type?: string;
text?: string;
index?: number;
}> | null;
};
message?: { content?: string | null }; message?: { content?: string | null };
finish_reason?: string | null; finish_reason?: string | null;
text?: string; text?: string;
@@ -67,6 +76,39 @@ type LLMResponseChunk = {
finish_reason?: string; finish_reason?: string;
}; };
/**
* Extract the textual payload from a single streaming chunk. Prefers
* `delta.content`; falls back to reasoning fields so reasoning-only models
* still produce usable aggregated text. Providers differ in the field name:
* - DeepSeek-style / Cloudflare gemma `delta.reasoning_content`
* - mimo (via 9router) streams reasoning in `delta.reasoning` +
* `delta.reasoning_details[].text` (content:"") without these fallbacks
* vision aggregation came back empty ("Vision API null response").
* Exported for unit tests.
*/
export function extractChunkText(
chunk: LLMResponseChunk | null | undefined,
): string {
if (!chunk) return "";
const choice = chunk.choices?.[0];
const reasoningDetails = choice?.delta?.reasoning_details
?.map((d) => d.text ?? "")
.filter(Boolean)
.join("");
return (
choice?.delta?.content ||
choice?.delta?.reasoning_content ||
choice?.delta?.reasoning ||
reasoningDetails ||
choice?.message?.content ||
choice?.text ||
chunk?.message?.content ||
chunk?.response ||
chunk?.content ||
""
);
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Lazy singleton — created on first use so that config is always resolved. // Lazy singleton — created on first use so that config is always resolved.
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -105,6 +147,11 @@ export interface LlmCallOpts {
top_p?: number; top_p?: number;
/** Force JSON output via response_format: { type: "json_object" }. */ /** Force JSON output via response_format: { type: "json_object" }. */
jsonResponse?: { type: "json_object" }; jsonResponse?: { type: "json_object" };
/**
* Disable LLM chain-of-thought (reasoning/thinking) for faster analysis.
* Defaults to config.AI_LLM_DISABLE_THINKING when omitted.
*/
disableThinking?: boolean;
/** Extra retries beyond DEFAULT_RETRIES (default 2). */ /** Extra retries beyond DEFAULT_RETRIES (default 2). */
retries?: number; retries?: number;
/** Whether to use streaming (if true, will consume stream and return aggregated result) */ /** Whether to use streaming (if true, will consume stream and return aggregated result) */
@@ -113,6 +160,61 @@ export interface LlmCallOpts {
signal?: AbortSignal; signal?: AbortSignal;
} }
/**
* Build the request params for an LLM chat completion. Pulled out of `llmChat`
* so the thinking-disable injection can be unit-tested without network access.
*
* Optional params (temperature/top_p/max_tokens) are only attached when
* explicitly provided, to maximise compatibility with various providers/local
* APIs. When `disableThinking` is set, we inject the common provider params
* used to switch OFF chain-of-thought reasoning. OpenAI-compatible routers
* ignore the variants their backend does not understand, so sending the
* OpenAI (`reasoning_effort`), OpenRouter (`reasoning.enabled`) and
* vLLM/Qwen/litellm (`chat_template_kwargs.enable_thinking`) forms together
* covers the popular reasoning backends behind a proxy.
*/
export function buildLlmParams(
opts: LlmCallOpts,
disableThinking: boolean,
): OpenAI.Chat.Completions.ChatCompletionCreateParams {
const {
messages,
model = config.AI_LLM_MODEL,
max_tokens,
temperature,
top_p,
jsonResponse,
stream,
} = opts;
const params = {
model,
messages,
...(stream !== undefined ? { stream } : {}),
} as OpenAI.Chat.Completions.ChatCompletionCreateParams;
if (temperature !== undefined) params.temperature = temperature;
if (top_p !== undefined) params.top_p = top_p;
if (max_tokens !== undefined) params.max_tokens = max_tokens;
if (jsonResponse) params.response_format = jsonResponse;
if (disableThinking) {
Object.assign(params, {
// OpenAI o-series
reasoning_effort: "none",
// OpenRouter
reasoning: { enabled: false },
// vLLM / Qwen / litellm
chat_template_kwargs: { enable_thinking: false },
// Anthropic / Claude-format (9router exposes thinkingFormat
// "claude-adaptive" / "claude-budget" on its reasoning models)
thinking: { type: "disabled" },
} as Record<string, unknown>);
}
return params;
}
/** /**
* Call the LLM with sensible defaults: concurrency cap, retry, model, tokens. * Call the LLM with sensible defaults: concurrency cap, retry, model, tokens.
* *
@@ -125,33 +227,12 @@ export async function llmChat(
const client = getClient(); const client = getClient();
if (!client) return null; if (!client) return null;
const { const { retries = DEFAULT_RETRIES, signal } = opts;
messages, const disableThinking =
model = config.AI_LLM_MODEL, opts.disableThinking ?? config.AI_LLM_DISABLE_THINKING;
max_tokens,
temperature,
top_p,
jsonResponse,
retries = DEFAULT_RETRIES,
stream,
signal,
} = opts;
const params = { const params = buildLlmParams(opts, disableThinking);
model, const model = params.model;
messages,
...(stream !== undefined ? { stream } : {}),
} as OpenAI.Chat.Completions.ChatCompletionCreateParams;
// Attach optional parameters only if explicitly provided to maintain
// maximum compatibility with various LLM providers and local APIs.
if (temperature !== undefined) params.temperature = temperature;
if (top_p !== undefined) params.top_p = top_p;
if (max_tokens !== undefined) params.max_tokens = max_tokens;
if (jsonResponse) {
params.response_format = jsonResponse;
}
return retryWithBackoff( return retryWithBackoff(
async () => { async () => {
@@ -167,15 +248,7 @@ export async function llmChat(
let finishReason = "stop"; let finishReason = "stop";
for await (const chunk of response as unknown as AsyncIterable<LLMResponseChunk>) { for await (const chunk of response as unknown as AsyncIterable<LLMResponseChunk>) {
const choice = chunk?.choices?.[0]; const choice = chunk?.choices?.[0];
const textChunk = content += extractChunkText(chunk);
choice?.delta?.content ||
choice?.message?.content ||
choice?.text ||
chunk?.message?.content ||
chunk?.response ||
chunk?.content ||
"";
content += textChunk;
const fr = choice?.finish_reason || chunk?.finish_reason; const fr = choice?.finish_reason || chunk?.finish_reason;
if (fr) finishReason = fr; if (fr) finishReason = fr;
} }
@@ -249,6 +322,12 @@ export async function llmChat(
* Convenience for vision (image/sticker/emoji) analysis. * Convenience for vision (image/sticker/emoji) analysis.
* Returns the raw completion content (trimmed) or null. * Returns the raw completion content (trimmed) or null.
* *
* Vision routes through the SAME router/base URL as text moderation
* (AI_LLM_BASE_URL) the dedicated NVIDIA multimodal endpoint was removed.
* It uses AI_LLM_VISION_MODEL (a different model alias from the text combo)
* so image analysis stays on a vision-capable model. Thinking-disable from
* config.AI_LLM_DISABLE_THINKING applies automatically via buildLlmParams.
*
* NOTE: retries are disabled here on purpose visionAnalyzer.ts already * NOTE: retries are disabled here on purpose visionAnalyzer.ts already
* wraps this call in its own 3-attempt loop with exponential backoff. * wraps this call in its own 3-attempt loop with exponential backoff.
* A second retry layer would multiply worst-case API calls (3×3=9/image). * A second retry layer would multiply worst-case API calls (3×3=9/image).
@@ -6,7 +6,6 @@
*/ */
export { export {
acquireMediaAnalysisLock, acquireMediaAnalysisLock,
computeImagePhash,
deleteCachedMediaAnalysis, deleteCachedMediaAnalysis,
getCachedMediaAnalysis, getCachedMediaAnalysis,
setCachedMediaAnalysis, setCachedMediaAnalysis,
@@ -16,8 +16,10 @@ import { getChannelCulture } from "./channelCultureStore.js";
import type { RetryState } from "./llmCaller.js"; import type { RetryState } from "./llmCaller.js";
import { callModerationLLM } from "./llmCaller.js"; import { callModerationLLM } from "./llmCaller.js";
import { prepareMediaMessage } from "./mediaAnalysisClient.js"; import { prepareMediaMessage } from "./mediaAnalysisClient.js";
import { buildUserProfilesBlock } from "./moderationBuilders.js";
import { buildSystemPrompt as buildSystemPromptModular } from "./moderationPrompt.js"; import { buildSystemPrompt as buildSystemPromptModular } from "./moderationPrompt.js";
import { buildCorrectedFewShotExamples } from "./textBatchProcessor.js"; import { buildCorrectedFewShotExamples } from "./textBatchProcessor.js";
import { getUserProfile } from "./userProfileStore.js";
const log = createChildLogger("mediaBatchProcessor"); const log = createChildLogger("mediaBatchProcessor");
@@ -26,7 +28,7 @@ const log = createChildLogger("mediaBatchProcessor");
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export async function runMediaBatch( export async function runMediaBatch(
targets: MessageRecord[], targets: MessageRecord[],
contextText: string, contextBlock: string,
attachments: AttachmentRecord[] | undefined, attachments: AttachmentRecord[] | undefined,
): Promise<{ results: AnalysisResult[]; raw: unknown }> { ): Promise<{ results: AnalysisResult[]; raw: unknown }> {
if (!targets.length) return { results: [], raw: null }; if (!targets.length) return { results: [], raw: null };
@@ -58,14 +60,41 @@ export async function runMediaBatch(
const channelCulture = channelCultureObj?.culture_summary; const channelCulture = channelCultureObj?.culture_summary;
const correctedExamples = await buildCorrectedFewShotExamples(); const correctedExamples = await buildCorrectedFewShotExamples();
const systemText = buildSystemPromptModular({ const systemText = buildSystemPromptModular({
contextText,
mode: "mixed", mode: "mixed",
correctedExamples, correctedExamples,
channelCulture, channelCulture,
}); });
// Gather user profiles ONCE for the whole batch and emit a deduplicated
// <user_profiles> map (with last-generated timestamp); per-message blocks
// (from prepareMediaMessage) reference it via <user_profile_ref>.
const profileByUser = new Map<
string,
{
text: string;
asOf?: number | null;
}
>();
for (const t of targets) {
if (profileByUser.has(t.user_id)) continue;
const profile = await getUserProfile(t.user_id);
profileByUser.set(t.user_id, {
text: profile?.profile_summary ?? "",
asOf: profile?.last_analyzed_at ?? null,
});
}
const userProfilesBlock = buildUserProfilesBlock(profileByUser);
const messagesBlock = prepared.map((p) => p.messageBlock).join("\n"); const messagesBlock = prepared.map((p) => p.messageBlock).join("\n");
const userContent = `<messages_to_analyze>\n${messagesBlock}\n</messages_to_analyze>`; // Data/instruction separation: the system prompt is stable per mode — all
// per-batch context (profiles, conversation) lives in the USER payload,
// ordered oldest-first so targets come last.
const userBlocks = [
userProfilesBlock?.trimEnd() ?? "",
contextBlock?.trimEnd() ?? "",
`<messages_to_analyze>\n${messagesBlock}\n</messages_to_analyze>`,
].filter((b) => b.trim().length > 0);
const userContent = userBlocks.join("\n\n");
const perMsgTimeout = config.AI_LLM_MEDIA_ANALYSIS_TIMEOUT_MS ?? 60000; const perMsgTimeout = config.AI_LLM_MEDIA_ANALYSIS_TIMEOUT_MS ?? 60000;
const batchTimeout = Math.min( const batchTimeout = Math.min(
@@ -7,28 +7,22 @@
import { LRUCache } from "lru-cache"; import { LRUCache } from "lru-cache";
import { import {
acquireMediaAnalysisLock, acquireMediaAnalysisLock,
computeImagePhash,
deleteCachedMediaAnalysis, deleteCachedMediaAnalysis,
getCachedMediaAnalysis, getCachedMediaAnalysis,
getCachedMediaByPhash,
makeCustomEmojiCacheKey, makeCustomEmojiCacheKey,
makeImageCacheKey, makeImageCacheKey,
makeStickerCacheKey, makeStickerCacheKey,
upsertCachedMediaAnalysis, upsertCachedMediaAnalysis,
upsertCachedMediaByPhash,
} from "./textCacheStore.js"; } from "./textCacheStore.js";
export { export {
acquireMediaAnalysisLock, acquireMediaAnalysisLock,
computeImagePhash,
deleteCachedMediaAnalysis, deleteCachedMediaAnalysis,
getCachedMediaAnalysis, getCachedMediaAnalysis,
getCachedMediaByPhash,
makeCustomEmojiCacheKey, makeCustomEmojiCacheKey,
makeImageCacheKey, makeImageCacheKey,
makeStickerCacheKey, makeStickerCacheKey,
upsertCachedMediaAnalysis, upsertCachedMediaAnalysis,
upsertCachedMediaByPhash,
}; };
/** Convenience alias for upsertCachedMediaAnalysis. */ /** Convenience alias for upsertCachedMediaAnalysis. */
@@ -296,13 +296,37 @@ export async function downloadAndExtractFrame(
imageMap: Map<string, MessageImagePart[]>, imageMap: Map<string, MessageImagePart[]>,
): Promise<void> { ): Promise<void> {
const log = createChildLogger("mediaAnalysis"); const log = createChildLogger("mediaAnalysis");
const urlToUse = att.uploaded_url ?? att.discord_url ?? null; // Prefer the upload proxy (uploaded_url); the Discord CDN link can expire
if (!urlToUse) return; // or be purged (404), and a non-OK response used to silently drop the image
// from vision analysis (no log, empty image map → text-only verdict). Try
// each candidate URL in order and surface failures.
const urlCandidates = [
att.uploaded_url,
att.discord_url && att.discord_url !== att.uploaded_url
? att.discord_url
: null,
].filter((u): u is string => Boolean(u));
if (urlCandidates.length === 0) return;
let imageBytes: Buffer | null = null;
let lastStatus = 0;
let lastError: string | null = null;
for (const urlToUse of urlCandidates) {
const { controller, clear } = createAbortControllerWithTimeout(15000); const { controller, clear } = createAbortControllerWithTimeout(15000);
try { try {
const res = await fetch(urlToUse, { signal: controller.signal }); const res = await fetch(urlToUse, { signal: controller.signal });
if (!res.ok || !res.body) return; if (!res.ok || !res.body) {
lastStatus = res.status;
log.warn(
{
attachmentId: att.id,
urlHost: new URL(urlToUse).host,
status: res.status,
},
"Attachment fetch non-OK — trying next URL",
);
continue;
}
let totalBytes = 0; let totalBytes = 0;
const chunks: Uint8Array[] = []; const chunks: Uint8Array[] = [];
@@ -319,17 +343,40 @@ export async function downloadAndExtractFrame(
chunks.push(value); chunks.push(value);
} }
} }
const imageBytes = Buffer.concat(chunks); imageBytes = Buffer.concat(chunks);
break;
} catch (err) {
lastError = err instanceof Error ? err.message : String(err);
log.warn(
{
attachmentId: att.id,
urlHost: new URL(urlToUse).host,
error: lastError,
},
"Attachment download failed — trying next URL",
);
} finally {
clear();
}
}
if (!imageBytes) {
log.warn(
{
attachmentId: att.id,
filename: att.filename,
lastStatus,
lastError,
},
"All attachment URLs failed — skipping media analysis",
);
return;
}
const sniffedMime = sniffImageMimeType(imageBytes); const sniffedMime = sniffImageMimeType(imageBytes);
if (!sniffedMime && att.type.startsWith("video/")) { if (!sniffedMime && att.type.startsWith("video/")) {
await extractVideoFrames( await extractVideoFrames(att, imageBytes, targetId, maxDimension, imageMap);
att,
imageBytes,
targetId,
maxDimension,
imageMap,
);
return; return;
} }
@@ -380,17 +427,6 @@ export async function downloadAndExtractFrame(
image_url: { url: dataUrl }, image_url: { url: dataUrl },
sourceLabel: `[gambar di atas adalah attachment ${att.filename} dari pesan id=${att.message_id}]`, sourceLabel: `[gambar di atas adalah attachment ${att.filename} dari pesan id=${att.message_id}]`,
}); });
} catch (err) {
log.warn(
{
attachmentId: att.id,
error: err instanceof Error ? err.message : String(err),
},
"Download failed",
);
} finally {
clear();
}
} }
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -414,7 +450,7 @@ export async function downloadMediaCandidate(
if (candidate.customEmojiId || candidate.stickerName) { if (candidate.customEmojiId || candidate.stickerName) {
const vck = candidate.customEmojiId const vck = candidate.customEmojiId
? makeCustomEmojiCacheKey(candidate.customEmojiId) ? makeCustomEmojiCacheKey(candidate.customEmojiId)
: makeStickerCacheKey(candidate.stickerName!); : makeStickerCacheKey(candidate.stickerName ?? "");
const cached = await getCachedMediaAnalysis(vck); const cached = await getCachedMediaAnalysis(vck);
if (cached) { if (cached) {
const existing = mediaAnalysisMap.get(targetId) ?? []; const existing = mediaAnalysisMap.get(targetId) ?? [];
@@ -494,8 +530,9 @@ export async function fetchUrlInline(
sourceLabel: `[gambar dari URL ${url} (inline), pesan id=${targetId}]`, sourceLabel: `[gambar dari URL ${url} (inline), pesan id=${targetId}]`,
}); });
} else if (result.type === "text" && result.textContent) { } else if (result.type === "text" && result.textContent) {
const titleAttr = result.title ? ` title="${escapeXml(result.title)}"` : "";
webTexts.push( webTexts.push(
`<web_content url="${escapeXml(url)}">${escapeXml(result.textContent.slice(0, 2000))}</web_content>`, `<web_content url="${escapeXml(url)}"${titleAttr}>${escapeXml(result.textContent.slice(0, 2000))}</web_content>`,
); );
} }
} }
@@ -9,6 +9,7 @@ import { renderDiscordMentions } from "../message-capture/messageMetadata.js";
import { messageStore } from "../message-capture/messageStore.js"; import { messageStore } from "../message-capture/messageStore.js";
import type { MessageRecord } from "../message-capture/types.js"; import type { MessageRecord } from "../message-capture/types.js";
import { sanitizeDiscordTokens } from "./discordTokens.js"; import { sanitizeDiscordTokens } from "./discordTokens.js";
import { sanitizeAiContent } from "./prompts/output.js";
/** Simple XML-escaping for content text. */ /** Simple XML-escaping for content text. */
export function escapeXml(s: string): string { export function escapeXml(s: string): string {
@@ -19,6 +20,216 @@ export function escapeXml(s: string): string {
.replace(/"/g, "&quot;"); .replace(/"/g, "&quot;");
} }
// ---------------------------------------------------------------------------
// Conversation context block — structured data for the USER message.
//
// All per-batch context lives in the USER message (not the SYSTEM prompt) so
// the system prompt is stable per mode (cacheable on routers/providers) and
// the role boundary is clean: instructions in SYSTEM, data in USER.
// ---------------------------------------------------------------------------
/** Outer char cap for the assembled `<conversation_context>` inner text. */
export const CONVERSATION_CONTEXT_MAX_CHARS = 40_000;
/**
* Wraps per-batch context data into structured XML blocks for the USER
* message:
*
* <location_context channel_id="..." channel_name="..." nsfw="..."/>
* <conversation_context>
* [conversation_flow] status=ongoing context_msgs=12 dropped=0
* [context] id=... time=... user=...: isi pesan
* ...
* </conversation_context>
*
* Empty blocks are omitted entirely (never emit a hollow `<conversation_context>`
* with no content). The inner text is AI/user-derived and passed through
* `sanitizeAiContent` (CDATA + XML-escape) to block prompt injection.
*/
export function buildConversationContextBlock(input: {
/** Pre-built `<location_context .../>` string (or ""). */
location?: string;
/** `[conversation_flow]` descriptor line from buildConversationContext. */
descriptor?: string;
/** `[context]` lines, oldest → newest. */
lines: string[];
}): string {
const blocks: string[] = [];
const location = input.location?.trim();
if (location) blocks.push(location);
const inner = [input.descriptor ?? "", ...input.lines]
.map((line) => line.trim())
.filter((line) => line.length > 0)
.join("\n");
if (inner) {
blocks.push(
`<conversation_context>\n${sanitizeAiContent(inner, CONVERSATION_CONTEXT_MAX_CHARS)}\n</conversation_context>`,
);
}
return blocks.join("\n");
}
// ---------------------------------------------------------------------------
// Per-message content bounds — protects the LLM token budget from a single
// huge paste (stack traces, log dumps, copypasta). Truncation is explicit so
// the model never mistakes the cut for a real message boundary.
// ---------------------------------------------------------------------------
/** Max characters of a message's content sent to the LLM `<content>` payload. */
export const AI_CONTENT_MAX_CHARS = 4000;
/** Marker appended when a message is longer than AI_CONTENT_MAX_CHARS. */
export const AI_CONTENT_TRUNC_MARKER = "\n…[pesan dipotong: terlalu panjang]";
/** Truncate a message's content for the LLM `<content>` payload. */
export function truncateForAi(content: string): string {
if (content.length <= AI_CONTENT_MAX_CHARS) return content;
return `${content.slice(0, AI_CONTENT_MAX_CHARS)}${AI_CONTENT_TRUNC_MARKER}`;
}
// ---------------------------------------------------------------------------
// User profile deduplication — a batch can contain many messages from the
// same user. Instead of repeating the (up to 3000-char) profile summary on
// every message, emit a single <user_profiles> map per batch and reference
// entries per message with <user_profile_ref user_id="..."/>.
// ---------------------------------------------------------------------------
export interface UserProfileEntry {
/** Profile summary text (from user_profiles.profile_summary). */
text: string;
/** Epoch ms when the profile was last generated staleness signal for
* the LLM (a profile from months ago may not reflect current behavior). */
asOf?: number | null;
}
/** Build a deduplicated `<user_profiles>` map block, keyed by Discord user id. */
export function buildUserProfilesBlock(
profiles: ReadonlyMap<string, UserProfileEntry>,
): string {
const entries = Array.from(profiles.entries()).filter(
([, entry]) => entry.text.trim().length > 0,
);
if (entries.length === 0) return "";
const lines = entries.map(([userId, entry]) => {
const asOfAttr =
typeof entry.asOf === "number" && entry.asOf > 0
? ` as_of="${new Date(entry.asOf).toISOString()}"`
: "";
return ` <user_profile user_id="${escapeXml(userId)}"${asOfAttr}>${sanitizeAiContent(entry.text)}</user_profile>`;
});
return `<user_profiles>\n${lines.join("\n")}\n</user_profiles>`;
}
/** Per-message reference tag pointing at an entry in the `<user_profiles>` map. */
export function buildUserProfileRef(userId: string): string {
return `<user_profile_ref user_id="${escapeXml(userId)}"/>`;
}
// ---------------------------------------------------------------------------
// User reputation — richer than a bare trust score.
//
// The trust model tracks total_infractions, a clean-message streak and the
// last infraction timestamp. Feeding all of it to the LLM lets it tell a
// first-timer (same score, 1 infraction) from a repeat offender (score 50,
// 3 infractions, last one yesterday) — the same score means very different
// things in those two contexts.
// ---------------------------------------------------------------------------
export interface ReputationAttrsSource {
trust_score: number;
total_infractions: number;
clean_message_streak: number;
last_infraction_at: number | null;
}
const DAY_MS = 24 * 60 * 60 * 1000;
const REPEAT_OFFENSE_WINDOW_MS = 7 * DAY_MS;
/**
* Formats reputation fields into XML attributes for `<user_reputation .../>`.
* Derived signals: last_offense_days_ago (0 = today) and repeat_offender
* (infraction within the last 7 days) are computed here so both the text and
* media paths emit the exact same shape.
*/
export function formatReputationAttrs(
rep: ReputationAttrsSource,
now: number = Date.now(),
): string {
const attrs = [
`trust_score="${rep.trust_score}"`,
`total_infractions="${rep.total_infractions}"`,
`clean_streak="${rep.clean_message_streak}"`,
];
if (
typeof rep.last_infraction_at === "number" &&
rep.last_infraction_at > 0
) {
const daysAgo = Math.max(
0,
Math.floor((now - rep.last_infraction_at) / DAY_MS),
);
attrs.push(`last_offense_days_ago="${daysAgo}"`);
const isRepeat =
rep.total_infractions > 0 &&
now - rep.last_infraction_at <= REPEAT_OFFENSE_WINDOW_MS;
if (isRepeat) attrs.push(`repeat_offender="true"`);
}
return attrs.join(" ");
}
/**
* Builds an optional `<user_history>` block (last flagged messages) from
* getUserRecentInfractions rows. Only emitted when there is real history
* lets the LLM see the PATTERN (e.g. the same scam link posted repeatedly)
* without treating old flags as proof for the current message.
*/
export function buildUserHistoryXml(
history: Array<{
content: string;
severity: string | null;
created_at: number;
}>,
now: number = Date.now(),
): string {
const filtered = history.filter((h) => h.content?.trim());
if (filtered.length === 0) return "";
const lines = filtered.map((h) => {
const daysAgo = Math.max(0, Math.floor((now - h.created_at) / DAY_MS));
const severityAttr = h.severity
? ` severity="${escapeXml(h.severity)}"`
: "";
const snippet =
h.content.length > 100
? `${h.content.slice(0, 100).trimEnd()}`
: h.content;
return ` <infraction${severityAttr} time_ago_days="${daysAgo}">${escapeXml(snippet)}</infraction>`;
});
return `<user_history>\n${lines.join("\n")}\n</user_history>`;
}
/**
* Whether the message author was a bot (captured in metadata.author.bot).
* Bot posts (logging bots, webhook-style automation) deserve different
* scrutiny than user posts expose the flag instead of hiding it.
*/
export function resolveIsBot(msg: MessageRecord): boolean {
if (!msg.metadata) return false;
try {
const meta = JSON.parse(msg.metadata) as {
author?: { bot?: boolean } | null;
};
return Boolean(meta?.author?.bot);
} catch {
return false;
}
}
/** Whether the shown content is an EDIT of the original post (evasion signal). */
export function resolveIsEdited(msg: MessageRecord): boolean {
return Boolean(msg.edited_content);
}
/** /**
* Returns the real text content for AI analysis, stripping fallback text * Returns the real text content for AI analysis, stripping fallback text
* that getDisplayContent() synthesized ("[Attachment: ...]", "[Sticker: ...]", * that getDisplayContent() synthesized ("[Attachment: ...]", "[Sticker: ...]",
@@ -36,6 +247,27 @@ export function getAnalysisContent(message: MessageRecord): string {
).trim(); ).trim();
} }
/**
* Server nickname (member.displayName) when captured, else the author
* username. Discord shows the server nickname to other members, so the LLM
* should see the same name the channel sees and a nickname can carry
* moderation signal itself (offensive nick + clean message low warn).
*/
export function resolveDisplayName(msg: MessageRecord): string {
if (msg.metadata) {
try {
const meta = JSON.parse(msg.metadata) as {
member?: { displayName?: string | null } | null;
};
const dn = meta?.member?.displayName;
if (dn && dn.trim().length > 0) return dn;
} catch {
// malformed metadata — fall back to username
}
}
return msg.username;
}
/** /**
* Builds a <reference> XML element for reply/forward/crosspost context. * Builds a <reference> XML element for reply/forward/crosspost context.
*/ */
@@ -35,7 +35,13 @@ const log = createChildLogger("moderationOrchestrator");
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export interface ModerationInput { export interface ModerationInput {
targets: MessageRecord[]; targets: MessageRecord[];
contextText: string; /**
* Pre-built XML context block for the USER message (from
* `buildConversationContextBlock`): `<location_context .../>` +
* `<conversation_context>...</conversation_context>`. Kept out of the
* system prompt so it stays stable/cacheable per mode.
*/
contextBlock: string;
attachments?: AttachmentRecord[]; attachments?: AttachmentRecord[];
} }
@@ -62,7 +68,7 @@ export interface ModerationOutput {
export async function runModerationAnalysis( export async function runModerationAnalysis(
input: ModerationInput, input: ModerationInput,
): Promise<ModerationOutput> { ): Promise<ModerationOutput> {
const { targets, contextText, attachments } = input; const { targets, contextBlock, attachments } = input;
initSearxngCache(config.REDIS_URL); initSearxngCache(config.REDIS_URL);
if (!targets.length) throw new Error("No targets provided for analysis"); if (!targets.length) throw new Error("No targets provided for analysis");
@@ -188,7 +194,7 @@ export async function runModerationAnalysis(
if (embeddings && embeddings.length === texts.length) { if (embeddings && embeddings.length === texts.length) {
// index-aligned with semanticCandidates // index-aligned with semanticCandidates
for (let i = 0; i < semanticCandidates.length; i++) { for (let i = 0; i < semanticCandidates.length; i++) {
const { target, cacheKey } = semanticCandidates[i]; const { cacheKey } = semanticCandidates[i];
embeddingsByKey.set(cacheKey, embeddings[i]); embeddingsByKey.set(cacheKey, embeddings[i]);
} }
@@ -320,10 +326,10 @@ export async function runModerationAnalysis(
// Run both paths in parallel // Run both paths in parallel
const [textBatchResult, mediaBatchResult] = await Promise.all([ const [textBatchResult, mediaBatchResult] = await Promise.all([
textOnlyTargets.length > 0 textOnlyTargets.length > 0
? runTextOnlyBatch(textOnlyTargets, contextText) ? runTextOnlyBatch(textOnlyTargets, contextBlock)
: Promise.resolve({ results: [] as AnalysisResult[], raw: null }), : Promise.resolve({ results: [] as AnalysisResult[], raw: null }),
mediaTargets.length > 0 mediaTargets.length > 0
? runMediaBatch(mediaTargets, contextText, attachments) ? runMediaBatch(mediaTargets, contextBlock, attachments)
: Promise.resolve({ results: [] as AnalysisResult[], raw: null }), : Promise.resolve({ results: [] as AnalysisResult[], raw: null }),
]); ]);
@@ -335,6 +335,26 @@ export const ALL_EXAMPLES: ExampleDef[] = [
'{"results":[{"message_id":"31313","status":"flagged","flags":["conflict_instigation","sara"],"score":0.95,"severity":"critical","confidence":0.95,"recommended_action":"delete","evidence":["gw sih dukung palestina"],"analysis":"Segala diskusi Israel/Palestina/Yahudi dilarang total — tidak ada debat, dukungan, atau berita. Dihapus."}]}', '{"results":[{"message_id":"31313","status":"flagged","flags":["conflict_instigation","sara"],"score":0.95,"severity":"critical","confidence":0.95,"recommended_action":"delete","evidence":["gw sih dukung palestina"],"analysis":"Segala diskusi Israel/Palestina/Yahudi dilarang total — tidak ada debat, dukungan, atau berita. Dihapus."}]}',
modes: ["text", "media", "mixed"], modes: ["text", "media", "mixed"],
}, },
// ── Physics / Technology Discussions (false positive prevention) ──
{
id: "32",
title: "Diskusi fisika/kinetik dalam konteks teknis (AMAN, bukan ancaman)",
input:
"[target] id=32323 user=physics_student: Cukup cuman tubuh manusia vs gravitasi. Konsep energy conservation di sini penting buat analisis statis.",
output:
'{"results":[{"message_id":"32323","status":"clean","flags":[],"score":0.0,"severity":"none","confidence":0.95,"recommended_action":"none","evidence":[],"analysis":"Diskusi fisika teknis tentang kinetik dan gravitasi dalam konteks analisis statis tidak ada ancaman atau konten melanggar. Penggunaan istilah fisika untuk perhitungan teknis adalah hal wajar."}]}',
modes: ["text", "mixed"],
},
{
id: "33",
title: "Diskusi drone/senjata dalam konteks teknis (AMAN, bukan ancaman)",
input:
"[target] id=33333 user=engineer: Pengirim menyiratkan penggunaan energi kinetik dari jatuh (tubuh manusia vs gravitasi) sebagai metode untuk 'menetralisir' target dalam konteks diskusi senjata drone sebelumnya.",
output:
'{"results":[{"message_id":"33333","status":"clean","flags":[],"score":0.0,"severity":"none","confidence":0.9,"recommended_action":"none","evidence":[],"analysis":"Diskusi teknis tentang drone dan aplikasi fisika dalam konteks engineering tidak ada ajuan aksi atau ancaman nyata. Penggunaan istilah senjata dalam konteks diskusi teori adalah hal wajar."}]}',
modes: ["text", "mixed"],
},
]; ];
// Derive per-mode strings from the single ALL_EXAMPLES array (zero duplication) // Derive per-mode strings from the single ALL_EXAMPLES array (zero duplication)
@@ -31,11 +31,16 @@ Struktur wajib:
] ]
} }
Instruksi per field:
- "message_id": WAJIB sama persis dengan id di input. Setiap <message> di <messages_to_analyze> menghasilkan SATU hasil. Jangan gabungkan beberapa pesan, jangan lewati, jangan karang id.
- "evidence": kutipan PERSIS frasa yang melanggar (maks 1 baris). Pelanggaran di gambar/sticker kutip deskripsi Media analysis. Pelanggaran lewat balasan/referensi sebut konteks pesan yang dibalas. Boleh tambah label sumber, mis. [media analysis] / [web_search] / [reply]. Kosong jika clean.
## PERSONALITY & MEMORI Profil Pengguna dan Kultur Channel ## PERSONALITY & MEMORI Profil Pengguna dan Kultur Channel
Data konteks tersedia: <user_profile> (ringkasan kepribadian pengguna) dan <channel_culture> (topik/vibe channel). Data konteks tersedia: <user_profiles> (peta ringkasan kepribadian, di pesan USER), <user_reputation> (skor trust), dan <channel_culture> (topik/vibe channel). Setiap <message> dapat memuat <user_profile_ref user_id="..."/> yang menunjuk ke entri di peta <user_profiles>.
Gunakan untuk personalisasi analysis, tapi: Gunakan untuk personalisasi analysis, tapi:
- Profil adalah KONTEKS, bukan bukti. Profil mencurigakan flag; profil bersih loloskan pelanggaran. - Profil adalah KONTEKS, bukan bukti. Profil mencurigakan flag; profil bersih loloskan pelanggaran.
- Perubahan perilaku mencolok (biasanya teknis tiba-tiba provokatif) layak dicatat di analysis. - Perubahan perilaku mencolok (biasanya teknis tiba-tiba provokatif) layak dicatat di analysis.
- <user_history> (kutipan pesan yang pernah di-flag) = pola pelanggaran lama. Gunakan untuk mendeteksi PENGULANGAN KEKONSISTEN (spam link yang SAMA, provokasi yang MENGULANG KONTEN NYATA YANG SAMA). JANGAN pernah gunakan history untuk "menginterpretasi ulang" pesan bersih yang TERPISAH DARI riwayat. Setiap pesan BARU dinilai TERSAMBUNG (standalone). Jika tidak ada pola pengulangan yang jelas CLEAN. Contoh: Jika sebelumnya ada pesan dengan link scam.example.com yang di-flag, dan pesan baru juga ada link scam.example.com FLAG. Tapi jika pesan baru tentang "energi kinetik dari jatuh" tanpa link yang sama CLEAN walaupun ada history lain.
- JANGAN paksa referensi profil jika tidak relevan analysis natural lebih baik. - JANGAN paksa referensi profil jika tidak relevan analysis natural lebih baik.
- Channel culture coding/teknis pesan teknis lebih wajar; channel santai slang lebih wajar. Jangan dipakai mengabaikan pelanggaran nyata. - Channel culture coding/teknis pesan teknis lebih wajar; channel santai slang lebih wajar. Jangan dipakai mengabaikan pelanggaran nyata.
@@ -54,6 +59,7 @@ Contoh buruk: "Pesan berisi teks dan gambar tanpa pelanggaran." (mengabaikan buk
- **conflict_instigation:** "Pengirim <ajakan memicu konflik>. <konteks>. Diberi peringatan karena berpotensi memicu drama." - **conflict_instigation:** "Pengirim <ajakan memicu konflik>. <konteks>. Diberi peringatan karena berpotensi memicu drama."
- **Username ofensif (pesan bersih):** "Pengirim memiliki username yang <alasan ofensif>. Isi pesan hanya <isi>. Diberi warning ringan." (pesan memperkuat): "<username SARA> + isi pesan memperkuat tone kebencian. Pelanggaran berat." - **Username ofensif (pesan bersih):** "Pengirim memiliki username yang <alasan ofensif>. Isi pesan hanya <isi>. Diberi warning ringan." (pesan memperkuat): "<username SARA> + isi pesan memperkuat tone kebencian. Pelanggaran berat."
- **Evasi (zalgo/leetspeak):** "Pengirim menggunakan teknik obfuscation untuk menyembunyikan <makna asli>. <dampak>. <kesimpulan>." - **Evasi (zalgo/leetspeak):** "Pengirim menggunakan teknik obfuscation untuk menyembunyikan <makna asli>. <dampak>. <kesimpulan>."
- **Spam (repetitions > 1):** "Pengirim mengirim teks yang sama sebanyak N kali dalam waktu singkat. <isi pesan>. Diberi peringatan karena spam berulang." nilai tetap dari isi; pengulangan saja (mis. "ok" x5 dalam obrolan aktif) bukan pelanggaran.
- **sexual_deviation:** "Pengirim <konten penyimpangan>. <konteks>. Melanggar kebijakan server." - **sexual_deviation:** "Pengirim <konten penyimpangan>. <konteks>. Melanggar kebijakan server."
- **SARA/penistaan agama:** "Pengirim <jenis penistaan spesifik: parodi ayat, mengaku Tuhan, mockery ritual, istilah agama sebagai joke, provokasi antar-agama>. <bukti>. Melanggar kebijakan SARA." JANGAN gunakan kata "bercanda" untuk SARA. - **SARA/penistaan agama:** "Pengirim <jenis penistaan spesifik: parodi ayat, mengaku Tuhan, mockery ritual, istilah agama sebagai joke, provokasi antar-agama>. <bukti>. Melanggar kebijakan SARA." JANGAN gunakan kata "bercanda" untuk SARA.
@@ -66,7 +72,7 @@ CRITICAL:
- Jika pesan adalah BALASAN (reply) ke pesan lain, jelaskan konteks balasannya: apa yang sedang dibicarakan, siapa yang dibalas (tanpa nama, cukup peran/isi pesan yang dibalas), dan bagaimana tanggapan pengirim terhadapnya. - Jika pesan adalah BALASAN (reply) ke pesan lain, jelaskan konteks balasannya: apa yang sedang dibicarakan, siapa yang dibalas (tanpa nama, cukup peran/isi pesan yang dibalas), dan bagaimana tanggapan pengirim terhadapnya.
- Gunakan informasi dari Media analysis untuk mendeskripsikan gambar. - Gunakan informasi dari Media analysis untuk mendeskripsikan gambar.
- Analisis harus MEMBERI KONTEKS, bukan hanya menyatakan status. - Analisis harus MEMBERI KONTEKS, bukan hanya menyatakan status.
- GUNAKAN <user_profile> untuk personalisasi analysis jadikan analysis terasa seperti sistem "mengenal" pengguna. - GUNAKAN <user_profile_ref>/<user_profiles> untuk personalisasi analysis jadikan analysis terasa seperti sistem "mengenal" pengguna.
- Jika perilaku pesan menyimpang dari profil yang diketahui, CATAT dalam analysis sebagai informasi kontekstual yang relevan. - Jika perilaku pesan menyimpang dari profil yang diketahui, CATAT dalam analysis sebagai informasi kontekstual yang relevan.
- JANGAN paksa referensi profil jika tidak relevan analysis natural lebih baik dari yang dipaksakan.`; - JANGAN paksa referensi profil jika tidak relevan analysis natural lebih baik dari yang dipaksakan.`;
@@ -12,6 +12,7 @@ export const SYSTEM_RULES = `Kamu adalah asisten moderasi konten untuk server Di
## Normalisasi & Pertahanan Lintas Bahasa (WAJIB) ## Normalisasi & Pertahanan Lintas Bahasa (WAJIB)
1. Campuran bahasa (Inggris/Indonesia/daerah) WAJIB dinormalisasi mental ke Bahasa Indonesia sebelum menilai intent. Jangan longgar hanya karena sintaksis campur (Polyglot Obfuscation). 1. Campuran bahasa (Inggris/Indonesia/daerah) WAJIB dinormalisasi mental ke Bahasa Indonesia sebelum menilai intent. Jangan longgar hanya karena sintaksis campur (Polyglot Obfuscation).
2. Lakukan Named Entity Recognition agresif nama orang/karakter (mis. "ren" setelah kata archaic "diagem") tetap dikenali sebagai nama. 2. Lakukan Named Entity Recognition agresif nama orang/karakter (mis. "ren" setelah kata archaic "diagem") tetap dikenali sebagai nama.
3. <term_glossary> (bila ada) = definisi kata/slang/jargon yang tidak umum. Baca dulu arti kata yang tidak kamu kenal dari sana jangan menebak dari bunyi/kemiripan. Kata yang tampak mencurigakan namun ternyata bermakna netral di glossary = AMAN; kata asing yang ternyata vulgar/terlarang di glossary = FLAG.
## Aturan Umum (AMAN jangan flag) ## Aturan Umum (AMAN jangan flag)
- Slang: anjay, wkwk, gws, gaskeun, santuy, njir, baka, woy/woi, hadeh, astaga = AMAN. - Slang: anjay, wkwk, gws, gaskeun, santuy, njir, baka, woy/woi, hadeh, astaga = AMAN.
@@ -29,15 +30,19 @@ export const SYSTEM_RULES = `Kamu adalah asisten moderasi konten untuk server Di
- Ekspresi religius (Astaghfirullah, Alhamdulillah, Subhanallah, Allahuakbar, MasyaAllah, Bismillah, InsyaAllah, Laa ilaha illallah + varian all-caps) = DOA NORMAL, bukan vulgar. AMAN. - Ekspresi religius (Astaghfirullah, Alhamdulillah, Subhanallah, Allahuakbar, MasyaAllah, Bismillah, InsyaAllah, Laa ilaha illallah + varian all-caps) = DOA NORMAL, bukan vulgar. AMAN.
- Discord custom emoji (<:hadeh:123>) = ekspresi, bukan pelanggaran teks. - Discord custom emoji (<:hadeh:123>) = ekspresi, bukan pelanggaran teks.
- Makian pada entitas eksternal (game, dev, perusahaan, benda mati: "game ini ampas") = AMAN. Harassment/hate_speech HANYA untuk anggota/kelompok server secara personal. - Makian pada entitas eksternal (game, dev, perusahaan, benda mati: "game ini ampas") = AMAN. Harassment/hate_speech HANYA untuk anggota/kelompok server secara personal.
- **Diskusi fisika, teknik, atau engineering dalam konteks teknis** (kinetik, gravitasi, energi, drone, senjata, drone warfare, physics simulations, CAD, CNC, 3D printing, robotics, aerospace, aerodynamika) = AMAN. Penggunaan istilah teknis untuk perhitungan atau analisis bukan ancaman. JANGAN flag hanya karena istilah "senjata" atau "drone" dalam konteks diskusi teori teknis. Flag HANYA jika ada ajuan aksi eksplisit atau ancaman nyata terarah.
- **Riwayat pengguna dengan pelanggaran sebelumnya** tidak boleh memengaruhi penilaian pesan bersih yang TERPISAH dan tidak mengandung pelanggaran aktual. Setiap pesan dinilai berdasarkan ISINYA SENDIRI.
## Zero Tolerance Vulgaritas Anatomi/Seksual ## Zero Tolerance Vulgaritas Anatomi/Seksual
Kata alat kelamin/anatomi seksual (kontol, memek, titten, tit, dick) atau istilah seksual eksplisit WAJIB di-flag sebagai vulgar_language/sexual_content TANPA pengecualian bercanda, slang, atau "santai". Kata alat kelamin/anatomi seksual (kontol, memek, titten, tit, dick) atau istilah seksual eksplisit WAJIB di-flag sebagai vulgar_language/sexual_content TANPA pengecualian bercanda, slang, atau "santai".
## Nilai Server Diskriminasi ## Nilai Server Diskriminasi
-Ketika sesuatu yang melanggar terjadi di channel, flag jika relevan. Setiap pesan dinilai BERDASARKAN ISINYA SENDIRI, bukan sekadar histori pengguna.
-Seksisme ("dasar perempuan", "logika cewek") hate_speech (umum) / harassment (terarah). -Seksisme ("dasar perempuan", "logika cewek") hate_speech (umum) / harassment (terarah).
-Ageisme ("dasar bocil", "tau aja lo tua") hate_speech / harassment. -Ageisme ("dasar bocil", "tau aja lo tua") hate_speech / harassment.
-Diskriminasi fisik ("gendut", "iteman", "cungkring") harassment jika terarah. -Diskriminasi fisik ("gendut", "iteman", "cungkring") harassment jika terarah.
-Serangan personal, penghinaan, merendahkan = tidak ditoleransi. Perbedaan pendapat wajar. -Serangan personal, penghinaan, merendahkan = tidak ditoleransi. Perbedaan pendapat wajar.
+**PESAN DINILAI SECARA STANDALONE:** Setiap pesan baru dinilai BERDASARKAN ISINYA SENDIRI. <user_history> (jika ada) HANYA untuk mendeteksi POLA PENGULANGAN dengan JAMAK (spam link yang SAMA, provokasi berulang yang MENGANDALKAN KONTEN YANG SAMA). JANGAN gunakan history untuk "menginterpretasi ulang" pesan bersih yang TERPISAH DARI riwayat pelanggaran sebelumnya. Jika pesan tidak mengandung unsur yang BERPANDUAN PADA riwayat tetap CLEAN.
## LARANGAN BERAT (ZERO TOLERANCE) ## LARANGAN BERAT (ZERO TOLERANCE)
- **LGBT:** Segala promosi, diskusi, pengakuan orientasi, coming out, atau curhat personal tentang LGBT WAJIB di-flag "sexual_deviation". Tidak ada pengecualian. - **LGBT:** Segala promosi, diskusi, pengakuan orientasi, coming out, atau curhat personal tentang LGBT WAJIB di-flag "sexual_deviation". Tidak ada pengecualian.
@@ -73,6 +78,7 @@ RENDAH: harassment, vulgar_language terarah, offensive_username (Scunthorpe: "Sa
## Web Sebagai Bukti Utama ## Web Sebagai Bukti Utama
- <web_searches> ADALAH BUKTI UTAMA. Jika ada, WAJIB pakai hasilnya (hentai/scam/narkoba flag; aman clean). JANGAN abaikan. Jika tidak ada gunakan pengetahuan internal. - <web_searches> ADALAH BUKTI UTAMA. Jika ada, WAJIB pakai hasilnya (hentai/scam/narkoba flag; aman clean). JANGAN abaikan. Jika tidak ada gunakan pengetahuan internal.
- <term_glossary> = REFERENSI ARTI KATA, bukan bukti pelanggaran. Dipakai untuk memahami istilah yang tidak dikenal sebelum memutuskan.
- Prioritas bukti: <web_searches> > <web_content> > <media_analysis> > pengetahuan internal. <web_content> (URL fetch): gunakan isi, jangan flag hanya dari domain name. - Prioritas bukti: <web_searches> > <web_content> > <media_analysis> > pengetahuan internal. <web_content> (URL fetch): gunakan isi, jangan flag hanya dari domain name.
## Pohon Keputusan ## Pohon Keputusan
@@ -39,7 +39,6 @@ Gambar/sticker/embed/preview link sudah DIDESKRIPSIKAN vision model sebelum batc
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export interface BuildSystemPromptOptions { export interface BuildSystemPromptOptions {
contextText: string;
/** Prompt mode — determines which sections are included. */ /** Prompt mode — determines which sections are included. */
mode: PromptMode; mode: PromptMode;
/** @deprecated Use `mode` instead. */ /** @deprecated Use `mode` instead. */
@@ -59,7 +58,6 @@ export interface BuildSystemPromptOptions {
export function buildSystemPrompt(options: BuildSystemPromptOptions): string { export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
const { const {
contextText,
mode, mode,
includeMediaInstructions, includeMediaInstructions,
correction, correction,
@@ -105,15 +103,40 @@ export function buildSystemPrompt(options: BuildSystemPromptOptions): string {
} }
parts.push( parts.push(
`## Konteks Pengguna\nSetiap pesan mungkin memiliki tag <user_reputation>. Tag ini hanya indikator **referensi**, bukan bukti pelanggaran. Nilai trust_score yang rendah bukan alasan untuk memflag pesan yang bersih. Nilai trust_score yang tinggi bukan alasan untuk mengabaikan pelanggaran nyata. **Setiap pesan harus dinilai berdasarkan isinya sendiri.**`, `## Blok Data di Pesan USER\n` +
`Semua data dinamis per-batch dikirim di pesan USER — system prompt ini TIDAK memuat data batch:\n` +
`- <location_context .../> = metadata channel/thread (channel_id, channel_name, thread_name, topic, nsfw, age_restricted). topic = deskripsi resmi channel — pakai untuk menilai kesesuaian pesan dengan tujuan channel.\n` +
`- <conversation_context> = obrolan SEBELUM pesan target. Baris "[context]" di dalamnya BUKAN yang dinilai.\n` +
`- <user_profiles> = peta ringkasan kepribadian per user_id (attr as_of = kapan profil terakhir dibuat — profil lama mungkin tidak mencerminkan perilaku terkini); setiap <message> merujuk lewat <user_profile_ref user_id="..."/>.\n` +
`- <web_searches> / <web_content> = bukti web (lihat "Web Sebagai Bukti Utama").\n` +
`- <term_glossary> = kamus istilah: definisi kata/slang/jargon yang jarang dikenal (hasil pencarian Wikipedia via SearXNG). Gunakan untuk memahami arti kata yang tidak kamu kenal — JANGAN menebak atau mengarang arti.\n` +
`- <messages_to_analyze> = pesan-pesan TARGET yang WAJIB dinilai. Atribut <message>: id, user (nama server), time (ISO — kapan pesan dikirim), repetitions (N = teks pendek sama muncul N kali di batch — sinyal spam), bot (true jika dari bot), edited (true jika konten adalah hasil edit setelah posting).`,
);
parts.push(
`## Konteks Pengguna (Referensi, Bukan Bukti)\n` +
`Konteks per pengguna hanya indikator **referensi** untuk personalisasi analisis, BUKAN bukti pelanggaran:\n` +
`- <user_reputation trust_score="..." total_infractions="..." clean_streak="..." last_offense_days_ago="..." repeat_offender="..."> = histori moderasi pengguna. Skor rendah BUKAN alasan memflag pesan bersih; skor tinggi BUKAN alasan mengabaikan pelanggaran nyata. repeat_offender="true" = ada pelanggaran dalam 7 hari terakhir.\n` +
`- <user_history> (di dalam <user_reputation>) = kutipan pesan-pesan pengguna yang PERNAH di-flag. Gunakan untuk mengenali POLA berulang (spam link sama, provokasi), tapi JANGAN memflag pesan bersih hanya karena riwayat.\n` +
`- <user_profiles> (di pesan USER) = peta ringkasan kepribadian per user_id. <user_profile_ref user_id="..."/> dalam sebuah pesan menunjuk ke peta itu. Tanpa ref = tidak ada profil untuk pengguna tersebut.\n` +
`- Profil berguna untuk mengenali penyimpangan perilaku mencolok (mis. pengguna teknis tiba-tiba provokatif), tapi JANGAN memflag atau meloloskan hanya karena profil.\n` +
`**Setiap pesan dinilai berdasarkan isinya sendiri.**`,
);
parts.push(
`## Framing: Konteks vs Target\n` +
`- Baris dalam <conversation_context> berformat "[context] id=... time=<ISO> user=<nama>: isi", diurutkan paling lama → paling baru. Baris pertama biasanya "[conversation_flow] status=... context_msgs=... dropped=..." — metadata sistem tentang status percakapan (ongoing/sparse/cold_start), BUKAN pesan yang dinilai.\n` +
`- <messages_to_analyze> berisi pesan-pesan TARGET yang WAJIB dinilai. Hasilkan SATU hasil per message_id — jangan menggabungkan beberapa pesan, jangan melewati, jangan mengarang id.\n` +
`- Setiap target dinilai berdasarkan isinya sendiri; konteks percakapan memengaruhi interpretasi, bukan menggantikan isi pesan.\n` +
`- Marker "…[pesan dipotong: terlalu panjang]" = konten TARGET sengaja dipotong; marker "…[konteks dipotong: terlalu panjang]" = konten pesan KONTEKS dipotong. Nilai dari bagian yang terlihat; pemotongan BUKAN pelanggaran dan BUKAN teknik evasi.\n` +
`- Atribut time= pada <message> target = kapan pesan dikirim (ISO). Pakai untuk menilai kerelevanan waktu (mis. pesan lama di-bump, spam beruntun dalam menit yang sama).\n` +
`- repetitions="N" pada <message> = teks pendek yang sama muncul N kali dalam batch — pertimbangkan sebagai sinyal spam, tapi nilai tetap dari isi pesan.\n` +
`- bot="true" = pengirim adalah bot (otomatisasi), bukan pengguna manusia — jangan perlakukan sebagai pelanggaran personal, tapi kontennya tetap dinilai.\n` +
`- edited="true" = konten yang ditampilkan adalah hasil edit setelah posting (sinyal potensi evasi), nilai konten saat ini apa adanya.`,
); );
parts.push(OUTPUT_INSTRUCTIONS); parts.push(OUTPUT_INSTRUCTIONS);
// XML-delimited context — prevents prompt injection
const delimitedContext = `<conversation_context>\n${sanitizeAiContent(contextText, 8000)}\n</conversation_context>`;
parts.push(delimitedContext);
let base = parts.join("\n\n"); let base = parts.join("\n\n");
if (correction) { if (correction) {
@@ -17,6 +17,18 @@ import { config } from "../../shared/config/config.js";
const log = createChildLogger("qdrant"); const log = createChildLogger("qdrant");
// ensureQdrantCollection performs a network round-trip (GET, possibly
// DELETE+PUT). Running it on every upsert adds 1-3 HTTP calls per
// moderation verdict, which under Qdrant load pushes the upsert past the
// request timeout and aborts it ("This operation was aborted"). Memoise the
// result so the collection is only verified once per process lifetime.
let ensureCollectionPromise: Promise<boolean> | null = null;
/** Reset the memoised ensure result (used by tests / config reload). */
export function resetQdrantCollectionCache(): void {
ensureCollectionPromise = null;
}
export interface QdrantVerdictPayload { export interface QdrantVerdictPayload {
text: string; text: string;
flags: string; // JSON string of the full moderation result flags: string; // JSON string of the full moderation result
@@ -97,13 +109,18 @@ export function qdrantPointId(cacheKey: string): number {
export async function ensureQdrantCollection( export async function ensureQdrantCollection(
vectorSize: number, vectorSize: number,
): Promise<boolean> { ): Promise<boolean> {
if (ensureCollectionPromise) return ensureCollectionPromise;
ensureCollectionPromise = (async () => {
try { try {
// 404 = collection doesn't exist yet → create it. // 404 = collection doesn't exist yet → create it.
let existing: { let existing: {
result?: { config?: { params?: { vectors?: { size?: number } } } }; result?: { config?: { params?: { vectors?: { size?: number } } } };
} | null = null; } | null = null;
try { try {
existing = (await request("GET", `/collections/${collectionName()}`)) as { existing = (await request(
"GET",
`/collections/${collectionName()}`,
)) as {
result?: { config?: { params?: { vectors?: { size?: number } } } }; result?: { config?: { params?: { vectors?: { size?: number } } } };
}; };
} catch (error) { } catch (error) {
@@ -137,6 +154,8 @@ export async function ensureQdrantCollection(
); );
return false; return false;
} }
})();
return ensureCollectionPromise;
} }
/** Upsert one embedding + verdict payload point. Returns false on failure. */ /** Upsert one embedding + verdict payload point. Returns false on failure. */
@@ -147,10 +166,15 @@ export async function upsertQdrantPoint(
): Promise<boolean> { ): Promise<boolean> {
try { try {
if (!(await ensureQdrantCollection(vector.length))) return false; if (!(await ensureQdrantCollection(vector.length))) return false;
await request("PUT", `/collections/${collectionName()}/points`, { await request(
"PUT",
`/collections/${collectionName()}/points`,
{
points: [{ id: qdrantPointId(cacheKey), vector, payload }], points: [{ id: qdrantPointId(cacheKey), vector, payload }],
wait: true, wait: true,
}); },
30_000,
);
return true; return true;
} catch (error) { } catch (error) {
log.warn( log.warn(
@@ -1,10 +1,11 @@
import Redis from "ioredis"; import Redis from "ioredis";
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
import { createAbortControllerWithTimeout } from "@/shared/utils/index"; import { createAbortControllerWithTimeout } from "@/shared/utils/index";
import { config } from "../../shared/config/config.js";
const log = createChildLogger("searxng-search"); const log = createChildLogger("searxng-search");
const SEARXNG_BASE_URL = "https://searxng.imrnes.team"; const SEARXNG_BASE_URL = config.SEARXNG_BASE_URL;
const MAX_RESULTS = 3; const MAX_RESULTS = 3;
const TIMEOUT_MS = 8000; const TIMEOUT_MS = 8000;
const CACHE_TTL = 86400; // 24 hours const CACHE_TTL = 86400; // 24 hours
@@ -12,6 +13,42 @@ const CACHE_PREFIX = "searxng:";
let redis: Redis | null = null; let redis: Redis | null = null;
/**
* Exposes the shared SearXNG Redis connection so other modules (e.g. the
* term glossary) reuse the same connection and cache prefix instead of
* opening their own. Returns null when Redis is unavailable.
*/
export function getSearxngRedis(): Redis | null {
return redis;
}
/** Builds a namespaced SearXNG cache key (shared across modules). */
export function makeSearxngCacheKey(namespace: string, key: string): string {
return `${CACHE_PREFIX}${namespace}:${key.toLowerCase().trim()}`;
}
/** Reads a value from the SearXNG Redis cache; null on miss/unavailable. */
export async function searxngCacheGet(key: string): Promise<string | null> {
if (!redis) return null;
try {
return await redis.get(key);
} catch {
return null;
}
}
/** Writes a value to the SearXNG Redis cache, fire-and-forget. */
export function searxngCacheSet(
key: string,
value: string,
ttlSeconds: number,
): void {
if (!redis) return;
redis.setex(key, ttlSeconds, value).catch(() => {
// Cache write failed silently
});
}
/** /**
* Initialize Redis connection for SearXNG cache. * Initialize Redis connection for SearXNG cache.
* Safe to call multiple times only creates one connection. * Safe to call multiple times only creates one connection.
@@ -51,19 +88,26 @@ export interface SearxngResult {
/** /**
* Search SearXNG for a query and return structured results. * Search SearXNG for a query and return structured results.
* Uses Redis cache when available same query within 24h returns cached results. * Uses Redis cache when available same query within 24h returns cached results.
*
* @param engines Optional comma-separated SearXNG engine list to constrain
* the search (e.g. "wikipedia"). When set, results are cached under a
* separate cache namespace so engine-specific results never collide.
*/ */
export async function searchSearxng( export async function searchSearxng(
query: string, query: string,
category: "general" | "news" | "science" = "general", category: "general" | "news" | "science" = "general",
engines?: string,
timeoutMs: number = TIMEOUT_MS,
): Promise<SearxngResult[]> { ): Promise<SearxngResult[]> {
const cacheKey = `${CACHE_PREFIX}${category}:${query.toLowerCase().trim()}`; const engineNs = engines ? `eng:${engines}` : "auto";
const cacheKey = makeSearxngCacheKey(`${category}:${engineNs}`, query);
// Try cache first // Try cache first
if (redis) { if (redis) {
try { try {
const cached = await redis.get(cacheKey); const cached = await redis.get(cacheKey);
if (cached) { if (cached) {
log.debug({ query, category }, "SearXNG cache HIT"); log.debug({ query, category, engines }, "SearXNG cache HIT");
return JSON.parse(cached) as SearxngResult[]; return JSON.parse(cached) as SearxngResult[];
} }
} catch { } catch {
@@ -73,8 +117,11 @@ export async function searchSearxng(
// Cache miss — hit SearXNG API // Cache miss — hit SearXNG API
try { try {
const url = `${SEARXNG_BASE_URL}/search?q=${encodeURIComponent(query)}&format=json&language=id&categories=${category}`; const engineParam = engines
const { controller, clear } = createAbortControllerWithTimeout(TIMEOUT_MS); ? `&engines=${encodeURIComponent(engines)}`
: "";
const url = `${SEARXNG_BASE_URL}/search?q=${encodeURIComponent(query)}&format=json&language=id&categories=${category}${engineParam}`;
const { controller, clear } = createAbortControllerWithTimeout(timeoutMs);
try { try {
const response = await fetch(url, { const response = await fetch(url, {
@@ -0,0 +1,551 @@
/**
* termGlossary.ts
*
* Per-word "kamus" enrichment for LLM moderation.
*
* Problem: the moderation LLM often meets words it does not know regional
* slang (Jawa/Sunda), foreign terms, niche anime/game jargon, or obscure
* technical vocabulary. When it guesses, it either invents a wrong meaning
* (false positive on a safe word) or misses a violation hidden in unfamiliar
* wording (false negative on an unknown vulgar/slang term).
*
* Solution: extract candidate "unknown-looking" words from message content,
* look each one up on Wikipedia via SearXNG, and inject the definitions into
* the LLM prompt as a `<term_glossary>` block so verdicts are based on facts
* instead of guesses.
*
* Cost control & persistence:
* - successfully resolved definitions are PERSISTED PERMANENTLY in Postgres
* (`term_glossary_cache`) definitions rarely change, so a resolved term
* is never searched again; only misses stay ephemeral (Redis/LRU, 1h);
* - in-memory LRU + Redis (shared with the SearXNG cache) sit in front of
* the DB as fast read caches, so repeat lookups are effectively free;
* - lookups per batch are bounded (AI_GLOSSARY_MAX_TERMS);
* - live SearXNG calls are rate-limit aware: concurrency 2 + stagger, retry
* once on empty results, and misses cached for only 1h so a limiter/
* network blip is not treated as a permanent miss;
* - only results that read like actual definitions are accepted (Wikipedia
* preferred; disambiguation/ads/translate-homepages rejected);
* - everything degrades gracefully: no Redis, no SearXNG, no match
* the block is simply omitted and moderation proceeds as before.
*/
import { LRUCache } from "lru-cache";
import pLimit from "p-limit";
import { createChildLogger } from "@/shared/logger/index";
import { delay } from "@/shared/utils/index";
import { config } from "../../shared/config/config.js";
import { escapeXml } from "./moderationBuilders.js";
import {
makeSearxngCacheKey,
searchSearxng,
searxngCacheGet,
searxngCacheSet,
} from "./searxngSearch.js";
import {
getTermDefinitionFromDb,
setTermDefinitionInDb,
} from "./termGlossaryStore.js";
const log = createChildLogger("term-glossary");
// ---------------------------------------------------------------------------
// Constants
// ---------------------------------------------------------------------------
/** Redis TTL for a successfully resolved definition (definitions are stable). */
const DEF_TTL_SECONDS = 7 * 24 * 60 * 60;
/**
* Redis TTL for a lookup that found nothing. Kept SHORT (1h): SearXNG
* instances silently return empty result sets when rate-limited, so an empty
* response is often a transient failure, not a real miss. A short TTL lets
* the term be retried on a later batch instead of poisoning it for a day.
*/
const MISS_TTL_SECONDS = 60 * 60;
const MISS_TTL_MS = MISS_TTL_SECONDS * 1000;
/** Sentinel stored in caches for "term has no resolvable definition". */
const EMPTY_SENTINEL = "__not_found__";
/** Per-search timeout — keep glossary lookups snappy even on a slow SearXNG. */
const GLOSSARY_SEARCH_TIMEOUT_MS = 5000;
/** Delay before retrying a search that returned zero results. */
const RETRY_DELAY_MS = 350;
/** Max definition snippet length kept in the prompt. */
const MAX_DEFINITION_CHARS = 300;
/**
* SearXNG rate-limits aggressive parallel bursts (returns 200 with empty
* results). Never fire all terms at once cap live searches at 2 concurrent
* and stagger the start times slightly.
*/
const LIVE_SEARCH_CONCURRENCY = 2;
const LIVE_SEARCH_STAGGER_MS = 250;
/** In-memory cache: term (lowercase) → definition | NOT_FOUND sentinel. */
const NOT_FOUND: TermDefinition = {
term: "__not_found__",
definition: "",
sourceUrl: "",
};
const termLru = new LRUCache<string, TermDefinition>({
max: 2000,
ttl: 24 * 60 * 60 * 1000,
});
/** Serializes live SearXNG lookups (rate-limit aware) with a small stagger. */
const liveSearchLimit = pLimit(LIVE_SEARCH_CONCURRENCY);
let lastLiveSearchAt = 0;
async function acquireLiveSlot(): Promise<void> {
const now = Date.now();
const wait = lastLiveSearchAt + LIVE_SEARCH_STAGGER_MS - now;
if (wait > 0) await delay(wait);
lastLiveSearchAt = Date.now();
}
// ---------------------------------------------------------------------------
// Term extraction
// ---------------------------------------------------------------------------
/** Word tokenizer letters/digits plus internal -_'· (handles "well-known",
* "node_modules", diacritics). */
const WORD_RE = /[\p{L}\p{N}]+(?:[-_'·][\p{L}\p{N}]+)*/gu;
/** Removes URLs, Discord mentions/custom emoji, code fences, markdown noise. */
function cleanContent(raw: string): string {
return raw
.replace(/https?:\/\/\S+/gi, " ")
.replace(/<@!?\d+>/g, " ")
.replace(/<#\d+>/g, " ")
.replace(/<a?:\w+:\d+>/g, " ")
.replace(/[`*_~|>[\]]/g, " ")
.replace(/[\p{Emoji}\p{Extended_Pictographic}]/gu, " ")
.replace(/\s+/g, " ")
.trim();
}
/** Filters out tokens that are useless as glossary candidates (numbers,
* repeated-char noise, mega-tokens). */
function isNoiseWord(word: string): boolean {
if (word.length > 28) return true;
if (/^\d+$/.test(word)) return true;
const lower = word.toLowerCase();
// "aaaa…", "wwwwww" — single repeated character
if (/^(.)\1{2,}$/.test(lower)) return true;
// "wkwk", "hehe", "69" alternations — repeated 23 char base. "meme" is
// the one legit 4-letter word this matches; it is whitelisted below.
if (/^([a-z]{2,3})\1{1,}$/.test(lower)) return true;
return false;
}
/** Deterministic bonus for words that look like proper nouns or foreign. */
function scoreWord(word: string): number {
let score = 1;
// Capitalized first letter (proper noun / title) but not ALL-CAPS acronyms
if (/^[A-Z]/.test(word) && !/^[A-Z]{2,}$/.test(word)) score += 3;
// Contains a letter outside basic latin → regional/foreign spelling
if (/[\p{L}]/u.test(word.replace(/[A-Za-z]/g, ""))) score += 2;
// Contains an internal apostrophe or hyphen → likely a named entity
if (/[-_'’·]/.test(word)) score += 2;
return score;
}
const STOPWORDS = new Set(
// ── Bahasa Indonesia ────────────────────────────────────────────────
(
" yang dan di ke dari ini itu dengan untuk pada dalam adalah akan telah sudah bisa dapat harus tidak juga saya kamu kita kami mereka dia aku kau gua lu lo gw gue elu anda kalian nya kah lah pun ya yah kan sih dong deh kok loh toh aja saja gitu gini begitu begini tapi tetapi namun atau karena sebab jika kalau bila maka supaya agar meski meskipun walau walaupun ketika saat setelah sebelum selama antara terhadap tentang mengenai bagi oleh secara sebagai seperti daripada tanpa hingga sampai sejak menuju bahwa padahal sebenarnya sepertinya mungkin memang jadi lalu terus akhirnya misalnya contohnya banyak sedikit semua seluruh setiap tiap beberapa ada bukan jangan boleh mau ingin pengen nggak ngak gak ga kagak ngga ndak nanti kemarin besok hari ini sekarang waktu itu masih sedang belum pernah sering selalu kadang jarang cepat lambat awal akhir baru lama besar kecil tinggi rendah panjang pendek baik buruk benar salah sama beda penting biasanya selamat terima kasih makasih sangat sekali paling cuma cuman hanya lebih kurang sekitar hampir ternyata rupanya begitu gimana bagaimana kenapa mengapa siapa apa mana kapan darimana kemana bilang ngomong omong kata tadi dulu terus lagi tetap pasti seharusnya sebaiknya seakan seolah kayaknya keliatan kelihatan ketahuan disini disitu disana kesini kesana bener pake pakai kayak emang lagian mulu istilah istilahnya banget" +
// ── English ───────────────────────────────────────────────────────
" the a an and or but if then else for to in on at by with without from of is are was were be been being have has had do does did will would can could should may might must shall this that these those it its i you he she we they them their there here when where why how what which who whom whose only very just about above after before below under over into onto within upon against between among during through across along around behind beyond near off out up down now then so as not no yes ok okay" +
// ── Common net slang / acronyms the LLM already knows ──────────────
" lol omg wtf idk btw tbh imo aka fyi nsfw smh nvm asap afk brb gg wp ty np mb sry thx kk oke okk ygy frfr"
).split(/\s+/),
);
/**
* Words that are either already defined by the moderation rules, or are so
* common (brands, tech vocabulary, project names) that a Wikipedia lookup is
* a guaranteed miss/waste. Keeps the glossary focused on genuinely unknown
* terms.
*/
const KNOWN_SAFE_TERMS = new Set(
(
"discord youtube google facebook instagram twitter tiktok whatsapp telegram netflix spotify steam github gitlab bitbucket chatgpt openai anthropic claude deepseek gemini llama copilot cursor vscode vscodium jetbrains intellij pycharm webstorm sublime codeblocks" +
" docker kubernetes k8s linux ubuntu debian arch fedora manjaro kali windows macos android ios chrome firefox safari edge opera brave" +
" react nextjs next vue svelte angular node nodejs deno bun pnpm yarn npm javascript typescript python golang go rust java kotlin swift cplusplus cpp css html json xml yaml toml regex backend frontend database mysql postgres postgresql mongodb redis qdrant sqlite nosql graphql rest websocket webhook" +
" bug crash error debug fix issue pr merge commit push pull branch main master dev staging production server client app website web browser" +
" stream streaming video audio voice call camera screen share screenshare gameplay gaming game play steam epic xbox playstation nintendo switch console" +
" bot discordbot moderation moderator admin member user profile avatar channel server guild message chat dm reply forward embed sticker emoji role permission" +
" meme code coding ngoding programmer program developer engineer software hardware cpu gpu ram rom storage disk network internet wifi lan ip dns vpn proxy cloud aws azure gcp vercel netlify heroku railway render vps hosting domain ssl login logout register account password email username" +
" anime manga waifu husbando tsundere moe otaku wibu weeb otome isekai shonen seinen josei manga manhwa manhua doujin" +
" anjay wkwk wkwkwk gws gaskeun santuy njir baka woy woi hadeh astaga asu anjing bangsat ngehe asal alay lebay caper mabar" +
" asus bete imphnen impnhen ngab" +
" syahadat sholat shalat solat puasa zakat haji umrah doa tuhan nabi allah yesus muhammad hashem" +
" loli shota incest exhibition furry fursuit cosplay costume" +
" gaza palestine israel yahudi yahud israel palestina israeli" +
" hokkian mandarin arabic jawa sunda betawi minang bugis batak melayu inggris indonesia"
).split(/\s+/),
);
function isKnownTerm(word: string): boolean {
return STOPWORDS.has(word) || KNOWN_SAFE_TERMS.has(word);
}
/** True when a quoted phrase is mostly filler words (skip it). */
function isMostlyStopwords(phrase: string): boolean {
const words = phrase
.toLowerCase()
.split(/[^a-zà-öø-ÿ]+/i)
.filter(Boolean);
if (words.length === 0) return true;
const stopCount = words.filter((w) => STOPWORDS.has(w)).length;
return stopCount / words.length >= 0.6;
}
export interface ExtractGlossaryOptions {
maxTerms?: number;
minWordLength?: number;
}
/**
* Extracts candidate terms that the LLM might not know from message content.
* Returns at most `maxTerms` terms (default from config), scored by how
* "unknown-looking" they are (proper nouns, foreign spelling, quoted phrases).
*/
export function extractGlossaryTerms(
contents: string[],
options: ExtractGlossaryOptions = {},
): string[] {
const maxTerms = options.maxTerms ?? config.AI_GLOSSARY_MAX_TERMS;
const minWordLength =
options.minWordLength ?? config.AI_GLOSSARY_MIN_WORD_LENGTH;
const candidates = new Map<string, { word: string; score: number }>();
const push = (rawWord: string, score: number): void => {
const clean = rawWord
.trim()
.replace(/^[^\p{L}\p{N}]+|[^\p{L}\p{N}]+$/gu, "");
if (clean.length < minWordLength) return;
const key = clean.toLowerCase();
if (isKnownTerm(key) || isNoiseWord(clean)) return;
const existing = candidates.get(key);
if (existing) {
existing.score += score + 1;
} else {
candidates.set(key, { word: clean, score });
}
};
for (const content of contents) {
if (!content) continue;
const cleaned = cleanContent(content);
if (!cleaned) continue;
// Quoted phrases — explicit terms the user called out
for (const m of cleaned.matchAll(/"([^"]{2,80})"/g)) {
const phrase = m[1].trim();
const wordCount = phrase.split(/\s+/).length;
if (wordCount >= 2 && wordCount <= 6 && !isMostlyStopwords(phrase)) {
push(phrase, 10);
}
}
// Individual words
for (const m of cleaned.matchAll(WORD_RE)) {
const w = m[0];
if (w.length < minWordLength) continue;
if (isNoiseWord(w)) continue;
const key = w.toLowerCase();
if (isKnownTerm(key)) continue;
push(w, scoreWord(w));
}
}
return Array.from(candidates.values())
.sort((a, b) => b.score - a.score)
.slice(0, maxTerms)
.map((c) => c.word);
}
// ---------------------------------------------------------------------------
// Definition lookup (cached: LRU → Redis → SearXNG/Wikipedia)
// ---------------------------------------------------------------------------
export interface TermDefinition {
term: string;
definition: string;
sourceUrl: string;
}
/** Definition-like markers for accepting a non-Wikipedia search result. */
const DEF_MARKERS =
/adalah|merupakan|istilah (?:untuk|yang|yg)|artinya|sebutan|berarti|refers? to|known as|also called|short for|a term (?:for|used)|istilah dalam|kata (?:asing|serapan)? ?untuk/i;
/** True when the term appears in the result text (or a 4+ char word in the
* result is part of the term). Lenient "kafircel" matches a "Kafir"
* article via substring, while a Google-Translate homepage snippet does not. */
function hasTermOverlap(term: string, title: string, snippet: string): boolean {
const termLower = term.toLowerCase();
const text = `${title} ${snippet}`.toLowerCase();
if (text.includes(termLower)) return true;
const words = text.match(/[a-z0-9]{4,}/gi) ?? [];
return words.some((w) => termLower.includes(w));
}
/** Quality gate: is this result good enough to quote as a definition? */
function isUsableDefinition(
r: { title: string; url: string; snippet: string },
term: string,
isWiki: boolean,
): boolean {
const text = `${r.title} ${r.snippet}`;
// Wikipedia disambiguation pages are not definitions
if (/disambiguasi|disambiguation/i.test(text)) return false;
if ((r.snippet ?? "").trim().length < 25) return false;
if (!hasTermOverlap(term, r.title, r.snippet)) return false;
// Wikipedia articles are accepted with just the overlap+length gate;
// everything else must read like an actual definition, not an ad,
// a translate homepage, or a navigation blurb.
if (isWiki) return true;
return DEF_MARKERS.test(r.snippet);
}
/** Picks the best definition from search results, preferring a genuine
* Wikipedia article; otherwise the first result that reads like a
* definition. Returns null when nothing qualifies. */
function pickDefinition(
results: Array<{ title: string; url: string; snippet: string }>,
term: string,
): TermDefinition | null {
const wiki = results.find((r) => /wikipedia\.org/i.test(r.url));
const best = wiki && isUsableDefinition(wiki, term, true) ? wiki : null;
if (!best) {
for (const r of results) {
if (isUsableDefinition(r, term, false)) {
return buildDefinition(r, term);
}
}
return null;
}
return buildDefinition(best, term);
}
function buildDefinition(
best: { title: string; url: string; snippet: string },
term: string,
): TermDefinition {
const snippet = (best.snippet || best.title || "").trim();
const definition =
snippet.length > MAX_DEFINITION_CHARS
? `${snippet.slice(0, MAX_DEFINITION_CHARS - 1).trimEnd()}`
: snippet;
return { term, definition, sourceUrl: best.url };
}
/** Live (network) lookup — runs under the shared SearXNG rate-limit gate. */
async function fetchDefinitionLive(
term: string,
key: string,
cacheKey: string,
): Promise<TermDefinition | null> {
return liveSearchLimit(async () => {
await acquireLiveSlot();
try {
let results = await searchSearxng(
key,
"general",
undefined,
GLOSSARY_SEARCH_TIMEOUT_MS,
);
let def = pickDefinition(results, term);
// Zero results is usually the limiter kicking in, not a real miss —
// retry once. Results-but-unusable = genuine miss, no retry.
if (!def && results.length === 0) {
await delay(RETRY_DELAY_MS);
results = await searchSearxng(
key,
"general",
undefined,
GLOSSARY_SEARCH_TIMEOUT_MS,
);
def = pickDefinition(results, term);
}
if (def) {
// Persist permanently (definitions rarely change) — best-effort,
// then warm the fast caches.
void setTermDefinitionInDb(key, def.definition, def.sourceUrl);
searxngCacheSet(
cacheKey,
JSON.stringify({
definition: def.definition,
sourceUrl: def.sourceUrl,
}),
DEF_TTL_SECONDS,
);
termLru.set(key, def);
log.debug({ term: key }, "Term glossary resolved definition");
return def;
}
} catch (err) {
log.debug(
{ term: key, error: err instanceof Error ? err.message : String(err) },
"Term glossary lookup failed — skipping term",
);
}
// No definition — cache the miss with a SHORT TTL so a transient
// limiter/network failure is retried on a later batch.
searxngCacheSet(cacheKey, EMPTY_SENTINEL, MISS_TTL_SECONDS);
termLru.set(key, NOT_FOUND, { ttl: MISS_TTL_MS });
return null;
});
}
/** Resolve one term: LRU Redis Postgres (permanent) live SearXNG
* (rate-limited). The fast caches sit in front of the DB; the DB is the
* source of truth for successfully resolved definitions. */
async function resolveTerm(term: string): Promise<TermDefinition | null> {
const key = term.toLowerCase().trim();
// 1. In-memory LRU — same process, instant
const lruHit = termLru.get(key);
if (lruHit) return lruHit === NOT_FOUND ? null : lruHit;
// 2. Redis — shared across processes/workers. A miss sentinel here is NOT
// a definitive answer: it may predate a permanent DB entry written by
// another process, so we keep going and let the DB decide.
const cacheKey = makeSearxngCacheKey("def", key);
const cached = await searxngCacheGet(cacheKey);
let redisMiss = false;
if (cached !== null) {
if (cached === EMPTY_SENTINEL) {
redisMiss = true;
} else {
try {
const parsed = JSON.parse(cached) as {
definition?: string;
sourceUrl?: string;
};
if (parsed.definition) {
const def: TermDefinition = {
term,
definition: parsed.definition,
sourceUrl: parsed.sourceUrl ?? "",
};
termLru.set(key, def);
return def;
}
} catch {
// malformed cache entry — fall through to DB/live
}
}
}
// 3. Postgres — permanent store for resolved definitions. A hit re-warms
// the fast caches so the DB is not hit on every batch.
const dbDef = await getTermDefinitionFromDb(key);
if (dbDef) {
const def: TermDefinition = {
term,
definition: dbDef.definition,
sourceUrl: dbDef.sourceUrl,
};
termLru.set(key, def);
searxngCacheSet(
cacheKey,
JSON.stringify({ definition: def.definition, sourceUrl: def.sourceUrl }),
DEF_TTL_SECONDS,
);
log.debug({ term: key }, "Term glossary DB hit");
return def;
}
// 4. Redis already said "miss" recently and the DB has nothing — respect
// that instead of hammering SearXNG again within the miss window.
if (redisMiss) {
termLru.set(key, NOT_FOUND, { ttl: MISS_TTL_MS });
return null;
}
// 5. Live search (rate-limited + staggered)
return fetchDefinitionLive(term, key, cacheKey);
}
/**
* Looks up definitions for a batch of terms, in parallel. Returns a map of
* term definition for the terms that resolved. Errors/misses are skipped.
* Live SearXNG calls are throttled internally (concurrency 2 + stagger).
*/
export async function lookupTermDefinitions(
terms: string[],
): Promise<Map<string, TermDefinition>> {
const map = new Map<string, TermDefinition>();
if (terms.length === 0) return map;
const results = await Promise.allSettled(terms.map(resolveTerm));
for (let i = 0; i < terms.length; i++) {
const r = results[i];
if (r.status === "fulfilled" && r.value) {
map.set(r.value.term, r.value);
}
}
return map;
}
// ---------------------------------------------------------------------------
// Prompt formatting
// ---------------------------------------------------------------------------
/**
* Formats definitions as a `<term_glossary>` XML block for the LLM prompt:
*
* <term_glossary>
* <term word="ngab" source="https://…">definisi</term>
* </term_glossary>
*
* Returns "" when there are no definitions (the block is then omitted).
*/
export function formatTermGlossary(
defs: ReadonlyMap<string, TermDefinition>,
): string {
if (!defs || defs.size === 0) return "";
const lines = Array.from(defs.values()).map(
(d) =>
` <term word="${escapeXml(d.term)}" source="${escapeXml(d.sourceUrl)}">${escapeXml(d.definition)}</term>`,
);
return `<term_glossary>\n${lines.join("\n")}\n</term_glossary>`;
}
// ---------------------------------------------------------------------------
// Convenience: full pipeline
// ---------------------------------------------------------------------------
export interface GlossaryBlockOptions extends ExtractGlossaryOptions {
enabled?: boolean;
}
/**
* One-shot helper: extract terms from message contents, look up definitions,
* and return the formatted `<term_glossary>` block ("" when disabled or no
* definitions found). Safe to call on every batch cached lookups make it
* cheap.
*/
export async function buildTermGlossaryBlock(
contents: string[],
options: GlossaryBlockOptions = {},
): Promise<string> {
const enabled = options.enabled ?? config.AI_GLOSSARY_ENABLED;
if (!enabled) return "";
if (contents.length === 0) return "";
const terms = extractGlossaryTerms(contents, options);
if (terms.length === 0) return "";
const defs = await lookupTermDefinitions(terms);
if (defs.size === 0) return "";
const block = formatTermGlossary(defs);
log.debug(
{ terms: terms.length, definitions: defs.size },
"Term glossary block built",
);
return block;
}
@@ -0,0 +1,86 @@
/**
* termGlossaryStore.ts
*
* Permanent Postgres layer for the term glossary. Resolved definitions
* (which carry content) are persisted here because they rarely change
* Redis/LRU only act as fast read caches in front of this table. Terms with
* no definition (misses) are deliberately NOT persisted; they stay ephemeral
* in Redis with a short TTL so transient lookup failures get retried.
*
* All calls are best-effort: any DB error degrades to a cache miss (the
* glossary then falls through to Redis/live search as if the DB layer
* didn't exist).
*/
import { createChildLogger } from "@/shared/logger/index";
import { executeAll, executeGet } from "../../shared/database/drizzle.js";
const log = createChildLogger("term-glossary-store");
export interface StoredTermDefinition {
definition: string;
sourceUrl: string;
}
/**
* Read a permanently stored definition for a term (lowercase key).
* Returns null when missing or on any DB error (callers fall through).
* A successful read bumps hit_count for observability (fire-and-forget).
*/
export async function getTermDefinitionFromDb(
term: string,
): Promise<StoredTermDefinition | null> {
try {
const row = await executeGet(
`SELECT definition, source_url FROM term_glossary_cache WHERE term = $1`,
[term.toLowerCase().trim()],
);
if (!row) return null;
try {
await executeAll(
`UPDATE term_glossary_cache SET hit_count = hit_count + 1 WHERE term = $1`,
[term.toLowerCase().trim()],
);
} catch {
// hit_count is observability only — never fail a read for it
}
return {
definition: row.definition as string,
sourceUrl: (row.source_url as string | null) ?? "",
};
} catch (error) {
log.debug(
{ error: error instanceof Error ? error.message : String(error) },
"getTermDefinitionFromDb failed — falling back to live search",
);
return null;
}
}
/**
* Persist a resolved definition permanently (UPSERT by term).
* Only called for successful resolutions never for misses.
* Best-effort: a DB write failure does not affect the returned definition.
*/
export async function setTermDefinitionInDb(
term: string,
definition: string,
sourceUrl: string,
): Promise<void> {
try {
await executeAll(
`INSERT INTO term_glossary_cache (term, definition, source_url, resolved_at, hit_count)
VALUES ($1, $2, $3, $4, 0)
ON CONFLICT (term) DO UPDATE SET
definition = EXCLUDED.definition,
source_url = EXCLUDED.source_url,
resolved_at = EXCLUDED.resolved_at`,
[term.toLowerCase().trim(), definition, sourceUrl, Date.now()],
);
} catch (error) {
log.warn(
{ error: error instanceof Error ? error.message : String(error) },
"setTermDefinitionInDb failed — definition stays memory/Redis only",
);
}
}
@@ -6,7 +6,9 @@
* the LLM for analysis. Extracted from moderationOrchestrator.ts. * the LLM for analysis. Extracted from moderationOrchestrator.ts.
*/ */
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
import { delay } from "@/shared/utils/index";
import { config } from "../../shared/config/config.js"; import { config } from "../../shared/config/config.js";
import { resizeImageForVision } from "../attachment-upload/imageResizer.js";
import type { import type {
AnalysisResult, AnalysisResult,
MessageRecord, MessageRecord,
@@ -14,25 +16,32 @@ import type {
import { getChannelCulture } from "./channelCultureStore.js"; import { getChannelCulture } from "./channelCultureStore.js";
import type { ModerationPromptContent, RetryState } from "./llmCaller.js"; import type { ModerationPromptContent, RetryState } from "./llmCaller.js";
import { callModerationLLM } from "./llmCaller.js"; import { callModerationLLM } from "./llmCaller.js";
import { analyzeSingleMediaImage } from "./mediaAnalysisClient.js";
import { import {
buildReferenceXml, buildReferenceXml,
buildUserProfileRef,
buildUserProfilesBlock,
escapeXml, escapeXml,
formatReputationAttrs,
getAnalysisContent, getAnalysisContent,
resolveDisplayName,
resolveIsBot,
resolveIsEdited,
truncateForAi,
} from "./moderationBuilders.js"; } from "./moderationBuilders.js";
import { import { buildSystemPrompt as buildSystemPromptModular } from "./moderationPrompt.js";
buildSystemPrompt as buildSystemPromptModular,
sanitizeAiContent,
} from "./moderationPrompt.js";
import { logModerationAnalysis } from "./responseLogger.js"; import { logModerationAnalysis } from "./responseLogger.js";
import { import {
extractSearchQueries, extractSearchQueries,
formatSearchResults, formatSearchResults,
searchSearxng, searchSearxng,
} from "./searxngSearch.js"; } from "./searxngSearch.js";
import { buildTermGlossaryBlock } from "./termGlossary.js";
import { getRecentCorrectedModerations } from "./textCacheStore.js"; import { getRecentCorrectedModerations } from "./textCacheStore.js";
import { extractUrlsFromText, fetchUrlSafely } from "./urlFetcher.js"; import { extractUrlsFromText, fetchUrlSafely } from "./urlFetcher.js";
import { getUserProfile } from "./userProfileStore.js"; import { getUserProfile } from "./userProfileStore.js";
import { initializeUserReputation } from "./userReputationStore.js"; import { initializeUserReputation } from "./userReputationStore.js";
import type { MessageImagePart } from "./visionAnalyzer.js";
const log = createChildLogger("textBatchProcessor"); const log = createChildLogger("textBatchProcessor");
@@ -69,7 +78,7 @@ export async function buildCorrectedFewShotExamples(): Promise<string> {
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
export async function runTextOnlyBatch( export async function runTextOnlyBatch(
targets: MessageRecord[], targets: MessageRecord[],
contextText: string, contextBlock: string,
): Promise<{ results: AnalysisResult[]; raw: unknown }> { ): Promise<{ results: AnalysisResult[]; raw: unknown }> {
if (!targets.length) return { results: [], raw: null }; if (!targets.length) return { results: [], raw: null };
@@ -84,22 +93,33 @@ export async function runTextOnlyBatch(
allUrls.add(url); allUrls.add(url);
} }
const urlArr = Array.from(allUrls).slice(0, 10); const urlArr = Array.from(allUrls).slice(0, 10);
if (urlArr.length === 0) return new Map<string, string>(); if (urlArr.length === 0) {
return {
text: new Map<string, string>(),
image: new Map<string, { data: Buffer; mimeType: string }>(),
title: new Map<string, string>(),
};
}
const results = await Promise.allSettled( const results = await Promise.allSettled(
urlArr.map((url) => fetchUrlSafely(url)), urlArr.map((url) => fetchUrlSafely(url)),
); );
const map = new Map<string, string>(); const textMap = new Map<string, string>();
const imageMap = new Map<string, { data: Buffer; mimeType: string }>();
const titleMap = new Map<string, string>();
for (let i = 0; i < urlArr.length; i++) { for (let i = 0; i < urlArr.length; i++) {
const r = results[i]; const r = results[i];
if ( if (r.status !== "fulfilled") continue;
r.status === "fulfilled" && const v = r.value;
r.value.type === "text" && if (v.type === "text" && v.textContent) {
r.value.textContent textMap.set(urlArr[i], v.textContent);
) { if (v.title) titleMap.set(urlArr[i], v.title);
map.set(urlArr[i], r.value.textContent); } else if (v.type === "image" && v.data && v.mimeType) {
// Direct image link (or og:image followed from an HTML page) —
// kept for vision analysis below.
imageMap.set(urlArr[i], { data: v.data, mimeType: v.mimeType });
} }
} }
return map; return { text: textMap, image: imageMap, title: titleMap };
})(); })();
const searxngPromise = (async () => { const searxngPromise = (async () => {
@@ -122,10 +142,19 @@ export async function runTextOnlyBatch(
return map; return map;
})(); })();
const [urlFetchMap, searxngResults] = await Promise.all([ // Term glossary — per-word Wikipedia lookups for words the LLM may not
// know (slang, jargon, regional language). Cached in Redis + in-memory, so
// repeat terms resolve instantly and only genuinely new words hit SearXNG.
const glossaryPromise = buildTermGlossaryBlock(
targets.map((msg) => getAnalysisContent(msg)),
).catch(() => "");
const [urlFetchMaps, searxngResults, glossaryBlock] = await Promise.all([
urlFetchPromise, urlFetchPromise,
searxngPromise, searxngPromise,
glossaryPromise,
]); ]);
const urlFetchMap = urlFetchMaps.text;
// Deduplicate identical short messages // Deduplicate identical short messages
const shortContentGroups = new Map<string, MessageRecord[]>(); const shortContentGroups = new Map<string, MessageRecord[]>();
@@ -171,25 +200,90 @@ export async function runTextOnlyBatch(
const batch = subBatches[i]; const batch = subBatches[i];
const targetIds = batch.map((t) => t.id); const targetIds = batch.map((t) => t.id);
// User reputation + profiles // User reputation + profiles (raw summary text — deduplicated into a
// single <user_profiles> map per batch; messages only reference it).
const userContexts = new Map<string, string>(); const userContexts = new Map<string, string>();
const userProfiles = new Map<string, string>(); const userProfiles = new Map<
string,
{
text: string;
asOf?: number | null;
}
>();
for (const msg of batch) { for (const msg of batch) {
if (!userContexts.has(msg.user_id)) { if (!userContexts.has(msg.user_id)) {
const rep = await initializeUserReputation(msg.user_id, msg.guild_id); const rep = await initializeUserReputation(msg.user_id, msg.guild_id);
userContexts.set( const repAttrs = formatReputationAttrs(rep);
msg.user_id, const repXml = `<user_reputation ${repAttrs}/>`;
`<user_reputation trust_score="${rep.trust_score}" />`, userContexts.set(msg.user_id, repXml);
);
} }
if (!userProfiles.has(msg.user_id)) { if (!userProfiles.has(msg.user_id)) {
const profile = await getUserProfile(msg.user_id); const profile = await getUserProfile(msg.user_id);
userProfiles.set( userProfiles.set(msg.user_id, {
msg.user_id, text: profile?.profile_summary ?? "",
profile asOf: profile?.last_analyzed_at ?? null,
? `<user_profile>${sanitizeAiContent(profile.profile_summary)}</user_profile>` });
: "", }
}
const userProfilesBlock = buildUserProfilesBlock(userProfiles);
// ── URL images → multimodal vision evidence ─────────────────────────
// The text batch fetches inline URLs; whenever one resolved to an image
// (direct image link, or og:image followed from an HTML page), run the
// vision model and append its description as media evidence. If any
// message in the sub-batch produced image evidence, the prompt switches
// to "mixed" mode so media-analysis instructions/examples are injected
// — a link to media is analyzed as media, not as bare text.
const batchImageEvidence = new Map<string, string[]>();
let batchHasImageEvidence = false;
const urlImages = urlFetchMaps.image;
const urlTitles = urlFetchMaps.title;
if (urlImages.size > 0) {
const maxDim = config.AI_LLM_IMAGE_MAX_DIMENSION ?? 1024;
const evidenceSets = await Promise.all(
batch.map(async (msg) => {
const content = getAnalysisContent(msg);
const pics = extractUrlsFromText(content)
.slice(0, 3)
.filter((url) => urlImages.has(url));
if (pics.length === 0) return { id: msg.id, lines: [] as string[] };
const lines = await Promise.all(
pics.map(async (url) => {
const img = urlImages.get(url);
if (!img) return null;
try {
const { data: resizedBuffer, mimeType: resizedMime } =
await resizeImageForVision(img.data, maxDim);
const part: MessageImagePart = {
type: "image_url",
image_url: {
url: `data:${resizedMime};base64,${resizedBuffer.toString("base64")}`,
},
sourceLabel: `[gambar dari URL ${url} (inline), pesan id=${msg.id}]`,
};
// Bound vision time so a dead vision model can't stall the
// whole text batch — a timeout just skips the evidence.
const timedOut = delay(15000).then(() => null as string | null);
return await Promise.race([
analyzeSingleMediaImage(msg.id, part),
timedOut,
]);
} catch {
return null;
}
}),
); );
return {
id: msg.id,
lines: lines.filter((l): l is string => Boolean(l)),
};
}),
);
for (const set of evidenceSets) {
if (set.lines.length > 0) {
batchImageEvidence.set(set.id, set.lines);
batchHasImageEvidence = true;
}
} }
} }
@@ -204,8 +298,7 @@ export async function runTextOnlyBatch(
: undefined; : undefined;
const correctedExamples = await buildCorrectedFewShotExamples(); const correctedExamples = await buildCorrectedFewShotExamples();
const systemText = buildSystemPromptModular({ const systemText = buildSystemPromptModular({
contextText, mode: batchHasImageEvidence ? "mixed" : "text",
mode: "text",
correction, correction,
correctedExamples, correctedExamples,
channelCulture, channelCulture,
@@ -214,38 +307,59 @@ export async function runTextOnlyBatch(
const messagesBlock = ( const messagesBlock = (
await Promise.all( await Promise.all(
batch.map(async (msg) => { batch.map(async (msg) => {
const content = getAnalysisContent(msg); const content = truncateForAi(getAnalysisContent(msg));
const msgUrls = extractUrlsFromText(content); const msgUrls = extractUrlsFromText(content);
const urlContexts = msgUrls const urlContexts = msgUrls
.map((url) => { .map((url) => {
const ft = urlFetchMap.get(url); const ft = urlFetchMap.get(url);
return ft if (!ft) return null;
? `<web_content url="${escapeXml(url)}">${escapeXml(ft)}</web_content>` const title = urlTitles.get(url);
: null; const titleAttr = title ? ` title="${escapeXml(title)}"` : "";
return `<web_content url="${escapeXml(url)}"${titleAttr}>${escapeXml(ft)}</web_content>`;
}) })
.filter(Boolean) .filter(Boolean)
.join("\n"); .join("\n");
const webContext = urlContexts ? `\n${urlContexts}` : ""; const webContext = urlContexts ? `\n${urlContexts}` : "";
const mediaEvidenceCtx = (batchImageEvidence.get(msg.id) ?? [])
.map((line) => `\n${line}`)
.join("");
const userCtx = userContexts.get(msg.user_id) ?? ""; const userCtx = userContexts.get(msg.user_id) ?? "";
const userProfileCtx = userProfiles.get(msg.user_id) ?? ""; const userProfileRef = (
userProfiles.get(msg.user_id)?.text ?? ""
).trim()
? buildUserProfileRef(msg.user_id)
: "";
const refXml = await buildReferenceXml(msg); const refXml = await buildReferenceXml(msg);
return `<message id="${msg.id}" user="${msg.username}">\n ${userCtx}${userProfileCtx ? `\n ${userProfileCtx}` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(content)}</content>${webContext}\n</message>`; const repetitionCount = groupMapping.get(msg.id)?.length ?? 1;
const isBot = resolveIsBot(msg);
const isEdited = resolveIsEdited(msg);
return `<message id="${escapeXml(msg.id)}" user="${escapeXml(resolveDisplayName(msg))}" time="${new Date(msg.created_at).toISOString()}"${repetitionCount > 1 ? ` repetitions="${repetitionCount}"` : ""}${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${userCtx}${userProfileRef ? `\n ${userProfileRef}` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(content)}</content>${webContext}${mediaEvidenceCtx}\n</message>`;
}), }),
) )
).join("\n"); ).join("\n");
const searxngBlock = const searxngBlock =
searxngResults.size > 0 searxngResults.size > 0
? `\n\n<web_searches>\n${Array.from(searxngResults.entries()) ? `<web_searches>\n${Array.from(searxngResults.entries())
.map( .map(
([q, xml]) => ([q, xml]) =>
` <search_query query="${escapeXml(q)}">\n${xml} </search_query>`, ` <search_query query="${escapeXml(q)}">\n${xml} </search_query>`,
) )
.join("\n")}\n</web_searches>` .join("\n")}\n</web_searches>`
: ""; : "";
// Data/instruction separation: the system prompt is stable per mode —
// all per-batch context (profiles, conversation, web evidence) lives in
// the USER payload, ordered oldest-first so targets come last.
const userBlocks = [
userProfilesBlock?.trimEnd() ?? "",
contextBlock?.trimEnd() ?? "",
searxngBlock,
glossaryBlock,
`<messages_to_analyze>\n${messagesBlock}\n</messages_to_analyze>`,
].filter((b) => b.trim().length > 0);
return { return {
system: systemText, system: systemText,
user: `${searxngBlock}\n\n<messages_to_analyze>\n${messagesBlock}\n</messages_to_analyze>`, user: userBlocks.join("\n\n"),
}; };
}; };
@@ -3,7 +3,6 @@ import { createChildLogger } from "@/shared/logger/index";
import { executeAll, executeGet } from "../../shared/database/drizzle.js"; import { executeAll, executeGet } from "../../shared/database/drizzle.js";
import { findBestEmbeddingMatch } from "./embeddingClient.js"; import { findBestEmbeddingMatch } from "./embeddingClient.js";
import { import {
deleteExpiredQdrantPoints,
deleteQdrantPoint, deleteQdrantPoint,
deleteQdrantPointsByContentHash, deleteQdrantPointsByContentHash,
isQdrantConfigured, isQdrantConfigured,
@@ -62,13 +61,30 @@ export function makeCustomEmojiCacheKey(emojiId: string): string {
} }
/** /**
* Generate a deterministic cache key for an image data URL. * Generate a deterministic cache key for an image from its source URL
* Hashes the first 128 chars of the data URL (enough to identify the image * (Discord CDN / embed URL / inline URL).
* without storing the full base64 string as the key). *
* The CDN URL is the stable identity of an attachment: re-analysis of the
* same message (recovery worker, retries) always hits the cache regardless
* of resize/encoding output. Query params are stripped (Discord signed
* tokens `?ex=&is=&hm=` and render variants `?format=&width=`) so the same
* attachment resolves to the same key even when fetched with different
* signatures or sizes.
*
* No SHA/phash the CDN URL is the cache key itself. This makes
* re-analysis of the SAME attachment cache-hit, while different attachments
* (different URLs) never collide.
*/ */
export function makeImageCacheKey(dataUrl: string): string { export function makeImageCacheKey(imageUrl: string): string {
const prefix = dataUrl.slice(0, 128); // Hash the URL to a fixed-length key. The raw Discord CDN URL is short,
const hash = createHash("sha256").update(prefix).digest("hex").slice(0, 16); // but callers sometimes pass base64 data URLs (can be multi-MB) or very
// long signed/external URLs. text_analysis_cache.text is the PK and lives
// in a B-tree index with an 8191-byte per-row limit — inserting a long URL
// as the key aborts the whole INSERT ("index row requires N bytes, maximum
// size is 8191"), which fails acquireMediaAnalysisLock and silently skips
// every media analysis. A 32-char sha256 keeps the key well under the limit
// and is still deterministic (same attachment → same key).
const hash = createHash("sha256").update(imageUrl).digest("hex").slice(0, 32);
return `image:${hash}`; return `image:${hash}`;
} }
@@ -515,64 +531,6 @@ export async function setCachedTextModeration(
} }
} }
// ---------------------------------------------------------------------------
// Perceptual hash helpers for image deduplication
// ---------------------------------------------------------------------------
/**
* Generate a deterministic cache key for a perceptual hash.
* The phash value is a string like "a1b2c3d4e5f6..." from the imghash library.
*/
export function makePhashCacheKey(phash: string): string {
return `phash:${phash.slice(0, 16)}`;
}
/**
* Look up a cached media analysis by perceptual hash.
* Returns the cached analysis string or null if not found/expired.
*/
export async function getCachedMediaByPhash(
phash: string,
): Promise<string | null> {
const cacheKey = makePhashCacheKey(phash);
return getCachedMediaAnalysis(cacheKey);
}
/**
* Store a media analysis result keyed by perceptual hash.
*/
export async function upsertCachedMediaByPhash(
phash: string,
analysisResult: string,
source: "vision_llm",
expiresAt: number,
): Promise<void> {
const cacheKey = makePhashCacheKey(phash);
return upsertCachedMediaAnalysis(cacheKey, analysisResult, source, expiresAt);
}
/**
* Compute perceptual hash from image buffer using imghash.
* Returns a hexadecimal string representation of the hash.
* Returns null if hashing fails (e.g., invalid image data).
*/
export async function computeImagePhash(
buffer: Buffer,
): Promise<string | null> {
try {
// Dynamic import — imghash is ESM with a default export containing { hash, hashRaw, ... }
const imghashModule: {
default?: { hash?: (buf: Buffer) => Promise<string> };
} = await import("imghash");
const hashFn = imghashModule.default?.hash;
if (typeof hashFn !== "function") return null;
const hash = await hashFn(buffer);
return hash;
} catch {
return null;
}
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Corrected Moderation (false-positive) helpers for dynamic few-shot injection // Corrected Moderation (false-positive) helpers for dynamic few-shot injection
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -11,6 +11,8 @@ export interface FetchedUrlContext {
data?: Buffer; data?: Buffer;
mimeType?: string; mimeType?: string;
textContent?: string; textContent?: string;
/** Page title from og:title / <title> — strong signal for the LLM. */
title?: string;
error?: string; error?: string;
} }
@@ -86,6 +88,50 @@ function extractOgImage(html: string): string | null {
return null; return null;
} }
export interface OgMeta {
title: string | null;
description: string | null;
siteName: string | null;
}
/**
* Extracts OpenGraph / twitter meta + <title> from raw HTML. Both attribute
* orders are accepted (<meta property=... content=...> and reversed).
*/
export function extractOgMeta(html: string): OgMeta {
const metaValue = (name: string): string | null => {
const re = new RegExp(
`<meta[^>]*(?:property|name)=["']${name}["'][^>]*content=["']([^"']+)["']`,
"i",
);
const m = html.match(re);
if (m?.[1]) return m[1].replace(/&amp;/g, "&").replace(/&quot;/g, '"');
const reRev = new RegExp(
`<meta[^>]*content=["']([^"']+)["'][^>]*(?:property|name)=["']${name}["']`,
"i",
);
const mRev = html.match(reRev);
return mRev?.[1]
? mRev[1].replace(/&amp;/g, "&").replace(/&quot;/g, '"')
: null;
};
const title =
metaValue("og:title") ||
metaValue("twitter:title") ||
html.match(/<title[^>]*>([^<]+)<\/title>/i)?.[1]?.trim() ||
null;
const description =
metaValue("og:description") ||
metaValue("twitter:description") ||
metaValue("description") ||
null;
const siteName =
metaValue("og:site_name") || metaValue("application-name") || null;
return { title, description, siteName };
}
function truncateAndCleanHtml(html: string, maxLen = 1000): string { function truncateAndCleanHtml(html: string, maxLen = 1000): string {
// Strip <script> and <style> entirely // Strip <script> and <style> entirely
let text = html.replace( let text = html.replace(
@@ -176,6 +222,7 @@ export async function fetchUrlSafely(
url, url,
type: "text", type: "text",
textContent: cleaned, textContent: cleaned,
title: extractOgMeta(text).title ?? undefined,
}; };
} }
@@ -16,19 +16,45 @@ import type {
import { llmVision } from "./llmClient.js"; import { llmVision } from "./llmClient.js";
import { import {
acquireMediaAnalysisLock, acquireMediaAnalysisLock,
computeImagePhash,
deleteCachedMediaAnalysis, deleteCachedMediaAnalysis,
FAILED_ANALYSIS_PREFIX, FAILED_ANALYSIS_PREFIX,
getCachedMediaAnalysis, getCachedMediaAnalysis,
getCachedMediaByPhash,
inFlightVisionCalls, inFlightVisionCalls,
makeCustomEmojiCacheKey, makeCustomEmojiCacheKey,
makeImageCacheKey, makeImageCacheKey,
makeStickerCacheKey, makeStickerCacheKey,
upsertCachedMediaAnalysis, upsertCachedMediaAnalysis,
upsertCachedMediaByPhash,
visionLruCache, visionLruCache,
} from "./mediaCache.js"; } from "./mediaCache.js";
/**
* Detect vision outputs where the model claims it saw no image at all
* ("Maaf, saya tidak melihat gambar apapun...", "Tidak ada gambar yang
* terlampir...", "I cannot see any image..."). Such text is NOT a valid
* analysis caching it poisons the image cache for 24h (image/phash keys),
* so every re-analysis of the same image returns the "no image" text and the
* moderation LLM writes "lampiran gagal terbaca". These outputs must be
* treated as failures: never cached, and ignored when read back from cache.
*/
export function isNoImageSeenText(text: string | null | undefined): boolean {
if (!text) return false;
const lower = text.toLowerCase();
return (
/tidak (?:melihat|ada|terlihat) (?:gambar|foto|image)/i.test(lower) ||
/tidak (?:ada )?(?:gambar|foto|image) (?:apapun|yang terlampir)/i.test(
lower,
) ||
/gambar apapun/i.test(lower) ||
/tanpa (?:input )?(?:visual|gambar|image)/i.test(lower) ||
/\bno image (?:provided|attached|detected|found|was provided)?/i.test(
lower,
) ||
/(?:cannot|can't) see (?:any |an |the )?image/i.test(lower) ||
/i (?:do not|don't) (?:see|detect) (?:any |an |the )?image/i.test(lower) ||
/there (?:is|are) no image/i.test(lower)
);
}
import { import {
buildMediaCandidates, buildMediaCandidates,
downloadAndExtractFrame, downloadAndExtractFrame,
@@ -37,21 +63,27 @@ import {
} from "./mediaDownloader.js"; } from "./mediaDownloader.js";
import { import {
buildReferenceXml, buildReferenceXml,
buildUserProfileRef,
escapeXml, escapeXml,
formatReputationAttrs,
getAnalysisContent, getAnalysisContent,
resolveDisplayName,
resolveIsBot,
resolveIsEdited,
truncateForAi,
} from "./moderationBuilders.js"; } from "./moderationBuilders.js";
import { import {
buildCustomEmojiVisionPrompt, buildCustomEmojiVisionPrompt,
buildGeneralImageVisionPrompt, buildGeneralImageVisionPrompt,
buildStickerTextOnlyWarning, buildStickerTextOnlyWarning,
buildStickerVisionPrompt, buildStickerVisionPrompt,
sanitizeAiContent,
} from "./moderationPrompt.js"; } from "./moderationPrompt.js";
import { import {
extractSearchQueries, extractSearchQueries,
formatSearchResults, formatSearchResults,
searchSearxng, searchSearxng,
} from "./searxngSearch.js"; } from "./searxngSearch.js";
import { buildTermGlossaryBlock } from "./termGlossary.js";
import { extractUrlsFromText } from "./urlFetcher.js"; import { extractUrlsFromText } from "./urlFetcher.js";
import { getUserProfile } from "./userProfileStore.js"; import { getUserProfile } from "./userProfileStore.js";
import { initializeUserReputation } from "./userReputationStore.js"; import { initializeUserReputation } from "./userReputationStore.js";
@@ -110,18 +142,35 @@ export const analyzeSingleMediaImage = async (
// Layer 0: LRU // Layer 0: LRU
const lruCached = visionLruCache.get(cacheKey); const lruCached = visionLruCache.get(cacheKey);
if (lruCached) { if (lruCached && !isNoImageSeenText(lruCached)) {
log.debug({ cacheKey }, "Vision LRU cache HIT (in-memory)"); log.debug({ cacheKey }, "Vision LRU cache HIT (in-memory)");
return `[Media analysis for message ${messageId}] ${image.sourceLabel}: ${lruCached}`; return `[Media analysis for message ${messageId}] ${image.sourceLabel}: ${lruCached}`;
} }
if (lruCached) {
// Poisoned entry ("I see no image") — drop it and re-analyze.
log.warn({ cacheKey }, "Vision LRU cache HIT was no-image-seen — dropping");
visionLruCache.delete(cacheKey);
}
// Layer 1: DB // Layer 1: DB
const cached = await getCachedMediaAnalysis(cacheKey); const cached = await getCachedMediaAnalysis(cacheKey);
if (cached) { if (cached && !isNoImageSeenText(cached)) {
visionLruCache.set(cacheKey, cached); visionLruCache.set(cacheKey, cached);
log.debug({ cacheKey }, "Media analysis cache HIT (DB → LRU)"); log.debug(
{ cacheKey, messageId, cachedLen: cached.length },
"Media analysis cache HIT (DB → LRU)",
);
return `[Media analysis for message ${messageId}] ${image.sourceLabel}: ${cached}`; return `[Media analysis for message ${messageId}] ${image.sourceLabel}: ${cached}`;
} }
if (cached) {
// Poisoned DB entry — purge it so later messages re-analyze.
log.warn(
{ cacheKey },
"Media analysis cache HIT was no-image-seen — purging",
);
await deleteCachedMediaAnalysis(cacheKey).catch(() => {});
visionLruCache.delete(cacheKey);
}
// In-flight dedupe // In-flight dedupe
const existing = inFlightVisionCalls.get(cacheKey); const existing = inFlightVisionCalls.get(cacheKey);
@@ -154,39 +203,19 @@ export const analyzeSingleMediaImage = async (
return FAILED_ANALYSIS_PREFIX; return FAILED_ANALYSIS_PREFIX;
} }
// phash check
let phash: string | null = null;
if (image.image_url.url.startsWith("data:")) {
try {
const base64Data = image.image_url.url.split(",")[1];
if (base64Data) {
const imgBuffer = Buffer.from(base64Data, "base64");
phash = await computeImagePhash(imgBuffer);
if (phash) {
const phashCached = await getCachedMediaByPhash(phash);
if (phashCached) {
visionLruCache.set(cacheKey, phashCached);
await upsertCachedMediaAnalysis(
cacheKey,
phashCached,
"vision_llm",
Date.now() + 24 * 60 * 60 * 1000,
).catch(() => {});
return phashCached;
}
}
}
} catch {
phash = null;
}
}
// Vision API call // Vision API call
let lastError: Error | null = null; let lastError: Error | null = null;
for (let attempt = 0; attempt < 3; attempt++) { for (let attempt = 0; attempt < 3; attempt++) {
try { try {
const content = await llmVision(promptText, image.image_url); const content = await llmVision(promptText, image.image_url);
if (content) { if (content && !isNoImageSeenText(content)) {
// Defensive: log when a vision analysis is cached so we can trace
// if the SAME analysis text is being stored for DIFFERENT cache keys
// (which would indicate the vision model is returning duplicates).
log.debug(
{ cacheKey, messageId, contentLen: content.length },
"Vision analysis cached (new entry)",
);
await upsertCachedMediaAnalysis( await upsertCachedMediaAnalysis(
cacheKey, cacheKey,
content, content,
@@ -194,17 +223,18 @@ export const analyzeSingleMediaImage = async (
Date.now() + 24 * 60 * 60 * 1000, Date.now() + 24 * 60 * 60 * 1000,
); );
visionLruCache.set(cacheKey, content); visionLruCache.set(cacheKey, content);
if (phash) {
upsertCachedMediaByPhash(
phash,
content,
"vision_llm",
Date.now() + 7 * 24 * 60 * 60 * 1000,
).catch(() => {});
}
return content; return content;
} }
if (content) {
// Model claims it saw no image — same as a null response: NOT a
// valid analysis, and caching it would poison the cache key.
log.warn(
{ messageId, cacheKey },
"Vision returned no-image-seen text — not caching",
);
} else {
log.warn({ messageId }, "Vision API null response"); log.warn({ messageId }, "Vision API null response");
}
break; break;
} catch (err) { } catch (err) {
lastError = err instanceof Error ? err : new Error(String(err)); lastError = err instanceof Error ? err : new Error(String(err));
@@ -231,6 +261,7 @@ export const analyzeSingleMediaImage = async (
"Vision failed after 3 attempts", "Vision failed after 3 attempts",
); );
await deleteCachedMediaAnalysis(cacheKey).catch(() => {}); await deleteCachedMediaAnalysis(cacheKey).catch(() => {});
visionLruCache.delete(cacheKey);
return FAILED_ANALYSIS_PREFIX; return FAILED_ANALYSIS_PREFIX;
})(); })();
@@ -344,6 +375,12 @@ export async function prepareMediaMessage(
searxngXml = `\n<web_searches>\n${parts.join("\n")}\n</web_searches>`; searxngXml = `\n<web_searches>\n${parts.join("\n")}\n</web_searches>`;
} }
// Term glossary — cached per-word Wikipedia definitions for words the LLM
// may not know. Bounded and cached (in-memory + Redis), so this adds no
// meaningful latency to the media path either.
const glossaryXml = await buildTermGlossaryBlock([content]).catch(() => "");
const glossaryCtx = glossaryXml ? `\n${glossaryXml}` : "";
// Build XML block // Build XML block
const webTexts = webTextMap.get(targetId) ?? []; const webTexts = webTextMap.get(targetId) ?? [];
const mediaAnalyses = mediaAnalysisMap.get(targetId) ?? []; const mediaAnalyses = mediaAnalysisMap.get(targetId) ?? [];
@@ -366,7 +403,19 @@ export async function prepareMediaMessage(
const rep = await initializeUserReputation(target.user_id, target.guild_id); const rep = await initializeUserReputation(target.user_id, target.guild_id);
const profile = await getUserProfile(target.user_id); const profile = await getUserProfile(target.user_id);
const refXml = await buildReferenceXml(target); const refXml = await buildReferenceXml(target);
// Profile is emitted ONCE per batch in a <user_profiles> map (see
// mediaBatchProcessor); here we only reference it to avoid repeating the
// full summary on every message of the same user.
const profileRef = profile?.profile_summary?.trim()
? buildUserProfileRef(target.user_id)
: "";
const messageBlock = `<message id="${escapeXml(target.id)}" user="${escapeXml(target.username)}">\n <user_reputation trust_score="${rep.trust_score}" />${profile ? `\n <user_profile>${sanitizeAiContent(profile.profile_summary)}</user_profile>` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(content)}</content>${mediaContext ? ` ${escapeXml(mediaContext)}` : ""}${webContext}${mediaAnalysisContext}${searxngXml}\n</message>`; // Rich reputation — attrs only, no user history injection (per channel context preference)
const repAttrs = formatReputationAttrs(rep);
const repXml = `<user_reputation ${repAttrs}/>`;
const isBot = resolveIsBot(target);
const isEdited = resolveIsEdited(target);
const messageBlock = `<message id="${escapeXml(target.id)}" user="${escapeXml(resolveDisplayName(target))}" time="${new Date(target.created_at).toISOString()}"${isBot ? ` bot="true"` : ""}${isEdited ? ` edited="true"` : ""}>\n ${repXml}${profileRef ? `\n ${profileRef}` : ""}${refXml ? `\n ${refXml}` : ""}\n <content>${escapeXml(truncateForAi(content))}</content>${mediaContext ? ` ${escapeXml(mediaContext)}` : ""}${webContext}${mediaAnalysisContext}${searxngXml}${glossaryCtx}\n</message>`;
return { targetId, messageBlock }; return { targetId, messageBlock };
} }
@@ -4,10 +4,17 @@ import { createChildLogger } from "@/shared/logger/index";
const log = createChildLogger("imageResizer"); const log = createChildLogger("imageResizer");
/** /**
* Prepare an image buffer for optimal vision LLM analysis. * Prepare an image buffer for vision LLM analysis.
* *
* - Resizes to maxDim x maxDim maintaining aspect ratio (only if larger) * - Resizes to maxDim x maxDim maintaining aspect ratio WITHOUT upscaling
* - Converts to PNG (lossless) to preserve full image detail * (small images such as stickers/emojis are passed through at original size,
* just re-encoded)
* - Encodes as JPEG (lossy, quality ~85). Photos compress VERY poorly in
* lossless PNG a 1024px Facebook photo balloons to multi-MB PNG base64 that
* the vision model silently rejects (empty response "Vision API null").
* JPEG keeps the same photo at ~100400KB, which the model processes fine and
* stays well below request/token size limits.
* - Falls back to original buffer if sharp fails * - Falls back to original buffer if sharp fails
* *
* @param buf - Raw image buffer * @param buf - Raw image buffer
@@ -20,22 +27,17 @@ export async function resizeImageForVision(
): Promise<{ data: Buffer; mimeType: string }> { ): Promise<{ data: Buffer; mimeType: string }> {
try { try {
const metadata = await sharp(buf).metadata(); const metadata = await sharp(buf).metadata();
const inputFormat = metadata.format ?? "jpeg";
// Skip resize entirely if already within max dimension // Always (re-)encode to JPEG and fit inside maxDim without upscaling.
if ((metadata.width ?? 0) <= maxDim && (metadata.height ?? 0) <= maxDim) { // Skipping the encode for already-small images left raw originals in
return { data: buf, mimeType: `image/${inputFormat}` }; // their native (often lossless PNG or full-quality) form, which could
} // still bloat data URLs and trip the vision model's size limit.
// Resize dimension only — convert to PNG lossless to preserve detail
const resized = await sharp(buf) const resized = await sharp(buf)
.resize(maxDim, maxDim, { .resize(maxDim, maxDim, { fit: "inside", withoutEnlargement: true })
fit: "inside", .jpeg({ quality: 85 })
withoutEnlargement: true,
})
.png()
.toBuffer(); .toBuffer();
const inputFormat = metadata.format ?? "jpeg";
log.debug( log.debug(
{ {
originalSize: buf.length, originalSize: buf.length,
@@ -45,10 +47,10 @@ export async function resizeImageForVision(
((buf.length - resized.length) / buf.length) * 100, ((buf.length - resized.length) / buf.length) * 100,
), ),
}, },
"Image resized for vision analysis (lossless PNG)", "Image resized for vision analysis (JPEG)",
); );
return { data: resized, mimeType: "image/png" }; return { data: resized, mimeType: "image/jpeg" };
} catch (error) { } catch (error) {
log.warn( log.warn(
{ error: error instanceof Error ? error.message : String(error) }, { error: error instanceof Error ? error.message : String(error) },
@@ -16,6 +16,7 @@ import {
createHandlerRegistry, createHandlerRegistry,
} from "./handler-registry.js"; } from "./handler-registry.js";
import { MediaHandler } from "./media.handler.js"; import { MediaHandler } from "./media.handler.js";
import { wireMediaStatusWriter } from "./mediaStatusSink.js";
import { ModerationHandler } from "./moderation.handler.js"; import { ModerationHandler } from "./moderation.handler.js";
import { VoiceHandler } from "./voice.handler.js"; import { VoiceHandler } from "./voice.handler.js";
@@ -82,12 +83,14 @@ export class CommandHandler {
// Create domain-specific handlers with their dependencies // Create domain-specific handlers with their dependencies
this.voiceHandler = new VoiceHandler(client, voiceController); this.voiceHandler = new VoiceHandler(client, voiceController);
this.mediaHandler = new MediaHandler(client, () => this.mediaHandler = new MediaHandler();
voiceController.getStatus(),
);
this.guildHandler = new GuildHandler(client); this.guildHandler = new GuildHandler(client);
this.moderationHandler = new ModerationHandler(client); this.moderationHandler = new ModerationHandler(client);
// Wire the media status sink so MediaHandler can persist status on
// queue advances that happen outside a command (natural track end).
wireMediaStatusWriter(this.redisPub);
// Build the command registry // Build the command registry
this.registry = createHandlerRegistry( this.registry = createHandlerRegistry(
this.voiceHandler, this.voiceHandler,
@@ -1,6 +1,7 @@
import { import {
COMMAND_GUILDS_LIST, COMMAND_GUILDS_LIST,
COMMAND_GUILDS_TEXT_CHANNELS, COMMAND_GUILDS_TEXT_CHANNELS,
COMMAND_MEDIA_LOOP,
COMMAND_MEDIA_QUEUE, COMMAND_MEDIA_QUEUE,
COMMAND_MEDIA_SKIP, COMMAND_MEDIA_SKIP,
COMMAND_MEDIA_STOP, COMMAND_MEDIA_STOP,
@@ -69,6 +70,7 @@ export function createHandlerRegistry(
registry.set(COMMAND_MEDIA_VOLUME, (cmd) => registry.set(COMMAND_MEDIA_VOLUME, (cmd) =>
mediaHandler.handleMediaVolume(cmd), mediaHandler.handleMediaVolume(cmd),
); );
registry.set(COMMAND_MEDIA_LOOP, (cmd) => mediaHandler.handleMediaLoop(cmd));
// Guild commands // Guild commands
registry.set(COMMAND_GUILDS_LIST, (cmd) => registry.set(COMMAND_GUILDS_LIST, (cmd) =>
@@ -1,21 +1,18 @@
import { randomUUID } from "node:crypto"; import { randomUUID } from "node:crypto";
import { StreamType } from "@discordjs/voice"; import { StreamType } from "@discordjs/voice";
import type { Client } from "discord.js-selfbot-v13";
import type { CommandMessage, CommandReply } from "../../shared/index.js"; import type { CommandMessage, CommandReply } from "../../shared/index.js";
import { createChildLogger } from "../../shared/logger/index.js"; import { createChildLogger } from "../../shared/logger/index.js";
import { import {
extractMediaInfo, extractMediaInfo,
resolveMediaUrl, resolveMediaUrl,
transcodeToHighQualityOgg,
} from "../voice-recording/mediaSource.js"; } from "../voice-recording/mediaSource.js";
import type { import type {
MediaMode, MediaMode,
MediaQueueItem, MediaQueueItem,
} from "../voice-recording/mediaTypes.js"; } from "../voice-recording/mediaTypes.js";
import { discordPlayer } from "../voice-recording/player.js"; import { discordPlayer } from "../voice-recording/player.js";
import { import { setMediaStatusKey } from "./mediaStatusSink.js";
ScreenShareController,
type ScreenShareVoiceStatus,
} from "../voice-recording/screenShareController.js";
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Types // Types
@@ -34,6 +31,7 @@ export interface MediaStatusPayload {
playing: boolean; playing: boolean;
activeMode: MediaMode | null; activeMode: MediaMode | null;
musicVolume: number; musicVolume: number;
loop: boolean;
current: MediaStatusItem | null; current: MediaStatusItem | null;
queue: MediaStatusItem[]; queue: MediaStatusItem[];
} }
@@ -44,6 +42,9 @@ export interface MediaStatusPayload {
const mediaQueue: MediaQueueItem[] = []; const mediaQueue: MediaQueueItem[] = [];
let currentTrackItem: MediaQueueItem | null = null; let currentTrackItem: MediaQueueItem | null = null;
let loopEnabled = false;
/** Active ffmpeg transcode (killed on stop/skip). */
let currentTranscodeCleanup: (() => void) | null = null;
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Helpers // Helpers
@@ -66,6 +67,7 @@ function buildStatusPayload(): MediaStatusPayload {
currentTrackItem !== null && discordPlayer.getStatus() === "playing", currentTrackItem !== null && discordPlayer.getStatus() === "playing",
activeMode: currentTrackItem?.mode ?? null, activeMode: currentTrackItem?.mode ?? null,
musicVolume: discordPlayer.getMusicVolume(), musicVolume: discordPlayer.getMusicVolume(),
loop: loopEnabled,
current: currentTrackItem ? mapToStatusItem(currentTrackItem) : null, current: currentTrackItem ? mapToStatusItem(currentTrackItem) : null,
queue: mediaQueue.map(mapToStatusItem), queue: mediaQueue.map(mapToStatusItem),
}; };
@@ -77,32 +79,45 @@ function buildStatusPayload(): MediaStatusPayload {
export class MediaHandler { export class MediaHandler {
private logger = createChildLogger("media-handler"); private logger = createChildLogger("media-handler");
private screenController: ScreenShareController | null = null;
private screenPlayback: { stop(): void } | null = null;
constructor( constructor() {
private readonly client: Client | null = null, // Register auto-advance on natural track end. advanceQueue mutates the
private readonly getVoiceStatus: () => ScreenShareVoiceStatus = () => ({ // module-level currentTrackItem/queue, so we must re-publish the status
connected: false, // key afterward: otherwise the backend's Redis `media:status` cache (and
activeGuildId: null, // the frontend's 10s polling) stays stuck on the finished track.
activeChannelId: null,
}),
) {
// Register auto-advance on natural track end
discordPlayer.onIdle(() => { discordPlayer.onIdle(() => {
this.advanceQueue().catch((err) => { this.advanceQueue()
.then(() => this.publishStatus())
.catch((err) => {
this.logger.error({ err }, "Auto-advance failed"); this.logger.error({ err }, "Auto-advance failed");
}); });
}); });
} }
/**
* Persist the latest media state to Redis so the backend/frontend see queue
* advances that happen outside a command (natural track end). CommandHandler
* owns the Redis status-key writes for command-triggered changes; this
* covers the side-effect-only path.
*/
private publishStatus(): void {
try {
setMediaStatusKey(this.getCurrentMediaStatus());
} catch (err: unknown) {
this.logger.warn(
{ error: err instanceof Error ? err.message : String(err) },
"Failed to publish media status on track end",
);
}
}
getCurrentMediaStatus(): MediaStatusPayload { getCurrentMediaStatus(): MediaStatusPayload {
return buildStatusPayload(); return buildStatusPayload();
} }
async handleMediaQueue(cmd: CommandMessage): Promise<CommandReply<unknown>> { async handleMediaQueue(cmd: CommandMessage): Promise<CommandReply<unknown>> {
const url = String(cmd.payload.url ?? "").trim(); // Accept both `url` (canonical) and `source` (legacy FE) for resilience.
const mode: MediaMode = cmd.payload.mode === "screen" ? "screen" : "music"; const url = String(cmd.payload.url ?? cmd.payload.source ?? "").trim();
const requestedBy = String(cmd.payload.requestedBy ?? "unknown"); const requestedBy = String(cmd.payload.requestedBy ?? "unknown");
if (!url) { if (!url) {
@@ -125,50 +140,6 @@ export class MediaHandler {
}; };
} }
// Screen share (GoLive) path — bypasses the audio queue entirely.
if (mode === "screen") {
try {
if (!this.client) {
return {
id: cmd.id,
success: false,
data: null,
error: "Gateway client not initialized",
};
}
if (!this.screenController) {
this.screenController = new ScreenShareController(
this.client,
this.getVoiceStatus,
);
}
const playback = await this.screenController.start(url);
this.screenPlayback = playback;
currentTrackItem = {
id: randomUUID(),
source: url,
title: url,
kind: "url",
mode: "screen",
requestedBy,
addedAt: Date.now(),
status: "playing",
};
playback.done.finally(() => {
this.screenPlayback = null;
if (currentTrackItem?.mode === "screen") {
currentTrackItem = null;
}
});
this.logger.info({ url }, "Screen share started");
return { id: cmd.id, success: true, data: buildStatusPayload() };
} catch (err) {
const message = err instanceof Error ? err.message : String(err);
this.logger.error({ error: message }, "Screen share failed to start");
return { id: cmd.id, success: false, data: null, error: message };
}
}
// Lightweight metadata fetch for display — the full resolve happens in playNext // Lightweight metadata fetch for display — the full resolve happens in playNext
let title: string = url; let title: string = url;
let duration: number | undefined; let duration: number | undefined;
@@ -190,7 +161,7 @@ export class MediaHandler {
source: url, source: url,
title, title,
kind: "url" as const, kind: "url" as const,
mode, mode: "music",
requestedBy, requestedBy,
addedAt: Date.now(), addedAt: Date.now(),
status: "queued", status: "queued",
@@ -234,12 +205,6 @@ export class MediaHandler {
} }
async handleMediaStop(cmd: CommandMessage): Promise<CommandReply<unknown>> { async handleMediaStop(cmd: CommandMessage): Promise<CommandReply<unknown>> {
// Stop screen share if active — playback.done.finally clears the item.
this.screenPlayback?.stop();
this.screenPlayback = null;
if (currentTrackItem?.mode === "screen") {
currentTrackItem = null;
}
discordPlayer.stop("music"); discordPlayer.stop("music");
currentTrackItem = null; currentTrackItem = null;
mediaQueue.length = 0; // Clear entire queue mediaQueue.length = 0; // Clear entire queue
@@ -268,6 +233,17 @@ export class MediaHandler {
}; };
} }
async handleMediaLoop(cmd: CommandMessage): Promise<CommandReply<unknown>> {
loopEnabled = Boolean(cmd.payload.loop);
this.logger.info({ loop: loopEnabled }, "Media loop toggled");
this.publishStatus();
return {
id: cmd.id,
success: true,
data: buildStatusPayload(),
};
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Internal // Internal
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -282,6 +258,8 @@ export class MediaHandler {
discordPlayer.stop("music"); discordPlayer.stop("music");
currentTrackItem = null; currentTrackItem = null;
} }
currentTranscodeCleanup?.();
currentTranscodeCleanup = null;
const next = mediaQueue.shift(); const next = mediaQueue.shift();
if (!next) { if (!next) {
@@ -304,11 +282,20 @@ export class MediaHandler {
next.title = resolution.title ?? next.title; next.title = resolution.title ?? next.title;
next.duration = resolution.duration ?? next.duration; next.duration = resolution.duration ?? next.duration;
discordPlayer.playStream(resolution.stream, "music", { // Music playback: transcode once to high-quality OggOpus (48kHz stereo,
inputType: StreamType.Arbitrary, // 192kbps) with volume baked into the encode. This avoids the double
inlineVolume: true, // lossy encode that inlineVolume would cause and gives Discord the
volume: discordPlayer.getMusicVolume(), // cleanest possible stream.
const transcoded = transcodeToHighQualityOgg(
resolution.stream,
discordPlayer.getMusicVolume(),
);
discordPlayer.playStream(transcoded.stream, "music", {
inputType: StreamType.OggOpus,
inlineVolume: false,
}); });
currentTranscodeCleanup = transcoded.cleanup;
this.logger.info({ title: next.title }, "Playback started"); this.logger.info({ title: next.title }, "Playback started");
} catch (err) { } catch (err) {
@@ -335,10 +322,21 @@ export class MediaHandler {
/** /**
* Called by the idle callback delegates to playNext since the player is * Called by the idle callback delegates to playNext since the player is
* already idle and currentTrackItem is already null. * already idle and currentTrackItem is already null. When loop mode is
* enabled and a music track ended naturally, requeue it so it plays again.
*/ */
private async advanceQueue(): Promise<void> { private async advanceQueue(): Promise<void> {
const finished = currentTrackItem;
currentTrackItem = null; currentTrackItem = null;
if (loopEnabled && finished && finished.mode === "music") {
mediaQueue.unshift(finished);
this.logger.info(
{ title: finished.title },
"Loop enabled — replaying finished track",
);
}
await this.playNext(); await this.playNext();
} }
} }
@@ -0,0 +1,41 @@
import type Redis from "ioredis";
import { createChildLogger } from "@/shared/logger/index";
import { MEDIA_STATUS_KEY } from "../../shared/redis-channels.js";
/**
* Shared sink for writing the media status Redis key.
*
* CommandHandler owns the publisher + status writes for command-triggered
* changes (`publishMediaStatus`). MediaHandler needs to also persist status
* when the queue advances *outside* a command (natural track end / screen-share
* done), so we expose the real publisher here and let CommandHandler wire it
* once at startup.
*/
const logger = createChildLogger("media-status-sink");
let _setMediaStatusKey: ((payload: unknown) => void) | null = null;
export function setMediaStatusWriter(writer: (payload: unknown) => void): void {
_setMediaStatusKey = writer;
}
export function setMediaStatusKey(payload: unknown): void {
if (!_setMediaStatusKey) {
logger.warn("Media status writer not wired — skipping status publish");
return;
}
_setMediaStatusKey(payload);
}
export { MEDIA_STATUS_KEY };
export function wireMediaStatusWriter(redisPub: Redis): void {
setMediaStatusWriter((payload) => {
redisPub
.set(MEDIA_STATUS_KEY, JSON.stringify(payload))
.catch((err: unknown) => {
const msg = err instanceof Error ? err.message : String(err);
logger.warn({ error: msg }, "Failed to update media status key");
});
});
}
@@ -128,6 +128,9 @@ export class VoiceHandler {
id: c.id, id: c.id,
name: c.name, name: c.name,
type: "voice" as const, type: "voice" as const,
// selfbot exposes joinable (permission check) — let FE filter
// channels the account actually may join.
joinable: (c as { joinable?: boolean }).joinable ?? true,
})); }));
return { id: cmd.id, success: true, data: voiceChannels }; return { id: cmd.id, success: true, data: voiceChannels };
@@ -35,6 +35,13 @@ export interface MessageLocationInput {
} }
const EXCLUDED_CHANNEL_IDS = new Set(config.EXCLUDED_CHANNEL_IDS); const EXCLUDED_CHANNEL_IDS = new Set(config.EXCLUDED_CHANNEL_IDS);
const BOT_EXCLUDED_CHANNEL_IDS = new Set(config.BOT_EXCLUDED_CHANNEL_IDS);
function isBotExcludedChannel(message: Message): boolean {
if (!message.author?.bot) return false;
const id = getParentChannelId(message) ?? message.channelId;
return id != null && BOT_EXCLUDED_CHANNEL_IDS.has(id);
}
const EXCLUDED_THREAD_IDS = new Set(config.EXCLUDED_THREAD_IDS); const EXCLUDED_THREAD_IDS = new Set(config.EXCLUDED_THREAD_IDS);
function isExcludedThread(message: { function isExcludedThread(message: {
@@ -274,7 +281,7 @@ export function registerMessageCapture(client: Client): void {
client.on("messageCreate", async (message) => { client.on("messageCreate", async (message) => {
if (!shouldCaptureForAnyTarget(message, targets)) return; if (!shouldCaptureForAnyTarget(message, targets)) return;
if (message.author?.bot) return; if (isBotExcludedChannel(message)) return;
if (isAgeRestrictedMessage(message)) return; if (isAgeRestrictedMessage(message)) return;
if (isExcludedThread(message)) return; if (isExcludedThread(message)) return;
@@ -293,7 +300,7 @@ export function registerMessageCapture(client: Client): void {
client.on("messageUpdate", async (_oldMessage, newMessage) => { client.on("messageUpdate", async (_oldMessage, newMessage) => {
if (!shouldCaptureForAnyTarget(newMessage, targets)) return; if (!shouldCaptureForAnyTarget(newMessage, targets)) return;
if (newMessage.author?.bot) return; if (isBotExcludedChannel(newMessage as Message)) return;
if (isAgeRestrictedMessage(newMessage as Message)) return; if (isAgeRestrictedMessage(newMessage as Message)) return;
if (isExcludedThread(newMessage)) return; if (isExcludedThread(newMessage)) return;
@@ -343,6 +350,20 @@ export function registerMessageCapture(client: Client): void {
id: newMessage.id, id: newMessage.id,
edited_content: getDisplayContent(newMessage as Message), edited_content: getDisplayContent(newMessage as Message),
edited_at: editedAt, edited_at: editedAt,
type: "edited",
// Match the DB update (updateMessageAsEdited resets analysis to
// pending) so the live UI reflects the same state instead of
// lingering on the stale pre-edit verdict.
ai_status: "pending",
ai_moderation_flags: null,
ai_moderation_score: null,
ai_analysis: null,
ai_categories: null,
ai_severity: null,
ai_confidence: null,
ai_recommended_action: null,
ai_analyzed_at: null,
ai_error: null,
}); });
} }
} else if (newMessage.author) { } else if (newMessage.author) {
@@ -9,6 +9,10 @@ export interface MessageLocation {
threadId: string | null; threadId: string | null;
threadName: string | null; threadName: string | null;
channelName: string | null; channelName: string | null;
/** Channel topic (resmi/deskripsi channel) strong context for judging
* whether a message fits the channel's purpose. Guarded: some channel
* types (threads on older API builds) expose no topic. */
topic?: string | null;
nsfw?: boolean; nsfw?: boolean;
nsfwLevel?: string | null; nsfwLevel?: string | null;
ageRestricted?: boolean; ageRestricted?: boolean;
@@ -107,12 +111,17 @@ export function getMessageLocation(message: Message): MessageLocation {
nsfw?: boolean; nsfw?: boolean;
nsfwLevel?: string | null; nsfwLevel?: string | null;
}; };
const topic =
"topic" in channel && typeof channel.topic === "string"
? channel.topic
: null;
if (!channel.isThread?.()) { if (!channel.isThread?.()) {
return { return {
channelId: message.channelId, channelId: message.channelId,
threadId: null, threadId: null,
threadName: null, threadName: null,
channelName: "name" in channel ? channel.name : null, channelName: "name" in channel ? channel.name : null,
topic,
nsfw: nsfw:
typeof safetyChannel.nsfw === "boolean" typeof safetyChannel.nsfw === "boolean"
? safetyChannel.nsfw ? safetyChannel.nsfw
@@ -133,6 +142,7 @@ export function getMessageLocation(message: Message): MessageLocation {
threadId: channel.id, threadId: channel.id,
threadName: channel.name, threadName: channel.name,
channelName: channel.parent?.name ?? null, channelName: channel.parent?.name ?? null,
topic,
nsfw: nsfw:
typeof safetyChannel.nsfw === "boolean" ? safetyChannel.nsfw : undefined, typeof safetyChannel.nsfw === "boolean" ? safetyChannel.nsfw : undefined,
nsfwLevel: nsfwLevel:
@@ -505,7 +515,7 @@ export function renderDiscordMentions(
content: string, content: string,
metadata: string | null | undefined, metadata: string | null | undefined,
): string { ): string {
if (!content || !content.includes("<")) return content; if (!content?.includes("<")) return content;
const parsed = parseRichMessageMetadata(metadata); const parsed = parseRichMessageMetadata(metadata);
const roleName = new Map( const roleName = new Map(
(parsed?.mentionedRoles ?? []).map((r) => [r.id, r.name] as const), (parsed?.mentionedRoles ?? []).map((r) => [r.id, r.name] as const),
@@ -26,6 +26,8 @@ export interface AIAnalysisUpdate {
confidence?: number | null; confidence?: number | null;
recommendedAction?: MessageRecord["ai_recommended_action"] | null; recommendedAction?: MessageRecord["ai_recommended_action"] | null;
analyzedAt?: number | null; analyzedAt?: number | null;
/** Wall-clock time the AI analysis (LLM call) took, in milliseconds. */
analysisDurationMs?: number | null;
error?: string | null; error?: string | null;
} }
@@ -42,6 +44,7 @@ function buildAIAnalysisSet(result: AIAnalysisUpdate, now?: number) {
ai_confidence: result.confidence ?? result.score ?? null, ai_confidence: result.confidence ?? result.score ?? null,
ai_recommended_action: result.recommendedAction ?? null, ai_recommended_action: result.recommendedAction ?? null,
ai_analyzed_at: result.analyzedAt ?? now ?? Date.now(), ai_analyzed_at: result.analyzedAt ?? now ?? Date.now(),
ai_analysis_duration_ms: result.analysisDurationMs ?? null,
ai_error: result.error ?? null, ai_error: result.error ?? null,
}; };
} }
@@ -1,4 +1,4 @@
import { and, desc, eq, inArray, type SQL, sql } from "drizzle-orm"; import { and, desc, eq, inArray, type SQL } from "drizzle-orm";
import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import type { NodePgDatabase } from "drizzle-orm/node-postgres";
import type * as schema from "../../shared/database/schema.js"; import type * as schema from "../../shared/database/schema.js";
import { moderationActionsTable } from "../../shared/database/schema.js"; import { moderationActionsTable } from "../../shared/database/schema.js";
@@ -1,4 +1,4 @@
import { and, eq, isNull, or } from "drizzle-orm"; import { and, eq, isNull } from "drizzle-orm";
import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import type { NodePgDatabase } from "drizzle-orm/node-postgres";
import { createChildLogger, type Logger } from "@/shared/logger/index"; import { createChildLogger, type Logger } from "@/shared/logger/index";
import type * as schema from "../../shared/database/schema.js"; import type * as schema from "../../shared/database/schema.js";
@@ -1,4 +1,4 @@
import { and, desc, eq, inArray, type SQL, sql } from "drizzle-orm"; import { and, desc, eq, inArray, type SQL } from "drizzle-orm";
import type { NodePgDatabase } from "drizzle-orm/node-postgres"; import type { NodePgDatabase } from "drizzle-orm/node-postgres";
import type * as schema from "../../shared/database/schema.js"; import type * as schema from "../../shared/database/schema.js";
import { messageReviewsTable } from "../../shared/database/schema.js"; import { messageReviewsTable } from "../../shared/database/schema.js";
@@ -1,4 +1,7 @@
import { type ChildProcess, spawn } from "node:child_process"; import { type ChildProcess, spawn } from "node:child_process";
import { chmodSync, existsSync, readFileSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { PassThrough, type Readable } from "node:stream"; import { PassThrough, type Readable } from "node:stream";
import { StreamType } from "@discordjs/voice"; import { StreamType } from "@discordjs/voice";
import { createChildLogger } from "@/shared/logger/index"; import { createChildLogger } from "@/shared/logger/index";
@@ -33,6 +36,87 @@ export interface ResolveOptions {
quality?: string; quality?: string;
} }
export interface TranscodeResult {
stream: Readable;
/** Kill the ffmpeg child (used on stop/skip). */
cleanup: () => void;
}
/**
* Re-encode a source stream to high-quality OggOpus (48kHz stereo, 192kbps).
*
* Discord voice downmixes whatever we feed it to the channel's bitrate, so the
* best we can do is hand it a clean 48kHz stereo Opus stream instead of the
* raw source (which may be mono, low-bitrate, or a non-Opus container). The
* volume is baked into the encode with `-af volume=` so the player does not
* need inlineVolume re-encoding (double lossy encode).
*/
export function transcodeToHighQualityOgg(
input: Readable,
volume: number,
): TranscodeResult {
const proc = spawn(
"ffmpeg",
[
"-hide_banner",
"-loglevel",
"error",
"-i",
"pipe:0",
"-vn",
"-ac",
"2",
"-ar",
"48000",
"-c:a",
"libopus",
"-b:a",
"192k",
"-af",
`volume=${volume}`,
"-f",
"ogg",
"pipe:1",
],
{ stdio: ["pipe", "pipe", "ignore"] },
);
input.pipe(proc.stdin);
// ffmpeg teardown closes stdin while the upstream source may still write —
// swallow EPIPE / destroyed-stream errors so they don't crash the gateway.
proc.stdin.on("error", (err: NodeJS.ErrnoException) => {
if (
err.code === "EPIPE" ||
err.code === "ERR_STREAM_DESTROYED" ||
err.code === "ERR_STREAM_WRITE_AFTER_END"
) {
logger.debug(
{ code: err.code },
"Transcode stdin closed during teardown",
);
} else {
logger.error({ error: err.message }, "Transcode stdin error");
}
});
activeProcesses.add(proc);
const cleanup = () => {
activeProcesses.delete(proc);
if (proc.exitCode === null) {
proc.kill("SIGKILL");
}
};
proc.once("exit", () => activeProcesses.delete(proc));
// If ffmpeg fails, surface the error to the consumer stream so the
// AudioPlayer's error handler can advance the queue.
const output = proc.stdout;
output.on("error", () => cleanup());
return { stream: output, cleanup };
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Internal state // Internal state
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -62,90 +146,67 @@ function parseSeconds(value: string): number {
*/ */
const MAX_HEADER_BUFFER = 65536; // 64KB safety limit for metadata headers const MAX_HEADER_BUFFER = 65536; // 64KB safety limit for metadata headers
function readFirstTwoLines( /**
stdout: Readable, * Read the title + duration header lines from yt-dlp's STDERR.
*
* When yt-dlp streams media to stdout (`-o -`) it redirects its `--print`
* output to STDERR so the media stream on stdout stays clean. The first two
* meaningful lines on stderr are then the title and (before_dl) duration.
*
* Blank lines and `[...]` info prefixes are skipped. An `ERROR:` line is
* reported via `onError` so a failing download surfaces as a resolution error
* instead of a silent empty stream.
*/
function readStderrHeader(
stderr: Readable,
onError: (message: string) => void,
maxBufferSize: number = MAX_HEADER_BUFFER, maxBufferSize: number = MAX_HEADER_BUFFER,
): Promise<{ ): Promise<{ title: string; duration: number }> {
title: string; return new Promise((resolve) => {
duration: number; let buffer = "";
remaining: Readable;
}> {
return new Promise((resolve, reject) => {
const passThrough = new PassThrough();
let buffer = Buffer.alloc(0);
let title = ""; let title = "";
let stage: "title" | "duration" | "done" = "title"; let duration = 0;
let done = false;
function cleanup() { const finish = () => {
stdout.removeListener("data", onData); if (done) return;
stdout.removeListener("error", onError); done = true;
stdout.removeListener("end", onEnd); stderr.removeListener("data", onData);
} resolve({ title, duration });
};
function onData(chunk: Buffer) { const onData = (chunk: Buffer) => {
if (stage === "done") return; if (done) return;
buffer = Buffer.concat([buffer, chunk]); buffer += chunk.toString("utf8");
if (buffer.length > maxBufferSize) { if (buffer.length > maxBufferSize) {
cleanup(); finish();
reject(new Error(`Metadata header exceeded ${maxBufferSize} bytes`));
return; return;
} }
processBuffer(); while (!done) {
const nl = buffer.indexOf("\n");
if (nl === -1) break; // need more data
const line = buffer.slice(0, nl).trim();
buffer = buffer.slice(nl + 1);
if (line.length === 0) continue; // blank line
if (line.startsWith("[")) continue; // "[info] ..." — not a header
if (line.startsWith("ERROR")) {
onError(line);
finish();
return;
} }
if (!title) {
function processBuffer() {
while (buffer.length > 0 && stage !== "done") {
const nl = buffer.indexOf(0x0a); // '\n' byte
if (nl === -1) break; // Need more data
const line = buffer.subarray(0, nl).toString("utf8").trim();
buffer = buffer.subarray(nl + 1);
if (stage === "title") {
title = line; title = line;
stage = "duration"; } else {
} else if (stage === "duration") { duration = parseSeconds(line);
const duration = parseSeconds(line); finish();
stage = "done";
cleanup();
// Write any buffered data that follows the second newline
if (buffer.length > 0) {
passThrough.write(buffer);
}
// Pipe the remainder of stdout into the pass-through
stdout.pipe(passThrough);
resolve({ title, duration, remaining: passThrough });
return; return;
} }
} }
} };
function onError(err: Error) { stderr.on("data", onData);
if (stage !== "done") { stderr.on("end", finish);
cleanup();
reject(err);
}
}
function onEnd() {
if (stage !== "done") {
cleanup();
reject(
new Error(
`yt-dlp stdout ended before metadata could be read. ` +
`Stage: ${stage}, partial title: "${title}"`,
),
);
}
}
stdout.on("data", onData);
stdout.on("error", onError);
stdout.on("end", onEnd);
}); });
} }
@@ -156,6 +217,85 @@ function buildNotInstalledError(): Error {
); );
} }
/**
* Build the yt-dlp --cookies args. YouTube blocks anonymous embeds with a
* "Sign in to confirm you're not a bot" 403 unless yt-dlp is given a logged-
* in account's cookies. The path is configurable via GMW_YT_COOKIES_PATH
* (default: the BWS-provided file the deploy writes to /etc/.../ytcookies.txt).
* If the file doesn't exist we pass nothing and fall back to anon (YouTube
* may 403 playback will fail gracefully, not crash).
*/
var _cachedCookiePath: string | null = null;
function buildCookieArgs(): string[] {
// Single source of truth: BWS injects the account cookies via env
// (gmw_yt_downloader_cookies → GMW_YT_DOWNLOADER_COOKIES by bws-exec).
// We materialize them to a temp Netscape file because yt-dlp --cookies
// only accepts a file path, not stdin, and multiline env values are not
// reliable to pass directly on the spawn argv. Falls back to the on-disk
// file at GMW_YT_COOKIES_PATH (or /etc/gmw-discord-gateway/ytcookies.txt)
// which the Nix deploy writes from BWS once at start.
if (_cachedCookiePath) return ["--cookies", _cachedCookiePath];
const envCookies = process.env.GMW_YT_DOWNLOADER_COOKIES?.trim();
if (envCookies?.includes("LOGIN_INFO")) {
const fdPath = join(tmpdir(), `gmw-ytcookies.${process.pid}.txt`);
writeFileSync(fdPath, envCookies);
try {
chmodSync(fdPath, 0o600);
} catch {
/* best-effort */
}
_cachedCookiePath = fdPath;
logger.info(
{ cookiePath: fdPath, source: "GMW_YT_DOWNLOADER_COOKIES env" },
"Using YouTube cookies (from BWS env)",
);
return ["--cookies", fdPath];
}
const cookiePath =
process.env.GMW_YT_COOKIES_PATH ?? "/etc/gmw-discord-gateway/ytcookies.txt";
try {
if (cookiePath && existsSync(cookiePath)) {
// Never hand the ORIGINAL system file to yt-dlp: recent yt-dlp rewrites
// the cookie file on close (`--cookies` implies write-back). The system
// file is owned by another user (root/deploy) and the service user
// cannot write it → PermissionError → yt-dlp exits 1 → playback
// fails for every attempt. Copy to a per-run temp file (like the env
// branch above) so write-back lands somewhere we own; if the original
// is not readable we fall back to anonymous (YouTube may 403 → the
// screen-share controller retries via Invidious mirrors without auth).
let cookieContents: string;
try {
cookieContents = readFileSync(cookiePath, "utf8");
} catch {
logger.warn(
{ cookiePath },
"Cookie file not readable; continuing without cookies (anon)",
);
return [];
}
const fdPath = join(tmpdir(), `gmw-ytcookies.${process.pid}.txt`);
writeFileSync(fdPath, cookieContents);
try {
chmodSync(fdPath, 0o600);
} catch {
/* best-effort */
}
_cachedCookiePath = fdPath;
logger.info(
{ cookiePath: fdPath, source: "on-disk file (copied)" },
"Using YouTube cookies for yt-dlp",
);
return ["--cookies", fdPath];
}
} catch {
/* ignore — fallback to anon */
}
logger.warn(
"No YouTube cookies available; yt-dlp will use anonymous (YouTube may 403)",
);
return [];
}
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
// Public API // Public API
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@@ -163,8 +303,10 @@ function buildNotInstalledError(): Error {
/** /**
* Resolve a media URL (YouTube, Spotify, etc.) to a playable audio stream. * Resolve a media URL (YouTube, Spotify, etc.) to a playable audio stream.
* *
* Spawns `yt-dlp`, extracts the title and duration from the first two stdout * Spawns `yt-dlp` with `-o -` so the raw audio bytes stream on stdout. Since
* lines, then pipes the remaining raw audio data into a Readable stream. * stdout is the media sink, yt-dlp emits its `--print before_dl:title` /
* `before_dl:duration` header lines on STDERR the title + duration are read
* from there and the stdout media stream is returned untouched.
* *
* The returned stream uses `StreamType.Arbitrary` suitable for * The returned stream uses `StreamType.Arbitrary` suitable for
* `DiscordPlayer.playStream()` with `inputType: StreamType.Arbitrary`. * `DiscordPlayer.playStream()` with `inputType: StreamType.Arbitrary`.
@@ -178,13 +320,15 @@ export function resolveMediaUrl(
): Promise<MediaSourceResolution> { ): Promise<MediaSourceResolution> {
return new Promise<MediaSourceResolution>((resolve, reject) => { return new Promise<MediaSourceResolution>((resolve, reject) => {
const format = options?.quality ?? "bestaudio"; const format = options?.quality ?? "bestaudio";
const cookieArgs = buildCookieArgs();
const args = [ const args = [
"-f", "-f",
format, format,
"--audio-format",
"best",
"-o", "-o",
"-", "-",
"--no-progress",
"--no-warnings",
...cookieArgs,
"--print", "--print",
"before_dl:title", "before_dl:title",
"--print", "--print",
@@ -195,11 +339,31 @@ export function resolveMediaUrl(
logger.info({ url }, "Spawning yt-dlp for media resolution"); logger.info({ url }, "Spawning yt-dlp for media resolution");
const proc = spawn("yt-dlp", args, { const proc = spawn("yt-dlp", args, {
stdio: ["pipe", "pipe", "pipe"], stdio: ["ignore", "pipe", "pipe"],
}); });
activeProcesses.add(proc); activeProcesses.add(proc);
// With `-o -` yt-dlp streams the raw audio on stdout and moves its
// `--print` headers to stderr — pipe stdout immediately so the child
// never blocks on a full pipe while we wait for the headers on stderr.
const mediaStream = new PassThrough();
// Teardown (player stop / ffmpeg exit) destroys this stream while
// yt-dlp may still push bytes — without a listener an EPIPE /
// ERR_STREAM_DESTROYED surfaces as an uncaughtException.
mediaStream.on("error", (err: NodeJS.ErrnoException) => {
if (
err.code === "EPIPE" ||
err.code === "ERR_STREAM_DESTROYED" ||
err.code === "ERR_STREAM_WRITE_AFTER_END"
) {
logger.debug({ code: err.code }, "Media stream closed during teardown");
} else {
logger.error({ error: err.message }, "Media stream error");
}
});
proc.stdout.pipe(mediaStream);
let stderrBuf = ""; let stderrBuf = "";
let resolved = false; let resolved = false;
@@ -209,9 +373,23 @@ export function resolveMediaUrl(
if (resolved) return; if (resolved) return;
resolved = true; resolved = true;
activeProcesses.delete(proc); activeProcesses.delete(proc);
mediaStream.destroy();
reject(err); reject(err);
}; };
const resolveOnce = (info: MediaInfo) => {
if (resolved) return;
resolved = true;
activeProcesses.delete(proc);
resolve({
stream: mediaStream,
type: StreamType.Arbitrary,
title: info.title,
duration: info.duration,
info,
});
};
// -- spawn error (ENOENT etc.) ---------------------------------------- // -- spawn error (ENOENT etc.) ----------------------------------------
proc.on("error", (err: NodeJS.ErrnoException) => { proc.on("error", (err: NodeJS.ErrnoException) => {
@@ -222,31 +400,25 @@ export function resolveMediaUrl(
} }
}); });
// -- stderr (capture for diagnostics, capped at 4KB) ---------------------------------- // -- stderr: title + duration headers ---------------------------------
// Capture raw stderr too, for the exit-diagnostics in the close handler.
const _MAX_STDERR = 4096; const _MAX_STDERR = 4096;
if (proc.stderr) { if (proc.stderr) {
proc.stderr.on("data", (chunk: Buffer) => { proc.stderr.on("data", (chunk: Buffer) => {
stderrBuf += chunk.toString("utf8"); if (stderrBuf.length < _MAX_STDERR) {
stderrBuf += chunk
.toString("utf8")
.slice(0, _MAX_STDERR - stderrBuf.length);
}
}); });
} }
// -- stdout: parse header, then stream audio --------------------------- readStderrHeader(proc.stderr, (message) => {
failOnce(new Error(message));
readFirstTwoLines(proc.stdout) })
.then(({ title, duration, remaining }) => { .then(({ title, duration }) => {
if (resolved) return; resolveOnce({ title: title || url, duration });
resolved = true;
activeProcesses.delete(proc);
const info: MediaInfo = { title, duration };
resolve({
stream: remaining,
type: StreamType.Arbitrary,
title,
duration,
info,
});
}) })
.catch((err: Error) => { .catch((err: Error) => {
failOnce(err); failOnce(err);
@@ -264,6 +436,10 @@ export function resolveMediaUrl(
failOnce(new Error(`yt-dlp exited with code ${code}${detail}`)); failOnce(new Error(`yt-dlp exited with code ${code}${detail}`));
} else if (signal) { } else if (signal) {
failOnce(new Error(`yt-dlp was killed by signal ${signal}`)); failOnce(new Error(`yt-dlp was killed by signal ${signal}`));
} else {
// Exited cleanly but the header lines never surfaced (e.g. a direct
// file URL with no duration) — keep the media stream alive anyway.
resolveOnce({ title: url, duration: 0 });
} }
}); });
@@ -282,89 +458,6 @@ export function resolveMediaUrl(
}); });
} }
/**
* Resolve a media URL to a directly playable video URL (for screen share /
* GoLive streaming). Uses yt-dlp `--get-url` with bestvideo+bestaudio.
*
* @throws If yt-dlp is not installed or the process exits with a non-zero code.
*/
export function getDirectVideoUrl(url: string): Promise<string> {
return new Promise<string>((resolve, reject) => {
const args = [
url,
"--get-url",
"--format",
"bestvideo[protocol^=http]+bestaudio[protocol^=http]/best[protocol^=http]/best",
"--no-playlist",
"--no-warnings",
"--quiet",
];
logger.info({ url }, "Spawning yt-dlp for direct video URL");
const proc = spawn("yt-dlp", args, {
stdio: ["pipe", "pipe", "pipe"],
});
activeProcesses.add(proc);
let stdoutBuf = "";
let stderrBuf = "";
const MAX_STDERR = 4096;
const MAX_STDOUT = 1_048_576;
if (proc.stdout) {
proc.stdout.on("data", (chunk: Buffer) => {
if (stdoutBuf.length < MAX_STDOUT) {
stdoutBuf += chunk
.toString("utf8")
.slice(0, MAX_STDOUT - stdoutBuf.length);
}
});
}
if (proc.stderr) {
proc.stderr.on("data", (chunk: Buffer) => {
if (stderrBuf.length < MAX_STDERR) {
stderrBuf += chunk
.toString("utf8")
.slice(0, MAX_STDERR - stderrBuf.length);
}
});
}
proc.on("error", (err: NodeJS.ErrnoException) => {
activeProcesses.delete(proc);
if (err.code === "ENOENT") {
reject(buildNotInstalledError());
} else {
reject(new Error(`yt-dlp failed to start: ${err.message}`));
}
});
proc.on("close", (code) => {
activeProcesses.delete(proc);
if (code !== 0) {
const detail = stderrBuf.trim() ? `: ${stderrBuf.trim()}` : "";
reject(
new Error(
`yt-dlp direct URL resolution exited with code ${code}${detail}`,
),
);
return;
}
const firstLine = stdoutBuf.trim().split("\n")[0];
if (!firstLine) {
reject(new Error("yt-dlp returned no direct video URL"));
return;
}
resolve(firstLine);
});
});
}
/** /**
* Extract metadata (title, duration, thumbnail) from a media URL * Extract metadata (title, duration, thumbnail) from a media URL
* without downloading the audio stream. * without downloading the audio stream.
@@ -452,7 +545,7 @@ export async function extractMediaInfo(url: string): Promise<MediaInfo> {
} }
/** /**
* Kill all active yt-dlp child processes. * Kill all active yt-dlp / screen-share merge ffmpeg child processes.
* *
* Call during graceful shutdown to ensure no orphan processes remain. * Call during graceful shutdown to ensure no orphan processes remain.
*/ */
@@ -1,7 +1,7 @@
import type { Readable } from "node:stream"; import type { Readable } from "node:stream";
import type { StreamType } from "@discordjs/voice"; import type { StreamType } from "@discordjs/voice";
export type MediaMode = "music" | "screen"; export type MediaMode = "music";
export type MediaSourceKind = export type MediaSourceKind =
| "url" | "url"
| "local" | "local"
@@ -32,6 +32,7 @@ export interface MediaState {
playing: boolean; playing: boolean;
activeMode: MediaMode | null; activeMode: MediaMode | null;
musicVolume: number; musicVolume: number;
loop: boolean;
current: MediaQueueItem | null; current: MediaQueueItem | null;
queue: MediaQueueItem[]; queue: MediaQueueItem[];
} }
@@ -50,17 +51,7 @@ export interface MusicPlayer {
play(source: ResolvedMediaSource): MusicPlayback; play(source: ResolvedMediaSource): MusicPlayback;
} }
export interface ScreenSharePlayback { export type DiscordPlayerOwner = "none" | "browser-bridge" | "music";
done: Promise<void>;
stop(): void;
}
export interface ScreenShareController {
isActive(): boolean;
start(source: string): Promise<ScreenSharePlayback>;
}
export type DiscordPlayerOwner = "none" | "browser-bridge" | "music" | "screen";
export interface DiscordPlayOptions { export interface DiscordPlayOptions {
inputType?: StreamType; inputType?: StreamType;
@@ -18,7 +18,7 @@ export class DiscordPlayer {
private connection: VoiceConnection | null = null; private connection: VoiceConnection | null = null;
private owner: DiscordPlayerOwner = "none"; private owner: DiscordPlayerOwner = "none";
private resource: AudioResource | null = null; private resource: AudioResource | null = null;
private musicVolume = 1; private musicVolume = 0.3;
private idleCallback: (() => void) | null = null; private idleCallback: (() => void) | null = null;
/** Set before manual stop() calls to distinguish from natural track end. */ /** Set before manual stop() calls to distinguish from natural track end. */
private manualStop = false; private manualStop = false;
@@ -1,99 +0,0 @@
import {
Encoders,
playStream,
prepareStream,
Streamer,
Utils,
} from "@dank074/discord-video-stream";
import type { Client } from "discord.js-selfbot-v13";
import { createChildLogger } from "@/shared/logger/index";
import { getDirectVideoUrl } from "./mediaSource.js";
import type { ScreenSharePlayback } from "./mediaTypes.js";
import { discordPlayer } from "./player.js";
const logger = createChildLogger("screen-share");
export interface ScreenShareVoiceStatus {
connected: boolean;
activeGuildId: string | null;
activeChannelId: string | null;
}
/**
* Discord Go Live (screenshare) via @dank074/discord-video-stream.
*
* Pipeline:
* URL (YouTube, dll.) yt-dlp direct video URL ffmpeg (H264 720p30)
* playStream({ type: "go-live" }) Discord voice channel as Go Live.
*
* Restored from the pre-microservices implementation (commit d50ce86,
* src/media/screenShareController.ts) the interface survived in
* mediaTypes.ts but the implementation was lost during the split.
*/
export class ScreenShareController {
private logger = createChildLogger("screen-share");
private streamer: Streamer | null = null;
private active: ScreenSharePlayback | null = null;
constructor(
private readonly client: Client,
private readonly getVoiceStatus: () => ScreenShareVoiceStatus,
) {}
isActive(): boolean {
return this.active !== null;
}
async start(source: string): Promise<ScreenSharePlayback> {
const status = this.getVoiceStatus();
if (!status.connected || !status.activeGuildId || !status.activeChannelId) {
throw new Error("Connect to a voice channel before sharing screen");
}
if (this.active || discordPlayer.getOwner() !== "none") {
throw new Error("Another media mode is active");
}
try {
const directUrl = await getDirectVideoUrl(source);
if (!this.streamer) {
this.streamer = new Streamer(this.client);
}
const { command, output } = prepareStream(directUrl, {
encoder: Encoders.software({ x264: { preset: "superfast" } }),
width: 1280,
height: 720,
frameRate: 30,
bitrateVideo: 2500,
bitrateVideoMax: 4000,
includeAudio: true,
videoCodec: Utils.normalizeVideoCodec("H264"),
});
let stopped = false;
const done = playStream(output, this.streamer, {
type: "go-live",
}).finally(() => {
this.active = null;
});
this.active = {
done,
stop: () => {
if (stopped) return;
stopped = true;
command.kill("SIGTERM");
this.active = null;
},
};
logger.info({ source }, "Screen share started");
return this.active;
} catch (error) {
this.active = null;
const message = error instanceof Error ? error.message : String(error);
logger.error({ error: message, source }, "Screen stream failed");
throw error;
}
}
}
@@ -27,8 +27,6 @@ export class VoiceTransmitter {
private gate = Promise.resolve(); private gate = Promise.resolve();
/** Set true before sending SIGTERM so exit handler knows it's intentional */ /** Set true before sending SIGTERM so exit handler knows it's intentional */
private _expectedExit = false; private _expectedExit = false;
/** True while the underlying stream is in a drain state (backpressure) */
private draining = false;
/** /**
* Start listening for PCM audio data from Redis and stream to Discord * Start listening for PCM audio data from Redis and stream to Discord
@@ -56,6 +54,24 @@ export class VoiceTransmitter {
// Create PCM input stream // Create PCM input stream
this.pcmStream = new PassThrough(); this.pcmStream = new PassThrough();
this.pcmStream.setMaxListeners(32); // drain listeners accumulate during backpressure this.pcmStream.setMaxListeners(32); // drain listeners accumulate during backpressure
// Voice teardown (stop / disconnect / ffmpeg exit) destroys this stream
// while Redis PCM messages may still be in flight. Without a listener,
// EPIPE / ERR_STREAM_DESTROYED / ERR_STREAM_WRITE_AFTER_END surface as
// an uncaughtException and crash the whole gateway.
this.pcmStream.on("error", (err: NodeJS.ErrnoException) => {
if (
err.code === "EPIPE" ||
err.code === "ERR_STREAM_DESTROYED" ||
err.code === "ERR_STREAM_WRITE_AFTER_END"
) {
logger.debug(
{ code: err.code },
"PCM stream closed during voice teardown — ignoring",
);
} else {
logger.error({ error: err.message }, "PCM stream error");
}
});
// Spawn FFmpeg to encode 24kHz mono PCM → OggOpus // Spawn FFmpeg to encode 24kHz mono PCM → OggOpus
// Input: 24kHz mono s16le (raw PCM) // Input: 24kHz mono s16le (raw PCM)
@@ -146,7 +162,12 @@ export class VoiceTransmitter {
); );
this.redisSub.on("message", (channel, message) => { this.redisSub.on("message", (channel, message) => {
if (channel !== this.TRANSMIT_CHANNEL || !this.pcmStream) return; if (
!this.isActive ||
channel !== this.TRANSMIT_CHANNEL ||
!this.pcmStream
)
return;
try { try {
const data = JSON.parse(message); const data = JSON.parse(message);
@@ -156,16 +177,24 @@ export class VoiceTransmitter {
const canContinue = stream.write(pcmBuffer); const canContinue = stream.write(pcmBuffer);
// Backpressure: queue until drain // Backpressure: queue until drain
if (!canContinue) { if (!canContinue) {
this.draining = true;
stream.once("drain", () => { stream.once("drain", () => {
this.draining = false;
// Re-acquire stream reference (could have been replaced by restart)
const currentStream = this.pcmStream; const currentStream = this.pcmStream;
if (!currentStream) return; if (!currentStream || !this.isActive) return;
// Flush queued chunks // Flush queued chunks
while (this.backpressureQueue.length > 0) { while (this.backpressureQueue.length > 0) {
const queued = this.backpressureQueue.shift()!; const queued = this.backpressureQueue.shift();
if (!queued) break;
try {
if (!currentStream.write(queued)) break; if (!currentStream.write(queued)) break;
} catch (err) {
logger.debug(
{
error: err instanceof Error ? err.message : String(err),
},
"PCM flush write failed during teardown — ignoring",
);
break;
}
} }
}); });
} }
@@ -204,7 +233,6 @@ export class VoiceTransmitter {
this.isActive = false; this.isActive = false;
this.backpressureQueue = []; this.backpressureQueue = [];
this.draining = false;
if (this.pcmStream) { if (this.pcmStream) {
this.pcmStream.removeAllListeners("drain"); this.pcmStream.removeAllListeners("drain");
@@ -32,6 +32,13 @@ export const configSchema = z
.default("") .default("")
.transform((v) => v.split(",").filter(Boolean)) .transform((v) => v.split(",").filter(Boolean))
.describe("Thread IDs to exclude from capture"), .describe("Thread IDs to exclude from capture"),
BOT_EXCLUDED_CHANNEL_IDS: z
.string()
.default("1206269771340058694")
.transform((v) => v.split(",").filter(Boolean))
.describe(
"Channel IDs where bot messages are NOT captured/analyzed (bot detection stays on everywhere else)",
),
// ── Legacy voice ───────────────────────────────────────────────────── // ── Legacy voice ─────────────────────────────────────────────────────
VOICE_GUILD_ID: z.string().min(1).optional(), VOICE_GUILD_ID: z.string().min(1).optional(),
@@ -92,6 +99,10 @@ export const configSchema = z
// ── Redis ──────────────────────────────────────────────────────────── // ── Redis ────────────────────────────────────────────────────────────
REDIS_URL: z.string().default("redis://localhost:6379"), REDIS_URL: z.string().default("redis://localhost:6379"),
// ── SearXNG ───────────────────────────────────────────────────────────
// Instance for web search + term glossary lookups. Override when the
// default instance is down/rate-limited.
SEARXNG_BASE_URL: z.string().url().default("https://searxng.imrnes.team"),
// ── Voice PCM WebSocket (direct gateway→backend, bypasses Redis) ──── // ── Voice PCM WebSocket (direct gateway→backend, bypasses Redis) ────
VOICE_PCM_WS_ENABLED: z VOICE_PCM_WS_ENABLED: z
.string() .string()
@@ -133,7 +144,17 @@ export const configSchema = z
.url() .url()
.default("https://9router.asepharyana.my.id/v1"), .default("https://9router.asepharyana.my.id/v1"),
AI_LLM_MODEL: z.string().default("text"), AI_LLM_MODEL: z.string().default("text"),
AI_LLM_VISION_MODEL: z.string().optional(), // Vision uses the SAME router/base URL as text moderation
// (AI_LLM_BASE_URL) but a different model alias. The dedicated NVIDIA
// multimodal endpoint was removed.
AI_LLM_VISION_MODEL: z.string().default("multimodal"),
AI_LLM_DISABLE_THINKING: z
.string()
.default("true")
.transform((v) => v === "true")
.describe(
"Disable LLM chain-of-thought (reasoning/thinking) to speed up AI analysis. Set false to restore thinking.",
),
AI_LLM_EMBEDDING_MODEL: z.string().optional(), AI_LLM_EMBEDDING_MODEL: z.string().optional(),
AI_LLM_EMBEDDING_MIN_SIMILARITY: z.coerce AI_LLM_EMBEDDING_MIN_SIMILARITY: z.coerce
.number() .number()
@@ -171,6 +192,24 @@ export const configSchema = z
.int() .int()
.positive() .positive()
.default(30000), .default(30000),
// Term glossary — per-word Wikipedia lookups (via SearXNG) for words the
// LLM may not know (slang, jargon, regional language, foreign terms).
// Definitions are cached (in-memory + Redis) so repeat lookups are fast.
// Disable to skip glossary lookups entirely and analyze without them.
AI_GLOSSARY_ENABLED: z
.string()
.optional()
.transform((v) => v === "true")
.default(true),
// Max glossary terms looked up per analysis batch (keeps latency bounded).
AI_GLOSSARY_MAX_TERMS: z.coerce.number().int().min(1).max(20).default(6),
// Min word length for a term to be considered glossary-worthy.
AI_GLOSSARY_MIN_WORD_LENGTH: z.coerce
.number()
.int()
.min(2)
.max(20)
.default(5),
// ── AI Analysis Timing ────────────────────────────────────────────── // ── AI Analysis Timing ──────────────────────────────────────────────
AI_ANALYSIS_DEBOUNCE_MS: z.coerce.number().positive().default(500), AI_ANALYSIS_DEBOUNCE_MS: z.coerce.number().positive().default(500),
@@ -189,6 +228,17 @@ export const configSchema = z
.int() .int()
.positive() .positive()
.default(20), .default(20),
// Recency gates for conversation context. A silence longer than GAP_MS
// between context messages = the conversation restarted (older messages
// dropped); MAX_AGE_MS caps how far back context is considered relevant.
AI_ANALYSIS_CONTEXT_GAP_MS: z.coerce
.number()
.positive()
.default(12 * 60 * 1000),
AI_ANALYSIS_CONTEXT_MAX_AGE_MS: z.coerce
.number()
.positive()
.default(45 * 60 * 1000),
AI_ANALYSIS_PROCESSING_TIMEOUT_MS: z.coerce AI_ANALYSIS_PROCESSING_TIMEOUT_MS: z.coerce
.number() .number()
.positive() .positive()
@@ -242,6 +292,19 @@ export const configSchema = z
.default(false), .default(false),
AUTO_DELETE_LOG_CHANNEL_ID: z.string().default(""), AUTO_DELETE_LOG_CHANNEL_ID: z.string().default(""),
// ── Nickname Reset (offensive_username enforcement) ────────────────
// When the only violation is the member's server nickname, reset the
// nickname to the default username instead of deleting the message.
AUTO_NICKNAME_RESET_ENABLED: z
.string()
.optional()
.transform((v) => v === "true")
.default(true),
AUTO_NICKNAME_RESET_COOLDOWN_MS: z.coerce
.number()
.positive()
.default(10 * 60 * 1000),
// ── Retention ─────────────────────────────────────────────────────── // ── Retention ───────────────────────────────────────────────────────
RETENTION_MESSAGES_DAYS: z.coerce.number().int().min(0).default(0), RETENTION_MESSAGES_DAYS: z.coerce.number().int().min(0).default(0),
RETENTION_ATTACHMENTS_DAYS: z.coerce.number().int().min(0).default(0), RETENTION_ATTACHMENTS_DAYS: z.coerce.number().int().min(0).default(0),
@@ -62,6 +62,9 @@ export const pgMessagesTable = pgTable(
enum: ["none", "monitor", "warn", "review", "delete", "escalate"], enum: ["none", "monitor", "warn", "review", "delete", "escalate"],
}), }),
ai_analyzed_at: pgBigint("ai_analyzed_at", { mode: "number" }), ai_analyzed_at: pgBigint("ai_analyzed_at", { mode: "number" }),
ai_analysis_duration_ms: pgBigint("ai_analysis_duration_ms", {
mode: "number",
}),
ai_error: pgText("ai_error"), ai_error: pgText("ai_error"),
}, },
(table) => ({ (table) => ({
@@ -435,6 +438,32 @@ export const pgStickerCacheTable = pgTable(
export const stickerCacheTable = pgStickerCacheTable; export const stickerCacheTable = pgStickerCacheTable;
/**
* Term Glossary Cache Table (PostgreSQL)
* Permanently stores resolved term definitions (Wikipedia/SearXNG lookups).
* Definitions rarely change, so once a term is successfully resolved it is
* persisted here forever Redis/LRU only act as fast read caches in front.
* Terms with NO definition (misses) are NOT stored here; they stay ephemeral
* in Redis with a short TTL so transient lookup failures get retried.
*/
export const pgTermGlossaryCacheTable = pgTable(
"term_glossary_cache",
{
term: pgText("term").primaryKey(),
definition: pgText("definition").notNull(),
source_url: pgText("source_url").notNull().default(""),
resolved_at: pgBigint("resolved_at", { mode: "number" }).notNull(),
hit_count: pgInteger("hit_count").notNull().default(0),
},
(table) => ({
resolvedAtIdx: pgIndex("idx_term_glossary_cache_resolved_at").on(
table.resolved_at,
),
}),
);
export const termGlossaryCacheTable = pgTermGlossaryCacheTable;
// ============================================================================= // =============================================================================
// Meta / System // Meta / System
// ============================================================================= // =============================================================================
@@ -580,6 +609,11 @@ export type TextAnalysisCacheInsert =
export type StickerCacheRecord = typeof stickerCacheTable.$inferSelect; export type StickerCacheRecord = typeof stickerCacheTable.$inferSelect;
export type StickerCacheInsert = typeof stickerCacheTable.$inferInsert; export type StickerCacheInsert = typeof stickerCacheTable.$inferInsert;
// Term Glossary Cache
export type TermGlossaryCache = typeof termGlossaryCacheTable.$inferSelect;
export type TermGlossaryCacheInsert =
typeof termGlossaryCacheTable.$inferInsert;
// Muxer Jobs // Muxer Jobs
export type MuxerJob = typeof muxerJobsTable.$inferSelect; export type MuxerJob = typeof muxerJobsTable.$inferSelect;
export type MuxerJobInsert = typeof muxerJobsTable.$inferInsert; export type MuxerJobInsert = typeof muxerJobsTable.$inferInsert;
@@ -615,6 +649,7 @@ export const pgModerationActionsTable = pgTable(
"warn_user", "warn_user",
"kick_user", "kick_user",
"ban_user", "ban_user",
"reset_nickname",
], ],
}).notNull(), }).notNull(),
reason: pgText("reason"), reason: pgText("reason"),
@@ -198,7 +198,8 @@ export type ModerationActionType =
| "mute_user" | "mute_user"
| "warn_user" | "warn_user"
| "kick_user" | "kick_user"
| "ban_user"; | "ban_user"
| "reset_nickname";
export interface ModerationAction { export interface ModerationAction {
id: string; id: string;
@@ -62,6 +62,7 @@ export const COMMAND_MEDIA_QUEUE = "media:queue";
export const COMMAND_MEDIA_SKIP = "media:skip"; export const COMMAND_MEDIA_SKIP = "media:skip";
export const COMMAND_MEDIA_STOP = "media:stop"; export const COMMAND_MEDIA_STOP = "media:stop";
export const COMMAND_MEDIA_VOLUME = "media:volume"; export const COMMAND_MEDIA_VOLUME = "media:volume";
export const COMMAND_MEDIA_LOOP = "media:loop";
export const COMMAND_MODERATION_ACTION = "moderation:action"; export const COMMAND_MODERATION_ACTION = "moderation:action";
export const DISCORD_VOICE_ANALYZED = "discord:voice:analyzed"; export const DISCORD_VOICE_ANALYZED = "discord:voice:analyzed";
@@ -108,5 +108,8 @@ export async function retryWithBackoff<T>(
}); });
} }
} }
throw lastError!; // lastError is always set: the for-loop only exits via break when attempt
// === retries, which only happens in the catch branch that sets lastError.
if (!lastError) throw new Error("Unknown retry error");
throw lastError;
} }
@@ -0,0 +1,209 @@
// ═══════════════════════════════════════════════════════════════════════════
// Context enrichment builders — rich <user_reputation> attrs, <user_history>,
// <user_profiles> as_of, bot/edited detection (pure, no DB)
// ═══════════════════════════════════════════════════════════════════════════
import { describe, expect, it } from "vitest";
import {
buildUserHistoryXml,
buildUserProfilesBlock,
formatReputationAttrs,
resolveIsBot,
resolveIsEdited,
} from "../src/modules/ai-moderation/moderationBuilders.js";
import type { MessageRecord } from "../src/modules/message-capture/types.js";
const NOW = 1_800_000_000_000;
function msg(overrides: Partial<MessageRecord> = {}): MessageRecord {
return {
id: "m1",
guild_id: "g1",
channel_id: "c1",
thread_id: null,
user_id: "u1",
username: "user1",
avatar_url: null,
content: "hai",
edited_content: null,
created_at: NOW,
edited_at: null,
deleted_at: null,
type: "text",
is_reply: null,
is_forward: null,
is_crosspost: null,
reference_message_id: null,
reference_channel_id: null,
reference_guild_id: null,
metadata: null,
...overrides,
};
}
const DAY_MS = 24 * 60 * 60 * 1000;
describe("formatReputationAttrs — rich reputation signal", () => {
it("emits trust, infraction count and clean streak", () => {
const attrs = formatReputationAttrs({
trust_score: 62,
total_infractions: 3,
clean_message_streak: 45,
last_infraction_at: null,
});
expect(attrs).toContain('trust_score="62"');
expect(attrs).toContain('total_infractions="3"');
expect(attrs).toContain('clean_streak="45"');
});
it("derives last_offense_days_ago and marks repeat offenders (7-day window)", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 2,
clean_message_streak: 0,
last_infraction_at: NOW - 2 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="2"');
expect(attrs).toContain('repeat_offender="true"');
});
it("does NOT mark repeat offender when the last offense is older than 7 days", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 2,
clean_message_streak: 10,
last_infraction_at: NOW - 30 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="30"');
expect(attrs).not.toContain("repeat_offender");
});
it("omits offense-derived attrs when the user has no recorded infraction date", () => {
const attrs = formatReputationAttrs({
trust_score: 85,
total_infractions: 0,
clean_message_streak: 120,
last_infraction_at: null,
});
expect(attrs).not.toContain("last_offense_days_ago");
expect(attrs).not.toContain("repeat_offender");
});
it("clamps a future/skewed timestamp to days_ago=0", () => {
const attrs = formatReputationAttrs(
{
trust_score: 50,
total_infractions: 1,
clean_message_streak: 0,
last_infraction_at: NOW + 5 * DAY_MS,
},
NOW,
);
expect(attrs).toContain('last_offense_days_ago="0"');
});
});
describe("buildUserHistoryXml — last flagged messages for repeat offenders", () => {
it("returns empty when there is no real history", () => {
expect(buildUserHistoryXml([])).toBe("");
expect(
buildUserHistoryXml([{ content: " ", severity: "low", created_at: 1 }]),
).toBe("");
});
it("renders <infraction> rows with severity and recency", () => {
const xml = buildUserHistoryXml(
[
{
content: "beli barang murah disini https://scam.example",
severity: "high",
created_at: NOW - 3 * DAY_MS,
},
],
NOW,
);
expect(xml).toContain("<user_history>");
expect(xml).toContain('severity="high"');
expect(xml).toContain('time_ago_days="3"');
expect(xml).toContain("beli barang murah disini");
});
it("caps long snippets and XML-escapes content", () => {
const xml = buildUserHistoryXml(
[
{
content: "x".repeat(300),
severity: "low",
created_at: NOW - DAY_MS,
},
],
NOW,
);
expect(xml.length).toBeLessThan(250);
});
});
describe("buildUserProfilesBlock — deduplicated map with staleness", () => {
it("emits as_of when the profile has a last-generated timestamp", () => {
const block = buildUserProfilesBlock(
new Map([
[
"u1",
{
text: "Developer teknis, bahasa Indonesia",
asOf: NOW - 3 * DAY_MS,
},
],
]),
);
expect(block).toContain('<user_profile user_id="u1"');
expect(block).toContain(
`as_of="${new Date(NOW - 3 * DAY_MS).toISOString()}"`,
);
expect(block).toContain("Developer teknis");
});
it("omits as_of when absent, and drops empty profiles", () => {
const block = buildUserProfilesBlock(
new Map([
["u1", { text: "profil aktif", asOf: null }],
["u2", { text: " " }],
]),
);
expect(block).toContain('user_id="u1"');
expect(block).not.toContain("as_of");
expect(block).not.toContain("u2");
});
it("returns empty for no profiles", () => {
expect(buildUserProfilesBlock(new Map())).toBe("");
});
});
describe("resolveIsBot / resolveIsEdited — message flags", () => {
it("reads author.bot from captured metadata", () => {
const bot = msg({
metadata: JSON.stringify({
author: { id: "x", username: "bot", bot: true },
}),
});
const human = msg({
metadata: JSON.stringify({
author: { id: "y", username: "user", bot: false },
}),
});
expect(resolveIsBot(bot)).toBe(true);
expect(resolveIsBot(human)).toBe(false);
expect(resolveIsBot(msg())).toBe(false);
});
it("flags edited content only when edited_content is present (the edit path)", () => {
expect(resolveIsEdited(msg({ edited_content: "versi baru" }))).toBe(true);
expect(resolveIsEdited(msg())).toBe(false);
});
});
@@ -0,0 +1,299 @@
// ═══════════════════════════════════════════════════════════════════════════
// Conversation context v2 — recency gating + location context (pure, no DB)
// ═══════════════════════════════════════════════════════════════════════════
import { describe, expect, it } from "vitest";
import {
buildConversationContext,
buildLocationContext,
formatMessageForPrompt,
truncateContextLine,
} from "../src/modules/ai-moderation/conversationContext.js";
import { buildConversationContextBlock } from "../src/modules/ai-moderation/moderationBuilders.js";
import { extractOgMeta } from "../src/modules/ai-moderation/urlFetcher.js";
import type { MessageRecord } from "../src/modules/message-capture/types.js";
const NOW = 1_800_000_000_000;
function msg(id: string, createdAt: number, content = "hai"): MessageRecord {
return {
id,
guild_id: "g1",
channel_id: "c1",
thread_id: null,
user_id: `u_${id}`,
username: `user_${id}`,
avatar_url: null,
content,
edited_content: null,
created_at: createdAt,
edited_at: null,
deleted_at: null,
type: "text",
is_reply: null,
is_forward: null,
is_crosspost: null,
reference_message_id: null,
reference_channel_id: null,
reference_guild_id: null,
metadata: null,
};
}
function target(id = "t1", createdAt = NOW): MessageRecord {
return {
...msg(id, createdAt),
content: "pesan yang dianalisis",
};
}
const MIN = 60_000;
describe("buildConversationContext — recency gating", () => {
it("keeps an ONGOING conversation — recent messages, small gaps", () => {
const context = [
msg("a", NOW - 8 * MIN),
msg("b", NOW - 6 * MIN),
msg("c", NOW - 4 * MIN),
msg("d", NOW - 2 * MIN),
];
const { lines, descriptor, dropped } = buildConversationContext({
contextBefore: context,
targets: [target()],
maxTokens: 8000,
gapMs: 12 * MIN,
maxAgeMs: 45 * MIN,
});
expect(lines).toHaveLength(4);
expect(dropped).toBe(0);
expect(descriptor).toContain("status=ongoing");
});
it("drops messages before a silence gap — conversation RESTARTED", () => {
const context = [
msg("old1", NOW - 40 * MIN),
msg("old2", NOW - 38 * MIN),
msg("fresh", NOW - 5 * MIN),
];
const { lines, descriptor, dropped } = buildConversationContext({
contextBefore: context,
targets: [target()],
maxTokens: 8000,
gapMs: 12 * MIN,
maxAgeMs: 45 * MIN,
});
// 40min-old messages are within maxAge but 33min before "fresh" → gap gate
expect(lines.some((l) => l.includes("old1"))).toBe(false);
expect(lines.some((l) => l.includes("fresh"))).toBe(true);
expect(dropped).toBe(2);
expect(descriptor).toContain("status=sparse");
expect(descriptor).toContain("gap_before_min=");
});
it("drops everything older than maxAge — stale noise, cold_start anchor kept", () => {
const context = [
msg("ancient", NOW - 120 * MIN),
msg("stale", NOW - 60 * MIN),
];
const { lines, descriptor, dropped } = buildConversationContext({
contextBefore: context,
targets: [target()],
maxTokens: 8000,
gapMs: 12 * MIN,
maxAgeMs: 45 * MIN,
});
// Age gate drops both from the real context block, but the cold-start
// anchor keeps the nearest 2 so the LLM still senses the channel.
expect(dropped).toBe(2);
expect(descriptor).toContain("status=cold_start");
expect(lines).toHaveLength(2);
});
it("keeps a 2-message anchor on cold start so the LLM senses the channel", () => {
const context = [
msg("far1", NOW - 100 * MIN),
msg("far2", NOW - 99 * MIN),
msg("near1", NOW - 50 * MIN),
];
const { lines, descriptor } = buildConversationContext({
contextBefore: context,
targets: [target()],
maxTokens: 8000,
gapMs: 12 * MIN,
maxAgeMs: 45 * MIN,
});
expect(lines).toHaveLength(2); // nearest 2 kept as anchor
expect(lines.some((l) => l.includes("near1"))).toBe(true);
expect(descriptor).toContain("status=cold_start");
});
it("respects the token budget (older lines dropped first)", () => {
const context = Array.from({ length: 20 }, (_, i) =>
msg(`m${i}`, NOW - (i + 1) * MIN),
);
const { lines } = buildConversationContext({
contextBefore: context,
targets: [target()],
maxTokens: 600,
gapMs: 12 * MIN,
maxAgeMs: 45 * MIN,
});
expect(lines.length).toBeLessThan(20);
expect(lines.length).toBeGreaterThan(0);
});
});
describe("formatMessageForPrompt — server nickname (displayName)", () => {
it("renders member.displayName when captured (per-server nickname)", () => {
const m = msg("n1", NOW - MIN);
m.metadata = JSON.stringify({
member: {
displayName: "Si Goblok Server",
roles: [],
joinedTimestamp: null,
},
});
const line = formatMessageForPrompt(m, "context");
expect(line).toContain("user=Si Goblok Server");
expect(line).not.toContain("user_user_n1");
});
it("falls back to global username when displayName missing", () => {
const line = formatMessageForPrompt(
msg("n2", NOW - MIN, "halo"),
"context",
);
expect(line).toContain("user=user_n2");
});
it("truncates an oversized context message so one paste cannot eat the whole budget", () => {
const huge = "A".repeat(5000);
const line = formatMessageForPrompt(msg("n3", NOW - MIN, huge), "context");
expect(line).toContain("…[konteks dipotong: terlalu panjang]");
expect(line.length).toBeLessThan(2000);
});
it("keeps short context content intact", () => {
expect(truncateContextLine("pendek")).toBe("pendek");
});
});
describe("buildLocationContext — channel/thread/nsfw enrichment", () => {
it("renders a structured <location_context/> element from captured metadata", () => {
const t = target();
t.metadata = JSON.stringify({
channel: {
channelName: "general",
threadName: "tanya coding",
nsfw: false,
ageRestricted: false,
},
});
const line = buildLocationContext([t]);
expect(line).toContain("<location_context");
expect(line).toContain('channel_id="c1"');
expect(line).toContain('channel_name="general"');
expect(line).toContain('thread_name="tanya coding"');
expect(line).toContain('nsfw="false"');
expect(line).toContain('age_restricted="false"');
});
it("includes the channel topic (escaped) when captured", () => {
const t = target();
t.metadata = JSON.stringify({
channel: {
channelName: "rules",
topic: "Diskusi coding & programming — no self-promo",
nsfw: false,
},
});
const line = buildLocationContext([t]);
expect(line).toContain(
'topic="Diskusi coding &amp; programming — no self-promo"',
);
});
it("caps an oversized topic and omits empty/absent topic", () => {
const t = target();
t.metadata = JSON.stringify({
channel: { channelName: "general", topic: "x".repeat(500), nsfw: false },
});
const line = buildLocationContext([t]);
const match = line.match(/topic="([^"]*)"/);
expect(match).not.toBeNull();
expect(match?.[1].length).toBeLessThanOrEqual(201);
const t2 = target();
t2.metadata = JSON.stringify({ channel: { channelName: "general" } });
expect(buildLocationContext([t2])).not.toContain("topic=");
});
it("returns empty when no metadata", () => {
expect(buildLocationContext([target()])).toBe("");
});
});
describe("buildConversationContextBlock — structured USER-message context", () => {
it("wraps location + descriptor + lines into XML blocks", () => {
const block = buildConversationContextBlock({
location: buildLocationContext(
(() => {
const t = target();
t.metadata = JSON.stringify({
channel: { channelName: "general", nsfw: false },
});
return [t];
})(),
),
descriptor: "[conversation_flow] status=ongoing context_msgs=1 dropped=0",
lines: ["[context] id=a time=2027-01-01T00:00:00.000Z user=user_a: hai"],
});
expect(block).toContain("<location_context");
expect(block).toContain("<conversation_context>");
expect(block).toContain("[conversation_flow] status=ongoing");
expect(block).toContain("[context] id=a");
// location block comes before conversation block
expect(block.indexOf("<location_context")).toBeLessThan(
block.indexOf("<conversation_context>"),
);
});
it("omits the conversation block when there are no lines", () => {
const block = buildConversationContextBlock({
location: "",
descriptor: "",
lines: [],
});
expect(block).toBe("");
});
it("keeps only the location block when lines are empty but location exists", () => {
const block = buildConversationContextBlock({
location: '<location_context channel_id="c1"/>',
descriptor: "",
lines: [],
});
expect(block).toBe('<location_context channel_id="c1"/>');
});
});
describe("extractOgMeta — page title/site for <web_content>", () => {
it("extracts og:title, og:description and og:site_name", () => {
const html = `
<html><head>
<title>Fallback title</title>
<meta property="og:title" content="Judul Halaman &amp; Keren" />
<meta property="og:description" content="Deskripsi halaman" />
<meta property="og:site_name" content="Contoh Site" />
<meta property="og:image" content="https://img.example.com/x.png" />
</head></html>`;
const meta = extractOgMeta(html);
expect(meta.title).toBe("Judul Halaman & Keren");
expect(meta.description).toBe("Deskripsi halaman");
expect(meta.siteName).toBe("Contoh Site");
});
it("falls back to <title> when og:title missing", () => {
const html = "<html><head><title>Plain Title</title></head></html>";
expect(extractOgMeta(html).title).toBe("Plain Title");
});
});
@@ -0,0 +1,51 @@
// ═══════════════════════════════════════════════════════════════════════════
// makeImageCacheKey — regression for hash collision bug
// ═══════════════════════════════════════════════════════════════════════════
// Bug (2026-08-12): makeImageCacheKey() only hashed the first 128 chars of the
// data URL. Since all resized images share the same MIME prefix
// ('data:image/png;base64,') + identical base64 header bytes, nearly every
// image got the SAME hash → 'image:<same-hash>' → all images reused the
// first cached vision analysis ("konten judi").
//
// Fix: hash the ENTIRE data URL. This test verifies the fix and prevents
// regression.
import { createHash } from "node:crypto";
import { describe, expect, it } from "vitest";
import { makeImageCacheKey } from "../src/modules/ai-moderation/textCacheStore.js";
function oldBuggyHash(dataUrl: string): string {
const prefix = dataUrl.slice(0, 128);
return "image:" + createHash("sha256").update(prefix).digest("hex").slice(0, 16);
}
describe("makeImageCacheKey — collision prevention", () => {
it("produces different keys for images whose first 128 chars are identical", () => {
// Two data URLs that SHARE the first 128 chars (same MIME + identical
// base64 header) but differ after — this is the real-world scenario
// that caused the collision bug.
const sharedPrefix =
"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg==" +
"xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"; // pad to >128 chars
const imgA = sharedPrefix + "UNIQUE_TO_A";
const imgB = sharedPrefix + "UNIQUE_TO_B";
// Under the OLD buggy scheme: same prefix → same hash → COLLISION
expect(oldBuggyHash(imgA)).toBe(oldBuggyHash(imgB));
// Under the FIXED scheme: full data URL hashed → different keys
const keyA = makeImageCacheKey(imgA);
const keyB = makeImageCacheKey(imgB);
expect(keyA).not.toBe(keyB);
});
it("produces same key for identical input", () => {
const dataUrl = "data:image/png;base64,samebase64dataheremari";
expect(makeImageCacheKey(dataUrl)).toBe(makeImageCacheKey(dataUrl));
});
it("prefix is always 'image:'", () => {
const key = makeImageCacheKey("data:image/png;base64,test");
expect(key.startsWith("image:")).toBe(true);
});
});

Some files were not shown because too many files have changed in this diff Show More