Track.setPacketizer(kind, ssrc, pt, clockRate, ...) builds the same
media-handler chain node-datachannel does for @dank074:
RtpPacketizer (Opus | H264 | H265 | AV1) → RtcpSrReporter →
RtcpNackResponder → PacingHandler(25Mbps, 1ms) for video
Track.sendFrame(encodedFrame) packetizes into RTP; addTimestamp(delta)
advances the RTP timestamp (node-datachannel contract).
Verified test-packetizer.js: two peers connected over tracks, real opus
frames + AnnexB H264 (SPS/PPS/IDR) flow through the chain without crash.
This removes the need for a JS RTP packetizer entirely — libdatachannel
0.24 has the full media stack built in.
Expose rtc::Track with send(binary) for raw RTP — verified:
- SDP from addTrack(audio)+addTrack(video) has m=audio (opus 120)
and m=video (H264 101 + H265/VP8/VP9/AV1 + RTX)
- libdatachannel Track::send() sends RAW RTP/RTCP when no media
handler is set (verified in src/track.cpp impl::Track::outgoing) —
so RTP packetization can live in pure JS, keeping the binding minimal
Also fix: Track class was missing from InitAll exports (crash on
TrackWrap::NewInstance — null FunctionReference).
Phase 0 of GoLive rewrite (drop @dank074/node-datachannel 771MB):
minimal N-API binding exposing PeerConnection/DataChannel/ICE/SDP,
built against libdatachannel 0.24.0 (from node-datachannel _deps source).
Verified: offer/answer/ICE/DataChannel roundtrip between two local
peers (test-handshake.js). Key findings:
- callbacks must be registered in ctor BEFORE createDataChannel
- SDP with candidates comes from localDescription() at gathering Complete
- answer auto-generates on setRemoteDescription(offer); do NOT call
setLocalDescription() after or role=actpass breaks the peer
prepareStream (from @dank074/discord-video-stream) unconditionally appends
audio filters 'volume@internal_lib' + 'azmq' that exist ONLY in its custom
node-av jellyfin-ffmpeg build. The Nix deployment runs plain ffmpeg-headless
on PATH, so fluent-ffmpeg died instantly with 'Filter not found' (exit 8),
the NUT output stream stayed empty, and playStream's node-av demux failed
with 'Failed to open input from Readable stream: Invalid data found when
processing input' — every screen share failed ~100ms after start.
Fix: pass customFfmpegFlags ['-filter:a','anull'] — ffmpeg applies the LAST
-filter:a for a stream, so the trailing no-op filter overrides the custom
chain (verified: command ends with '-filter:a anull', transcode runs, node-av
demux finds video+audio). Realtime volume control was already removed from
GMW (a690e5b), so dropping the filters is lossless.
Verified end-to-end with the failing URL (youtu.be/fONoh7Pc6VU, AV1+Opus
DASH): getDirectScreenInput → NUT merge → patched prepareStream → node-av
demux finds H264 video + Opus audio streams.
Root cause (3rd layer after 50371bd + 4f4c435): a vision model run
(2026-08-10) returned 'Maaf, saya tidak melihat gambar apapun yang terlampir...'
and that text was cached as a VALID vision_llm result (image + phash keys,
24h/7d TTL). Every subsequent analysis of the same image (same hash/phash)
hit the poisoned cache, so image analysis looked broken forever even though
9router responded fine — the moderation LLM wrote 'lampiran yang gagal
terbaca' from a cache hit.
Also: mimo via 9router streams reasoning in delta.reasoning +
delta.reasoning_details[].text (content:"") — extractChunkText only read
delta.reasoning_content, so those runs aggregated empty → 'Vision API null
response' (observed 08:54/09:07/09:38).
Fixes:
- llmClient.extractChunkText: fall back to delta.reasoning and
reasoning_details[].text (mimo), on top of reasoning_content (gemma).
- visionAnalyzer: isNoImageSeenText() detects 'no image' style outputs;
such results are NEVER cached, and poisoned entries are purged when hit
(LRU/DB/phash) so re-analysis actually re-runs vision.
- Tests: reasoning/reasoning_details extraction + isNoImageSeenText
(Indonesian + English, no false positives on real descriptions).
Root cause (2nd layer after 50371bd): the analysis worker could pick up an
image message while its attachment upload was still in flight
(upload_status='pending'). downloadAndExtractFrame then fell back to the
Discord CDN URL (cdn.discordapp.com), which often 404s for old/purged links,
and 'if (!res.ok) return' silently dropped the image — no log, no vision
call, empty image map, and the LLM produced a text-only verdict like
'lampiran yang gagal terbaca oleh sistem'.
Fixes:
- ai-analysis-worker: skip targets whose attachment upload is still pending
(both batch + individual paths) — they stay ai_status='pending' and the
next 15s cycle analyzes them after the upload lands.
- mediaDownloader.downloadAndExtractFrame: try uploaded_url first, then
discord_url as fallback; log non-OK responses (status + host) instead of
silently returning; log when all candidate URLs fail.
Root cause: 9router combo 'multimodal' routes to cloudflare-ai/@cf/google/
gemma-4-26b-a4b-it which streams ALL output in delta.reasoning_content
(content:"") and finishes with 'length' at max_tokens. llmClient only read
delta.content, so llmVision returned empty → every image moderation fell back
to text-only analysis ('Meskipun analisis gambar gagal' in every ai_analysis).
Fix: extractChunkText() prefers delta.content then falls back to
delta.reasoning_content (also handles message/text/response fields), with
unit tests for the exact 9router chunk shape. Verified live against a real
DB image: oc/mimo-v2.5-free (new first model in the multimodal combo) returns
a proper description in delta.content.
- <message> targets now carry time (ISO), repetitions (N identical short texts = spam signal), bot and edited flags; escape id/user XML
- rich <user_reputation>: total_infractions, clean_streak, last_offense_days_ago, repeat_offender (7-day window)
- <user_history> with last flagged messages for repeat offenders (wires dead getUserRecentInfractions)
- <user_profile as_of> staleness signal; <location_context topic> from captured channel topic
- prompt framing + output instructions teach the LLM to use the new signals without treating history as proof
- tests: contextEnrichment.test.ts (13) + topic cases in conversationContext.test.ts
When the ONLY violation is offensive_username (message content clean):
- Message is NOT deleted (nickname-only violation bypasses auto-delete)
- Member's server nickname is reset to default username via
setNickname(null) (Discord shows the global username again)
- Action 'reset_nickname' logged to moderation_actions; cooldown
10min per guild:user (LRU) so repeated messages by same member
don't hammer the Discord PATCH
- Config: AUTO_NICKNAME_RESET_ENABLED / AUTO_NICKNAME_RESET_COOLDOWN_MS
- resolveDisplayName(): member.displayName from captured metadata,
falls back to global username
- Applied to context lines, target message blocks, and media message
blocks — LLM sees the name the channel actually sees (nickname can
carry moderation signal itself)
- Conversation context recency gates (GAP_MS/MAX_AGE_MS): drop stale
messages before silence gaps; cold_start anchor + flow descriptor
tells LLM whether conversation is ongoing or restarted
- [location] block: channel name, thread name, nsfw/age flags from
captured metadata (thread names instead of bare IDs)
- Link media -> multimodal: text-batch URL fetches that resolve to
images now run vision analysis (bounded 15s) and switch prompt to
mixed mode; <web_content> gains og:title for page context
- pnpm-workspace.yaml: approve sharp build script (unblocks install)
A single long guild name was clipped (whitespace-nowrap + narrow min-width),
so the dropdown rendered as a tiny 144px box with truncated text. Size the
popup to fit-content up to the available width and let option text wrap.
Select dropdowns (voice tab, guild selector) rendered with a transparent
background because --color-popover/--color-input were undefined, and the
popup overlapped the trigger due to alignItemWithTrigger. Define the missing
theme tokens (popover, popover-foreground, input, secondary) and default the
select popup to open below the trigger.
- Implemented RadioGroup and RadioGroupItem for radio button functionality.
- Created ResizablePanelGroup, ResizablePanel, and ResizableHandle for resizable panels.
- Developed Sidebar component with context for state management and various subcomponents (SidebarTrigger, SidebarMenu, etc.).
- Added Spinner component for loading indicators.
- Introduced Table component with TableHeader, TableBody, TableFooter, and related subcomponents for structured data display.
- Built Toast component for notifications with customizable actions and icons.
- Implemented ToggleGroup and ToggleGroupItem for toggle button functionality with context support.
- Loop: toggle via POST /api/media/loop → COMMAND_MEDIA_LOOP; gateway
replays finished music track on natural end (queue untouched); status
payload exposes loop flag; FE tombol Loop di music-player + mini-player.
- Kualitas suara: music playback sekarang di-transcode sekali via ffmpeg ke
OggOpus 48kHz stereo 192kbps dengan volume di-bake ke encode — menghindari
double lossy encode (inlineVolume) yang bikin suara buram. Screen share
tetap pakai jalur lama.
- Backend: MediaState.loop, setLoop service, route + schema validation.
Volume sudah di-set default 0.3 di gateway (suara kecil saat play), dan
user bisa naikin sendiri di Discord (command media:volume) — jadi kontrol
volume lewat dashboard tak perlu. Hapus:
- BE: POST /api/media/volume route, mediaVolumeSchema, setVolume service
- FE: useMediaVolume hook, mediaApi.volume, slider volume di music-player
+ mini-player, field volume/setVolume di MediaPlayerProvider
Pertahankan COMMAND_MEDIA_VOLUME di gateway (masih dipakai command DC)
dan mic volume (terpisah, tetap di voice page).
Volume play music terlalu besar buat user — default sekarang 0.3 (30%)
di semua layer: player gateway (musicVolume=0.3), backend state/schema
(default 0.3), dan UI slider (fallback 0.3). User tetap bisa naikin
manual via slider volume di dashboard.
Rombak total alur data frontend: dari static-export CSR (tiap browser
fetch sendiri + akumulasi state voice per-tab) jadi server-side rendering.
Frontend (Next.js):
- next.config: output export -> standalone; halaman jadi server components
- server data layer baru src/lib/api/server.ts (GMW_BACKEND_URL, no window)
- dashboard/media/messages/moderation/recordings/voice page -> RSC yang
fetch backend di render-time, seed ke client view (SWR fallbackData)
- hook-hook utama terima initialData -> first paint data server, revalidate
SWR setelahnya, tanpa spinner-blank-load
- messages: guild/channel/tab/selected dibaca dari URL di server, page awal
di-fetch server-side
Shared realtime state (voice) server-authoritative:
- backend src/modules/voice/live-speaker.ts: agregat voice_active_user dari
gateway jadi snapshot authoritatif (single source of truth semua browser)
- GET /api/voice/status kini include activeSpeakers
- WS initial states kirim voice_state snapshot saat connect (late join
langsung dapat state yang sama, bukan daftar kosong)
- useSpeakers seed dari server snapshot + voice_state full-replace +
voice_active_user delta upsert
Deploy:
- flake.nix: frontend package build SSR standalone (server.js wrapper,
GMW_FRONTEND_PORT=4017); proxy nginx template proxy / -> Next server,
/api + /ws tetap ke backend :4001
Music playback produced no audio: with `-o -` yt-dlp streams media on
stdout and emits its `--print` title/duration headers on stderr, but
resolveMediaUrl read them from stdout — stripping two binary 'lines' off
the WebM container and corrupting the stream (player 'playing' but silent).
Now headers are read from stderr and the stdout media stream is returned
untouched.
Screenshare was failing with EACCES: getDirectScreenInput used --no-simulate,
making yt-dlp write .f*.part files into the read-only Nix store CWD. Dropped
it — simulate mode still returns requested_formats[].url in the JSON.
Adds tests/mediaResolve.test.ts (stderr-header + untouched-stream regression).
shutdown() awaited httpServer.close(), which waits for ALL open
connections. A lingering WS/keep-alive socket left the process zombie
forever after an uncaughtException (e.g. pg 'Connection terminated
unexpectedly' to imrnes) — no exit, so systemd Restart=always could
never revive it; /api/guilds returned 502 until manual restart.
Add 10s force-exit timer in shutdown(); clear it on clean completion.
getDirectVideoUrl used yt-dlp --get-url with bestvideo+bestaudio, which
prints the video-only and audio-only URLs on SEPARATE lines. Only the
first (video-only) line was used, so ffmpeg had no audio track and the
GoLive stream had no sound.
Replace with getDirectScreenInput which:
- uses --dump-single-json to fetch both fresh URLs in ONE yt-dlp run
(signature URLs expire quickly)
- returns the merged progressive URL directly when one exists
- otherwise merges the video-only + audio-only DASH URLs locally via a
child ffmpeg into a single NUT stream consumed as a Readable
- tracks the merge ffmpeg process in cleanup() so shutdown kills it too
Verified end-to-end with real YouTube URLs: yt-dlp pair → live ffmpeg
merge (NUT) → H264+opus transcode yields both streams. Added
tests/screenShareInput.test.ts covering URL / DASH-pair / error paths.
Gateway publishes {playing, activeMode, musicVolume, current, queue} but the
backend MediaState interface + normalizeMediaState dropped activeMode, so the
FE's 'Screen share active' / 'Music playing' badge never rendered. Carry it
through so the FE↔BE media contract stays in sync.
GoLive (dank074 Streamer) needs the single voice session; after the stream
ends, an automatic @discordjs/voice reconnect often races Discord's session
teardown and times out (AbortError). Restore is now best-effort with a 5s
delay; if it fails the FE shows disconnected and the user clicks Connect —
an accepted tradeoff for one-voice-session-per-user.
Immediate reconnect after Streamer.stop() races Discord's voice session
teardown → AbortError. Wait 4s so the old session is fully released before
re-joining with @discordjs/voice.
The restore callback previously read getVoiceStatus() AFTER disconnectGuild
had already cleared it, so it never knew which guild/channel to reconnect.
Now release/restore receive the status captured BEFORE the audio connection
is released, so reconnect actually happens after the GoLive stream ends.
The dank074 Streamer creates its own WebRTC voice connection, but Discord
allows only ONE voice session per user. When VoiceController (audio) was
already connected, the Streamer join hung forever (never got
VOICE_SERVER_UPDATE). Now:
1. ScreenShareController takes releaseVoice/restoreVoice callbacks.
2. Before joining, it disconnects the @discordjs audio connection via
VoiceController.disconnectGuild.
3. Streamer joins + streams GoLive.
4. After the stream ends, restoreVoice reconnects the audio connection so
mic/listen keep working.
5. media.handler wires these via a new setVoiceController accessor from
commandHandler; VoiceController is the single source of truth.
Also adds caller-bound timeouts & safe .catch() everywhere so a stream
failure can never become an unhandledRejection again.
Root cause: ScreenShareController created @dank074 Streamer but never called
streamer.joinVoiceChannel() — playStream threw 'Bot is not connected to a
voice channel', and since the code only used .finally() (no .catch), the
rejection became an unhandledRejection that took down the whole gateway
(graceful shutdown triggered, systemd restarted).
Fixes:
1. Resolve active channel + streamer.joinVoiceChannel(channel) before
prepareStream/playStream (dank074 needs its OWN WebRTC voice connection).
2. .catch() on the playStream done promise — log + kill ffmpeg instead of
crashing the process.
3. .catch() on playback.done in media.handler too.
4. stop() now kills ffmpeg AND stops the streamer's voice connection.
Audit voice (kirim/terima/music/screenshare) menemukan 3 masalah:
1. media:queue SILENT no-op — backend publish {source,mode} tapi gateway
handler baca payload.url → selalu 'received without a URL'. Backend
sekarang kirim {url,mode}, gateway terima url ATAU source (robust).
2. Voice connect gagal diam-diam saat user pilih channel tanpa permission
(joinable=false, contoh Music 32/64/128/256k). Backend+gateway sekarang
expose joinable; FE disable channel 'no akses' + empty state.
3. FE tidak kasih feedback saat connect gagal — tambah toast.error dengan
pesan dari backend.
Verified live: @discordjs/voice connect ke Lofi Radio joinable sukses
(VOICE READY, DAVE session OK) — pipeline voice sebenarnya sehat, masalah
utama UX. media:queue fix akan di-verify setelah deploy.
- chatbot-container: remove chatOpen layer — the minimized bubble now
expands straight into the chat panel (single click, no extra button)
- Remove the redundant 'Tanya soal server...' quick-prompt button and
the PanelLeft collapse toggle (one less state to fight)
- Bubble fixed h-[460px], chat panel flex-fills remaining space
- chat-panel: suggestion chips on empty state (suasana server, channel
paling ramai, total pesan, pesan bermasalah) so first-time users can
start with a single click
- Input: roomier padding, more descriptive placeholder, bigger send
button, autoComplete off
- Drop chatOpen/setChatOpen from context (dead state)
Backend:
- New chatbot.tools.ts: 4 tools (get_server_stats, get_top_channels,
get_recent_activity, get_top_flagged) with real DB executors
- chatbot.service: agentic loop — stream:true, parse SSE, execute
tool_calls, feed results back, up to 4 rounds
- controller: resolve userId from X-User-Id header (no-login device
uuid) with auth middleware precedence; history/clear scoped per user
Frontend:
- use-chatbot-user: mint UUID in localStorage, send as X-User-Id
- chatbotApi.send/getHistory/clearHistory accept userId header
- client.ts: apiRequest supports custom headers per call
- provider: history load + send + clear keyed to device user id
- Delete chatbot-canvas.tsx (placeholder face canvas) + its export
- Remove canvas area from chatbot container; chat panel gets the freed
space (248px → 300px) and bubble height drops 440px → 400px
- Chatbot bubble: 220px → 320px wide, 440px tall; proper header with
drag handle; quick-prompt row when chat is closed
- ChatPanel: show ALL history (not last 8), Indonesian placeholder/empty
state/error copy (backend speaks Indonesian), timestamps (id-ID),
clear-history button, typing indicator bubbles, Enter-to-send
- Send active guildId as context so backend answers reference the real
server (serverInsights path in chatbot.service)
- GuildId sync from layout → ChatbotProvider via ChatbotGuildSync
- chatbotApi.send(message, guildId) → POST /api/chat {message, context}
matching BE zod schema (guildId optional)
The Redis media:status key was only rewritten after a command received via
Redis. When the last track ended naturally (AudioPlayer Idle -> advanceQueue
with an empty queue), currentTrackItem was cleared but the status key was not
persisted — so the backend's cached status and the frontend's 10s polling
stayed stuck showing the finished track as 'playing' forever.
Wire a media-status sink (commandHandler provides the real redisPub to
MediaHandler) and re-publish status after auto-advance, so natural track end
updates the UI.
asepharyana