Commit Graph
25 Commits
Author SHA1 Message Date
asepharyana f999be4fa0 feat(dashboard): add moderation log page + message edit history
- Backend moderation module: GET /api/moderation/stats (per-status + failed rate)
  + GET /api/moderation/actions (filter by status/actionType, cursor paging),
  joins messages for target username + content
- Message GET /api/messages/detail/:id now returns edit_count + edit_history
  (old_content snapshots from message_edits, newest first)
- FE: new /moderation page — summary cards (total/executed/failed/pending +
  failed-rate), status+type filter chips, timeline rows with action icon,
  target user, reason, status badge, timestamps, error text
- FE: message detail shows 'Riwayat edit' panel with previous versions
2026-08-05 11:11:10 +07:00
asepharyana 2f51f94610 fix(moderation): remove manual reanalyze triggers — auto-recovery only
Manual per-message and batch reanalyze buttons/endpoints let anyone
re-queue arbitrary messages for LLM analysis, burning AI credits on
spam. Removed:
- FE: Reanalyze buttons in message list, search panel, and messages page
- FE: useReanalyze/useReanalyzeBatch hooks + messagesApi methods
- BE: POST /api/messages/:id/reanalyze and /reanalyze-batch endpoints
- BE: markForReanalysis/reanalyzeErrorBatch service+repository methods

Recovery of failed messages is fully automatic: the discord-gateway
startPendingAIAnalysisWorker retries 'pending' (batch path) and
'error/analysis_incomplete' (individual path) messages on
AI_ANALYSIS_RECOVERY_INTERVAL_MS.
2026-08-04 15:37:56 +07:00
asepharyana 6293d588bc chore(lint): biome cleanup across services — format, sort imports, drop unused
- discord-gateway: 74 lint errors -> 0 (format, import sorting, unused
  imports/vars, dead breath var)
- backend: format + sort imports (11 warnings left: noExplicitAny)
- frontend: remove unused imports, drop dead breathing var, fix
  useExhaustiveDependencies (scroll keyed on messages), a11y biome-ignore
  for drag surface + stopPropagation container (mouse-only gestures)
- remaining warnings are false positives: index keys on static lists,
  <img> in static export (next/image unsupported), noExplicitAny

tsc --noEmit clean on all 3 services; vitest green (60+36).
2026-08-01 22:09:02 +07:00
asepharyana 8e7ba67248 fix: message detail shows only its own images, not everyone's
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m47s
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m12s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 3m12s
useMessageDetail fetch attachments per-channel -> detail view nampilin
10 image terbaru di channel itu (semua orang), bukan image pesan yang
diklik. Root cause: getAttachmentsByChannel cuma filter channel_id.

- backend: messageQuerySchema + messageId optional; repository
  tambah eq(message_id) pas messageId ada. Sudah ada index
  idx_attachments_message.
- frontend: getAttachments(channelId, limit, cursor, messageId);
  useMessageDetail pass id pesan.

Verifikasi: backend tsc PASS, frontend tsc PASS + next build PASS.
E2E local (tsx + env prod, port 3901): tanpa messageId -> 3 row
(termasuk image.png punya message lain); dengan messageId -> tepat
2 row milik pesan itu. psql juga konfirmasi channel-only 3 vs
message-filtered 2.
2026-08-01 10:59:54 +07:00
Developer dcd13482c2 refactor: break monorepo into 3 standalone services (gateway, backend, frontend)
Build & Deploy / build-and-push (backend) (push) Failing after 35s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 25s
Build & Deploy / build-and-push (proxy) (push) Failing after 25s
- Remove pnpm workspace, moon repo, and all monorepo tooling
- Delete packages/shared/, embed shared code directly into each service
- Copy packages/shared/src/* -> services/backend/src/shared/ and services/discord-gateway/src/shared/
- Replace all @bete/shared imports with @/shared/ path alias
- Remove @bete/shared workspace dependency from both services
- Update root package.json scripts from --filter to --prefix
- Rewrite Dockerfiles to build each service standalone
- Clean up biome.json, .gitignore, remove root drizzle.config.ts
2026-07-30 11:50:48 +07:00
DeveloperandClaude Opus 4.8 5802d02e29 refactor: large codebase cleanup - consolidate schemas, migrate to Drizzle ORM, extract frontend components, modernize Docker builds
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
- Consolidate all DB schema definitions into packages/shared as single source of truth
- Migrate backend from raw SQL to Drizzle ORM across all modules
- Extract frontend inline UI into separate component files
- Refactor discord-gateway circuitBreaker into conversationState + moderationState
- Convert messageStore to Proxy singleton pattern
- Add validateBody/validateQuery middleware + Zod schemas for API endpoints
- Modernize Docker builds with multi-stage + pnpm deploy
- Migrate CI/CD from deployment to image-based pipeline
- Remove 60+ unused/dead files (~15K lines)
- Update color scheme from sky-blue to teal-cyan
- Move DB connection management to @bete/shared/database

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 21:54:31 +07:00
asepharyana 9621317a3c chore: format files for lint
Deploy to VPS / deploy (push) Failing after 8m8s
2026-07-08 01:37:46 +07:00
asepharyana c0edb0fa40 feat: enhance deployment instructions and exclude spam threads from message capture 2026-07-05 06:58:17 +07:00
asepharyana c3b2b3f334 feat: add /api/messages/images endpoint + ImageGrid fetch
Backend:
- New GET /api/messages/images?guildId=&limit= endpoint
- Queries attachments table for image/* MIME type, returns distinct messages
- Repository uses Drizzle subquery + join pattern

Frontend:
- New get_images() API client function
- ImageGrid fetches from /api/messages/images when Images tab selected
- Separate image_messages signal, fetched on tab switch

Deploy: hot-patched JS/WASM files to running containers
2026-07-05 04:48:29 +07:00
MythEclipse f502918b27 feat: multimodal video detection + reply/forward/crosspost + batch optimization
- Video frame extraction via ffmpeg (4 key frames per video → vision LLM)
- Video display in FE MessageCard with HTML5 <video> player
- Reply/forward/crosspost indicator in FE + pipeline in DG/BE
- Fix: missing sanitizeAiContent + escapeXml in media path (prompt injection)
- Optimize: text-only batch results saved to DB immediately, no longer wait for media analysis
- BE mapper/schema/repo: add reference fields (is_reply, is_forward, etc.)
2026-06-22 09:46:06 +07:00
MythEclipseandClaude Opus 4.8 07032ab521 refactor: atomic, DRY, and logging improvements across codebase
- Split llmModerationClient.ts (2170 lines) into 5 focused sub-modules
- Split aiAnalyzer.ts (1282 lines) into 4 modular pipelines
- Split messages.db.ts (826 lines) into 5 domain-specific modules
- Moved shared schema to @bete/shared, eliminated backend duplication
- Added createChildLogger to all voice-recording and AI moderation modules
- Extracted tryCommandThenFallback, normalizeMediaState, DEFAULT_VOICE_STATUS
- Created shared pagination.ts utility, eliminated 5+ cursor-pagination duplications
- Created shared messageMapper.ts for row mapping
- Standardized backend error handling with asyncHandler
- Added frontend createLogger utility and useAsyncAction hook
- Added structured logging to frontend hooks, socket, and API client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 19:46:08 +07:00
MythEclipseandClaude Opus 4.8 b68789fffc refactor: atomic, DRY, and logging improvements
- Shared Redis channel constants as single source of truth (redis-channels.ts)
- commandHandler.ts split into VoiceHandler, MediaHandler, GuildHandler,
  ModerationHandler with handler-registry.ts dispatch
- messageStore.ts (1322 lines) split into domain-specific DB files:
  messages.db.ts, attachments.db.ts, reviews.db.ts,
  moderation-actions.db.ts, retention.db.ts
- recorder.ts startSpeaking callback extracted into speakingHandler.ts,
  streamSetup.ts, segmentFinalizer.ts
- autoDeleteManager.ts split into autoDeleteEligibility.ts,
  autoDeleteNotify.ts, autoDeleteLogger.ts
- Added createChildLogger() logging across 8 service files
- Backend messages.repository.ts migrated from raw SQL to Drizzle ORM
- Fixed biome.json to exclude packages/**/dist/* from lint
- Fixed config.ts GUILD_ID pre-existing type error

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 17:34:18 +07:00
MythEclipse 7108f6bb47 feat(system): implement graceful shutdown and expand websocket events
Improve system reliability and real-time capabilities by implementing a robust lifecycle management system and adding new broadcast events for attachments and voice recordings.

- Implement asynchronous graceful shutdown in backend to close HTTP, WebSocket, Redis, and database connections.
- Add new WebSocket broadcast events: `attachment_created`, `voice_recording_started`, `voice_recording_stopped`, `voice_recording_uploaded`, and `analysis_queue_status`.
- Refactor media status handling to use boolean `playing` state instead of string-based status.
- Centralize `PageResult` and `VoiceRecording` types to improve consistency between frontend and backend.
- Update frontend API client to include `listRecordings` and handle new WebSocket event types.
- Fix type mismatches in voice command handling and media status reporting.
2026-06-09 16:56:44 +07:00
MythEclipseandClaude Opus 4.8 7d6612bb2d refactor: resolve architecture disconnects and codebase weaknesses
- Consolidate eventTypes.ts as single source of truth for Redis channels:
  - Remove duplicate DiscordGatewayEvent interface from eventBroadcaster.ts
  - Replace all hardcoded channel strings with EventChannels constants
  - eventTypes.ts is no longer an orphan file

- Remove dangerous moderation action feature (selfbot safety):
  - Remove /messages/:id/moderate endpoint from backend
  - Remove moderation:action handler from commandHandler.ts
  - Remove publishFireAndForgetCommand (wrong envelope format)
  - Verified no remaining references to moderation:action in code

- Remove unused getCommandPublisher import from redis-bridge.ts

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 13:22:09 +07:00
MythEclipseandClaude Opus 4.8 4becf0d6f1 refactor: comprehensive codebase cleanup and architecture hardening
- Sprint 1 (Quick Wins): Remove dead analytics modules, fix 4 unresolved
  imports, replace 3 console.warn with logger, remove mock-crc import
- Sprint 2 (Architecture): Create MascotChatRepository, AnalysisRepository,
  3 Zod schemas (mascot-chat, analysis, voice), deduplicate error classes,
  move 3 SQL queries from routes to repository
- Sprint 3 (Complexity): Replace 7 any types with proper interfaces,
  extract 6 helpers from prepareMediaMessage (CC 85 -> ~15)
- Sprint 4 (Config): Remove 22 dead env vars from .env, add 30 missing
  vars to .env.example, standardize naming

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 10:16:04 +07:00
MythEclipse e00b23f9b8 fix(concurrency): eliminate all identified double-queue and race conditions
Implements full audit from double_queue_audit.md.

## Critical

fix(backend): reanalyze-batch — per-scope in-flight guard (messages.routes.ts)
  Two concurrent admin sessions clicking 'Retry All Errors' simultaneously now
  get a 409 REANALYZE_BATCH_IN_PROGRESS for the same guildId:channelId scope.
  Prevents the recovery worker from being triggered twice for the same set of
  error messages.

fix(discord-gateway): messageUpdate embed resolution skip (messageCapture.ts)
  Discord fires messageUpdate when link previews resolve 1-2s after send
  even though the message body is unchanged. Compare newContent vs
  existingContent before resetting ai_status to pending and re-queueing LLM.
  Eliminates a spurious duplicate analysis that could overwrite a valid result.

## High

fix(discord-gateway): scheduleAutoDelete idempotency (aiAnalyzer.ts)
  Add autoDeleteInFlight Set. Both processBatch and processIndividualFallback
  call scheduleAutoDelete; without the guard, a message that races through
  both paths launches two concurrent attemptAutoDeleteFlaggedMessage calls,
  producing a duplicate moderation-action log entry and a Discord 10008 error.
  The Set is cleaned up in a .finally() block after each attempt completes.

## Medium

fix(frontend): revert optimistic pending state on HTTP failure (useMessages.ts)
  Capture the prior MessageRecord inside the setMessages functional updater
  (no extra useCallback deps needed). If reanalyzeMessage() throws, restore
  the snapshot so the UI reflects the real DB state instead of lying.

fix(discord-gateway): exclude individual_analysis_exhausted from recovery queries
  Both getConversationKeysWithIncompleteAnalysis and
  getIncompleteMessagesByConversation now add a NOT LIKE guard for
  individual_analysis_exhausted. Prevents an infinite recovery loop if a bug
  ever writes both flags to the same row.

## Low

fix(discord-gateway): unified timer path in scheduleConversationAnalysis
  Remove the separate cooldown-path that created a secondary timer calling
  scheduleConversationAnalysis recursively. Replace with a single
  clear-and-reset pattern where delayMs = max(cooldownRemainder+500, debounce).
  Eliminates the edge case where both timers were live simultaneously.

## Misc

fix(backend): cast req.params.id to String() to satisfy Express typings
  Pre-existing tsc error (string | string[]) exposed by our edit.
  String() is correct; route params are always scalar strings at runtime.
2026-06-05 16:09:23 +07:00
MythEclipse 5c0a837cf0 fix(ai-moderation): eliminate double-queue on failure and spam-retry
Three root causes patched:

1. aiAnalyzer.ts — processBatch apiFailedMessages path:
   After reverting messages to 'pending', suppress shouldScheduleNext
   (was true by default) to prevent scheduleConversationAnalysis from
   firing immediately and racing with the recovery worker that will pick
   up those same pending messages on its next poll cycle.
   Also release the conversationProcessing lock immediately after the
   revert so the cooldown timer (not the full processing-timeout) gates
   the next attempt.

2. messages.routes.ts — POST /messages/:id/reanalyze:
   Add a per-message reanalyzeInFlight Set.  Concurrent requests for
   the same ID now return HTTP 409 instead of issuing duplicate UPDATEs
   and triggering multiple recovery worker activations.
   Also narrow the SQL predicate to 'WHERE id =  AND ai_status != pending'
   so a click that arrives while the recovery worker already picked the
   message up is a no-op at the DB level.
2026-06-05 15:38:31 +07:00
MythEclipseandClaude Opus 4.8 6da8a32c9b feat: add "Retry All Errors" batch reanalyze button
Backend:
- POST /api/messages/reanalyze-batch — bulk reset error messages to pending
- messages.repository.reanalyzeErrorBatch() — scoped by guildId/channelId/messageIds
- messages.service.reanalyzeErrorBatch() — validation layer

Frontend:
- reanalyzeErrorBatch() API client function
- useMessages.reanalyzeAllErrors() — optimistic state + batch call
- MessagesPanel: destructive button visible when error count > 0
- Shows confirmation text with queued count

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 00:17:51 +07:00
MythEclipse 2d79d8aefd feat(shared): reorder AppError constructor to (message, code, status), add singleton logger, retry and TTL cache utilities 2026-06-02 21:06:42 +07:00
MythEclipseandClaude Opus 4.8 c5c667b147 feat: comprehensive UX improvements — messages, analysis, moderation
Fix #1 (CRITICAL): Wire message_analyzed through Redis EventBroadcaster
- DG aiAnalyzer.ts: Add broadcastAnalysisCompleted() helper that publishes
  to both in-memory WS broadcaster AND Redis EventBroadcaster
- DG bootstrap.ts: Pass eventBroadcaster to startPendingAIAnalysisWorker()
- Fixes broken real-time chain so analysis results appear instantly

Fix #2: Collapsible AI Analysis with Rich Formatting
- MessageCard: AI analysis now collapsible with color-coded severity border
  (red/yellow/blue), summary line showing categories + confidence + severity
- Default collapsed for clean, expanded for warn/flagged

Fix #3: Toast Notifications for Flagged Content
- Wrap app in ToastProvider; ModerationAlertListener component listens for
  moderation_alert custom events and shows toast with emoji + details

Fix #4: Discord-style Message Grouping
- MessageFeed: Group consecutive messages from same user within 5 min
- MessageCard: Compact variant hides avatar, reduces padding for non-first

Fix #5: Moderation Action Buttons in UI
- BE: POST /api/messages/:id/moderate endpoint + publishCommand() for Redis
- FE: Delete/Warn buttons on flagged/warned cards with confirmation dialog

Fix #6: Search Results Include Full Data
- BE analysis.service.ts: SELECT all 26 message columns instead of just 11
- Search results now render with full MessageCard including images/analysis

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 12:17:51 +07:00
MythEclipseandClaude Opus 4.8 f1ddca5eee fix: false positive gambling detection + route collision + WS events (#1)
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 10:44:27 +07:00
MythEclipseandClaude Opus 4.8 3b2709455e feat: implement all real backend endpoints, Redis pub/sub bridge, and gateway command handler
Backend (real implementations, no more stubs):
- Redis pub/sub bridge: subscribes to discord-gateway events (message/attachment/voice) and broadcasts to WS clients
- Redis command channel: backend publishes voice/media commands, discord-gateway executes and replies
- Voice service: connectVoice/disconnectVoice/getVoiceStatus via Redis commands with graceful fallback
- Media service: queue/skip/stop/volume via Redis commands, reads status from Redis cache
- Messages repository: ALL 7 methods now use real PostgreSQL queries (findMany, findById, findByChannel, create, update, delete, getAttachmentsByChannel)
- Analytics: period returns {start,end} epoch millis, overview includes hourly/topics/top_users, worst_flags as string[]
- Health check: actually queries SELECT 1 against database
- VoiceStatus type fixed: {connected, activeGuildId, activeChannelId, activeChannelName}
- Guild type: includes icon: string | null
- asyncHandler: accepts Promise<unknown> instead of Promise<void>

Discord Gateway:
- CommandHandler: subscribes to 'backend:command' Redis channel, executes voice/media commands, publishes replies
- Publishes voice:status and media:status to Redis for backend caching
- Shutdown handler updated to close command handler

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 00:32:38 +07:00
MythEclipseandClaude Opus 4.8 9b41eb9c12 feat(backend): implement all missing endpoints, real analytics queries, and WebSocket server
- Replace stub analytics.repository.ts with real PostgreSQL queries using pg.Pool
- Add /api/guilds, /api/config, /api/auth/login, /api/ui-state (GET/POST)
- Add /api/review, /api/recordings, /api/analysis/search
- Add /api/messages/:id/reanalyze endpoint
- Add /api/analytics/heatmap and /api/analytics/topics
- Implement media routes (stub responses, backend has no Discord voice client)
- Add WebSocket server at /ws with heartbeat and broadcast functions
- Fix analytics route paths to match frontend contract (dual paths for backward compat)
- Export getPool() from database module for raw SQL queries
- Register all new routers in app.ts

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-02 00:11:29 +07:00
MythEclipseandClaude Opus 4.6 f20c2389a4 refactor(backend): flatten module structure, move routes to .routes.ts files
- Remove empty controllers/, services/, repositories/ subdirectories
- Move routes/index.ts into module.routes.ts (flat naming convention)
- Update imports in http/app.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-06-01 21:53:08 +07:00
MythEclipseandClaude Opus 4.8 c48a0c5e3b refactor: split monolith into 3 microservices (frontend, backend, discord-gateway)
- Extract services into services/{frontend,backend,discord-gateway}
- Create packages/shared/ for shared logger, errors, utils, types
- Setup Modular MVC pattern in backend (controller→service→repository)
- Setup event-driven architecture in discord-gateway with Redis pub/sub
- Move Docker files to infra/docker/ with per-service Dockerfiles
- Update docker-compose.yml to use Traefik-only routing (no port exposes)
- Update GitHub Actions deploy workflow for multi-service matrix build
- Fix all import paths and resolve type errors across all services
- All 3 services pass tsc --noEmit clean

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:29 +07:00