Commit Graph
24 Commits
Author SHA1 Message Date
asepharyana 185d81f0e0 feat(ai-moderation): reset offensive nickname instead of deleting message
When the ONLY violation is offensive_username (message content clean):
- Message is NOT deleted (nickname-only violation bypasses auto-delete)
- Member's server nickname is reset to default username via
  setNickname(null) (Discord shows the global username again)
- Action 'reset_nickname' logged to moderation_actions; cooldown
  10min per guild:user (LRU) so repeated messages by same member
  don't hammer the Discord PATCH
- Config: AUTO_NICKNAME_RESET_ENABLED / AUTO_NICKNAME_RESET_COOLDOWN_MS
2026-08-10 11:48:27 +07:00
asepharyana 6293d588bc chore(lint): biome cleanup across services — format, sort imports, drop unused
- discord-gateway: 74 lint errors -> 0 (format, import sorting, unused
  imports/vars, dead breath var)
- backend: format + sort imports (11 warnings left: noExplicitAny)
- frontend: remove unused imports, drop dead breathing var, fix
  useExhaustiveDependencies (scroll keyed on messages), a11y biome-ignore
  for drag surface + stopPropagation container (mouse-only gestures)
- remaining warnings are false positives: index keys on static lists,
  <img> in static export (next/image unsupported), noExplicitAny

tsc --noEmit clean on all 3 services; vitest green (60+36).
2026-08-01 22:09:02 +07:00
Developer 1249ae81d8 perf(automod): compress prompts ~40% + semantic cache via AI_LLM_EMBEDDING_MODEL
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m4s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m29s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m33s
Prompt overhaul (token-frugal, same quality):
- rules.ts 28KB -> 10.3KB: every normative rule kept (safe lists, SARA
  6 kategori, LGBT/Israel zero tolerance, anti-evasion, decision tree,
  evasi hierarchy, image rules) with duplicated phrasing removed
- examples.ts 24.7KB -> 20KB: all 31 teaching examples kept; analysis
  strings shortened, redundant categories/policy_version dropped from
  example outputs (both optional in the response schema)
- output.ts 13.8KB -> 6.8KB: compressed schema + personality + format
  rules; CRITICAL bans on generic analysis and reply-context requirement
  retained
- system.ts: MEDIA_INSTRUCTIONS compressed, key rules kept

Semantic moderation cache (AI_LLM_EMBEDDING_MODEL):
- New embeddingClient.ts: OpenAI-compatible embeddings + cosine
  similarity; degrades gracefully when model/key unset
- textCacheStore: stores embedding JSON per verdict, findSimilarTextModeration
  reuses near-duplicate verdicts (min 0.97 cosine, processing locks skipped)
- moderationOrchestrator: after exact-hash miss, embed text-only targets
  and reuse stored verdict for near-duplicates -> skips expensive chat
  completion for spam variants; fresh verdicts written back with embedding
- Config: AI_LLM_EMBEDDING_MODEL / MIN_SIMILARITY (0.97) / MAX_CANDIDATES (30)
- Migration 0012: ADD COLUMN embedding to text_analysis_cache (idempotent)
- .env.example documents the new vars
2026-07-31 19:37:53 +07:00
Developer 0c6e18db77 perf: optimize Docker images - backend 294MB, gateway 1.89GB
Build & Deploy / build-and-push (backend) (push) Failing after 1m25s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 26s
Build & Deploy / build-and-push (proxy) (push) Failing after 25s
Backend:
- node:22-alpine runtime (from 1.82GB to 294MB, 84% reduction)
- COPY --chown instead of chown -R layer
- pnpm install --prod after build for clean .pnpm store
- Pinned pnpm@10 to avoid v11 build-script lockout

Gateway:
- Fixed TypeScript errors (missing table defs in shared schema)
- Restored node-crc Rust patch for native compilation
- Multi-stage build with COPY --chown
- pnpm install --prod after build
- ffmpeg with --no-install-recommends

Proxy:
- Restructured for standalone build (build errors are pre-existing)

General:
- pnpm@10 in all builds (avoids ERR_PNPM_IGNORED_BUILDS)
- Clean .dockerignore (excludes docs/design/scripts)
2026-07-30 13:18:44 +07:00
Developer dcd13482c2 refactor: break monorepo into 3 standalone services (gateway, backend, frontend)
Build & Deploy / build-and-push (backend) (push) Failing after 35s
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 25s
Build & Deploy / build-and-push (proxy) (push) Failing after 25s
- Remove pnpm workspace, moon repo, and all monorepo tooling
- Delete packages/shared/, embed shared code directly into each service
- Copy packages/shared/src/* -> services/backend/src/shared/ and services/discord-gateway/src/shared/
- Replace all @bete/shared imports with @/shared/ path alias
- Remove @bete/shared workspace dependency from both services
- Update root package.json scripts from --filter to --prefix
- Rewrite Dockerfiles to build each service standalone
- Clean up biome.json, .gitignore, remove root drizzle.config.ts
2026-07-30 11:50:48 +07:00
DeveloperandClaude Opus 4.8 5802d02e29 refactor: large codebase cleanup - consolidate schemas, migrate to Drizzle ORM, extract frontend components, modernize Docker builds
Build & Deploy / build-and-push (discord-gateway) (push) Failing after 2m22s
Build & Deploy / build-and-push (backend) (push) Failing after 3m22s
Build & Deploy / build-and-push (proxy) (push) Successful in 1m36s
Build & Deploy / deploy (push) Skipped
- Consolidate all DB schema definitions into packages/shared as single source of truth
- Migrate backend from raw SQL to Drizzle ORM across all modules
- Extract frontend inline UI into separate component files
- Refactor discord-gateway circuitBreaker into conversationState + moderationState
- Convert messageStore to Proxy singleton pattern
- Add validateBody/validateQuery middleware + Zod schemas for API endpoints
- Modernize Docker builds with multi-stage + pnpm deploy
- Migrate CI/CD from deployment to image-based pipeline
- Remove 60+ unused/dead files (~15K lines)
- Update color scheme from sky-blue to teal-cyan
- Move DB connection management to @bete/shared/database

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-27 21:54:31 +07:00
asepharyana 0a6a9fd982 feat: update dependencies and improve dashboard functionality
Deploy to VPS / deploy (push) Failing after 1m43s
- Added new dependencies for Next.js and lucide-react in pnpm-workspace.yaml.
- Refactored DashboardPage component to improve readability and error handling.
- Enhanced Header component to display error status with an alert icon.
- Updated MobileTabBar and Sidebar components to use a centralized tabs definition.
- Improved ChannelsView in dashboard-panel to handle channel fetching more cleanly.
- Fixed ActiveSpeaker type to use camelCase for userId.
- Updated MessagesPanel to handle guildId checks more gracefully.
- Adjusted API calls in dashboard and messages to align with backend expectations.
- Refined type definitions across various interfaces for consistency and clarity.
2026-07-26 14:27:36 +07:00
MythEclipse 28baeda5ad chore(services): update components based on recent changes
Changes:
 packages/shared/src/config/index.ts                                          |   7 +++
 packages/shared/src/moderation-types.ts                                      |   1 +
 services/backend/src/modules/recordings/recordings.service.ts                |   3 +-
 services/discord-gateway/drizzle/migrations/0011_add_voice_transcription.sql |   1 +
 services/discord-gateway/src/modules/voice-recording/recorder/segment.ts     |  10 ++++
 services/discord-gateway/src/modules/voice-recording/recorder/uploader.ts    |  15 ++++++
 services/discord-gateway/src/modules/voice-recording/voiceTranscriber.ts     |  51 ++++++++++++++++++
 services/discord-gateway/src/shared/database/schema.ts                       |   1 +
 services/discord-gateway/src/shared/database/voiceRecordingRepo.ts           |  17 ++++++
 services/frontend/src/features/live/components/RecordingsSubPanel.tsx        | 108 +++++++++++++++++++-------------------
2026-06-13 17:08:54 +07:00
MythEclipse 2e14b5e4ed chore(auto): task completed - unknown 2026-06-13 13:35:11 +07:00
MythEclipseandClaude fbc2184c6e feat(ai-moderation): add user profile self-learning system
Add user_profiles table, store, and background learner worker
that summarizes user communication style, topics, and personality.

- New user_profiles table (user_id PK, guild_id, profile_summary, last_analyzed_at)
- userProfileStore.ts — CRUD (get/update) following channelCultureStore pattern
- userProfileLearner.ts — background worker: queries 100 recent msgs per user,
  calls LLM for personality summary, updates every 12h
- Inject <user_profile> XML tag per-message in moderation prompt
- Start worker alongside cultureLearner in aiAnalyzer.ts
- Migration 0008 for user_profiles table

Co-Authored-By: Claude <noreply@anthropic.com>
2026-06-12 20:11:34 +07:00
MythEclipseandClaude Opus 4.8 07032ab521 refactor: atomic, DRY, and logging improvements across codebase
- Split llmModerationClient.ts (2170 lines) into 5 focused sub-modules
- Split aiAnalyzer.ts (1282 lines) into 4 modular pipelines
- Split messages.db.ts (826 lines) into 5 domain-specific modules
- Moved shared schema to @bete/shared, eliminated backend duplication
- Added createChildLogger to all voice-recording and AI moderation modules
- Extracted tryCommandThenFallback, normalizeMediaState, DEFAULT_VOICE_STATUS
- Created shared pagination.ts utility, eliminated 5+ cursor-pagination duplications
- Created shared messageMapper.ts for row mapping
- Standardized backend error handling with asyncHandler
- Added frontend createLogger utility and useAsyncAction hook
- Added structured logging to frontend hooks, socket, and API client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 19:46:08 +07:00
MythEclipseandClaude Opus 4.8 3614d32701 fix: resolve architecture disconnects and codebase weaknesses
- Standardize MessageRecord types — single source of truth from @bete/shared
- Clean up config: remove unused GUILD_ID/TEXT_GUILD_ID/TEXT_CHANNEL_ID, fix WEBSERVER_PORT default (3001), remove default admin password
- Move mascot_chat_messages table to Drizzle schema with proper migration
- Remove runtime DDL (CREATE TABLE IF NOT EXISTS) from mascot-chat repository
- Remove phantom analytics/ module from documentation
- Add better-sqlite3 dependency to root devDependencies
- Replace 'as any' casts with proper type assertions across AI moderation
- Add error logging to silent catch blocks in LLM client
- Apply Biome formatting and import organization

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 13:07:01 +07:00
MythEclipse 71a8a9e6e1 fix(ai-moderation): patch structural bypass vulnerabilities in NLP pipeline
- Implement Pre-computation Normalization for Polyglot Obfuscation.

- Inject Ontological Graph for literal translation evasion (e.g., 'kostum hewan').

- Enforce Entropy-Triggered Routing to deny softmax fallback exploitation.

- Format discord-gateway codebase.
2026-06-06 15:38:36 +07:00
MythEclipse 6a3d0eb17c feat(discord-gateway): replace base64 sticker cache column with image URL storage 2026-06-06 14:10:55 +07:00
MythEclipse 2f3d7e1d61 feat(ai-moderation): introduce user reputation and channel culture context
Implements a context-aware moderation system by tracking user behavior
and channel-specific norms to improve AI decision-making accuracy.

- Adds `user_reputations` table to track trust scores, clean streaks,
  and infraction history.
- Adds `channel_cultures` table to store AI-generated summaries of
  channel-specific norms and slang.
- Implements `userReputationStore` to autonomously update user scores
  based on moderation outcomes (clean vs. flagged).
- Implements `cultureLearner` and `channelCultureStore` to manage
  evolving channel contexts.
- Enhances LLM prompts to inject user reputation (trust scores,
  history) and channel culture summaries, enabling "wisdom-based"
  moderation (e.g., giving benefit of the doubt to high-trust users).
- Integrates reputation and culture updates into the existing
  `aiAnalyzer` pipeline.
2026-06-05 18:04:57 +07:00
MythEclipse 09f6e80ddd fix(ai-moderation): resolve race conditions and implement processing state
Introduces a `processing` state to the AI analysis lifecycle to prevent
duplicate processing of the same messages.

- Implements row-level locking using `FOR UPDATE SKIP LOCKED` in
  `messageStore.ts` to ensure atomic message acquisition.
- Adds a `processing` status to the `AIStatus` type and database schema.
- Fixes a TOCTOU race condition in `aiAnalyzer.ts` by synchronizing
  the conversation processing lock before async database operations.
- Implements `revertStuckProcessingMessages` to recover messages stuck
  in the `processing` state due to worker crashes or timeouts.
- Updates `processBatch` and scheduling logic to correctly manage and
  release conversation-level locks.
2026-06-05 16:36:14 +07:00
MythEclipse d1d4510ce8 chore(format): auto-format files with biome 2026-06-04 18:52:56 +07:00
MythEclipse f10fa9aa88 feat(discord-gateway): export correctedModerationsTable from database schema 2026-06-02 23:05:37 +07:00
MythEclipse eb16c1f504 ``` 2026-06-02 23:05:33 +07:00
MythEclipse 0cd4f197e9 No staged changes found — the staging area is empty. Stage some files first with git add, then I can help craft the commit message. 2026-06-02 23:05:06 +07:00
MythEclipse b23ccf0837 chore: session auto-commit 2026-06-02 17:12:42 +07:00
MythEclipse 406a6cbf79 refactor: migrate sticker cache to PostgreSQL, remove versioning
- Replace file-based sticker cache (.dat + index.json) with PostgreSQL sticker_cache table
- Remove model_version column and all versioning logic (git branch detection, GitHub API, CACHE_MODEL_VERSION)
- Strip VISION_MODEL_VERSION, logCacheEvent calls, and version filtering from SQL queries
- Remove STICKER_CACHE_DIR and STICKER_CACHE_MAX_SIZE_MB config variables
- Delete dead migration add_model_version_to_cache.sql

textCacheStore.ts reduced 482→204 lines (-57%)
stickerCache.ts reduced 210→144 lines (-31%)
Total: 11 files changed, 233 insertions, 591 deletions
2026-06-02 14:05:51 +07:00
MythEclipse 385fc0bbbf feat: add response logger for moderation/vision audit trail + cache model versioning
- Add responseLogger.ts with comprehensive logging: vision analysis, moderation
  analysis batches, cache hit/miss events, errors, retries, false positives,
  model version changes, cache invalidation
- Integrate responseLogger calls into llmModerationClient.ts (cache events,
  moderation results, errors, retries)
- Add model_version column to text_analysis_cache schema with composite
  index for efficient filtering
- Add migration SQL for model_version column
- Document CACHE_MODEL_VERSION in .env.example
2026-06-02 12:23:52 +07:00
MythEclipseandClaude Opus 4.8 c48a0c5e3b refactor: split monolith into 3 microservices (frontend, backend, discord-gateway)
- Extract services into services/{frontend,backend,discord-gateway}
- Create packages/shared/ for shared logger, errors, utils, types
- Setup Modular MVC pattern in backend (controller→service→repository)
- Setup event-driven architecture in discord-gateway with Redis pub/sub
- Move Docker files to infra/docker/ with per-service Dockerfiles
- Update docker-compose.yml to use Traefik-only routing (no port exposes)
- Update GitHub Actions deploy workflow for multi-service matrix build
- Fix all import paths and resolve type errors across all services
- All 3 services pass tsc --noEmit clean

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-01 21:44:29 +07:00