Commit Graph
34 Commits
Author SHA1 Message Date
Developer dc119b5d5a chore(env): switch AI_LLM_BASE_URL to omniroute (imrnes:20128)
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 1m38s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m22s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m38s
9router → omniroute router. /api/v1 exposes OpenAI-compatible chat
(model 'text' → gemma-4-31b-it, verified) + embeddings (verified:
gemini-embedding-001/-2, nemotron-embed-vl-1b-v2:free, qwen3-embedding).
Runtime env already switched via /etc/gmw/discord-gateway.env +
GATEWAY_ENV secret; gateway restarted 21:10, embedding writes flowing.
2026-07-31 21:11:39 +07:00
Developer 67e432564d ci(deploy): declarative env — CI writes service env from Gitea secrets
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 1m34s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m22s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m37s
Previously /etc/gmw/*.env was managed by hand on the VPS; AI_LLM_* and
other vars drifted silently (AI_LLM_EMBEDDING_MODEL was missing until
added manually today). Now:
- BACKEND_ENV / GATEWAY_ENV secrets hold the full env block per service
- deploy.yml streams the secret to the VPS via stdin before the deploy
  (never through argv — no shell escaping issues, no secret exposure)
- env file chown'd gmw:gmw, chmod 600; empty secret = skip (no clobber)
- .env.example documents the declarative workflow

Update production env = edit the Gitea secret + push, never SSH by hand.
2026-07-31 20:08:46 +07:00
Developer 1249ae81d8 perf(automod): compress prompts ~40% + semantic cache via AI_LLM_EMBEDDING_MODEL
Build & Deploy (Nix) / build-and-deploy (backend) (push) Successful in 3m4s
Build & Deploy (Nix) / build-and-deploy (discord-gateway) (push) Successful in 2m29s
Build & Deploy (Nix) / build-and-deploy (proxy) (push) Successful in 2m33s
Prompt overhaul (token-frugal, same quality):
- rules.ts 28KB -> 10.3KB: every normative rule kept (safe lists, SARA
  6 kategori, LGBT/Israel zero tolerance, anti-evasion, decision tree,
  evasi hierarchy, image rules) with duplicated phrasing removed
- examples.ts 24.7KB -> 20KB: all 31 teaching examples kept; analysis
  strings shortened, redundant categories/policy_version dropped from
  example outputs (both optional in the response schema)
- output.ts 13.8KB -> 6.8KB: compressed schema + personality + format
  rules; CRITICAL bans on generic analysis and reply-context requirement
  retained
- system.ts: MEDIA_INSTRUCTIONS compressed, key rules kept

Semantic moderation cache (AI_LLM_EMBEDDING_MODEL):
- New embeddingClient.ts: OpenAI-compatible embeddings + cosine
  similarity; degrades gracefully when model/key unset
- textCacheStore: stores embedding JSON per verdict, findSimilarTextModeration
  reuses near-duplicate verdicts (min 0.97 cosine, processing locks skipped)
- moderationOrchestrator: after exact-hash miss, embed text-only targets
  and reuse stored verdict for near-duplicates -> skips expensive chat
  completion for spam variants; fresh verdicts written back with embedding
- Config: AI_LLM_EMBEDDING_MODEL / MIN_SIMILARITY (0.97) / MAX_CANDIDATES (30)
- Migration 0012: ADD COLUMN embedding to text_analysis_cache (idempotent)
- .env.example documents the new vars
2026-07-31 19:37:53 +07:00
asepharyana 831fddd1bf refactor(backend): remove auth for public API
Deploy to VPS / deploy (push) Failing after 35s
- Remove auth module (auth.routes.ts, /api/auth/login)
- Remove adminAuth middleware from voice and media routes
- Remove adminAuth() function from shared middlewares
- Remove auth-related e2e test
- Clean up .env.example
2026-07-26 11:33:45 +07:00
asepharyana 7efaf00c93 Revert "feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config"
This reverts commit d59b59a7a7.
2026-07-02 03:54:44 +07:00
asepharyana d59b59a7a7 feat: migrate frontend to Astro + expand AI moderation + backend admin/runtime config
Frontend:
- migrate from Vite to Astro (astro.config.mjs, pages/, layouts/)
- add admin panel, settings page, command palette, error boundary
- refactor App.tsx, MascotChatbot, Sidebar, Header, DashboardLayout
- update API client, WebSocket, auth, dashboard features

Backend:
- add admin module and config routes
- refactor middlewares, Redis connection, WebSocket server/bridge
- add runtime config loader

Discord Gateway:
- refactor AI moderation: circuit breaker, concurrency limiter, fallback processor
- add media analysis client, Seaxng search, user profile learner
- add new drizzle migration

Shared:
- extend database schema, add new config fields
2026-07-02 00:02:41 +07:00
MythEclipse 92d73fe12d chore(services): update components based on recent changes
Changes:
 .env.example                                                     | 5 +++++
 services/discord-gateway/src/modules/voice-recording/recorder.ts | 2 +-
 2 files changed, 6 insertions(+), 1 deletion(-)
2026-06-14 00:57:08 +07:00
MythEclipse 2557a07916 feat(core): implement data retention, metrics, and enhanced media handling
This commit introduces several significant improvements across the backend and gateway services:

- **Data Retention**: Added an automated cleanup scheduler in `discord-gateway` to prune expired messages, attachments, and voice recordings based on configurable retention policies.
- **Observability**: Integrated `prom-client` in the `backend` service to expose Prometheus metrics via `/api/metrics` and added default Node.js runtime metrics.
- **Media Handling**: Enhanced `MediaHandler` in `discord-gateway` to support media URL resolution and improved playback status tracking.
- **API & Config**: Expanded the configuration endpoint to expose more system settings and reorganized `.env.example` for better readability.
- **Refactoring & Cleanup**:
    - Removed unused `better-sqlite3` dependency.
    - Refactored voice channel routing.
    - Improved error handling and testing coverage with comprehensive unit tests for shared utilities and error classes.
- **Documentation**: Added `MEMORY.md` for project context.
2026-06-10 20:56:16 +07:00
MythEclipse 67d66bb5dd refactor(shared): centralize configuration and types
Migrate configuration validation and core moderation types from individual services to the `@bete/shared` package to ensure consistency across the monorepo.

- Move `AppConfig` and moderation-related interfaces to `packages/shared`.
- Replace service-specific Zod schemas with the centralized shared configuration.
- Refactor `services/backend` and `services/discord-gateway` to consume shared config and types.
- Remove redundant type definitions and local configuration logic in services.
- Update `packages/shared` exports to include new `config` and `moderation-types` modules.
- Clean up unused files and deprecated utility functions in `packages/shared`.
2026-06-09 11:56:03 +07:00
MythEclipseandClaude Opus 4.8 4becf0d6f1 refactor: comprehensive codebase cleanup and architecture hardening
- Sprint 1 (Quick Wins): Remove dead analytics modules, fix 4 unresolved
  imports, replace 3 console.warn with logger, remove mock-crc import
- Sprint 2 (Architecture): Create MascotChatRepository, AnalysisRepository,
  3 Zod schemas (mascot-chat, analysis, voice), deduplicate error classes,
  move 3 SQL queries from routes to repository
- Sprint 3 (Complexity): Replace 7 any types with proper interfaces,
  extract 6 helpers from prepareMediaMessage (CC 85 -> ~15)
- Sprint 4 (Config): Remove 22 dead env vars from .env, add 30 missing
  vars to .env.example, standardize naming

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 10:16:04 +07:00
MythEclipse 5d9713b162 chore(config): update TELE_UPLOAD_URL domain across all services 2026-06-04 17:12:02 +07:00
MythEclipse b23ccf0837 chore: session auto-commit 2026-06-02 17:12:42 +07:00
MythEclipse 385fc0bbbf feat: add response logger for moderation/vision audit trail + cache model versioning
- Add responseLogger.ts with comprehensive logging: vision analysis, moderation
  analysis batches, cache hit/miss events, errors, retries, false positives,
  model version changes, cache invalidation
- Integrate responseLogger calls into llmModerationClient.ts (cache events,
  moderation results, errors, retries)
- Add model_version column to text_analysis_cache schema with composite
  index for efficient filtering
- Add migration SQL for model_version column
- Document CACHE_MODEL_VERSION in .env.example
2026-06-02 12:23:52 +07:00
MythEclipse 7607282db2 feat(database): add automatic migration on startup and enhance text analysis cache source 2026-05-31 23:01:38 +07:00
MythEclipse 34c4e3e017 feat(moderation): add support for vision model in moderation analysis 2026-05-31 20:12:16 +07:00
MythEclipse 47bac6ff8f feat: add Groq Llama Prompt Guard moderation integration
- Updated .env.example to include Groq API configuration.
- Enhanced config schema in config.ts to support Groq API keys and settings.
- Implemented Groq moderation API call in indonesianTextNormalizer.ts as a fallback for badword detection.
- Removed the Indonesian slang normalization function and related tests to streamline moderation logic.
- Updated tests to reflect changes in moderation strategy, focusing on emoji normalization and badword detection.
2026-05-31 19:42:39 +07:00
MythEclipse 8a712cff9b Refactor database handling to exclusively support PostgreSQL
- Removed SQLite support from the configuration and database initialization logic.
- Updated database migration scripts to focus solely on PostgreSQL migrations.
- Simplified logging messages to reflect PostgreSQL usage.
- Adjusted database schema definitions to remove SQLite-specific types and structures.
- Modified tests to ensure compatibility with PostgreSQL, including changes to table creation and data types.
- Cleaned up unused imports and code related to SQLite.
2026-05-31 15:46:18 +07:00
MythEclipse d1ef036358 feat(redis): integrate Redis for job queue management and persistent storage 2026-05-31 14:21:43 +07:00
MythEclipseandClaude Opus 4.8 202ea311a5 feat: lower auto-delete confidence threshold to 0.50, add allowed severities to defaults
- AUTO_DELETE_MIN_CONFIDENCE default 0.92 -> 0.50
- AUTO_DELETE_ALLOWED_SEVERITIES default 'critical' -> 'critical,high,medium'
- Add auto-delete config section to .env.example

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-05-30 15:35:07 +07:00
MythEclipse 8f6a35f591 feat: integrate NVIDIA Nemotron-3 Content Safety API for Indonesian badword detection
- Added configuration options for NVIDIA Nemotron API key, model, and base URL.
- Refactored badword detection to utilize NVIDIA API, with a fallback to a local badword list.
- Updated moderation functions to handle asynchronous operations for text evidence generation.
- Removed dependency on the `indonesian-badwords` package and implemented custom detection logic.
- Enhanced tests to accommodate asynchronous behavior and validate new detection methods.
2026-05-30 14:48:50 +07:00
MythEclipse 7eb01606b7 feat(moderation): enhance attachment handling and AI analysis integration 2026-05-21 02:39:28 +07:00
MythEclipse 3d64228d6a feat(config): add AI analysis tuning parameters and update related logic 2026-05-21 01:55:50 +07:00
MythEclipseandClaude Opus 4.7 ed438e6fc0 feat: split text and voice channel selection
Separate text moderation and voice recording guild/channel state so each workflow can persist and operate independently.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 15:58:38 +07:00
MythEclipse 818a059121 feat: add PostgreSQL config and dependencies 2026-05-14 14:44:01 +07:00
MythEclipse 6e203604ec feat: remove OpenAI moderation configuration and update AI analysis logic 2026-05-14 02:44:26 +07:00
MythEclipse be6c9f8132 feat: add AI analysis integration with moderation and LLM processing 2026-05-14 02:31:16 +07:00
MythEclipse 0b8111de81 feat(moderation): implement backlog message synchronization and enhance message metadata handling 2026-05-13 21:04:45 +07:00
MythEclipse b13dfb2ece config: add moderation watcher configuration variables
- Add MONITOR_GUILD_ID for target server monitoring
- Add PICSER_UPLOAD_URL for attachment upload endpoint
- Add attachment upload timeout and size limit settings
- Add retry attempts configuration for upload failures
2026-05-13 19:34:14 +07:00
MythEclipse 220c3b93d2 feat: implement Opus decoder runtime checks and add tests for decoder functionality 2026-05-13 18:56:44 +07:00
MythEclipse f9a4b4a92d feat: add muxer job queue with SQLite integration and metrics tracking 2026-05-13 17:06:22 +07:00
MythEclipse 978c2c468d feat: enhance configuration management and add error handling 2026-05-13 16:57:07 +07:00
MythEclipse 9497e721e0 docs: add env example configuration 2026-05-13 16:05:56 +07:00
baharsah c911b06b95 feat: implement Opus demuxing for browser streams, cache headers for playback, and enable microphone input in recorder 2026-05-13 01:03:46 +07:00
root 1a5449f16d init 2026-05-12 19:39:51 +07:00