docs: add telemetry stack documentation to CLAUDE.md, METRICS.md, and README.md

2026-06-07 21:00:22 +07:00
parent 2c41b80f97
commit 36216601e3
3 changed files with 173 additions and 0 deletions
@@ -107,6 +107,31 @@ Run the ML service directly:
 cd apps/ml-service && cargo run
 ```

+Run the Telemetry stack:
+
+```bash
+# Start all telemetry services (Prometheus, Ingester, Vector, ClickHouse, Query Proxy, Telemetry UI)
+make telemetry-up
+
+# Local dev mode (port bindings exposed)
+make telemetry-up-local
+
+# Check health of all telemetry services
+make telemetry-status
+
+# View telemetry logs
+make telemetry-logs [s=<service>]
+
+# Build telemetry components
+make telemetry-build
+
+# Send a test metric
+make telemetry-test-metric
+
+# Stop telemetry
+make telemetry-down
+```
+
 ## High-level architecture

 - `Machine_Learning/preprocessing.py` prepares the training dataset locally. It extracts three source ZIP files, merges selected class folders into `dataset/`, maps selected Mandarin labels from Dataset 3 via `desc.json`, removes known problematic image files, then creates `dataset.zip` for upload to Google Drive/Colab.
@@ -116,6 +141,18 @@ cd apps/ml-service && cargo run
 - TensorFlow.js export is intentionally done with the `tensorflowjs_converter` CLI rather than from Python to avoid protobuf/runtime conflicts documented in the README.
 - `apps/ml-service/` is a Rust/Axum service that loads the ONNX model and serves HTTP endpoints for health checks, metadata, and image classification predictions. It uses ONNX Runtime for cross-platform inference performance.

+## Telemetry architecture
+
+The repository includes a full Prometheus → ClickHouse metric pipeline as a git submodule at `telemetry/`. Each ZeaVis Edu service exposes a `GET /metrics` endpoint:
+
+- **Web app** (`apps/web`): In dev mode, a Vite plugin serves client-side session metrics (page views, Web Vitals). In production, nginx proxies `/metrics` to the API service. Source: `apps/web/src/lib/telemetry.ts`, `apps/web/vite-plugin-metrics.ts`.
+- **API** (`apps/api`): Uses `prom-client` for Node.js default metrics plus custom HTTP, auth, classification, and diagnosis counters/histograms. Source: `apps/api/src/lib/telemetry.ts`, exposed via `apps/api/src/routes/metrics.ts`.
+- **ML service** (`apps/ml-service`): Uses the `prometheus` Rust crate for HTTP metrics, prediction counts, and model load status. Source: `apps/ml-service/src/telemetry.rs`.
+
+All three share the `zeavis_` metric prefix and are scraped by the Telemetry Prometheus instance via `file_sd_configs` (see `telemetry/prometheus/targets/zeavis-edu.json`).
+
+The telemetry stack is managed from the project root via `make telemetry-*` targets (see `Makefile`). The Docker Compose files in `telemetry/deploy/` define 6 services (Prometheus, Metric Ingester, Vector, ClickHouse, Query Proxy, Telemetry UI).
+
 ## Fullstack application architecture

 The root TypeScript workspace is a Bun + Moon monorepo:
@@ -17,6 +17,12 @@ application stack and the payload each service provides.
 > In production all metrics are scraped by the Prometheus collector running in the
 > Telemetry stack. See [`telemetry/prometheus/targets/`](./telemetry/prometheus/targets/)
 > for the auto‑discovery configuration.
+>
+> In production (nginx), the web app proxies `/metrics` to the API service:
+> see [`apps/web/nginx.conf`](apps/web/nginx.conf).
+>
+> For local development the Vite plugin `vite-plugin-metrics.ts` serves
+> client‑side session metrics at `GET /metrics` on the Vite dev server.

 ---

@@ -77,6 +77,17 @@ Model klasifikasi menargetkan empat label berbahasa Indonesia:
 - GitHub Container Registry
 - Traefik labels untuk routing deployment

+### Telemetry & Observability
+
+- Prometheus — metric scraping & remote_write
+- Metric Ingester (Go) — enrichment, filtering, aggregation
+- Vector — buffering & backpressure
+- ClickHouse — columnar analytical storage
+- Query Proxy (Go) — read-only SQL proxy
+- Telemetry UI (Vue 3) — metrics dashboard
+- Semua service ZeaVis Edu (web, api, ml-service) mengekspos metrik Prometheus di `/metrics`
+- Client-side Web Vitals (CLS, FCP, INP, LCP, TTFB) dikumpulkan di frontend
+
 ## Prasyarat

 Untuk menjalankan seluruh project secara lokal, siapkan:
@@ -203,6 +214,125 @@ Contoh menjalankan compose setelah environment dan network siap:
 docker compose up -d
 ```

+## Telemetry Stack
+
+Proyek ini menyertakan pipeline telemetry metric sebagai git submodule di `telemetry/`. Pipeline mengalirkan metrik dari seluruh service ZeaVis Edu ke ClickHouse untuk analisis dan visualisasi jangka panjang.
+
+### Arsitektur
+
+```mermaid
+flowchart LR
+    subgraph Apps["ZeaVis Edu"]
+        W[Web / React]
+        A[API / Elysia]
+        M[ML Service / Axum]
+    end
+
+    subgraph Telemetry["Telemetry Pipeline"]
+        P[Prometheus]
+        MI[Metric Ingester]
+        V[Vector]
+        CH[ClickHouse]
+        QP[Query Proxy]
+        TUI[Telemetry UI]
+    end
+
+    W -->|"GET /metrics"| P
+    A -->|"GET /metrics"| P
+    M -->|"GET /metrics"| P
+    P -->|remote_write| MI
+    MI -->|HTTP POST| V
+    V -->|JSONEachRow| CH
+    QP -->|SQL| CH
+    TUI -->|/proxy/query| QP
+```
+
+Setiap service ZeaVis Edu mengekspos endpoint `/metrics` dalam format Prometheus text:
+
+| Service               | Endpoint           | Port (lokal) |
+|-----------------------|--------------------|--------------|
+| Web (Vite dev)        | `GET /metrics`     | 5173         |
+| API (Elysia)          | `GET /metrics`     | 3000         |
+| ML Service (Axum)     | `GET /metrics`     | 8000         |
+
+Lihat [`METRICS.md`](./METRICS.md) untuk daftar lengkap metrik yang diekspos.
+
+### Service Telemetry
+
+| # | Service | Peran | Port |
+|---|---------|------|------|
+| 1 | **Prometheus** | Metric scraping & remote_write | 9090 |
+| 2 | **Metric Ingester** | Enrichment, filtering, aggregation | 9091 |
+| 3 | **Vector** | Buffering, backpressure, retry | 9001 |
+| 4 | **ClickHouse** | Columnar analytical storage | 8123 / 9000 |
+| 5 | **Query Proxy** | Read-only SQL proxy, tenant isolation | 9092 |
+| 6 | **Telemetry UI** | Vue 3 metrics dashboard | 8181 |
+
+### Menjalankan Telemetry Stack
+
+Semua operasi telemetry dijalankan dari **root proyek** melalui Makefile:
+
+```bash
+# Build komponen telemetry (metric-ingester + telemetry-ui)
+make telemetry-build
+
+# Start semua service telemetry (mode produksi, via Tailscale)
+make telemetry-up
+
+# Start semua service telemetry (mode lokal — port langsung terbuka)
+make telemetry-up-local
+
+# Cek status kesehatan semua service
+make telemetry-status
+
+# Lihat log (semua service, atau filter dengan s=)
+make telemetry-logs
+make telemetry-logs s=metric-ingester
+
+# Restart service tertentu
+make telemetry-restart s=prometheus
+
+# Kirim test metric
+make telemetry-test-metric
+
+# Stop semua service
+make telemetry-down
+```
+
+Untuk development lokal:
+
+```bash
+# Setup network jika belum ada
+docker network create telemetry-net
+docker network create app-shared-net
+
+# Build & start
+make telemetry-build
+make telemetry-up-local
+
+# Buka dashboard di http://localhost:8181
+```
+
+### Prometheus Auto-Discovery
+
+Prometheus menggunakan `file_sd_configs` untuk menemukan target secara dinamis. Cukup letakkan file JSON di `telemetry/prometheus/targets/` dan Prometheus akan otomatis mendeteksinya dalam 15 detik — tanpa restart.
+
+File target ZeaVis Edu sudah tersedia di [`telemetry/prometheus/targets/zeavis-edu.json`](telemetry/prometheus/targets/zeavis-edu.json):
+
+```json
+[
+  { "targets": ["zeavis-api:3000"], "labels": { "service": "zeavis-api", "component": "backend" } },
+  { "targets": ["zeavis-ml:8000"],  "labels": { "service": "zeavis-ml", "component": "inference" } }
+]
+```
+
+### Environment Variables Telemetry
+
+| Variable | Default | Deskripsi |
+|----------|---------|-----------|
+| `CLICKHOUSE_USER` | `telemetry` | User ClickHouse |
+| `CLICKHOUSE_PASSWORD` | `telemetry` | Password ClickHouse |
+
 ## Workflow Machine Learning

 Detail lengkap tersedia di [`Machine_Learning/README.md`](Machine_Learning/README.md). Ringkasnya: