diff --git a/CLAUDE.md b/CLAUDE.md
index 4adcacf..196231f 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -151,8 +151,12 @@ The repository includes a full Prometheus → ClickHouse metric pipeline as a gi
All three share the `zeavis_` metric prefix and are scraped by the Telemetry Prometheus instance via `file_sd_configs` (see `telemetry/prometheus/targets/zeavis-edu.json`).
+**IMPORTANT — Production architecture:** ZeaVis Edu apps and the Telemetry stack run on **separate VPS instances** connected via **Tailscale** (mesh VPN). Prometheus scrapes the API and ML service through their **Tailscale IPs** (e.g. `100.x.x.a:3000`), not via Docker hostnames. The target file `telemetry/prometheus/targets/zeavis-edu.json` has `__CHANGE_ME__` placeholders — before deploying, replace with the actual Tailscale IPs of the app VPS.
+
The telemetry stack is managed from the project root via `make telemetry-*` targets (see `Makefile`). The Docker Compose files in `telemetry/deploy/` define 6 services (Prometheus, Metric Ingester, Vector, ClickHouse, Query Proxy, Telemetry UI).
+For **local single-host dev**, Prometheus can reach app services via a shared Docker network (`app-shared-net`). Use `make telemetry-up-local` for this mode — it includes the `docker-compose.telemetry.yml` override.
+
## Fullstack application architecture
The root TypeScript workspace is a Bun + Moon monorepo:
diff --git a/METRICS.md b/METRICS.md
index 2e1f21d..af8ba31 100644
--- a/METRICS.md
+++ b/METRICS.md
@@ -15,8 +15,11 @@ application stack and the payload each service provides.
| Prometheus Collector | — | `GET /metrics` (self) | 9090 |
> In production all metrics are scraped by the Prometheus collector running in the
-> Telemetry stack. See [`telemetry/prometheus/targets/`](./telemetry/prometheus/targets/)
-> for the auto‑discovery configuration.
+> Telemetry stack on a **separate VPS** connected via **Tailscale**.
+> See [`telemetry/prometheus/targets/`](./telemetry/prometheus/targets/)
+> for the auto‑discovery configuration. Target files must use **Tailscale IPs**
+> (e.g. `100.x.x.a:3000`), not Docker hostnames, because the services are on
+> different hosts.
>
> In production (nginx), the web app proxies `/metrics` to the API service:
> see [`apps/web/nginx.conf`](apps/web/nginx.conf).
@@ -98,16 +101,21 @@ The Telemetry submodule includes a Prometheus instance that uses
```json
[
{
- "targets": ["zeavis-api:3000"],
- "labels": { "service": "zeavis-api", "component": "backend" }
+ "targets": ["100.x.x.a:3000"],
+ "labels": { "service": "zeavis-api", "component": "backend", "env": "production" }
},
{
- "targets": ["zeavis-ml:8000"],
- "labels": { "service": "zeavis-ml", "component": "inference" }
+ "targets": ["100.x.x.b:8000"],
+ "labels": { "service": "zeavis-ml", "component": "inference", "env": "production" }
}
]
```
+> ⚠️ **Cross-VPS:** Gunakan **IP Tailscale** (bukan Docker hostname) karena
+> Prometheus dan ZeaVis Edu berjalan di VPS berbeda. Pastikan port service
+> (`:3000`, `:8000`) terekspos di `0.0.0.0` atau diizinkan oleh aturan
+> `iptables`/`ufw` untuk interface Tailscale (`tailscale0`/`100.x.x.x/10`).
+
The Prometheus config (in `telemetry/prometheus/prometheus.yml`) will
automatically pick up new files within its 15‑second scrape interval —
no restart required.
diff --git a/Makefile b/Makefile
index 3c6c015..8559fa3 100644
--- a/Makefile
+++ b/Makefile
@@ -44,21 +44,23 @@ TELEMETRY_COMPOSE := telemetry/deploy/docker-compose.yml
TELEMETRY_LOCAL := telemetry/deploy/docker-compose.local.yml
TELEMETRY_ZEAVIS := docker-compose.telemetry.yml
-# Start all telemetry services
+# Start all telemetry services (standalone — cross-VPS production mode)
+# Prometheus scrapes ZeaVis Edu via Tailscale IPs, not Docker network.
telemetry-up:
- @echo ">> Starting Telemetry stack..."
+ @echo ">> Starting Telemetry stack (standalone)..."
CLICKHOUSE_USER=$${CLICKHOUSE_USER:-telemetry} \
CLICKHOUSE_PASSWORD=$${CLICKHOUSE_PASSWORD:-telemetry} \
- docker compose -f $(TELEMETRY_COMPOSE) -f $(TELEMETRY_ZEAVIS) up -d
+ docker compose -f $(TELEMETRY_COMPOSE) up -d
@echo ">> Telemetry stack started. Use 'make telemetry-logs' to view output."
-# Start telemetry services with local port overrides (no Tailscale)
+# Start telemetry services with ZeaVis Edu network sharing (local single-host dev)
+# Prometheus can scrape app services via app-shared-net Docker network.
telemetry-up-local:
- @echo ">> Starting Telemetry stack (local mode)..."
+ @echo ">> Starting Telemetry stack (local dev mode)..."
CLICKHOUSE_USER=$${CLICKHOUSE_USER:-telemetry} \
CLICKHOUSE_PASSWORD=$${CLICKHOUSE_PASSWORD:-telemetry} \
docker compose -f $(TELEMETRY_COMPOSE) -f $(TELEMETRY_LOCAL) -f $(TELEMETRY_ZEAVIS) up -d
- @echo ">> Telemetry stack started in local mode."
+ @echo ">> Telemetry stack started in local dev mode."
# Stop all telemetry services
telemetry-down:
diff --git a/README.md b/README.md
index c134bc0..a50dbf4 100644
--- a/README.md
+++ b/README.md
@@ -218,33 +218,34 @@ docker compose up -d
Proyek ini menyertakan pipeline telemetry metric sebagai git submodule di `telemetry/`. Pipeline mengalirkan metrik dari seluruh service ZeaVis Edu ke ClickHouse untuk analisis dan visualisasi jangka panjang.
-### Arsitektur
+### Arsitektur (Production)
+
+Di production, aplikasi dan telemetry berjalan di **VPS terpisah** dan terhubung via **Tailscale** (mesh VPN). Prometheus di VPS telemetry melakukan scrape ke service ZeaVis Edu melalui IP Tailscale masing-masing.
```mermaid
flowchart LR
- subgraph Apps["ZeaVis Edu"]
- W[Web / React]
- A[API / Elysia]
- M[ML Service / Axum]
+ subgraph VPS1["VPS — ZeaVis Edu (App)"]
+ W[Web / React
api-zeavisedu.asepharyana.id]
+ A[API / Elysia
:3000]
+ M[ML Service / Axum
:8000]
end
- subgraph Telemetry["Telemetry Pipeline"]
- P[Prometheus]
- MI[Metric Ingester]
- V[Vector]
- CH[ClickHouse]
- QP[Query Proxy]
- TUI[Telemetry UI]
+ subgraph VPS2["VPS — Telemetry Stack"]
+ P[Prometheus
:9090]
+ MI[Metric Ingester
:9091]
+ V[Vector
:9001]
+ CH[ClickHouse
:8123]
+ QP[Query Proxy
:9092]
+ TUI[Telemetry UI
:8181]
end
- W -->|"GET /metrics"| P
- A -->|"GET /metrics"| P
- M -->|"GET /metrics"| P
+ P -.->|"scrape via Tailscale IP
100.x.x.a:3000/metrics"| A
+ P -.->|"scrape via Tailscale IP
100.x.x.a:8000/metrics"| M
P -->|remote_write| MI
- MI -->|HTTP POST| V
- V -->|JSONEachRow| CH
- QP -->|SQL| CH
- TUI -->|/proxy/query| QP
+ MI --> V
+ V --> CH
+ QP --> CH
+ TUI --> QP
```
Setiap service ZeaVis Edu mengekspos endpoint `/metrics` dalam format Prometheus text:
@@ -255,6 +256,8 @@ Setiap service ZeaVis Edu mengekspos endpoint `/metrics` dalam format Prometheus
| API (Elysia) | `GET /metrics` | 3000 |
| ML Service (Axum) | `GET /metrics` | 8000 |
+Prometheus di VPS telemetry melakukan **scrape langsung** ke API dan ML service melalui IP Tailscale mereka, bukan melalui domain publik. Konfigurasi target ada di `telemetry/prometheus/targets/zeavis-edu.json` — isi dengan IP Tailscale dari service yang dituju.
+
Lihat [`METRICS.md`](./METRICS.md) untuk daftar lengkap metrik yang diekspos.
### Service Telemetry
@@ -268,6 +271,18 @@ Lihat [`METRICS.md`](./METRICS.md) untuk daftar lengkap metrik yang diekspos.
| 5 | **Query Proxy** | Read-only SQL proxy, tenant isolation | 9092 |
| 6 | **Telemetry UI** | Vue 3 metrics dashboard | 8181 |
+### Arsitektur (Local Dev)
+
+Untuk development lokal di satu mesin, telemetry dan app bisa jalan bareng di satu Docker host. Prometheus bisa scrape service lewat Docker network yang sama.
+
+```bash
+# Setup network
+docker network create app-shared-net
+
+# Build & start telemetry (dengan network sharing)
+make telemetry-up-local
+```
+
### Menjalankan Telemetry Stack
Semua operasi telemetry dijalankan dari **root proyek** melalui Makefile:
@@ -317,15 +332,17 @@ make telemetry-up-local
Prometheus menggunakan `file_sd_configs` untuk menemukan target secara dinamis. Cukup letakkan file JSON di `telemetry/prometheus/targets/` dan Prometheus akan otomatis mendeteksinya dalam 15 detik — tanpa restart.
-File target ZeaVis Edu sudah tersedia di [`telemetry/prometheus/targets/zeavis-edu.json`](telemetry/prometheus/targets/zeavis-edu.json):
+File template sudah tersedia di [`telemetry/prometheus/targets/zeavis-edu.json`](telemetry/prometheus/targets/zeavis-edu.json). **Sebelum production, isi `__CHANGE_ME__` dengan IP Tailscale masing-masing service:**
```json
[
- { "targets": ["zeavis-api:3000"], "labels": { "service": "zeavis-api", "component": "backend" } },
- { "targets": ["zeavis-ml:8000"], "labels": { "service": "zeavis-ml", "component": "inference" } }
+ { "targets": ["100.x.x.a:3000"], "labels": { "service": "zeavis-api", "component": "backend", "env": "production" } },
+ { "targets": ["100.x.x.a:8000"], "labels": { "service": "zeavis-ml", "component": "inference", "env": "production" } }
]
```
+> **Catatan:** Aplikasi ZeaVis Edu mengekspose port Docker-nya (`:3000`, `:8000`) langsung ke host via `docker-compose.yml`. Pastikan port-port tersebut terbuka di network Tailscale (biasanya iptables Tailscale mengizinkan koneksi ke port localhost).
+
### Environment Variables Telemetry
| Variable | Default | Deskripsi |