Arsitektur asepharyana-hub

Topologi Fisik

Dua node terhubung via Tailscale overlay network:

┌──────────────────────────────┐       ┌──────────────────────────────┐
│         orangevps (VPS)       │       │        imrnes (Bare-metal)   │
│  IP: 45.127.35.244           │       │  Tailscale: 100.121.180.82   │
│  Tailscale: 100.x.x.x        │◄──────┤                              │
│                              │       │  Layanan:                    │
│  Layanan:                    │       │  ├─ PostgreSQL (port 6432)   │
│  ├─ Caddy (port 80/443)     │       │  └─ Redis (port 6379)        │
│  ├─ NATS + JetStream         │       │                              │
│  ├─ Dapr Placement           │       └──────────────────────────────┘
│  ├─ Redis (cache, Dapr)      │
│  ├─ Scraper API + Dapr       │
│  └─ Hub (Next.js SPA)       │
└──────────────────────────────┘

Konektivitas Container ke Tailscale

Container di orangevps tidak bisa langsung mencapai IP Tailscale (100.x.x.x). Route Tailscale harus ditambahkan ke tabel routing utama (main) via tailscale-routes.service agar traffic dari container bisa melewati host ke Tailscale.

Alur Request HTTP (External)

Internet
   │
   ▼ Port 443
Caddy 2.11.4 (auto-TLS LE, HTTP/3)
   ├─ TLS termination (sertifikat dari volume mount)
   ├─ Middleware chain: secure-headers → compress → retry → rate-limit → buffer
   ├─ Plugin: real-ip (Cloudflare), block-sensitive-paths
   │
   ▼ Router matching
Host(`asepharyana.my.id`) || Host(`www.asepharyana.my.id`) → hub
host(`hub.asepharyana.my.id`) → hub (SPA + dashboard)
Host(`scraper.asepharyana.my.id`) || Host(`api.asepharyana.my.id`) → scraper-api
   │
   ├─ hub (Next.js, port 4003)
   │   ├─ / — Portfolio SPA
   │   ├─ /dashboard — Ops dashboard (client-side, auto-refresh 15s)
   │   ├─ /api/dashboard — JSON: systemd services, Jaeger traces, Prometheus metrics
   │   └─ Metrics via node-exporter + app endpoints
   │
   ▼ Service load balancer
http://scraper-api:4091
   │
   ▼
Scraper API (Rust / Axum)
   ├─ Health check: GET /, respon 200
   ├─ REST endpoints
   ├─ Database via `DATABASE_URL` (Tailscale → PostgreSQL di imrnes)
   ├─ Cache via `REDIS_URL` (Redis lokal di container)
   └─ Pub/sub via Dapr sidecar (localhost:3500)

Infrastruktur Internal

Docker Compose Project

Semua service berjalan dalam satu Docker Compose project bernama compose dan bergabung di network app-shared-net:

File	Service	Peran
`traefik.yml`	`traefik`	Reverse proxy + TLS + metrics Prometheus
`shared.yml`	`redis`	Cache, session store, backend Dapr pub/sub & state
`nats.yml`	`nats`	Message broker + JetStream persistent streaming
`dapr.yml`	`dapr-placement`	Koordinasi actor placement untuk sidecar Dapr
`scraper.yml`	`scraper-api` + `scraper-api-dapr`	Aplikasi Rust + sidecar Dapr
systemd hub	`hub`	Next.js SPA portfolio + dashboard
`observability.yml`	`otel-collector`, `jaeger`, `prometheus`, `node-exporter`	Tracing, metrics, observability

Dapr Sidecar Pattern

Setiap aplikasi yang menggunakan Dapr mendapat sidecar container daprd:

┌─────────────────────┐
│  scraper-api        │
│  (app port 4091)    │
└────────┬────────────┘
         │ localhost:3500 (HTTP)
         │ localhost:50001 (gRPC)
┌────────▼────────────┐
│  scraper-api-dapr   │
│  (daprd sidecar)    │
│                     │
│  Dapr components:   │
│  ├─ pubsub.redis    │
│  └─ state.redis     │
└─────────────────────┘

Komponen Dapr:

Komponen	Tipe	Backend
`pubsub`	`pubsub.redis`	`redis:6379`
`statestore`	`state.redis`	`redis:6379` (prefix `dapr`)

Monitoring & Auto-Discovery

Prometheus Docker Auto-Discovery

Prometheus menggunakan docker_sd_configs untuk auto-detect container yang perlu di-scrape. Cukup tambah label pada container:

labels:
  - 'prometheus.io/scrape=true'
  - 'prometheus.io/port=8080'       # port metrics endpoint
  - 'prometheus.io/path=/metrics'   # optional, default /metrics

Prometheus akan auto-detect dan mulai scrape container dalam 15 detik.

Traefik Metrics

Traefik mengekspos metrics Prometheus di port 8080 (--metrics.prometheus=true). Metrics yang tersedia:

Metric	Query untuk dashboard
Request rate	`sum(rate(traefik_service_requests_total[1m]))`
Latency	`avg(traefik_service_request_duration_seconds_sum / traefik_service_request_duration_seconds_count) * 1000`
Error rate	`sum(rate(traefik_service_requests_total{code=~"5.."}[1m]))`

Dashboard di /api/dashboard returns node metrics + Traefik range data untuk 4 sparkline charts (RPS, latency, errors, trace volume).

Docker Socket Access

Container yang perlu akses Docker socket (/var/run/docker.sock) harus punya group docker (GID 988):

volumes:
  - /var/run/docker.sock:/var/run/docker.sock:ro
group_add:
  - '988'
```di compose atau `--group-add 988` via CLI. Berlaku untuk `hub` (container list) dan `prometheus` (Docker SD).

### NATS + JetStream

NATS berjalan dengan flag `-js` untuk mengaktifkan JetStream. Persistent stream disimpan di volume `nats_data`. Dapr pub/sub routing:

Service → Dapr sidecar (pubsub.redis) → Redis streams


> **Catatan:** Saat ini Dapr pub/sub menggunakan Redis, bukan NATS. Jika ingin migrasi ke NATS untuk pub/sub, komponen Dapr perlu diganti dengan `pubsub.nats`.

## Arsitektur CI/CD

Push ke main (apps/, infra/) │ ▼ docker-build-push.yml ├─ Phase 1: Detect changed services ├─ Phase 2: Build & Push image ke GHCR └─ Phase 3: Update compose manifest + submodule pointer │ ▼ (workflow_run trigger) deploy-docker.yml ├─ SSH ke orangevps ├─ Git sync, pull images ├─ Remove stale containers └─ Selective restart service


Submodule update dari remote repo via `repository_dispatch`:

Push ke asepharyana-hub-scraper │ ▼ (repository_dispatch) update-submodule.yml ├─ Update submodule pointer └─ Commit & push ke hub repo │ ▼ (repository_dispatch trigger) docker-build-push.yml └─ Build, push, deploy


## Image Tagging Strategy

| Tag | Contoh | Penggunaan |
|-----|--------|------------|
| `sha-<short>` | `sha-a3c5d74` | Immutable, deterministic rollback |
| `latest` | `latest` | Mutable, convenience |
| `buildcache` | `sha-a3c5d74-buildcache` | Registry-based build cache (internal) |

## Networking

### Port Map

| Port | Service | Deskripsi |
|------|---------|-----------|
| 443 | Traefik | HTTPS eksternal |
| 80 | Traefik | Redirect ke HTTPS |
| 4222 | NATS | Client connections |
| 8222 | NATS | HTTP monitor / health |
| 6379 | Redis | Internal container network |
| 3500 | Dapr sidecar | Dapr HTTP API (per service) |
| 50001 | Dapr sidecar | Dapr gRPC API (per service) |
| 50005 | Dapr placement | Actor placement |
| 4091 | Scraper API | Aplikasi HTTP |

## Event Topics Convention

Semua event menggunakan prefix `hub.`:

| Topic | Payload | Deskripsi |
|-------|---------|-----------|
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |

## Service Registry (Traefik)

Domain routing:

| Subdomain | Service | URL Backend |
|-----------|---------|-------------|
| `asepharyana.my.id` (root) | Hub SPA + dashboard | `http://hub:3000` |
| `www.*` | Hub (alias) | `http://hub:3000` |
| `hub.*` | Hub (alias) | `http://hub:3000` |
| `scraper.*` | Scraper API | `http://scraper-api:4091` |
| `api.*` | Scraper API (alias) | `http://scraper-api:4091` |
| `traefik.*` | Traefik Dashboard | `api@internal` |
| `jaeger.*` | Jaeger UI | `http://jaeger:16686` |

Semua domain tersedia di:
- `<service>.asepharyana.my.id`
- `<service>.asepharyana.web.id`

8.6 KiB Raw Blame History