Add comprehensive documentation for CI/CD pipeline, NATS, security practices, Tailscale networking, and troubleshooting guide
- Created CI/CD pipeline documentation detailing workflows, triggers, and actions for `asepharyana-hub`. - Added NATS + JetStream guide covering architecture, configuration, CLI tools, and event topics. - Introduced a security guide outlining best practices for secrets management, TLS, container security, and access control. - Documented Tailscale networking setup and troubleshooting for connectivity between VPS and bare-metal nodes. - Compiled a troubleshooting guide addressing common issues across deployment, Dapr, NATS, Traefik, Tailscale, Docker, database, and submodules.
This commit is contained in:
asepharyana
@@ -0,0 +1,186 @@
|
|||||||
|
# Arsitektur asepharyana-hub
|
||||||
|
|
||||||
|
## Topologi Fisik
|
||||||
|
|
||||||
|
Dua node terhubung via **Tailscale** overlay network:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌──────────────────────────────┐ ┌──────────────────────────────┐
|
||||||
|
│ orangevps (VPS) │ │ imrnes (Bare-metal) │
|
||||||
|
│ IP: 45.127.35.244 │ │ Tailscale: 100.121.180.82 │
|
||||||
|
│ Tailscale: 100.x.x.x │◄──────┤ │
|
||||||
|
│ │ │ Layanan: │
|
||||||
|
│ Layanan: │ │ ├─ PostgreSQL (port 6432) │
|
||||||
|
│ ├─ Traefik (port 80/443) │ │ └─ Redis (port 6379) │
|
||||||
|
│ ├─ NATS + JetStream │ │ │
|
||||||
|
│ ├─ Dapr Placement │ └──────────────────────────────┘
|
||||||
|
│ ├─ Redis (cache, Dapr) │
|
||||||
|
│ ├─ Scraper API + Dapr │
|
||||||
|
│ └─ (future services) │
|
||||||
|
└──────────────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
### Konektivitas Container ke Tailscale
|
||||||
|
|
||||||
|
Container di `orangevps` tidak bisa langsung mencapai IP Tailscale (`100.x.x.x`). Route Tailscale harus ditambahkan ke tabel routing utama (`main`) via `tailscale-routes.service` agar traffic dari container bisa melewati host ke Tailscale.
|
||||||
|
|
||||||
|
## Alur Request HTTP (External)
|
||||||
|
|
||||||
|
```
|
||||||
|
Internet
|
||||||
|
│
|
||||||
|
▼ Port 443
|
||||||
|
Traefik (v3.6)
|
||||||
|
├─ TLS termination (sertifikat dari volume mount)
|
||||||
|
├─ Middleware chain: secure-headers → compress → retry → rate-limit → buffer
|
||||||
|
├─ Plugin: real-ip (Cloudflare), block-sensitive-paths
|
||||||
|
│
|
||||||
|
▼ Router matching
|
||||||
|
Host(`scraper.asepharyana.my.id`) || Host(`api.asepharyana.my.id`)
|
||||||
|
│
|
||||||
|
▼ Service load balancer
|
||||||
|
http://scraper-api:4091
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
Scraper API (Rust / Axum)
|
||||||
|
├─ Health check: GET /, respon 200
|
||||||
|
├─ REST endpoints
|
||||||
|
├─ Database via `DATABASE_URL` (Tailscale → PostgreSQL di imrnes)
|
||||||
|
├─ Cache via `REDIS_URL` (Redis lokal di container)
|
||||||
|
└─ Pub/sub via Dapr sidecar (localhost:3500)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Infrastruktur Internal
|
||||||
|
|
||||||
|
### Docker Compose Project
|
||||||
|
|
||||||
|
Semua service berjalan dalam satu Docker Compose project bernama `compose` dan bergabung di network `app-shared-net`:
|
||||||
|
|
||||||
|
| File | Service | Peran |
|
||||||
|
|------|---------|-------|
|
||||||
|
| `traefik.yml` | `traefik` | Reverse proxy, TLS termination, middleware |
|
||||||
|
| `shared.yml` | `redis` | Cache, session store, backend Dapr pub/sub & state |
|
||||||
|
| `nats.yml` | `nats` | Message broker + JetStream persistent streaming |
|
||||||
|
| `dapr.yml` | `dapr-placement` | Koordinasi actor placement untuk sidecar Dapr |
|
||||||
|
| `scraper.yml` | `scraper-api` + `scraper-api-dapr` | Aplikasi Rust + sidecar Dapr |
|
||||||
|
|
||||||
|
### Dapr Sidecar Pattern
|
||||||
|
|
||||||
|
Setiap aplikasi yang menggunakan Dapr mendapat sidecar container `daprd`:
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────────────┐
|
||||||
|
│ scraper-api │
|
||||||
|
│ (app port 4091) │
|
||||||
|
└────────┬────────────┘
|
||||||
|
│ localhost:3500 (HTTP)
|
||||||
|
│ localhost:50001 (gRPC)
|
||||||
|
┌────────▼────────────┐
|
||||||
|
│ scraper-api-dapr │
|
||||||
|
│ (daprd sidecar) │
|
||||||
|
│ │
|
||||||
|
│ Dapr components: │
|
||||||
|
│ ├─ pubsub.redis │
|
||||||
|
│ └─ state.redis │
|
||||||
|
└─────────────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
Komponen Dapr:
|
||||||
|
|
||||||
|
| Komponen | Tipe | Backend |
|
||||||
|
|----------|------|---------|
|
||||||
|
| `pubsub` | `pubsub.redis` | `redis:6379` |
|
||||||
|
| `statestore` | `state.redis` | `redis:6379` (prefix `dapr`) |
|
||||||
|
|
||||||
|
### NATS + JetStream
|
||||||
|
|
||||||
|
NATS berjalan dengan flag `-js` untuk mengaktifkan JetStream. Persistent stream disimpan di volume `nats_data`. Dapr pub/sub routing:
|
||||||
|
|
||||||
|
```
|
||||||
|
Service → Dapr sidecar (pubsub.redis) → Redis streams
|
||||||
|
```
|
||||||
|
|
||||||
|
> **Catatan:** Saat ini Dapr pub/sub menggunakan Redis, bukan NATS. Jika ingin migrasi ke NATS untuk pub/sub, komponen Dapr perlu diganti dengan `pubsub.nats`.
|
||||||
|
|
||||||
|
## Arsitektur CI/CD
|
||||||
|
|
||||||
|
```
|
||||||
|
Push ke main (apps/**, infra/**)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
docker-build-push.yml
|
||||||
|
├─ Phase 1: Detect changed services
|
||||||
|
├─ Phase 2: Build & Push image ke GHCR
|
||||||
|
└─ Phase 3: Update compose manifest + submodule pointer
|
||||||
|
│
|
||||||
|
▼ (workflow_run trigger)
|
||||||
|
deploy-docker.yml
|
||||||
|
├─ SSH ke orangevps
|
||||||
|
├─ Git sync, pull images
|
||||||
|
├─ Remove stale containers
|
||||||
|
└─ Selective restart service
|
||||||
|
```
|
||||||
|
|
||||||
|
Submodule update dari remote repo via `repository_dispatch`:
|
||||||
|
|
||||||
|
```
|
||||||
|
Push ke asepharyana-hub-scraper
|
||||||
|
│
|
||||||
|
▼ (repository_dispatch)
|
||||||
|
update-submodule.yml
|
||||||
|
├─ Update submodule pointer
|
||||||
|
└─ Commit & push ke hub repo
|
||||||
|
│
|
||||||
|
▼ (repository_dispatch trigger)
|
||||||
|
docker-build-push.yml
|
||||||
|
└─ Build, push, deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
## Image Tagging Strategy
|
||||||
|
|
||||||
|
| Tag | Contoh | Penggunaan |
|
||||||
|
|-----|--------|------------|
|
||||||
|
| `sha-<short>` | `sha-a3c5d74` | Immutable, deterministic rollback |
|
||||||
|
| `latest` | `latest` | Mutable, convenience |
|
||||||
|
| `buildcache` | `sha-a3c5d74-buildcache` | Registry-based build cache (internal) |
|
||||||
|
|
||||||
|
## Networking
|
||||||
|
|
||||||
|
### Port Map
|
||||||
|
|
||||||
|
| Port | Service | Deskripsi |
|
||||||
|
|------|---------|-----------|
|
||||||
|
| 443 | Traefik | HTTPS eksternal |
|
||||||
|
| 80 | Traefik | Redirect ke HTTPS |
|
||||||
|
| 4222 | NATS | Client connections |
|
||||||
|
| 8222 | NATS | HTTP monitor / health |
|
||||||
|
| 6379 | Redis | Internal container network |
|
||||||
|
| 3500 | Dapr sidecar | Dapr HTTP API (per service) |
|
||||||
|
| 50001 | Dapr sidecar | Dapr gRPC API (per service) |
|
||||||
|
| 50005 | Dapr placement | Actor placement |
|
||||||
|
| 4091 | Scraper API | Aplikasi HTTP |
|
||||||
|
|
||||||
|
## Event Topics Convention
|
||||||
|
|
||||||
|
Semua event menggunakan prefix `hub.`:
|
||||||
|
|
||||||
|
| Topic | Payload | Deskripsi |
|
||||||
|
|-------|---------|-----------|
|
||||||
|
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
|
||||||
|
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
|
||||||
|
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
|
||||||
|
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |
|
||||||
|
|
||||||
|
## Service Registry (Traefik)
|
||||||
|
|
||||||
|
Domain routing:
|
||||||
|
|
||||||
|
| Subdomain | Service | URL Backend |
|
||||||
|
|-----------|---------|-------------|
|
||||||
|
| `scraper.*` | Scraper API | `http://scraper-api:4091` |
|
||||||
|
| `api.*` | Scraper API (alias) | `http://scraper-api:4091` |
|
||||||
|
| `traefik.*` | Traefik Dashboard | `api@internal` |
|
||||||
|
|
||||||
|
Semua domain tersedia di:
|
||||||
|
- `<service>.asepharyana.my.id`
|
||||||
|
- `<service>.asepharyana.web.id`
|
||||||
@@ -0,0 +1,206 @@
|
|||||||
|
# Development Guide
|
||||||
|
|
||||||
|
Panduan setup lingkungan development lokal untuk kontributor `asepharyana-hub`.
|
||||||
|
|
||||||
|
## Prasyarat
|
||||||
|
|
||||||
|
| Tool | Versi Minimal | Catatan |
|
||||||
|
|------|---------------|---------|
|
||||||
|
| Git | 2.40+ | Submodule support |
|
||||||
|
| Docker | 24+ | Dengan Docker Compose v2 plugin |
|
||||||
|
| Rust | 1.85+ | Hanya untuk `apps/scraper` |
|
||||||
|
| Bun | 1.x | Root tooling (Biome) |
|
||||||
|
| Dapr CLI | 1.14+ | Opsional, untuk development dengan Dapr |
|
||||||
|
|
||||||
|
## Setup Awal
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Clone repo
|
||||||
|
git clone https://github.com/asepharyana/asepharyana-hub.git
|
||||||
|
cd asepharyana-hub
|
||||||
|
|
||||||
|
# 2. Init submodules
|
||||||
|
make init-submodules
|
||||||
|
|
||||||
|
# 3. Setup environment
|
||||||
|
cp .env.example .env
|
||||||
|
# Edit .env sesuai kebutuhan lokal
|
||||||
|
|
||||||
|
# 4. Install root dependencies
|
||||||
|
bun install
|
||||||
|
```
|
||||||
|
|
||||||
|
## Menjalankan Infrastruktur Lokal
|
||||||
|
|
||||||
|
Beberapa service membutuhkan Redis. Jalankan dengan:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make dev
|
||||||
|
# atau equivalen:
|
||||||
|
docker compose -f infra/compose/shared.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Ini akan menjalankan Redis Alpine di `localhost:6379`.
|
||||||
|
|
||||||
|
### (Opsional) NATS Lokal
|
||||||
|
|
||||||
|
Jika service membutuhkan pub/sub:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f infra/compose/nats.yml up -d
|
||||||
|
# NATS client: localhost:4222
|
||||||
|
# NATS monitor: localhost:8222
|
||||||
|
```
|
||||||
|
|
||||||
|
### (Opsional) Dapr Placement Lokal
|
||||||
|
|
||||||
|
Jika service membutuhkan sidecar Dapr:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f infra/compose/dapr.yml up -d
|
||||||
|
# Dapr placement: localhost:50005
|
||||||
|
```
|
||||||
|
|
||||||
|
## Menjalankan Service Lokal
|
||||||
|
|
||||||
|
### Scraper API (Rust)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Pastikan Redis sudah running (make dev)
|
||||||
|
cd apps/scraper
|
||||||
|
|
||||||
|
# Cargo run
|
||||||
|
cargo run
|
||||||
|
|
||||||
|
# Dengan Dapr sidecar (jika placement running)
|
||||||
|
dapr run \
|
||||||
|
--app-id scraper-api \
|
||||||
|
--app-port 4091 \
|
||||||
|
--dapr-http-port 3500 \
|
||||||
|
--resources-path ../../infra/dapr/components \
|
||||||
|
-- cargo run
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dengan Docker Compose (Full Stack)
|
||||||
|
|
||||||
|
Untuk menjalankan semua service sekaligus:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose \
|
||||||
|
-f infra/compose/shared.yml \
|
||||||
|
-f infra/compose/nats.yml \
|
||||||
|
-f infra/compose/dapr.yml \
|
||||||
|
-f infra/compose/scraper.yml \
|
||||||
|
--env-file .env \
|
||||||
|
up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Untuk service baru, tambahkan compose file-nya ke daftar.
|
||||||
|
|
||||||
|
## Update Submodules
|
||||||
|
|
||||||
|
### Pull latest dari semua submodule
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make update-submodules
|
||||||
|
# atau:
|
||||||
|
git submodule update --remote --merge --recursive
|
||||||
|
```
|
||||||
|
|
||||||
|
### Check status submodule
|
||||||
|
|
||||||
|
```bash
|
||||||
|
make status
|
||||||
|
# atau:
|
||||||
|
git submodule status
|
||||||
|
```
|
||||||
|
|
||||||
|
### Sync .env ke submodule
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/2updateenv.sh
|
||||||
|
# Copy .env root ke apps/*/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Linting & Formatting
|
||||||
|
|
||||||
|
Root repo menggunakan **Biome** untuk linting dan formatting:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bun run check # Lint + format + write
|
||||||
|
bun run ci # CI mode (no write, exit code on issues)
|
||||||
|
bun run lint # Lint only
|
||||||
|
bun run format # Format only
|
||||||
|
```
|
||||||
|
|
||||||
|
## Build Docker Image Lokal
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Scraper API
|
||||||
|
docker build -f infra/docker/scraper.Dockerfile -t scraper-api:local .
|
||||||
|
|
||||||
|
# Service baru: tambahkan Dockerfile di infra/docker/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Saat ini belum ada test runner di root level. Masing-masing submodule mengelola testing sendiri:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Scraper API (Rust)
|
||||||
|
cd apps/scraper && cargo test
|
||||||
|
```
|
||||||
|
|
||||||
|
## Validasi YAML
|
||||||
|
|
||||||
|
Sebelum commit perubahan infra, validasi semua file YAML:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -c "
|
||||||
|
import pathlib, yaml
|
||||||
|
for p in pathlib.Path('infra').rglob('*.yml'):
|
||||||
|
with open(p) as f: yaml.safe_load(f)
|
||||||
|
print(f'OK {p}')
|
||||||
|
for p in pathlib.Path('infra').rglob('*.yaml'):
|
||||||
|
with open(p) as f: yaml.safe_load(f)
|
||||||
|
print(f'OK {p}')
|
||||||
|
"
|
||||||
|
|
||||||
|
for f in infra/compose/*.yml; do
|
||||||
|
docker compose -f "$f" config >/dev/null && echo "OK $f"
|
||||||
|
done
|
||||||
|
```
|
||||||
|
|
||||||
|
## Git Workflow
|
||||||
|
|
||||||
|
### Commit Convention
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <description>
|
||||||
|
```
|
||||||
|
|
||||||
|
Type: `feat`, `fix`, `chore`, `docs`, `refactor`, `test`, `ci`, `perf`, `style`
|
||||||
|
Scope: `scraper`, `infra`, `ci`, `dapr`, `nats`, `docs`, `deps`, `scripts`, `root`
|
||||||
|
|
||||||
|
Contoh:
|
||||||
|
```
|
||||||
|
feat(scraper): add image cache endpoint
|
||||||
|
fix(infra): correct Traefik rate-limit config
|
||||||
|
chore(deps): bump biome to 2.5.0
|
||||||
|
```
|
||||||
|
|
||||||
|
### Branch Strategy
|
||||||
|
|
||||||
|
- `main` — production branch, push triggers CI/CD
|
||||||
|
- Fitur baru: branch dari `main`, PR ke `main`
|
||||||
|
- Submodule development: dilakukan di repo masing-masing, hub hanya update pointer
|
||||||
|
|
||||||
|
## Deployment ke VPS
|
||||||
|
|
||||||
|
Push ke `main` otomatis trigger CI/CD. Untuk trigger manual:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
gh workflow run deploy-docker.yml
|
||||||
|
```
|
||||||
|
|
||||||
|
Lihat `docs/DEPLOYMENT.md` untuk detail.
|
||||||
@@ -0,0 +1,234 @@
|
|||||||
|
# Backup & Disaster Recovery
|
||||||
|
|
||||||
|
## Aset yang Perlu di-Backup
|
||||||
|
|
||||||
|
| Aset | Lokasi | Frekuensi | Metode |
|
||||||
|
|------|--------|-----------|--------|
|
||||||
|
| Database PostgreSQL | `imrnes` (100.121.180.82:6432) | Harian | `pg_dump` |
|
||||||
|
| Volume Redis | `orangevps` (Docker volume) | Opsional | Redis RDB / AOF |
|
||||||
|
| Volume NATS JetStream | `orangevps` (Docker volume) | Opsional | File copy |
|
||||||
|
| Docker Compose manifests | GitHub (hub repo) | Real-time | Git |
|
||||||
|
| Environment variables | GitHub secret `ENV_FILE_PRODUCTION` | Manual | `gh secret set` |
|
||||||
|
| TLS certificates | `orangevps` (`/root/*.pem`, `*.key`) | Saat renew | SCP |
|
||||||
|
| Tailscale auth | Tailscale admin console | - | Cloud-managed |
|
||||||
|
| GitHub Actions secrets | GitHub UI | Manual | Backup list |
|
||||||
|
|
||||||
|
## Database PostgreSQL (Prioritas Tertinggi)
|
||||||
|
|
||||||
|
### Backup Manual
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dari orangevps (via Tailscale)
|
||||||
|
pg_dump -h 100.121.180.82 -p 6432 -U asephs -d hub \
|
||||||
|
--no-owner --no-acl \
|
||||||
|
-F c -f /root/db-backups/hub-$(date +%Y%m%d-%H%M%S).dump
|
||||||
|
|
||||||
|
# Atau dari imrnes langsung
|
||||||
|
pg_dump -U asephs -d hub \
|
||||||
|
-F c -f /backup/hub/hub-$(date +%Y%m%d-%H%M%S).dump
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restore
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Drop dan recreate database
|
||||||
|
dropdb -h 100.121.180.82 -p 6432 -U asephs hub
|
||||||
|
createdb -h 100.121.180.82 -p 6432 -U asephs hub
|
||||||
|
|
||||||
|
# Restore dari dump
|
||||||
|
pg_restore -h 100.121.180.82 -p 6432 -U asephs -d hub \
|
||||||
|
--no-owner --no-acl \
|
||||||
|
/path/to/backup/hub-20260101-120000.dump
|
||||||
|
```
|
||||||
|
|
||||||
|
### Backup Otomatis (via Cron di imrnes)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# /etc/cron.d/hub-db-backup
|
||||||
|
0 2 * * * root pg_dump -U asephs -d hub -F c -f /backup/hub/hub-$(date +\%Y\%m\%d).dump && find /backup/hub -name "hub-*.dump" -mtime +30 -delete
|
||||||
|
```
|
||||||
|
|
||||||
|
## Volume Docker
|
||||||
|
|
||||||
|
### Redis
|
||||||
|
|
||||||
|
Redis data bisa di-recover dari NATS events (event sourcing). Jika tidak ada persistence requirement, cukup restart:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker volume rm redis_data
|
||||||
|
docker compose -f infra/compose/shared.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Jika perlu backup:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Save RDB snapshot
|
||||||
|
docker exec redis redis-cli SAVE
|
||||||
|
|
||||||
|
# Copy dari volume
|
||||||
|
docker run --rm -v redis_data:/data -v /backup:/backup alpine cp /data/dump.rdb /backup/redis-$(date +%Y%m%d).rdb
|
||||||
|
```
|
||||||
|
|
||||||
|
### NATS JetStream
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Backup volume
|
||||||
|
docker run --rm -v nats_data:/data -v /backup:/backup alpine \
|
||||||
|
tar czf /backup/nats-$(date +%Y%m%d).tar.gz -C /data .
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
### Backup `.env` dari VPS
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Simpan current .env dari VPS
|
||||||
|
ssh root@45.127.35.244 "cat /root/asepharyana-hub/.env" > .env.backup.$(date +%Y%m%d)
|
||||||
|
|
||||||
|
# Update GitHub secret
|
||||||
|
cat .env.backup.$(date +%Y%m%d) | gh secret set ENV_FILE_PRODUCTION --repo asepharyana/asepharyana-hub
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restore `.env` jika hilang
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Buat .env baru dari template
|
||||||
|
cp .env.example .env
|
||||||
|
|
||||||
|
# Edit secrets (manual dari password manager atau GitHub secret)
|
||||||
|
# Atau download dari GitHub secret
|
||||||
|
gh secret list --repo asepharyana/asepharyana-hub
|
||||||
|
```
|
||||||
|
|
||||||
|
## TLS Certificates
|
||||||
|
|
||||||
|
### Backup
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Di orangevps
|
||||||
|
tar czf /root/cert-backup-$(date +%Y%m%d).tar.gz \
|
||||||
|
/root/asepharyana.my.id.pem \
|
||||||
|
/root/asepharyana.my.id.key \
|
||||||
|
/root/asepharyana.web.id.pem \
|
||||||
|
/root/asepharyana.web.id.key \
|
||||||
|
/root/asepharyana-hub/infra/traefik/dynamic/ssl.yaml
|
||||||
|
|
||||||
|
# SCP ke local
|
||||||
|
scp root@45.127.35.244:/root/cert-backup-*.tar.gz .
|
||||||
|
```
|
||||||
|
|
||||||
|
### Restore
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# SCP ke VPS
|
||||||
|
scp cert-backup-20260101.tar.gz root@45.127.35.244:/root/
|
||||||
|
|
||||||
|
# Extract
|
||||||
|
ssh root@45.127.35.244 "tar xzf /root/cert-backup-20260101.tar.gz -C / && docker restart traefik"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Disaster Recovery Scenarios
|
||||||
|
|
||||||
|
### Skenario 1: VPS (orangevps) mati total
|
||||||
|
|
||||||
|
**Dampak:** Semua service down.
|
||||||
|
|
||||||
|
**Recovery:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Provision VPS baru (atau restore dari snapshot)
|
||||||
|
# 2. Install Docker + Tailscale
|
||||||
|
# 3. Clone repo
|
||||||
|
git clone https://github.com/asepharyana/asepharyana-hub.git /root/asepharyana-hub
|
||||||
|
|
||||||
|
# 4. Setup Tailscale, route service
|
||||||
|
# 5. Restore .env
|
||||||
|
echo "<ENV_FILE_PRODUCTION>" > /root/asepharyana-hub/.env
|
||||||
|
|
||||||
|
# 6. Restore TLS certs
|
||||||
|
# 7. Create network
|
||||||
|
docker network create app-shared-net
|
||||||
|
|
||||||
|
# 8. Start services sesuai urutan
|
||||||
|
cd /root/asepharyana-hub
|
||||||
|
for f in shared.yml nats.yml dapr.yml traefik.yml scraper.yml; do
|
||||||
|
docker compose -f infra/compose/$f --env-file .env up -d
|
||||||
|
done
|
||||||
|
|
||||||
|
# 9. Update DNS jika IP baru
|
||||||
|
```
|
||||||
|
|
||||||
|
### Skenario 2: Database (imrnes) mati total
|
||||||
|
|
||||||
|
**Dampak:** Semua service yang butuh database error.
|
||||||
|
|
||||||
|
**Recovery:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Fix imrnes atau provision server baru
|
||||||
|
# 2. Setup PostgreSQL
|
||||||
|
# 3. Restore dari backup terakhir
|
||||||
|
# 4. Update Tailscale IP jika perlu
|
||||||
|
# 5. Update .env dan GitHub secret
|
||||||
|
# 6. Redeploy
|
||||||
|
```
|
||||||
|
|
||||||
|
### Skenario 3: GitHub repository hilang
|
||||||
|
|
||||||
|
**Dampak:** Kehilangan CI/CD, tapi Docker images masih ada di GHCR.
|
||||||
|
|
||||||
|
**Recovery:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Create repo baru di GitHub
|
||||||
|
# 2. Push dari local clone
|
||||||
|
git remote add origin-new https://github.com/asepharyana/asepharyana-hub-new.git
|
||||||
|
git push origin-new main
|
||||||
|
|
||||||
|
# 3. Re-create GitHub secrets
|
||||||
|
# 4. Re-create workflows
|
||||||
|
# 5. Update VPS remote
|
||||||
|
ssh root@45.127.35.244 "cd /root/asepharyana-hub && git remote set-url origin https://github.com/asepharyana/asepharyana-hub-new.git"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Skenario 4: GHCR registry tidak bisa diakses
|
||||||
|
|
||||||
|
**Dampak:** Tidak bisa pull image.
|
||||||
|
|
||||||
|
**Recovery:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Build image langsung di VPS
|
||||||
|
docker build -f infra/docker/scraper.Dockerfile -t ghcr.io/asepharyana/asepharyana-hub/scraper-api:local .
|
||||||
|
|
||||||
|
# 2. Update compose file untuk sementara
|
||||||
|
sed -i 's|image: ghcr.io/.*|image: ghcr.io/asepharyana/asepharyana-hub/scraper-api:local|' infra/compose/scraper.yml
|
||||||
|
|
||||||
|
# 3. Start
|
||||||
|
docker compose -f infra/compose/scraper.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Skenario 5: Semua server mati (total loss)
|
||||||
|
|
||||||
|
**Recovery:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Provision VPS baru
|
||||||
|
# 2. Provision server database baru
|
||||||
|
# 3. Setup Tailscale
|
||||||
|
# 4. Clone repo, restore .env, certs
|
||||||
|
# 5. Restore database dari backup (jika ada)
|
||||||
|
# 6. Jika tidak ada backup database:
|
||||||
|
# - Build image dari GHCR
|
||||||
|
# - Start service dengan database kosong
|
||||||
|
# - Data akan terisi ulang dari scraping
|
||||||
|
```
|
||||||
|
|
||||||
|
## Checklist Pencegahan
|
||||||
|
|
||||||
|
- [ ] Cron job backup database berjalan
|
||||||
|
- [ ] Backup `.env` disimpan di luar VPS (password manager)
|
||||||
|
- [ ] TLS certificates backup disimpan di luar VPS
|
||||||
|
- [ ] GitHub secrets terdaftar (tidak hanya diingat)
|
||||||
|
- [ ] Docker images bisa di-rebuild dari CI (GHCR sebagai source of truth)
|
||||||
|
- [ ] Tailscale admin access via multiple accounts
|
||||||
@@ -0,0 +1,248 @@
|
|||||||
|
# CI/CD Pipeline
|
||||||
|
|
||||||
|
Dokumentasi pipeline CI/CD untuk `asepharyana-hub`. Terdiri dari 5 GitHub Actions workflow yang saling terhubung.
|
||||||
|
|
||||||
|
## Workflow Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
┌─────────────┐
|
||||||
|
│ Lint │ (PR/push → Biome)
|
||||||
|
└──────┬──────┘
|
||||||
|
│
|
||||||
|
Push ke main ─────┼────── repository_dispatch
|
||||||
|
│
|
||||||
|
┌──────▼──────────────────┐
|
||||||
|
│ docker-build-push.yml │
|
||||||
|
│ │
|
||||||
|
│ Phase 1: Detect │
|
||||||
|
│ Phase 2: Build & Push │
|
||||||
|
│ Phase 3: Update │
|
||||||
|
│ manifests │
|
||||||
|
└──────┬──────────────────┘
|
||||||
|
│ workflow_run
|
||||||
|
┌──────▼──────────────┐
|
||||||
|
│ deploy-docker.yml │
|
||||||
|
│ SSH → VPS │
|
||||||
|
│ Pull → Restart │
|
||||||
|
└─────────────────────┘
|
||||||
|
|
||||||
|
repository_dispatch ──► update-submodule.yml
|
||||||
|
(dari submodule) (update pointer → commit)
|
||||||
|
│
|
||||||
|
▼
|
||||||
|
docker-build-push.yml
|
||||||
|
(triggered by push)
|
||||||
|
```
|
||||||
|
|
||||||
|
## Workflow Detail
|
||||||
|
|
||||||
|
### 1. Lint (`lint.yml`)
|
||||||
|
|
||||||
|
**Trigger:** PR/push ke `main` yang mengubah `*.json`, `*.js`, `biome.json`
|
||||||
|
|
||||||
|
**Aksi:**
|
||||||
|
- Checkout repo dengan submodules
|
||||||
|
- Setup Bun
|
||||||
|
- `bun install --frozen-lockfile`
|
||||||
|
- `bun run ci` (Biome CI mode)
|
||||||
|
|
||||||
|
**Permissions:** read-only
|
||||||
|
|
||||||
|
### 2. Build and Push Docker Images (`docker-build-push.yml`)
|
||||||
|
|
||||||
|
**Trigger:**
|
||||||
|
- Push ke `main` yang mengubah `apps/**`, `infra/**`, atau file workflow
|
||||||
|
- `repository_dispatch` tipe `submodule-updated`
|
||||||
|
- `workflow_dispatch` (manual)
|
||||||
|
|
||||||
|
**Concurrency:** Satu workflow per branch (cancel-in-progress=false)
|
||||||
|
|
||||||
|
#### Phase 1: Detect Changes
|
||||||
|
|
||||||
|
Job `changes` mendeteksi service mana yang perlu di-build:
|
||||||
|
|
||||||
|
- **Push event:** `git diff --name-only` antara `before` dan `after` SHA
|
||||||
|
- **repository_dispatch:** Parse payload `{service, sha}` dan validasi
|
||||||
|
- **workflow_dispatch:** Build semua service
|
||||||
|
|
||||||
|
Output format matrix:
|
||||||
|
```json
|
||||||
|
[{"id":"scraper-api","target":"docker-scraper","path":"apps/scraper"}]
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Phase 2: Build & Push (Matrix)
|
||||||
|
|
||||||
|
Job `build` berjalan paralel per service (matrix strategy):
|
||||||
|
|
||||||
|
1. Checkout repo + sync submodule
|
||||||
|
2. Jika `repository_dispatch`, checkout submodule ke SHA tertentu
|
||||||
|
3. Login ke GHCR
|
||||||
|
4. Setup Docker Buildx
|
||||||
|
5. Build & push dengan tag:
|
||||||
|
- `ghcr.io/asepharyana/asepharyana-hub/<service>:latest`
|
||||||
|
- `ghcr.io/asepharyana/asepharyana-hub/<service>:sha-<shortsha>`
|
||||||
|
6. Build cache: registry-based (`:<service>:buildcache`)
|
||||||
|
|
||||||
|
#### Phase 3: Update Manifests
|
||||||
|
|
||||||
|
Job `update-manifest`:
|
||||||
|
|
||||||
|
1. Update image tag di compose file (`infra/compose/<service>.yml`)
|
||||||
|
2. Jika `repository_dispatch`, update submodule pointer
|
||||||
|
3. Commit dengan message `chore: update manifests and submodules [skip ci]`
|
||||||
|
4. Push dengan retry (3 attempts, rebase jika conflict)
|
||||||
|
|
||||||
|
### 3. Deploy Docker to VPS (`deploy-docker.yml`)
|
||||||
|
|
||||||
|
**Trigger:**
|
||||||
|
- `workflow_run` setelah `docker-build-push.yml` selesai
|
||||||
|
- Push ke `main` yang mengubah `infra/**`
|
||||||
|
- `workflow_dispatch` (manual)
|
||||||
|
|
||||||
|
**Concurrency:** Satu deployment dalam satu waktu (`group: deploy-vps`)
|
||||||
|
|
||||||
|
**Aksi di VPS (via SSH):**
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Setup SSH multiplexing
|
||||||
|
2. SCP .env dari GitHub secret ke VPS
|
||||||
|
3. Docker login ke GHCR
|
||||||
|
4. Git sync (fetch + reset --hard)
|
||||||
|
5. Detect changed files:
|
||||||
|
├─ Compose stack changes → selective container update
|
||||||
|
├─ Traefik dynamic config → SIGHUP
|
||||||
|
└─ Other infra → full deploy
|
||||||
|
6. Pull images (retry 3x)
|
||||||
|
7. Remove stale containers
|
||||||
|
8. Up services
|
||||||
|
9. SIGHUP Traefik jika perlu
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Security Scan (`security.yml`)
|
||||||
|
|
||||||
|
**Trigger:**
|
||||||
|
- PR ke `main`
|
||||||
|
- Jadwal: Setiap Senin (`0 6 * * 1`)
|
||||||
|
|
||||||
|
**Aksi:**
|
||||||
|
- Checkout dengan fetch-depth 2
|
||||||
|
- CodeQL init untuk Rust
|
||||||
|
- `cargo build` di `apps/scraper`
|
||||||
|
- CodeQL analyze
|
||||||
|
|
||||||
|
### 5. Update Submodule Pointer (`update-submodule.yml`)
|
||||||
|
|
||||||
|
**Trigger:** `repository_dispatch` tipe `submodule-updated`
|
||||||
|
|
||||||
|
**Aksi:**
|
||||||
|
1. Validasi payload (`service`, `sha`)
|
||||||
|
2. Map service ke submodule path (e.g., `scraper-api` → `apps/scraper`)
|
||||||
|
3. Update submodule ke SHA yang diberikan
|
||||||
|
4. Commit sebagai `monrepo-bot` dengan message:
|
||||||
|
`chore: update <service> to <shortsha>`
|
||||||
|
5. Push dengan retry (3 attempts)
|
||||||
|
|
||||||
|
## Flow Submodule Update
|
||||||
|
|
||||||
|
Flow lengkap ketika code berubah di submodule repo:
|
||||||
|
|
||||||
|
```
|
||||||
|
1. Developer push ke asepharyana-hub-scraper
|
||||||
|
2. GitHub Action di scraper repo kirim repository_dispatch
|
||||||
|
ke asepharyana-hub
|
||||||
|
3. update-submodule.yml terima dispatch, update pointer
|
||||||
|
4. Commit masuk ke hub repo main
|
||||||
|
5. Commit ini trigger docker-build-push.yml
|
||||||
|
(push ke main dengan path apps/scraper/**)
|
||||||
|
6. Build image baru, update compose file
|
||||||
|
7. Deploy ke VPS
|
||||||
|
```
|
||||||
|
|
||||||
|
## Secrets yang Diperlukan
|
||||||
|
|
||||||
|
| Secret | Workflow | Deskripsi |
|
||||||
|
|--------|----------|-----------|
|
||||||
|
| `SSH_PRIVATE_KEY` | deploy-docker | SSH key untuk akses VPS |
|
||||||
|
| `VPS_HOST` | deploy-docker | IP VPS (`45.127.35.244`) |
|
||||||
|
| `VPS_USER` | deploy-docker | User SSH (`root`) |
|
||||||
|
| `VPS_TARGET_DIR` | deploy-docker | Dir di VPS (`/root/asepharyana-hub`) |
|
||||||
|
| `ENV_FILE_PRODUCTION` | deploy-docker | Full `.env` production |
|
||||||
|
|
||||||
|
## Menambahkan Service Baru ke Pipeline
|
||||||
|
|
||||||
|
Untuk menambahkan service baru, update:
|
||||||
|
|
||||||
|
### `docker-build-push.yml`
|
||||||
|
|
||||||
|
1. **Phase 1 — `changes` job:** Tambah detection logic untuk service baru:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
echo "new-service=$(changed '^(apps/new-service(/|$)|\.github/workflows/docker-build-push\.yml$|infra/docker/new-service\.Dockerfile$)')" >> "$GITHUB_OUTPUT"
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Phase 1 — `repository_dispatch`:** Tambah case:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
case "$SERVICE" in
|
||||||
|
scraper-api|new-service) ;;
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Phase 1 — `set-matrix`:** Tambah service:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
if [ "${{ ...['new-service'] == 'true' ... }}" == "true" ]; then add_service "new-service" "docker-new-service" "apps/new-service"; fi
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Phase 2 — `meta` step:** Tambah mapping Dockerfile:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
"new-service") echo "dockerfile=infra/docker/new-service.Dockerfile" >> $GITHUB_OUTPUT ;;
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Phase 3 — `update-manifest`:** Tambah mapping:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
SERVICES["new-service"]="new-service.yml"
|
||||||
|
PATHS["new-service"]="apps/new-service"
|
||||||
|
```
|
||||||
|
|
||||||
|
### `deploy-docker.yml`
|
||||||
|
|
||||||
|
Tambah compose file ke `ALL_COMPOSE_FILES`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
ALL_COMPOSE_FILES="infra/compose/traefik.yml infra/compose/shared.yml infra/compose/scraper.yml infra/compose/nats.yml infra/compose/dapr.yml infra/compose/new-service.yml"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Rollback
|
||||||
|
|
||||||
|
### Rollback Image
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cari SHA tag sebelumnya di GHCR packages
|
||||||
|
# Update compose file ke tag tersebut
|
||||||
|
sed -i 's|sha-badcommit|sha-goodcommit|g' infra/compose/scraper.yml
|
||||||
|
git commit -am "fix: rollback scraper-api to sha-goodcommit"
|
||||||
|
git push
|
||||||
|
```
|
||||||
|
|
||||||
|
### Rollback via Git Revert
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git revert HEAD
|
||||||
|
git push origin main
|
||||||
|
# Pipeline otomatis build dan deploy
|
||||||
|
```
|
||||||
|
|
||||||
|
## Monitoring Pipeline
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek status workflow terbaru
|
||||||
|
gh run list --limit 5
|
||||||
|
|
||||||
|
# Lihat log workflow tertentu
|
||||||
|
gh run view <run-id> --log
|
||||||
|
|
||||||
|
# Trigger workflow manual
|
||||||
|
gh workflow run deploy-docker.yml
|
||||||
|
```
|
||||||
@@ -0,0 +1,286 @@
|
|||||||
|
# NATS + JetStream Guide
|
||||||
|
|
||||||
|
Dokumentasi konfigurasi, penggunaan, dan troubleshooting NATS di infrastruktur `asepharyana-hub`.
|
||||||
|
|
||||||
|
## Arsitektur
|
||||||
|
|
||||||
|
NATS berjalan di container `nats` dengan JetStream diaktifkan (`-js`). Data persistent disimpan di volume Docker `nats_data`.
|
||||||
|
|
||||||
|
```
|
||||||
|
Service ──► NATS (port 4222) ──► JetStream (disk)
|
||||||
|
│
|
||||||
|
├─ Monitoring HTTP: port 8222
|
||||||
|
└─ Client connections: port 4222
|
||||||
|
```
|
||||||
|
|
||||||
|
### Hubungan dengan Dapr
|
||||||
|
|
||||||
|
Saat ini Dapr pub/sub menggunakan **Redis** (`pubsub.redis`), bukan NATS. NATS berfungsi sebagai message broker independen untuk:
|
||||||
|
|
||||||
|
- Event streaming antar service
|
||||||
|
- Persistent job queues
|
||||||
|
- Pub/sub untuk service yang tidak menggunakan Dapr
|
||||||
|
|
||||||
|
Jika ingin Dapr menggunakan NATS sebagai backend pub/sub, ganti komponen `pubsub.yaml`:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: pubsub
|
||||||
|
spec:
|
||||||
|
type: pubsub.nats
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: natsURL
|
||||||
|
value: nats://nats:4222
|
||||||
|
```
|
||||||
|
|
||||||
|
## Konfigurasi Compose
|
||||||
|
|
||||||
|
File: `infra/compose/nats.yml`
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
nats:
|
||||||
|
container_name: nats
|
||||||
|
image: nats:latest
|
||||||
|
restart: always
|
||||||
|
networks:
|
||||||
|
app-shared-net:
|
||||||
|
aliases:
|
||||||
|
- nats
|
||||||
|
ports:
|
||||||
|
- '4222:4222' # client connections
|
||||||
|
- '8222:8222' # HTTP monitor
|
||||||
|
command:
|
||||||
|
- '-js' # enable JetStream
|
||||||
|
- '-sd'
|
||||||
|
- '/data' # storage directory
|
||||||
|
volumes:
|
||||||
|
- nats_data:/data
|
||||||
|
```
|
||||||
|
|
||||||
|
## CLI Tools
|
||||||
|
|
||||||
|
### Install NATS CLI
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Linux
|
||||||
|
curl -sf https://bin.nats.dev/nats | sh
|
||||||
|
sudo mv nats /usr/local/bin/
|
||||||
|
|
||||||
|
# Atau via package manager
|
||||||
|
# brew install nats-io/nats-tools/nats (macOS)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Koneksi ke NATS
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dari host (port 4222 ter-expose)
|
||||||
|
nats context save hub --server nats://localhost:4222 --description "Hub Production"
|
||||||
|
nats context select hub
|
||||||
|
|
||||||
|
# Test koneksi
|
||||||
|
nats server check
|
||||||
|
nats server info
|
||||||
|
```
|
||||||
|
|
||||||
|
### Manage Streams (JetStream)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# List semua stream
|
||||||
|
nats stream list
|
||||||
|
|
||||||
|
# Lihat detail stream
|
||||||
|
nats stream info <stream-name>
|
||||||
|
|
||||||
|
# Buat stream
|
||||||
|
nats stream add <stream-name> \
|
||||||
|
--subjects "hub.>" \
|
||||||
|
--storage file \
|
||||||
|
--max-msgs 1000000 \
|
||||||
|
--max-bytes 1G \
|
||||||
|
--retention limits
|
||||||
|
|
||||||
|
# Hapus stream
|
||||||
|
nats stream rm <stream-name>
|
||||||
|
|
||||||
|
# Purge (hapus semua message, retain stream)
|
||||||
|
nats stream purge <stream-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
### Pub/Sub
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Subscribe ke subject
|
||||||
|
nats sub "hub.>"
|
||||||
|
nats sub "hub.image.cached"
|
||||||
|
|
||||||
|
# Publish message
|
||||||
|
nats pub "hub.test" '{"message": "hello"}'
|
||||||
|
nats pub "hub.image.cached" '{"original_url": "https://example.com/img.jpg", "cdn_url": "https://cdn.example.com/img.jpg"}'
|
||||||
|
|
||||||
|
# Request-reply
|
||||||
|
nats request "hub.service.do" '{"task": "process"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
### Monitoring via HTTP API
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Server info
|
||||||
|
curl http://localhost:8222/
|
||||||
|
|
||||||
|
# JetStream info
|
||||||
|
curl http://localhost:8222/jszetstream
|
||||||
|
|
||||||
|
# Stream detail
|
||||||
|
curl http://localhost:8222/jszetstream?stream=<stream-name>
|
||||||
|
|
||||||
|
# Consumer info
|
||||||
|
curl http://localhost:8222/jszetstream?stream=<stream-name>&consumer=<consumer-name>
|
||||||
|
|
||||||
|
# Server stats
|
||||||
|
curl http://localhost:8222/varz
|
||||||
|
|
||||||
|
# Connections
|
||||||
|
curl http://localhost:8222/connz
|
||||||
|
```
|
||||||
|
|
||||||
|
## Event Topics Convention
|
||||||
|
|
||||||
|
Semua topik menggunakan prefix `hub.`:
|
||||||
|
|
||||||
|
| Subject | Payload | Deskripsi |
|
||||||
|
|---------|---------|-----------|
|
||||||
|
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
|
||||||
|
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
|
||||||
|
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
|
||||||
|
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |
|
||||||
|
| `hub.test` | Any | Testing |
|
||||||
|
|
||||||
|
### Wildcard Subjects
|
||||||
|
|
||||||
|
NATS mendukung wildcard:
|
||||||
|
|
||||||
|
- `hub.>` — semua event hub (multi-level)
|
||||||
|
- `hub.image.*` — semua event image (single-level)
|
||||||
|
- `hub.*.done` — semua event yang selesai (single-level)
|
||||||
|
|
||||||
|
## JetStream Configuration
|
||||||
|
|
||||||
|
### Storage
|
||||||
|
|
||||||
|
Data JetStream disimpan di volume Docker `nats_data`.
|
||||||
|
|
||||||
|
Lokasi di VPS:
|
||||||
|
```bash
|
||||||
|
docker volume inspect nats_data
|
||||||
|
# atau
|
||||||
|
ls -la /var/lib/docker/volumes/nats_data/_data/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Memory & Limits
|
||||||
|
|
||||||
|
NATS tidak memiliki konfigurasi limit memori default. Untuk production, pertimbangkan:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
command:
|
||||||
|
- '-js'
|
||||||
|
- '-sd'
|
||||||
|
- '/data'
|
||||||
|
- '--max_pending_size=64MB'
|
||||||
|
- '--max_payload=1MB'
|
||||||
|
```
|
||||||
|
|
||||||
|
Atau gunakan NATS configuration file:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# nats-server.conf
|
||||||
|
jetstream:
|
||||||
|
max_memory_store: 256MB
|
||||||
|
max_file_store: 10GB
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Stream data tidak muncul
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Cek koneksi NATS
|
||||||
|
nats server check
|
||||||
|
|
||||||
|
# 2. Cek apakah JetStream aktif
|
||||||
|
curl http://localhost:8222/jszetstream
|
||||||
|
|
||||||
|
# 3. Cek stream dan message count
|
||||||
|
nats stream list
|
||||||
|
|
||||||
|
# 4. Subscribe langsung untuk test
|
||||||
|
nats sub ">"
|
||||||
|
```
|
||||||
|
|
||||||
|
### NATS tidak bisa start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek log
|
||||||
|
docker logs nats
|
||||||
|
|
||||||
|
# Cek apakah port 4222 sudah dipakai
|
||||||
|
ss -tlnp | grep 4222
|
||||||
|
|
||||||
|
# Cek volume data korup
|
||||||
|
docker run --rm -v nats_data:/data alpine ls -la /data
|
||||||
|
|
||||||
|
# Restart
|
||||||
|
docker compose -f infra/compose/nats.yml up -d --force-recreate
|
||||||
|
```
|
||||||
|
|
||||||
|
### Disk JetStream penuh
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek ukuran volume
|
||||||
|
docker system df -v | grep nats_data
|
||||||
|
|
||||||
|
# Purge stream jika perlu
|
||||||
|
nats stream purge <stream-name>
|
||||||
|
|
||||||
|
# Atau hapus volume (data hilang!)
|
||||||
|
docker compose -f infra/compose/nats.yml down
|
||||||
|
docker volume rm nats_data
|
||||||
|
docker compose -f infra/compose/nats.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### Slow consumer
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek consumer lag
|
||||||
|
nats stream info <stream-name>
|
||||||
|
# Lihat fields: "Pending" dan "Acknowledgment"
|
||||||
|
|
||||||
|
# Lihat stats server
|
||||||
|
curl http://localhost:8222/varz | jq '.slow_consumers'
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration: Redis Pub/Sub ke NATS
|
||||||
|
|
||||||
|
Jika ingin migrasi dari Dapr pub/sub Redis ke NATS:
|
||||||
|
|
||||||
|
1. Buat stream NATS untuk topik `hub.>`
|
||||||
|
2. Update `infra/dapr/components/pubsub.yaml` dari `pubsub.redis` ke `pubsub.nats`
|
||||||
|
3. Deploy ulang semua service (Dapr sidecar akan reconnect)
|
||||||
|
4. Verifikasi event flow
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# infra/dapr/components/pubsub.yaml (setelah migrasi)
|
||||||
|
apiVersion: dapr.io/v1alpha1
|
||||||
|
kind: Component
|
||||||
|
metadata:
|
||||||
|
name: pubsub
|
||||||
|
spec:
|
||||||
|
type: pubsub.nats
|
||||||
|
version: v1
|
||||||
|
metadata:
|
||||||
|
- name: natsURL
|
||||||
|
value: nats://nats:4222
|
||||||
|
```
|
||||||
@@ -0,0 +1,211 @@
|
|||||||
|
# Security Guide
|
||||||
|
|
||||||
|
Praktik keamanan untuk infrastruktur `asepharyana-hub`.
|
||||||
|
|
||||||
|
## Ringkasan
|
||||||
|
|
||||||
|
| Area | Status | Prioritas |
|
||||||
|
|------|--------|-----------|
|
||||||
|
| Secrets management | GitHub encrypted secrets | Tinggi |
|
||||||
|
| TLS termination | Traefik + cert volume mounts | Tinggi |
|
||||||
|
| Container security | Non-root user (scraper-api) | Sedang |
|
||||||
|
| Network security | Tailscale overlay, app-shared-net | Sedang |
|
||||||
|
| Access control | SSH key, GitHub permissions | Sedang |
|
||||||
|
| Monitoring | Belum ada alert system | Rendah |
|
||||||
|
| Firewall | UFW/iptables (manual) | Sedang |
|
||||||
|
| Backup | lihat `docs/backup-recovery.md` | Sedang |
|
||||||
|
|
||||||
|
## Secrets Management
|
||||||
|
|
||||||
|
### Yang Tidak Boleh di-Commit
|
||||||
|
|
||||||
|
- [ ] `.env` production (disimpan sebagai GitHub secret `ENV_FILE_PRODUCTION`)
|
||||||
|
- [ ] SSH private keys
|
||||||
|
- [ ] API tokens, JWT secret
|
||||||
|
- [ ] Docker registry tokens
|
||||||
|
- [ ] Database passwords
|
||||||
|
- [ ] TLS certificate private keys
|
||||||
|
|
||||||
|
### GitHub Secrets
|
||||||
|
|
||||||
|
Setting di Settings > Secrets and variables > Actions:
|
||||||
|
|
||||||
|
| Secret | Tujuan | Rotasi |
|
||||||
|
|--------|--------|--------|
|
||||||
|
| `SSH_PRIVATE_KEY` | Akses SSH ke VPS | 6 bulan |
|
||||||
|
| `VPS_HOST` | IP VPS | Tidak berubah |
|
||||||
|
| `VPS_USER` | User SSH | Tidak berubah |
|
||||||
|
| `VPS_TARGET_DIR` | Directory di VPS | Tidak berubah |
|
||||||
|
| `ENV_FILE_PRODUCTION` | Full `.env` production | Saat ada perubahan |
|
||||||
|
|
||||||
|
### Update Secrets dengan aman
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Baca current .env dari VPS via SSH
|
||||||
|
ssh root@45.127.35.244 "cat /root/asepharyana-hub/.env" | gh secret set ENV_FILE_PRODUCTION --repo asepharyana/asepharyana-hub --repos
|
||||||
|
```
|
||||||
|
|
||||||
|
### Production `.env` tidak boleh di-commit
|
||||||
|
|
||||||
|
`.env` di root repo adalah untuk development lokal. Production `.env` hanya ada di:
|
||||||
|
1. GitHub secret `ENV_FILE_PRODUCTION`
|
||||||
|
2. File `/root/asepharyana-hub/.env` di VPS (hasil SCP dari CI/CD)
|
||||||
|
|
||||||
|
## TLS / SSL
|
||||||
|
|
||||||
|
### Konfigurasi
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
# Traefik TLS certs dari file mount (bukan auto-ACME)
|
||||||
|
volumes:
|
||||||
|
- ${TRAEFIK_CERT_MY_ID_PEM:-/root/asepharyana.my.id.pem}:/etc/traefik/certs/asepharyana.my.id.pem:ro
|
||||||
|
- ${TRAEFIK_CERT_MY_ID_KEY:-/root/asepharyana.my.id.key}:/etc/traefik/certs/asepharyana.my.id.key:ro
|
||||||
|
```
|
||||||
|
|
||||||
|
### Best Practices
|
||||||
|
|
||||||
|
- Certificates disimpan di host (`/root/`), bukan di repo
|
||||||
|
- Volume mount read-only (`:ro`)
|
||||||
|
- Private key hanya bisa dibaca oleh root (chmod 600)
|
||||||
|
- Renew certificates sebelum expired (monitor expiry)
|
||||||
|
- Dua domain: `asepharyana.my.id` + `asepharyana.web.id`
|
||||||
|
|
||||||
|
## Container Security
|
||||||
|
|
||||||
|
### Non-Root User
|
||||||
|
|
||||||
|
Scraper API berjalan sebagai `appuser` (UID 1001):
|
||||||
|
|
||||||
|
```dockerfile
|
||||||
|
RUN groupadd -g 1001 appgroup && \
|
||||||
|
useradd -u 1001 -g appgroup -s /bin/sh appuser
|
||||||
|
USER appuser
|
||||||
|
```
|
||||||
|
|
||||||
|
Service baru harus mengikuti pattern yang sama.
|
||||||
|
|
||||||
|
### Read-Only Filesystem
|
||||||
|
|
||||||
|
Untuk container yang tidak perlu write ke filesystem:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
image: app:latest
|
||||||
|
read_only: true
|
||||||
|
tmpfs:
|
||||||
|
- /tmp
|
||||||
|
```
|
||||||
|
|
||||||
|
### Docker Socket
|
||||||
|
|
||||||
|
Hanya Traefik yang perlu akses ke Docker socket (read-only):
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
```
|
||||||
|
|
||||||
|
Service lain tidak boleh mount Docker socket.
|
||||||
|
|
||||||
|
### Image Security
|
||||||
|
|
||||||
|
- Build dari base image resmi dan minimal (`debian:bookworm-slim`, `redis:alpine`, `nats:latest`)
|
||||||
|
- Multi-stage build untuk production image (tidak include build tools)
|
||||||
|
- Update base image secara berkala
|
||||||
|
|
||||||
|
## Network Security
|
||||||
|
|
||||||
|
### Firewall (UFW/iptables)
|
||||||
|
|
||||||
|
Di VPS (`orangevps`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Hanya buka port yang diperlukan
|
||||||
|
sudo ufw default deny incoming
|
||||||
|
sudo ufw default allow outgoing
|
||||||
|
sudo ufw allow 22/tcp # SSH
|
||||||
|
sudo ufw allow 80/tcp # HTTP redirect
|
||||||
|
sudo ufw allow 443/tcp # HTTPS
|
||||||
|
sudo ufw allow 4222/tcp # NATS (jika perlu external akses)
|
||||||
|
sudo ufw enable
|
||||||
|
```
|
||||||
|
|
||||||
|
Di `imrnes`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Hanya dari Tailscale interface
|
||||||
|
sudo ufw allow in on tailscale0 to any port 6432 proto tcp # PostgreSQL
|
||||||
|
sudo ufw allow in on tailscale0 to any port 6379 proto tcp # Redis
|
||||||
|
sudo ufw enable
|
||||||
|
```
|
||||||
|
|
||||||
|
### Network Segmentation
|
||||||
|
|
||||||
|
- Semua container di network `app-shared-net` (internal bridge)
|
||||||
|
- Tidak ada port yang di-expose ke host kecuali Traefik (80,443)
|
||||||
|
- Redis hanya accessible via Docker DNS (`redis:6379`) — tidak di-expose
|
||||||
|
- Database hanya via Tailscale — tidak accessible dari public internet
|
||||||
|
|
||||||
|
### SSH Hardening
|
||||||
|
|
||||||
|
Konfigurasi di `/etc/ssh/sshd_config`:
|
||||||
|
|
||||||
|
```
|
||||||
|
Port 22
|
||||||
|
PermitRootLogin prohibit-password
|
||||||
|
PasswordAuthentication no
|
||||||
|
PubkeyAuthentication yes
|
||||||
|
AllowUsers root
|
||||||
|
MaxAuthTries 3
|
||||||
|
ClientAliveInterval 300
|
||||||
|
ClientAliveCountMax 2
|
||||||
|
```
|
||||||
|
|
||||||
|
## Access Control
|
||||||
|
|
||||||
|
### GitHub Repository
|
||||||
|
|
||||||
|
- `contents: write` hanya untuk workflow `update-manifest` dan `update-submodule`
|
||||||
|
- `packages: write` hanya untuk workflow `build`
|
||||||
|
- `security-events: write` hanya untuk workflow `security`
|
||||||
|
- Branch protection di `main`: require PR review, status checks
|
||||||
|
|
||||||
|
### VPS
|
||||||
|
|
||||||
|
- SSH hanya dengan key-based authentication
|
||||||
|
- Key disimpan di GitHub secret, bukan di repo
|
||||||
|
- Rotate SSH key secara berkala (minimal 6 bulan)
|
||||||
|
- Jangan gunakan password login
|
||||||
|
|
||||||
|
## Monitoring Keamanan
|
||||||
|
|
||||||
|
### Saat Ini
|
||||||
|
|
||||||
|
- Traefik access logs (format JSON, buffer size 100)
|
||||||
|
- Docker logs via `docker logs`
|
||||||
|
- CodeQL analysis untuk Rust code (setiap PR + weekly)
|
||||||
|
|
||||||
|
### Rekomendasi
|
||||||
|
|
||||||
|
- [ ] Alert untuk SSH failed login (fail2ban)
|
||||||
|
- [ ] Log monitoring (Loki / Promtail)
|
||||||
|
- [ ] Container vulnerability scanning (Trivy / Snyk)
|
||||||
|
- [ ] Certificate expiry monitoring
|
||||||
|
- [ ] Disk usage alert
|
||||||
|
- [ ] Unauthorized access detection
|
||||||
|
|
||||||
|
## Checklist Security
|
||||||
|
|
||||||
|
- [ ] SSH password authentication disabled
|
||||||
|
- [ ] Root login via SSH key only
|
||||||
|
- [ ] UFW/iptables configured
|
||||||
|
- [ ] Docker socket only mounted where necessary (read-only)
|
||||||
|
- [ ] Container berjalan sebagai non-root user
|
||||||
|
- [ ] `.env` tidak di-commit
|
||||||
|
- [ ] GitHub secrets ter-encrypt
|
||||||
|
- [ ] TLS certificates valid dan belum expired
|
||||||
|
- [ ] CodeQL analysis berjalan
|
||||||
|
- [ ] Backup database berjalan
|
||||||
|
- [ ] SSH key di-rotate
|
||||||
|
- [ ] Docker image di-scan untuk vulnerability
|
||||||
@@ -0,0 +1,196 @@
|
|||||||
|
# Tailscale Networking
|
||||||
|
|
||||||
|
Dokumentasi setup dan troubleshooting konektivitas Tailscale antara node `orangevps` (VPS) dan `imrnes` (bare-metal).
|
||||||
|
|
||||||
|
## Topologi
|
||||||
|
|
||||||
|
```
|
||||||
|
orangevps (VPS)
|
||||||
|
├─ Tailscale IP: 100.x.x.x (dynamic)
|
||||||
|
├─ Public IP: 45.127.35.244
|
||||||
|
├─ Docker containers (app-shared-net)
|
||||||
|
│ └─ perlu akses ke imrnes via Tailscale
|
||||||
|
└─ tailscale-routes.service
|
||||||
|
└─ menambahkan route 100.x.x.x ke tabel routing main
|
||||||
|
|
||||||
|
imrnes (Bare-metal)
|
||||||
|
├─ Tailscale IP: 100.121.180.82
|
||||||
|
├─ Layanan:
|
||||||
|
│ ├─ PostgreSQL (port 6432)
|
||||||
|
│ └─ Redis (port 6379)
|
||||||
|
└─ Layanan hanya listen di Tailscale interface
|
||||||
|
```
|
||||||
|
|
||||||
|
## Masalah: Container Tidak Bisa Mencapai Tailscale IP
|
||||||
|
|
||||||
|
Docker container secara default hanya bisa mencapai IP di Docker bridge network dan network host. Tailscale menggunakan interface virtual `tailscale0` yang tidak secara otomatis di-route ke container.
|
||||||
|
|
||||||
|
### Solusi: `tailscale-routes.service`
|
||||||
|
|
||||||
|
Service systemd yang menambahkan route Tailscale ke tabel routing `main` agar traffic dari container bisa melewati host ke Tailscale.
|
||||||
|
|
||||||
|
```ini
|
||||||
|
# /etc/systemd/system/tailscale-routes.service
|
||||||
|
[Unit]
|
||||||
|
Description=Add Tailscale routes to main routing table
|
||||||
|
After=tailscaled.service
|
||||||
|
Requires=tailscaled.service
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
RemainAfterExit=yes
|
||||||
|
ExecStart=/bin/sh -c 'ip rule add from all lookup main priority 10000 2>/dev/null; ip route add 100.64.0.0/10 dev tailscale0 table main 2>/dev/null || true'
|
||||||
|
ExecStop=/bin/sh -c 'ip rule del from all lookup main priority 10000 2>/dev/null; ip route del 100.64.0.0/10 dev tailscale0 table main 2>/dev/null || true'
|
||||||
|
RemainAfterExit=yes
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
```
|
||||||
|
|
||||||
|
### Verifikasi
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek apakah route sudah ada
|
||||||
|
ip route show table main | grep tailscale
|
||||||
|
|
||||||
|
# Test dari dalam container
|
||||||
|
docker run --rm alpine ping -c 3 100.121.180.82
|
||||||
|
|
||||||
|
# Test koneksi PostgreSQL dari container
|
||||||
|
docker run --rm alpine sh -c "apk add postgresql-client && psql -h 100.121.180.82 -p 6432 -U asephs -d hub -c 'SELECT 1'"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Setup Tailscale di Node Baru
|
||||||
|
|
||||||
|
### 1. Install Tailscale
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -fsSL https://tailscale.com/install.sh | sh
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Authenticate
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo tailscale up --advertise-routes=<LAN_SUBNET_CIDR>
|
||||||
|
```
|
||||||
|
|
||||||
|
Untuk node yang hanya sebagai client (tidak advertise routes):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo tailscale up
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3. Enable dan Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl enable --now tailscaled
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4. Setup Route Service (khusus node dengan Docker)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Buat service file
|
||||||
|
sudo nano /etc/systemd/system/tailscale-routes.service
|
||||||
|
# Paste content di atas
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now tailscale-routes.service
|
||||||
|
```
|
||||||
|
|
||||||
|
### 5. Konfigurasi ACL di Tailscale Admin
|
||||||
|
|
||||||
|
Pastikan ACL di [Tailscale Admin Console](https://login.tailscale.com/admin/acls) mengizinkan traffic antar node:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"acls": [
|
||||||
|
{"action": "accept", "src": ["*"], "dst": ["*:*"]}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Atau jika ingin lebih ketat:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"acls": [
|
||||||
|
{"action": "accept", "src": ["tag:server"], "dst": ["tag:server:*"]}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Konfigurasi iptables/ufw
|
||||||
|
|
||||||
|
Pastikan port yang diperlukan terbuka di `imrnes`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# PostgreSQL
|
||||||
|
sudo ufw allow in on tailscale0 to any port 6432 proto tcp
|
||||||
|
|
||||||
|
# Redis
|
||||||
|
sudo ufw allow in on tailscale0 to any port 6379 proto tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
Atau menggunakan iptables langsung:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo iptables -A INPUT -i tailscale0 -p tcp --dport 6432 -j ACCEPT
|
||||||
|
sudo iptables -A INPUT -i tailscale0 -p tcp --dport 6379 -j ACCEPT
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Container timeout connect ke Tailscale IP
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Cek apakah route service berjalan
|
||||||
|
systemctl status tailscale-routes.service
|
||||||
|
|
||||||
|
# 2. Cek route di host
|
||||||
|
ip route show table main | grep 100.64
|
||||||
|
|
||||||
|
# 3. Cek apakah host bisa ping ke target
|
||||||
|
ping 100.121.180.82
|
||||||
|
|
||||||
|
# 4. Test dari container dengan --network host
|
||||||
|
docker run --rm --network host alpine ping -c 3 100.121.180.82
|
||||||
|
|
||||||
|
# 5. Pastikan tidak ada firewall blocking
|
||||||
|
iptables -L FORWARD -n -v
|
||||||
|
```
|
||||||
|
|
||||||
|
### Tailscale disconnect
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek status
|
||||||
|
tailscale status
|
||||||
|
|
||||||
|
# Restart
|
||||||
|
sudo systemctl restart tailscaled
|
||||||
|
```
|
||||||
|
|
||||||
|
### IP Tailscale berubah
|
||||||
|
|
||||||
|
Tailscale IP bisa berubah jika node dire-auth. Update:
|
||||||
|
|
||||||
|
1. `.env` production di VPS (via GitHub secret `ENV_FILE_PRODUCTION`)
|
||||||
|
2. Database connection strings
|
||||||
|
3. Redis connection strings
|
||||||
|
4. Trigger redeploy
|
||||||
|
|
||||||
|
### MagicDNS tidak resolve
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek DNS
|
||||||
|
tailscale dns status
|
||||||
|
|
||||||
|
# Flush DNS cache
|
||||||
|
sudo resolvectl flush-caches
|
||||||
|
```
|
||||||
|
|
||||||
|
## Catatan Keamanan
|
||||||
|
|
||||||
|
- Interface Tailscale (`tailscale0`) hanya boleh diakses oleh node yang terautentikasi dalam network yang sama
|
||||||
|
- Jangan expose port database ke public interface (`eth0`), hanya ke Tailscale
|
||||||
|
- Gunakan ACL untuk membatasi akses antar node jika diperlukan
|
||||||
|
- Rotate auth key secara berkala di Tailscale admin console
|
||||||
@@ -0,0 +1,493 @@
|
|||||||
|
# Troubleshooting
|
||||||
|
|
||||||
|
Kumpulan solusi untuk masalah umum yang spesifik di infrastruktur `asepharyana-hub`.
|
||||||
|
|
||||||
|
## Daftar Isi
|
||||||
|
|
||||||
|
- [Deployment](#deployment)
|
||||||
|
- [Dapr](#dapr)
|
||||||
|
- [NATS](#nats)
|
||||||
|
- [Traefik](#traefik)
|
||||||
|
- [Tailscale / Networking](#tailscale--networking)
|
||||||
|
- [Docker / Container](#docker--container)
|
||||||
|
- [Database](#database)
|
||||||
|
- [Submodule](#submodule)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Deployment
|
||||||
|
|
||||||
|
### Workflow deploy gagal: "Secrets not fully configured"
|
||||||
|
|
||||||
|
**Penyebab:** Salah satu GitHub secrets tidak diset.
|
||||||
|
|
||||||
|
**Solusi:** Cek secrets di Settings > Secrets and variables > Actions:
|
||||||
|
|
||||||
|
| Secret | Status |
|
||||||
|
|--------|--------|
|
||||||
|
| `SSH_PRIVATE_KEY` | Wajib |
|
||||||
|
| `VPS_HOST` | Wajib (`45.127.35.244`) |
|
||||||
|
| `VPS_USER` | Wajib (`root`) |
|
||||||
|
| `VPS_TARGET_DIR` | Wajib (`/root/asepharyana-hub`) |
|
||||||
|
| `ENV_FILE_PRODUCTION` | Wajib |
|
||||||
|
|
||||||
|
### Workflow build gagal: "Submodule commit not fetchable"
|
||||||
|
|
||||||
|
**Penyebab:** Commit SHA dari `repository_dispatch` belum tersedia di remote submodule repo (eventual consistency).
|
||||||
|
|
||||||
|
**Solusi:** Workflow akan retry hingga 5 menit. Jika masih gagal:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek apakah commit ada di remote
|
||||||
|
git ls-remote https://github.com/asepharyana/asepharyana-hub-scraper.git <SHA>
|
||||||
|
|
||||||
|
# Trigger ulang dispatch dari submodule repo, atau push langsung ke hub
|
||||||
|
```
|
||||||
|
|
||||||
|
### Push manifest gagal: conflict di main
|
||||||
|
|
||||||
|
**Penyebab:** Ada commit lain yang masuk sebelum workflow selesai.
|
||||||
|
|
||||||
|
**Solusi:** Workflow otomatis retry rebase 3 kali. Jika semua gagal:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Manual fix di lokal
|
||||||
|
git pull --rebase origin main
|
||||||
|
# resolve conflict
|
||||||
|
git push origin main
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dapr
|
||||||
|
|
||||||
|
### Dapr sidecar tidak connect ke placement
|
||||||
|
|
||||||
|
**Gejala:** Container `scraper-api-dapr` restart loop. Log: `failed to connect to placement`
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek log sidecar
|
||||||
|
docker logs scraper-api-dapr --tail 50
|
||||||
|
|
||||||
|
# Cek apakah placement service running
|
||||||
|
docker ps -a | grep dapr-placement
|
||||||
|
docker logs dapr-placement --tail 20
|
||||||
|
|
||||||
|
# Cek konektivitas
|
||||||
|
docker exec scraper-api-dapr curl -s http://dapr-placement:50005
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Restart placement dulu, lalu sidecar
|
||||||
|
docker compose -f infra/compose/dapr.yml up -d --force-recreate
|
||||||
|
sleep 5
|
||||||
|
docker compose -f infra/compose/scraper.yml up -d --force-recreate scraper-api-dapr
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dapr pub/sub tidak bekerja
|
||||||
|
|
||||||
|
**Gejala:** Event di-publish tapi tidak sampai ke subscriber.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek komponen Dapr
|
||||||
|
curl http://localhost:3500/v1.0/components
|
||||||
|
|
||||||
|
# Cek health sidecar
|
||||||
|
curl http://localhost:3500/v1.0/healthz
|
||||||
|
|
||||||
|
# Cek Redis (backend pub/sub)
|
||||||
|
docker exec redis redis-cli ping
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Restart sidecar
|
||||||
|
docker restart scraper-api-dapr
|
||||||
|
|
||||||
|
# Jika Redis bermasalah, restart juga
|
||||||
|
docker restart redis
|
||||||
|
```
|
||||||
|
|
||||||
|
### Dapr state store error: "key not found"
|
||||||
|
|
||||||
|
**Penyebab:** Key belum ada di state store, atau prefix berbeda.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek state langsung di Redis
|
||||||
|
docker exec redis redis-cli KEYS 'dapr*'
|
||||||
|
|
||||||
|
# State store menggunakan prefix "dapr"
|
||||||
|
# Format key: dapr || <app-id> || <key>
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## NATS
|
||||||
|
|
||||||
|
### NATS tidak bisa start
|
||||||
|
|
||||||
|
**Gejala:** Container NATS restart loop.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs nats --tail 50
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:** Kemungkinan korupsi data JetStream:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Backup dulu volume data
|
||||||
|
docker run --rm -v nats_data:/data -v /tmp:/backup alpine cp -r /data /backup/nats_data_backup
|
||||||
|
|
||||||
|
# Hapus volume dan recreate
|
||||||
|
docker compose -f infra/compose/nats.yml down
|
||||||
|
docker volume rm asepharyana-hub_nats_data
|
||||||
|
docker compose -f infra/compose/nats.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
### JetStream stream overflow
|
||||||
|
|
||||||
|
**Gejala:** Disk penuh, NATS lambat.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek ukuran volume
|
||||||
|
docker system df | grep nats_data
|
||||||
|
du -sh /var/lib/docker/volumes/nats_data/_data/
|
||||||
|
|
||||||
|
# Cek stream info
|
||||||
|
nats stream list
|
||||||
|
nats stream info <stream-name>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Purge stream tertentu (data hilang)
|
||||||
|
nats stream purge <stream-name>
|
||||||
|
|
||||||
|
# Atau tambah limit stream via NATS config
|
||||||
|
```
|
||||||
|
|
||||||
|
### "Slow Consumer" warning
|
||||||
|
|
||||||
|
**Gejala:** Log NATS menampilkan "slow consumer".
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8222/varz | jq '.slow_consumers'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
- Scale consumer (tambah worker)
|
||||||
|
- Percepat processing message
|
||||||
|
- Kurangi ukuran payload
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Traefik
|
||||||
|
|
||||||
|
### Traefik tidak routing ke service
|
||||||
|
|
||||||
|
**Gejala:** 404 atau 503 dari Traefik.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek apakah service container running
|
||||||
|
docker ps -a | grep scraper-api
|
||||||
|
|
||||||
|
# Cek log Traefik
|
||||||
|
docker logs traefik --tail 50
|
||||||
|
|
||||||
|
# Cek apakah container ada di network yang benar
|
||||||
|
docker network inspect app-shared-net | grep scraper-api
|
||||||
|
|
||||||
|
# Test routing langsung
|
||||||
|
curl -H "Host: scraper.asepharyana.my.id" http://localhost/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Pastikan service terdaftar di apps.yaml
|
||||||
|
# Pastikan container join app-shared-net
|
||||||
|
# Restart Traefik
|
||||||
|
docker compose -f infra/compose/traefik.yml up -d --force-recreate
|
||||||
|
```
|
||||||
|
|
||||||
|
### TLS certificate error
|
||||||
|
|
||||||
|
**Gejala:** Browser menampilkan warning certificate.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek sertifikat di host
|
||||||
|
ls -la /root/asepharyana.my.id.pem
|
||||||
|
openssl x509 -in /root/asepharyana.my.id.pem -text -noout | head -20
|
||||||
|
|
||||||
|
# Cek apakah Traefik bisa mount
|
||||||
|
docker exec traefik ls -la /etc/traefik/certs/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
- Update sertifikat di host
|
||||||
|
- Restart Traefik
|
||||||
|
- Jika path berbeda, set environment variable `TRAEFIK_CERT_*`
|
||||||
|
|
||||||
|
### Rate limit terlalu ketat
|
||||||
|
|
||||||
|
**Gejala:** Request legitimate di-block.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek rate-limit config di middlewares.yaml
|
||||||
|
# Current: average 100, burst 50
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:** Ubah nilai `average` dan `burst` di `infra/traefik/dynamic/middlewares.yaml`, lalu reload:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker kill --signal HUP traefik
|
||||||
|
# atau
|
||||||
|
docker exec traefik kill -HUP 1
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tailscale / Networking
|
||||||
|
|
||||||
|
### Container tidak bisa connect ke Tailscale IP
|
||||||
|
|
||||||
|
**Gejala:** Timeout saat container connect ke `100.121.180.82:6432`.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek route service dari host
|
||||||
|
systemctl status tailscale-routes.service
|
||||||
|
|
||||||
|
# Cek route di host
|
||||||
|
ip route show table main | grep 100.64
|
||||||
|
|
||||||
|
# Cek koneksi dari host
|
||||||
|
ping 100.121.180.82
|
||||||
|
|
||||||
|
# Test dari container (dengan --network host)
|
||||||
|
docker run --rm --network host alpine ping -c 3 100.121.180.82
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Restart route service
|
||||||
|
sudo systemctl restart tailscale-routes.service
|
||||||
|
|
||||||
|
# Atau tambah route manual
|
||||||
|
sudo ip rule add from all lookup main priority 10000
|
||||||
|
sudo ip route add 100.64.0.0/10 dev tailscale0 table main
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database connection refused
|
||||||
|
|
||||||
|
**Gejala:** Service tidak bisa konek ke PostgreSQL.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek apakah DB listen di Tailscale (dari imrnes)
|
||||||
|
ss -tlnp | grep 6432
|
||||||
|
|
||||||
|
# Cek dari orangevps
|
||||||
|
nc -zv 100.121.180.82 6432
|
||||||
|
|
||||||
|
# Cek firewall di imrnes
|
||||||
|
sudo ufw status
|
||||||
|
sudo iptables -L -n | grep 6432
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Di imrnes: pastikan PostgreSQL bind ke Tailscale interface
|
||||||
|
# Di postgresql.conf:
|
||||||
|
listen_addresses = 'localhost,100.121.180.82'
|
||||||
|
|
||||||
|
# Di pg_hba.conf:
|
||||||
|
host hub asephs 100.0.0.0/8 md5
|
||||||
|
|
||||||
|
# Restart PostgreSQL
|
||||||
|
sudo systemctl restart postgresql
|
||||||
|
```
|
||||||
|
|
||||||
|
### Redis connection refused dari container
|
||||||
|
|
||||||
|
**Gejala:** Service tidak bisa connect ke `redis://redis:6379`.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek apakah container Redis running
|
||||||
|
docker ps -a | grep redis
|
||||||
|
|
||||||
|
# Cek apakah container target join network yang sama
|
||||||
|
docker inspect <container> | grep -A5 Networks
|
||||||
|
|
||||||
|
# Cek DNS resolve dari container
|
||||||
|
docker exec <container> getent hosts redis
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Pastikan Redis ada di network app-shared-net
|
||||||
|
docker network inspect app-shared-net | grep redis
|
||||||
|
|
||||||
|
# Jika tidak, attach
|
||||||
|
docker network connect app-shared-net redis
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Docker / Container
|
||||||
|
|
||||||
|
### Container restart loop
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs <container> --tail 50
|
||||||
|
docker inspect <container> | jq '.[].State'
|
||||||
|
```
|
||||||
|
|
||||||
|
**Penyebab umum:**
|
||||||
|
- Health check gagal
|
||||||
|
- Dependency service belum siap
|
||||||
|
- Environment variable tidak diset
|
||||||
|
|
||||||
|
### Image pull gagal dari GHCR
|
||||||
|
|
||||||
|
**Gejala:** `docker pull` gagal di VPS.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek login
|
||||||
|
cat ~/.docker/config.json | grep ghcr
|
||||||
|
|
||||||
|
# Cek visibility package
|
||||||
|
# Buka https://github.com/orgs/asepharyana/packages
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Re-login
|
||||||
|
echo $GITHUB_TOKEN | docker login ghcr.io -u asepharyana --password-stdin
|
||||||
|
|
||||||
|
# Pastikan package visibility public atau di-share ke org
|
||||||
|
```
|
||||||
|
|
||||||
|
### Disk penuh
|
||||||
|
|
||||||
|
**Gejala:** Container crash, write error.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
df -h
|
||||||
|
docker system df
|
||||||
|
du -sh /var/lib/docker/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Bersihkan container/image/volume yang tidak dipakai
|
||||||
|
docker system prune -a -f
|
||||||
|
|
||||||
|
# Hapus image lama
|
||||||
|
docker image prune -a -f
|
||||||
|
|
||||||
|
# Lihat volume terbesar
|
||||||
|
docker system df -v | grep -E "(nats_data|redis_data)"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Database
|
||||||
|
|
||||||
|
### Koneksi PostgreSQL lambat
|
||||||
|
|
||||||
|
**Gejala:** Query time high, connection timeout.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Dari container, test latency
|
||||||
|
docker exec scraper-api ping -c 5 100.121.180.82
|
||||||
|
|
||||||
|
# Cek koneksi aktif
|
||||||
|
docker exec scraper-api psql $DATABASE_URL -c "SELECT count(*) FROM pg_stat_activity;"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
- Cek Tailscale latency
|
||||||
|
- Adjust connection pool size
|
||||||
|
- Cek resource PostgreSQL di `imrnes`
|
||||||
|
|
||||||
|
### Migration gagal
|
||||||
|
|
||||||
|
**Gejala:** Service error setelah image update.
|
||||||
|
|
||||||
|
**Diagnosis:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek log service
|
||||||
|
docker logs scraper-api --tail 100 | grep -i migration
|
||||||
|
```
|
||||||
|
|
||||||
|
**Solusi:**
|
||||||
|
- Migration ada di submodule `apps/scraper`, bukan di hub
|
||||||
|
- Pastikan schema sesuai dengan versi code
|
||||||
|
- Rollback image jika migration tidak backward-compatible
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Submodule
|
||||||
|
|
||||||
|
### HEAD detached di submodule
|
||||||
|
|
||||||
|
**Gejala:** `git status` di `apps/scraper` menunjukkan "HEAD detached".
|
||||||
|
|
||||||
|
**Penyebab:** Normal. Submodule selalu dalam keadaan detached HEAD karena mengacu pada commit spesifik.
|
||||||
|
|
||||||
|
**Solusi:** Jangan commit perubahan dari dalam submodule. Selalu bekerja di repo asli.
|
||||||
|
|
||||||
|
### Submodule tidak ter-update setelah pull
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git submodule update --init --recursive
|
||||||
|
```
|
||||||
|
|
||||||
|
### Konflik submodule saat rebase/merge
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Resolve dengan memilih versi yang benar
|
||||||
|
git add apps/scraper
|
||||||
|
git rebase --continue
|
||||||
|
```
|
||||||
Reference in New Issue
Block a user