Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 asepharyana
|
d26923c9b0 |
@@ -1,73 +0,0 @@
|
|||||||
{
|
|
||||||
"skills": [
|
|
||||||
{
|
|
||||||
"name": "clean-code",
|
|
||||||
"filePattern": ".claude/skills/clean-code/SKILL.md",
|
|
||||||
"description": "Clean Code, Clean Architecture, SOLID, TDD — dari kana-best-practice-engineering"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "hub-rules",
|
|
||||||
"filePattern": ".claude/skills/hub-rules.md",
|
|
||||||
"description": "Aturan repository hub, submodule, infra patterns, dan arsitektur"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "commit-convention",
|
|
||||||
"filePattern": ".claude/skills/commit-convention.md",
|
|
||||||
"description": "Commit message convention — type(scope): description"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "event-driven",
|
|
||||||
"filePattern": ".claude/skills/event-driven.md",
|
|
||||||
"description": "Event-driven patterns with Dapr + NATS untuk hub services"
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "deploy-workflow",
|
|
||||||
"filePattern": ".claude/skills/deploy-workflow.md",
|
|
||||||
"description": "CI/CD pipeline, Docker patterns, deployment guide"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"hooks": {
|
|
||||||
"PreToolUse": [
|
|
||||||
{
|
|
||||||
"matcher": "",
|
|
||||||
"hooks": [
|
|
||||||
{
|
|
||||||
"type": "command",
|
|
||||||
"command": "serena-hooks remind --client=claude-code"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"matcher": "mcp__serena__*",
|
|
||||||
"hooks": [
|
|
||||||
{
|
|
||||||
"type": "command",
|
|
||||||
"command": "serena-hooks auto-approve --client=claude-code"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"SessionStart": [
|
|
||||||
{
|
|
||||||
"matcher": "",
|
|
||||||
"hooks": [
|
|
||||||
{
|
|
||||||
"type": "command",
|
|
||||||
"command": "serena-hooks activate --client=claude-code"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"SessionEnd": [
|
|
||||||
{
|
|
||||||
"matcher": "",
|
|
||||||
"hooks": [
|
|
||||||
{
|
|
||||||
"type": "command",
|
|
||||||
"command": "serena-hooks cleanup --client=claude-code"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1 +0,0 @@
|
|||||||
/home/asephs/kana-best-practice-engineering/skills/clean-code
|
|
||||||
@@ -1,65 +0,0 @@
|
|||||||
---
|
|
||||||
name: commit-convention
|
|
||||||
description: Enforce commit message convention untuk Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Commit Convention — Asepharyana Hub
|
|
||||||
|
|
||||||
## Format
|
|
||||||
|
|
||||||
```
|
|
||||||
<type>(<scope>): <description>
|
|
||||||
|
|
||||||
[optional body]
|
|
||||||
|
|
||||||
[optional footer]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Types
|
|
||||||
|
|
||||||
| Type | Usage |
|
|
||||||
| ---------- | ------------------------------------ |
|
|
||||||
| `feat` | Fitur baru |
|
|
||||||
| `fix` | Bug fix |
|
|
||||||
| `chore` | Maintenance, config, tooling |
|
|
||||||
| `docs` | Dokumentasi |
|
|
||||||
| `refactor` | Perubahan kode tanpa fungsional baru |
|
|
||||||
| `test` | Nambah/update test |
|
|
||||||
| `ci` | CI/CD workflows |
|
|
||||||
| `perf` | Optimasi performa |
|
|
||||||
| `style` | Formatting (tanda kutip, dll) |
|
|
||||||
|
|
||||||
## Scopes
|
|
||||||
|
|
||||||
| Scope | Area |
|
|
||||||
| ------------- | --------------------------------- |
|
|
||||||
| `scraper` | apps/scraper submodule |
|
|
||||||
| `infra` | infra/ (compose, traefik, docker) |
|
|
||||||
| `ci` | .github/workflows/ |
|
|
||||||
| `dapr` | Dapr config & sidecar |
|
|
||||||
| `nats` | NATS message bus |
|
|
||||||
| `docs` | Dokumentasi |
|
|
||||||
| `deps` | Dependencies |
|
|
||||||
| `scripts` | Utility scripts |
|
|
||||||
| `root` | Root config files |
|
|
||||||
|
|
||||||
## Contoh
|
|
||||||
|
|
||||||
```
|
|
||||||
feat(scraper): add anime detail caching via Dapr pubsub
|
|
||||||
fix(infra): correct NATS CLI flags for JetStream
|
|
||||||
chore(deps): update biome to v2.5.3
|
|
||||||
docs(infra): add deployment order for Dapr services
|
|
||||||
ci(deploy): add nats.yml to ALL_COMPOSE_FILES
|
|
||||||
refactor(scraper): migrate EventBus from tokio broadcast to Dapr pubsub
|
|
||||||
```
|
|
||||||
|
|
||||||
## Aturan
|
|
||||||
|
|
||||||
1. **Wajib** menyertakan scope dalam tanda kurung
|
|
||||||
2. **Wajib** `Co-Authored-By` untuk commit yang digenerate AI
|
|
||||||
3. **Gunakan imperative mood**: "add" bukan "added" / "adds"
|
|
||||||
4. **Jangan capitalize** type: `feat:` bukan `Feat:`
|
|
||||||
5. **No period** di akhir subject baris
|
|
||||||
6. Body explain **why** dan **what**, bukan **how**
|
|
||||||
7. Refer issue dengan `Closes #123` atau `Fixes #123` di footer
|
|
||||||
@@ -1,103 +0,0 @@
|
|||||||
---
|
|
||||||
name: deploy-workflow
|
|
||||||
description: Panduan deploy, CI/CD, dan Nix/systemd patterns untuk Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Deploy & Workflow — Asepharyana Hub
|
|
||||||
|
|
||||||
## CI/CD Pipeline
|
|
||||||
|
|
||||||
### Build Pipeline (`docker-build-push.yml`)
|
|
||||||
Trigger: push ke `main` yang touch `apps/**`, `infra/**`, `infra/docker/**`
|
|
||||||
|
|
||||||
1. **changes** — detect service mana yg berubah via git diff
|
|
||||||
2. **wait-submodule-ref** — (repository_dispatch only) tunggu SHA commit fetchable
|
|
||||||
3. **build** — matrix build per service, push ke GHCR (`sha-<short>` + `latest`)
|
|
||||||
4. **update-manifest** — update image tag di compose file, commit + push
|
|
||||||
|
|
||||||
### Deploy Pipeline (`deploy-docker.yml`)
|
|
||||||
Trigger: build selesai, atau push ke `main` touch `infra/**`
|
|
||||||
|
|
||||||
1. SSH ke `orangevps` (via `secrets.VPS_HOST`)
|
|
||||||
2. Sync repo (`git fetch --depth=1 + reset`)
|
|
||||||
3. Login ke GHCR
|
|
||||||
4. Deteksi compose file yg berubah
|
|
||||||
5. Pull images + restart container selektif
|
|
||||||
|
|
||||||
### Secrets Required
|
|
||||||
| Secret | Untuk |
|
|
||||||
|--------|-------|
|
|
||||||
| `SSH_PRIVATE_KEY` | SSH ke VPS |
|
|
||||||
| `VPS_HOST` | IP/host VPS (tailscale IP) |
|
|
||||||
| `VPS_USER` | SSH user, biasanya `root` |
|
|
||||||
| `VPS_TARGET_DIR` | Lokasi repo di VPS |
|
|
||||||
| `ENV_FILE_PRODUCTION` | .env content untuk production |
|
|
||||||
|
|
||||||
### Selective Deployment
|
|
||||||
- Hanya compose file yg berubah yang di-redeploy
|
|
||||||
- Selective: `UP_FLAGS="-d"` (tanpa `--remove-orphans`)
|
|
||||||
- Full deploy: `UP_FLAGS="-d --remove-orphans"`
|
|
||||||
|
|
||||||
## Docker Patterns
|
|
||||||
|
|
||||||
### Build dengan cargo-chef (Rust)
|
|
||||||
```dockerfile
|
|
||||||
FROM lukemathwalker/cargo-chef:latest-rust-1.89.0 AS chef
|
|
||||||
WORKDIR /app
|
|
||||||
FROM chef AS planner
|
|
||||||
COPY apps/scraper .
|
|
||||||
RUN cargo chef prepare --recipe-path recipe.json
|
|
||||||
FROM chef AS builder
|
|
||||||
COPY --from=planner /app/recipe.json recipe.json
|
|
||||||
RUN cargo chef cook --release --recipe-path recipe.json
|
|
||||||
COPY apps/scraper .
|
|
||||||
RUN cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
### Runtime minimal untuk Rust binary
|
|
||||||
```dockerfile
|
|
||||||
FROM debian:bookworm-slim AS runtime
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
ca-certificates curl libssl3 && rm -rf /var/lib/apt/lists/*
|
|
||||||
```
|
|
||||||
|
|
||||||
## Image Tagging
|
|
||||||
- `sha-<short-sha>` — immutable, untuk rollback
|
|
||||||
- `latest` — mutable, untuk convenience
|
|
||||||
- Build cache: `sha-<short>-buildcache`
|
|
||||||
- Registry: `ghcr.io/asepharyana/asepharyana-hub/<service>`
|
|
||||||
|
|
||||||
## Manual Deploy Steps
|
|
||||||
```bash
|
|
||||||
# 1. Login GHCR
|
|
||||||
echo $GITHUB_TOKEN | docker login ghcr.io -u asepharyana --password-stdin
|
|
||||||
|
|
||||||
# 2. Full stack
|
|
||||||
docker compose -f infra/compose/traefik.yml \
|
|
||||||
-f infra/compose/shared.yml \
|
|
||||||
-f infra/compose/nats.yml \
|
|
||||||
-f infra/compose/dapr.yml \
|
|
||||||
-f infra/compose/scraper.yml \
|
|
||||||
--env-file .env up -d --remove-orphans
|
|
||||||
|
|
||||||
# 3. Selective (hanya satu service)
|
|
||||||
docker compose -f infra/compose/scraper.yml --env-file .env up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Container reach Tailscale
|
|
||||||
Pastikan route ke Tailscale di main table:
|
|
||||||
```bash
|
|
||||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
|
||||||
systemctl restart tailscale-routes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Healthcheck gagal di scratch images
|
|
||||||
NATS dan Dapr placement pake scratch — tidak bisa healthcheck. Cukup `service_started` di depends_on.
|
|
||||||
|
|
||||||
### Dapr sidecar crash
|
|
||||||
```bash
|
|
||||||
docker logs scraper-api-dapr | grep -iE "fatal|error"
|
|
||||||
```
|
|
||||||
Penyebab umum: komponen config salah, NATS/Dapr placement belum siap.
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
---
|
|
||||||
name: event-driven
|
|
||||||
description: Event-driven patterns dengan Dapr + NATS untuk Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Event-Driven Architecture — Asepharyana Hub
|
|
||||||
|
|
||||||
## Stack
|
|
||||||
- **Message Backbone**: NATS + JetStream (untuk streaming & job queue)
|
|
||||||
- **Pub/Sub Runtime**: Dapr sidecar per service (pubsub via Redis built-in)
|
|
||||||
- **State Store**: Dapr → Redis
|
|
||||||
|
|
||||||
## Event Topics Convention
|
|
||||||
|
|
||||||
```
|
|
||||||
hub.<domain>.<action>
|
|
||||||
|
|
||||||
Contoh:
|
|
||||||
hub.image.cached → Image selesai di-cache ke CDN
|
|
||||||
hub.image.repaired → Image diperbaiki (CNAME change)
|
|
||||||
hub.scrape.anime.done → Scrape anime selesai
|
|
||||||
hub.system.alert → Error/alert dari service
|
|
||||||
```
|
|
||||||
|
|
||||||
## CloudEvents Format
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"specversion": "1.0",
|
|
||||||
"type": "hub.image.cached",
|
|
||||||
"source": "scraper-api",
|
|
||||||
"subject": "anime-poster",
|
|
||||||
"id": "uuid-v4",
|
|
||||||
"time": "2026-07-21T10:00:00Z",
|
|
||||||
"datacontenttype": "application/json",
|
|
||||||
"data": { ... }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Publish Event (Rust via HTTP API)
|
|
||||||
|
|
||||||
Gunakan `reqwest` langsung ke Dapr sidecar (SDK Rust masih experimental):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let event = serde_json::json!({
|
|
||||||
"specversion": "1.0",
|
|
||||||
"type": "hub.image.cached",
|
|
||||||
"source": "scraper-api",
|
|
||||||
"id": Uuid::new_v4().to_string(),
|
|
||||||
"time": chrono::Utc::now().to_rfc3339(),
|
|
||||||
"datacontenttype": "application/json",
|
|
||||||
"data": { "original_url": url, "cdn_url": cdn_url }
|
|
||||||
});
|
|
||||||
|
|
||||||
reqwest::Client::new()
|
|
||||||
.post("http://localhost:3500/v1.0/publish/pubsub/hub.image.cached")
|
|
||||||
.json(&event)
|
|
||||||
.send()
|
|
||||||
.await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Service Invocation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://localhost:3500/v1.0/invoke/<app-id>/method/<path>
|
|
||||||
```
|
|
||||||
|
|
||||||
## State Store
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Set
|
|
||||||
curl -X POST http://localhost:3500/v1.0/state/statestore \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '[{"key": "mykey", "value": "myvalue"}]'
|
|
||||||
|
|
||||||
# Get
|
|
||||||
curl http://localhost:3500/v1.0/state/statestore/mykey
|
|
||||||
|
|
||||||
# Delete
|
|
||||||
curl -X DELETE http://localhost:3500/v1.0/state/statestore/mykey
|
|
||||||
```
|
|
||||||
|
|
||||||
## Scraper Event Integration
|
|
||||||
|
|
||||||
File yang perlu dimodifikasi untuk event-driven:
|
|
||||||
|
|
||||||
| File | Perubahan |
|
|
||||||
|------|-----------|
|
|
||||||
| `src/events/bus.rs` | Ganti backend dari tokio broadcast ke Dapr pub/sub |
|
|
||||||
| `src/bootstrap/mod.rs` | Init DaprClient, inject ke AppState |
|
|
||||||
| `src/presentation/state.rs` | Tambah `dapr_client` field |
|
|
||||||
| `src/proxy/use_cases.rs` | Publish `ImageRepaired` & `ImageCached` events |
|
|
||||||
| `src/infrastructure/services/images/cache.rs` | Emit event tiap cache selesai |
|
|
||||||
| `Cargo.toml` | Tambah `reqwest`, `uuid`, `chrono` (jika belum ada) |
|
|
||||||
|
|
||||||
## Event Handlers (Subscribe)
|
|
||||||
|
|
||||||
Buat `src/subscribers/` untuk handler:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// src/subscribers/image_handler.rs
|
|
||||||
pub async fn handle_image_cached(event: CloudEvent) -> Result<()> {
|
|
||||||
// Log, notifikasi, update status
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Daftarkan subscribers di `bootstrap/mod.rs` dengan spawn task:
|
|
||||||
```rust
|
|
||||||
tokio::spawn(async move {
|
|
||||||
let mut stream = dapr_client.subscribe("pubsub", "hub.image.cached");
|
|
||||||
while let Some(event) = stream.next().await {
|
|
||||||
handle_image_cached(event).await;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing Event-Driven Code
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_publish_event() {
|
|
||||||
let client = MockDaprClient::new();
|
|
||||||
client.expect_publish()
|
|
||||||
.with(...)
|
|
||||||
.returning(|_| Ok(()));
|
|
||||||
// ... test
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
@@ -1,96 +0,0 @@
|
|||||||
---
|
|
||||||
name: hub-rules
|
|
||||||
description: Aturan repository, arsitektur hub, submodule, dan workflow Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Asepharyana Hub — Repository Rules
|
|
||||||
|
|
||||||
## Struktur Repository
|
|
||||||
|
|
||||||
```
|
|
||||||
asepharyana-hub/
|
|
||||||
├── apps/ # Git submodules — source code aplikasi
|
|
||||||
├── docs/ # Dokumentasi, ADR, deployment guide
|
|
||||||
├── infra/ # Infrastructure as code
|
|
||||||
│ ├── compose/ # Satu compose file per service
|
|
||||||
│ ├── dapr/ # Dapr component configs
|
|
||||||
│ ├── docker/ # Dockerfiles (LEGACY — Docker dihapus)
|
|
||||||
│ ├── traefik/ # Traefik config (LEGACY — diganti Caddy)
|
|
||||||
│ └── caddy/ # Caddyfile.prod (reverse proxy produksi)
|
|
||||||
├── scripts/ # Utility scripts (cleanup, update-deps)
|
|
||||||
└── .github/workflows/ # CI/CD pipelines
|
|
||||||
```
|
|
||||||
|
|
||||||
### Aturan Submodule
|
|
||||||
- Setiap aplikasi di `apps/` adalah **submodule** ke repo terpisah.
|
|
||||||
- Perubahan kode aplikasi dilakukan di **repo masing-masing**, bukan di sini.
|
|
||||||
- Submodule pointer diupdate oleh CI/CD (bukan manual).
|
|
||||||
|
|
||||||
## Infrastructure Patterns
|
|
||||||
|
|
||||||
### Networking
|
|
||||||
- Semua service Nix/systemd, inter-service via 127.0.0.1:<port>
|
|
||||||
- Caddy sebagai ingress untuk HTTP/S eksternal (auto-TLS LE, HTTP/3)
|
|
||||||
- Tailscale untuk cross-VPS (PostgreSQL, Redis)
|
|
||||||
|
|
||||||
### Compose File Pattern
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
<service>:
|
|
||||||
container_name: <service>
|
|
||||||
image: ghcr.io/asepharyana/asepharyana-hub/<service>:sha-<sha>
|
|
||||||
restart: always
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
aliases:
|
|
||||||
- <service>
|
|
||||||
env_file:
|
|
||||||
- ../../.env
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dapr Sidecar Pattern
|
|
||||||
```yaml
|
|
||||||
<service>-dapr:
|
|
||||||
container_name: <service>-dapr
|
|
||||||
image: daprio/daprd:latest
|
|
||||||
restart: always
|
|
||||||
depends_on:
|
|
||||||
nats:
|
|
||||||
condition: service_started
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_started
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
command:
|
|
||||||
- './daprd'
|
|
||||||
- '--app-id=<service>'
|
|
||||||
- '--app-port=<port>'
|
|
||||||
- '--dapr-http-port=3500'
|
|
||||||
- '--dapr-grpc-port=50001'
|
|
||||||
- '--placement-host-address=dapr-placement:50005'
|
|
||||||
- '--resources-path=/components'
|
|
||||||
volumes:
|
|
||||||
- ../../infra/dapr/components:/components
|
|
||||||
```
|
|
||||||
|
|
||||||
### Caddy Routing
|
|
||||||
- Site block di `/etc/caddy/Caddyfile` (ref `infra/caddy/Caddyfile.prod`)
|
|
||||||
- Subdomain pattern: `<service>.asepharyana.my.id` + `<service>.asepharyana.web.id`
|
|
||||||
- TLS cert dari volume mount (bukan auto-acme)
|
|
||||||
|
|
||||||
### CI/CD
|
|
||||||
- `docker-build-push.yml` — build per service, push ke GHCR, update compose manifest
|
|
||||||
- `deploy-docker.yml` — SSH ke orangevps, pull images, restart
|
|
||||||
- Selective deploy: hanya compose file yg berubah
|
|
||||||
|
|
||||||
## Deployment Order
|
|
||||||
1. `shared.yml` (Redis)
|
|
||||||
2. `nats.yml` (NATS message bus)
|
|
||||||
3. `dapr.yml` (Dapr placement)
|
|
||||||
4. Caddy (reverse proxy)
|
|
||||||
5. Service compose files (apps + Dapr sidecar)
|
|
||||||
@@ -0,0 +1,7 @@
|
|||||||
|
{
|
||||||
|
"image": "mcr.microsoft.com/devcontainers/universal:2",
|
||||||
|
"features": {
|
||||||
|
"ghcr.io/devcontainers/features/rust:1": {},
|
||||||
|
"ghcr.io/devcontainers/features/node:2": {}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,254 @@
|
|||||||
|
name: Deploy Docker to VPS
|
||||||
|
|
||||||
|
on:
|
||||||
|
workflow_run:
|
||||||
|
workflows: ['Build and Push Docker Images']
|
||||||
|
types:
|
||||||
|
- completed
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- 'infra/**'
|
||||||
|
- '.github/workflows/deploy-docker.yml'
|
||||||
|
- '.github/workflows/docker-build-push.yml'
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
# Prevent multiple deployments from running simultaneously
|
||||||
|
concurrency:
|
||||||
|
group: deploy-vps
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
packages: read
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
deploy:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 30
|
||||||
|
if: github.event_name == 'workflow_dispatch' || github.event_name == 'push' || github.event.workflow_run.conclusion == 'success'
|
||||||
|
steps:
|
||||||
|
- name: Checkout repository
|
||||||
|
uses: actions/checkout@v6
|
||||||
|
with:
|
||||||
|
fetch-depth: 1
|
||||||
|
submodules: false
|
||||||
|
|
||||||
|
- name: Deploy to VPS
|
||||||
|
env:
|
||||||
|
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
|
||||||
|
VPS_HOST: ${{ secrets.VPS_HOST }}
|
||||||
|
VPS_USER: ${{ secrets.VPS_USER }}
|
||||||
|
VPS_TARGET_DIR: ${{ secrets.VPS_TARGET_DIR }}
|
||||||
|
ENV_FILE_PRODUCTION: ${{ secrets.ENV_FILE_PRODUCTION }}
|
||||||
|
GHCR_USERNAME: ${{ github.actor }}
|
||||||
|
GHCR_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
echo "Deploy event: ${{ github.event_name }}"
|
||||||
|
echo "Deploy ref: ${{ github.ref }}"
|
||||||
|
echo "Deploy sha: ${{ github.sha }}"
|
||||||
|
|
||||||
|
if [ -z "${SSH_PRIVATE_KEY:-}" ] || [ -z "${VPS_HOST:-}" ] || [ -z "${VPS_USER:-}" ] || [ -z "${VPS_TARGET_DIR:-}" ]; then
|
||||||
|
echo "❌ Deployment secrets are not fully configured. Please set SSH_PRIVATE_KEY, VPS_HOST, VPS_USER, and VPS_TARGET_DIR."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p ~/.ssh
|
||||||
|
echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
|
||||||
|
chmod 600 ~/.ssh/id_rsa
|
||||||
|
ssh-keyscan -H -t ed25519,rsa "$VPS_HOST" >> ~/.ssh/known_hosts
|
||||||
|
|
||||||
|
# Use SSH multiplexing for faster subsequent commands
|
||||||
|
SSH_OPTS=(-o ControlMaster=auto -o ControlPath=/tmp/ssh-%r@%h:%p -o ControlPersist=600 -o StrictHostKeyChecking=yes)
|
||||||
|
|
||||||
|
ssh "${SSH_OPTS[@]}" "$VPS_USER@$VPS_HOST" "mkdir -p $VPS_TARGET_DIR && mkdir -p $VPS_TARGET_DIR/infra/compose"
|
||||||
|
echo "$ENV_FILE_PRODUCTION" > .env.prod
|
||||||
|
scp "${SSH_OPTS[@]}" .env.prod "$VPS_USER@$VPS_HOST:$VPS_TARGET_DIR/.env"
|
||||||
|
|
||||||
|
echo "🔐 Logging in to GitHub Container Registry..."
|
||||||
|
printf '%s' "$GHCR_TOKEN" | ssh "${SSH_OPTS[@]}" "$VPS_USER@$VPS_HOST" "docker login ghcr.io -u '$GHCR_USERNAME' --password-stdin"
|
||||||
|
|
||||||
|
ssh "${SSH_OPTS[@]}" "$VPS_USER@$VPS_HOST" "export VPS_TARGET_DIR=$VPS_TARGET_DIR; bash -s" <<'EOF'
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$VPS_TARGET_DIR"
|
||||||
|
|
||||||
|
# Ensure shared network exists
|
||||||
|
docker network inspect app-shared-net >/dev/null 2>&1 || docker network create app-shared-net
|
||||||
|
|
||||||
|
echo "🔄 Synchronizing repository..."
|
||||||
|
if [ ! -d ".git" ]; then
|
||||||
|
echo "Initializing git repository..."
|
||||||
|
git init
|
||||||
|
git remote add origin https://github.com/asepharyana/asepharyana-hub.git
|
||||||
|
fi
|
||||||
|
git fetch origin main --depth=1 || true
|
||||||
|
|
||||||
|
# Detect changed files before resetting
|
||||||
|
ALL_COMPOSE_FILES="infra/compose/traefik.yml infra/compose/shared.yml infra/compose/scraper.yml infra/compose/elysia.yml infra/compose/react.yml infra/compose/rust-auth.yml"
|
||||||
|
TRAEFIK_DYNAMIC_DIR="infra/traefik/dynamic"
|
||||||
|
|
||||||
|
if git rev-parse HEAD >/dev/null 2>&1; then
|
||||||
|
BEFORE_REV=$(git rev-parse HEAD)
|
||||||
|
git reset --hard FETCH_HEAD
|
||||||
|
AFTER_REV=$(git rev-parse HEAD)
|
||||||
|
|
||||||
|
if [ "$BEFORE_REV" = "$AFTER_REV" ]; then
|
||||||
|
echo "ℹ️ No new commits detected. Using full file list for safety."
|
||||||
|
TARGET_COMPOSE=""
|
||||||
|
else
|
||||||
|
CHANGED=$(git diff --name-only "$BEFORE_REV" "$AFTER_REV" || true)
|
||||||
|
echo "📄 Changed files:"
|
||||||
|
echo "$CHANGED"
|
||||||
|
|
||||||
|
# Detect compose stack changes
|
||||||
|
CHANGED_COMPOSE=$(echo "$CHANGED" | grep '^infra/compose/.*\.yml$' || true)
|
||||||
|
TARGET_COMPOSE=""
|
||||||
|
for f in $CHANGED_COMPOSE; do
|
||||||
|
case " $ALL_COMPOSE_FILES " in
|
||||||
|
*" $f "*) TARGET_COMPOSE="$TARGET_COMPOSE $f" ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
TARGET_COMPOSE=$(printf '%s' "$TARGET_COMPOSE" | xargs || true)
|
||||||
|
if [ -n "$TARGET_COMPOSE" ]; then
|
||||||
|
echo "🎯 Detected compose stack changes in: $TARGET_COMPOSE"
|
||||||
|
else
|
||||||
|
echo "ℹ️ No stack compose files changed."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Detect Traefik dynamic config changes
|
||||||
|
CHANGED_TRAEFIK=$(echo "$CHANGED" | grep "^$TRAEFIK_DYNAMIC_DIR/" || true)
|
||||||
|
if [ -n "$CHANGED_TRAEFIK" ]; then
|
||||||
|
echo "🎯 Detected Traefik dynamic config changes:"
|
||||||
|
echo "$CHANGED_TRAEFIK"
|
||||||
|
RELOAD_TRAEFIK="true"
|
||||||
|
else
|
||||||
|
echo "ℹ️ No Traefik dynamic config changes."
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Detect infra file changes (Dockerfiles, config, traefik static)
|
||||||
|
CHANGED_INFRA=$(echo "$CHANGED" | grep '^infra/' | grep -v '^infra/compose/' || true)
|
||||||
|
if [ -n "$CHANGED_INFRA" ]; then
|
||||||
|
echo "📦 Detected other infra file changes:"
|
||||||
|
echo "$CHANGED_INFRA"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
git reset --hard FETCH_HEAD
|
||||||
|
TARGET_COMPOSE=""
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Submodule update removed for faster VPS deployments
|
||||||
|
|
||||||
|
if command -v "docker" >/dev/null 2>&1 && docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_CMD="docker compose"
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1; then
|
||||||
|
COMPOSE_CMD="docker-compose"
|
||||||
|
else
|
||||||
|
echo "❌ docker compose is not installed on the remote host."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Construct compose arguments
|
||||||
|
docker rm -f imphenbot-app || true
|
||||||
|
|
||||||
|
if [ -n "$TARGET_COMPOSE" ]; then
|
||||||
|
COMPOSE_ARGS=""
|
||||||
|
for f in $TARGET_COMPOSE; do
|
||||||
|
if [ -f "$f" ]; then
|
||||||
|
COMPOSE_ARGS="$COMPOSE_ARGS -f $f"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
UP_FLAGS="-d" # No --remove-orphans for selective updates to avoid killing other services
|
||||||
|
else
|
||||||
|
echo "🚀 Performing full deployment of all services..."
|
||||||
|
COMPOSE_ARGS=""
|
||||||
|
for f in $ALL_COMPOSE_FILES; do
|
||||||
|
COMPOSE_ARGS="$COMPOSE_ARGS -f $f"
|
||||||
|
done
|
||||||
|
UP_FLAGS="-d --remove-orphans"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "📥 Pulling images for target services..."
|
||||||
|
export DOCKER_CLI_EXPERIMENTAL=enabled
|
||||||
|
PULL_SUCCESS=false
|
||||||
|
# Retry pull up to 3 times to handle transient Docker attestation lease errors
|
||||||
|
for attempt in 1 2 3; do
|
||||||
|
echo "Pull attempt $attempt/3..."
|
||||||
|
if $COMPOSE_CMD $COMPOSE_ARGS --env-file .env pull; then
|
||||||
|
echo "✅ Pull succeeded on attempt $attempt"
|
||||||
|
PULL_SUCCESS=true
|
||||||
|
break
|
||||||
|
else
|
||||||
|
echo "⚠️ Pull attempt $attempt failed. Retrying in 5s..."
|
||||||
|
sleep 5
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$PULL_SUCCESS" != "true" ]; then
|
||||||
|
echo "❌ Failed to pull images after 3 attempts."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "🧹 Clearing Git locks..."
|
||||||
|
rm -f .git/shallow.lock || true
|
||||||
|
|
||||||
|
# Cooldown to allow daemon to settle (optional but kept for safety)
|
||||||
|
# echo "⏳ Waiting for Docker daemon to settle..."
|
||||||
|
# sleep 2
|
||||||
|
|
||||||
|
echo "🧹 Removing stale target containers by container_name..."
|
||||||
|
# Extract all explicitly defined container_names from compose files and remove them to prevent conflicts
|
||||||
|
if [ -n "$TARGET_COMPOSE" ]; then
|
||||||
|
for f in $TARGET_COMPOSE; do
|
||||||
|
if [ -f "$f" ]; then
|
||||||
|
grep "container_name:" "$f" | awk '{print $2}' | while read -r cname; do
|
||||||
|
docker rm -f "$cname" >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
else
|
||||||
|
for f in $ALL_COMPOSE_FILES; do
|
||||||
|
if [ -f "$f" ]; then
|
||||||
|
grep "container_name:" "$f" | awk '{print $2}' | while read -r cname; do
|
||||||
|
docker rm -f "$cname" >/dev/null 2>&1 || true
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "🆙 Starting services..."
|
||||||
|
echo "🔍 Debug: Current docker containers:"
|
||||||
|
docker ps -a
|
||||||
|
$COMPOSE_CMD $COMPOSE_ARGS --env-file .env up $UP_FLAGS
|
||||||
|
|
||||||
|
# ── Traefik reload ──
|
||||||
|
if [ "${RELOAD_TRAEFIK:-false}" = "true" ]; then
|
||||||
|
echo "🔄 Traefik dynamic config changed — reloading Traefik..."
|
||||||
|
# Traefik watches the dynamic config dir (providers.file.watch=true),
|
||||||
|
# but send SIGHUP as insurance
|
||||||
|
docker kill --signal HUP traefik 2>/dev/null || docker exec traefik kill -HUP 1 2>/dev/null || true
|
||||||
|
echo "✅ Traefik reload signal sent"
|
||||||
|
fi
|
||||||
|
EOF
|
||||||
|
|
||||||
|
# - name: Restart coolify-proxy (delay 1 min)
|
||||||
|
# env:
|
||||||
|
# SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
|
||||||
|
# VPS_HOST: ${{ secrets.VPS_HOST }}
|
||||||
|
# VPS_USER: ${{ secrets.VPS_USER }}
|
||||||
|
# run: |
|
||||||
|
# mkdir -p ~/.ssh
|
||||||
|
# echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
|
||||||
|
# chmod 600 ~/.ssh/id_rsa
|
||||||
|
# ssh-keyscan -H "$VPS_HOST" >> ~/.ssh/known_hosts
|
||||||
|
#
|
||||||
|
# echo "⏳ Waiting 60s for containers to settle before restarting proxy..."
|
||||||
|
# sleep 60
|
||||||
|
#
|
||||||
|
# echo "🔄 Restarting coolify-proxy..."
|
||||||
|
# ssh "${VPS_USER}@${VPS_HOST}" "docker restart coolify-proxy"
|
||||||
|
# echo "✅ coolify-proxy restarted."
|
||||||
@@ -0,0 +1,334 @@
|
|||||||
|
name: Build and Push Docker Images
|
||||||
|
|
||||||
|
on:
|
||||||
|
push:
|
||||||
|
branches:
|
||||||
|
- main
|
||||||
|
paths:
|
||||||
|
- 'apps/**'
|
||||||
|
- '.github/workflows/docker-build-push.yml'
|
||||||
|
- 'infra/**'
|
||||||
|
- '!infra/compose/**'
|
||||||
|
repository_dispatch:
|
||||||
|
types: [submodule-updated]
|
||||||
|
workflow_dispatch:
|
||||||
|
|
||||||
|
concurrency:
|
||||||
|
group: ${{ github.workflow }}-${{ github.ref }}
|
||||||
|
cancel-in-progress: false
|
||||||
|
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
|
||||||
|
env:
|
||||||
|
REGISTRY: ghcr.io
|
||||||
|
IMAGE_NAME_PREFIX: asepharyana/asepharyana-hub
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
# ──────────────────────────────────────────────
|
||||||
|
# Phase 1: Detect which services have changed
|
||||||
|
# ──────────────────────────────────────────────
|
||||||
|
changes:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 10
|
||||||
|
outputs:
|
||||||
|
matrix: ${{ steps.set-matrix.outputs.matrix }}
|
||||||
|
scraper-api: ${{ steps.filter.outputs['scraper-api'] == 'true' || steps.dispatch.outputs['scraper-api'] == 'true' || github.event_name == 'workflow_dispatch' }}
|
||||||
|
elysia-api: ${{ steps.filter.outputs['elysia-api'] == 'true' || steps.dispatch.outputs['elysia-api'] == 'true' || github.event_name == 'workflow_dispatch' }}
|
||||||
|
react-web: ${{ steps.filter.outputs['react-web'] == 'true' || steps.dispatch.outputs['react-web'] == 'true' || github.event_name == 'workflow_dispatch' }}
|
||||||
|
rust-auth: ${{ steps.filter.outputs['rust-auth'] == 'true' || steps.dispatch.outputs['rust-auth'] == 'true' || github.event_name == 'workflow_dispatch' }}
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v6
|
||||||
|
with:
|
||||||
|
submodules: false
|
||||||
|
fetch-depth: 2
|
||||||
|
|
||||||
|
- name: Detect changed services
|
||||||
|
id: filter
|
||||||
|
if: github.event_name == 'push'
|
||||||
|
env:
|
||||||
|
BEFORE: ${{ github.event.before }}
|
||||||
|
AFTER: ${{ github.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
if [ -z "${BEFORE:-}" ] || [[ "$BEFORE" =~ ^0+$ ]]; then
|
||||||
|
CHANGED_FILES=$(git ls-files)
|
||||||
|
else
|
||||||
|
git fetch --no-tags --depth=2 origin "$BEFORE" || true
|
||||||
|
CHANGED_FILES=$(git diff --name-only "$BEFORE" "$AFTER")
|
||||||
|
fi
|
||||||
|
|
||||||
|
changed() {
|
||||||
|
printf '%s\n' "$CHANGED_FILES" | grep -Eq "$1" && echo true || echo false
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "scraper-api=$(changed '^(apps/scraper(/|$)|\.github/workflows/docker-build-push\.yml$|infra/docker/scraper\.Dockerfile$)')" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "elysia-api=$(changed '^(apps/elysia(/|$)|\.github/workflows/docker-build-push\.yml$|infra/docker/elysia\.Dockerfile$)')" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "react-web=$(changed '^(apps/react(/|$)|\.github/workflows/docker-build-push\.yml$|infra/docker/react\.Dockerfile$)')" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "rust-auth=$(changed '^(apps/rust-auth(/|$)|infra/docker/rust\.Dockerfile$|\.github/workflows/docker-build-push\.yml$|\.gitmodules$)')" >> "$GITHUB_OUTPUT"
|
||||||
|
|
||||||
|
- name: Parse repository_dispatch payload
|
||||||
|
id: dispatch
|
||||||
|
if: github.event_name == 'repository_dispatch'
|
||||||
|
env:
|
||||||
|
SERVICE: ${{ github.event.client_payload.service }}
|
||||||
|
SHA: ${{ github.event.client_payload.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
if [ -z "${SERVICE:-}" ]; then
|
||||||
|
echo "::error::repository_dispatch payload missing service"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -z "${SHA:-}" ]; then
|
||||||
|
echo "::error::repository_dispatch payload missing sha"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
case "$SERVICE" in
|
||||||
|
scraper-api|elysia-api|react-web|rust-auth) ;;
|
||||||
|
*)
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if ! [[ "$SHA" =~ ^[0-9a-fA-F]{40}$ ]]; then echo "::error::Invalid sha '$SHA'. Expected 40 hex characters"; exit 1; fi
|
||||||
|
|
||||||
|
SERVICES=(scraper-api elysia-api react-web rust-auth)
|
||||||
|
for svc in "${SERVICES[@]}"; do
|
||||||
|
if [ "$SERVICE" = "$svc" ]; then
|
||||||
|
echo "${svc}=true" >> "$GITHUB_OUTPUT"
|
||||||
|
else
|
||||||
|
echo "${svc}=false" >> "$GITHUB_OUTPUT"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
- name: Set matrix
|
||||||
|
id: set-matrix
|
||||||
|
run: |
|
||||||
|
SERVICES=()
|
||||||
|
add_service() {
|
||||||
|
SERVICES+=("{\"id\":\"$1\",\"target\":\"$2\",\"path\":\"$3\"}")
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ "${{ steps.filter.outputs['scraper-api'] == 'true' || steps.dispatch.outputs['scraper-api'] == 'true' || github.event_name == 'workflow_dispatch' }}" == "true" ]; then add_service "scraper-api" "docker-scraper" "apps/scraper"; fi
|
||||||
|
if [ "${{ steps.filter.outputs['elysia-api'] == 'true' || steps.dispatch.outputs['elysia-api'] == 'true' || github.event_name == 'workflow_dispatch' }}" == "true" ]; then add_service "elysia-api" "docker-elysia" "apps/elysia"; fi
|
||||||
|
if [ "${{ steps.filter.outputs['react-web'] == 'true' || steps.dispatch.outputs['react-web'] == 'true' || github.event_name == 'workflow_dispatch' }}" == "true" ]; then add_service "react-web" "docker-react" "apps/react"; fi
|
||||||
|
if [ "${{ steps.filter.outputs['rust-auth'] == 'true' || steps.dispatch.outputs['rust-auth'] == 'true' || github.event_name == 'workflow_dispatch' }}" == "true" ]; then add_service "rust-auth" "docker-rust-auth" "apps/rust-auth"; fi
|
||||||
|
|
||||||
|
JSON_ARRAY="[$(IFS=,; echo "${SERVICES[*]}")]"
|
||||||
|
echo "matrix=$JSON_ARRAY" >> $GITHUB_OUTPUT
|
||||||
|
|
||||||
|
wait-submodule-ref:
|
||||||
|
needs: [changes]
|
||||||
|
if: github.event_name == 'repository_dispatch'
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 10
|
||||||
|
steps:
|
||||||
|
- name: Wait for submodule ref
|
||||||
|
env:
|
||||||
|
SERVICE: ${{ github.event.client_payload.service }}
|
||||||
|
SHA: ${{ github.event.client_payload.sha }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
case "$SERVICE" in
|
||||||
|
"scraper-api") REPO="https://github.com/asepharyana/asepharyana-hub-scraper.git" ;;
|
||||||
|
"elysia-api") REPO="https://github.com/asepharyana/asepharyana-hub-elysia.git" ;;
|
||||||
|
"react-web") REPO="https://github.com/asepharyana/asepharyana-hub-react.git" ;;
|
||||||
|
"rust-auth") REPO="https://github.com/asepharyana/asepharyana-hub-rust-auth.git" ;;
|
||||||
|
*)
|
||||||
|
echo "::error::Unsupported service '$SERVICE'"
|
||||||
|
exit 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
echo "Waiting for $SERVICE commit $SHA in $REPO"
|
||||||
|
TMPDIR=$(mktemp -d)
|
||||||
|
git init "$TMPDIR/probe" >/dev/null
|
||||||
|
git -C "$TMPDIR/probe" remote add origin "$REPO"
|
||||||
|
for attempt in {1..30}; do
|
||||||
|
if git -C "$TMPDIR/probe" fetch --depth=1 origin "$SHA" >/dev/null 2>&1; then
|
||||||
|
echo "Submodule commit $SHA is fetchable for $SERVICE"
|
||||||
|
rm -rf "$TMPDIR"
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "Attempt $attempt/30: $SHA not fetchable yet; waiting 10s"
|
||||||
|
sleep 10
|
||||||
|
done
|
||||||
|
rm -rf "$TMPDIR"
|
||||||
|
|
||||||
|
echo "::error::Submodule commit $SHA for $SERVICE was not fetchable after 300s"
|
||||||
|
exit 1
|
||||||
|
|
||||||
|
# ─────────────────────────────────────────────────
|
||||||
|
# Phase 2: Build and Push Images (Matrix)
|
||||||
|
# ─────────────────────────────────────────────────
|
||||||
|
build:
|
||||||
|
needs: [changes, wait-submodule-ref]
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 30
|
||||||
|
strategy:
|
||||||
|
fail-fast: false
|
||||||
|
matrix:
|
||||||
|
include: ${{ fromJson(needs.changes.outputs.matrix) }}
|
||||||
|
if: |
|
||||||
|
always() &&
|
||||||
|
needs.changes.result == 'success' &&
|
||||||
|
(needs.wait-submodule-ref.result == 'success' || needs.wait-submodule-ref.result == 'skipped') &&
|
||||||
|
needs.changes.outputs.matrix != '[]'
|
||||||
|
permissions:
|
||||||
|
contents: read
|
||||||
|
packages: write
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v6
|
||||||
|
with:
|
||||||
|
submodules: false
|
||||||
|
|
||||||
|
- name: Sync submodule locally
|
||||||
|
env:
|
||||||
|
EVENT_NAME: ${{ github.event_name }}
|
||||||
|
DISPATCH_SHA: ${{ github.event.client_payload.sha }}
|
||||||
|
SUBMODULE_PATH: ${{ matrix.path }}
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
git submodule update --init --recursive "$SUBMODULE_PATH"
|
||||||
|
if [ "$EVENT_NAME" = "repository_dispatch" ] && [ -n "${DISPATCH_SHA:-}" ]; then
|
||||||
|
cd "$SUBMODULE_PATH"
|
||||||
|
git fetch origin "$DISPATCH_SHA"
|
||||||
|
git checkout "$DISPATCH_SHA"
|
||||||
|
fi
|
||||||
|
|
||||||
|
- uses: docker/login-action@v4
|
||||||
|
with:
|
||||||
|
registry: ${{ env.REGISTRY }}
|
||||||
|
username: ${{ github.actor }}
|
||||||
|
password: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
|
||||||
|
- name: Set up Docker Buildx
|
||||||
|
uses: docker/setup-buildx-action@v4
|
||||||
|
|
||||||
|
- name: Docker metadata
|
||||||
|
id: meta
|
||||||
|
run: |
|
||||||
|
SVC_NAME="${{ matrix.id }}"
|
||||||
|
SHORT=${GITHUB_SHA:0:7}
|
||||||
|
echo "image=${REGISTRY}/${IMAGE_NAME_PREFIX}/${SVC_NAME}" >> $GITHUB_OUTPUT
|
||||||
|
echo "tags=${REGISTRY}/${IMAGE_NAME_PREFIX}/${SVC_NAME}:latest,${REGISTRY}/${IMAGE_NAME_PREFIX}/${SVC_NAME}:sha-${SHORT}" >> $GITHUB_OUTPUT
|
||||||
|
echo "cache-registry=${REGISTRY}/${IMAGE_NAME_PREFIX}/${SVC_NAME}:buildcache" >> $GITHUB_OUTPUT
|
||||||
|
case "$SVC_NAME" in
|
||||||
|
"scraper-api") echo "dockerfile=infra/docker/scraper.Dockerfile" >> $GITHUB_OUTPUT ;;
|
||||||
|
"elysia-api") echo "dockerfile=infra/docker/elysia.Dockerfile" >> $GITHUB_OUTPUT ;;
|
||||||
|
"react-web") echo "dockerfile=infra/docker/react.Dockerfile" >> $GITHUB_OUTPUT ;;
|
||||||
|
"rust-auth") echo "dockerfile=infra/docker/rust.Dockerfile" >> $GITHUB_OUTPUT ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
- name: Build and Push Docker image
|
||||||
|
uses: docker/build-push-action@v7
|
||||||
|
with:
|
||||||
|
context: .
|
||||||
|
file: ${{ steps.meta.outputs.dockerfile }}
|
||||||
|
push: true
|
||||||
|
tags: ${{ steps.meta.outputs.tags }}
|
||||||
|
build-args: |
|
||||||
|
COMMIT_COUNT=${{ env.NR_COMMIT_COUNT || github.run_number }}
|
||||||
|
COMMIT_SHA=${{ env.NR_COMMIT_SHA || github.sha }}
|
||||||
|
cache-from: type=registry,ref=${{ steps.meta.outputs['cache-registry'] }}
|
||||||
|
cache-to: type=registry,ref=${{ steps.meta.outputs['cache-registry'] }},mode=max
|
||||||
|
|
||||||
|
# ──────────────────────────────────────────────
|
||||||
|
# Phase 3: Update Manifests and Submodule Refs
|
||||||
|
# ──────────────────────────────────────────────
|
||||||
|
update-manifest:
|
||||||
|
needs: [changes, wait-submodule-ref, build]
|
||||||
|
if: |
|
||||||
|
always() &&
|
||||||
|
needs.changes.result == 'success' &&
|
||||||
|
(needs.wait-submodule-ref.result == 'success' || needs.wait-submodule-ref.result == 'skipped') &&
|
||||||
|
(needs.build.result == 'success' || needs.build.result == 'skipped')
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 10
|
||||||
|
permissions:
|
||||||
|
contents: write
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v6
|
||||||
|
with:
|
||||||
|
submodules: false
|
||||||
|
token: ${{ secrets.GITHUB_TOKEN }}
|
||||||
|
ref: main
|
||||||
|
|
||||||
|
- name: Update tags and submodules
|
||||||
|
run: |
|
||||||
|
SHORT_SHA=${GITHUB_SHA:0:7}
|
||||||
|
TAG="sha-$SHORT_SHA"
|
||||||
|
CHANGED=false
|
||||||
|
|
||||||
|
declare -A SERVICES
|
||||||
|
SERVICES["scraper-api"]="scraper.yml"
|
||||||
|
SERVICES["elysia-api"]="elysia.yml"
|
||||||
|
SERVICES["react-web"]="react.yml"
|
||||||
|
SERVICES["rust-auth"]="rust-auth.yml"
|
||||||
|
|
||||||
|
declare -A PATHS
|
||||||
|
PATHS["scraper-api"]="apps/scraper"
|
||||||
|
PATHS["elysia-api"]="apps/elysia"
|
||||||
|
PATHS["react-web"]="apps/react"
|
||||||
|
PATHS["rust-auth"]="apps/rust-auth"
|
||||||
|
|
||||||
|
# Use git config for possible commits
|
||||||
|
git config --local user.email "action@github.com"
|
||||||
|
git config --local user.name "GitHub Action"
|
||||||
|
|
||||||
|
for id in "${!SERVICES[@]}"; do
|
||||||
|
SHOULD_HAVE_RUN=false
|
||||||
|
if [ "${{ needs.changes.outputs['scraper-api'] }}" == "true" ] && [ "$id" == "scraper-api" ]; then SHOULD_HAVE_RUN=true; fi
|
||||||
|
if [ "${{ needs.changes.outputs['elysia-api'] }}" == "true" ] && [ "$id" == "elysia-api" ]; then SHOULD_HAVE_RUN=true; fi
|
||||||
|
if [ "${{ needs.changes.outputs['react-web'] }}" == "true" ] && [ "$id" == "react-web" ]; then SHOULD_HAVE_RUN=true; fi
|
||||||
|
if [ "${{ needs.changes.outputs['rust-auth'] }}" == "true" ] && [ "$id" == "rust-auth" ]; then SHOULD_HAVE_RUN=true; fi
|
||||||
|
|
||||||
|
if [ "${{ github.event_name }}" == "workflow_dispatch" ]; then SHOULD_HAVE_RUN=true; fi
|
||||||
|
|
||||||
|
if [ "$SHOULD_HAVE_RUN" == "true" ]; then
|
||||||
|
COMPOSE_FILE="infra/compose/${SERVICES[$id]}"
|
||||||
|
if [ -f "$COMPOSE_FILE" ]; then
|
||||||
|
echo "Updating $COMPOSE_FILE to $TAG"
|
||||||
|
sed -i "s|image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_PREFIX }}/$id:.*|image: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME_PREFIX }}/$id:$TAG|g" "$COMPOSE_FILE"
|
||||||
|
git add "$COMPOSE_FILE"
|
||||||
|
CHANGED=true
|
||||||
|
fi
|
||||||
|
|
||||||
|
# If it's a repository_dispatch for this specific service, update its submodule pointer
|
||||||
|
if [ "${{ github.event_name }}" == "repository_dispatch" ] && [ "${{ github.event.client_payload.service }}" == "$id" ]; then
|
||||||
|
SHA_DISPATCH="${{ github.event.client_payload.sha }}"
|
||||||
|
SUB_PATH="${PATHS[$id]}"
|
||||||
|
if [ -n "$SHA_DISPATCH" ]; then
|
||||||
|
echo "Updating submodule $SUB_PATH to $SHA_DISPATCH"
|
||||||
|
git submodule update --init "$SUB_PATH"
|
||||||
|
git -C "$SUB_PATH" fetch origin "$SHA_DISPATCH"
|
||||||
|
git -C "$SUB_PATH" checkout "$SHA_DISPATCH"
|
||||||
|
git add "$SUB_PATH"
|
||||||
|
CHANGED=true
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
if [ "$CHANGED" == "true" ]; then
|
||||||
|
git commit -m "chore: update manifests and submodules [skip ci]"
|
||||||
|
|
||||||
|
for attempt in {1..3}; do
|
||||||
|
if git pull --rebase origin main && git push origin main; then
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
echo "Manifest push attempt $attempt/3 failed; retrying"
|
||||||
|
git rebase --abort || true
|
||||||
|
git pull --rebase origin main || true
|
||||||
|
sleep 5
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "::error::Failed to push manifest update after 3 attempts"
|
||||||
|
exit 1
|
||||||
|
else
|
||||||
|
echo "No changes detected."
|
||||||
|
fi
|
||||||
@@ -1,20 +0,0 @@
|
|||||||
name: Publish to FlakeHub
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main, master]
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
flakehub-publish:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
permissions:
|
|
||||||
id-token: write
|
|
||||||
contents: read
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v6
|
|
||||||
- uses: DeterminateSystems/determinate-nix-action@main
|
|
||||||
- uses: DeterminateSystems/flakehub-push@main
|
|
||||||
with:
|
|
||||||
visibility: public
|
|
||||||
rolling: true
|
|
||||||
@@ -5,24 +5,26 @@ on:
|
|||||||
pull_request:
|
pull_request:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
paths:
|
paths:
|
||||||
|
- 'apps/*/src/**/*.ts'
|
||||||
|
- 'apps/*/src/**/*.tsx'
|
||||||
- 'biome.json'
|
- 'biome.json'
|
||||||
- '*.json'
|
|
||||||
- '*.js'
|
|
||||||
push:
|
push:
|
||||||
branches: [main]
|
branches: [main]
|
||||||
paths:
|
paths:
|
||||||
|
- 'apps/*/src/**/*.ts'
|
||||||
|
- 'apps/*/src/**/*.tsx'
|
||||||
- 'biome.json'
|
- 'biome.json'
|
||||||
- '*.json'
|
|
||||||
- '*.js'
|
|
||||||
|
|
||||||
jobs:
|
jobs:
|
||||||
biome:
|
biome:
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
timeout-minutes: 10
|
timeout-minutes: 10
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v7
|
- uses: actions/checkout@v6
|
||||||
with:
|
with:
|
||||||
submodules: recursive
|
submodules: recursive
|
||||||
- uses: oven-sh/setup-bun@v2
|
- uses: actions/setup-node@v4
|
||||||
- run: bun install --frozen-lockfile
|
with:
|
||||||
- run: bun run ci
|
node-version: 22
|
||||||
|
- run: npm install -g @biomejs/biome
|
||||||
|
- run: biome ci . --no-errors-on-unmatched
|
||||||
|
|||||||
@@ -1,84 +0,0 @@
|
|||||||
name: Nix Build & Deploy — All Services
|
|
||||||
|
|
||||||
on:
|
|
||||||
# No `paths` filter: GitHub's path filters do not match submodule gitlink
|
|
||||||
# changes, so a submodule pointer update (e.g. from update-submodule.yml)
|
|
||||||
# would never trigger this deploy. Run on every push to main instead.
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
concurrency:
|
|
||||||
group: nix-deploy
|
|
||||||
cancel-in-progress: false
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
id-token: write
|
|
||||||
|
|
||||||
env:
|
|
||||||
VPS_HOST: ${{ secrets.VPS_HOST }}
|
|
||||||
VPS_USER: ${{ secrets.VPS_USER }}
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build-and-deploy:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
matrix:
|
|
||||||
service: [hub, scraper, tools-gateway, tools-workers, tools-frontend, llm-api]
|
|
||||||
|
|
||||||
steps:
|
|
||||||
- name: Checkout
|
|
||||||
uses: actions/checkout@v7
|
|
||||||
with:
|
|
||||||
submodules: recursive
|
|
||||||
fetch-depth: 0
|
|
||||||
|
|
||||||
- name: Install Nix
|
|
||||||
uses: DeterminateSystems/nix-installer-action@v22
|
|
||||||
with:
|
|
||||||
determinate: false
|
|
||||||
extra-conf: |
|
|
||||||
sandbox = false
|
|
||||||
accept-flake-config = true
|
|
||||||
|
|
||||||
- name: Cache Nix
|
|
||||||
uses: DeterminateSystems/magic-nix-cache-action@v14
|
|
||||||
with:
|
|
||||||
use-flakehub: false
|
|
||||||
|
|
||||||
- name: Build ${{ matrix.service }}
|
|
||||||
id: build
|
|
||||||
run: |
|
|
||||||
nix build .#${{ matrix.service }} --impure --option sandbox false --print-build-logs
|
|
||||||
STORE_PATH=$(readlink result)
|
|
||||||
echo "store-path=$STORE_PATH" >> "$GITHUB_OUTPUT"
|
|
||||||
echo "✅ ${{ matrix.service }}: $STORE_PATH"
|
|
||||||
|
|
||||||
- name: Setup SSH key
|
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
env:
|
|
||||||
SSH_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
|
|
||||||
run: |
|
|
||||||
mkdir -p ~/.ssh
|
|
||||||
echo "$SSH_KEY" > ~/.ssh/id_ed25519
|
|
||||||
chmod 600 ~/.ssh/id_ed25519
|
|
||||||
sed -i 's/\r$//' ~/.ssh/id_ed25519
|
|
||||||
ssh-keygen -y -f ~/.ssh/id_ed25519 >/dev/null 2>&1 || { echo "SSH key invalid"; exit 1; }
|
|
||||||
ssh-keyscan -H "$VPS_HOST" >> ~/.ssh/known_hosts 2>/dev/null
|
|
||||||
|
|
||||||
- name: Deploy ${{ matrix.service }} to VPS
|
|
||||||
if: github.ref == 'refs/heads/main'
|
|
||||||
run: |
|
|
||||||
STORE_PATH="${{ steps.build.outputs.store-path }}"
|
|
||||||
echo "=== Copying ${{ matrix.service }}: $STORE_PATH ==="
|
|
||||||
nix copy --to "ssh://$VPS_USER@$VPS_HOST" "$STORE_PATH"
|
|
||||||
|
|
||||||
echo "=== Updating profile ==="
|
|
||||||
ssh "$VPS_USER@$VPS_HOST" "sudo /nix/var/nix/profiles/default/bin/nix-env --profile /nix/var/nix/profiles/${{ matrix.service }} --set '$STORE_PATH'"
|
|
||||||
|
|
||||||
echo "=== Restarting service ==="
|
|
||||||
ssh "$VPS_USER@$VPS_HOST" "sudo systemctl restart ${{ matrix.service }}" || echo " ⚠️ restart failed (may not be enabled yet)"
|
|
||||||
|
|
||||||
echo "✅ ${{ matrix.service }} deployed"
|
|
||||||
@@ -13,18 +13,8 @@ jobs:
|
|||||||
permissions:
|
permissions:
|
||||||
security-events: write
|
security-events: write
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v7
|
- uses: actions/checkout@v6
|
||||||
|
- uses: github/codeql-action/init@v3
|
||||||
with:
|
with:
|
||||||
fetch-depth: 2
|
languages: javascript-typescript, rust
|
||||||
submodules: recursive
|
- uses: github/codeql-action/analyze@v3
|
||||||
|
|
||||||
- uses: github/codeql-action/init@v4
|
|
||||||
with:
|
|
||||||
languages: rust
|
|
||||||
|
|
||||||
- name: Build Rust projects for CodeQL analysis
|
|
||||||
run: |
|
|
||||||
cargo build --manifest-path apps/scraper/Cargo.toml
|
|
||||||
cargo build --manifest-path apps/llm-api/Cargo.toml
|
|
||||||
|
|
||||||
- uses: github/codeql-action/analyze@v4
|
|
||||||
|
|||||||
@@ -0,0 +1,36 @@
|
|||||||
|
name: TypeCheck
|
||||||
|
|
||||||
|
on:
|
||||||
|
pull_request:
|
||||||
|
branches: [main]
|
||||||
|
paths:
|
||||||
|
- 'apps/react/src/**/*.ts'
|
||||||
|
- 'apps/react/src/**/*.tsx'
|
||||||
|
- 'apps/elysia/src/**/*.ts'
|
||||||
|
- 'apps/elysia/tsconfig.json'
|
||||||
|
- 'tsconfig.base.json'
|
||||||
|
|
||||||
|
jobs:
|
||||||
|
typecheck:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
timeout-minutes: 10
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v6
|
||||||
|
with:
|
||||||
|
submodules: recursive
|
||||||
|
|
||||||
|
- uses: oven/setup-bun@v2
|
||||||
|
with:
|
||||||
|
bun-version: latest
|
||||||
|
|
||||||
|
- name: TypeCheck apps/react
|
||||||
|
working-directory: apps/react
|
||||||
|
run: |
|
||||||
|
bun install
|
||||||
|
npx tsc --noEmit
|
||||||
|
|
||||||
|
- name: TypeCheck apps/elysia
|
||||||
|
working-directory: apps/elysia
|
||||||
|
run: |
|
||||||
|
bun install
|
||||||
|
bun run typecheck
|
||||||
@@ -11,7 +11,7 @@ jobs:
|
|||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
timeout-minutes: 10
|
timeout-minutes: 10
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v7
|
- uses: actions/checkout@v6
|
||||||
|
|
||||||
- name: Validate payload
|
- name: Validate payload
|
||||||
env:
|
env:
|
||||||
@@ -36,7 +36,7 @@ jobs:
|
|||||||
fi
|
fi
|
||||||
|
|
||||||
case "$SERVICE" in
|
case "$SERVICE" in
|
||||||
scraper-api|hub|llm-api|tools) ;;
|
scraper-api|elysia-api|react-web|rust-auth) ;;
|
||||||
*)
|
*)
|
||||||
echo "::error::Unsupported service '$SERVICE'"
|
echo "::error::Unsupported service '$SERVICE'"
|
||||||
exit 1
|
exit 1
|
||||||
@@ -52,20 +52,14 @@ jobs:
|
|||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
# Map service name to submodule path
|
echo "Updating ${SERVICE} to ${SHA}"
|
||||||
case "$SERVICE" in
|
git submodule update --init "apps/${SERVICE}"
|
||||||
scraper-api) SUBMODULE_PATH="apps/scraper" ;;
|
cd "apps/${SERVICE}"
|
||||||
llm-api) SUBMODULE_PATH="apps/llm-api" ;;
|
# full fetch so we get tree objects for the target SHA
|
||||||
*) SUBMODULE_PATH="apps/${SERVICE}" ;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
echo "Updating ${SUBMODULE_PATH} to ${SHA}"
|
|
||||||
git submodule update --init "${SUBMODULE_PATH}"
|
|
||||||
cd "${SUBMODULE_PATH}"
|
|
||||||
git fetch --depth=1 origin master 2>/dev/null || git fetch --depth=1 origin main
|
git fetch --depth=1 origin master 2>/dev/null || git fetch --depth=1 origin main
|
||||||
git checkout "${SHA}"
|
git checkout "${SHA}"
|
||||||
cd "${GITHUB_WORKSPACE}"
|
cd "${GITHUB_WORKSPACE}"
|
||||||
git add "${SUBMODULE_PATH}"
|
git add "apps/${SERVICE}"
|
||||||
git diff --cached --quiet && exit 0
|
git diff --cached --quiet && exit 0
|
||||||
|
|
||||||
git config user.name "monrepo-bot"
|
git config user.name "monrepo-bot"
|
||||||
|
|||||||
+3
-6
@@ -1,3 +1,4 @@
|
|||||||
|
apps/gmw/
|
||||||
# See https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files for more about ignoring files.
|
# See https://docs.github.com/en/get-started/getting-started-with-git/ignoring-files for more about ignoring files.
|
||||||
|
|
||||||
# compiled output
|
# compiled output
|
||||||
@@ -11,7 +12,6 @@ node_modules
|
|||||||
.turbo/
|
.turbo/
|
||||||
# IDEs and editors
|
# IDEs and editors
|
||||||
/.idea
|
/.idea
|
||||||
.serena/
|
|
||||||
.project
|
.project
|
||||||
.classpath
|
.classpath
|
||||||
.c9/
|
.c9/
|
||||||
@@ -35,18 +35,16 @@ npm-debug.log
|
|||||||
yarn-error.log
|
yarn-error.log
|
||||||
testem.log
|
testem.log
|
||||||
/typings
|
/typings
|
||||||
.playwright-mcp/
|
|
||||||
|
|
||||||
# System Files
|
# System Files
|
||||||
.DS_Store
|
.DS_Store
|
||||||
Thumbs.db
|
Thumbs.db
|
||||||
.claude/*
|
.claude
|
||||||
!.claude/skills/
|
|
||||||
!.claude/settings.json
|
|
||||||
# Next.js
|
# Next.js
|
||||||
.next
|
.next
|
||||||
out
|
out
|
||||||
**/.codegraph/**
|
**/.codegraph/**
|
||||||
|
**/.claude/**
|
||||||
test-output
|
test-output
|
||||||
**/**.env
|
**/**.env
|
||||||
**/**.env.**
|
**/**.env.**
|
||||||
@@ -64,4 +62,3 @@ docs/todo.md
|
|||||||
**/vendor/
|
**/vendor/
|
||||||
|
|
||||||
.codegraph/
|
.codegraph/
|
||||||
result
|
|
||||||
|
|||||||
+9
-12
@@ -1,15 +1,12 @@
|
|||||||
|
[submodule "apps/elysia"]
|
||||||
|
path = apps/elysia
|
||||||
|
url = https://github.com/asepharyana/asepharyana-hub-elysia.git
|
||||||
|
[submodule "apps/react"]
|
||||||
|
path = apps/react
|
||||||
|
url = https://github.com/asepharyana/asepharyana-hub-react.git
|
||||||
[submodule "apps/scraper"]
|
[submodule "apps/scraper"]
|
||||||
path = apps/scraper
|
path = apps/scraper
|
||||||
url = https://github.com/asepharyana/asepharyana-hub-scraper.git
|
url = https://github.com/asepharyana/asepharyana-hub-scraper.git
|
||||||
[submodule "apps/hub"]
|
[submodule "apps/rust-auth"]
|
||||||
path = apps/hub
|
path = apps/rust-auth
|
||||||
url = https://github.com/asepharyana/asepharyana-hub-hub.git
|
url = https://github.com/asepharyana/asepharyana-hub-rust-auth.git
|
||||||
[submodule "apps/tools"]
|
|
||||||
path = apps/tools
|
|
||||||
url = https://github.com/asepharyana/asepharyana-hub-tools.git
|
|
||||||
[submodule "plugins/hub-guide"]
|
|
||||||
path = plugins/hub-guide
|
|
||||||
url = https://github.com/asepharyana/asepharyana-hub-guide.git
|
|
||||||
[submodule "apps/llm-api"]
|
|
||||||
path = apps/llm-api
|
|
||||||
url = https://github.com/asepharyana/asepharyana-hub-llm-api.git
|
|
||||||
|
|||||||
@@ -1,65 +0,0 @@
|
|||||||
---
|
|
||||||
name: commit-convention
|
|
||||||
description: Commit message convention — type(scope): description for Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Commit Convention — Asepharyana Hub
|
|
||||||
|
|
||||||
## Format
|
|
||||||
|
|
||||||
```
|
|
||||||
<type>(<scope>): <description>
|
|
||||||
|
|
||||||
[optional body]
|
|
||||||
|
|
||||||
[optional footer]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Types
|
|
||||||
|
|
||||||
| Type | Usage |
|
|
||||||
| ---------- | ------------------------------------ |
|
|
||||||
| `feat` | Fitur baru |
|
|
||||||
| `fix` | Bug fix |
|
|
||||||
| `chore` | Maintenance, config, tooling |
|
|
||||||
| `docs` | Dokumentasi |
|
|
||||||
| `refactor` | Perubahan kode tanpa fungsional baru |
|
|
||||||
| `test` | Nambah/update test |
|
|
||||||
| `ci` | CI/CD workflows |
|
|
||||||
| `perf` | Optimasi performa |
|
|
||||||
| `style` | Formatting (tanda kutip, dll) |
|
|
||||||
|
|
||||||
## Scopes
|
|
||||||
|
|
||||||
| Scope | Area |
|
|
||||||
| ------------- | --------------------------------- |
|
|
||||||
| `scraper` | apps/scraper submodule |
|
|
||||||
| `infra` | infra/ (compose, traefik, docker) |
|
|
||||||
| `ci` | .github/workflows/ |
|
|
||||||
| `dapr` | Dapr config & sidecar |
|
|
||||||
| `nats` | NATS message bus |
|
|
||||||
| `docs` | Dokumentasi |
|
|
||||||
| `deps` | Dependencies |
|
|
||||||
| `scripts` | Utility scripts |
|
|
||||||
| `root` | Root config files |
|
|
||||||
|
|
||||||
## Contoh
|
|
||||||
|
|
||||||
```
|
|
||||||
feat(scraper): add anime detail caching via Dapr pubsub
|
|
||||||
fix(infra): correct NATS CLI flags for JetStream
|
|
||||||
chore(deps): update biome to v2.5.3
|
|
||||||
docs(infra): add deployment order for Dapr services
|
|
||||||
ci(deploy): add nats.yml to ALL_COMPOSE_FILES
|
|
||||||
refactor(scraper): migrate EventBus from tokio broadcast to Dapr pubsub
|
|
||||||
```
|
|
||||||
|
|
||||||
## Aturan
|
|
||||||
|
|
||||||
1. **Wajib** menyertakan scope dalam tanda kurung
|
|
||||||
2. **Wajib** `Co-Authored-By` untuk commit yang digenerate AI
|
|
||||||
3. **Gunakan imperative mood**: "add" bukan "added" / "adds"
|
|
||||||
4. **Jangan capitalize** type: `feat:` bukan `Feat:`
|
|
||||||
5. **No period** di akhir subject baris
|
|
||||||
6. Body explain **why** dan **what**, bukan **how**
|
|
||||||
7. Refer issue dengan `Closes #123` atau `Fixes #123` di footer
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
---
|
|
||||||
name: deploy-workflow
|
|
||||||
description: CI/CD pipeline, Docker build patterns, manual deploy steps, and troubleshooting for Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Deploy & Workflow — Asepharyana Hub
|
|
||||||
|
|
||||||
## CI/CD Pipeline
|
|
||||||
|
|
||||||
### Build Pipeline (`docker-build-push.yml`)
|
|
||||||
Trigger: push ke `main` yang touch `apps/**`, `infra/**`, `infra/docker/**`
|
|
||||||
|
|
||||||
1. **changes** — detect service mana yg berubah via git diff
|
|
||||||
2. **wait-submodule-ref** — (repository_dispatch only) tunggu SHA commit fetchable
|
|
||||||
3. **build** — matrix build per service, push ke GHCR (`sha-<short>` + `latest`)
|
|
||||||
4. **update-manifest** — update image tag di compose file, commit + push
|
|
||||||
|
|
||||||
### Deploy Pipeline (`deploy-docker.yml`)
|
|
||||||
Trigger: build selesai, atau push ke `main` touch `infra/**`
|
|
||||||
|
|
||||||
1. SSH ke `orangevps` (via `secrets.VPS_HOST`)
|
|
||||||
2. Sync repo (`git fetch --depth=1 + reset`)
|
|
||||||
3. Login ke GHCR
|
|
||||||
4. Deteksi compose file yg berubah
|
|
||||||
5. Pull images + restart container selektif
|
|
||||||
|
|
||||||
### Secrets Required
|
|
||||||
| Secret | Untuk |
|
|
||||||
|--------|-------|
|
|
||||||
| `SSH_PRIVATE_KEY` | SSH ke VPS |
|
|
||||||
| `VPS_HOST` | IP/host VPS (tailscale IP) |
|
|
||||||
| `VPS_USER` | SSH user, biasanya `root` |
|
|
||||||
| `VPS_TARGET_DIR` | Lokasi repo di VPS |
|
|
||||||
| `ENV_FILE_PRODUCTION` | .env content untuk production |
|
|
||||||
|
|
||||||
### Selective Deployment
|
|
||||||
- Hanya compose file yg berubah yang di-redeploy
|
|
||||||
- Selective: `UP_FLAGS="-d"` (tanpa `--remove-orphans`)
|
|
||||||
- Full deploy: `UP_FLAGS="-d --remove-orphans"`
|
|
||||||
|
|
||||||
## Docker Patterns
|
|
||||||
|
|
||||||
### Build dengan cargo-chef (Rust)
|
|
||||||
```dockerfile
|
|
||||||
FROM lukemathwalker/cargo-chef:latest-rust-1.89.0 AS chef
|
|
||||||
WORKDIR /app
|
|
||||||
FROM chef AS planner
|
|
||||||
COPY apps/scraper .
|
|
||||||
RUN cargo chef prepare --recipe-path recipe.json
|
|
||||||
FROM chef AS builder
|
|
||||||
COPY --from=planner /app/recipe.json recipe.json
|
|
||||||
RUN cargo chef cook --release --recipe-path recipe.json
|
|
||||||
COPY apps/scraper .
|
|
||||||
RUN cargo build --release
|
|
||||||
```
|
|
||||||
|
|
||||||
### Runtime minimal untuk Rust binary
|
|
||||||
```dockerfile
|
|
||||||
FROM debian:bookworm-slim AS runtime
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
ca-certificates curl libssl3 && rm -rf /var/lib/apt/lists/*
|
|
||||||
```
|
|
||||||
|
|
||||||
## Manual Deploy Steps
|
|
||||||
```bash
|
|
||||||
# 1. Login GHCR
|
|
||||||
echo $GITHUB_TOKEN | docker login ghcr.io -u asepharyana --password-stdin
|
|
||||||
|
|
||||||
# 2. Full stack
|
|
||||||
docker compose -f infra/compose/traefik.yml \
|
|
||||||
-f infra/compose/shared.yml \
|
|
||||||
-f infra/compose/nats.yml \
|
|
||||||
-f infra/compose/dapr.yml \
|
|
||||||
-f infra/compose/scraper.yml \
|
|
||||||
--env-file .env up -d --remove-orphans
|
|
||||||
|
|
||||||
# 3. Selective (hanya satu service)
|
|
||||||
docker compose -f infra/compose/scraper.yml --env-file .env up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Container reach Tailscale
|
|
||||||
Pastikan route ke Tailscale di main table:
|
|
||||||
```bash
|
|
||||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
|
||||||
systemctl restart tailscale-routes
|
|
||||||
```
|
|
||||||
|
|
||||||
### Healthcheck gagal di scratch images
|
|
||||||
NATS dan Dapr placement pake scratch — tidak bisa healthcheck. Cukup `service_started` di depends_on.
|
|
||||||
|
|
||||||
### Dapr sidecar crash
|
|
||||||
```bash
|
|
||||||
docker logs scraper-api-dapr | grep -iE "fatal|error"
|
|
||||||
```
|
|
||||||
Penyebab umum: komponen config salah, NATS/Dapr placement belum siap.
|
|
||||||
@@ -1,133 +0,0 @@
|
|||||||
---
|
|
||||||
name: event-driven
|
|
||||||
description: Event-driven architecture patterns with Dapr + NATS for Asepharyana Hub services
|
|
||||||
---
|
|
||||||
|
|
||||||
# Event-Driven Architecture — Asepharyana Hub
|
|
||||||
|
|
||||||
## Stack
|
|
||||||
- **Message Backbone**: NATS + JetStream (untuk streaming & job queue)
|
|
||||||
- **Pub/Sub Runtime**: Dapr sidecar per service (pubsub via Redis built-in)
|
|
||||||
- **State Store**: Dapr → Redis
|
|
||||||
|
|
||||||
## Event Topics Convention
|
|
||||||
|
|
||||||
```
|
|
||||||
hub.<domain>.<action>
|
|
||||||
|
|
||||||
Contoh:
|
|
||||||
hub.image.cached → Image selesai di-cache ke CDN
|
|
||||||
hub.image.repaired → Image diperbaiki (CNAME change)
|
|
||||||
hub.scrape.anime.done → Scrape anime selesai
|
|
||||||
hub.system.alert → Error/alert dari service
|
|
||||||
```
|
|
||||||
|
|
||||||
## CloudEvents Format
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"specversion": "1.0",
|
|
||||||
"type": "hub.image.cached",
|
|
||||||
"source": "scraper-api",
|
|
||||||
"subject": "anime-poster",
|
|
||||||
"id": "uuid-v4",
|
|
||||||
"time": "2026-07-21T10:00:00Z",
|
|
||||||
"datacontenttype": "application/json",
|
|
||||||
"data": { ... }
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## Publish Event (Rust via HTTP API)
|
|
||||||
|
|
||||||
Gunakan `reqwest` langsung ke Dapr sidecar (SDK Rust masih experimental):
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let event = serde_json::json!({
|
|
||||||
"specversion": "1.0",
|
|
||||||
"type": "hub.image.cached",
|
|
||||||
"source": "scraper-api",
|
|
||||||
"id": Uuid::new_v4().to_string(),
|
|
||||||
"time": chrono::Utc::now().to_rfc3339(),
|
|
||||||
"datacontenttype": "application/json",
|
|
||||||
"data": { "original_url": url, "cdn_url": cdn_url }
|
|
||||||
});
|
|
||||||
|
|
||||||
reqwest::Client::new()
|
|
||||||
.post("http://localhost:3500/v1.0/publish/pubsub/hub.image.cached")
|
|
||||||
.json(&event)
|
|
||||||
.send()
|
|
||||||
.await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
## Service Invocation
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://localhost:3500/v1.0/invoke/<app-id>/method/<path>
|
|
||||||
```
|
|
||||||
|
|
||||||
## State Store
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Set
|
|
||||||
curl -X POST http://localhost:3500/v1.0/state/statestore \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '[{"key": "mykey", "value": "myvalue"}]'
|
|
||||||
|
|
||||||
# Get
|
|
||||||
curl http://localhost:3500/v1.0/state/statestore/mykey
|
|
||||||
|
|
||||||
# Delete
|
|
||||||
curl -X DELETE http://localhost:3500/v1.0/state/statestore/mykey
|
|
||||||
```
|
|
||||||
|
|
||||||
## Scraper Event Integration
|
|
||||||
|
|
||||||
File yang perlu dimodifikasi untuk event-driven:
|
|
||||||
|
|
||||||
| File | Perubahan |
|
|
||||||
|------|-----------|
|
|
||||||
| `src/events/bus.rs` | Ganti backend dari tokio broadcast ke Dapr pub/sub |
|
|
||||||
| `src/bootstrap/mod.rs` | Init DaprClient, inject ke AppState |
|
|
||||||
| `src/presentation/state.rs` | Tambah `dapr_client` field |
|
|
||||||
| `src/proxy/use_cases.rs` | Publish `ImageRepaired` & `ImageCached` events |
|
|
||||||
| `src/infrastructure/services/images/cache.rs` | Emit event tiap cache selesai |
|
|
||||||
| `Cargo.toml` | Tambah `reqwest`, `uuid`, `chrono` (jika belum ada) |
|
|
||||||
|
|
||||||
## Event Handlers (Subscribe)
|
|
||||||
|
|
||||||
Buat `src/subscribers/` untuk handler:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
// src/subscribers/image_handler.rs
|
|
||||||
pub async fn handle_image_cached(event: CloudEvent) -> Result<()> {
|
|
||||||
// Log, notifikasi, update status
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
Daftarkan subscribers di `bootstrap/mod.rs` dengan spawn task:
|
|
||||||
```rust
|
|
||||||
tokio::spawn(async move {
|
|
||||||
let mut stream = dapr_client.subscribe("pubsub", "hub.image.cached");
|
|
||||||
while let Some(event) = stream.next().await {
|
|
||||||
handle_image_cached(event).await;
|
|
||||||
}
|
|
||||||
});
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing Event-Driven Code
|
|
||||||
|
|
||||||
```rust
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[tokio::test]
|
|
||||||
async fn test_publish_event() {
|
|
||||||
let client = MockDaprClient::new();
|
|
||||||
client.expect_publish()
|
|
||||||
.with(...)
|
|
||||||
.returning(|_| Ok(()));
|
|
||||||
// ... test
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
@@ -1,101 +0,0 @@
|
|||||||
---
|
|
||||||
name: hub-rules
|
|
||||||
description: Repository structure, submodule strategy, infrastructure patterns, and architecture of Asepharyana Hub
|
|
||||||
---
|
|
||||||
|
|
||||||
# Asepharyana Hub — Repository Rules
|
|
||||||
|
|
||||||
## Struktur Repository
|
|
||||||
|
|
||||||
```
|
|
||||||
asepharyana-hub/
|
|
||||||
├── apps/ # Git submodules — source code aplikasi
|
|
||||||
├── docs/ # Dokumentasi, ADR, deployment guide
|
|
||||||
├── infra/ # Infrastructure as code
|
|
||||||
│ ├── compose/ # Satu compose file per service
|
|
||||||
│ ├── dapr/ # Dapr component configs
|
|
||||||
│ ├── docker/ # Dockerfiles per service
|
|
||||||
│ └── traefik/ # Static & dynamic Traefik config
|
|
||||||
├── scripts/ # Utility scripts (cleanup, update-deps)
|
|
||||||
└── .github/workflows/ # CI/CD pipelines
|
|
||||||
```
|
|
||||||
|
|
||||||
### Aturan Submodule
|
|
||||||
- Setiap aplikasi di `apps/` adalah **submodule** ke repo terpisah.
|
|
||||||
- Perubahan kode aplikasi dilakukan di **repo masing-masing**, bukan di sini.
|
|
||||||
- Submodule pointer diupdate oleh CI/CD (bukan manual).
|
|
||||||
|
|
||||||
## Infrastructure Patterns
|
|
||||||
|
|
||||||
### Networking
|
|
||||||
- Semua service join **`app-shared-net`** (external Docker bridge)
|
|
||||||
- Service discovery via Docker DNS (container alias)
|
|
||||||
- Traefik sebagai ingress untuk HTTP/S eksternal
|
|
||||||
- Tailscale untuk cross-VPS (PostgreSQL, Redis)
|
|
||||||
|
|
||||||
### Compose File Pattern
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
<service>:
|
|
||||||
container_name: <service>
|
|
||||||
image: ghcr.io/asepharyana/asepharyana-hub/<service>:sha-<sha>
|
|
||||||
restart: always
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
aliases:
|
|
||||||
- <service>
|
|
||||||
env_file:
|
|
||||||
- ../../.env
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dapr Sidecar Pattern
|
|
||||||
```yaml
|
|
||||||
<service>-dapr:
|
|
||||||
container_name: <service>-dapr
|
|
||||||
image: daprio/daprd:latest
|
|
||||||
restart: always
|
|
||||||
depends_on:
|
|
||||||
nats:
|
|
||||||
condition: service_started
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_started
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
command:
|
|
||||||
- './daprd'
|
|
||||||
- '--app-id=<service>'
|
|
||||||
- '--app-port=<port>'
|
|
||||||
- '--dapr-http-port=3500'
|
|
||||||
- '--dapr-grpc-port=50001'
|
|
||||||
- '--placement-host-address=dapr-placement:50005'
|
|
||||||
- '--resources-path=/components'
|
|
||||||
volumes:
|
|
||||||
- ../../infra/dapr/components:/components
|
|
||||||
```
|
|
||||||
|
|
||||||
### Traefik Routing
|
|
||||||
- Router + service definition di `infra/traefik/dynamic/apps.yaml`
|
|
||||||
- Subdomain pattern: `<service>.asepharyana.my.id` + `<service>.asepharya.web.id`
|
|
||||||
- TLS cert dari volume mount (bukan auto-acme)
|
|
||||||
|
|
||||||
### Image Tagging
|
|
||||||
- `sha-<short-sha>` — immutable, untuk rollback
|
|
||||||
- `latest` — mutable, untuk convenience
|
|
||||||
- Registry: `ghcr.io/asepharyana/asepharyana-hub/<service>`
|
|
||||||
|
|
||||||
### CI/CD
|
|
||||||
- `docker-build-push.yml` — build per service, push ke GHCR, update compose manifest
|
|
||||||
- `deploy-docker.yml` — SSH ke orangevps, pull images, restart
|
|
||||||
- Selective deploy: hanya compose file yg berubah
|
|
||||||
|
|
||||||
## Deployment Order
|
|
||||||
1. `shared.yml` (Redis)
|
|
||||||
2. `nats.yml` (NATS message bus)
|
|
||||||
3. `dapr.yml` (Dapr placement)
|
|
||||||
4. `traefik.yml` (Reverse proxy)
|
|
||||||
5. Service compose files (apps + Dapr sidecar)
|
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"mcpServers": {
|
||||||
|
"next-devtools": {
|
||||||
|
"command": "npx",
|
||||||
|
"args": ["-y", "next-devtools-mcp@latest"]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
# Agent Protocol v10: Strategic Execution and Diagnostic Directives
|
||||||
|
|
||||||
|
1. **Mandatory Executability.**
|
||||||
|
- **Directive:** All generated outputs must be syntactically correct and directly executable by the target system's interpreter, compiler, or shell.
|
||||||
|
- **Constraint:** Placeholders and pseudo-code are forbidden. Every output must be a complete, functional artifact.
|
||||||
|
|
||||||
|
2. **Data and State Integrity.**
|
||||||
|
- **Directive:** All generated code must strictly adhere to declared data structures, schemas, and the target language's type system.
|
||||||
|
- **Constraint:** Any operation producing a type mismatch, schema violation, or logical inconsistency is an invalid operation and must be discarded.
|
||||||
|
|
||||||
|
3. **Atomic and Consistent State Modification.**
|
||||||
|
- **Directive:** Modification of a shared resource must be performed as an atomic operation or within an ACID-compliant transaction.
|
||||||
|
- **Constraint:** Operations that can lead to race conditions or inconsistent state are prohibited. Immutability is the required default.
|
||||||
|
|
||||||
|
4. **Zero-Trust Security (Inviolable Safety Constraint).**
|
||||||
|
- **Directive:** Secrets must not be stored as literal values in source code. They must be loaded at runtime from a secure external source.
|
||||||
|
- **Constraint:** Generated access policies must adhere to the Principle of Least Privilege.
|
||||||
|
|
||||||
|
5. **Supply Chain Security (Inviolable Safety Constraint).**
|
||||||
|
- **Directive:** All external dependencies must be sourced from trusted repositories and defined in a lockfile for deterministic resolution.
|
||||||
|
- **Constraint:** The dependency graph must be scanned for known CVEs. Dependencies with critical vulnerabilities are prohibited.
|
||||||
|
|
||||||
|
6. **Deterministic and Reproducible Builds.**
|
||||||
|
- **Directive:** From a given source commit, the build process must produce a byte-for-byte identical artifact in every execution.
|
||||||
|
- **Constraint:** All automated tests must be deterministic. A regression test codifying the fixed bug's failure condition must be included with the fix.
|
||||||
|
|
||||||
|
7. **Structured, Traceable Logging.**
|
||||||
|
- **Directive:** All processes must emit structured (JSON) logs for significant events. All log entries for a request must contain the same unique trace ID.
|
||||||
|
- **Constraint:** Error conditions must be explicitly logged with context and propagated. Errors must not be silently suppressed.
|
||||||
|
|
||||||
|
8. **Strict API Contract Enforcement.**
|
||||||
|
- **Directive:** All network communication must strictly conform to its published, versioned API contract.
|
||||||
|
- **Constraint:** Any network call violating the contract must be rejected. Breaking changes require a major version increment (SemVer).
|
||||||
|
|
||||||
|
9. **Distributed System Consensus.**
|
||||||
|
- **Directive:** Changes to shared state across a distributed system are committed only after a formal consensus algorithm confirms quorum.
|
||||||
|
- **Constraint:** Nodes in a minority partition must enter a read-only or unavailable state to prevent a split-brain scenario.
|
||||||
|
|
||||||
|
10. **Execution Planning and Pre-flight Validation (Think Before Acting).**
|
||||||
|
- **Directive:** For any multi-step task, a detailed execution plan (sequence of commands and file modifications) must be formulated before any state-modifying action is taken.
|
||||||
|
- **Constraint:** Before executing a command, the agent must first use a validation or dry-run flag (e.g., `--dry-run`, `--check`) if available. The operation may only proceed if the pre-flight check passes without error.
|
||||||
|
|
||||||
|
11. **Post-Failure Root Cause Analysis (Evaluate Mistakes from Logs).**
|
||||||
|
- **Directive:** Upon command execution failure (non-zero exit code), the current execution plan must be halted, and the agent must enter a diagnostic mode.
|
||||||
|
- **Constraint:** In diagnostic mode, the agent is required to: 1) Capture and parse the complete `stdout` and `stderr` logs. 2) Identify the specific error message or stack trace. 3) Correlate the error with the last command to form a root cause hypothesis. 4) Formulate a new, corrective execution plan based on the analysis.
|
||||||
|
|
||||||
|
12. **Context-Aware File System Operations.**
|
||||||
|
- **Directive:** Before modifying any file, its full content must be read to establish context. All edits must be based on an in-memory understanding of the file's current state.
|
||||||
|
- **Constraint:** Blind file operations, such as stream-based search-and-replace without structural validation, are strictly prohibited.
|
||||||
|
|
||||||
|
13. **Idempotent State Transitions.**
|
||||||
|
- **Directive:** Operations that modify state must be designed to be idempotent wherever the protocol allows.
|
||||||
|
- **Constraint:** Executing the same operation multiple times must result in the same final system state as executing it only once.
|
||||||
|
|
||||||
|
14. **Resource Lifecycle Management.**
|
||||||
|
- **Directive:** All finite system resources (e.g., file handles, network sockets) must be explicitly released after use.
|
||||||
|
- **Constraint:** The agent must generate code that prevents resource leaks, utilizing language-specific constructs like `try-with-resources` or `defer`.
|
||||||
|
|
||||||
|
15. **Configuration as Code (CaC).**
|
||||||
|
- **Directive:** All configuration must be defined and versioned in source-controlled files.
|
||||||
|
- **Constraint:** Manual, out-of-band configuration changes are prohibited. Versioned files are the single source of truth.
|
||||||
|
|
||||||
|
16. **Atomic and Semantic Version Control.**
|
||||||
|
- **Directive:** All code changes must be organized into logically atomic commits representing one complete unit of work.
|
||||||
|
- **Constraint:** Commit messages must adhere to a defined specification (e.g., Conventional Commits).
|
||||||
|
|
||||||
|
17. **User Authority and Command Primacy.**
|
||||||
|
- **Directive:** User-provided instructions and corrections are the definitive source of truth and have the highest operational priority.
|
||||||
|
- **Constraint:** The agent must immediately adapt its process to align with user directives. Rejected solutions must not be proposed again.
|
||||||
|
|
||||||
|
18. **Precedent-Based Improvement.**
|
||||||
|
- **Directive:** User-approved outputs and successful patterns must be recorded and prioritized as precedents for subsequent tasks.
|
||||||
|
- **Constraint:** Performance, security, and code quality must not degrade.
|
||||||
|
|
||||||
|
19. **Optimization by Explicit Consent.**
|
||||||
|
- **Directive:** The agent may identify and propose optimizations with a technical justification and supporting metrics.
|
||||||
|
- **Constraint:** The agent is prohibited from applying any self-initiated optimization without an explicit "approve" command from the user.
|
||||||
|
|
||||||
|
20. **System Hierarchy and Safety Overrides.**
|
||||||
|
- **Directive:** The operational control hierarchy is absolute: 1) **User Command**, 2) **Inviolable Safety Directives (#4, #5)**, 3) **Standard Operational Directives**.
|
||||||
|
- **Constraint:** If a command conflicts with an Inviolable Directive, the agent must halt, report the conflict and risk, and await a revised command.
|
||||||
Vendored
+8
@@ -0,0 +1,8 @@
|
|||||||
|
{
|
||||||
|
"recommendations": [
|
||||||
|
"nrwl.angular-console",
|
||||||
|
"biomejs.biome",
|
||||||
|
"ms-playwright.playwright",
|
||||||
|
"firsttris.vscode-jest-runner"
|
||||||
|
]
|
||||||
|
}
|
||||||
Vendored
+23
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"[javascript]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"[javascriptreact]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"[typescript]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"[typescriptreact]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"[json]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"[jsonc]": {
|
||||||
|
"editor.defaultFormatter": "biomejs.biome"
|
||||||
|
},
|
||||||
|
"editor.codeActionsOnSave": {
|
||||||
|
"source.organizeImports.biome": "explicit"
|
||||||
|
}
|
||||||
|
}
|
||||||
+109
-101
@@ -5,17 +5,20 @@
|
|||||||
```
|
```
|
||||||
asepharyana-hub/
|
asepharyana-hub/
|
||||||
├── apps/ # Application services (Git submodules)
|
├── apps/ # Application services (Git submodules)
|
||||||
|
│ ├── elysia/ # Realtime API (Bun/Elysia/Drizzle/Redis)
|
||||||
|
│ ├── react/ # Frontend SPA (React/Vite/TanStack)
|
||||||
|
│ ├── rust-auth/ # IAM & auth service (Axum/SeaORM)
|
||||||
│ └── scraper/ # Web scraper service
|
│ └── scraper/ # Web scraper service
|
||||||
├── docs/ # Documentation
|
├── docs/ # Documentation
|
||||||
│ ├── adr/ # Architecture Decision Records
|
│ ├── adr/ # Architecture Decision Records
|
||||||
│ ├── add-new-app.md # Guide for adding new services
|
│ ├── add-new-app.md # Guide for adding new services
|
||||||
│ └── superpowers/ # Project capabilities tracking
|
│ └── superpowers/ # Project capabilities tracking
|
||||||
├── infra/ # Infrastructure as code (LEGACY Docker layout)
|
├── infra/ # Infrastructure as code
|
||||||
│ ├── compose/ # Docker Compose files (LEGACY — Docker dihapus 2026-08-02)
|
│ ├── compose/ # Docker Compose files per service
|
||||||
│ ├── config/ # Infrastructure configuration
|
│ ├── config/ # Infrastructure configuration
|
||||||
│ ├── docker/ # Dockerfiles (LEGACY)
|
│ ├── docker/ # Dockerfiles per service
|
||||||
│ ├── traefik/ # Traefik config (LEGACY — diganti Caddy)
|
│ └── traefik/ # Traefik reverse proxy config
|
||||||
│ └── caddy/ # Caddyfile.prod (reverse proxy produksi)
|
│ └── dynamic/ # Dynamic routing rules (YAML)
|
||||||
├── scripts/ # Utility scripts
|
├── scripts/ # Utility scripts
|
||||||
│ ├── git-hooks/ # Git hook scripts
|
│ ├── git-hooks/ # Git hook scripts
|
||||||
│ ├── cleanup-ghcr.sh # GHCR image cleanup
|
│ ├── cleanup-ghcr.sh # GHCR image cleanup
|
||||||
@@ -28,80 +31,77 @@ asepharyana-hub/
|
|||||||
|
|
||||||
## Technology Stack
|
## Technology Stack
|
||||||
|
|
||||||
### Services
|
### Backend Services
|
||||||
|
|
||||||
| Service | Language/Runtime | Framework | Database | Key Libraries |
|
| Service | Language/Runtime | Framework | Database | Key Libraries |
|
||||||
| --------- | ---------------- | --------- | -------- | ------------- |
|
| ------------- | ---------------- | ---------- | ------------------------ | ------------------------------------------------------------ |
|
||||||
| **scraper** | _(submodule)_ | — | — | — |
|
| **rust-auth** | Rust | Axum 0.8 | PostgreSQL (sqlx) | sqlx, jsonwebtoken, argon2, redis, opentelemetry, prometheus |
|
||||||
|
| **elysia** | TypeScript/Bun | Elysia 1.4 | PostgreSQL (Drizzle ORM) | Drizzle ORM, Redis (ioredis), JWT (jose), OTel, Swagger |
|
||||||
|
| **scraper** | _(submodule)_ | — | — | — |
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
|
||||||
|
| Service | Framework | Build Tool | Key Libraries |
|
||||||
|
| --------- | --------- | ---------- | --------------------------------------------------------------------------------------------------- |
|
||||||
|
| **react** | React 19 | Vite 7 | TanStack Router + Query, Three.js/React Three Fiber, Tailwind CSS 4, Zustand, Recharts, tsParticles |
|
||||||
|
|
||||||
### Infrastructure
|
### Infrastructure
|
||||||
|
|
||||||
| Component | Technology | Purpose |
|
| Component | Technology | Purpose |
|
||||||
| ------------------ | ----------------------- | ---------------------------------------------------------------- |
|
| ------------------ | ----------------------- | ---------------------------------------------------------------- |
|
||||||
| Reverse Proxy | Caddy 2.11.4 | TLS termination (auto-LE), routing, HTTP/3, keep-alive tuning |
|
| Reverse Proxy | Traefik v3.6 | TLS termination, routing, middleware (rate-limit, headers, auth) |
|
||||||
| Runtime | Nix + systemd | Service isolation and orchestration (Docker dihapus 2026-08-02) |
|
| Container Runtime | Docker + Docker Compose | Service isolation and orchestration |
|
||||||
| Deployment | GitHub Actions | nix build → nix copy ssh:// → systemctl restart |
|
| Container Registry | GHCR (ghcr.io) | Docker image storage |
|
||||||
| Secrets | Bitwarden Secrets Manager (BWS) | Central secret store, bws-exec wrapper |
|
|
||||||
| Networking | Tailscale | Secure overlay network between VPS nodes |
|
| Networking | Tailscale | Secure overlay network between VPS nodes |
|
||||||
| Message Bus | NATS + JetStream | Event-driven pub/sub, job queues, streaming |
|
| Cache | Redis (Alpine) | Session store, rate limit counters, caching |
|
||||||
| Runtime Sidecar | Dapr | Service invocation, pub/sub abstraction, state management |
|
|
||||||
| Cache & State | Redis (Alpine) | Session store, rate limit counters, caching, Dapr state store |
|
|
||||||
| CI/CD | GitHub Actions | Build, test, deploy automation |
|
| CI/CD | GitHub Actions | Build, test, deploy automation |
|
||||||
|
|
||||||
## Infrastructure
|
## Infrastructure
|
||||||
|
|
||||||
### Caddy Reverse Proxy
|
### Traefik Reverse Proxy
|
||||||
|
|
||||||
Caddy 2.11.4 runs as the entry point for all HTTP/S traffic (systemd `caddy.service`, `/etc/caddy/Caddyfile`). It is configured via:
|
Traefik runs as the entry point for all HTTP/S traffic. It is configured via:
|
||||||
|
|
||||||
- **Auto-TLS**: Let's Encrypt per-domain (email asepharyana@gmail.com)
|
- **Static config**: `infra/traefik/traefik.yaml` — entry points, providers, plugins
|
||||||
- **HTTP/3**: h3 enabled on :443 (QUIC)
|
- **Dynamic config**: `infra/traefik/dynamic/` — routers, services, middlewares, TLS
|
||||||
- **Snippet `(proxy)`**: shared handler — `encode zstd gzip`, security headers, keep-alive upstream (keepalive 120s, max_conns_per_host 100, dial_timeout 3s)
|
- **Docker provider**: Auto-discovers containers with `traefik.enable=true` labels
|
||||||
- **Upload domain** (`upload.asepharyana.my.id`): `flush_interval -1` (streaming), `request_body max_size 0` (unlimited)
|
- **File provider**: Loads `apps.yaml` (routers/services), `middlewares.yaml`, `ssl.yaml`
|
||||||
|
|
||||||
Reference: `infra/caddy/Caddyfile.prod`. Legacy Traefik configs stay under `infra/traefik/` for reference only.
|
Key middleware chains (`infra/traefik/dynamic/middlewares.yaml`):
|
||||||
|
|
||||||
### Port Mapping (Produksi)
|
- `secure-headers` — SSL redirect, HSTS, XSS protection, CSP
|
||||||
|
- `compress` — Gzip compression for responses over 256 bytes
|
||||||
|
- `rate-limit` — 100 avg / 50 burst requests
|
||||||
|
- `buffer` — 10MB request/response body limit
|
||||||
|
- `block-sensitive-paths` — blocks `.env`, `.git`, `/wp-admin` etc.
|
||||||
|
- `common-chain` — composes secure-headers + compress + retry + rate-limit + buffer
|
||||||
|
|
||||||
| Service | Port | Domain |
|
All services route through Traefik on port 443 (TLS), with automatic HTTP-to-HTTPS redirect.
|
||||||
|---------|------|--------|
|
|
||||||
| TeleUploader | 4000 | upload.asepharyana.my.id |
|
|
||||||
| GMW backend | 4001 | (internal) |
|
|
||||||
| pr-agent | 4002 | pr-agent.asepharyana.my.id |
|
|
||||||
| hub frontend | 4003 | asepharyana.my.id |
|
|
||||||
| lidm frontend | 4004 | lidm.asepharyana.my.id |
|
|
||||||
| lidm backend | 4005 | lidm-api.asepharyana.my.id |
|
|
||||||
| zeavis API | 4006 | api-zeavisedu.asepharyana.my.id |
|
|
||||||
| tools frontend | 4007 | tools.asepharyana.my.id |
|
|
||||||
| tools gateway | 4008 | (internal) |
|
|
||||||
| GMW proxy | 4009 | imphnen.asepharyana.my.id |
|
|
||||||
| llm-api | 4010 | ai.asepharyana.my.id |
|
|
||||||
| zeavisedu nginx | 4011 | zeavisedu.asepharyana.my.id |
|
|
||||||
| zeavis ML | 4012 | ml-zeavisedu.asepharyana.my.id |
|
|
||||||
| dashboard | 4013 | dashboard.asepharyana.my.id |
|
|
||||||
| 9router | 4014 | 9router.asepharyana.my.id |
|
|
||||||
| scraper | 4091 | scraper.asepharyana.my.id |
|
|
||||||
|
|
||||||
### Nix + systemd Deployment
|
### Docker Compose
|
||||||
|
|
||||||
Docker dihapus dari produksi (2026-08-02). Semua service deploy via Nix flakes + systemd:
|
Each service has its own Compose file under `infra/compose/`. All services join the `app-shared-net` external Docker network, enabling inter-service communication by container name.
|
||||||
|
|
||||||
|
Shared services:
|
||||||
|
|
||||||
|
- `infra/compose/shared.yml` — Redis (alias: `redis`)
|
||||||
|
- `infra/compose/traefik.yml` — Traefik reverse proxy
|
||||||
|
|
||||||
|
Service compose files are combined during deployment:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
nix build .#default --impure --option sandbox false
|
docker compose -f traefik.yml -f shared.yml -f elysia.yml -f react.yml ... up -d
|
||||||
nix copy --to ssh://vps /nix/store/<hash>
|
|
||||||
systemctl restart <service>
|
|
||||||
```
|
```
|
||||||
|
|
||||||
CI/CD: GitHub Actions (`deploy.yml`) → nix build → nix copy → systemctl restart. Flake dibatasi `x86_64-linux` (nixpkgs 26.11 drop darwin).
|
|
||||||
|
|
||||||
### Tailscale Networking
|
### Tailscale Networking
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
graph TB
|
graph TB
|
||||||
subgraph "Tailnet (100.64.0.0/10)"
|
subgraph "Tailnet (100.64.0.0/10)"
|
||||||
IMRNES["imrnes (100.121.180.82)"]
|
IMRNES["imrnes (100.108.1.124)"]
|
||||||
ORANGEVPS["orangevps (100.79.111.61)"]
|
ORANGE["orange (100.96.248.86)"]
|
||||||
ARCH["archlinux (100.84.39.83)"]
|
ARCH["archlinux (100.114.19.66)"]
|
||||||
|
LAPTOP["laptop-2f6e1iph (100.86.195.29)"]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph "imrnes Services"
|
subgraph "imrnes Services"
|
||||||
@@ -109,16 +109,28 @@ graph TB
|
|||||||
REDIS[Redis]
|
REDIS[Redis]
|
||||||
end
|
end
|
||||||
|
|
||||||
subgraph "orangevps Services (Nix)"
|
subgraph "orange Containers"
|
||||||
CADDY[Caddy :443]
|
TRAEFIK[Traefik :443]
|
||||||
|
RUST_AUTH[rust-auth :3000]
|
||||||
|
ELYSIA[elysia-api :4092]
|
||||||
|
REACT[react-web :80]
|
||||||
SCRAPER[scraper-api :4091]
|
SCRAPER[scraper-api :4091]
|
||||||
end
|
end
|
||||||
|
|
||||||
CADDY --> SCRAPER
|
TRAEFIK --> RUST_AUTH
|
||||||
|
TRAEFIK --> ELYSIA
|
||||||
|
TRAEFIK --> REACT
|
||||||
|
TRAEFIK --> SCRAPER
|
||||||
|
|
||||||
|
RUST_AUTH -.->|Tailscale IP| PG
|
||||||
|
ELYSIA -.->|Tailscale IP| PG
|
||||||
|
RUST_AUTH -.->|Tailscale IP| REDIS
|
||||||
|
ELYSIA -.->|Tailscale IP| REDIS
|
||||||
|
|
||||||
style IMRNES fill:#3a7,color:#fff
|
style IMRNES fill:#3a7,color:#fff
|
||||||
style ORANGEVPS fill:#37a,color:#fff
|
style ORANGE fill:#37a,color:#fff
|
||||||
style ARCH fill:#773,color:#fff
|
style ARCH fill:#773,color:#fff
|
||||||
|
style LAPTOP fill:#777,color:#fff
|
||||||
```
|
```
|
||||||
|
|
||||||
Container-to-Tailscale connectivity requires a systemd service that adds a route to the main routing table:
|
Container-to-Tailscale connectivity requires a systemd service that adds a route to the main routing table:
|
||||||
@@ -127,7 +139,7 @@ Container-to-Tailscale connectivity requires a systemd service that adds a route
|
|||||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
ip route add 100.64.0.0/10 dev tailscale0 table main
|
||||||
```
|
```
|
||||||
|
|
||||||
This is managed by `/etc/systemd/system/tailscale-routes.service` on the `orangevps` VPS.
|
This is managed by `/etc/systemd/system/tailscale-routes.service` on the `orange` VPS.
|
||||||
|
|
||||||
## Data Flow
|
## Data Flow
|
||||||
|
|
||||||
@@ -137,28 +149,28 @@ This is managed by `/etc/systemd/system/tailscale-routes.service` on the `orange
|
|||||||
sequenceDiagram
|
sequenceDiagram
|
||||||
participant User as Browser/Client
|
participant User as Browser/Client
|
||||||
participant DNS as Cloudflare DNS
|
participant DNS as Cloudflare DNS
|
||||||
participant Caddy as Caddy Proxy
|
participant Traefik as Traefik Proxy
|
||||||
participant App as Application Container
|
participant App as Application Container
|
||||||
participant DB as PostgreSQL (imrnes via Tailscale)
|
participant DB as PostgreSQL (imrnes via Tailscale)
|
||||||
participant Redis as Redis (imrnes via Tailscale)
|
participant Redis as Redis (imrnes via Tailscale)
|
||||||
|
|
||||||
User->>DNS: asepharyana.my.id
|
User->>DNS: asepharyana.my.id
|
||||||
DNS->>User: A/AAAA record → orangevps VPS IP
|
DNS->>User: A/AAAA record → orange VPS IP
|
||||||
User->>Caddy: HTTPS request :443
|
User->>Traefik: HTTPS request :443
|
||||||
Caddy->>Caddy: TLS termination
|
Traefik->>Traefik: TLS termination
|
||||||
Caddy->>Caddy: encode + headers
|
Traefik->>Traefik: Middleware chain (headers, rate-limit, buffer)
|
||||||
Caddy->>App: HTTP reverse-proxy (127.0.0.1:<port>)
|
Traefik->>App: HTTP reverse-proxy (internal network)
|
||||||
|
|
||||||
alt Database query
|
alt Database query
|
||||||
App->>DB: sqlx/Drizzle query via Tailscale
|
App->>DB: sqlx/Drizzle query via Tailscale
|
||||||
DB-->>App: Result set
|
DB-->>App: Result set
|
||||||
else Cache lookup
|
else Cache lookup
|
||||||
App->>Cache: GET/SET via Tailscale
|
App->>Redis: GET/SET via Tailscale
|
||||||
Cache-->>App: Cached value
|
Redis-->>App: Cached value
|
||||||
end
|
end
|
||||||
|
|
||||||
App-->>Caddy: HTTP response
|
App-->>Traefik: HTTP response
|
||||||
Caddy-->>User: HTTPS response
|
Traefik-->>User: HTTPS response
|
||||||
```
|
```
|
||||||
|
|
||||||
### CI/CD Pipeline
|
### CI/CD Pipeline
|
||||||
@@ -203,7 +215,7 @@ flowchart LR
|
|||||||
|
|
||||||
### VPS Deployment
|
### VPS Deployment
|
||||||
|
|
||||||
The `orangevps` VPS (Tailscale `100.79.111.61`) hosts all application containers:
|
The `orange` VPS (Tailscale `100.96.248.86`) hosts all application containers:
|
||||||
|
|
||||||
1. GitHub Actions SSHes into the VPS
|
1. GitHub Actions SSHes into the VPS
|
||||||
2. Production secrets are written as `.env`
|
2. Production secrets are written as `.env`
|
||||||
@@ -216,7 +228,7 @@ The `orangevps` VPS (Tailscale `100.79.111.61`) hosts all application containers
|
|||||||
|
|
||||||
### Selective Deployment
|
### Selective Deployment
|
||||||
|
|
||||||
The deploy workflow supports selective updates — if only one compose file changed, only the corresponding service is pulled and recreated, avoiding disruption to other services.
|
The deploy workflow supports selective updates — if only `infra/compose/elysia.yml` changed, only `elysia-api` is pulled and recreated, avoiding disruption to other services.
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
graph TB
|
graph TB
|
||||||
@@ -253,7 +265,7 @@ Each application lives in its own Git repository and is imported as a submodule
|
|||||||
|
|
||||||
### Submodule Lifecycle
|
### Submodule Lifecycle
|
||||||
|
|
||||||
1. Developer pushes to a submodule (e.g., `apps/scraper`)
|
1. Developer pushes to a submodule (e.g., `apps/elysia`)
|
||||||
2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA
|
2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA
|
||||||
3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service
|
3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service
|
||||||
4. The compose manifest is updated and committed with the new SHA tag
|
4. The compose manifest is updated and committed with the new SHA tag
|
||||||
@@ -263,24 +275,21 @@ Each application lives in its own Git repository and is imported as a submodule
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Update a single submodule to latest
|
# Update a single submodule to latest
|
||||||
cd apps/scraper
|
cd apps/elysia
|
||||||
git checkout main
|
git checkout main
|
||||||
git pull
|
git pull
|
||||||
cd ../..
|
cd ../..
|
||||||
git add apps/scraper
|
git add apps/elysia
|
||||||
git commit -m "chore(scraper): update submodule to latest"
|
git commit -m "chore(elysia): update submodule to latest"
|
||||||
|
|
||||||
|
# Update all submodules
|
||||||
|
git submodule update --remote --merge
|
||||||
```
|
```
|
||||||
|
|
||||||
## Service Mesh & Inter-Service Communication
|
## Service Mesh & Inter-Service Communication
|
||||||
|
|
||||||
### HTTP (External + Internal via Traefik)
|
|
||||||
External traffic and internal HTTP calls route through Traefik. Services on `app-shared-net` can also communicate directly by container name.
|
|
||||||
|
|
||||||
### Event-Driven (NATS + Dapr)
|
|
||||||
NATS with JetStream provides a persistent message backbone. Each service has a Dapr sidecar that abstracts pub/sub, service invocation, and state management.
|
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
graph TB
|
graph LR
|
||||||
subgraph "External"
|
subgraph "External"
|
||||||
WWW[Internet]
|
WWW[Internet]
|
||||||
end
|
end
|
||||||
@@ -289,36 +298,35 @@ graph TB
|
|||||||
TRAEFIK[Traefik :443]
|
TRAEFIK[Traefik :443]
|
||||||
|
|
||||||
subgraph "app-shared-net"
|
subgraph "app-shared-net"
|
||||||
NATS[NATS + JetStream<br/>:4222]
|
REACT[react-web<br/>:80]
|
||||||
DAPR_PLACEMENT[Dapr Placement<br/>:50005]
|
ELYSIA[elysia-api<br/>:4092]
|
||||||
|
RUST_AUTH[rust-auth<br/>:3000]
|
||||||
subgraph "Service: scraper-api"
|
SCRAPER[scraper-api<br/>:4091]
|
||||||
SCRAPER[scraper-api<br/>:4091]
|
REDIS[redis<br/>:6379]
|
||||||
DAPR_SIDECAR[Dapr Sidecar<br/>:3500]
|
|
||||||
SCRAPER --- DAPR_SIDECAR
|
|
||||||
end
|
|
||||||
end
|
end
|
||||||
|
end
|
||||||
|
|
||||||
DAPR_SIDECAR -.->|gRPC pub/sub| NATS
|
subgraph "Imrnes VPS"
|
||||||
DAPR_SIDECAR -.->|placement| DAPR_PLACEMENT
|
PG[(PostgreSQL<br/>:5432)]
|
||||||
|
REDIS_IMR[Redis<br/>:6379]
|
||||||
end
|
end
|
||||||
|
|
||||||
WWW -->|HTTPS| TRAEFIK
|
WWW -->|HTTPS| TRAEFIK
|
||||||
|
TRAEFIK --> REACT
|
||||||
|
TRAEFIK --> ELYSIA
|
||||||
|
TRAEFIK --> RUST_AUTH
|
||||||
TRAEFIK --> SCRAPER
|
TRAEFIK --> SCRAPER
|
||||||
|
|
||||||
|
ELYSIA -->|Tailscale| PG
|
||||||
|
RUST_AUTH -->|Tailscale| PG
|
||||||
|
ELYSIA -->|internal| REDIS
|
||||||
|
RUST_AUTH -->|internal| REDIS
|
||||||
|
ELYSIA -->|Tailscale| REDIS_IMR
|
||||||
|
RUST_AUTH -->|Tailscale| REDIS_IMR
|
||||||
```
|
```
|
||||||
|
|
||||||
### Communication Patterns
|
|
||||||
|
|
||||||
| Pattern | Mechanism | Use Case |
|
|
||||||
|---------|-----------|----------|
|
|
||||||
| External HTTP | Traefik → Service | User requests, API calls |
|
|
||||||
| Internal HTTP | Service → Service (via Traefik or direct) | Synchronous queries |
|
|
||||||
| Pub/Sub Event | Dapr sidecar → NATS JetStream | Async notifications, image cache events |
|
|
||||||
| Service Invocation | Dapr sidecar gRPC | Cross-service RPC with retry & observability |
|
|
||||||
| State Store | Dapr → Redis | Shared state, job progress |
|
|
||||||
|
|
||||||
## Observability
|
## Observability
|
||||||
|
|
||||||
- **Traefik access logs**: JSON format, logged at INFO level
|
- **Prometheus metrics**: Available on rust-auth via `axum-prometheus`
|
||||||
- **Traefik access logs**: JSON format, logged at INFO level
|
- **Traefik access logs**: JSON format, logged at INFO level
|
||||||
- **Dashboard**: Traefik dashboard at `traefik.asepharyana.my.id` (secured)
|
- **Dashboard**: Traefik dashboard at `traefik.asepharyana.my.id` (secured)
|
||||||
|
|||||||
@@ -5,16 +5,6 @@ All notable changes to this project will be documented in this file.
|
|||||||
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
||||||
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
||||||
|
|
||||||
## [2026-08-02]
|
|
||||||
|
|
||||||
### Changed
|
|
||||||
|
|
||||||
- **Infra overhaul**: Docker + Traefik dihapus dari produksi → Caddy 2.11.4 (reverse proxy, auto-TLS LE, HTTP/3) + Nix/systemd services.
|
|
||||||
- **Port migration**: semua service pindah ke port 4000-an (hub 4003, tools 4007/4008, scraper 4091, llm-api 4010, dll).
|
|
||||||
- **DB via PgBouncer pool**: semua service konek ke imrnes 100.121.180.82:6432 (bukan :5432 langsung).
|
|
||||||
- **Secrets**: Bitwarden Secrets Manager (BWS) sebagai central secret store, wrapper bws-exec.
|
|
||||||
- **Flake**: dibatasi x86_64-linux (nixpkgs 26.11 drop darwin).
|
|
||||||
|
|
||||||
## [Unreleased]
|
## [Unreleased]
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
@@ -22,22 +12,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|||||||
- Restructured the repository into a lightweight hub repo with standalone app submodules.
|
- Restructured the repository into a lightweight hub repo with standalone app submodules.
|
||||||
- Simplified root tooling to plain `package.json` scripts and per-service commands.
|
- Simplified root tooling to plain `package.json` scripts and per-service commands.
|
||||||
- Kept infrastructure, deployment workflows, and documentation in the root hub repo.
|
- Kept infrastructure, deployment workflows, and documentation in the root hub repo.
|
||||||
- Cleaned up Traefik SSL config: removed legacy `asephstech`/`asephscloud` cert references, synced volume mounts, fixed `api.insecure`.
|
|
||||||
- Pruned `.env.example` from 145 legacy vars (Firebase, Discord, Coolify, Portainer, YouTube, etc.) to 30 focused vars.
|
|
||||||
- Optimized `scraper.Dockerfile`: removed Node.js and Chromium from runtime image.
|
|
||||||
- Simplified CI/CD workflows: removed orphan container reference, commented code blocks.
|
|
||||||
- Cleaned up scripts: removed stale MySQL config, fixed package references, simplified update-deps.
|
|
||||||
- Added NATS + JetStream message broker infrastructure (`infra/compose/nats.yml`).
|
|
||||||
- Added Dapr runtime infrastructure: placement service, sidecar pattern, pub/sub + state store components.
|
|
||||||
- Integrated Dapr sidecar into scraper service (`infra/compose/scraper.yml`).
|
|
||||||
- Updated deployment order: shared → NATS → Dapr → Traefik → apps.
|
|
||||||
- Added `docs/add-dapr-service.md` guide for adding Dapr to new services.
|
|
||||||
|
|
||||||
### Removed
|
### Removed
|
||||||
|
|
||||||
- Removed deprecated services from apps, compose files, Dockerfiles, Traefik routes, and workflows.
|
- Removed deprecated services from apps, compose files, Dockerfiles, Traefik routes, and workflows.
|
||||||
- Removed stale monorepo orchestration configs and hook tooling from the root repo.
|
- Removed stale monorepo orchestration configs and hook tooling from the root repo.
|
||||||
- Removed `.nvmrc` (duplicate of `.node-version`), `renovate.json` (using dependabot).
|
|
||||||
- Removed stale documentation: `dependency-map.md`, `observability.md`, `handoff-log.jsonl`, `quality-gates.json`, `workflow-state.json`.
|
|
||||||
- Removed deprecated `docs/superpowers/` design docs.
|
|
||||||
- Removed `infra/config/mysql/` (no active MySQL service) and `docs/config/squid.conf.archived`.
|
|
||||||
|
|||||||
@@ -1,130 +0,0 @@
|
|||||||
# CLAUDE.md
|
|
||||||
|
|
||||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
|
||||||
|
|
||||||
## Repository Overview
|
|
||||||
|
|
||||||
Asepharyana Hub is a **hub monorepo** for Asep Haryana Saputra's portfolio ecosystem. Application services live in separate repos imported as Git submodules under `apps/`. Production infrastructure: Caddy reverse proxy + Nix/systemd services (Docker/Traefik removed 2026-08-02; legacy configs under `infra/` marked LEGACY).
|
|
||||||
|
|
||||||
```
|
|
||||||
asepharyana-hub/
|
|
||||||
├── apps/ # Git submodules — each app is its own repo
|
|
||||||
│ ├── hub/ # Personal portfolio SPA (asepharyana-hub-hub)
|
|
||||||
│ └── scraper/ # Rust scraper API (asepharyana-hub-scraper)
|
|
||||||
├── docs/ # ADRs, deployment guide, new-app guide
|
|
||||||
├── infra/
|
|
||||||
│ ├── compose/ # Docker Compose files (LEGACY — Docker dihapus)
|
|
||||||
│ ├── dapr/ # Dapr config + component definitions
|
|
||||||
│ ├── docker/ # Dockerfiles (LEGACY)
|
|
||||||
│ └── traefik/ # Reverse proxy config (static + dynamic)
|
|
||||||
├── scripts/ # Utility scripts (cleanup, update-deps, git hooks)
|
|
||||||
└── .github/workflows/ # CI/CD pipelines
|
|
||||||
```
|
|
||||||
|
|
||||||
### Submodule Strategy
|
|
||||||
- Each app in `apps/` is a separate Git repo imported as a submodule. Code changes happen in the submodule repo, not here.
|
|
||||||
- Submodule pointers are updated by CI/CD (via `repository_dispatch` or manual commit).
|
|
||||||
- Current submodules:
|
|
||||||
- `apps/hub` → `asepharyana/asepharyana-hub-hub`
|
|
||||||
- `apps/scraper` → `asepharyana/asepharyana-hub-scraper`
|
|
||||||
- `apps/llm-api` → `asepharyana/asepharyana-hub-llm-api`
|
|
||||||
- `apps/tools` → `asepharyana/asepharyana-hub-tools`.
|
|
||||||
|
|
||||||
### Infrastructure Stack
|
|
||||||
- **Caddy 2.11.4** — reverse proxy, TLS termination (auto-LE), HTTP/3, zstd/gzip, keep-alive tuning (`/etc/caddy/Caddyfile`, ref `infra/caddy/Caddyfile.prod`)
|
|
||||||
- **NATS + JetStream** — message broker with persistent streaming
|
|
||||||
- **Dapr** — sidecar runtime (pub/sub abstraction, state management, service invocation)
|
|
||||||
- **Redis (Alpine)** — cache, session store, Dapr state store & pub/sub backend
|
|
||||||
- **Prometheus** — metrics backend with `file_sd_configs` target files.
|
|
||||||
- **Jaeger** — distributed tracing backend (all-in-one), OTLP receiver
|
|
||||||
- **Tailscale** — secure overlay network between VPS nodes (PostgreSQL on `imrnes`, containers on `orangevps`)
|
|
||||||
|
|
||||||
### Monitoring
|
|
||||||
- **Hub dashboard** at `/dashboard` (Next.js client page, auto-refresh 15s)
|
|
||||||
- **Dashboard API** at `/api/dashboard` — returns JSON with systemd services, Jaeger traces, Prometheus metrics (RPS, latency, errors, node CPU/RAM/Disk)
|
|
||||||
- **Prometheus** scrapes node-exporter + app metrics endpoints
|
|
||||||
|
|
||||||
### Networking
|
|
||||||
- All services run as Nix/systemd units; inter-service via 127.0.0.1:<port>.
|
|
||||||
- Caddy handles all external HTTP/S traffic on port 443 (and HTTP/3 UDP).
|
|
||||||
- Cross-VPS traffic (DB, Redis) goes through Tailscale (`100.64.0.0/10`). Container-to-Tailscale connectivity requires a route in the main routing table (managed by `tailscale-routes.service`).
|
|
||||||
|
|
||||||
## Commands
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make init-submodules # Initialize submodules after clone
|
|
||||||
make dev # Start shared dev infrastructure (Redis)
|
|
||||||
make update-submodules # Update all submodules to latest
|
|
||||||
|
|
||||||
bun run check # Biome lint + format + write
|
|
||||||
bun run ci # Biome CI mode (no writes, exit code on issues)
|
|
||||||
bun run format # Format only
|
|
||||||
bun run lint # Lint only
|
|
||||||
|
|
||||||
# Nix build (produksi): nix build .#default --impure --option sandbox false
|
|
||||||
```
|
|
||||||
|
|
||||||
### Validate YAML
|
|
||||||
```bash
|
|
||||||
python -c "import pathlib, yaml; [yaml.safe_load(open(p)) for p in pathlib.Path('infra').rglob('*.yml')]"
|
|
||||||
for f in infra/compose/*.yml; do docker compose -f "$f" config >/dev/null && echo "OK $f"; done
|
|
||||||
```
|
|
||||||
|
|
||||||
## CI/CD Workflows
|
|
||||||
|
|
||||||
| Workflow | Trigger | Action |
|
|
||||||
|----------|---------|--------|
|
|
||||||
| `lint.yml` | PR/push to main touching `*.json`, `*.js`, `biome.json` | `bun run ci` (Biome lint) |
|
|
||||||
| `deploy.yml` | Push to main | nix build → nix copy ssh:// → systemctl restart |
|
|
||||||
| `docker-build-push.yml` | LEGACY (Docker dihapus) | LEGACY |
|
|
||||||
| `security.yml` | PR to main + weekly Monday | CodeQL analysis (Rust) |
|
|
||||||
| `update-submodule.yml` | `repository_dispatch` | Update submodule pointer in hub repo |
|
|
||||||
|
|
||||||
### Deployment Order
|
|
||||||
1. `shared.yml` (Redis)
|
|
||||||
2. `nats.yml` (NATS + JetStream)
|
|
||||||
3. `dapr.yml` (Dapr placement)
|
|
||||||
4. `traefik.yml` (Reverse proxy)
|
|
||||||
5. Service compose files (app + Dapr sidecar)
|
|
||||||
|
|
||||||
### Secrets Required for Deploy
|
|
||||||
`SSH_PRIVATE_KEY`, `VPS_HOST`, `VPS_USER`, `VPS_TARGET_DIR`, `ENV_FILE_PRODUCTION`
|
|
||||||
|
|
||||||
## Infrastructure Patterns
|
|
||||||
|
|
||||||
### Compose File Pattern
|
|
||||||
Each service gets one compose file. Containers join `app-shared-net` with a `container_name` alias for DNS. The network is declared `external: true`.
|
|
||||||
|
|
||||||
### Dapr Sidecar Pattern
|
|
||||||
Each app gets a companion `daprd` sidecar container. Dapr components (pubsub, statestore) are mounted from `infra/dapr/components/`. The sidecar communicates with NATS for pub/sub and Dapr placement for actor coordination.
|
|
||||||
|
|
||||||
### Caddy Routing
|
|
||||||
- Site blocks in `/etc/caddy/Caddyfile` (ref `infra/caddy/Caddyfile.prod`)
|
|
||||||
- Subdomain pattern: `<service>.asepharyana.my.id` and `<service>.asepharya.web.id`
|
|
||||||
- Auto-TLS via Let's Encrypt
|
|
||||||
- Shared handler snippet `(proxy)`: `encode zstd gzip` + security headers + keep-alive tuning
|
|
||||||
|
|
||||||
### Image Tagging
|
|
||||||
- `sha-<short-sha>` — immutable, for deterministic rollbacks
|
|
||||||
- `latest` — mutable, for convenience
|
|
||||||
- Registry: `ghcr.io/asepharyana/asepharyana-hub/<service>`
|
|
||||||
- Build cache: `sha-<short>-buildcache` (registry-based caching)
|
|
||||||
|
|
||||||
## Adding a New Service
|
|
||||||
|
|
||||||
1. Create a separate repo for the app code
|
|
||||||
2. Add as submodule: `git submodule add <url> apps/<name>`
|
|
||||||
3. Create Nix flake package + systemd unit
|
|
||||||
4. Create compose file in `infra/compose/` (app + Dapr sidecar)
|
|
||||||
5. Add Caddy site block in `/etc/caddy/Caddyfile`
|
|
||||||
6. Add build job in `.github/workflows/docker-build-push.yml`
|
|
||||||
7. See `docs/add-new-app.md` for full guide
|
|
||||||
|
|
||||||
## Commit Convention
|
|
||||||
|
|
||||||
Format: `<type>(<scope>): <description>`
|
|
||||||
|
|
||||||
Types: `feat`, `fix`, `chore`, `docs`, `refactor`, `test`, `ci`, `perf`, `style`
|
|
||||||
Scopes: `scraper`, `infra`, `ci`, `dapr`, `nats`, `docs`, `deps`, `scripts`, `root`
|
|
||||||
|
|
||||||
Scope is required. Use imperative mood. No period at end of subject line. Co-Authored-By footer for AI-generated commits.
|
|
||||||
+69
-17
@@ -14,8 +14,9 @@
|
|||||||
## Prerequisites
|
## Prerequisites
|
||||||
|
|
||||||
- **Git** with LFS support
|
- **Git** with LFS support
|
||||||
- **Node.js** >= 22.11.0 (via `.node-version`)
|
- **Node.js** >= 22.11.0 (via `.node-version` or `.nvmrc`)
|
||||||
- **Bun** >= 1.3.11 (package manager)
|
- **Bun** >= 1.3.11 (package manager)
|
||||||
|
- **Rust** >= 1.89.0 (for Rust services)
|
||||||
- **Docker** and **Docker Compose** (for shared infrastructure)
|
- **Docker** and **Docker Compose** (for shared infrastructure)
|
||||||
|
|
||||||
## Local Setup
|
## Local Setup
|
||||||
@@ -39,14 +40,18 @@ This checks out all submodules at the pinned commit (not `main`). The submodules
|
|||||||
|
|
||||||
| Path | Remote |
|
| Path | Remote |
|
||||||
| ---------------- | --------------------------------------- |
|
| ---------------- | --------------------------------------- |
|
||||||
|
| `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
|
||||||
| `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
|
| `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
|
||||||
|
| `apps/react` | `asepharyana/asepharyana-hub-react` |
|
||||||
|
| `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
|
||||||
|
|
||||||
### 3. Install Dependencies per Service
|
### 3. Install Dependencies per Service
|
||||||
|
|
||||||
Install dependencies for TypeScript/Bun services:
|
Install dependencies for TypeScript/Bun services:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cd apps/scraper && bun install && cd ../..
|
cd apps/elysia && bun install && cd ../..
|
||||||
|
cd apps/react && npm install && cd ../..
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4. Start Shared Infrastructure
|
### 4. Start Shared Infrastructure
|
||||||
@@ -70,42 +75,76 @@ Key variables to configure:
|
|||||||
| Variable | Description |
|
| Variable | Description |
|
||||||
| -------------- | ----------------------------------------------------- |
|
| -------------- | ----------------------------------------------------- |
|
||||||
| `DATABASE_URL` | PostgreSQL connection (Tailscale IP to `imrnes` VPS) |
|
| `DATABASE_URL` | PostgreSQL connection (Tailscale IP to `imrnes` VPS) |
|
||||||
|
| `REDIS_URL` | Redis connection (`redis://localhost:6379` for local) |
|
||||||
|
| `JWT_SECRET` | JWT signing secret |
|
||||||
| `GITHUB_TOKEN` | GitHub personal access token |
|
| `GITHUB_TOKEN` | GitHub personal access token |
|
||||||
|
|
||||||
## Development Workflow
|
## Development Workflow
|
||||||
|
|
||||||
### Running Services
|
### Running Services
|
||||||
|
|
||||||
Refer to each service's own documentation for setup and development instructions.
|
**Rust API (rust-auth):**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/rust-auth
|
||||||
|
cargo run
|
||||||
|
```
|
||||||
|
|
||||||
|
**Elysia API (elysia):**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/elysia
|
||||||
|
bun run dev
|
||||||
|
```
|
||||||
|
|
||||||
|
**React Frontend (react):**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd apps/react
|
||||||
|
npm run dev
|
||||||
|
```
|
||||||
|
|
||||||
### API Documentation
|
### API Documentation
|
||||||
|
|
||||||
Refer to each service's own documentation for API docs and endpoints.
|
- Rust OpenAPI: `http://localhost:4091/docs`
|
||||||
|
- Elysia Swagger: `http://localhost:4092/docs`
|
||||||
|
- Elysia AsyncAPI: `http://localhost:4092/docs-ws`
|
||||||
|
|
||||||
## Coding Standards
|
## Coding Standards
|
||||||
|
|
||||||
### Linting
|
### Linting
|
||||||
|
|
||||||
- **Biome** for TypeScript/JavaScript formatting and linting
|
- **ESLint** with `@antfu/eslint-config` for TypeScript/JavaScript
|
||||||
|
- **Cargo Clippy** for Rust
|
||||||
|
|
||||||
Run linting:
|
Run linting:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# TypeScript/JavaScript
|
# TypeScript/JavaScript
|
||||||
bun run check
|
eslint . --no-error-on-unmatched-pattern
|
||||||
|
|
||||||
|
# Rust specific
|
||||||
|
cd apps/rust-auth && cargo clippy -- -D warnings
|
||||||
```
|
```
|
||||||
|
|
||||||
### Formatting
|
### Formatting
|
||||||
|
|
||||||
- **Biome** for TypeScript/JavaScript
|
- **Prettier** for TypeScript/JavaScript/Markdown (config in `.prettierrc`)
|
||||||
|
- Single quotes, 100 print width, 2-space indent, trailing commas
|
||||||
|
- **Cargo fmt** for Rust
|
||||||
- **EditorConfig** for general formatting (`.editorconfig`)
|
- **EditorConfig** for general formatting (`.editorconfig`)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Format all
|
# Prettier
|
||||||
bun run format
|
prettier --write .
|
||||||
|
|
||||||
|
# Rust
|
||||||
|
cd apps/rust-auth && cargo fmt
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Rust Configuration
|
||||||
|
|
||||||
|
Rust services use edition `2024` with stable toolchain (nightly features may be used).
|
||||||
|
|
||||||
## Commit Message Format
|
## Commit Message Format
|
||||||
|
|
||||||
@@ -138,15 +177,18 @@ This project enforces **Conventional Commits** for all commit messages.
|
|||||||
### Examples
|
### Examples
|
||||||
|
|
||||||
```
|
```
|
||||||
feat(scraper): add new data source integration
|
feat(rust-auth): add OAuth2 Google login flow
|
||||||
chore: update biome config to v10
|
fix(elysia): handle null JWT payload in auth middleware
|
||||||
|
chore: update eslint config to v10
|
||||||
docs: add API endpoint documentation for scraper
|
docs: add API endpoint documentation for scraper
|
||||||
|
refactor(react): extract Header component from App
|
||||||
|
test(elysia): add unit tests for rate limiter
|
||||||
ci: migrate to CodeQL v3
|
ci: migrate to CodeQL v3
|
||||||
```
|
```
|
||||||
|
|
||||||
### Scopes
|
### Scopes
|
||||||
|
|
||||||
Common scopes: `scraper`, `infra`, `ci`, `deps`
|
Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
|
||||||
|
|
||||||
## Pull Request Process
|
## Pull Request Process
|
||||||
|
|
||||||
@@ -159,9 +201,10 @@ Common scopes: `scraper`, `infra`, `ci`, `deps`
|
|||||||
|
|
||||||
3. **Run checks locally** before pushing:
|
3. **Run checks locally** before pushing:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
bun run check
|
cd apps/react && npx tsc --noEmit
|
||||||
```
|
eslint . --no-error-on-unmatched-pattern
|
||||||
|
```
|
||||||
|
|
||||||
4. **Push and open a PR** against `main`. CI will automatically run:
|
4. **Push and open a PR** against `main`. CI will automatically run:
|
||||||
- **Lint** — ESLint across changed TypeScript files
|
- **Lint** — ESLint across changed TypeScript files
|
||||||
@@ -175,7 +218,7 @@ Common scopes: `scraper`, `infra`, `ci`, `deps`
|
|||||||
- Updates compose manifests to use the new SHA tags
|
- Updates compose manifests to use the new SHA tags
|
||||||
|
|
||||||
6. **Deployment Pipeline** triggers after a successful Docker build:
|
6. **Deployment Pipeline** triggers after a successful Docker build:
|
||||||
- SSHes into the VPS (`orangevps`, Tailscale IP `100.79.111.61`)
|
- SSHes into the VPS (`orange`, Tailscale IP `100.96.248.86`)
|
||||||
- Pulls updated Docker images
|
- Pulls updated Docker images
|
||||||
- Recreates only the changed containers
|
- Recreates only the changed containers
|
||||||
- All services share the `app-shared-net` Docker network
|
- All services share the `app-shared-net` Docker network
|
||||||
@@ -184,4 +227,13 @@ Common scopes: `scraper`, `infra`, `ci`, `deps`
|
|||||||
|
|
||||||
## Adding a New Service
|
## Adding a New Service
|
||||||
|
|
||||||
See `docs/add-new-app.md` for the complete step-by-step guide.
|
See `docs/add-new-app.md` for the complete step-by-step guide. In summary:
|
||||||
|
|
||||||
|
1. Create the app in `apps/<name>`
|
||||||
|
2. Add it as a Git submodule in `.gitmodules`
|
||||||
|
3. Register it in `infra/compose/<name>.yml`
|
||||||
|
4. Add a Dockerfile at `infra/docker/<name>.Dockerfile`
|
||||||
|
5. Add Traefik routing config in `infra/traefik/dynamic/apps.yaml`
|
||||||
|
6. Add CI entries in `.github/workflows/docker-build-push.yml`
|
||||||
|
7. Add compose file to the deploy script in `deploy-docker.yml`
|
||||||
|
8. Add any required GitHub secrets for the service
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
.PHONY: help dev update-submodules deploy init-submodules status
|
.PHONY: help dev lint format test clean update-submodules deploy init-submodules status
|
||||||
|
|
||||||
SHELL := /bin/bash
|
SHELL := /bin/bash
|
||||||
|
|
||||||
@@ -8,6 +8,16 @@ help: ## Show this help
|
|||||||
dev: ## Start development infrastructure (Redis etc.)
|
dev: ## Start development infrastructure (Redis etc.)
|
||||||
docker compose -f infra/compose/shared.yml up -d
|
docker compose -f infra/compose/shared.yml up -d
|
||||||
|
|
||||||
|
lint: ## Run Biome linter
|
||||||
|
biome lint .
|
||||||
|
|
||||||
|
format: ## Format code with Biome
|
||||||
|
biome format --write .
|
||||||
|
|
||||||
|
clean: ## Clean build artifacts
|
||||||
|
rm -rf apps/*/dist apps/*/.next apps/*/target 2>/dev/null || true
|
||||||
|
rm -rf node_modules 2>/dev/null || true
|
||||||
|
|
||||||
update-submodules: ## Update all git submodules to latest remote
|
update-submodules: ## Update all git submodules to latest remote
|
||||||
git submodule update --remote --merge --recursive
|
git submodule update --remote --merge --recursive
|
||||||
|
|
||||||
|
|||||||
@@ -1,232 +1,202 @@
|
|||||||
# Architecture
|
# Asepharyana Hub
|
||||||
|
|
||||||
## Hub Repository Structure Overview
|
Hub repo untuk ekosistem portfolio dan layanan pendukung milik Asep Haryana Saputra.
|
||||||
|
Aplikasi dipisah sebagai submodule agar frontend, API, dan service pendukung bisa dikembangkan serta di-deploy secara independen.
|
||||||
|
|
||||||
```diff
|
## Services
|
||||||
asepharyana-hub/
|
|
||||||
├── apps/ # Application services (Git submodules)
|
|
||||||
│ └── scraper/ # Web scraper service
|
|
||||||
├── docs/ # Documentation
|
|
||||||
│ ├── adr/ # Architecture Decision Records
|
|
||||||
│ ├── add-new-app.md # Guide for adding new services
|
|
||||||
│ └── superpowers/ # Project capabilities tracking
|
|
||||||
├── infra/ # Infrastructure as code
|
|
||||||
│ ├── compose/ # Docker Compose files per service
|
|
||||||
│ ├── config/ # Infrastructure configuration
|
|
||||||
│ ├── docker/ # Dockerfiles per service
|
|
||||||
│ └── traefik/ # Traefik reverse proxy config
|
|
||||||
│ └── dynamic/ # Dynamic routing rules (YAML)
|
|
||||||
├── scripts/ # Utility scripts
|
|
||||||
│ ├── git-hooks/ # Git hook scripts
|
|
||||||
│ ├── cleanup-ghcr.sh # GHCR image cleanup
|
|
||||||
│ └── update-deps.sh # Dependency update helper
|
|
||||||
├── .github/workflows/ # CI/CD pipelines
|
|
||||||
├── eslint.config.mjs # Root ESLint config
|
|
||||||
├── package.json # Root formatting/lint helper scripts
|
|
||||||
└── .prettierrc # Prettier formatting rules
|
|
||||||
```
|
|
||||||
|
|
||||||
## Technology Stack
|
| Service | Path | Default Local Port | Notes |
|
||||||
|
| :---------- | :--------------- | :----------------- | :---------------------------------------------------------------------------- |
|
||||||
|
| Rust API | `apps/rust-auth` | `4091` | API utama (Axum + SeaORM), scraping, image proxy/cache, metrics, OpenAPI docs |
|
||||||
|
| Elysia API | `apps/elysia` | `4092` | API realtime/auth/chat/quiz (Elysia + Bun + Drizzle + Redis) |
|
||||||
|
| React Web | `apps/react` | `3000` | Frontend React/Vite |
|
||||||
|
| Scraper | `apps/scraper` | — | Web scraper service |
|
||||||
|
|
||||||
### Services
|
## Infrastructure
|
||||||
|
|
||||||
|| Service | Path | Language/Runtime | Framework | Database | Key Libraries |
|
File compose berada di `infra/compose/`:
|
||||||
||---------|----------------|------------------|-----------|----------|---------------|
|
|
||||||
|| **scraper** | `apps/scraper` | — | — | — | — |
|
|
||||||
|
|
||||||
### Infrastructure
|
- `traefik.yml`: reverse proxy Traefik untuk semua layanan.
|
||||||
|
- `shared.yml`: Redis.
|
||||||
|
- `rust-auth.yml`, `elysia.yml`, `react.yml`, `scraper.yml`: manifest deploy per service (image GHCR bertag SHA).
|
||||||
|
|
||||||
|| Component | Technology | Purpose |
|
Dockerfile per service berada di `infra/docker/`.
|
||||||
||---------------------|-------------------------|------------------------------------------------------------------|
|
|
||||||
|| Reverse Proxy | Traefik v3.6 | TLS termination, routing, middleware (rate-limit, headers, auth) |
|
|
||||||
|| Container Runtime | Docker + Docker Compose | Service isolation and orchestration |
|
|
||||||
|| Container Registry | GHCR (ghcr.io) | Docker image storage |
|
|
||||||
|| Networking | Tailscale | Secure overlay network between VPS nodes |
|
|
||||||
|| Message Bus | NATS + JetStream | Event-driven pub/sub, job queues, streaming |
|
|
||||||
|| Runtime Sidecar | Dapr | Service invocation, pub/sub abstraction, state management |
|
|
||||||
|| Cache & State | Redis (Alpine) | Session store, rate limit counters, caching, Dapr state store |
|
|
||||||
|| CI/CD | GitHub Actions | Build, test, deploy automation |
|
|
||||||
|
|
||||||
### Infrastructure
|
## Docker Image Builds
|
||||||
|
|
||||||
### Traefik Reverse Proxy
|
Build image via Dockerfile:
|
||||||
|
|
||||||
Traefik runs as the entry point for all HTTP/S traffic. It is configured via:
|
|
||||||
|
|
||||||
- **Static config**: CLI arguments in `infra/compose/traefik.yml` — entry points, providers, plugins
|
|
||||||
- **Dynamic config**: `infra/traefik/dynamic/` — routers, services, middlewares, TLS
|
|
||||||
- **Docker provider**: Auto-discovers containers with `traefik.enable=true` labels
|
|
||||||
- **File provider**: Loads `apps.yaml` (routers/services), `middlewares.yaml`, `ssl.yaml`
|
|
||||||
|
|
||||||
Key middleware chains (`infra/traefik/dynamic/middlewares.yaml`):
|
|
||||||
|
|
||||||
- `secure-headers` — SSL redirect, HSTS, XSS protection, CSP
|
|
||||||
- `compress` — Gzip compression for responses over 256 bytes
|
|
||||||
- `rate-limit` — 100 avg / 50 burst requests
|
|
||||||
- `buffer` — 10MB request/response body limit
|
|
||||||
- `block-sensitive-paths` — blocks `.env`, `.git`, `/wp-admin` etc.
|
|
||||||
- `common-chain` — composes secure-headers + compress + retry + rate-limit + buffer
|
|
||||||
|
|
||||||
All services route through Traefik on port 443 (TLS), with automatic HTTP-to-HTTPS redirect.
|
|
||||||
|
|
||||||
### Docker Compose
|
|
||||||
|
|
||||||
Each service has its own Compose file under `infra/compose/`. All services join the `app-shared-net` external Docker network, enabling inter-service communication by container name.
|
|
||||||
|
|
||||||
Shared services:
|
|
||||||
|
|
||||||
- `infra/compose/shared.yml` — Redis (alias: `redis`)
|
|
||||||
- `infra/compose/traefik.yml` — Traefik reverse proxy
|
|
||||||
|
|
||||||
Service compose files are combined during deployment:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker compose -f traefik.yml -f shared.yml -f scraper.yml up -d
|
docker build -f infra/docker/rust.Dockerfile -t rust-auth:latest .
|
||||||
|
docker build -f infra/docker/elysia.Dockerfile -t elysia-api:latest .
|
||||||
|
docker build -f infra/docker/react.Dockerfile -t react-web:latest .
|
||||||
|
docker build -f infra/docker/scraper.Dockerfile -t scraper-api:latest .
|
||||||
```
|
```
|
||||||
|
|
||||||
### Tailscale Networking
|
Tag and push:
|
||||||
|
|
||||||
### Arsitektur
|
|
||||||
|
|
||||||
Semua VPS terhubung via **Tailscale**. Setiap VPS punya IP Tailscale dan service berkomunikasi antar VPS melalui Tailscale network (`100.64.0.0/10`). Container-to-Tailscale connectivity requires a systemd service that adds a route to the main routing table:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
SHORT_SHA=$(git rev-parse --short HEAD)
|
||||||
|
|
||||||
|
docker tag rust-auth:latest ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA
|
||||||
|
docker push ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA
|
||||||
|
# repeat for elysia-api, react-web, scraper-api
|
||||||
```
|
```
|
||||||
|
|
||||||
This is managed by `/etc/systemd/system/tailscale-routes.service` on the `orangevps` VPS.
|
## Local Development
|
||||||
|
|
||||||
### Data Flow
|
### 1) Jalankan dependency bersama
|
||||||
|
|
||||||
### Request Flow (Production)
|
```bash
|
||||||
|
docker compose -f infra/compose/shared.yml up -d
|
||||||
```mermaid
|
|
||||||
sequenceDiagram
|
|
||||||
participant User as Browser/Client
|
|
||||||
participant DNS as Cloudflare DNS
|
|
||||||
participant Traefik as Traefik Proxy
|
|
||||||
participant App as Application Container
|
|
||||||
participant DB as PostgreSQL (imrnes via Tailscale)
|
|
||||||
participant Redis as Redis (imrnes via Tailscale)
|
|
||||||
|
|
||||||
User->>DNS: asepharyana.my.id
|
|
||||||
DNS->>User: A/AAAA record → orangevps VPS IP
|
|
||||||
User->>Traefik: HTTPS request :443
|
|
||||||
Traefik->>Traefik: TLS termination
|
|
||||||
Traefik->>Traefik: Middleware chain (headers, rate-limit, buffer)
|
|
||||||
Traefik->>App: HTTP reverse-proxy (internal network)
|
|
||||||
|
|
||||||
alt Database query
|
|
||||||
App->>DB: sqlx/Drizzle query via Tailscale
|
|
||||||
DB-->>App: Result set
|
|
||||||
else Cache lookup
|
|
||||||
App->>Cache: GET/SET via Tailscale
|
|
||||||
Cache-->>App: Cached value
|
|
||||||
end
|
|
||||||
|
|
||||||
App-->>Traefik: HTTP response
|
|
||||||
Traefik-->>User: HTTPS response
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### CI/CD Pipeline
|
### 2) Jalankan service yang dibutuhkan
|
||||||
|
|
||||||
```mermaid
|
```bash
|
||||||
flowchart LR
|
# Rust API
|
||||||
A[Push to main] --> B{Changed paths?}
|
cd apps/rust-auth
|
||||||
B -->|apps/** or infra/docker/**| C[Build Docker Images]
|
cargo run
|
||||||
B -->|infra/compose/**| D[Deploy to VPS]
|
|
||||||
B -->|apps/*/src/**/*.ts| E[Lint + TypeCheck]
|
|
||||||
|
|
||||||
C --> F[Push to GHCR]
|
# Elysia API
|
||||||
F --> G[Update Compose tags]
|
cd apps/elysia
|
||||||
G --> D
|
bun install
|
||||||
|
bun run dev
|
||||||
|
|
||||||
D --> H[SSH into VPS]
|
# React web
|
||||||
H --> I[Pull images]
|
cd apps/react
|
||||||
I --> J[docker compose up -d]
|
npm install
|
||||||
|
npm run dev
|
||||||
subgraph "Build Phase"
|
|
||||||
C
|
|
||||||
F
|
|
||||||
G
|
|
||||||
end
|
|
||||||
|
|
||||||
subgraph "Deploy Phase"
|
|
||||||
D
|
|
||||||
H
|
|
||||||
I
|
|
||||||
J
|
|
||||||
end
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Deployment Architecture
|
## API Docs and Monitoring
|
||||||
|
|
||||||
### Image Tags
|
- Rust OpenAPI: `/docs`
|
||||||
|
- Elysia Swagger: `/docs`
|
||||||
- `latest` — mutable, for convenience
|
- Elysia AsyncAPI viewer: `/docs-ws`
|
||||||
- `sha-<short-sha>` — immutable, for deterministic rollbacks
|
|
||||||
- Build cache: `sha-<short>-buildcache`
|
|
||||||
|
|
||||||
Registry: `ghcr.io/asepharyana/asepharyana-hub/<service>`
|
|
||||||
|
|
||||||
## Deployment Notes
|
## Deployment Notes
|
||||||
|
|
||||||
- Pipeline memakai image tag berbasis commit SHA (`sha-<short-sha>`), bukan `latest`.
|
- Pipeline memakai image tag berbasis commit SHA (`sha-<short-sha>`), bukan `latest`.
|
||||||
- Deploy Compose sekarang mencakup `infra/compose/*.yml` dan `deploy-docker.yml` akan berjalan langsung ketika `infra/compose/**` berubah.
|
- Deploy Compose sekarang mencakup `infra/compose/*.yml` dan `deploy-docker.yml` akan berjalan langsung ketika `infra/compose/**` berubah.
|
||||||
- Selective deployment: hanya compose file yg berubah yang di-redeploy.
|
|
||||||
|
|
||||||
## Networking & Tailscale
|
## Networking & Tailscale
|
||||||
|
|
||||||
### Arsitektur
|
### Arsitektur
|
||||||
|
|
||||||
Semua VPS terhubung via **Tailscale**. Setiap VPS punya IP Tailscale dan service berkomunikasi antar VPS melalui Tailscale network (`100.64.0.0/10`). Container-to-Tailscale connectivity requires a systemd service that adds a route to the main routing table:
|
Semua VPS terhubung via **Tailscale**. Setiap VPS punya IP Tailscale dan service berkomunikasi antar VPS melalui Tailscale network (`100.64.0.0/10`).
|
||||||
|
|
||||||
|
| VPS | Tailscale IP | Service |
|
||||||
|
| :---------------- | :-------------- | :------------------------------------- |
|
||||||
|
| `imrnes` | `100.108.1.124` | PostgreSQL (`hub`), Redis |
|
||||||
|
| `orange` | `100.96.248.86` | App containers (Traefik, 9Router, dll) |
|
||||||
|
| `archlinux` | `100.114.19.66` | _(development machine)_ |
|
||||||
|
| `laptop-2f6e1iph` | `100.86.195.29` | _(offline)_ |
|
||||||
|
|
||||||
|
### Container → Tailscale Connectivity
|
||||||
|
|
||||||
|
Docker containers di bridge network (`app-shared-net`) **tidak otomatis bisa access Tailscale IPs** karena Tailscale menggunakan **custom policy routing** (routes di `table 52`, bukan `main` table).
|
||||||
|
|
||||||
|
#### Fix: Tailscale Route di Main Table
|
||||||
|
|
||||||
|
Agar container bisa reach Tailscale IPs (untuk DB, Redis, dll), tambahkan route ke `main` routing table:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
# Manual (hilang setelah reboot)
|
||||||
ip route add 100.64.0.0/10 dev tailscale0 table main
|
ip route add 100.64.0.0/10 dev tailscale0 table main
|
||||||
|
|
||||||
|
# Persistent (systemd service)
|
||||||
|
# Sudah dikonfigurasi sebagai /etc/systemd/system/tailscale-routes.service
|
||||||
|
# Service ini berjalan otomatis setelah tailscaled start
|
||||||
|
systemctl enable tailscale-routes.service
|
||||||
|
systemctl start tailscale-routes.service
|
||||||
```
|
```
|
||||||
|
|
||||||
This is managed by `/etc/systemd/system/tailscale-routes.service` on the `orangevps` VPS.
|
#### Environment Variables
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
Service yang connect ke Tailscale IP:
|
Service yang connect ke Tailscale IP:
|
||||||
|
|
||||||
```env
|
```env
|
||||||
# PostgreSQL di imrnes
|
# PostgreSQL di imrnes
|
||||||
DATABASE_URL=postgres://user:***@100.121.180.82:6432/dbname
|
DATABASE_URL=postgres://user:pass@100.108.1.124:5432/dbname
|
||||||
|
|
||||||
# Redis di imrnes
|
# Redis di imrnes
|
||||||
REDIS_URL=redis://100.121.180.82:6379
|
REDIS_URL=redis://100.108.1.124:6379
|
||||||
```
|
```
|
||||||
|
|
||||||
## Submodule Strategy
|
#### Persistent Systemd Service
|
||||||
|
|
||||||
Each application lives in its own Git repository and is imported as a submodule into `apps/`. This approach:
|
File: `/etc/systemd/system/tailscale-routes.service`
|
||||||
|
|
||||||
- **Enables independent development** — each service can be developed, tested, and versioned separately
|
```ini
|
||||||
- **Pins exact commits** — the super-repository tracks exact submodule SHAs, enabling reproducible deployments
|
[Unit]
|
||||||
- **Supports `repository_dispatch`** — when a submodule receives a push, it can trigger the super-repository to build and deploy only that service
|
Description=Add Tailscale routes to main routing table
|
||||||
|
After=tailscaled.service
|
||||||
|
Requires=tailscaled.service
|
||||||
|
|
||||||
### Submodule Lifecycle
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/bin/bash -c '/usr/sbin/ip route add 100.64.0.0/10 dev tailscale0 table main 2>/dev/null || /usr/sbin/ip route replace 100.64.0.0/10 dev tailscale0 table main'
|
||||||
|
RemainAfterExit=yes
|
||||||
|
|
||||||
1. Developer pushes to a submodule (e.g., `apps/scraper`)
|
[Install]
|
||||||
2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA
|
WantedBy=multi-user.target
|
||||||
3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service
|
```
|
||||||
4. The compose manifest is updated and committed with the new SHA tag
|
|
||||||
5. The deploy workflow runs and updates only the changed containers
|
|
||||||
|
|
||||||
### Updating Submodules
|
Install & enable:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Update a single submodule to latest
|
sudo tee /etc/systemd/system/tailscale-routes.service > /dev/null << 'EOF'
|
||||||
cd apps/scraper
|
[Unit]
|
||||||
git checkout main
|
Description=Add Tailscale routes to main routing table
|
||||||
git pull
|
After=tailscaled.service
|
||||||
cd ../..
|
Requires=tailscaled.service
|
||||||
git add apps/scraper
|
|
||||||
git commit -m "chore(scraper): update submodule to latest"
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/bin/bash -c '/usr/sbin/ip route add 100.64.0.0/10 dev tailscale0 table main 2>/dev/null || /usr/sbin/ip route replace 100.64.0.0/10 dev tailscale0 table main'
|
||||||
|
RemainAfterExit=yes
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
EOF
|
||||||
|
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable tailscale-routes.service
|
||||||
|
sudo systemctl start tailscale-routes.service
|
||||||
```
|
```
|
||||||
|
|
||||||
|
#### Troubleshooting
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Cek Tailscale peers
|
||||||
|
tailscale status
|
||||||
|
|
||||||
|
# Cek route table 52 (Tailscale internal)
|
||||||
|
ip route show table 52
|
||||||
|
|
||||||
|
# Cek route table main (yang dipakai container)
|
||||||
|
ip route show table main | grep 100.
|
||||||
|
|
||||||
|
# Test connectivity dari dalam container
|
||||||
|
docker exec <container> node -e "
|
||||||
|
const net = require('net');
|
||||||
|
const c = new net.Socket();
|
||||||
|
c.setTimeout(5000);
|
||||||
|
c.connect(5432, '100.108.1.124', () => { console.log('OK'); c.end(); });
|
||||||
|
c.on('error', e => { console.log('FAIL:', e.code); });
|
||||||
|
c.on('timeout', () => { console.log('TIMEOUT'); c.destroy(); });
|
||||||
|
"
|
||||||
|
|
||||||
|
# Cek service tailscale-routes
|
||||||
|
systemctl status tailscale-routes.service
|
||||||
|
```
|
||||||
|
|
||||||
|
## Menambahkan Aplikasi Baru
|
||||||
|
|
||||||
|
Panduan langkah demi langkah untuk menambahkan aplikasi baru ada di `docs/add-new-app.md`.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT
|
MIT
|
||||||
|
# PR-Agent Test
|
||||||
|
|||||||
Submodule
+1
Submodule apps/elysia added at b8f4b5806e
-1
Submodule apps/hub deleted from ad6b1847c5
-1
Submodule apps/llm-api deleted from 5f7ead5503
Submodule
+1
Submodule apps/react added at 124d2f88bd
Submodule
+1
Submodule apps/rust-auth added at 6e695f07e8
+1
-1
Submodule apps/scraper updated: 62aa5b0e52...80c96eaa42
-1
Submodule apps/tools deleted from 036f67d05a
@@ -1,238 +0,0 @@
|
|||||||
# Arsitektur asepharyana-hub
|
|
||||||
|
|
||||||
## Topologi Fisik
|
|
||||||
|
|
||||||
Dua node terhubung via **Tailscale** overlay network:
|
|
||||||
|
|
||||||
```
|
|
||||||
┌──────────────────────────────┐ ┌──────────────────────────────┐
|
|
||||||
│ orangevps (VPS) │ │ imrnes (Bare-metal) │
|
|
||||||
│ IP: 45.127.35.244 │ │ Tailscale: 100.121.180.82 │
|
|
||||||
│ Tailscale: 100.x.x.x │◄──────┤ │
|
|
||||||
│ │ │ Layanan: │
|
|
||||||
│ Layanan: │ │ ├─ PostgreSQL (port 6432) │
|
|
||||||
│ ├─ Caddy (port 80/443) │ │ └─ Redis (port 6379) │
|
|
||||||
│ ├─ NATS + JetStream │ │ │
|
|
||||||
│ ├─ Dapr Placement │ └──────────────────────────────┘
|
|
||||||
│ ├─ Redis (cache, Dapr) │
|
|
||||||
│ ├─ Scraper API + Dapr │
|
|
||||||
│ └─ Hub (Next.js SPA) │
|
|
||||||
└──────────────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
### Konektivitas Container ke Tailscale
|
|
||||||
|
|
||||||
Container di `orangevps` tidak bisa langsung mencapai IP Tailscale (`100.x.x.x`). Route Tailscale harus ditambahkan ke tabel routing utama (`main`) via `tailscale-routes.service` agar traffic dari container bisa melewati host ke Tailscale.
|
|
||||||
|
|
||||||
## Alur Request HTTP (External)
|
|
||||||
|
|
||||||
```
|
|
||||||
Internet
|
|
||||||
│
|
|
||||||
▼ Port 443
|
|
||||||
Caddy 2.11.4 (auto-TLS LE, HTTP/3)
|
|
||||||
├─ TLS termination (sertifikat dari volume mount)
|
|
||||||
├─ Middleware chain: secure-headers → compress → retry → rate-limit → buffer
|
|
||||||
├─ Plugin: real-ip (Cloudflare), block-sensitive-paths
|
|
||||||
│
|
|
||||||
▼ Router matching
|
|
||||||
Host(`asepharyana.my.id`) || Host(`www.asepharyana.my.id`) → hub
|
|
||||||
host(`hub.asepharyana.my.id`) → hub (SPA + dashboard)
|
|
||||||
Host(`scraper.asepharyana.my.id`) || Host(`api.asepharyana.my.id`) → scraper-api
|
|
||||||
│
|
|
||||||
├─ hub (Next.js, port 4003)
|
|
||||||
│ ├─ / — Portfolio SPA
|
|
||||||
│ ├─ /dashboard — Ops dashboard (client-side, auto-refresh 15s)
|
|
||||||
│ ├─ /api/dashboard — JSON: systemd services, Jaeger traces, Prometheus metrics
|
|
||||||
│ └─ Metrics via node-exporter + app endpoints
|
|
||||||
│
|
|
||||||
▼ Service load balancer
|
|
||||||
http://scraper-api:4091
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
Scraper API (Rust / Axum)
|
|
||||||
├─ Health check: GET /, respon 200
|
|
||||||
├─ REST endpoints
|
|
||||||
├─ Database via `DATABASE_URL` (Tailscale → PostgreSQL di imrnes)
|
|
||||||
├─ Cache via `REDIS_URL` (Redis lokal di container)
|
|
||||||
└─ Pub/sub via Dapr sidecar (localhost:3500)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Infrastruktur Internal
|
|
||||||
|
|
||||||
### Docker Compose Project
|
|
||||||
|
|
||||||
Semua service berjalan dalam satu Docker Compose project bernama `compose` dan bergabung di network `app-shared-net`:
|
|
||||||
|
|
||||||
| File | Service | Peran |
|
|
||||||
|------|---------|-------|
|
|
||||||
| `traefik.yml` | `traefik` | Reverse proxy + TLS + metrics Prometheus |
|
|
||||||
| `shared.yml` | `redis` | Cache, session store, backend Dapr pub/sub & state |
|
|
||||||
| `nats.yml` | `nats` | Message broker + JetStream persistent streaming |
|
|
||||||
| `dapr.yml` | `dapr-placement` | Koordinasi actor placement untuk sidecar Dapr |
|
|
||||||
| `scraper.yml` | `scraper-api` + `scraper-api-dapr` | Aplikasi Rust + sidecar Dapr |
|
|
||||||
| systemd hub | `hub` | Next.js SPA portfolio + dashboard |
|
|
||||||
| `observability.yml` | `otel-collector`, `jaeger`, `prometheus`, `node-exporter` | Tracing, metrics, observability |
|
|
||||||
|
|
||||||
### Dapr Sidecar Pattern
|
|
||||||
|
|
||||||
Setiap aplikasi yang menggunakan Dapr mendapat sidecar container `daprd`:
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────────────┐
|
|
||||||
│ scraper-api │
|
|
||||||
│ (app port 4091) │
|
|
||||||
└────────┬────────────┘
|
|
||||||
│ localhost:3500 (HTTP)
|
|
||||||
│ localhost:50001 (gRPC)
|
|
||||||
┌────────▼────────────┐
|
|
||||||
│ scraper-api-dapr │
|
|
||||||
│ (daprd sidecar) │
|
|
||||||
│ │
|
|
||||||
│ Dapr components: │
|
|
||||||
│ ├─ pubsub.redis │
|
|
||||||
│ └─ state.redis │
|
|
||||||
└─────────────────────┘
|
|
||||||
```
|
|
||||||
|
|
||||||
Komponen Dapr:
|
|
||||||
|
|
||||||
| Komponen | Tipe | Backend |
|
|
||||||
|----------|------|---------|
|
|
||||||
| `pubsub` | `pubsub.redis` | `redis:6379` |
|
|
||||||
| `statestore` | `state.redis` | `redis:6379` (prefix `dapr`) |
|
|
||||||
|
|
||||||
### Monitoring & Auto-Discovery
|
|
||||||
|
|
||||||
#### Prometheus Docker Auto-Discovery
|
|
||||||
|
|
||||||
Prometheus menggunakan `docker_sd_configs` untuk auto-detect container yang perlu di-scrape. Cukup tambah label pada container:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
labels:
|
|
||||||
- 'prometheus.io/scrape=true'
|
|
||||||
- 'prometheus.io/port=8080' # port metrics endpoint
|
|
||||||
- 'prometheus.io/path=/metrics' # optional, default /metrics
|
|
||||||
```
|
|
||||||
|
|
||||||
Prometheus akan auto-detect dan mulai scrape container dalam 15 detik.
|
|
||||||
|
|
||||||
#### Traefik Metrics
|
|
||||||
|
|
||||||
Traefik mengekspos metrics Prometheus di port 8080 (`--metrics.prometheus=true`). Metrics yang tersedia:
|
|
||||||
|
|
||||||
| Metric | Query untuk dashboard |
|
|
||||||
|--------|----------------------|
|
|
||||||
| Request rate | `sum(rate(traefik_service_requests_total[1m]))` |
|
|
||||||
| Latency | `avg(traefik_service_request_duration_seconds_sum / traefik_service_request_duration_seconds_count) * 1000` |
|
|
||||||
| Error rate | `sum(rate(traefik_service_requests_total{code=~"5.."}[1m]))` |
|
|
||||||
|
|
||||||
Dashboard di `/api/dashboard` returns node metrics + Traefik range data untuk 4 sparkline charts (RPS, latency, errors, trace volume).
|
|
||||||
|
|
||||||
#### Docker Socket Access
|
|
||||||
|
|
||||||
Container yang perlu akses Docker socket (`/var/run/docker.sock`) harus punya group docker (GID 988):
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
volumes:
|
|
||||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
||||||
group_add:
|
|
||||||
- '988'
|
|
||||||
```di compose atau `--group-add 988` via CLI. Berlaku untuk `hub` (container list) dan `prometheus` (Docker SD).
|
|
||||||
|
|
||||||
### NATS + JetStream
|
|
||||||
|
|
||||||
NATS berjalan dengan flag `-js` untuk mengaktifkan JetStream. Persistent stream disimpan di volume `nats_data`. Dapr pub/sub routing:
|
|
||||||
|
|
||||||
```
|
|
||||||
Service → Dapr sidecar (pubsub.redis) → Redis streams
|
|
||||||
```
|
|
||||||
|
|
||||||
> **Catatan:** Saat ini Dapr pub/sub menggunakan Redis, bukan NATS. Jika ingin migrasi ke NATS untuk pub/sub, komponen Dapr perlu diganti dengan `pubsub.nats`.
|
|
||||||
|
|
||||||
## Arsitektur CI/CD
|
|
||||||
|
|
||||||
```
|
|
||||||
Push ke main (apps/**, infra/**)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
docker-build-push.yml
|
|
||||||
├─ Phase 1: Detect changed services
|
|
||||||
├─ Phase 2: Build & Push image ke GHCR
|
|
||||||
└─ Phase 3: Update compose manifest + submodule pointer
|
|
||||||
│
|
|
||||||
▼ (workflow_run trigger)
|
|
||||||
deploy-docker.yml
|
|
||||||
├─ SSH ke orangevps
|
|
||||||
├─ Git sync, pull images
|
|
||||||
├─ Remove stale containers
|
|
||||||
└─ Selective restart service
|
|
||||||
```
|
|
||||||
|
|
||||||
Submodule update dari remote repo via `repository_dispatch`:
|
|
||||||
|
|
||||||
```
|
|
||||||
Push ke asepharyana-hub-scraper
|
|
||||||
│
|
|
||||||
▼ (repository_dispatch)
|
|
||||||
update-submodule.yml
|
|
||||||
├─ Update submodule pointer
|
|
||||||
└─ Commit & push ke hub repo
|
|
||||||
│
|
|
||||||
▼ (repository_dispatch trigger)
|
|
||||||
docker-build-push.yml
|
|
||||||
└─ Build, push, deploy
|
|
||||||
```
|
|
||||||
|
|
||||||
## Image Tagging Strategy
|
|
||||||
|
|
||||||
| Tag | Contoh | Penggunaan |
|
|
||||||
|-----|--------|------------|
|
|
||||||
| `sha-<short>` | `sha-a3c5d74` | Immutable, deterministic rollback |
|
|
||||||
| `latest` | `latest` | Mutable, convenience |
|
|
||||||
| `buildcache` | `sha-a3c5d74-buildcache` | Registry-based build cache (internal) |
|
|
||||||
|
|
||||||
## Networking
|
|
||||||
|
|
||||||
### Port Map
|
|
||||||
|
|
||||||
| Port | Service | Deskripsi |
|
|
||||||
|------|---------|-----------|
|
|
||||||
| 443 | Traefik | HTTPS eksternal |
|
|
||||||
| 80 | Traefik | Redirect ke HTTPS |
|
|
||||||
| 4222 | NATS | Client connections |
|
|
||||||
| 8222 | NATS | HTTP monitor / health |
|
|
||||||
| 6379 | Redis | Internal container network |
|
|
||||||
| 3500 | Dapr sidecar | Dapr HTTP API (per service) |
|
|
||||||
| 50001 | Dapr sidecar | Dapr gRPC API (per service) |
|
|
||||||
| 50005 | Dapr placement | Actor placement |
|
|
||||||
| 4091 | Scraper API | Aplikasi HTTP |
|
|
||||||
|
|
||||||
## Event Topics Convention
|
|
||||||
|
|
||||||
Semua event menggunakan prefix `hub.`:
|
|
||||||
|
|
||||||
| Topic | Payload | Deskripsi |
|
|
||||||
|-------|---------|-----------|
|
|
||||||
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
|
|
||||||
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
|
|
||||||
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
|
|
||||||
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |
|
|
||||||
|
|
||||||
## Service Registry (Traefik)
|
|
||||||
|
|
||||||
Domain routing:
|
|
||||||
|
|
||||||
| Subdomain | Service | URL Backend |
|
|
||||||
|-----------|---------|-------------|
|
|
||||||
| `asepharyana.my.id` (root) | Hub SPA + dashboard | `http://hub:3000` |
|
|
||||||
| `www.*` | Hub (alias) | `http://hub:3000` |
|
|
||||||
| `hub.*` | Hub (alias) | `http://hub:3000` |
|
|
||||||
| `scraper.*` | Scraper API | `http://scraper-api:4091` |
|
|
||||||
| `api.*` | Scraper API (alias) | `http://scraper-api:4091` |
|
|
||||||
| `traefik.*` | Traefik Dashboard | `api@internal` |
|
|
||||||
| `jaeger.*` | Jaeger UI | `http://jaeger:16686` |
|
|
||||||
|
|
||||||
Semua domain tersedia di:
|
|
||||||
- `<service>.asepharyana.my.id`
|
|
||||||
- `<service>.asepharyana.web.id`
|
|
||||||
@@ -1,681 +0,0 @@
|
|||||||
# Deployment Guide
|
|
||||||
|
|
||||||
Panduan deploy aplikasi apapun menggunakan **Docker + Docker Compose + GitHub Actions + VPS**.
|
|
||||||
|
|
||||||
## Arsitektur
|
|
||||||
|
|
||||||
```
|
|
||||||
GitHub Repo ──► GitHub Actions ──► Registry (GHCR / Docker Hub / ECR / dll.)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
VPS (<VPS_HOST>)
|
|
||||||
docker compose pull + up
|
|
||||||
```
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Docker Engine >= 24.x
|
|
||||||
- Docker Compose v2 (plugin)
|
|
||||||
- Git
|
|
||||||
- Akun GitHub dengan akses repo
|
|
||||||
- SSH key di `~/.ssh/<KEY_NAME>` (default: `id_ed25519`)
|
|
||||||
|
|
||||||
## Konfigurasi VPS Target
|
|
||||||
|
|
||||||
Buat berkas `~/orangevps` (atau sesuaikan dengan env Anda):
|
|
||||||
|
|
||||||
```text
|
|
||||||
ssh <USER>@<VPS_HOST>
|
|
||||||
```
|
|
||||||
|
|
||||||
Contoh isi `~/orangevps`:
|
|
||||||
|
|
||||||
```text
|
|
||||||
ssh root@45.127.35.244
|
|
||||||
```
|
|
||||||
|
|
||||||
| Parameter | Nilai | Contoh |
|
|
||||||
|-----------|-------|--------|
|
|
||||||
| User | `<USER>` | `root` |
|
|
||||||
| Host | `<VPS_HOST>` | `45.127.35.244` |
|
|
||||||
| SSH Key | `~/.ssh/<KEY_NAME>` | `~/.ssh/id_ed25519` |
|
|
||||||
| Target Dir di VPS | `<VPS_TARGET_DIR>` | `/opt/app` atau `/root/app` |
|
|
||||||
|
|
||||||
> Tip: Jika SSH key menggunakan nama selain default, sesuaikan path dan `ssh -i` sesuai.
|
|
||||||
|
|
||||||
## Registry
|
|
||||||
|
|
||||||
Pilih registry untuk menyimpan image Docker. Sesuaikan dengan proyek:
|
|
||||||
|
|
||||||
| Registry | URL | Auth |
|
|
||||||
|----------|-----|------|
|
|
||||||
| GitHub Container Registry | `ghcr.io` | `GITHUB_TOKEN` |
|
|
||||||
| Docker Hub | `docker.io` | username / PAT |
|
|
||||||
| AWS ECR | `<account>.dkr.ecr.<region>.amazonaws.com` | `aws ecr get-login-password` |
|
|
||||||
| Google GCR | `gcr.io` | `gcloud auth print-access-token` |
|
|
||||||
| Azure ACR | `<registry>.azurecr.io` | `az acr login` |
|
|
||||||
|
|
||||||
Contoh namespace untuk GHCR:
|
|
||||||
|
|
||||||
```text
|
|
||||||
Registry : ghcr.io
|
|
||||||
Namespace: <GITHUB_USERNAME_OR_ORG>
|
|
||||||
Repo : <REPO_NAME>
|
|
||||||
```
|
|
||||||
|
|
||||||
Pastikan package/visibility di registry mengizinkan akses pull dari VPS.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deploy Otomatis (Recommended)
|
|
||||||
|
|
||||||
Gunakan GitHub Actions untuk otomatisasi build, push, dan deploy.
|
|
||||||
|
|
||||||
### Workflow 1: Build dan Push Image
|
|
||||||
|
|
||||||
File: `.github/workflows/docker-build-push.yml`
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: Build and Push Docker Images
|
|
||||||
|
|
||||||
on:
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
permissions:
|
|
||||||
contents: read
|
|
||||||
packages: write
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
build:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v6
|
|
||||||
|
|
||||||
- uses: docker/login-action@v4
|
|
||||||
with:
|
|
||||||
registry: ghcr.io
|
|
||||||
username: ${{ github.actor }}
|
|
||||||
password: ${{ secrets.GITHUB_TOKEN }}
|
|
||||||
|
|
||||||
- uses: docker/setup-buildx-action@v4
|
|
||||||
|
|
||||||
- uses: docker/build-push-action@v7
|
|
||||||
with:
|
|
||||||
context: .
|
|
||||||
file: Dockerfile
|
|
||||||
push: true
|
|
||||||
tags: |
|
|
||||||
ghcr.io/${{ github.repository }}/<SERVICE_NAME>:latest
|
|
||||||
ghcr.io/${{ github.repository }}/<SERVICE_NAME>:sha-${{ github.sha }}
|
|
||||||
cache-from: type=registry,ref=ghcr.io/${{ github.repository }}/<SERVICE_NAME>:buildcache
|
|
||||||
cache-to: type=registry,ref=ghcr.io/${{ github.repository }}/<SERVICE_NAME>:buildcache,mode=max
|
|
||||||
```
|
|
||||||
|
|
||||||
Ubah `<SERVICE_NAME>` sesuai service (misal: `app`, `web`, `api`). Jika monorepo, gunakan matrix strategy untuk build beberapa service sekaligus.
|
|
||||||
|
|
||||||
### Workflow 2: Deploy ke VPS
|
|
||||||
|
|
||||||
File: `.github/workflows/deploy-docker.yml`
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
name: Deploy Docker to VPS
|
|
||||||
|
|
||||||
on:
|
|
||||||
workflow_run:
|
|
||||||
workflows: ['Build and Push Docker Images']
|
|
||||||
types: [completed]
|
|
||||||
push:
|
|
||||||
branches: [main]
|
|
||||||
workflow_dispatch:
|
|
||||||
|
|
||||||
jobs:
|
|
||||||
deploy:
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v6
|
|
||||||
|
|
||||||
- name: Deploy to VPS
|
|
||||||
env:
|
|
||||||
SSH_PRIVATE_KEY: ${{ secrets.SSH_PRIVATE_KEY }}
|
|
||||||
VPS_HOST: ${{ secrets.VPS_HOST }}
|
|
||||||
VPS_USER: ${{ secrets.VPS_USER }}
|
|
||||||
VPS_TARGET_DIR: ${{ secrets.VPS_TARGET_DIR }}
|
|
||||||
ENV_FILE_PRODUCTION: ${{ secrets.ENV_FILE_PRODUCTION }}
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
mkdir -p ~/.ssh
|
|
||||||
echo "$SSH_PRIVATE_KEY" > ~/.ssh/id_rsa
|
|
||||||
chmod 600 ~/.ssh/id_rsa
|
|
||||||
ssh-keyscan -H -t ed25519,rsa "$VPS_HOST" >> ~/.ssh/known_hosts
|
|
||||||
|
|
||||||
SSH_OPTS=(-o ControlMaster=auto -o ControlPath=/tmp/ssh-%r@%h:%p -o ControlPersist=600 -o StrictHostKeyChecking=yes)
|
|
||||||
|
|
||||||
ssh "${SSH_OPTS[@]}" "$VPS_USER@$VPS_HOST" "mkdir -p $VPS_TARGET_DIR && mkdir -p $VPS_TARGET_DIR/infra/compose"
|
|
||||||
echo "$ENV_FILE_PRODUCTION" > .env.prod
|
|
||||||
scp "${SSH_OPTS[@]}" .env.prod "$VPS_USER@$VPS_HOST:$VPS_TARGET_DIR/.env"
|
|
||||||
|
|
||||||
ssh "${SSH_OPTS[@]}" "$VPS_USER@$VPS_HOST" bash -s <<'EOF'
|
|
||||||
set -euo pipefail
|
|
||||||
cd "$VPS_TARGET_DIR"
|
|
||||||
|
|
||||||
docker network inspect app-shared-net >/dev/null 2>&1 || docker network create app-shared-net
|
|
||||||
|
|
||||||
if [ ! -d ".git" ]; then
|
|
||||||
git init
|
|
||||||
git remote add origin https://github.com/<GITHUB_USER>/<REPO_NAME>.git
|
|
||||||
fi
|
|
||||||
git fetch origin main --depth=1 || true
|
|
||||||
git reset --hard FETCH_HEAD
|
|
||||||
|
|
||||||
docker compose --env-file .env pull
|
|
||||||
docker compose --env-file .env up -d --remove-orphans
|
|
||||||
EOF
|
|
||||||
```
|
|
||||||
|
|
||||||
### Secrets GitHub yang Diperlukan
|
|
||||||
|
|
||||||
Buka **Settings > Secrets and variables > Actions**:
|
|
||||||
|
|
||||||
| Secret | Deskripsi |
|
|
||||||
|--------|-----------|
|
|
||||||
| `SSH_PRIVATE_KEY` | Isi dengan `cat ~/.ssh/<KEY_NAME>` |
|
|
||||||
| `VPS_HOST` | IP atau domain VPS |
|
|
||||||
| `VPS_USER` | User SSH (misal: `root`, `ubuntu`, `deploy`) |
|
|
||||||
| `VPS_TARGET_DIR` | Direktori aplikasi di VPS |
|
|
||||||
| `ENV_FILE_PRODUCTION` | Isi dengan environment production |
|
|
||||||
|
|
||||||
### Trigger Manual
|
|
||||||
|
|
||||||
```bash
|
|
||||||
gh workflow run deploy-docker.yml
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Dockerfile Patterns
|
|
||||||
|
|
||||||
Pilih pattern sesuai jenis aplikasi.
|
|
||||||
|
|
||||||
### Pattern 1: Multi-stage Build (SPA / static assets)
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM oven/bun:1 AS builder
|
|
||||||
WORKDIR /app
|
|
||||||
COPY package.json bun.lock ./
|
|
||||||
RUN bun install --frozen-lockfile
|
|
||||||
COPY . .
|
|
||||||
RUN bun run build
|
|
||||||
|
|
||||||
FROM nginx:alpine
|
|
||||||
COPY --from=builder /app/dist /usr/share/nginx/html
|
|
||||||
COPY nginx.conf /etc/nginx/conf.d/default.conf
|
|
||||||
EXPOSE 80
|
|
||||||
CMD ["nginx", "-g", "daemon off;"]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pattern 2: Single-stage (runtime image)
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM oven/bun:1
|
|
||||||
WORKDIR /app
|
|
||||||
COPY package.json bun.lock ./
|
|
||||||
RUN bun install --frozen-lockfile
|
|
||||||
COPY . .
|
|
||||||
EXPOSE 3000
|
|
||||||
CMD ["bun", "run", "start"]
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pattern 3: Compiled binary (Rust / Go / Zig)
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
FROM rust:1 AS builder
|
|
||||||
WORKDIR /app
|
|
||||||
COPY . .
|
|
||||||
RUN cargo build --release
|
|
||||||
|
|
||||||
FROM debian:bookworm-slim
|
|
||||||
COPY --from=builder /app/target/release/app /usr/local/bin/app
|
|
||||||
EXPOSE 8080
|
|
||||||
CMD ["app"]
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Docker Compose Patterns
|
|
||||||
|
|
||||||
### Single service
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
container_name: app
|
|
||||||
image: registry.example.com/org/app:latest
|
|
||||||
restart: always
|
|
||||||
ports:
|
|
||||||
- "3000:3000"
|
|
||||||
environment:
|
|
||||||
- NODE_ENV=production
|
|
||||||
```
|
|
||||||
|
|
||||||
### Multi-service dengan shared network
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
container_name: app
|
|
||||||
image: registry.example.com/org/app:latest
|
|
||||||
restart: always
|
|
||||||
networks: [app-shared-net]
|
|
||||||
|
|
||||||
redis:
|
|
||||||
container_name: redis
|
|
||||||
image: redis:7-alpine
|
|
||||||
restart: always
|
|
||||||
networks: [app-shared-net]
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dengan reverse proxy (Traefik / Caddy / Nginx)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
container_name: app
|
|
||||||
image: registry.example.com/org/app:latest
|
|
||||||
restart: always
|
|
||||||
networks: [app-shared-net]
|
|
||||||
labels:
|
|
||||||
- 'traefik.enable=true'
|
|
||||||
- 'traefik.http.routers.app.rule=Host(`app.example.com`)'
|
|
||||||
- 'traefik.http.routers.app.entrypoints=websecure'
|
|
||||||
- 'traefik.http.routers.app.tls=true'
|
|
||||||
- 'traefik.http.services.app.loadbalancer.server.port=3000'
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deploy Manual (Lokal)
|
|
||||||
|
|
||||||
### 1. Build dan Push ke Registry
|
|
||||||
|
|
||||||
Login ke registry:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
echo $GITHUB_TOKEN | docker login ghcr.io -u <GITHUB_USERNAME> --password-stdin
|
|
||||||
```
|
|
||||||
|
|
||||||
Build dan push:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker build -t ghcr.io/<GITHUB_USERNAME>/<REPO_NAME>/<SERVICE_NAME>:latest -f Dockerfile .
|
|
||||||
|
|
||||||
docker push ghcr.io/<GITHUB_USERNAME>/<REPO_NAME>/<SERVICE_NAME>:latest
|
|
||||||
```
|
|
||||||
|
|
||||||
Tag tambahan dengan SHA commit:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
SHORT_SHA=$(git rev-parse --short HEAD)
|
|
||||||
docker tag ghcr.io/<GITHUB_USERNAME>/<REPO_NAME>/<SERVICE_NAME>:latest \
|
|
||||||
ghcr.io/<GITHUB_USERNAME>/<REPO_NAME>/<SERVICE_NAME>:sha-${SHORT_SHA}
|
|
||||||
docker push ghcr.io/<GITHUB_USERNAME>/<REPO_NAME>/<SERVICE_NAME>:sha-${SHORT_SHA}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Pull dan Deploy di VPS
|
|
||||||
|
|
||||||
SSH ke VPS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ssh -i ~/.ssh/<KEY_NAME> <USER>@<VPS_HOST>
|
|
||||||
```
|
|
||||||
|
|
||||||
Clone repo (jika belum):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git clone https://github.com/<GITHUB_USER>/<REPO_NAME>.git <VPS_TARGET_DIR>
|
|
||||||
cd <VPS_TARGET_DIR>
|
|
||||||
```
|
|
||||||
|
|
||||||
Buat shared network (hanya sekali):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker network create app-shared-net
|
|
||||||
```
|
|
||||||
|
|
||||||
Siapkan environment:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cp .env.example .env
|
|
||||||
# Edit .env sesuai nilai production
|
|
||||||
nano .env
|
|
||||||
```
|
|
||||||
|
|
||||||
Login ke registry di VPS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
echo $GITHUB_TOKEN | docker login ghcr.io -u <GITHUB_USERNAME> --password-stdin
|
|
||||||
```
|
|
||||||
|
|
||||||
Pull gambar terbaru:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd <VPS_TARGET_DIR>
|
|
||||||
docker compose -f docker-compose.yml --env-file .env pull
|
|
||||||
```
|
|
||||||
|
|
||||||
Deploy (up):
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f docker-compose.yml --env-file .env up -d --remove-orphans
|
|
||||||
```
|
|
||||||
|
|
||||||
Verifikasi:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f docker-compose.yml ps
|
|
||||||
docker compose -f docker-compose.yml logs -f <SERVICE_NAME>
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Deployment Order (Manual)
|
|
||||||
|
|
||||||
Jika deploy bertahap, gunakan urutan ini:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Shared services (Redis, database, dll.)
|
|
||||||
docker compose -f infra/compose/shared.yml up -d
|
|
||||||
|
|
||||||
# 2. Reverse proxy
|
|
||||||
docker compose -f infra/compose/traefik.yml up -d
|
|
||||||
|
|
||||||
# 3. Aplikasi
|
|
||||||
docker compose \
|
|
||||||
-f infra/compose/app1.yml \
|
|
||||||
-f infra/compose/app2.yml \
|
|
||||||
up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Perintah Berguna di VPS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Lihat semua container
|
|
||||||
docker ps -a
|
|
||||||
|
|
||||||
# Log service
|
|
||||||
docker logs -f <container_name>
|
|
||||||
|
|
||||||
# Restart satu service
|
|
||||||
docker compose -f <compose_file> up -d --force-recreate
|
|
||||||
|
|
||||||
# Hapus network lama (hati-hati)
|
|
||||||
docker network rm app-shared-net
|
|
||||||
docker network create app-shared-net
|
|
||||||
|
|
||||||
# Bersihkan image unused
|
|
||||||
docker image prune -a -f
|
|
||||||
docker system prune -a -f
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Image tidak bisa di-pull
|
|
||||||
|
|
||||||
Pastikan sudah login ke registry di VPS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logout ghcr.io
|
|
||||||
echo $GITHUB_TOKEN | docker login ghcr.io -u <GITHUB_USERNAME> --password-stdin
|
|
||||||
```
|
|
||||||
|
|
||||||
Periksa visibility package di registry (harus `Public` atau akses diberikan).
|
|
||||||
|
|
||||||
### Port sudah dipakai
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker ps | grep :80
|
|
||||||
docker ps | grep :443
|
|
||||||
```
|
|
||||||
|
|
||||||
### Reverse proxy tidak routing
|
|
||||||
|
|
||||||
Periksa label di compose file dan pastikan shared network ada:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker network inspect app-shared-net
|
|
||||||
docker logs traefik
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Environment Variable Management
|
|
||||||
|
|
||||||
### Pola 1: `.env` di VPS (recommended untuk production)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Di VPS
|
|
||||||
cd <VPS_TARGET_DIR>
|
|
||||||
cp .env.example .env
|
|
||||||
# Edit sesuai production
|
|
||||||
nano .env
|
|
||||||
```
|
|
||||||
|
|
||||||
CI/CD upload `.env` via secret, tidak simpan di repo.
|
|
||||||
|
|
||||||
### Pola 2: Docker secrets (Swarm mode)
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
image: app:latest
|
|
||||||
secrets:
|
|
||||||
- db_password
|
|
||||||
|
|
||||||
secrets:
|
|
||||||
db_password:
|
|
||||||
file: ./secrets/db_password.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pola 3: External secret manager
|
|
||||||
|
|
||||||
- **HashiCorp Vault**: inject via env atau file
|
|
||||||
- **AWS Secrets Manager**: `aws secretsmanager get-secret-value`
|
|
||||||
- **Doppler / Infisical**: unified secret management
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Tagging dan Versioning
|
|
||||||
|
|
||||||
### Strategy yang umum
|
|
||||||
|
|
||||||
| Strategy | Contoh tag | Kegunaan |
|
|
||||||
|----------|-----------|----------|
|
|
||||||
| Latest + SHA | `latest`, `sha-abc1234` | CI/CD cepat, traceable |
|
|
||||||
| SemVer | `1.2.3`, `1.2`, `1` | Release publik |
|
|
||||||
| Git tag mirror | `v1.2.3` | Sync dengan git tag |
|
|
||||||
| Branch mirror | `main`, `develop` | Preview / staging |
|
|
||||||
|
|
||||||
### Contoh git tag driven deploy
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git tag v1.2.3
|
|
||||||
git push origin v1.2.3
|
|
||||||
```
|
|
||||||
|
|
||||||
CI/CD membaca tag, build image dengan tag yang sama, dan deploy.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
|
|
||||||
### Rollback via registry
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Lihat tag yang tersedia
|
|
||||||
docker manifest inspect ghcr.io/org/app:latest
|
|
||||||
# atau lihat UI registry
|
|
||||||
|
|
||||||
# Di VPS, edit compose file ke tag sebelumnya
|
|
||||||
# lalu:
|
|
||||||
docker compose --env-file .env pull
|
|
||||||
docker compose --env-file .env up -d --remove-orphans
|
|
||||||
```
|
|
||||||
|
|
||||||
### Rollback via git
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git revert HEAD
|
|
||||||
git push origin main
|
|
||||||
# CI/CD otomatis build dan deploy versi sebelumnya
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Health Checks
|
|
||||||
|
|
||||||
### Di Dockerfile
|
|
||||||
|
|
||||||
```dockerfile
|
|
||||||
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
|
|
||||||
CMD curl -f http://localhost:3000/health || exit 1
|
|
||||||
```
|
|
||||||
|
|
||||||
### Di Docker Compose
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
image: app:latest
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 3s
|
|
||||||
retries: 3
|
|
||||||
start_period: 10s
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Monitoring & Observability
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Log aggregated
|
|
||||||
docker compose logs -f --tail=100
|
|
||||||
|
|
||||||
# Resource usage
|
|
||||||
docker stats
|
|
||||||
|
|
||||||
# Disk usage
|
|
||||||
docker system df
|
|
||||||
|
|
||||||
# Cleanup
|
|
||||||
docker system prune -a -f
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Catatan Keamanan
|
|
||||||
|
|
||||||
- Jangan commit `.env` atau SSH private key ke repo.
|
|
||||||
- Gunakan GitHub Secrets (atau secret manager) untuk credential di CI/CD.
|
|
||||||
- Rotate token dan key secara berkala.
|
|
||||||
- Batasi akses SSH ke VPS (ubah port default, gunakan fail2ban).
|
|
||||||
- Set `StrictHostKeyChecking=yes` pada SSH opsional deployment.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Proyek Ini: asepharyana-hub
|
|
||||||
|
|
||||||
> Dokumentasi spesifik untuk repo ini. Lihat juga [ADR-0002](adr/0002-env-file-via-github-secret.md).
|
|
||||||
|
|
||||||
### Topologi
|
|
||||||
|
|
||||||
| Host | IP | Peran |
|
|
||||||
|------|----|-------|
|
|
||||||
| `orangevps` (VPS) | `45.127.35.244` | Docker host: Traefik, scraper-api, Redis, NATS, Dapr |
|
|
||||||
| `imrnes` (bare-metal) | `100.121.180.82` (Tailscale) | PostgreSQL (port 6432), Redis (port 6379) |
|
|
||||||
|
|
||||||
### Environment Variables
|
|
||||||
|
|
||||||
**Production `.env` tidak pernah di-commit.** File ini disimpan sebagai GitHub secret `ENV_FILE_PRODUCTION` dan di-SCP ke VPS saat deploy via `deploy-docker.yml`.
|
|
||||||
|
|
||||||
Cara update:
|
|
||||||
```bash
|
|
||||||
# Baca current .env dari VPS
|
|
||||||
ssh root@45.127.35.244 "cat /root/asepharyana-hub/.env"
|
|
||||||
|
|
||||||
# Update GitHub secret (dari output di atas)
|
|
||||||
cat > /tmp/env-updated << 'EOF'
|
|
||||||
<paste content, edit, lalu>
|
|
||||||
EOF
|
|
||||||
cat /tmp/env-updated | gh secret set ENV_FILE_PRODUCTION --repo asepharyana/asepharyana-hub
|
|
||||||
```
|
|
||||||
|
|
||||||
**Jangan manual edit `.env` di VPS tanpa update GitHub secret juga** — nanti ke- overwrite pas deploy berikutnya.
|
|
||||||
|
|
||||||
### Database
|
|
||||||
|
|
||||||
| Variable | Value |
|
|
||||||
|----------|-------|
|
|
||||||
| `DATABASE_URL` | `postgres://asephs:hunterz@100.121.180.82:6432/hub` |
|
|
||||||
| `REDIS_URL` | `redis://redis:6379` (Docker network) |
|
|
||||||
|
|
||||||
### Kompose
|
|
||||||
|
|
||||||
Proyek compose bernama `compose`, terdiri dari 5 file yang selalu di-include bersamaan:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
/root/asepharyana-hub/infra/compose/
|
|
||||||
├── traefik.yml # Reverse proxy
|
|
||||||
├── shared.yml # Redis
|
|
||||||
├── nats.yml # NATS
|
|
||||||
├── dapr.yml # Dapr placement
|
|
||||||
├── scraper.yml # Scraper API
|
|
||||||
└── observability.yml # OTel Collector, Jaeger, Dashboard
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd /root/asepharyana-hub
|
|
||||||
docker compose \
|
|
||||||
-p compose \
|
|
||||||
--env-file .env \
|
|
||||||
-f infra/compose/traefik.yml \
|
|
||||||
-f infra/compose/shared.yml \
|
|
||||||
-f infra/compose/scraper.yml \
|
|
||||||
-f infra/compose/nats.yml \
|
|
||||||
-f infra/compose/dapr.yml \
|
|
||||||
-f infra/compose/observability.yml \
|
|
||||||
up -d --remove-orphans
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Checklist Deploy Proyek Baru
|
|
||||||
|
|
||||||
1. [ ] Dockerfile ditest lokal (`docker build`, `docker run`)
|
|
||||||
2. [ ] Docker Compose file valid (`docker compose config`)
|
|
||||||
3. [ ] `.dockerignore` sesuai (node_modules, .git, .env)
|
|
||||||
4. [ ] Registry dibuat (GHCR package / Docker Hub repo / ECR / dll.)
|
|
||||||
5. [ ] GitHub Actions workflow dibuat dengan permission `packages: write`
|
|
||||||
6. [ ] VPS siap: Docker, Docker Compose, SSH key
|
|
||||||
7. [ ] Shared network dibuat (`docker network create`)
|
|
||||||
8. [ ] `.env` production di-VPS atau via secret manager
|
|
||||||
9. [ ] Reverse proxy (Traefik / Caddy / Nginx) routing ke container
|
|
||||||
10. [ ] Health check endpoint aktif
|
|
||||||
@@ -1,206 +0,0 @@
|
|||||||
# Development Guide
|
|
||||||
|
|
||||||
Panduan setup lingkungan development lokal untuk kontributor `asepharyana-hub`.
|
|
||||||
|
|
||||||
## Prasyarat
|
|
||||||
|
|
||||||
| Tool | Versi Minimal | Catatan |
|
|
||||||
|------|---------------|---------|
|
|
||||||
| Git | 2.40+ | Submodule support |
|
|
||||||
| Docker | 24+ | Dengan Docker Compose v2 plugin |
|
|
||||||
| Rust | 1.85+ | Hanya untuk `apps/scraper` |
|
|
||||||
| Bun | 1.x | Root tooling (Biome) |
|
|
||||||
| Dapr CLI | 1.14+ | Opsional, untuk development dengan Dapr |
|
|
||||||
|
|
||||||
## Setup Awal
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Clone repo
|
|
||||||
git clone https://github.com/asepharyana/asepharyana-hub.git
|
|
||||||
cd asepharyana-hub
|
|
||||||
|
|
||||||
# 2. Init submodules
|
|
||||||
make init-submodules
|
|
||||||
|
|
||||||
# 3. Setup environment
|
|
||||||
cp .env.example .env
|
|
||||||
# Edit .env sesuai kebutuhan lokal
|
|
||||||
|
|
||||||
# 4. Install root dependencies
|
|
||||||
bun install
|
|
||||||
```
|
|
||||||
|
|
||||||
## Menjalankan Infrastruktur Lokal
|
|
||||||
|
|
||||||
Beberapa service membutuhkan Redis. Jalankan dengan:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make dev
|
|
||||||
# atau equivalen:
|
|
||||||
docker compose -f infra/compose/shared.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Ini akan menjalankan Redis Alpine di `localhost:6379`.
|
|
||||||
|
|
||||||
### (Opsional) NATS Lokal
|
|
||||||
|
|
||||||
Jika service membutuhkan pub/sub:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f infra/compose/nats.yml up -d
|
|
||||||
# NATS client: localhost:4222
|
|
||||||
# NATS monitor: localhost:8222
|
|
||||||
```
|
|
||||||
|
|
||||||
### (Opsional) Dapr Placement Lokal
|
|
||||||
|
|
||||||
Jika service membutuhkan sidecar Dapr:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f infra/compose/dapr.yml up -d
|
|
||||||
# Dapr placement: localhost:50005
|
|
||||||
```
|
|
||||||
|
|
||||||
## Menjalankan Service Lokal
|
|
||||||
|
|
||||||
### Scraper API (Rust)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Pastikan Redis sudah running (make dev)
|
|
||||||
cd apps/scraper
|
|
||||||
|
|
||||||
# Cargo run
|
|
||||||
cargo run
|
|
||||||
|
|
||||||
# Dengan Dapr sidecar (jika placement running)
|
|
||||||
dapr run \
|
|
||||||
--app-id scraper-api \
|
|
||||||
--app-port 4091 \
|
|
||||||
--dapr-http-port 3500 \
|
|
||||||
--resources-path ../../infra/dapr/components \
|
|
||||||
-- cargo run
|
|
||||||
```
|
|
||||||
|
|
||||||
### Dengan Docker Compose (Full Stack)
|
|
||||||
|
|
||||||
Untuk menjalankan semua service sekaligus:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose \
|
|
||||||
-f infra/compose/shared.yml \
|
|
||||||
-f infra/compose/nats.yml \
|
|
||||||
-f infra/compose/dapr.yml \
|
|
||||||
-f infra/compose/scraper.yml \
|
|
||||||
--env-file .env \
|
|
||||||
up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Untuk service baru, tambahkan compose file-nya ke daftar.
|
|
||||||
|
|
||||||
## Update Submodules
|
|
||||||
|
|
||||||
### Pull latest dari semua submodule
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make update-submodules
|
|
||||||
# atau:
|
|
||||||
git submodule update --remote --merge --recursive
|
|
||||||
```
|
|
||||||
|
|
||||||
### Check status submodule
|
|
||||||
|
|
||||||
```bash
|
|
||||||
make status
|
|
||||||
# atau:
|
|
||||||
git submodule status
|
|
||||||
```
|
|
||||||
|
|
||||||
### Sync .env ke submodule
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/2updateenv.sh
|
|
||||||
# Copy .env root ke apps/*/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Linting & Formatting
|
|
||||||
|
|
||||||
Root repo menggunakan **Biome** untuk linting dan formatting:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bun run check # Lint + format + write
|
|
||||||
bun run ci # CI mode (no write, exit code on issues)
|
|
||||||
bun run lint # Lint only
|
|
||||||
bun run format # Format only
|
|
||||||
```
|
|
||||||
|
|
||||||
## Build Docker Image Lokal
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Scraper API
|
|
||||||
docker build -f infra/docker/scraper.Dockerfile -t scraper-api:local .
|
|
||||||
|
|
||||||
# Service baru: tambahkan Dockerfile di infra/docker/
|
|
||||||
```
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Saat ini belum ada test runner di root level. Masing-masing submodule mengelola testing sendiri:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Scraper API (Rust)
|
|
||||||
cd apps/scraper && cargo test
|
|
||||||
```
|
|
||||||
|
|
||||||
## Validasi YAML
|
|
||||||
|
|
||||||
Sebelum commit perubahan infra, validasi semua file YAML:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python -c "
|
|
||||||
import pathlib, yaml
|
|
||||||
for p in pathlib.Path('infra').rglob('*.yml'):
|
|
||||||
with open(p) as f: yaml.safe_load(f)
|
|
||||||
print(f'OK {p}')
|
|
||||||
for p in pathlib.Path('infra').rglob('*.yaml'):
|
|
||||||
with open(p) as f: yaml.safe_load(f)
|
|
||||||
print(f'OK {p}')
|
|
||||||
"
|
|
||||||
|
|
||||||
for f in infra/compose/*.yml; do
|
|
||||||
docker compose -f "$f" config >/dev/null && echo "OK $f"
|
|
||||||
done
|
|
||||||
```
|
|
||||||
|
|
||||||
## Git Workflow
|
|
||||||
|
|
||||||
### Commit Convention
|
|
||||||
|
|
||||||
```
|
|
||||||
<type>(<scope>): <description>
|
|
||||||
```
|
|
||||||
|
|
||||||
Type: `feat`, `fix`, `chore`, `docs`, `refactor`, `test`, `ci`, `perf`, `style`
|
|
||||||
Scope: `scraper`, `infra`, `ci`, `dapr`, `nats`, `docs`, `deps`, `scripts`, `root`
|
|
||||||
|
|
||||||
Contoh:
|
|
||||||
```
|
|
||||||
feat(scraper): add image cache endpoint
|
|
||||||
fix(infra): correct Traefik rate-limit config
|
|
||||||
chore(deps): bump biome to 2.5.0
|
|
||||||
```
|
|
||||||
|
|
||||||
### Branch Strategy
|
|
||||||
|
|
||||||
- `main` — production branch, push triggers CI/CD
|
|
||||||
- Fitur baru: branch dari `main`, PR ke `main`
|
|
||||||
- Submodule development: dilakukan di repo masing-masing, hub hanya update pointer
|
|
||||||
|
|
||||||
## Deployment ke VPS
|
|
||||||
|
|
||||||
Push ke `main` otomatis trigger CI/CD. Untuk trigger manual:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
gh workflow run deploy-docker.yml
|
|
||||||
```
|
|
||||||
|
|
||||||
Lihat `docs/DEPLOYMENT.md` untuk detail.
|
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
# Menambahkan Dapr ke Service Baru
|
|
||||||
|
|
||||||
Panduan integrasi Dapr runtime sidecar untuk service di `asepharyana-hub`.
|
|
||||||
|
|
||||||
## Prasyarat
|
|
||||||
|
|
||||||
- NATS server berjalan (`infra/compose/nats.yml`)
|
|
||||||
- Dapr placement service berjalan (`infra/compose/dapr.yml`)
|
|
||||||
|
|
||||||
## 1. Compose File
|
|
||||||
|
|
||||||
Setiap service butuh sidecar container Dapr. Contoh:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
app:
|
|
||||||
container_name: app
|
|
||||||
image: ghcr.io/asepharyana/asepharyana-hub/app:latest
|
|
||||||
restart: always
|
|
||||||
depends_on:
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_healthy
|
|
||||||
nats:
|
|
||||||
condition: service_healthy
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
aliases:
|
|
||||||
- app
|
|
||||||
env_file:
|
|
||||||
- ../../.env
|
|
||||||
|
|
||||||
app-dapr:
|
|
||||||
container_name: app-dapr
|
|
||||||
image: daprio/daprd:latest
|
|
||||||
restart: always
|
|
||||||
depends_on:
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_healthy
|
|
||||||
nats:
|
|
||||||
condition: service_healthy
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
depends_on:
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_healthy
|
|
||||||
nats:
|
|
||||||
condition: service_healthy
|
|
||||||
otel-collector:
|
|
||||||
condition: service_started
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
command:
|
|
||||||
- './daprd'
|
|
||||||
- '--app-id=app'
|
|
||||||
- '--app-port=3000'
|
|
||||||
- '--dapr-http-port=3500'
|
|
||||||
- '--dapr-grpc-port=50001'
|
|
||||||
- '--placement-host-address=dapr-placement:50005'
|
|
||||||
- '--config=/dapr/config.yaml'
|
|
||||||
- '--resources-path=/dapr/components'
|
|
||||||
volumes:
|
|
||||||
- ../../infra/dapr:/dapr:ro
|
|
||||||
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
name: app-shared-net
|
|
||||||
external: true
|
|
||||||
```
|
|
||||||
|
|
||||||
## 2. Mengakses Dapr dari Service
|
|
||||||
|
|
||||||
### Via HTTP API (semua bahasa)
|
|
||||||
|
|
||||||
Sidecar listen di `localhost:3500`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Publish event
|
|
||||||
curl -X POST http://localhost:3500/v1.0/publish/pubsub/hub.event.type \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"key": "value"}'
|
|
||||||
|
|
||||||
# Service invocation
|
|
||||||
curl http://localhost:3500/v1.0/invoke/app/method/endpoint
|
|
||||||
|
|
||||||
# State store
|
|
||||||
curl -X POST http://localhost:3500/v1.0/state/statestore \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '[{"key": "mykey", "value": "myvalue"}]'
|
|
||||||
```
|
|
||||||
|
|
||||||
### Via Dapr SDK (Rust)
|
|
||||||
|
|
||||||
Tambah ke `Cargo.toml`:
|
|
||||||
|
|
||||||
```toml
|
|
||||||
dapr-sdk = { version = "0.15", features = ["pubsub", "http"] }
|
|
||||||
tokio-stream = "0.1"
|
|
||||||
```
|
|
||||||
|
|
||||||
Contoh publish event:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
use dapr_sdk::client::{Client, Event};
|
|
||||||
use dapr_sdk::DaprClient;
|
|
||||||
|
|
||||||
let client = DaprClient::new("127.0.0.1", 3500).await?;
|
|
||||||
client.publish_event("pubsub", "hub.image.cached", serde_json::json!({
|
|
||||||
"original_url": url,
|
|
||||||
"cdn_url": cdn_url,
|
|
||||||
})).await?;
|
|
||||||
```
|
|
||||||
|
|
||||||
Contoh subscribe event:
|
|
||||||
|
|
||||||
```rust
|
|
||||||
let mut stream = client.subscribe_events("pubsub", "hub.image.cached").await?;
|
|
||||||
while let Some(event) = stream.next().await {
|
|
||||||
let data: MyEvent = serde_json::from_slice(&event.data)?;
|
|
||||||
// handle event
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## 3. Event Topics Convention
|
|
||||||
|
|
||||||
Gunakan prefix `hub.` untuk semua event:
|
|
||||||
|
|
||||||
| Topic | Payload | Description |
|
|
||||||
|-------|---------|-------------|
|
|
||||||
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
|
|
||||||
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
|
|
||||||
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
|
|
||||||
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |
|
|
||||||
|
|
||||||
## 4. Local Development
|
|
||||||
|
|
||||||
Untuk development tanpa Docker:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Install Dapr CLI
|
|
||||||
# 2. Init Dapr local
|
|
||||||
dapr init
|
|
||||||
|
|
||||||
# 3. Run service dengan sidecar
|
|
||||||
dapr run --app-id app --app-port 3000 --dapr-http-port 3500 \
|
|
||||||
--resources-path ./infra/dapr/components \
|
|
||||||
-- cargo run
|
|
||||||
```
|
|
||||||
|
|
||||||
## 5. Verifikasi
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Sidecar health
|
|
||||||
curl http://localhost:3500/v1.0/healthz
|
|
||||||
|
|
||||||
# Publish test event
|
|
||||||
curl -X POST http://localhost:3500/v1.0/publish/pubsub/hub.test \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-d '{"test": true}'
|
|
||||||
|
|
||||||
# NATS stream stats
|
|
||||||
curl http://localhost:8222/jszetstream
|
|
||||||
```
|
|
||||||
@@ -52,49 +52,6 @@ networks:
|
|||||||
|
|
||||||
Gunakan `app-shared-net` agar service dapat diakses oleh Traefik dan service lain.
|
Gunakan `app-shared-net` agar service dapat diakses oleh Traefik dan service lain.
|
||||||
|
|
||||||
## 3.5. Tambahkan Dapr sidecar (wajib untuk pub/sub)
|
|
||||||
|
|
||||||
Setiap service yang ingin menggunakan Dapr pub/sub atau service invocation harus punya sidecar.
|
|
||||||
Tambah di `infra/compose/<nama-app>.yml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
<nama-app>-dapr:
|
|
||||||
container_name: <nama-app>-dapr
|
|
||||||
image: daprio/daprd:latest
|
|
||||||
restart: always
|
|
||||||
depends_on:
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_healthy
|
|
||||||
nats:
|
|
||||||
condition: service_healthy
|
|
||||||
otel-collector:
|
|
||||||
condition: service_started
|
|
||||||
networks:
|
|
||||||
- app-shared-net
|
|
||||||
command:
|
|
||||||
- './daprd'
|
|
||||||
- '--app-id=<nama-app>'
|
|
||||||
- '--app-port=<port>'
|
|
||||||
- '--dapr-http-port=3500'
|
|
||||||
- '--dapr-grpc-port=50001'
|
|
||||||
- '--placement-host-address=dapr-placement:50005'
|
|
||||||
- '--config=/dapr/config.yaml'
|
|
||||||
- '--resources-path=/dapr/components'
|
|
||||||
volumes:
|
|
||||||
- ../../infra/dapr:/dapr:ro
|
|
||||||
```
|
|
||||||
|
|
||||||
Pastikan juga app container punya `depends_on` ke dapr-placement, nats, dan otel-collector:
|
|
||||||
```yaml
|
|
||||||
depends_on:
|
|
||||||
dapr-placement:
|
|
||||||
condition: service_healthy
|
|
||||||
nats:
|
|
||||||
condition: service_healthy
|
|
||||||
otel-collector:
|
|
||||||
condition: service_started
|
|
||||||
```
|
|
||||||
|
|
||||||
## 4. Tambahkan route Traefik
|
## 4. Tambahkan route Traefik
|
||||||
|
|
||||||
Update `infra/traefik/dynamic/apps.yaml`:
|
Update `infra/traefik/dynamic/apps.yaml`:
|
||||||
|
|||||||
@@ -22,9 +22,12 @@ Use `asepharyana-hub` as the root hub repository.
|
|||||||
|
|
||||||
Current app submodules:
|
Current app submodules:
|
||||||
|
|
||||||
| Service | Path | Remote |
|
| Service | Path | Remote |
|
||||||
| ----------- | -------------- | --------------------------------------- |
|
| -------------- | ---------------- | --------------------------------------- |
|
||||||
| Scraper API | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
|
| Elysia API | `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
|
||||||
|
| React frontend | `apps/react` | `asepharyana/asepharyana-hub-react` |
|
||||||
|
| Rust auth API | `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
|
||||||
|
| Scraper API | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
|
||||||
|
|
||||||
## Consequences
|
## Consequences
|
||||||
|
|
||||||
|
|||||||
@@ -1,124 +0,0 @@
|
|||||||
# ADR 0002: Production `.env` via GitHub Encrypted Secret
|
|
||||||
|
|
||||||
## Status
|
|
||||||
|
|
||||||
Accepted
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
The project runs on a remote VPS (`orangevps`, IP `45.127.35.244`) that hosts multiple services via Docker Compose. These services require environment variables (database credentials, API keys, tokens) that must not be committed to the repository.
|
|
||||||
|
|
||||||
The production `.env` file on the VPS is **not** a copy of the committed `.env` in the repo root — it contains additional secrets (Portainer tokens, Discord bot tokens, etc.) that only exist in production.
|
|
||||||
|
|
||||||
Previously, the `.env` file on the VPS was edited manually via SSH, which led to drift between the local `.env` and the production `.env`. When the database server IP or port changed in the local `.env`, the production `.env` was not updated, causing service outages.
|
|
||||||
|
|
||||||
## Decision
|
|
||||||
|
|
||||||
The production `.env` file is stored as a **GitHub Actions encrypted secret** named `ENV_FILE_PRODUCTION`. During deployment, the `.github/workflows/deploy-docker.yml` workflow writes this secret to a file and SCPs it to the VPS.
|
|
||||||
|
|
||||||
### Flow
|
|
||||||
|
|
||||||
```
|
|
||||||
GitHub Secret (ENV_FILE_PRODUCTION)
|
|
||||||
│
|
|
||||||
▼ (deploy-docker.yml)
|
|
||||||
echo "$ENV_FILE_PRODUCTION" > .env.prod
|
|
||||||
scp .env.prod → VPS:$VPS_TARGET_DIR/.env
|
|
||||||
│
|
|
||||||
▼ (docker compose --env-file .env up)
|
|
||||||
Container reads $DATABASE_URL, $JWT_SECRET, etc.
|
|
||||||
```
|
|
||||||
|
|
||||||
### How to update
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Read current content from the VPS
|
|
||||||
ssh root@45.127.35.244 "cat /root/asepharyana-hub/.env"
|
|
||||||
|
|
||||||
# 2. Pipe updated content to the GitHub secret
|
|
||||||
# (requires gh CLI with repo access)
|
|
||||||
cat /path/to/updated-env | gh secret set ENV_FILE_PRODUCTION --repo asepharyana/asepharyana-hub
|
|
||||||
|
|
||||||
# 3. Trigger a redeploy to push it to the VPS
|
|
||||||
gh workflow run deploy-docker.yml
|
|
||||||
|
|
||||||
# OR apply immediately on the VPS (for hotfix):
|
|
||||||
ssh root@45.127.35.244 "sed -i 's|OLD_VALUE|NEW_VALUE|' /root/asepharyana-hub/.env"
|
|
||||||
# Then restart affected containers
|
|
||||||
```
|
|
||||||
|
|
||||||
## Server Topology
|
|
||||||
|
|
||||||
| Host | IP | Role |
|
|
||||||
|------|----|------|
|
|
||||||
| `orangevps` (VPS) | `45.127.35.244` | Docker host: Traefik, scraper-api, Redis, NATS, Dapr |
|
|
||||||
| `imrnes` (bare-metal) | `100.121.180.82` (Tailscale) | PostgreSQL (port 6432), Redis (port 6379), Browserless |
|
|
||||||
|
|
||||||
## Database
|
|
||||||
|
|
||||||
| Variable | Value |
|
|
||||||
|----------|-------|
|
|
||||||
| `DATABASE_URL` | `postgres://asephs:hunterz@100.121.180.82:6432/hub` |
|
|
||||||
| `REDIS_URL` | `redis://redis:6379` (Docker network, overridden per-service) |
|
|
||||||
| `EXTERNAL_BROWSERLESS_WS` | `ws://43.134.105.109:3001/?token=...` (external proxy) |
|
|
||||||
|
|
||||||
> **Important:** The Docker Compose `environment:` section uses variable interpolation (`${DATABASE_URL}`), which is resolved from the `--env-file .env` at compose time — NOT from the service's `env_file`. Both must be kept in sync.
|
|
||||||
|
|
||||||
## Docker Compose Project Structure
|
|
||||||
|
|
||||||
The VPS runs a single Docker Compose project named `compose` composed of multiple files:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
/root/asepharyana-hub/infra/compose/
|
|
||||||
├── traefik.yml # Reverse proxy (TLS termination, routing)
|
|
||||||
├── shared.yml # Redis
|
|
||||||
├── nats.yml # NATS message broker + JetStream
|
|
||||||
├── dapr.yml # Dapr placement service
|
|
||||||
├── scraper.yml # Scraper API + Dapr sidecar
|
|
||||||
└── observability.yml # OTel Collector + Jaeger + Dashboard
|
|
||||||
```
|
|
||||||
|
|
||||||
All files are always included together for dependency resolution:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose \
|
|
||||||
--env-file .env \
|
|
||||||
-f infra/compose/traefik.yml \
|
|
||||||
-f infra/compose/shared.yml \
|
|
||||||
-f infra/compose/scraper.yml \
|
|
||||||
-f infra/compose/nats.yml \
|
|
||||||
-f infra/compose/dapr.yml \
|
|
||||||
-f infra/compose/observability.yml \
|
|
||||||
up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
## GitHub Secrets Required
|
|
||||||
|
|
||||||
| Secret | Description |
|
|
||||||
|--------|-------------|
|
|
||||||
| `SSH_PRIVATE_KEY` | SSH key for VPS access |
|
|
||||||
| `VPS_HOST` | `45.127.35.244` |
|
|
||||||
| `VPS_USER` | `root` |
|
|
||||||
| `VPS_TARGET_DIR` | `/root/asepharyana-hub` |
|
|
||||||
| `ENV_FILE_PRODUCTION` | Full `.env` content for production |
|
|
||||||
|
|
||||||
## Consequences
|
|
||||||
|
|
||||||
### Positive
|
|
||||||
|
|
||||||
- Environment is version-controlled via GitHub Secrets audit log.
|
|
||||||
- No risk of committing secrets to the repo.
|
|
||||||
- Deployment is fully automated — `.env` is pushed on every deploy.
|
|
||||||
- Easy to rotate secrets: update `ENV_FILE_PRODUCTION` and redeploy.
|
|
||||||
|
|
||||||
### Negative
|
|
||||||
|
|
||||||
- The secret is opaque — you cannot diff it or review changes via PR.
|
|
||||||
- If the secret falls out of sync with the local `.env`, services silently break on next deploy.
|
|
||||||
- Requires `gh` CLI or GitHub UI to update — not a simple file edit.
|
|
||||||
|
|
||||||
### Mitigations
|
|
||||||
|
|
||||||
- Keep the **committed `.env`** in the repo root as the source of truth for non-secret values (database URL, ports, API endpoints).
|
|
||||||
- Document any manual SSH hotfix at the same time as updating the GitHub secret.
|
|
||||||
- Run `gh secret set ENV_FILE_PRODUCTION` with the latest server `.env` content after any hotfix.
|
|
||||||
@@ -1,234 +0,0 @@
|
|||||||
# Backup & Disaster Recovery
|
|
||||||
|
|
||||||
## Aset yang Perlu di-Backup
|
|
||||||
|
|
||||||
| Aset | Lokasi | Frekuensi | Metode |
|
|
||||||
|------|--------|-----------|--------|
|
|
||||||
| Database PostgreSQL | `imrnes` (100.121.180.82:6432) | Harian | `pg_dump` |
|
|
||||||
| Volume Redis | `orangevps` (Docker volume) | Opsional | Redis RDB / AOF |
|
|
||||||
| Volume NATS JetStream | `orangevps` (Docker volume) | Opsional | File copy |
|
|
||||||
| Docker Compose manifests | GitHub (hub repo) | Real-time | Git |
|
|
||||||
| Environment variables | GitHub secret `ENV_FILE_PRODUCTION` | Manual | `gh secret set` |
|
|
||||||
| TLS certificates | `orangevps` (`/root/*.pem`, `*.key`) | Saat renew | SCP |
|
|
||||||
| Tailscale auth | Tailscale admin console | - | Cloud-managed |
|
|
||||||
| GitHub Actions secrets | GitHub UI | Manual | Backup list |
|
|
||||||
|
|
||||||
## Database PostgreSQL (Prioritas Tertinggi)
|
|
||||||
|
|
||||||
### Backup Manual
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Dari orangevps (via Tailscale)
|
|
||||||
pg_dump -h 100.121.180.82 -p 6432 -U asephs -d hub \
|
|
||||||
--no-owner --no-acl \
|
|
||||||
-F c -f /root/db-backups/hub-$(date +%Y%m%d-%H%M%S).dump
|
|
||||||
|
|
||||||
# Atau dari imrnes langsung
|
|
||||||
pg_dump -U asephs -d hub \
|
|
||||||
-F c -f /backup/hub/hub-$(date +%Y%m%d-%H%M%S).dump
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restore
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Drop dan recreate database
|
|
||||||
dropdb -h 100.121.180.82 -p 6432 -U asephs hub
|
|
||||||
createdb -h 100.121.180.82 -p 6432 -U asephs hub
|
|
||||||
|
|
||||||
# Restore dari dump
|
|
||||||
pg_restore -h 100.121.180.82 -p 6432 -U asephs -d hub \
|
|
||||||
--no-owner --no-acl \
|
|
||||||
/path/to/backup/hub-20260101-120000.dump
|
|
||||||
```
|
|
||||||
|
|
||||||
### Backup Otomatis (via Cron di imrnes)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# /etc/cron.d/hub-db-backup
|
|
||||||
0 2 * * * root pg_dump -U asephs -d hub -F c -f /backup/hub/hub-$(date +\%Y\%m\%d).dump && find /backup/hub -name "hub-*.dump" -mtime +30 -delete
|
|
||||||
```
|
|
||||||
|
|
||||||
## Volume Docker
|
|
||||||
|
|
||||||
### Redis
|
|
||||||
|
|
||||||
Redis data bisa di-recover dari NATS events (event sourcing). Jika tidak ada persistence requirement, cukup restart:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker volume rm redis_data
|
|
||||||
docker compose -f infra/compose/shared.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Jika perlu backup:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Save RDB snapshot
|
|
||||||
docker exec redis redis-cli SAVE
|
|
||||||
|
|
||||||
# Copy dari volume
|
|
||||||
docker run --rm -v redis_data:/data -v /backup:/backup alpine cp /data/dump.rdb /backup/redis-$(date +%Y%m%d).rdb
|
|
||||||
```
|
|
||||||
|
|
||||||
### NATS JetStream
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Backup volume
|
|
||||||
docker run --rm -v nats_data:/data -v /backup:/backup alpine \
|
|
||||||
tar czf /backup/nats-$(date +%Y%m%d).tar.gz -C /data .
|
|
||||||
```
|
|
||||||
|
|
||||||
## Environment Variables
|
|
||||||
|
|
||||||
### Backup `.env` dari VPS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Simpan current .env dari VPS
|
|
||||||
ssh root@45.127.35.244 "cat /root/asepharyana-hub/.env" > .env.backup.$(date +%Y%m%d)
|
|
||||||
|
|
||||||
# Update GitHub secret
|
|
||||||
cat .env.backup.$(date +%Y%m%d) | gh secret set ENV_FILE_PRODUCTION --repo asepharyana/asepharyana-hub
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restore `.env` jika hilang
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Buat .env baru dari template
|
|
||||||
cp .env.example .env
|
|
||||||
|
|
||||||
# Edit secrets (manual dari password manager atau GitHub secret)
|
|
||||||
# Atau download dari GitHub secret
|
|
||||||
gh secret list --repo asepharyana/asepharyana-hub
|
|
||||||
```
|
|
||||||
|
|
||||||
## TLS Certificates
|
|
||||||
|
|
||||||
### Backup
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Di orangevps
|
|
||||||
tar czf /root/cert-backup-$(date +%Y%m%d).tar.gz \
|
|
||||||
/root/asepharyana.my.id.pem \
|
|
||||||
/root/asepharyana.my.id.key \
|
|
||||||
/root/asepharyana.web.id.pem \
|
|
||||||
/root/asepharyana.web.id.key \
|
|
||||||
/root/asepharyana-hub/infra/traefik/dynamic/ssl.yaml
|
|
||||||
|
|
||||||
# SCP ke local
|
|
||||||
scp root@45.127.35.244:/root/cert-backup-*.tar.gz .
|
|
||||||
```
|
|
||||||
|
|
||||||
### Restore
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# SCP ke VPS
|
|
||||||
scp cert-backup-20260101.tar.gz root@45.127.35.244:/root/
|
|
||||||
|
|
||||||
# Extract
|
|
||||||
ssh root@45.127.35.244 "tar xzf /root/cert-backup-20260101.tar.gz -C / && docker restart traefik"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Disaster Recovery Scenarios
|
|
||||||
|
|
||||||
### Skenario 1: VPS (orangevps) mati total
|
|
||||||
|
|
||||||
**Dampak:** Semua service down.
|
|
||||||
|
|
||||||
**Recovery:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Provision VPS baru (atau restore dari snapshot)
|
|
||||||
# 2. Install Docker + Tailscale
|
|
||||||
# 3. Clone repo
|
|
||||||
git clone https://github.com/asepharyana/asepharyana-hub.git /root/asepharyana-hub
|
|
||||||
|
|
||||||
# 4. Setup Tailscale, route service
|
|
||||||
# 5. Restore .env
|
|
||||||
echo "<ENV_FILE_PRODUCTION>" > /root/asepharyana-hub/.env
|
|
||||||
|
|
||||||
# 6. Restore TLS certs
|
|
||||||
# 7. Create network
|
|
||||||
docker network create app-shared-net
|
|
||||||
|
|
||||||
# 8. Start services sesuai urutan
|
|
||||||
cd /root/asepharyana-hub
|
|
||||||
for f in shared.yml nats.yml dapr.yml traefik.yml scraper.yml; do
|
|
||||||
docker compose -f infra/compose/$f --env-file .env up -d
|
|
||||||
done
|
|
||||||
|
|
||||||
# 9. Update DNS jika IP baru
|
|
||||||
```
|
|
||||||
|
|
||||||
### Skenario 2: Database (imrnes) mati total
|
|
||||||
|
|
||||||
**Dampak:** Semua service yang butuh database error.
|
|
||||||
|
|
||||||
**Recovery:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Fix imrnes atau provision server baru
|
|
||||||
# 2. Setup PostgreSQL
|
|
||||||
# 3. Restore dari backup terakhir
|
|
||||||
# 4. Update Tailscale IP jika perlu
|
|
||||||
# 5. Update .env dan GitHub secret
|
|
||||||
# 6. Redeploy
|
|
||||||
```
|
|
||||||
|
|
||||||
### Skenario 3: GitHub repository hilang
|
|
||||||
|
|
||||||
**Dampak:** Kehilangan CI/CD, tapi Docker images masih ada di GHCR.
|
|
||||||
|
|
||||||
**Recovery:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Create repo baru di GitHub
|
|
||||||
# 2. Push dari local clone
|
|
||||||
git remote add origin-new https://github.com/asepharyana/asepharyana-hub-new.git
|
|
||||||
git push origin-new main
|
|
||||||
|
|
||||||
# 3. Re-create GitHub secrets
|
|
||||||
# 4. Re-create workflows
|
|
||||||
# 5. Update VPS remote
|
|
||||||
ssh root@45.127.35.244 "cd /root/asepharyana-hub && git remote set-url origin https://github.com/asepharyana/asepharyana-hub-new.git"
|
|
||||||
```
|
|
||||||
|
|
||||||
### Skenario 4: GHCR registry tidak bisa diakses
|
|
||||||
|
|
||||||
**Dampak:** Tidak bisa pull image.
|
|
||||||
|
|
||||||
**Recovery:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Build image langsung di VPS
|
|
||||||
docker build -f infra/docker/scraper.Dockerfile -t ghcr.io/asepharyana/asepharyana-hub/scraper-api:local .
|
|
||||||
|
|
||||||
# 2. Update compose file untuk sementara
|
|
||||||
sed -i 's|image: ghcr.io/.*|image: ghcr.io/asepharyana/asepharyana-hub/scraper-api:local|' infra/compose/scraper.yml
|
|
||||||
|
|
||||||
# 3. Start
|
|
||||||
docker compose -f infra/compose/scraper.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### Skenario 5: Semua server mati (total loss)
|
|
||||||
|
|
||||||
**Recovery:**
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Provision VPS baru
|
|
||||||
# 2. Provision server database baru
|
|
||||||
# 3. Setup Tailscale
|
|
||||||
# 4. Clone repo, restore .env, certs
|
|
||||||
# 5. Restore database dari backup (jika ada)
|
|
||||||
# 6. Jika tidak ada backup database:
|
|
||||||
# - Build image dari GHCR
|
|
||||||
# - Start service dengan database kosong
|
|
||||||
# - Data akan terisi ulang dari scraping
|
|
||||||
```
|
|
||||||
|
|
||||||
## Checklist Pencegahan
|
|
||||||
|
|
||||||
- [ ] Cron job backup database berjalan
|
|
||||||
- [ ] Backup `.env` disimpan di luar VPS (password manager)
|
|
||||||
- [ ] TLS certificates backup disimpan di luar VPS
|
|
||||||
- [ ] GitHub secrets terdaftar (tidak hanya diingat)
|
|
||||||
- [ ] Docker images bisa di-rebuild dari CI (GHCR sebagai source of truth)
|
|
||||||
- [ ] Tailscale admin access via multiple accounts
|
|
||||||
@@ -1,248 +0,0 @@
|
|||||||
# CI/CD Pipeline
|
|
||||||
|
|
||||||
Dokumentasi pipeline CI/CD untuk `asepharyana-hub`. Terdiri dari 5 GitHub Actions workflow yang saling terhubung.
|
|
||||||
|
|
||||||
## Workflow Overview
|
|
||||||
|
|
||||||
```
|
|
||||||
┌─────────────┐
|
|
||||||
│ Lint │ (PR/push → Biome)
|
|
||||||
└──────┬──────┘
|
|
||||||
│
|
|
||||||
Push ke main ─────┼────── repository_dispatch
|
|
||||||
│
|
|
||||||
┌──────▼──────────────────┐
|
|
||||||
│ docker-build-push.yml │
|
|
||||||
│ │
|
|
||||||
│ Phase 1: Detect │
|
|
||||||
│ Phase 2: Build & Push │
|
|
||||||
│ Phase 3: Update │
|
|
||||||
│ manifests │
|
|
||||||
└──────┬──────────────────┘
|
|
||||||
│ workflow_run
|
|
||||||
┌──────▼──────────────┐
|
|
||||||
│ deploy-docker.yml │
|
|
||||||
│ SSH → VPS │
|
|
||||||
│ Pull → Restart │
|
|
||||||
└─────────────────────┘
|
|
||||||
|
|
||||||
repository_dispatch ──► update-submodule.yml
|
|
||||||
(dari submodule) (update pointer → commit)
|
|
||||||
│
|
|
||||||
▼
|
|
||||||
docker-build-push.yml
|
|
||||||
(triggered by push)
|
|
||||||
```
|
|
||||||
|
|
||||||
## Workflow Detail
|
|
||||||
|
|
||||||
### 1. Lint (`lint.yml`)
|
|
||||||
|
|
||||||
**Trigger:** PR/push ke `main` yang mengubah `*.json`, `*.js`, `biome.json`
|
|
||||||
|
|
||||||
**Aksi:**
|
|
||||||
- Checkout repo dengan submodules
|
|
||||||
- Setup Bun
|
|
||||||
- `bun install --frozen-lockfile`
|
|
||||||
- `bun run ci` (Biome CI mode)
|
|
||||||
|
|
||||||
**Permissions:** read-only
|
|
||||||
|
|
||||||
### 2. Build and Push Docker Images (`docker-build-push.yml`)
|
|
||||||
|
|
||||||
**Trigger:**
|
|
||||||
- Push ke `main` yang mengubah `apps/**`, `infra/**`, atau file workflow
|
|
||||||
- `repository_dispatch` tipe `submodule-updated`
|
|
||||||
- `workflow_dispatch` (manual)
|
|
||||||
|
|
||||||
**Concurrency:** Satu workflow per branch (cancel-in-progress=false)
|
|
||||||
|
|
||||||
#### Phase 1: Detect Changes
|
|
||||||
|
|
||||||
Job `changes` mendeteksi service mana yang perlu di-build:
|
|
||||||
|
|
||||||
- **Push event:** `git diff --name-only` antara `before` dan `after` SHA
|
|
||||||
- **repository_dispatch:** Parse payload `{service, sha}` dan validasi
|
|
||||||
- **workflow_dispatch:** Build semua service
|
|
||||||
|
|
||||||
Output format matrix:
|
|
||||||
```json
|
|
||||||
[{"id":"scraper-api","target":"docker-scraper","path":"apps/scraper"}]
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Phase 2: Build & Push (Matrix)
|
|
||||||
|
|
||||||
Job `build` berjalan paralel per service (matrix strategy):
|
|
||||||
|
|
||||||
1. Checkout repo + sync submodule
|
|
||||||
2. Jika `repository_dispatch`, checkout submodule ke SHA tertentu
|
|
||||||
3. Login ke GHCR
|
|
||||||
4. Setup Docker Buildx
|
|
||||||
5. Build & push dengan tag:
|
|
||||||
- `ghcr.io/asepharyana/asepharyana-hub/<service>:latest`
|
|
||||||
- `ghcr.io/asepharyana/asepharyana-hub/<service>:sha-<shortsha>`
|
|
||||||
6. Build cache: registry-based (`:<service>:buildcache`)
|
|
||||||
|
|
||||||
#### Phase 3: Update Manifests
|
|
||||||
|
|
||||||
Job `update-manifest`:
|
|
||||||
|
|
||||||
1. Update image tag di compose file (`infra/compose/<service>.yml`)
|
|
||||||
2. Jika `repository_dispatch`, update submodule pointer
|
|
||||||
3. Commit dengan message `chore: update manifests and submodules [skip ci]`
|
|
||||||
4. Push dengan retry (3 attempts, rebase jika conflict)
|
|
||||||
|
|
||||||
### 3. Deploy Docker to VPS (`deploy-docker.yml`)
|
|
||||||
|
|
||||||
**Trigger:**
|
|
||||||
- `workflow_run` setelah `docker-build-push.yml` selesai
|
|
||||||
- Push ke `main` yang mengubah `infra/**`
|
|
||||||
- `workflow_dispatch` (manual)
|
|
||||||
|
|
||||||
**Concurrency:** Satu deployment dalam satu waktu (`group: deploy-vps`)
|
|
||||||
|
|
||||||
**Aksi di VPS (via SSH):**
|
|
||||||
|
|
||||||
```
|
|
||||||
1. Setup SSH multiplexing
|
|
||||||
2. SCP .env dari GitHub secret ke VPS
|
|
||||||
3. Docker login ke GHCR
|
|
||||||
4. Git sync (fetch + reset --hard)
|
|
||||||
5. Detect changed files:
|
|
||||||
├─ Compose stack changes → selective container update
|
|
||||||
├─ Traefik dynamic config → SIGHUP
|
|
||||||
└─ Other infra → full deploy
|
|
||||||
6. Pull images (retry 3x)
|
|
||||||
7. Remove stale containers
|
|
||||||
8. Up services
|
|
||||||
9. SIGHUP Traefik jika perlu
|
|
||||||
```
|
|
||||||
|
|
||||||
### 4. Security Scan (`security.yml`)
|
|
||||||
|
|
||||||
**Trigger:**
|
|
||||||
- PR ke `main`
|
|
||||||
- Jadwal: Setiap Senin (`0 6 * * 1`)
|
|
||||||
|
|
||||||
**Aksi:**
|
|
||||||
- Checkout dengan fetch-depth 2
|
|
||||||
- CodeQL init untuk Rust
|
|
||||||
- `cargo build` di `apps/scraper`
|
|
||||||
- CodeQL analyze
|
|
||||||
|
|
||||||
### 5. Update Submodule Pointer (`update-submodule.yml`)
|
|
||||||
|
|
||||||
**Trigger:** `repository_dispatch` tipe `submodule-updated`
|
|
||||||
|
|
||||||
**Aksi:**
|
|
||||||
1. Validasi payload (`service`, `sha`)
|
|
||||||
2. Map service ke submodule path (e.g., `scraper-api` → `apps/scraper`)
|
|
||||||
3. Update submodule ke SHA yang diberikan
|
|
||||||
4. Commit sebagai `monrepo-bot` dengan message:
|
|
||||||
`chore: update <service> to <shortsha>`
|
|
||||||
5. Push dengan retry (3 attempts)
|
|
||||||
|
|
||||||
## Flow Submodule Update
|
|
||||||
|
|
||||||
Flow lengkap ketika code berubah di submodule repo:
|
|
||||||
|
|
||||||
```
|
|
||||||
1. Developer push ke asepharyana-hub-scraper
|
|
||||||
2. GitHub Action di scraper repo kirim repository_dispatch
|
|
||||||
ke asepharyana-hub
|
|
||||||
3. update-submodule.yml terima dispatch, update pointer
|
|
||||||
4. Commit masuk ke hub repo main
|
|
||||||
5. Commit ini trigger docker-build-push.yml
|
|
||||||
(push ke main dengan path apps/scraper/**)
|
|
||||||
6. Build image baru, update compose file
|
|
||||||
7. Deploy ke VPS
|
|
||||||
```
|
|
||||||
|
|
||||||
## Secrets yang Diperlukan
|
|
||||||
|
|
||||||
| Secret | Workflow | Deskripsi |
|
|
||||||
|--------|----------|-----------|
|
|
||||||
| `SSH_PRIVATE_KEY` | deploy-docker | SSH key untuk akses VPS |
|
|
||||||
| `VPS_HOST` | deploy-docker | IP VPS (`45.127.35.244`) |
|
|
||||||
| `VPS_USER` | deploy-docker | User SSH (`root`) |
|
|
||||||
| `VPS_TARGET_DIR` | deploy-docker | Dir di VPS (`/root/asepharyana-hub`) |
|
|
||||||
| `ENV_FILE_PRODUCTION` | deploy-docker | Full `.env` production |
|
|
||||||
|
|
||||||
## Menambahkan Service Baru ke Pipeline
|
|
||||||
|
|
||||||
Untuk menambahkan service baru, update:
|
|
||||||
|
|
||||||
### `docker-build-push.yml`
|
|
||||||
|
|
||||||
1. **Phase 1 — `changes` job:** Tambah detection logic untuk service baru:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
echo "new-service=$(changed '^(apps/new-service(/|$)|\.github/workflows/docker-build-push\.yml$|infra/docker/new-service\.Dockerfile$)')" >> "$GITHUB_OUTPUT"
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Phase 1 — `repository_dispatch`:** Tambah case:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
case "$SERVICE" in
|
|
||||||
scraper-api|new-service) ;;
|
|
||||||
```
|
|
||||||
|
|
||||||
3. **Phase 1 — `set-matrix`:** Tambah service:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
if [ "${{ ...['new-service'] == 'true' ... }}" == "true" ]; then add_service "new-service" "docker-new-service" "apps/new-service"; fi
|
|
||||||
```
|
|
||||||
|
|
||||||
4. **Phase 2 — `meta` step:** Tambah mapping Dockerfile:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
"new-service") echo "dockerfile=infra/docker/new-service.Dockerfile" >> $GITHUB_OUTPUT ;;
|
|
||||||
```
|
|
||||||
|
|
||||||
5. **Phase 3 — `update-manifest`:** Tambah mapping:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
SERVICES["new-service"]="new-service.yml"
|
|
||||||
PATHS["new-service"]="apps/new-service"
|
|
||||||
```
|
|
||||||
|
|
||||||
### `deploy-docker.yml`
|
|
||||||
|
|
||||||
Tambah compose file ke `ALL_COMPOSE_FILES`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
ALL_COMPOSE_FILES="infra/compose/traefik.yml infra/compose/shared.yml infra/compose/scraper.yml infra/compose/nats.yml infra/compose/dapr.yml infra/compose/new-service.yml"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Rollback
|
|
||||||
|
|
||||||
### Rollback Image
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cari SHA tag sebelumnya di GHCR packages
|
|
||||||
# Update compose file ke tag tersebut
|
|
||||||
sed -i 's|sha-badcommit|sha-goodcommit|g' infra/compose/scraper.yml
|
|
||||||
git commit -am "fix: rollback scraper-api to sha-goodcommit"
|
|
||||||
git push
|
|
||||||
```
|
|
||||||
|
|
||||||
### Rollback via Git Revert
|
|
||||||
|
|
||||||
```bash
|
|
||||||
git revert HEAD
|
|
||||||
git push origin main
|
|
||||||
# Pipeline otomatis build dan deploy
|
|
||||||
```
|
|
||||||
|
|
||||||
## Monitoring Pipeline
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cek status workflow terbaru
|
|
||||||
gh run list --limit 5
|
|
||||||
|
|
||||||
# Lihat log workflow tertentu
|
|
||||||
gh run view <run-id> --log
|
|
||||||
|
|
||||||
# Trigger workflow manual
|
|
||||||
gh workflow run deploy-docker.yml
|
|
||||||
```
|
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Squid Proxy Configuration (Archived)
|
||||||
|
|
||||||
|
**Status**: Archived; not referenced by active infrastructure.
|
||||||
|
|
||||||
|
**Date archived**: 2026-06-04
|
||||||
|
|
||||||
|
**Original location**: `infra/squid.conf`
|
||||||
|
|
||||||
|
**Reason**: No active Docker Compose service mounts or starts Squid. The config is preserved here for reference if Squid proxying is restored later.
|
||||||
|
|
||||||
|
## Original configuration
|
||||||
|
|
||||||
|
```squid
|
||||||
|
# Allow localhost access only
|
||||||
|
acl localhost src 127.0.0.1/32 ::1/128
|
||||||
|
http_access allow localhost
|
||||||
|
|
||||||
|
# Allow Docker bridge subnets (Adjusted for our expected subnets)
|
||||||
|
acl docker_bridge src 172.16.0.0/12
|
||||||
|
http_access allow docker_bridge
|
||||||
|
|
||||||
|
# Deny all other access
|
||||||
|
http_access deny all
|
||||||
|
|
||||||
|
# Standard port - Explicitly bind to all interfaces
|
||||||
|
http_port 0.0.0.0:3128
|
||||||
|
|
||||||
|
# Hide source info
|
||||||
|
forwarded_for off
|
||||||
|
via off
|
||||||
|
```
|
||||||
|
|
||||||
|
## Restore checklist
|
||||||
|
|
||||||
|
1. Copy this config back to `infra/squid.conf`.
|
||||||
|
2. Add a Squid service to an active compose file.
|
||||||
|
3. Mount the config into the Squid container.
|
||||||
|
4. Document which services should use the proxy.
|
||||||
|
5. Validate network exposure and authentication before production use.
|
||||||
@@ -1,286 +0,0 @@
|
|||||||
# NATS + JetStream Guide
|
|
||||||
|
|
||||||
Dokumentasi konfigurasi, penggunaan, dan troubleshooting NATS di infrastruktur `asepharyana-hub`.
|
|
||||||
|
|
||||||
## Arsitektur
|
|
||||||
|
|
||||||
NATS berjalan di container `nats` dengan JetStream diaktifkan (`-js`). Data persistent disimpan di volume Docker `nats_data`.
|
|
||||||
|
|
||||||
```
|
|
||||||
Service ──► NATS (port 4222) ──► JetStream (disk)
|
|
||||||
│
|
|
||||||
├─ Monitoring HTTP: port 8222
|
|
||||||
└─ Client connections: port 4222
|
|
||||||
```
|
|
||||||
|
|
||||||
### Hubungan dengan Dapr
|
|
||||||
|
|
||||||
Saat ini Dapr pub/sub menggunakan **Redis** (`pubsub.redis`), bukan NATS. NATS berfungsi sebagai message broker independen untuk:
|
|
||||||
|
|
||||||
- Event streaming antar service
|
|
||||||
- Persistent job queues
|
|
||||||
- Pub/sub untuk service yang tidak menggunakan Dapr
|
|
||||||
|
|
||||||
Jika ingin Dapr menggunakan NATS sebagai backend pub/sub, ganti komponen `pubsub.yaml`:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: pubsub
|
|
||||||
spec:
|
|
||||||
type: pubsub.nats
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: natsURL
|
|
||||||
value: nats://nats:4222
|
|
||||||
```
|
|
||||||
|
|
||||||
## Konfigurasi Compose
|
|
||||||
|
|
||||||
File: `infra/compose/nats.yml`
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
services:
|
|
||||||
nats:
|
|
||||||
container_name: nats
|
|
||||||
image: nats:latest
|
|
||||||
restart: always
|
|
||||||
networks:
|
|
||||||
app-shared-net:
|
|
||||||
aliases:
|
|
||||||
- nats
|
|
||||||
ports:
|
|
||||||
- '4222:4222' # client connections
|
|
||||||
- '8222:8222' # HTTP monitor
|
|
||||||
command:
|
|
||||||
- '-js' # enable JetStream
|
|
||||||
- '-sd'
|
|
||||||
- '/data' # storage directory
|
|
||||||
volumes:
|
|
||||||
- nats_data:/data
|
|
||||||
```
|
|
||||||
|
|
||||||
## CLI Tools
|
|
||||||
|
|
||||||
### Install NATS CLI
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Linux
|
|
||||||
curl -sf https://bin.nats.dev/nats | sh
|
|
||||||
sudo mv nats /usr/local/bin/
|
|
||||||
|
|
||||||
# Atau via package manager
|
|
||||||
# brew install nats-io/nats-tools/nats (macOS)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Koneksi ke NATS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Dari host (port 4222 ter-expose)
|
|
||||||
nats context save hub --server nats://localhost:4222 --description "Hub Production"
|
|
||||||
nats context select hub
|
|
||||||
|
|
||||||
# Test koneksi
|
|
||||||
nats server check
|
|
||||||
nats server info
|
|
||||||
```
|
|
||||||
|
|
||||||
### Manage Streams (JetStream)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# List semua stream
|
|
||||||
nats stream list
|
|
||||||
|
|
||||||
# Lihat detail stream
|
|
||||||
nats stream info <stream-name>
|
|
||||||
|
|
||||||
# Buat stream
|
|
||||||
nats stream add <stream-name> \
|
|
||||||
--subjects "hub.>" \
|
|
||||||
--storage file \
|
|
||||||
--max-msgs 1000000 \
|
|
||||||
--max-bytes 1G \
|
|
||||||
--retention limits
|
|
||||||
|
|
||||||
# Hapus stream
|
|
||||||
nats stream rm <stream-name>
|
|
||||||
|
|
||||||
# Purge (hapus semua message, retain stream)
|
|
||||||
nats stream purge <stream-name>
|
|
||||||
```
|
|
||||||
|
|
||||||
### Pub/Sub
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Subscribe ke subject
|
|
||||||
nats sub "hub.>"
|
|
||||||
nats sub "hub.image.cached"
|
|
||||||
|
|
||||||
# Publish message
|
|
||||||
nats pub "hub.test" '{"message": "hello"}'
|
|
||||||
nats pub "hub.image.cached" '{"original_url": "https://example.com/img.jpg", "cdn_url": "https://cdn.example.com/img.jpg"}'
|
|
||||||
|
|
||||||
# Request-reply
|
|
||||||
nats request "hub.service.do" '{"task": "process"}'
|
|
||||||
```
|
|
||||||
|
|
||||||
### Monitoring via HTTP API
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Server info
|
|
||||||
curl http://localhost:8222/
|
|
||||||
|
|
||||||
# JetStream info
|
|
||||||
curl http://localhost:8222/jszetstream
|
|
||||||
|
|
||||||
# Stream detail
|
|
||||||
curl http://localhost:8222/jszetstream?stream=<stream-name>
|
|
||||||
|
|
||||||
# Consumer info
|
|
||||||
curl http://localhost:8222/jszetstream?stream=<stream-name>&consumer=<consumer-name>
|
|
||||||
|
|
||||||
# Server stats
|
|
||||||
curl http://localhost:8222/varz
|
|
||||||
|
|
||||||
# Connections
|
|
||||||
curl http://localhost:8222/connz
|
|
||||||
```
|
|
||||||
|
|
||||||
## Event Topics Convention
|
|
||||||
|
|
||||||
Semua topik menggunakan prefix `hub.`:
|
|
||||||
|
|
||||||
| Subject | Payload | Deskripsi |
|
|
||||||
|---------|---------|-----------|
|
|
||||||
| `hub.image.cached` | `{original_url, cdn_url, source}` | Image selesai di-cache |
|
|
||||||
| `hub.image.repaired` | `{old_url, new_url}` | CNAME image diperbaiki |
|
|
||||||
| `hub.scrape.anime.done` | `{source, slug, duration}` | Scrape anime selesai |
|
|
||||||
| `hub.system.alert` | `{service, level, message}` | Error/alert dari service |
|
|
||||||
| `hub.test` | Any | Testing |
|
|
||||||
|
|
||||||
### Wildcard Subjects
|
|
||||||
|
|
||||||
NATS mendukung wildcard:
|
|
||||||
|
|
||||||
- `hub.>` — semua event hub (multi-level)
|
|
||||||
- `hub.image.*` — semua event image (single-level)
|
|
||||||
- `hub.*.done` — semua event yang selesai (single-level)
|
|
||||||
|
|
||||||
## JetStream Configuration
|
|
||||||
|
|
||||||
### Storage
|
|
||||||
|
|
||||||
Data JetStream disimpan di volume Docker `nats_data`.
|
|
||||||
|
|
||||||
Lokasi di VPS:
|
|
||||||
```bash
|
|
||||||
docker volume inspect nats_data
|
|
||||||
# atau
|
|
||||||
ls -la /var/lib/docker/volumes/nats_data/_data/
|
|
||||||
```
|
|
||||||
|
|
||||||
### Memory & Limits
|
|
||||||
|
|
||||||
NATS tidak memiliki konfigurasi limit memori default. Untuk production, pertimbangkan:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
command:
|
|
||||||
- '-js'
|
|
||||||
- '-sd'
|
|
||||||
- '/data'
|
|
||||||
- '--max_pending_size=64MB'
|
|
||||||
- '--max_payload=1MB'
|
|
||||||
```
|
|
||||||
|
|
||||||
Atau gunakan NATS configuration file:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# nats-server.conf
|
|
||||||
jetstream:
|
|
||||||
max_memory_store: 256MB
|
|
||||||
max_file_store: 10GB
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Stream data tidak muncul
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. Cek koneksi NATS
|
|
||||||
nats server check
|
|
||||||
|
|
||||||
# 2. Cek apakah JetStream aktif
|
|
||||||
curl http://localhost:8222/jszetstream
|
|
||||||
|
|
||||||
# 3. Cek stream dan message count
|
|
||||||
nats stream list
|
|
||||||
|
|
||||||
# 4. Subscribe langsung untuk test
|
|
||||||
nats sub ">"
|
|
||||||
```
|
|
||||||
|
|
||||||
### NATS tidak bisa start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cek log
|
|
||||||
docker logs nats
|
|
||||||
|
|
||||||
# Cek apakah port 4222 sudah dipakai
|
|
||||||
ss -tlnp | grep 4222
|
|
||||||
|
|
||||||
# Cek volume data korup
|
|
||||||
docker run --rm -v nats_data:/data alpine ls -la /data
|
|
||||||
|
|
||||||
# Restart
|
|
||||||
docker compose -f infra/compose/nats.yml up -d --force-recreate
|
|
||||||
```
|
|
||||||
|
|
||||||
### Disk JetStream penuh
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cek ukuran volume
|
|
||||||
docker system df -v | grep nats_data
|
|
||||||
|
|
||||||
# Purge stream jika perlu
|
|
||||||
nats stream purge <stream-name>
|
|
||||||
|
|
||||||
# Atau hapus volume (data hilang!)
|
|
||||||
docker compose -f infra/compose/nats.yml down
|
|
||||||
docker volume rm nats_data
|
|
||||||
docker compose -f infra/compose/nats.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
### Slow consumer
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Cek consumer lag
|
|
||||||
nats stream info <stream-name>
|
|
||||||
# Lihat fields: "Pending" dan "Acknowledgment"
|
|
||||||
|
|
||||||
# Lihat stats server
|
|
||||||
curl http://localhost:8222/varz | jq '.slow_consumers'
|
|
||||||
```
|
|
||||||
|
|
||||||
## Migration: Redis Pub/Sub ke NATS
|
|
||||||
|
|
||||||
Jika ingin migrasi dari Dapr pub/sub Redis ke NATS:
|
|
||||||
|
|
||||||
1. Buat stream NATS untuk topik `hub.>`
|
|
||||||
2. Update `infra/dapr/components/pubsub.yaml` dari `pubsub.redis` ke `pubsub.nats`
|
|
||||||
3. Deploy ulang semua service (Dapr sidecar akan reconnect)
|
|
||||||
4. Verifikasi event flow
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
# infra/dapr/components/pubsub.yaml (setelah migrasi)
|
|
||||||
apiVersion: dapr.io/v1alpha1
|
|
||||||
kind: Component
|
|
||||||
metadata:
|
|
||||||
name: pubsub
|
|
||||||
spec:
|
|
||||||
type: pubsub.nats
|
|
||||||
version: v1
|
|
||||||
metadata:
|
|
||||||
- name: natsURL
|
|
||||||
value: nats://nats:4222
|
|
||||||
```
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user