MythEclipse/asepharyana-hub
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

refactor: update architecture and documentation for scraper service, remove deprecated services

Browse Source
This commit is contained in:
asepharyana
2026-07-21 12:54:57 +07:00
parent cdad9ae720
commit 5d6bdf6ce3
5 changed files with 47 additions and 168 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ARCHITECTURE.md
+8 -41
View File
@@ -100,22 +100,11 @@ graph TB
subgraph "orange Containers"
TRAEFIK[Traefik :443]
RUST_AUTH[rust-auth :3000]
ELYSIA[elysia-api :4092]
REACT[react-web :80]
SCRAPER[scraper-api :4091]
end
TRAEFIK --> RUST_AUTH
TRAEFIK --> ELYSIA
TRAEFIK --> REACT
TRAEFIK --> SCRAPER
RUST_AUTH -.->|Tailscale IP| PG
ELYSIA -.->|Tailscale IP| PG
RUST_AUTH -.->|Tailscale IP| REDIS
ELYSIA -.->|Tailscale IP| REDIS
style IMRNES fill:#3a7,color:#fff
style ORANGE fill:#37a,color:#fff
style ARCH fill:#773,color:#fff
@@ -154,8 +143,8 @@ sequenceDiagram
App->>DB: sqlx/Drizzle query via Tailscale
DB-->>App: Result set
else Cache lookup
App->>Redis: GET/SET via Tailscale
Redis-->>App: Cached value
App->>Cache: GET/SET via Tailscale
Cache-->>App: Cached value
end
App-->>Traefik: HTTP response
@@ -217,7 +206,7 @@ The `orange` VPS (Tailscale `100.96.248.86`) hosts all application containers:
### Selective Deployment
The deploy workflow supports selective updates — if only `infra/compose/elysia.yml` changed, only `elysia-api` is pulled and recreated, avoiding disruption to other services.
The deploy workflow supports selective updates — if only one compose file changed, only the corresponding service is pulled and recreated, avoiding disruption to other services.
```mermaid
graph TB
@@ -254,7 +243,7 @@ Each application lives in its own Git repository and is imported as a submodule
### Submodule Lifecycle
1. Developer pushes to a submodule (e.g., `apps/elysia`)
1. Developer pushes to a submodule (e.g., `apps/scraper`)
2. Submodule's GitHub Action dispatches `repository_dispatch` to the super-repo with the service name and new SHA
3. Super-repo detects the dispatch, waits for the SHA to be fetchable, then builds only that service
4. The compose manifest is updated and committed with the new SHA tag
@@ -264,15 +253,12 @@ Each application lives in its own Git repository and is imported as a submodule
```bash
# Update a single submodule to latest
cd apps/elysia
cd apps/scraper
git checkout main
git pull
cd ../..
git add apps/elysia
git commit -m "chore(elysia): update submodule to latest"
# Update all submodules
git submodule update --remote --merge
git add apps/scraper
git commit -m "chore(scraper): update submodule to latest"
```
## Service Mesh & Inter-Service Communication
@@ -287,35 +273,16 @@ graph LR
TRAEFIK[Traefik :443]
subgraph "app-shared-net"
REACT[react-web<br/>:80]
ELYSIA[elysia-api<br/>:4092]
RUST_AUTH[rust-auth<br/>:3000]
SCRAPER[scraper-api<br/>:4091]
REDIS[redis<br/>:6379]
end
end
subgraph "Imrnes VPS"
PG[(PostgreSQL<br/>:5432)]
REDIS_IMR[Redis<br/>:6379]
end
WWW -->|HTTPS| TRAEFIK
TRAEFIK --> REACT
TRAEFIK --> ELYSIA
TRAEFIK --> RUST_AUTH
TRAEFIK --> SCRAPER
ELYSIA -->|Tailscale| PG
RUST_AUTH -->|Tailscale| PG
ELYSIA -->|internal| REDIS
RUST_AUTH -->|internal| REDIS
ELYSIA -->|Tailscale| REDIS_IMR
RUST_AUTH -->|Tailscale| REDIS_IMR
```
## Observability
- **Prometheus metrics**: Available on rust-auth via `axum-prometheus`
- **Traefik access logs**: JSON format, logged at INFO level
- **Traefik access logs**: JSON format, logged at INFO level
- **Dashboard**: Traefik dashboard at `traefik.asepharyana.my.id` (secured)
CONTRIBUTING.md
+13 -65
View File
@@ -16,7 +16,6 @@
- **Git** with LFS support
- **Node.js** >= 22.11.0 (via `.node-version` or `.nvmrc`)
- **Bun** >= 1.3.11 (package manager)
- **Rust** >= 1.89.0 (for Rust services)
- **Docker** and **Docker Compose** (for shared infrastructure)
## Local Setup
@@ -40,18 +39,14 @@ This checks out all submodules at the pinned commit (not `main`). The submodules
| Path | Remote |
| ---------------- | --------------------------------------- |
| `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
| `apps/react` | `asepharyana/asepharyana-hub-react` |
| `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
### 3. Install Dependencies per Service
Install dependencies for TypeScript/Bun services:
```bash
cd apps/elysia && bun install && cd ../..
cd apps/react && npm install && cd ../..
cd apps/scraper && bun install && cd ../..
```
### 4. Start Shared Infrastructure
@@ -75,76 +70,42 @@ Key variables to configure:
| Variable | Description |
| -------------- | ----------------------------------------------------- |
| `DATABASE_URL` | PostgreSQL connection (Tailscale IP to `imrnes` VPS) |
| `REDIS_URL` | Redis connection (`redis://localhost:6379` for local) |
| `JWT_SECRET` | JWT signing secret |
| `GITHUB_TOKEN` | GitHub personal access token |
## Development Workflow
### Running Services
**Rust API (rust-auth):**
```bash
cd apps/rust-auth
cargo run
```
**Elysia API (elysia):**
```bash
cd apps/elysia
bun run dev
```
**React Frontend (react):**
```bash
cd apps/react
npm run dev
```
Refer to each service's own documentation for setup and development instructions.
### API Documentation
- Rust OpenAPI: `http://localhost:4091/docs`
- Elysia Swagger: `http://localhost:4092/docs`
- Elysia AsyncAPI: `http://localhost:4092/docs-ws`
Refer to each service's own documentation for API docs and endpoints.
## Coding Standards
### Linting
- **ESLint** with `@antfu/eslint-config` for TypeScript/JavaScript
- **Cargo Clippy** for Rust
- **Biome** for TypeScript/JavaScript formatting and linting
Run linting:
```bash
# TypeScript/JavaScript
eslint . --no-error-on-unmatched-pattern
# Rust specific
cd apps/rust-auth && cargo clippy -- -D warnings
bun run check
```
### Formatting
- **Prettier** for TypeScript/JavaScript/Markdown (config in `.prettierrc`)
- Single quotes, 100 print width, 2-space indent, trailing commas
- **Cargo fmt** for Rust
- **Biome** for TypeScript/JavaScript
- **EditorConfig** for general formatting (`.editorconfig`)
```bash
# Prettier
prettier --write .
# Rust
cd apps/rust-auth && cargo fmt
# Format all
bun run format
```
### Rust Configuration
Rust services use edition `2024` with stable toolchain (nightly features may be used).
## Commit Message Format
@@ -177,18 +138,15 @@ This project enforces **Conventional Commits** for all commit messages.
### Examples
```
feat(rust-auth): add OAuth2 Google login flow
fix(elysia): handle null JWT payload in auth middleware
chore: update eslint config to v10
feat(scraper): add new data source integration
chore: update biome config to v10
docs: add API endpoint documentation for scraper
refactor(react): extract Header component from App
test(elysia): add unit tests for rate limiter
ci: migrate to CodeQL v3
```
### Scopes
Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
Common scopes: `scraper`, `infra`, `ci`, `deps`
## Pull Request Process
@@ -202,8 +160,7 @@ Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
3. **Run checks locally** before pushing:
```bash
cd apps/react && npx tsc --noEmit
eslint . --no-error-on-unmatched-pattern
bun run check
```
4. **Push and open a PR** against `main`. CI will automatically run:
@@ -227,13 +184,4 @@ Common scopes: `rust-auth`, `elysia`, `react`, `scraper`, `infra`, `ci`, `deps`
## Adding a New Service
See `docs/add-new-app.md` for the complete step-by-step guide. In summary:
1. Create the app in `apps/<name>`
2. Add it as a Git submodule in `.gitmodules`
3. Register it in `infra/compose/<name>.yml`
4. Add a Dockerfile at `infra/docker/<name>.Dockerfile`
5. Add Traefik routing config in `infra/traefik/dynamic/apps.yaml`
6. Add CI entries in `.github/workflows/docker-build-push.yml`
7. Add compose file to the deploy script in `deploy-docker.yml`
8. Add any required GitHub secrets for the service
See `docs/add-new-app.md` for the complete step-by-step guide.
README.md
+8 -31
View File
@@ -5,12 +5,9 @@ Aplikasi dipisah sebagai submodule agar frontend, API, dan service pendukung bis
## Services
| Service | Path | Default Local Port | Notes |
| :---------- | :--------------- | :----------------- | :---------------------------------------------------------------------------- |
| Rust API | `apps/rust-auth` | `4091` | API utama (Axum + SeaORM), scraping, image proxy/cache, metrics, OpenAPI docs |
| Elysia API | `apps/elysia` | `4092` | API realtime/auth/chat/quiz (Elysia + Bun + Drizzle + Redis) |
| React Web | `apps/react` | `3000` | Frontend React/Vite |
| Scraper | `apps/scraper` | — | Web scraper service |
| Service | Path | Notes |
| :------ | :------------- | :------------------ |
| Scraper | `apps/scraper` | Web scraper service |
## Infrastructure
@@ -18,7 +15,7 @@ File compose berada di `infra/compose/`:
- `traefik.yml`: reverse proxy Traefik untuk semua layanan.
- `shared.yml`: Redis.
- `rust-auth.yml`, `elysia.yml`, `react.yml`, `scraper.yml`: manifest deploy per service (image GHCR bertag SHA).
- `scraper.yml`: manifest deploy per service (image GHCR bertag SHA).
Dockerfile per service berada di `infra/docker/`.
@@ -27,9 +24,6 @@ Dockerfile per service berada di `infra/docker/`.
Build image via Dockerfile:
```bash
docker build -f infra/docker/rust.Dockerfile -t rust-auth:latest .
docker build -f infra/docker/elysia.Dockerfile -t elysia-api:latest .
docker build -f infra/docker/react.Dockerfile -t react-web:latest .
docker build -f infra/docker/scraper.Dockerfile -t scraper-api:latest .
```
@@ -38,9 +32,8 @@ Tag and push:
```bash
SHORT_SHA=$(git rev-parse --short HEAD)
docker tag rust-auth:latest ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA
docker push ghcr.io/asepharyana/asepharyana-hub/rust-auth:sha-$SHORT_SHA
# repeat for elysia-api, react-web, scraper-api
docker tag scraper-api:latest ghcr.io/asepharyana/asepharyana-hub/scraper-api:sha-$SHORT_SHA
docker push ghcr.io/asepharyana/asepharyana-hub/scraper-api:sha-$SHORT_SHA
```
## Local Development
@@ -53,27 +46,11 @@ docker compose -f infra/compose/shared.yml up -d
### 2) Jalankan service yang dibutuhkan
```bash
# Rust API
cd apps/rust-auth
cargo run
# Elysia API
cd apps/elysia
bun install
bun run dev
# React web
cd apps/react
npm install
npm run dev
```
Refer to each service's own documentation for development setup.
## API Docs and Monitoring
- Rust OpenAPI: `/docs`
- Elysia Swagger: `/docs`
- Elysia AsyncAPI viewer: `/docs-ws`
Refer to each service's own documentation for API docs.
## Deployment Notes
docs/adr/0001-use-hub-repo-with-submodules.md
+1 -4
View File
@@ -23,10 +23,7 @@ Use `asepharyana-hub` as the root hub repository.
Current app submodules:
| Service | Path | Remote |
| -------------- | ---------------- | --------------------------------------- |
| Elysia API | `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| React frontend | `apps/react` | `asepharyana/asepharyana-hub-react` |
| Rust auth API | `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
| ----------- | -------------- | --------------------------------------- |
| Scraper API | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
## Consequences
docs/superpowers/specs/2026-07-09-monorepo-restructure-design.md
+13 -23
View File
@@ -21,10 +21,7 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
| Service | Path | Git remote baru |
|---------|------|----------------|
| Elysia API | `apps/elysia` | `asepharyana/asepharyana-hub-elysia` |
| React Frontend | `apps/react` | `asepharyana/asepharyana-hub-react` |
| Scraper | `apps/scraper` | `asepharyana/asepharyana-hub-scraper` |
| Rust Auth | `apps/rust-auth` | `asepharyana/asepharyana-hub-rust-auth` |
## Service yang dihapus
@@ -32,8 +29,11 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
|---------|------|
| Docker Manager | `apps/docker-manager/` |
| TeleUploader | `apps/teleuploader/` |
| Elysia API | `apps/elysia/` |
| React Frontend | `apps/react/` |
| Rust Auth | `apps/rust-auth/` |
## File yang akan dihapus
## File yang akan dihapus (gelombang 1)
- `apps/docker-manager/` (seluruh direktori)
- `apps/teleuploader/` (seluruh direktori)
@@ -68,10 +68,10 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
| Lama | Baru |
|------|------|
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/elysia-api:*` | `ghcr.io/asepharyana/asepharyana-hub/elysia-api:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/react-web:*` | `ghcr.io/asepharyana/asepharyana-hub/react-web:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/scraper-api:*` | `ghcr.io/asepharyana/asepharyana-hub/scraper-api:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/rust-auth:*` | `ghcr.io/asepharyana/asepharyana-hub/rust-auth:*` |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/elysia-api:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/react-web:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/rust-auth:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/docker-manager:*` | — (dihapus) |
| `ghcr.io/mytheclipse/ultimate-asepharyana.tech/teleuploader:*` | — (dihapus) |
@@ -79,35 +79,25 @@ Proyek ini sebelumnya bernama `ultimate-asepharyana.tech` — sebuah monorepo ya
### `docker-build-push.yml`
- `IMAGE_NAME_PREFIX`: `mytheclipse/ultimate-asepharyana.tech` → `asepharyana/asepharyana-hub`
- Hapus service entries: `docker-manager`, `teleuploader`
- Hapus service entries: `docker-manager`, `teleuploader`, `elysia-api`, `react-web`, `rust-auth`
- Update repo URLs di `wait-submodule-ref` dari `MythEclipse/*` ke `asepharyana/*`
- Update `update-manifest` phase — hapus service docker-manager & teleuploader
- Update `update-manifest` phase — hapus service docker-manager, teleuploader, elysia-api, react-web, rust-auth
### `deploy-docker.yml`
- Update `git remote add origin`
- Hapus docker-manager & teleuploader dari compose list dan checkout
- Hapus docker-manager, teleuploader, elysia-api, react-web, rust-auth dari compose list dan checkout
### `update-submodule.yml`
- Hapus service docker-manager & teleuploader
- Update nama workflow
- Hapus service docker-manager, teleuploader, elysia-api, react-web, rust-auth
## `.gitmodules`
Hanya berisi 4 apps dengan remote baru:
Hanya berisi 1 app scraper:
```ini
[submodule "apps/elysia"]
path = apps/elysia
url = https://github.com/asepharyana/asepharyana-hub-elysia.git
[submodule "apps/react"]
path = apps/react
url = https://github.com/asepharyana/asepharyana-hub-react.git
[submodule "apps/scraper"]
path = apps/scraper
url = https://github.com/asepharyana/asepharyana-hub-scraper.git
[submodule "apps/rust-auth"]
path = apps/rust-auth
url = https://github.com/asepharyana/asepharyana-hub-rust-auth.git
```
## Execution plan
@@ -129,7 +119,7 @@ Hanya berisi 4 apps dengan remote baru:
## Post-execution state
- Root direktori `asepharyana-hub/` dengan source code apps utuh (tanpa git)
- 4 app submodule terdaftar di `.gitmodules` dengan remote baru
- 1 app submodule (scraper) terdaftar di `.gitmodules` dengan remote baru
- Infra/docs/scripts tetap menyatu di root
- 0 references ke `MythEclipse/ultimate-asepharyana.tech` di file konfigurasi
- Siap untuk `git init && git add && git commit` kapan saja