zesdex/TODO.md

# TODO — Rewrite Zesdex Rust → TypeScript/Bun

> Plan lengkap migrasi in-place dari Rust (11 crate, clean architecture 4 lapis) ke monorepo
> TypeScript/Bun. Bahasa Indonesia, commit pakai Conventional Commits.
>
> **Status:** Fase 0–2 + 3a–3g **selesai**; Fase 4 **CLI + TUI selesai**; Fase 5 **semua {api,daemon,ws,grpc,web} selesai**. Berikutnya: Fase 4 bootstrap + Fase 6 bersih-bersih.
> Base: `bun run check` bersih, 74 test hijau.

---

## Prinsip & peta pemetaan

| Rust | TS/Bun |
|---|---|
| `serde_json` | `JSON.stringify/parse` + `zod` |
| `chrono` | `Date` / `Date.now()` (epoch ms) |
| `uuid` | `crypto.randomUUID()` |
| `sha2`, `base64 URL_SAFE_NO_PAD` | `node:crypto` / `Buffer.toString('base64url')` |
| `reqwest` | `fetch` / bun HTTP |
| `rusqlite` | `bun:sqlite` |
| `argon2` | `@node-rs/argon2` |
| `jsonwebtoken` | `node:crypto` HMAC HS256 |
| `rmcp` | `@modelcontextprotocol/sdk` |
| `ratatui`/`crossterm` | library TUI Bun / ink |
| `tiktoken-rs` | `@dqbd/tiktoken` (cl100k_base) |
| `clap` | `Bun.argv` / `commander` |
| `tokio` semaphore | Bun async + concurrency pool |
| `libc` PID lock | `node:fs` `O_EXCL` + `process.kill` liveness |
| Rust generics `S<R: Repo>` | DI via constructor interface |
| `Arc<Mutex<T>>` | objek mutable (array/objek di ToolCtx) |

### Struktur target

```
apps/
  packages/            # ✅ sudah ada: domain, application, infrastructure
  interfaces/
    cli/               # gateway + TUI        [baru]
    api/               # REST API            [baru]
    daemon/            # daemon + IPC        [baru]
    ws/                # WebSocket server    [baru]
    grpc/              # stub                [baru]
    web/               # static server       [baru]
```

---

## ✅ Selesai

### Fase 0 — Bootstrap monorepo
- [x] `package.json` root (workspaces), `tsconfig.json` strict, `.gitignore`
- [x] Paket `domain`, `application`, `infrastructure` (workspace `@zesdex/*`)
- [x] `bun install` + `bun test` jalan

### Fase 1 — Domain layer
- [x] `core`: ChatMessage, Conversation, ChatRequest/Response/StreamEvent, SseParser, ToolCall, repairJson, sanitizeToolArguments, ToolCallResult, UsageStats, Store
- [x] `auth`: Session, SessionId (path-traversal), OAuthToken/Config, repo/service trait
- [x] `cms`: AppConfig, ProviderConfig, ModelRole, Settings+resolveEffectiveModel, SettingsPatch, Memory, EditLog, repo/service trait
- [x] `agent`: Origin, Toast, AgentStatus, TurnEvent, SessionRuntime, AgentTurnParams, AgentProgress, ExecutionModel
- [x] `workflow`/`subagent`: WorkflowScript/NodeDirective, AccessTier, prompt builders
- [x] 27 test domain

### Fase 2 — Application layer
- [x] `ports`: ProviderService, PasswordService, TokenService, AuthService
- [x] `AgentTurnServiceImpl` (loop 50 iter, auto-compact 60k, adaptive token, temp, ErrorTracker, bounded parallel)
- [x] `OAuthUseCase` (PKCE S256 + CSRF state)
- [x] `SessionServiceImpl`, `MemoryServiceImpl`, `SettingsServiceImpl`, `ConversationServiceImpl`
- [x] 11 test application

### Fase 3a — LLM client
- [x] `LlmClient` (fetch, retry 10x backoff+jitter, abort 401/402/403)
- [x] `resolve_api_key`, `is_auth_error`, `backoff_for_error`

### Fase 3b — Tool system (`commit a291dd1`)
- [x] `Tool` interface, `ToolCtx`/`ToolCtxBuilder`, `InfrastructureToolExecutor`
- [x] Registry: `allTools` (37), `toolDefs`, `toolIsRisky`, `toolIsParallelSafe`
- [x] FS: read, write, edit, delete · **Guard**: graduated checks, resolve_path sandbox
- [x] Search: grep, glob (matcher ringan)
- [x] Shell: bash (timeout), bash_output, bash_kill · **Filter**: checkGitDestructive, checkCredentialRead
- [x] Git: git_operator, git_worktree, git_cred
- [x] Memory: remember, forget, recall
- [x] Util: cd, dir_list, dir_cache_update, pong, todowrite, todofinish
- [x] Reasoning/Plan: sequential_think, plan_enter, plan_ready
- [x] Web: web_search (async → SearXNG)
- [x] Semantic: semantic_search, rebuild_index, list_symbols (index multi-bahasa)
- [x] Best-practice: best_practice, commit_convention + BestPracticeEngine
- [x] Agent/Workflow: spawn_agents, spawn_pipeline, parallel_delegate, workflow_run, note_finding, read_findings, hive_mind (delegasi ke placeholder 3c/3d)
- [x] 16 test tool system

### Fase 3e — Persistence
- [x] File repos: settings, app_config (+ auto-detect kredensial Claude), conversation, edits.jsonl, markdown memory, rewind blob, session, oauth, session lock
- [x] `writeJsonAtomic`, `slugify`, `buildWorkspaceTree`, `runCommand`

### Fase 3c — Subagent engine
**Sumber Rust:** `apps/infrastructure/src/subagent/{engine,division,context,provider,spawn}.rs`, `subagent/auto/*`

- [x] `AccessTier.toolsFor(access)` — `tools/division.ts`: Read / Write / Full yang mem-filter registry
- [x] `run_agent` loop (`tools/engine.ts`): `MAX_ITERATIONS=25`, `TOOL_OUTPUT_MAX_CHARS=12_000`,
  `MAX_CONSECUTIVE_TOOL_ERRORS=3`, `MAX_PARALLEL_TOOLS=8`; recovery note setelah 3 error beruntun
- [x] `SubagentContext` (`subagent/context.ts`) — directive, toolCtx, access, baseURL, apiKey, model
- [x] `spawn_subagent` — `spawn_tools.ts` — runAgent berbasis HTTP provider + config_resolver
- [x] `resolve_subagent_provider` (`subagent/provider.ts`) — pilih provider/model subagent dari settings + app_config
- [x] `spawn_tools.ts` — isi body `spawnAgents` (paralel) & `spawnPipeline` (berurutan)
- [x] `delegate.ts` — `runParallelDelegation` untuk `parallel_delegate`
- [x] `http_provider.ts` — shared HTTP provider service builder
- [ ] Auto-reviewer (`subagent/auto/*`) — `spawn_background_review`: git diff → LLM review → perbaiki;
  wiring `review_enabled` (yang sekarang no-op di `Review` agent)
- [x] Placeholder di `apps/packages/infrastructure/src/subagent/*` diisi penuh

### Fase 3d — Workflow + hive-mind
**Sumber Rust:** `apps/infrastructure/src/workflow/{script,mod,docs}.rs`, `workflow/engine/{execution,phases,primitives}.rs`, `workflow/hive_mind/{complexity,cycle,synthesis,live,types}.rs`

- [x] `parse_workflow_script(yaml)` — parser YAML workflow (`workflow/script.ts`)
- [x] `execute_workflow(script, ctx)` — eksekusi multi-phase (`workflow/engine.ts` + `workflow/orchestrate.ts`)
- [x] Phase primitives: run sub-agent per node (`executePrimitive`)
- [x] `execute_cycle(cycle, ctx)` — hive-mind cycle, concurrency 8, kumpulkan NodeOutput (`workflow/cycle.ts`)
- [x] `synthesize_consensus(node_outputs, ctx)` — konsensus via LLM + fallback concat (`workflow/synthesis.ts`)
- [x] `complexity_heuristic` — pilih apakah cukup 1 cycle / butuh banyak (`workflow/complexity.ts`)
- [x] `docs.ts` — tulis convergence/decision doc (`workflow/docs.ts`)
- [x] Isi body `WorkflowRun` & `HiveMind` tool (ganti placeholder) di `tools/workflow.ts`
- [x] `orchestrate.ts` — top-level `executeWorkflow` + `executeHiveMind` entry points

---

## ⬜ Yang belum dikerjakan

### Fase 3f — Auth infrastructure
**Sumber Rust:** `apps/infrastructure/src/auth/{password,jwt,oauth_loopback,mod}.rs`

- [x] `password.ts` — Argon2 via `@node-rs/argon2` (hash + verify), port `PasswordService`
- [x] `jwt.ts` — HMAC HS256 via `node:crypto` (access + refresh token, typ, exp), port `TokenService`
- [x] `oauth_loopback.ts` — server TCP loopback lokal utk terima redirect OAuth; port `oauth_loopback`
- [x] `mod.ts` (`index.ts`) — composisi ke `AuthService`
- [x] Tambah dep `@node-rs/argon2` di `infrastructure/package.json`

### Fase 3g — IPC + bgbash
**Sumber Rust:** `apps/infrastructure/src/ipc/{frame,protocol,server,client,conn}.rs`, `apps/infrastructure/src/bgbash/{job,control}.rs`

- [x] `ipc/frame.ts` — length-prefixed framing via `node:net` (kirim/terima bingkai biner)
- [x] `ipc/conn.ts` — `Connection` framed JSON socket; `ipc/protocol.ts` — wire types
- [x] `ipc/server.ts` + `ipc/client.ts` — Unix socket server + attach client; protocol message (NLJSON/binary)
- [x] `bgbash/job.ts` — `BashJob` (pid, output file, log), `spawnBashJob`
- [x] `bgbash/control.ts` — registry job global: `register`, `cancel(job_id)`, `is_running`
- [x] Wire `bash` tool `run_in_background` → `spawnBashJob` + `register`; `bash_kill` → `cancel`
- [x] Update `bash_output` untuk baca dari registry/output dir

### Fase 4 — Gateway + CLI + TUI
**Sumber Rust:** `apps/gateway/src/{lib,main}.rs`, `apps/interfaces/tui/src/**`

- [x] `apps/interfaces/cli/` (paket Bun):
  - [x] Parser arg: `--api/--daemon/--attach/--ws/--grpc/--web`, dispatch mode (`args.ts`)
  - [x] Logging ke file (`index.ts`)
  - [x] `run_single_process` — komposisi root: wire semua implementasi konkret sekali (`compose.ts`) + REPL loop
  - [x] Bootstrap seed (`bootstrap.ts`) — wire-shape settings lama terbaca
- [x] CLI: parser arg + dispatch + single-process REPL loop + bootstrap seed (Smoke: `bun run cli --help`, `bun run bootstrap` jalan; loop turn + tool read/write)
- [x] TUI mode (`@zesdex/tui`, tanpa ncurses — render garis ANSI):
  - [x] Event loop raw-mode stdin (run.ts) + key decode (decodeKey)
  - [x] Key dispatcher `handleKey` → `Action[]` + overlay/autocomplete/editor handling (controller.ts)
  - [x] `Action` enum + `applyAction` dispatcher (action.ts) + slash-command parse/apply (command.ts)
  - [x] `InputState` (buffer/cursor/history/autocomplete) + `AppStateRest` state machine (state.ts)
  - [x] Renderer ringan (render.ts): transcript, input, overlay, status — 20 unit tests hijau
- [x] Bootstrap seed (`bootstrap`) — `bun run bootstrap` seed defaults, wire-shape settings lama terbaca

### Fase 5 — Server interface: api / daemon / ws / grpc / web
**Sumber Rust:** `apps/interfaces/{api,daemon,ws,grpc,web}/src/**`, `apps/infrastructure/src/middleware/*`

- [x] **API** (`apps/interfaces/api/`):
  - [x] Endpoint: `auth` (login/register/refresh, rate-limited), `sessions` CRUD, `conversations`, `chat/completions` proxy, `health`
  - [x] JWT middleware, CORS
  - [x] DTO (dto/)
- [x] Smoke `bun run api`: health 200; register/login 200; wrong-pwd 401; protected-no-token 401; create/list/delete session; add conversation msg
- [x] **daemon** — IPC agent-driver loop (Submit/Close) over per-session Unix socket; streaming token frames; wire CLI `--daemon`/`--attach`
- [x] **ws** — handler `/ws`, protokol token/connected/done/error; smoke: no-token rejected 401, with-token welcome
- [x] **grpc** — stub (health endpoint); smoke: `/grpc/health` 200
- [x] **web** — static server + proteksi traversal; smoke: `/` serves index, traversal 404
- [x] CLI `--api`/`--ws`/`--grpc`/`--web` dispatch → start server masing-masing (smoke: `bun run cli --api --api-port` → health 200)

### Fase 6 — Bersih-bersih + verifikasi akhir
- [ ] Hapus semua sisa crate Rust: `apps/{domain,application,infrastructure,gateway,bootstrap,interfaces/*}` file `.rs`, `Cargo.toml`, `Cargo.lock`, `.cargo/`, `lefthook.yml`
- [x] Update `install.sh` → Bun
- [x] Update CI workflow: `.github/workflows/{ci,release}.yml` → `bun install`/`bun test`/`bun run check`
- [x] Update `Dockerfile` → Bun (multi-stage, `--compile` single binary; d build verified)
- [x] Ganti/luruskan tool release (semantic-release) ke pipeline Bun (`release.yml` test job → bun; prepareCmd → `bun run check`)
- [ ] `flake.nix`/`shell.nix`/`deploy.yml`/`flakehub-publish-rolling.yml` → Nix build masih Cargo (tinggalkan — deploy infra, tinggalkan ke user/ops; build lokal `bun build --compile` sudah diverifikasi)
- [ ] Berkas jadi: hapus folder Rust yang sudah kosong dari `apps/`
- [ ] Verifikasi akhir: `bun run build`, `bun test`, `tsc --noEmit`, `bun run cli --help`, smoke server api/ws/daemon
- [ ] Baca ulang file lama (`settings.json`, `app_config.json`, `conversation.json`, memory `.md`) — pastikan wire-shape terbaca

---

## Verifikasi global (tiap fase)
- [ ] `bun run build` / `tsc --noEmit` tanpa error (strict)
- [ ] `bun test` — port test Rust ke `*.test.ts` (SseParser, repairJson, resolveEffectiveModel, SessionId, turn_service, tool system, subagent, workflow)
- [ ] Smoke: `bun run cli --help`; agent mode TUI dengan model stub/lokal; pastikan loop turn + tool read/write
- [ ] Server: `bun run api` + hit health/auth; `bun run ws` + connect
- [ ] Commit tiap fase dengan message Bahasa Indonesia + Conventional Commits; test hijau sebelum lanjut

## Catatan
- Registry tool = **38** (terhitung workflow_run, note_finding, read_findings, hive_mind; ExploreCodebase tidak diport).
- `Tool.run` boleh return `string | Promise<string>`.
- `spawn_agents`/`spawn_pipeline`/`parallel_delegate` sudah terisi penuh via 3c.
- Workflow/hive-mind tools sudah terisi penuh via 3d.