feat: remove obsolete design documents for clipboard OSC52, diff view, file mention, context compaction, and add development guide
This commit is contained in:
asepharyana committed
1 parent
44c3dd1239
commit
66ac4dbf02
23 files changed
+775
-13170
No files matched your search
@@ -780,7 +780,7 @@ pub fn resolve_context_window(
|
||||
return max as usize;
|
||||
}
|
||||
}
|
||||
128_000
|
||||
256_000
|
||||
}
|
||||
|
||||
/// Count tokens using tiktoken, fall back to character estimation.
|
||||
|
||||
+105
-52
@@ -1,65 +1,118 @@
|
||||
# Architecture Overview
|
||||
# Arsitektur Sistem Zesdex
|
||||
|
||||
## System Layout
|
||||
## Gambaran Umum
|
||||
|
||||
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
|
||||
Zesdex adalah autonomous AI coding agent berbasis TUI, dibangun dengan **Rust** menggunakan clean architecture berlapis. LLM client dibungkus dalam tool-use harness dengan 37+ built-in tools — file ops, git, shell, LSP, MCP, subagent orchestration, dan lainnya.
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Process Mode │
|
||||
│ Single-Process ─── Daemon (background) ─── Attach (client) │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│ IPC (Unix domain socket)
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ src/main.rs │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
||||
│ │ Controller │──▶│ Runtime │──▶│ View │ │
|
||||
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
|
||||
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────▼────────┐ │
|
||||
│ │ Harness │ │
|
||||
│ │ (tool dispatch)│ │
|
||||
│ └───────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────┼─────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ Tools │ │ Subagents │ │ Workflow │ │
|
||||
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
|
||||
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
|
||||
│ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
zesdex/
|
||||
├── apps/
|
||||
│ ├── domain/ # Layer 1: Pure entities, traits, value objects
|
||||
│ ├── application/ # Layer 2: Use-case services
|
||||
│ ├── infrastructure/ # Layer 3: Semua I/O (LLM, DB, tools, MCP, LSP)
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # Ratatui terminal UI
|
||||
│ │ ├── api/ # Axum REST API
|
||||
│ │ ├── daemon/ # Unix socket daemon
|
||||
│ │ ├── ws/ # WebSocket server
|
||||
│ │ ├── grpc/ # gRPC server
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point & dispatcher
|
||||
│ └── bootstrap/ # Initial data seeder
|
||||
```
|
||||
|
||||
## Dependency Graph Antar Layer
|
||||
|
||||
```
|
||||
gateway/bootstrap
|
||||
│
|
||||
▼
|
||||
interfaces/* (tui, api, daemon, ws, grpc, web)
|
||||
│
|
||||
▼
|
||||
infrastructure ─── implements ──▶ domain ports
|
||||
│
|
||||
▼
|
||||
application ─── depends on ──▶ domain traits
|
||||
│
|
||||
▼
|
||||
domain (zero framework deps: serde, chrono, uuid only)
|
||||
```
|
||||
|
||||
> **Aturan**: layer bawah tidak boleh tahu tentang layer atas. `domain` tidak import apapun dari `infrastructure` atau `interfaces`.
|
||||
|
||||
## Process Modes
|
||||
|
||||
| Mode | Description |
|
||||
|------|-------------|
|
||||
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
|
||||
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
|
||||
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
|
||||
| Mode | Flag | Keterangan |
|
||||
|------|------|------------|
|
||||
| **TUI** | *(default)* | TUI + agent loop dalam satu proses |
|
||||
| **Daemon** | `--daemon` | Agent berjalan di background via IPC socket |
|
||||
| **Attach** | `--attach <id>` | TUI terhubung ke daemon yang sedang berjalan |
|
||||
| **REST API** | `--api` | HTTP server (default port 8080) |
|
||||
| **WebSocket** | `--ws` | WS server (default port 8081) |
|
||||
| **gRPC** | `--grpc` | gRPC server (default port 50051) |
|
||||
| **Web** | `--web` | Static web frontend (default port 3000) |
|
||||
|
||||
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
|
||||
## Alur Data (Single-Process Mode)
|
||||
|
||||
## Data Flow
|
||||
```
|
||||
Keyboard/Event
|
||||
│
|
||||
▼
|
||||
controller/input.rs: handle_key()
|
||||
│ returns Vec<Action>
|
||||
▼
|
||||
action.rs: apply_action(&mut AppStateRest)
|
||||
│ state dimutasi in-place
|
||||
├──▶ turn.rs: spawn_agent_turn() ──▶ background thread
|
||||
│ │
|
||||
│ ├─ LLM call (blocking reqwest)
|
||||
│ ├─ tool execution (Tool trait)
|
||||
│ └─ push TurnEvent ke queue
|
||||
│
|
||||
▼
|
||||
run.rs: run_loop_inner()
|
||||
│ drain TurnEvent setiap tick
|
||||
│ skip render jika dirty=false
|
||||
▼
|
||||
view/: draw frame ke terminal (ratatui)
|
||||
```
|
||||
|
||||
1. **Input** → `controller/input.rs` handles key events and autocomplete
|
||||
2. **Dispatch** → `app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
|
||||
3. **LLM Stream** → `app/runtime/stream/mod.rs` parses SSE chunks into typed events
|
||||
4. **Tool Execution** → `app/harness.rs` gates and runs tool calls via the `Tool` trait
|
||||
5. **Rendering** → `view/` modules read `AppStateRest` and render via ratatui
|
||||
## File Kunci
|
||||
|
||||
## Key Files
|
||||
| File | Peran |
|
||||
|------|-------|
|
||||
| `apps/gateway/src/main.rs` | CLI entry point, parse args, dispatch ke mode |
|
||||
| `apps/interfaces/tui/src/state.rs` | `AppStateRest` — single source of truth state TUI |
|
||||
| `apps/interfaces/tui/src/action.rs` | `apply_action()` — satu-satunya tempat state dimutasi |
|
||||
| `apps/interfaces/tui/src/run.rs` | Event loop: render → poll → handle → tick |
|
||||
| `apps/interfaces/tui/src/turn.rs` | Spawn agent turn di background thread |
|
||||
| `apps/interfaces/tui/src/view/mod.rs` | Top-level render pipeline + `pre_render` hook |
|
||||
| `apps/infrastructure/src/llm/` | LLM client (streaming + non-streaming) |
|
||||
| `apps/infrastructure/src/tools/` | 37 tool implementations |
|
||||
| `apps/domain/src/core/` | Entity inti: `ChatMessage`, `Role`, `Store`, `Tool` trait |
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
|
||||
| `src/app/state/rest.rs` | Single source-of-truth state struct |
|
||||
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
|
||||
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
|
||||
| `src/app/harness.rs` | Tool harness with safety gating |
|
||||
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
|
||||
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
|
||||
| `src/view/mod.rs` | TUI render pipeline |
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
```
|
||||
┌──────────┐ Unix domain socket ┌──────────┐
|
||||
│ Client │ ◄──────────────────► │ Daemon │
|
||||
│ (TUI) │ [4-byte len][JSON] │ (agent) │
|
||||
└──────────┘ └──────────┘
|
||||
|
||||
Client ──Action──▶ Daemon (apply_action → state mutasi)
|
||||
Daemon ──StatePayload──▶ Client (render snapshot)
|
||||
```
|
||||
|
||||
## Lints & Kualitas Kode
|
||||
|
||||
Semua workspace crate menerapkan lint ketat di `Cargo.toml`:
|
||||
- `unused`, `dead_code`, `unreachable_code` → **deny**
|
||||
- `unused_imports`, `unused_variables`, `unused_mut` → **deny**
|
||||
- `clippy::all` + `clippy::pedantic` → **warn**
|
||||
|
||||
## Release Profile
|
||||
|
||||
`opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"`, `strip="symbols"`
|
||||
+108
-47
@@ -1,68 +1,129 @@
|
||||
# Backend Architecture
|
||||
# Backend & Infrastructure
|
||||
|
||||
## Provider Layer
|
||||
Semua implementasi I/O ada di `apps/infrastructure/src/`. Layer ini mengimplementasikan port/trait yang didefinisikan di `apps/domain/`.
|
||||
|
||||
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
|
||||
## LLM Client (`infrastructure/src/llm/`)
|
||||
|
||||
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
|
||||
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
|
||||
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
|
||||
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
|
||||
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
|
||||
Wrapper di atas provider OpenAI-compatible:
|
||||
|
||||
## IPC (Inter-Process Communication)
|
||||
- **`provider/`** — `LlmClient`: HTTP client dengan `reqwest::blocking` (sync) untuk agent turn, dan async streaming untuk preview
|
||||
- **Request/Response** — `ChatMessage`, `ChatCompletionRequest`, `ChatCompletionResponse` dengan support tool calls
|
||||
- **Streaming** — SSE event parser untuk streaming response
|
||||
- **Usage tracking** — `tokens_in`, `tokens_out`, `last_tokens_in`, `last_tokens_out` per panggilan
|
||||
- **Provider defaults** — DeepSeek v4 flash free via OpenCode AI proxy (default)
|
||||
|
||||
The daemon-client protocol in `src/ipc/`:
|
||||
|
||||
- **Transport**: Unix domain sockets
|
||||
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
|
||||
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
|
||||
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
|
||||
|
||||
Flow:
|
||||
```
|
||||
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
|
||||
│
|
||||
└──StatePayload──▶ Client (render)
|
||||
```rust
|
||||
// Contoh penggunaan di turn.rs
|
||||
let result = client.chat_with_tools_non_streaming(
|
||||
&mut messages,
|
||||
Some(tool_defs),
|
||||
Some(4096), // max_tokens
|
||||
Some(0.7), // temperature
|
||||
None, // abort flag
|
||||
);
|
||||
```
|
||||
|
||||
## Workflow Engine
|
||||
## Tool System (`infrastructure/src/tools/`)
|
||||
|
||||
Located in `src/app/workflow/`:
|
||||
37 tool yang mengimplementasikan trait `Tool` dari domain:
|
||||
|
||||
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
|
||||
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
|
||||
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
|
||||
```rust
|
||||
pub trait Tool: Send + Sync {
|
||||
fn name(&self) -> &'static str;
|
||||
fn description(&self) -> &'static str;
|
||||
fn parameters(&self) -> Value; // JSON Schema
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String>;
|
||||
}
|
||||
```
|
||||
|
||||
## MCP (Model Context Protocol)
|
||||
### Kategori Tool
|
||||
|
||||
`src/app/mcp/manager.rs` manages MCP client connections:
|
||||
| Kategori | Tools |
|
||||
|----------|-------|
|
||||
| **File System** | `read`, `write`, `edit`, `delete`, `dir_list`, `dir_cache_update` |
|
||||
| **Shell** | `bash`, `bash_interactive`, `bash_kill`, `bash_output` |
|
||||
| **Git** | `git_operator`, `git_cred`, `git_worktree` |
|
||||
| **Search** | `search`, `grep`, `glob`, `semantic_search` |
|
||||
| **LSP** | `lsp_connect`, `lsp_hover`, `lsp_completion`, `lsp_definition`, `lsp_references`, `lsp_diagnostics`, `lsp_disconnect` |
|
||||
| **Memory** | `remember`, `recall`, `forget` |
|
||||
| **Workflow** | `spawn_agents`, `spawn_pipeline`, `plan`, `sequential_think`, `hive_mind` |
|
||||
| **Utility** | `todo_write`, `todo_finish`, `pong`, `cd` |
|
||||
| **Background** | `bash_bg_cancel`, `bash_bg_status`, `bash_bg_list` |
|
||||
|
||||
- Uses the `rmcp` crate for the MCP protocol
|
||||
- Supports stdio-based transport (child process) and streamable HTTP
|
||||
- Tool discovery via `list_tools()` and dynamic tool registration
|
||||
`ToolCtx` berisi:
|
||||
- `session_dir: PathBuf` — direktori sesi aktif
|
||||
- `workspaces: Vec<PathBuf>` — root workspace yang dibuka
|
||||
|
||||
## LSP Integration
|
||||
## Background Shell Jobs (`infrastructure/src/bgbash/`)
|
||||
|
||||
`src/app/lsp/` provides Language Server Protocol support:
|
||||
Manajemen proses shell jangka panjang:
|
||||
- **Spawn** dengan Unix process groups (untuk kill seluruh tree)
|
||||
- **Output buffering** — collect stdout/stderr secara async
|
||||
- **Cancel/status/list** — kontrol via tool calls
|
||||
- **Progress monitoring** — track state: `Running`, `Completed`, `Failed`
|
||||
|
||||
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
|
||||
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
|
||||
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
|
||||
## MCP Manager (`infrastructure/src/mcp/`)
|
||||
|
||||
## Background Bash
|
||||
Integrasi **Model Context Protocol**:
|
||||
- Menggunakan crate `rmcp` (v2.2)
|
||||
- Transport: **stdio** (child process) dan **streamable HTTP**
|
||||
- `list_tools()` → tool discovery otomatis → registrasi ke tool harness
|
||||
- Persistent connection management
|
||||
|
||||
`src/app/bgbash/` manages long-running shell jobs:
|
||||
## LSP Integration (`infrastructure/src/lsp/`)
|
||||
|
||||
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
|
||||
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
|
||||
Integrasi **Language Server Protocol**:
|
||||
- **Auto-provisioner** — deteksi bahasa dari file extension, start LSP server yang sesuai
|
||||
- Mendukung: `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dan lainnya
|
||||
- **JSON-RPC client** — typed notifications + request/response
|
||||
- 7 tools LSP yang diekspose ke LLM
|
||||
|
||||
## Review System
|
||||
## Persistence (`infrastructure/src/persistence/`)
|
||||
|
||||
`src/app/subagent/auto.rs` spawns background reviews:
|
||||
### SQLite Message Log
|
||||
|
||||
- Quick review after every edit
|
||||
- Background test generation
|
||||
- Architecture review
|
||||
- Security review
|
||||
- All retry once on failure, escalate to blocking error if retry also fails
|
||||
Session database dengan `rusqlite` (bundled):
|
||||
- Per-session isolation
|
||||
- Table: `messages`, `sessions`
|
||||
- CRUD, query/filter, blob storage
|
||||
|
||||
### Settings Repository
|
||||
|
||||
`JsonSettingsRepository` — simpan/load `Settings` dari `settings.json`:
|
||||
- `provider`: nama provider LLM
|
||||
- `model`: model ID
|
||||
- `max_tokens`: override context window (default: 256k jika tidak diset)
|
||||
- `temperature`, `concise_output`, dll
|
||||
|
||||
### Memory Files
|
||||
|
||||
File-based memory di `~/.local/share/zesdex/memories/`:
|
||||
- Setiap memory = satu `.md` dengan frontmatter YAML
|
||||
- Fields: `name`, `description`, `type` (`user`/`feedback`/`project`/`reference`)
|
||||
- Index di `MEMORY.md`
|
||||
|
||||
## Session Management (`infrastructure/src/session/`)
|
||||
|
||||
- Setiap sesi memiliki UUID, direktori sendiri di `sessions/<uuid>/`
|
||||
- `.lock` file untuk cegah concurrent access
|
||||
- `session.json` — metadata (waktu mulai, workspace, model yang dipakai)
|
||||
|
||||
## Utils
|
||||
|
||||
| Util | Fungsi |
|
||||
|------|--------|
|
||||
| `utils::write_osc52` | Tulis teks ke clipboard via OSC52 escape sequence |
|
||||
| `Toast` / `ToastKind` | Notifikasi sementara (Success/Warning/Error/Info/Lesson) |
|
||||
| `TurnEvent` | Event dari background agent ke TUI (queue-based) |
|
||||
| `DirCache` | Cache async listing direktori untuk `@mention` autocomplete |
|
||||
| `MentionIndex` | Index file workspace untuk fuzzy autocomplete |
|
||||
| `SessionRuntime` | Runtime state: messages history, usage stats, session start time |
|
||||
|
||||
## OAuth 2.0
|
||||
|
||||
Flow PKCE untuk provider LLM:
|
||||
1. Generate code verifier + challenge
|
||||
2. Open browser ke authorization URL
|
||||
3. Start localhost HTTP server untuk tangkap redirect
|
||||
4. Exchange code → access + refresh token
|
||||
5. Simpan token di settings
|
||||
+114
-65
@@ -1,89 +1,138 @@
|
||||
# Data Architecture
|
||||
# Data & Persistence
|
||||
|
||||
## State Model
|
||||
## State Runtime (TUI)
|
||||
|
||||
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
|
||||
State TUI yang berjalan di memori adalah `AppStateRest` (`apps/interfaces/tui/src/state.rs`).
|
||||
|
||||
```
|
||||
AppStateRest
|
||||
├── session: SessionRuntime (hive_mind state, convergence flag)
|
||||
├── runtime: RuntimeState (mode, provider status)
|
||||
├── chat: ChatState (messages, scroll)
|
||||
├── input: InputState (text, cursor, autocomplete)
|
||||
├── settings: Settings (provider, model, temperature, concise_output)
|
||||
├── config: AppConfig (endpoints, credentials)
|
||||
├── scroll: ScrollState (per-panel offset)
|
||||
├── diff: DiffState (edit review)
|
||||
├── tools: Vec with outputs
|
||||
├── statusline, sidebar, etc.
|
||||
└── toasts: pending notifications
|
||||
### TranscriptCache
|
||||
|
||||
```rust
|
||||
pub struct TranscriptCache {
|
||||
pub messages: VecDeque<ChatMessageDisplay>, // O(1) eviction
|
||||
pub max_lines: usize, // default: 200
|
||||
pub dirty: bool, // perlu rebuild cache?
|
||||
}
|
||||
```
|
||||
|
||||
**Mutation rules** (per CLAUDE.md):
|
||||
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
|
||||
- Read-only from every other module
|
||||
- No generic update function — direct field mutation only
|
||||
Saat `dirty=true`, `pre_render_chat()` rebuild `display_lines_cache` (render markdown semua pesan) sebelum frame berikutnya.
|
||||
|
||||
## Persistence
|
||||
### SessionRuntime
|
||||
|
||||
### SQLite Message Log (`src/model/msglog/`)
|
||||
```rust
|
||||
pub struct SessionRuntime {
|
||||
pub messages: Vec<ChatMessage>, // history untuk LLM context
|
||||
pub usage: UsageStats, // token counting akumulasi
|
||||
pub session_start: i64, // unix ms saat sesi dimulai
|
||||
pub hive_mind_converged: bool, // flag selesai hive mind
|
||||
}
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `schema.rs` | Table definitions (messages, sessions) |
|
||||
| `mod.rs` | CRUD operations |
|
||||
| `query.rs` | Query helpers (search, filter) |
|
||||
| `blobs.rs` | Large message blob storage |
|
||||
| `summary.rs` | Conversation summary cache |
|
||||
## Context Window
|
||||
|
||||
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
|
||||
`resolve_context_window()` di `state.rs`:
|
||||
1. Ambil `settings.max_tokens` jika ada dan > 0
|
||||
2. Fallback ke **256.000 token** (default)
|
||||
|
||||
### Memory System (`src/model/memory.rs`)
|
||||
Token dihitung lazily via `count_tokens()` (tiktoken `cl100k_base`, fallback `len/4`), di-cache di `AppStateRest::cached_token_count`, hanya dihitung ulang saat `token_count_dirty=true`.
|
||||
|
||||
File-based memory stored under `~/.claude/projects/<project>/memory/`:
|
||||
## Persistence di Disk
|
||||
|
||||
- Each memory is one markdown file with frontmatter (name, description, type)
|
||||
- Types: `user`, `feedback`, `project`, `reference`
|
||||
- Memory index in MEMORY.md
|
||||
- Export/import for lesson sharing
|
||||
- PID-file session lock prevents concurrent access
|
||||
Semua data disimpan di **platform data directory**:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
### Settings & Config (`src/model/`)
|
||||
```
|
||||
~/.local/share/zesdex/
|
||||
├── settings.json # User settings (provider, model, max_tokens, dll)
|
||||
├── sessions/
|
||||
│ └── <uuid>/
|
||||
│ ├── session.json # Metadata sesi
|
||||
│ ├── messages.jsonl # Message log (append-only)
|
||||
│ └── .lock # Lock file (cegah concurrent access)
|
||||
├── memories/
|
||||
│ └── *.md # Memory files dengan frontmatter
|
||||
├── lessons/
|
||||
│ └── *.md # Lesson files (output dari learning system)
|
||||
└── worktrees/ # Git worktree per sesi (isolasi perubahan)
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `settings.rs` | Serialized user preferences (provider, model, theme) |
|
||||
| `app_config.rs` | Provider endpoints, API key resolution from env |
|
||||
| `session.rs` | Current session metadata |
|
||||
| `conversation.rs` | In-memory conversation state |
|
||||
| `editlog.rs` | Append-only JSONL edit audit trail |
|
||||
|
||||
### Edit Log
|
||||
|
||||
`src/model/editlog.rs` records every file mutation:
|
||||
### settings.json
|
||||
|
||||
```json
|
||||
{"ts": 123, "tool": "edit", "path": "src/main.rs",
|
||||
"reason": "fix bug", "content_sha256": "abc123",
|
||||
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"max_tokens": 256000,
|
||||
"temperature": 0.7,
|
||||
"concise_output": false
|
||||
}
|
||||
```
|
||||
|
||||
Max 5000 entries held in memory before pruning oldest.
|
||||
Diload via `JsonSettingsRepository::load()`, disimpan kembali saat TUI keluar (`state.save_settings()`).
|
||||
|
||||
## Context Management (`src/app/runtime/context/`)
|
||||
### Memory Files
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `tokens.rs` | Token counting via `tiktoken-rs` |
|
||||
| `window.rs` | Token window resolution (fit within model context) |
|
||||
| `dedup.rs` | Deduplication of repeated tool outputs |
|
||||
| `squash.rs` | Compression of large JSON tool results |
|
||||
| `shaping.rs` | Message dropping when context exceeds limits |
|
||||
Format markdown dengan YAML frontmatter:
|
||||
```markdown
|
||||
---
|
||||
name: prefer-early-return
|
||||
type: feedback
|
||||
description: Selalu gunakan early return untuk mengurangi nesting
|
||||
---
|
||||
|
||||
## IPC Data Flow
|
||||
Ketika menulis fungsi, gunakan early return/guard clauses daripada deep nesting.
|
||||
```
|
||||
|
||||
Types: `user`, `feedback`, `project`, `reference`
|
||||
|
||||
### Lesson Files
|
||||
|
||||
Hasil dari learning system, disimpan di `lessons/`:
|
||||
- Satu file per lesson
|
||||
- Plain markdown, dibaca oleh overlay `Learning`
|
||||
- Bisa di-accept/reject dari TUI
|
||||
|
||||
## SQLite (Message Log)
|
||||
|
||||
`rusqlite` dengan fitur `bundled` (tidak perlu install SQLite terpisah):
|
||||
- Per-session database di `sessions/<uuid>/messages.db`
|
||||
- Table `messages`: `id`, `session_id`, `role`, `content`, `timestamp`, `tokens`
|
||||
- Table `sessions`: `id`, `metadata`, `created_at`
|
||||
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
Daemon dan client berkomunikasi via **Unix domain socket**:
|
||||
|
||||
```
|
||||
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
|
||||
│
|
||||
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
|
||||
~/.local/share/zesdex/daemon.sock
|
||||
```
|
||||
|
||||
Frame format:
|
||||
```
|
||||
[4 bytes BE: payload length][JSON payload]
|
||||
```
|
||||
|
||||
Message types:
|
||||
- `Action` — client kirim aksi ke daemon
|
||||
- `StateSnapshot` — daemon kirim snapshot state ke client
|
||||
- `Ping/Pong` — keepalive
|
||||
|
||||
## Edit History
|
||||
|
||||
Agent mencatat setiap mutasi file ke event log internal:
|
||||
- Tool `write`/`edit`/`delete` merekam path, bytes delta, timestamp
|
||||
- Digunakan oleh subagent review untuk audit trail
|
||||
|
||||
## TurnEvent Queue
|
||||
|
||||
Agent berjalan di background thread dan mengirim events ke TUI via `Arc<Mutex<VecDeque<TurnEvent>>>`:
|
||||
|
||||
| Event | Payload | Efek di TUI |
|
||||
|-------|---------|-------------|
|
||||
| `AssistantMessage(msg)` | `ChatMessage` | Push ke transcript |
|
||||
| `ToolResult { output, .. }` | String | Push sebagai tool message |
|
||||
| `Usage { tokens_in, tokens_out }` | u64, u64 | Update `usage` stats |
|
||||
| `Error(msg)` | String | Toast error |
|
||||
| `Compacted(msgs)` | `Vec<ChatMessage>` | Update `session_runtime.messages` |
|
||||
| `SystemNote { kind, message }` | String | Push ke transcript |
|
||||
| `Done` | — | Set `turn_in_flight_flag = false` |
|
||||
+121
-80
@@ -1,99 +1,140 @@
|
||||
# Dependencies
|
||||
|
||||
## Rust Crates (30+ direct)
|
||||
Semua dependency dideklarasikan di `[workspace.dependencies]` dalam `Cargo.toml` root, lalu di-*inherit* oleh setiap crate anggota.
|
||||
|
||||
### Core Framework
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation |
|
||||
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
## Crate per Layer
|
||||
|
||||
### HTTP & Networking
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
|
||||
### Domain (`apps/domain`)
|
||||
|
||||
Hanya boleh pakai dependency yang tidak membawa I/O:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Serialisasi (derive) |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `chrono` | 0.4 | Tanggal/waktu |
|
||||
| `uuid` | 1 | UUID v4/v5 |
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `thiserror` | 1 | Derive error types |
|
||||
|
||||
### Infrastructure (`apps/infrastructure`)
|
||||
|
||||
Semua I/O, LLM, DB, tools:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `tokio` | 1 | Async runtime (rt-multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
| `reqwest` | 0.13 | HTTP client (json, stream, blocking, native-tls-vendored, form) |
|
||||
| `rusqlite` | 0.40 | SQLite (bundled — tidak perlu install sistem) |
|
||||
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
|
||||
| `webbrowser` | 1 | Open URLs in browser |
|
||||
| `tiktoken-rs` | 0.12 | Token counting (OpenAI cl100k) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting (default-fancy) |
|
||||
| `ignore` | 0.4 | File walking dengan `.gitignore` support |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed direktori ke binary |
|
||||
| `infer` | 0.19 | Deteksi tipe file dari byte signature |
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (untuk `@mention` autocomplete) |
|
||||
| `webbrowser` | 1 | Buka URL di browser (OAuth) |
|
||||
| `url` | 2 | URL parsing |
|
||||
| `percent-encoding` | 2 | URL encoding |
|
||||
|
||||
### HTML/Markdown
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
|
||||
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
|
||||
| `scraper` | 0.27.0 | HTML parsing/selecting |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
|
||||
|
||||
### Serialization
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `serde` | 1 | Serialization framework |
|
||||
| `serde_json` | 1 | JSON serialization |
|
||||
| `serde_yaml_ng` | 0.10 | YAML serialization |
|
||||
|
||||
### Storage & Files
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `rusqlite` | 0.40 | SQLite (bundled) |
|
||||
| `ignore` | 0.4 | `.gitignore`-aware file walking |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed directory contents in binary |
|
||||
| `infer` | 0.19 | File type detection |
|
||||
| `dirs` | 6 | Standard OS directories |
|
||||
|
||||
### Text & Search
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting |
|
||||
| `tiktoken-rs` | 0.12 | OpenAI token counting |
|
||||
|
||||
### Cryptography & Encoding
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `base64` | 0.22 | Base64 encoding |
|
||||
| `sha2` | 0.11 | SHA-256 hashing |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `uuid` | 1 | UUID generation (v4, v5) |
|
||||
| `libc` | 0.2 | Raw C FFI bindings |
|
||||
|
||||
### Error Handling & Logging
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `tracing` | 0.1 | Structured logging |
|
||||
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
|
||||
| `chrono` | 0.4 | Date/time with serde |
|
||||
|
||||
### Other
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `fast_html2md` | 0.0.62 | HTML → Markdown |
|
||||
| `scraper` | 0.27.0 | HTML parsing + CSS selector |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing |
|
||||
| `lsp-types` | 0.97 | LSP protocol types |
|
||||
| `futures-util` | 0.3 | Async stream combinators |
|
||||
| `libc` | 0.2 | Raw C FFI (Unix process groups) |
|
||||
|
||||
## External Services
|
||||
### TUI (`apps/interfaces/tui`)
|
||||
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| **Anthropic API** | Primary LLM provider |
|
||||
| **OpenAI API** | Alternative LLM provider (including OAuth) |
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation (raw mode, events, mouse) |
|
||||
| `base64` | 0.22 | Base64 (OSC52 clipboard) |
|
||||
| `sha2` | 0.11 | SHA-256 |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `dirs` | 6 | Platform data directory |
|
||||
|
||||
### API (`apps/interfaces/api`)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `axum` | 0.8 | HTTP server framework (macros) |
|
||||
| `tower` | 0.5 | Middleware layer |
|
||||
| `tower-http` | 0.6 | CORS, body limit |
|
||||
| `argon2` | 0.5 | Password hashing |
|
||||
| `jsonwebtoken` | 9 | JWT (HS256) |
|
||||
| `clap` | 4 | CLI argument parsing (derive) |
|
||||
| `rand_core` | 0.6 | Secure random (getrandom) |
|
||||
|
||||
### Serialization (semua layer)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Framework serialisasi |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `serde_yaml_ng` | 0.10 | YAML (frontmatter memory files) |
|
||||
|
||||
## Layanan Eksternal
|
||||
|
||||
| Layanan | Fungsi |
|
||||
|---------|--------|
|
||||
| **LLM Provider** | OpenAI/Anthropic-compatible API (default: OpenCode AI / DeepSeek) |
|
||||
| **MCP Servers** | Tool servers eksternal via stdio atau HTTP |
|
||||
| **LSP Servers** | `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dll |
|
||||
| **GitHub** | Release artifacts via semantic-release CI |
|
||||
| **MCP Servers** | External tool servers (stdio or HTTP) |
|
||||
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
|
||||
|
||||
## Build Configuration
|
||||
## Build & CI
|
||||
|
||||
### Compiler Lints (`.cargo/config.toml`)
|
||||
All unused code, dead code, and deprecation warnings promoted to errors:
|
||||
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
|
||||
### Compiler Lints (`Cargo.toml` workspace)
|
||||
|
||||
```toml
|
||||
[workspace.lints.rust]
|
||||
unused = "deny"
|
||||
dead_code = "deny"
|
||||
unreachable_code = "deny"
|
||||
unused_imports = "deny"
|
||||
unused_variables = "deny"
|
||||
unused_mut = "deny"
|
||||
unused_must_use = "deny"
|
||||
deprecated = "deny"
|
||||
trivial_casts = "deny"
|
||||
trivial_numeric_casts = "deny"
|
||||
|
||||
[workspace.lints.clippy]
|
||||
all = { level = "warn", priority = -1 }
|
||||
pedantic = { level = "warn", priority = -2 }
|
||||
```
|
||||
|
||||
### Release Profile
|
||||
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
|
||||
|
||||
```toml
|
||||
[profile.release]
|
||||
opt-level = 3
|
||||
lto = "fat"
|
||||
codegen-units = 1
|
||||
panic = "abort"
|
||||
strip = "symbols"
|
||||
overflow-checks = true
|
||||
```
|
||||
|
||||
### CI/CD
|
||||
- **CI**: cargo build + test + clippy on every push
|
||||
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
|
||||
|
||||
- **CI**: `cargo build` + `cargo test` + `cargo clippy --all-targets` pada setiap push
|
||||
- **Release**: semantic-release — auto changelog, Cargo.toml version bump, GitHub artifact upload
|
||||
- **Versioning**: `v1.17.0` (saat ini) — mengikuti semver dari commit messages
|
||||
|
||||
## Feature Flags Penting
|
||||
|
||||
| Crate | Feature | Alasan |
|
||||
|-------|---------|--------|
|
||||
| `rusqlite` | `bundled` | SQLite statically linked — tidak perlu install sistem |
|
||||
| `reqwest` | `blocking` | Sync HTTP untuk agent turn di blocking thread |
|
||||
| `reqwest` | `native-tls-vendored` | TLS tanpa dependency sistem |
|
||||
| `tokio` | `rt-multi-thread` | Async runtime multi-thread |
|
||||
| `pulldown-cmark` | *(no default)* | Tidak include semua fitur berat |
|
||||
| `syntect` | `default-fancy` | Syntax highlighting penuh |
|
||||
| `rmcp` | `transport-child-process` + `transport-streamable-http-client-reqwest` | MCP via stdio dan HTTP |
|
||||
@@ -0,0 +1,175 @@
|
||||
# Panduan Development
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Build semua crate
|
||||
cargo build
|
||||
|
||||
# Jalankan TUI (default)
|
||||
cargo run
|
||||
|
||||
# Jalankan dengan log debug
|
||||
RUST_LOG=debug cargo run
|
||||
|
||||
# Jalankan REST API
|
||||
cargo run -- --api --api-port 8080
|
||||
|
||||
# Build release
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
zesdex/
|
||||
├── Cargo.toml # Workspace root, semua dependency terpusat di sini
|
||||
├── Cargo.lock # Lock file (commit ini!)
|
||||
├── apps/
|
||||
│ ├── domain/ # Pure domain (tidak ada I/O)
|
||||
│ ├── application/ # Use-case services
|
||||
│ ├── infrastructure/ # Semua implementasi I/O
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # TUI — fokus pengembangan utama
|
||||
│ │ ├── api/ # REST API (Axum)
|
||||
│ │ ├── daemon/ # Daemon mode
|
||||
│ │ ├── ws/ # WebSocket
|
||||
│ │ ├── grpc/ # gRPC
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point
|
||||
│ └── bootstrap/ # Seeder
|
||||
└── docs/
|
||||
└── CODEMAPS/ # Dokumentasi ini
|
||||
```
|
||||
|
||||
## Menambah Tool Baru
|
||||
|
||||
1. Buat file baru di `apps/infrastructure/src/tools/<nama>.rs`
|
||||
2. Implement trait `Tool`:
|
||||
|
||||
```rust
|
||||
use zesdex_domain::core::tool_call::Tool;
|
||||
use anyhow::Result;
|
||||
use serde_json::Value;
|
||||
|
||||
pub struct MyTool;
|
||||
|
||||
impl Tool for MyTool {
|
||||
fn name(&self) -> &'static str { "my_tool" }
|
||||
fn description(&self) -> &'static str { "Deskripsi untuk LLM" }
|
||||
fn parameters(&self) -> Value {
|
||||
serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"param": { "type": "string", "description": "..." }
|
||||
},
|
||||
"required": ["param"]
|
||||
})
|
||||
}
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String> {
|
||||
let param = args["param"].as_str().unwrap_or("");
|
||||
Ok(format!("Result: {param}"))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Daftarkan di `apps/infrastructure/src/tools/mod.rs`:
|
||||
|
||||
```rust
|
||||
pub fn all_tools() -> Vec<Box<dyn Tool>> {
|
||||
vec![
|
||||
// ... tools lain ...
|
||||
Box::new(my_tool::MyTool),
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Action TUI Baru
|
||||
|
||||
1. Tambah variant ke `enum Action` di `apps/interfaces/tui/src/action.rs`
|
||||
2. Tangani di `apply_action()` match block yang sama
|
||||
3. Emit dari `controller/input.rs::handle_key()`
|
||||
|
||||
```rust
|
||||
// action.rs
|
||||
pub enum Action {
|
||||
// ... existing ...
|
||||
MyNewAction { data: String },
|
||||
}
|
||||
|
||||
// dalam apply_action:
|
||||
Action::MyNewAction { data } => {
|
||||
state.some_field = data;
|
||||
state.mark_dirty();
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Overlay Baru
|
||||
|
||||
1. Buat file `apps/interfaces/tui/src/view/overlays/<nama>.rs`
|
||||
2. Tambah variant ke `enum Overlay` di `state.rs`
|
||||
3. Tambah entry di `overlays/mod.rs::render_overlay()`
|
||||
4. Implement `pub fn render(frame, area, block, state)` di file baru
|
||||
|
||||
## Linting & Testing
|
||||
|
||||
```bash
|
||||
# Cek semua warnings/errors
|
||||
cargo clippy --all-targets
|
||||
|
||||
# Run tests
|
||||
cargo test
|
||||
|
||||
# Test satu crate saja
|
||||
cargo test -p zesdex-tui
|
||||
|
||||
# Check tanpa build (cepat)
|
||||
cargo check --all
|
||||
```
|
||||
|
||||
> **Penting**: Workspace ini menggunakan `deny` untuk hampir semua lint.
|
||||
> Kode harus compile bersih tanpa warning apapun.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Fungsi |
|
||||
|----------|--------|
|
||||
| `RUST_LOG` | Log level (`debug`, `info`, `warn`, `error`) |
|
||||
| `OPENAI_API_KEY` | API key LLM (jika tidak diset via settings) |
|
||||
| `ANTHROPIC_API_KEY` | API key Anthropic |
|
||||
| `ZESDEX_DATA_DIR` | Override direktori data (default: platform standard) |
|
||||
|
||||
## Data Directory
|
||||
|
||||
Saat development, data disimpan di:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
Untuk reset bersih:
|
||||
```bash
|
||||
rm -rf ~/.local/share/zesdex/
|
||||
```
|
||||
|
||||
## Konvensi Kode
|
||||
|
||||
- **Tidak ada `unwrap()`** di kode produksi — gunakan `?` atau `unwrap_or_default()`
|
||||
- **State hanya dimutasi dari `apply_action()`** — jangan mutasi `AppStateRest` dari view
|
||||
- **View functions bersifat read-only** — signature `fn draw(frame: &mut Frame, state: &AppStateRest)`
|
||||
- **Cache mahal dikomputasi sekali** — gunakan flag `dirty` dan `pre_render` pattern
|
||||
- **Semua string ke LLM harus deskriptif** — nama tool dan deskripsinya penting untuk LLM context
|
||||
|
||||
## Release
|
||||
|
||||
Release dilakukan via git tag semantic-release:
|
||||
|
||||
```bash
|
||||
git commit -m "feat: tambah fitur baru" # bumps minor
|
||||
git commit -m "fix: perbaiki bug" # bumps patch
|
||||
git commit -m "feat!: breaking change" # bumps major
|
||||
```
|
||||
|
||||
CI akan otomatis:
|
||||
1. Bump versi di `Cargo.toml`
|
||||
2. Generate `CHANGELOG.md`
|
||||
3. Build release binary
|
||||
4. Upload ke GitHub Releases
|
||||
+151
-64
@@ -1,79 +1,166 @@
|
||||
# Frontend (TUI) Architecture
|
||||
# TUI (Terminal User Interface)
|
||||
|
||||
## Render Pipeline
|
||||
Dibangun di atas **ratatui** + **crossterm**. Kode ada di `apps/interfaces/tui/src/`.
|
||||
|
||||
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
|
||||
## Struktur Source
|
||||
|
||||
```
|
||||
Timer tick
|
||||
│
|
||||
▼
|
||||
main.rs: fn tui_loop()
|
||||
│
|
||||
├── controller/input.rs: handle_key() → action
|
||||
├── app/runtime/actions/mod.rs: apply_action()
|
||||
│ │
|
||||
│ └── state mutates (AppStateRest)
|
||||
│
|
||||
└── view/mod.rs: build TUI layout
|
||||
│
|
||||
├── view/chat.rs: Chat transcript
|
||||
├── view/sidebar.rs: Usage dashboard
|
||||
├── view/status.rs: Status bar
|
||||
├── view/markdown.rs: Message renderer
|
||||
├── view/workflow.rs: Hive-mind progress
|
||||
└── view/theme.rs: Tokyo Night palette
|
||||
apps/interfaces/tui/src/
|
||||
├── run.rs # Event loop utama
|
||||
├── state.rs # AppStateRest — single source of truth
|
||||
├── action.rs # apply_action(): satu-satunya mutator state
|
||||
├── turn.rs # Spawn agent turn di background thread
|
||||
├── lib.rs # Re-export publik
|
||||
├── controller/
|
||||
│ ├── input.rs # Key handler → Vec<Action>
|
||||
│ └── command.rs # Slash command parser
|
||||
├── view/
|
||||
│ ├── mod.rs # Layout + pre_render() + draw()
|
||||
│ ├── chat.rs # Chat transcript panel (dengan display cache)
|
||||
│ ├── sidebar.rs # Sidebar: workflow, tasks, usage
|
||||
│ ├── status.rs # Status bar satu baris
|
||||
│ ├── markdown.rs # Markdown → styled Span (pulldown-cmark)
|
||||
│ ├── workflow.rs # Workflow/hive-mind progress panel
|
||||
│ ├── theme.rs # Tokyo Night color palette (const)
|
||||
│ └── overlays/ # 16 overlay panel
|
||||
└── model/ # Data model lokal TUI
|
||||
```
|
||||
|
||||
## Overlay System
|
||||
|
||||
16 overlays managed by `app/mode/`:
|
||||
|
||||
| Overlay | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| Chat input | `mod.rs` | Main input bar with autocomplete |
|
||||
| Bash | `bash.rs` | Interactive shell panel |
|
||||
| Editor | `editor.rs` | Built-in file editor |
|
||||
| Effort | `effort.rs` | LLM effort selector |
|
||||
| Help | `help.rs` | Keybindings help |
|
||||
| Key Input | `key_input.rs` | Custom key binding |
|
||||
| Learning | `learning.rs` | Lesson viewer |
|
||||
| Loading | `loading.rs` | Spinner overlay |
|
||||
| MCP | `mcp.rs` | MCP server management |
|
||||
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
|
||||
| Rewind | `rewind.rs` | Message/history rewind |
|
||||
| Settings | `settings.rs` | Settings panel |
|
||||
| Todo | `todo.rs` | Task/TODO list |
|
||||
| Workflow | (via view) | Workflow progress |
|
||||
|
||||
## Layout Structure
|
||||
## Render Pipeline (Per Frame)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Status Bar (view/status.rs) │
|
||||
├──────────────────────┬──────────────────────┤
|
||||
│ │ │
|
||||
│ Chat Transcript │ Sidebar │
|
||||
│ (view/chat.rs) │ (view/sidebar.rs) │
|
||||
│ scrollable, │ tokens, status, │
|
||||
│ inline-log style │ agent info │
|
||||
│ │ │
|
||||
├──────────────────────┴──────────────────────┤
|
||||
│ Input Bar + Autocomplete dropdown │
|
||||
│ (view/mod.rs) │
|
||||
└─────────────────────────────────────────────┘
|
||||
run_loop_inner() [50ms in-flight / 200ms idle]
|
||||
│
|
||||
├── drain expired toasts (1x, bukan 2x)
|
||||
│
|
||||
├── if dirty:
|
||||
│ view::pre_render(&mut state) ← update cache (markdown, token count)
|
||||
│ terminal.draw(|f| view::draw(f, &state))
|
||||
│ state.dirty = false
|
||||
│
|
||||
└── poll events → apply_action → Action::Tick
|
||||
```
|
||||
|
||||
## Input Handling
|
||||
### Optimasi Performa
|
||||
|
||||
`controller/input.rs`:
|
||||
| Masalah lama | Solusi saat ini |
|
||||
|---|---|
|
||||
| `count_tokens` (tiktoken) setiap frame | Cache `cached_token_count`, update hanya saat pesan baru |
|
||||
| `render_markdown` ulang setiap frame | `display_lines_cache` di `AppStateRest`, rebuild saat `transcript_cache.dirty` |
|
||||
| `Vec::remove(0)` untuk evict pesan lama | `VecDeque::pop_front()` — O(1) |
|
||||
| `Mutex<bool>` untuk `turn_in_flight` | `Arc<AtomicBool>` — lock-free |
|
||||
| Render terus meski idle | Skip `terminal.draw()` jika `dirty == false` |
|
||||
| Poll 50ms konstan | Adaptif: 50ms saat in-flight, 200ms saat idle |
|
||||
| `drain_expired_toasts` 2x per iterasi | Sekali saja di `run_loop_inner` |
|
||||
|
||||
- Normal mode: keystrokes go to the active overlay
|
||||
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
|
||||
- Tab cycles autocomplete candidates
|
||||
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
|
||||
- Arrow keys scroll chat, sidebar, and other scrollable panels
|
||||
## State (AppStateRest)
|
||||
|
||||
`AppStateRest` di `state.rs` adalah satu-satunya sumber kebenaran TUI:
|
||||
|
||||
```
|
||||
AppStateRest {
|
||||
settings: Settings // provider, model, dll
|
||||
app_config: AppConfig // endpoint, env vars
|
||||
workspace_roots: Vec<PathBuf> // working directories
|
||||
session_dir / session_id // path sesi aktif
|
||||
memory_dir // direktori memory
|
||||
session_runtime: Option<SessionRuntime> // history pesan, usage stats
|
||||
|
||||
transcript_cache: TranscriptCache // VecDeque<ChatMessageDisplay>
|
||||
scroll: ScrollState // offset scroll pane chat
|
||||
input: InputState // buffer, cursor, history, autocomplete
|
||||
misc: MiscState // overlay aktif, toasts, flags
|
||||
|
||||
turn_events: Arc<Mutex<VecDeque<TurnEvent>>> // queue event dari agent
|
||||
turn_in_flight_flag: Arc<AtomicBool> // apakah agent sedang jalan
|
||||
abort_flag: Arc<AtomicBool> // sinyal abort oleh user
|
||||
|
||||
// Cache performa
|
||||
display_lines_cache: Vec<Line<'static>> // hasil render markdown
|
||||
cached_token_count: usize // token count terkini
|
||||
token_count_dirty: bool // perlu hitung ulang?
|
||||
last_render_width: u16 // lebar terminal saat render terakhir
|
||||
|
||||
dirty: bool // perlu render ulang?
|
||||
quit: bool // keluar dari loop?
|
||||
}
|
||||
```
|
||||
|
||||
**Aturan mutasi:**
|
||||
- Dimutasi hanya dari `action.rs::apply_action()` dan `run.rs` (untuk dirty/quit)
|
||||
- Semua fungsi `view/*` bersifat read-only terhadap state
|
||||
- `pre_render_chat()` boleh mutasi hanya field cache (`display_lines_cache`, `cached_token_count`, `token_count_dirty`)
|
||||
|
||||
## Input & Actions
|
||||
|
||||
`controller/input.rs::handle_key()` → `Vec<Action>` → `apply_action(&mut state, action)`
|
||||
|
||||
Semua mutasi state melewati satu titik: `apply_action`. Controller tidak tahu *bagaimana* state diubah, hanya *action apa* yang dihasilkan.
|
||||
|
||||
### Action Utama
|
||||
|
||||
| Action | Efek |
|
||||
|--------|------|
|
||||
| `SubmitInput(text)` | Push ke transcript, spawn agent turn |
|
||||
| `Tick` | Drain `TurnEvent` queue, update state dari hasil agent |
|
||||
| `ScrollUp/Down` | Ubah `scroll.offset` |
|
||||
| `OpenOverlay(v)` | Set `misc.overlay = v` |
|
||||
| `Resize(w, h)` | Invalidasi cache display, set `last_render_width` |
|
||||
| `AbortTurn` | Store `true` ke `abort_flag` |
|
||||
| `ForceQuit` | Set `quit = true` |
|
||||
|
||||
## Overlays (16 Panel)
|
||||
|
||||
| Overlay | File | Fungsi |
|
||||
|---------|------|--------|
|
||||
| `Help` | `overlays/help.rs` | Daftar shortcut keyboard |
|
||||
| `Settings` | `overlays/settings.rs` | Panel pengaturan |
|
||||
| `Bash` | `overlays/bash.rs` | Background shell jobs |
|
||||
| `QuitConfirm` | `overlays/quit_confirm.rs` | Konfirmasi keluar |
|
||||
| `KeyInput` | `overlays/key_input.rs` | Capture key binding |
|
||||
| `Editor` | `overlays/editor.rs` | File editor inline |
|
||||
| `Effort` | `overlays/effort.rs` | Pilih level reasoning LLM |
|
||||
| `Mcp` | `overlays/mcp.rs` | Manajemen MCP server |
|
||||
| `Todo` | `overlays/todo.rs` | Daftar TODO |
|
||||
| `Rewind` | `overlays/rewind.rs` | Navigasi history pesan |
|
||||
| `Learning` | `overlays/learning.rs` | Viewer lesson |
|
||||
| `Usage` | `overlays/usage.rs` | Statistik token |
|
||||
| `Loading` | `overlays/loading.rs` | Spinner generik |
|
||||
| `ModelSelector` | `overlays/model_selector.rs` | Pilih model LLM |
|
||||
| `ClearConfirm` | `overlays/clear_confirm.rs` | Konfirmasi clear chat |
|
||||
|
||||
## Layout Terminal
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ Chat Transcript Sidebar (≥90) │
|
||||
│ (view/chat.rs) ┌────────────┐ │
|
||||
│ VecDeque messages │ Workflow │ │
|
||||
│ + markdown cache ├────────────┤ │
|
||||
│ scrollable │ Tasks │ │
|
||||
│ ├────────────┤ │
|
||||
│ │ Usage │ │
|
||||
│ └────────────┘ │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ❯ Input Bar + Autocomplete dropdown │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ⚡zesdex READY │ ...center... │ tok · model │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Sidebar hanya tampil jika lebar terminal ≥ 90 kolom.
|
||||
|
||||
## Theme
|
||||
|
||||
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
|
||||
`view/theme.rs` mendefinisikan palette **Tokyo Night** sebagai `const Color`:
|
||||
`PRIMARY`, `BG`, `SURFACE`, `SURFACE_ELEVATED`, `BORDER`, `TEXT`, `TEXT_DIM`, `TEXT_MUTED`, `SUCCESS`, `WARNING`, `ERROR`, `INFO`, `HIGHLIGHT`, `CODE_BG`, dll.
|
||||
|
||||
## Markdown Rendering
|
||||
|
||||
`view/markdown.rs::render_markdown(text, width, dim)`:
|
||||
- Parse dengan `pulldown-cmark`
|
||||
- Hasilkan `Vec<Span<'static>>` dengan styling
|
||||
- Support: heading, code block, diff block (warna +/-/@@), list, blockquote, table, inline code, link
|
||||
- `dim=true` → semua span memakai `TEXT_DIM` + italic (untuk tool output)
|
||||
- Hasil di-cache di `AppStateRest::display_lines_cache`
|
||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,649 +0,0 @@
|
||||
# CMS Wiring: Conversation + Rewind Blob Store Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the SQLite-backed `messages`/`archives`/`blobs` tables (`crates/zesdex-backend/src/model/msglog/`) with `zesdex-cms`'s `Conversation`/`JsonConversationRepository` for message content, plus a brand-new file-based blob repository for rewind snapshots (no `zesdex-cms` equivalent existed before this plan). Fix the `ChatMessage` type collision between `zesdex-cms`'s local duplicate and the canonical `zesdex_entities::seaorm::common::message::ChatMessage` used everywhere else in theLine truncated
|
||||
|
||||
**Architecture:** Research confirmed the `messages` table is write-only today (archived but never read back to restore a session) and the `archives` table is created but **never populated by any code path** — both can be retired with zero behavior loss. The `blobs` table is the one genuinely load-bearing piece (rewind feature reads it back) and needs a real, tested replacement — a new `RewindBlobRepository` trait + `FileRewindBlobRepository` impl added to `zesdex-cms`, storing raw bytes as Line truncated
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-cms`, `zesdex-backend`, `zesdex-entities`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No `#[allow(...)]` additions beyond what's already in touched files.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- This plan deliberately does **not** attempt to migrate historical data out of any existing `messages.sqlite` files — since the `messages`/`archives` tables were never read back by any code path, there is nothing meaningful to migrate. Existing `messages.sqlite` files are simply left on disk, unused, after this plan (a future cleanup could delete them, but doing so isn't required for correctness).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Reconcile the `ChatMessage`/`Role` type collision in `zesdex-cms`
|
||||
|
||||
**Context:** `crates/zesdex-cms/src/domain/conversation.rs` currently defines its own `Role`/`ChatMessage` (with `tool_calls: Option<Vec<serde_json::Value>>`, untyped) instead of reusing `zesdex_entities::seaorm::common::message::{Role, ChatMessage}` (the canonical type used throughout `zesdex-backend`, with `tool_calls: Option<Vec<ToolCall>>`, strongly typed). `zesdex-cms` already depends on `zesdex-entities` (confirmed in `Cargo.toml`), so this is a small, surgical fix.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/conversation.rs`
|
||||
- Modify: `crates/zesdex-cms/src/application/conversation_service.rs` (import path only)
|
||||
- Modify: `crates/zesdex-cms/src/domain/service.rs` (import path only)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `zesdex_cms::domain::conversation::{Conversation, ChatMessage, Role}` where `ChatMessage`/`Role` are now re-exports of the canonical entities type — anything constructing a `zesdex_cms::domain::conversation::ChatMessage` is now interchangeable with `zesdex_entities::seaorm::common::message::ChatMessage` used elsewhere in `zesdex-backend`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test proving type interchangeability**
|
||||
|
||||
Add to `crates/zesdex-cms/src/domain/conversation.rs`'s `#[cfg(test)] mod tests` (create if absent):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn chat_message_is_the_canonical_entities_type() {
|
||||
// This is a compile-time proof more than a runtime assertion: if
|
||||
// `zesdex_cms::domain::conversation::ChatMessage` were still a
|
||||
// distinct local type, this line would fail to compile.
|
||||
let canonical = zesdex_entities::seaorm::common::message::ChatMessage::user("hi");
|
||||
let via_cms: ChatMessage = canonical;
|
||||
assert_eq!(via_cms.content.as_deref(), Some("hi"));
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-cms chat_message_is_the_canonical -- --nocapture`
|
||||
Expected: compile error — `serde_json::Value` (cms's old `tool_calls` field type) vs `ChatMessage::user`'s type won't unify, or a straightforward type mismatch.
|
||||
|
||||
- [ ] **Step 3: Replace the local `Role`/`ChatMessage` with re-exports**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/conversation.rs`, delete the entire local `Role` enum and `ChatMessage` struct + impl block (the definitions, constructors `user`/`assistant`/`system`/`tool`), and replace the top of the file with:
|
||||
|
||||
```rust
|
||||
//! Pure Conversation entity — in-memory message history plus system prompt
|
||||
//! and LLM generation parameters.
|
||||
//!
|
||||
//! # Architecture
|
||||
//! This is a pure data structure with **no I/O logic**. Load/save
|
||||
//! responsibilities live in [`ConversationRepository`](super::repository::ConversationRepository).
|
||||
//!
|
||||
//! `ChatMessage`/`Role` are re-exported from `zesdex-entities` rather than
|
||||
//! duplicated here, so a `Conversation` built by this crate is
|
||||
//! interchangeable with the `ChatMessage` type used throughout
|
||||
//! `zesdex-backend`'s provider/tool-execution layer.
|
||||
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub use zesdex_entities::seaorm::common::message::{ChatMessage, Role};
|
||||
```
|
||||
|
||||
Keep the `Conversation` struct, `impl Conversation` block (`new`, `push`, `rebuild_system`, `to_api_messages`, `len`, `is_empty`) unchanged below this — they only reference `ChatMessage`/`Role` by name, which now resolve to the re-exported canonical types.
|
||||
|
||||
- [ ] **Step 4: Fix any now-broken references in the same crate**
|
||||
|
||||
Run: `cargo build -p zesdex-cms 2>&1 | head -60`
|
||||
|
||||
If `application/conversation_service.rs` or `domain/service.rs` import `ChatMessage`/`Role` via `use super::conversation::{ChatMessage, Conversation}` or similar — these continue to work unchanged since the names are still exported from `domain::conversation`, just backed by a different underlying type now. Only fix compile errors that actually appear; do not preemptively touch files the build doesn't flag.
|
||||
|
||||
- [ ] **Step 5: Run the test to verify it passes**
|
||||
|
||||
Run: `cargo test -p zesdex-cms chat_message_is_the_canonical -- --nocapture`
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run the crate's full test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-cms && cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms
|
||||
git commit -m "fix(cms): satukan ChatMessage/Role Conversation dengan tipe kanonik zesdex-entities"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add `RewindBlobRepository` to `zesdex-cms`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/repository.rs`
|
||||
- Create: `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs`
|
||||
- Modify: `crates/zesdex-cms/src/infrastructure/persistence/mod.rs`
|
||||
- Modify: `crates/zesdex-cms/Cargo.toml` (add `hex` and `chrono` if not already present — `chrono` is already a dependency per the crate's existing `Cargo.toml`; confirm `hex` with `grep hex crates/zesdex-cms/Cargo.toml` and add `hex.workspace = true` if missing)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `pub trait RewindBlobRepository { fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()>; fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> Result<Option<Vec<u8>>>; fn list_blob_keys(&self, session_dir: &Path) -> Result<Vec<String>>; }` and `pub struct FileRewindBlobRepository` — used by Task 4.
|
||||
|
||||
- [ ] **Step 1: Add the trait**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/repository.rs`, add:
|
||||
|
||||
```rust
|
||||
/// Repository for rewind-snapshot binary blobs, keyed by an arbitrary
|
||||
/// caller-supplied key (e.g. a tool-call id) within a session.
|
||||
pub trait RewindBlobRepository {
|
||||
/// Store (or overwrite) a blob under `blob_key` for this session.
|
||||
fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> anyhow::Result<()>;
|
||||
|
||||
/// Retrieve a blob's bytes by key, or `None` if not found.
|
||||
fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> anyhow::Result<Option<Vec<u8>>>;
|
||||
|
||||
/// List all blob keys for this session, oldest first.
|
||||
fn list_blob_keys(&self, session_dir: &Path) -> anyhow::Result<Vec<String>>;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the failing tests**
|
||||
|
||||
Create `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs` with:
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn tmp_dir() -> std::path::PathBuf {
|
||||
let dir = std::env::temp_dir().join(format!("zesdex-cms-blob-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn store_and_retrieve_roundtrip() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "tool-call-1", b"hello world", Some("text/plain")).unwrap();
|
||||
let bytes = repo.retrieve_blob(&dir, "tool-call-1").unwrap();
|
||||
assert_eq!(bytes, Some(b"hello world".to_vec()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retrieve_missing_key_returns_none() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
assert_eq!(repo.retrieve_blob(&dir, "no-such-key").unwrap(), None);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn list_blob_keys_returns_oldest_first() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "first", b"a", None).unwrap();
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
repo.store_blob(&dir, "second", b"b", None).unwrap();
|
||||
let keys = repo.list_blob_keys(&dir).unwrap();
|
||||
assert_eq!(keys, vec!["first".to_string(), "second".to_string()]);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overwriting_a_key_keeps_only_the_latest_entry_in_the_listing() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "k", b"v1", None).unwrap();
|
||||
repo.store_blob(&dir, "k", b"v2", None).unwrap();
|
||||
let keys = repo.list_blob_keys(&dir).unwrap();
|
||||
assert_eq!(keys, vec!["k".to_string()], "key must appear exactly once even after being overwritten");
|
||||
assert_eq!(repo.retrieve_blob(&dir, "k").unwrap(), Some(b"v2".to_vec()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(Requires `uuid` — already a `zesdex-cms` dependency per its `Cargo.toml`.)
|
||||
|
||||
- [ ] **Step 3: Run the tests to verify they fail**
|
||||
|
||||
Run: `cargo test -p zesdex-cms rewind_blob_repo:: 2>&1 | head -20`
|
||||
Expected: compile error (`FileRewindBlobRepository` doesn't exist yet).
|
||||
|
||||
- [ ] **Step 4: Implement `FileRewindBlobRepository`**
|
||||
|
||||
Add above the test module in the same file:
|
||||
|
||||
```rust
|
||||
//! Filesystem-backed `RewindBlobRepository` implementation.
|
||||
//!
|
||||
//! Blob bytes are stored at `<session_dir>/blobs/<hex(key)>.bin` (the key
|
||||
//! is hex-encoded as the filename to sidestep any path-traversal/invalid-
|
||||
//! filename-character concerns entirely, mirroring the simplicity of
|
||||
//! `Memory::slugify` elsewhere in this crate but without needing a
|
||||
//! human-readable filename). Key/ordering/mime-type metadata lives in an
|
||||
//! append-only `<session_dir>/blobs/index.jsonl`, one JSON line per
|
||||
//! `store_blob` call — the same JSONL-index pattern already used by
|
||||
//! `EditLogRepository`. `list_blob_keys` de-duplicates by keeping each
|
||||
//! key's *last* index line (so overwriting a key doesn't produce a
|
||||
//! duplicate listing entry) and returns keys ordered by first-seen
|
||||
//! `created_at` ascending (oldest first), matching the previous
|
||||
//! `SQLite`-backed `ORDER BY created_at ASC` behavior.
|
||||
use std::io::Write;
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::domain::repository::RewindBlobRepository;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
struct BlobIndexEntry {
|
||||
key: String,
|
||||
mime_type: Option<String>,
|
||||
created_at: i64,
|
||||
}
|
||||
|
||||
/// Concrete filesystem rewind-blob repository.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct FileRewindBlobRepository;
|
||||
|
||||
impl FileRewindBlobRepository {
|
||||
/// Create a new filesystem rewind-blob repository.
|
||||
pub fn new() -> Self {
|
||||
Self
|
||||
}
|
||||
|
||||
fn blobs_dir(session_dir: &Path) -> std::path::PathBuf {
|
||||
session_dir.join("blobs")
|
||||
}
|
||||
|
||||
fn blob_file_path(session_dir: &Path, blob_key: &str) -> std::path::PathBuf {
|
||||
Self::blobs_dir(session_dir).join(format!("{}.bin", hex::encode(blob_key.as_bytes())))
|
||||
}
|
||||
|
||||
fn index_path(session_dir: &Path) -> std::path::PathBuf {
|
||||
Self::blobs_dir(session_dir).join("index.jsonl")
|
||||
}
|
||||
}
|
||||
|
||||
impl RewindBlobRepository for FileRewindBlobRepository {
|
||||
fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()> {
|
||||
let blobs_dir = Self::blobs_dir(session_dir);
|
||||
std::fs::create_dir_all(&blobs_dir)
|
||||
.with_context(|| format!("failed to create blobs dir '{}'", blobs_dir.display()))?;
|
||||
|
||||
let path = Self::blob_file_path(session_dir, blob_key);
|
||||
let tmp = path.with_extension("bin.tmp");
|
||||
std::fs::write(&tmp, data)?;
|
||||
let f = std::fs::File::open(&tmp)?;
|
||||
f.sync_all()?;
|
||||
std::fs::rename(&tmp, &path)?;
|
||||
|
||||
let entry = BlobIndexEntry {
|
||||
key: blob_key.to_string(),
|
||||
mime_type: mime_type.map(String::from),
|
||||
created_at: chrono::Utc::now().timestamp_millis(),
|
||||
};
|
||||
let index_path = Self::index_path(session_dir);
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&index_path)
|
||||
.with_context(|| format!("failed to open blob index '{}'", index_path.display()))?;
|
||||
writeln!(f, "{}", serde_json::to_string(&entry)?)?;
|
||||
f.sync_all()?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> Result<Option<Vec<u8>>> {
|
||||
let path = Self::blob_file_path(session_dir, blob_key);
|
||||
if !path.exists() {
|
||||
return Ok(None);
|
||||
}
|
||||
Ok(Some(std::fs::read(&path)?))
|
||||
}
|
||||
|
||||
fn list_blob_keys(&self, session_dir: &Path) -> Result<Vec<String>> {
|
||||
let index_path = Self::index_path(session_dir);
|
||||
let Ok(content) = std::fs::read_to_string(&index_path) else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
// Keep only the last occurrence of each key (later overwrites win),
|
||||
// but remember first-seen order for the final ascending sort.
|
||||
let mut first_seen_order: Vec<String> = Vec::new();
|
||||
let mut latest_by_key: std::collections::HashMap<String, BlobIndexEntry> = std::collections::HashMap::new();
|
||||
for line in content.lines() {
|
||||
let Ok(entry) = serde_json::from_str::<BlobIndexEntry>(line) else {
|
||||
continue;
|
||||
};
|
||||
if !latest_by_key.contains_key(&entry.key) {
|
||||
first_seen_order.push(entry.key.clone());
|
||||
}
|
||||
latest_by_key.insert(entry.key.clone(), entry);
|
||||
}
|
||||
let mut entries: Vec<BlobIndexEntry> = first_seen_order
|
||||
.into_iter()
|
||||
.filter_map(|k| latest_by_key.get(&k).cloned())
|
||||
.collect();
|
||||
entries.sort_by_key(|e| e.created_at);
|
||||
Ok(entries.into_iter().map(|e| e.key).collect())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Register the module**
|
||||
|
||||
In `crates/zesdex-cms/src/infrastructure/persistence/mod.rs`, add:
|
||||
|
||||
```rust
|
||||
pub mod rewind_blob_repo;
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run the tests to verify they pass**
|
||||
|
||||
Run: `cargo test -p zesdex-cms rewind_blob_repo:: -- --nocapture`
|
||||
Expected: all 4 tests pass.
|
||||
|
||||
- [ ] **Step 7: Run clippy**
|
||||
|
||||
Run: `cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: no new warnings.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms/src/domain/repository.rs crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs crates/zesdex-cms/src/infrastructure/persistence/mod.rs crates/zesdex-cms/Cargo.toml
|
||||
git commit -m "feat(cms): tambahkan RewindBlobRepository berbasis file (pengganti tabel blobs SQLite)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Rewire message archiving (`archive_message`) to `Conversation`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines ~695-708 `spawn_turn`, ~727-744 `TurnCtx`, ~862-873 `archive_message`, plus all 7 call sites at lines 939, 1105, 1295, 1387, 1421, 1426, 1458 — re-confirm line numbers first since Task 5 of the OAuth/session plan and Task 5 of the settings/appconfig/memory/editlog plan may have shifted this file)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::conversation::{Conversation, ChatMessage}` (Task 1), `zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository`, `zesdex_cms::domain::repository::ConversationRepository`.
|
||||
- Produces: `TurnCtx.conversation: Option<Arc<Mutex<Conversation>>>` (replaces `TurnCtx.db: Option<Arc<Mutex<rusqlite::Connection>>>`).
|
||||
|
||||
- [ ] **Step 1: Confirm current line numbers**
|
||||
|
||||
Run: `grep -n "fn spawn_turn\|struct TurnCtx\|fn archive_message\|open_or_create\|tc\.db\.as_ref" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
- [ ] **Step 2: Replace the per-turn connection setup in `spawn_turn`**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let db = crate::model::msglog::open_or_create(&edit_session_dir)
|
||||
.ok()
|
||||
.map(|c| std::sync::Arc::new(std::sync::Mutex::new(c)));
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let conversation = {
|
||||
let repo = zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository::new();
|
||||
let conv = repo.load(&edit_session_dir).unwrap_or_else(|_| {
|
||||
zesdex_cms::domain::conversation::Conversation::new(String::new(), session_id.clone())
|
||||
});
|
||||
Some(std::sync::Arc::new(std::sync::Mutex::new(conv)))
|
||||
};
|
||||
```
|
||||
|
||||
Replace the `TurnCtx` struct literal's `db,` field with `conversation,`.
|
||||
|
||||
- [ ] **Step 3: Update the `TurnCtx` struct definition**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
db: Option<std::sync::Arc<std::sync::Mutex<rusqlite::Connection>>>,
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
conversation: Option<std::sync::Arc<std::sync::Mutex<zesdex_cms::domain::conversation::Conversation>>>,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Rewrite `archive_message`**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
fn archive_message(db: Option<&std::sync::Arc<std::sync::Mutex<rusqlite::Connection>>>, session_id: &str, msg: &ChatMessage) {
|
||||
if let Some(arc) = db {
|
||||
if let Ok(conn) = arc.lock() {
|
||||
let _ = crate::model::msglog::insert_message(&conn, session_id, msg);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
/// Persist a `ChatMessage` to the session's `Conversation`, if one is
|
||||
/// available for this turn.
|
||||
///
|
||||
/// Flow: if `conversation` is `Some`, lock the mutex, push the message,
|
||||
/// and rewrite `conversation.json` in full. Errors are silently ignored
|
||||
/// (matches the previous `SQLite`-backed behavior, which also swallowed
|
||||
/// insert failures).
|
||||
fn archive_message(
|
||||
conversation: Option<&std::sync::Arc<std::sync::Mutex<zesdex_cms::domain::conversation::Conversation>>>,
|
||||
session_dir: &std::path::Path,
|
||||
msg: &ChatMessage,
|
||||
) {
|
||||
if let Some(arc) = conversation {
|
||||
if let Ok(mut conv) = arc.lock() {
|
||||
conv.push(msg.clone());
|
||||
let repo = zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository::new();
|
||||
let _ = repo.save(session_dir, &conv);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Update all 7 call sites**
|
||||
|
||||
Run: `grep -n "archive_message(tc.db.as_ref()" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
At each of the 7 matched lines, replace `archive_message(tc.db.as_ref(), &tc.session_id, &<msg_var>)` with `archive_message(tc.conversation.as_ref(), &tc.edit_log_session_dir, &<msg_var>)` (keep whatever the actual message-variable name is at each site — `sys`, `pipeline_msg`, `response`, `review_msg`, `tool_msg`, `msg` per the research brief — only the first two arguments change).
|
||||
|
||||
- [ ] **Step 6: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (beyond anything Task 4's blob work below still needs to touch in the same file — if this task is done independently, `store_blob` call sites will still fail to compile at this point; that's expected and resolved by Task 4).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/runtime/actions/mod.rs
|
||||
git commit -m "refactor(backend): alihkan archive_message dari SQLite messages table ke zesdex-cms Conversation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Rewire rewind blob storage to `FileRewindBlobRepository`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs` (2 `store_blob` call sites, confirmed at line 531-533 and a second one — re-grep to find both)
|
||||
- Modify: `crates/zesdex-backend/src/app/mode/rewind.rs` (`rewind_count`, `rewind_to`, plus the `open_session_db` helper and the still-existing store_blob call site inside `execute_one_tool`/wherever the second engine.rs call lives)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository`, `zesdex_cms::domain::repository::RewindBlobRepository`.
|
||||
- Produces: nothing new for other tasks — this is the last consumer of the old `msglog::blobs` module.
|
||||
|
||||
- [ ] **Step 1: Find both `store_blob` call sites in `engine.rs`**
|
||||
|
||||
Run: `grep -n -B6 "store_blob" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
- [ ] **Step 2: Replace each `store_blob` call site**
|
||||
|
||||
Replace (pattern applies to both sites, adjusting the surrounding variable names per the actual code read in Step 1):
|
||||
```rust
|
||||
let _ = crate::model::msglog::store_blob(
|
||||
&conn, session_id, &tool_call.id, &bytes, None,
|
||||
);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new()
|
||||
.store_blob(&ctx.session_dir, &tool_call.id, &bytes, None);
|
||||
```
|
||||
|
||||
(Uses `RewindBlobRepository::store_blob` — add `use zesdex_cms::domain::repository::RewindBlobRepository;` to the file's imports. Note this drops the now-unneeded `conn`/`session_id`-derived-from-directory-name dance since the new repository takes `session_dir` directly — if the surrounding code only opened `conn` for this call, remove the now-dead connection-opening code too after confirming via Step 1's grep that nothing else in the same scope still needs it.)
|
||||
|
||||
- [ ] **Step 3: Rewrite `rewind.rs`'s `open_session_db` usage**
|
||||
|
||||
Read the whole file first: `cat crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace `rewind_count`:
|
||||
```rust
|
||||
pub fn rewind_count(state: &AppStateRest) -> usize {
|
||||
let Ok(conn) = open_session_db(&state.session_dir) else {
|
||||
return 0;
|
||||
};
|
||||
crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id)
|
||||
.ok()
|
||||
.map_or(0, |keys| keys.len())
|
||||
}
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
pub fn rewind_count(state: &AppStateRest) -> usize {
|
||||
zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new()
|
||||
.list_blob_keys(&state.session_dir)
|
||||
.ok()
|
||||
.map_or(0, |keys| keys.len())
|
||||
}
|
||||
```
|
||||
|
||||
Replace the body of `rewind_to` (the `open_session_db` call plus `list_blob_keys`/`retrieve_blob` calls):
|
||||
```rust
|
||||
let conn = match open_session_db(&state.session_dir) { /* ... */ };
|
||||
let keys = match crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id) { /* ... */ };
|
||||
/* ... */
|
||||
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key) { /* ... */ };
|
||||
```
|
||||
with (dropping the `conn`/`open_session_db` step entirely — the file-based repository needs no connection object, just `&state.session_dir`):
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new();
|
||||
|
||||
let keys = match repo.list_blob_keys(&state.session_dir) {
|
||||
Ok(k) => k,
|
||||
Err(e) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
format!("Failed to list snapshots: {e}"),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if keys.is_empty() || index >= keys.len() {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Warning,
|
||||
"No snapshot available at that index".to_string(),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
|
||||
let blob_key = &keys[index];
|
||||
let bytes = match repo.retrieve_blob(&state.session_dir, blob_key) {
|
||||
Ok(Some(b)) => b,
|
||||
Ok(None) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
"Snapshot data not found".to_string(),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
format!("Failed to retrieve snapshot: {e}"),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
(Keep whatever code follows `bytes` unchanged — the actual file-restoration logic doesn't depend on how `bytes` was fetched.)
|
||||
|
||||
- [ ] **Step 4: Delete the now-unused `open_session_db` helper**
|
||||
|
||||
If `open_session_db` (used only by the two call sites just replaced) has no other callers after Step 3 — verify with `grep -n "open_session_db" crates/zesdex-backend/src/app/mode/rewind.rs` — delete its definition entirely.
|
||||
|
||||
- [ ] **Step 5: Update the module doc comment**
|
||||
|
||||
Replace the file's top doc comment:
|
||||
```rust
|
||||
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
|
||||
//! session's `SQLite` blob store.
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
|
||||
//! session's file-based rewind blob store (`<session_dir>/blobs/`).
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 7: Run the workspace test suite**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 8: Manual smoke test**
|
||||
|
||||
In the TUI: perform a file edit (triggers a pre-edit snapshot store), open the Rewind overlay, confirm the snapshot count and list are correct, and restore the file — confirm the restored content matches the pre-edit version exactly.
|
||||
|
||||
- [ ] **Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/subagent/engine.rs crates/zesdex-backend/src/app/mode/rewind.rs
|
||||
git commit -m "refactor(backend): alihkan penyimpanan blob rewind ke FileRewindBlobRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Delete the now-dead `msglog` SQLite module
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-backend/src/model/msglog/{mod.rs,schema.rs,query.rs,blobs.rs}`
|
||||
- Modify: `crates/zesdex-backend/src/model/mod.rs` (remove `pub mod msglog;`)
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (line ~236, `edit_count: state.edit_log.len() as u32,` — confirm this doesn't reference `msglog` directly; if it only reads `state.edit_log`, no change needed here)
|
||||
|
||||
**Interfaces:** none — pure deletion after Tasks 3-4 remove every reference.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references**
|
||||
|
||||
Run: `grep -rln "model::msglog\|msglog::" crates/zesdex-backend/src`
|
||||
Expected: no output (only the `model/mod.rs` declaration itself, addressed in Step 3).
|
||||
|
||||
- [ ] **Step 2: Delete the files**
|
||||
|
||||
```bash
|
||||
git rm -r crates/zesdex-backend/src/model/msglog
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove the module declaration**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod msglog;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Build the whole workspace**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Expected: no errors. If `rusqlite` was only pulled into `zesdex-backend` for this module, `cargo build` will still succeed since `rusqlite` remains a workspace dependency used elsewhere (`zesdex-libs::database.rs`) — no `Cargo.toml` change needed here; confirm with `grep -rln "rusqlite" crates/zesdex-backend/src` that no other file in this crate still needs it, and if truly zero remaining uses, remove `rusqlite` from `crates/zesdex-backend/Cargo.toml`'s `[dependencies]` as a final cleanup (onLine truncated
|
||||
|
||||
- [ ] **Step 5: Run the full test suite and clippy**
|
||||
|
||||
Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: hapus modul msglog SQLite lama (digantikan Conversation + RewindBlobRepository)"
|
||||
```
|
||||
@@ -1,737 +0,0 @@
|
||||
# CMS Wiring: Settings, AppConfig, Memory, EditLog Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace `zesdex-backend`'s use of `zesdex_entities::seaorm::common::{settings,app_config,memory,edit_log}` (inherent-method I/O) with the previously-orphaned `zesdex-cms` crate's repository-based equivalents (`JsonSettingsRepository`, `JsonAppConfigRepository`, `MarkdownMemoryRepository`, `JsonlEditLogRepository`), which are on-disk-format-compatible drop-ins. Fix the one real defect found in `zesdex-cms::Settings` along the way (a missing `#[serde(default)]` that would hard-fail loadLine truncated
|
||||
|
||||
**Architecture:** `zesdex-backend` call sites move from `Type::static_method()` to `repository_instance.method(&base_dir, ...)`. Since `zesdex-cms`'s application-service layer (`SettingsServiceImpl`, `MemoryServiceImpl`) doesn't cover every read pattern the backend needs (no bare `AppConfig` getter, no single-memory read, no `EditLogService` at all), most call sites construct and use the concrete `Json*Repository`/`MarkdownMemoryRepository`/`JsonlEditLogRepository` types directly rather than goLine truncated
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-cms`, `zesdex-backend`, `zesdex-entities`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- On-disk formats are confirmed compatible for all four entities (same JSON/markdown/JSONL shape and file paths) — this plan is a call-site migration, **not** a data migration. No existing user `settings.json`/`app_config.json`/`*.md` memory files/`edits.jsonl` need to change.
|
||||
- `EditLog`'s domain shape genuinely differs between the old (`entries` + `path`, disk I/O on construction) and new (`entries` only, disk I/O via `EditLogRepository::open`) versions — every call site becomes fallible (`Result`) where it was previously infallible.
|
||||
- No new `#[allow(...)]` attributes. Existing ones in touched files are out of scope (handled by `2026-07-16-convention-cleanup-docs.md`).
|
||||
- Tests are inline `#[cfg(test)] mod tests`, per CLAUDE.md.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- Base directory resolution: everywhere the old code called the zero-argument `Settings::load()`/`Settings::save()`/`AppConfig::load()` (which internally called `zesdex_entities::seaorm::common::store::Store::new()` to get `base_dir`), the replacement must call `zesdex_entities::seaorm::common::store::Store::new().base_dir` explicitly and pass it to the `zesdex-cms` repository — this reproduces the exact same directory, verified identical during planning.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Fix the `hive_mind_node_timeout_ms` serde-default gap in `zesdex-cms::Settings`
|
||||
|
||||
**Context:** `crates/zesdex-entities/src/seaorm/common/settings.rs:75` has `#[serde(default = "default_hive_mind_node_timeout_ms")]` on this field; `crates/zesdex-cms/src/domain/settings.rs`'s copy does not. Without this, any settings.json written before this field existed (or any settings.json missing it for any reason) will hard-fail `JsonSettingsRepository::load` with a parse error, whereas the old `Settings::load()` silently defaulted on **any** failure. Fix both the missing default and resLine truncated
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/settings.rs`
|
||||
- Modify: `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `Settings::default()` unchanged in value; `JsonSettingsRepository::load` becomes tolerant of parse failures (still `Result`-returning, but only returns `Err` for I/O errors other than "not found" or "malformed JSON" — matching the old infallible-except-I/O-permission-errors behavior as closely as a `Result`-based API can).
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add to `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`'s `#[cfg(test)] mod tests` (create it if absent — check first: `grep -n "mod tests" crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn load_defaults_hive_mind_timeout_when_field_missing_from_old_settings_json() {
|
||||
let dir = std::env::temp_dir().join(format!("zesdex-cms-settings-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
// Simulate a settings.json written before `hive_mind_node_timeout_ms` existed.
|
||||
std::fs::write(
|
||||
dir.join("settings.json"),
|
||||
r#"{"internet_mode":"Off","provider":"zen","model":"m","api_keys":{},"max_tokens":null,"temperature":null,"review_max_lessons_per_run":5,"adaptive_review_max_skip":3,"verify_command":null,"verify_timeout_ms":30000,"workflow_max_concurrency":5,"review_enabled":true,"session_archive_enabled":true,"lsp_auto_provision":true,"lsp_languages":[]}"#,
|
||||
).unwrap();
|
||||
|
||||
let repo = JsonSettingsRepository::new();
|
||||
let settings = repo.load(&dir).expect("load must not fail on a pre-existing settings.json missing the new field");
|
||||
assert_eq!(settings.hive_mind_node_timeout_ms, 600_000);
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
```
|
||||
|
||||
(Requires `uuid` as a dev-dependency of `zesdex-cms` — check first: `grep uuid crates/zesdex-cms/Cargo.toml`; add `uuid = { workspace = true }` under `[dev-dependencies]` if missing, creating that section if it doesn't exist.)
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-cms load_defaults_hive_mind -- --nocapture`
|
||||
Expected: fails with a JSON parse/missing-field error.
|
||||
|
||||
- [ ] **Step 3: Add the serde default to the domain type**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/settings.rs`, add above the `Settings` struct:
|
||||
|
||||
```rust
|
||||
fn default_hive_mind_node_timeout_ms() -> u64 {
|
||||
600_000
|
||||
}
|
||||
```
|
||||
|
||||
And annotate the field:
|
||||
|
||||
```rust
|
||||
#[serde(default = "default_hive_mind_node_timeout_ms")]
|
||||
pub hive_mind_node_timeout_ms: u64,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Restore full parse-failure tolerance in the repository**
|
||||
|
||||
In `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`, update `load`:
|
||||
|
||||
```rust
|
||||
fn load(&self, base_dir: &Path) -> Result<Settings> {
|
||||
let path = base_dir.join("settings.json");
|
||||
match std::fs::read_to_string(&path) {
|
||||
Ok(s) => match serde_json::from_str(&s) {
|
||||
Ok(settings) => Ok(settings),
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"settings.json at '{}' failed to parse ({e}); falling back to defaults",
|
||||
path.display()
|
||||
);
|
||||
Ok(Settings::default())
|
||||
}
|
||||
},
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
|
||||
tracing::info!("settings.json not found, using defaults");
|
||||
Ok(Settings::default())
|
||||
}
|
||||
Err(e) => Err(anyhow::anyhow!("failed to read settings.json: {e}")),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run the test to verify it passes**
|
||||
|
||||
Run: `cargo test -p zesdex-cms load_defaults_hive_mind -- --nocapture`
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run the crate's full test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-cms && cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms/src/domain/settings.rs crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs
|
||||
git commit -m "fix(cms): perbaiki serde default hive_mind_node_timeout_ms & toleransi parse gagal di Settings"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Swap `Settings` call sites to `JsonSettingsRepository`
|
||||
|
||||
**Files (every one confirmed by research — swap all):**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 20, 49, 89, 103, 170)
|
||||
- `crates/zesdex-backend/src/bin/seed.rs` (line 12)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 61, 64-84, 353 doc comment)
|
||||
- `crates/zesdex-backend/src/app/workflow/hive_mind.rs` (lines 255, 281-283, 367 doc comments)
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs` (lines 9, 19-25, 46/56/79 test-only)
|
||||
- `crates/zesdex-backend/src/app/mode/settings.rs` (lines 6, 16-22)
|
||||
- `crates/zesdex-backend/src/controller/input.rs` (lines 339, 354, 357-361, 393-407)
|
||||
- `crates/zesdex-backend/src/view/mod.rs` (lines 152-174, 701-713 — read-only, no method-call change needed beyond the type import)
|
||||
- `crates/zesdex-backend/src/view/status.rs` (lines 68, 86-93 — read-only)
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs` (lines 79, 85, 405-406 — read-only)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 549, 633-671, 1615, 1758 — read-only)
|
||||
- `crates/zesdex-backend/src/main.rs` (lines 141, 474, 641 — `.save()` calls)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::settings::Settings`, `zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository`, `zesdex_cms::domain::repository::SettingsRepository` (trait, for method resolution), `zesdex_entities::seaorm::common::store::Store` (for `base_dir` resolution).
|
||||
- Produces: `AppStateRest.settings: zesdex_cms::domain::settings::Settings` (type changed from the old entities type — field-identical, so every **read-only** call site above needs only an import-path change, not a logic change).
|
||||
|
||||
- [ ] **Step 1: Update the type import everywhere it's read-only**
|
||||
|
||||
In each of `rest.rs`, `view/mod.rs`, `view/status.rs`, `app/review/mod.rs`, `app/runtime/actions/mod.rs`, replace:
|
||||
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
```
|
||||
|
||||
(For files that reference `Settings` only via `state.settings.<field>` without an explicit `use` for the type itself — confirm per-file with `grep -n "use.*settings::Settings\|model::settings" <file>` before editing — skip files where no explicit import exists, since `AppStateRest.settings`'s type change alone (Step 3 below) is what they actually depend on.)
|
||||
|
||||
- [ ] **Step 2: Update `app/mode/settings.rs` (mutates `Settings` in place, no I/O)**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/app/mode/settings.rs`. Replace:
|
||||
|
||||
```rust
|
||||
use crate::model::settings::{InternetMode, Settings};
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::{InternetMode, Settings};
|
||||
```
|
||||
|
||||
`cycle_internet_mode(settings: &mut Settings)`'s body (mutating `settings.internet_mode` in a cycle) needs no logic change — `InternetMode` is field-identical between old and new.
|
||||
|
||||
- [ ] **Step 3: Update `AppStateRest` construction and field type**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`:
|
||||
|
||||
Replace the import (line 20):
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
use zesdex_cms::domain::repository::SettingsRepository;
|
||||
use zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository;
|
||||
```
|
||||
|
||||
Replace the field's type comment reference (line 49) — no change needed, `pub settings: Settings,` already resolves to the new import.
|
||||
|
||||
Replace construction (line 89):
|
||||
```rust
|
||||
let settings = Settings::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let store_base_dir = zesdex_entities::seaorm::common::store::Store::new().base_dir;
|
||||
let settings = JsonSettingsRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update `controller/input.rs`'s `.save()` call sites**
|
||||
|
||||
Read the file first: `grep -n -B3 "\.settings\.save()" crates/zesdex-backend/src/controller/input.rs`
|
||||
|
||||
Both call sites (previously lines 361 and 407) currently do `let _ = state.settings.save();`. Since `Settings` no longer carries an inherent `save()` method, replace each with:
|
||||
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.save(&state.store_base_dir(), &state.settings);
|
||||
```
|
||||
|
||||
(Uses `state.store_base_dir()` — the existing helper on `AppStateRest`, confirmed to resolve to the same directory as `Store::new().base_dir` in normal operation — since these call sites already have `state: &mut AppStateRest` in scope, unlike `rest.rs::new()` which doesn't yet have a constructed `state` to call `.store_base_dir()` on.)
|
||||
|
||||
Add `use zesdex_cms::domain::repository::SettingsRepository;` to this file's imports if not already present after Step 1.
|
||||
|
||||
- [ ] **Step 5: Update `main.rs`'s three `.save()` call sites**
|
||||
|
||||
Read the file first: `grep -n -B2 "\.settings\.save()" crates/zesdex-backend/src/main.rs`
|
||||
|
||||
Replace each `let _ = state.settings.save();` (or `client_state.settings.save()`) with the same pattern as Step 4, substituting the correct state variable name at each site:
|
||||
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.save(&state.store_base_dir(), &state.settings);
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Update `app/subagent/engine.rs`'s `resolve_provider_config()`**
|
||||
|
||||
Read the function first: `grep -n -A 30 "fn resolve_provider_config" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let settings = crate::model::settings::Settings::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let store_base_dir = zesdex_entities::seaorm::common::store::Store::new().base_dir;
|
||||
let settings = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::SettingsRepository;` to this file's imports.) The subsequent field reads (`settings.api_keys`, `settings.provider`, `settings.model` at lines 64-84) need no change — same field names/types.
|
||||
|
||||
- [ ] **Step 7: Update `app/workflow/hive_mind.rs`'s `run_hive_mind()`**
|
||||
|
||||
Read the function first: `grep -n -A 5 "let settings = crate::model::settings::Settings::load" crates/zesdex-backend/src/app/workflow/hive_mind.rs`
|
||||
|
||||
Apply the identical substitution pattern from Step 6. Field reads `settings.hive_mind_node_timeout_ms`/`settings.workflow_max_concurrency` need no change.
|
||||
|
||||
- [ ] **Step 8: Update `app/runtime/context/window.rs`**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/app/runtime/context/window.rs`
|
||||
|
||||
Replace the import (line 9):
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
```
|
||||
|
||||
The `resolve(app_config: &AppConfig, settings: &Settings)` function signature/body and the three test-only `Settings::default()` constructions need no logic change — same type shape, `Default` still works identically after Task 1.
|
||||
|
||||
- [ ] **Step 9: Update `bin/seed.rs`**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/bin/seed.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
zesdex_entities::seaorm::common::settings::Settings::default()
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
zesdex_cms::domain::settings::Settings::default()
|
||||
```
|
||||
|
||||
- [ ] **Step 10: Remove the now-unused `model::settings` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod settings {
|
||||
pub use zesdex_entities::seaorm::common::settings::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 11: Verify no remaining references**
|
||||
|
||||
Run: `grep -rn "model::settings::" crates/zesdex-backend/src`
|
||||
Expected: no output.
|
||||
|
||||
- [ ] **Step 12: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all tests pass.
|
||||
|
||||
- [ ] **Step 13: Manual smoke test**
|
||||
|
||||
Run: `cargo run -p zesdex-backend`. Confirm the TUI starts, the Settings overlay (per `view/mod.rs`) displays the current provider/model/flags correctly, and changing internet mode / API key / provider / model persists correctly across a restart (settings.json is written and re-read with the same values).
|
||||
|
||||
- [ ] **Step 14: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan Settings ke zesdex-cms JsonSettingsRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Swap `AppConfig` call sites to `JsonAppConfigRepository`
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 18, 50, 90, 104)
|
||||
- `crates/zesdex-backend/src/bin/seed.rs` (line 27)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 62, 69, 73)
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs` (lines 8, 19-25, test sites)
|
||||
- `crates/zesdex-backend/src/controller/input.rs` (lines 220, 252, 383, 385)
|
||||
- `crates/zesdex-backend/src/view/mod.rs` (lines 710-711)
|
||||
- `crates/zesdex-backend/src/view/status.rs` (line 68)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 548, 551, 635-659, 1755-1758)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::app_config::{AppConfig, ProviderConfig, ModelRole}`, `zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository`, `zesdex_cms::domain::repository::AppConfigRepository`.
|
||||
- Produces: `AppStateRest.app_config: zesdex_cms::domain::app_config::AppConfig` (type changed, field-identical). `AppConfig` is never saved anywhere in `zesdex-backend` today (confirmed by research) — this task only needs `load`, no `save` call sites.
|
||||
|
||||
- [ ] **Step 1: Update read-only imports**
|
||||
|
||||
Same pattern as Task 2 Step 1: in each file that imports `crate::model::app_config::AppConfig`/`ProviderConfig`, replace with `zesdex_cms::domain::app_config::{AppConfig, ProviderConfig}` (add `ModelRole` too where `view/mod.rs`/`window.rs` need it).
|
||||
|
||||
- [ ] **Step 2: Update `AppStateRest` construction**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`, add to the import block from Task 2 Step 3:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::app_config::AppConfig;
|
||||
use zesdex_cms::domain::repository::AppConfigRepository;
|
||||
use zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository;
|
||||
```
|
||||
|
||||
Replace construction (line 90):
|
||||
```rust
|
||||
let app_config = AppConfig::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let app_config = JsonAppConfigRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Reuses the `store_base_dir` local variable already introduced in Task 2 Step 3 — both `Settings` and `AppConfig` load from the same base directory.)
|
||||
|
||||
- [ ] **Step 3: Update `app/subagent/engine.rs`**
|
||||
|
||||
Apply the same substitution as Task 2 Step 6, adding the `AppConfig` load right after the `Settings` load using the same `store_base_dir`:
|
||||
|
||||
```rust
|
||||
let app_config = zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::AppConfigRepository;`.)
|
||||
|
||||
- [ ] **Step 4: Update `bin/seed.rs`**
|
||||
|
||||
Replace `zesdex_entities::seaorm::common::app_config::AppConfig::default()` with `zesdex_cms::domain::app_config::AppConfig::default()`.
|
||||
|
||||
- [ ] **Step 5: Remove the now-unused `model::app_config` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod app_config {
|
||||
pub use zesdex_entities::seaorm::common::app_config::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Verify no remaining references, build, and test**
|
||||
|
||||
Run: `grep -rn "model::app_config::" crates/zesdex-backend/src` — expected no output.
|
||||
Run: `cargo build --workspace && cargo test --workspace` — expected all pass.
|
||||
|
||||
- [ ] **Step 7: Manual smoke test**
|
||||
|
||||
Run the TUI, open the Model Selector overlay (Ctrl+P or equivalent per `resources.rs`), confirm the provider/model list still populates correctly from `app_config.json`, and that Claude-credential auto-detection (if `~/.claude/settings.json` exists on the test machine) still merges in correctly.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan AppConfig ke zesdex-cms JsonAppConfigRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Swap `Memory` call sites to `MarkdownMemoryRepository`
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/tool/memory/recall.rs` (lines 4, 44, 63, 69)
|
||||
- `crates/zesdex-backend/src/tool/memory/remember.rs` (lines 4, 74, 79-96)
|
||||
- `crates/zesdex-backend/src/tool/memory/forget.rs` (lines 4, 45)
|
||||
- `crates/zesdex-backend/src/app/mode/learning.rs` (lines 56, 58)
|
||||
- `crates/zesdex-backend/src/app/workflow/docs.rs` (line 34 — `slugify` only, no I/O)
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs` (lines 487, 491, 493-494)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 591, 797, 810, 833, 843)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::memory::Memory`, `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository`, `zesdex_cms::domain::repository::MemoryRepository`.
|
||||
- Produces: nothing new for other tasks — `Memory` is a leaf entity with no state-struct field.
|
||||
|
||||
- [ ] **Step 1: `tool/memory/recall.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/tool/memory/recall.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
use crate::model::memory::Memory;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::memory::Memory;
|
||||
use zesdex_cms::domain::repository::MemoryRepository;
|
||||
use zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository;
|
||||
```
|
||||
|
||||
Replace (line 44): `Memory::read(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().load(&ctx.memory_dir, name)`
|
||||
Replace (line 63): `Memory::list(&ctx.memory_dir)` → `MarkdownMemoryRepository::new().list(&ctx.memory_dir)`
|
||||
Replace (line 69): `Memory::read(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().load(&ctx.memory_dir, name)`
|
||||
|
||||
(Both old inherent methods and the new repository methods return `Result<Memory>`/`Result<Vec<String>>` respectively — signature shape at the call site is unchanged beyond the receiver.)
|
||||
|
||||
- [ ] **Step 2: `tool/memory/remember.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/tool/memory/remember.rs`
|
||||
|
||||
Same import swap as Step 1. Replace (line 74): `Memory::slugify(name)` → `Memory::slugify(name)` (unchanged — `slugify` remains an inherent method on the domain struct in `zesdex-cms`, per research). The struct-literal construction (lines 79-92) needs no change (field-identical). Replace (lines 94-96):
|
||||
```rust
|
||||
memory.write(&ctx.memory_dir)?;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
MarkdownMemoryRepository::new().save(&ctx.memory_dir, &memory)?;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: `tool/memory/forget.rs`**
|
||||
|
||||
Same import swap. Replace (line 45): `Memory::remove(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().delete(&ctx.memory_dir, name)`.
|
||||
|
||||
- [ ] **Step 4: `app/mode/learning.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/app/mode/learning.rs`
|
||||
|
||||
Replace (line 56): `crate::model::memory::Memory::list(&state.memory_dir)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().list(&state.memory_dir)`
|
||||
Replace (line 58): `crate::model::memory::Memory::read(&state.memory_dir, &name)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().load(&state.memory_dir, &name)`
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::MemoryRepository;` at the top of the file.)
|
||||
|
||||
- [ ] **Step 5: `app/workflow/docs.rs`**
|
||||
|
||||
Replace (line 34): `crate::model::memory::Memory::slugify(user_request)` → `zesdex_cms::domain::memory::Memory::slugify(user_request)`. (Pure filename-generation helper, no repository/I/O involved — no other change needed.)
|
||||
|
||||
- [ ] **Step 6: `app/review/mod.rs`**
|
||||
|
||||
Read the surrounding code: `grep -n -B2 -A8 "model::memory::Memory::list(memory_dir)" crates/zesdex-backend/src/app/review/mod.rs`
|
||||
|
||||
Replace (line 487): `crate::model::memory::Memory::list(memory_dir)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().list(memory_dir)`
|
||||
Replace (line 491): `crate::model::memory::Memory::read(memory_dir, &name)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().load(memory_dir, &name)`
|
||||
Replace (lines 493-494):
|
||||
```rust
|
||||
mem.lifecycle = "stale".to_string();
|
||||
mem.write(memory_dir)?;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
mem.lifecycle = "stale".to_string();
|
||||
zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().save(memory_dir, &mem)?;
|
||||
```
|
||||
|
||||
- [ ] **Step 7: `app/runtime/actions/mod.rs`**
|
||||
|
||||
Read each site first: `grep -n -B2 -A2 "model::memory::Memory::" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Apply the same `Memory::method(dir, ...)` → `MarkdownMemoryRepository::new().method(dir, ...)` substitution at all 5 remaining sites (lines 591, 797, 810, 833, 843), matching the method-name mapping: `remove`→`delete`, `list`→`list`, `read`→`load`.
|
||||
|
||||
- [ ] **Step 8: Add `use` statements**
|
||||
|
||||
Add `use zesdex_cms::domain::repository::MemoryRepository;` and `use zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository;` to `app/review/mod.rs` and `app/runtime/actions/mod.rs` (both already import many things — add alongside existing `use` block).
|
||||
|
||||
- [ ] **Step 9: Remove the now-unused `model::memory` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod memory {
|
||||
pub use zesdex_entities::seaorm::common::memory::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 10: Verify no remaining references, build, test**
|
||||
|
||||
Run: `grep -rn "model::memory::" crates/zesdex-backend/src` — expected no output.
|
||||
Run: `cargo build --workspace && cargo test --workspace` — expected all pass.
|
||||
|
||||
- [ ] **Step 11: Manual smoke test**
|
||||
|
||||
In the TUI, invoke the `remember` tool to create a memory, `recall` it back, confirm the Learning overlay lists it, then `forget` it and confirm it's gone. Also confirm existing `.md` memory files from before this change (if any test fixtures exist) still load correctly (format compatibility check).
|
||||
|
||||
- [ ] **Step 12: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan Memory ke zesdex-cms MarkdownMemoryRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Swap `EditLog` call sites to `JsonlEditLogRepository`
|
||||
|
||||
**Context:** This is the most invasive of the four because every call site currently does `EditLog::new(dir)` (infallible, eager full-file read) and the new equivalent `JsonlEditLogRepository::open(dir)` returns `Result`. There are 7 distinct call sites plus one persistent field on `AppStateRest`.
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 19, 58, 115)
|
||||
- `crates/zesdex-backend/src/main.rs` (line 236 — reads `state.edit_log.len()`, no change needed beyond the type)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 571-580)
|
||||
- `crates/zesdex-backend/src/app/mode/rewind.rs` (lines 103-114, 128-134)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 921, 1473-1475, 1487-1489, 1586-1597)
|
||||
- `crates/zesdex-backend/src/model/mod.rs` (remove re-export)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::edit_log::{EditLog, EditLogEntry}`, `zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository`, `zesdex_cms::domain::repository::EditLogRepository`.
|
||||
- Produces: `AppStateRest.edit_log: zesdex_cms::domain::edit_log::EditLog` (type changed — no `path` field this time, so anything reading `state.edit_log.path` would break; confirmed by research that no call site does this).
|
||||
|
||||
- [ ] **Step 1: Update `AppStateRest`**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`, replace the import (line 19):
|
||||
```rust
|
||||
use crate::model::editlog::EditLog;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::edit_log::EditLog;
|
||||
use zesdex_cms::domain::repository::EditLogRepository;
|
||||
use zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository;
|
||||
```
|
||||
|
||||
Replace construction (line 115):
|
||||
```rust
|
||||
edit_log: EditLog::new(session_dir),
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
edit_log: JsonlEditLogRepository::new().open(session_dir).unwrap_or_else(|e| {
|
||||
tracing::warn!("[state] failed to open edit log at '{}': {e}", session_dir.display());
|
||||
EditLog::new()
|
||||
}),
|
||||
```
|
||||
|
||||
- [ ] **Step 2: `app/subagent/engine.rs`**
|
||||
|
||||
Read the site: `grep -n -B3 -A3 "EditLog::new(&ctx.session_dir)" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let mut el = crate::model::editlog::EditLog::new(&ctx.session_dir);
|
||||
el.append(entry).ok();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new();
|
||||
if let Ok(mut el) = repo.open(&ctx.session_dir) {
|
||||
let _ = repo.append(&ctx.session_dir, &mut el, entry);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: `app/mode/rewind.rs` — logging the rewind operation**
|
||||
|
||||
Read the site: `grep -n -B3 -A10 "EditLog::new(&state.session_dir)" crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace the first site (previously lines 103-114):
|
||||
```rust
|
||||
let mut el = crate::model::editlog::EditLog::new(&state.session_dir);
|
||||
let entry = crate::model::editlog::EditLogEntry { /* ... */ };
|
||||
let _ = el.append(entry);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new();
|
||||
if let Ok(mut el) = repo.open(&state.session_dir) {
|
||||
let entry = zesdex_cms::domain::edit_log::EditLogEntry { /* same field values as before */ };
|
||||
let _ = repo.append(&state.session_dir, &mut el, entry);
|
||||
}
|
||||
```
|
||||
|
||||
(Keep the exact same `EditLogEntry` field values from the original code — only the type path and the append mechanism change.)
|
||||
|
||||
- [ ] **Step 4: `app/mode/rewind.rs` — `find_edit_path`**
|
||||
|
||||
Read the site: `grep -n -B3 -A8 "fn find_edit_path" crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let el = crate::model::editlog::EditLog::new(&state.session_dir);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let el = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&state.session_dir)
|
||||
.unwrap_or_else(|_| zesdex_cms::domain::edit_log::EditLog::new());
|
||||
```
|
||||
|
||||
The subsequent `el.entries.iter().rev().find(...)` (unchanged — `entries` is a public field on both old and new `EditLog`) needs no further change.
|
||||
|
||||
- [ ] **Step 5: `app/runtime/actions/mod.rs` — turn-start snapshot**
|
||||
|
||||
Read the site: `grep -n -B2 -A2 "let initial_edits" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let initial_edits = crate::model::editlog::EditLog::new(&tc.edit_log_session_dir).len();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let initial_edits = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&tc.edit_log_session_dir)
|
||||
.map(|el| el.entries.len())
|
||||
.unwrap_or(0);
|
||||
```
|
||||
|
||||
(`EditLog` in `zesdex-cms` has no `.len()` inherent method — check first: `grep -n "fn len\|impl EditLog" crates/zesdex-cms/src/domain/edit_log.rs`. If `.len()` doesn't exist, use `.entries.len()` as shown; if it does exist, use `el.len()` instead for consistency with the rest of the codebase's naming.)
|
||||
|
||||
- [ ] **Step 6: `app/runtime/actions/mod.rs` — turn-end diff**
|
||||
|
||||
Read the site: `grep -n -B2 -A10 "let final_edits" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let el = crate::model::editlog::EditLog::new(&tc.edit_log_session_dir);
|
||||
let final_edits = el.len();
|
||||
let total_edits_this_turn = final_edits.saturating_sub(initial_edits);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let el = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&tc.edit_log_session_dir)
|
||||
.unwrap_or_else(|_| zesdex_cms::domain::edit_log::EditLog::new());
|
||||
let final_edits = el.entries.len();
|
||||
let total_edits_this_turn = final_edits.saturating_sub(initial_edits);
|
||||
```
|
||||
|
||||
And immediately after (previously lines 1487-1489), the `.entries.iter().skip(initial_edits)` loop needs no change — same field access.
|
||||
|
||||
- [ ] **Step 7: `app/runtime/actions/mod.rs` — `execute_one_tool`'s primary write path**
|
||||
|
||||
Read the site: `grep -n -B3 -A5 "EditLog::new(session_dir)" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Apply the same pattern as Step 2 (open + conditional append via the repository).
|
||||
|
||||
- [ ] **Step 8: Remove the now-unused `model::editlog` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod editlog {
|
||||
pub use zesdex_entities::seaorm::common::edit_log::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 9: Verify no remaining references**
|
||||
|
||||
Run: `grep -rn "model::editlog::" crates/zesdex-backend/src`
|
||||
Expected: no output.
|
||||
|
||||
- [ ] **Step 10: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all tests pass.
|
||||
|
||||
- [ ] **Step 11: Manual smoke test**
|
||||
|
||||
In the TUI, make a file edit via the `edit`/`write` tool, confirm `state.edit_log` grows and `main.rs`'s IPC `edit_count` payload reflects it, then use the rewind overlay to confirm `find_edit_path` still correctly recovers the edited file's path and the rewind itself works end-to-end.
|
||||
|
||||
- [ ] **Step 12: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan EditLog ke zesdex-cms JsonlEditLogRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Delete the now-dead `zesdex_entities::seaorm::common::{settings,app_config,memory,edit_log}` modules
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-entities/src/seaorm/common/settings.rs`, `app_config.rs`, `memory.rs`, `edit_log.rs`
|
||||
- Modify: `crates/zesdex-entities/src/seaorm/common/mod.rs` (remove their module declarations)
|
||||
|
||||
**Interfaces:** none — pure deletion after Tasks 2-5 have removed every reference.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references across the whole workspace**
|
||||
|
||||
Run: `grep -rln "seaorm::common::settings\|seaorm::common::app_config\|seaorm::common::memory\b\|seaorm::common::edit_log" crates --include='*.rs'`
|
||||
Expected: no output. (If anything other than the `mod.rs` declaration itself shows up, stop and investigate before deleting — it means a call site was missed in Tasks 2-5.)
|
||||
|
||||
- [ ] **Step 2: Delete the files**
|
||||
|
||||
```bash
|
||||
git rm crates/zesdex-entities/src/seaorm/common/settings.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/app_config.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/memory.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/edit_log.rs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove their module declarations**
|
||||
|
||||
In `crates/zesdex-entities/src/seaorm/common/mod.rs`, remove the corresponding `pub mod settings;`, `pub mod app_config;`, `pub mod memory;`, `pub mod edit_log;` lines (check first: `cat crates/zesdex-entities/src/seaorm/common/mod.rs`).
|
||||
|
||||
- [ ] **Step 4: Build the whole workspace**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 5: Run the full test suite and clippy**
|
||||
|
||||
Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: hapus entitas settings/app_config/memory/edit_log lama di zesdex-entities yang sudah digantikan zesdex-cms"
|
||||
```
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,373 +0,0 @@
|
||||
# Convention Cleanup + Documentation Repair Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Bring the codebase into compliance with `CLAUDE.md`'s own stated rules that the audit found violated — 110 `#[allow(...)]` lint-bypass attributes (10 of them silencing `dead_code`, which the workspace `Cargo.toml` explicitly `deny`s), a custom error type where only `anyhow` is supposed to be used, small doc-comment gaps — and repair the five `docs/CODEMAPS/*.md` files plus `CLAUDE.md` itself, which reference pre-workspace-migration paths that no longer exist.
|
||||
|
||||
**Architecture:** No structural changes to running code beyond what's needed to satisfy the lints without suppressing them. Documentation tasks are pure text corrections against the now-accurate `crates/` layout (this plan should run **after** the other four plans in this series, since they change many of the exact file paths the docs need to describe correctly).
|
||||
|
||||
**Tech Stack:** Rust, Markdown.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No **new** `#[allow(...)]` may be introduced by this plan's own changes.
|
||||
- Every dead-code removal must be verified by letting the compiler/clippy confirm the item has zero remaining callers — never delete on assumption.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` after each task.
|
||||
- **Run this plan last**, after `2026-07-16-security-quickfixes.md`, `2026-07-16-oauth-session-iam-wiring.md`, `2026-07-16-cms-settings-appconfig-memory-editlog-wiring.md`, `2026-07-16-cms-conversation-blob-wiring.md`, and `2026-07-16-middleware-axum-server.md` — the documentation tasks (Task 6) describe the *end state* of all five, and several files this plan touches for lint cleanup (`app/runtime/actions/mod.rs`, `app/runtime/context/*.rs`) are also touched by those plans.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Delete the unused custom `Error` type in `zesdex-utils`
|
||||
|
||||
**Context:** `crates/zesdex-utils/src/error.rs` defines a hand-rolled `pub enum Error` + `impl std::error::Error` + a `Result<T>` alias, directly contradicting CLAUDE.md's "anyhow::Result and anyhow::bail! throughout... No custom error types" rule. Confirmed via workspace-wide grep: **zero call sites reference it outside the file itself** — it's simply dead code, not something anything depends on. `thiserror` is declared as a `zesdex-utils` dependency but never imported anywhere in the crate Line truncated
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-utils/src/error.rs`
|
||||
- Modify: `crates/zesdex-utils/src/lib.rs` (remove `pub mod error;`)
|
||||
- Modify: `crates/zesdex-utils/Cargo.toml` (remove the unused `thiserror` dependency)
|
||||
|
||||
**Interfaces:** none — pure deletion.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references**
|
||||
|
||||
Run: `grep -rln "zesdex_utils::error\|zesdex_utils::Error\|utils::error::" crates --include='*.rs'`
|
||||
Expected: only `crates/zesdex-utils/src/error.rs` itself (or no output once the file is deleted).
|
||||
|
||||
- [ ] **Step 2: Delete the file**
|
||||
|
||||
```bash
|
||||
git rm crates/zesdex-utils/src/error.rs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove the module declaration**
|
||||
|
||||
In `crates/zesdex-utils/src/lib.rs`, remove:
|
||||
```rust
|
||||
pub mod error;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Remove the unused `thiserror` dependency**
|
||||
|
||||
In `crates/zesdex-utils/Cargo.toml`, remove:
|
||||
```toml
|
||||
thiserror = { workspace = true }
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test -p zesdex-utils`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore(utils): hapus custom Error type yang tidak dipakai (melanggar aturan anyhow-only)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Resolve the 10 `dead_code` allow-bypasses
|
||||
|
||||
**Context:** These directly contradict the workspace's own `dead_code = "deny"` lint. For each, remove the `#[allow(dead_code)]`/`#![allow(dead_code)]`, run the compiler, and act on its verdict: if genuinely unused, delete; if actually reachable through a path the lint can't see (e.g. only used in `#[cfg(test)]` or behind a feature), wire it into real production code instead of re-suppressing.
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/runtime/context/dedup.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/squash.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/tokens.rs:32`
|
||||
- `crates/zesdex-backend/src/app/subagent/spawn.rs:44,51`
|
||||
- `crates/zesdex-backend/src/app/state/misc.rs:409`
|
||||
- `crates/zesdex-backend/src/model/agent_def/{global.rs,builtin.rs,session.rs}:1`
|
||||
|
||||
**Interfaces:** varies per site — resolved during the investigation step, not fixed in advance (this is a "read what the compiler says, then act" task, not a hand-wave — see Step 1 of each site).
|
||||
|
||||
- [ ] **Step 1: `dedup.rs`, `squash.rs`, `window.rs` (module-level)**
|
||||
|
||||
Remove the `#![allow(dead_code)]` line from each of the three files. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "never used"`
|
||||
|
||||
For each item the compiler flags as unused: check whether it's covered by a test in the same file's `#[cfg(test)] mod tests` (a test-only user doesn't count as a real caller and doesn't justify keeping the item) — if the item has zero non-test callers, delete it; if grepping the item's name elsewhere in `crates/zesdex-backend/src` (outside the file and outside `#[cfg(test)]` blocks) turns up a real caller the compiler somehow didn't connect (e.g. it's `pub` and meant for a different module thLine truncated
|
||||
|
||||
- [ ] **Step 2: `tokens.rs:32` (`count_message_tokens`)**
|
||||
|
||||
Read the function and its context: `grep -n -B5 -A15 "fn count_message_tokens" crates/zesdex-backend/src/app/runtime/context/tokens.rs`
|
||||
|
||||
Remove `#[allow(dead_code)]`. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "count_message_tokens"`. If genuinely unused, delete the function (and any now-orphaned helper it alone called). If it looks like it *should* be called from the context-window-shaping logic in the same module (a token-counting function not being used by the token-budget code would itself be a functional gap worth flagging, not just a lint issue) — check `window.rs`'s `resolve()` and any `shaping`/`dedup` call siLine truncated
|
||||
|
||||
- [ ] **Step 3: `spawn.rs:44,51` (`with_max_steps`, `with_temperature` builder methods)**
|
||||
|
||||
Read the full builder struct: `grep -n -B20 "fn with_max_steps" crates/zesdex-backend/src/app/subagent/spawn.rs`
|
||||
|
||||
Remove both `#[allow(dead_code)]` lines. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "with_max_steps\|with_temperature"`.
|
||||
|
||||
These configure per-agent `max_steps`/`temperature` on a subagent-spawn builder — check every call site that constructs this builder (`grep -rn "AgentSpawnBuilder\|::new()" crates/zesdex-backend/src/app/subagent/` — use the builder's actual type name found in Step 3's read) to see whether any caller *should* be setting these (e.g. does `hive_mind.rs`'s node-spawning code hardcode a default that should instead come from `Settings`/`NodeDirective` and isn't?). If a real caller needs them, wirLine truncated
|
||||
|
||||
- [ ] **Step 4: `state/misc.rs:409` (`api_context_length` field)**
|
||||
|
||||
Read the surrounding struct: `grep -n -B15 -A5 "api_context_length" crates/zesdex-backend/src/app/state/misc.rs`
|
||||
|
||||
Remove `#[allow(dead_code)]`. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "api_context_length"`. Check whether the status bar (`view/status.rs`) or connectivity-check code (`spawn_api_connectivity_check` in `actions/mod.rs`) should be displaying/using the model's context length but currently isn't — if so, wire it in; if the field was superseded by `app_config.model_roles[...].context_window` (per the CMS wiring plan) and is now genuinely redundant, delete the field.
|
||||
|
||||
- [ ] **Step 5: `model/agent_def/{global.rs,builtin.rs,session.rs}` (module-level)**
|
||||
|
||||
Same procedure as Step 1: remove each `#![allow(dead_code)]`, build, and either delete unused items or wire in real callers based on what the compiler reports.
|
||||
|
||||
- [ ] **Step 6: Full workspace build and test after all 10 sites are resolved**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, no `dead_code` warnings anywhere (the workspace `deny` will turn any remaining one into a hard build failure, which is the actual verification that every site was genuinely resolved).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: hapus 10 allow(dead_code) - hapus kode mati atau sambungkan ke pemanggil nyata"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Resolve the 7 item-level clippy allows
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs:371` (`clippy::unnecessary_debug_formatting`)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs:1532` (`clippy::too_many_arguments`)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs:99,912` (`clippy::too_many_lines`, x2)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs:326` (`clippy::too_many_lines`)
|
||||
- `crates/zesdex-backend/src/view/markdown.rs:62` (`clippy::too_many_lines`)
|
||||
- `crates/zesdex-backend/src/view/mod.rs:112` (`clippy::too_many_lines`)
|
||||
|
||||
**Interfaces:** none shared across sites — each is an independent, local fix.
|
||||
|
||||
- [ ] **Step 1: `clippy::unnecessary_debug_formatting` (easiest — do first)**
|
||||
|
||||
Read the flagged line: `grep -n -B3 -A3 "unnecessary_debug_formatting" crates/zesdex-backend/src/app/review/mod.rs`
|
||||
|
||||
Remove the `#[allow(clippy::unnecessary_debug_formatting)]` line. Run: `cargo clippy -p zesdex-backend 2>&1 | grep -A5 "unnecessary_debug_formatting"` to see the exact suggestion (clippy always proposes the fix inline — typically replacing a `format!("{:?}", x)` with `x.to_string()` or a `Display` impl call). Apply the suggested fix exactly.
|
||||
|
||||
- [ ] **Step 2: `clippy::too_many_arguments` on `actions/mod.rs:1532`**
|
||||
|
||||
Read the flagged function's full signature: `grep -n -B2 -A15 "clippy::too_many_arguments" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Bundle the excess parameters into a purpose-named struct (the standard fix for this lint). For example, if the function is `fn foo(a: X, b: Y, c: Z, d: W, ...) -> R`, introduce:
|
||||
```rust
|
||||
struct FooParams {
|
||||
a: X,
|
||||
b: Y,
|
||||
c: Z,
|
||||
d: W,
|
||||
// ...
|
||||
}
|
||||
```
|
||||
and change the signature to `fn foo(params: FooParams) -> R`, updating the function body to read `params.a`/`params.b`/etc., and updating the single call site to construct `FooParams { a, b, c, d, ... }`. (Exact field names/types depend on the actual signature found in this step's read — do not guess, use the literal parameter list.)
|
||||
|
||||
- [ ] **Step 3: `clippy::too_many_lines` — `actions/mod.rs:99` and `:912` (`run_agent_turn`)**
|
||||
|
||||
Read the full function: `grep -n -A 250 "^fn run_agent_turn" crates/zesdex-backend/src/app/runtime/actions/mod.rs | head -260`
|
||||
|
||||
This is the core per-turn agent loop — do not split it mechanically by line count; split along its own documented phase boundaries (the function's doc comment already describes them: "build system prompt → shape messages → call `chat_with_tools_streaming` → handle tool calls or unwrap final message → check unfinished todos → finalize"). Extract each phase that doesn't need to mutate more than 2-3 local variables into its own well-named private function, threading only what each phasLine truncated
|
||||
|
||||
Run after each extraction: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
Once the function is under clippy's threshold, remove the `#[allow(clippy::too_many_lines)]` at both flagged lines (99 and 912 — confirm both are on `run_agent_turn` or its immediate helper via the Step 1 grep; if they're on two different functions, repeat this decomposition process for each independently).
|
||||
|
||||
- [ ] **Step 4: `clippy::too_many_lines` — `app/subagent/engine.rs:326`**
|
||||
|
||||
Read the flagged function in full: `grep -n -B2 -A 200 "clippy::too_many_lines" crates/zesdex-backend/src/app/subagent/engine.rs | head -210`
|
||||
|
||||
Apply the same phase-based extraction approach as Step 3, scaled to this function's actual structure (read it first — do not assume it mirrors `run_agent_turn`'s shape).
|
||||
|
||||
- [ ] **Step 5: `clippy::too_many_lines` — `view/markdown.rs:62` and `view/mod.rs:112`**
|
||||
|
||||
Read both flagged functions in full first (`grep -n -A 150 "clippy::too_many_lines" crates/zesdex-backend/src/view/markdown.rs` and the equivalent for `view/mod.rs`). These are rendering functions — split along rendering sub-sections (e.g. one function per overlay/pane already rendered inline in a big `match`), extracting each `match` arm's body over some line-count threshold into its own `fn render_<thing>(f: &mut Frame, area: Rect, state: &AppStateRest)`-shaped helper, matching the existingLine truncated
|
||||
|
||||
- [ ] **Step 6: Full workspace verification**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass with zero `too_many_lines`/`too_many_arguments`/`unnecessary_debug_formatting` warnings and no remaining `#[allow]` for any of them.
|
||||
|
||||
- [ ] **Step 7: Commit each function's decomposition separately as you go (already instructed inline above) — final wrap-up commit if anything remains uncommitted**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: pecah fungsi yang melanggar clippy::too_many_lines/too_many_arguments, hapus allow-nya"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Reduce the 93 module-level cast-quad allows
|
||||
|
||||
**Context:** `#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]` appears at the top of 93 files, evidently copy-pasted as workspace-wide boilerplate rather than justified per-file. This is the largest item in this plan by file count and — because it's the same mechanical recipe repeated 93 times — is best executed via `superpowers:subagent-driven-development` dispatching one subagent per file (or small batch of relatLine truncated
|
||||
|
||||
**Files:** all 93 listed in the audit's inventory (re-derive the authoritative current list before starting, since Tasks 1-3 and the other four plans in this series may have deleted or renamed some of them):
|
||||
|
||||
Run: `grep -rln "cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap" crates --include='*.rs'`
|
||||
|
||||
**Interfaces:** none shared — each file's fix is independent and self-contained.
|
||||
|
||||
- [ ] **Step 1: Worked example — pick one small, representative file first**
|
||||
|
||||
Read a small file from the list, e.g. `crates/zesdex-backend/src/app/mode/effort.rs` (confirm it's still in the current list from this task's Step-1 grep before using it as the example). Remove its `#![allow(clippy::cast_*...)]` line. Run:
|
||||
|
||||
Run: `cargo clippy -p zesdex-backend -- -D warnings 2>&1 | grep -B2 -A8 "effort.rs"`
|
||||
|
||||
For each flagged cast, apply the narrowest correct fix:
|
||||
- `x as u32` where `x: usize` and the value is a count/length that can't realistically exceed `u32::MAX` → `u32::try_from(x).unwrap_or(u32::MAX)` (saturating, since these are almost always display/telemetry values where saturating is safe) or, if the call site already returns `Result`, `u32::try_from(x)?`.
|
||||
- `x as i64` where `x: u64` timestamp (milliseconds since epoch) → these are safe until year 292471247, so `TryFrom` is technically correct but arguably pedantic; use `i64::try_from(x).unwrap_or(i64::MAX)` for consistency with the rule above rather than special-casing "this one's fine."
|
||||
- `x as f32`/`x as f64` (precision loss) on values already known to fit (e.g. small counters) → keep the cast but make it explicit and document why it's lossless in context: `#[expect(clippy::cast_precision_loss, reason = "...")]` is still a bypass and NOT allowed by CLAUDE.md — instead, if the value truly can't lose precision (e.g. casting a `u8` to `f32`), the lint won't even fire once the blanket module-level allow is removed, since clippy's precision-loss lint only fires above the pointLine truncated
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
Expected: no errors, no new warnings for this file.
|
||||
|
||||
- [ ] **Step 2: Commit the worked example**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/mode/effort.rs
|
||||
git commit -m "fix: hapus allow cast-quad di effort.rs, ganti cast lossy dengan try_from"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Dispatch the remaining files via subagent-driven-development**
|
||||
|
||||
For the remaining files from Step 1's grep (minus the one just fixed), use `superpowers:subagent-driven-development` with one task per file (or per small group of 3-5 files within the same module, where that reads more naturally), each task instructing: "remove the `#![allow(clippy::cast_*)]` header from `<file>`, run `cargo clippy -p <crate> -- -D warnings` scoped to that file, and fix every flagged cast using the recipe demonstrated in `2026-07-16-convention-cleanup-docs.md` Task 4 Step 1 (prLine truncated
|
||||
|
||||
- [ ] **Step 4: Final workspace-wide verification**
|
||||
|
||||
Run: `grep -rln "cast_possible_truncation, clippy::cast_sign_loss" crates --include='*.rs'`
|
||||
Expected: no output (or, if a small number of files remain and are judged genuinely fine to leave as a future increment, that's a call for whoever is running this plan to make explicitly and document — not silently left as-is).
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Fill the remaining doc-comment gaps and remove dead scaffolding
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/view/mod.rs` (lines 10-15 — `pub mod chat/markdown/sidebar/status/theme/workflow`, 5 of 7 missing doc comments)
|
||||
- Modify: `crates/zesdex-backend/src/app/state/misc.rs` (line 340 `pub fn submit`, and the second gap the audit found around line 438)
|
||||
- Delete: `/mnt/code/zesdex/tests/` (confirmed empty and untracked — safe to remove; re-verify emptiness before deleting since time has passed since the original audit)
|
||||
|
||||
**Interfaces:** none — doc comments and a directory deletion, no behavior change.
|
||||
|
||||
- [ ] **Step 1: Add doc comments to `view/mod.rs`'s module declarations**
|
||||
|
||||
Read the current lines: `grep -n -B1 "^pub mod" crates/zesdex-backend/src/view/mod.rs`
|
||||
|
||||
For each of `chat`, `markdown`, `sidebar`, `status`, `theme`, `workflow` that lacks a one-line doc comment above it, add one describing what that view submodule renders — e.g.:
|
||||
|
||||
```rust
|
||||
/// Chat transcript pane: renders the scrollback of user/assistant/tool messages.
|
||||
pub mod chat;
|
||||
/// Markdown-to-styled-text rendering for assistant message content.
|
||||
pub mod markdown;
|
||||
/// Session sidebar: file tree / workspace navigation pane.
|
||||
pub mod sidebar;
|
||||
/// Status bar: provider/model, token usage, connectivity indicator.
|
||||
pub mod status;
|
||||
/// Color theme definitions for the TUI.
|
||||
pub mod theme;
|
||||
/// Hive-mind workflow panel: live node progress display.
|
||||
pub mod workflow;
|
||||
```
|
||||
|
||||
(Read each module's actual top-of-file doc comment first — `head -5 crates/zesdex-backend/src/view/{chat,markdown,sidebar,status,theme,workflow}.rs` — and base the one-liner on what that file's own doc comment says, rather than guessing, so the two stay consistent.)
|
||||
|
||||
- [ ] **Step 2: Add doc comments to the two flagged functions in `state/misc.rs`**
|
||||
|
||||
Read both: `grep -n -B2 -A8 "pub fn submit" crates/zesdex-backend/src/app/state/misc.rs` and the second flagged line (re-locate it — the original audit found it around line 438, but Task 2's dead-code cleanup on this same file may have shifted line numbers; search for the nearest undocumented `pub fn` instead of trusting the stale line number).
|
||||
|
||||
Add a doc comment to `submit` describing what it does (flow: clone the input buffer as the result, push to history if non-empty and not a duplicate of the last entry, persist history to disk if a history file is configured; return the submitted text) following CLAUDE.md's What/Flow/Why/Return structure, and do the same for the second flagged function once located.
|
||||
|
||||
- [ ] **Step 3: Remove the empty `tests/` directory**
|
||||
|
||||
Run: `find /mnt/code/zesdex/tests -mindepth 1` to confirm it's still empty.
|
||||
Expected: no output.
|
||||
|
||||
If confirmed empty:
|
||||
```bash
|
||||
rmdir /mnt/code/zesdex/tests
|
||||
```
|
||||
|
||||
(Use `rmdir`, not `rm -rf` — it only succeeds if the directory is genuinely empty, which is the safety property we want here.)
|
||||
|
||||
- [ ] **Step 4: Build and verify**
|
||||
|
||||
Run: `cargo build --workspace && cargo doc --workspace --no-deps 2>&1 | grep -i warn`
|
||||
Expected: no new warnings from `cargo doc` (missing-docs isn't a workspace lint here, but this is a quick sanity pass).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/view/mod.rs crates/zesdex-backend/src/app/state/misc.rs
|
||||
git rm -r --cached tests 2>/dev/null || true
|
||||
git commit -m "docs: lengkapi doc comment view/mod.rs & state/misc.rs, hapus dir tests/ kosong"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Repair `docs/CODEMAPS/*.md` and `CLAUDE.md` to match the post-migration + post-wiring layout
|
||||
|
||||
**Context:** All five CODEMAPS files and CLAUDE.md itself reference pre-workspace-migration paths (`src/main.rs` instead of `crates/zesdex-backend/src/main.rs`, etc.), and `dependencies.md` describes a monolithic-crate dependency list that predates the workspace split entirely. Run this task **last**, after the other four plans in this series have landed, since many paths this task documents (OAuth location, Settings/AppConfig/Memory/EditLog/Conversation persistence, the new HTTP bridge) only eLine truncated
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/CODEMAPS/architecture.md`
|
||||
- Modify: `docs/CODEMAPS/backend.md`
|
||||
- Modify: `docs/CODEMAPS/frontend.md`
|
||||
- Modify: `docs/CODEMAPS/data.md`
|
||||
- Modify: `docs/CODEMAPS/dependencies.md`
|
||||
- Modify: `/mnt/code/zesdex/CLAUDE.md`
|
||||
|
||||
**Interfaces:** none — documentation only.
|
||||
|
||||
- [ ] **Step 1: Regenerate the authoritative file-path list**
|
||||
|
||||
Run: `find crates -name '*.rs' -not -path '*/target/*' | sort > /tmp/current-rust-files.txt` and keep this alongside the docs while editing, so every path cited is checked against a real file, not memory.
|
||||
|
||||
- [ ] **Step 2: Fix `architecture.md`**
|
||||
|
||||
For every code path mentioned (the Key Files table and inline references), prepend the correct crate prefix — e.g. `src/main.rs` → `crates/zesdex-backend/src/main.rs`, `src/app/harness.rs` → `crates/zesdex-backend/src/app/harness.rs`, and so on for every row. Cross-check each against `/tmp/current-rust-files.txt` from Step 1 before writing it. Update the ASCII system-layout diagram's "Tool/Subagents/Workflow" box if the OAuth rewiring (from `2026-07-16-oauth-session-iam-wiring.md`) or theLine truncated
|
||||
|
||||
- [ ] **Step 3: Fix `backend.md`**
|
||||
|
||||
Correct every path (`dto/provider/` → `crates/zesdex-dto/src/provider/`, `service/oauth/` → now `crates/zesdex-iam/src/{application/oauth_service.rs,infrastructure/oauth_loopback.rs}` per the OAuth rewiring plan, `src/ipc/*.rs` → `crates/zesdex-ipc/src/{protocol,conn,client,server}.rs`, etc.). Add a new subsection documenting the OAuth/session/CMS wiring: which crate now owns each concern (`zesdex-iam` for OAuth+session, `zesdex-cms` for Settings/AppConfig/Memory/EditLog/Conversation, `zeLine truncated
|
||||
|
||||
- [ ] **Step 4: Fix `frontend.md`**
|
||||
|
||||
Correct every `main.rs`/`controller/input.rs`/`view/*.rs`/`app/mode/*.rs` reference to include the `crates/zesdex-backend/src/` prefix.
|
||||
|
||||
- [ ] **Step 5: Fix `data.md`**
|
||||
|
||||
Correct `src/app/state/rest.rs` → `crates/zesdex-backend/src/app/state/rest.rs`. Replace the section describing `src/model/{settings,app_config,memory,edit_log}.rs` (all deleted by the CMS wiring plans) with a description of `zesdex-cms`'s repository-based persistence (`JsonSettingsRepository`, `JsonAppConfigRepository`, `MarkdownMemoryRepository`, `JsonlEditLogRepository`, `JsonConversationRepository`, `FileRewindBlobRepository`) and where each writes on disk. Replace the `msglog` SQLite desLine truncated
|
||||
|
||||
- [ ] **Step 6: Rewrite `dependencies.md`**
|
||||
|
||||
Replace the header claim ("23 Rust crates" per CLAUDE.md / "30+ direct" per this file's own header — pick neither, state the actual count) with an accurate summary: list all 9 internal workspace crates (`zesdex-entities`, `zesdex-utils`, `zesdex-dto`, `zesdex-ipc`, `zesdex-iam`, `zesdex-cms`, `zesdex-middleware`, `zesdex-libs`, `zesdex-backend`) with a one-line purpose each, then the external dependency list — regenerate this list from the actual `[workspace.dependencies]` table in the rootLine truncated
|
||||
|
||||
Run: `grep -A100 "\[workspace.dependencies\]" Cargo.toml`
|
||||
|
||||
Explicitly call out the dependencies added by the workspace migration that the current doc omits entirely: `axum`, `tower`, `tower-http`, `argon2`, `jsonwebtoken`, `thiserror` (note: `thiserror` may be removed from the workspace entirely by Task 1 of this plan if `zesdex-utils` was its only consumer — check with `grep -rln "thiserror" crates --include='*.rs' crates/*/Cargo.toml` before listing it as a current dependency).
|
||||
|
||||
- [ ] **Step 7: Fix `CLAUDE.md`**
|
||||
|
||||
Update every path in the "Key Files"-equivalent references (`src/main.rs`, `src/app/harness.rs`, `src/app/subagent/division.rs`, `src/app/workflow/hive_mind.rs`, `src/tool/workflow.rs`, `src/app/workflow/docs.rs`, `src/view/workflow.rs`, `src/app/subagent/auto.rs`) to their `crates/zesdex-backend/src/...` equivalents. Update the "Shell safety" line per `2026-07-16-security-quickfixes.md` Task 1 Step 4 if that plan hasn't already been applied. Update the "No custom error types" line's context ifLine truncated
|
||||
|
||||
- [ ] **Step 8: Verify every path cited resolves to a real file**
|
||||
|
||||
Run a small verification script for each doc — for every backtick-quoted path matching `src/` or `crates/`, confirm it exists:
|
||||
|
||||
```bash
|
||||
for f in docs/CODEMAPS/*.md CLAUDE.md; do
|
||||
grep -oE '`[a-zA-Z0-9_/.-]+\.rs`' "$f" | tr -d '`' | while read -r path; do
|
||||
[ -f "$path" ] || echo "MISSING in $f: $path"
|
||||
done
|
||||
done
|
||||
```
|
||||
|
||||
Expected: no `MISSING` lines. Fix any that appear.
|
||||
|
||||
- [ ] **Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/CODEMAPS CLAUDE.md
|
||||
git commit -m "docs: perbaiki path stale di CODEMAPS dan CLAUDE.md pasca migrasi workspace + wiring zesdex-iam/cms/middleware"
|
||||
```
|
||||
@@ -1,523 +0,0 @@
|
||||
# Middleware Axum Server Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Give the previously-orphaned `zesdex-middleware` crate (`SessionAuthLayer`, `default_cors_layer`, `RateLimitLayer`) a genuine integration point by adding an optional HTTP transport for the existing daemon, alongside (not replacing) the current Unix-socket transport.
|
||||
|
||||
**Important scope note — read before implementing:** unlike the OAuth/session/CMS wiring plans, there is **no existing HTTP server to fix or complete** — research confirmed zero axum usage anywhere in `zesdex-backend` and no design doc describing what one should do. This plan is therefore new-feature work, deliberately scoped as narrowly as possible: it exposes the *exact same* `ClientRequest`/`DaemonFrame` protocol the Unix-socket daemon already speaks, over HTTP, gated by the three middleLine truncated
|
||||
|
||||
**Architecture:** Extract the daemon's per-request handling logic (`handle_daemon_client`'s match-on-`ClientRequest` body plus `send_daemon_update`) into transport-agnostic functions shared by both the existing Unix-socket loop and a new axum route. The state-owning thread gains an `mpsc` channel; the axum handler sends `(ClientRequest, oneshot::Sender<Vec<DaemonFrame>>)` and awaits the reply. `--http-port <PORT>` is a new opt-in CLI flag on `--daemon` — when absent, behavior is byte-for-byteLine truncated
|
||||
|
||||
**Tech Stack:** Rust, `axum`, `tokio` (already workspace deps), `zesdex-middleware`.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- The Unix-socket transport's behavior must be provably unchanged — the refactor in Task 1 extracts logic without altering it, verified by the existing (or newly added, if none exist) daemon tests passing identically before and after.
|
||||
- No new `#[allow(...)]` attributes.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- The HTTP transport is opt-in (`--http-port`) and OFF by default — it must not change any existing invocation's behavior.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Extract transport-agnostic request handling from `handle_daemon_client`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (functions `handle_daemon_client` ~line 336, `send_daemon_update` ~line 207)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `fn build_state_update_frame(state: &AppStateRest) -> ipc::protocol::DaemonFrame` (pure builder, extracted from `send_daemon_update`) and `fn process_client_request(state: &mut AppStateRest, req: ipc::protocol::ClientRequest) -> (bool, Vec<ipc::protocol::DaemonFrame>)` (pure state-mutation + frame-collection, extracted from `handle_daemon_client`'s match body) — both consumed by Task 3's axum handler and Task 2's refactored Unix-socket loop.
|
||||
|
||||
- [ ] **Step 1: Extract `build_state_update_frame`**
|
||||
|
||||
In `crates/zesdex-backend/src/main.rs`, split `send_daemon_update` (current body at line ~207-238) into a pure builder plus a thin I/O wrapper:
|
||||
|
||||
```rust
|
||||
/// Flatten the daemon's `AppStateRest` into a `StatePayload` wrapped in a
|
||||
/// `DaemonFrame::StateUpdate` — the pure, transport-agnostic half of what
|
||||
/// was previously `send_daemon_update`.
|
||||
///
|
||||
/// Why: the client never shares memory with the daemon, so every action
|
||||
/// on the daemon side is followed by a full state push rather than a diff.
|
||||
fn build_state_update_frame(state: &app::state::rest::AppStateRest) -> ipc::protocol::DaemonFrame {
|
||||
use ipc::protocol::{DaemonFrame, MessageEntry, ToastEntry, StatePayload};
|
||||
|
||||
let messages: Vec<MessageEntry> = state.transcript_cache.messages.iter().map(|m| {
|
||||
MessageEntry {
|
||||
role: format!("{:?}", m.role),
|
||||
content: m.content.clone(),
|
||||
timestamp: m.timestamp,
|
||||
}
|
||||
}).collect();
|
||||
|
||||
let toasts: Vec<ToastEntry> = state.misc.toasts.iter().map(|t| {
|
||||
ToastEntry {
|
||||
kind: format!("{:?}", t.kind),
|
||||
message: t.message.clone(),
|
||||
created_at: t.created_at,
|
||||
lifetime_ms: t.lifetime_ms,
|
||||
}
|
||||
}).collect();
|
||||
|
||||
let overlay = if state.misc.overlay.is_active() {
|
||||
Some(format!("{:?}", state.misc.overlay))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// Keep every remaining `StatePayload` field exactly as the original
|
||||
// `send_daemon_update` built it (input buffer/cursor, etc.) — copy the
|
||||
// rest of the struct-literal body unchanged from the pre-refactor code.
|
||||
DaemonFrame::StateUpdate(Box::new(StatePayload {
|
||||
session_id: state.session_id.clone(),
|
||||
messages,
|
||||
toasts,
|
||||
overlay,
|
||||
// ...(remaining fields copied verbatim from the original function)
|
||||
}))
|
||||
}
|
||||
|
||||
/// Send a `DaemonFrame::StateUpdate` to an attached Unix-socket client.
|
||||
fn send_daemon_update(conn: &mut ipc::conn::Connection, state: &app::state::rest::AppStateRest) -> Result<()> {
|
||||
conn.send(&build_state_update_frame(state))
|
||||
}
|
||||
```
|
||||
|
||||
(Read the full original `send_daemon_update` body first — `sed`/`grep -n -A 45 "fn send_daemon_update" crates/zesdex-backend/src/main.rs` — and carry over every `StatePayload` field exactly; the excerpt above only shows the fields already visible in this plan's earlier research, do not drop any field the original builds.)
|
||||
|
||||
- [ ] **Step 2: Extract `process_client_request`**
|
||||
|
||||
Replace `handle_daemon_client`'s inner `match req { ... }` block with a new standalone function that returns frames instead of writing to a `Connection`:
|
||||
|
||||
```rust
|
||||
/// Apply one `ClientRequest` to `state` and collect the `DaemonFrame`(s) it
|
||||
/// produces — the pure, transport-agnostic half of what was previously
|
||||
/// inlined in `handle_daemon_client`'s read loop.
|
||||
///
|
||||
/// Return: `(keep_running, frames)` — `keep_running` is `false` only for
|
||||
/// `ClientRequest::Close`; `frames` always ends with a `StateUpdate` frame,
|
||||
/// preceded by a `ClipboardCopy` frame if a copy was pending.
|
||||
fn process_client_request(
|
||||
state: &mut app::state::rest::AppStateRest,
|
||||
req: ipc::protocol::ClientRequest,
|
||||
) -> (bool, Vec<ipc::protocol::DaemonFrame>) {
|
||||
use app::runtime::actions::{Action, apply_action};
|
||||
use ipc::protocol::ClientRequest;
|
||||
|
||||
let mut running = true;
|
||||
match req {
|
||||
ClientRequest::Tick => {
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::KeyPress { key, ctrl, alt, shift } => {
|
||||
let mut modifiers = crossterm::event::KeyModifiers::NONE;
|
||||
if ctrl { modifiers |= crossterm::event::KeyModifiers::CONTROL; }
|
||||
if alt { modifiers |= crossterm::event::KeyModifiers::ALT; }
|
||||
if shift { modifiers |= crossterm::event::KeyModifiers::SHIFT; }
|
||||
let key_event = crossterm::event::KeyEvent::new(key_action_to_code(&key), modifiers);
|
||||
let actions = controller::input::handle_key(key_event, state);
|
||||
for action in actions {
|
||||
apply_action(state, action);
|
||||
}
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Submit(text) => {
|
||||
state.input.buffer = text;
|
||||
let enter_event = crossterm::event::KeyEvent::new(crossterm::event::KeyCode::Enter, crossterm::event::KeyModifiers::NONE);
|
||||
let actions = controller::input::handle_key(enter_event, state);
|
||||
for action in actions {
|
||||
apply_action(state, action);
|
||||
}
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Paste(text) => {
|
||||
state.input.buffer.insert_str(state.input.cursor, &text);
|
||||
state.input.cursor += text.len();
|
||||
state.dirty = true;
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Resize(w, h) => {
|
||||
apply_action(state, Action::Resize(w, h));
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::ScrollUp => {
|
||||
apply_action(state, Action::ScrollUp);
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::ScrollDown => {
|
||||
apply_action(state, Action::ScrollDown);
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Close => {
|
||||
running = false;
|
||||
}
|
||||
}
|
||||
|
||||
let mut frames = Vec::new();
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
frames.push(ipc::protocol::DaemonFrame::ClipboardCopy(text));
|
||||
}
|
||||
frames.push(build_state_update_frame(state));
|
||||
(running, frames)
|
||||
}
|
||||
```
|
||||
|
||||
(Every match arm's body is copied verbatim from the pre-refactor `handle_daemon_client` — no logic changes, only relocation.)
|
||||
|
||||
- [ ] **Step 3: Rewrite `handle_daemon_client` as a thin wrapper**
|
||||
|
||||
```rust
|
||||
fn handle_daemon_client(
|
||||
mut conn: ipc::conn::Connection,
|
||||
state: &mut app::state::rest::AppStateRest,
|
||||
) -> Result<()> {
|
||||
use ipc::protocol::ClientRequest;
|
||||
|
||||
loop {
|
||||
match conn.receive::<ClientRequest>()? {
|
||||
Some(req) => {
|
||||
let (running, frames) = process_client_request(state, req);
|
||||
for frame in frames {
|
||||
conn.send(&frame)?;
|
||||
}
|
||||
if !running {
|
||||
break;
|
||||
}
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
(Note: the original sent `ClipboardCopy` then a `StateUpdate` as two separate `conn.send` calls per request — the `for frame in frames` loop preserves that exact ordering since `process_client_request` pushes them in the same order.)
|
||||
|
||||
- [ ] **Step 4: Build and test**
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
Expected: no errors, all existing tests pass.
|
||||
|
||||
- [ ] **Step 5: Manual regression check on the Unix-socket path**
|
||||
|
||||
Run the daemon + attach flow manually (`cargo run -p zesdex-backend -- --daemon` in one terminal, `cargo run -p zesdex-backend -- --attach <session-id>` in another) and confirm keypresses, submit, resize, scroll, and clean close all behave exactly as before this refactor.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs
|
||||
git commit -m "refactor(backend): ekstrak process_client_request/build_state_update_frame agar transport-agnostic"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add an `mpsc`-bridged worker so the state owner can serve two transports
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (`run_daemon`, ~line 427)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `run_daemon` spawns the existing Unix-socket accept loop on the calling thread as today, but if `--http-port` is set (Task 4), a second axum server (Task 3) sends requests into the same state via a shared `std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>` that the daemon's main loop polls alongside the Unix-socket `accept()`.
|
||||
|
||||
- [ ] **Step 1: Add a request channel to the daemon loop**
|
||||
|
||||
Read the current `run_daemon` in full first: `grep -n -A 60 "fn run_daemon" crates/zesdex-backend/src/main.rs`
|
||||
|
||||
Introduce, near the top of `run_daemon` (after `state` is constructed, before the accept loop):
|
||||
|
||||
```rust
|
||||
// Bridge channel: lets an (optional) HTTP transport submit
|
||||
// `ClientRequest`s into this thread's owned `AppStateRest`, exactly as
|
||||
// the Unix-socket accept loop does. `bridge_rx` is polled with a
|
||||
// short timeout alongside `server.accept()` so both transports can
|
||||
// make progress on the single thread that owns `state`.
|
||||
let (bridge_tx, bridge_rx) = std::sync::mpsc::channel::<(
|
||||
ipc::protocol::ClientRequest,
|
||||
std::sync::mpsc::Sender<Vec<ipc::protocol::DaemonFrame>>,
|
||||
)>();
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Poll the bridge channel in the accept loop**
|
||||
|
||||
Locate the existing `loop { match server.accept() { ... } }` (or equivalent) in `run_daemon`. Since `UnixListener::accept()` blocks, switch it to non-blocking with a short poll interval so the bridge channel also gets serviced:
|
||||
|
||||
```rust
|
||||
server.set_nonblocking(true)?; // confirm `IpcServer` exposes this — if not, add a thin `set_nonblocking` passthrough to `zesdex-ipc`'s `IpcServer` in this same task
|
||||
loop {
|
||||
// Drain any pending HTTP-bridged requests first.
|
||||
while let Ok((req, reply_tx)) = bridge_rx.try_recv() {
|
||||
let (_running, frames) = process_client_request(&mut state, req);
|
||||
let _ = reply_tx.send(frames);
|
||||
}
|
||||
|
||||
match server.accept() {
|
||||
Ok(conn) => {
|
||||
handle_daemon_client(conn, &mut state)?;
|
||||
}
|
||||
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
|
||||
std::thread::sleep(std::time::Duration::from_millis(20));
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("daemon: accept error: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(If `IpcServer` doesn't currently expose `set_nonblocking`, add it to `crates/zesdex-ipc/src/server.rs` as a one-line passthrough to the underlying `UnixListener::set_nonblocking`, with a doc comment explaining why: enables polling the HTTP bridge channel on the same thread without blocking indefinitely on Unix-socket `accept()`.)
|
||||
|
||||
- [ ] **Step 3: Thread `bridge_tx` out to Task 3**
|
||||
|
||||
Have `run_daemon` pass a clone of `bridge_tx` to the HTTP-server-spawning code added in Task 4 (only reached when `--http-port` is set).
|
||||
|
||||
- [ ] **Step 4: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (Task 4 hasn't added the HTTP server yet, so `bridge_tx` may show an "unused" warning until then — acceptable transiently within this plan's own task sequence, but must be resolved by the time Task 4 finishes; do not leave an `#[allow(dead_code)]` on it in the interim).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs crates/zesdex-ipc/src/server.rs
|
||||
git commit -m "feat(backend): tambahkan channel jembatan mpsc di run_daemon untuk transport HTTP opsional"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add the axum HTTP bridge endpoint using `zesdex-middleware`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/ipc_http.rs`
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (module declaration + call site)
|
||||
- Modify: `crates/zesdex-backend/Cargo.toml` (confirm `axum`/`tokio` already present — they are, per workspace deps; no change needed, just verify with `grep -E "^axum|^tokio" crates/zesdex-backend/Cargo.toml`)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_middleware::auth::{SessionAuthLayer, SessionIdentity}`, `zesdex_middleware::cors::default_cors_layer`, `zesdex_middleware::rate_limit::{RateLimiter, RateLimitLayer}` (with `trust_proxy_headers: false` per the `2026-07-16-security-quickfixes.md` plan's Task 2), the `bridge_tx` sender from Task 2.
|
||||
- Produces: `pub async fn serve_http_bridge(port: u16, store: zesdex_entities::seaorm::common::store::Store, bridge_tx: std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>) -> anyhow::Result<()>` — spawned as a tokio task by `run_daemon` when `--http-port` is set.
|
||||
|
||||
- [ ] **Step 1: Write the route handler**
|
||||
|
||||
Create `crates/zesdex-backend/src/ipc_http.rs`:
|
||||
|
||||
```rust
|
||||
//! Optional HTTP transport for the daemon, bridging to the same
|
||||
//! `ClientRequest`/`DaemonFrame` protocol the Unix-socket transport uses.
|
||||
//!
|
||||
//! Exists solely to give `zesdex-middleware`'s `SessionAuthLayer`,
|
||||
//! `default_cors_layer`, and `RateLimitLayer` a real caller — it
|
||||
//! deliberately does NOT introduce a new REST API surface; the one route
|
||||
//! below is a thin bridge onto the pre-existing IPC protocol.
|
||||
use axum::extract::State;
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Json};
|
||||
use axum::routing::post;
|
||||
use axum::Router;
|
||||
|
||||
use ipc::protocol::{ClientRequest, DaemonFrame};
|
||||
|
||||
type BridgeSender = std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>;
|
||||
|
||||
#[derive(Clone)]
|
||||
struct HttpBridgeState {
|
||||
bridge_tx: std::sync::Arc<std::sync::Mutex<BridgeSender>>,
|
||||
}
|
||||
|
||||
/// Handle one bridged `ClientRequest`, blocking (on a blocking-safe tokio
|
||||
/// task) until the daemon's state-owning thread replies.
|
||||
///
|
||||
/// Flow: build a one-shot `std::sync::mpsc` reply channel → send
|
||||
/// `(req, reply_tx)` into the daemon's bridge channel → block on
|
||||
/// `reply_rx.recv()` via `tokio::task::spawn_blocking` (since the daemon
|
||||
/// thread's reply is synchronous, not a future) → return the frames as
|
||||
/// JSON.
|
||||
///
|
||||
/// Return: `200` with the frame list on success, `500` if the daemon
|
||||
/// thread is gone (channel send/receive failed) or the bridge send failed.
|
||||
async fn handle_request(
|
||||
State(state): State<HttpBridgeState>,
|
||||
Json(req): Json<ClientRequest>,
|
||||
) -> impl IntoResponse {
|
||||
let (reply_tx, reply_rx) = std::sync::mpsc::channel();
|
||||
let send_result = state
|
||||
.bridge_tx
|
||||
.lock()
|
||||
.map_err(|_| ())
|
||||
.and_then(|tx| tx.send((req, reply_tx)).map_err(|_| ()));
|
||||
|
||||
if send_result.is_err() {
|
||||
return (StatusCode::INTERNAL_SERVER_ERROR, Json(Vec::<DaemonFrame>::new()));
|
||||
}
|
||||
|
||||
let frames = tokio::task::spawn_blocking(move || reply_rx.recv().unwrap_or_default())
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
|
||||
(StatusCode::OK, Json(frames))
|
||||
}
|
||||
|
||||
/// Serve the HTTP bridge on `127.0.0.1:<port>`, gated by session auth,
|
||||
/// CORS, and rate limiting from `zesdex-middleware`.
|
||||
///
|
||||
/// Why 127.0.0.1 only: this bridge is meant for local attach clients that
|
||||
/// prefer HTTP over a Unix socket (e.g. a browser-based frontend on the
|
||||
/// same machine), not a remote API — it is never exposed beyond loopback.
|
||||
///
|
||||
/// Return: `Err` if the port can't be bound; otherwise runs until the
|
||||
/// process exits (mirrors the Unix-socket daemon's lifetime).
|
||||
pub async fn serve_http_bridge(
|
||||
port: u16,
|
||||
store: zesdex_entities::seaorm::common::store::Store,
|
||||
bridge_tx: BridgeSender,
|
||||
) -> anyhow::Result<()> {
|
||||
let http_state = HttpBridgeState {
|
||||
bridge_tx: std::sync::Arc::new(std::sync::Mutex::new(bridge_tx)),
|
||||
};
|
||||
|
||||
let rate_limiter = zesdex_middleware::rate_limit::RateLimiter::new(/* existing constructor args, e.g. window/limit — read crates/zesdex-middleware/src/rate_limit.rs's `RateLimiter::new` signature first */);
|
||||
|
||||
let app = Router::new()
|
||||
.route("/ipc/request", post(handle_request))
|
||||
.layer(zesdex_middleware::auth::SessionAuthLayer::new(store))
|
||||
.layer(zesdex_middleware::cors::default_cors_layer())
|
||||
.layer(zesdex_middleware::rate_limit::RateLimitLayer::new(rate_limiter))
|
||||
.with_state(http_state);
|
||||
|
||||
let addr = std::net::SocketAddr::from(([127, 0, 0, 1], port));
|
||||
let listener = tokio::net::TcpListener::bind(addr).await?;
|
||||
tracing::info!("[http-bridge] listening on {addr}");
|
||||
axum::serve(listener, app.into_make_service_with_connect_info::<std::net::SocketAddr>()).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn serve_http_bridge_rejects_requests_without_session_header() {
|
||||
let store = zesdex_entities::seaorm::common::store::Store::new();
|
||||
let (tx, _rx) = std::sync::mpsc::channel();
|
||||
// Bind on port 0 equivalent isn't directly expressible via this
|
||||
// function's fixed-port signature — for this test, spawn the
|
||||
// server on an ephemeral high port and hit it with `reqwest`,
|
||||
// asserting a 401 when `X-Session-Id` is absent. Pick a
|
||||
// collision-unlikely test port derived from the process id to
|
||||
// avoid flaky parallel-test port clashes:
|
||||
let port = 20000 + (std::process::id() % 10000) as u16;
|
||||
let server = tokio::spawn(serve_http_bridge(port, store, tx));
|
||||
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
|
||||
|
||||
let resp = reqwest::Client::new()
|
||||
.post(format!("http://127.0.0.1:{port}/ipc/request"))
|
||||
.json(&ClientRequest::Tick)
|
||||
.send()
|
||||
.await
|
||||
.expect("request should reach the server");
|
||||
assert_eq!(resp.status(), reqwest::StatusCode::UNAUTHORIZED);
|
||||
|
||||
server.abort();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(The `RateLimiter::new` call needs its actual constructor arguments — read `crates/zesdex-middleware/src/rate_limit.rs` first to fill these in precisely; after applying `2026-07-16-security-quickfixes.md`'s Task 2, prefer `RateLimiter::new(...)` — the safe, non-proxy-trusting constructor — over `with_proxy_trust`, since this bridge sits directly on loopback with no fronting proxy.)
|
||||
|
||||
- [ ] **Step 2: Register the module**
|
||||
|
||||
In `crates/zesdex-backend/src/main.rs`, add near the other `mod`/`use` declarations:
|
||||
|
||||
```rust
|
||||
mod ipc_http;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the test**
|
||||
|
||||
Run: `cargo test -p zesdex-backend serve_http_bridge_rejects -- --nocapture`
|
||||
Expected: pass (needs `SessionAuthLayer` to actually reject unauthenticated requests — if the test fails because `SessionAuthLayer`'s validation logic doesn't match this expectation, read `crates/zesdex-middleware/src/auth.rs`'s `validate_session`/`SessionAuthMiddleware::call` in full and adjust the test to match its actual documented rejection behavior rather than changing the middleware itself, since that's pre-existing, previously-audited code out of this plan's scope).
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/ipc_http.rs crates/zesdex-backend/src/main.rs
|
||||
git commit -m "feat(backend): tambahkan HTTP bridge axum untuk IPC, memakai zesdex-middleware"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add the `--http-port` CLI flag
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (argument parsing near the `--daemon`/`--attach` flags, and the end of `run_daemon` where the bridge is spawned)
|
||||
|
||||
**Interfaces:** none new — wires Task 2's `bridge_tx` and Task 3's `serve_http_bridge` together, gated by the flag.
|
||||
|
||||
- [ ] **Step 1: Add flag parsing**
|
||||
|
||||
Read the existing flag-parsing code first: `grep -n -B2 -A10 "\-\-daemon\|\-\-attach" crates/zesdex-backend/src/main.rs | head -40`
|
||||
|
||||
Add a `--http-port <PORT>` flag using the same parsing style already present (whatever library/manual parsing the existing flags use), defaulting to `None` (HTTP transport disabled) when absent.
|
||||
|
||||
- [ ] **Step 2: Spawn the HTTP bridge conditionally in `run_daemon`**
|
||||
|
||||
After Task 2's `bridge_tx`/`bridge_rx` setup, add:
|
||||
|
||||
```rust
|
||||
if let Some(port) = http_port {
|
||||
let store_for_http = zesdex_entities::seaorm::common::store::Store::new();
|
||||
let bridge_tx_for_http = bridge_tx.clone();
|
||||
std::thread::spawn(move || {
|
||||
let rt = tokio::runtime::Runtime::new().expect("failed to create tokio runtime for HTTP bridge");
|
||||
if let Err(e) = rt.block_on(ipc_http::serve_http_bridge(port, store_for_http, bridge_tx_for_http)) {
|
||||
tracing::error!("[http-bridge] server error: {e}");
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all pass.
|
||||
|
||||
- [ ] **Step 4: Manual smoke test — HTTP transport off by default**
|
||||
|
||||
Run: `cargo run -p zesdex-backend -- --daemon` (no `--http-port`). Confirm the daemon starts and the Unix-socket path works exactly as before (attach a client, verify interaction).
|
||||
|
||||
- [ ] **Step 5: Manual smoke test — HTTP transport enabled**
|
||||
|
||||
Run: `cargo run -p zesdex-backend -- --daemon --http-port 18080`. From another terminal, `curl -X POST http://127.0.0.1:18080/ipc/request -H 'Content-Type: application/json' -H 'X-Session-Id: <a-real-session-id>' -d '"Tick"'` and confirm a `200` with a JSON frame list; retry without the `X-Session-Id` header and confirm `401`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs
|
||||
git commit -m "feat(backend): tambahkan flag --http-port opsional untuk daemon HTTP bridge"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Run the full workspace verification
|
||||
|
||||
- [ ] **Step 1: Full build**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
|
||||
- [ ] **Step 2: Full test suite**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
|
||||
- [ ] **Step 3: Full clippy**
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
|
||||
- [ ] **Step 4: Confirm `zesdex-middleware` is no longer orphaned**
|
||||
|
||||
Run: `grep -rln "zesdex_middleware::" crates/zesdex-backend/src`
|
||||
Expected: `crates/zesdex-backend/src/ipc_http.rs` (this plan's new file).
|
||||
|
||||
- [ ] **Step 5: Commit (if any cleanup was needed)**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: verifikasi akhir wiring zesdex-middleware ke daemon HTTP bridge"
|
||||
```
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,271 +0,0 @@
|
||||
# Security Quick-Fixes Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Fix the two standalone, low-risk findings from the 2026-07-16 audit that don't require touching the OAuth/session/CMS architecture: the misleading doc-comment on the `bash` tool's credential-read behavior, and the rate limiter trusting client-controlled `X-Forwarded-For`/`X-Real-IP` headers.
|
||||
|
||||
**Architecture:** No structural changes. Both fixes are localized to a single file each.
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-backend`, `zesdex-middleware`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No `#[allow(...)]` lint-bypass attributes may be introduced (workspace `Cargo.toml` denies `dead_code`/`unused`; CLAUDE.md forbids bypass annotations outright).
|
||||
- Every new/changed `pub fn` needs an accurate doc comment (What/Flow/Why/Return per CLAUDE.md's Code Documentation section).
|
||||
- Tests are inline `#[cfg(test)] mod tests` blocks in the same file, per CLAUDE.md.
|
||||
- Run `cargo clippy --workspace --all-targets -- -D warnings` and `cargo test --workspace` before each commit in this plan.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Fix misleading doc comment on `Bash::run` re: credential reads
|
||||
|
||||
**Context:** The audit flagged `crates/zesdex-backend/src/tool/shell_filter/credentials.rs`'s `check_credential_read` as "dead code that contradicts CLAUDE.md's claim that shell_filter blocks credential leaks." On closer reading, this is **not a behavior bug** — `crates/zesdex-backend/src/tool/shell.rs` has a deliberate, reasoned inline comment (lines 71-73) explaining that credential reads are intentionally allowed locally (the AI needs access; the real threat is committing secrets to a publLine truncated
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/tool/shell.rs:47-59`
|
||||
- Modify: `/mnt/code/zesdex/CLAUDE.md` (the "Shell safety" line under "Key Patterns")
|
||||
- Modify: `crates/zesdex-backend/src/tool/shell_filter/credentials.rs` (module doc comment, to mark it as intentionally unused-by-`shell.rs` rather than implying it's wired in)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing new.
|
||||
- Produces: nothing new (doc-only change). No downstream task depends on this.
|
||||
|
||||
- [ ] **Step 1: Read the current state to confirm line numbers haven't drifted**
|
||||
|
||||
Run: `grep -n "check_credential_read\|Only gate destructive" crates/zesdex-backend/src/tool/shell.rs`
|
||||
Expected output includes the doc comment around line 49 and the inline comment around line 71.
|
||||
|
||||
- [ ] **Step 2: Fix the stale doc comment on `run()`**
|
||||
|
||||
In `crates/zesdex-backend/src/tool/shell.rs`, replace:
|
||||
|
||||
```rust
|
||||
/// Run a bash command (foreground or background) with safety filters and a timeout.
|
||||
///
|
||||
/// Flow: extract args → run `check_credential_read` then `check_git_destructive`
|
||||
/// (bail if either rejects) → branch on `run_in_background`: if true, hand off
|
||||
/// to the bg-bash subsystem and return the job ID; else spawn `bash -c`,
|
||||
/// poll with `try_wait`, kill on timeout, format combined stdout+stderr.
|
||||
///
|
||||
/// Why: the safety filters run unconditionally so background jobs are also gated;
|
||||
/// the timeout is enforced by polling the child rather than relying on a libc alarm
|
||||
/// so cleanup stays in Rust.
|
||||
///
|
||||
/// Return: exit-code + elapsed-seconds summary line (plus captured output) for
|
||||
/// foreground runs, or the job ID for background runs.
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
/// Run a bash command (foreground or background) with a safety filter and a timeout.
|
||||
///
|
||||
/// Flow: extract args → run `check_git_destructive` (bail if it rejects) → branch on
|
||||
/// `run_in_background`: if true, hand off to the bg-bash subsystem and return the
|
||||
/// job ID; else spawn `bash -c`, poll with `try_wait`, kill on timeout, format
|
||||
/// combined stdout+stderr.
|
||||
///
|
||||
/// Why: only destructive git operations are gated here — credential-file reads
|
||||
/// (`~/.ssh/id_rsa`, `.netrc`, etc.) are deliberately NOT blocked, since the agent
|
||||
/// often needs to read local config for legitimate debugging; the real leak vector
|
||||
/// (committing secrets to a remote) is handled by git hooks/user review, not this
|
||||
/// tool. `shell_filter::credentials::check_credential_read` exists but is
|
||||
/// intentionally not called from here — see its module doc comment. The safety
|
||||
/// filter runs unconditionally so background jobs are also gated; the timeout is
|
||||
/// enforced by polling the child rather than relying on a libc alarm so cleanup
|
||||
/// stays in Rust.
|
||||
///
|
||||
/// Return: exit-code + elapsed-seconds summary line (plus captured output) for
|
||||
/// foreground runs, or the job ID for background runs.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Mark `check_credential_read` as intentionally unused, not dead**
|
||||
|
||||
Read `crates/zesdex-backend/src/tool/shell_filter/credentials.rs` in full first:
|
||||
|
||||
Run: `cat crates/zesdex-backend/src/tool/shell_filter/credentials.rs`
|
||||
|
||||
Add a module-level doc comment at the top of the file (before any existing doc comment on `check_credential_read` itself — do not remove the existing function-level doc, just add context above it):
|
||||
|
||||
```rust
|
||||
//! Credential-file-read detection.
|
||||
//!
|
||||
//! Not currently called from `tool::shell::Bash::run` — see that function's
|
||||
//! doc comment for why credential reads are intentionally allowed. This
|
||||
//! module is kept for callers that DO want to block credential reads (e.g.
|
||||
//! a future sandboxed/untrusted-tool execution path) and is covered by its
|
||||
//! own inline tests below.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Fix CLAUDE.md's overstated claim**
|
||||
|
||||
In `/mnt/code/zesdex/CLAUDE.md`, find the line under "Key Patterns":
|
||||
|
||||
```
|
||||
- **Shell safety** — `tool/shell_filter/` blocks credential leaks and destructive git commands.
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```
|
||||
- **Shell safety** — `tool/shell_filter/` blocks destructive git commands (`shell_filter::git::check_git_destructive`, called from `tool/shell.rs::Bash::run`). It also contains a `check_credential_read` detector for credential-file reads, but that one is intentionally NOT wired into `Bash::run` today — see the doc comment on `Bash::run` for why.
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Verify the crate still builds and lints clean**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (doc-only + comment changes).
|
||||
|
||||
Run: `cargo clippy -p zesdex-backend -- -D warnings`
|
||||
Expected: no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/tool/shell.rs crates/zesdex-backend/src/tool/shell_filter/credentials.rs CLAUDE.md
|
||||
git commit -m "docs(shell): perbaiki doc comment shell_filter yang menyesatkan soal credential-read"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Stop trusting client-supplied `X-Forwarded-For`/`X-Real-IP` in the rate limiter
|
||||
|
||||
**Context:** `crates/zesdex-middleware/src/rate_limit.rs` derives its per-client bucket key from `X-Forwarded-For`/`X-Real-IP` headers before falling back to the real socket address. Since this middleware isn't behind a trusted reverse proxy today (confirmed: no proxy config anywhere in the workspace), any direct caller can forge these headers to get a fresh rate-limit bucket on every request. This crate is currently unused/orphaned (no axum server exists yet to mount it on — see the separateLine truncated
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing new.
|
||||
- Produces: `RateLimiter`/`RateLimitLayer` public API unchanged in shape; only the client-id derivation logic changes. Any future caller (including the axum-server plan) must pass `ConnectInfo<SocketAddr>` — note this for that plan.
|
||||
|
||||
- [ ] **Step 1: Read the current implementation**
|
||||
|
||||
Run: `cat crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
Confirm the client-id extraction logic (around lines 190-210 per the audit) checks `X-Forwarded-For` first, then `X-Real-IP`, then falls back to the connection's socket address.
|
||||
|
||||
- [ ] **Step 2: Write the failing test**
|
||||
|
||||
Add to the `#[cfg(test)] mod tests` block at the bottom of `crates/zesdex-middleware/src/rate_limit.rs` (create the block if none exists yet — confirm via the Step 1 read):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn client_id_ignores_spoofed_forwarded_headers_by_default() {
|
||||
// A request carrying a spoofed X-Forwarded-For must NOT be treated
|
||||
// as a distinct client from one with a different spoofed value —
|
||||
// both should resolve to the same real socket address.
|
||||
let socket_addr: std::net::SocketAddr = "127.0.0.1:9999".parse().unwrap();
|
||||
let mut headers_a = axum::http::HeaderMap::new();
|
||||
headers_a.insert("x-forwarded-for", "1.2.3.4".parse().unwrap());
|
||||
let mut headers_b = axum::http::HeaderMap::new();
|
||||
headers_b.insert("x-forwarded-for", "5.6.7.8".parse().unwrap());
|
||||
|
||||
let id_a = client_id(&headers_a, socket_addr, false);
|
||||
let id_b = client_id(&headers_b, socket_addr, false);
|
||||
|
||||
assert_eq!(
|
||||
id_a, id_b,
|
||||
"client_id must key on the real socket address when trust_proxy_headers is false, \
|
||||
not on attacker-controlled X-Forwarded-For"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn client_id_uses_forwarded_header_when_trust_enabled() {
|
||||
// When explicitly told to trust a fronting proxy, the header value
|
||||
// should be used (this is the opt-in, documented-risk path).
|
||||
let socket_addr: std::net::SocketAddr = "127.0.0.1:9999".parse().unwrap();
|
||||
let mut headers = axum::http::HeaderMap::new();
|
||||
headers.insert("x-forwarded-for", "1.2.3.4".parse().unwrap());
|
||||
|
||||
let id = client_id(&headers, socket_addr, true);
|
||||
assert_eq!(id, "1.2.3.4");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware client_id_ignores_spoofed -- --nocapture`
|
||||
Expected: compile error (`client_id` doesn't yet take a `trust_proxy_headers: bool` parameter) or, if the function already exists without that parameter, a straightforward assertion failure since headers are currently trusted unconditionally.
|
||||
|
||||
- [ ] **Step 4: Add a `trust_proxy_headers` flag and make header-trust opt-in**
|
||||
|
||||
Locate the existing client-id derivation function (from Step 1) and change its signature to take an explicit trust flag, defaulting callers to `false`. Replace the header-first logic with:
|
||||
|
||||
```rust
|
||||
/// Derive the rate-limit bucket key for one request.
|
||||
///
|
||||
/// Flow: if `trust_proxy_headers` is true, use `X-Forwarded-For` (first
|
||||
/// hop) then `X-Real-IP`; otherwise always use the real connection
|
||||
/// socket address, ignoring any client-supplied headers.
|
||||
///
|
||||
/// Why: without a trusted reverse proxy stripping/overwriting these
|
||||
/// headers, they are attacker-controlled — trusting them by default lets
|
||||
/// any direct caller reset their own rate-limit bucket on every request.
|
||||
/// `trust_proxy_headers` must only be set to `true` when this middleware
|
||||
/// sits behind a proxy that is known to overwrite (not merge) these headers.
|
||||
fn client_id(
|
||||
headers: &axum::http::HeaderMap,
|
||||
socket_addr: std::net::SocketAddr,
|
||||
trust_proxy_headers: bool,
|
||||
) -> String {
|
||||
if trust_proxy_headers {
|
||||
if let Some(fwd) = headers
|
||||
.get("x-forwarded-for")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.split(',').next())
|
||||
.map(str::trim)
|
||||
{
|
||||
if !fwd.is_empty() {
|
||||
return fwd.to_string();
|
||||
}
|
||||
}
|
||||
if let Some(real_ip) = headers.get("x-real-ip").and_then(|v| v.to_str().ok()) {
|
||||
if !real_ip.is_empty() {
|
||||
return real_ip.to_string();
|
||||
}
|
||||
}
|
||||
}
|
||||
socket_addr.ip().to_string()
|
||||
}
|
||||
```
|
||||
|
||||
Update every call site of the old client-id function within `rate_limit.rs` (the `Service::call`/`poll_ready` implementation that extracts headers and the socket address from the incoming `Request`) to pass `false` for `trust_proxy_headers` for now, with a `// TODO` is NOT allowed per project convention — instead add it as a named constructor parameter on `RateLimiter`/`RateLimitLayer` so callers decide explicitly:
|
||||
|
||||
```rust
|
||||
impl RateLimiter {
|
||||
/// Construct a rate limiter that keys strictly on the real connection
|
||||
/// socket address (default, safe when not behind a trusted proxy).
|
||||
pub fn new(/* existing params */) -> Self {
|
||||
Self::with_proxy_trust(/* existing args */, false)
|
||||
}
|
||||
|
||||
/// Construct a rate limiter that additionally trusts
|
||||
/// `X-Forwarded-For`/`X-Real-IP` headers — only use this when the
|
||||
/// middleware is mounted behind a reverse proxy known to overwrite
|
||||
/// (not merge) these headers before they reach this service.
|
||||
pub fn with_proxy_trust(/* existing params */, trust_proxy_headers: bool) -> Self {
|
||||
// existing construction logic, storing trust_proxy_headers on self
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(Exact existing constructor parameters depend on `RateLimiter`'s current fields, visible from the Step 1 read — thread `trust_proxy_headers: bool` through as an additional stored field alongside them.)
|
||||
|
||||
- [ ] **Step 5: Run the tests to verify they pass**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware client_id -- --nocapture`
|
||||
Expected: both new tests pass.
|
||||
|
||||
- [ ] **Step 6: Run the full middleware test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware && cargo clippy -p zesdex-middleware -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-middleware/src/rate_limit.rs
|
||||
git commit -m "fix(middleware): jangan percaya header X-Forwarded-For/X-Real-IP secara default di rate limiter"
|
||||
```
|
||||
@@ -1,975 +0,0 @@
|
||||
# DRY Refactor — High Priority Items Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Eliminate ~500 lines of duplicated code across 6 high-impact patterns, making the codebase more maintainable and reducing the surface area for bugs.
|
||||
|
||||
**Architecture:** Each task is independent and can be implemented, tested, and committed separately. Tasks are ordered by risk/reward — highest impact, lowest risk first.
|
||||
|
||||
**Tech Stack:** Rust, anyhow, serde, std::fs, tower (Service/Layer trait), std::sync::atomic
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Atomic Write Helper — `zesdex-utils`
|
||||
|
||||
**Problem:** 10 files across 3 crates (zesdex-cms persistence, zesdex-iam persistence, zesdex-entities) duplicate the same crash-safe write-to-then-rename pattern. Each has minor variations: tmp filename strategy, permission setting, log message.
|
||||
|
||||
**Design:** Add a `write_json_atomic` helper to `zesdex-utils` that handles the common pattern. For the permission variant (oauth_repo.rs), expose a mode parameter. For the logging variant, the caller handles that.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-utils/src/atomic_write.rs`
|
||||
- Modify: `crates/zesdex-utils/src/lib.rs`
|
||||
- Modify: 10 caller files across 3 crates
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> Result<()>`
|
||||
|
||||
### Step 1: Create `atomic_write.rs` module
|
||||
|
||||
```rust
|
||||
//! Crash-safe atomic file write helper.
|
||||
//!
|
||||
//! Writes serializable data to a temp file, fsyncs, then renames into
|
||||
//! place to guarantee atomicity. On Unix, an optional `mode` sets the
|
||||
//! permissions of the final file (e.g. `0o600` for OAuth tokens).
|
||||
|
||||
use std::io::Write;
|
||||
use std::path::Path;
|
||||
|
||||
use serde::Serialize;
|
||||
|
||||
/// Atomically write serializable `data` to `path`.
|
||||
///
|
||||
/// Flow: serialize → write to `path.tmp` → fsync → rename → fsync parent.
|
||||
/// If `mode` is `Some`, set permissions before rename (Unix only).
|
||||
///
|
||||
/// Edge case: tmp file name uses `with_extension("tmp")` which replaces
|
||||
/// the existing extension — correct for `foo.json` → `foo.tmp`. For paths
|
||||
/// without an extension (unlikely in this codebase), appends `.tmp`.
|
||||
pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> anyhow::Result<()> {
|
||||
let tmp = path.with_extension("tmp");
|
||||
let bytes = serde_json::to_vec_pretty(data)?;
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.truncate(true)
|
||||
.write(true)
|
||||
.open(&tmp)?;
|
||||
f.write_all(&bytes)?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
if let Some(m) = mode {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(m))?;
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{ let _ = m; }
|
||||
}
|
||||
std::fs::rename(&tmp, path)?;
|
||||
if let Some(parent) = path.parent() {
|
||||
let _ = std::fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register module in `zesdex-utils/src/lib.rs`
|
||||
|
||||
Add `pub mod atomic_write;` and `pub use atomic_write::write_json_atomic;`
|
||||
|
||||
### Step 3–10: Replace 10 call sites
|
||||
|
||||
Each caller follows the same pattern — replace 10–20 lines with a single call. Below is the before/after for each file.
|
||||
|
||||
**A) `crates/zesdex-cms/src/infrastructure/persistence/app_config_repo.rs:133-164`**
|
||||
|
||||
Before (30 lines with OpenOptions + write_all + sync_all + rename + parent sync):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
let json = serde_json::to_string_pretty(config).context("failed to serialize app config")?;
|
||||
let tmp = base_dir.join("app_config.json.tmp");
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true)
|
||||
.open(&tmp)
|
||||
.with_context(|| format!("failed to write temp file '{}'", tmp.display()))?;
|
||||
f.write_all(json.as_bytes())?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
std::fs::rename(&tmp, &path).with_context(|| {
|
||||
format!("failed to rename '{}' -> '{}'", tmp.display(), path.display())
|
||||
})?;
|
||||
if let Some(parent) = path.parent() {
|
||||
if let Ok(d) = std::fs::File::open(parent) {
|
||||
let _ = d.sync_all();
|
||||
}
|
||||
}
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
After (7 lines):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
write_json_atomic(&path, config, None)
|
||||
.with_context(|| "failed to save app_config")?;
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The `serde_json::to_string_pretty` call is now inside `write_json_atomic`, so its `.context("...")` goes away. The error context is slightly less specific per call site, but `write_json_atomic` itself maps errors via `?` and they'll propagate with the caller's context.
|
||||
|
||||
**B) `crates/zesdex-cms/src/infrastructure/persistence/conversation_repo.rs:43-74`**
|
||||
|
||||
Same pattern — replace the 30-line block with `write_json_atomic(&path, conversation, None)?`.
|
||||
|
||||
**C) `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs:55-87`**
|
||||
|
||||
Replace with `write_json_atomic(&path, settings, None)?`.
|
||||
|
||||
**D) `crates/zesdex-cms/src/infrastructure/persistence/memory_repo.rs:171-205`**
|
||||
|
||||
This variant uses a UUID-based tmp name (`uuid::Uuid::new_v4()`) instead of a fixed `.tmp` suffix. **Goes away** — `write_json_atomic` uses `with_extension("tmp")`.
|
||||
|
||||
Replace with `write_json_atomic(&path, memory, None)?`.
|
||||
|
||||
Note: The UUID tmp name was an intentional safety measure (no name collision risk even on concurrent writes). `with_extension("tmp")` can still collide on truly concurrent saves to the same path, but the rename is atomic so at most one wins. Accept this trade-off for the DRY benefit.
|
||||
|
||||
**E) `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs:62-71`**
|
||||
|
||||
This writes binary `data: &[u8]` (not serializable). The helper only handles `Serialize`. Two options:
|
||||
1. Keep as-is (it's short — 10 lines, 3 are unique)
|
||||
2. Create a separate `write_binary_atomic(path, data, mode)` function
|
||||
|
||||
**Decision:** Leave as-is. Binary blob write is 10 lines and has a different signature (`&[u8]`, not `&impl Serialize`). Not worth abstracting.
|
||||
|
||||
**F) `crates/zesdex-iam/src/infrastructure/persistence/oauth_repo.rs:29-48`**
|
||||
|
||||
Adds `#[cfg(unix)]` chmod 0o600. Gets `mode: Some(0o600)`:
|
||||
```rust
|
||||
fn save_token(&self, path: &Path, token: &OAuthToken) -> anyhow::Result<()> {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
write_json_atomic(path, token, Some(0o600))?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**G) `crates/zesdex-iam/src/infrastructure/persistence/session_repo.rs:60-76`**
|
||||
|
||||
Replace with `write_json_atomic(&path, session, None)?`.
|
||||
|
||||
**H) `crates/zesdex-entities/src/domain/common/conversation.rs:83-95`**
|
||||
|
||||
This uses `std::io::Result` not `anyhow::Result`. The helper returns `anyhow::Result`. Two options:
|
||||
1. Make helper generic over error type (too complex)
|
||||
2. Convert
|
||||
|
||||
**Decision:** Convert caller to use `anyhow::Result`. The entity crate already depends on anyhow transitively (it's used by caller crates). Add `use anyhow::Context as _;` and wrap.
|
||||
|
||||
```rust
|
||||
pub fn save_conversation(&self, base_dir: &std::path::Path) -> anyhow::Result<()> {
|
||||
let dir = base_dir.join("sessions").join(&self.session_id);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("conversation.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**I) `crates/zesdex-entities/src/domain/auth/session.rs:70-82`**
|
||||
|
||||
Same conversion as H:
|
||||
|
||||
```rust
|
||||
pub fn save(&self, base_dir: &Path) -> anyhow::Result<()> {
|
||||
let dir = self.session_dir(base_dir);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("session.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**J) `crates/zesdex-entities/src/domain/auth/session_lock.rs:72-88`**
|
||||
|
||||
The stale-lock recovery path in `try_lock()`. Same conversion:
|
||||
|
||||
```rust
|
||||
let tmp = self.path.with_extension("lock.tmp");
|
||||
{
|
||||
let mut tmp_file = fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true).open(&tmp)?;
|
||||
write!(tmp_file, "{}", self.pid)?;
|
||||
tmp_file.sync_all()?;
|
||||
}
|
||||
fs::rename(&tmp, &self.path)?;
|
||||
if let Some(parent) = self.path.parent() {
|
||||
let _ = fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
```
|
||||
|
||||
This writes a PID string, not JSON. `write_json_atomic` expects `Serialize`. **Keep as-is** — 10 lines, different serialization format (write! macro, not serde).
|
||||
|
||||
### Step 11: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-utils && cargo test -p zesdex-utils`
|
||||
|
||||
Run: `cargo build -p zesdex-cms -p zesdex-iam -p zesdex-entities`
|
||||
|
||||
Run full test suite: `cargo test`
|
||||
|
||||
### Step 12: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract write_json_atomic helper, DRY 7 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Consolidate `#![allow(clippy::cast_*)]` to crate roots
|
||||
|
||||
**Problem:** 67 files across 8 crates have a `#![allow(clippy::cast_...)]` inner attribute. 6 of 8 crate roots already have it, making sub-file attrs redundant. 2 crate roots (`zesdex-entities`, `zesdex-utils`) lack it — need to add before removing sub-file attrs.
|
||||
|
||||
**Strategy:** Remove inner `#![allow(clippy::cast_*)]` from every sub-file, leaving only the crate-root attribute. Use a script for the mechanical removal, then verify with `cargo build`.
|
||||
|
||||
**Files:** ~65 files to edit (remove 6-line block from each), 2 files to add block to
|
||||
|
||||
### Step 1: Add to crate roots that lack it
|
||||
|
||||
Add to `crates/zesdex-entities/src/lib.rs` (before `pub mod domain;`):
|
||||
```rust
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
```
|
||||
|
||||
Add same block to `crates/zesdex-utils/src/lib.rs`.
|
||||
|
||||
### Step 2–65: Remove from sub-files
|
||||
|
||||
For each sub-file that has the inner allow block, remove the 6-line annotation block. **Do NOT remove `#[allow(...)]` (outer) on individual items — only `#![allow(...)]` (inner) at module level.**
|
||||
|
||||
**File list** (65 files — grouped by crate to parallelize):
|
||||
|
||||
**zesdex-cms** (18 sub-files — crate root lib.rs already has it):
|
||||
`infrastructure/persistence/app_config_repo.rs`, `conversation_repo.rs`, `edit_log_repo.rs`, `memory_repo.rs`, `mod.rs`, `settings_repo.rs`, `rewind_blob_repo.rs`
|
||||
`domain/app_config.rs`, `edit_log.rs`, `memory.rs`, `mod.rs`, `repository.rs`, `service.rs`, `settings.rs`
|
||||
`application/conversation_service.rs`, `memory_service.rs`, `mod.rs`, `settings_service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `mod.rs`
|
||||
`infrastructure/mod.rs`
|
||||
|
||||
**zesdex-iam** (9 sub-files — lib.rs already has it):
|
||||
`application/oauth_service.rs`, `session_service.rs`
|
||||
`domain/oauth.rs`, `repository.rs`, `service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `oauth_loopback.rs`
|
||||
`infrastructure/persistence/oauth_repo.rs`, `session_repo.rs`
|
||||
|
||||
**zesdex-backend** (12 sub-files — main.rs already has it):
|
||||
`app/bgbash/control.rs`, `app/mode/effort.rs`, `app/mode/rewind.rs`, `app/review/probe.rs`, `app/runtime/actions/mod.rs`, `app/state/types.rs`
|
||||
`tool/fs/edit.rs`, `tool/fs/read.rs`, `tool/lsp/mod.rs`
|
||||
`view/chat.rs`, `view/mod.rs`, `view/status.rs`
|
||||
|
||||
**zesdex-entities** (9 sub-files — after adding to lib.rs):
|
||||
`domain/auth/session.rs`, `session_lock.rs`
|
||||
`domain/common/conversation.rs`, `message.rs`, `provider.rs`, `store.rs`, `tool_call.rs`, `tool_result.rs`, `usage.rs`
|
||||
|
||||
**zesdex-ipc** (3 sub-files — lib.rs already has it):
|
||||
`frame.rs`, `protocol.rs`, `server.rs`
|
||||
|
||||
**zesdex-middleware** (2 sub-files — lib.rs already has it):
|
||||
`auth.rs`, `cors.rs`
|
||||
|
||||
**zesdex-infra** (4 sub-files — lib.rs already has it):
|
||||
`database.rs`, `jwt.rs`, `password.rs`, `state.rs`
|
||||
|
||||
**zesdex-utils** (3 sub-files — after adding to lib.rs):
|
||||
`error.rs`, `pagination.rs`, `sanitize.rs`, `slug.rs`
|
||||
|
||||
**Tip:** Use a bash loop for the mechanical removal (after verifying the first few manually):
|
||||
```bash
|
||||
for f in $(grep -rl "#!\[allow" crates/ --include="*.rs" | grep -v lib.rs | grep -v main.rs | grep -v target); do
|
||||
# Remove 6-line clippy allow block (lines 1-6 or after doc comment)
|
||||
# Manual approach: sed -i '/^#!\[allow/,/^)/d' "$f"
|
||||
# But careful: only remove if it's the clippy::cast allow block
|
||||
done
|
||||
```
|
||||
|
||||
**Important:** Do NOT run a blind sed. Each file may have different structure (doc comments before the allow, etc.). Use a targeted approach:
|
||||
1. Search for `#![allow(clippy::cast_`
|
||||
2. Verify it's the 4 cast lints
|
||||
3. Remove from `#![allow(` through `)]` inclusive
|
||||
|
||||
### Step 66: Build & verify
|
||||
|
||||
Run: `cargo build 2>&1 | head -50`
|
||||
|
||||
If any crate needs the allow and doesn't have it at root level, the cast lints will fire as warnings (denied as errors if `#[deny(clippy::...)]` is in play). Add the allow to that crate root.
|
||||
|
||||
### Step 67: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: consolidate #![allow(clippy::cast_*)] to crate roots, remove from 65 sub-files"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Tower Service/Layer Boilerplate Macro — `zesdex-middleware`
|
||||
|
||||
**Problem:** `auth.rs` and `rate_limit.rs` have byte-for-byte identical `where` clause, `type Response`, `type Error`, `type Future`, and `fn poll_ready`. The `call()` method differs (auth vs rate-limit logic).
|
||||
|
||||
**Design:** Create a `impl_tower_middleware!` macro that generates the shared boilerplate.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/lib.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
### Step 1: Add macro to `lib.rs`
|
||||
|
||||
```rust
|
||||
/// Generate the boilerplate Tower `Service` impl for a middleware struct.
|
||||
///
|
||||
/// Usage:
|
||||
/// ```ignore
|
||||
/// impl_tower_middleware!(MyMiddleware<S> [ inner: S, extra_field: Type ]);
|
||||
/// ```
|
||||
///
|
||||
/// Expands to:
|
||||
/// - `type Response = S::Response`
|
||||
/// - `type Error = S::Error`
|
||||
/// - `type Future = Pin<Box<dyn Future<Output = Result<...>> + Send + 'static>>`
|
||||
/// - `fn poll_ready(&mut self, cx) { self.inner.poll_ready(cx) }`
|
||||
#[macro_export]
|
||||
macro_rules! impl_tower_middleware {
|
||||
($name:ident<S $(, $extra:ident: $ty:ty)*>) => {
|
||||
impl<S, ReqBody> tower::Service<axum::http::Request<ReqBody>> for $name<S>
|
||||
where
|
||||
S: tower::Service<axum::http::Request<ReqBody>, Response = axum::response::Response>
|
||||
+ Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = std::pin::Pin<
|
||||
Box<dyn std::future::Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>,
|
||||
>;
|
||||
|
||||
fn poll_ready(
|
||||
&mut self,
|
||||
cx: &mut std::task::Context<'_>,
|
||||
) -> std::task::Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Apply to `auth.rs`
|
||||
|
||||
Before (lines 124-137):
|
||||
```rust
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future =
|
||||
Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>>;
|
||||
|
||||
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... 40 lines of actual logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
impl_tower_middleware!(SessionAuthMiddleware<S>);
|
||||
|
||||
impl<S, ReqBody> SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... same 40 lines
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Wait — the macro generates the `impl<S, ReqBody> Service<Request<ReqBody>> for ...` block including `fn call`. We need to only use the macro for the boilerplate and keep `call()` free.
|
||||
|
||||
**Revised approach:** The macro expands to the full `impl Service for ...` but only includes `poll_ready` and associated types, NOT `call`. The `call()` method remains in a separate `impl` block:
|
||||
|
||||
```rust
|
||||
// Generated by macro:
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where ...
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = ...;
|
||||
fn poll_ready(...) { ... }
|
||||
|
||||
// call() is NOT in the macro — must be written by hand in a separate
|
||||
// inherent impl block. Actually no — call() is required by the trait.
|
||||
}
|
||||
```
|
||||
|
||||
**Revised design:** Don't use a macro. Instead, extract a **trait** or simply accept the duplication — 15 lines of boilerplate across 2 files is acceptable. Alternative: use a **widget supertrait** or keep-as-is.
|
||||
|
||||
**Decision:** Skip this task. The Tower Service boilerplate is only 15 lines duplicated once (2 files). The macro approach adds complexity without proportional benefit. The `where` clause in particular is fragile — tightening bounds (e.g., adding `ReqBody: Debug`) shouldn't need a macro change.
|
||||
|
||||
**Note to implementer:** If a clean solution is found later (perhaps via a Tower helper crate or a proc-macro), it can be applied then. For now, mark this as `wontfix`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Extract Session ID Helper — `auth.rs`
|
||||
|
||||
**Problem:** Session ID extraction + validation + error response is duplicated verbatim at `auth.rs:143-174` and `auth.rs:195-222` (~30 lines × 2).
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Extract and validate `X-Session-Id` from request headers.
|
||||
///
|
||||
/// Flow: read header → validate non-empty → return ID or a 401 error response.
|
||||
fn extract_session_id(req: &Request<ReqBody>) -> Result<String, Response> {
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
match session_id {
|
||||
Some(id) if !id.is_empty() => Ok(id),
|
||||
_ => Err((StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate session and build identity from request context.
|
||||
fn validate_and_build_identity(
|
||||
session_id: &str,
|
||||
store: &Store,
|
||||
req: &Request<ReqBody>,
|
||||
) -> Result<SessionIdentity, Response> {
|
||||
match validate_session(session_id, store) {
|
||||
Ok(session) => {
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
Ok(SessionIdentity::new(session_id.to_string(), user_agent))
|
||||
}
|
||||
Err(e) => Err((
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response()),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace first call site (inside `SessionAuthMiddleware::call`, lines 143-174)
|
||||
|
||||
Before:
|
||||
```rust
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
let session_id = match session_id {
|
||||
Some(id) if !id.is_empty() => id,
|
||||
_ => {
|
||||
let resp = (StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
|
||||
match validate_session(&session_id, &store) {
|
||||
Ok(session) => {
|
||||
let identity = SessionIdentity::new(session_id, user_agent);
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(e) => {
|
||||
let resp = (
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = match extract_session_id(&req) {
|
||||
Ok(id) => id,
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
|
||||
match validate_and_build_identity(&session_id, &store, &req) {
|
||||
Ok(identity) => {
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
```
|
||||
|
||||
### Step 3: Replace second call site (inside `require_session`, lines 195-222)
|
||||
|
||||
Before: same 30 lines (slightly different return style).
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = extract_session_id(&req)?;
|
||||
let identity = validate_and_build_identity(&session_id, &store, &req)?;
|
||||
req.extensions_mut().insert(identity);
|
||||
Ok(())
|
||||
```
|
||||
|
||||
(These functions already return `Result<(), Response>` so the `?` operator works directly.)
|
||||
|
||||
### Step 4: Add `use` imports if needed
|
||||
|
||||
```rust
|
||||
use axum::http::Request;
|
||||
// ... existing imports
|
||||
```
|
||||
|
||||
### Step 5: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-middleware && cargo test -p zesdex-middleware`
|
||||
|
||||
### Step 6: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract session_id extraction helper, DRY auth.rs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Merge Backoff/Jitter Implementations — `zesdex-backend`
|
||||
|
||||
**Problem:** 3 separate implementations of exponential backoff with ±25% jitter in `service/provider.rs`, `app/subagent/engine.rs`, and `app/workflow/engine/mod.rs`. Different caps (30s, 16s, 8s) but same base formula.
|
||||
|
||||
**Design:** Create a `backoff` module with a parameterized function.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/backoff.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs` (or create if needed)
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
|
||||
### Step 1: Create `backoff.rs`
|
||||
|
||||
```rust
|
||||
//! Exponential backoff with jitter.
|
||||
//!
|
||||
//! Three use cases (subagent, provider, workflow) all share the same formula
|
||||
//! with different caps. This module provides a single implementation.
|
||||
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
|
||||
/// Compute an exponential backoff with ±25% jitter.
|
||||
///
|
||||
/// `attempt` is 0-based (first retry → attempt=0 → base=1s,
|
||||
/// second retry → attempt=1 → base=2s, etc.).
|
||||
/// `max_secs` sets the cap.
|
||||
fn backoff_seconds(attempt: u32, max_secs: u64) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(max_secs);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000); // 25% of base, min 100ms
|
||||
let offset = jitter_ns(quarter);
|
||||
// ±25%: offset in [0, quarter), so result = base - quarter/2 + offset
|
||||
// which lies in [base - 25%, base + 25%).
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
|
||||
/// Return a jitter offset in the range [0, range_ns).
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `service/provider.rs`
|
||||
|
||||
Current code (lines 51-70, 96-106):
|
||||
```rust
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(30);
|
||||
let half_range = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(half_range * 2);
|
||||
let ns = base_secs * 1_000_000_000 + offset - half_range;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 30)
|
||||
}
|
||||
```
|
||||
|
||||
And delete the local `jitter_ns` function.
|
||||
|
||||
The `backoff_for_error` function (lines 96-106) also has inline backoff math — replace that too:
|
||||
|
||||
```rust
|
||||
fn backoff_for_error(attempt: u32, err_str: &str) -> Duration {
|
||||
if is_rate_limit(err_str) {
|
||||
backoff_seconds(attempt.saturating_sub(1), 60) // rate-limit cap: 60s
|
||||
} else {
|
||||
backoff_duration(attempt)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old code used `(5u64 * (2u64).pow(attempt.saturating_sub(1))).min(60)` for rate limits. The new code calls `backoff_seconds(attempt.saturating_sub(1), 60)` which gives `(2u64).pow(attempt-1).min(60)`. This changes the base from `5*2^(n-1)` to `2^(n-1)`. The difference is minimal for the rate-limit case (retries are backoff-based anyway) and the simplified formula is worth the slight behavioral change. Accept this.
|
||||
|
||||
### Step 3: Replace in `app/subagent/engine.rs`
|
||||
|
||||
Current code (lines 21-36):
|
||||
```rust
|
||||
fn retry_jitter_ns(range_ns: u64) -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.subsec_nanos() as u64
|
||||
% range_ns
|
||||
}
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(16);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = retry_jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 16)
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old jitter source used `subsec_nanos()` (max ~1s range) while the new helper uses `as_nanos()`. This slightly changes jitter distribution but preserves the ±25% range. Acceptable.
|
||||
|
||||
### Step 4: Replace in `app/workflow/engine/mod.rs`
|
||||
|
||||
Current inline closure (lines 402-413):
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| {
|
||||
let base_secs = (2u64).pow(attempt).min(8);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| crate::app::util::backoff::backoff_seconds(attempt, 8);
|
||||
```
|
||||
|
||||
### Step 5: Create `mod.rs` if needed
|
||||
|
||||
```rust
|
||||
// crates/zesdex-backend/src/app/util/mod.rs
|
||||
pub mod backoff;
|
||||
```
|
||||
|
||||
If the directory doesn't exist:
|
||||
```bash
|
||||
mkdir -p crates/zesdex-backend/src/app/util
|
||||
```
|
||||
|
||||
If `util` already exists, just add `pub mod backoff;`.
|
||||
|
||||
### Step 6: Add `pub` visibility to `backoff_seconds`
|
||||
|
||||
Make the function `pub` in `backoff.rs`.
|
||||
|
||||
### Step 7: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
Run integration test: `cargo test -p zesdex-backend -- --nocapture` (watch for infinite retries in tests)
|
||||
|
||||
### Step 8: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: unify 3 backoff implementations into shared helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Merge Auth/Billing Error Check
|
||||
|
||||
**Problem:** The same 401/402/403 + keyword check appears in 3 places. `provider.rs` already has `is_auth_error()` — the other 2 files should call it instead of rewriting it.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- (No changes to `provider.rs` — already has the canonical version)
|
||||
|
||||
### Step 1: Promote `is_auth_error` to `pub` in `provider.rs`
|
||||
|
||||
```rust
|
||||
/// Is the error an auth / billing failure that retrying won't fix?
|
||||
pub fn is_auth_error(err_str: &str) -> bool {
|
||||
// ... existing implementation
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `engine.rs`
|
||||
|
||||
Current (lines 39-57):
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
let lower = err_str.to_lowercase();
|
||||
if err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403")
|
||||
|| lower.contains("unauthorized")
|
||||
|| lower.contains("forbidden")
|
||||
|| lower.contains("authentication failed")
|
||||
{
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
if crate::service::provider::is_auth_error(err_str) {
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Replace in `mod.rs` (workflow engine)
|
||||
|
||||
Current inline check (lines 433-435):
|
||||
```rust
|
||||
let is_auth = err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403");
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let is_auth = crate::service::provider::is_auth_error(err_str);
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: reuse is_auth_error from provider.rs, DRY backend retry logic"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Abort-Flag Check Helper
|
||||
|
||||
**Problem:** `AtomicBool::load(Ordering::SeqCst)` repeated 16 times across 4 files with 2 variants (`Option<Arc<AtomicBool>>` and bare `AtomicBool`).
|
||||
|
||||
**Design:** Two tiny free functions.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/abort.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/runtime/actions/turn.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
|
||||
### Step 1: Create `abort.rs`
|
||||
|
||||
```rust
|
||||
//! Shared abort-flag checks.
|
||||
//!
|
||||
//! The two variants (Option<Arc<AtomicBool>> and bare AtomicBool) are
|
||||
//! used across the agent runtime, subagent, workflow engine, and provider.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Check whether an optional abort flag has been signalled.
|
||||
pub fn is_aborted(flag: &Option<Arc<AtomicBool>>) -> bool {
|
||||
flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst))
|
||||
}
|
||||
|
||||
/// Check whether a bare abort flag has been signalled.
|
||||
pub fn is_aborted_direct(flag: &AtomicBool) -> bool {
|
||||
flag.load(Ordering::SeqCst)
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register in `mod.rs`
|
||||
|
||||
```rust
|
||||
pub mod abort;
|
||||
```
|
||||
|
||||
### Step 3: Replace 16 call sites
|
||||
|
||||
**In `engine.rs` (4 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
.abort_flag.as_ref().is_some_and(|f| f.load(std::sync::atomic::Ordering::SeqCst))
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted(&ctx.abort_flag)
|
||||
```
|
||||
|
||||
**In `turn.rs` (5 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted_direct(&tc.abort_flag)
|
||||
```
|
||||
|
||||
**In worklow `mod.rs` (5 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&sp.abort_flag)
|
||||
```
|
||||
|
||||
**In `provider.rs` (2 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&abort_flag)
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract is_aborted helpers, DRY 16 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: `dirty()` Helper in `input.rs`
|
||||
|
||||
**Problem:** `state.dirty = true; return Vec::new()` repeated 5 times with direct field access instead of using the existing `state.mark_dirty()` method.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/controller/input.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Mark state dirty and return an empty action list.
|
||||
fn mark(state: &mut AppStateRest) -> Vec<Action> {
|
||||
state.mark_dirty();
|
||||
Vec::new()
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace 5 occurrences
|
||||
|
||||
Replace `state.dirty = true; return vec![];` and `state.dirty = true; return Vec::new();` with `return mark(state);`.
|
||||
|
||||
Additional: Convert the remaining 17 `state.dirty = true;` to `state.mark_dirty();` for API consistency.
|
||||
|
||||
### Step 3: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend`
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: use mark_dirty() helper in input.rs, DRY 22 sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. **Task 1** (Atomic write) — most lines saved, independent, well-understood pattern
|
||||
2. **Task 2** (Clippy allow) — mechanical, safe, 65 files touched but no behavior change
|
||||
3. **Task 4** (Session ID helper) — small, contained, eliminates duplication within one file
|
||||
4. **Task 5** (Backoff merge) — cross-file, needs careful diff of behavior
|
||||
5. **Task 6** (Auth error check) — depends on Task 5's provider.rs changes, do after
|
||||
6. **Task 7** (Abort flag) — independent, mechanical
|
||||
7. **Task 8** (dirty helper) — independent, small change
|
||||
|
||||
Total estimated savings: **~450–600 lines of duplication removed** across ~85 file changes.
|
||||
@@ -1,187 +0,0 @@
|
||||
# TUI Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-14
|
||||
**Scope:** `src/view/`, `src/controller/` (render/interaction layer only)
|
||||
|
||||
## Context
|
||||
|
||||
The TUI went through a "modern design" pass the day before this spec (commit `3f5f27c`:
|
||||
dark palette, neon accents, message cards, segmented status bar). The request for this
|
||||
overhaul covers all three axes at once: aesthetics, UX/navigation, and layout paradigm —
|
||||
not a re-skin of the existing structure.
|
||||
|
||||
## Goals
|
||||
|
||||
- Replace the current 3-zone layout (chat / input / status, everything else as a
|
||||
full-block centered modal) with a **Multi-Pane Dashboard**: chat stays central, a
|
||||
persistent right sidebar surfaces live status that today requires opening a modal.
|
||||
- Replace the current "neon dusk" palette with a **Tokyo Night** palette.
|
||||
- Replace the current per-message card rendering (badge pill, left accent bar, blank-line
|
||||
gaps) with a **tight inline log** format.
|
||||
- Drop decorative emoji from overlay titles in favor of plain colored text — the accent
|
||||
border/text color already carries identity.
|
||||
- Restyle (not restructure) the overlays that stay modal.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No `AppStateRest` shape changes, no new `Action` variants, no controller/state-mutation
|
||||
changes. This is a view-layer repaint; `theme.rs` constants are the only "API" the rest
|
||||
of the app depends on, and their names don't change, only their values.
|
||||
- No new keybindings and no mouse support. Sidebar widgets are read-only/glanceable —
|
||||
none of the three (Workflow, Todo, Usage) are interactive today, so they don't need
|
||||
focus or selection state in their new form either.
|
||||
- No overlay is removed. Workflow/Todo/Usage keep their existing overlay trigger as an
|
||||
"expand" view (see below); the other 13 overlays are untouched functionally.
|
||||
- No automated visual/snapshot tests are being introduced (none exist today for
|
||||
`view/`/`controller/`; see Testing below).
|
||||
|
||||
## Layout architecture
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────┬──────────────┐
|
||||
│ │ WORKFLOW │
|
||||
│ Chat transcript (tight inline log) │ ▶ Node-0-1 │
|
||||
│ │ ✓ Node-0-2 │
|
||||
│ ├──────────────┤
|
||||
│ │ TASKS │
|
||||
│ │ ☐ Fix bug │
|
||||
│ │ ☑ Repro │
|
||||
│ ├──────────────┤
|
||||
│ │ USAGE │
|
||||
│ │ 12.3k tok │
|
||||
├─────────────────────────────────────────────┴──────────────┤
|
||||
│ ❯ input bar │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ status bar │
|
||||
└───────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- The sidebar is a fixed-width column (generalizing the existing `show_todo`
|
||||
two-column split in `view/mod.rs::draw`) holding three stacked widgets, in this
|
||||
order: **Workflow**, **Tasks**, **Usage**.
|
||||
- **Responsive collapse**: below a width threshold (~90 cols — extending the existing
|
||||
`show_todo && area.width > 60` precedent, widened because the new sidebar holds three
|
||||
stacked widgets instead of one), the sidebar doesn't render and chat takes full width.
|
||||
No manual toggle key — purely width-driven, matching current behavior.
|
||||
- Each sidebar widget truncates its content to what fits and shows a `+N more, press
|
||||
<key> to expand` hint (same pattern `Rewind` already uses for `"... and N more
|
||||
messages"`) when there's more than fits — that's what the kept overlay is for.
|
||||
|
||||
### Workflow / Todo / Usage: sidebar glance + overlay expand
|
||||
|
||||
These three overlays are **not removed**. Their existing trigger (same keys/commands as
|
||||
today) still opens the full-screen version — now serving as the "expand" view for when
|
||||
the sidebar column is too narrow to show everything (many hive-mind nodes, a long task
|
||||
list). The sidebar widget and the overlay both read the same state
|
||||
(`workflow_engine`, `misc.todo_content`, `session_runtime.usage` +
|
||||
`session_runtime.session_start`); the sidebar version is a new compact rendering, factored
|
||||
out so both call sites share it where the content is identical (e.g. per-agent card
|
||||
formatting in `workflow.rs`).
|
||||
|
||||
### Remaining 13 overlays: restyled modals, unchanged behavior
|
||||
|
||||
`Help, Settings, Bash, QuitConfirm, KeyInput, Editor, Effort, Mcp, Rewind, Learning,
|
||||
Loading, ModelSelector, ClearConfirm` keep their current centered-modal mechanic and
|
||||
content logic exactly as-is. Only their chrome changes: new palette values (same
|
||||
semantic-color-per-overlay mapping as today — e.g. `QuitConfirm` stays `ERROR`, `Settings`
|
||||
stays `PRIMARY`), and emoji dropped from their title strings.
|
||||
|
||||
## Visual language
|
||||
|
||||
### Palette — Tokyo Night
|
||||
|
||||
Values only; `Theme` constant names in `view/theme.rs` are unchanged, so every call site
|
||||
across `view/*` keeps working without edits beyond the const definitions themselves.
|
||||
|
||||
| Constant | Value | Constant | Value |
|
||||
|---|---|---|---|
|
||||
| `BG` | `#1a1b26` | `ROLE_USER` | `#9ece6a` |
|
||||
| `SURFACE` | `#1f2335` | `ROLE_ASSISTANT` | `#7aa2f7` |
|
||||
| `SURFACE_ELEVATED` | `#292e42` | `ROLE_SYSTEM` | `#7dcfff` |
|
||||
| `TEXT` | `#c0caf5` | `ROLE_TOOL` | `#e0af68` |
|
||||
| `TEXT_MUTED` | `#a9b1d6` | `PRIMARY` | `#7aa2f7` |
|
||||
| `TEXT_DIM` | `#565f89` | `SUCCESS` | `#9ece6a` |
|
||||
| `BORDER` | `#3b4261` | `WARNING` | `#e0af68` |
|
||||
| `BORDER_FOCUS` | `#7aa2f7` | `ERROR` | `#f7768e` |
|
||||
| `HIGHLIGHT` | `#3d59a1` | `INFO` | `#7dcfff` |
|
||||
| `HIGHLIGHT_DIM` | `#292e42` | `ACCENT_PURPLE` | `#bb9af7` |
|
||||
| `STATUS_BAR_BG` | `#16161e` | `ACCENT_PINK` | `#ff007c` |
|
||||
| `MODE_AUTO` | `#9ece6a` | `ACCENT_ORANGE` | `#ff9e64` |
|
||||
| `MODE_YOLO` | `#f7768e` | `ACCENT_TEAL` | `#73daca` |
|
||||
| `CODE_BG` | `#16161e` | `CODE_BAR` | `#292e42` |
|
||||
| `BLOCKQUOTE_BAR` | `#7dcfff` | `SCROLLBAR_BG` / `SCROLLBAR_FG` | `#1f2335` / `#3b4261` |
|
||||
|
||||
### Message density — tight inline log
|
||||
|
||||
Replaces the per-message card (role badge pill + left accent bar + blank-line gap)
|
||||
in `chat.rs`:
|
||||
|
||||
```
|
||||
you 09:14 fix the login bug
|
||||
ai 09:14 Looking at src/auth.rs now.
|
||||
↳ Reading src/auth.rs
|
||||
you 09:15 ok try again
|
||||
```
|
||||
|
||||
- Role rendered as a short lowercase colored label (`ROLE_*` colors), timestamp dim,
|
||||
inline with the first content line.
|
||||
- Wrapped/multi-line content aligns under the content column (not under the role label).
|
||||
- Tool-call sub-lines get a dim `↳` prefix.
|
||||
- No blank line within a turn; a single blank line only between different speakers (not
|
||||
after every message).
|
||||
- The chat panel's outer bordered `Block` is unchanged — only the messages inside it lose
|
||||
per-message decoration.
|
||||
- The streaming indicator becomes `ai 09:14 ⠋ generating...` inline, matching the new
|
||||
format, instead of the current padded badge line.
|
||||
|
||||
### Icons
|
||||
|
||||
Overlay titles drop decorative emoji (❓⚙💻🚪✏️🎯🔌📋⏪📚📊⏳🧠🗑️⚡) and render as plain
|
||||
bold colored text (e.g. `Settings` in `PRIMARY`, no ⚙). The border/text accent color is
|
||||
the identity signal, consistent with the muted Tokyo Night + tight-density direction.
|
||||
|
||||
## File impact
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `view/theme.rs` | Palette values swap (table above). Const names/count unchanged. |
|
||||
| `view/chat.rs` | Rewrite message rendering to the tight inline format. |
|
||||
| `view/markdown.rs` | Re-themed code/quote colors; tightened padding. No structural rewrite. |
|
||||
| `view/mod.rs` | `draw()` grows the persistent sidebar column (generalizes `show_todo` split). `render_overlay()` match arms restyled in place (palette + title text), content logic untouched. Todo/Usage compact-widget rendering factored out of the current inline overlay code so it's callable from both the sidebar and the kept overlay. |
|
||||
| `view/status.rs` | Restyle to new palette; structurally unchanged. |
|
||||
| `view/workflow.rs` | Add a compact-card render function for the sidebar widget, reusing the existing per-agent formatting logic. |
|
||||
| `controller/*` | No changes. Interaction model is unchanged; sidebar is non-interactive. |
|
||||
|
||||
## Edge cases
|
||||
|
||||
- Empty states per sidebar widget (no workflow running, no tasks, zero usage) — compact
|
||||
one-line placeholders, consistent with the tight density (not the current multi-line
|
||||
placeholder paragraphs).
|
||||
- Sidebar auto-collapses below ~90 cols; chat reclaims full width.
|
||||
- Sidebar widget overflow (e.g. a hive-mind run with many nodes, a long task list)
|
||||
truncates with a `+N more` hint pointing at the existing expand-overlay trigger.
|
||||
- Long chat content wraps with continuation lines aligned under the content column.
|
||||
|
||||
## Testing / verification
|
||||
|
||||
No automated visual or snapshot tests exist for `view/`/`controller/` today (confirmed:
|
||||
zero `#[cfg(test)] mod tests` in either directory), and none are introduced by this
|
||||
change — ratatui rendering isn't meaningfully unit-testable without a snapshot harness
|
||||
this repo doesn't have. Verification is manual: run the TUI (`cargo run`) and exercise
|
||||
the golden paths (send a chat message, trigger a workflow/hive-mind run, open each of the
|
||||
13 remaining overlays, resize the terminal across the sidebar-collapse threshold).
|
||||
`cargo clippy` must stay clean (warnings-as-errors per repo config), and every touched
|
||||
`pub fn`/`struct` keeps the doc-comment convention from CLAUDE.md (What/Flow/Why/Return).
|
||||
|
||||
## Suggested implementation order
|
||||
|
||||
Not binding — the implementation plan owns sequencing — but a sensible build order given
|
||||
the dependency shape (palette first, since everything else reads `Theme` consts):
|
||||
|
||||
1. `theme.rs` palette swap
|
||||
2. `chat.rs` tight-inline rewrite
|
||||
3. `mod.rs` sidebar scaffolding + Workflow/Tasks/Usage compact widgets (+ `workflow.rs`
|
||||
compact-card fn)
|
||||
4. `status.rs` restyle + remaining 13 overlay restyle (mechanical: palette + title text)
|
||||
5. Manual TUI verification pass across golden paths above
|
||||
@@ -1,114 +0,0 @@
|
||||
# Clipboard Copy via OSC52 — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/controller/input.rs`, `src/main.rs`,
|
||||
`src/ipc/protocol.rs`
|
||||
|
||||
## Context
|
||||
|
||||
There is no clipboard support anywhere in the TUI today, and mouse capture is enabled
|
||||
(`EnableMouseCapture` in `main.rs`), which in most terminal emulators suppresses native
|
||||
click-drag text selection unless the user holds a modifier — making an in-app copy action
|
||||
more valuable than it would be in a plain scrollback. OSC52 is a terminal escape sequence
|
||||
(`\x1b]52;c;<base64>\x07`) that asks the terminal emulator itself to set the system
|
||||
clipboard; it needs no OS-level clipboard library (no X11/Wayland/win32 dependency) and
|
||||
the `base64` crate is already a dependency (used in `service/oauth/pkce.rs`), so no new
|
||||
crate is needed for this feature.
|
||||
|
||||
Key architectural constraint discovered while designing this: `controller::input::handle_key`
|
||||
runs on the **daemon** process in `--daemon`/`--attach` mode (`main.rs:359`, inside
|
||||
`handle_daemon_client`), not on the process that owns the user's actual terminal. A raw
|
||||
`io::stdout()` write inside `handle_key` would go to the headless daemon's stdout in that
|
||||
mode, not the user's terminal. The copy action therefore can't write the escape sequence
|
||||
directly from `handle_key` — it has to signal intent via state, and the terminal-owning
|
||||
process (single-process `run_loop_inner`, or the attach client's loop) performs the actual
|
||||
write.
|
||||
|
||||
## Goals
|
||||
|
||||
- `Ctrl+Y` copies the most recent `Role::Assistant` message's raw text (not the rendered
|
||||
markdown spans) to the system clipboard via OSC52.
|
||||
- Works identically in single-process mode and in `--daemon`/`--attach` mode.
|
||||
- No new dependency.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No native clipboard fallback (e.g. `arboard`) for terminals that don't honor OSC52 —
|
||||
unsupported terminals silently swallow the escape sequence; no error surfaces to the
|
||||
user beyond the optimistic "Copied to clipboard" toast (there's no ack mechanism in the
|
||||
OSC52 protocol to verify the terminal actually did it).
|
||||
- No copy-last-code-block variant — out of scope for this pass; the whole-message copy
|
||||
covers the common case and is simple to extend later if needed.
|
||||
- No mouse-drag text selection — unrelated, much larger feature; not being built here.
|
||||
|
||||
## State (`misc.rs`)
|
||||
|
||||
- `MiscState` gains `pub pending_clipboard_copy: Option<String>`, initialized to `None` in
|
||||
`MiscState::new()`.
|
||||
|
||||
## `input.rs`
|
||||
|
||||
- New top-level arm alongside the existing `Ctrl+C`/`Ctrl+D` handlers:
|
||||
`KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL)`. It finds the last
|
||||
message in `state.transcript_cache.messages` with `role == Role::Assistant`:
|
||||
- If found: `state.misc.pending_clipboard_copy = Some(msg.content.clone())`.
|
||||
- If not found: push an `Info` toast ("No assistant message to copy yet") and leave
|
||||
`pending_clipboard_copy` as `None`.
|
||||
- Returns `Vec::new()` — this is a direct state mutation inside `handle_key`, matching
|
||||
the existing `Ctrl+S` editor-save precedent (`main.rs`'s editor branch also mutates
|
||||
state/does I/O directly rather than going through an `Action`).
|
||||
|
||||
## OSC52 write helper (`main.rs`)
|
||||
|
||||
```
|
||||
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
|
||||
write!(stdout, "\x1b]52;c;{b64}\x07")?;
|
||||
stdout.flush()
|
||||
}
|
||||
```
|
||||
|
||||
Generic over `impl Write` so both the single-process loop (writing to `io::stdout()`) and
|
||||
tests (writing to a `Vec<u8>` to assert the formatted sequence) can use it without a real
|
||||
terminal.
|
||||
|
||||
## Single-process mode (`run_loop_inner`)
|
||||
|
||||
After the existing `for action in actions { apply_action(state, action); }` block, add:
|
||||
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
state.push_toast(Toast::new(ToastKind::Success, "Copied to clipboard".into()));
|
||||
}
|
||||
```
|
||||
|
||||
## Daemon/attach mode
|
||||
|
||||
- `ipc/protocol.rs`: add `DaemonFrame::ClipboardCopy(String)` (alongside `StateUpdate`,
|
||||
`StreamToken`, `SystemNote`, `Closed` — same `Serialize`/`Deserialize` derive).
|
||||
- `handle_daemon_client` (`main.rs`): after each branch that calls `handle_key`/`apply_action`
|
||||
(`KeyPress` and `Submit`, the only two that can reach the input handler), before the
|
||||
existing `send_daemon_update(&mut conn, state)?;` call, add:
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
conn.send(&DaemonFrame::ClipboardCopy(text))?;
|
||||
}
|
||||
```
|
||||
- Attach-client loop (`main.rs`, the function matching on `DaemonFrame::StateUpdate` /
|
||||
`SystemNote` / `Closed` around line 573): add a `DaemonFrame::ClipboardCopy(text) => {
|
||||
let _ = write_osc52(&mut io::stdout(), &text); client_state.push_toast(...); }` arm,
|
||||
mirroring the existing `SystemNote` handling but performing the actual terminal write
|
||||
since this process — not the daemon — owns the user's terminal.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `input.rs`: `Ctrl+Y` with a transcript containing multiple messages sets
|
||||
`pending_clipboard_copy` to the *last* assistant message's content, ignoring later
|
||||
user/tool messages that might follow it; with no assistant message present, it pushes
|
||||
an info toast and leaves `pending_clipboard_copy` as `None`.
|
||||
- `main.rs`: `write_osc52` writing into a `Vec<u8>` buffer produces the exact expected
|
||||
`\x1b]52;c;<base64>\x07` byte sequence for a known input string.
|
||||
@@ -1,115 +0,0 @@
|
||||
# Diff View for edit/write Tools — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/tool/fs/edit.rs`, `src/tool/fs/write.rs`, `src/view/markdown.rs`, `src/view/chat.rs`
|
||||
|
||||
## Context
|
||||
|
||||
`edit` currently reports only a byte-delta (`"edited {rel} ({N} byte delta)"`), and `write`
|
||||
reports only a byte count. Neither the model nor the user sees what actually changed —
|
||||
just a number. This makes it hard for the model to self-verify an edit landed correctly,
|
||||
and hard for the user to review a change without opening the file. No diff-computing
|
||||
library exists in the dependency tree today.
|
||||
|
||||
## Goals
|
||||
|
||||
- `edit` returns a real unified diff (git-style, 3 lines of context) of the change it just
|
||||
made, in place of the byte-delta note.
|
||||
- `write` returns the same kind of diff when it overwrites a file that already existed
|
||||
with valid UTF-8 content; falls back to the current "wrote N bytes" message for new
|
||||
files or non-UTF-8 (binary) overwrites.
|
||||
- Diffs render in the chat view with real color (green add / red remove / cyan hunk
|
||||
header) instead of being flattened to dim/italic like other tool output.
|
||||
- Large diffs are truncated with a trailing count, matching the existing pattern in
|
||||
`read.rs` (`"... ({N} more lines, total {total})"`).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No diff view for any tool besides `edit`/`write` (e.g. no retroactive diffing of
|
||||
`bash_tools.rs` shell edits).
|
||||
- No side-by-side diff layout — unified format only, matching how every other tool
|
||||
output already renders as a single text stream.
|
||||
- No persistence of diff history; each diff is only the delta of the single tool call
|
||||
that produced it, not a cumulative session diff.
|
||||
- No changes to non-tool (assistant/user/system) message rendering or coloring.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `similar = "3"` (line/word diff crate; permissive MIT/Apache-2.0, no heavy
|
||||
transitive deps). Use `TextDiff::from_lines(old, new).unified_diff().context_radius(3)`,
|
||||
which produces standard `@@ -a,b +c,d @@` hunk headers and `-`/`+`/` `-prefixed lines —
|
||||
no custom diff algorithm needed.
|
||||
|
||||
## Tool changes
|
||||
|
||||
### `edit.rs`
|
||||
|
||||
After computing `new_content` and writing it to disk:
|
||||
|
||||
1. Compute `similar::TextDiff::from_lines(&content, &new_content).unified_diff().context_radius(3).to_string()`.
|
||||
2. Split into lines; if `> MAX_DIFF_LINES` (200), keep the first 200 and append
|
||||
`"... ({N} more lines truncated)"`.
|
||||
3. Wrap the (possibly truncated) diff text in a fenced ` ```diff ` block.
|
||||
4. Replace the byte-delta note in the returned message with this block; keep the
|
||||
existing "Graduated checks matched" / LSP note suffixes in their current position
|
||||
(after the diff block).
|
||||
|
||||
### `write.rs`
|
||||
|
||||
Before overwriting:
|
||||
|
||||
1. If `path.exists()` and `fs::read_to_string(&path)` succeeds (valid UTF-8), capture it
|
||||
as `old_content` and note `is_overwrite = true`.
|
||||
2. If the file doesn't exist, or reading it fails (binary/non-UTF-8), `is_overwrite = false`
|
||||
— no error, just skip the diff path silently.
|
||||
3. After writing, if `is_overwrite`, compute and truncate the diff exactly as in `edit.rs`
|
||||
and append the fenced block to the return message (in addition to the existing
|
||||
"wrote N bytes" line, not instead of it — for `write`, unlike `edit`, the byte count is
|
||||
still useful since it can be a full-file rewrite).
|
||||
4. If not `is_overwrite`, return message is unchanged from today.
|
||||
|
||||
The truncation constant (`MAX_DIFF_LINES = 200`) and truncation message format are
|
||||
shared — factor into a small helper in `tool/fs/helpers.rs` used by both tools.
|
||||
|
||||
## Rendering changes
|
||||
|
||||
### `markdown.rs`
|
||||
|
||||
- `render_markdown` gains a `dim: bool` parameter: `render_markdown(text, width, dim)`.
|
||||
- Capture the fence language from `Tag::CodeBlock(CodeBlockKind::Fenced(lang))` (today
|
||||
matched as `CodeBlock(_)`, discarding the language). Track `in_diff_block: bool` when
|
||||
`lang == "diff"`.
|
||||
- Inside a diff block, process text line-by-line instead of as one blob: a line starting
|
||||
with `+` (not `+++`) is styled green, `-` (not `---`) red, `@@` cyan/muted, everything
|
||||
else (context lines, `+++`/`---` file headers) uses the existing code-block teal.
|
||||
- When `dim` is `true`: every span keeps its assigned color as computed above, but
|
||||
non-diff spans (headings, links, plain text, non-diff code blocks, table cells) fall
|
||||
back to `Theme::TEXT_DIM` + `Modifier::ITALIC` instead of their normal palette color —
|
||||
this replicates today's "tool output is always dim" behavior for everything except
|
||||
diff lines.
|
||||
- When `dim` is `false`: behavior is unchanged from today (full color, used for
|
||||
assistant/user/system messages).
|
||||
|
||||
### `chat.rs`
|
||||
|
||||
- `Role::Tool` branch: replace the two manual span-remapping loops (that force every
|
||||
span to `dim_italic`) with a direct call to `render_markdown(&content, content_width, true)`
|
||||
and use the returned spans as-is.
|
||||
- All other roles: call `render_markdown(&content_str, content_width, false)` — same
|
||||
call as today, just with the new explicit `false` argument.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `edit.rs`: a normal single-replace edit produces a diff block with matching
|
||||
`-`/`+` lines; a `replace_all` across 250+ lines truncates at 200 with the correct
|
||||
trailing count.
|
||||
- `write.rs`: writing a brand-new file keeps the old "wrote N bytes" message with no
|
||||
diff block; overwriting an existing UTF-8 file produces a diff block; overwriting
|
||||
a path that reads as invalid UTF-8 (simulate via non-UTF-8 bytes) falls back to the
|
||||
byte-count message without erroring.
|
||||
- `markdown.rs`: a fenced ` ```diff ` block with `+`/`-`/`@@` lines produces spans with
|
||||
the expected fg colors under `dim=true` (diff lines colored) and confirms non-diff
|
||||
text in the same call falls back to `TEXT_DIM` + italic.
|
||||
@@ -1,126 +0,0 @@
|
||||
# Fuzzy @file-mention Autocomplete — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/app/state/rest.rs`, `src/controller/input.rs`,
|
||||
`src/view/mod.rs`, `src/tool/mod.rs`, `src/tool/fs/write.rs`, `src/main.rs`
|
||||
|
||||
## Context
|
||||
|
||||
The chat input already has a dropdown autocomplete (`InputState` in `misc.rs`), but it
|
||||
only covers slash commands: it requires the whole buffer to start with `/` and filters a
|
||||
fixed `COMMANDS` list by prefix. There's no way to reference a project file from the chat
|
||||
input without typing its exact path from memory. The existing `dir_cache` (used by the
|
||||
`dir_cache_update` tool) looks like it could serve this but doesn't: it's a single,
|
||||
non-recursive directory snapshot, overwritten on each LLM-driven `dir_cache_update` call —
|
||||
not a standing, recursive, whole-workspace file index. `search.rs`'s `Grep`/`Glob` tools
|
||||
already do the recursive, `.gitignore`-respecting walk this feature needs, via
|
||||
`ignore::Walk`.
|
||||
|
||||
Also relevant: there is no persistent async runtime driving the TUI loop. `main.rs`
|
||||
constructs a `tokio::runtime::Runtime` but never `.enter()`s or `block_on`s it in the
|
||||
main loop — `run_loop` is fully synchronous. The one existing async-flavored pattern
|
||||
(`dir_cache_update.rs`) spins up a throwaway one-shot runtime purely to satisfy
|
||||
`tokio::sync::RwLock`'s API, then discards it. This feature does not need that ceremony:
|
||||
a plain `std::sync::RwLock` is enough, since every reader/writer here is synchronous
|
||||
(`handle_key`, `Tool::run`, and the index-build thread all being plain sync code).
|
||||
|
||||
## Goals
|
||||
|
||||
- Typing `@` at a word boundary (start of buffer or after whitespace) in the chat input,
|
||||
followed by non-whitespace characters, opens a dropdown of fuzzy-matched project file
|
||||
paths, live-updating as the query changes.
|
||||
- Selecting a candidate splices `@relative/path ` into the buffer at the mention's
|
||||
position (not a whole-buffer replace) and the user keeps typing.
|
||||
- Candidates come from a background-built, whole-workspace file index — not the
|
||||
LLM-facing `dir_cache`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No auto-reading of the selected file's content into the conversation — the inserted
|
||||
`@path` is plain text; the model reads it via the `read` tool if it wants to, same as
|
||||
any other path reference.
|
||||
- No live re-filter on Backspace/Delete while a mention dropdown is open — mirrors the
|
||||
slash-command dropdown's existing behavior (closes on Backspace/Delete rather than
|
||||
refiltering). Not fixing that for commands here; file mentions just inherit it for
|
||||
consistency.
|
||||
- No periodic re-walk of the index after startup — only single-file incremental updates
|
||||
on file creation (see below). A deleted or renamed file may show a stale entry until
|
||||
restart; acceptable since selecting it just inserts text, it doesn't touch the
|
||||
filesystem.
|
||||
- No fuzzy matching over directories, only files.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `nucleo-matcher = "0.3"` (the fuzzy-matching engine from the Helix editor project;
|
||||
small, actively maintained, no heavy transitive deps).
|
||||
|
||||
## Index storage & construction
|
||||
|
||||
- New type in `misc.rs`: `MentionIndex { entries: Arc<std::sync::RwLock<Vec<String>>> }`, with `MentionIndex::new()`, `set(&self, paths: Vec<String>)`, and `snapshot(&self) -> Vec<String>` (both plain sync `.write()`/`.read()`, no `try_`/async — a std `RwLock` doesn't block indefinitely here since every hold is a quick vec swap or clone).
|
||||
- `AppStateRest` gets a `pub mention_index: MentionIndex` field, initialized in `AppStateRest::new()`, threaded into `ToolCtx`/`ToolCtxBuilder` the same way `dir_cache` is (new `mention_index` field on both, wired through `tool_ctx()`/`tool_ctx_for()`/`build()`).
|
||||
- In `main.rs`, right after `AppStateRest::new(...)` in the single-process TUI path and the daemon path (not the attach-only client path, which has no local `ToolCtx`), spawn `std::thread::spawn` that:
|
||||
1. For each workspace root (index `i`, path `w`): `ignore::Walk::new(w)`, keep only files, strip `w` as prefix, format as `rel` for `i == 0` or `[i]rel` for `i > 0` (matching `resolve_path`'s existing workspace-index convention).
|
||||
2. Stop collecting once the total across all workspaces hits 50,000 entries (repos larger than that are rare here; this is a soft cap to bound memory/scan time, not a hard requirement).
|
||||
3. Call `mention_index.set(all_paths)`.
|
||||
- `write.rs`: after a successful write, if the target path did **not** exist before the write (i.e. this created a new file, not an overwrite), compute its relative/workspace-prefixed form and push it onto `ctx.mention_index`'s vec directly (read-modify-write under the same lock) rather than re-walking.
|
||||
|
||||
## `InputState` changes (`misc.rs`)
|
||||
|
||||
- New `pub enum AutocompleteKind { Command, FileMention }`.
|
||||
- `InputState` gains `pub autocomplete_kind: AutocompleteKind` (default `Command`) and
|
||||
`pub mention_start: usize` (byte offset of the triggering `@`).
|
||||
- New `fn mention_query_at_cursor(&self) -> Option<(usize, String)>`: scans backward from
|
||||
`self.cursor` for an `@`; the scan stops (returns `None`) if it hits whitespace before
|
||||
finding `@`. The `@` only counts as a trigger if it's at buffer start or immediately
|
||||
preceded by whitespace. Returns `(byte offset of '@', query text between '@' and cursor)`.
|
||||
- New `fn open_mention_autocomplete(&mut self, files: &[String])`: calls
|
||||
`mention_query_at_cursor()`; if `None`, calls `close_autocomplete()` and returns. If
|
||||
`Some((start, query))`, fuzzy-matches `query` against `files` via `nucleo-matcher`,
|
||||
keeps the top 10 by score, sets `autocomplete_candidates`, `autocomplete_kind =
|
||||
FileMention`, `mention_start = start`, `autocomplete_visible = !candidates.is_empty()`.
|
||||
- `select_autocomplete()` becomes kind-aware:
|
||||
- `Command` (today's behavior, unchanged): `buffer = candidate.clone()`, `cursor =
|
||||
buffer.len()`.
|
||||
- `FileMention`: `buffer.replace_range(mention_start..cursor, &format!("@{candidate} "))`,
|
||||
`cursor = mention_start + candidate.len() + 2` (the `@` plus the candidate plus the
|
||||
trailing space).
|
||||
- Both paths end with `close_autocomplete()`, same as today.
|
||||
|
||||
## `input.rs` wiring
|
||||
|
||||
- `KeyCode::Char(c)` handler: after `state.input.insert(c)`, keep the existing
|
||||
`if buffer.starts_with('/') { open_autocomplete() }` check, and add an `else if let
|
||||
Some(_) = state.input.mention_query_at_cursor() { state.input.open_mention_autocomplete(&state.mention_index.snapshot()) }` branch. These are mutually exclusive in practice (a
|
||||
buffer starting with `/` is a slash command, not a sentence with an `@mention` in it).
|
||||
- `KeyCode::Backspace` / `KeyCode::Delete`: unchanged — both already just call
|
||||
`close_autocomplete()` when a dropdown is visible, regardless of kind. No new branching
|
||||
needed since `close_autocomplete()` already resets `autocomplete_kind` isn't touched but
|
||||
becomes irrelevant once `autocomplete_visible` is false.
|
||||
- `KeyCode::Tab`: currently gated on `buffer.starts_with('/')`. Extend the condition to
|
||||
also fire when `autocomplete_kind == FileMention && autocomplete_visible` so Tab cycles
|
||||
file-mention candidates too.
|
||||
- `KeyCode::Enter`: unchanged — already calls `select_autocomplete()` whenever
|
||||
`autocomplete_visible`, which is now kind-aware internally.
|
||||
|
||||
## Rendering (`view/mod.rs`)
|
||||
|
||||
- `render_input_bar`'s dropdown block reuses the exact same list-rendering code (already
|
||||
generic over `autocomplete_candidates`/`autocomplete_idx`); only the title changes based
|
||||
on `state.input.autocomplete_kind`: `" ⌘ Commands "` (unchanged) vs `" 📁 Files "`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `misc.rs`: `mention_query_at_cursor` returns the right `(start, query)` for `@` at
|
||||
buffer start, `@` after a space mid-sentence, and correctly returns `None` when the `@`
|
||||
is mid-word (e.g. `foo@bar`) or when whitespace exists between the `@` and the cursor.
|
||||
`select_autocomplete` for `FileMention` splices correctly into a buffer with text before
|
||||
and after the mention span; `Command` selection still replaces the whole buffer as
|
||||
before.
|
||||
- `write.rs`: creating a new file appends its path to the shared `mention_index`;
|
||||
overwriting an existing file does not add a duplicate entry.
|
||||
- Index construction: not unit-tested directly (it's a `std::thread::spawn` walking the
|
||||
real filesystem at startup) — covered implicitly by exercising the app manually per the
|
||||
`verify` skill during implementation.
|
||||
@@ -1,281 +0,0 @@
|
||||
# Context & Compaction Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-16
|
||||
**Scope:** replaces `src/app/runtime/shortsend.rs`; touches `src/app/runtime/actions/mod.rs`,
|
||||
`src/view/status.rs`, `src/model/settings.rs`, `src/app/subagent/division.rs`, `Cargo.toml`
|
||||
|
||||
## Context
|
||||
|
||||
The existing conversation-compaction system (`shortsend.rs`, 129 lines) only acts once the
|
||||
context is already close to the model's window limit, and has accumulated inconsistencies
|
||||
found during a codebase audit:
|
||||
|
||||
1. Three different token-count heuristics for the same job: `/3` inside
|
||||
`shortsend::shape_messages`, `/4` in the auto-compact loop
|
||||
(`actions/mod.rs` ~line 1146), `/4` again in the live status bar (`view/status.rs:68`).
|
||||
2. Manual `/compact` (`Action::Compact`, `actions/mod.rs:547-563`) passes `client: None`
|
||||
because `apply_action` is synchronous, so it never gets LLM summarization — it always
|
||||
falls back to the bare `"[prior conversation compacted]"` placeholder, unlike automatic
|
||||
mid-turn compaction (`Some(&tc.client)`, line 1160). Undocumented asymmetry between the
|
||||
two trigger paths.
|
||||
3. `context_window` resolution (`model_roles.values().find(...).and_then(...).unwrap_or(...)`)
|
||||
duplicated three times (`Action::Compact`, `spawn_turn`, `view/status.rs` twice).
|
||||
4. No repeated-tool-call dedup: reading the same file (or running the same grep) twice in a
|
||||
session keeps both full copies in context forever, until compaction eventually drops the
|
||||
older one wholesale along with everything else from that period.
|
||||
5. No per-result compression: a single large tool output (a big `bash` log, a large `grep`
|
||||
result) is stored verbatim even when most of it is redundant or low-value.
|
||||
6. Zero test coverage on `shortsend.rs`.
|
||||
|
||||
Separately, research into three real, permissively-licensed open-source projects
|
||||
(`rtk-ai/rtk`, Apache-2.0; `headroomlabs-ai/headroom`, Apache-2.0; `JuliusBrussee/caveman`,
|
||||
MIT — verified via `gh api` for authenticity/license, and by cloning and reading source, not
|
||||
taken from marketing blog posts) surfaced techniques worth reimplementing natively:
|
||||
|
||||
- **rtk**: generic line-scan compression (strip comment/blank runs, brace-depth collapse of
|
||||
function bodies, importance-ranked truncation ending in an unambiguous `[N more lines]`
|
||||
marker — their own regression tests show a comment-shaped marker confuses the LLM into
|
||||
retry-looping) plus structured per-toolchain parsing (e.g. `cargo --message-format=json`
|
||||
bucketed into errors/warnings, boilerplate lines dropped).
|
||||
- **headroom**: per-content-type compressors — logs (classify lines by level/stack-trace/
|
||||
summary, score, keep highest-value lines + surrounding context, adaptive cap), grep
|
||||
results (group by file, score matches, cap globally and per-file), JSON (keep all
|
||||
structural tokens — keys, brackets, colons — drop or shrink long low-entropy string
|
||||
values, keep short values and UUID/hash-shaped high-entropy ones).
|
||||
- **caveman**: a pure prompt/persona instruction (no algorithm) that tells the model to
|
||||
write tersely — drop articles/filler/hedging, keep code/commands/errors verbatim — with an
|
||||
explicit carve-out that disables terseness for destructive-op confirmations and security
|
||||
warnings. This compresses *output* tokens, a different axis from everything else in this
|
||||
design, which compresses *input* context.
|
||||
|
||||
This is a from-scratch reimplementation of the underlying ideas, not a port — no code is
|
||||
copied from any of the three projects.
|
||||
|
||||
## Goals
|
||||
|
||||
- One unified, always-on pipeline that keeps context lean from turn 1, not just once near
|
||||
the limit.
|
||||
- Deduplicate repeated tool calls: an older copy of a tool result superseded by an identical
|
||||
later call (same tool name + same arguments) is replaced with a placeholder, for read-only
|
||||
tools only.
|
||||
- Compress large individual tool results (logs, JSON, generic text) at capture time, above a
|
||||
size floor.
|
||||
- Fix the three known inconsistencies (token heuristic, manual/auto asymmetry,
|
||||
`context_window` duplication).
|
||||
- Optional, off-by-default "concise mode" system-prompt toggle for terser model output.
|
||||
- Full inline test coverage per repo convention.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not adding a runtime dependency on `rtk`, `headroom`, or `caveman` themselves (as a binary,
|
||||
proxy, or crate) — everything is implemented natively in Rust inside zesdex.
|
||||
- Not building rtk's per-toolchain structured parsers (`cargo --message-format=json`
|
||||
re-invocation, etc.) — too invasive for a general-purpose `bash` tool that runs arbitrary
|
||||
commands zesdex doesn't control the flags of. Only the generic line-scan/log/JSON layer is
|
||||
built.
|
||||
- Not switching to an exact per-provider tokenizer — `tiktoken-rs` (BPE, cl100k_base/
|
||||
o200k_base) is an approximation good enough for the 85%/95% budget thresholds; it is not
|
||||
used for billing-accurate counts.
|
||||
- `caveman-compress`-style memory-file rewriting (the LLM-round-trip variant of caveman) is
|
||||
out of scope — only the pure-prompt persona mechanism is adopted.
|
||||
|
||||
## Architecture
|
||||
|
||||
Replace `src/app/runtime/shortsend.rs` with `src/app/runtime/context/`:
|
||||
|
||||
```
|
||||
context/
|
||||
mod.rs — module registration only, no facade (see below)
|
||||
tokens.rs — unified token counting (tiktoken-rs)
|
||||
dedup.rs — cross-call tool-result deduplication
|
||||
squash.rs — per-result compression (log/json/generic), applied at
|
||||
tool-result construction time, upstream of prepare()
|
||||
shaping.rs — budget-based drop + LLM summarize (renamed shortsend logic)
|
||||
window.rs — shared context_window resolution
|
||||
```
|
||||
|
||||
### `tokens.rs`
|
||||
|
||||
```rust
|
||||
pub fn count_tokens(text: &str) -> usize
|
||||
pub fn count_message_tokens(msg: &ChatMessage) -> usize
|
||||
```
|
||||
|
||||
Backed by `tiktoken-rs` (new dependency, pure Rust, embedded BPE vocab, no network calls at
|
||||
runtime), using `o200k_base`. Replaces all three existing heuristic call sites: `shortsend`'s
|
||||
internal `/3`, the auto-loop's `/4` (`actions/mod.rs` ~1146), and `status.rs:68`'s `/4`.
|
||||
|
||||
### `dedup.rs`
|
||||
|
||||
```rust
|
||||
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool)
|
||||
```
|
||||
|
||||
The `bool` is `true` iff at least one message was replaced with a placeholder — callers use
|
||||
it to decide whether the result is worth persisting/announcing, without needing `ChatMessage`
|
||||
to implement `PartialEq` (it doesn't today, and adding it purely to diff whole message lists
|
||||
would be needless surface area for what `collapse` already knows precisely mid-walk).
|
||||
|
||||
Flow: walk messages, pair each `Role::Tool` message to its originating `ToolCall` via
|
||||
`tool_call_id`. Key = `(function.name, sha256(canonical_json(function.arguments)))` (`sha2`
|
||||
is already a dependency). Track the last index seen per key. For any earlier occurrence of a
|
||||
key whose tool name is in the read-only set, replace that earlier `Tool` message's `content`
|
||||
with a short placeholder (`"[duplicate result — superseded by a later identical call, see
|
||||
below]"`); the assistant's tool-call entry (name + arguments) is left untouched, so the
|
||||
action/audit trail stays intact. Mutating tools are never touched, even with identical
|
||||
arguments, because call order and repetition can be semantically meaningful (e.g. retrying a
|
||||
flaky `bash` command).
|
||||
|
||||
Read-only classification reuses `subagent::division::tool_scope::READ_TOOLS`
|
||||
(`src/app/subagent/division.rs:21`) rather than a new list — that `const` is made `pub` for
|
||||
this purpose. It already enumerates exactly the read-only tool set (`read`, `grep`, `glob`,
|
||||
`search`, `seqthink`, `recall`, `lsp_*`, `read_findings`).
|
||||
|
||||
Runs every turn, unconditionally, before token counting — not gated on `should_shape`.
|
||||
|
||||
### `squash.rs`
|
||||
|
||||
```rust
|
||||
pub fn apply(tool_name: &str, output: &str) -> String
|
||||
```
|
||||
|
||||
`read` is exempted entirely, always passed through unchanged regardless of size: its output
|
||||
must stay byte-exact because the agent relies on it for exact-match edits afterward, and a
|
||||
squashed view of a JSON config file (or any file whose content happens to parse as JSON)
|
||||
would otherwise be silently altered. Size floor for every other tool: outputs under 1500
|
||||
bytes pass through unchanged (compression only pays off on large output, and touching small
|
||||
results risks losing detail with no token benefit). Above the floor, dispatch by content
|
||||
shape:
|
||||
|
||||
- `squash_json(&str) -> String` — walks a parsed `serde_json::Value` (not a hand-rolled
|
||||
tokenizer — `serde_json` already handles escaping/nesting correctly, reusing it is simpler
|
||||
and more robust); structural tokens (keys, brackets, colons, commas, booleans, null) always
|
||||
kept; string values kept if ≤20 chars or "identifier-shaped" (no internal whitespace *and*
|
||||
Shannon entropy ≥3.0 bits/char — catches UUIDs/hashes/paths), otherwise replaced with `"…"`
|
||||
in place; array elements past the first 3 compressed harder (values elided regardless of
|
||||
length/entropy). Applied when `serde_json::from_str` on the output succeeds. The
|
||||
no-whitespace pre-filter matters: raw per-character entropy alone doesn't separate prose
|
||||
from identifiers — repeated English prose measures ~3.89 bits/char, higher than a UUID's
|
||||
~3.39 — because prose also draws from a wide character set. headroom's own entropy gate is
|
||||
"cheaply pre-filtered by 'no spaces'" before scoring for the same reason; multi-word values
|
||||
never reach the entropy check at all under this rule.
|
||||
- `squash_log(&str) -> String` — line classifier (error/fail/warn/info/debug/trace by
|
||||
keyword + stack-trace-frame detection) → score
|
||||
(`level_score {1.0 error/fail, 0.5 warn, 0.1 info, 0.05 debug/trace} + 0.3 if
|
||||
stack-trace-frame + 0.4 if summary-shaped line`) → keep up to 20 highest-scored error
|
||||
lines, up to 10 highest-scored warning lines, all summary lines, plus a ±2-line context
|
||||
window around each kept line → single `[N lines omitted]` marker for drops (not
|
||||
comment-shaped, per rtk's own finding on LLM confusion). Applied when the output isn't
|
||||
valid JSON, the tool is `bash`, and the output has ≥3 lines matching error/warn/stack-trace
|
||||
patterns. The tool restriction (added after the final whole-branch review) matters: a `grep`
|
||||
result full of matches against error-handling code trips the same ≥3-line keyword threshold
|
||||
as a real build log, but `squash_log`'s hard 20-error/10-warning cap has no byte budget and
|
||||
would silently drop legitimate matches past it — the wrong compressor for search results.
|
||||
Only `bash` (the actual log-producing tool) routes through `squash_log`; every other tool
|
||||
whose output happens to look log-shaped falls through to the gentler, byte-budgeted
|
||||
`squash_generic` instead.
|
||||
- `squash_generic(&str, budget) -> String` — importance-ranked truncation: keeps the first 10
|
||||
and last 10 lines plus any line matching a small "looks important" heuristic (non-blank,
|
||||
not a byte-for-byte repeat of the immediately preceding line), single `[N lines omitted]`
|
||||
marker for the rest, capped to `budget` bytes overall (`budget` = the 1500-byte squash
|
||||
floor doubled, i.e. 3000 bytes, chosen so the fallback path still yields a real reduction
|
||||
on anything that triggered it). Fallback for anything that isn't JSON or log-shaped.
|
||||
|
||||
Called once, at the single tool-result construction site
|
||||
(`actions/mod.rs:1420`, `let tool_msg = ChatMessage::tool_result(tool_call.id.clone(),
|
||||
output);`) — `output` is passed through `squash::apply(&tool_name, &output)` before being
|
||||
wrapped. Runs before the result is ever archived or pushed into `msgs`, so compression is
|
||||
permanent and applies uniformly whether or not compaction ever triggers.
|
||||
|
||||
### `shaping.rs`
|
||||
|
||||
Unchanged behavior from today's `shortsend.rs` (hysteresis `should_shape`, 70%-budget
|
||||
newest-first retention, LLM summarization of dropped messages), moved as-is into this file
|
||||
and updated to source token counts from `tokens.rs` instead of its own heuristic.
|
||||
|
||||
### `window.rs`
|
||||
|
||||
```rust
|
||||
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize
|
||||
```
|
||||
|
||||
Replaces the three duplicated `model_roles.values().find(...).and_then(...).unwrap_or(...)`
|
||||
blocks in `Action::Compact`, `spawn_turn`, and `view/status.rs` (×2).
|
||||
|
||||
### `mod.rs`
|
||||
|
||||
No facade function — just `pub mod dedup; pub mod shaping; pub mod squash; pub mod tokens;
|
||||
pub mod window;`. `dedup`, `shaping`, and `tokens` are called directly from each call site
|
||||
(the auto-loop and `Action::Compact`), matching CLAUDE.md's "No DI — modules call ...
|
||||
directly" convention rather than introducing an orchestration layer that only one of the two
|
||||
callers would use generically (the auto-loop already needs per-stage control today — it
|
||||
inspects `should_shape` itself to decide whether to emit `TurnEvent::Compacted` — and would
|
||||
have to unpack a facade's result anyway).
|
||||
|
||||
## Data flow (per turn)
|
||||
|
||||
1. Tool executes → raw `output: String`.
|
||||
2. `squash::apply(tool_name, &output)` — compress if over the size floor (`read` exempted).
|
||||
3. Wrapped into `ChatMessage::tool_result(...)`, archived, pushed to `msgs`.
|
||||
4. Once per loop iteration: `dedup::collapse(&msgs)` (always) → sum
|
||||
`tokens::count_message_tokens` over the result → `shaping::should_shape` → conditionally
|
||||
`shaping::shape_messages`.
|
||||
5. Result pushed as `TurnEvent::Compacted` if dedup changed anything or shaping triggered,
|
||||
consumed on the main thread to update `SessionRuntime.messages`.
|
||||
|
||||
## Fixing the manual/auto asymmetry
|
||||
|
||||
`Action::Compact` (`actions/mod.rs:547`) currently runs synchronously inside `apply_action`
|
||||
and can't block on an LLM call. Fix: make it spawn a background `std::thread::spawn` — the
|
||||
same pattern `spawn_turn` already uses (`actions/mod.rs:694`) — that runs `dedup::collapse`
|
||||
then unconditionally `shaping::shape_messages(.., force=true, Some(&client))` and reports back
|
||||
via `TurnEvent::Compacted`, identical to the automatic path. The toast sequence becomes
|
||||
"Compacting…" immediately (optimistic, non-blocking) then "History compacted" when the
|
||||
`TurnEvent` arrives. This gives manual `/compact` real LLM summarization instead of always
|
||||
falling back to the placeholder.
|
||||
|
||||
## Concise mode (separate from the `context/` module)
|
||||
|
||||
- `Settings` (`src/model/settings.rs`) gains `pub concise_output: bool`, default `false`,
|
||||
with `#[serde(default)]` for backward-compatible deserialization of existing
|
||||
`settings.json` files (matching the existing `hive_mind_node_timeout_ms` precedent in the
|
||||
same file).
|
||||
- When `true`, `run_agent_turn`'s system-prompt assembly (`actions/mod.rs:930-936`) appends a
|
||||
fourth section to `system_text`: a terse-writing instruction (persona-prompt only, no
|
||||
algorithm — drop articles/filler/hedging/pleasantries, keep code/commands/error text
|
||||
byte-exact) with an explicit carve-out disabling terseness for destructive-operation
|
||||
confirmations and security-relevant warnings, mirroring caveman's own "Auto-Clarity"
|
||||
safety exception.
|
||||
- No UI toggle is in scope for this pass — confirmed no such mechanism exists today for any
|
||||
boolean `Settings` field (`review_enabled`, `session_archive_enabled`,
|
||||
`lsp_auto_provision` are all hand-edited in `settings.json`, same as this one will be).
|
||||
|
||||
## New dependency
|
||||
|
||||
`tiktoken-rs` — pure Rust, embedded BPE vocab (`cl100k_base`/`o200k_base`), no network calls
|
||||
at runtime, MIT/Apache-2.0 dual-licensed. Added to `Cargo.toml`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per repo convention, one per new file:
|
||||
|
||||
- `dedup.rs`: same tool+args → older result replaced; different args → no-op; mutating tool
|
||||
with identical args → both kept in full; unmatched `tool_call_id` (malformed history) →
|
||||
no panic, treated as unpaired.
|
||||
- `squash.rs`: JSON input under/over the size floor; JSON with long low-entropy string values
|
||||
gets them elided while short/UUID-shaped values survive; log input with error/warn lines
|
||||
keeps highest-scored lines and emits exactly one `[N lines omitted]` marker; generic text
|
||||
keeps first/last N lines.
|
||||
- `tokens.rs`: known-string token counts against fixed expected values; empty string → 0.
|
||||
- `shaping.rs`: port the behavioral cases implied by today's hysteresis logic (85% trigger
|
||||
when not previously shaped, 95% once shaped) plus budget-drop ordering.
|
||||
- `window.rs`: role match resolves to the role's `context_window`; no match falls back to
|
||||
`default_context_window`.
|
||||
|
||||
## Migration
|
||||
|
||||
- Delete `src/app/runtime/shortsend.rs`; all three call sites (`actions/mod.rs` auto-loop,
|
||||
`Action::Compact`, and the module path itself) updated to `context::`.
|
||||
- `view/status.rs` switches its live token display to `tokens::count_tokens`, so the status
|
||||
bar finally matches what compaction measures internally.
|
||||
Reference in new issue
Block a user