feat: remove obsolete design documents for clipboard OSC52, diff view, file mention, context compaction, and add development guide
This commit is contained in:
asepharyana
@@ -780,7 +780,7 @@ pub fn resolve_context_window(
|
||||
return max as usize;
|
||||
}
|
||||
}
|
||||
128_000
|
||||
256_000
|
||||
}
|
||||
|
||||
/// Count tokens using tiktoken, fall back to character estimation.
|
||||
|
||||
+104
-51
@@ -1,65 +1,118 @@
|
||||
# Architecture Overview
|
||||
# Arsitektur Sistem Zesdex
|
||||
|
||||
## System Layout
|
||||
## Gambaran Umum
|
||||
|
||||
Zesdex is an autonomous AI coding agent with a TUI — an LLM client wrapped in a tool-use harness with 37 built-in tools.
|
||||
Zesdex adalah autonomous AI coding agent berbasis TUI, dibangun dengan **Rust** menggunakan clean architecture berlapis. LLM client dibungkus dalam tool-use harness dengan 37+ built-in tools — file ops, git, shell, LSP, MCP, subagent orchestration, dan lainnya.
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Process Mode │
|
||||
│ Single-Process ─── Daemon (background) ─── Attach (client) │
|
||||
└──────────────────────────┬──────────────────────────────────┘
|
||||
│ IPC (Unix domain socket)
|
||||
zesdex/
|
||||
├── apps/
|
||||
│ ├── domain/ # Layer 1: Pure entities, traits, value objects
|
||||
│ ├── application/ # Layer 2: Use-case services
|
||||
│ ├── infrastructure/ # Layer 3: Semua I/O (LLM, DB, tools, MCP, LSP)
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # Ratatui terminal UI
|
||||
│ │ ├── api/ # Axum REST API
|
||||
│ │ ├── daemon/ # Unix socket daemon
|
||||
│ │ ├── ws/ # WebSocket server
|
||||
│ │ ├── grpc/ # gRPC server
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point & dispatcher
|
||||
│ └── bootstrap/ # Initial data seeder
|
||||
```
|
||||
|
||||
## Dependency Graph Antar Layer
|
||||
|
||||
```
|
||||
gateway/bootstrap
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ src/main.rs │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌────────────────┐ │
|
||||
│ │ Controller │──▶│ Runtime │──▶│ View │ │
|
||||
│ │ (input.rs) │ │ (actions.rs) │ │ (chat,status,…)│ │
|
||||
│ └──────────────┘ └──────┬───────┘ └────────────────┘ │
|
||||
│ │ │
|
||||
│ ┌───────▼────────┐ │
|
||||
│ │ Harness │ │
|
||||
│ │ (tool dispatch)│ │
|
||||
│ └───────┬────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────────────────┼─────────────────┐ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌─────────┐ ┌────────────┐ ┌───────────────┐ │
|
||||
│ │ Tools │ │ Subagents │ │ Workflow │ │
|
||||
│ │ (37x) │ │ (auto/gen) │ │ Engine │ │
|
||||
│ └─────────┘ └────────────┘ │ (hive_mind) │ │
|
||||
│ └───────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
interfaces/* (tui, api, daemon, ws, grpc, web)
|
||||
│
|
||||
▼
|
||||
infrastructure ─── implements ──▶ domain ports
|
||||
│
|
||||
▼
|
||||
application ─── depends on ──▶ domain traits
|
||||
│
|
||||
▼
|
||||
domain (zero framework deps: serde, chrono, uuid only)
|
||||
```
|
||||
|
||||
> **Aturan**: layer bawah tidak boleh tahu tentang layer atas. `domain` tidak import apapun dari `infrastructure` atau `interfaces`.
|
||||
|
||||
## Process Modes
|
||||
|
||||
| Mode | Description |
|
||||
|------|-------------|
|
||||
| **Single-process** | TUI + agent run in the same process. Simplest mode. |
|
||||
| **Daemon** | `--daemon` flag. Agent processes state in background; clients attach to render. |
|
||||
| **Attach** | `--attach <id>` flag. Connect to existing daemon with IPC. |
|
||||
| Mode | Flag | Keterangan |
|
||||
|------|------|------------|
|
||||
| **TUI** | *(default)* | TUI + agent loop dalam satu proses |
|
||||
| **Daemon** | `--daemon` | Agent berjalan di background via IPC socket |
|
||||
| **Attach** | `--attach <id>` | TUI terhubung ke daemon yang sedang berjalan |
|
||||
| **REST API** | `--api` | HTTP server (default port 8080) |
|
||||
| **WebSocket** | `--ws` | WS server (default port 8081) |
|
||||
| **gRPC** | `--grpc` | gRPC server (default port 50051) |
|
||||
| **Web** | `--web` | Static web frontend (default port 3000) |
|
||||
|
||||
In daemon mode, the daemon runs the full agent loop; clients are stateless renderers that sync via Unix domain sockets with diff-based state synchronization.
|
||||
## Alur Data (Single-Process Mode)
|
||||
|
||||
## Data Flow
|
||||
```
|
||||
Keyboard/Event
|
||||
│
|
||||
▼
|
||||
controller/input.rs: handle_key()
|
||||
│ returns Vec<Action>
|
||||
▼
|
||||
action.rs: apply_action(&mut AppStateRest)
|
||||
│ state dimutasi in-place
|
||||
├──▶ turn.rs: spawn_agent_turn() ──▶ background thread
|
||||
│ │
|
||||
│ ├─ LLM call (blocking reqwest)
|
||||
│ ├─ tool execution (Tool trait)
|
||||
│ └─ push TurnEvent ke queue
|
||||
│
|
||||
▼
|
||||
run.rs: run_loop_inner()
|
||||
│ drain TurnEvent setiap tick
|
||||
│ skip render jika dirty=false
|
||||
▼
|
||||
view/: draw frame ke terminal (ratatui)
|
||||
```
|
||||
|
||||
1. **Input** → `controller/input.rs` handles key events and autocomplete
|
||||
2. **Dispatch** → `app/runtime/actions/mod.rs` applies actions to state (`AppStateRest`)
|
||||
3. **LLM Stream** → `app/runtime/stream/mod.rs` parses SSE chunks into typed events
|
||||
4. **Tool Execution** → `app/harness.rs` gates and runs tool calls via the `Tool` trait
|
||||
5. **Rendering** → `view/` modules read `AppStateRest` and render via ratatui
|
||||
## File Kunci
|
||||
|
||||
## Key Files
|
||||
| File | Peran |
|
||||
|------|-------|
|
||||
| `apps/gateway/src/main.rs` | CLI entry point, parse args, dispatch ke mode |
|
||||
| `apps/interfaces/tui/src/state.rs` | `AppStateRest` — single source of truth state TUI |
|
||||
| `apps/interfaces/tui/src/action.rs` | `apply_action()` — satu-satunya tempat state dimutasi |
|
||||
| `apps/interfaces/tui/src/run.rs` | Event loop: render → poll → handle → tick |
|
||||
| `apps/interfaces/tui/src/turn.rs` | Spawn agent turn di background thread |
|
||||
| `apps/interfaces/tui/src/view/mod.rs` | Top-level render pipeline + `pre_render` hook |
|
||||
| `apps/infrastructure/src/llm/` | LLM client (streaming + non-streaming) |
|
||||
| `apps/infrastructure/src/tools/` | 37 tool implementations |
|
||||
| `apps/domain/src/core/` | Entity inti: `ChatMessage`, `Role`, `Store`, `Tool` trait |
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `src/main.rs` | Entry point, process mode dispatch, TUI init |
|
||||
| `src/app/state/rest.rs` | Single source-of-truth state struct |
|
||||
| `src/app/runtime/actions/mod.rs` | State reducer (`apply_action`) |
|
||||
| `src/app/runtime/stream/mod.rs` | SSE stream parser |
|
||||
| `src/app/harness.rs` | Tool harness with safety gating |
|
||||
| `src/app/workflow/hive_mind.rs` | Multi-agent orchestration |
|
||||
| `src/tool/mod.rs` | Tool trait + registry (37 tools) |
|
||||
| `src/view/mod.rs` | TUI render pipeline |
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
```
|
||||
┌──────────┐ Unix domain socket ┌──────────┐
|
||||
│ Client │ ◄──────────────────► │ Daemon │
|
||||
│ (TUI) │ [4-byte len][JSON] │ (agent) │
|
||||
└──────────┘ └──────────┘
|
||||
|
||||
Client ──Action──▶ Daemon (apply_action → state mutasi)
|
||||
Daemon ──StatePayload──▶ Client (render snapshot)
|
||||
```
|
||||
|
||||
## Lints & Kualitas Kode
|
||||
|
||||
Semua workspace crate menerapkan lint ketat di `Cargo.toml`:
|
||||
- `unused`, `dead_code`, `unreachable_code` → **deny**
|
||||
- `unused_imports`, `unused_variables`, `unused_mut` → **deny**
|
||||
- `clippy::all` + `clippy::pedantic` → **warn**
|
||||
|
||||
## Release Profile
|
||||
|
||||
`opt-level=3`, `lto="fat"`, `codegen-units=1`, `panic="abort"`, `strip="symbols"`
|
||||
|
||||
+108
-47
@@ -1,68 +1,129 @@
|
||||
# Backend Architecture
|
||||
# Backend & Infrastructure
|
||||
|
||||
## Provider Layer
|
||||
Semua implementasi I/O ada di `apps/infrastructure/src/`. Layer ini mengimplementasikan port/trait yang didefinisikan di `apps/domain/`.
|
||||
|
||||
The provider abstraction in `dto/provider/` and `service/provider.rs` wraps LLM API calls:
|
||||
## LLM Client (`infrastructure/src/llm/`)
|
||||
|
||||
- **Configuration**: `model/app_config.rs` loads Anthropic/OpenAI-compatible endpoint settings
|
||||
- **Authentication**: `service/oauth/` handles OAuth 2.0 with PKCE flow and token management
|
||||
- **Requests**: `dto/provider/request.rs` builds provider-agnostic request structs
|
||||
- **Responses**: `dto/provider/response.rs` parses streaming and non-streaming responses
|
||||
- **Token tracking**: `dto/provider/usage.rs` tracks token consumption
|
||||
Wrapper di atas provider OpenAI-compatible:
|
||||
|
||||
## IPC (Inter-Process Communication)
|
||||
- **`provider/`** — `LlmClient`: HTTP client dengan `reqwest::blocking` (sync) untuk agent turn, dan async streaming untuk preview
|
||||
- **Request/Response** — `ChatMessage`, `ChatCompletionRequest`, `ChatCompletionResponse` dengan support tool calls
|
||||
- **Streaming** — SSE event parser untuk streaming response
|
||||
- **Usage tracking** — `tokens_in`, `tokens_out`, `last_tokens_in`, `last_tokens_out` per panggilan
|
||||
- **Provider defaults** — DeepSeek v4 flash free via OpenCode AI proxy (default)
|
||||
|
||||
The daemon-client protocol in `src/ipc/`:
|
||||
|
||||
- **Transport**: Unix domain sockets
|
||||
- **Framing**: Length-prefixed frames with `serde_json` serialization (`ipc/frame.rs`)
|
||||
- **State Sync**: Full state push from daemon after each action (`ipc/snapshot.rs`); diff-based updates for efficiency (`ipc/diff.rs`)
|
||||
- **Protocol**: `ipc/protocol.rs` defines message types (Action, StateSnapshot, etc.)
|
||||
|
||||
Flow:
|
||||
```
|
||||
Client ──Action──▶ Daemon ──apply_action()──▶ State mutated
|
||||
│
|
||||
└──StatePayload──▶ Client (render)
|
||||
```rust
|
||||
// Contoh penggunaan di turn.rs
|
||||
let result = client.chat_with_tools_non_streaming(
|
||||
&mut messages,
|
||||
Some(tool_defs),
|
||||
Some(4096), // max_tokens
|
||||
Some(0.7), // temperature
|
||||
None, // abort flag
|
||||
);
|
||||
```
|
||||
|
||||
## Workflow Engine
|
||||
## Tool System (`infrastructure/src/tools/`)
|
||||
|
||||
Located in `src/app/workflow/`:
|
||||
37 tool yang mengimplementasikan trait `Tool` dari domain:
|
||||
|
||||
- **Script DSL** (`engine.rs`): Executes the workflow script language (agent/parallel/pipeline/phase). Supports subagent spawning with schema-validated output, concurrency limiting, and budget tracking.
|
||||
- **Hive Mind** (`hive_mind.rs`): Core Intelligence spawns a CognitiveCyclePlan — ordered cycles of parallel processing nodes. Each node has a directive and access tier (`read`/`write`/`full`). Node outputs merge into a shared collective state in real time. Final consensus synthesis completes the convergence.
|
||||
- **Docs** (`docs.rs`): Deterministic (not LLM) convergence writer — records every node's output + final consensus to `docs/runs/`.
|
||||
```rust
|
||||
pub trait Tool: Send + Sync {
|
||||
fn name(&self) -> &'static str;
|
||||
fn description(&self) -> &'static str;
|
||||
fn parameters(&self) -> Value; // JSON Schema
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String>;
|
||||
}
|
||||
```
|
||||
|
||||
## MCP (Model Context Protocol)
|
||||
### Kategori Tool
|
||||
|
||||
`src/app/mcp/manager.rs` manages MCP client connections:
|
||||
| Kategori | Tools |
|
||||
|----------|-------|
|
||||
| **File System** | `read`, `write`, `edit`, `delete`, `dir_list`, `dir_cache_update` |
|
||||
| **Shell** | `bash`, `bash_interactive`, `bash_kill`, `bash_output` |
|
||||
| **Git** | `git_operator`, `git_cred`, `git_worktree` |
|
||||
| **Search** | `search`, `grep`, `glob`, `semantic_search` |
|
||||
| **LSP** | `lsp_connect`, `lsp_hover`, `lsp_completion`, `lsp_definition`, `lsp_references`, `lsp_diagnostics`, `lsp_disconnect` |
|
||||
| **Memory** | `remember`, `recall`, `forget` |
|
||||
| **Workflow** | `spawn_agents`, `spawn_pipeline`, `plan`, `sequential_think`, `hive_mind` |
|
||||
| **Utility** | `todo_write`, `todo_finish`, `pong`, `cd` |
|
||||
| **Background** | `bash_bg_cancel`, `bash_bg_status`, `bash_bg_list` |
|
||||
|
||||
- Uses the `rmcp` crate for the MCP protocol
|
||||
- Supports stdio-based transport (child process) and streamable HTTP
|
||||
- Tool discovery via `list_tools()` and dynamic tool registration
|
||||
`ToolCtx` berisi:
|
||||
- `session_dir: PathBuf` — direktori sesi aktif
|
||||
- `workspaces: Vec<PathBuf>` — root workspace yang dibuka
|
||||
|
||||
## LSP Integration
|
||||
## Background Shell Jobs (`infrastructure/src/bgbash/`)
|
||||
|
||||
`src/app/lsp/` provides Language Server Protocol support:
|
||||
Manajemen proses shell jangka panjang:
|
||||
- **Spawn** dengan Unix process groups (untuk kill seluruh tree)
|
||||
- **Output buffering** — collect stdout/stderr secara async
|
||||
- **Cancel/status/list** — kontrol via tool calls
|
||||
- **Progress monitoring** — track state: `Running`, `Completed`, `Failed`
|
||||
|
||||
- **Auto-provisioner** (`provisioner.rs`): Detects and starts LSP servers for Rust, TypeScript, Python, Go, and other languages
|
||||
- **Client** (`client.rs`): JSON-RPC-based LSP client with typed notifications
|
||||
- **Tools** (`tool/lsp/mod.rs`): 7 LSP tools (connect, hover, completion, definition, references, diagnostics, disconnect)
|
||||
## MCP Manager (`infrastructure/src/mcp/`)
|
||||
|
||||
## Background Bash
|
||||
Integrasi **Model Context Protocol**:
|
||||
- Menggunakan crate `rmcp` (v2.2)
|
||||
- Transport: **stdio** (child process) dan **streamable HTTP**
|
||||
- `list_tools()` → tool discovery otomatis → registrasi ke tool harness
|
||||
- Persistent connection management
|
||||
|
||||
`src/app/bgbash/` manages long-running shell jobs:
|
||||
## LSP Integration (`infrastructure/src/lsp/`)
|
||||
|
||||
- **Control** (`control.rs`): Job lifecycle management (spawn, signal, terminate) using Unix process groups
|
||||
- **Job** (`job.rs`): Individual job state tracking with output buffering and progress monitoring
|
||||
Integrasi **Language Server Protocol**:
|
||||
- **Auto-provisioner** — deteksi bahasa dari file extension, start LSP server yang sesuai
|
||||
- Mendukung: `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dan lainnya
|
||||
- **JSON-RPC client** — typed notifications + request/response
|
||||
- 7 tools LSP yang diekspose ke LLM
|
||||
|
||||
## Review System
|
||||
## Persistence (`infrastructure/src/persistence/`)
|
||||
|
||||
`src/app/subagent/auto.rs` spawns background reviews:
|
||||
### SQLite Message Log
|
||||
|
||||
- Quick review after every edit
|
||||
- Background test generation
|
||||
- Architecture review
|
||||
- Security review
|
||||
- All retry once on failure, escalate to blocking error if retry also fails
|
||||
Session database dengan `rusqlite` (bundled):
|
||||
- Per-session isolation
|
||||
- Table: `messages`, `sessions`
|
||||
- CRUD, query/filter, blob storage
|
||||
|
||||
### Settings Repository
|
||||
|
||||
`JsonSettingsRepository` — simpan/load `Settings` dari `settings.json`:
|
||||
- `provider`: nama provider LLM
|
||||
- `model`: model ID
|
||||
- `max_tokens`: override context window (default: 256k jika tidak diset)
|
||||
- `temperature`, `concise_output`, dll
|
||||
|
||||
### Memory Files
|
||||
|
||||
File-based memory di `~/.local/share/zesdex/memories/`:
|
||||
- Setiap memory = satu `.md` dengan frontmatter YAML
|
||||
- Fields: `name`, `description`, `type` (`user`/`feedback`/`project`/`reference`)
|
||||
- Index di `MEMORY.md`
|
||||
|
||||
## Session Management (`infrastructure/src/session/`)
|
||||
|
||||
- Setiap sesi memiliki UUID, direktori sendiri di `sessions/<uuid>/`
|
||||
- `.lock` file untuk cegah concurrent access
|
||||
- `session.json` — metadata (waktu mulai, workspace, model yang dipakai)
|
||||
|
||||
## Utils
|
||||
|
||||
| Util | Fungsi |
|
||||
|------|--------|
|
||||
| `utils::write_osc52` | Tulis teks ke clipboard via OSC52 escape sequence |
|
||||
| `Toast` / `ToastKind` | Notifikasi sementara (Success/Warning/Error/Info/Lesson) |
|
||||
| `TurnEvent` | Event dari background agent ke TUI (queue-based) |
|
||||
| `DirCache` | Cache async listing direktori untuk `@mention` autocomplete |
|
||||
| `MentionIndex` | Index file workspace untuk fuzzy autocomplete |
|
||||
| `SessionRuntime` | Runtime state: messages history, usage stats, session start time |
|
||||
|
||||
## OAuth 2.0
|
||||
|
||||
Flow PKCE untuk provider LLM:
|
||||
1. Generate code verifier + challenge
|
||||
2. Open browser ke authorization URL
|
||||
3. Start localhost HTTP server untuk tangkap redirect
|
||||
4. Exchange code → access + refresh token
|
||||
5. Simpan token di settings
|
||||
|
||||
+114
-65
@@ -1,89 +1,138 @@
|
||||
# Data Architecture
|
||||
# Data & Persistence
|
||||
|
||||
## State Model
|
||||
## State Runtime (TUI)
|
||||
|
||||
The single source of truth is `AppStateRest` (`src/app/state/rest.rs`):
|
||||
State TUI yang berjalan di memori adalah `AppStateRest` (`apps/interfaces/tui/src/state.rs`).
|
||||
|
||||
```
|
||||
AppStateRest
|
||||
├── session: SessionRuntime (hive_mind state, convergence flag)
|
||||
├── runtime: RuntimeState (mode, provider status)
|
||||
├── chat: ChatState (messages, scroll)
|
||||
├── input: InputState (text, cursor, autocomplete)
|
||||
├── settings: Settings (provider, model, temperature, concise_output)
|
||||
├── config: AppConfig (endpoints, credentials)
|
||||
├── scroll: ScrollState (per-panel offset)
|
||||
├── diff: DiffState (edit review)
|
||||
├── tools: Vec with outputs
|
||||
├── statusline, sidebar, etc.
|
||||
└── toasts: pending notifications
|
||||
### TranscriptCache
|
||||
|
||||
```rust
|
||||
pub struct TranscriptCache {
|
||||
pub messages: VecDeque<ChatMessageDisplay>, // O(1) eviction
|
||||
pub max_lines: usize, // default: 200
|
||||
pub dirty: bool, // perlu rebuild cache?
|
||||
}
|
||||
```
|
||||
|
||||
**Mutation rules** (per CLAUDE.md):
|
||||
- Mutated in-place from exactly two locations: `actions/mod.rs` (apply_action) and `controller/input.rs` (key handlers)
|
||||
- Read-only from every other module
|
||||
- No generic update function — direct field mutation only
|
||||
Saat `dirty=true`, `pre_render_chat()` rebuild `display_lines_cache` (render markdown semua pesan) sebelum frame berikutnya.
|
||||
|
||||
## Persistence
|
||||
### SessionRuntime
|
||||
|
||||
### SQLite Message Log (`src/model/msglog/`)
|
||||
```rust
|
||||
pub struct SessionRuntime {
|
||||
pub messages: Vec<ChatMessage>, // history untuk LLM context
|
||||
pub usage: UsageStats, // token counting akumulasi
|
||||
pub session_start: i64, // unix ms saat sesi dimulai
|
||||
pub hive_mind_converged: bool, // flag selesai hive mind
|
||||
}
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `schema.rs` | Table definitions (messages, sessions) |
|
||||
| `mod.rs` | CRUD operations |
|
||||
| `query.rs` | Query helpers (search, filter) |
|
||||
| `blobs.rs` | Large message blob storage |
|
||||
| `summary.rs` | Conversation summary cache |
|
||||
## Context Window
|
||||
|
||||
Schema uses `rusqlite` (bundled) with per-session isolation — each session gets its own database.
|
||||
`resolve_context_window()` di `state.rs`:
|
||||
1. Ambil `settings.max_tokens` jika ada dan > 0
|
||||
2. Fallback ke **256.000 token** (default)
|
||||
|
||||
### Memory System (`src/model/memory.rs`)
|
||||
Token dihitung lazily via `count_tokens()` (tiktoken `cl100k_base`, fallback `len/4`), di-cache di `AppStateRest::cached_token_count`, hanya dihitung ulang saat `token_count_dirty=true`.
|
||||
|
||||
File-based memory stored under `~/.claude/projects/<project>/memory/`:
|
||||
## Persistence di Disk
|
||||
|
||||
- Each memory is one markdown file with frontmatter (name, description, type)
|
||||
- Types: `user`, `feedback`, `project`, `reference`
|
||||
- Memory index in MEMORY.md
|
||||
- Export/import for lesson sharing
|
||||
- PID-file session lock prevents concurrent access
|
||||
Semua data disimpan di **platform data directory**:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
### Settings & Config (`src/model/`)
|
||||
```
|
||||
~/.local/share/zesdex/
|
||||
├── settings.json # User settings (provider, model, max_tokens, dll)
|
||||
├── sessions/
|
||||
│ └── <uuid>/
|
||||
│ ├── session.json # Metadata sesi
|
||||
│ ├── messages.jsonl # Message log (append-only)
|
||||
│ └── .lock # Lock file (cegah concurrent access)
|
||||
├── memories/
|
||||
│ └── *.md # Memory files dengan frontmatter
|
||||
├── lessons/
|
||||
│ └── *.md # Lesson files (output dari learning system)
|
||||
└── worktrees/ # Git worktree per sesi (isolasi perubahan)
|
||||
```
|
||||
|
||||
| File | Purpose |
|
||||
|------|---------|
|
||||
| `settings.rs` | Serialized user preferences (provider, model, theme) |
|
||||
| `app_config.rs` | Provider endpoints, API key resolution from env |
|
||||
| `session.rs` | Current session metadata |
|
||||
| `conversation.rs` | In-memory conversation state |
|
||||
| `editlog.rs` | Append-only JSONL edit audit trail |
|
||||
|
||||
### Edit Log
|
||||
|
||||
`src/model/editlog.rs` records every file mutation:
|
||||
### settings.json
|
||||
|
||||
```json
|
||||
{"ts": 123, "tool": "edit", "path": "src/main.rs",
|
||||
"reason": "fix bug", "content_sha256": "abc123",
|
||||
"bytes_delta": 15, "origin": "chat", "session_id": "sess-1"}
|
||||
{
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"max_tokens": 256000,
|
||||
"temperature": 0.7,
|
||||
"concise_output": false
|
||||
}
|
||||
```
|
||||
|
||||
Max 5000 entries held in memory before pruning oldest.
|
||||
Diload via `JsonSettingsRepository::load()`, disimpan kembali saat TUI keluar (`state.save_settings()`).
|
||||
|
||||
## Context Management (`src/app/runtime/context/`)
|
||||
### Memory Files
|
||||
|
||||
| Module | Purpose |
|
||||
|--------|---------|
|
||||
| `tokens.rs` | Token counting via `tiktoken-rs` |
|
||||
| `window.rs` | Token window resolution (fit within model context) |
|
||||
| `dedup.rs` | Deduplication of repeated tool outputs |
|
||||
| `squash.rs` | Compression of large JSON tool results |
|
||||
| `shaping.rs` | Message dropping when context exceeds limits |
|
||||
Format markdown dengan YAML frontmatter:
|
||||
```markdown
|
||||
---
|
||||
name: prefer-early-return
|
||||
type: feedback
|
||||
description: Selalu gunakan early return untuk mengurangi nesting
|
||||
---
|
||||
|
||||
## IPC Data Flow
|
||||
Ketika menulis fungsi, gunakan early return/guard clauses daripada deep nesting.
|
||||
```
|
||||
|
||||
Types: `user`, `feedback`, `project`, `reference`
|
||||
|
||||
### Lesson Files
|
||||
|
||||
Hasil dari learning system, disimpan di `lessons/`:
|
||||
- Satu file per lesson
|
||||
- Plain markdown, dibaca oleh overlay `Learning`
|
||||
- Bisa di-accept/reject dari TUI
|
||||
|
||||
## SQLite (Message Log)
|
||||
|
||||
`rusqlite` dengan fitur `bundled` (tidak perlu install SQLite terpisah):
|
||||
- Per-session database di `sessions/<uuid>/messages.db`
|
||||
- Table `messages`: `id`, `session_id`, `role`, `content`, `timestamp`, `tokens`
|
||||
- Table `sessions`: `id`, `metadata`, `created_at`
|
||||
|
||||
## IPC Protocol (Daemon Mode)
|
||||
|
||||
Daemon dan client berkomunikasi via **Unix domain socket**:
|
||||
|
||||
```
|
||||
Daemon State ──diff──▶ serialize ──frame──▶ socket ──▶ Client
|
||||
│
|
||||
Client State ◀── apply_diff ◀── deserialize ◀──── socket ─┘
|
||||
~/.local/share/zesdex/daemon.sock
|
||||
```
|
||||
|
||||
Frame format:
|
||||
```
|
||||
[4 bytes BE: payload length][JSON payload]
|
||||
```
|
||||
|
||||
Message types:
|
||||
- `Action` — client kirim aksi ke daemon
|
||||
- `StateSnapshot` — daemon kirim snapshot state ke client
|
||||
- `Ping/Pong` — keepalive
|
||||
|
||||
## Edit History
|
||||
|
||||
Agent mencatat setiap mutasi file ke event log internal:
|
||||
- Tool `write`/`edit`/`delete` merekam path, bytes delta, timestamp
|
||||
- Digunakan oleh subagent review untuk audit trail
|
||||
|
||||
## TurnEvent Queue
|
||||
|
||||
Agent berjalan di background thread dan mengirim events ke TUI via `Arc<Mutex<VecDeque<TurnEvent>>>`:
|
||||
|
||||
| Event | Payload | Efek di TUI |
|
||||
|-------|---------|-------------|
|
||||
| `AssistantMessage(msg)` | `ChatMessage` | Push ke transcript |
|
||||
| `ToolResult { output, .. }` | String | Push sebagai tool message |
|
||||
| `Usage { tokens_in, tokens_out }` | u64, u64 | Update `usage` stats |
|
||||
| `Error(msg)` | String | Toast error |
|
||||
| `Compacted(msgs)` | `Vec<ChatMessage>` | Update `session_runtime.messages` |
|
||||
| `SystemNote { kind, message }` | String | Push ke transcript |
|
||||
| `Done` | — | Set `turn_in_flight_flag = false` |
|
||||
|
||||
+121
-80
@@ -1,99 +1,140 @@
|
||||
# Dependencies
|
||||
|
||||
## Rust Crates (30+ direct)
|
||||
Semua dependency dideklarasikan di `[workspace.dependencies]` dalam `Cargo.toml` root, lalu di-*inherit* oleh setiap crate anggota.
|
||||
|
||||
### Core Framework
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation |
|
||||
| `tokio` | 1 | Async runtime (multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
## Crate per Layer
|
||||
|
||||
### HTTP & Networking
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `reqwest` | 0.13 | HTTP client (JSON, streaming, native-tls-vendored, form) |
|
||||
### Domain (`apps/domain`)
|
||||
|
||||
Hanya boleh pakai dependency yang tidak membawa I/O:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Serialisasi (derive) |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `chrono` | 0.4 | Tanggal/waktu |
|
||||
| `uuid` | 1 | UUID v4/v5 |
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `thiserror` | 1 | Derive error types |
|
||||
|
||||
### Infrastructure (`apps/infrastructure`)
|
||||
|
||||
Semua I/O, LLM, DB, tools:
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `tokio` | 1 | Async runtime (rt-multi-thread, macros, sync, time, net, io-util, signal) |
|
||||
| `reqwest` | 0.13 | HTTP client (json, stream, blocking, native-tls-vendored, form) |
|
||||
| `rusqlite` | 0.40 | SQLite (bundled — tidak perlu install sistem) |
|
||||
| `rmcp` | 2.2 | MCP client (child-process, streamable HTTP) |
|
||||
| `webbrowser` | 1 | Open URLs in browser |
|
||||
| `tiktoken-rs` | 0.12 | Token counting (OpenAI cl100k) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting (default-fancy) |
|
||||
| `ignore` | 0.4 | File walking dengan `.gitignore` support |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed direktori ke binary |
|
||||
| `infer` | 0.19 | Deteksi tipe file dari byte signature |
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (untuk `@mention` autocomplete) |
|
||||
| `webbrowser` | 1 | Buka URL di browser (OAuth) |
|
||||
| `url` | 2 | URL parsing |
|
||||
| `percent-encoding` | 2 | URL encoding |
|
||||
|
||||
### HTML/Markdown
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `dom_smoothie` | 0.18.0 | HTML DOM manipulation |
|
||||
| `fast_html2md` | 0.0.62 | HTML-to-Markdown conversion |
|
||||
| `scraper` | 0.27.0 | HTML parsing/selecting |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing (no default features) |
|
||||
|
||||
### Serialization
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `serde` | 1 | Serialization framework |
|
||||
| `serde_json` | 1 | JSON serialization |
|
||||
| `serde_yaml_ng` | 0.10 | YAML serialization |
|
||||
|
||||
### Storage & Files
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `rusqlite` | 0.40 | SQLite (bundled) |
|
||||
| `ignore` | 0.4 | `.gitignore`-aware file walking |
|
||||
| `globset` | 0.4 | Glob pattern matching |
|
||||
| `include_dir` | 0.7 | Embed directory contents in binary |
|
||||
| `infer` | 0.19 | File type detection |
|
||||
| `dirs` | 6 | Standard OS directories |
|
||||
|
||||
### Text & Search
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `regex` | 1 | Regular expressions |
|
||||
| `nucleo-matcher` | 0.3 | Fuzzy matching (for @mention autocomplete) |
|
||||
| `similar` | 3 | Diff computation |
|
||||
| `syntect` | 5 | Syntax highlighting |
|
||||
| `tiktoken-rs` | 0.12 | OpenAI token counting |
|
||||
|
||||
### Cryptography & Encoding
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `base64` | 0.22 | Base64 encoding |
|
||||
| `sha2` | 0.11 | SHA-256 hashing |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `uuid` | 1 | UUID generation (v4, v5) |
|
||||
| `libc` | 0.2 | Raw C FFI bindings |
|
||||
|
||||
### Error Handling & Logging
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `anyhow` | 1 | Error handling |
|
||||
| `tracing` | 0.1 | Structured logging |
|
||||
| `tracing-subscriber` | 0.3 | Log subscriber with env-filter |
|
||||
| `chrono` | 0.4 | Date/time with serde |
|
||||
|
||||
### Other
|
||||
| Crate | Version | Purpose |
|
||||
|-------|---------|---------|
|
||||
| `fast_html2md` | 0.0.62 | HTML → Markdown |
|
||||
| `scraper` | 0.27.0 | HTML parsing + CSS selector |
|
||||
| `pulldown-cmark` | 0.13 | Markdown parsing |
|
||||
| `lsp-types` | 0.97 | LSP protocol types |
|
||||
| `futures-util` | 0.3 | Async stream combinators |
|
||||
| `libc` | 0.2 | Raw C FFI (Unix process groups) |
|
||||
|
||||
## External Services
|
||||
### TUI (`apps/interfaces/tui`)
|
||||
|
||||
| Service | Purpose |
|
||||
|---------|---------|
|
||||
| **Anthropic API** | Primary LLM provider |
|
||||
| **OpenAI API** | Alternative LLM provider (including OAuth) |
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `ratatui` | 0.30.2 | TUI framework |
|
||||
| `crossterm` | 0.29 | Terminal manipulation (raw mode, events, mouse) |
|
||||
| `base64` | 0.22 | Base64 (OSC52 clipboard) |
|
||||
| `sha2` | 0.11 | SHA-256 |
|
||||
| `hex` | 0.4 | Hex encoding |
|
||||
| `dirs` | 6 | Platform data directory |
|
||||
|
||||
### API (`apps/interfaces/api`)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `axum` | 0.8 | HTTP server framework (macros) |
|
||||
| `tower` | 0.5 | Middleware layer |
|
||||
| `tower-http` | 0.6 | CORS, body limit |
|
||||
| `argon2` | 0.5 | Password hashing |
|
||||
| `jsonwebtoken` | 9 | JWT (HS256) |
|
||||
| `clap` | 4 | CLI argument parsing (derive) |
|
||||
| `rand_core` | 0.6 | Secure random (getrandom) |
|
||||
|
||||
### Serialization (semua layer)
|
||||
|
||||
| Crate | Versi | Fungsi |
|
||||
|-------|-------|--------|
|
||||
| `serde` | 1 | Framework serialisasi |
|
||||
| `serde_json` | 1 | JSON |
|
||||
| `serde_yaml_ng` | 0.10 | YAML (frontmatter memory files) |
|
||||
|
||||
## Layanan Eksternal
|
||||
|
||||
| Layanan | Fungsi |
|
||||
|---------|--------|
|
||||
| **LLM Provider** | OpenAI/Anthropic-compatible API (default: OpenCode AI / DeepSeek) |
|
||||
| **MCP Servers** | Tool servers eksternal via stdio atau HTTP |
|
||||
| **LSP Servers** | `rust-analyzer`, `typescript-language-server`, `pyright`, `gopls`, dll |
|
||||
| **GitHub** | Release artifacts via semantic-release CI |
|
||||
| **MCP Servers** | External tool servers (stdio or HTTP) |
|
||||
| **LSP Servers** | Language servers (rust-analyzer, TypeScript, Pyright, gopls, etc.) |
|
||||
|
||||
## Build Configuration
|
||||
## Build & CI
|
||||
|
||||
### Compiler Lints (`.cargo/config.toml`)
|
||||
All unused code, dead code, and deprecation warnings promoted to errors:
|
||||
`-W unused`, `-W dead_code`, `-W unreachable_code`, `-D warnings`
|
||||
### Compiler Lints (`Cargo.toml` workspace)
|
||||
|
||||
```toml
|
||||
[workspace.lints.rust]
|
||||
unused = "deny"
|
||||
dead_code = "deny"
|
||||
unreachable_code = "deny"
|
||||
unused_imports = "deny"
|
||||
unused_variables = "deny"
|
||||
unused_mut = "deny"
|
||||
unused_must_use = "deny"
|
||||
deprecated = "deny"
|
||||
trivial_casts = "deny"
|
||||
trivial_numeric_casts = "deny"
|
||||
|
||||
[workspace.lints.clippy]
|
||||
all = { level = "warn", priority = -1 }
|
||||
pedantic = { level = "warn", priority = -2 }
|
||||
```
|
||||
|
||||
### Release Profile
|
||||
`opt-level=3`, LTO="fat", `codegen-units=1`, `panic="abort"`, `strip="symbols"`, `overflow-checks=true`
|
||||
|
||||
```toml
|
||||
[profile.release]
|
||||
opt-level = 3
|
||||
lto = "fat"
|
||||
codegen-units = 1
|
||||
panic = "abort"
|
||||
strip = "symbols"
|
||||
overflow-checks = true
|
||||
```
|
||||
|
||||
### CI/CD
|
||||
- **CI**: cargo build + test + clippy on every push
|
||||
- **Release**: semantic-release with changelog generation, Cargo.toml version bump, GitHub artifact upload
|
||||
|
||||
- **CI**: `cargo build` + `cargo test` + `cargo clippy --all-targets` pada setiap push
|
||||
- **Release**: semantic-release — auto changelog, Cargo.toml version bump, GitHub artifact upload
|
||||
- **Versioning**: `v1.17.0` (saat ini) — mengikuti semver dari commit messages
|
||||
|
||||
## Feature Flags Penting
|
||||
|
||||
| Crate | Feature | Alasan |
|
||||
|-------|---------|--------|
|
||||
| `rusqlite` | `bundled` | SQLite statically linked — tidak perlu install sistem |
|
||||
| `reqwest` | `blocking` | Sync HTTP untuk agent turn di blocking thread |
|
||||
| `reqwest` | `native-tls-vendored` | TLS tanpa dependency sistem |
|
||||
| `tokio` | `rt-multi-thread` | Async runtime multi-thread |
|
||||
| `pulldown-cmark` | *(no default)* | Tidak include semua fitur berat |
|
||||
| `syntect` | `default-fancy` | Syntax highlighting penuh |
|
||||
| `rmcp` | `transport-child-process` + `transport-streamable-http-client-reqwest` | MCP via stdio dan HTTP |
|
||||
|
||||
@@ -0,0 +1,175 @@
|
||||
# Panduan Development
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# Build semua crate
|
||||
cargo build
|
||||
|
||||
# Jalankan TUI (default)
|
||||
cargo run
|
||||
|
||||
# Jalankan dengan log debug
|
||||
RUST_LOG=debug cargo run
|
||||
|
||||
# Jalankan REST API
|
||||
cargo run -- --api --api-port 8080
|
||||
|
||||
# Build release
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
## Struktur Workspace
|
||||
|
||||
```
|
||||
zesdex/
|
||||
├── Cargo.toml # Workspace root, semua dependency terpusat di sini
|
||||
├── Cargo.lock # Lock file (commit ini!)
|
||||
├── apps/
|
||||
│ ├── domain/ # Pure domain (tidak ada I/O)
|
||||
│ ├── application/ # Use-case services
|
||||
│ ├── infrastructure/ # Semua implementasi I/O
|
||||
│ ├── interfaces/
|
||||
│ │ ├── tui/ # TUI — fokus pengembangan utama
|
||||
│ │ ├── api/ # REST API (Axum)
|
||||
│ │ ├── daemon/ # Daemon mode
|
||||
│ │ ├── ws/ # WebSocket
|
||||
│ │ ├── grpc/ # gRPC
|
||||
│ │ └── web/ # Web frontend
|
||||
│ ├── gateway/ # CLI entry point
|
||||
│ └── bootstrap/ # Seeder
|
||||
└── docs/
|
||||
└── CODEMAPS/ # Dokumentasi ini
|
||||
```
|
||||
|
||||
## Menambah Tool Baru
|
||||
|
||||
1. Buat file baru di `apps/infrastructure/src/tools/<nama>.rs`
|
||||
2. Implement trait `Tool`:
|
||||
|
||||
```rust
|
||||
use zesdex_domain::core::tool_call::Tool;
|
||||
use anyhow::Result;
|
||||
use serde_json::Value;
|
||||
|
||||
pub struct MyTool;
|
||||
|
||||
impl Tool for MyTool {
|
||||
fn name(&self) -> &'static str { "my_tool" }
|
||||
fn description(&self) -> &'static str { "Deskripsi untuk LLM" }
|
||||
fn parameters(&self) -> Value {
|
||||
serde_json::json!({
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"param": { "type": "string", "description": "..." }
|
||||
},
|
||||
"required": ["param"]
|
||||
})
|
||||
}
|
||||
fn run(&self, ctx: &ToolCtx, args: &Value) -> Result<String> {
|
||||
let param = args["param"].as_str().unwrap_or("");
|
||||
Ok(format!("Result: {param}"))
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Daftarkan di `apps/infrastructure/src/tools/mod.rs`:
|
||||
|
||||
```rust
|
||||
pub fn all_tools() -> Vec<Box<dyn Tool>> {
|
||||
vec![
|
||||
// ... tools lain ...
|
||||
Box::new(my_tool::MyTool),
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Action TUI Baru
|
||||
|
||||
1. Tambah variant ke `enum Action` di `apps/interfaces/tui/src/action.rs`
|
||||
2. Tangani di `apply_action()` match block yang sama
|
||||
3. Emit dari `controller/input.rs::handle_key()`
|
||||
|
||||
```rust
|
||||
// action.rs
|
||||
pub enum Action {
|
||||
// ... existing ...
|
||||
MyNewAction { data: String },
|
||||
}
|
||||
|
||||
// dalam apply_action:
|
||||
Action::MyNewAction { data } => {
|
||||
state.some_field = data;
|
||||
state.mark_dirty();
|
||||
}
|
||||
```
|
||||
|
||||
## Menambah Overlay Baru
|
||||
|
||||
1. Buat file `apps/interfaces/tui/src/view/overlays/<nama>.rs`
|
||||
2. Tambah variant ke `enum Overlay` di `state.rs`
|
||||
3. Tambah entry di `overlays/mod.rs::render_overlay()`
|
||||
4. Implement `pub fn render(frame, area, block, state)` di file baru
|
||||
|
||||
## Linting & Testing
|
||||
|
||||
```bash
|
||||
# Cek semua warnings/errors
|
||||
cargo clippy --all-targets
|
||||
|
||||
# Run tests
|
||||
cargo test
|
||||
|
||||
# Test satu crate saja
|
||||
cargo test -p zesdex-tui
|
||||
|
||||
# Check tanpa build (cepat)
|
||||
cargo check --all
|
||||
```
|
||||
|
||||
> **Penting**: Workspace ini menggunakan `deny` untuk hampir semua lint.
|
||||
> Kode harus compile bersih tanpa warning apapun.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Fungsi |
|
||||
|----------|--------|
|
||||
| `RUST_LOG` | Log level (`debug`, `info`, `warn`, `error`) |
|
||||
| `OPENAI_API_KEY` | API key LLM (jika tidak diset via settings) |
|
||||
| `ANTHROPIC_API_KEY` | API key Anthropic |
|
||||
| `ZESDEX_DATA_DIR` | Override direktori data (default: platform standard) |
|
||||
|
||||
## Data Directory
|
||||
|
||||
Saat development, data disimpan di:
|
||||
- **Linux**: `~/.local/share/zesdex/`
|
||||
- **macOS**: `~/Library/Application Support/zesdex/`
|
||||
|
||||
Untuk reset bersih:
|
||||
```bash
|
||||
rm -rf ~/.local/share/zesdex/
|
||||
```
|
||||
|
||||
## Konvensi Kode
|
||||
|
||||
- **Tidak ada `unwrap()`** di kode produksi — gunakan `?` atau `unwrap_or_default()`
|
||||
- **State hanya dimutasi dari `apply_action()`** — jangan mutasi `AppStateRest` dari view
|
||||
- **View functions bersifat read-only** — signature `fn draw(frame: &mut Frame, state: &AppStateRest)`
|
||||
- **Cache mahal dikomputasi sekali** — gunakan flag `dirty` dan `pre_render` pattern
|
||||
- **Semua string ke LLM harus deskriptif** — nama tool dan deskripsinya penting untuk LLM context
|
||||
|
||||
## Release
|
||||
|
||||
Release dilakukan via git tag semantic-release:
|
||||
|
||||
```bash
|
||||
git commit -m "feat: tambah fitur baru" # bumps minor
|
||||
git commit -m "fix: perbaiki bug" # bumps patch
|
||||
git commit -m "feat!: breaking change" # bumps major
|
||||
```
|
||||
|
||||
CI akan otomatis:
|
||||
1. Bump versi di `Cargo.toml`
|
||||
2. Generate `CHANGELOG.md`
|
||||
3. Build release binary
|
||||
4. Upload ke GitHub Releases
|
||||
+154
-67
@@ -1,79 +1,166 @@
|
||||
# Frontend (TUI) Architecture
|
||||
# TUI (Terminal User Interface)
|
||||
|
||||
## Render Pipeline
|
||||
Dibangun di atas **ratatui** + **crossterm**. Kode ada di `apps/interfaces/tui/src/`.
|
||||
|
||||
The TUI is built with [ratatui](https://github.com/ratatui-org/ratatui) and [crossterm](https://github.com/crossterm-rs/crossterm).
|
||||
## Struktur Source
|
||||
|
||||
```
|
||||
Timer tick
|
||||
apps/interfaces/tui/src/
|
||||
├── run.rs # Event loop utama
|
||||
├── state.rs # AppStateRest — single source of truth
|
||||
├── action.rs # apply_action(): satu-satunya mutator state
|
||||
├── turn.rs # Spawn agent turn di background thread
|
||||
├── lib.rs # Re-export publik
|
||||
├── controller/
|
||||
│ ├── input.rs # Key handler → Vec<Action>
|
||||
│ └── command.rs # Slash command parser
|
||||
├── view/
|
||||
│ ├── mod.rs # Layout + pre_render() + draw()
|
||||
│ ├── chat.rs # Chat transcript panel (dengan display cache)
|
||||
│ ├── sidebar.rs # Sidebar: workflow, tasks, usage
|
||||
│ ├── status.rs # Status bar satu baris
|
||||
│ ├── markdown.rs # Markdown → styled Span (pulldown-cmark)
|
||||
│ ├── workflow.rs # Workflow/hive-mind progress panel
|
||||
│ ├── theme.rs # Tokyo Night color palette (const)
|
||||
│ └── overlays/ # 16 overlay panel
|
||||
└── model/ # Data model lokal TUI
|
||||
```
|
||||
|
||||
## Render Pipeline (Per Frame)
|
||||
|
||||
```
|
||||
run_loop_inner() [50ms in-flight / 200ms idle]
|
||||
│
|
||||
▼
|
||||
main.rs: fn tui_loop()
|
||||
├── drain expired toasts (1x, bukan 2x)
|
||||
│
|
||||
├── controller/input.rs: handle_key() → action
|
||||
├── app/runtime/actions/mod.rs: apply_action()
|
||||
├── if dirty:
|
||||
│ view::pre_render(&mut state) ← update cache (markdown, token count)
|
||||
│ terminal.draw(|f| view::draw(f, &state))
|
||||
│ state.dirty = false
|
||||
│
|
||||
└── poll events → apply_action → Action::Tick
|
||||
```
|
||||
|
||||
### Optimasi Performa
|
||||
|
||||
| Masalah lama | Solusi saat ini |
|
||||
|---|---|
|
||||
| `count_tokens` (tiktoken) setiap frame | Cache `cached_token_count`, update hanya saat pesan baru |
|
||||
| `render_markdown` ulang setiap frame | `display_lines_cache` di `AppStateRest`, rebuild saat `transcript_cache.dirty` |
|
||||
| `Vec::remove(0)` untuk evict pesan lama | `VecDeque::pop_front()` — O(1) |
|
||||
| `Mutex<bool>` untuk `turn_in_flight` | `Arc<AtomicBool>` — lock-free |
|
||||
| Render terus meski idle | Skip `terminal.draw()` jika `dirty == false` |
|
||||
| Poll 50ms konstan | Adaptif: 50ms saat in-flight, 200ms saat idle |
|
||||
| `drain_expired_toasts` 2x per iterasi | Sekali saja di `run_loop_inner` |
|
||||
|
||||
## State (AppStateRest)
|
||||
|
||||
`AppStateRest` di `state.rs` adalah satu-satunya sumber kebenaran TUI:
|
||||
|
||||
```
|
||||
AppStateRest {
|
||||
settings: Settings // provider, model, dll
|
||||
app_config: AppConfig // endpoint, env vars
|
||||
workspace_roots: Vec<PathBuf> // working directories
|
||||
session_dir / session_id // path sesi aktif
|
||||
memory_dir // direktori memory
|
||||
session_runtime: Option<SessionRuntime> // history pesan, usage stats
|
||||
|
||||
transcript_cache: TranscriptCache // VecDeque<ChatMessageDisplay>
|
||||
scroll: ScrollState // offset scroll pane chat
|
||||
input: InputState // buffer, cursor, history, autocomplete
|
||||
misc: MiscState // overlay aktif, toasts, flags
|
||||
|
||||
turn_events: Arc<Mutex<VecDeque<TurnEvent>>> // queue event dari agent
|
||||
turn_in_flight_flag: Arc<AtomicBool> // apakah agent sedang jalan
|
||||
abort_flag: Arc<AtomicBool> // sinyal abort oleh user
|
||||
|
||||
// Cache performa
|
||||
display_lines_cache: Vec<Line<'static>> // hasil render markdown
|
||||
cached_token_count: usize // token count terkini
|
||||
token_count_dirty: bool // perlu hitung ulang?
|
||||
last_render_width: u16 // lebar terminal saat render terakhir
|
||||
|
||||
dirty: bool // perlu render ulang?
|
||||
quit: bool // keluar dari loop?
|
||||
}
|
||||
```
|
||||
|
||||
**Aturan mutasi:**
|
||||
- Dimutasi hanya dari `action.rs::apply_action()` dan `run.rs` (untuk dirty/quit)
|
||||
- Semua fungsi `view/*` bersifat read-only terhadap state
|
||||
- `pre_render_chat()` boleh mutasi hanya field cache (`display_lines_cache`, `cached_token_count`, `token_count_dirty`)
|
||||
|
||||
## Input & Actions
|
||||
|
||||
`controller/input.rs::handle_key()` → `Vec<Action>` → `apply_action(&mut state, action)`
|
||||
|
||||
Semua mutasi state melewati satu titik: `apply_action`. Controller tidak tahu *bagaimana* state diubah, hanya *action apa* yang dihasilkan.
|
||||
|
||||
### Action Utama
|
||||
|
||||
| Action | Efek |
|
||||
|--------|------|
|
||||
| `SubmitInput(text)` | Push ke transcript, spawn agent turn |
|
||||
| `Tick` | Drain `TurnEvent` queue, update state dari hasil agent |
|
||||
| `ScrollUp/Down` | Ubah `scroll.offset` |
|
||||
| `OpenOverlay(v)` | Set `misc.overlay = v` |
|
||||
| `Resize(w, h)` | Invalidasi cache display, set `last_render_width` |
|
||||
| `AbortTurn` | Store `true` ke `abort_flag` |
|
||||
| `ForceQuit` | Set `quit = true` |
|
||||
|
||||
## Overlays (16 Panel)
|
||||
|
||||
| Overlay | File | Fungsi |
|
||||
|---------|------|--------|
|
||||
| `Help` | `overlays/help.rs` | Daftar shortcut keyboard |
|
||||
| `Settings` | `overlays/settings.rs` | Panel pengaturan |
|
||||
| `Bash` | `overlays/bash.rs` | Background shell jobs |
|
||||
| `QuitConfirm` | `overlays/quit_confirm.rs` | Konfirmasi keluar |
|
||||
| `KeyInput` | `overlays/key_input.rs` | Capture key binding |
|
||||
| `Editor` | `overlays/editor.rs` | File editor inline |
|
||||
| `Effort` | `overlays/effort.rs` | Pilih level reasoning LLM |
|
||||
| `Mcp` | `overlays/mcp.rs` | Manajemen MCP server |
|
||||
| `Todo` | `overlays/todo.rs` | Daftar TODO |
|
||||
| `Rewind` | `overlays/rewind.rs` | Navigasi history pesan |
|
||||
| `Learning` | `overlays/learning.rs` | Viewer lesson |
|
||||
| `Usage` | `overlays/usage.rs` | Statistik token |
|
||||
| `Loading` | `overlays/loading.rs` | Spinner generik |
|
||||
| `ModelSelector` | `overlays/model_selector.rs` | Pilih model LLM |
|
||||
| `ClearConfirm` | `overlays/clear_confirm.rs` | Konfirmasi clear chat |
|
||||
|
||||
## Layout Terminal
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ │
|
||||
│ └── state mutates (AppStateRest)
|
||||
│
|
||||
└── view/mod.rs: build TUI layout
|
||||
│
|
||||
├── view/chat.rs: Chat transcript
|
||||
├── view/sidebar.rs: Usage dashboard
|
||||
├── view/status.rs: Status bar
|
||||
├── view/markdown.rs: Message renderer
|
||||
├── view/workflow.rs: Hive-mind progress
|
||||
└── view/theme.rs: Tokyo Night palette
|
||||
│ Chat Transcript Sidebar (≥90) │
|
||||
│ (view/chat.rs) ┌────────────┐ │
|
||||
│ VecDeque messages │ Workflow │ │
|
||||
│ + markdown cache ├────────────┤ │
|
||||
│ scrollable │ Tasks │ │
|
||||
│ ├────────────┤ │
|
||||
│ │ Usage │ │
|
||||
│ └────────────┘ │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ❯ Input Bar + Autocomplete dropdown │
|
||||
├───────────────────────────────────────────────┤
|
||||
│ ⚡zesdex READY │ ...center... │ tok · model │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Overlay System
|
||||
|
||||
16 overlays managed by `app/mode/`:
|
||||
|
||||
| Overlay | File | Purpose |
|
||||
|---------|------|---------|
|
||||
| Chat input | `mod.rs` | Main input bar with autocomplete |
|
||||
| Bash | `bash.rs` | Interactive shell panel |
|
||||
| Editor | `editor.rs` | Built-in file editor |
|
||||
| Effort | `effort.rs` | LLM effort selector |
|
||||
| Help | `help.rs` | Keybindings help |
|
||||
| Key Input | `key_input.rs` | Custom key binding |
|
||||
| Learning | `learning.rs` | Lesson viewer |
|
||||
| Loading | `loading.rs` | Spinner overlay |
|
||||
| MCP | `mcp.rs` | MCP server management |
|
||||
| Quit Confirm | `quit_confirm.rs` | Exit confirmation dialog |
|
||||
| Rewind | `rewind.rs` | Message/history rewind |
|
||||
| Settings | `settings.rs` | Settings panel |
|
||||
| Todo | `todo.rs` | Task/TODO list |
|
||||
| Workflow | (via view) | Workflow progress |
|
||||
|
||||
## Layout Structure
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Status Bar (view/status.rs) │
|
||||
├──────────────────────┬──────────────────────┤
|
||||
│ │ │
|
||||
│ Chat Transcript │ Sidebar │
|
||||
│ (view/chat.rs) │ (view/sidebar.rs) │
|
||||
│ scrollable, │ tokens, status, │
|
||||
│ inline-log style │ agent info │
|
||||
│ │ │
|
||||
├──────────────────────┴──────────────────────┤
|
||||
│ Input Bar + Autocomplete dropdown │
|
||||
│ (view/mod.rs) │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Input Handling
|
||||
|
||||
`controller/input.rs`:
|
||||
|
||||
- Normal mode: keystrokes go to the active overlay
|
||||
- `@mention` triggers fuzzy autocomplete (via `nucleo-matcher`)
|
||||
- Tab cycles autocomplete candidates
|
||||
- `Ctrl+Y` copies selected text to clipboard (via OSC52 escape sequence)
|
||||
- Arrow keys scroll chat, sidebar, and other scrollable panels
|
||||
Sidebar hanya tampil jika lebar terminal ≥ 90 kolom.
|
||||
|
||||
## Theme
|
||||
|
||||
`view/theme.rs` defines a Tokyo Night color palette as constants (`Theme::PRIMARY`, `Theme::ERROR`, `Theme::TEXT_MUTED`, etc.) rather than using a theme enum or hot-reloadable config. All view modules import and apply these constants directly.
|
||||
`view/theme.rs` mendefinisikan palette **Tokyo Night** sebagai `const Color`:
|
||||
`PRIMARY`, `BG`, `SURFACE`, `SURFACE_ELEVATED`, `BORDER`, `TEXT`, `TEXT_DIM`, `TEXT_MUTED`, `SUCCESS`, `WARNING`, `ERROR`, `INFO`, `HIGHLIGHT`, `CODE_BG`, dll.
|
||||
|
||||
## Markdown Rendering
|
||||
|
||||
`view/markdown.rs::render_markdown(text, width, dim)`:
|
||||
- Parse dengan `pulldown-cmark`
|
||||
- Hasilkan `Vec<Span<'static>>` dengan styling
|
||||
- Support: heading, code block, diff block (warna +/-/@@), list, blockquote, table, inline code, link
|
||||
- `dim=true` → semua span memakai `TEXT_DIM` + italic (untuk tool output)
|
||||
- Hasil di-cache di `AppStateRest::display_lines_cache`
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,649 +0,0 @@
|
||||
# CMS Wiring: Conversation + Rewind Blob Store Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace the SQLite-backed `messages`/`archives`/`blobs` tables (`crates/zesdex-backend/src/model/msglog/`) with `zesdex-cms`'s `Conversation`/`JsonConversationRepository` for message content, plus a brand-new file-based blob repository for rewind snapshots (no `zesdex-cms` equivalent existed before this plan). Fix the `ChatMessage` type collision between `zesdex-cms`'s local duplicate and the canonical `zesdex_entities::seaorm::common::message::ChatMessage` used everywhere else in the codebase.
|
||||
|
||||
**Architecture:** Research confirmed the `messages` table is write-only today (archived but never read back to restore a session) and the `archives` table is created but **never populated by any code path** — both can be retired with zero behavior loss. The `blobs` table is the one genuinely load-bearing piece (rewind feature reads it back) and needs a real, tested replacement — a new `RewindBlobRepository` trait + `FileRewindBlobRepository` impl added to `zesdex-cms`, storing raw bytes as `<session_dir>/blobs/<hex(key)>.bin` plus an append-only `<session_dir>/blobs/index.jsonl` for key/mime_type/ordering metadata (mirroring the JSONL-index pattern `zesdex-cms`'s own `EditLogRepository` already uses).
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-cms`, `zesdex-backend`, `zesdex-entities`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No `#[allow(...)]` additions beyond what's already in touched files.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- This plan deliberately does **not** attempt to migrate historical data out of any existing `messages.sqlite` files — since the `messages`/`archives` tables were never read back by any code path, there is nothing meaningful to migrate. Existing `messages.sqlite` files are simply left on disk, unused, after this plan (a future cleanup could delete them, but doing so isn't required for correctness).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Reconcile the `ChatMessage`/`Role` type collision in `zesdex-cms`
|
||||
|
||||
**Context:** `crates/zesdex-cms/src/domain/conversation.rs` currently defines its own `Role`/`ChatMessage` (with `tool_calls: Option<Vec<serde_json::Value>>`, untyped) instead of reusing `zesdex_entities::seaorm::common::message::{Role, ChatMessage}` (the canonical type used throughout `zesdex-backend`, with `tool_calls: Option<Vec<ToolCall>>`, strongly typed). `zesdex-cms` already depends on `zesdex-entities` (confirmed in `Cargo.toml`), so this is a small, surgical fix.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/conversation.rs`
|
||||
- Modify: `crates/zesdex-cms/src/application/conversation_service.rs` (import path only)
|
||||
- Modify: `crates/zesdex-cms/src/domain/service.rs` (import path only)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `zesdex_cms::domain::conversation::{Conversation, ChatMessage, Role}` where `ChatMessage`/`Role` are now re-exports of the canonical entities type — anything constructing a `zesdex_cms::domain::conversation::ChatMessage` is now interchangeable with `zesdex_entities::seaorm::common::message::ChatMessage` used elsewhere in `zesdex-backend`.
|
||||
|
||||
- [ ] **Step 1: Write the failing test proving type interchangeability**
|
||||
|
||||
Add to `crates/zesdex-cms/src/domain/conversation.rs`'s `#[cfg(test)] mod tests` (create if absent):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn chat_message_is_the_canonical_entities_type() {
|
||||
// This is a compile-time proof more than a runtime assertion: if
|
||||
// `zesdex_cms::domain::conversation::ChatMessage` were still a
|
||||
// distinct local type, this line would fail to compile.
|
||||
let canonical = zesdex_entities::seaorm::common::message::ChatMessage::user("hi");
|
||||
let via_cms: ChatMessage = canonical;
|
||||
assert_eq!(via_cms.content.as_deref(), Some("hi"));
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-cms chat_message_is_the_canonical -- --nocapture`
|
||||
Expected: compile error — `serde_json::Value` (cms's old `tool_calls` field type) vs `ChatMessage::user`'s type won't unify, or a straightforward type mismatch.
|
||||
|
||||
- [ ] **Step 3: Replace the local `Role`/`ChatMessage` with re-exports**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/conversation.rs`, delete the entire local `Role` enum and `ChatMessage` struct + impl block (the definitions, constructors `user`/`assistant`/`system`/`tool`), and replace the top of the file with:
|
||||
|
||||
```rust
|
||||
//! Pure Conversation entity — in-memory message history plus system prompt
|
||||
//! and LLM generation parameters.
|
||||
//!
|
||||
//! # Architecture
|
||||
//! This is a pure data structure with **no I/O logic**. Load/save
|
||||
//! responsibilities live in [`ConversationRepository`](super::repository::ConversationRepository).
|
||||
//!
|
||||
//! `ChatMessage`/`Role` are re-exported from `zesdex-entities` rather than
|
||||
//! duplicated here, so a `Conversation` built by this crate is
|
||||
//! interchangeable with the `ChatMessage` type used throughout
|
||||
//! `zesdex-backend`'s provider/tool-execution layer.
|
||||
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
pub use zesdex_entities::seaorm::common::message::{ChatMessage, Role};
|
||||
```
|
||||
|
||||
Keep the `Conversation` struct, `impl Conversation` block (`new`, `push`, `rebuild_system`, `to_api_messages`, `len`, `is_empty`) unchanged below this — they only reference `ChatMessage`/`Role` by name, which now resolve to the re-exported canonical types.
|
||||
|
||||
- [ ] **Step 4: Fix any now-broken references in the same crate**
|
||||
|
||||
Run: `cargo build -p zesdex-cms 2>&1 | head -60`
|
||||
|
||||
If `application/conversation_service.rs` or `domain/service.rs` import `ChatMessage`/`Role` via `use super::conversation::{ChatMessage, Conversation}` or similar — these continue to work unchanged since the names are still exported from `domain::conversation`, just backed by a different underlying type now. Only fix compile errors that actually appear; do not preemptively touch files the build doesn't flag.
|
||||
|
||||
- [ ] **Step 5: Run the test to verify it passes**
|
||||
|
||||
Run: `cargo test -p zesdex-cms chat_message_is_the_canonical -- --nocapture`
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run the crate's full test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-cms && cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms
|
||||
git commit -m "fix(cms): satukan ChatMessage/Role Conversation dengan tipe kanonik zesdex-entities"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add `RewindBlobRepository` to `zesdex-cms`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/repository.rs`
|
||||
- Create: `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs`
|
||||
- Modify: `crates/zesdex-cms/src/infrastructure/persistence/mod.rs`
|
||||
- Modify: `crates/zesdex-cms/Cargo.toml` (add `hex` and `chrono` if not already present — `chrono` is already a dependency per the crate's existing `Cargo.toml`; confirm `hex` with `grep hex crates/zesdex-cms/Cargo.toml` and add `hex.workspace = true` if missing)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `pub trait RewindBlobRepository { fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()>; fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> Result<Option<Vec<u8>>>; fn list_blob_keys(&self, session_dir: &Path) -> Result<Vec<String>>; }` and `pub struct FileRewindBlobRepository` — used by Task 4.
|
||||
|
||||
- [ ] **Step 1: Add the trait**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/repository.rs`, add:
|
||||
|
||||
```rust
|
||||
/// Repository for rewind-snapshot binary blobs, keyed by an arbitrary
|
||||
/// caller-supplied key (e.g. a tool-call id) within a session.
|
||||
pub trait RewindBlobRepository {
|
||||
/// Store (or overwrite) a blob under `blob_key` for this session.
|
||||
fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> anyhow::Result<()>;
|
||||
|
||||
/// Retrieve a blob's bytes by key, or `None` if not found.
|
||||
fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> anyhow::Result<Option<Vec<u8>>>;
|
||||
|
||||
/// List all blob keys for this session, oldest first.
|
||||
fn list_blob_keys(&self, session_dir: &Path) -> anyhow::Result<Vec<String>>;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Write the failing tests**
|
||||
|
||||
Create `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs` with:
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn tmp_dir() -> std::path::PathBuf {
|
||||
let dir = std::env::temp_dir().join(format!("zesdex-cms-blob-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
dir
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn store_and_retrieve_roundtrip() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "tool-call-1", b"hello world", Some("text/plain")).unwrap();
|
||||
let bytes = repo.retrieve_blob(&dir, "tool-call-1").unwrap();
|
||||
assert_eq!(bytes, Some(b"hello world".to_vec()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn retrieve_missing_key_returns_none() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
assert_eq!(repo.retrieve_blob(&dir, "no-such-key").unwrap(), None);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn list_blob_keys_returns_oldest_first() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "first", b"a", None).unwrap();
|
||||
std::thread::sleep(std::time::Duration::from_millis(5));
|
||||
repo.store_blob(&dir, "second", b"b", None).unwrap();
|
||||
let keys = repo.list_blob_keys(&dir).unwrap();
|
||||
assert_eq!(keys, vec!["first".to_string(), "second".to_string()]);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn overwriting_a_key_keeps_only_the_latest_entry_in_the_listing() {
|
||||
let dir = tmp_dir();
|
||||
let repo = FileRewindBlobRepository::new();
|
||||
repo.store_blob(&dir, "k", b"v1", None).unwrap();
|
||||
repo.store_blob(&dir, "k", b"v2", None).unwrap();
|
||||
let keys = repo.list_blob_keys(&dir).unwrap();
|
||||
assert_eq!(keys, vec!["k".to_string()], "key must appear exactly once even after being overwritten");
|
||||
assert_eq!(repo.retrieve_blob(&dir, "k").unwrap(), Some(b"v2".to_vec()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(Requires `uuid` — already a `zesdex-cms` dependency per its `Cargo.toml`.)
|
||||
|
||||
- [ ] **Step 3: Run the tests to verify they fail**
|
||||
|
||||
Run: `cargo test -p zesdex-cms rewind_blob_repo:: 2>&1 | head -20`
|
||||
Expected: compile error (`FileRewindBlobRepository` doesn't exist yet).
|
||||
|
||||
- [ ] **Step 4: Implement `FileRewindBlobRepository`**
|
||||
|
||||
Add above the test module in the same file:
|
||||
|
||||
```rust
|
||||
//! Filesystem-backed `RewindBlobRepository` implementation.
|
||||
//!
|
||||
//! Blob bytes are stored at `<session_dir>/blobs/<hex(key)>.bin` (the key
|
||||
//! is hex-encoded as the filename to sidestep any path-traversal/invalid-
|
||||
//! filename-character concerns entirely, mirroring the simplicity of
|
||||
//! `Memory::slugify` elsewhere in this crate but without needing a
|
||||
//! human-readable filename). Key/ordering/mime-type metadata lives in an
|
||||
//! append-only `<session_dir>/blobs/index.jsonl`, one JSON line per
|
||||
//! `store_blob` call — the same JSONL-index pattern already used by
|
||||
//! `EditLogRepository`. `list_blob_keys` de-duplicates by keeping each
|
||||
//! key's *last* index line (so overwriting a key doesn't produce a
|
||||
//! duplicate listing entry) and returns keys ordered by first-seen
|
||||
//! `created_at` ascending (oldest first), matching the previous
|
||||
//! `SQLite`-backed `ORDER BY created_at ASC` behavior.
|
||||
use std::io::Write;
|
||||
use std::path::Path;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use serde::{Deserialize, Serialize};
|
||||
|
||||
use crate::domain::repository::RewindBlobRepository;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
struct BlobIndexEntry {
|
||||
key: String,
|
||||
mime_type: Option<String>,
|
||||
created_at: i64,
|
||||
}
|
||||
|
||||
/// Concrete filesystem rewind-blob repository.
|
||||
#[derive(Debug, Clone, Default)]
|
||||
pub struct FileRewindBlobRepository;
|
||||
|
||||
impl FileRewindBlobRepository {
|
||||
/// Create a new filesystem rewind-blob repository.
|
||||
pub fn new() -> Self {
|
||||
Self
|
||||
}
|
||||
|
||||
fn blobs_dir(session_dir: &Path) -> std::path::PathBuf {
|
||||
session_dir.join("blobs")
|
||||
}
|
||||
|
||||
fn blob_file_path(session_dir: &Path, blob_key: &str) -> std::path::PathBuf {
|
||||
Self::blobs_dir(session_dir).join(format!("{}.bin", hex::encode(blob_key.as_bytes())))
|
||||
}
|
||||
|
||||
fn index_path(session_dir: &Path) -> std::path::PathBuf {
|
||||
Self::blobs_dir(session_dir).join("index.jsonl")
|
||||
}
|
||||
}
|
||||
|
||||
impl RewindBlobRepository for FileRewindBlobRepository {
|
||||
fn store_blob(&self, session_dir: &Path, blob_key: &str, data: &[u8], mime_type: Option<&str>) -> Result<()> {
|
||||
let blobs_dir = Self::blobs_dir(session_dir);
|
||||
std::fs::create_dir_all(&blobs_dir)
|
||||
.with_context(|| format!("failed to create blobs dir '{}'", blobs_dir.display()))?;
|
||||
|
||||
let path = Self::blob_file_path(session_dir, blob_key);
|
||||
let tmp = path.with_extension("bin.tmp");
|
||||
std::fs::write(&tmp, data)?;
|
||||
let f = std::fs::File::open(&tmp)?;
|
||||
f.sync_all()?;
|
||||
std::fs::rename(&tmp, &path)?;
|
||||
|
||||
let entry = BlobIndexEntry {
|
||||
key: blob_key.to_string(),
|
||||
mime_type: mime_type.map(String::from),
|
||||
created_at: chrono::Utc::now().timestamp_millis(),
|
||||
};
|
||||
let index_path = Self::index_path(session_dir);
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.append(true)
|
||||
.open(&index_path)
|
||||
.with_context(|| format!("failed to open blob index '{}'", index_path.display()))?;
|
||||
writeln!(f, "{}", serde_json::to_string(&entry)?)?;
|
||||
f.sync_all()?;
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
fn retrieve_blob(&self, session_dir: &Path, blob_key: &str) -> Result<Option<Vec<u8>>> {
|
||||
let path = Self::blob_file_path(session_dir, blob_key);
|
||||
if !path.exists() {
|
||||
return Ok(None);
|
||||
}
|
||||
Ok(Some(std::fs::read(&path)?))
|
||||
}
|
||||
|
||||
fn list_blob_keys(&self, session_dir: &Path) -> Result<Vec<String>> {
|
||||
let index_path = Self::index_path(session_dir);
|
||||
let Ok(content) = std::fs::read_to_string(&index_path) else {
|
||||
return Ok(Vec::new());
|
||||
};
|
||||
// Keep only the last occurrence of each key (later overwrites win),
|
||||
// but remember first-seen order for the final ascending sort.
|
||||
let mut first_seen_order: Vec<String> = Vec::new();
|
||||
let mut latest_by_key: std::collections::HashMap<String, BlobIndexEntry> = std::collections::HashMap::new();
|
||||
for line in content.lines() {
|
||||
let Ok(entry) = serde_json::from_str::<BlobIndexEntry>(line) else {
|
||||
continue;
|
||||
};
|
||||
if !latest_by_key.contains_key(&entry.key) {
|
||||
first_seen_order.push(entry.key.clone());
|
||||
}
|
||||
latest_by_key.insert(entry.key.clone(), entry);
|
||||
}
|
||||
let mut entries: Vec<BlobIndexEntry> = first_seen_order
|
||||
.into_iter()
|
||||
.filter_map(|k| latest_by_key.get(&k).cloned())
|
||||
.collect();
|
||||
entries.sort_by_key(|e| e.created_at);
|
||||
Ok(entries.into_iter().map(|e| e.key).collect())
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Register the module**
|
||||
|
||||
In `crates/zesdex-cms/src/infrastructure/persistence/mod.rs`, add:
|
||||
|
||||
```rust
|
||||
pub mod rewind_blob_repo;
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Run the tests to verify they pass**
|
||||
|
||||
Run: `cargo test -p zesdex-cms rewind_blob_repo:: -- --nocapture`
|
||||
Expected: all 4 tests pass.
|
||||
|
||||
- [ ] **Step 7: Run clippy**
|
||||
|
||||
Run: `cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: no new warnings.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms/src/domain/repository.rs crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs crates/zesdex-cms/src/infrastructure/persistence/mod.rs crates/zesdex-cms/Cargo.toml
|
||||
git commit -m "feat(cms): tambahkan RewindBlobRepository berbasis file (pengganti tabel blobs SQLite)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Rewire message archiving (`archive_message`) to `Conversation`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines ~695-708 `spawn_turn`, ~727-744 `TurnCtx`, ~862-873 `archive_message`, plus all 7 call sites at lines 939, 1105, 1295, 1387, 1421, 1426, 1458 — re-confirm line numbers first since Task 5 of the OAuth/session plan and Task 5 of the settings/appconfig/memory/editlog plan may have shifted this file)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::conversation::{Conversation, ChatMessage}` (Task 1), `zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository`, `zesdex_cms::domain::repository::ConversationRepository`.
|
||||
- Produces: `TurnCtx.conversation: Option<Arc<Mutex<Conversation>>>` (replaces `TurnCtx.db: Option<Arc<Mutex<rusqlite::Connection>>>`).
|
||||
|
||||
- [ ] **Step 1: Confirm current line numbers**
|
||||
|
||||
Run: `grep -n "fn spawn_turn\|struct TurnCtx\|fn archive_message\|open_or_create\|tc\.db\.as_ref" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
- [ ] **Step 2: Replace the per-turn connection setup in `spawn_turn`**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let db = crate::model::msglog::open_or_create(&edit_session_dir)
|
||||
.ok()
|
||||
.map(|c| std::sync::Arc::new(std::sync::Mutex::new(c)));
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let conversation = {
|
||||
let repo = zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository::new();
|
||||
let conv = repo.load(&edit_session_dir).unwrap_or_else(|_| {
|
||||
zesdex_cms::domain::conversation::Conversation::new(String::new(), session_id.clone())
|
||||
});
|
||||
Some(std::sync::Arc::new(std::sync::Mutex::new(conv)))
|
||||
};
|
||||
```
|
||||
|
||||
Replace the `TurnCtx` struct literal's `db,` field with `conversation,`.
|
||||
|
||||
- [ ] **Step 3: Update the `TurnCtx` struct definition**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
db: Option<std::sync::Arc<std::sync::Mutex<rusqlite::Connection>>>,
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
conversation: Option<std::sync::Arc<std::sync::Mutex<zesdex_cms::domain::conversation::Conversation>>>,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Rewrite `archive_message`**
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
fn archive_message(db: Option<&std::sync::Arc<std::sync::Mutex<rusqlite::Connection>>>, session_id: &str, msg: &ChatMessage) {
|
||||
if let Some(arc) = db {
|
||||
if let Ok(conn) = arc.lock() {
|
||||
let _ = crate::model::msglog::insert_message(&conn, session_id, msg);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
/// Persist a `ChatMessage` to the session's `Conversation`, if one is
|
||||
/// available for this turn.
|
||||
///
|
||||
/// Flow: if `conversation` is `Some`, lock the mutex, push the message,
|
||||
/// and rewrite `conversation.json` in full. Errors are silently ignored
|
||||
/// (matches the previous `SQLite`-backed behavior, which also swallowed
|
||||
/// insert failures).
|
||||
fn archive_message(
|
||||
conversation: Option<&std::sync::Arc<std::sync::Mutex<zesdex_cms::domain::conversation::Conversation>>>,
|
||||
session_dir: &std::path::Path,
|
||||
msg: &ChatMessage,
|
||||
) {
|
||||
if let Some(arc) = conversation {
|
||||
if let Ok(mut conv) = arc.lock() {
|
||||
conv.push(msg.clone());
|
||||
let repo = zesdex_cms::infrastructure::persistence::conversation_repo::JsonConversationRepository::new();
|
||||
let _ = repo.save(session_dir, &conv);
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Update all 7 call sites**
|
||||
|
||||
Run: `grep -n "archive_message(tc.db.as_ref()" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
At each of the 7 matched lines, replace `archive_message(tc.db.as_ref(), &tc.session_id, &<msg_var>)` with `archive_message(tc.conversation.as_ref(), &tc.edit_log_session_dir, &<msg_var>)` (keep whatever the actual message-variable name is at each site — `sys`, `pipeline_msg`, `response`, `review_msg`, `tool_msg`, `msg` per the research brief — only the first two arguments change).
|
||||
|
||||
- [ ] **Step 6: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (beyond anything Task 4's blob work below still needs to touch in the same file — if this task is done independently, `store_blob` call sites will still fail to compile at this point; that's expected and resolved by Task 4).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/runtime/actions/mod.rs
|
||||
git commit -m "refactor(backend): alihkan archive_message dari SQLite messages table ke zesdex-cms Conversation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Rewire rewind blob storage to `FileRewindBlobRepository`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs` (2 `store_blob` call sites, confirmed at line 531-533 and a second one — re-grep to find both)
|
||||
- Modify: `crates/zesdex-backend/src/app/mode/rewind.rs` (`rewind_count`, `rewind_to`, plus the `open_session_db` helper and the still-existing store_blob call site inside `execute_one_tool`/wherever the second engine.rs call lives)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository`, `zesdex_cms::domain::repository::RewindBlobRepository`.
|
||||
- Produces: nothing new for other tasks — this is the last consumer of the old `msglog::blobs` module.
|
||||
|
||||
- [ ] **Step 1: Find both `store_blob` call sites in `engine.rs`**
|
||||
|
||||
Run: `grep -n -B6 "store_blob" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
- [ ] **Step 2: Replace each `store_blob` call site**
|
||||
|
||||
Replace (pattern applies to both sites, adjusting the surrounding variable names per the actual code read in Step 1):
|
||||
```rust
|
||||
let _ = crate::model::msglog::store_blob(
|
||||
&conn, session_id, &tool_call.id, &bytes, None,
|
||||
);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new()
|
||||
.store_blob(&ctx.session_dir, &tool_call.id, &bytes, None);
|
||||
```
|
||||
|
||||
(Uses `RewindBlobRepository::store_blob` — add `use zesdex_cms::domain::repository::RewindBlobRepository;` to the file's imports. Note this drops the now-unneeded `conn`/`session_id`-derived-from-directory-name dance since the new repository takes `session_dir` directly — if the surrounding code only opened `conn` for this call, remove the now-dead connection-opening code too after confirming via Step 1's grep that nothing else in the same scope still needs it.)
|
||||
|
||||
- [ ] **Step 3: Rewrite `rewind.rs`'s `open_session_db` usage**
|
||||
|
||||
Read the whole file first: `cat crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace `rewind_count`:
|
||||
```rust
|
||||
pub fn rewind_count(state: &AppStateRest) -> usize {
|
||||
let Ok(conn) = open_session_db(&state.session_dir) else {
|
||||
return 0;
|
||||
};
|
||||
crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id)
|
||||
.ok()
|
||||
.map_or(0, |keys| keys.len())
|
||||
}
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
pub fn rewind_count(state: &AppStateRest) -> usize {
|
||||
zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new()
|
||||
.list_blob_keys(&state.session_dir)
|
||||
.ok()
|
||||
.map_or(0, |keys| keys.len())
|
||||
}
|
||||
```
|
||||
|
||||
Replace the body of `rewind_to` (the `open_session_db` call plus `list_blob_keys`/`retrieve_blob` calls):
|
||||
```rust
|
||||
let conn = match open_session_db(&state.session_dir) { /* ... */ };
|
||||
let keys = match crate::model::msglog::blobs::list_blob_keys(&conn, &state.session_id) { /* ... */ };
|
||||
/* ... */
|
||||
let bytes = match crate::model::msglog::blobs::retrieve_blob(&conn, &state.session_id, blob_key) { /* ... */ };
|
||||
```
|
||||
with (dropping the `conn`/`open_session_db` step entirely — the file-based repository needs no connection object, just `&state.session_dir`):
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::rewind_blob_repo::FileRewindBlobRepository::new();
|
||||
|
||||
let keys = match repo.list_blob_keys(&state.session_dir) {
|
||||
Ok(k) => k,
|
||||
Err(e) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
format!("Failed to list snapshots: {e}"),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
};
|
||||
|
||||
if keys.is_empty() || index >= keys.len() {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Warning,
|
||||
"No snapshot available at that index".to_string(),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
|
||||
let blob_key = &keys[index];
|
||||
let bytes = match repo.retrieve_blob(&state.session_dir, blob_key) {
|
||||
Ok(Some(b)) => b,
|
||||
Ok(None) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
"Snapshot data not found".to_string(),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
Err(e) => {
|
||||
state.push_toast(crate::app::state::types::Toast::new(
|
||||
crate::app::state::types::ToastKind::Error,
|
||||
format!("Failed to retrieve snapshot: {e}"),
|
||||
));
|
||||
state.dirty = true;
|
||||
return;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
(Keep whatever code follows `bytes` unchanged — the actual file-restoration logic doesn't depend on how `bytes` was fetched.)
|
||||
|
||||
- [ ] **Step 4: Delete the now-unused `open_session_db` helper**
|
||||
|
||||
If `open_session_db` (used only by the two call sites just replaced) has no other callers after Step 3 — verify with `grep -n "open_session_db" crates/zesdex-backend/src/app/mode/rewind.rs` — delete its definition entirely.
|
||||
|
||||
- [ ] **Step 5: Update the module doc comment**
|
||||
|
||||
Replace the file's top doc comment:
|
||||
```rust
|
||||
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
|
||||
//! session's `SQLite` blob store.
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
//! Rewind mode: restores a file to a pre-edit snapshot stored in the
|
||||
//! session's file-based rewind blob store (`<session_dir>/blobs/`).
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 7: Run the workspace test suite**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
Expected: all pass.
|
||||
|
||||
- [ ] **Step 8: Manual smoke test**
|
||||
|
||||
In the TUI: perform a file edit (triggers a pre-edit snapshot store), open the Rewind overlay, confirm the snapshot count and list are correct, and restore the file — confirm the restored content matches the pre-edit version exactly.
|
||||
|
||||
- [ ] **Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/subagent/engine.rs crates/zesdex-backend/src/app/mode/rewind.rs
|
||||
git commit -m "refactor(backend): alihkan penyimpanan blob rewind ke FileRewindBlobRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Delete the now-dead `msglog` SQLite module
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-backend/src/model/msglog/{mod.rs,schema.rs,query.rs,blobs.rs}`
|
||||
- Modify: `crates/zesdex-backend/src/model/mod.rs` (remove `pub mod msglog;`)
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (line ~236, `edit_count: state.edit_log.len() as u32,` — confirm this doesn't reference `msglog` directly; if it only reads `state.edit_log`, no change needed here)
|
||||
|
||||
**Interfaces:** none — pure deletion after Tasks 3-4 remove every reference.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references**
|
||||
|
||||
Run: `grep -rln "model::msglog\|msglog::" crates/zesdex-backend/src`
|
||||
Expected: no output (only the `model/mod.rs` declaration itself, addressed in Step 3).
|
||||
|
||||
- [ ] **Step 2: Delete the files**
|
||||
|
||||
```bash
|
||||
git rm -r crates/zesdex-backend/src/model/msglog
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove the module declaration**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod msglog;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Build the whole workspace**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Expected: no errors. If `rusqlite` was only pulled into `zesdex-backend` for this module, `cargo build` will still succeed since `rusqlite` remains a workspace dependency used elsewhere (`zesdex-libs::database.rs`) — no `Cargo.toml` change needed here; confirm with `grep -rln "rusqlite" crates/zesdex-backend/src` that no other file in this crate still needs it, and if truly zero remaining uses, remove `rusqlite` from `crates/zesdex-backend/Cargo.toml`'s `[dependencies]` as a final cleanup (only if the grep comes back empty).
|
||||
|
||||
- [ ] **Step 5: Run the full test suite and clippy**
|
||||
|
||||
Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: hapus modul msglog SQLite lama (digantikan Conversation + RewindBlobRepository)"
|
||||
```
|
||||
@@ -1,737 +0,0 @@
|
||||
# CMS Wiring: Settings, AppConfig, Memory, EditLog Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Replace `zesdex-backend`'s use of `zesdex_entities::seaorm::common::{settings,app_config,memory,edit_log}` (inherent-method I/O) with the previously-orphaned `zesdex-cms` crate's repository-based equivalents (`JsonSettingsRepository`, `JsonAppConfigRepository`, `MarkdownMemoryRepository`, `JsonlEditLogRepository`), which are on-disk-format-compatible drop-ins. Fix the one real defect found in `zesdex-cms::Settings` along the way (a missing `#[serde(default)]` that would hard-fail loading any pre-existing `settings.json`).
|
||||
|
||||
**Architecture:** `zesdex-backend` call sites move from `Type::static_method()` to `repository_instance.method(&base_dir, ...)`. Since `zesdex-cms`'s application-service layer (`SettingsServiceImpl`, `MemoryServiceImpl`) doesn't cover every read pattern the backend needs (no bare `AppConfig` getter, no single-memory read, no `EditLogService` at all), most call sites construct and use the concrete `Json*Repository`/`MarkdownMemoryRepository`/`JsonlEditLogRepository` types directly rather than going through the service traits — this matches the actual usage shape better than forcing everything through an ill-fitting service abstraction.
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-cms`, `zesdex-backend`, `zesdex-entities`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- On-disk formats are confirmed compatible for all four entities (same JSON/markdown/JSONL shape and file paths) — this plan is a call-site migration, **not** a data migration. No existing user `settings.json`/`app_config.json`/`*.md` memory files/`edits.jsonl` need to change.
|
||||
- `EditLog`'s domain shape genuinely differs between the old (`entries` + `path`, disk I/O on construction) and new (`entries` only, disk I/O via `EditLogRepository::open`) versions — every call site becomes fallible (`Result`) where it was previously infallible.
|
||||
- No new `#[allow(...)]` attributes. Existing ones in touched files are out of scope (handled by `2026-07-16-convention-cleanup-docs.md`).
|
||||
- Tests are inline `#[cfg(test)] mod tests`, per CLAUDE.md.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- Base directory resolution: everywhere the old code called the zero-argument `Settings::load()`/`Settings::save()`/`AppConfig::load()` (which internally called `zesdex_entities::seaorm::common::store::Store::new()` to get `base_dir`), the replacement must call `zesdex_entities::seaorm::common::store::Store::new().base_dir` explicitly and pass it to the `zesdex-cms` repository — this reproduces the exact same directory, verified identical during planning.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Fix the `hive_mind_node_timeout_ms` serde-default gap in `zesdex-cms::Settings`
|
||||
|
||||
**Context:** `crates/zesdex-entities/src/seaorm/common/settings.rs:75` has `#[serde(default = "default_hive_mind_node_timeout_ms")]` on this field; `crates/zesdex-cms/src/domain/settings.rs`'s copy does not. Without this, any settings.json written before this field existed (or any settings.json missing it for any reason) will hard-fail `JsonSettingsRepository::load` with a parse error, whereas the old `Settings::load()` silently defaulted on **any** failure. Fix both the missing default and restore full parse-failure tolerance to avoid a behavior regression for existing users.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-cms/src/domain/settings.rs`
|
||||
- Modify: `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `Settings::default()` unchanged in value; `JsonSettingsRepository::load` becomes tolerant of parse failures (still `Result`-returning, but only returns `Err` for I/O errors other than "not found" or "malformed JSON" — matching the old infallible-except-I/O-permission-errors behavior as closely as a `Result`-based API can).
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add to `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`'s `#[cfg(test)] mod tests` (create it if absent — check first: `grep -n "mod tests" crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn load_defaults_hive_mind_timeout_when_field_missing_from_old_settings_json() {
|
||||
let dir = std::env::temp_dir().join(format!("zesdex-cms-settings-test-{}", uuid::Uuid::new_v4()));
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
// Simulate a settings.json written before `hive_mind_node_timeout_ms` existed.
|
||||
std::fs::write(
|
||||
dir.join("settings.json"),
|
||||
r#"{"internet_mode":"Off","provider":"zen","model":"m","api_keys":{},"max_tokens":null,"temperature":null,"review_max_lessons_per_run":5,"adaptive_review_max_skip":3,"verify_command":null,"verify_timeout_ms":30000,"workflow_max_concurrency":5,"review_enabled":true,"session_archive_enabled":true,"lsp_auto_provision":true,"lsp_languages":[]}"#,
|
||||
).unwrap();
|
||||
|
||||
let repo = JsonSettingsRepository::new();
|
||||
let settings = repo.load(&dir).expect("load must not fail on a pre-existing settings.json missing the new field");
|
||||
assert_eq!(settings.hive_mind_node_timeout_ms, 600_000);
|
||||
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
```
|
||||
|
||||
(Requires `uuid` as a dev-dependency of `zesdex-cms` — check first: `grep uuid crates/zesdex-cms/Cargo.toml`; add `uuid = { workspace = true }` under `[dev-dependencies]` if missing, creating that section if it doesn't exist.)
|
||||
|
||||
- [ ] **Step 2: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-cms load_defaults_hive_mind -- --nocapture`
|
||||
Expected: fails with a JSON parse/missing-field error.
|
||||
|
||||
- [ ] **Step 3: Add the serde default to the domain type**
|
||||
|
||||
In `crates/zesdex-cms/src/domain/settings.rs`, add above the `Settings` struct:
|
||||
|
||||
```rust
|
||||
fn default_hive_mind_node_timeout_ms() -> u64 {
|
||||
600_000
|
||||
}
|
||||
```
|
||||
|
||||
And annotate the field:
|
||||
|
||||
```rust
|
||||
#[serde(default = "default_hive_mind_node_timeout_ms")]
|
||||
pub hive_mind_node_timeout_ms: u64,
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Restore full parse-failure tolerance in the repository**
|
||||
|
||||
In `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs`, update `load`:
|
||||
|
||||
```rust
|
||||
fn load(&self, base_dir: &Path) -> Result<Settings> {
|
||||
let path = base_dir.join("settings.json");
|
||||
match std::fs::read_to_string(&path) {
|
||||
Ok(s) => match serde_json::from_str(&s) {
|
||||
Ok(settings) => Ok(settings),
|
||||
Err(e) => {
|
||||
tracing::warn!(
|
||||
"settings.json at '{}' failed to parse ({e}); falling back to defaults",
|
||||
path.display()
|
||||
);
|
||||
Ok(Settings::default())
|
||||
}
|
||||
},
|
||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
|
||||
tracing::info!("settings.json not found, using defaults");
|
||||
Ok(Settings::default())
|
||||
}
|
||||
Err(e) => Err(anyhow::anyhow!("failed to read settings.json: {e}")),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run the test to verify it passes**
|
||||
|
||||
Run: `cargo test -p zesdex-cms load_defaults_hive_mind -- --nocapture`
|
||||
Expected: pass.
|
||||
|
||||
- [ ] **Step 6: Run the crate's full test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-cms && cargo clippy -p zesdex-cms -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-cms/src/domain/settings.rs crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs
|
||||
git commit -m "fix(cms): perbaiki serde default hive_mind_node_timeout_ms & toleransi parse gagal di Settings"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Swap `Settings` call sites to `JsonSettingsRepository`
|
||||
|
||||
**Files (every one confirmed by research — swap all):**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 20, 49, 89, 103, 170)
|
||||
- `crates/zesdex-backend/src/bin/seed.rs` (line 12)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 61, 64-84, 353 doc comment)
|
||||
- `crates/zesdex-backend/src/app/workflow/hive_mind.rs` (lines 255, 281-283, 367 doc comments)
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs` (lines 9, 19-25, 46/56/79 test-only)
|
||||
- `crates/zesdex-backend/src/app/mode/settings.rs` (lines 6, 16-22)
|
||||
- `crates/zesdex-backend/src/controller/input.rs` (lines 339, 354, 357-361, 393-407)
|
||||
- `crates/zesdex-backend/src/view/mod.rs` (lines 152-174, 701-713 — read-only, no method-call change needed beyond the type import)
|
||||
- `crates/zesdex-backend/src/view/status.rs` (lines 68, 86-93 — read-only)
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs` (lines 79, 85, 405-406 — read-only)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 549, 633-671, 1615, 1758 — read-only)
|
||||
- `crates/zesdex-backend/src/main.rs` (lines 141, 474, 641 — `.save()` calls)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::settings::Settings`, `zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository`, `zesdex_cms::domain::repository::SettingsRepository` (trait, for method resolution), `zesdex_entities::seaorm::common::store::Store` (for `base_dir` resolution).
|
||||
- Produces: `AppStateRest.settings: zesdex_cms::domain::settings::Settings` (type changed from the old entities type — field-identical, so every **read-only** call site above needs only an import-path change, not a logic change).
|
||||
|
||||
- [ ] **Step 1: Update the type import everywhere it's read-only**
|
||||
|
||||
In each of `rest.rs`, `view/mod.rs`, `view/status.rs`, `app/review/mod.rs`, `app/runtime/actions/mod.rs`, replace:
|
||||
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
```
|
||||
|
||||
(For files that reference `Settings` only via `state.settings.<field>` without an explicit `use` for the type itself — confirm per-file with `grep -n "use.*settings::Settings\|model::settings" <file>` before editing — skip files where no explicit import exists, since `AppStateRest.settings`'s type change alone (Step 3 below) is what they actually depend on.)
|
||||
|
||||
- [ ] **Step 2: Update `app/mode/settings.rs` (mutates `Settings` in place, no I/O)**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/app/mode/settings.rs`. Replace:
|
||||
|
||||
```rust
|
||||
use crate::model::settings::{InternetMode, Settings};
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::{InternetMode, Settings};
|
||||
```
|
||||
|
||||
`cycle_internet_mode(settings: &mut Settings)`'s body (mutating `settings.internet_mode` in a cycle) needs no logic change — `InternetMode` is field-identical between old and new.
|
||||
|
||||
- [ ] **Step 3: Update `AppStateRest` construction and field type**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`:
|
||||
|
||||
Replace the import (line 20):
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
use zesdex_cms::domain::repository::SettingsRepository;
|
||||
use zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository;
|
||||
```
|
||||
|
||||
Replace the field's type comment reference (line 49) — no change needed, `pub settings: Settings,` already resolves to the new import.
|
||||
|
||||
Replace construction (line 89):
|
||||
```rust
|
||||
let settings = Settings::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let store_base_dir = zesdex_entities::seaorm::common::store::Store::new().base_dir;
|
||||
let settings = JsonSettingsRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Update `controller/input.rs`'s `.save()` call sites**
|
||||
|
||||
Read the file first: `grep -n -B3 "\.settings\.save()" crates/zesdex-backend/src/controller/input.rs`
|
||||
|
||||
Both call sites (previously lines 361 and 407) currently do `let _ = state.settings.save();`. Since `Settings` no longer carries an inherent `save()` method, replace each with:
|
||||
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.save(&state.store_base_dir(), &state.settings);
|
||||
```
|
||||
|
||||
(Uses `state.store_base_dir()` — the existing helper on `AppStateRest`, confirmed to resolve to the same directory as `Store::new().base_dir` in normal operation — since these call sites already have `state: &mut AppStateRest` in scope, unlike `rest.rs::new()` which doesn't yet have a constructed `state` to call `.store_base_dir()` on.)
|
||||
|
||||
Add `use zesdex_cms::domain::repository::SettingsRepository;` to this file's imports if not already present after Step 1.
|
||||
|
||||
- [ ] **Step 5: Update `main.rs`'s three `.save()` call sites**
|
||||
|
||||
Read the file first: `grep -n -B2 "\.settings\.save()" crates/zesdex-backend/src/main.rs`
|
||||
|
||||
Replace each `let _ = state.settings.save();` (or `client_state.settings.save()`) with the same pattern as Step 4, substituting the correct state variable name at each site:
|
||||
|
||||
```rust
|
||||
let _ = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.save(&state.store_base_dir(), &state.settings);
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Update `app/subagent/engine.rs`'s `resolve_provider_config()`**
|
||||
|
||||
Read the function first: `grep -n -A 30 "fn resolve_provider_config" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let settings = crate::model::settings::Settings::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let store_base_dir = zesdex_entities::seaorm::common::store::Store::new().base_dir;
|
||||
let settings = zesdex_cms::infrastructure::persistence::settings_repo::JsonSettingsRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::SettingsRepository;` to this file's imports.) The subsequent field reads (`settings.api_keys`, `settings.provider`, `settings.model` at lines 64-84) need no change — same field names/types.
|
||||
|
||||
- [ ] **Step 7: Update `app/workflow/hive_mind.rs`'s `run_hive_mind()`**
|
||||
|
||||
Read the function first: `grep -n -A 5 "let settings = crate::model::settings::Settings::load" crates/zesdex-backend/src/app/workflow/hive_mind.rs`
|
||||
|
||||
Apply the identical substitution pattern from Step 6. Field reads `settings.hive_mind_node_timeout_ms`/`settings.workflow_max_concurrency` need no change.
|
||||
|
||||
- [ ] **Step 8: Update `app/runtime/context/window.rs`**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/app/runtime/context/window.rs`
|
||||
|
||||
Replace the import (line 9):
|
||||
```rust
|
||||
use crate::model::settings::Settings;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::settings::Settings;
|
||||
```
|
||||
|
||||
The `resolve(app_config: &AppConfig, settings: &Settings)` function signature/body and the three test-only `Settings::default()` constructions need no logic change — same type shape, `Default` still works identically after Task 1.
|
||||
|
||||
- [ ] **Step 9: Update `bin/seed.rs`**
|
||||
|
||||
Read the file first: `cat crates/zesdex-backend/src/bin/seed.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
zesdex_entities::seaorm::common::settings::Settings::default()
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
zesdex_cms::domain::settings::Settings::default()
|
||||
```
|
||||
|
||||
- [ ] **Step 10: Remove the now-unused `model::settings` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod settings {
|
||||
pub use zesdex_entities::seaorm::common::settings::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 11: Verify no remaining references**
|
||||
|
||||
Run: `grep -rn "model::settings::" crates/zesdex-backend/src`
|
||||
Expected: no output.
|
||||
|
||||
- [ ] **Step 12: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all tests pass.
|
||||
|
||||
- [ ] **Step 13: Manual smoke test**
|
||||
|
||||
Run: `cargo run -p zesdex-backend`. Confirm the TUI starts, the Settings overlay (per `view/mod.rs`) displays the current provider/model/flags correctly, and changing internet mode / API key / provider / model persists correctly across a restart (settings.json is written and re-read with the same values).
|
||||
|
||||
- [ ] **Step 14: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan Settings ke zesdex-cms JsonSettingsRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Swap `AppConfig` call sites to `JsonAppConfigRepository`
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 18, 50, 90, 104)
|
||||
- `crates/zesdex-backend/src/bin/seed.rs` (line 27)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 62, 69, 73)
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs` (lines 8, 19-25, test sites)
|
||||
- `crates/zesdex-backend/src/controller/input.rs` (lines 220, 252, 383, 385)
|
||||
- `crates/zesdex-backend/src/view/mod.rs` (lines 710-711)
|
||||
- `crates/zesdex-backend/src/view/status.rs` (line 68)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 548, 551, 635-659, 1755-1758)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::app_config::{AppConfig, ProviderConfig, ModelRole}`, `zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository`, `zesdex_cms::domain::repository::AppConfigRepository`.
|
||||
- Produces: `AppStateRest.app_config: zesdex_cms::domain::app_config::AppConfig` (type changed, field-identical). `AppConfig` is never saved anywhere in `zesdex-backend` today (confirmed by research) — this task only needs `load`, no `save` call sites.
|
||||
|
||||
- [ ] **Step 1: Update read-only imports**
|
||||
|
||||
Same pattern as Task 2 Step 1: in each file that imports `crate::model::app_config::AppConfig`/`ProviderConfig`, replace with `zesdex_cms::domain::app_config::{AppConfig, ProviderConfig}` (add `ModelRole` too where `view/mod.rs`/`window.rs` need it).
|
||||
|
||||
- [ ] **Step 2: Update `AppStateRest` construction**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`, add to the import block from Task 2 Step 3:
|
||||
|
||||
```rust
|
||||
use zesdex_cms::domain::app_config::AppConfig;
|
||||
use zesdex_cms::domain::repository::AppConfigRepository;
|
||||
use zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository;
|
||||
```
|
||||
|
||||
Replace construction (line 90):
|
||||
```rust
|
||||
let app_config = AppConfig::load();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let app_config = JsonAppConfigRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Reuses the `store_base_dir` local variable already introduced in Task 2 Step 3 — both `Settings` and `AppConfig` load from the same base directory.)
|
||||
|
||||
- [ ] **Step 3: Update `app/subagent/engine.rs`**
|
||||
|
||||
Apply the same substitution as Task 2 Step 6, adding the `AppConfig` load right after the `Settings` load using the same `store_base_dir`:
|
||||
|
||||
```rust
|
||||
let app_config = zesdex_cms::infrastructure::persistence::app_config_repo::JsonAppConfigRepository::new()
|
||||
.load(&store_base_dir)
|
||||
.unwrap_or_default();
|
||||
```
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::AppConfigRepository;`.)
|
||||
|
||||
- [ ] **Step 4: Update `bin/seed.rs`**
|
||||
|
||||
Replace `zesdex_entities::seaorm::common::app_config::AppConfig::default()` with `zesdex_cms::domain::app_config::AppConfig::default()`.
|
||||
|
||||
- [ ] **Step 5: Remove the now-unused `model::app_config` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod app_config {
|
||||
pub use zesdex_entities::seaorm::common::app_config::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Verify no remaining references, build, and test**
|
||||
|
||||
Run: `grep -rn "model::app_config::" crates/zesdex-backend/src` — expected no output.
|
||||
Run: `cargo build --workspace && cargo test --workspace` — expected all pass.
|
||||
|
||||
- [ ] **Step 7: Manual smoke test**
|
||||
|
||||
Run the TUI, open the Model Selector overlay (Ctrl+P or equivalent per `resources.rs`), confirm the provider/model list still populates correctly from `app_config.json`, and that Claude-credential auto-detection (if `~/.claude/settings.json` exists on the test machine) still merges in correctly.
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan AppConfig ke zesdex-cms JsonAppConfigRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Swap `Memory` call sites to `MarkdownMemoryRepository`
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/tool/memory/recall.rs` (lines 4, 44, 63, 69)
|
||||
- `crates/zesdex-backend/src/tool/memory/remember.rs` (lines 4, 74, 79-96)
|
||||
- `crates/zesdex-backend/src/tool/memory/forget.rs` (lines 4, 45)
|
||||
- `crates/zesdex-backend/src/app/mode/learning.rs` (lines 56, 58)
|
||||
- `crates/zesdex-backend/src/app/workflow/docs.rs` (line 34 — `slugify` only, no I/O)
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs` (lines 487, 491, 493-494)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 591, 797, 810, 833, 843)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::memory::Memory`, `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository`, `zesdex_cms::domain::repository::MemoryRepository`.
|
||||
- Produces: nothing new for other tasks — `Memory` is a leaf entity with no state-struct field.
|
||||
|
||||
- [ ] **Step 1: `tool/memory/recall.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/tool/memory/recall.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
use crate::model::memory::Memory;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::memory::Memory;
|
||||
use zesdex_cms::domain::repository::MemoryRepository;
|
||||
use zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository;
|
||||
```
|
||||
|
||||
Replace (line 44): `Memory::read(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().load(&ctx.memory_dir, name)`
|
||||
Replace (line 63): `Memory::list(&ctx.memory_dir)` → `MarkdownMemoryRepository::new().list(&ctx.memory_dir)`
|
||||
Replace (line 69): `Memory::read(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().load(&ctx.memory_dir, name)`
|
||||
|
||||
(Both old inherent methods and the new repository methods return `Result<Memory>`/`Result<Vec<String>>` respectively — signature shape at the call site is unchanged beyond the receiver.)
|
||||
|
||||
- [ ] **Step 2: `tool/memory/remember.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/tool/memory/remember.rs`
|
||||
|
||||
Same import swap as Step 1. Replace (line 74): `Memory::slugify(name)` → `Memory::slugify(name)` (unchanged — `slugify` remains an inherent method on the domain struct in `zesdex-cms`, per research). The struct-literal construction (lines 79-92) needs no change (field-identical). Replace (lines 94-96):
|
||||
```rust
|
||||
memory.write(&ctx.memory_dir)?;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
MarkdownMemoryRepository::new().save(&ctx.memory_dir, &memory)?;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: `tool/memory/forget.rs`**
|
||||
|
||||
Same import swap. Replace (line 45): `Memory::remove(&ctx.memory_dir, name)` → `MarkdownMemoryRepository::new().delete(&ctx.memory_dir, name)`.
|
||||
|
||||
- [ ] **Step 4: `app/mode/learning.rs`**
|
||||
|
||||
Read the file: `cat crates/zesdex-backend/src/app/mode/learning.rs`
|
||||
|
||||
Replace (line 56): `crate::model::memory::Memory::list(&state.memory_dir)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().list(&state.memory_dir)`
|
||||
Replace (line 58): `crate::model::memory::Memory::read(&state.memory_dir, &name)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().load(&state.memory_dir, &name)`
|
||||
|
||||
(Add `use zesdex_cms::domain::repository::MemoryRepository;` at the top of the file.)
|
||||
|
||||
- [ ] **Step 5: `app/workflow/docs.rs`**
|
||||
|
||||
Replace (line 34): `crate::model::memory::Memory::slugify(user_request)` → `zesdex_cms::domain::memory::Memory::slugify(user_request)`. (Pure filename-generation helper, no repository/I/O involved — no other change needed.)
|
||||
|
||||
- [ ] **Step 6: `app/review/mod.rs`**
|
||||
|
||||
Read the surrounding code: `grep -n -B2 -A8 "model::memory::Memory::list(memory_dir)" crates/zesdex-backend/src/app/review/mod.rs`
|
||||
|
||||
Replace (line 487): `crate::model::memory::Memory::list(memory_dir)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().list(memory_dir)`
|
||||
Replace (line 491): `crate::model::memory::Memory::read(memory_dir, &name)` → `zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().load(memory_dir, &name)`
|
||||
Replace (lines 493-494):
|
||||
```rust
|
||||
mem.lifecycle = "stale".to_string();
|
||||
mem.write(memory_dir)?;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
mem.lifecycle = "stale".to_string();
|
||||
zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository::new().save(memory_dir, &mem)?;
|
||||
```
|
||||
|
||||
- [ ] **Step 7: `app/runtime/actions/mod.rs`**
|
||||
|
||||
Read each site first: `grep -n -B2 -A2 "model::memory::Memory::" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Apply the same `Memory::method(dir, ...)` → `MarkdownMemoryRepository::new().method(dir, ...)` substitution at all 5 remaining sites (lines 591, 797, 810, 833, 843), matching the method-name mapping: `remove`→`delete`, `list`→`list`, `read`→`load`.
|
||||
|
||||
- [ ] **Step 8: Add `use` statements**
|
||||
|
||||
Add `use zesdex_cms::domain::repository::MemoryRepository;` and `use zesdex_cms::infrastructure::persistence::memory_repo::MarkdownMemoryRepository;` to `app/review/mod.rs` and `app/runtime/actions/mod.rs` (both already import many things — add alongside existing `use` block).
|
||||
|
||||
- [ ] **Step 9: Remove the now-unused `model::memory` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod memory {
|
||||
pub use zesdex_entities::seaorm::common::memory::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 10: Verify no remaining references, build, test**
|
||||
|
||||
Run: `grep -rn "model::memory::" crates/zesdex-backend/src` — expected no output.
|
||||
Run: `cargo build --workspace && cargo test --workspace` — expected all pass.
|
||||
|
||||
- [ ] **Step 11: Manual smoke test**
|
||||
|
||||
In the TUI, invoke the `remember` tool to create a memory, `recall` it back, confirm the Learning overlay lists it, then `forget` it and confirm it's gone. Also confirm existing `.md` memory files from before this change (if any test fixtures exist) still load correctly (format compatibility check).
|
||||
|
||||
- [ ] **Step 12: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan Memory ke zesdex-cms MarkdownMemoryRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Swap `EditLog` call sites to `JsonlEditLogRepository`
|
||||
|
||||
**Context:** This is the most invasive of the four because every call site currently does `EditLog::new(dir)` (infallible, eager full-file read) and the new equivalent `JsonlEditLogRepository::open(dir)` returns `Result`. There are 7 distinct call sites plus one persistent field on `AppStateRest`.
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/state/rest.rs` (lines 19, 58, 115)
|
||||
- `crates/zesdex-backend/src/main.rs` (line 236 — reads `state.edit_log.len()`, no change needed beyond the type)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs` (lines 571-580)
|
||||
- `crates/zesdex-backend/src/app/mode/rewind.rs` (lines 103-114, 128-134)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs` (lines 921, 1473-1475, 1487-1489, 1586-1597)
|
||||
- `crates/zesdex-backend/src/model/mod.rs` (remove re-export)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_cms::domain::edit_log::{EditLog, EditLogEntry}`, `zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository`, `zesdex_cms::domain::repository::EditLogRepository`.
|
||||
- Produces: `AppStateRest.edit_log: zesdex_cms::domain::edit_log::EditLog` (type changed — no `path` field this time, so anything reading `state.edit_log.path` would break; confirmed by research that no call site does this).
|
||||
|
||||
- [ ] **Step 1: Update `AppStateRest`**
|
||||
|
||||
In `crates/zesdex-backend/src/app/state/rest.rs`, replace the import (line 19):
|
||||
```rust
|
||||
use crate::model::editlog::EditLog;
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
use zesdex_cms::domain::edit_log::EditLog;
|
||||
use zesdex_cms::domain::repository::EditLogRepository;
|
||||
use zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository;
|
||||
```
|
||||
|
||||
Replace construction (line 115):
|
||||
```rust
|
||||
edit_log: EditLog::new(session_dir),
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
edit_log: JsonlEditLogRepository::new().open(session_dir).unwrap_or_else(|e| {
|
||||
tracing::warn!("[state] failed to open edit log at '{}': {e}", session_dir.display());
|
||||
EditLog::new()
|
||||
}),
|
||||
```
|
||||
|
||||
- [ ] **Step 2: `app/subagent/engine.rs`**
|
||||
|
||||
Read the site: `grep -n -B3 -A3 "EditLog::new(&ctx.session_dir)" crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let mut el = crate::model::editlog::EditLog::new(&ctx.session_dir);
|
||||
el.append(entry).ok();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new();
|
||||
if let Ok(mut el) = repo.open(&ctx.session_dir) {
|
||||
let _ = repo.append(&ctx.session_dir, &mut el, entry);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: `app/mode/rewind.rs` — logging the rewind operation**
|
||||
|
||||
Read the site: `grep -n -B3 -A10 "EditLog::new(&state.session_dir)" crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace the first site (previously lines 103-114):
|
||||
```rust
|
||||
let mut el = crate::model::editlog::EditLog::new(&state.session_dir);
|
||||
let entry = crate::model::editlog::EditLogEntry { /* ... */ };
|
||||
let _ = el.append(entry);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let repo = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new();
|
||||
if let Ok(mut el) = repo.open(&state.session_dir) {
|
||||
let entry = zesdex_cms::domain::edit_log::EditLogEntry { /* same field values as before */ };
|
||||
let _ = repo.append(&state.session_dir, &mut el, entry);
|
||||
}
|
||||
```
|
||||
|
||||
(Keep the exact same `EditLogEntry` field values from the original code — only the type path and the append mechanism change.)
|
||||
|
||||
- [ ] **Step 4: `app/mode/rewind.rs` — `find_edit_path`**
|
||||
|
||||
Read the site: `grep -n -B3 -A8 "fn find_edit_path" crates/zesdex-backend/src/app/mode/rewind.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let el = crate::model::editlog::EditLog::new(&state.session_dir);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let el = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&state.session_dir)
|
||||
.unwrap_or_else(|_| zesdex_cms::domain::edit_log::EditLog::new());
|
||||
```
|
||||
|
||||
The subsequent `el.entries.iter().rev().find(...)` (unchanged — `entries` is a public field on both old and new `EditLog`) needs no further change.
|
||||
|
||||
- [ ] **Step 5: `app/runtime/actions/mod.rs` — turn-start snapshot**
|
||||
|
||||
Read the site: `grep -n -B2 -A2 "let initial_edits" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let initial_edits = crate::model::editlog::EditLog::new(&tc.edit_log_session_dir).len();
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let initial_edits = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&tc.edit_log_session_dir)
|
||||
.map(|el| el.entries.len())
|
||||
.unwrap_or(0);
|
||||
```
|
||||
|
||||
(`EditLog` in `zesdex-cms` has no `.len()` inherent method — check first: `grep -n "fn len\|impl EditLog" crates/zesdex-cms/src/domain/edit_log.rs`. If `.len()` doesn't exist, use `.entries.len()` as shown; if it does exist, use `el.len()` instead for consistency with the rest of the codebase's naming.)
|
||||
|
||||
- [ ] **Step 6: `app/runtime/actions/mod.rs` — turn-end diff**
|
||||
|
||||
Read the site: `grep -n -B2 -A10 "let final_edits" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Replace:
|
||||
```rust
|
||||
let el = crate::model::editlog::EditLog::new(&tc.edit_log_session_dir);
|
||||
let final_edits = el.len();
|
||||
let total_edits_this_turn = final_edits.saturating_sub(initial_edits);
|
||||
```
|
||||
with:
|
||||
```rust
|
||||
let el = zesdex_cms::infrastructure::persistence::edit_log_repo::JsonlEditLogRepository::new()
|
||||
.open(&tc.edit_log_session_dir)
|
||||
.unwrap_or_else(|_| zesdex_cms::domain::edit_log::EditLog::new());
|
||||
let final_edits = el.entries.len();
|
||||
let total_edits_this_turn = final_edits.saturating_sub(initial_edits);
|
||||
```
|
||||
|
||||
And immediately after (previously lines 1487-1489), the `.entries.iter().skip(initial_edits)` loop needs no change — same field access.
|
||||
|
||||
- [ ] **Step 7: `app/runtime/actions/mod.rs` — `execute_one_tool`'s primary write path**
|
||||
|
||||
Read the site: `grep -n -B3 -A5 "EditLog::new(session_dir)" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Apply the same pattern as Step 2 (open + conditional append via the repository).
|
||||
|
||||
- [ ] **Step 8: Remove the now-unused `model::editlog` re-export**
|
||||
|
||||
In `crates/zesdex-backend/src/model/mod.rs`, remove:
|
||||
```rust
|
||||
pub mod editlog {
|
||||
pub use zesdex_entities::seaorm::common::edit_log::*;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 9: Verify no remaining references**
|
||||
|
||||
Run: `grep -rn "model::editlog::" crates/zesdex-backend/src`
|
||||
Expected: no output.
|
||||
|
||||
- [ ] **Step 10: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all tests pass.
|
||||
|
||||
- [ ] **Step 11: Manual smoke test**
|
||||
|
||||
In the TUI, make a file edit via the `edit`/`write` tool, confirm `state.edit_log` grows and `main.rs`'s IPC `edit_count` payload reflects it, then use the rewind overlay to confirm `find_edit_path` still correctly recovers the edited file's path and the rewind itself works end-to-end.
|
||||
|
||||
- [ ] **Step 12: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor(backend): alihkan EditLog ke zesdex-cms JsonlEditLogRepository"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Delete the now-dead `zesdex_entities::seaorm::common::{settings,app_config,memory,edit_log}` modules
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-entities/src/seaorm/common/settings.rs`, `app_config.rs`, `memory.rs`, `edit_log.rs`
|
||||
- Modify: `crates/zesdex-entities/src/seaorm/common/mod.rs` (remove their module declarations)
|
||||
|
||||
**Interfaces:** none — pure deletion after Tasks 2-5 have removed every reference.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references across the whole workspace**
|
||||
|
||||
Run: `grep -rln "seaorm::common::settings\|seaorm::common::app_config\|seaorm::common::memory\b\|seaorm::common::edit_log" crates --include='*.rs'`
|
||||
Expected: no output. (If anything other than the `mod.rs` declaration itself shows up, stop and investigate before deleting — it means a call site was missed in Tasks 2-5.)
|
||||
|
||||
- [ ] **Step 2: Delete the files**
|
||||
|
||||
```bash
|
||||
git rm crates/zesdex-entities/src/seaorm/common/settings.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/app_config.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/memory.rs
|
||||
git rm crates/zesdex-entities/src/seaorm/common/edit_log.rs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove their module declarations**
|
||||
|
||||
In `crates/zesdex-entities/src/seaorm/common/mod.rs`, remove the corresponding `pub mod settings;`, `pub mod app_config;`, `pub mod memory;`, `pub mod edit_log;` lines (check first: `cat crates/zesdex-entities/src/seaorm/common/mod.rs`).
|
||||
|
||||
- [ ] **Step 4: Build the whole workspace**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 5: Run the full test suite and clippy**
|
||||
|
||||
Run: `cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: hapus entitas settings/app_config/memory/edit_log lama di zesdex-entities yang sudah digantikan zesdex-cms"
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,373 +0,0 @@
|
||||
# Convention Cleanup + Documentation Repair Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Bring the codebase into compliance with `CLAUDE.md`'s own stated rules that the audit found violated — 110 `#[allow(...)]` lint-bypass attributes (10 of them silencing `dead_code`, which the workspace `Cargo.toml` explicitly `deny`s), a custom error type where only `anyhow` is supposed to be used, small doc-comment gaps — and repair the five `docs/CODEMAPS/*.md` files plus `CLAUDE.md` itself, which reference pre-workspace-migration paths that no longer exist.
|
||||
|
||||
**Architecture:** No structural changes to running code beyond what's needed to satisfy the lints without suppressing them. Documentation tasks are pure text corrections against the now-accurate `crates/` layout (this plan should run **after** the other four plans in this series, since they change many of the exact file paths the docs need to describe correctly).
|
||||
|
||||
**Tech Stack:** Rust, Markdown.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No **new** `#[allow(...)]` may be introduced by this plan's own changes.
|
||||
- Every dead-code removal must be verified by letting the compiler/clippy confirm the item has zero remaining callers — never delete on assumption.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` after each task.
|
||||
- **Run this plan last**, after `2026-07-16-security-quickfixes.md`, `2026-07-16-oauth-session-iam-wiring.md`, `2026-07-16-cms-settings-appconfig-memory-editlog-wiring.md`, `2026-07-16-cms-conversation-blob-wiring.md`, and `2026-07-16-middleware-axum-server.md` — the documentation tasks (Task 6) describe the *end state* of all five, and several files this plan touches for lint cleanup (`app/runtime/actions/mod.rs`, `app/runtime/context/*.rs`) are also touched by those plans.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Delete the unused custom `Error` type in `zesdex-utils`
|
||||
|
||||
**Context:** `crates/zesdex-utils/src/error.rs` defines a hand-rolled `pub enum Error` + `impl std::error::Error` + a `Result<T>` alias, directly contradicting CLAUDE.md's "anyhow::Result and anyhow::bail! throughout... No custom error types" rule. Confirmed via workspace-wide grep: **zero call sites reference it outside the file itself** — it's simply dead code, not something anything depends on. `thiserror` is declared as a `zesdex-utils` dependency but never imported anywhere in the crate either.
|
||||
|
||||
**Files:**
|
||||
- Delete: `crates/zesdex-utils/src/error.rs`
|
||||
- Modify: `crates/zesdex-utils/src/lib.rs` (remove `pub mod error;`)
|
||||
- Modify: `crates/zesdex-utils/Cargo.toml` (remove the unused `thiserror` dependency)
|
||||
|
||||
**Interfaces:** none — pure deletion.
|
||||
|
||||
- [ ] **Step 1: Verify zero remaining references**
|
||||
|
||||
Run: `grep -rln "zesdex_utils::error\|zesdex_utils::Error\|utils::error::" crates --include='*.rs'`
|
||||
Expected: only `crates/zesdex-utils/src/error.rs` itself (or no output once the file is deleted).
|
||||
|
||||
- [ ] **Step 2: Delete the file**
|
||||
|
||||
```bash
|
||||
git rm crates/zesdex-utils/src/error.rs
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Remove the module declaration**
|
||||
|
||||
In `crates/zesdex-utils/src/lib.rs`, remove:
|
||||
```rust
|
||||
pub mod error;
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Remove the unused `thiserror` dependency**
|
||||
|
||||
In `crates/zesdex-utils/Cargo.toml`, remove:
|
||||
```toml
|
||||
thiserror = { workspace = true }
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test -p zesdex-utils`
|
||||
Expected: no errors.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore(utils): hapus custom Error type yang tidak dipakai (melanggar aturan anyhow-only)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Resolve the 10 `dead_code` allow-bypasses
|
||||
|
||||
**Context:** These directly contradict the workspace's own `dead_code = "deny"` lint. For each, remove the `#[allow(dead_code)]`/`#![allow(dead_code)]`, run the compiler, and act on its verdict: if genuinely unused, delete; if actually reachable through a path the lint can't see (e.g. only used in `#[cfg(test)]` or behind a feature), wire it into real production code instead of re-suppressing.
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/runtime/context/dedup.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/squash.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/window.rs:1`
|
||||
- `crates/zesdex-backend/src/app/runtime/context/tokens.rs:32`
|
||||
- `crates/zesdex-backend/src/app/subagent/spawn.rs:44,51`
|
||||
- `crates/zesdex-backend/src/app/state/misc.rs:409`
|
||||
- `crates/zesdex-backend/src/model/agent_def/{global.rs,builtin.rs,session.rs}:1`
|
||||
|
||||
**Interfaces:** varies per site — resolved during the investigation step, not fixed in advance (this is a "read what the compiler says, then act" task, not a hand-wave — see Step 1 of each site).
|
||||
|
||||
- [ ] **Step 1: `dedup.rs`, `squash.rs`, `window.rs` (module-level)**
|
||||
|
||||
Remove the `#![allow(dead_code)]` line from each of the three files. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "never used"`
|
||||
|
||||
For each item the compiler flags as unused: check whether it's covered by a test in the same file's `#[cfg(test)] mod tests` (a test-only user doesn't count as a real caller and doesn't justify keeping the item) — if the item has zero non-test callers, delete it; if grepping the item's name elsewhere in `crates/zesdex-backend/src` (outside the file and outside `#[cfg(test)]` blocks) turns up a real caller the compiler somehow didn't connect (e.g. it's `pub` and meant for a different module that has a typo'd import), fix the import instead of deleting.
|
||||
|
||||
- [ ] **Step 2: `tokens.rs:32` (`count_message_tokens`)**
|
||||
|
||||
Read the function and its context: `grep -n -B5 -A15 "fn count_message_tokens" crates/zesdex-backend/src/app/runtime/context/tokens.rs`
|
||||
|
||||
Remove `#[allow(dead_code)]`. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "count_message_tokens"`. If genuinely unused, delete the function (and any now-orphaned helper it alone called). If it looks like it *should* be called from the context-window-shaping logic in the same module (a token-counting function not being used by the token-budget code would itself be a functional gap worth flagging, not just a lint issue) — check `window.rs`'s `resolve()` and any `shaping`/`dedup` call sites for where a token count is needed but computed some other way, and wire `count_message_tokens` in there if that's the case; otherwise delete.
|
||||
|
||||
- [ ] **Step 3: `spawn.rs:44,51` (`with_max_steps`, `with_temperature` builder methods)**
|
||||
|
||||
Read the full builder struct: `grep -n -B20 "fn with_max_steps" crates/zesdex-backend/src/app/subagent/spawn.rs`
|
||||
|
||||
Remove both `#[allow(dead_code)]` lines. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "with_max_steps\|with_temperature"`.
|
||||
|
||||
These configure per-agent `max_steps`/`temperature` on a subagent-spawn builder — check every call site that constructs this builder (`grep -rn "AgentSpawnBuilder\|::new()" crates/zesdex-backend/src/app/subagent/` — use the builder's actual type name found in Step 3's read) to see whether any caller *should* be setting these (e.g. does `hive_mind.rs`'s node-spawning code hardcode a default that should instead come from `Settings`/`NodeDirective` and isn't?). If a real caller needs them, wire them in (this may surface an actual functional gap, not just unused code — document what you find). If truly no caller has a legitimate need for per-agent overrides today, delete both methods and their backing struct fields if those fields are then also unused.
|
||||
|
||||
- [ ] **Step 4: `state/misc.rs:409` (`api_context_length` field)**
|
||||
|
||||
Read the surrounding struct: `grep -n -B15 -A5 "api_context_length" crates/zesdex-backend/src/app/state/misc.rs`
|
||||
|
||||
Remove `#[allow(dead_code)]`. Run: `cargo build -p zesdex-backend 2>&1 | grep -A3 "api_context_length"`. Check whether the status bar (`view/status.rs`) or connectivity-check code (`spawn_api_connectivity_check` in `actions/mod.rs`) should be displaying/using the model's context length but currently isn't — if so, wire it in; if the field was superseded by `app_config.model_roles[...].context_window` (per the CMS wiring plan) and is now genuinely redundant, delete the field.
|
||||
|
||||
- [ ] **Step 5: `model/agent_def/{global.rs,builtin.rs,session.rs}` (module-level)**
|
||||
|
||||
Same procedure as Step 1: remove each `#![allow(dead_code)]`, build, and either delete unused items or wire in real callers based on what the compiler reports.
|
||||
|
||||
- [ ] **Step 6: Full workspace build and test after all 10 sites are resolved**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, no `dead_code` warnings anywhere (the workspace `deny` will turn any remaining one into a hard build failure, which is the actual verification that every site was genuinely resolved).
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: hapus 10 allow(dead_code) - hapus kode mati atau sambungkan ke pemanggil nyata"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Resolve the 7 item-level clippy allows
|
||||
|
||||
**Files:**
|
||||
- `crates/zesdex-backend/src/app/review/mod.rs:371` (`clippy::unnecessary_debug_formatting`)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs:1532` (`clippy::too_many_arguments`)
|
||||
- `crates/zesdex-backend/src/app/runtime/actions/mod.rs:99,912` (`clippy::too_many_lines`, x2)
|
||||
- `crates/zesdex-backend/src/app/subagent/engine.rs:326` (`clippy::too_many_lines`)
|
||||
- `crates/zesdex-backend/src/view/markdown.rs:62` (`clippy::too_many_lines`)
|
||||
- `crates/zesdex-backend/src/view/mod.rs:112` (`clippy::too_many_lines`)
|
||||
|
||||
**Interfaces:** none shared across sites — each is an independent, local fix.
|
||||
|
||||
- [ ] **Step 1: `clippy::unnecessary_debug_formatting` (easiest — do first)**
|
||||
|
||||
Read the flagged line: `grep -n -B3 -A3 "unnecessary_debug_formatting" crates/zesdex-backend/src/app/review/mod.rs`
|
||||
|
||||
Remove the `#[allow(clippy::unnecessary_debug_formatting)]` line. Run: `cargo clippy -p zesdex-backend 2>&1 | grep -A5 "unnecessary_debug_formatting"` to see the exact suggestion (clippy always proposes the fix inline — typically replacing a `format!("{:?}", x)` with `x.to_string()` or a `Display` impl call). Apply the suggested fix exactly.
|
||||
|
||||
- [ ] **Step 2: `clippy::too_many_arguments` on `actions/mod.rs:1532`**
|
||||
|
||||
Read the flagged function's full signature: `grep -n -B2 -A15 "clippy::too_many_arguments" crates/zesdex-backend/src/app/runtime/actions/mod.rs`
|
||||
|
||||
Bundle the excess parameters into a purpose-named struct (the standard fix for this lint). For example, if the function is `fn foo(a: X, b: Y, c: Z, d: W, ...) -> R`, introduce:
|
||||
```rust
|
||||
struct FooParams {
|
||||
a: X,
|
||||
b: Y,
|
||||
c: Z,
|
||||
d: W,
|
||||
// ...
|
||||
}
|
||||
```
|
||||
and change the signature to `fn foo(params: FooParams) -> R`, updating the function body to read `params.a`/`params.b`/etc., and updating the single call site to construct `FooParams { a, b, c, d, ... }`. (Exact field names/types depend on the actual signature found in this step's read — do not guess, use the literal parameter list.)
|
||||
|
||||
- [ ] **Step 3: `clippy::too_many_lines` — `actions/mod.rs:99` and `:912` (`run_agent_turn`)**
|
||||
|
||||
Read the full function: `grep -n -A 250 "^fn run_agent_turn" crates/zesdex-backend/src/app/runtime/actions/mod.rs | head -260`
|
||||
|
||||
This is the core per-turn agent loop — do not split it mechanically by line count; split along its own documented phase boundaries (the function's doc comment already describes them: "build system prompt → shape messages → call `chat_with_tools_streaming` → handle tool calls or unwrap final message → check unfinished todos → finalize"). Extract each phase that doesn't need to mutate more than 2-3 local variables into its own well-named private function, threading only what each phase actually needs as parameters (not the whole `TurnCtx` if a phase only reads one field). After extraction, re-add doc comments to each new function per CLAUDE.md's Code Documentation rules. Do this incrementally: extract one phase, build, test, commit; repeat rather than one giant rewrite, so a regression is easy to bisect.
|
||||
|
||||
Run after each extraction: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
Once the function is under clippy's threshold, remove the `#[allow(clippy::too_many_lines)]` at both flagged lines (99 and 912 — confirm both are on `run_agent_turn` or its immediate helper via the Step 1 grep; if they're on two different functions, repeat this decomposition process for each independently).
|
||||
|
||||
- [ ] **Step 4: `clippy::too_many_lines` — `app/subagent/engine.rs:326`**
|
||||
|
||||
Read the flagged function in full: `grep -n -B2 -A 200 "clippy::too_many_lines" crates/zesdex-backend/src/app/subagent/engine.rs | head -210`
|
||||
|
||||
Apply the same phase-based extraction approach as Step 3, scaled to this function's actual structure (read it first — do not assume it mirrors `run_agent_turn`'s shape).
|
||||
|
||||
- [ ] **Step 5: `clippy::too_many_lines` — `view/markdown.rs:62` and `view/mod.rs:112`**
|
||||
|
||||
Read both flagged functions in full first (`grep -n -A 150 "clippy::too_many_lines" crates/zesdex-backend/src/view/markdown.rs` and the equivalent for `view/mod.rs`). These are rendering functions — split along rendering sub-sections (e.g. one function per overlay/pane already rendered inline in a big `match`), extracting each `match` arm's body over some line-count threshold into its own `fn render_<thing>(f: &mut Frame, area: Rect, state: &AppStateRest)`-shaped helper, matching the existing `view/` module's established per-pane function naming convention (check `view/chat.rs`/`view/status.rs` for the naming pattern already in use and follow it).
|
||||
|
||||
- [ ] **Step 6: Full workspace verification**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass with zero `too_many_lines`/`too_many_arguments`/`unnecessary_debug_formatting` warnings and no remaining `#[allow]` for any of them.
|
||||
|
||||
- [ ] **Step 7: Commit each function's decomposition separately as you go (already instructed inline above) — final wrap-up commit if anything remains uncommitted**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: pecah fungsi yang melanggar clippy::too_many_lines/too_many_arguments, hapus allow-nya"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Reduce the 93 module-level cast-quad allows
|
||||
|
||||
**Context:** `#![allow(clippy::cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap)]` appears at the top of 93 files, evidently copy-pasted as workspace-wide boilerplate rather than justified per-file. This is the largest item in this plan by file count and — because it's the same mechanical recipe repeated 93 times — is best executed via `superpowers:subagent-driven-development` dispatching one subagent per file (or small batch of related files within the same crate) using the worked recipe below, rather than as one sequential task list here.
|
||||
|
||||
**Files:** all 93 listed in the audit's inventory (re-derive the authoritative current list before starting, since Tasks 1-3 and the other four plans in this series may have deleted or renamed some of them):
|
||||
|
||||
Run: `grep -rln "cast_possible_truncation, clippy::cast_sign_loss, clippy::cast_precision_loss, clippy::cast_possible_wrap" crates --include='*.rs'`
|
||||
|
||||
**Interfaces:** none shared — each file's fix is independent and self-contained.
|
||||
|
||||
- [ ] **Step 1: Worked example — pick one small, representative file first**
|
||||
|
||||
Read a small file from the list, e.g. `crates/zesdex-backend/src/app/mode/effort.rs` (confirm it's still in the current list from this task's Step-1 grep before using it as the example). Remove its `#![allow(clippy::cast_*...)]` line. Run:
|
||||
|
||||
Run: `cargo clippy -p zesdex-backend -- -D warnings 2>&1 | grep -B2 -A8 "effort.rs"`
|
||||
|
||||
For each flagged cast, apply the narrowest correct fix:
|
||||
- `x as u32` where `x: usize` and the value is a count/length that can't realistically exceed `u32::MAX` → `u32::try_from(x).unwrap_or(u32::MAX)` (saturating, since these are almost always display/telemetry values where saturating is safe) or, if the call site already returns `Result`, `u32::try_from(x)?`.
|
||||
- `x as i64` where `x: u64` timestamp (milliseconds since epoch) → these are safe until year 292471247, so `TryFrom` is technically correct but arguably pedantic; use `i64::try_from(x).unwrap_or(i64::MAX)` for consistency with the rule above rather than special-casing "this one's fine."
|
||||
- `x as f32`/`x as f64` (precision loss) on values already known to fit (e.g. small counters) → keep the cast but make it explicit and document why it's lossless in context: `#[expect(clippy::cast_precision_loss, reason = "...")]` is still a bypass and NOT allowed by CLAUDE.md — instead, if the value truly can't lose precision (e.g. casting a `u8` to `f32`), the lint won't even fire once the blanket module-level allow is removed, since clippy's precision-loss lint only fires above the point where precision loss is actually possible for the source type; if it does fire, use the same `try_from`-then-`as` pattern, or restructure to avoid the float conversion entirely if it's just for display (`format!("{x}")` instead of casting to display as a percentage, etc.).
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
Expected: no errors, no new warnings for this file.
|
||||
|
||||
- [ ] **Step 2: Commit the worked example**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/app/mode/effort.rs
|
||||
git commit -m "fix: hapus allow cast-quad di effort.rs, ganti cast lossy dengan try_from"
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Dispatch the remaining files via subagent-driven-development**
|
||||
|
||||
For the remaining files from Step 1's grep (minus the one just fixed), use `superpowers:subagent-driven-development` with one task per file (or per small group of 3-5 files within the same module, where that reads more naturally), each task instructing: "remove the `#![allow(clippy::cast_*)]` header from `<file>`, run `cargo clippy -p <crate> -- -D warnings` scoped to that file, and fix every flagged cast using the recipe demonstrated in `2026-07-16-convention-cleanup-docs.md` Task 4 Step 1 (prefer `TryFrom`/`try_from` with a saturating fallback for lossy integer casts; restructure to avoid unnecessary float casts where the value is just being displayed)." Review each file's diff before merging — this is exactly the kind of large, repetitive, low-per-item-risk task the subagent-driven workflow is for.
|
||||
|
||||
- [ ] **Step 4: Final workspace-wide verification**
|
||||
|
||||
Run: `grep -rln "cast_possible_truncation, clippy::cast_sign_loss" crates --include='*.rs'`
|
||||
Expected: no output (or, if a small number of files remain and are judged genuinely fine to leave as a future increment, that's a call for whoever is running this plan to make explicitly and document — not silently left as-is).
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace && cargo clippy --workspace --all-targets -- -D warnings`
|
||||
Expected: all pass.
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Fill the remaining doc-comment gaps and remove dead scaffolding
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/view/mod.rs` (lines 10-15 — `pub mod chat/markdown/sidebar/status/theme/workflow`, 5 of 7 missing doc comments)
|
||||
- Modify: `crates/zesdex-backend/src/app/state/misc.rs` (line 340 `pub fn submit`, and the second gap the audit found around line 438)
|
||||
- Delete: `/mnt/code/zesdex/tests/` (confirmed empty and untracked — safe to remove; re-verify emptiness before deleting since time has passed since the original audit)
|
||||
|
||||
**Interfaces:** none — doc comments and a directory deletion, no behavior change.
|
||||
|
||||
- [ ] **Step 1: Add doc comments to `view/mod.rs`'s module declarations**
|
||||
|
||||
Read the current lines: `grep -n -B1 "^pub mod" crates/zesdex-backend/src/view/mod.rs`
|
||||
|
||||
For each of `chat`, `markdown`, `sidebar`, `status`, `theme`, `workflow` that lacks a one-line doc comment above it, add one describing what that view submodule renders — e.g.:
|
||||
|
||||
```rust
|
||||
/// Chat transcript pane: renders the scrollback of user/assistant/tool messages.
|
||||
pub mod chat;
|
||||
/// Markdown-to-styled-text rendering for assistant message content.
|
||||
pub mod markdown;
|
||||
/// Session sidebar: file tree / workspace navigation pane.
|
||||
pub mod sidebar;
|
||||
/// Status bar: provider/model, token usage, connectivity indicator.
|
||||
pub mod status;
|
||||
/// Color theme definitions for the TUI.
|
||||
pub mod theme;
|
||||
/// Hive-mind workflow panel: live node progress display.
|
||||
pub mod workflow;
|
||||
```
|
||||
|
||||
(Read each module's actual top-of-file doc comment first — `head -5 crates/zesdex-backend/src/view/{chat,markdown,sidebar,status,theme,workflow}.rs` — and base the one-liner on what that file's own doc comment says, rather than guessing, so the two stay consistent.)
|
||||
|
||||
- [ ] **Step 2: Add doc comments to the two flagged functions in `state/misc.rs`**
|
||||
|
||||
Read both: `grep -n -B2 -A8 "pub fn submit" crates/zesdex-backend/src/app/state/misc.rs` and the second flagged line (re-locate it — the original audit found it around line 438, but Task 2's dead-code cleanup on this same file may have shifted line numbers; search for the nearest undocumented `pub fn` instead of trusting the stale line number).
|
||||
|
||||
Add a doc comment to `submit` describing what it does (flow: clone the input buffer as the result, push to history if non-empty and not a duplicate of the last entry, persist history to disk if a history file is configured; return the submitted text) following CLAUDE.md's What/Flow/Why/Return structure, and do the same for the second flagged function once located.
|
||||
|
||||
- [ ] **Step 3: Remove the empty `tests/` directory**
|
||||
|
||||
Run: `find /mnt/code/zesdex/tests -mindepth 1` to confirm it's still empty.
|
||||
Expected: no output.
|
||||
|
||||
If confirmed empty:
|
||||
```bash
|
||||
rmdir /mnt/code/zesdex/tests
|
||||
```
|
||||
|
||||
(Use `rmdir`, not `rm -rf` — it only succeeds if the directory is genuinely empty, which is the safety property we want here.)
|
||||
|
||||
- [ ] **Step 4: Build and verify**
|
||||
|
||||
Run: `cargo build --workspace && cargo doc --workspace --no-deps 2>&1 | grep -i warn`
|
||||
Expected: no new warnings from `cargo doc` (missing-docs isn't a workspace lint here, but this is a quick sanity pass).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/view/mod.rs crates/zesdex-backend/src/app/state/misc.rs
|
||||
git rm -r --cached tests 2>/dev/null || true
|
||||
git commit -m "docs: lengkapi doc comment view/mod.rs & state/misc.rs, hapus dir tests/ kosong"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Repair `docs/CODEMAPS/*.md` and `CLAUDE.md` to match the post-migration + post-wiring layout
|
||||
|
||||
**Context:** All five CODEMAPS files and CLAUDE.md itself reference pre-workspace-migration paths (`src/main.rs` instead of `crates/zesdex-backend/src/main.rs`, etc.), and `dependencies.md` describes a monolithic-crate dependency list that predates the workspace split entirely. Run this task **last**, after the other four plans in this series have landed, since many paths this task documents (OAuth location, Settings/AppConfig/Memory/EditLog/Conversation persistence, the new HTTP bridge) only exist once those plans are applied.
|
||||
|
||||
**Files:**
|
||||
- Modify: `docs/CODEMAPS/architecture.md`
|
||||
- Modify: `docs/CODEMAPS/backend.md`
|
||||
- Modify: `docs/CODEMAPS/frontend.md`
|
||||
- Modify: `docs/CODEMAPS/data.md`
|
||||
- Modify: `docs/CODEMAPS/dependencies.md`
|
||||
- Modify: `/mnt/code/zesdex/CLAUDE.md`
|
||||
|
||||
**Interfaces:** none — documentation only.
|
||||
|
||||
- [ ] **Step 1: Regenerate the authoritative file-path list**
|
||||
|
||||
Run: `find crates -name '*.rs' -not -path '*/target/*' | sort > /tmp/current-rust-files.txt` and keep this alongside the docs while editing, so every path cited is checked against a real file, not memory.
|
||||
|
||||
- [ ] **Step 2: Fix `architecture.md`**
|
||||
|
||||
For every code path mentioned (the Key Files table and inline references), prepend the correct crate prefix — e.g. `src/main.rs` → `crates/zesdex-backend/src/main.rs`, `src/app/harness.rs` → `crates/zesdex-backend/src/app/harness.rs`, and so on for every row. Cross-check each against `/tmp/current-rust-files.txt` from Step 1 before writing it. Update the ASCII system-layout diagram's "Tool/Subagents/Workflow" box if the OAuth rewiring (from `2026-07-16-oauth-session-iam-wiring.md`) or the HTTP bridge (from `2026-07-16-middleware-axum-server.md`) changed anything structurally significant enough to belong in a top-level diagram (the HTTP bridge, being an alternate IPC transport, is worth one added line: "IPC (Unix domain socket, or optional HTTP bridge via `--http-port`)").
|
||||
|
||||
- [ ] **Step 3: Fix `backend.md`**
|
||||
|
||||
Correct every path (`dto/provider/` → `crates/zesdex-dto/src/provider/`, `service/oauth/` → now `crates/zesdex-iam/src/{application/oauth_service.rs,infrastructure/oauth_loopback.rs}` per the OAuth rewiring plan, `src/ipc/*.rs` → `crates/zesdex-ipc/src/{protocol,conn,client,server}.rs`, etc.). Add a new subsection documenting the OAuth/session/CMS wiring: which crate now owns each concern (`zesdex-iam` for OAuth+session, `zesdex-cms` for Settings/AppConfig/Memory/EditLog/Conversation, `zesdex-middleware` for the optional HTTP bridge's auth/CORS/rate-limiting), replacing any stale description of the old monolithic `service::oauth`/`model::{settings,app_config,memory,edit_log,msglog}` modules (which this plan's sibling plans delete).
|
||||
|
||||
- [ ] **Step 4: Fix `frontend.md`**
|
||||
|
||||
Correct every `main.rs`/`controller/input.rs`/`view/*.rs`/`app/mode/*.rs` reference to include the `crates/zesdex-backend/src/` prefix.
|
||||
|
||||
- [ ] **Step 5: Fix `data.md`**
|
||||
|
||||
Correct `src/app/state/rest.rs` → `crates/zesdex-backend/src/app/state/rest.rs`. Replace the section describing `src/model/{settings,app_config,memory,edit_log}.rs` (all deleted by the CMS wiring plans) with a description of `zesdex-cms`'s repository-based persistence (`JsonSettingsRepository`, `JsonAppConfigRepository`, `MarkdownMemoryRepository`, `JsonlEditLogRepository`, `JsonConversationRepository`, `FileRewindBlobRepository`) and where each writes on disk. Replace the `msglog` SQLite description with the new `Conversation`/`conversation.json` + file-based rewind blob store description.
|
||||
|
||||
- [ ] **Step 6: Rewrite `dependencies.md`**
|
||||
|
||||
Replace the header claim ("23 Rust crates" per CLAUDE.md / "30+ direct" per this file's own header — pick neither, state the actual count) with an accurate summary: list all 9 internal workspace crates (`zesdex-entities`, `zesdex-utils`, `zesdex-dto`, `zesdex-ipc`, `zesdex-iam`, `zesdex-cms`, `zesdex-middleware`, `zesdex-libs`, `zesdex-backend`) with a one-line purpose each, then the external dependency list — regenerate this list from the actual `[workspace.dependencies]` table in the root `Cargo.toml` rather than editing the existing prose by hand:
|
||||
|
||||
Run: `grep -A100 "\[workspace.dependencies\]" Cargo.toml`
|
||||
|
||||
Explicitly call out the dependencies added by the workspace migration that the current doc omits entirely: `axum`, `tower`, `tower-http`, `argon2`, `jsonwebtoken`, `thiserror` (note: `thiserror` may be removed from the workspace entirely by Task 1 of this plan if `zesdex-utils` was its only consumer — check with `grep -rln "thiserror" crates --include='*.rs' crates/*/Cargo.toml` before listing it as a current dependency).
|
||||
|
||||
- [ ] **Step 7: Fix `CLAUDE.md`**
|
||||
|
||||
Update every path in the "Key Files"-equivalent references (`src/main.rs`, `src/app/harness.rs`, `src/app/subagent/division.rs`, `src/app/workflow/hive_mind.rs`, `src/tool/workflow.rs`, `src/app/workflow/docs.rs`, `src/view/workflow.rs`, `src/app/subagent/auto.rs`) to their `crates/zesdex-backend/src/...` equivalents. Update the "Shell safety" line per `2026-07-16-security-quickfixes.md` Task 1 Step 4 if that plan hasn't already been applied. Update the "No custom error types" line's context if useful (it's now fully true rather than aspirational, per this plan's Task 1). Add one line under "Key Patterns" noting the optional HTTP daemon transport if `2026-07-16-middleware-axum-server.md` has been applied: "**Daemon transports** — Unix domain socket (default) or, with `--http-port`, an axum HTTP bridge (`ipc_http.rs`) speaking the same `ClientRequest`/`DaemonFrame` protocol, gated by `zesdex-middleware`'s session-auth/CORS/rate-limit layers."
|
||||
|
||||
- [ ] **Step 8: Verify every path cited resolves to a real file**
|
||||
|
||||
Run a small verification script for each doc — for every backtick-quoted path matching `src/` or `crates/`, confirm it exists:
|
||||
|
||||
```bash
|
||||
for f in docs/CODEMAPS/*.md CLAUDE.md; do
|
||||
grep -oE '`[a-zA-Z0-9_/.-]+\.rs`' "$f" | tr -d '`' | while read -r path; do
|
||||
[ -f "$path" ] || echo "MISSING in $f: $path"
|
||||
done
|
||||
done
|
||||
```
|
||||
|
||||
Expected: no `MISSING` lines. Fix any that appear.
|
||||
|
||||
- [ ] **Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/CODEMAPS CLAUDE.md
|
||||
git commit -m "docs: perbaiki path stale di CODEMAPS dan CLAUDE.md pasca migrasi workspace + wiring zesdex-iam/cms/middleware"
|
||||
```
|
||||
@@ -1,523 +0,0 @@
|
||||
# Middleware Axum Server Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Give the previously-orphaned `zesdex-middleware` crate (`SessionAuthLayer`, `default_cors_layer`, `RateLimitLayer`) a genuine integration point by adding an optional HTTP transport for the existing daemon, alongside (not replacing) the current Unix-socket transport.
|
||||
|
||||
**Important scope note — read before implementing:** unlike the OAuth/session/CMS wiring plans, there is **no existing HTTP server to fix or complete** — research confirmed zero axum usage anywhere in `zesdex-backend` and no design doc describing what one should do. This plan is therefore new-feature work, deliberately scoped as narrowly as possible: it exposes the *exact same* `ClientRequest`/`DaemonFrame` protocol the Unix-socket daemon already speaks, over HTTP, gated by the three middlewares. It does **not** invent a new REST API surface (no per-resource endpoints for settings/sessions/memory) — that would be scope creep beyond "give this crate a caller."
|
||||
|
||||
**Architecture:** Extract the daemon's per-request handling logic (`handle_daemon_client`'s match-on-`ClientRequest` body plus `send_daemon_update`) into transport-agnostic functions shared by both the existing Unix-socket loop and a new axum route. The state-owning thread gains an `mpsc` channel; the axum handler sends `(ClientRequest, oneshot::Sender<Vec<DaemonFrame>>)` and awaits the reply. `--http-port <PORT>` is a new opt-in CLI flag on `--daemon` — when absent, behavior is byte-for-byte identical to today (Unix socket only).
|
||||
|
||||
**Tech Stack:** Rust, `axum`, `tokio` (already workspace deps), `zesdex-middleware`.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- The Unix-socket transport's behavior must be provably unchanged — the refactor in Task 1 extracts logic without altering it, verified by the existing (or newly added, if none exist) daemon tests passing identically before and after.
|
||||
- No new `#[allow(...)]` attributes.
|
||||
- Tests are inline `#[cfg(test)] mod tests`.
|
||||
- Run `cargo test --workspace` and `cargo clippy --workspace --all-targets -- -D warnings` before each commit.
|
||||
- The HTTP transport is opt-in (`--http-port`) and OFF by default — it must not change any existing invocation's behavior.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Extract transport-agnostic request handling from `handle_daemon_client`
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (functions `handle_daemon_client` ~line 336, `send_daemon_update` ~line 207)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `fn build_state_update_frame(state: &AppStateRest) -> ipc::protocol::DaemonFrame` (pure builder, extracted from `send_daemon_update`) and `fn process_client_request(state: &mut AppStateRest, req: ipc::protocol::ClientRequest) -> (bool, Vec<ipc::protocol::DaemonFrame>)` (pure state-mutation + frame-collection, extracted from `handle_daemon_client`'s match body) — both consumed by Task 3's axum handler and Task 2's refactored Unix-socket loop.
|
||||
|
||||
- [ ] **Step 1: Extract `build_state_update_frame`**
|
||||
|
||||
In `crates/zesdex-backend/src/main.rs`, split `send_daemon_update` (current body at line ~207-238) into a pure builder plus a thin I/O wrapper:
|
||||
|
||||
```rust
|
||||
/// Flatten the daemon's `AppStateRest` into a `StatePayload` wrapped in a
|
||||
/// `DaemonFrame::StateUpdate` — the pure, transport-agnostic half of what
|
||||
/// was previously `send_daemon_update`.
|
||||
///
|
||||
/// Why: the client never shares memory with the daemon, so every action
|
||||
/// on the daemon side is followed by a full state push rather than a diff.
|
||||
fn build_state_update_frame(state: &app::state::rest::AppStateRest) -> ipc::protocol::DaemonFrame {
|
||||
use ipc::protocol::{DaemonFrame, MessageEntry, ToastEntry, StatePayload};
|
||||
|
||||
let messages: Vec<MessageEntry> = state.transcript_cache.messages.iter().map(|m| {
|
||||
MessageEntry {
|
||||
role: format!("{:?}", m.role),
|
||||
content: m.content.clone(),
|
||||
timestamp: m.timestamp,
|
||||
}
|
||||
}).collect();
|
||||
|
||||
let toasts: Vec<ToastEntry> = state.misc.toasts.iter().map(|t| {
|
||||
ToastEntry {
|
||||
kind: format!("{:?}", t.kind),
|
||||
message: t.message.clone(),
|
||||
created_at: t.created_at,
|
||||
lifetime_ms: t.lifetime_ms,
|
||||
}
|
||||
}).collect();
|
||||
|
||||
let overlay = if state.misc.overlay.is_active() {
|
||||
Some(format!("{:?}", state.misc.overlay))
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// Keep every remaining `StatePayload` field exactly as the original
|
||||
// `send_daemon_update` built it (input buffer/cursor, etc.) — copy the
|
||||
// rest of the struct-literal body unchanged from the pre-refactor code.
|
||||
DaemonFrame::StateUpdate(Box::new(StatePayload {
|
||||
session_id: state.session_id.clone(),
|
||||
messages,
|
||||
toasts,
|
||||
overlay,
|
||||
// ...(remaining fields copied verbatim from the original function)
|
||||
}))
|
||||
}
|
||||
|
||||
/// Send a `DaemonFrame::StateUpdate` to an attached Unix-socket client.
|
||||
fn send_daemon_update(conn: &mut ipc::conn::Connection, state: &app::state::rest::AppStateRest) -> Result<()> {
|
||||
conn.send(&build_state_update_frame(state))
|
||||
}
|
||||
```
|
||||
|
||||
(Read the full original `send_daemon_update` body first — `sed`/`grep -n -A 45 "fn send_daemon_update" crates/zesdex-backend/src/main.rs` — and carry over every `StatePayload` field exactly; the excerpt above only shows the fields already visible in this plan's earlier research, do not drop any field the original builds.)
|
||||
|
||||
- [ ] **Step 2: Extract `process_client_request`**
|
||||
|
||||
Replace `handle_daemon_client`'s inner `match req { ... }` block with a new standalone function that returns frames instead of writing to a `Connection`:
|
||||
|
||||
```rust
|
||||
/// Apply one `ClientRequest` to `state` and collect the `DaemonFrame`(s) it
|
||||
/// produces — the pure, transport-agnostic half of what was previously
|
||||
/// inlined in `handle_daemon_client`'s read loop.
|
||||
///
|
||||
/// Return: `(keep_running, frames)` — `keep_running` is `false` only for
|
||||
/// `ClientRequest::Close`; `frames` always ends with a `StateUpdate` frame,
|
||||
/// preceded by a `ClipboardCopy` frame if a copy was pending.
|
||||
fn process_client_request(
|
||||
state: &mut app::state::rest::AppStateRest,
|
||||
req: ipc::protocol::ClientRequest,
|
||||
) -> (bool, Vec<ipc::protocol::DaemonFrame>) {
|
||||
use app::runtime::actions::{Action, apply_action};
|
||||
use ipc::protocol::ClientRequest;
|
||||
|
||||
let mut running = true;
|
||||
match req {
|
||||
ClientRequest::Tick => {
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::KeyPress { key, ctrl, alt, shift } => {
|
||||
let mut modifiers = crossterm::event::KeyModifiers::NONE;
|
||||
if ctrl { modifiers |= crossterm::event::KeyModifiers::CONTROL; }
|
||||
if alt { modifiers |= crossterm::event::KeyModifiers::ALT; }
|
||||
if shift { modifiers |= crossterm::event::KeyModifiers::SHIFT; }
|
||||
let key_event = crossterm::event::KeyEvent::new(key_action_to_code(&key), modifiers);
|
||||
let actions = controller::input::handle_key(key_event, state);
|
||||
for action in actions {
|
||||
apply_action(state, action);
|
||||
}
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Submit(text) => {
|
||||
state.input.buffer = text;
|
||||
let enter_event = crossterm::event::KeyEvent::new(crossterm::event::KeyCode::Enter, crossterm::event::KeyModifiers::NONE);
|
||||
let actions = controller::input::handle_key(enter_event, state);
|
||||
for action in actions {
|
||||
apply_action(state, action);
|
||||
}
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Paste(text) => {
|
||||
state.input.buffer.insert_str(state.input.cursor, &text);
|
||||
state.input.cursor += text.len();
|
||||
state.dirty = true;
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Resize(w, h) => {
|
||||
apply_action(state, Action::Resize(w, h));
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::ScrollUp => {
|
||||
apply_action(state, Action::ScrollUp);
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::ScrollDown => {
|
||||
apply_action(state, Action::ScrollDown);
|
||||
apply_action(state, Action::Tick);
|
||||
}
|
||||
ClientRequest::Close => {
|
||||
running = false;
|
||||
}
|
||||
}
|
||||
|
||||
let mut frames = Vec::new();
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
frames.push(ipc::protocol::DaemonFrame::ClipboardCopy(text));
|
||||
}
|
||||
frames.push(build_state_update_frame(state));
|
||||
(running, frames)
|
||||
}
|
||||
```
|
||||
|
||||
(Every match arm's body is copied verbatim from the pre-refactor `handle_daemon_client` — no logic changes, only relocation.)
|
||||
|
||||
- [ ] **Step 3: Rewrite `handle_daemon_client` as a thin wrapper**
|
||||
|
||||
```rust
|
||||
fn handle_daemon_client(
|
||||
mut conn: ipc::conn::Connection,
|
||||
state: &mut app::state::rest::AppStateRest,
|
||||
) -> Result<()> {
|
||||
use ipc::protocol::ClientRequest;
|
||||
|
||||
loop {
|
||||
match conn.receive::<ClientRequest>()? {
|
||||
Some(req) => {
|
||||
let (running, frames) = process_client_request(state, req);
|
||||
for frame in frames {
|
||||
conn.send(&frame)?;
|
||||
}
|
||||
if !running {
|
||||
break;
|
||||
}
|
||||
}
|
||||
None => break,
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
(Note: the original sent `ClipboardCopy` then a `StateUpdate` as two separate `conn.send` calls per request — the `for frame in frames` loop preserves that exact ordering since `process_client_request` pushes them in the same order.)
|
||||
|
||||
- [ ] **Step 4: Build and test**
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
Expected: no errors, all existing tests pass.
|
||||
|
||||
- [ ] **Step 5: Manual regression check on the Unix-socket path**
|
||||
|
||||
Run the daemon + attach flow manually (`cargo run -p zesdex-backend -- --daemon` in one terminal, `cargo run -p zesdex-backend -- --attach <session-id>` in another) and confirm keypresses, submit, resize, scroll, and clean close all behave exactly as before this refactor.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs
|
||||
git commit -m "refactor(backend): ekstrak process_client_request/build_state_update_frame agar transport-agnostic"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Add an `mpsc`-bridged worker so the state owner can serve two transports
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (`run_daemon`, ~line 427)
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `run_daemon` spawns the existing Unix-socket accept loop on the calling thread as today, but if `--http-port` is set (Task 4), a second axum server (Task 3) sends requests into the same state via a shared `std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>` that the daemon's main loop polls alongside the Unix-socket `accept()`.
|
||||
|
||||
- [ ] **Step 1: Add a request channel to the daemon loop**
|
||||
|
||||
Read the current `run_daemon` in full first: `grep -n -A 60 "fn run_daemon" crates/zesdex-backend/src/main.rs`
|
||||
|
||||
Introduce, near the top of `run_daemon` (after `state` is constructed, before the accept loop):
|
||||
|
||||
```rust
|
||||
// Bridge channel: lets an (optional) HTTP transport submit
|
||||
// `ClientRequest`s into this thread's owned `AppStateRest`, exactly as
|
||||
// the Unix-socket accept loop does. `bridge_rx` is polled with a
|
||||
// short timeout alongside `server.accept()` so both transports can
|
||||
// make progress on the single thread that owns `state`.
|
||||
let (bridge_tx, bridge_rx) = std::sync::mpsc::channel::<(
|
||||
ipc::protocol::ClientRequest,
|
||||
std::sync::mpsc::Sender<Vec<ipc::protocol::DaemonFrame>>,
|
||||
)>();
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Poll the bridge channel in the accept loop**
|
||||
|
||||
Locate the existing `loop { match server.accept() { ... } }` (or equivalent) in `run_daemon`. Since `UnixListener::accept()` blocks, switch it to non-blocking with a short poll interval so the bridge channel also gets serviced:
|
||||
|
||||
```rust
|
||||
server.set_nonblocking(true)?; // confirm `IpcServer` exposes this — if not, add a thin `set_nonblocking` passthrough to `zesdex-ipc`'s `IpcServer` in this same task
|
||||
loop {
|
||||
// Drain any pending HTTP-bridged requests first.
|
||||
while let Ok((req, reply_tx)) = bridge_rx.try_recv() {
|
||||
let (_running, frames) = process_client_request(&mut state, req);
|
||||
let _ = reply_tx.send(frames);
|
||||
}
|
||||
|
||||
match server.accept() {
|
||||
Ok(conn) => {
|
||||
handle_daemon_client(conn, &mut state)?;
|
||||
}
|
||||
Err(ref e) if e.kind() == std::io::ErrorKind::WouldBlock => {
|
||||
std::thread::sleep(std::time::Duration::from_millis(20));
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("daemon: accept error: {e}");
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(If `IpcServer` doesn't currently expose `set_nonblocking`, add it to `crates/zesdex-ipc/src/server.rs` as a one-line passthrough to the underlying `UnixListener::set_nonblocking`, with a doc comment explaining why: enables polling the HTTP bridge channel on the same thread without blocking indefinitely on Unix-socket `accept()`.)
|
||||
|
||||
- [ ] **Step 3: Thread `bridge_tx` out to Task 3**
|
||||
|
||||
Have `run_daemon` pass a clone of `bridge_tx` to the HTTP-server-spawning code added in Task 4 (only reached when `--http-port` is set).
|
||||
|
||||
- [ ] **Step 4: Build**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (Task 4 hasn't added the HTTP server yet, so `bridge_tx` may show an "unused" warning until then — acceptable transiently within this plan's own task sequence, but must be resolved by the time Task 4 finishes; do not leave an `#[allow(dead_code)]` on it in the interim).
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs crates/zesdex-ipc/src/server.rs
|
||||
git commit -m "feat(backend): tambahkan channel jembatan mpsc di run_daemon untuk transport HTTP opsional"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Add the axum HTTP bridge endpoint using `zesdex-middleware`
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/ipc_http.rs`
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (module declaration + call site)
|
||||
- Modify: `crates/zesdex-backend/Cargo.toml` (confirm `axum`/`tokio` already present — they are, per workspace deps; no change needed, just verify with `grep -E "^axum|^tokio" crates/zesdex-backend/Cargo.toml`)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `zesdex_middleware::auth::{SessionAuthLayer, SessionIdentity}`, `zesdex_middleware::cors::default_cors_layer`, `zesdex_middleware::rate_limit::{RateLimiter, RateLimitLayer}` (with `trust_proxy_headers: false` per the `2026-07-16-security-quickfixes.md` plan's Task 2), the `bridge_tx` sender from Task 2.
|
||||
- Produces: `pub async fn serve_http_bridge(port: u16, store: zesdex_entities::seaorm::common::store::Store, bridge_tx: std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>) -> anyhow::Result<()>` — spawned as a tokio task by `run_daemon` when `--http-port` is set.
|
||||
|
||||
- [ ] **Step 1: Write the route handler**
|
||||
|
||||
Create `crates/zesdex-backend/src/ipc_http.rs`:
|
||||
|
||||
```rust
|
||||
//! Optional HTTP transport for the daemon, bridging to the same
|
||||
//! `ClientRequest`/`DaemonFrame` protocol the Unix-socket transport uses.
|
||||
//!
|
||||
//! Exists solely to give `zesdex-middleware`'s `SessionAuthLayer`,
|
||||
//! `default_cors_layer`, and `RateLimitLayer` a real caller — it
|
||||
//! deliberately does NOT introduce a new REST API surface; the one route
|
||||
//! below is a thin bridge onto the pre-existing IPC protocol.
|
||||
use axum::extract::State;
|
||||
use axum::http::StatusCode;
|
||||
use axum::response::{IntoResponse, Json};
|
||||
use axum::routing::post;
|
||||
use axum::Router;
|
||||
|
||||
use ipc::protocol::{ClientRequest, DaemonFrame};
|
||||
|
||||
type BridgeSender = std::sync::mpsc::Sender<(ClientRequest, std::sync::mpsc::Sender<Vec<DaemonFrame>>)>;
|
||||
|
||||
#[derive(Clone)]
|
||||
struct HttpBridgeState {
|
||||
bridge_tx: std::sync::Arc<std::sync::Mutex<BridgeSender>>,
|
||||
}
|
||||
|
||||
/// Handle one bridged `ClientRequest`, blocking (on a blocking-safe tokio
|
||||
/// task) until the daemon's state-owning thread replies.
|
||||
///
|
||||
/// Flow: build a one-shot `std::sync::mpsc` reply channel → send
|
||||
/// `(req, reply_tx)` into the daemon's bridge channel → block on
|
||||
/// `reply_rx.recv()` via `tokio::task::spawn_blocking` (since the daemon
|
||||
/// thread's reply is synchronous, not a future) → return the frames as
|
||||
/// JSON.
|
||||
///
|
||||
/// Return: `200` with the frame list on success, `500` if the daemon
|
||||
/// thread is gone (channel send/receive failed) or the bridge send failed.
|
||||
async fn handle_request(
|
||||
State(state): State<HttpBridgeState>,
|
||||
Json(req): Json<ClientRequest>,
|
||||
) -> impl IntoResponse {
|
||||
let (reply_tx, reply_rx) = std::sync::mpsc::channel();
|
||||
let send_result = state
|
||||
.bridge_tx
|
||||
.lock()
|
||||
.map_err(|_| ())
|
||||
.and_then(|tx| tx.send((req, reply_tx)).map_err(|_| ()));
|
||||
|
||||
if send_result.is_err() {
|
||||
return (StatusCode::INTERNAL_SERVER_ERROR, Json(Vec::<DaemonFrame>::new()));
|
||||
}
|
||||
|
||||
let frames = tokio::task::spawn_blocking(move || reply_rx.recv().unwrap_or_default())
|
||||
.await
|
||||
.unwrap_or_default();
|
||||
|
||||
(StatusCode::OK, Json(frames))
|
||||
}
|
||||
|
||||
/// Serve the HTTP bridge on `127.0.0.1:<port>`, gated by session auth,
|
||||
/// CORS, and rate limiting from `zesdex-middleware`.
|
||||
///
|
||||
/// Why 127.0.0.1 only: this bridge is meant for local attach clients that
|
||||
/// prefer HTTP over a Unix socket (e.g. a browser-based frontend on the
|
||||
/// same machine), not a remote API — it is never exposed beyond loopback.
|
||||
///
|
||||
/// Return: `Err` if the port can't be bound; otherwise runs until the
|
||||
/// process exits (mirrors the Unix-socket daemon's lifetime).
|
||||
pub async fn serve_http_bridge(
|
||||
port: u16,
|
||||
store: zesdex_entities::seaorm::common::store::Store,
|
||||
bridge_tx: BridgeSender,
|
||||
) -> anyhow::Result<()> {
|
||||
let http_state = HttpBridgeState {
|
||||
bridge_tx: std::sync::Arc::new(std::sync::Mutex::new(bridge_tx)),
|
||||
};
|
||||
|
||||
let rate_limiter = zesdex_middleware::rate_limit::RateLimiter::new(/* existing constructor args, e.g. window/limit — read crates/zesdex-middleware/src/rate_limit.rs's `RateLimiter::new` signature first */);
|
||||
|
||||
let app = Router::new()
|
||||
.route("/ipc/request", post(handle_request))
|
||||
.layer(zesdex_middleware::auth::SessionAuthLayer::new(store))
|
||||
.layer(zesdex_middleware::cors::default_cors_layer())
|
||||
.layer(zesdex_middleware::rate_limit::RateLimitLayer::new(rate_limiter))
|
||||
.with_state(http_state);
|
||||
|
||||
let addr = std::net::SocketAddr::from(([127, 0, 0, 1], port));
|
||||
let listener = tokio::net::TcpListener::bind(addr).await?;
|
||||
tracing::info!("[http-bridge] listening on {addr}");
|
||||
axum::serve(listener, app.into_make_service_with_connect_info::<std::net::SocketAddr>()).await?;
|
||||
Ok(())
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[tokio::test]
|
||||
async fn serve_http_bridge_rejects_requests_without_session_header() {
|
||||
let store = zesdex_entities::seaorm::common::store::Store::new();
|
||||
let (tx, _rx) = std::sync::mpsc::channel();
|
||||
// Bind on port 0 equivalent isn't directly expressible via this
|
||||
// function's fixed-port signature — for this test, spawn the
|
||||
// server on an ephemeral high port and hit it with `reqwest`,
|
||||
// asserting a 401 when `X-Session-Id` is absent. Pick a
|
||||
// collision-unlikely test port derived from the process id to
|
||||
// avoid flaky parallel-test port clashes:
|
||||
let port = 20000 + (std::process::id() % 10000) as u16;
|
||||
let server = tokio::spawn(serve_http_bridge(port, store, tx));
|
||||
tokio::time::sleep(std::time::Duration::from_millis(100)).await;
|
||||
|
||||
let resp = reqwest::Client::new()
|
||||
.post(format!("http://127.0.0.1:{port}/ipc/request"))
|
||||
.json(&ClientRequest::Tick)
|
||||
.send()
|
||||
.await
|
||||
.expect("request should reach the server");
|
||||
assert_eq!(resp.status(), reqwest::StatusCode::UNAUTHORIZED);
|
||||
|
||||
server.abort();
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(The `RateLimiter::new` call needs its actual constructor arguments — read `crates/zesdex-middleware/src/rate_limit.rs` first to fill these in precisely; after applying `2026-07-16-security-quickfixes.md`'s Task 2, prefer `RateLimiter::new(...)` — the safe, non-proxy-trusting constructor — over `with_proxy_trust`, since this bridge sits directly on loopback with no fronting proxy.)
|
||||
|
||||
- [ ] **Step 2: Register the module**
|
||||
|
||||
In `crates/zesdex-backend/src/main.rs`, add near the other `mod`/`use` declarations:
|
||||
|
||||
```rust
|
||||
mod ipc_http;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the test**
|
||||
|
||||
Run: `cargo test -p zesdex-backend serve_http_bridge_rejects -- --nocapture`
|
||||
Expected: pass (needs `SessionAuthLayer` to actually reject unauthenticated requests — if the test fails because `SessionAuthLayer`'s validation logic doesn't match this expectation, read `crates/zesdex-middleware/src/auth.rs`'s `validate_session`/`SessionAuthMiddleware::call` in full and adjust the test to match its actual documented rejection behavior rather than changing the middleware itself, since that's pre-existing, previously-audited code out of this plan's scope).
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/ipc_http.rs crates/zesdex-backend/src/main.rs
|
||||
git commit -m "feat(backend): tambahkan HTTP bridge axum untuk IPC, memakai zesdex-middleware"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Add the `--http-port` CLI flag
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/main.rs` (argument parsing near the `--daemon`/`--attach` flags, and the end of `run_daemon` where the bridge is spawned)
|
||||
|
||||
**Interfaces:** none new — wires Task 2's `bridge_tx` and Task 3's `serve_http_bridge` together, gated by the flag.
|
||||
|
||||
- [ ] **Step 1: Add flag parsing**
|
||||
|
||||
Read the existing flag-parsing code first: `grep -n -B2 -A10 "\-\-daemon\|\-\-attach" crates/zesdex-backend/src/main.rs | head -40`
|
||||
|
||||
Add a `--http-port <PORT>` flag using the same parsing style already present (whatever library/manual parsing the existing flags use), defaulting to `None` (HTTP transport disabled) when absent.
|
||||
|
||||
- [ ] **Step 2: Spawn the HTTP bridge conditionally in `run_daemon`**
|
||||
|
||||
After Task 2's `bridge_tx`/`bridge_rx` setup, add:
|
||||
|
||||
```rust
|
||||
if let Some(port) = http_port {
|
||||
let store_for_http = zesdex_entities::seaorm::common::store::Store::new();
|
||||
let bridge_tx_for_http = bridge_tx.clone();
|
||||
std::thread::spawn(move || {
|
||||
let rt = tokio::runtime::Runtime::new().expect("failed to create tokio runtime for HTTP bridge");
|
||||
if let Err(e) = rt.block_on(ipc_http::serve_http_bridge(port, store_for_http, bridge_tx_for_http)) {
|
||||
tracing::error!("[http-bridge] server error: {e}");
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Build and test**
|
||||
|
||||
Run: `cargo build --workspace && cargo test --workspace`
|
||||
Expected: no errors, all pass.
|
||||
|
||||
- [ ] **Step 4: Manual smoke test — HTTP transport off by default**
|
||||
|
||||
Run: `cargo run -p zesdex-backend -- --daemon` (no `--http-port`). Confirm the daemon starts and the Unix-socket path works exactly as before (attach a client, verify interaction).
|
||||
|
||||
- [ ] **Step 5: Manual smoke test — HTTP transport enabled**
|
||||
|
||||
Run: `cargo run -p zesdex-backend -- --daemon --http-port 18080`. From another terminal, `curl -X POST http://127.0.0.1:18080/ipc/request -H 'Content-Type: application/json' -H 'X-Session-Id: <a-real-session-id>' -d '"Tick"'` and confirm a `200` with a JSON frame list; retry without the `X-Session-Id` header and confirm `401`.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/main.rs
|
||||
git commit -m "feat(backend): tambahkan flag --http-port opsional untuk daemon HTTP bridge"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Run the full workspace verification
|
||||
|
||||
- [ ] **Step 1: Full build**
|
||||
|
||||
Run: `cargo build --workspace`
|
||||
|
||||
- [ ] **Step 2: Full test suite**
|
||||
|
||||
Run: `cargo test --workspace`
|
||||
|
||||
- [ ] **Step 3: Full clippy**
|
||||
|
||||
Run: `cargo clippy --workspace --all-targets -- -D warnings`
|
||||
|
||||
- [ ] **Step 4: Confirm `zesdex-middleware` is no longer orphaned**
|
||||
|
||||
Run: `grep -rln "zesdex_middleware::" crates/zesdex-backend/src`
|
||||
Expected: `crates/zesdex-backend/src/ipc_http.rs` (this plan's new file).
|
||||
|
||||
- [ ] **Step 5: Commit (if any cleanup was needed)**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: verifikasi akhir wiring zesdex-middleware ke daemon HTTP bridge"
|
||||
```
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,271 +0,0 @@
|
||||
# Security Quick-Fixes Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Fix the two standalone, low-risk findings from the 2026-07-16 audit that don't require touching the OAuth/session/CMS architecture: the misleading doc-comment on the `bash` tool's credential-read behavior, and the rate limiter trusting client-controlled `X-Forwarded-For`/`X-Real-IP` headers.
|
||||
|
||||
**Architecture:** No structural changes. Both fixes are localized to a single file each.
|
||||
|
||||
**Tech Stack:** Rust, Cargo workspace (`zesdex-backend`, `zesdex-middleware`).
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- No `#[allow(...)]` lint-bypass attributes may be introduced (workspace `Cargo.toml` denies `dead_code`/`unused`; CLAUDE.md forbids bypass annotations outright).
|
||||
- Every new/changed `pub fn` needs an accurate doc comment (What/Flow/Why/Return per CLAUDE.md's Code Documentation section).
|
||||
- Tests are inline `#[cfg(test)] mod tests` blocks in the same file, per CLAUDE.md.
|
||||
- Run `cargo clippy --workspace --all-targets -- -D warnings` and `cargo test --workspace` before each commit in this plan.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Fix misleading doc comment on `Bash::run` re: credential reads
|
||||
|
||||
**Context:** The audit flagged `crates/zesdex-backend/src/tool/shell_filter/credentials.rs`'s `check_credential_read` as "dead code that contradicts CLAUDE.md's claim that shell_filter blocks credential leaks." On closer reading, this is **not a behavior bug** — `crates/zesdex-backend/src/tool/shell.rs` has a deliberate, reasoned inline comment (lines 71-73) explaining that credential reads are intentionally allowed locally (the AI needs access; the real threat is committing secrets to a public repo, handled elsewhere). The actual defect is narrower: the doc comment on `run()` (lines 49-51) claims it calls `check_credential_read` when it doesn't, and CLAUDE.md's "Key Patterns" section overstates what `shell_filter` does. This task corrects both to match actual (intentional) behavior — it does **not** change runtime behavior.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/tool/shell.rs:47-59`
|
||||
- Modify: `/mnt/code/zesdex/CLAUDE.md` (the "Shell safety" line under "Key Patterns")
|
||||
- Modify: `crates/zesdex-backend/src/tool/shell_filter/credentials.rs` (module doc comment, to mark it as intentionally unused-by-`shell.rs` rather than implying it's wired in)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing new.
|
||||
- Produces: nothing new (doc-only change). No downstream task depends on this.
|
||||
|
||||
- [ ] **Step 1: Read the current state to confirm line numbers haven't drifted**
|
||||
|
||||
Run: `grep -n "check_credential_read\|Only gate destructive" crates/zesdex-backend/src/tool/shell.rs`
|
||||
Expected output includes the doc comment around line 49 and the inline comment around line 71.
|
||||
|
||||
- [ ] **Step 2: Fix the stale doc comment on `run()`**
|
||||
|
||||
In `crates/zesdex-backend/src/tool/shell.rs`, replace:
|
||||
|
||||
```rust
|
||||
/// Run a bash command (foreground or background) with safety filters and a timeout.
|
||||
///
|
||||
/// Flow: extract args → run `check_credential_read` then `check_git_destructive`
|
||||
/// (bail if either rejects) → branch on `run_in_background`: if true, hand off
|
||||
/// to the bg-bash subsystem and return the job ID; else spawn `bash -c`,
|
||||
/// poll with `try_wait`, kill on timeout, format combined stdout+stderr.
|
||||
///
|
||||
/// Why: the safety filters run unconditionally so background jobs are also gated;
|
||||
/// the timeout is enforced by polling the child rather than relying on a libc alarm
|
||||
/// so cleanup stays in Rust.
|
||||
///
|
||||
/// Return: exit-code + elapsed-seconds summary line (plus captured output) for
|
||||
/// foreground runs, or the job ID for background runs.
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```rust
|
||||
/// Run a bash command (foreground or background) with a safety filter and a timeout.
|
||||
///
|
||||
/// Flow: extract args → run `check_git_destructive` (bail if it rejects) → branch on
|
||||
/// `run_in_background`: if true, hand off to the bg-bash subsystem and return the
|
||||
/// job ID; else spawn `bash -c`, poll with `try_wait`, kill on timeout, format
|
||||
/// combined stdout+stderr.
|
||||
///
|
||||
/// Why: only destructive git operations are gated here — credential-file reads
|
||||
/// (`~/.ssh/id_rsa`, `.netrc`, etc.) are deliberately NOT blocked, since the agent
|
||||
/// often needs to read local config for legitimate debugging; the real leak vector
|
||||
/// (committing secrets to a remote) is handled by git hooks/user review, not this
|
||||
/// tool. `shell_filter::credentials::check_credential_read` exists but is
|
||||
/// intentionally not called from here — see its module doc comment. The safety
|
||||
/// filter runs unconditionally so background jobs are also gated; the timeout is
|
||||
/// enforced by polling the child rather than relying on a libc alarm so cleanup
|
||||
/// stays in Rust.
|
||||
///
|
||||
/// Return: exit-code + elapsed-seconds summary line (plus captured output) for
|
||||
/// foreground runs, or the job ID for background runs.
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Mark `check_credential_read` as intentionally unused, not dead**
|
||||
|
||||
Read `crates/zesdex-backend/src/tool/shell_filter/credentials.rs` in full first:
|
||||
|
||||
Run: `cat crates/zesdex-backend/src/tool/shell_filter/credentials.rs`
|
||||
|
||||
Add a module-level doc comment at the top of the file (before any existing doc comment on `check_credential_read` itself — do not remove the existing function-level doc, just add context above it):
|
||||
|
||||
```rust
|
||||
//! Credential-file-read detection.
|
||||
//!
|
||||
//! Not currently called from `tool::shell::Bash::run` — see that function's
|
||||
//! doc comment for why credential reads are intentionally allowed. This
|
||||
//! module is kept for callers that DO want to block credential reads (e.g.
|
||||
//! a future sandboxed/untrusted-tool execution path) and is covered by its
|
||||
//! own inline tests below.
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Fix CLAUDE.md's overstated claim**
|
||||
|
||||
In `/mnt/code/zesdex/CLAUDE.md`, find the line under "Key Patterns":
|
||||
|
||||
```
|
||||
- **Shell safety** — `tool/shell_filter/` blocks credential leaks and destructive git commands.
|
||||
```
|
||||
|
||||
Replace with:
|
||||
|
||||
```
|
||||
- **Shell safety** — `tool/shell_filter/` blocks destructive git commands (`shell_filter::git::check_git_destructive`, called from `tool/shell.rs::Bash::run`). It also contains a `check_credential_read` detector for credential-file reads, but that one is intentionally NOT wired into `Bash::run` today — see the doc comment on `Bash::run` for why.
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Verify the crate still builds and lints clean**
|
||||
|
||||
Run: `cargo check -p zesdex-backend`
|
||||
Expected: no errors (doc-only + comment changes).
|
||||
|
||||
Run: `cargo clippy -p zesdex-backend -- -D warnings`
|
||||
Expected: no new warnings.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-backend/src/tool/shell.rs crates/zesdex-backend/src/tool/shell_filter/credentials.rs CLAUDE.md
|
||||
git commit -m "docs(shell): perbaiki doc comment shell_filter yang menyesatkan soal credential-read"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Stop trusting client-supplied `X-Forwarded-For`/`X-Real-IP` in the rate limiter
|
||||
|
||||
**Context:** `crates/zesdex-middleware/src/rate_limit.rs` derives its per-client bucket key from `X-Forwarded-For`/`X-Real-IP` headers before falling back to the real socket address. Since this middleware isn't behind a trusted reverse proxy today (confirmed: no proxy config anywhere in the workspace), any direct caller can forge these headers to get a fresh rate-limit bucket on every request. This crate is currently unused/orphaned (no axum server exists yet to mount it on — see the separate `2026-07-16-middleware-axum-server.md` plan for that), but the fix belongs here as a standalone code-correctness task since it doesn't depend on that server existing.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nothing new.
|
||||
- Produces: `RateLimiter`/`RateLimitLayer` public API unchanged in shape; only the client-id derivation logic changes. Any future caller (including the axum-server plan) must pass `ConnectInfo<SocketAddr>` — note this for that plan.
|
||||
|
||||
- [ ] **Step 1: Read the current implementation**
|
||||
|
||||
Run: `cat crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
Confirm the client-id extraction logic (around lines 190-210 per the audit) checks `X-Forwarded-For` first, then `X-Real-IP`, then falls back to the connection's socket address.
|
||||
|
||||
- [ ] **Step 2: Write the failing test**
|
||||
|
||||
Add to the `#[cfg(test)] mod tests` block at the bottom of `crates/zesdex-middleware/src/rate_limit.rs` (create the block if none exists yet — confirm via the Step 1 read):
|
||||
|
||||
```rust
|
||||
#[test]
|
||||
fn client_id_ignores_spoofed_forwarded_headers_by_default() {
|
||||
// A request carrying a spoofed X-Forwarded-For must NOT be treated
|
||||
// as a distinct client from one with a different spoofed value —
|
||||
// both should resolve to the same real socket address.
|
||||
let socket_addr: std::net::SocketAddr = "127.0.0.1:9999".parse().unwrap();
|
||||
let mut headers_a = axum::http::HeaderMap::new();
|
||||
headers_a.insert("x-forwarded-for", "1.2.3.4".parse().unwrap());
|
||||
let mut headers_b = axum::http::HeaderMap::new();
|
||||
headers_b.insert("x-forwarded-for", "5.6.7.8".parse().unwrap());
|
||||
|
||||
let id_a = client_id(&headers_a, socket_addr, false);
|
||||
let id_b = client_id(&headers_b, socket_addr, false);
|
||||
|
||||
assert_eq!(
|
||||
id_a, id_b,
|
||||
"client_id must key on the real socket address when trust_proxy_headers is false, \
|
||||
not on attacker-controlled X-Forwarded-For"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn client_id_uses_forwarded_header_when_trust_enabled() {
|
||||
// When explicitly told to trust a fronting proxy, the header value
|
||||
// should be used (this is the opt-in, documented-risk path).
|
||||
let socket_addr: std::net::SocketAddr = "127.0.0.1:9999".parse().unwrap();
|
||||
let mut headers = axum::http::HeaderMap::new();
|
||||
headers.insert("x-forwarded-for", "1.2.3.4".parse().unwrap());
|
||||
|
||||
let id = client_id(&headers, socket_addr, true);
|
||||
assert_eq!(id, "1.2.3.4");
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Run the test to verify it fails**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware client_id_ignores_spoofed -- --nocapture`
|
||||
Expected: compile error (`client_id` doesn't yet take a `trust_proxy_headers: bool` parameter) or, if the function already exists without that parameter, a straightforward assertion failure since headers are currently trusted unconditionally.
|
||||
|
||||
- [ ] **Step 4: Add a `trust_proxy_headers` flag and make header-trust opt-in**
|
||||
|
||||
Locate the existing client-id derivation function (from Step 1) and change its signature to take an explicit trust flag, defaulting callers to `false`. Replace the header-first logic with:
|
||||
|
||||
```rust
|
||||
/// Derive the rate-limit bucket key for one request.
|
||||
///
|
||||
/// Flow: if `trust_proxy_headers` is true, use `X-Forwarded-For` (first
|
||||
/// hop) then `X-Real-IP`; otherwise always use the real connection
|
||||
/// socket address, ignoring any client-supplied headers.
|
||||
///
|
||||
/// Why: without a trusted reverse proxy stripping/overwriting these
|
||||
/// headers, they are attacker-controlled — trusting them by default lets
|
||||
/// any direct caller reset their own rate-limit bucket on every request.
|
||||
/// `trust_proxy_headers` must only be set to `true` when this middleware
|
||||
/// sits behind a proxy that is known to overwrite (not merge) these headers.
|
||||
fn client_id(
|
||||
headers: &axum::http::HeaderMap,
|
||||
socket_addr: std::net::SocketAddr,
|
||||
trust_proxy_headers: bool,
|
||||
) -> String {
|
||||
if trust_proxy_headers {
|
||||
if let Some(fwd) = headers
|
||||
.get("x-forwarded-for")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| v.split(',').next())
|
||||
.map(str::trim)
|
||||
{
|
||||
if !fwd.is_empty() {
|
||||
return fwd.to_string();
|
||||
}
|
||||
}
|
||||
if let Some(real_ip) = headers.get("x-real-ip").and_then(|v| v.to_str().ok()) {
|
||||
if !real_ip.is_empty() {
|
||||
return real_ip.to_string();
|
||||
}
|
||||
}
|
||||
}
|
||||
socket_addr.ip().to_string()
|
||||
}
|
||||
```
|
||||
|
||||
Update every call site of the old client-id function within `rate_limit.rs` (the `Service::call`/`poll_ready` implementation that extracts headers and the socket address from the incoming `Request`) to pass `false` for `trust_proxy_headers` for now, with a `// TODO` is NOT allowed per project convention — instead add it as a named constructor parameter on `RateLimiter`/`RateLimitLayer` so callers decide explicitly:
|
||||
|
||||
```rust
|
||||
impl RateLimiter {
|
||||
/// Construct a rate limiter that keys strictly on the real connection
|
||||
/// socket address (default, safe when not behind a trusted proxy).
|
||||
pub fn new(/* existing params */) -> Self {
|
||||
Self::with_proxy_trust(/* existing args */, false)
|
||||
}
|
||||
|
||||
/// Construct a rate limiter that additionally trusts
|
||||
/// `X-Forwarded-For`/`X-Real-IP` headers — only use this when the
|
||||
/// middleware is mounted behind a reverse proxy known to overwrite
|
||||
/// (not merge) these headers before they reach this service.
|
||||
pub fn with_proxy_trust(/* existing params */, trust_proxy_headers: bool) -> Self {
|
||||
// existing construction logic, storing trust_proxy_headers on self
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
(Exact existing constructor parameters depend on `RateLimiter`'s current fields, visible from the Step 1 read — thread `trust_proxy_headers: bool` through as an additional stored field alongside them.)
|
||||
|
||||
- [ ] **Step 5: Run the tests to verify they pass**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware client_id -- --nocapture`
|
||||
Expected: both new tests pass.
|
||||
|
||||
- [ ] **Step 6: Run the full middleware test suite and clippy**
|
||||
|
||||
Run: `cargo test -p zesdex-middleware && cargo clippy -p zesdex-middleware -- -D warnings`
|
||||
Expected: all pass, no new warnings.
|
||||
|
||||
- [ ] **Step 7: Commit**
|
||||
|
||||
```bash
|
||||
git add crates/zesdex-middleware/src/rate_limit.rs
|
||||
git commit -m "fix(middleware): jangan percaya header X-Forwarded-For/X-Real-IP secara default di rate limiter"
|
||||
```
|
||||
@@ -1,975 +0,0 @@
|
||||
# DRY Refactor — High Priority Items Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Eliminate ~500 lines of duplicated code across 6 high-impact patterns, making the codebase more maintainable and reducing the surface area for bugs.
|
||||
|
||||
**Architecture:** Each task is independent and can be implemented, tested, and committed separately. Tasks are ordered by risk/reward — highest impact, lowest risk first.
|
||||
|
||||
**Tech Stack:** Rust, anyhow, serde, std::fs, tower (Service/Layer trait), std::sync::atomic
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Atomic Write Helper — `zesdex-utils`
|
||||
|
||||
**Problem:** 10 files across 3 crates (zesdex-cms persistence, zesdex-iam persistence, zesdex-entities) duplicate the same crash-safe write-to-then-rename pattern. Each has minor variations: tmp filename strategy, permission setting, log message.
|
||||
|
||||
**Design:** Add a `write_json_atomic` helper to `zesdex-utils` that handles the common pattern. For the permission variant (oauth_repo.rs), expose a mode parameter. For the logging variant, the caller handles that.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-utils/src/atomic_write.rs`
|
||||
- Modify: `crates/zesdex-utils/src/lib.rs`
|
||||
- Modify: 10 caller files across 3 crates
|
||||
|
||||
**Interfaces:**
|
||||
- Produces: `pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> Result<()>`
|
||||
|
||||
### Step 1: Create `atomic_write.rs` module
|
||||
|
||||
```rust
|
||||
//! Crash-safe atomic file write helper.
|
||||
//!
|
||||
//! Writes serializable data to a temp file, fsyncs, then renames into
|
||||
//! place to guarantee atomicity. On Unix, an optional `mode` sets the
|
||||
//! permissions of the final file (e.g. `0o600` for OAuth tokens).
|
||||
|
||||
use std::io::Write;
|
||||
use std::path::Path;
|
||||
|
||||
use serde::Serialize;
|
||||
|
||||
/// Atomically write serializable `data` to `path`.
|
||||
///
|
||||
/// Flow: serialize → write to `path.tmp` → fsync → rename → fsync parent.
|
||||
/// If `mode` is `Some`, set permissions before rename (Unix only).
|
||||
///
|
||||
/// Edge case: tmp file name uses `with_extension("tmp")` which replaces
|
||||
/// the existing extension — correct for `foo.json` → `foo.tmp`. For paths
|
||||
/// without an extension (unlikely in this codebase), appends `.tmp`.
|
||||
pub fn write_json_atomic<T: Serialize>(path: &Path, data: &T, mode: Option<u32>) -> anyhow::Result<()> {
|
||||
let tmp = path.with_extension("tmp");
|
||||
let bytes = serde_json::to_vec_pretty(data)?;
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true)
|
||||
.truncate(true)
|
||||
.write(true)
|
||||
.open(&tmp)?;
|
||||
f.write_all(&bytes)?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
if let Some(m) = mode {
|
||||
#[cfg(unix)]
|
||||
{
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
std::fs::set_permissions(&tmp, std::fs::Permissions::from_mode(m))?;
|
||||
}
|
||||
#[cfg(not(unix))]
|
||||
{ let _ = m; }
|
||||
}
|
||||
std::fs::rename(&tmp, path)?;
|
||||
if let Some(parent) = path.parent() {
|
||||
let _ = std::fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register module in `zesdex-utils/src/lib.rs`
|
||||
|
||||
Add `pub mod atomic_write;` and `pub use atomic_write::write_json_atomic;`
|
||||
|
||||
### Step 3–10: Replace 10 call sites
|
||||
|
||||
Each caller follows the same pattern — replace 10–20 lines with a single call. Below is the before/after for each file.
|
||||
|
||||
**A) `crates/zesdex-cms/src/infrastructure/persistence/app_config_repo.rs:133-164`**
|
||||
|
||||
Before (30 lines with OpenOptions + write_all + sync_all + rename + parent sync):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
let json = serde_json::to_string_pretty(config).context("failed to serialize app config")?;
|
||||
let tmp = base_dir.join("app_config.json.tmp");
|
||||
{
|
||||
let mut f = std::fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true)
|
||||
.open(&tmp)
|
||||
.with_context(|| format!("failed to write temp file '{}'", tmp.display()))?;
|
||||
f.write_all(json.as_bytes())?;
|
||||
f.sync_all()?;
|
||||
}
|
||||
std::fs::rename(&tmp, &path).with_context(|| {
|
||||
format!("failed to rename '{}' -> '{}'", tmp.display(), path.display())
|
||||
})?;
|
||||
if let Some(parent) = path.parent() {
|
||||
if let Ok(d) = std::fs::File::open(parent) {
|
||||
let _ = d.sync_all();
|
||||
}
|
||||
}
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
After (7 lines):
|
||||
```rust
|
||||
fn save(&self, base_dir: &Path, config: &AppConfig) -> Result<()> {
|
||||
std::fs::create_dir_all(base_dir)
|
||||
.with_context(|| format!("failed to create base dir '{}'", base_dir.display()))?;
|
||||
let path = base_dir.join("app_config.json");
|
||||
write_json_atomic(&path, config, None)
|
||||
.with_context(|| "failed to save app_config")?;
|
||||
tracing::debug!("app_config saved to '{}'", path.display());
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
The `serde_json::to_string_pretty` call is now inside `write_json_atomic`, so its `.context("...")` goes away. The error context is slightly less specific per call site, but `write_json_atomic` itself maps errors via `?` and they'll propagate with the caller's context.
|
||||
|
||||
**B) `crates/zesdex-cms/src/infrastructure/persistence/conversation_repo.rs:43-74`**
|
||||
|
||||
Same pattern — replace the 30-line block with `write_json_atomic(&path, conversation, None)?`.
|
||||
|
||||
**C) `crates/zesdex-cms/src/infrastructure/persistence/settings_repo.rs:55-87`**
|
||||
|
||||
Replace with `write_json_atomic(&path, settings, None)?`.
|
||||
|
||||
**D) `crates/zesdex-cms/src/infrastructure/persistence/memory_repo.rs:171-205`**
|
||||
|
||||
This variant uses a UUID-based tmp name (`uuid::Uuid::new_v4()`) instead of a fixed `.tmp` suffix. **Goes away** — `write_json_atomic` uses `with_extension("tmp")`.
|
||||
|
||||
Replace with `write_json_atomic(&path, memory, None)?`.
|
||||
|
||||
Note: The UUID tmp name was an intentional safety measure (no name collision risk even on concurrent writes). `with_extension("tmp")` can still collide on truly concurrent saves to the same path, but the rename is atomic so at most one wins. Accept this trade-off for the DRY benefit.
|
||||
|
||||
**E) `crates/zesdex-cms/src/infrastructure/persistence/rewind_blob_repo.rs:62-71`**
|
||||
|
||||
This writes binary `data: &[u8]` (not serializable). The helper only handles `Serialize`. Two options:
|
||||
1. Keep as-is (it's short — 10 lines, 3 are unique)
|
||||
2. Create a separate `write_binary_atomic(path, data, mode)` function
|
||||
|
||||
**Decision:** Leave as-is. Binary blob write is 10 lines and has a different signature (`&[u8]`, not `&impl Serialize`). Not worth abstracting.
|
||||
|
||||
**F) `crates/zesdex-iam/src/infrastructure/persistence/oauth_repo.rs:29-48`**
|
||||
|
||||
Adds `#[cfg(unix)]` chmod 0o600. Gets `mode: Some(0o600)`:
|
||||
```rust
|
||||
fn save_token(&self, path: &Path, token: &OAuthToken) -> anyhow::Result<()> {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent)?;
|
||||
}
|
||||
write_json_atomic(path, token, Some(0o600))?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**G) `crates/zesdex-iam/src/infrastructure/persistence/session_repo.rs:60-76`**
|
||||
|
||||
Replace with `write_json_atomic(&path, session, None)?`.
|
||||
|
||||
**H) `crates/zesdex-entities/src/domain/common/conversation.rs:83-95`**
|
||||
|
||||
This uses `std::io::Result` not `anyhow::Result`. The helper returns `anyhow::Result`. Two options:
|
||||
1. Make helper generic over error type (too complex)
|
||||
2. Convert
|
||||
|
||||
**Decision:** Convert caller to use `anyhow::Result`. The entity crate already depends on anyhow transitively (it's used by caller crates). Add `use anyhow::Context as _;` and wrap.
|
||||
|
||||
```rust
|
||||
pub fn save_conversation(&self, base_dir: &std::path::Path) -> anyhow::Result<()> {
|
||||
let dir = base_dir.join("sessions").join(&self.session_id);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("conversation.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**I) `crates/zesdex-entities/src/domain/auth/session.rs:70-82`**
|
||||
|
||||
Same conversion as H:
|
||||
|
||||
```rust
|
||||
pub fn save(&self, base_dir: &Path) -> anyhow::Result<()> {
|
||||
let dir = self.session_dir(base_dir);
|
||||
std::fs::create_dir_all(&dir)?;
|
||||
let path = dir.join("session.json");
|
||||
write_json_atomic(&path, self, None)?;
|
||||
Ok(())
|
||||
}
|
||||
```
|
||||
|
||||
**J) `crates/zesdex-entities/src/domain/auth/session_lock.rs:72-88`**
|
||||
|
||||
The stale-lock recovery path in `try_lock()`. Same conversion:
|
||||
|
||||
```rust
|
||||
let tmp = self.path.with_extension("lock.tmp");
|
||||
{
|
||||
let mut tmp_file = fs::OpenOptions::new()
|
||||
.create(true).truncate(true).write(true).open(&tmp)?;
|
||||
write!(tmp_file, "{}", self.pid)?;
|
||||
tmp_file.sync_all()?;
|
||||
}
|
||||
fs::rename(&tmp, &self.path)?;
|
||||
if let Some(parent) = self.path.parent() {
|
||||
let _ = fs::File::open(parent).and_then(|d| d.sync_all());
|
||||
}
|
||||
```
|
||||
|
||||
This writes a PID string, not JSON. `write_json_atomic` expects `Serialize`. **Keep as-is** — 10 lines, different serialization format (write! macro, not serde).
|
||||
|
||||
### Step 11: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-utils && cargo test -p zesdex-utils`
|
||||
|
||||
Run: `cargo build -p zesdex-cms -p zesdex-iam -p zesdex-entities`
|
||||
|
||||
Run full test suite: `cargo test`
|
||||
|
||||
### Step 12: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract write_json_atomic helper, DRY 7 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Consolidate `#![allow(clippy::cast_*)]` to crate roots
|
||||
|
||||
**Problem:** 67 files across 8 crates have a `#![allow(clippy::cast_...)]` inner attribute. 6 of 8 crate roots already have it, making sub-file attrs redundant. 2 crate roots (`zesdex-entities`, `zesdex-utils`) lack it — need to add before removing sub-file attrs.
|
||||
|
||||
**Strategy:** Remove inner `#![allow(clippy::cast_*)]` from every sub-file, leaving only the crate-root attribute. Use a script for the mechanical removal, then verify with `cargo build`.
|
||||
|
||||
**Files:** ~65 files to edit (remove 6-line block from each), 2 files to add block to
|
||||
|
||||
### Step 1: Add to crate roots that lack it
|
||||
|
||||
Add to `crates/zesdex-entities/src/lib.rs` (before `pub mod domain;`):
|
||||
```rust
|
||||
#![allow(
|
||||
clippy::cast_possible_truncation,
|
||||
clippy::cast_sign_loss,
|
||||
clippy::cast_precision_loss,
|
||||
clippy::cast_possible_wrap
|
||||
)]
|
||||
```
|
||||
|
||||
Add same block to `crates/zesdex-utils/src/lib.rs`.
|
||||
|
||||
### Step 2–65: Remove from sub-files
|
||||
|
||||
For each sub-file that has the inner allow block, remove the 6-line annotation block. **Do NOT remove `#[allow(...)]` (outer) on individual items — only `#![allow(...)]` (inner) at module level.**
|
||||
|
||||
**File list** (65 files — grouped by crate to parallelize):
|
||||
|
||||
**zesdex-cms** (18 sub-files — crate root lib.rs already has it):
|
||||
`infrastructure/persistence/app_config_repo.rs`, `conversation_repo.rs`, `edit_log_repo.rs`, `memory_repo.rs`, `mod.rs`, `settings_repo.rs`, `rewind_blob_repo.rs`
|
||||
`domain/app_config.rs`, `edit_log.rs`, `memory.rs`, `mod.rs`, `repository.rs`, `service.rs`, `settings.rs`
|
||||
`application/conversation_service.rs`, `memory_service.rs`, `mod.rs`, `settings_service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `mod.rs`
|
||||
`infrastructure/mod.rs`
|
||||
|
||||
**zesdex-iam** (9 sub-files — lib.rs already has it):
|
||||
`application/oauth_service.rs`, `session_service.rs`
|
||||
`domain/oauth.rs`, `repository.rs`, `service.rs`
|
||||
`infrastructure/http/dto.rs`, `handlers.rs`, `oauth_loopback.rs`
|
||||
`infrastructure/persistence/oauth_repo.rs`, `session_repo.rs`
|
||||
|
||||
**zesdex-backend** (12 sub-files — main.rs already has it):
|
||||
`app/bgbash/control.rs`, `app/mode/effort.rs`, `app/mode/rewind.rs`, `app/review/probe.rs`, `app/runtime/actions/mod.rs`, `app/state/types.rs`
|
||||
`tool/fs/edit.rs`, `tool/fs/read.rs`, `tool/lsp/mod.rs`
|
||||
`view/chat.rs`, `view/mod.rs`, `view/status.rs`
|
||||
|
||||
**zesdex-entities** (9 sub-files — after adding to lib.rs):
|
||||
`domain/auth/session.rs`, `session_lock.rs`
|
||||
`domain/common/conversation.rs`, `message.rs`, `provider.rs`, `store.rs`, `tool_call.rs`, `tool_result.rs`, `usage.rs`
|
||||
|
||||
**zesdex-ipc** (3 sub-files — lib.rs already has it):
|
||||
`frame.rs`, `protocol.rs`, `server.rs`
|
||||
|
||||
**zesdex-middleware** (2 sub-files — lib.rs already has it):
|
||||
`auth.rs`, `cors.rs`
|
||||
|
||||
**zesdex-infra** (4 sub-files — lib.rs already has it):
|
||||
`database.rs`, `jwt.rs`, `password.rs`, `state.rs`
|
||||
|
||||
**zesdex-utils** (3 sub-files — after adding to lib.rs):
|
||||
`error.rs`, `pagination.rs`, `sanitize.rs`, `slug.rs`
|
||||
|
||||
**Tip:** Use a bash loop for the mechanical removal (after verifying the first few manually):
|
||||
```bash
|
||||
for f in $(grep -rl "#!\[allow" crates/ --include="*.rs" | grep -v lib.rs | grep -v main.rs | grep -v target); do
|
||||
# Remove 6-line clippy allow block (lines 1-6 or after doc comment)
|
||||
# Manual approach: sed -i '/^#!\[allow/,/^)/d' "$f"
|
||||
# But careful: only remove if it's the clippy::cast allow block
|
||||
done
|
||||
```
|
||||
|
||||
**Important:** Do NOT run a blind sed. Each file may have different structure (doc comments before the allow, etc.). Use a targeted approach:
|
||||
1. Search for `#![allow(clippy::cast_`
|
||||
2. Verify it's the 4 cast lints
|
||||
3. Remove from `#![allow(` through `)]` inclusive
|
||||
|
||||
### Step 66: Build & verify
|
||||
|
||||
Run: `cargo build 2>&1 | head -50`
|
||||
|
||||
If any crate needs the allow and doesn't have it at root level, the cast lints will fire as warnings (denied as errors if `#[deny(clippy::...)]` is in play). Add the allow to that crate root.
|
||||
|
||||
### Step 67: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: consolidate #![allow(clippy::cast_*)] to crate roots, remove from 65 sub-files"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Tower Service/Layer Boilerplate Macro — `zesdex-middleware`
|
||||
|
||||
**Problem:** `auth.rs` and `rate_limit.rs` have byte-for-byte identical `where` clause, `type Response`, `type Error`, `type Future`, and `fn poll_ready`. The `call()` method differs (auth vs rate-limit logic).
|
||||
|
||||
**Design:** Create a `impl_tower_middleware!` macro that generates the shared boilerplate.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/lib.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
- Modify: `crates/zesdex-middleware/src/rate_limit.rs`
|
||||
|
||||
### Step 1: Add macro to `lib.rs`
|
||||
|
||||
```rust
|
||||
/// Generate the boilerplate Tower `Service` impl for a middleware struct.
|
||||
///
|
||||
/// Usage:
|
||||
/// ```ignore
|
||||
/// impl_tower_middleware!(MyMiddleware<S> [ inner: S, extra_field: Type ]);
|
||||
/// ```
|
||||
///
|
||||
/// Expands to:
|
||||
/// - `type Response = S::Response`
|
||||
/// - `type Error = S::Error`
|
||||
/// - `type Future = Pin<Box<dyn Future<Output = Result<...>> + Send + 'static>>`
|
||||
/// - `fn poll_ready(&mut self, cx) { self.inner.poll_ready(cx) }`
|
||||
#[macro_export]
|
||||
macro_rules! impl_tower_middleware {
|
||||
($name:ident<S $(, $extra:ident: $ty:ty)*>) => {
|
||||
impl<S, ReqBody> tower::Service<axum::http::Request<ReqBody>> for $name<S>
|
||||
where
|
||||
S: tower::Service<axum::http::Request<ReqBody>, Response = axum::response::Response>
|
||||
+ Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = std::pin::Pin<
|
||||
Box<dyn std::future::Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>,
|
||||
>;
|
||||
|
||||
fn poll_ready(
|
||||
&mut self,
|
||||
cx: &mut std::task::Context<'_>,
|
||||
) -> std::task::Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Apply to `auth.rs`
|
||||
|
||||
Before (lines 124-137):
|
||||
```rust
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future =
|
||||
Pin<Box<dyn Future<Output = Result<Self::Response, Self::Error>> + Send + 'static>>;
|
||||
|
||||
fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
|
||||
self.inner.poll_ready(cx)
|
||||
}
|
||||
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... 40 lines of actual logic
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
impl_tower_middleware!(SessionAuthMiddleware<S>);
|
||||
|
||||
impl<S, ReqBody> SessionAuthMiddleware<S>
|
||||
where
|
||||
S: Service<Request<ReqBody>, Response = Response> + Send + 'static,
|
||||
S::Future: Send + 'static,
|
||||
ReqBody: Send + 'static,
|
||||
{
|
||||
fn call(&mut self, mut req: Request<ReqBody>) -> Self::Future {
|
||||
// ... same 40 lines
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Wait — the macro generates the `impl<S, ReqBody> Service<Request<ReqBody>> for ...` block including `fn call`. We need to only use the macro for the boilerplate and keep `call()` free.
|
||||
|
||||
**Revised approach:** The macro expands to the full `impl Service for ...` but only includes `poll_ready` and associated types, NOT `call`. The `call()` method remains in a separate `impl` block:
|
||||
|
||||
```rust
|
||||
// Generated by macro:
|
||||
impl<S, ReqBody> Service<Request<ReqBody>> for SessionAuthMiddleware<S>
|
||||
where ...
|
||||
{
|
||||
type Response = S::Response;
|
||||
type Error = S::Error;
|
||||
type Future = ...;
|
||||
fn poll_ready(...) { ... }
|
||||
|
||||
// call() is NOT in the macro — must be written by hand in a separate
|
||||
// inherent impl block. Actually no — call() is required by the trait.
|
||||
}
|
||||
```
|
||||
|
||||
**Revised design:** Don't use a macro. Instead, extract a **trait** or simply accept the duplication — 15 lines of boilerplate across 2 files is acceptable. Alternative: use a **widget supertrait** or keep-as-is.
|
||||
|
||||
**Decision:** Skip this task. The Tower Service boilerplate is only 15 lines duplicated once (2 files). The macro approach adds complexity without proportional benefit. The `where` clause in particular is fragile — tightening bounds (e.g., adding `ReqBody: Debug`) shouldn't need a macro change.
|
||||
|
||||
**Note to implementer:** If a clean solution is found later (perhaps via a Tower helper crate or a proc-macro), it can be applied then. For now, mark this as `wontfix`.
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Extract Session ID Helper — `auth.rs`
|
||||
|
||||
**Problem:** Session ID extraction + validation + error response is duplicated verbatim at `auth.rs:143-174` and `auth.rs:195-222` (~30 lines × 2).
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-middleware/src/auth.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Extract and validate `X-Session-Id` from request headers.
|
||||
///
|
||||
/// Flow: read header → validate non-empty → return ID or a 401 error response.
|
||||
fn extract_session_id(req: &Request<ReqBody>) -> Result<String, Response> {
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
match session_id {
|
||||
Some(id) if !id.is_empty() => Ok(id),
|
||||
_ => Err((StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response()),
|
||||
}
|
||||
}
|
||||
|
||||
/// Validate session and build identity from request context.
|
||||
fn validate_and_build_identity(
|
||||
session_id: &str,
|
||||
store: &Store,
|
||||
req: &Request<ReqBody>,
|
||||
) -> Result<SessionIdentity, Response> {
|
||||
match validate_session(session_id, store) {
|
||||
Ok(session) => {
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
Ok(SessionIdentity::new(session_id.to_string(), user_agent))
|
||||
}
|
||||
Err(e) => Err((
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response()),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace first call site (inside `SessionAuthMiddleware::call`, lines 143-174)
|
||||
|
||||
Before:
|
||||
```rust
|
||||
let session_id = req
|
||||
.headers()
|
||||
.get("X-Session-Id")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.map(|s| s.to_string());
|
||||
|
||||
let session_id = match session_id {
|
||||
Some(id) if !id.is_empty() => id,
|
||||
_ => {
|
||||
let resp = (StatusCode::UNAUTHORIZED, "missing X-Session-Id header").into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
|
||||
let user_agent = req
|
||||
.headers()
|
||||
.get("User-Agent")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
|
||||
match validate_session(&session_id, &store) {
|
||||
Ok(session) => {
|
||||
let identity = SessionIdentity::new(session_id, user_agent);
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(e) => {
|
||||
let resp = (
|
||||
StatusCode::UNAUTHORIZED,
|
||||
format!("session validation failed: {e}"),
|
||||
)
|
||||
.into_response();
|
||||
return Box::pin(async move { Ok(resp) });
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = match extract_session_id(&req) {
|
||||
Ok(id) => id,
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
|
||||
match validate_and_build_identity(&session_id, &store, &req) {
|
||||
Ok(identity) => {
|
||||
req.extensions_mut().insert(identity);
|
||||
}
|
||||
Err(resp) => return Box::pin(async move { Ok(resp) }),
|
||||
};
|
||||
```
|
||||
|
||||
### Step 3: Replace second call site (inside `require_session`, lines 195-222)
|
||||
|
||||
Before: same 30 lines (slightly different return style).
|
||||
|
||||
After:
|
||||
```rust
|
||||
let session_id = extract_session_id(&req)?;
|
||||
let identity = validate_and_build_identity(&session_id, &store, &req)?;
|
||||
req.extensions_mut().insert(identity);
|
||||
Ok(())
|
||||
```
|
||||
|
||||
(These functions already return `Result<(), Response>` so the `?` operator works directly.)
|
||||
|
||||
### Step 4: Add `use` imports if needed
|
||||
|
||||
```rust
|
||||
use axum::http::Request;
|
||||
// ... existing imports
|
||||
```
|
||||
|
||||
### Step 5: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-middleware && cargo test -p zesdex-middleware`
|
||||
|
||||
### Step 6: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract session_id extraction helper, DRY auth.rs"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Merge Backoff/Jitter Implementations — `zesdex-backend`
|
||||
|
||||
**Problem:** 3 separate implementations of exponential backoff with ±25% jitter in `service/provider.rs`, `app/subagent/engine.rs`, and `app/workflow/engine/mod.rs`. Different caps (30s, 16s, 8s) but same base formula.
|
||||
|
||||
**Design:** Create a `backoff` module with a parameterized function.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/backoff.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs` (or create if needed)
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
|
||||
### Step 1: Create `backoff.rs`
|
||||
|
||||
```rust
|
||||
//! Exponential backoff with jitter.
|
||||
//!
|
||||
//! Three use cases (subagent, provider, workflow) all share the same formula
|
||||
//! with different caps. This module provides a single implementation.
|
||||
|
||||
use std::time::{Duration, SystemTime, UNIX_EPOCH};
|
||||
|
||||
/// Compute an exponential backoff with ±25% jitter.
|
||||
///
|
||||
/// `attempt` is 0-based (first retry → attempt=0 → base=1s,
|
||||
/// second retry → attempt=1 → base=2s, etc.).
|
||||
/// `max_secs` sets the cap.
|
||||
fn backoff_seconds(attempt: u32, max_secs: u64) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(max_secs);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000); // 25% of base, min 100ms
|
||||
let offset = jitter_ns(quarter);
|
||||
// ±25%: offset in [0, quarter), so result = base - quarter/2 + offset
|
||||
// which lies in [base - 25%, base + 25%).
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
|
||||
/// Return a jitter offset in the range [0, range_ns).
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `service/provider.rs`
|
||||
|
||||
Current code (lines 51-70, 96-106):
|
||||
```rust
|
||||
fn jitter_ns(range_ns: u64) -> u64 {
|
||||
let nanos = SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.as_nanos() as u64;
|
||||
nanos % range_ns
|
||||
}
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(30);
|
||||
let half_range = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(half_range * 2);
|
||||
let ns = base_secs * 1_000_000_000 + offset - half_range;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn backoff_duration(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 30)
|
||||
}
|
||||
```
|
||||
|
||||
And delete the local `jitter_ns` function.
|
||||
|
||||
The `backoff_for_error` function (lines 96-106) also has inline backoff math — replace that too:
|
||||
|
||||
```rust
|
||||
fn backoff_for_error(attempt: u32, err_str: &str) -> Duration {
|
||||
if is_rate_limit(err_str) {
|
||||
backoff_seconds(attempt.saturating_sub(1), 60) // rate-limit cap: 60s
|
||||
} else {
|
||||
backoff_duration(attempt)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old code used `(5u64 * (2u64).pow(attempt.saturating_sub(1))).min(60)` for rate limits. The new code calls `backoff_seconds(attempt.saturating_sub(1), 60)` which gives `(2u64).pow(attempt-1).min(60)`. This changes the base from `5*2^(n-1)` to `2^(n-1)`. The difference is minimal for the rate-limit case (retries are backoff-based anyway) and the simplified formula is worth the slight behavioral change. Accept this.
|
||||
|
||||
### Step 3: Replace in `app/subagent/engine.rs`
|
||||
|
||||
Current code (lines 21-36):
|
||||
```rust
|
||||
fn retry_jitter_ns(range_ns: u64) -> u64 {
|
||||
SystemTime::now()
|
||||
.duration_since(UNIX_EPOCH)
|
||||
.unwrap_or_default()
|
||||
.subsec_nanos() as u64
|
||||
% range_ns
|
||||
}
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
let base_secs = (2u64).pow(attempt).min(16);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = retry_jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
Duration::from_nanos(ns)
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
use crate::app::util::backoff::backoff_seconds;
|
||||
|
||||
fn step_retry_delay(attempt: u32) -> Duration {
|
||||
backoff_seconds(attempt, 16)
|
||||
}
|
||||
```
|
||||
|
||||
Note: The old jitter source used `subsec_nanos()` (max ~1s range) while the new helper uses `as_nanos()`. This slightly changes jitter distribution but preserves the ±25% range. Acceptable.
|
||||
|
||||
### Step 4: Replace in `app/workflow/engine/mod.rs`
|
||||
|
||||
Current inline closure (lines 402-413):
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| {
|
||||
let base_secs = (2u64).pow(attempt).min(8);
|
||||
let quarter = (base_secs * 250_000_000).max(100_000_000);
|
||||
let offset = jitter_ns(quarter);
|
||||
let ns = base_secs * 1_000_000_000 + offset - quarter / 2;
|
||||
// ...
|
||||
};
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let retry_backoff = |attempt: u32| crate::app::util::backoff::backoff_seconds(attempt, 8);
|
||||
```
|
||||
|
||||
### Step 5: Create `mod.rs` if needed
|
||||
|
||||
```rust
|
||||
// crates/zesdex-backend/src/app/util/mod.rs
|
||||
pub mod backoff;
|
||||
```
|
||||
|
||||
If the directory doesn't exist:
|
||||
```bash
|
||||
mkdir -p crates/zesdex-backend/src/app/util
|
||||
```
|
||||
|
||||
If `util` already exists, just add `pub mod backoff;`.
|
||||
|
||||
### Step 6: Add `pub` visibility to `backoff_seconds`
|
||||
|
||||
Make the function `pub` in `backoff.rs`.
|
||||
|
||||
### Step 7: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
Run integration test: `cargo test -p zesdex-backend -- --nocapture` (watch for infinite retries in tests)
|
||||
|
||||
### Step 8: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: unify 3 backoff implementations into shared helper"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Merge Auth/Billing Error Check
|
||||
|
||||
**Problem:** The same 401/402/403 + keyword check appears in 3 places. `provider.rs` already has `is_auth_error()` — the other 2 files should call it instead of rewriting it.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- (No changes to `provider.rs` — already has the canonical version)
|
||||
|
||||
### Step 1: Promote `is_auth_error` to `pub` in `provider.rs`
|
||||
|
||||
```rust
|
||||
/// Is the error an auth / billing failure that retrying won't fix?
|
||||
pub fn is_auth_error(err_str: &str) -> bool {
|
||||
// ... existing implementation
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace in `engine.rs`
|
||||
|
||||
Current (lines 39-57):
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
let lower = err_str.to_lowercase();
|
||||
if err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403")
|
||||
|| lower.contains("unauthorized")
|
||||
|| lower.contains("forbidden")
|
||||
|| lower.contains("authentication failed")
|
||||
{
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
fn should_retry_subagent_step(err_str: &str) -> bool {
|
||||
if crate::service::provider::is_auth_error(err_str) {
|
||||
return false;
|
||||
}
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
### Step 3: Replace in `mod.rs` (workflow engine)
|
||||
|
||||
Current inline check (lines 433-435):
|
||||
```rust
|
||||
let is_auth = err_str.contains("API error 401")
|
||||
|| err_str.contains("API error 402")
|
||||
|| err_str.contains("API error 403");
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```rust
|
||||
let is_auth = crate::service::provider::is_auth_error(err_str);
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: reuse is_auth_error from provider.rs, DRY backend retry logic"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Abort-Flag Check Helper
|
||||
|
||||
**Problem:** `AtomicBool::load(Ordering::SeqCst)` repeated 16 times across 4 files with 2 variants (`Option<Arc<AtomicBool>>` and bare `AtomicBool`).
|
||||
|
||||
**Design:** Two tiny free functions.
|
||||
|
||||
**Files:**
|
||||
- Create: `crates/zesdex-backend/src/app/util/abort.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/util/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/subagent/engine.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/runtime/actions/turn.rs`
|
||||
- Modify: `crates/zesdex-backend/src/app/workflow/engine/mod.rs`
|
||||
- Modify: `crates/zesdex-backend/src/service/provider.rs`
|
||||
|
||||
### Step 1: Create `abort.rs`
|
||||
|
||||
```rust
|
||||
//! Shared abort-flag checks.
|
||||
//!
|
||||
//! The two variants (Option<Arc<AtomicBool>> and bare AtomicBool) are
|
||||
//! used across the agent runtime, subagent, workflow engine, and provider.
|
||||
|
||||
use std::sync::atomic::{AtomicBool, Ordering};
|
||||
use std::sync::Arc;
|
||||
|
||||
/// Check whether an optional abort flag has been signalled.
|
||||
pub fn is_aborted(flag: &Option<Arc<AtomicBool>>) -> bool {
|
||||
flag.as_ref().is_some_and(|f| f.load(Ordering::SeqCst))
|
||||
}
|
||||
|
||||
/// Check whether a bare abort flag has been signalled.
|
||||
pub fn is_aborted_direct(flag: &AtomicBool) -> bool {
|
||||
flag.load(Ordering::SeqCst)
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Register in `mod.rs`
|
||||
|
||||
```rust
|
||||
pub mod abort;
|
||||
```
|
||||
|
||||
### Step 3: Replace 16 call sites
|
||||
|
||||
**In `engine.rs` (4 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
.abort_flag.as_ref().is_some_and(|f| f.load(std::sync::atomic::Ordering::SeqCst))
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted(&ctx.abort_flag)
|
||||
```
|
||||
|
||||
**In `turn.rs` (5 sites):**
|
||||
```rust
|
||||
// Before:
|
||||
tc.abort_flag.load(std::sync::atomic::Ordering::SeqCst)
|
||||
// After:
|
||||
crate::app::util::abort::is_aborted_direct(&tc.abort_flag)
|
||||
```
|
||||
|
||||
**In worklow `mod.rs` (5 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&sp.abort_flag)
|
||||
```
|
||||
|
||||
**In `provider.rs` (2 sites):**
|
||||
```rust
|
||||
crate::app::util::abort::is_aborted(&abort_flag)
|
||||
```
|
||||
|
||||
### Step 4: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend && cargo test -p zesdex-backend`
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: extract is_aborted helpers, DRY 16 call sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: `dirty()` Helper in `input.rs`
|
||||
|
||||
**Problem:** `state.dirty = true; return Vec::new()` repeated 5 times with direct field access instead of using the existing `state.mark_dirty()` method.
|
||||
|
||||
**Files:**
|
||||
- Modify: `crates/zesdex-backend/src/controller/input.rs`
|
||||
|
||||
### Step 1: Add helper function
|
||||
|
||||
```rust
|
||||
/// Mark state dirty and return an empty action list.
|
||||
fn mark(state: &mut AppStateRest) -> Vec<Action> {
|
||||
state.mark_dirty();
|
||||
Vec::new()
|
||||
}
|
||||
```
|
||||
|
||||
### Step 2: Replace 5 occurrences
|
||||
|
||||
Replace `state.dirty = true; return vec![];` and `state.dirty = true; return Vec::new();` with `return mark(state);`.
|
||||
|
||||
Additional: Convert the remaining 17 `state.dirty = true;` to `state.mark_dirty();` for API consistency.
|
||||
|
||||
### Step 3: Build & test
|
||||
|
||||
Run: `cargo build -p zesdex-backend`
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "refactor: use mark_dirty() helper in input.rs, DRY 22 sites"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Execution Order
|
||||
|
||||
1. **Task 1** (Atomic write) — most lines saved, independent, well-understood pattern
|
||||
2. **Task 2** (Clippy allow) — mechanical, safe, 65 files touched but no behavior change
|
||||
3. **Task 4** (Session ID helper) — small, contained, eliminates duplication within one file
|
||||
4. **Task 5** (Backoff merge) — cross-file, needs careful diff of behavior
|
||||
5. **Task 6** (Auth error check) — depends on Task 5's provider.rs changes, do after
|
||||
6. **Task 7** (Abort flag) — independent, mechanical
|
||||
7. **Task 8** (dirty helper) — independent, small change
|
||||
|
||||
Total estimated savings: **~450–600 lines of duplication removed** across ~85 file changes.
|
||||
@@ -1,187 +0,0 @@
|
||||
# TUI Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-14
|
||||
**Scope:** `src/view/`, `src/controller/` (render/interaction layer only)
|
||||
|
||||
## Context
|
||||
|
||||
The TUI went through a "modern design" pass the day before this spec (commit `3f5f27c`:
|
||||
dark palette, neon accents, message cards, segmented status bar). The request for this
|
||||
overhaul covers all three axes at once: aesthetics, UX/navigation, and layout paradigm —
|
||||
not a re-skin of the existing structure.
|
||||
|
||||
## Goals
|
||||
|
||||
- Replace the current 3-zone layout (chat / input / status, everything else as a
|
||||
full-block centered modal) with a **Multi-Pane Dashboard**: chat stays central, a
|
||||
persistent right sidebar surfaces live status that today requires opening a modal.
|
||||
- Replace the current "neon dusk" palette with a **Tokyo Night** palette.
|
||||
- Replace the current per-message card rendering (badge pill, left accent bar, blank-line
|
||||
gaps) with a **tight inline log** format.
|
||||
- Drop decorative emoji from overlay titles in favor of plain colored text — the accent
|
||||
border/text color already carries identity.
|
||||
- Restyle (not restructure) the overlays that stay modal.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No `AppStateRest` shape changes, no new `Action` variants, no controller/state-mutation
|
||||
changes. This is a view-layer repaint; `theme.rs` constants are the only "API" the rest
|
||||
of the app depends on, and their names don't change, only their values.
|
||||
- No new keybindings and no mouse support. Sidebar widgets are read-only/glanceable —
|
||||
none of the three (Workflow, Todo, Usage) are interactive today, so they don't need
|
||||
focus or selection state in their new form either.
|
||||
- No overlay is removed. Workflow/Todo/Usage keep their existing overlay trigger as an
|
||||
"expand" view (see below); the other 13 overlays are untouched functionally.
|
||||
- No automated visual/snapshot tests are being introduced (none exist today for
|
||||
`view/`/`controller/`; see Testing below).
|
||||
|
||||
## Layout architecture
|
||||
|
||||
```
|
||||
┌───────────────────────────────────────────┬──────────────┐
|
||||
│ │ WORKFLOW │
|
||||
│ Chat transcript (tight inline log) │ ▶ Node-0-1 │
|
||||
│ │ ✓ Node-0-2 │
|
||||
│ ├──────────────┤
|
||||
│ │ TASKS │
|
||||
│ │ ☐ Fix bug │
|
||||
│ │ ☑ Repro │
|
||||
│ ├──────────────┤
|
||||
│ │ USAGE │
|
||||
│ │ 12.3k tok │
|
||||
├─────────────────────────────────────────────┴──────────────┤
|
||||
│ ❯ input bar │
|
||||
├───────────────────────────────────────────────────────────┤
|
||||
│ status bar │
|
||||
└───────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
- The sidebar is a fixed-width column (generalizing the existing `show_todo`
|
||||
two-column split in `view/mod.rs::draw`) holding three stacked widgets, in this
|
||||
order: **Workflow**, **Tasks**, **Usage**.
|
||||
- **Responsive collapse**: below a width threshold (~90 cols — extending the existing
|
||||
`show_todo && area.width > 60` precedent, widened because the new sidebar holds three
|
||||
stacked widgets instead of one), the sidebar doesn't render and chat takes full width.
|
||||
No manual toggle key — purely width-driven, matching current behavior.
|
||||
- Each sidebar widget truncates its content to what fits and shows a `+N more, press
|
||||
<key> to expand` hint (same pattern `Rewind` already uses for `"... and N more
|
||||
messages"`) when there's more than fits — that's what the kept overlay is for.
|
||||
|
||||
### Workflow / Todo / Usage: sidebar glance + overlay expand
|
||||
|
||||
These three overlays are **not removed**. Their existing trigger (same keys/commands as
|
||||
today) still opens the full-screen version — now serving as the "expand" view for when
|
||||
the sidebar column is too narrow to show everything (many hive-mind nodes, a long task
|
||||
list). The sidebar widget and the overlay both read the same state
|
||||
(`workflow_engine`, `misc.todo_content`, `session_runtime.usage` +
|
||||
`session_runtime.session_start`); the sidebar version is a new compact rendering, factored
|
||||
out so both call sites share it where the content is identical (e.g. per-agent card
|
||||
formatting in `workflow.rs`).
|
||||
|
||||
### Remaining 13 overlays: restyled modals, unchanged behavior
|
||||
|
||||
`Help, Settings, Bash, QuitConfirm, KeyInput, Editor, Effort, Mcp, Rewind, Learning,
|
||||
Loading, ModelSelector, ClearConfirm` keep their current centered-modal mechanic and
|
||||
content logic exactly as-is. Only their chrome changes: new palette values (same
|
||||
semantic-color-per-overlay mapping as today — e.g. `QuitConfirm` stays `ERROR`, `Settings`
|
||||
stays `PRIMARY`), and emoji dropped from their title strings.
|
||||
|
||||
## Visual language
|
||||
|
||||
### Palette — Tokyo Night
|
||||
|
||||
Values only; `Theme` constant names in `view/theme.rs` are unchanged, so every call site
|
||||
across `view/*` keeps working without edits beyond the const definitions themselves.
|
||||
|
||||
| Constant | Value | Constant | Value |
|
||||
|---|---|---|---|
|
||||
| `BG` | `#1a1b26` | `ROLE_USER` | `#9ece6a` |
|
||||
| `SURFACE` | `#1f2335` | `ROLE_ASSISTANT` | `#7aa2f7` |
|
||||
| `SURFACE_ELEVATED` | `#292e42` | `ROLE_SYSTEM` | `#7dcfff` |
|
||||
| `TEXT` | `#c0caf5` | `ROLE_TOOL` | `#e0af68` |
|
||||
| `TEXT_MUTED` | `#a9b1d6` | `PRIMARY` | `#7aa2f7` |
|
||||
| `TEXT_DIM` | `#565f89` | `SUCCESS` | `#9ece6a` |
|
||||
| `BORDER` | `#3b4261` | `WARNING` | `#e0af68` |
|
||||
| `BORDER_FOCUS` | `#7aa2f7` | `ERROR` | `#f7768e` |
|
||||
| `HIGHLIGHT` | `#3d59a1` | `INFO` | `#7dcfff` |
|
||||
| `HIGHLIGHT_DIM` | `#292e42` | `ACCENT_PURPLE` | `#bb9af7` |
|
||||
| `STATUS_BAR_BG` | `#16161e` | `ACCENT_PINK` | `#ff007c` |
|
||||
| `MODE_AUTO` | `#9ece6a` | `ACCENT_ORANGE` | `#ff9e64` |
|
||||
| `MODE_YOLO` | `#f7768e` | `ACCENT_TEAL` | `#73daca` |
|
||||
| `CODE_BG` | `#16161e` | `CODE_BAR` | `#292e42` |
|
||||
| `BLOCKQUOTE_BAR` | `#7dcfff` | `SCROLLBAR_BG` / `SCROLLBAR_FG` | `#1f2335` / `#3b4261` |
|
||||
|
||||
### Message density — tight inline log
|
||||
|
||||
Replaces the per-message card (role badge pill + left accent bar + blank-line gap)
|
||||
in `chat.rs`:
|
||||
|
||||
```
|
||||
you 09:14 fix the login bug
|
||||
ai 09:14 Looking at src/auth.rs now.
|
||||
↳ Reading src/auth.rs
|
||||
you 09:15 ok try again
|
||||
```
|
||||
|
||||
- Role rendered as a short lowercase colored label (`ROLE_*` colors), timestamp dim,
|
||||
inline with the first content line.
|
||||
- Wrapped/multi-line content aligns under the content column (not under the role label).
|
||||
- Tool-call sub-lines get a dim `↳` prefix.
|
||||
- No blank line within a turn; a single blank line only between different speakers (not
|
||||
after every message).
|
||||
- The chat panel's outer bordered `Block` is unchanged — only the messages inside it lose
|
||||
per-message decoration.
|
||||
- The streaming indicator becomes `ai 09:14 ⠋ generating...` inline, matching the new
|
||||
format, instead of the current padded badge line.
|
||||
|
||||
### Icons
|
||||
|
||||
Overlay titles drop decorative emoji (❓⚙💻🚪✏️🎯🔌📋⏪📚📊⏳🧠🗑️⚡) and render as plain
|
||||
bold colored text (e.g. `Settings` in `PRIMARY`, no ⚙). The border/text accent color is
|
||||
the identity signal, consistent with the muted Tokyo Night + tight-density direction.
|
||||
|
||||
## File impact
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `view/theme.rs` | Palette values swap (table above). Const names/count unchanged. |
|
||||
| `view/chat.rs` | Rewrite message rendering to the tight inline format. |
|
||||
| `view/markdown.rs` | Re-themed code/quote colors; tightened padding. No structural rewrite. |
|
||||
| `view/mod.rs` | `draw()` grows the persistent sidebar column (generalizes `show_todo` split). `render_overlay()` match arms restyled in place (palette + title text), content logic untouched. Todo/Usage compact-widget rendering factored out of the current inline overlay code so it's callable from both the sidebar and the kept overlay. |
|
||||
| `view/status.rs` | Restyle to new palette; structurally unchanged. |
|
||||
| `view/workflow.rs` | Add a compact-card render function for the sidebar widget, reusing the existing per-agent formatting logic. |
|
||||
| `controller/*` | No changes. Interaction model is unchanged; sidebar is non-interactive. |
|
||||
|
||||
## Edge cases
|
||||
|
||||
- Empty states per sidebar widget (no workflow running, no tasks, zero usage) — compact
|
||||
one-line placeholders, consistent with the tight density (not the current multi-line
|
||||
placeholder paragraphs).
|
||||
- Sidebar auto-collapses below ~90 cols; chat reclaims full width.
|
||||
- Sidebar widget overflow (e.g. a hive-mind run with many nodes, a long task list)
|
||||
truncates with a `+N more` hint pointing at the existing expand-overlay trigger.
|
||||
- Long chat content wraps with continuation lines aligned under the content column.
|
||||
|
||||
## Testing / verification
|
||||
|
||||
No automated visual or snapshot tests exist for `view/`/`controller/` today (confirmed:
|
||||
zero `#[cfg(test)] mod tests` in either directory), and none are introduced by this
|
||||
change — ratatui rendering isn't meaningfully unit-testable without a snapshot harness
|
||||
this repo doesn't have. Verification is manual: run the TUI (`cargo run`) and exercise
|
||||
the golden paths (send a chat message, trigger a workflow/hive-mind run, open each of the
|
||||
13 remaining overlays, resize the terminal across the sidebar-collapse threshold).
|
||||
`cargo clippy` must stay clean (warnings-as-errors per repo config), and every touched
|
||||
`pub fn`/`struct` keeps the doc-comment convention from CLAUDE.md (What/Flow/Why/Return).
|
||||
|
||||
## Suggested implementation order
|
||||
|
||||
Not binding — the implementation plan owns sequencing — but a sensible build order given
|
||||
the dependency shape (palette first, since everything else reads `Theme` consts):
|
||||
|
||||
1. `theme.rs` palette swap
|
||||
2. `chat.rs` tight-inline rewrite
|
||||
3. `mod.rs` sidebar scaffolding + Workflow/Tasks/Usage compact widgets (+ `workflow.rs`
|
||||
compact-card fn)
|
||||
4. `status.rs` restyle + remaining 13 overlay restyle (mechanical: palette + title text)
|
||||
5. Manual TUI verification pass across golden paths above
|
||||
@@ -1,114 +0,0 @@
|
||||
# Clipboard Copy via OSC52 — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/controller/input.rs`, `src/main.rs`,
|
||||
`src/ipc/protocol.rs`
|
||||
|
||||
## Context
|
||||
|
||||
There is no clipboard support anywhere in the TUI today, and mouse capture is enabled
|
||||
(`EnableMouseCapture` in `main.rs`), which in most terminal emulators suppresses native
|
||||
click-drag text selection unless the user holds a modifier — making an in-app copy action
|
||||
more valuable than it would be in a plain scrollback. OSC52 is a terminal escape sequence
|
||||
(`\x1b]52;c;<base64>\x07`) that asks the terminal emulator itself to set the system
|
||||
clipboard; it needs no OS-level clipboard library (no X11/Wayland/win32 dependency) and
|
||||
the `base64` crate is already a dependency (used in `service/oauth/pkce.rs`), so no new
|
||||
crate is needed for this feature.
|
||||
|
||||
Key architectural constraint discovered while designing this: `controller::input::handle_key`
|
||||
runs on the **daemon** process in `--daemon`/`--attach` mode (`main.rs:359`, inside
|
||||
`handle_daemon_client`), not on the process that owns the user's actual terminal. A raw
|
||||
`io::stdout()` write inside `handle_key` would go to the headless daemon's stdout in that
|
||||
mode, not the user's terminal. The copy action therefore can't write the escape sequence
|
||||
directly from `handle_key` — it has to signal intent via state, and the terminal-owning
|
||||
process (single-process `run_loop_inner`, or the attach client's loop) performs the actual
|
||||
write.
|
||||
|
||||
## Goals
|
||||
|
||||
- `Ctrl+Y` copies the most recent `Role::Assistant` message's raw text (not the rendered
|
||||
markdown spans) to the system clipboard via OSC52.
|
||||
- Works identically in single-process mode and in `--daemon`/`--attach` mode.
|
||||
- No new dependency.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No native clipboard fallback (e.g. `arboard`) for terminals that don't honor OSC52 —
|
||||
unsupported terminals silently swallow the escape sequence; no error surfaces to the
|
||||
user beyond the optimistic "Copied to clipboard" toast (there's no ack mechanism in the
|
||||
OSC52 protocol to verify the terminal actually did it).
|
||||
- No copy-last-code-block variant — out of scope for this pass; the whole-message copy
|
||||
covers the common case and is simple to extend later if needed.
|
||||
- No mouse-drag text selection — unrelated, much larger feature; not being built here.
|
||||
|
||||
## State (`misc.rs`)
|
||||
|
||||
- `MiscState` gains `pub pending_clipboard_copy: Option<String>`, initialized to `None` in
|
||||
`MiscState::new()`.
|
||||
|
||||
## `input.rs`
|
||||
|
||||
- New top-level arm alongside the existing `Ctrl+C`/`Ctrl+D` handlers:
|
||||
`KeyCode::Char('y') if key.modifiers.contains(KeyModifiers::CONTROL)`. It finds the last
|
||||
message in `state.transcript_cache.messages` with `role == Role::Assistant`:
|
||||
- If found: `state.misc.pending_clipboard_copy = Some(msg.content.clone())`.
|
||||
- If not found: push an `Info` toast ("No assistant message to copy yet") and leave
|
||||
`pending_clipboard_copy` as `None`.
|
||||
- Returns `Vec::new()` — this is a direct state mutation inside `handle_key`, matching
|
||||
the existing `Ctrl+S` editor-save precedent (`main.rs`'s editor branch also mutates
|
||||
state/does I/O directly rather than going through an `Action`).
|
||||
|
||||
## OSC52 write helper (`main.rs`)
|
||||
|
||||
```
|
||||
fn write_osc52(stdout: &mut impl Write, text: &str) -> io::Result<()> {
|
||||
let b64 = base64::engine::general_purpose::STANDARD.encode(text);
|
||||
write!(stdout, "\x1b]52;c;{b64}\x07")?;
|
||||
stdout.flush()
|
||||
}
|
||||
```
|
||||
|
||||
Generic over `impl Write` so both the single-process loop (writing to `io::stdout()`) and
|
||||
tests (writing to a `Vec<u8>` to assert the formatted sequence) can use it without a real
|
||||
terminal.
|
||||
|
||||
## Single-process mode (`run_loop_inner`)
|
||||
|
||||
After the existing `for action in actions { apply_action(state, action); }` block, add:
|
||||
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
let _ = write_osc52(&mut io::stdout(), &text);
|
||||
state.push_toast(Toast::new(ToastKind::Success, "Copied to clipboard".into()));
|
||||
}
|
||||
```
|
||||
|
||||
## Daemon/attach mode
|
||||
|
||||
- `ipc/protocol.rs`: add `DaemonFrame::ClipboardCopy(String)` (alongside `StateUpdate`,
|
||||
`StreamToken`, `SystemNote`, `Closed` — same `Serialize`/`Deserialize` derive).
|
||||
- `handle_daemon_client` (`main.rs`): after each branch that calls `handle_key`/`apply_action`
|
||||
(`KeyPress` and `Submit`, the only two that can reach the input handler), before the
|
||||
existing `send_daemon_update(&mut conn, state)?;` call, add:
|
||||
```
|
||||
if let Some(text) = state.misc.pending_clipboard_copy.take() {
|
||||
conn.send(&DaemonFrame::ClipboardCopy(text))?;
|
||||
}
|
||||
```
|
||||
- Attach-client loop (`main.rs`, the function matching on `DaemonFrame::StateUpdate` /
|
||||
`SystemNote` / `Closed` around line 573): add a `DaemonFrame::ClipboardCopy(text) => {
|
||||
let _ = write_osc52(&mut io::stdout(), &text); client_state.push_toast(...); }` arm,
|
||||
mirroring the existing `SystemNote` handling but performing the actual terminal write
|
||||
since this process — not the daemon — owns the user's terminal.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `input.rs`: `Ctrl+Y` with a transcript containing multiple messages sets
|
||||
`pending_clipboard_copy` to the *last* assistant message's content, ignoring later
|
||||
user/tool messages that might follow it; with no assistant message present, it pushes
|
||||
an info toast and leaves `pending_clipboard_copy` as `None`.
|
||||
- `main.rs`: `write_osc52` writing into a `Vec<u8>` buffer produces the exact expected
|
||||
`\x1b]52;c;<base64>\x07` byte sequence for a known input string.
|
||||
@@ -1,115 +0,0 @@
|
||||
# Diff View for edit/write Tools — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/tool/fs/edit.rs`, `src/tool/fs/write.rs`, `src/view/markdown.rs`, `src/view/chat.rs`
|
||||
|
||||
## Context
|
||||
|
||||
`edit` currently reports only a byte-delta (`"edited {rel} ({N} byte delta)"`), and `write`
|
||||
reports only a byte count. Neither the model nor the user sees what actually changed —
|
||||
just a number. This makes it hard for the model to self-verify an edit landed correctly,
|
||||
and hard for the user to review a change without opening the file. No diff-computing
|
||||
library exists in the dependency tree today.
|
||||
|
||||
## Goals
|
||||
|
||||
- `edit` returns a real unified diff (git-style, 3 lines of context) of the change it just
|
||||
made, in place of the byte-delta note.
|
||||
- `write` returns the same kind of diff when it overwrites a file that already existed
|
||||
with valid UTF-8 content; falls back to the current "wrote N bytes" message for new
|
||||
files or non-UTF-8 (binary) overwrites.
|
||||
- Diffs render in the chat view with real color (green add / red remove / cyan hunk
|
||||
header) instead of being flattened to dim/italic like other tool output.
|
||||
- Large diffs are truncated with a trailing count, matching the existing pattern in
|
||||
`read.rs` (`"... ({N} more lines, total {total})"`).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No diff view for any tool besides `edit`/`write` (e.g. no retroactive diffing of
|
||||
`bash_tools.rs` shell edits).
|
||||
- No side-by-side diff layout — unified format only, matching how every other tool
|
||||
output already renders as a single text stream.
|
||||
- No persistence of diff history; each diff is only the delta of the single tool call
|
||||
that produced it, not a cumulative session diff.
|
||||
- No changes to non-tool (assistant/user/system) message rendering or coloring.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `similar = "3"` (line/word diff crate; permissive MIT/Apache-2.0, no heavy
|
||||
transitive deps). Use `TextDiff::from_lines(old, new).unified_diff().context_radius(3)`,
|
||||
which produces standard `@@ -a,b +c,d @@` hunk headers and `-`/`+`/` `-prefixed lines —
|
||||
no custom diff algorithm needed.
|
||||
|
||||
## Tool changes
|
||||
|
||||
### `edit.rs`
|
||||
|
||||
After computing `new_content` and writing it to disk:
|
||||
|
||||
1. Compute `similar::TextDiff::from_lines(&content, &new_content).unified_diff().context_radius(3).to_string()`.
|
||||
2. Split into lines; if `> MAX_DIFF_LINES` (200), keep the first 200 and append
|
||||
`"... ({N} more lines truncated)"`.
|
||||
3. Wrap the (possibly truncated) diff text in a fenced ` ```diff ` block.
|
||||
4. Replace the byte-delta note in the returned message with this block; keep the
|
||||
existing "Graduated checks matched" / LSP note suffixes in their current position
|
||||
(after the diff block).
|
||||
|
||||
### `write.rs`
|
||||
|
||||
Before overwriting:
|
||||
|
||||
1. If `path.exists()` and `fs::read_to_string(&path)` succeeds (valid UTF-8), capture it
|
||||
as `old_content` and note `is_overwrite = true`.
|
||||
2. If the file doesn't exist, or reading it fails (binary/non-UTF-8), `is_overwrite = false`
|
||||
— no error, just skip the diff path silently.
|
||||
3. After writing, if `is_overwrite`, compute and truncate the diff exactly as in `edit.rs`
|
||||
and append the fenced block to the return message (in addition to the existing
|
||||
"wrote N bytes" line, not instead of it — for `write`, unlike `edit`, the byte count is
|
||||
still useful since it can be a full-file rewrite).
|
||||
4. If not `is_overwrite`, return message is unchanged from today.
|
||||
|
||||
The truncation constant (`MAX_DIFF_LINES = 200`) and truncation message format are
|
||||
shared — factor into a small helper in `tool/fs/helpers.rs` used by both tools.
|
||||
|
||||
## Rendering changes
|
||||
|
||||
### `markdown.rs`
|
||||
|
||||
- `render_markdown` gains a `dim: bool` parameter: `render_markdown(text, width, dim)`.
|
||||
- Capture the fence language from `Tag::CodeBlock(CodeBlockKind::Fenced(lang))` (today
|
||||
matched as `CodeBlock(_)`, discarding the language). Track `in_diff_block: bool` when
|
||||
`lang == "diff"`.
|
||||
- Inside a diff block, process text line-by-line instead of as one blob: a line starting
|
||||
with `+` (not `+++`) is styled green, `-` (not `---`) red, `@@` cyan/muted, everything
|
||||
else (context lines, `+++`/`---` file headers) uses the existing code-block teal.
|
||||
- When `dim` is `true`: every span keeps its assigned color as computed above, but
|
||||
non-diff spans (headings, links, plain text, non-diff code blocks, table cells) fall
|
||||
back to `Theme::TEXT_DIM` + `Modifier::ITALIC` instead of their normal palette color —
|
||||
this replicates today's "tool output is always dim" behavior for everything except
|
||||
diff lines.
|
||||
- When `dim` is `false`: behavior is unchanged from today (full color, used for
|
||||
assistant/user/system messages).
|
||||
|
||||
### `chat.rs`
|
||||
|
||||
- `Role::Tool` branch: replace the two manual span-remapping loops (that force every
|
||||
span to `dim_italic`) with a direct call to `render_markdown(&content, content_width, true)`
|
||||
and use the returned spans as-is.
|
||||
- All other roles: call `render_markdown(&content_str, content_width, false)` — same
|
||||
call as today, just with the new explicit `false` argument.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `edit.rs`: a normal single-replace edit produces a diff block with matching
|
||||
`-`/`+` lines; a `replace_all` across 250+ lines truncates at 200 with the correct
|
||||
trailing count.
|
||||
- `write.rs`: writing a brand-new file keeps the old "wrote N bytes" message with no
|
||||
diff block; overwriting an existing UTF-8 file produces a diff block; overwriting
|
||||
a path that reads as invalid UTF-8 (simulate via non-UTF-8 bytes) falls back to the
|
||||
byte-count message without erroring.
|
||||
- `markdown.rs`: a fenced ` ```diff ` block with `+`/`-`/`@@` lines produces spans with
|
||||
the expected fg colors under `dim=true` (diff lines colored) and confirms non-diff
|
||||
text in the same call falls back to `TEXT_DIM` + italic.
|
||||
@@ -1,126 +0,0 @@
|
||||
# Fuzzy @file-mention Autocomplete — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-15
|
||||
**Scope:** `src/app/state/misc.rs`, `src/app/state/rest.rs`, `src/controller/input.rs`,
|
||||
`src/view/mod.rs`, `src/tool/mod.rs`, `src/tool/fs/write.rs`, `src/main.rs`
|
||||
|
||||
## Context
|
||||
|
||||
The chat input already has a dropdown autocomplete (`InputState` in `misc.rs`), but it
|
||||
only covers slash commands: it requires the whole buffer to start with `/` and filters a
|
||||
fixed `COMMANDS` list by prefix. There's no way to reference a project file from the chat
|
||||
input without typing its exact path from memory. The existing `dir_cache` (used by the
|
||||
`dir_cache_update` tool) looks like it could serve this but doesn't: it's a single,
|
||||
non-recursive directory snapshot, overwritten on each LLM-driven `dir_cache_update` call —
|
||||
not a standing, recursive, whole-workspace file index. `search.rs`'s `Grep`/`Glob` tools
|
||||
already do the recursive, `.gitignore`-respecting walk this feature needs, via
|
||||
`ignore::Walk`.
|
||||
|
||||
Also relevant: there is no persistent async runtime driving the TUI loop. `main.rs`
|
||||
constructs a `tokio::runtime::Runtime` but never `.enter()`s or `block_on`s it in the
|
||||
main loop — `run_loop` is fully synchronous. The one existing async-flavored pattern
|
||||
(`dir_cache_update.rs`) spins up a throwaway one-shot runtime purely to satisfy
|
||||
`tokio::sync::RwLock`'s API, then discards it. This feature does not need that ceremony:
|
||||
a plain `std::sync::RwLock` is enough, since every reader/writer here is synchronous
|
||||
(`handle_key`, `Tool::run`, and the index-build thread all being plain sync code).
|
||||
|
||||
## Goals
|
||||
|
||||
- Typing `@` at a word boundary (start of buffer or after whitespace) in the chat input,
|
||||
followed by non-whitespace characters, opens a dropdown of fuzzy-matched project file
|
||||
paths, live-updating as the query changes.
|
||||
- Selecting a candidate splices `@relative/path ` into the buffer at the mention's
|
||||
position (not a whole-buffer replace) and the user keeps typing.
|
||||
- Candidates come from a background-built, whole-workspace file index — not the
|
||||
LLM-facing `dir_cache`.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- No auto-reading of the selected file's content into the conversation — the inserted
|
||||
`@path` is plain text; the model reads it via the `read` tool if it wants to, same as
|
||||
any other path reference.
|
||||
- No live re-filter on Backspace/Delete while a mention dropdown is open — mirrors the
|
||||
slash-command dropdown's existing behavior (closes on Backspace/Delete rather than
|
||||
refiltering). Not fixing that for commands here; file mentions just inherit it for
|
||||
consistency.
|
||||
- No periodic re-walk of the index after startup — only single-file incremental updates
|
||||
on file creation (see below). A deleted or renamed file may show a stale entry until
|
||||
restart; acceptable since selecting it just inserts text, it doesn't touch the
|
||||
filesystem.
|
||||
- No fuzzy matching over directories, only files.
|
||||
|
||||
## Dependency
|
||||
|
||||
Add `nucleo-matcher = "0.3"` (the fuzzy-matching engine from the Helix editor project;
|
||||
small, actively maintained, no heavy transitive deps).
|
||||
|
||||
## Index storage & construction
|
||||
|
||||
- New type in `misc.rs`: `MentionIndex { entries: Arc<std::sync::RwLock<Vec<String>>> }`, with `MentionIndex::new()`, `set(&self, paths: Vec<String>)`, and `snapshot(&self) -> Vec<String>` (both plain sync `.write()`/`.read()`, no `try_`/async — a std `RwLock` doesn't block indefinitely here since every hold is a quick vec swap or clone).
|
||||
- `AppStateRest` gets a `pub mention_index: MentionIndex` field, initialized in `AppStateRest::new()`, threaded into `ToolCtx`/`ToolCtxBuilder` the same way `dir_cache` is (new `mention_index` field on both, wired through `tool_ctx()`/`tool_ctx_for()`/`build()`).
|
||||
- In `main.rs`, right after `AppStateRest::new(...)` in the single-process TUI path and the daemon path (not the attach-only client path, which has no local `ToolCtx`), spawn `std::thread::spawn` that:
|
||||
1. For each workspace root (index `i`, path `w`): `ignore::Walk::new(w)`, keep only files, strip `w` as prefix, format as `rel` for `i == 0` or `[i]rel` for `i > 0` (matching `resolve_path`'s existing workspace-index convention).
|
||||
2. Stop collecting once the total across all workspaces hits 50,000 entries (repos larger than that are rare here; this is a soft cap to bound memory/scan time, not a hard requirement).
|
||||
3. Call `mention_index.set(all_paths)`.
|
||||
- `write.rs`: after a successful write, if the target path did **not** exist before the write (i.e. this created a new file, not an overwrite), compute its relative/workspace-prefixed form and push it onto `ctx.mention_index`'s vec directly (read-modify-write under the same lock) rather than re-walking.
|
||||
|
||||
## `InputState` changes (`misc.rs`)
|
||||
|
||||
- New `pub enum AutocompleteKind { Command, FileMention }`.
|
||||
- `InputState` gains `pub autocomplete_kind: AutocompleteKind` (default `Command`) and
|
||||
`pub mention_start: usize` (byte offset of the triggering `@`).
|
||||
- New `fn mention_query_at_cursor(&self) -> Option<(usize, String)>`: scans backward from
|
||||
`self.cursor` for an `@`; the scan stops (returns `None`) if it hits whitespace before
|
||||
finding `@`. The `@` only counts as a trigger if it's at buffer start or immediately
|
||||
preceded by whitespace. Returns `(byte offset of '@', query text between '@' and cursor)`.
|
||||
- New `fn open_mention_autocomplete(&mut self, files: &[String])`: calls
|
||||
`mention_query_at_cursor()`; if `None`, calls `close_autocomplete()` and returns. If
|
||||
`Some((start, query))`, fuzzy-matches `query` against `files` via `nucleo-matcher`,
|
||||
keeps the top 10 by score, sets `autocomplete_candidates`, `autocomplete_kind =
|
||||
FileMention`, `mention_start = start`, `autocomplete_visible = !candidates.is_empty()`.
|
||||
- `select_autocomplete()` becomes kind-aware:
|
||||
- `Command` (today's behavior, unchanged): `buffer = candidate.clone()`, `cursor =
|
||||
buffer.len()`.
|
||||
- `FileMention`: `buffer.replace_range(mention_start..cursor, &format!("@{candidate} "))`,
|
||||
`cursor = mention_start + candidate.len() + 2` (the `@` plus the candidate plus the
|
||||
trailing space).
|
||||
- Both paths end with `close_autocomplete()`, same as today.
|
||||
|
||||
## `input.rs` wiring
|
||||
|
||||
- `KeyCode::Char(c)` handler: after `state.input.insert(c)`, keep the existing
|
||||
`if buffer.starts_with('/') { open_autocomplete() }` check, and add an `else if let
|
||||
Some(_) = state.input.mention_query_at_cursor() { state.input.open_mention_autocomplete(&state.mention_index.snapshot()) }` branch. These are mutually exclusive in practice (a
|
||||
buffer starting with `/` is a slash command, not a sentence with an `@mention` in it).
|
||||
- `KeyCode::Backspace` / `KeyCode::Delete`: unchanged — both already just call
|
||||
`close_autocomplete()` when a dropdown is visible, regardless of kind. No new branching
|
||||
needed since `close_autocomplete()` already resets `autocomplete_kind` isn't touched but
|
||||
becomes irrelevant once `autocomplete_visible` is false.
|
||||
- `KeyCode::Tab`: currently gated on `buffer.starts_with('/')`. Extend the condition to
|
||||
also fire when `autocomplete_kind == FileMention && autocomplete_visible` so Tab cycles
|
||||
file-mention candidates too.
|
||||
- `KeyCode::Enter`: unchanged — already calls `select_autocomplete()` whenever
|
||||
`autocomplete_visible`, which is now kind-aware internally.
|
||||
|
||||
## Rendering (`view/mod.rs`)
|
||||
|
||||
- `render_input_bar`'s dropdown block reuses the exact same list-rendering code (already
|
||||
generic over `autocomplete_candidates`/`autocomplete_idx`); only the title changes based
|
||||
on `state.input.autocomplete_kind`: `" ⌘ Commands "` (unchanged) vs `" 📁 Files "`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per CLAUDE.md convention:
|
||||
|
||||
- `misc.rs`: `mention_query_at_cursor` returns the right `(start, query)` for `@` at
|
||||
buffer start, `@` after a space mid-sentence, and correctly returns `None` when the `@`
|
||||
is mid-word (e.g. `foo@bar`) or when whitespace exists between the `@` and the cursor.
|
||||
`select_autocomplete` for `FileMention` splices correctly into a buffer with text before
|
||||
and after the mention span; `Command` selection still replaces the whole buffer as
|
||||
before.
|
||||
- `write.rs`: creating a new file appends its path to the shared `mention_index`;
|
||||
overwriting an existing file does not add a duplicate entry.
|
||||
- Index construction: not unit-tested directly (it's a `std::thread::spawn` walking the
|
||||
real filesystem at startup) — covered implicitly by exercising the app manually per the
|
||||
`verify` skill during implementation.
|
||||
@@ -1,281 +0,0 @@
|
||||
# Context & Compaction Overhaul — Design
|
||||
|
||||
**Status:** Approved, pending implementation plan
|
||||
**Date:** 2026-07-16
|
||||
**Scope:** replaces `src/app/runtime/shortsend.rs`; touches `src/app/runtime/actions/mod.rs`,
|
||||
`src/view/status.rs`, `src/model/settings.rs`, `src/app/subagent/division.rs`, `Cargo.toml`
|
||||
|
||||
## Context
|
||||
|
||||
The existing conversation-compaction system (`shortsend.rs`, 129 lines) only acts once the
|
||||
context is already close to the model's window limit, and has accumulated inconsistencies
|
||||
found during a codebase audit:
|
||||
|
||||
1. Three different token-count heuristics for the same job: `/3` inside
|
||||
`shortsend::shape_messages`, `/4` in the auto-compact loop
|
||||
(`actions/mod.rs` ~line 1146), `/4` again in the live status bar (`view/status.rs:68`).
|
||||
2. Manual `/compact` (`Action::Compact`, `actions/mod.rs:547-563`) passes `client: None`
|
||||
because `apply_action` is synchronous, so it never gets LLM summarization — it always
|
||||
falls back to the bare `"[prior conversation compacted]"` placeholder, unlike automatic
|
||||
mid-turn compaction (`Some(&tc.client)`, line 1160). Undocumented asymmetry between the
|
||||
two trigger paths.
|
||||
3. `context_window` resolution (`model_roles.values().find(...).and_then(...).unwrap_or(...)`)
|
||||
duplicated three times (`Action::Compact`, `spawn_turn`, `view/status.rs` twice).
|
||||
4. No repeated-tool-call dedup: reading the same file (or running the same grep) twice in a
|
||||
session keeps both full copies in context forever, until compaction eventually drops the
|
||||
older one wholesale along with everything else from that period.
|
||||
5. No per-result compression: a single large tool output (a big `bash` log, a large `grep`
|
||||
result) is stored verbatim even when most of it is redundant or low-value.
|
||||
6. Zero test coverage on `shortsend.rs`.
|
||||
|
||||
Separately, research into three real, permissively-licensed open-source projects
|
||||
(`rtk-ai/rtk`, Apache-2.0; `headroomlabs-ai/headroom`, Apache-2.0; `JuliusBrussee/caveman`,
|
||||
MIT — verified via `gh api` for authenticity/license, and by cloning and reading source, not
|
||||
taken from marketing blog posts) surfaced techniques worth reimplementing natively:
|
||||
|
||||
- **rtk**: generic line-scan compression (strip comment/blank runs, brace-depth collapse of
|
||||
function bodies, importance-ranked truncation ending in an unambiguous `[N more lines]`
|
||||
marker — their own regression tests show a comment-shaped marker confuses the LLM into
|
||||
retry-looping) plus structured per-toolchain parsing (e.g. `cargo --message-format=json`
|
||||
bucketed into errors/warnings, boilerplate lines dropped).
|
||||
- **headroom**: per-content-type compressors — logs (classify lines by level/stack-trace/
|
||||
summary, score, keep highest-value lines + surrounding context, adaptive cap), grep
|
||||
results (group by file, score matches, cap globally and per-file), JSON (keep all
|
||||
structural tokens — keys, brackets, colons — drop or shrink long low-entropy string
|
||||
values, keep short values and UUID/hash-shaped high-entropy ones).
|
||||
- **caveman**: a pure prompt/persona instruction (no algorithm) that tells the model to
|
||||
write tersely — drop articles/filler/hedging, keep code/commands/errors verbatim — with an
|
||||
explicit carve-out that disables terseness for destructive-op confirmations and security
|
||||
warnings. This compresses *output* tokens, a different axis from everything else in this
|
||||
design, which compresses *input* context.
|
||||
|
||||
This is a from-scratch reimplementation of the underlying ideas, not a port — no code is
|
||||
copied from any of the three projects.
|
||||
|
||||
## Goals
|
||||
|
||||
- One unified, always-on pipeline that keeps context lean from turn 1, not just once near
|
||||
the limit.
|
||||
- Deduplicate repeated tool calls: an older copy of a tool result superseded by an identical
|
||||
later call (same tool name + same arguments) is replaced with a placeholder, for read-only
|
||||
tools only.
|
||||
- Compress large individual tool results (logs, JSON, generic text) at capture time, above a
|
||||
size floor.
|
||||
- Fix the three known inconsistencies (token heuristic, manual/auto asymmetry,
|
||||
`context_window` duplication).
|
||||
- Optional, off-by-default "concise mode" system-prompt toggle for terser model output.
|
||||
- Full inline test coverage per repo convention.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Not adding a runtime dependency on `rtk`, `headroom`, or `caveman` themselves (as a binary,
|
||||
proxy, or crate) — everything is implemented natively in Rust inside zesdex.
|
||||
- Not building rtk's per-toolchain structured parsers (`cargo --message-format=json`
|
||||
re-invocation, etc.) — too invasive for a general-purpose `bash` tool that runs arbitrary
|
||||
commands zesdex doesn't control the flags of. Only the generic line-scan/log/JSON layer is
|
||||
built.
|
||||
- Not switching to an exact per-provider tokenizer — `tiktoken-rs` (BPE, cl100k_base/
|
||||
o200k_base) is an approximation good enough for the 85%/95% budget thresholds; it is not
|
||||
used for billing-accurate counts.
|
||||
- `caveman-compress`-style memory-file rewriting (the LLM-round-trip variant of caveman) is
|
||||
out of scope — only the pure-prompt persona mechanism is adopted.
|
||||
|
||||
## Architecture
|
||||
|
||||
Replace `src/app/runtime/shortsend.rs` with `src/app/runtime/context/`:
|
||||
|
||||
```
|
||||
context/
|
||||
mod.rs — module registration only, no facade (see below)
|
||||
tokens.rs — unified token counting (tiktoken-rs)
|
||||
dedup.rs — cross-call tool-result deduplication
|
||||
squash.rs — per-result compression (log/json/generic), applied at
|
||||
tool-result construction time, upstream of prepare()
|
||||
shaping.rs — budget-based drop + LLM summarize (renamed shortsend logic)
|
||||
window.rs — shared context_window resolution
|
||||
```
|
||||
|
||||
### `tokens.rs`
|
||||
|
||||
```rust
|
||||
pub fn count_tokens(text: &str) -> usize
|
||||
pub fn count_message_tokens(msg: &ChatMessage) -> usize
|
||||
```
|
||||
|
||||
Backed by `tiktoken-rs` (new dependency, pure Rust, embedded BPE vocab, no network calls at
|
||||
runtime), using `o200k_base`. Replaces all three existing heuristic call sites: `shortsend`'s
|
||||
internal `/3`, the auto-loop's `/4` (`actions/mod.rs` ~1146), and `status.rs:68`'s `/4`.
|
||||
|
||||
### `dedup.rs`
|
||||
|
||||
```rust
|
||||
pub fn collapse(messages: &[ChatMessage]) -> (Vec<ChatMessage>, bool)
|
||||
```
|
||||
|
||||
The `bool` is `true` iff at least one message was replaced with a placeholder — callers use
|
||||
it to decide whether the result is worth persisting/announcing, without needing `ChatMessage`
|
||||
to implement `PartialEq` (it doesn't today, and adding it purely to diff whole message lists
|
||||
would be needless surface area for what `collapse` already knows precisely mid-walk).
|
||||
|
||||
Flow: walk messages, pair each `Role::Tool` message to its originating `ToolCall` via
|
||||
`tool_call_id`. Key = `(function.name, sha256(canonical_json(function.arguments)))` (`sha2`
|
||||
is already a dependency). Track the last index seen per key. For any earlier occurrence of a
|
||||
key whose tool name is in the read-only set, replace that earlier `Tool` message's `content`
|
||||
with a short placeholder (`"[duplicate result — superseded by a later identical call, see
|
||||
below]"`); the assistant's tool-call entry (name + arguments) is left untouched, so the
|
||||
action/audit trail stays intact. Mutating tools are never touched, even with identical
|
||||
arguments, because call order and repetition can be semantically meaningful (e.g. retrying a
|
||||
flaky `bash` command).
|
||||
|
||||
Read-only classification reuses `subagent::division::tool_scope::READ_TOOLS`
|
||||
(`src/app/subagent/division.rs:21`) rather than a new list — that `const` is made `pub` for
|
||||
this purpose. It already enumerates exactly the read-only tool set (`read`, `grep`, `glob`,
|
||||
`search`, `seqthink`, `recall`, `lsp_*`, `read_findings`).
|
||||
|
||||
Runs every turn, unconditionally, before token counting — not gated on `should_shape`.
|
||||
|
||||
### `squash.rs`
|
||||
|
||||
```rust
|
||||
pub fn apply(tool_name: &str, output: &str) -> String
|
||||
```
|
||||
|
||||
`read` is exempted entirely, always passed through unchanged regardless of size: its output
|
||||
must stay byte-exact because the agent relies on it for exact-match edits afterward, and a
|
||||
squashed view of a JSON config file (or any file whose content happens to parse as JSON)
|
||||
would otherwise be silently altered. Size floor for every other tool: outputs under 1500
|
||||
bytes pass through unchanged (compression only pays off on large output, and touching small
|
||||
results risks losing detail with no token benefit). Above the floor, dispatch by content
|
||||
shape:
|
||||
|
||||
- `squash_json(&str) -> String` — walks a parsed `serde_json::Value` (not a hand-rolled
|
||||
tokenizer — `serde_json` already handles escaping/nesting correctly, reusing it is simpler
|
||||
and more robust); structural tokens (keys, brackets, colons, commas, booleans, null) always
|
||||
kept; string values kept if ≤20 chars or "identifier-shaped" (no internal whitespace *and*
|
||||
Shannon entropy ≥3.0 bits/char — catches UUIDs/hashes/paths), otherwise replaced with `"…"`
|
||||
in place; array elements past the first 3 compressed harder (values elided regardless of
|
||||
length/entropy). Applied when `serde_json::from_str` on the output succeeds. The
|
||||
no-whitespace pre-filter matters: raw per-character entropy alone doesn't separate prose
|
||||
from identifiers — repeated English prose measures ~3.89 bits/char, higher than a UUID's
|
||||
~3.39 — because prose also draws from a wide character set. headroom's own entropy gate is
|
||||
"cheaply pre-filtered by 'no spaces'" before scoring for the same reason; multi-word values
|
||||
never reach the entropy check at all under this rule.
|
||||
- `squash_log(&str) -> String` — line classifier (error/fail/warn/info/debug/trace by
|
||||
keyword + stack-trace-frame detection) → score
|
||||
(`level_score {1.0 error/fail, 0.5 warn, 0.1 info, 0.05 debug/trace} + 0.3 if
|
||||
stack-trace-frame + 0.4 if summary-shaped line`) → keep up to 20 highest-scored error
|
||||
lines, up to 10 highest-scored warning lines, all summary lines, plus a ±2-line context
|
||||
window around each kept line → single `[N lines omitted]` marker for drops (not
|
||||
comment-shaped, per rtk's own finding on LLM confusion). Applied when the output isn't
|
||||
valid JSON, the tool is `bash`, and the output has ≥3 lines matching error/warn/stack-trace
|
||||
patterns. The tool restriction (added after the final whole-branch review) matters: a `grep`
|
||||
result full of matches against error-handling code trips the same ≥3-line keyword threshold
|
||||
as a real build log, but `squash_log`'s hard 20-error/10-warning cap has no byte budget and
|
||||
would silently drop legitimate matches past it — the wrong compressor for search results.
|
||||
Only `bash` (the actual log-producing tool) routes through `squash_log`; every other tool
|
||||
whose output happens to look log-shaped falls through to the gentler, byte-budgeted
|
||||
`squash_generic` instead.
|
||||
- `squash_generic(&str, budget) -> String` — importance-ranked truncation: keeps the first 10
|
||||
and last 10 lines plus any line matching a small "looks important" heuristic (non-blank,
|
||||
not a byte-for-byte repeat of the immediately preceding line), single `[N lines omitted]`
|
||||
marker for the rest, capped to `budget` bytes overall (`budget` = the 1500-byte squash
|
||||
floor doubled, i.e. 3000 bytes, chosen so the fallback path still yields a real reduction
|
||||
on anything that triggered it). Fallback for anything that isn't JSON or log-shaped.
|
||||
|
||||
Called once, at the single tool-result construction site
|
||||
(`actions/mod.rs:1420`, `let tool_msg = ChatMessage::tool_result(tool_call.id.clone(),
|
||||
output);`) — `output` is passed through `squash::apply(&tool_name, &output)` before being
|
||||
wrapped. Runs before the result is ever archived or pushed into `msgs`, so compression is
|
||||
permanent and applies uniformly whether or not compaction ever triggers.
|
||||
|
||||
### `shaping.rs`
|
||||
|
||||
Unchanged behavior from today's `shortsend.rs` (hysteresis `should_shape`, 70%-budget
|
||||
newest-first retention, LLM summarization of dropped messages), moved as-is into this file
|
||||
and updated to source token counts from `tokens.rs` instead of its own heuristic.
|
||||
|
||||
### `window.rs`
|
||||
|
||||
```rust
|
||||
pub fn resolve(app_config: &AppConfig, settings: &Settings) -> usize
|
||||
```
|
||||
|
||||
Replaces the three duplicated `model_roles.values().find(...).and_then(...).unwrap_or(...)`
|
||||
blocks in `Action::Compact`, `spawn_turn`, and `view/status.rs` (×2).
|
||||
|
||||
### `mod.rs`
|
||||
|
||||
No facade function — just `pub mod dedup; pub mod shaping; pub mod squash; pub mod tokens;
|
||||
pub mod window;`. `dedup`, `shaping`, and `tokens` are called directly from each call site
|
||||
(the auto-loop and `Action::Compact`), matching CLAUDE.md's "No DI — modules call ...
|
||||
directly" convention rather than introducing an orchestration layer that only one of the two
|
||||
callers would use generically (the auto-loop already needs per-stage control today — it
|
||||
inspects `should_shape` itself to decide whether to emit `TurnEvent::Compacted` — and would
|
||||
have to unpack a facade's result anyway).
|
||||
|
||||
## Data flow (per turn)
|
||||
|
||||
1. Tool executes → raw `output: String`.
|
||||
2. `squash::apply(tool_name, &output)` — compress if over the size floor (`read` exempted).
|
||||
3. Wrapped into `ChatMessage::tool_result(...)`, archived, pushed to `msgs`.
|
||||
4. Once per loop iteration: `dedup::collapse(&msgs)` (always) → sum
|
||||
`tokens::count_message_tokens` over the result → `shaping::should_shape` → conditionally
|
||||
`shaping::shape_messages`.
|
||||
5. Result pushed as `TurnEvent::Compacted` if dedup changed anything or shaping triggered,
|
||||
consumed on the main thread to update `SessionRuntime.messages`.
|
||||
|
||||
## Fixing the manual/auto asymmetry
|
||||
|
||||
`Action::Compact` (`actions/mod.rs:547`) currently runs synchronously inside `apply_action`
|
||||
and can't block on an LLM call. Fix: make it spawn a background `std::thread::spawn` — the
|
||||
same pattern `spawn_turn` already uses (`actions/mod.rs:694`) — that runs `dedup::collapse`
|
||||
then unconditionally `shaping::shape_messages(.., force=true, Some(&client))` and reports back
|
||||
via `TurnEvent::Compacted`, identical to the automatic path. The toast sequence becomes
|
||||
"Compacting…" immediately (optimistic, non-blocking) then "History compacted" when the
|
||||
`TurnEvent` arrives. This gives manual `/compact` real LLM summarization instead of always
|
||||
falling back to the placeholder.
|
||||
|
||||
## Concise mode (separate from the `context/` module)
|
||||
|
||||
- `Settings` (`src/model/settings.rs`) gains `pub concise_output: bool`, default `false`,
|
||||
with `#[serde(default)]` for backward-compatible deserialization of existing
|
||||
`settings.json` files (matching the existing `hive_mind_node_timeout_ms` precedent in the
|
||||
same file).
|
||||
- When `true`, `run_agent_turn`'s system-prompt assembly (`actions/mod.rs:930-936`) appends a
|
||||
fourth section to `system_text`: a terse-writing instruction (persona-prompt only, no
|
||||
algorithm — drop articles/filler/hedging/pleasantries, keep code/commands/error text
|
||||
byte-exact) with an explicit carve-out disabling terseness for destructive-operation
|
||||
confirmations and security-relevant warnings, mirroring caveman's own "Auto-Clarity"
|
||||
safety exception.
|
||||
- No UI toggle is in scope for this pass — confirmed no such mechanism exists today for any
|
||||
boolean `Settings` field (`review_enabled`, `session_archive_enabled`,
|
||||
`lsp_auto_provision` are all hand-edited in `settings.json`, same as this one will be).
|
||||
|
||||
## New dependency
|
||||
|
||||
`tiktoken-rs` — pure Rust, embedded BPE vocab (`cl100k_base`/`o200k_base`), no network calls
|
||||
at runtime, MIT/Apache-2.0 dual-licensed. Added to `Cargo.toml`.
|
||||
|
||||
## Testing
|
||||
|
||||
Inline `#[cfg(test)] mod tests` per repo convention, one per new file:
|
||||
|
||||
- `dedup.rs`: same tool+args → older result replaced; different args → no-op; mutating tool
|
||||
with identical args → both kept in full; unmatched `tool_call_id` (malformed history) →
|
||||
no panic, treated as unpaired.
|
||||
- `squash.rs`: JSON input under/over the size floor; JSON with long low-entropy string values
|
||||
gets them elided while short/UUID-shaped values survive; log input with error/warn lines
|
||||
keeps highest-scored lines and emits exactly one `[N lines omitted]` marker; generic text
|
||||
keeps first/last N lines.
|
||||
- `tokens.rs`: known-string token counts against fixed expected values; empty string → 0.
|
||||
- `shaping.rs`: port the behavioral cases implied by today's hysteresis logic (85% trigger
|
||||
when not previously shaped, 95% once shaped) plus budget-drop ordering.
|
||||
- `window.rs`: role match resolves to the role's `context_window`; no match falls back to
|
||||
`default_context_window`.
|
||||
|
||||
## Migration
|
||||
|
||||
- Delete `src/app/runtime/shortsend.rs`; all three call sites (`actions/mod.rs` auto-loop,
|
||||
`Action::Compact`, and the module path itself) updated to `context::`.
|
||||
- `view/status.rs` switches its live token display to `tokens::count_tokens`, so the status
|
||||
bar finally matches what compaction measures internally.
|
||||
Reference in New Issue
Block a user