diff --git a/.reports/codemap-diff.txt b/.reports/codemap-diff.txt new file mode 100644 index 0000000..b14625b --- /dev/null +++ b/.reports/codemap-diff.txt @@ -0,0 +1,20 @@ +Codemap Update Report — 2026-07-12 +==================================== + +Status: FIRST GENERATION (no previous codemaps to compare) + +Files created: + - docs/CODEMAPS/architecture.md (new) + - docs/CODEMAPS/backend.md (new) + - docs/CODEMAPS/frontend.md (new) + - docs/CODEMAPS/data.md (new) + - docs/CODEMAPS/dependencies.md (new) + +Source scanned: + - 124 Rust source files + - 30 directories + - 103 modules + - 10,402 lines total + +No previous codemaps found — diff calculation skipped. +Freshness: all documents generated 2026-07-12. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..9aef797 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,114 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Build & Test + +```bash +# Build (debug) +cargo build + +# Release build +cargo build --release + +# Run all tests +cargo test + +# Run a single test +cargo test test_name + +# Lint +cargo clippy + +# Lint with warnings-as-errors +cargo clippy -- -D warnings +``` + +Test modules are located inline in production files (not a separate `tests/` dir): +- `src/app/harness.rs` — guard/verdict parsing tests +- `src/app/runtime/stream/mod.rs` — SSE parser tests +- `src/model/memory.rs` — memory CRUD + slugify tests +- `src/model/editlog.rs` — edit log append/reload tests +- `src/tool/fs/helpers.rs` — tool argument extraction tests + +Tests use `#[cfg(test)] mod tests` blocks. There are 37 unit tests total. + +Tracing output goes to `~/.local/share/zesdex/zesdex.log`. Set `RUST_LOG=debug` for verbose logging. + +## Architecture Overview + +Zesdex is an autonomous AI coding agent with a TUI — an OpenAI/Anthropic-compatible LLM client wrapped in a tool-use harness with 28 built-in tools. + +Detailed architecture documentation is in `docs/CODEMAPS/`: + +| File | Covers | +|------|--------| +| [`docs/CODEMAPS/architecture.md`](docs/CODEMAPS/architecture.md) | System layout, process modes, data flow, key files | +| [`docs/CODEMAPS/backend.md`](docs/CODEMAPS/backend.md) | Provider, OAuth, IPC, workflow engine, MCP, review, bg bash | +| [`docs/CODEMAPS/frontend.md`](docs/CODEMAPS/frontend.md) | TUI render pipeline, 16 overlays, toasts, input handling | +| [`docs/CODEMAPS/data.md`](docs/CODEMAPS/data.md) | Persistence, SQLite msglog, memory files, settings/config | +| [`docs/CODEMAPS/dependencies.md`](docs/CODEMAPS/dependencies.md) | 23 Rust crates, 5 external services | + +### Entry Points + +`src/main.rs` — three modes: +- **Single-process** (default): TUI + agent loop in one process +- **Daemon** (`--daemon`): background Unix socket server, handles LLM calls +- **Attach** (`--attach `): TUI-only client that connects to a daemon + +### Core Flow + +``` +Controller (key input → Action) → Event Loop → LLM stream → Tool execution → State mutation → TUI render + │ │ │ + │ src/controller/input.rs │ src/app/runtime/actions/ │ src/tool/ + └── maps keys to Action enum │── dispatches Action::* └── 28 tool impls + │ matching on Action variant + │── applies state mutations +``` + +### Key Patterns + +- **State mutation** — `AppStateRest` is mutable in-place from `actions/mod.rs` and `controller/input.rs`. No generic update function. +- **No DI** — modules call `Settings::load()`, `AppConfig::load()`, `all_tools()` directly. +- **Logging** — `tracing::warn!` to `~/.local/share/zesdex/zesdex.log` (not stderr, avoids TUI corruption). +- **Error handling** — `anyhow::Result` and `anyhow::bail!` throughout. No custom error types. +- **Static strings** — MCP tool descriptions use `Box::leak` + `OnceLock` cache. +- **Tools** — `trait Tool { fn name() -> &str, fn run() -> Result }`, 28 impls, gated by `Harness`. +- **Shell safety** — `tool/shell_filter/` blocks credential leaks and destructive git commands. + +## Code Documentation + +Every function, struct, enum, trait, module, and significant code block must have a doc comment (`///` or `//!`) that explains: + +- **What** the function/module does (purpose, not how) +- **Flow** — a brief ASCII or prose description of the code flow / data flow above each non-trivial function +- **Why** — non-obvious decisions, edge cases, invariants +- **Return** — what the caller gets back, especially for `Result` types + +Examples: + +```rust +/// Parse an SSE data chunk into one or more StreamEvents. +/// +/// Flow: buffer → split on '\n' → flush on blank line → JSON parse → match event type +/// → return Token / ToolCallDelta / Usage / Done. +/// +/// Edge case: chunk may split mid-line; remaining bytes stay in buffer +/// for the next feed() call. +fn feed(&mut self, chunk: &str) -> Vec { ... } + +/// The single source-of-truth state struct for the entire application. +/// +/// Mutated in-place from two locations: actions/mod.rs (apply_action) +/// and controller/input.rs (key event handlers). Read-only from +/// every other module. +struct AppStateRest { ... } +``` + +Rules: +- Every `pub fn` needs a doc comment +- Every `pub struct` / `pub enum` / `pub trait` needs a doc comment +- Non-trivial private functions (≥10 lines) need a doc comment +- Write the comment above the code it documents (not inline in the body) +- Update comments when code behavior changes — stale docs are worse than no docs diff --git a/docs/CODEMAPS/architecture.md b/docs/CODEMAPS/architecture.md new file mode 100644 index 0000000..b0728d4 --- /dev/null +++ b/docs/CODEMAPS/architecture.md @@ -0,0 +1,61 @@ + + +# Architecture + +Zesdex is a single-process terminal AI coding agent with optional daemon/client split. + +## System Layout + +``` +┌──────────────────────────────────────────────────────┐ +│ main.rs │ +│ single-process ─┬── daemon ── Unix socket ── client │ +│ └── attach (TUI-only client) │ +└──────────────────────┬───────────────────────────────┘ + │ +┌──────────────────────▼───────────────────────────────┐ +│ Event Loop │ +│ ┌────────┐ ┌───────────┐ ┌──────┐ ┌────────┐ │ +│ │Input │──▶│ Actions │──▶│State │──▶│ TUI │ │ +│ │Handler │ │ (dispatch)│ │ │ │ Render │ │ +│ └────────┘ └─────┬─────┘ └──────┘ └────────┘ │ +│ │ │ +│ ┌──────▼──────┐ │ +│ │ LLM Stream │ │ +│ │ + Tool Exec │ │ +│ └──────┬──────┘ │ +│ ┌────┴────┐ │ +│ │ │ │ +│ ┌─────▼──┐ ┌───▼────┐ │ +│ │ Tools │ │Sub- │ │ +│ │ (28) │ │agents │ │ +│ └────────┘ └────────┘ │ +└───────────────────────────────────────────────────────┘ +``` + +## Data Flow + +``` +User keystroke → Controller (KeyEvent → Action) + → apply_action() mutates AppStateRest + → TUI redraws (ratatui Frame) + → On submit: LLM request → SSE stream → tool calls → tool results → more LLM + → Session persisted to disk (editlog, msglog, memory) +``` + +## Process Modes + +| Mode | Impl | Process | IPC | +|------|------|---------|-----| +| Single | `run_single_process()` | One | No | +| Daemon | `run_daemon()` | Server | `ipc/server.rs` | +| Attach | `run_attach()` | Client | `ipc/client.rs` | + +## Key Files + +| File | Lines | Role | +|------|-------|------| +| `src/main.rs` | 530 | Entry, TUI setup, daemon loop, attach loop | +| `src/app/runtime/actions/mod.rs` | 1022 | Action dispatch + LLM stream loop + tool execution | +| `src/controller/input.rs` | 281 | Key event → Action mapping | +| `src/view/mod.rs` | 623 | TUI rendering (ratatui) | diff --git a/docs/CODEMAPS/backend.md b/docs/CODEMAPS/backend.md new file mode 100644 index 0000000..ddb8566 --- /dev/null +++ b/docs/CODEMAPS/backend.md @@ -0,0 +1,68 @@ + + +# Backend / Service Layer + +## AI Provider + +`src/service/provider.rs` (258 lines) +- `LlmClient::new(api_key, model, base_url)` — constructs blocking reqwest client +- `chat_with_tools()` — non-streaming with tool definitions +- `chat_stream()` — SSE streaming, returns `SseParser` yielding `StreamEvent` +- Retry logic: up to 3 attempts on transient errors, exponential backoff + +## OAuth + +`src/service/oauth/manager.rs` (113 lines) + `loopback.rs` + `pkce.rs` +- PKCE flow: `CodeVerifier` → challenge → browser auth → loopback server → token exchange +- Configurable via `app_config.json` provider definitions (auth URL, token URL, scopes) + +## IPC / Daemon + +`src/ipc/` (7 files, ~300 lines total) +- Unix domain socket, length-prefixed JSON frames +- Daemon sends `DaemonFrame { state: StatePayload, diff, tasks }` to clients +- Clients send `ClientRequest { action: Action }` back +- State sync uses snapshots + binary diffs (rsync-style, not git) + +## Workflow Engine + +`src/app/workflow/engine.rs` (251 lines) + `script.rs` +- Inline JS-style DSL executed by a lightweight runtime +- `agent()`, `parallel()`, `pipeline()`, `phase()`, `log()` — spawns sub-agents +- Max concurrency configurable via `workflow_max_concurrency` setting + +## Sub-Agent System + +`src/app/subagent/` (4 files, ~250 lines) +- `run_subagent()` — spawns independent agent with its own tool set & context +- Communicates via `mpsc` channel (tool calls, results, completion) +- Uses `LlmClient` (same as main agent) with tool-use API + +## MCP Client + +`src/app/mcp/manager.rs` (371 lines) +- Stdio transport: spawns child process, JSON-RPC via stdin/stdout +- HTTP transport: streaming HTTP with JSON-RPC +- Tool registration: `tools/list` → `McpToolAdapter` implements `crate::tool::Tool` +- Persistent child handle for stdio (reuses connection across calls) + +## Self-Review + +`src/app/review/mod.rs` (437 lines) +- Post-tool execution quality check against learned lessons +- Invokes `run_subagent()` with reviewer prompt +- Staleness detection: skips review after N consecutive empty results + +## Background Bash + +`src/app/bgbash/` (2 files) +- `spawn_bash_job()` — runs `sh -c` in a thread, collects stdout line-by-line +- Channels: output via `mpsc`, PID via `mpsc` +- Killable via PID + +## Gate Guard / Harness + +`src/app/harness.rs` (127 lines) +- `Harness::gate_tool_call()` — verdict-based tool gating (allow/block) +- Parses LLM verdicts (JSON or plain-text) +- `test_parse_verdict_*` tests for 6 verdict formats diff --git a/docs/CODEMAPS/data.md b/docs/CODEMAPS/data.md new file mode 100644 index 0000000..e86f5e9 --- /dev/null +++ b/docs/CODEMAPS/data.md @@ -0,0 +1,51 @@ + + +# Data / Persistence Layer + +## Storage Overview + +Base directory: `~/.config/zesdex/` (via `dirs::data_dir()`) + +``` +~/.config/zesdex/ +├── settings.json # User preferences (provider, model, tokens) +├── app_config.json # Provider definitions (API base, auth, models) +├── agents/ # Global agent definitions +│ └── *.json +├── memory/ # Persistent lesson/reference store +│ └── *.md # Markdown with YAML frontmatter +├── sessions/ # Per-session data +│ └── / +│ ├── editlog.json # Edit history +│ ├── msglog.db # SQLite message log +│ ├── transcript.json # Chat transcript +│ ├── session.json # Session metadata +│ ├── agents.json # Session-local agent defs +│ └── snapshot.dat # State snapshot (daemon mode) +├── run/ # Unix domain sockets +│ └── zesdex-*.sock +└── store.json # Legacy session index +``` + +## Key Files + +| File | Lines | Role | +|------|-------|------| +| `src/model/store.rs` | ~50 | File-system storage (ensure_dirs, base_dir resolution) | +| `src/model/settings.rs` | ~60 | `Settings` — load/save JSON, API keys map | +| `src/model/app_config.rs` | ~80 | `AppConfig` — provider definitions, model roles, auth | +| `src/model/memory.rs` | 332 | Memory CRUD — markdown files with frontmatter | +| `src/model/editlog.rs` | 121 | Edit log — append-only JSON array | +| `src/model/msglog/` | 4 files | SQLite-backed message log (schema, query, blobs) | +| `src/model/session.rs` | ~60 | Session CRUD, listing, archival | +| `src/model/session_lock.rs` | ~50 | flock-based session lock | +| `src/model/agent_def/` | 3 files | Agent definitions (builtin, global, session-local) | + +## Key Patterns + +- **No ORM** — raw JSON files + SQLite via rusqlite +- **settings.json** — loaded at startup, saved on quit / mode switches +- **Memory format** — Markdown files with YAML frontmatter (`---\nname: ...\ndescription: ...\n---\ncontent`) +- **Edit log** — append-only, stores `(file, old, new, timestamp, tool)` +- **Session locking** — flock-based, prevents concurrent access to same session dir +- **Message log** — SQLite with attached blobs for tool arguments/outputs diff --git a/docs/CODEMAPS/dependencies.md b/docs/CODEMAPS/dependencies.md new file mode 100644 index 0000000..87b313d --- /dev/null +++ b/docs/CODEMAPS/dependencies.md @@ -0,0 +1,41 @@ + + +# Dependencies + +## Rust Crates (Cargo.toml) + +| Crate | Version | Purpose | +|-------|---------|---------| +| ratatui | 0.30 | TUI framework (tui-rs successor) | +| crossterm | 0.28 | Terminal manipulation (raw mode, alt screen) | +| tokio | 1 | Async runtime (daemon, OAuth loopback) | +| reqwest | 0.12 | HTTP client (blocking + streaming, vendored native-tls) | +| serde / serde_json | 1 | JSON serialization (state, DTOs, IPC, config) | +| serde_yaml_ng | 0.9 | YAML frontmatter parsing (memory files) | +| anyhow | 1 | Error handling (no custom error types) | +| tracing / tracing-subscriber | 0.1/0.3 | Structured logging → file | +| rusqlite | 0.32 | SQLite (bundled, for message log) | +| pulldown-cmark | 0.13 | Markdown → HTML (chat rendering) | +| syntect | 5 | Syntax highlighting (code blocks in chat) | +| sha2 | 0.10 | SHA-256 for PKCE challenge | +| base64 | 0.22 | URL-safe base64 for PKCE | +| libc | 0.2 | daemon PID file locking | +| rmcp | 1.8 | MCP client (stdio + HTTP transports) | +| uuid | 1 | Session IDs, job IDs | +| chrono | 0.4 | Timestamps (ISO 8601, millis) | +| dirs | 5 | Platform data directories | +| dom_smoothie | 0.18 | HTML → plain text (web scraping) | +| scraper | 0.27 | HTML parsing (web scraping) | +| ignore | 0.4 | .gitignore-aware file walking (glob tool) | +| regex / globset | 0.4 | Pattern matching (grep/glob tools) | +| url / percent-encoding | 2 | URL parsing + encoding (OAuth) | + +## External Services + +| Service | Integration | Notes | +|---------|-------------|-------| +| **LLM providers** | HTTP API (OpenAI-compatible) | Configurable via app_config.json | +| **MCP servers** | stdio or HTTP | Model Context Protocol | +| **git** | CLI (spawns `git`) | Via git_operator/git_worktree/git_cred tools | +| **sh** | CLI (spawns `sh`) | Via bash tool | +| **webbrowser** | opens URL | OAuth browser flow | diff --git a/docs/CODEMAPS/frontend.md b/docs/CODEMAPS/frontend.md new file mode 100644 index 0000000..24ba518 --- /dev/null +++ b/docs/CODEMAPS/frontend.md @@ -0,0 +1,64 @@ + + +# Frontend / TUI + +## Render Pipeline + +``` +ratatui::Terminal::draw(|frame|) + → view::draw(frame, AppStateRest) + → render_main_panel / render_overlay (based on overlay state) + → render_input_bar + → draw_status_bar + → render_toasts (top-right floating notifications) +``` + +## Layout + +``` +┌──────────────────────────────────────────────┐ +│ Chat Panel (main_area: Min 3) │ +│ ┌────────────────────────────────────────┐ │ +│ │ User: Hello │ │ +│ │ Agent: Hi there, how can I help? │ │ +│ │ │ │ +│ │ Toast notifications (top-right) │ │ +│ └────────────────────────────────────────┘ │ +├──────────────────────────────────────────────┤ +│ Input Bar (3 lines) │ +│ > Some text... │ +├──────────────────────────────────────────────┤ +│ Status Bar (1 line) │ +│ ┌ Provider │ Model │ Tokens │ Mode │ Quit ─┤ +└──────────────────────────────────────────────┘ +``` + +## Key Files + +| File | Lines | Purpose | +|------|-------|---------| +| `src/view/mod.rs` | 623 | Frame draw, overlays (16 types), input bar, toasts | +| `src/view/chat.rs` | 155 | Chat transcript rendering with markdown | +| `src/view/markdown.rs` | 144 | Markdown → ratatui `Span` rendering (pulldown-cmark + syntect) | +| `src/view/status.rs` | ~50 | Status bar with provider/model/tokens | +| `src/view/workflow.rs` | 88 | Workflow progress visualization | +| `src/view/theme.rs` | 23 | Color palette (23 named colors) | +| `src/controller/input.rs` | 281 | Key event → Action mapping | + +## Overlays (16 types) + +`Overlay::Help | Settings | Bash | QuitConfirm | Workflow | KeyInput | Editor | Effort | Mcp | Todo | Rewind | Learning | Usage | Loading | ModelSelector | ClearConfirm` + +Each overlay renders a centered popup via `render_overlay()`. + +## State Mutations + +State is mutated in-place from two locations: +- `src/controller/input.rs` — keyboard shortcuts and overlay interactions +- `src/app/runtime/actions/mod.rs` — `apply_action()` reducer for all programmatic actions + +## Toast Notifications + +`render_toasts()` — floating stack at top-right, color-coded by severity: +- Info: blue, Success: green, Warning: yellow, Error: red, Lesson: cyan +- Max 4 visible, auto-expire after 5s lifetime