feat: add initial codemap documentation and architecture overview; include backend, data, dependencies, and frontend details

2026-07-12 11:10:16 +07:00
parent 89b63b1bc2
commit 7158d362fd
7 changed files with 419 additions and 0 deletions
@@ -0,0 +1,61 @@
+<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~750 -->
+
+# Architecture
+
+Zesdex is a single-process terminal AI coding agent with optional daemon/client split.
+
+## System Layout
+
+```
+┌──────────────────────────────────────────────────────┐
+│                    main.rs                            │
+│  single-process ─┬── daemon ── Unix socket ── client  │
+│                  └── attach <id> (TUI-only client)    │
+└──────────────────────┬───────────────────────────────┘
+                       │
+┌──────────────────────▼───────────────────────────────┐
+│                   Event Loop                          │
+│  ┌────────┐   ┌───────────┐   ┌──────┐   ┌────────┐  │
+│  │Input   │──▶│  Actions  │──▶│State │──▶│  TUI   │  │
+│  │Handler │   │ (dispatch)│   │      │   │ Render │  │
+│  └────────┘   └─────┬─────┘   └──────┘   └────────┘  │
+│                     │                                 │
+│              ┌──────▼──────┐                          │
+│              │ LLM Stream  │                          │
+│              │ + Tool Exec │                          │
+│              └──────┬──────┘                          │
+│                ┌────┴────┐                            │
+│                │         │                            │
+│          ┌─────▼──┐ ┌───▼────┐                        │
+│          │ Tools  │ │Sub-    │                        │
+│          │ (28)   │ │agents  │                        │
+│          └────────┘ └────────┘                        │
+└───────────────────────────────────────────────────────┘
+```
+
+## Data Flow
+
+```
+User keystroke → Controller (KeyEvent → Action)
+  → apply_action() mutates AppStateRest
+  → TUI redraws (ratatui Frame)
+  → On submit: LLM request → SSE stream → tool calls → tool results → more LLM
+  → Session persisted to disk (editlog, msglog, memory)
+```
+
+## Process Modes
+
+| Mode | Impl | Process | IPC |
+|------|------|---------|-----|
+| Single | `run_single_process()` | One | No |
+| Daemon | `run_daemon()` | Server | `ipc/server.rs` |
+| Attach | `run_attach()` | Client | `ipc/client.rs` |
+
+## Key Files
+
+| File | Lines | Role |
+|------|-------|------|
+| `src/main.rs` | 530 | Entry, TUI setup, daemon loop, attach loop |
+| `src/app/runtime/actions/mod.rs` | 1022 | Action dispatch + LLM stream loop + tool execution |
+| `src/controller/input.rs` | 281 | Key event → Action mapping |
+| `src/view/mod.rs` | 623 | TUI rendering (ratatui) |
@@ -0,0 +1,68 @@
+<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~850 -->
+
+# Backend / Service Layer
+
+## AI Provider
+
+`src/service/provider.rs` (258 lines)
+- `LlmClient::new(api_key, model, base_url)` — constructs blocking reqwest client
+- `chat_with_tools()` — non-streaming with tool definitions
+- `chat_stream()` — SSE streaming, returns `SseParser` yielding `StreamEvent`
+- Retry logic: up to 3 attempts on transient errors, exponential backoff
+
+## OAuth
+
+`src/service/oauth/manager.rs` (113 lines) + `loopback.rs` + `pkce.rs`
+- PKCE flow: `CodeVerifier` → challenge → browser auth → loopback server → token exchange
+- Configurable via `app_config.json` provider definitions (auth URL, token URL, scopes)
+
+## IPC / Daemon
+
+`src/ipc/` (7 files, ~300 lines total)
+- Unix domain socket, length-prefixed JSON frames
+- Daemon sends `DaemonFrame { state: StatePayload, diff, tasks }` to clients
+- Clients send `ClientRequest { action: Action }` back
+- State sync uses snapshots + binary diffs (rsync-style, not git)
+
+## Workflow Engine
+
+`src/app/workflow/engine.rs` (251 lines) + `script.rs`
+- Inline JS-style DSL executed by a lightweight runtime
+- `agent()`, `parallel()`, `pipeline()`, `phase()`, `log()` — spawns sub-agents
+- Max concurrency configurable via `workflow_max_concurrency` setting
+
+## Sub-Agent System
+
+`src/app/subagent/` (4 files, ~250 lines)
+- `run_subagent()` — spawns independent agent with its own tool set & context
+- Communicates via `mpsc<SubagentEvent>` channel (tool calls, results, completion)
+- Uses `LlmClient` (same as main agent) with tool-use API
+
+## MCP Client
+
+`src/app/mcp/manager.rs` (371 lines)
+- Stdio transport: spawns child process, JSON-RPC via stdin/stdout
+- HTTP transport: streaming HTTP with JSON-RPC
+- Tool registration: `tools/list` → `McpToolAdapter` implements `crate::tool::Tool`
+- Persistent child handle for stdio (reuses connection across calls)
+
+## Self-Review
+
+`src/app/review/mod.rs` (437 lines)
+- Post-tool execution quality check against learned lessons
+- Invokes `run_subagent()` with reviewer prompt
+- Staleness detection: skips review after N consecutive empty results
+
+## Background Bash
+
+`src/app/bgbash/` (2 files)
+- `spawn_bash_job()` — runs `sh -c` in a thread, collects stdout line-by-line
+- Channels: output via `mpsc<String>`, PID via `mpsc<u32>`
+- Killable via PID
+
+## Gate Guard / Harness
+
+`src/app/harness.rs` (127 lines)
+- `Harness::gate_tool_call()` — verdict-based tool gating (allow/block)
+- Parses LLM verdicts (JSON or plain-text)
+- `test_parse_verdict_*` tests for 6 verdict formats
@@ -0,0 +1,51 @@
+<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~600 -->
+
+# Data / Persistence Layer
+
+## Storage Overview
+
+Base directory: `~/.config/zesdex/` (via `dirs::data_dir()`)
+
+```
+~/.config/zesdex/
+├── settings.json        # User preferences (provider, model, tokens)
+├── app_config.json      # Provider definitions (API base, auth, models)
+├── agents/              # Global agent definitions
+│   └── *.json
+├── memory/              # Persistent lesson/reference store
+│   └── *.md             # Markdown with YAML frontmatter
+├── sessions/            # Per-session data
+│   └── <session-uuid>/
+│       ├── editlog.json      # Edit history
+│       ├── msglog.db         # SQLite message log
+│       ├── transcript.json   # Chat transcript
+│       ├── session.json      # Session metadata
+│       ├── agents.json       # Session-local agent defs
+│       └── snapshot.dat      # State snapshot (daemon mode)
+├── run/                 # Unix domain sockets
+│   └── zesdex-*.sock
+└── store.json           # Legacy session index
+```
+
+## Key Files
+
+| File | Lines | Role |
+|------|-------|------|
+| `src/model/store.rs` | ~50 | File-system storage (ensure_dirs, base_dir resolution) |
+| `src/model/settings.rs` | ~60 | `Settings` — load/save JSON, API keys map |
+| `src/model/app_config.rs` | ~80 | `AppConfig` — provider definitions, model roles, auth |
+| `src/model/memory.rs` | 332 | Memory CRUD — markdown files with frontmatter |
+| `src/model/editlog.rs` | 121 | Edit log — append-only JSON array |
+| `src/model/msglog/` | 4 files | SQLite-backed message log (schema, query, blobs) |
+| `src/model/session.rs` | ~60 | Session CRUD, listing, archival |
+| `src/model/session_lock.rs` | ~50 | flock-based session lock |
+| `src/model/agent_def/` | 3 files | Agent definitions (builtin, global, session-local) |
+
+## Key Patterns
+
+- **No ORM** — raw JSON files + SQLite via rusqlite
+- **settings.json** — loaded at startup, saved on quit / mode switches
+- **Memory format** — Markdown files with YAML frontmatter (`---\nname: ...\ndescription: ...\n---\ncontent`)
+- **Edit log** — append-only, stores `(file, old, new, timestamp, tool)`
+- **Session locking** — flock-based, prevents concurrent access to same session dir
+- **Message log** — SQLite with attached blobs for tool arguments/outputs
@@ -0,0 +1,41 @@
+<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~400 -->
+
+# Dependencies
+
+## Rust Crates (Cargo.toml)
+
+| Crate | Version | Purpose |
+|-------|---------|---------|
+| ratatui | 0.30 | TUI framework (tui-rs successor) |
+| crossterm | 0.28 | Terminal manipulation (raw mode, alt screen) |
+| tokio | 1 | Async runtime (daemon, OAuth loopback) |
+| reqwest | 0.12 | HTTP client (blocking + streaming, vendored native-tls) |
+| serde / serde_json | 1 | JSON serialization (state, DTOs, IPC, config) |
+| serde_yaml_ng | 0.9 | YAML frontmatter parsing (memory files) |
+| anyhow | 1 | Error handling (no custom error types) |
+| tracing / tracing-subscriber | 0.1/0.3 | Structured logging → file |
+| rusqlite | 0.32 | SQLite (bundled, for message log) |
+| pulldown-cmark | 0.13 | Markdown → HTML (chat rendering) |
+| syntect | 5 | Syntax highlighting (code blocks in chat) |
+| sha2 | 0.10 | SHA-256 for PKCE challenge |
+| base64 | 0.22 | URL-safe base64 for PKCE |
+| libc | 0.2 | daemon PID file locking |
+| rmcp | 1.8 | MCP client (stdio + HTTP transports) |
+| uuid | 1 | Session IDs, job IDs |
+| chrono | 0.4 | Timestamps (ISO 8601, millis) |
+| dirs | 5 | Platform data directories |
+| dom_smoothie | 0.18 | HTML → plain text (web scraping) |
+| scraper | 0.27 | HTML parsing (web scraping) |
+| ignore | 0.4 | .gitignore-aware file walking (glob tool) |
+| regex / globset | 0.4 | Pattern matching (grep/glob tools) |
+| url / percent-encoding | 2 | URL parsing + encoding (OAuth) |
+
+## External Services
+
+| Service | Integration | Notes |
+|---------|-------------|-------|
+| **LLM providers** | HTTP API (OpenAI-compatible) | Configurable via app_config.json |
+| **MCP servers** | stdio or HTTP | Model Context Protocol |
+| **git** | CLI (spawns `git`) | Via git_operator/git_worktree/git_cred tools |
+| **sh** | CLI (spawns `sh`) | Via bash tool |
+| **webbrowser** | opens URL | OAuth browser flow |
@@ -0,0 +1,64 @@
+<!-- Generated: 2026-07-12 | Files scanned: 124 | Token estimate: ~700 -->
+
+# Frontend / TUI
+
+## Render Pipeline
+
+```
+ratatui::Terminal::draw(|frame|)
+  → view::draw(frame, AppStateRest)
+    → render_main_panel / render_overlay (based on overlay state)
+    → render_input_bar
+    → draw_status_bar
+    → render_toasts (top-right floating notifications)
+```
+
+## Layout
+
+```
+┌──────────────────────────────────────────────┐
+│ Chat Panel (main_area: Min 3)                │
+│  ┌────────────────────────────────────────┐  │
+│  │ User: Hello                           │  │
+│  │ Agent: Hi there, how can I help?      │  │
+│  │                                       │  │
+│  │ Toast notifications (top-right)       │  │
+│  └────────────────────────────────────────┘  │
+├──────────────────────────────────────────────┤
+│ Input Bar (3 lines)                          │
+│ > Some text...                               │
+├──────────────────────────────────────────────┤
+│ Status Bar (1 line)                          │
+│  ┌ Provider │ Model │ Tokens │ Mode │ Quit ─┤
+└──────────────────────────────────────────────┘
+```
+
+## Key Files
+
+| File | Lines | Purpose |
+|------|-------|---------|
+| `src/view/mod.rs` | 623 | Frame draw, overlays (16 types), input bar, toasts |
+| `src/view/chat.rs` | 155 | Chat transcript rendering with markdown |
+| `src/view/markdown.rs` | 144 | Markdown → ratatui `Span` rendering (pulldown-cmark + syntect) |
+| `src/view/status.rs` | ~50 | Status bar with provider/model/tokens |
+| `src/view/workflow.rs` | 88 | Workflow progress visualization |
+| `src/view/theme.rs` | 23 | Color palette (23 named colors) |
+| `src/controller/input.rs` | 281 | Key event → Action mapping |
+
+## Overlays (16 types)
+
+`Overlay::Help | Settings | Bash | QuitConfirm | Workflow | KeyInput | Editor | Effort | Mcp | Todo | Rewind | Learning | Usage | Loading | ModelSelector | ClearConfirm`
+
+Each overlay renders a centered popup via `render_overlay()`.
+
+## State Mutations
+
+State is mutated in-place from two locations:
+- `src/controller/input.rs` — keyboard shortcuts and overlay interactions
+- `src/app/runtime/actions/mod.rs` — `apply_action()` reducer for all programmatic actions
+
+## Toast Notifications
+
+`render_toasts()` — floating stack at top-right, color-coded by severity:
+- Info: blue, Success: green, Warning: yellow, Error: red, Lesson: cyan
+- Max 4 visible, auto-expire after 5s lifetime