MythEclipse/zesdex
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

perf(agent): rombak alur AI agent — adaptif, hemat token, self-healing

Browse Source 
Ganti explore phase MANDATORY (3 subagent tiap turn, boros) dengan
tool explore_codebase yang DIPUTUSKAN agent sendiri (lazy, token-aware):
- hapus ExploreService trait + with_explore + Phase 0 dari turn loop
- ExploreServiceImpl kini jadi tool 'explore_codebase' (1 context-scout
  subagent, read-only, cap output 4k chars)
- system prompt: instruksi TOKEN BUDGET (jawab langsung utk query simple,
  panggil explore_codebase sekali utk task kompleks)

Loop utama kini adaptif & self-healing:
- max_tokens adaptif (800/1600/4096 by request length) — bukan selalu 4096
- temperature 0.2 saat tool-calling, 0.7 utk final answer
- ErrorTracker: deteksi tool error berulang → inject recovery note,
  stop setelah 8 error total (bukan 50 iterasi sia-sia)
- auto-compact history > 60k chars sebelum LLM call
- tool output di-truncate ke 12k chars sebelum masuk konteks

Tambah 8 unit test (truncation, adaptive tokens, error tracker).
This commit is contained in:
asepharyana
2026-08-27 23:06:37 +07:00
parent 14f3eae62a
commit eac0443c4c
9 changed files with 422 additions and 448 deletions
Download Patch File Download Diff File Expand all files Collapse all files
apps/application/src/agent/turn_service.rs
+252 -85
View File
@@ -7,12 +7,29 @@ use zesdex_domain::agent::{AgentTurnParams, TurnEvent};
use zesdex_domain::core::{ChatMessage, StreamEvent, ToolDef};
use zesdex_domain::main_agent_prompt;
use super::{ExploreService, ToolExecutor};
use super::ToolExecutor;
use crate::ports::ProviderService;
/// Maximum tool-call iterations per agent turn before forcing termination.
const MAX_TURN_ITERATIONS: u32 = 50;
/// Maximum number of consecutive identical tool errors before the loop
/// injects a recovery note and forces a different approach.
const MAX_CONSECUTIVE_TOOL_ERRORS: usize = 3;
/// Total tool-call errors tolerated per turn before the loop is stopped.
const MAX_TOTAL_TOOL_ERRORS: usize = 8;
/// Ceiling for a single tool-result message inserted into context.
///
/// Tool outputs can be huge (read / semantic_search). Truncating keeps the
/// context window from exploding while preserving the important head.
const TOOL_OUTPUT_MAX_CHARS: usize = 12_000;
/// Total conversation characters that trigger auto-compaction before the
/// next LLM call.
const AUTO_COMPACT_CHARS: usize = 60_000;
// ---------------------------------------------------------------------------
// Helper: push a TurnEvent onto the shared queue.
// ---------------------------------------------------------------------------
@@ -51,6 +68,89 @@ fn make_stream_callback(
})
}
// ---------------------------------------------------------------------------
// Helper: truncate a long tool output before it enters the conversation
// context. Preserves the head and appends a clear truncation marker.
// ---------------------------------------------------------------------------
fn truncate_tool_output(output: String) -> String {
if output.len() <= TOOL_OUTPUT_MAX_CHARS {
return output;
}
let mut result: String = output.chars().take(TOOL_OUTPUT_MAX_CHARS).collect();
result.push_str(&format!(
"\n...[truncated {} chars]",
output.len() - TOOL_OUTPUT_MAX_CHARS
));
result
}
// ---------------------------------------------------------------------------
// Helper: adaptive generation parameters.
// ---------------------------------------------------------------------------
/// Pick a `max_tokens` budget for the turn's next LLM call based on the
/// length of the user's request. Short requests need far fewer tokens than
/// the current hardcoded 4096 — big savings on small tasks.
fn adaptive_max_tokens(request_len: usize) -> u32 {
if request_len <= 80 {
800
} else if request_len <= 400 {
1600
} else {
4096
}
}
/// Sum the character length of the conversation (user + assistant +
/// tool content) as a cheap proxy for context size.
fn conversation_chars(messages: &[ChatMessage]) -> usize {
messages
.iter()
.map(|m| m.content.as_deref().map(str::len).unwrap_or(0))
.sum()
}
/// Track repeated tool-call errors so the loop can recover instead of
/// burning iterations retrying the same failing tool.
#[derive(Default)]
struct ErrorTracker {
consecutive: usize,
total: usize,
last_tool: String,
last_error: String,
}
impl ErrorTracker {
fn record(&mut self, tool_name: &str, error: &str, messages: &mut Vec<ChatMessage>) {
if self.last_tool == tool_name {
self.consecutive += 1;
} else {
self.consecutive = 1;
}
self.last_tool = tool_name.to_string();
self.last_error = error.to_string();
self.total += 1;
// Inject a recovery note once the same tool keeps failing.
if self.consecutive >= MAX_CONSECUTIVE_TOOL_ERRORS
&& !messages.iter().any(|m| {
m.content
.as_deref()
.is_some_and(|c| c.contains("[System note]"))
})
{
messages.push(ChatMessage::system(
zesdex_domain::agent::prompt::error_recovery_note(tool_name, error),
));
}
}
fn should_stop(&self) -> bool {
self.consecutive >= MAX_CONSECUTIVE_TOOL_ERRORS * 2 || self.total >= MAX_TOTAL_TOOL_ERRORS
}
}
// ---------------------------------------------------------------------------
// Helper: execute a single tool call, push events, return the result string.
// ---------------------------------------------------------------------------
@@ -71,6 +171,7 @@ async fn execute_tool_call<T: ToolExecutor>(
};
let is_error = output.starts_with("Error:");
let output = truncate_tool_output(output);
push_event(
turn_events,
@@ -108,19 +209,19 @@ fn emit_usage(turn_events: &Arc<Mutex<VecDeque<TurnEvent>>>, usage: Option<(u64,
/// Service implementation for executing an agent turn asynchronously.
///
/// # Explore phase
///
/// Before the main LLM loop begins, [`AgentTurnServiceImpl`] runs a mandatory
/// explore phase that spawns ≥3 parallel subagents (code structure, symbol
/// index, semantic context) and injects their consolidated findings as a
/// system message. See [`ExploreService`] for the trait contract.
/// The turn loop is adaptive and token-aware:
/// - No mandatory explore phase — the *agent* decides when to call the
/// `explore_codebase` tool (see the main prompt), so simple queries skip
/// exploration entirely.
/// - `max_tokens` / `temperature` adapt to the request length and phase.
/// - Repeated tool errors trigger a system recovery note and eventually
/// stop the loop instead of burning iterations.
/// - Tool outputs are truncated before entering context.
/// - Oversized histories are auto-compacted before the next LLM call.
pub struct AgentTurnServiceImpl<P: ProviderService, T: ToolExecutor> {
provider: Arc<P>,
tool_executor: Arc<T>,
tool_defs: Vec<ToolDef>,
/// Optional explore-phase service. When `Some`, the explore phase runs
/// before every turn; when `None` it is skipped (tests, daemon mode).
explore_service: Option<Arc<dyn ExploreService>>,
}
impl<P: ProviderService, T: ToolExecutor> AgentTurnServiceImpl<P, T> {
@@ -129,19 +230,9 @@ impl<P: ProviderService, T: ToolExecutor> AgentTurnServiceImpl<P, T> {
provider,
tool_executor,
tool_defs,
explore_service: None,
}
}
/// Attach an optional explore-phase service.
///
/// When set, every call to `run_turn` will first run the explore phase
/// and inject the consolidated context as a system message.
pub fn with_explore(mut self, service: Arc<dyn ExploreService>) -> Self {
self.explore_service = Some(service);
self
}
/// Execute a single LLM call with the current message list, handling
/// streaming events and error reporting.
async fn call_llm(
@@ -149,6 +240,8 @@ impl<P: ProviderService, T: ToolExecutor> AgentTurnServiceImpl<P, T> {
messages: &[ChatMessage],
abort: &Arc<AtomicBool>,
turn_events: &Arc<Mutex<VecDeque<TurnEvent>>>,
max_tokens: u32,
temperature: f32,
) -> Result<(ChatMessage, Option<(u64, u64)>), String> {
let on_event = make_stream_callback(abort, turn_events);
@@ -156,13 +249,39 @@ impl<P: ProviderService, T: ToolExecutor> AgentTurnServiceImpl<P, T> {
.chat_stream(
messages,
Some(self.tool_defs.clone()),
Some(4096),
Some(0.7),
Some(max_tokens),
Some(temperature),
on_event,
)
.await
.map_err(|e| format!("LLM error: {e}"))
}
/// Auto-compact the history in place if it exceeds the threshold.
///
/// Runs at most once per turn. Skips the synthetic system prompt that
/// this service inserts at index 0.
async fn auto_compact_if_needed(&self, messages: &mut Vec<ChatMessage>) {
if conversation_chars(messages) <= AUTO_COMPACT_CHARS {
return;
}
// Keep the system prompt (index 0) out of compaction.
let sys = messages[0].clone();
let mut rest: Vec<ChatMessage> = messages.drain(1..).collect();
let before = rest.len();
if let Err(e) = super::compact_messages_with_ai(&mut rest, self.provider.as_ref()).await {
warn!("auto-compact failed (non-fatal): {e}");
}
info!(
"auto-compacted history: {} messages -> {}",
before,
rest.len()
);
let mut rebuilt = Vec::with_capacity(rest.len() + 1);
rebuilt.push(sys);
rebuilt.extend(rest);
*messages = rebuilt;
}
}
impl<P: ProviderService, T: ToolExecutor> super::AgentTurnService for AgentTurnServiceImpl<P, T> {
@@ -173,73 +292,26 @@ impl<P: ProviderService, T: ToolExecutor> super::AgentTurnService for AgentTurnS
params.model
);
// ── Phase 0: Mandatory explore ──────────────────────────────────
// Spawn ≥3 parallel subagents to discover code structure, symbols,
// and semantic context. The consolidated summary is injected as a
// system message before the main agent prompt.
if let Some(ref explorer) = self.explore_service {
// Determine workspace root from the first message's context or
// the first workspace root in params.
let user_query = params
.messages
.last()
.map(|m| m.content.clone().unwrap_or_default())
.unwrap_or_default();
let workspace_root = params
.workspace_roots
.first()
.map(|p| p.to_string_lossy().to_string())
.unwrap_or_else(|| ".".to_string());
push_event(
&params.turn_events,
TurnEvent::SystemNote {
kind: "info".into(),
message: "🔍 Exploring codebase structure...".into(),
},
);
match explorer
.explore(&user_query, &workspace_root, &params.turn_events)
.await
{
Ok(output) => {
// Insert each context message as a system message.
// They go at index 0 and are removed after the turn
// like the main agent prompt.
for ctx_msg in &output.context_messages {
params
.messages
.insert(0, ChatMessage::system(ctx_msg.clone()));
}
info!(
"Explore phase complete: {} context messages, {}",
output.context_messages.len(),
output.summary
);
}
Err(e) => {
warn!("Explore phase failed (non-fatal): {e}");
push_event(
&params.turn_events,
TurnEvent::SystemNote {
kind: "warn".into(),
message: format!("Explore phase failed: {e}"),
},
);
}
}
}
// Insert system prompt at position 0 once and keep it there for the
// entire turn, avoiding per-iteration clones of the full message list.
// It is removed before emitting the Compacted event so persistence
// does not store the prompt redundantly.
params
.messages
.insert(0, ChatMessage::system(main_agent_prompt()));
let original_count = params.messages.len();
// Estimate request complexity from the last user message.
let request_len = params
.messages
.last()
.and_then(|m| m.content.as_deref())
.map(str::len)
.unwrap_or(0);
let mut errors = ErrorTracker::default();
// Track whether the previous call produced tool calls — used to
// lower temperature once the agent starts producing a final answer.
let mut saw_tool_calls = false;
for iteration in 0..MAX_TURN_ITERATIONS {
// ── Check abort flag ────────────────────────────────────────
if params.abort.load(Ordering::SeqCst) {
@@ -254,15 +326,40 @@ impl<P: ProviderService, T: ToolExecutor> super::AgentTurnService for AgentTurnS
break;
}
if errors.should_stop() {
push_event(
&params.turn_events,
TurnEvent::SystemNote {
kind: "warn".into(),
message: "Stopping: repeated tool errors without progress".into(),
},
);
break;
}
debug!("agent turn iteration {iteration}");
// ── Auto-compact oversized history before the LLM call ─────
self.auto_compact_if_needed(&mut params.messages).await;
// ── Adaptive generation parameters ─────────────────────────
let max_tokens = adaptive_max_tokens(request_len);
// Lower temperature while the agent is still choosing tools to
// keep tool selection deterministic; raise it for the final
// free-form answer.
let temperature = if saw_tool_calls { 0.2 } else { 0.7 };
// ── Stream start + call LLM ─────────────────────────────────
push_event(&params.turn_events, TurnEvent::StreamStart);
// Uses params.messages directly (sys_msg[0] already in place
// from the insert above) — no per-iteration clone needed.
let result = self
.call_llm(&params.messages, &params.abort, &params.turn_events)
.call_llm(
&params.messages,
&params.abort,
&params.turn_events,
max_tokens,
temperature,
)
.await;
match result {
@@ -283,6 +380,7 @@ impl<P: ProviderService, T: ToolExecutor> super::AgentTurnService for AgentTurnS
break;
}
saw_tool_calls = true;
params.messages.push(assistant_msg);
// ── Execute each tool call ──────────────────────────
@@ -290,6 +388,9 @@ impl<P: ProviderService, T: ToolExecutor> super::AgentTurnService for AgentTurnS
let output =
execute_tool_call(self.tool_executor.as_ref(), &params.turn_events, tc)
.await;
if output.starts_with("Error:") {
errors.record(&tc.function.name, &output, &mut params.messages);
}
params
.messages
.push(ChatMessage::tool(tc.id.clone(), output));
@@ -370,3 +471,69 @@ pub async fn compact_messages_with_ai<P: ProviderService>(
}
}
}
// ---------------------------------------------------------------------------
// Tests
// ---------------------------------------------------------------------------
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn truncate_short_output_is_unchanged() {
let out = "short".to_string();
assert_eq!(truncate_tool_output(out.clone()), out);
}
#[test]
fn truncate_long_output_preserves_head_and_marks_cut() {
let long = "x".repeat(TOOL_OUTPUT_MAX_CHARS + 500);
let truncated = truncate_tool_output(long.clone());
assert!(truncated.len() < long.len());
assert!(truncated.contains("...[truncated"));
assert!(truncated.starts_with("xxx"));
}
#[test]
fn adaptive_max_tokens_scales_with_request_len() {
assert_eq!(adaptive_max_tokens(10), 800);
assert_eq!(adaptive_max_tokens(200), 1600);
assert_eq!(adaptive_max_tokens(5000), 4096);
}
#[test]
fn error_tracker_injects_recovery_note_after_repeats() {
let mut tracker = ErrorTracker::default();
let mut messages: Vec<ChatMessage> = Vec::new();
tracker.record("read", "Error: File not found", &mut messages);
tracker.record("read", "Error: File not found", &mut messages);
assert!(!tracker.should_stop());
// Third consecutive failure → recovery note injected.
tracker.record("read", "Error: File not found", &mut messages);
assert!(messages.iter().any(|m| m
.content
.as_deref()
.is_some_and(|c| c.contains("[System note]"))));
}
#[test]
fn error_tracker_stops_after_too_many_errors() {
let mut tracker = ErrorTracker::default();
let mut messages: Vec<ChatMessage> = Vec::new();
for i in 0..MAX_TOTAL_TOOL_ERRORS {
tracker.record("bash", &format!("Error: boom {i}"), &mut messages);
}
assert!(tracker.should_stop());
}
#[test]
fn conversation_chars_sums_content_only() {
let messages = vec![
ChatMessage::system("sys".to_string()),
ChatMessage::user("hello world".to_string()),
ChatMessage::tool("id".to_string(), "output".to_string()),
];
assert_eq!(conversation_chars(&messages), 3 + 11 + 6);
}
}