6 Commits

Include renames

Author	SHA1	Message	Date
Asep Haryana	67861f384b	fix(stream): add anti-buffer headers for real SSE streaming ... - Add X-Accel-Buffering: no, Cache-Control: no-cache, Connection: keep-alive - Prevents Traefik/proxy from buffering SSE events - Each token is flushed immediately as generated	2026-07-26 19:13:00 +07:00
Asep Haryana	5cba76d280	fix(stream): separate reasoning_content from content in SSE streaming ... - Before </think>: tokens sent as delta.reasoning_content - After </think>: tokens sent as delta.content - Handles </think> split across token boundaries - Clean <|im_start|>/<|im_end|>/<think>/</think> from streamed text - Normal models (no thinking) send all as content as before	2026-07-26 19:04:29 +07:00
Asep Haryana	6ff31b5f62	feat: render GGUF Jinja template via minijinja crate ... - Replaced manual prompt building with GGUF's chat template rendered through minijinja (Rust Jinja2 engine) - Simplified template: removed multi-step tool detection, reasoning extraction (not needed at template level) - Added reasoning_content separation: clean_text() returns (reasoning, answer) tuple - Added reasoning_content field to ResponseMessage and SseDelta for OpenAI-compatible output - Embedded template at build time via include_str! from templates/chat_template.jinja - Built-in support for enable_thinking, tool_definitions, tool_calls, tool_response	2026-07-26 18:21:35 +07:00
Asep Haryana	9a63ff1601	feat: use GGUF chat template via apply_chat_template ... - Replaced manual prompt building with LlamaModel::apply_chat_template - Uses model's baked-in Jinja template (system/user/assistant/tool format) - Added <think> trigger after template for thinking mode - Clean_text strips only <|im_end|>, <|im_start|>, <think>, </think>	2026-07-26 17:10:42 +07:00
Asep Haryana	59c77108a5	fix: skip leading EOS tokens in streaming + non-streaming ... - Skip <|im_end|> generated as first token (prevents empty responses) - Clean <think> tags as plain text in generated output	2026-07-26 16:09:04 +07:00
asepharyanaandClaude Code	e351d74fa4	refactor(llm-api): implement clean architecture following scraper pattern ... Split monolithic 1012-line main.rs into layered hexagonal architecture: - Domain: entity types and LlmError enum - Application: prompt building, sampler construction, tool call parsing - Infrastructure: LlamaEngine wrapping llama-cpp-2 with isolated unsafe transmute - Presentation: Axum handlers, middleware (auth), error chain, router - Config: type-safe AppConfig with LazyLock - Bootstrap: Application struct with build() + run() Resolves build_sampler/build_sampler_params duplication. Adds simple web chat UI at GET /. Co-Authored-By: Claude Code <noreply@anthropic.com>	2026-07-25 15:07:19 +07:00