The incremental </think> search missed tags split across tokens, inverting
reasoning/content classification. Detect the first </think> on the full
buffer and track the content boundary as a byte offset. If the model never
closes </think>, flush the buffered text as content so clients always
receive the response. Chat UI now renders reasoning_content too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
- Unified synchronous generate() core with callback; both streaming and
non-streaming paths run it via spawn_blocking (context: std Mutex).
- build_prompt now passes tool definitions to the template (was dead) and
embeds assistant tool-call history as XML matching the parser format;
fixes double <tool_response> wrap and template set-scoping bug.
- Tokenize with AddBos::Never (template owns <s>) to remove double BOS.
- Streaming: preserve inter-word spaces (per-chunk trim removed), add
[DONE] + usage chunk, emit error events, single-shot tool_calls delta.
- Strict model validation (400 on unknown model); health/UI/README aligned
to minicpm5-1b-fable5-v2-thinking; auth returns JSON errors; n_ctx/
n_batch/n_threads env-configurable.
- Added 18 unit tests; cargo check/clippy/fmt clean.
- scripts/smoke-test.sh for post-deploy verification on the VPS.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Notify Parent Repo / dispatch (push) Canceled after 0s
- Add X-Accel-Buffering: no, Cache-Control: no-cache, Connection: keep-alive
- Prevents Traefik/proxy from buffering SSE events
- Each token is flushed immediately as generated
- Before </think>: tokens sent as delta.reasoning_content
- After </think>: tokens sent as delta.content
- Handles </think> split across token boundaries
- Clean <|im_start|>/<|im_end|>/<think>/</think> from streamed text
- Normal models (no thinking) send all as content as before
- Replaced manual prompt building with GGUF's chat template rendered through minijinja (Rust Jinja2 engine)
- Simplified template: removed multi-step tool detection, reasoning extraction (not needed at template level)
- Added reasoning_content separation: clean_text() returns (reasoning, answer) tuple
- Added reasoning_content field to ResponseMessage and SseDelta for OpenAI-compatible output
- Embedded template at build time via include_str! from templates/chat_template.jinja
- Built-in support for enable_thinking, tool_definitions, tool_calls, tool_response
- Removed /think trigger from prompt (model generates better without it)
- Added <think> and </think> back to clean_text (model uses plain text
thinking tags, not special tokens)
- Updated model path + model ID for new 1B thinking model
- Updated prompt builder to use MiniCPM5 native /think trigger
- Updated clean_text to strip MiniCPM5 special tokens
- Bumped n_ctx from 2048 to 8192
- Token-by-token streaming via Server-Sent Events
- tokio::sync::mpsc channel + ReceiverStream for clean async
- OpenAI SSE format: role chunk → content chunks → finish chunk
- Non-streaming still works with the same code path
- Uses spawn_blocking pattern for CPU-bound inference
