asepharyanaandClaude Opus 5
b636496497
- Unified synchronous generate() core with callback; both streaming and non-streaming paths run it via spawn_blocking (context: std Mutex). - build_prompt now passes tool definitions to the template (was dead) and embeds assistant tool-call history as XML matching the parser format; fixes double <tool_response> wrap and template set-scoping bug. - Tokenize with AddBos::Never (template owns <s>) to remove double BOS. - Streaming: preserve inter-word spaces (per-chunk trim removed), add [DONE] + usage chunk, emit error events, single-shot tool_calls delta. - Strict model validation (400 on unknown model); health/UI/README aligned to minicpm5-1b-fable5-v2-thinking; auth returns JSON errors; n_ctx/ n_batch/n_threads env-configurable. - Added 18 unit tests; cargo check/clippy/fmt clean. - scripts/smoke-test.sh for post-deploy verification on the VPS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
45 lines
1.9 KiB
Django/Jinja
45 lines
1.9 KiB
Django/Jinja
{{- bos_token }}
|
|
{%- if tools %}
|
|
{%- set tool_definitions %}
|
|
{{- "# Tools\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
|
|
{%- for tool in tools %}
|
|
{{- "\n" }}
|
|
{{- tool | tojson }}
|
|
{%- endfor %}
|
|
{{- '\n</tools>\n\nTool usage guidelines:\n- You may call zero or more functions. If no function calls are needed, just answer normally.\n- When calling a function, wrap each call in <tool_call> tags:\n<tool_call>\n<function=name>\n<parameter=key>value</parameter>\n</function>\n</tool_call>' }}
|
|
{%- endset %}
|
|
|
|
{{- '<|im_start|>system\n' }}
|
|
{%- if messages[0].role == 'system' %}
|
|
{{- messages[0].content + '\n\n' + tool_definitions }}
|
|
{%- else %}
|
|
{{- tool_definitions }}
|
|
{%- endif %}
|
|
{{- '<|im_end|>\n' }}
|
|
{%- else %}
|
|
{%- if messages[0].role == 'system' %}
|
|
{{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
|
|
{%- endif %}
|
|
{%- endif %}
|
|
{%- for message in messages %}
|
|
{%- set content = message.content %}
|
|
{%- if (message.role == "user") or (message.role == "system" and not loop.first) %}
|
|
{{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}
|
|
{%- elif message.role == "assistant" %}
|
|
{#- Tool-call XML from history is pre-embedded in content by build_prompt. #}
|
|
{{- '<|im_start|>assistant\n' + content + '<|im_end|>\n' }}
|
|
{%- elif message.role == "tool" %}
|
|
{{- '<|im_start|>user\n<tool_response>\n' + content + '\n</tool_response><|im_end|>\n' }}
|
|
{%- endif %}
|
|
{%- endfor %}
|
|
{%- if add_generation_prompt %}
|
|
{{- '<|im_start|>assistant\n' }}
|
|
{%- if enable_thinking is defined %}
|
|
{%- if enable_thinking is false %}
|
|
{{- '<think>\n\n</think>\n\n' }}
|
|
{%- elif enable_thinking is true %}
|
|
{{- '<think>\n' }}
|
|
{%- endif %}
|
|
{%- endif %}
|
|
{%- endif %}
|