8 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Asep Haryana	254532458b	Revert "fix: remove /think trigger from prompt, use plain assistant prefix" ... This reverts commit 7c8f747faf.	2026-07-26 15:49:04 +07:00
Asep Haryana	7c8f747faf	fix: remove /think trigger from prompt, use plain assistant prefix	2026-07-26 15:48:44 +07:00
Asep Haryana	d8425ea3b3	feat: switch to MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-Q8_0 GGUF ... - Updated model path + model ID for new 1B thinking model - Updated prompt builder to use MiniCPM5 native /think trigger - Updated clean_text to strip MiniCPM5 special tokens - Bumped n_ctx from 2048 to 8192	2026-07-26 15:38:05 +07:00
asepharyanaandGitHub	7e717cc808	Merge pull request #1 from asepharyana/refactor/clean-architecture ... refactor(llm-api): implement clean architecture following scraper pat…	2026-07-25 15:09:04 +07:00
Asep Haryana	c77ceee83f	feat: OpenAI-compatible tool calling + sampling params ... - Tool calling: tools array, tool_choice, tool call parsing, round-trip - Sampling: temperature, top_p, top_k, min_p, seed - Penalties: repeat_penalty, frequency_penalty, presence_penalty - Stop sequences support - Chat history with assistant tool_calls and tool responses - GenerateStream with SendSampler wrapper for thread-safe async	2026-07-25 11:42:57 +07:00
Asep Haryana	e1f5195407	feat: add SSE streaming support (OpenAI-compatible) ... - Token-by-token streaming via Server-Sent Events - tokio::sync::mpsc channel + ReceiverStream for clean async - OpenAI SSE format: role chunk → content chunks → finish chunk - Non-streaming still works with the same code path - Uses spawn_blocking pattern for CPU-bound inference	2026-07-25 11:33:59 +07:00
Asep Haryana	6edf6cee5d	fix: use token_to_piece_bytes for proper token decoding	2026-07-25 11:17:58 +07:00
Asep Haryana	33b42b789c	feat: initial release - OpenAI-compatible LLM server with llama-cpp-2	2026-07-25 11:00:03 +07:00