5 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
asepharyana	dfd6fa66a7	chore: remove infra files managed by hub monorepo ... Dockerfile, docker-compose.yml, llm-api.yml, and traefik-router.yaml are now managed centrally in asepharyana-hub's infra/ directory.	2026-07-25 12:58:12 +07:00
Asep Haryana	c77ceee83f	feat: OpenAI-compatible tool calling + sampling params ... - Tool calling: tools array, tool_choice, tool call parsing, round-trip - Sampling: temperature, top_p, top_k, min_p, seed - Penalties: repeat_penalty, frequency_penalty, presence_penalty - Stop sequences support - Chat history with assistant tool_calls and tool responses - GenerateStream with SendSampler wrapper for thread-safe async	2026-07-25 11:42:57 +07:00
Asep Haryana	e1f5195407	feat: add SSE streaming support (OpenAI-compatible) ... - Token-by-token streaming via Server-Sent Events - tokio::sync::mpsc channel + ReceiverStream for clean async - OpenAI SSE format: role chunk → content chunks → finish chunk - Non-streaming still works with the same code path - Uses spawn_blocking pattern for CPU-bound inference	2026-07-25 11:33:59 +07:00
Asep Haryana	6edf6cee5d	fix: use token_to_piece_bytes for proper token decoding	2026-07-25 11:17:58 +07:00
Asep Haryana	33b42b789c	feat: initial release - OpenAI-compatible LLM server with llama-cpp-2	2026-07-25 11:00:03 +07:00