Asep Haryana
254532458b
Revert "fix: remove /think trigger from prompt, use plain assistant prefix"
...
This reverts commit 7c8f747faf
2026-07-26 15:49:04 +07:00
Asep Haryana
7c8f747faf
fix: remove /think trigger from prompt, use plain assistant prefix
2026-07-26 15:48:44 +07:00
Asep Haryana
d8425ea3b3
feat: switch to MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-Q8_0 GGUF
...
- Updated model path + model ID for new 1B thinking model
- Updated prompt builder to use MiniCPM5 native /think trigger
- Updated clean_text to strip MiniCPM5 special tokens
- Bumped n_ctx from 2048 to 8192
2026-07-26 15:38:05 +07:00
asepharyana
14032db705
ci: add workflow_dispatch to notify-parent for manual triggering
2026-07-25 18:13:35 +07:00
asepharyana and Claude Code
2a5ab8b6d5
ci(llm-api): add notify-parent workflow
...
Trigger repository_dispatch to parent hub repo on push to master.
Co-Authored-By: Claude Code <noreply@anthropic.com >
2026-07-25 15:11:38 +07:00
asepharyana and GitHub
7e717cc808
Merge pull request #1 from asepharyana/refactor/clean-architecture
...
refactor(llm-api): implement clean architecture following scraper pat…
2026-07-25 15:09:04 +07:00
asepharyana and Claude Code
e351d74fa4
refactor(llm-api): implement clean architecture following scraper pattern
...
Split monolithic 1012-line main.rs into layered hexagonal architecture:
- Domain: entity types and LlmError enum
- Application: prompt building, sampler construction, tool call parsing
- Infrastructure: LlamaEngine wrapping llama-cpp-2 with isolated unsafe transmute
- Presentation: Axum handlers, middleware (auth), error chain, router
- Config: type-safe AppConfig with LazyLock
- Bootstrap: Application struct with build() + run()
Resolves build_sampler/build_sampler_params duplication.
Adds simple web chat UI at GET /.
Co-Authored-By: Claude Code <noreply@anthropic.com >
2026-07-25 15:07:19 +07:00
asepharyana
dfd6fa66a7
chore: remove infra files managed by hub monorepo
...
Dockerfile, docker-compose.yml, llm-api.yml, and traefik-router.yaml
are now managed centrally in asepharyana-hub's infra/ directory.
2026-07-25 12:58:12 +07:00
Asep Haryana
c77ceee83f
feat: OpenAI-compatible tool calling + sampling params
...
- Tool calling: tools array, tool_choice, tool call parsing, round-trip
- Sampling: temperature, top_p, top_k, min_p, seed
- Penalties: repeat_penalty, frequency_penalty, presence_penalty
- Stop sequences support
- Chat history with assistant tool_calls and tool responses
- GenerateStream with SendSampler wrapper for thread-safe async
2026-07-25 11:42:57 +07:00
Asep Haryana
e1f5195407
feat: add SSE streaming support (OpenAI-compatible)
...
- Token-by-token streaming via Server-Sent Events
- tokio::sync::mpsc channel + ReceiverStream for clean async
- OpenAI SSE format: role chunk → content chunks → finish chunk
- Non-streaming still works with the same code path
- Uses spawn_blocking pattern for CPU-bound inference
2026-07-25 11:33:59 +07:00
Asep Haryana
6edf6cee5d
fix: use token_to_piece_bytes for proper token decoding
2026-07-25 11:17:58 +07:00
Asep Haryana
33b42b789c
feat: initial release - OpenAI-compatible LLM server with llama-cpp-2
2026-07-25 11:00:03 +07:00
Asep Haryana