MythEclipse/asepharyana-hub-llm-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
14 Commits 2 Branches 0 Tags
59c77108a55b86617c0e40a67ff2805e71e343c9
Commit Graph
10 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Asep Haryana 59c77108a5 fix: skip leading EOS tokens in streaming + non-streaming 
- Skip <|im_end|> generated as first token (prevents empty responses)
- Clean <think> tags as plain text in generated output
 2026-07-26 16:09:04 +07:00
Asep Haryana 495b9ed126 fix: remove /think trigger, add back <think> in clean_text 
- Removed /think trigger from prompt (model generates better without it)
- Added <think> and </think> back to clean_text (model uses plain text
  thinking tags, not special tokens)
 2026-07-26 16:03:18 +07:00
Asep Haryana 254532458b Revert "fix: remove /think trigger from prompt, use plain assistant prefix" 
This reverts commit 7c8f747faf.
 2026-07-26 15:49:04 +07:00
Asep Haryana 7c8f747faf fix: remove /think trigger from prompt, use plain assistant prefix 2026-07-26 15:48:44 +07:00
Asep Haryana d8425ea3b3 feat: switch to MiniCPM5-1B-Claude-Opus-Fable5-V2-Thinking-Q8_0 GGUF 
- Updated model path + model ID for new 1B thinking model
- Updated prompt builder to use MiniCPM5 native /think trigger
- Updated clean_text to strip MiniCPM5 special tokens
- Bumped n_ctx from 2048 to 8192
 2026-07-26 15:38:05 +07:00
asepharyanaandGitHub 7e717cc808 Merge pull request #1 from asepharyana/refactor/clean-architecture 
refactor(llm-api): implement clean architecture following scraper pat…
 2026-07-25 15:09:04 +07:00
Asep Haryana c77ceee83f feat: OpenAI-compatible tool calling + sampling params 
- Tool calling: tools array, tool_choice, tool call parsing, round-trip
- Sampling: temperature, top_p, top_k, min_p, seed
- Penalties: repeat_penalty, frequency_penalty, presence_penalty
- Stop sequences support
- Chat history with assistant tool_calls and tool responses
- GenerateStream with SendSampler wrapper for thread-safe async
 2026-07-25 11:42:57 +07:00
Asep Haryana e1f5195407 feat: add SSE streaming support (OpenAI-compatible) 
- Token-by-token streaming via Server-Sent Events
- tokio::sync::mpsc channel + ReceiverStream for clean async
- OpenAI SSE format: role chunk → content chunks → finish chunk
- Non-streaming still works with the same code path
- Uses spawn_blocking pattern for CPU-bound inference
 2026-07-25 11:33:59 +07:00
Asep Haryana 6edf6cee5d fix: use token_to_piece_bytes for proper token decoding 2026-07-25 11:17:58 +07:00
Asep Haryana 33b42b789c feat: initial release - OpenAI-compatible LLM server with llama-cpp-2 2026-07-25 11:00:03 +07:00