MythEclipse/asepharyana-hub-llm-api
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
5 Commits 2 Branches 0 Tags
dfd6fa66a7e0a4b08d2a3c7d69cc9c4407e8eb0a
Commit Graph
2 Commits
Author SHA1 Message Date
Asep Haryana e1f5195407 feat: add SSE streaming support (OpenAI-compatible) 
- Token-by-token streaming via Server-Sent Events
- tokio::sync::mpsc channel + ReceiverStream for clean async
- OpenAI SSE format: role chunk → content chunks → finish chunk
- Non-streaming still works with the same code path
- Uses spawn_blocking pattern for CPU-bound inference
 2026-07-25 11:33:59 +07:00
Asep Haryana 33b42b789c feat: initial release - OpenAI-compatible LLM server with llama-cpp-2 2026-07-25 11:00:03 +07:00