feat(rag): lite-model document summaries — non-markdown docs summarized at import, summary chunk retrieves and resolves to the full source doc

This commit is contained in:
2026-08-25 17:48:37 -04:00
parent 9809482a4b
commit 572a4190a6
32 changed files with 1806 additions and 26 deletions
+16
View File
@@ -0,0 +1,16 @@
# Qwen 3.8 on llama.cpp — deployment notes
## Optimal parameters
The optimal parameters for qwen 3.8 on llama.cpp came out of a week of
benchmarks on the homelab GPU. Context length, flash attention and the
batch size matter more than the mmap knob. The full flag set lives in
`quadlet/qwen-llamacpp.yaml` — the yaml is the source of truth, these
notes are the reasoning behind each picked value.
## What changed since last month
Switched the server image to the 0.1.43 release and moved the model
files to the NVMe cache drive. Pinned the quant to q4_k_m; the repeat
penalty is the knob that kept the rambles honest. The webui now fronts
the raw server so chat sessions survive a container restart.