ducoterra
4971e2859d
Remove the blanket .agent/ gitignore so the phase roadmap, user stories, reports, and PLAN.md are versioned with the code. Only runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log) remain ignored. Update AGENTS.md git protocol rule to match.
2.2 KiB
2.2 KiB
Phase 11 — Long Answers (No Truncation)
Story: .agent/user_stories/long-answers.md
Context: owner report 2026-08-22 — "responses keep getting cut off.
It should be allowed to respond up to 32768 tokens."
Goal
Remove the hard 700-token output cap on chat answers; the model may
respond up to 32 768 tokens (BOR_MAX_OUTPUT_TOKENS, default
32 768).
Diagnosis
app/rag/llm.py::chat_stream calls chat.completions.create(..., max_tokens=700, ...). Long answers die mid-sentence at ~700 tokens.
Implementation steps
- Config (
app/config.py):max_output_tokens: int = 32_768in the RAG-tuning section (envBOR_MAX_OUTPUT_TOKENS); document in.env.example. - LLM client (
app/rag/llm.py):chat_streampassesmax_tokens=self.settings.max_output_tokens. - E2E mock (
tests/e2e/mock_llm.py):- Honor
max_tokensdeterministically: token ≈ whitespace word; if the composed answer is longer, truncate to the first N words. (With the old 700 cap a long answer loses its tail — the mock now behaves like the real endpoint.) - New trigger: user message containing
write a long answer→ deterministic ~4 000-word numbered answer ending in a unique final line (LONG-ANSWER-END). - No behavior change for existing (short) answers: they fit under any sane cap.
- Honor
- Tests:
- Unit: settings default + env override (
test_config.py);chat_streamforwards the configuredmax_tokens(fake client intest_llm_client.py). - E2E:
tests/e2e/test_long_answers.pyper the story mapping.
- Unit: settings default + env override (
Locked decisions
None touched. A5 (aipi endpoint) unchanged; turbo accepts the larger
cap per owner instruction.
Testing & Quality
- Unit + integration green;
uv run pytest --cov=app --cov-report=term-missing>90%; E2E in isolation:uv run pytest tests/e2e/test_long_answers.py -v --no-cov. - No regressions:
test_chat_rag.py+test_chat_api.pygreen (short answers unaffected by the mock's newmax_tokenshonoring).
Commit
git add -A .agent/ app/ tests/ .env.example && git commit --no-gpg-sign -m "fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off"