brain-of-reese/.agent/phases/complete/11_long_answers.md

Phase 11 — Long Answers (No Truncation)

Story: .agent/user_stories/long-answers.md Context: owner report 2026-08-22 — "responses keep getting cut off. It should be allowed to respond up to 32768 tokens."

Goal

Remove the hard 700-token output cap on chat answers; the model may respond up to 32 768 tokens (BOR_MAX_OUTPUT_TOKENS, default 32 768).

Diagnosis

app/rag/llm.py::chat_stream calls chat.completions.create(..., max_tokens=700, ...). Long answers die mid-sentence at ~700 tokens.

Implementation steps

1. Config (app/config.py): max_output_tokens: int = 32_768 in the RAG-tuning section (env BOR_MAX_OUTPUT_TOKENS); document in .env.example.
2. LLM client (app/rag/llm.py): chat_stream passes max_tokens=self.settings.max_output_tokens.
3. E2E mock (tests/e2e/mock_llm.py):
   • Honor max_tokens deterministically: token ≈ whitespace word; if the composed answer is longer, truncate to the first N words. (With the old 700 cap a long answer loses its tail — the mock now behaves like the real endpoint.)
   • New trigger: user message containing write a long answer → deterministic ~4 000-word numbered answer ending in a unique final line (LONG-ANSWER-END).
   • No behavior change for existing (short) answers: they fit under any sane cap.
4. Tests:
   • Unit: settings default + env override (test_config.py); chat_stream forwards the configured max_tokens (fake client in test_llm_client.py).
   • E2E: tests/e2e/test_long_answers.py per the story mapping.

Locked decisions

None touched. A5 (aipi endpoint) unchanged; turbo accepts the larger cap per owner instruction.

Testing & Quality

• Unit + integration green; uv run pytest --cov=app --cov-report=term-missing >90%; E2E in isolation: uv run pytest tests/e2e/test_long_answers.py -v --no-cov.
• No regressions: test_chat_rag.py + test_chat_api.py green (short answers unaffected by the mock's new max_tokens honoring).

Commit

git add -A .agent/ app/ tests/ .env.example && git commit --no-gpg-sign -m "fix(rag): lift chat output cap to 32768 tokens — long answers no longer cut off"