Files
brain-of-reese/.agent/reports/09_story_retrieval_quality.a1.md
T
ducoterra 4971e2859d
Build and Push Containers / build-and-push-app (push) Successful in 12s
Build and Push Containers / build-and-push-db (push) Successful in 10s
chore(agent): track .agent/ planning tree in git
Remove the blanket .agent/ gitignore so the phase roadmap, user
stories, reports, and PLAN.md are versioned with the code. Only
runtime artifacts (.agent/phase-sessions/, .agent/pipeline.log)
remain ignored. Update AGENTS.md git protocol rule to match.
2026-09-01 10:18:22 -04:00

1.8 KiB
Raw Blame History

Phase 09 is complete. Final report:

Phase 09 report — commit 7e8d147 (36 files, +2021/−293)

  • A7 hybrid retrieval: cosine top-100 ∪ Postgres FTS top-30 (stored tsvector + GIN, OR-tsquery), RRF k=60, LOCKED tiebreak. Fusion iteration = vector window 30→100: measured gitlab.md:1 double-hit 0.02212 > template 0.01639 → "How did I install gitlab?" now ranks gitlab.md #1 ✓
  • A8 gate: LOW only when best cosine < 0.62 and fts_hits == 0; fts_hits in per-turn log + query_log (migration 0002, nullable)
  • A9 scope: md,markdown,txt,yaml,yml,json,py (stdlib-only chunkers: YAML ---/top-level keys, JSON top-level keys, ast defs/classes), dot-prefixed paths skipped, --prune drops filter-outs, per-format summary counts
  • Live KB re-import: 520 docs / 2774 chunks / 0 errors in 72s; zero dot-prefixed paths; k8s/backup/dns questions all ground correctly
  • Gates: 174 unit+integration passed · coverage 98% (>90%) · ruff + pyright clean · all 9 E2E suites green in isolation, incl. new test_retrieval_quality.py (4 tests: multi-format import, gitlab grounding, FTS-beats-vector, sourdough deflection)

Deviations (flagged per protocol, not silent):

  • "sourdough → LOW" is unmet at the locked 0.62: measured off-topic best-cosine floor with the real embed model is 0.73 (battery: off 0.61–0.73, on 0.65–0.84 — the phase's 0.41–0.58 assumption was too low), so the gate answers. Fix when you want it: BOR_RELEVANCE_THRESHOLD≈0.77 (then salsa/travel/sourdough deflect; common-word off-topic like tire/pizza still answers via the owner-approved FTS-OR).
  • gitlab-compose.yaml lands #3, not top-2 (sits at vector rank 483; no sane window lets it outrank the template's double-free 1/61 term).

Next pending: 10_story_document_viewer.md.