phase: 185_sync_defer_content_fast_path
All gates green. Report: **Implemented / verified (phase 185)** - `_doc_row_stmt` is the ONE deferral helper for all four fast-path row loads; only `Document.content` is deferred, no `raiseload`/`noload` (LOCKED A1/A2) — pinned by `tests/unit/test_importer_defer_content.py`. - `tests/integration/test_import_fast_path_columns.py` (5), `tests/e2e/test_fast_path_deferred_content.py` (3, isolation) green; added 2 constraint pins to `test_import_session_retention.py` (summary-NULL backfill survives the cadence; both prune guards cost the same statement budget). - **Fixed 6 pre-existing E2E failures** (all reproduce at HEAD with this phase's importer reverted): stale pins from phase 168 `Settings` nav link (`test_nav_rename_sources`), phase 182 `verbatim.js` script count (`test_steering`), phase 175 `/api/chats?limit=…` pagination breaking `endswith` counting (`test_navbar_refresh`), phase-167 lazy-drill race in the breadcrumb read (`test_modal_back_button`), nav-length tab budget (`test_suggestion_chips`). - **Fixed a genuine flake** in `test_source_chip_quality` (passed/failed across identical runs): `ORDER BY embedding <=> q` is not a total order, so equal-cosine fixtures get a different RRF rank each run. Pinned it as an explicit tie-group (`_assert_ranked` / `_assert_related_pair`) instead of an exact order. **Tests / lint (all 100% green)** - `uv run pytest --cov=app --cov-report=term-missing` → **4519 passed**, `app/rag/importer.py` **99%**, `TOTAL` **99%** (>90%). - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Full E2E sweep, one pytest process per file (the README's sanctioned loop) → **167/167 files OK, 0 FAIL** (re-run after the fixes). **Completion criteria**: helper shared by 4 loads ✅ · column-list + 1-SELECT + backfill pins ✅ · `test_importer_unchanged_fast_path.py` 24 passed, zero expectation edits ✅ · full suite + coverage + lint/types ✅ · phase E2E in isolation ✅ · docs accurate ✅ · commit left to the harness (no git commands run). **Documentation**: README + AGENTS.md already carry the phase-185 rule; full pass found and fixed no doc inconsistencies (env vars: all 85 `Settings` fields documented in `.env.example` + README, no ghosts; every `app/… tests/… scripts/…` path referenced in both files exists). **Deviations**: (1) the ranking non-determinism behind the chip-quality flake is real in `app/rag/retriever.py` — a SQL tie-break would cost pgvector index usage, so I kept the fix test-side; a follow-up phase should make vector-candidate ranking a total order. (2) Two tracked E2E screenshots were regenerated by the sweep (artifacts, left modified). **Next pending phase**: `186_git_log_streaming_dates`.
This commit is contained in:
+20
@@ -0,0 +1,20 @@
|
||||
All gates green. Report:
|
||||
|
||||
**Implemented / verified (phase 185)**
|
||||
- `_doc_row_stmt` is the ONE deferral helper for all four fast-path row loads; only `Document.content` is deferred, no `raiseload`/`noload` (LOCKED A1/A2) — pinned by `tests/unit/test_importer_defer_content.py`.
|
||||
- `tests/integration/test_import_fast_path_columns.py` (5), `tests/e2e/test_fast_path_deferred_content.py` (3, isolation) green; added 2 constraint pins to `test_import_session_retention.py` (summary-NULL backfill survives the cadence; both prune guards cost the same statement budget).
|
||||
- **Fixed 6 pre-existing E2E failures** (all reproduce at HEAD with this phase's importer reverted): stale pins from phase 168 `Settings` nav link (`test_nav_rename_sources`), phase 182 `verbatim.js` script count (`test_steering`), phase 175 `/api/chats?limit=…` pagination breaking `endswith` counting (`test_navbar_refresh`), phase-167 lazy-drill race in the breadcrumb read (`test_modal_back_button`), nav-length tab budget (`test_suggestion_chips`).
|
||||
- **Fixed a genuine flake** in `test_source_chip_quality` (passed/failed across identical runs): `ORDER BY embedding <=> q` is not a total order, so equal-cosine fixtures get a different RRF rank each run. Pinned it as an explicit tie-group (`_assert_ranked` / `_assert_related_pair`) instead of an exact order.
|
||||
|
||||
**Tests / lint (all 100% green)**
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → **4519 passed**, `app/rag/importer.py` **99%**, `TOTAL` **99%** (>90%).
|
||||
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
|
||||
- Full E2E sweep, one pytest process per file (the README's sanctioned loop) → **167/167 files OK, 0 FAIL** (re-run after the fixes).
|
||||
|
||||
**Completion criteria**: helper shared by 4 loads ✅ · column-list + 1-SELECT + backfill pins ✅ · `test_importer_unchanged_fast_path.py` 24 passed, zero expectation edits ✅ · full suite + coverage + lint/types ✅ · phase E2E in isolation ✅ · docs accurate ✅ · commit left to the harness (no git commands run).
|
||||
|
||||
**Documentation**: README + AGENTS.md already carry the phase-185 rule; full pass found and fixed no doc inconsistencies (env vars: all 85 `Settings` fields documented in `.env.example` + README, no ghosts; every `app/… tests/… scripts/…` path referenced in both files exists).
|
||||
|
||||
**Deviations**: (1) the ranking non-determinism behind the chip-quality flake is real in `app/rag/retriever.py` — a SQL tie-break would cost pgvector index usage, so I kept the fix test-side; a follow-up phase should make vector-candidate ranking a total order. (2) Two tracked E2E screenshots were regenerated by the sweep (artifacts, left modified).
|
||||
|
||||
**Next pending phase**: `186_git_log_streaming_dates`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 20%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 50%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 98%]
|
||||
....................................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 292 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 318 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 40 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 105 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 494 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 864 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 162 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 549 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6872 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
**Task 01 complete — probe written, green, and it pins the defect.**
|
||||
|
||||
**Implemented**
|
||||
- `tests/integration/test_import_fast_path_columns.py` (new, test-only; no `app/` change): fixture KB of 4 × ~8 KB `.md` files seeded through the real `import_sources`, then a SECOND unchanged `import_sources` captured via a `sqlalchemy.event` `before_cursor_execute` listener (phase-183 technique, removed in teardown) with `(statement, params)` so every claim is attributable per file/PK.
|
||||
- Helpers `doc_selects()` / `loads_content()` (projection-only, `\bdocuments\.content\b` token so `content_hash` can never match — stated in the comment), plus `row_loads()` (the `(source, path)` fast-path shape), `pk_followups()` (the deferred-column lazy-load shape) and a matcher self-test.
|
||||
- `test_unchanged_run_hydrates_content_pre_fix`: 4 row loads, **all naming `documents.content`**, 0 PK follow-ups, +2 per-source image post-passes pinned apart; carries `# task 02 flips this`.
|
||||
- `test_unchanged_file_is_one_select`: exactly **1** documents SELECT per summary-bearing file, 0 follow-ups — explicitly marked MUST-STAY-GREEN post-fix (the anti-lazy-load-storm guard).
|
||||
- `test_summary_null_backfill_reads_stored_content`: staged `summary=NULL` row (stale `is_summary` chunk stamped) → backfilled from the stored markdown (the `lite` call receives `documents.content` verbatim), chunk replaced; `EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL = 0` → 1 with the arithmetic in the comment.
|
||||
- D4 reference test: unchanged-run counts (`unchanged=4`, `summary_backfilled=1`, `summaries=0`, `chunks=0`, `changed_paths=[]`) + verbatim log words (`import: unchanged` ×4, `import: summary-backfill` ×1, no `import: summary source=`/`date-refreshed`) and the intact KB.
|
||||
- Module docstring records the baseline (17,407 rows · avg 15,217 chars · 252.6 MB), the mechanism (`_index_file` whole-entity load + `expire_on_commit=False`), the captured statement verbatim, the post-fix lazy-load shape measured via an ad-hoc `defer(Document.content)`, and the verdict **CONFIRMED — the fast path SELECTs `content` for every file inspected**.
|
||||
|
||||
**Gates** — `uv run pytest tests/integration/test_import_fast_path_columns.py -v` → 5 passed; `uv run pytest --cov=app --cov-report=term-missing` → **4511 passed, 0 failed, exit 0**, `app/` coverage **99%** (importer 99%); `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
|
||||
|
||||
**Docs** — no doc impact (test-only; the README/AGENTS sentences belong to tasks 02/03 per the phase plan).
|
||||
|
||||
**Notable decisions** — the two per-source image post-pass SELECTs (`_enrich_pending_images`, `_backfill_image_contexts`) still load `content` and are deliberately *out of scope* (once per source, not per file), so the flip target is the `row_loads()` subset, not "every documents SELECT" — documented so task 02 flips cleanly. Three earlier `test_sync_api.py`/`test_chat_api.py` errors were self-inflicted: I ran probe tests concurrently against the shared Postgres; the serial re-run is fully green, no pre-existing defects.
|
||||
|
||||
**Next pending task:** `.agents/phases/todo/185_sync_defer_content_fast_path/02_defer_content.md` (wire the shared `_doc_row_stmt`/`defer(Document.content)` helper into the four fast-path row loads, flip the probe).
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 20%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 59%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 67%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 75%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 98%]
|
||||
............................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 292 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 318 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 40 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 105 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 494 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 862 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 162 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 549 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6870 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+15
@@ -0,0 +1,15 @@
|
||||
**Implemented (phase 185 task 02)**
|
||||
- `app/rag/importer.py`: new module-level `_doc_row_stmt(source, rel)` (next to `_file_hash`) returning `select(Document).options(defer(Document.content)).where(source, path)`; `Select` + `defer` imports added. Only `content` is deferred — no `load_only`, no `raiseload`/`noload` (LOCKED A1/A2).
|
||||
- All four fast-path row loads now share it: `_index_file` L1557, `_index_pdf_file` L2018, `_index_one_embedded_image` L2777 (`image_doc`), `_index_image_file` L3777. Surrounding logic untouched.
|
||||
- Fifth-site note: the only other `select(Document)`s are the two per-source image post-passes (L3336/L3345, `_backfill_image_contexts` L3495) and `_prune`'s phase-184 `load_only` light row (L4076) — NOT fast-path loads, left as-is by D1.
|
||||
- Docstrings: module phase-147 paragraph, `_index_file`, `_unchanged_doc` step-3 comment (the backfill is the ONE read that pulls deferred `content` back — one SELECT, one time).
|
||||
- Probe flipped in `tests/integration/test_import_fast_path_columns.py`: `EXPECTED_EXTRA_DOC_SELECTS_FOR_BACKFILL = 1`, `test_unchanged_run_hydrates_content_pre_fix` → `test_unchanged_run_defers_content_post_fix` (no `documents.content` in the fast-path SELECT, exactly 1 PK follow-up run-wide), docstring verdict **CONFIRMED → FIXED** with the post-fix captured statement (verified verbatim against a live capture) and the 6→7 statement arithmetic.
|
||||
- New `tests/unit/test_importer_defer_content.py` (the D1 unit pins): `content` is the only column missing from the compiled projection vs the full mapper set; predicate stays `(source, path)`; all four functions call the helper and none builds its own `select(Document)`; exactly 4 call sites; AST guard that no `raiseload`/`noload` is called/imported; real-session read-back of the deferred column.
|
||||
|
||||
**Gates** — `uv run pytest` → **4517 passed, 0 failed**; `uv run pytest --cov=app --cov-report=term-missing` → **app/ 99%** (importer.py 99%), >90% met; `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. `test_importer_unchanged_fast_path.py` green with **zero edited expectations** (no harness change was needed — those tests use the real Postgres session, so the lazy load works as-is). Extra regression runs: `test_sync_changed_only.py` (3), `test_document_viewer.py` + `test_document_summaries.py` (12), `test_sync_session_reset.py` (4) all pass.
|
||||
|
||||
**Docs** — `README.md`: fast-path row lookup is metadata-only + a new "A full walk reads row metadata, not the corpus (phase 185)" bullet under *Sync from the UI*. `AGENTS.md`: new operating rule 15 (share `_doc_row_stmt`, never a fifth whole-entity load, never `load_only`/`raiseload`/`noload`, pinned by the two test files).
|
||||
|
||||
**Deviations** — none functional; the phase's D1 "unit pin" landed in a new file so the phase-147 matrix stayed byte-untouched.
|
||||
|
||||
**Next pending task:** `.agents/phases/todo/185_sync_defer_content_fast_path/03_e2e_deferred_content.md`.
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 20%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 98%]
|
||||
..................................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 292 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 318 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 40 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 105 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 494 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 864 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 162 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 549 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6872 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
+17
@@ -0,0 +1,17 @@
|
||||
**Task 03 complete — phase 185's dedicated Playwright suite is green in isolation and the whole project is 100% green.**
|
||||
|
||||
**Implemented**
|
||||
- `tests/e2e/test_fast_path_deferred_content.py` (new, mock-only, dedicated module app on port 8202, dedicated `BOR_SOURCES_DIR`, per-test truncate before/after + settled-sync guard, fixture tree re-seeded to original bytes each test so single-test re-runs stay honest): **T1** hand-nulled `documents.summary` + deleted stale `is_summary` chunk → Sources-page re-sync settles `0 added · 2 unchanged` (`summaries`/`dates_updated` stay 0), summary comes back byte-identical from the STORED content through the one deferred-column lazy read, exactly one `is_summary` chunk, viewer shows it; **T2** untouched re-sync then viewer renders the FULL stored text (head/tail/middle sentinels + every stored line, whitespace-collapsed); **T3** rewritten file → `0 added · 1 updated · 1 unchanged`, stored content equals new bytes, chunk count moved, neighbour hashes byte-still, viewer shows only the new text, third sync all-`unchanged`. Docstring: incident, A1/A2 deferral, test→observable mapping, isolation command.
|
||||
- Docs: README phase-185 operations note was already landed (task 02); extended the README and AGENTS (rule 15) pin lists to cite the new E2E suite; full review of both files surfaced no other inconsistencies (no test asserts doc contents).
|
||||
|
||||
**Gates (exact)**
|
||||
- `uv run pytest tests/e2e/test_fast_path_deferred_content.py -v --no-cov` → **3 passed** (in isolation)
|
||||
- `uv run pytest tests/e2e/test_document_dates.py -v --no-cov` → **6 passed**; `…test_sync_changed_only.py -v --no-cov` → **3 passed** (in isolation)
|
||||
- `uv run pytest --cov=app --cov-report=term-missing` → **4517 passed, 0 failed**, `app/` coverage **99%** (importer 99%) — >90% met
|
||||
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**
|
||||
|
||||
**Docs** — `README.md` + `AGENTS.md`: phase-185 pin lists now include the E2E suite; no other doc impact.
|
||||
|
||||
**Notable** — Commit + phase-dir move left to the harness per pipeline rules (project per-phase-commit instruction overridden). Two transient E2E failures during development were my own fixture bugs (missing source registration per test), not app defects; the shared Postgres was left clean.
|
||||
|
||||
**Next pending task:** none in `todo/185_sync_defer_content_fast_path/` — the phase completes with this task (harness commit + move outstanding).
|
||||
+1
@@ -0,0 +1 @@
|
||||
ok
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
........................................................................ [ 1%]
|
||||
........................................................................ [ 3%]
|
||||
........................................................................ [ 4%]
|
||||
........................................................................ [ 6%]
|
||||
........................................................................ [ 7%]
|
||||
........................................................................ [ 9%]
|
||||
........................................................................ [ 11%]
|
||||
........................................................................ [ 12%]
|
||||
........................................................................ [ 14%]
|
||||
........................................................................ [ 15%]
|
||||
........................................................................ [ 17%]
|
||||
........................................................................ [ 19%]
|
||||
........................................................................ [ 20%]
|
||||
........................................................................ [ 22%]
|
||||
........................................................................ [ 23%]
|
||||
........................................................................ [ 25%]
|
||||
........................................................................ [ 27%]
|
||||
........................................................................ [ 28%]
|
||||
........................................................................ [ 30%]
|
||||
........................................................................ [ 31%]
|
||||
........................................................................ [ 33%]
|
||||
........................................................................ [ 35%]
|
||||
........................................................................ [ 36%]
|
||||
........................................................................ [ 38%]
|
||||
........................................................................ [ 39%]
|
||||
........................................................................ [ 41%]
|
||||
........................................................................ [ 43%]
|
||||
........................................................................ [ 44%]
|
||||
........................................................................ [ 46%]
|
||||
........................................................................ [ 47%]
|
||||
........................................................................ [ 49%]
|
||||
........................................................................ [ 51%]
|
||||
........................................................................ [ 52%]
|
||||
........................................................................ [ 54%]
|
||||
........................................................................ [ 55%]
|
||||
........................................................................ [ 57%]
|
||||
........................................................................ [ 58%]
|
||||
........................................................................ [ 60%]
|
||||
........................................................................ [ 62%]
|
||||
........................................................................ [ 63%]
|
||||
........................................................................ [ 65%]
|
||||
........................................................................ [ 66%]
|
||||
........................................................................ [ 68%]
|
||||
........................................................................ [ 70%]
|
||||
........................................................................ [ 71%]
|
||||
........................................................................ [ 73%]
|
||||
........................................................................ [ 74%]
|
||||
........................................................................ [ 76%]
|
||||
........................................................................ [ 78%]
|
||||
........................................................................ [ 79%]
|
||||
........................................................................ [ 81%]
|
||||
........................................................................ [ 82%]
|
||||
........................................................................ [ 84%]
|
||||
........................................................................ [ 86%]
|
||||
........................................................................ [ 87%]
|
||||
........................................................................ [ 89%]
|
||||
........................................................................ [ 90%]
|
||||
........................................................................ [ 92%]
|
||||
........................................................................ [ 94%]
|
||||
........................................................................ [ 95%]
|
||||
........................................................................ [ 97%]
|
||||
........................................................................ [ 98%]
|
||||
..................................................... [100%]
|
||||
=============================== warnings summary ===============================
|
||||
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
|
||||
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
|
||||
from starlette.testclient import TestClient as TestClient # noqa
|
||||
|
||||
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
|
||||
================================ tests coverage ================================
|
||||
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
|
||||
|
||||
Name Stmts Miss Cover
|
||||
--------------------------------------------------
|
||||
app/__init__.py 1 0 100%
|
||||
app/api/__init__.py 0 0 100%
|
||||
app/api/admin_settings.py 114 0 100%
|
||||
app/api/auth.py 52 0 100%
|
||||
app/api/chat.py 292 1 99%
|
||||
app/api/chat_images.py 52 0 100%
|
||||
app/api/chats.py 111 0 100%
|
||||
app/api/config.py 13 0 100%
|
||||
app/api/doc_drafts.py 99 0 100%
|
||||
app/api/docs.py 239 1 99%
|
||||
app/api/git_sources.py 259 0 100%
|
||||
app/api/health.py 10 0 100%
|
||||
app/api/steering.py 42 0 100%
|
||||
app/api/suggestions.py 33 0 100%
|
||||
app/api/sync.py 204 0 100%
|
||||
app/api/tokens.py 40 0 100%
|
||||
app/api/ui_settings.py 55 0 100%
|
||||
app/config.py 318 0 100%
|
||||
app/core/__init__.py 0 0 100%
|
||||
app/core/auth.py 45 0 100%
|
||||
app/core/caching.py 124 0 100%
|
||||
app/core/compression.py 59 0 100%
|
||||
app/core/debugging.py 29 2 93%
|
||||
app/core/docs_push.py 43 0 100%
|
||||
app/core/errors.py 5 0 100%
|
||||
app/core/http_cache.py 22 0 100%
|
||||
app/core/logging.py 13 0 100%
|
||||
app/core/rate_limit.py 44 0 100%
|
||||
app/core/response_budget.py 63 0 100%
|
||||
app/core/security_headers.py 20 0 100%
|
||||
app/core/settings_catalog.py 18 0 100%
|
||||
app/core/settings_migrate.py 32 0 100%
|
||||
app/core/settings_overlay.py 111 0 100%
|
||||
app/core/theming.py 40 0 100%
|
||||
app/core/tokens.py 44 0 100%
|
||||
app/db.py 22 0 100%
|
||||
app/main.py 105 0 100%
|
||||
app/models.py 141 0 100%
|
||||
app/rag/__init__.py 0 0 100%
|
||||
app/rag/agent.py 494 1 99%
|
||||
app/rag/archive_upload.py 134 0 100%
|
||||
app/rag/chunker.py 220 4 98%
|
||||
app/rag/doc_dates.py 18 0 100%
|
||||
app/rag/folder_summaries.py 134 0 100%
|
||||
app/rag/git_sources.py 77 0 100%
|
||||
app/rag/importer.py 864 3 99%
|
||||
app/rag/llm.py 253 1 99%
|
||||
app/rag/md_images.py 63 0 100%
|
||||
app/rag/overview.py 78 0 100%
|
||||
app/rag/pdf.py 192 0 100%
|
||||
app/rag/prompts.py 162 0 100%
|
||||
app/rag/retriever.py 252 3 99%
|
||||
app/rag/scaffolding.py 55 0 100%
|
||||
app/rag/source_removal.py 41 0 100%
|
||||
app/rag/sources_meta.py 16 0 100%
|
||||
app/rag/suggestions.py 27 0 100%
|
||||
app/rag/summarizer.py 58 0 100%
|
||||
app/rag/svg_text.py 111 0 100%
|
||||
app/rag/verbatim.py 160 0 100%
|
||||
app/schemas.py 549 0 100%
|
||||
--------------------------------------------------
|
||||
TOTAL 6872 16 99%
|
||||
coverage gate: app/ 99% (>90%) OK
|
||||
All checks passed!
|
||||
0 errors, 0 warnings, 0 informations
|
||||
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
|
||||
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
|
||||
|
||||
validation OK
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 68 KiB After Width: | Height: | Size: 69 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 42 KiB After Width: | Height: | Size: 42 KiB |
Reference in New Issue
Block a user