brain-of-reese

300 Commits 1 Branch 0 Tags

3 Commits

Include renames

Author	SHA1	Message	Date
ducoterra	a45f744604	phase: 194_sync_pending_count Build and Push Containers / build-and-push-app (push) Successful in 2m5s Details Build and Push Containers / build-and-push-db (push) Successful in 13s Details Phase 194 — final verification pass: all green. Verified (all 5 tasks already complete; nothing re-implemented needed) - `app/rag/importer.py`: `_prescan_pending` + `_pending_hash_stmt` (scalar `documents.file_hash`, own session, ONE `asyncio.to_thread` span, REPLACES the stats pre-walk), `on_scan_progress`, `progress` pair = pending position, `0` branch = byte-identical phase-193 walk. - `app/config.py`/`settings_catalog.py`/`.env.example`: `sync_pending_count=1` + validator; `app/api/sync.py`: `"scan"` phase, `scan_done`/`scan_total` (additive after `files_total`, kept at every terminal), forced/throttled RSS split; `frontend/assets/sources.js`: `fmtSyncPhaseLabel` scan branch first-arm, `fmtSyncLabel` untouched. Tests / lint / coverage (exact commands) - `uv run pytest --cov=app --cov-report=term-missing` → 4835 passed, `app/` 99% (7240 stmts / 16 miss), exit 0 - `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → 8 passed in 150.81s (phase suite, isolated) - 14 sync-lineage E2E suites, one pytest process each (`test_sync_remaining_count`, `…_chunk_progress`, `…_progress_box`, `…_peak_memory_visible`, `…_label_mobile`, `…_upload_progress`, `test_archive_upload_sources`, `…_image_progress`, `…_summary_visibility`, `…_status_freshness`, `test_smoke`, `test_fast_path_deferred_content`, `test_sync_session_reset`, `test_sync_git_freeze`) → all exit 0 - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors Completion criteria — all met: `N remaining` = predicted change set (E2E arms 1/2/5, first import counts the walk, no-change → no count part); `scan` phase visible + flips to `import`, scan pair reset/kept at terminals (`test_sync_scan_phase.py`); `BOR_SYNC_PENDING_COUNT=0` walk semantics ("7 remaining" arm) + negative refuses startup naming the field (verified live by importing `app.main`); pre-scan off-loop + cancel lands `cancelled` (integration pins); CLI/upload byte-identical (unedited pins green); payload additive (`_PRE_187_STATUS_KEYS \| _MEMORY_KEYS \| _SCAN_KEYS`); suite green, coverage >90%, lint/types clean. Documentation — 3 fixes: README sync "States" bullet had an unclosed parenthesis (pre-existing, 193/194) and the scan-label "total is 0" clause was ambiguous about a `=0` run (it still counts the walk once the import starts); AGENTS.md rule 17 had a stray `)` (pre-existing from phase 191). Also verified consistency both directions for all 87 settings ↔ README ↔ `.env.example` ↔ settings catalog, the pages table ↔ `app/main.py` routes, no stale `BOR_` names, and README/AGENTS claims through phase 194. Notes/deviations — no live-infra changes (prod sync knob ships default-on, no deployment touched); no git index/history commands run (all changes left in the working tree for the harness); commit criterion left to the harness as instructed. Next pending phase: none — `.agents/phases/todo/` holds only `194_sync_pending_count`, which is now ready to move to `complete/`.	2026-10-20 06:36:21 -04:00
ducoterra	632fb88e43	phase: 185_sync_defer_content_fast_path Build and Push Containers / build-and-push-app (push) Successful in 1m41s Details Build and Push Containers / build-and-push-db (push) Successful in 11s Details All gates green. Report: Implemented / verified (phase 185) - `_doc_row_stmt` is the ONE deferral helper for all four fast-path row loads; only `Document.content` is deferred, no `raiseload`/`noload` (LOCKED A1/A2) — pinned by `tests/unit/test_importer_defer_content.py`. - `tests/integration/test_import_fast_path_columns.py` (5), `tests/e2e/test_fast_path_deferred_content.py` (3, isolation) green; added 2 constraint pins to `test_import_session_retention.py` (summary-NULL backfill survives the cadence; both prune guards cost the same statement budget). - Fixed 6 pre-existing E2E failures (all reproduce at HEAD with this phase's importer reverted): stale pins from phase 168 `Settings` nav link (`test_nav_rename_sources`), phase 182 `verbatim.js` script count (`test_steering`), phase 175 `/api/chats?limit=…` pagination breaking `endswith` counting (`test_navbar_refresh`), phase-167 lazy-drill race in the breadcrumb read (`test_modal_back_button`), nav-length tab budget (`test_suggestion_chips`). - Fixed a genuine flake in `test_source_chip_quality` (passed/failed across identical runs): `ORDER BY embedding <=> q` is not a total order, so equal-cosine fixtures get a different RRF rank each run. Pinned it as an explicit tie-group (`_assert_ranked` / `_assert_related_pair`) instead of an exact order. Tests / lint (all 100% green) - `uv run pytest --cov=app --cov-report=term-missing` → 4519 passed, `app/rag/importer.py` 99%, `TOTAL` 99% (>90%). - `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors. - Full E2E sweep, one pytest process per file (the README's sanctioned loop) → 167/167 files OK, 0 FAIL (re-run after the fixes). Completion criteria: helper shared by 4 loads ✅ · column-list + 1-SELECT + backfill pins ✅ · `test_importer_unchanged_fast_path.py` 24 passed, zero expectation edits ✅ · full suite + coverage + lint/types ✅ · phase E2E in isolation ✅ · docs accurate ✅ · commit left to the harness (no git commands run). Documentation: README + AGENTS.md already carry the phase-185 rule; full pass found and fixed no doc inconsistencies (env vars: all 85 `Settings` fields documented in `.env.example` + README, no ghosts; every `app/… tests/… scripts/…` path referenced in both files exists). Deviations: (1) the ranking non-determinism behind the chip-quality flake is real in `app/rag/retriever.py` — a SQL tie-break would cost pgvector index usage, so I kept the fix test-side; a follow-up phase should make vector-candidate ranking a total order. (2) Two tracked E2E screenshots were regenerated by the sweep (artifacts, left modified). Next pending phase: `186_git_log_streaming_dates`.	2026-10-16 04:54:01 -04:00
ducoterra	065304b52f	phase: 184_sync_bounded_session Build and Push Containers / build-and-push-app (push) Successful in 1m53s Details Build and Push Containers / build-and-push-db (push) Successful in 13s Details All gates green; no code changes were needed — the prior failure was a missing final report, not a test failure. Verified / implemented (phase 184 final pass) - All 4 tasks present in `.agents/phases/complete/184_sync_bounded_session/`; confirmed their output in the worktree: `sync_session_reset_files=200` / `sync_prune_batch=500` on `Settings` (+ non-negative validators), both `SettingMeta` rows under `"Import & sources"`, `.env.example` + `README.md` entries. - `import_sources` expunges at exactly two sites (per-file loop top L972, source boundary L1012) — grep proves no third `expunge`/`expire_all` anywhere in `app/`; `app/db.py:30` untouched. - `_prune` uses `load_only(id, source, path, is_image, image_path, page_count).order_by(Document.id)` + per-batch commits, with the phase-122/126 guards and `changed_paths` intact. Tests / lint / coverage (all 100% green) - `uv run pytest --cov=app --cov-report=term-missing` → 4506 passed, `app/` TOTAL 99% (>90%). - `uv run pytest tests/integration/test_import_session_retention.py -v --no-cov` → 12 passed. - `uv run pytest tests/e2e/test_sync_session_reset.py -v --no-cov` → 4 passed in isolation. - Regression sweep of neighbouring sync suites (each isolated): `test_sync_changed_only` 3, `test_sync_summary_visibility` 3, `test_prune_cascade_cleanup` 3, `test_sync_git_freeze` 2 — all passed. - `uv run ruff check .` → All checks passed. `uv run pyright` → 0 errors, 0 warnings. Completion criteria 1. Settings + CATALOG + `.env.example` + README; `test_settings_catalog.py` → ✅ green 2. Retention test green with the post-fix bound and both measured numbers in the docstring (pre-fix 12-row monotone staircase / ≈265 MB extrapolated; post-fix ends at 0, add-path peak 14 vs 21→49) → ✅ 3. Expunge only at the two documented-safe points; light-row batched prune with guards intact → ✅ 4. Every existing importer/sync/removal test green with unchanged expectations → ✅ 5. Full suite green, coverage >90%, ruff + pyright clean → ✅ 6. Phase E2E green in isolation → ✅ 7. README + AGENTS accurate/consistent → ✅ 8. Atomic commit — left to the harness (I ran no git write commands). Documentation — Full README (1886 lines) + AGENTS.md (189 lines) review: verified consistent — no changes needed. Programmatic cross-checks: every `Settings` field appears in `.env.example` and in the README config reference; every `.env.example` var is documented in that reference; every `tests/…` and `app/…` path cited in README/AGENTS exists; all 10 README page URLs are registered in `app/main.py`; documented E2E port vars (`E2E_APP_PORT_SYNCRESET[_ZERO]`) exist in the test. The only reference-var without a `Settings` field is `BOR_THEME`, which the README explicitly documents as retired (correct as written). Notable decisions/deviations — None. No code, test, or doc edits were made this pass; changes remain entirely in the working tree for the harness to stage. `00_phase.md` still sits in `todo/` while tasks 01–04 are in `complete/` — harness-owned, so I left it untouched. Next pending phase: `185_sync_defer_content_fast_path`.	2026-10-15 23:19:27 -04:00

Author

SHA1

Message

Date

ducoterra

a45f744604

phase: 194_sync_pending_count

Build and Push Containers / build-and-push-app (push) Successful in 2m5s

Details

Build and Push Containers / build-and-push-db (push) Successful in 13s

Details

**Phase 194 — final verification pass: all green.**

**Verified (all 5 tasks already complete; nothing re-implemented needed)**
- `app/rag/importer.py`: `_prescan_pending` + `_pending_hash_stmt` (scalar `documents.file_hash`, own session, ONE `asyncio.to_thread` span, REPLACES the stats pre-walk), `on_scan_progress`, `progress` pair = pending position, `0` branch = byte-identical phase-193 walk.
- `app/config.py`/`settings_catalog.py`/`.env.example`: `sync_pending_count=1` + validator; `app/api/sync.py`: `"scan"` phase, `scan_done`/`scan_total` (additive after `files_total`, kept at every terminal), forced/throttled RSS split; `frontend/assets/sources.js`: `fmtSyncPhaseLabel` scan branch first-arm, `fmtSyncLabel` untouched.

**Tests / lint / coverage (exact commands)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4835 passed**, `app/` 99% (7240 stmts / 16 miss), exit 0
- `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → **8 passed in 150.81s** (phase suite, isolated)
- 14 sync-lineage E2E suites, one pytest process each (`test_sync_remaining_count`, `…_chunk_progress`, `…_progress_box`, `…_peak_memory_visible`, `…_label_mobile`, `…_upload_progress`, `test_archive_upload_sources`, `…_image_progress`, `…_summary_visibility`, `…_status_freshness`, `test_smoke`, `test_fast_path_deferred_content`, `test_sync_session_reset`, `test_sync_git_freeze`) → **all exit 0**
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**

**Completion criteria** — all met: `N remaining` = predicted change set (E2E arms 1/2/5, first import counts the walk, no-change → no count part); `scan` phase visible + flips to `import`, scan pair reset/kept at terminals (`test_sync_scan_phase.py`); `BOR_SYNC_PENDING_COUNT=0` walk semantics ("7 remaining" arm) + **negative refuses startup naming the field (verified live by importing `app.main`)**; pre-scan off-loop + cancel lands `cancelled` (integration pins); CLI/upload byte-identical (unedited pins green); payload additive (`_PRE_187_STATUS_KEYS | _MEMORY_KEYS | _SCAN_KEYS`); suite green, coverage >90%, lint/types clean.

**Documentation** — 3 fixes: README sync "States" bullet had an **unclosed parenthesis** (pre-existing, 193/194) and the scan-label "total is 0" clause was ambiguous about a `=0` run (it still counts the walk once the import starts); **AGENTS.md rule 17 had a stray `)`** (pre-existing from phase 191). Also verified consistency both directions for all 87 settings ↔ README ↔ `.env.example` ↔ settings catalog, the pages table ↔ `app/main.py` routes, no stale `BOR_` names, and README/AGENTS claims through phase 194.

**Notes/deviations** — no live-infra changes (prod sync knob ships default-on, no deployment touched); no git index/history commands run (all changes left in the working tree for the harness); commit criterion left to the harness as instructed.

**Next pending phase:** none — `.agents/phases/todo/` holds only `194_sync_pending_count`, which is now ready to move to `complete/`.

2026-10-20 06:36:21 -04:00

ducoterra

632fb88e43

phase: 185_sync_defer_content_fast_path

Build and Push Containers / build-and-push-app (push) Successful in 1m41s

Details

Build and Push Containers / build-and-push-db (push) Successful in 11s

Details

All gates green. Report:

**Implemented / verified (phase 185)**
- `_doc_row_stmt` is the ONE deferral helper for all four fast-path row loads; only `Document.content` is deferred, no `raiseload`/`noload` (LOCKED A1/A2) — pinned by `tests/unit/test_importer_defer_content.py`.
- `tests/integration/test_import_fast_path_columns.py` (5), `tests/e2e/test_fast_path_deferred_content.py` (3, isolation) green; added 2 constraint pins to `test_import_session_retention.py` (summary-NULL backfill survives the cadence; both prune guards cost the same statement budget).
- **Fixed 6 pre-existing E2E failures** (all reproduce at HEAD with this phase's importer reverted): stale pins from phase 168 `Settings` nav link (`test_nav_rename_sources`), phase 182 `verbatim.js` script count (`test_steering`), phase 175 `/api/chats?limit=…` pagination breaking `endswith` counting (`test_navbar_refresh`), phase-167 lazy-drill race in the breadcrumb read (`test_modal_back_button`), nav-length tab budget (`test_suggestion_chips`).
- **Fixed a genuine flake** in `test_source_chip_quality` (passed/failed across identical runs): `ORDER BY embedding <=> q` is not a total order, so equal-cosine fixtures get a different RRF rank each run. Pinned it as an explicit tie-group (`_assert_ranked` / `_assert_related_pair`) instead of an exact order.

**Tests / lint (all 100% green)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4519 passed**, `app/rag/importer.py` **99%**, `TOTAL` **99%** (>90%).
- `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors.
- Full E2E sweep, one pytest process per file (the README's sanctioned loop) → **167/167 files OK, 0 FAIL** (re-run after the fixes).

**Completion criteria**: helper shared by 4 loads ✅ · column-list + 1-SELECT + backfill pins ✅ · `test_importer_unchanged_fast_path.py` 24 passed, zero expectation edits ✅ · full suite + coverage + lint/types ✅ · phase E2E in isolation ✅ · docs accurate ✅ · commit left to the harness (no git commands run).

**Documentation**: README + AGENTS.md already carry the phase-185 rule; full pass found and fixed no doc inconsistencies (env vars: all 85 `Settings` fields documented in `.env.example` + README, no ghosts; every `app/… tests/… scripts/…` path referenced in both files exists).

**Deviations**: (1) the ranking non-determinism behind the chip-quality flake is real in `app/rag/retriever.py` — a SQL tie-break would cost pgvector index usage, so I kept the fix test-side; a follow-up phase should make vector-candidate ranking a total order. (2) Two tracked E2E screenshots were regenerated by the sweep (artifacts, left modified).

**Next pending phase**: `186_git_log_streaming_dates`.

2026-10-16 04:54:01 -04:00

ducoterra

065304b52f

phase: 184_sync_bounded_session

Build and Push Containers / build-and-push-app (push) Successful in 1m53s

Details

Build and Push Containers / build-and-push-db (push) Successful in 13s

Details

All gates green; no code changes were needed — the prior failure was a missing final report, not a test failure.

**Verified / implemented (phase 184 final pass)**
- All 4 tasks present in `.agents/phases/complete/184_sync_bounded_session/`; confirmed their output in the worktree: `sync_session_reset_files=200` / `sync_prune_batch=500` on `Settings` (+ non-negative validators), both `SettingMeta` rows under `"Import & sources"`, `.env.example` + `README.md` entries.
- `import_sources` expunges at **exactly two** sites (per-file loop top L972, source boundary L1012) — grep proves no third `expunge`/`expire_all` anywhere in `app/`; `app/db.py:30` untouched.
- `_prune` uses `load_only(id, source, path, is_image, image_path, page_count).order_by(Document.id)` + per-batch commits, with the phase-122/126 guards and `changed_paths` intact.

**Tests / lint / coverage (all 100% green)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4506 passed**, `app/` TOTAL **99%** (>90%).
- `uv run pytest tests/integration/test_import_session_retention.py -v --no-cov` → **12 passed**.
- `uv run pytest tests/e2e/test_sync_session_reset.py -v --no-cov` → **4 passed in isolation**.
- Regression sweep of neighbouring sync suites (each isolated): `test_sync_changed_only` 3, `test_sync_summary_visibility` 3, `test_prune_cascade_cleanup` 3, `test_sync_git_freeze` 2 — all passed.
- `uv run ruff check .` → All checks passed. `uv run pyright` → 0 errors, 0 warnings.

**Completion criteria**
1. Settings + CATALOG + `.env.example` + README; `test_settings_catalog.py` → ✅ green
2. Retention test green with the post-fix bound and both measured numbers in the docstring (pre-fix 12-row monotone staircase / ≈265 MB extrapolated; post-fix ends at 0, add-path peak 14 vs 21→49) → ✅
3. Expunge only at the two documented-safe points; light-row batched prune with guards intact → ✅
4. Every existing importer/sync/removal test green with unchanged expectations → ✅
5. Full suite green, coverage >90%, ruff + pyright clean → ✅
6. Phase E2E green in isolation → ✅
7. README + AGENTS accurate/consistent → ✅
8. Atomic commit — left to the harness (I ran no git write commands).

**Documentation** — Full README (1886 lines) + AGENTS.md (189 lines) review: **verified consistent — no changes needed.** Programmatic cross-checks: every `Settings` field appears in `.env.example` and in the README config reference; every `.env.example` var is documented in that reference; every `tests/…` and `app/…` path cited in README/AGENTS exists; all 10 README page URLs are registered in `app/main.py`; documented E2E port vars (`E2E_APP_PORT_SYNCRESET[_ZERO]`) exist in the test. The only reference-var without a `Settings` field is `BOR_THEME`, which the README explicitly documents as retired (correct as written).

**Notable decisions/deviations** — None. No code, test, or doc edits were made this pass; changes remain entirely in the working tree for the harness to stage. `00_phase.md` still sits in `todo/` while tasks 01–04 are in `complete/` — harness-owned, so I left it untouched.

**Next pending phase:** `185_sync_defer_content_fast_path`.

2026-10-15 23:19:27 -04:00