Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 194_sync_pending_count
Build and Push Containers / build-and-push-app (push) Successful in 2m5s
Details
Build and Push Containers / build-and-push-db (push) Successful in 13s
Details

Browse Source 
**Phase 194 — final verification pass: all green.**

**Verified (all 5 tasks already complete; nothing re-implemented needed)**
- `app/rag/importer.py`: `_prescan_pending` + `_pending_hash_stmt` (scalar `documents.file_hash`, own session, ONE `asyncio.to_thread` span, REPLACES the stats pre-walk), `on_scan_progress`, `progress` pair = pending position, `0` branch = byte-identical phase-193 walk.
- `app/config.py`/`settings_catalog.py`/`.env.example`: `sync_pending_count=1` + validator; `app/api/sync.py`: `"scan"` phase, `scan_done`/`scan_total` (additive after `files_total`, kept at every terminal), forced/throttled RSS split; `frontend/assets/sources.js`: `fmtSyncPhaseLabel` scan branch first-arm, `fmtSyncLabel` untouched.

**Tests / lint / coverage (exact commands)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4835 passed**, `app/` 99% (7240 stmts / 16 miss), exit 0
- `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → **8 passed in 150.81s** (phase suite, isolated)
- 14 sync-lineage E2E suites, one pytest process each (`test_sync_remaining_count`, `…_chunk_progress`, `…_progress_box`, `…_peak_memory_visible`, `…_label_mobile`, `…_upload_progress`, `test_archive_upload_sources`, `…_image_progress`, `…_summary_visibility`, `…_status_freshness`, `test_smoke`, `test_fast_path_deferred_content`, `test_sync_session_reset`, `test_sync_git_freeze`) → **all exit 0**
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**

**Completion criteria** — all met: `N remaining` = predicted change set (E2E arms 1/2/5, first import counts the walk, no-change → no count part); `scan` phase visible + flips to `import`, scan pair reset/kept at terminals (`test_sync_scan_phase.py`); `BOR_SYNC_PENDING_COUNT=0` walk semantics ("7 remaining" arm) + **negative refuses startup naming the field (verified live by importing `app.main`)**; pre-scan off-loop + cancel lands `cancelled` (integration pins); CLI/upload byte-identical (unedited pins green); payload additive (`_PRE_187_STATUS_KEYS | _MEMORY_KEYS | _SCAN_KEYS`); suite green, coverage >90%, lint/types clean.

**Documentation** — 3 fixes: README sync "States" bullet had an **unclosed parenthesis** (pre-existing, 193/194) and the scan-label "total is 0" clause was ambiguous about a `=0` run (it still counts the walk once the import starts); **AGENTS.md rule 17 had a stray `)`** (pre-existing from phase 191). Also verified consistency both directions for all 87 settings ↔ README ↔ `.env.example` ↔ settings catalog, the pages table ↔ `app/main.py` routes, no stale `BOR_` names, and README/AGENTS claims through phase 194.

**Notes/deviations** — no live-infra changes (prod sync knob ships default-on, no deployment touched); no git index/history commands run (all changes left in the working tree for the harness); commit criterion left to the harness as instructed.

**Next pending phase:** none — `.agents/phases/todo/` holds only `194_sync_pending_count`, which is now ready to move to `complete/`.
This commit is contained in:
ducoterra
2026-10-20 06:36:21 -04:00
parent ae080ac7f3
commit a45f744604
55 changed files with 7610 additions and 179 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/phases/complete/194_sync_pending_count/00_phase.md
+70
View File
@@ -0,0 +1,70 @@
# Phase 194 — "N remaining" counts the files that will actually change
**Source:** owner report 2026-10-19 — "I just synced the prod brain of reese and it said '1216 remaining' when it only had like 10 files to sync" (the prod KB holds ~10,000 importable files; that run changed ~10).
**Story:** n/a (owner directive — phase 193's confirmed assumption A1 proved wrong in production)
**Depends on:** `193_sync_remaining_count` (complete) — builds on the phase-64/98/128/129/152/179/184/185/187 sync lineage.
## Evidence (diagnosed from the code, read this not the chat)
1. **The phase-193 label counts the WALK, not the WORK.** `frontend/assets/sources.js` `fmtSyncLabel` renders `N remaining` with `N = files_total − files_done` (the arithmetic is fine). But `app/rag/importer.py`'s pre-walk (L949–962) sets `total` to the COMBINED COUNT OF EVERY IMPORTABLE FILE across all sources (the 10,000), and the per-file hook (L1034–1039) reports `done = summary.files` (the 1-based position across ALL files walked). The fast path (phase 147/185) skips re-embedding for the unchanged 9,990 but still COUNTS them. Prod: "1216 remaining" = "8,784 files left to walk", while the actual work was ~10 files. Phase 193's E2E suite passed because its fixture KB was a FRESH import where walk == work — the incremental case the owner just hit was never pinned.
2. **The change signal already exists per file.** `_file_hash` (L704–724 — the sha256 of the RAW bytes, streamed in 1 MiB chunks) compared against `documents.file_hash` (nullable `String(64)`, no index — looked up through the unique `(source, path)` constraint, `app/models.py` L119/L138–142) is EXACTLY the fast path's change predicate: stored NULL (no row, or a pre-phase row) or `!= digest` ⇔ the file will be re-indexed (added/updated). A pre-scan reusing that predicate predicts the change set with the SAME semantics the processing loop then applies — pre-phase NULL rows are predicted changed AND actually re-indexed (stamped) by the loop, so the prediction never lies about the stored-hash contract.
3. **The pre-scan must be OFF the loop.** A 10,000-file hash+SELECT span run inline would park the single uvicorn loop for seconds — the exact phase-179 defect class pinned by `tests/integration/test_sync_git_freeze.py` (the 3 s git stub froze the 2 s status poll) and `tests/e2e/test_sync_git_freeze.py`. The git span moved to `asyncio.to_thread` (phase 179); the pre-scan does the same (the worker runs to completion on cancel; the run stops at the next phase-134 boundary — the `CancelledError` clause in `_run_sync`, L178–190 contract).
4. **Only the UI sync pays anything.** `import_sources` has exactly two callers: `app/api/sync.py` (the UI sync — the ONLY one that passes `progress`) and `scripts/import_docs.py` L407 (the CLI — no `progress`). Uploads have no file-level progress at all (phase 90 A2 — `app/api/git_sources.py` L59; their label renders bare, the `test_archive_upload_sources.py` pins). Gating the pre-scan on `progress is not None` (the phase-64 skip rule) keeps every non-UI caller byte-identical.
5. **The count is a prediction, and the drift edges are bounded.** The pre-scan runs seconds before the processing loop reaches a given file. A predicted file that turns out unchanged (reverted mid-run) still advances the count (it was predicted — `total` counted it) so the count reaches 0; a file changed AFTER the scan is indexed for real but does not advance the count, and the phase-193 `total − done > 0` guard hides the excess (never negative, never a stale "0 remaining"). The terminal `counts_line` ("N added · M updated …") stays the exact surface — untouched.
## Design (shared by all tasks — the executor reads this, not the chat)
- **D1 — the pre-scan (task 01).** `import_sources` gains ONE optional callback `on_scan_progress: Callable[[int, int], None] | None = None` (the 1-based scan position + the importable total — the "only when provided", "hooks only assign fields", no-try/except contract of its three siblings). While `progress is not None` AND `llm.settings.sync_pending_count` is truthy (read ONCE before the source loop, the phase-184 read-once discipline — `0` = rollback to the phase-193 walk-position semantics, negative value refuses startup loudly, the house kill-switch pattern):
- the EXISTING stats-only pre-walk is REPLACED (not augmented — no third walk) by ONE `asyncio.to_thread` span that, per root with the EXACT `iter_importable_files` rules of the processing loop (same `ignore` / `include_hidden` / `image_extension_set` / images-toggle gating): (a) takes the per-root file list and its length (the scan total — the same number the old pre-walk computed), then (b) per file: `_file_hash(path)` (the existing streamed helper) + ONE scalar `SELECT documents.file_hash WHERE source = :s AND path = :p` (the only column — NO entity load, NO `defer`/`load_only` row, no session retention, one light metadata read per file; phase 184/185 compliant: it is not a whole-entity `select(Document)`, it defers nothing because it loads no row, and the processing loop's `_doc_row_stmt` fast-path row load stays the only ROW load) → `pending = {(source, rel)}` for every file whose stored hash is NULL or `!= digest`;
- `on_scan_progress(position, total)` fires per file (the scan position for the UI's "Scanning… (n/total)" label — called from the worker thread; field assignments are GIL-atomic, the phase-179 off-loop contract), and `files_total` becomes `len(pending)` while the per-file `progress` call's `done` becomes the 1-based POSITION IN THE PENDING SET (it advances for every predicted file, whether or not it turns out changed at processing time — the count and its denominator stay on the SAME set so the arithmetic never runs away);
- `summary.files` (the `import: summary … files=` log count — every file the loop processed, append-only log discipline) is UNTOUCHED; the `sync: done/cancelled/failed … files_done=… files_total=…` lines keep their EXACT shape, now carrying the pending counts (the module docstring names the semantic shift);
- the processing loop's FRESH per-file `_file_hash` is KEPT (the double hash is deliberate: the pre-scan's digest is the prediction, the loop's is the truth — a file edited between the two is still detected and re-indexed; the second read is page-cached — the cost the owner accepted, A3);
- `on_scan_progress` is called only when provided; `progress=None` callers (the CLI) skip the pre-scan entirely — byte-identical behavior (the phase-64 skip rule, extended);
- the degenerate empty KB: the scan total is 0, no callback fires, the run's `phase` stays `"scan"` until the first `progress` call (which there is none) or the next transition/terminal — the UI renders bare "Scanning…" (the `total === 0` guard → no count part) and the terminal clears the phase; documented, pinned in the task-05 arm;
- docstrings: the module `progress` paragraph (the `done`/`total` semantic shift), the new `on_scan_progress` paragraph, the `to_thread`/cancel/cost paragraphs, the `sync_pending_count` gating clause.
- **D2 — the kill switch (task 01).** `app/config.py` gains `sync_pending_count: int = 1` (`BOR_SYNC_PENDING_COUNT`): `1` (default) = the pre-scan pending-count semantics; `0` = the phase-193 WALK-position semantics (the stats pre-walk + `done = summary.files` — the full rollback the owner gets if the cost ever bites); a negative value refuses startup with the field named (the `agent_max_rounds` house pattern). `.env.example` + README settings documentation. The switch is read ONCE per run (the phase-184 discipline — a mid-run settings save cannot change the semantics between two files of the same walk).
- **D3 — the phase machine + status (task 02).** `SyncStatus.phase` gains `"scan"` (additive — the machine: the model-check + clone/pull prelude reports `null`, then `"scan"` (set immediately before `import_sources`, REPLACING today's `= "import"`), `"import"` (flipped by the FIRST `progress` call while `phase == "scan"` — the flip point is the existing per-file hook, so no extra callback), `"overview"`, `"summaries"` — the no-gap summaries skip still keeps the run in `"import"`). The flip + the pre-`import_sources` set are phase transitions, so each gets the phase-187 FORCED `_sample_rss()` (the per-file `_scan_hook` calls the UNFORCED throttled sample, the progress-hook convention — one `/proc` read per 5 s, never one per file). `SyncStatus` gains `scan_done` / `scan_total` (`int`, default 0; reset with the run; KEPT at every terminal — the phase-64 keep-final-values convention, beside the final file counts). The status payload gains the same two keys ADDITIVELY (placed after `files_total`; every pre-existing key keeps its name, type, position, and meaning — the phase-187 additive-only discipline). `current_file` stays `null` during the scan (the scan names no file). Docstrings: the module phase-machine paragraph (L188–200 region), the `SyncStatus` class paragraph (the phase-64/98/187 clauses gain the scan clause), the `GET /api/sync/status` docstring field paragraph.
- **D4 — the UI scan label (task 03).** `fmtSyncPhaseLabel` (the phase-aware wrapper — `frontend/assets/sources.js`) gains ONE explicit `"scan"` branch in its decision tree: main text `Scanning…` (no file — `current_file` is null in the phase, the branch must not read it), count part `(n/total)` rendered in the SAME count slot the import phase's `N remaining` uses (`n = scan_done`, `total = scan_total`; the `total === 0` guard → no count part, the phase-64 contract); the caller passes the scan pair through the existing call sites that already forward the summaries/chunk/image parts (the `phase` value selects which pair the wrapper consumes — the summaries-branch pattern). The `null` prelude + `"import"` fallback clause now covers ONLY those two. `fmtSyncLabel`'s `N remaining` arithmetic is BYTE-IDENTICAL (the phase-193 pins stay unedited — the server-side semantic change flows through the SAME builder unchanged). The A4 untruncated placement (button title + `#sync-progress`) is untouched — `Scanning… (n/total)` rides it automatically; the phase-187 `· peak N MB` trailing clause still lands on the `#sync-progress` counts line while the scan label is visible. The docstring's decision-tree paragraph (L157–167 region) is updated. The walk position surviving as `(n/total)` is HONEST for a scan — it is labeled "Scanning", never "Syncing" (the owner's objection, TODO L6, was the sync count wearing the total's shape).
- **D5 — the pinned suites move (task 04).** See the task for the definitive list (the count-frame pins retarget to the pending-set semantics where the fixture KB is NOT fresh; the fresh-fixture pins (chunk progress, label mobile, upload) stay unedited; the phase-184/185/179 integration pins get the pre-scan's statements distinguished or are verified unaffected).
## ASSUMPTIONS (owner-confirmed 2026-10-19, questionnaire)
- **A1:** `N remaining` counts down the pre-scan's PREDICTED change set (`files_total = len(pending)`, `files_done` = the 1-based position in it); the count advances on the prediction set (a predicted-then-unchanged file still advances it — the count reaches 0; a post-scan change is indexed but uncounted — the phase-193 guard hides the excess). First-time imports (empty DB) count the full walk exactly as phase 193 did; a no-change run shows no count part.
- **A2:** the pre-scan is a NEW VISIBLE `"scan"` phase — `Scanning… (n/total)` before the `import` phase; the scan's walk position is the only surviving `n/m` shape, and it is labeled a scan.
- **A3:** the cost is accepted: one extra full read + sha256 + one scalar metadata SELECT per importable file per run (the second read is page-cached; the span is off-loop via `to_thread`, the phase-179 rule). The rollback knob `BOR_SYNC_PENDING_COUNT=0` restores the phase-193 walk-position semantics; the CLI and the upload job pay nothing.
## Context
- `app/rag/importer.py` — `import_sources` (L756+, the pre-walk L949–962, the per-file `progress` call L1034–1039), `_file_hash` (L704–724), `_doc_row_stmt` (L725+), the module docstring's progress/chunk/image hook paragraphs.
- `app/api/sync.py` — the module docstring (the phase-179/134/98 paragraphs, L100–200 region), `SyncStatus` (L308+), `GET /api/sync/status` (L495+, the payload dict L548+), `_run_sync` (L651+, the reset block, the `_hook`/`_scan_hook`/`_chunk_hook`/`_image_hook` closures L815–860 region, the `_status.phase = "import"` set L880, the terminal log lines L1112/L1133).
- `app/config.py` — the `BOR_SYNC_*` settings block + validators (the `agent_max_rounds` kill-switch pattern); `.env.example`.
- `app/models.py` — `Document` (L117+, the unique `(source, path)`, the `file_hash` column contract L128–142).
- `frontend/assets/sources.js` — `fmtSyncPhaseLabel` (L157–167 docstring, the decision tree), `fmtSyncLabel` (L1043–1071 — UNTOUCHED), the status call sites (L1121–1127, L1449–1458, L1621–1652), `setSyncLabel`/A4 (UNTOUCHED).
- `scripts/import_docs.py` L407 (the CLI caller — `progress` absent — the byte-identity pin), `app/api/git_sources.py` L59 (the upload's no-file-progress contract — UNTOUCHED).
- Pinned suites: `tests/e2e/test_sync_remaining_count.py` (phase 193's own — the count-frame pins + the status key-set pin), `tests/e2e/test_sync_chunk_progress.py`, `test_sync_label_mobile.py`, `test_sync_upload_progress.py`, `test_archive_upload_sources.py` (the upload's "no `N remaining`" pins), `tests/integration/test_import_fast_path_columns.py` (phase 185 — the per-file SELECT pins), `test_import_session_retention.py` (phase 184), `test_sync_git_freeze.py` (phase 179 — the loop-freeze pin), `test_sync_session_reset.py`, `test_fast_path_deferred_content.py`, `test_sync_summary_visibility.py`, `test_sync_image_progress.py`, `test_sync_progress_box.py`, `test_sync_status_freshness.py`, `test_sync_peak_memory_visible.py`, `test_smoke.py`.
## Tasks
1. `01_prescan_importer.md` — the importer's pre-scan (the D1 spec) + the `BOR_SYNC_PENDING_COUNT` kill switch (the D2 spec) + the unit/integration pins.
2. `02_sync_phase_status.md` — the `"scan"` phase + `scan_done`/`scan_total` status fields + the additive payload + the docstrings (the D3 spec) + the integration pins.
3. `03_ui_scan_label.md` — the `fmtSyncPhaseLabel` scan branch (the D4 spec) + the JS marker pins.
4. `04_retarget_pinned_suites.md` — the pinned-suite retarget (the D5 list); the full suite green.
5. `05_e2e_and_gates.md` — the phase's own E2E suite (the pending-count-down through the real UI), the full gates, the docs pass, the atomic commit.
## Testing & Quality
- **Unit (marker pins):** the pre-scan gate (the `progress is not None` + `sync_pending_count` read-once), the scalar SELECT's column shape (no entity load), the `pending` predicate (NULL / mismatch / match), the `done`-as-pending-position contract, the kill-switch value handling (1 / 0 / negative).
- **Integration:** the pending set on fresh / unchanged / changed / pre-phase-NULL fixtures; the off-loop pin (the delayed pre-scan span keeps the loop responsive — the phase-179 pattern); the cancel-during-prescan terminal; the phase-184/185/179 pins retargeted or verified; the phase machine sequence (null → scan → import → … → terminal null) + the kept scan counts + the forced/throttled RSS samples.
- **E2E (regression, task 04):** the retargeted pins green; the fresh-fixture + upload pins green UNEDITED.
- **E2E (phase, AGENTS rule 4, task 05):** `tests/e2e/test_sync_pending_count.py`, run in isolation — the first-sync arm, the INCREMENTAL arm (the owner's prod case: K of N changed, the count holds at `K remaining` while the walk moves over unchanged files, hidden at the final predicted file, terminal `K updated`), the no-change arm (no count part), the kill-switch arm (`0` → the phase-193 walk semantics), the empty-KB arm, the peak-clause-on-scan-line pin.
- **Coverage:** **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`).
- **Lint/types:** `uv run ruff check . && uv run pyright`.
## Documentation & Consistency
- Final pass reviews `README.md` and `AGENTS.md` in their entirety against the post-phase app and fixes every inconsistency found, pre-existing ones included.
- `README.md`: the sync section's running-label clause gains the `scan` phase + the pending-count semantics (the phase-193 `N remaining` example becomes the predicted-change-set one) and the `BOR_SYNC_PENDING_COUNT` setting (the `.env.example` entry is part of task 01); `AGENTS.md` — verify whether any clause names the running count part (phase 193 verified it does not — if still absent, do NOT add; the `BOR_SYNC_PENDING_COUNT` setting row joins the settings documentation wherever the README/AGENTS settings table lives).
## Completion Criteria
- [ ] `N remaining` counts down the pre-scan's predicted change set (`files_total = len(pending)`, `files_done` = the pending position) on a progress caller; a first-time import counts the full walk (A1's degenerate parity); a no-change run shows no count part — the E2E arms a/b/c.
- [ ] the `scan` phase is visible (`Scanning… (n/total)`, the position climbs, the `total === 0` guard) and flips to `import` on the first progress call; `scan_done`/`scan_total` reset with the run and survive the terminals — the E2E arms a/e + the integration phase-machine pin.
- [ ] `BOR_SYNC_PENDING_COUNT=0` restores the phase-193 walk-position semantics byte-for-byte (the E2E kill-switch arm + the marker pins); a negative value refuses startup with the field named.
- [ ] the pre-scan span never parks the loop (the off-loop pin); a cancel during it lands the `cancelled` terminal (the phase-134 contract); the CLI and the upload job are byte-identical (the unedited pins).
- [ ] the chunk / image / summaries parts, the `fmtSyncLabel` arithmetic, the terminal `counts_line`, `summary.files` and every pre-existing log line are byte-identical (the unedited pins green); the status payload is additive-only (the key-set pin).
- [ ] test suite green, coverage >90%; `uv run ruff check . && uv run pyright` clean; README + AGENTS consistent (the final pass).
- [ ] one atomic Conventional Commits commit with `--no-gpg-sign`.
.agents/phases/complete/194_sync_pending_count/01_prescan_importer.md
+48
View File
@@ -0,0 +1,48 @@
# Task 01 — The importer's pre-scan + the `BOR_SYNC_PENDING_COUNT` kill switch (D1/D2)
**Phase:** `194_sync_pending_count` · **Source:** owner report 2026-10-19 — "1216 remaining" on a ~10,000-file KB with ~10 changed files.
**Story:** n/a
## Objective
The server-side core: `import_sources` predicts the run's change set BEFORE the walk (the pre-scan, D1) so the `progress` hook's `done`/`total` are the PENDING position (the files that will actually be re-indexed), with the `BOR_SYNC_PENDING_COUNT` kill switch (D2) — `0` restoring the phase-193 walk-position semantics byte-for-byte, negative refusing startup.
## Work
1. `app/config.py` — `sync_pending_count: int = 1` (env `BOR_SYNC_PENDING_COUNT`), the house pattern next to `sync_session_reset_files` (phase 184) / `agent_max_rounds`: `1` = the pre-scan pending-count semantics (default), `0` = the phase-193 walk-position rollback (the stats pre-walk + `done = summary.files`), a NEGATIVE value refuses startup loudly with the field named (the validator). `.env.example` — the entry, documented like the other `BOR_SYNC_*` rows.
2. `app/rag/importer.py` — `import_sources`:
- the signature gains `on_scan_progress: Callable[[int, int], None] | None = None` (the 1-based scan position + the importable total — the "only when provided", "the hooks only assign fields", no-try/except contract of `progress` / `on_chunk_progress` / `on_image_progress`);
- read `llm.settings.sync_pending_count` ONCE before the source loop (the phase-184 read-once discipline — a mid-run settings save cannot flip the semantics between two files of the same walk);
- the GATE: `progress is not None` AND the switch truthy → the PRE-SCAN; `progress is None` (the CLI, `scripts/import_docs.py` L407) → skipped entirely (the phase-64 skip rule, extended — byte-identical); the switch `0` on a progress caller → the EXISTING stats-only pre-walk runs and `done = summary.files` (the phase-193 rollback, byte-for-byte);
- the pre-scan REPLACES the existing stats-only pre-walk (L949–962) — no third walk: ONE `asyncio.to_thread` span (the phase-179 off-loop rule — a 10,000-file hash+SELECT span must not park the single uvicorn loop; the span's worker runs to completion on cancel, the phase-134 contract) that, per root with the EXACT `iter_importable_files` rules of the processing loop (the same `ignore` / `include_hidden` / `image_extension_set` / images-toggle gating — the L930–962 region's rules, byte-identical): (a) the per-root file list + length (the scan total — the same number the old pre-walk computed), then (b) per file: `_file_hash(path)` (the existing streamed 1 MiB-chunk helper, L704–724) + ONE scalar `SELECT documents.file_hash WHERE source = :s AND path = :p` (the ONLY column — NO entity load, NO `defer`/`load_only` row, nothing lands in the session's `identity_map`, no retention — phase 184/185 compliant: it is not a whole-entity `select(Document)` and it defers nothing because it loads no row; the processing loop's `_doc_row_stmt` stays the ONLY row load) → `pending = {(source, rel)}` where stored is NULL (no row, or a pre-phase row) or `!= digest` — the SAME predicate the fast path applies at processing time, so the prediction is consistent with the stored-hash contract (a pre-phase NULL row is predicted changed AND actually re-indexed + stamped by the loop);
- `on_scan_progress(position_1based, total)` fires per file from inside the worker span (the call is a plain field-assignment closure — GIL-atomic; document the thread-safety clause in the docstring);
- `files_total` = `len(pending)` (the `progress` hook's denominator); the per-file `progress(source, rel, done, total)` call's `done` becomes the 1-based POSITION IN THE PENDING SET — it advances for every file in `pending` (whether or not the file turns out changed at processing time — count and denominator stay on the SAME set so the arithmetic never runs away; a file NOT in `pending` does not advance it — the label holds at `K remaining` while the walk moves over the unchanged `N − K`);
- the processing loop's FRESH per-file `_file_hash` stays (the double hash is deliberate: the pre-scan's digest is the prediction, the loop's is the truth — a file edited between the two is still detected; the second read is page-cached — the cost the owner accepted, A3);
- `summary.files` (the `import: summary … files=` count — every file the loop processed) is UNTOUCHED; the prune post-pass, the date refresh, the phase-184 `expunge_all` cadence, the `seen` set: all UNTOUCHED (the pre-scan builds `pending`, not `seen` — the loop keeps building `seen` exactly as today);
- the degenerate empty KB: scan total 0, no callback fires, `pending` empty, `files_total` 0 (the phase-64 `total === 0` no-count-part contract);
- docstrings: the module `progress` paragraph (the `done`/`total` semantic shift + the drift edges — a predicted-then-unchanged file still advances the count so it reaches 0; a post-scan change is indexed but uncounted and the UI's `total − done > 0` guard hides the excess), the new `on_scan_progress` paragraph (the "only when provided" rule, the worker-thread call, the no-try/except contract), the `to_thread`/cancel/cost paragraph (the phase-179/134 clause + the double-hash + the page-cache note + the kill-switch gate), the `sync_pending_count` read-once clause.
3. `tests/unit/` — the marker pins (the importer pin-module convention, the phase-184/185 pin style):
- the gate: `progress is None` → no pre-scan (the callback never fires, no scalar statement — a mocked-callback pin); the switch `0` on a progress caller → the stats pre-walk path (a source marker pin: the old pre-walk code is the `0` branch);
- the scalar SELECT shape: a captured statement names EXACTLY `documents.file_hash` (column-list pin, the phase-185 style — no other column, no entity);
- the `pending` predicate: fresh (no rows → all pending), unchanged (hash match → not pending), changed (mismatch → pending), pre-phase NULL row (→ pending), an ignore-rule file (→ never in the walk, never pending);
- the `done`-as-pending-position: a 3-file fixture (2 predicted, 1 not) → the `progress` calls' `done` sequence is the pending position (the non-predicted file reports the previous position; `total = 2`);
- the switch values: `1`/`0`/negative (the validator names the field on the negative value).
4. `tests/integration/` —
- the pending set through a real `import_sources` call (a `file://`-style temp source dir, the house fixture convention): fresh / unchanged / one-changed fixtures assert the `progress` hook's `done`/`total` pairs AND the terminal `summary.added`/`summary.updated` (the parity pin: the prediction's size equals the terminal's `added + updated` on a stable KB);
- NO entity hydration: `len(session.identity_map) == 0` immediately after the pre-scan span (the `progress` + `on_scan_progress` caller, before any processing-loop row load);
- OFF-LOOP (the phase-179 pattern, `test_sync_git_freeze.py` style): a monkeypatched `_file_hash` that blocks ~3 s over a small multi-file fixture → a loop-side heartbeat (a coroutine timing `asyncio.sleep(0)` / a probe task) stays responsive through the pre-scan span;
- CANCEL: a cancel registered before the (delayed) pre-scan → the worker runs to completion, the run lands the `cancelled` terminal (the phase-134 contract, `test_sync_git_freeze.py`'s cancel-during-clone pattern).
5. Verify: `uv run pytest tests/unit tests/integration -v` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the marker pins above (the gate, the scalar shape, the predicate, the `done` contract, the switch values).
- Integration: the pending-set parity, the no-hydration pin, the off-loop pin, the cancel pin.
- Coverage: **>90%** on this task's new/modified code (the whole gate — `uv run pytest --cov=app --cov-report=term-missing` at the phase's final pass).
## Completion Criteria
- [ ] the pre-scan runs on a progress caller with the switch truthy, is skipped for `progress=None` callers, and the `0` switch runs the phase-193 walk-position path (the marker pins).
- [ ] the scalar SELECT's column list is exactly `[documents.file_hash]` with no entity hydration (the integration pins).
- [ ] `done`/`total` are the pending position on a stable KB (the parity pin: `files_total == summary.added + summary.updated`).
- [ ] the pre-scan span never parks the loop; a cancel during it lands `cancelled` (the integration pins).
- [ ] `summary.files`, `seen`, the prune, the date refresh and every log line are byte-identical (the unedited pins green).
- [ ] `BOR_SYNC_PENDING_COUNT` in `.env.example` + the config validator (negative refuses startup with the field named).
- [ ] full test suite **100% passing**; README/AGENTS: the setting is documented (the phase's final pass re-checks both project-wide).
- [ ] no behavior change in completed work (the CLI, the upload job, the switch-`0` path).
.agents/phases/complete/194_sync_pending_count/02_sync_phase_status.md
+43
View File
@@ -0,0 +1,43 @@
# Task 02 — The `"scan"` phase + `scan_done` / `scan_total` (D3)
**Phase:** `194_sync_pending_count` · **Source:** owner-confirmed A2 — the pre-scan is a visible phase.
**Story:** n/a
## Objective
The status half of D1: `SyncStatus` carries the scan phase (`"scan"`) and the scan position (`scan_done` / `scan_total`), the `GET /api/sync/status` payload gains the two keys ADDITIVELY, and every docstring names the new machine stage — the phase-187 forced/throttled sampling discipline extended to the scan transition.
## Work
1. `app/api/sync.py` — `SyncStatus`:
- `phase: Literal["scan", "import", "overview", "summaries"] | None = None` (additive value — the machine: prelude `null` → `"scan"` (set immediately before `import_sources`, REPLACING today's `_status.phase = "import"` at L880) → `"import"` (flipped by the FIRST `progress` call while `phase == "scan"` — the flip lives in the existing `_hook` closure: `if _status.phase == "scan": _status.phase = "import"`) → `"overview"` → `"summaries"`; the no-gap summaries skip still keeps the run in `"import"`);
- `scan_done: int = 0` / `scan_total: int = 0` (placed after `files_total` — reset with the run in the `_run_sync` reset block L664–679 region, kept at EVERY terminal like the file counts — the phase-64 keep-final-values convention);
- the class docstring: the phase-64 paragraph (the `files_done`/`files_total` clause now names the pending-set semantics — "the hook's pending position (phase 194: the pre-scan's predicted change set)"), the phase-98 paragraph (the machine gains the `"scan"` stage), a new phase-194 paragraph (the scan fields, the reset/keep convention, the `null` `current_file` during the scan, the empty-KB degenerate clause — the phase stays `"scan"` until the next transition/terminal, the UI renders bare "Scanning…").
2. `app/api/sync.py` — `_run_sync`:
- the reset block: `scan_done = 0` / `scan_total = 0` with the run;
- the pre-`import_sources` point (L880): `_status.phase = "scan"` + the phase-187 FORCED `_sample_rss()` (a phase transition — the forced-sample convention, replacing the current `= "import"` set + forced sample);
- a new `_scan_hook(scan: int, total: int)` closure passed as `on_scan_progress`: sets `scan_done` / `scan_total` + calls the UNFORCED `_sample_rss()` (the per-file throttled convention — one `/proc` read per 5 s, never one per file, LOCKED A4's throttle discipline);
- the `_hook` (progress) closure: the flip clause (above) + a FORCED `_sample_rss()` on the flip (the scan→import transition is a phase transition);
- the `import_sources` call (L881–893): `on_scan_progress=_scan_hook` added;
- the terminal paths (`sync: done/cancelled by user/failed` — L1112/L1133 region): UNTOUCHED in shape (the `files_done=… files_total=…` fields keep their names — now carrying the pending counts; the scan fields keep their final values beside them — the keep-final convention);
- the module docstring: the phase-machine paragraph (L188–200 region — the `"scan"` stage + the flip point), the phase-64 progress paragraph (L100–115 region — the `files_done`/`files_total` semantic shift), the phase-187 sampler paragraph (the scan transition's forced sample + the per-file scan hook's throttled sample), the cancel paragraph (a cancel during the pre-scan span: the `to_thread` worker runs to completion, the run stops at the next boundary — the phase-134/179 contract, the `CancelledError` clause's two paths unchanged).
3. `app/api/sync.py` — `GET /api/sync/status`:
- the payload dict: `"scan_done"` / `"scan_total"` added after `"files_total"` (ADDITIVE ONLY — every pre-existing key keeps its name, type, and position; the phase-187 additive-only discipline);
- the endpoint docstring: the new fields' paragraph (the scan position — 0/0 before the scan starts and in terminal states the final values are kept; `current_file` null during the scan).
4. `tests/integration/` —
- the PHASE MACHINE sequence (a real app, a small git-source fixture with one changed file): a held first status GET (the prelude — `phase: null`, `scan_*: 0/0`) → a `phase: "scan"` frame with `scan_total` set and `scan_done` climbing (≥2 distinct positions on a multi-file fixture) → a `phase: "import"` frame (the flip — `files_total` = the predicted count) → the terminal (`phase: null`, the scan counts KEPT beside the final file counts — the keep-final pin);
- the PAYLOAD key-set pin: the phase-193 key set + `scan_done` / `scan_total` (the additive pin — every pre-existing key byte-identical in name/type);
- the FORCED samples: a forced `_sample_rss` at the pre-`import_sources` set (the `"scan"` transition) and at the flip (the `"import"` transition) — the phase-187 sampling-pin pattern (`test_sync_peak_memory_visible.py` convention);
- the DEGENERATE empty-KB run: the phase reports `"scan"` (no `on_scan_progress` fires — `scan_*` 0/0), never `"import"` (no progress call), the terminal clears the phase (the pin names the D1 degenerate clause);
- the CANCEL-during-scan terminal (a delayed pre-scan, the task-01 cancel pin's app-side twin): `state: "cancelled"`, the phase cleared, the scan counts kept.
5. Verify: `uv run pytest tests/integration -v` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Integration: the phase-machine sequence, the payload key-set, the forced/throttled samples, the degenerate + cancel terminals.
- Coverage: **>90%** on this task's new/modified code.
## Completion Criteria
- [ ] the machine is `null → "scan" → "import" → "overview" → "summaries"` with the flip on the first progress call (the integration sequence pin).
- [ ] `scan_done` / `scan_total` reset with the run, are kept at every terminal, and are additive-only in the payload (the pins).
- [ ] the phase-187 sampling discipline holds (forced at the scan + import transitions, throttled per-file in `_scan_hook`) (the pins).
- [ ] every pre-existing log line, payload key and terminal behavior is byte-identical (the unedited pins green).
- [ ] full test suite **100% passing**; no user-visible doc changes this task (the phase's final pass reviews README/AGENTS).
- [ ] no behavior change in completed work.
.agents/phases/complete/194_sync_pending_count/03_ui_scan_label.md
+35
View File
@@ -0,0 +1,35 @@
# Task 03 — The `fmtSyncPhaseLabel` scan branch (D4)
**Phase:** `194_sync_pending_count` · **Source:** owner-confirmed A2 — the visible `Scanning… (n/total)` label.
**Story:** n/a
## Objective
The UI half of D1/D3: `fmtSyncPhaseLabel` (the phase-aware wrapper) gains ONE explicit `"scan"` branch — `Scanning…` + the `(n/total)` count part in the SAME count slot the import phase's `N remaining` uses — while `fmtSyncLabel`'s arithmetic stays BYTE-IDENTICAL and the A4 untruncated placement + the phase-187 peak clause flow through untouched.
## Work
1. `frontend/assets/sources.js` — `fmtSyncPhaseLabel` (the decision tree + its docstring, L157–167 region):
- the new `"scan"` branch: main text `Scanning…` (NO file — `current_file` is null in the phase; the branch must not read it), count part `(n/total)` with `n = scan_done`, `total = scan_total` — rendered in the SAME count slot (the `{ main, count }` part shape the import phase's `N remaining` uses, so `setSyncLabel`'s hidden-toggle + the A4 untruncated placement — button title + `#sync-progress` — carry `Scanning… (n/total)` automatically); the `total === 0` guard → NO count part (the phase-64 contract, the empty-KB clause — bare "Scanning…");
- the wrapper consumes the scan pair the way it already consumes the summaries pair (`summaries_done` / `summaries_total` / `current_summary` — the `phase` value selects which pair the branch reads); the status call sites (L1121–1127, L1449–1458, L1621–1652 region) forward `scan_done` / `scan_total` alongside the existing fields (the small caller pass-through — the summaries-forwarding pattern);
- the `null` prelude + `"import"` fallback clause now covers ONLY those two (the docstring's decision-tree paragraph updated: the new branch is named, the fallback narrowed, the phase-193 `N remaining` clause kept);
- the walk position surviving as `(n/total)` is HONEST for a scan — it is labeled "Scanning", never "Syncing" (TODO L6's objection was the sync count wearing the total's shape — the docstring says so);
- `fmtSyncLabel` (L1043–1071): UNTOUCHED (the `N remaining` arithmetic + its guards are byte-identical — the phase-193 pins stay unedited); the chunk / image parts, `setSyncLabel`, the terminal `counts_line`: UNTOUCHED;
- the phase-187 peak clause (`· peak N MB` on the `#sync-progress` counts line — `fmtSyncPeakClause`): UNTOUCHED (it lands on the scan label's line while the scan is visible — no new element, no new live region).
2. `tests/unit/` — the JS marker pins (the sources-JS pin-module convention, the phase-193 marker style):
- the `"scan"` branch marker (the `Scanning…` main + the `phase === "scan"` selection in the decision tree);
- the `(n/total)` shape marker (the exact template over `scan_done` / `scan_total` — the source marker asserts the expression);
- the `total === 0` guard marker (no count part before the scan knows the total);
- the caller pass-through marker (the call sites forward the scan pair);
- the UNTOUCHED markers: `fmtSyncLabel`'s `N remaining` expression (the phase-193 pin — byte-identical), the chunk / image / summaries part builders, `setSyncLabel`, the `counts_line` helper.
3. Verify: `uv run pytest tests/unit -v` green; `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- Unit: the marker pins above (the branch, the shape, the guard, the pass-through, the untouched set).
- E2E: task 04 (the retargeted pins) + task 05 (the phase arms — the scan label through the real UI).
- Coverage: **>90%** on `app/` (the task is frontend — the app-side gate stays green at the phase's final pass).
## Completion Criteria
- [ ] the `"scan"` branch renders `Scanning…` + `(n/total)` in the count slot with the `total === 0` guard (the marker pins).
- [ ] `fmtSyncLabel`'s `N remaining` arithmetic is byte-identical (the unedited phase-193 pins green).
- [ ] the A4 untruncated placement + the phase-187 peak clause are untouched (the unedited pins green).
- [ ] full test suite **100% passing**; no user-visible doc changes this task (the phase's final pass reviews README/AGENTS).
- [ ] no behavior change in completed work (the prelude / import / overview / summaries labels byte-identical).
.agents/phases/complete/194_sync_pending_count/04_retarget_pinned_suites.md
+38
View File
@@ -0,0 +1,38 @@
# Task 04 — The pinned suites move (D5)
**Phase:** `194_sync_pending_count` · **Source:** the D5 retarget list.
**Story:** n/a
## Objective
Every existing pin that asserts the phase-193 WALK-position semantics (or the pre-phase statement counts the pre-scan now disturbs) is retargeted to the pending-set semantics — and every pin that is unaffected by the change (fresh-fixture count pins, the upload's bare-label pins, the terminal-line pins) is verified green UNEDITED. The full suite ends green.
## Work
The definitive list (per suite: decide from the fixture's KB state — a FRESH-DB fixture has pending == walk, so its count pins survive unedited; a RE-SYNC fixture's walk-position pins retarget):
1. `tests/e2e/test_sync_remaining_count.py` (phase 193's own suite — the primary retarget):
- the docstring's `files_done` / `files_total` clause (the "the 1-based …" wording, L15–16 region) retargets to the PENDING-set semantics (the pre-scan's predicted change set; the phase-193 walk wording moves to the kill-switch history note);
- the status key-set pin (the "the phase-64 key set" clause) moves to the phase-64 set + `scan_done` / `scan_total` (the additive pin);
- the arm's phase-sequence pins: the prelude → `import` sequence gains the `"scan"` stage (a `phase: "scan"` frame with `scan_total` set before the first `files_total`-bearing frame — the D3 machine); the pre-scan's label frames (`Scanning… (n/total)`) are pinned where the arm's held-first-GET / pre-first-file region stands;
- the count frames: on the suite's FRESH-DB fixture, pending == walk — the `1 … N−1` frames and the "hidden at the final file" pin survive (the phase-193 "Notable" deviation note retargets to the pending-position wording); ADD the re-sync arm if the suite's fixture allows it (a second sync of the same repo: no count part — `files_total = 0`, the phase-193 A1 exception now driven by the empty pending set) — otherwise the re-sync pin lives in the task-05 suite;
- the upload arm (arm c): UNEDITED (the upload status endpoint gains no scan fields — phase 90 A2 stands).
2. `tests/integration/test_import_fast_path_columns.py` (phase 185 — the per-file SELECT pins): inspect the invocation. If it calls `import_sources` WITHOUT `progress` → the pre-scan never runs → verify green UNEDITED. If WITH `progress` → retarget the "exactly one document-wide `SELECT` per unchanged file" pin to distinguish the pre-scan's scalar statement (column list exactly `[documents.file_hash]` — the task-01 shape pin) from the fast-path ROW statement (content deferred — the phase-185 pin intact); the "one PK follow-up for the backfill" pin is UNTOUCHED either way (the pre-scan's scalar is a `(source, path)`-keyed statement, not a PK follow-up).
3. `tests/integration/test_import_session_retention.py` (phase 184 — the bounded/unbounded retention pins): the pre-scan hydrates NO entities (the task-01 no-hydration pin) → the retention bound is UNCHANGED; verify green (if the test's fixture invokes with `progress`, confirm the worker-thread scalar statements disturb no retention assertion — the scalar loads no row).
4. `tests/integration/test_sync_git_freeze.py` (phase 179 — the loop-freeze pins): the pre-scan span is itself off-loop (the task-01 off-loop pin lives in its own module, the phase-179 pattern) — verify this suite's git-span pins stay green; if its measurement window now overlaps the pre-scan, extend it with the pre-scan heartbeat (the loop must stay responsive through BOTH off-loop spans).
5. `tests/e2e/test_sync_chunk_progress.py` (the `Syncing… chunkprog/big.md 1 remaining · chunk n/m` pins): the fixture is a fresh-DB import → pending == walk → verify green UNEDITED (the `1 remaining` part reads identically — the pending denominator equals the walk denominator on a fresh KB).
6. `tests/e2e/test_sync_label_mobile.py` (the `IMPORT_FULL_RE` pin + the label docstring): inspect the fixture's KB state — fresh-DB → verify green UNEDITED; any re-sync walk-position pin retargets (the count part absent on a no-change re-sync).
7. `tests/e2e/test_sync_upload_progress.py` + `tests/e2e/test_archive_upload_sources.py` (the upload's "no `N remaining` count" pins — phase 90 A2): the upload job has no pre-scan and no scan fields → verify green UNEDITED.
8. `tests/e2e/test_sync_session_reset.py`, `tests/e2e/test_fast_path_deferred_content.py`, `tests/e2e/test_sync_summary_visibility.py`, `tests/e2e/test_sync_image_progress.py`, `tests/e2e/test_sync_progress_box.py`, `tests/e2e/test_sync_status_freshness.py`, `tests/e2e/test_sync_peak_memory_visible.py`, `tests/e2e/test_smoke.py` — the sync-lineage sweep: run each in isolation (`--no-cov`), verify green; a pin that breaks on the new `"scan"` stage of the phase sequence or on the pre-scan's seconds (e.g. a freshness window that now spans the scan span) retargets minimally (the window/sequence pin names the scan stage — never the arithmetic).
9. `AGENTS.md` / `README.md` are NOT edited this task (the phase's final pass owns the docs — task 05).
10. Verify: `uv run pytest -v` (unit + integration) green; the ISOLATED `--no-cov` sweep of items 1–8 green (the phase-193 14-suite convention, extended by this phase's suite in task 05); `uv run ruff check . && uv run pyright` clean.
## Testing & Quality
- The retargeted pins (the pending-set wording, the key-set pin, the phase-sequence pins).
- The verified-unedited pins (fresh-fixture count pins, the upload's bare-label pins, the terminal `counts_line` pins, the phase-184/185/179 pins).
- Coverage: **>90%** on `app/` (the gate stays green — this task edits tests + docstrings).
## Completion Criteria
- [ ] the retargeted list above is settled: every broken pin retargeted with the reason in its docstring, every unaffected pin green UNEDITED (the sweep).
- [ ] `tests/e2e/test_sync_remaining_count.py` pins the new machine (the `"scan"` stage + the key-set + the pending-position wording) and its fresh-fixture count frames survive.
- [ ] the terminal `counts_line` pins, the chunk / image / summaries pins and the upload pins are green UNEDITED.
- [ ] full test suite **100% passing** (unit + integration + the isolated E2E sweep).
- [ ] no behavior change in completed work (only test files + their docstrings move).
.agents/phases/complete/194_sync_pending_count/05_e2e_and_gates.md
+42
View File
@@ -0,0 +1,42 @@
# Task 05 — The phase's E2E suite + gates + docs + commit
**Phase:** `194_sync_pending_count` · **Source:** AGENTS rule 4 — "One Story, One Phase" (the owner-directive phase's own suite).
**Story:** n/a
## Objective
The owner-visible proof through the real UI: the count counts down the PREDICTED change set (the owner's prod case — K of N changed), the scan phase is visible and honest, the kill switch rolls back to phase 193, and the no-change / first-sync / empty-KB edges behave per D1 — then the full gates, the docs pass, and the atomic commit.
## Work
1. `tests/e2e/test_sync_pending_count.py` (NEW — run in isolation: `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov`):
- the house fixture convention (the phase-193/107 suite shape): a real git repo over the `file://` transport, the mock-LLM proxy with the house per-request delay, a unique E2E port pair (check the existing allocations in the sibling suites — the one-suite-at-a-time rule makes collisions harmless, but pick a free pair anyway), a `file://`-repo N = 8 importable files (keep the suite under ~2 min; ≥2 multi-chunk files so the chunk part coexists, the phase-129 convention);
- the module app: the default `BOR_SYNC_PENDING_COUNT` (1); a second module app for the kill-switch arm (`BOR_SYNC_PENDING_COUNT=0`) — the phase-184 two-module-app convention;
- **arm a — the first sync (fresh KB):** the phase sequence `null` → `"scan"` (the label `Scanning… (n/8)` — the position climbs: pin ≥2 distinct `scan_done` values at `scan_total = 8`; `current_file` null in the phase) → `"import"` (the count part `N remaining` with `files_total = 8` — pending == walk on a fresh KB, the phase-193 parity: the frames count down, hidden at the final file) → the terminal (`phase: null`, `scan_done = scan_total = 8` KEPT beside `files_done = 8`, the `sync: done` counts line `8 added`);
- **arm b — the incremental sync (the owner's prod case — the phase's heart):** modify K = 2 of the 8 (the fixture's changed-set), re-sync: the scan phase climbs `(n/8)` (the full walk is scanned), then the import: the count part reads `2 remaining` and HOLDS while `current_file` moves over UNCHANGED files (pin: ≥2 consecutive unchanged-file frames at `2 remaining` — the count does not advance on them), advances to `1 remaining` on the first predicted file, HIDES on the final predicted file (the phase-193 A1 edge — the main part names it), terminal `2 updated`;
- **arm c — the no-change re-sync:** the scan completes, the import renders BARE `Syncing… <file>` (NO count part — `files_total = 0`, the phase-64 guard), the terminal `0 added · 0 updated` (the `counts_line` byte-identical shape, the phase-64 pin);
- **arm d — the kill switch:** the second module app (`BOR_SYNC_PENDING_COUNT=0`), arm b's fixture: NO `"scan"` phase (the machine is `null` → `import` — the phase-193 machine), the count part counts down the WALK (`8 remaining` → hidden at the final file — the phase-193 semantics, the rollback pin), the terminal `2 updated`;
- **arm e — the empty KB:** a repo with zero importable files: `phase: "scan"` with `scan_*` 0/0 (bare `Scanning…` — the `total === 0` guard, NO count part), never `phase: "import"`, the terminal clears the phase (the D1 degenerate pin);
- **the phase-187 peak clause** (the `· peak N MB` / `fmtSyncPeakClause` trailing text on the `#sync-progress` counts line) is visible while the scan label is showing (the clause lands on the scan line — no new element, the phase-187 pin);
- **the upload arm:** a background upload run renders its bare label with NO file and NO count part (the phase-90 A2 pin — the `test_archive_upload_sources.py` convention; the upload status endpoint carries no scan fields);
- the `#sync-progress` A4 untruncated placement (the button `title` + `#sync-progress` text equal the full `Scanning… (n/total)` label, the phase-64 A4 pin).
2. The full gates (the phase's final pass — run in order, all green before the commit):
- `uv run pytest -q --cov=app --cov-report=term-missing` → exit 0, coverage **>90%** on `app/`;
- `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → all arms pass in ISOLATION;
- the isolated `--no-cov` sweep of the sync-lineage suites (task 04's list + this suite): `test_sync_remaining_count`, `test_sync_label_mobile`, `test_sync_chunk_progress`, `test_sync_image_progress`, `test_sync_progress_box`, `test_sync_summary_visibility`, `test_archive_upload_sources`, `test_sync_upload_progress`, `test_sync_status_freshness`, `test_sync_peak_memory_visible`, `test_sync_session_reset`, `test_fast_path_deferred_content`, `test_sync_git_freeze`, `test_smoke`;
- `uv run ruff check . && uv run pyright` clean.
3. The documentation & consistency pass (AGENTS rule + the phase's D-docs clause):
- `README.md` in full: the sync section's running-label clause — the phase-193 `N remaining` example becomes the PREDICTED-change-set one (the `scan` phase + `Scanning… (n/total)` + `N remaining` over the pending set + the no-change/first-sync/empty-KB edges + the `BOR_SYNC_PENDING_COUNT` setting: `1` default, `0` = the phase-193 walk-position rollback, negative = startup refusal); the `.env.example` entry was added in task 01 — verify it is documented here too; every pre-existing inconsistency found in the full review is fixed (the phase-193 docs-pass convention);
- `AGENTS.md` in full: verify whether any clause names the running count part (phase 193 verified one does NOT — if still absent, do NOT add one; if one exists, it gains the pending-set wording + the kill switch); the `BOR_SYNC_PENDING_COUNT` row joins wherever the settings documentation lives.
4. The atomic commit (AGENTS rule 8): ONE Conventional Commit, `--no-gpg-sign`, the phase directory moved `todo/` → `complete/` in the SAME commit (the house convention — the phase-193 commit shape: the `phase: 194_sync_pending_count` message with the verified-evidence block). Suggested subject: `feat(sync): count down the sync's predicted change set (pre-scan)`.
## Testing & Quality
- E2E: this suite, isolated (the six arms + the peak-clause + the A4 + the upload pins).
- Gates: the full suite + coverage >90% + the 15-suite sweep + lint/types (above).
- Docs: README + AGENTS reviewed in full, every inconsistency fixed.
## Completion Criteria
- [ ] arm b proves the owner's prod case through the real UI (the count holds at `2 remaining` over the unchanged files, hides at the final predicted file, terminal `2 updated`).
- [ ] arms a/c/d/e pin the first-sync parity, the no-change no-count edge, the kill-switch rollback and the empty-KB degenerate.
- [ ] the peak clause + the A4 placement pins hold on the scan label.
- [ ] all gates green (above) — full suite 100%, coverage >90%, the sweep, lint/types.
- [ ] README + AGENTS accurate and consistent with the whole project (the final pass).
- [ ] ONE atomic Conventional Commit with `--no-gpg-sign`, the phase directory in `complete/`.
.agents/reports/194_sync_pending_count/194_sync_pending_count__00_phase.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__00_phase.a1.md
+19
View File
@@ -0,0 +1,19 @@
**Phase 194 — final verification pass: all green.**
**Verified (all 5 tasks already complete; nothing re-implemented needed)**
- `app/rag/importer.py`: `_prescan_pending` + `_pending_hash_stmt` (scalar `documents.file_hash`, own session, ONE `asyncio.to_thread` span, REPLACES the stats pre-walk), `on_scan_progress`, `progress` pair = pending position, `0` branch = byte-identical phase-193 walk.
- `app/config.py`/`settings_catalog.py`/`.env.example`: `sync_pending_count=1` + validator; `app/api/sync.py`: `"scan"` phase, `scan_done`/`scan_total` (additive after `files_total`, kept at every terminal), forced/throttled RSS split; `frontend/assets/sources.js`: `fmtSyncPhaseLabel` scan branch first-arm, `fmtSyncLabel` untouched.
**Tests / lint / coverage (exact commands)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4835 passed**, `app/` 99% (7240 stmts / 16 miss), exit 0
- `uv run pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → **8 passed in 150.81s** (phase suite, isolated)
- 14 sync-lineage E2E suites, one pytest process each (`test_sync_remaining_count`, `…_chunk_progress`, `…_progress_box`, `…_peak_memory_visible`, `…_label_mobile`, `…_upload_progress`, `test_archive_upload_sources`, `…_image_progress`, `…_summary_visibility`, `…_status_freshness`, `test_smoke`, `test_fast_path_deferred_content`, `test_sync_session_reset`, `test_sync_git_freeze`) → **all exit 0**
- `uv run ruff check .` → All checks passed; `uv run pyright` → **0 errors**
**Completion criteria** — all met: `N remaining` = predicted change set (E2E arms 1/2/5, first import counts the walk, no-change → no count part); `scan` phase visible + flips to `import`, scan pair reset/kept at terminals (`test_sync_scan_phase.py`); `BOR_SYNC_PENDING_COUNT=0` walk semantics ("7 remaining" arm) + **negative refuses startup naming the field (verified live by importing `app.main`)**; pre-scan off-loop + cancel lands `cancelled` (integration pins); CLI/upload byte-identical (unedited pins green); payload additive (`_PRE_187_STATUS_KEYS | _MEMORY_KEYS | _SCAN_KEYS`); suite green, coverage >90%, lint/types clean.
**Documentation** — 3 fixes: README sync "States" bullet had an **unclosed parenthesis** (pre-existing, 193/194) and the scan-label "total is 0" clause was ambiguous about a `=0` run (it still counts the walk once the import starts); **AGENTS.md rule 17 had a stray `)`** (pre-existing from phase 191). Also verified consistency both directions for all 87 settings ↔ README ↔ `.env.example` ↔ settings catalog, the pages table ↔ `app/main.py` routes, no stale `BOR_` names, and README/AGENTS claims through phase 194.
**Notes/deviations** — no live-infra changes (prod sync knob ships default-on, no deployment touched); no git index/history commands run (all changes left in the working tree for the harness); commit criterion left to the harness as instructed.
**Next pending phase:** none — `.agents/phases/todo/` holds only `194_sync_pending_count`, which is now ready to move to `complete/`.
.agents/reports/194_sync_pending_count/194_sync_pending_count__00_phase.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__00_phase.a1.validate
+151
View File
@@ -0,0 +1,151 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 242 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7240 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/194_sync_pending_count/194_sync_pending_count__01_prescan_importer.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__01_prescan_importer.a1.md
+14
View File
@@ -0,0 +1,14 @@
Task 01 is complete and green.
**Server-side (phase 194 task 01)**
- `app/config.py` — `sync_pending_count: int = 1` (`BOR_SYNC_PENDING_COUNT`), negative refuses startup naming the field; `app/core/settings_catalog.py` — the `Import & sources` row.
- `app/rag/importer.py` — new `_pending_hash_stmt` (scalar `SELECT documents.file_hash`, `(source, path)` predicate, no entity/identity-map load) and `_prescan_pending` (per-file `_file_hash` + one scalar read → `{(source, rel): 1-based pending position}`, its own session, `on_scan_progress` per file). `import_sources` gained `on_scan_progress`, reads the switch **once** before the loop, and REPLACES the phase-64 stats pre-walk with ONE `asyncio.to_thread` pre-scan; `progress` now reports the pending position (holds across unchanged files), `0` runs the byte-identical phase-193 walk-position branch, `progress=None` pays nothing. `summary.files`, `seen`, prune, date refresh, log lines untouched.
- Docs: `.env.example`, `README.md` (label semantics + a phase-194 bounds bullet + the settings table), `AGENTS.md` (new rule 18).
**Tests**
- New: `tests/unit/test_sync_pending_count.py` (13 — gate both ways, read-once + `to_thread` markers, exactly-`file_hash` projection, 4 predicate states + ignore parity, unreadable arm, scan-hook per root/empty/worker-thread, `done` position, CLI byte-identity) and `tests/integration/test_sync_pending_count.py` (7 — fresh/unchanged/incremental parity `total == added+updated`, 2-file count-down, one-column + `identity_map == 0`, off-loop heartbeat + thread identity, cancel-at-the-await).
- Retargeted the two phase-64/129 pair pins in `tests/unit/test_importer.py`; extended `test_config.py` / `test_settings_catalog.py`.
**Gates**: `ruff check .` clean · `pyright` 0 errors · `pytest --cov=app` **exit 0**, app coverage **99 %** (importer 99 %) · 20 sync/progress E2E suites + smoke green.
**Deferred by design**: the `scan` status phase / `Scanning…` label (tasks 02–03), the phase's E2E file (task 05). Task 04's pinned suites needed **no** retargeting — they run fresh-import fixtures where the walk *is* the work. Nothing committed (per instructions), no task files touched.
.agents/reports/194_sync_pending_count/194_sync_pending_count__01_prescan_importer.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__01_prescan_importer.a1.validate
+150
View File
@@ -0,0 +1,150 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
............................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 231 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7229 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/194_sync_pending_count/194_sync_pending_count__02_sync_phase_status.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__02_sync_phase_status.a1.md
+17
View File
@@ -0,0 +1,17 @@
All gates green. Final state:
**Implemented (task 02 — D3, the status half)**
- `app/api/sync.py`: `SyncStatus.phase` → `Literal["scan","import","overview","summaries"]|None`; new `scan_done`/`scan_total` (after `files_total`, reset with the run, kept at every terminal); `_run_sync` sets `"scan"` immediately before `import_sources` (replacing `= "import"`) with a FORCED `_sample_rss`, adds ONE `_scan_hook` (unforced/throttled sample) passed as `on_scan_progress`, and the `_hook` closure now flips `scan → import` on the first progress call with a forced sample; payload gains `"scan_done"`/`"scan_total"` ADDITIVELY; module / class / endpoint / `_run_sync` docstrings + the `_RSS_SAMPLE_INTERVAL_S` note + the three terminals' keep-final comments updated. Terminals, log lines and every pre-existing key untouched.
- New pin suite `tests/integration/test_sync_scan_phase.py` (7): real-run scan frames (≥2 positions, `current_file` null) + both pairs kept at the terminal and **divergent** on the one-changed-file arm (scan 3/3 vs files 1/1, `updated=1`); choreographed flip frame carrying the predicted `files_total`; additive key set idle→mid-run→terminal; forced/throttled sample sequence `null→scan→import→overview→summaries→null` with exactly ONE `/proc` read inside the scan phase; empty-KB arm on both sides; cancel-during-scan terminal (counts kept, nothing indexed, `sync: cancelled by user …` prefix intact); `BOR_SYNC_PENDING_COUNT=0` = `0/0` scan + WALK file count.
- Retargeted stale pins (defects my change exposed): `test_sync_api.py` (idle dict + both `_PRE_187_STATUS_KEYS` assertions, `FakeImportSources`/`_GatedImport`/`failing_import` accept `on_scan_progress`, 409-concurrency pin now waits `"scan"` — a canned hook-less import legitimately stays pre-import), `test_sync_git_freeze.py`, `test_sync_button.py` (idle shape, key set, the two sampling pins now count the scan transition: 3 reads start→scan→flip), `test_sync_peak_memory_visible.py` (browser key-set pin + `_COUNT_KEYS`).
**Gates (exact)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4822 passed**, TOTAL **99 %**, `app/api/sync.py` **100 % (0 miss)**, exit 0.
- `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors.
- E2E regression (30 modules, all passed): `test_sync_remaining_count`, `test_sync_button`, `test_sync_progress_box`, `test_sync_label_mobile`, `test_sync_cancel`, `test_sync_status_freshness`, `test_sync_peak_memory_visible`, `test_sync_chunk_progress`, `test_sync_upload_progress`, `test_archive_upload_sources`, `test_sync_summary_visibility`, `test_sync_image_progress`, `test_sync_changed_only`, `test_sync_session_reset`, `test_fast_path_deferred_content`, `test_sync_model_down`, `test_sync_git_freeze`, `test_force_push_resync`, `test_source_name_labels`, `test_local_directory_sources`, `test_source_ignore_paths`, `test_folder_summary_voice`, `test_incremental_folder_summaries`, `test_upload_no_scan`, `test_git_source_branch`, `test_hidden_folders_toggle`, `test_prune_cascade_cleanup`, `test_git_log_stream_dates`, `test_kb_tree`, `test_smoke`.
**Docs**: README (`phase` machine + additive `scan_done`/`scan_total` clause; the phase-187 sampling bullet now lists `scan → import → overview → summaries`) and `AGENTS.md` rule 18 (status half: machine, flip-in-`_hook` rule, additive chain, forced-vs-throttled sampling, degenerate edge, new pinned suites). No new env var; nothing committed (harness commits).
**Notes/deviations**: D3 kept exactly (the flip lives in the existing per-file hook, no extra importer callback), so a canned import that fires no hook stays in `"scan"` — that is why the pre-existing 409 pin retargets. The real-run arm pins terminal values instead of an `"import"` frame because an in-process import never yields the loop (documented in the test module docstring); the flip frame gets a deterministic choreographed pin. `Scanning…` label copy intentionally left to task 03.
**Next pending task**: `.agents/phases/todo/194_sync_pending_count/03_ui_scan_label.md` (`fmtSyncPhaseLabel` scan branch).
.agents/reports/194_sync_pending_count/194_sync_pending_count__02_sync_phase_status.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__02_sync_phase_status.a1.validate
+150
View File
@@ -0,0 +1,150 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 73%]
........................................................................ [ 74%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 91%]
........................................................................ [ 92%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 97%]
........................................................................ [ 98%]
...................................................................... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 242 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7240 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/194_sync_pending_count/194_sync_pending_count__03_ui_scan_label.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__03_ui_scan_label.a1.md
+17
View File
@@ -0,0 +1,17 @@
All gates green. Final state:
**Implemented (task 03 — D4, the UI half)**
- `frontend/assets/sources.js`: `fmtSyncPhaseLabel` gains ONE `"scan"` branch as its **first** arm — main `Scanning…` (never a file; the branch never reads `current_file`) + count `(scan_done/scan_total)` in the SAME count slot the import `N remaining` uses, with the phase-64 `scan_total > 0` guard (0/0 → bare `Scanning…`) and `chunk`/`image` empty; `fmtSyncLabel`'s `N remaining` arithmetic, the chunk/image/summaries builders, `setSyncLabel`, `fmtSyncResult`, `fmtSyncPeakClause` untouched (whole-status pass-through at both call sites already carries the scan pair — comments + docstrings now name it); module header, phase-98 header, the section decision tree and the wrapper docstring updated (fallback narrowed to the null prelude + `import`, TODO L6 honesty clause added).
- New pin suite `tests/unit/test_sync_scan_label_ui.py` (13): branch order, the exact `(n/total)` shape + guard, no-file/no-other-pair reads, the tick + re-attach pass-through, the untouched builder/writer set, docstring/header markers, plus a **node run of the real label path** (`Scanning… (3/9)` identical on main/count/title/box/aria; bare `Scanning…` at `0/0`; every other phase byte-identical).
- No server-side change (tasks 01/02 already shipped the phase + payload keys).
**Gates (exact)**
- `uv run pytest --cov=app --cov-report=term` → **4835 passed**, TOTAL **99 %** (`app/api/sync.py` 100 %), exit 0.
- `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors · `node --check sources.js` → OK.
- E2E regression (32 modules, all passed, none edited): `test_sync_remaining_count` (phase-193 pins byte-identical), `test_sync_progress_box`, `test_sync_label_mobile`, `test_sync_chunk_progress`, `test_sync_summary_visibility`, `test_sync_image_progress`, `test_sync_status_freshness`, `test_sync_button`, `test_sync_cancel`, `test_sync_peak_memory_visible`, `test_sync_upload_progress`, `test_upload_no_scan`, `test_archive_upload_sources`, `test_sync_model_down`, `test_sync_changed_only`, `test_sync_session_reset`, `test_sync_git_freeze`, `test_fast_path_deferred_content`, `test_force_push_resync`, `test_prune_cascade_cleanup`, `test_local_directory_sources`, `test_source_ignore_paths`, `test_hidden_folders_toggle`, `test_kb_tree`, `test_git_source_branch`, `test_git_log_stream_dates`, `test_source_name_labels`, `test_folder_summary_voice`, `test_incremental_folder_summaries`, `test_answer_images`, `test_pdf_support`, `test_smoke`.
**Docs**: README (the running-label bullet now leads with `"Scanning… (n/total)"`, the phase-194 bullet names the branch + its A4 placement) and `AGENTS.md` rule 18 (new "UI half" clause: first-arm rule, no-file/no-chunk-image, `total === 0` bare label, honesty rule, whole-status pass-through, untouched set, the new pin module).
**Notes/deviations**: the `(n/total)` branch is placed before `overview`/`summaries` so every pre-existing branch slice in the phase-128/152 pins stays byte-identical; the D4 "caller pass-through" needed no argument-list change (the whole `status` object is already the 5th arg — the same carrier as the summaries/chunk/image pairs), so it is pinned + documented rather than re-listed. No pinned E2E suite went stale under this change, so nothing was retargeted here (task 04 stays as authored).
**Next pending task**: `.agents/phases/todo/194_sync_pending_count/04_retarget_pinned_suites.md`.
.agents/reports/194_sync_pending_count/194_sync_pending_count__03_ui_scan_label.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__03_ui_scan_label.a1.validate
+151
View File
@@ -0,0 +1,151 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 242 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7240 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/194_sync_pending_count/194_sync_pending_count__04_retarget_pinned_suites.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__04_retarget_pinned_suites.a1.md
+18
View File
@@ -0,0 +1,18 @@
Everything is green. Final verification summary:
**Implemented (task 04 — the D5 pinned-suite retarget)**
- `tests/e2e/test_sync_remaining_count.py` (phase 193's own suite) retargeted: docstring's "Server untouched / phase-64 key set / walk position" clauses now state the pending-set semantics (the phase-193 walk wording kept as the `BOR_SYNC_PENDING_COUNT=0` history); new `EXPECTED_STATUS_KEYS` additive key-set pin (phase-64 set + memory pair + `scan_done`/`scan_total`) asserted on every real tick AND on the served stub; new `_assert_phase_machine_tick_series()` pins the `"scan"` stage (file-less scan ticks carrying `scan_total`, `0/0` null prelude, one-directional sequence, terminal keeping BOTH pairs, `files_total ≤ scan_total`); test 2's pre-first-file region now renders/serves `Scanning… (2/5)`, the bare `Scanning…` (`scan_total===0`) and the null prelude; **new test 4** syncs the same repo twice — run 2's wire pair is `0/0` (empty pending set) while `scan_done == scan_total == 5` is kept at the terminal, terminal line `0 added · 5 unchanged` byte-identical, no count part rendered. `_TickRecorder` gained `wait_for_running` (+ `started_at` ownership) — a recorder started on a previous run's terminal was silently handing that body back (the bug test 4 exposed).
- Verified green **UNEDITED** (pins untouched): upload arm (test 3), `test_sync_chunk_progress`, `test_sync_label_mobile`, `test_sync_upload_progress`, `test_archive_upload_sources`, `test_sync_session_reset`, `test_fast_path_deferred_content`, `test_sync_summary_visibility`, `test_sync_image_progress`, `test_sync_progress_box`, `test_sync_status_freshness`, `test_sync_peak_memory_visible`, `test_smoke`; phase-185 `-230`→`-228` per-file SELECT pins and phase-184 retention bound unaffected (the pre-scan reads a scalar projection through its own session). Docstring-only notes recorded that inspection in `test_sync_chunk_progress.py`, `test_sync_progress_box.py`, `test_import_fast_path_columns.py`, `test_import_session_retention.py`, `test_sync_git_freeze.py` (whose stubbed import means the pre-scan never enters its measurement window — the span's own off-loop/cancel pins live in `test_sync_pending_count.py`).
- No app, README or AGENTS change (item 9: task 05 owns the docs).
**Gates (exact)**
- `uv run pytest --cov=app --cov-report=term-missing` → **4835 passed**, TOTAL **99 %** (`app/api/sync.py` 100 %, `app/config.py` 100 %, `app/rag/importer.py` 99 %), exit 0.
- `uv run ruff check .` → All checks passed · `uv run pyright` → **0 errors** · `node --check frontend/assets/sources.js` → OK.
- Isolated `--no-cov` sweep, items 1/5–8: 13/13 PASS (`test_sync_remaining_count` 97 s with 4 tests). Adjacent sync-lineage batch: 12/12 PASS (`test_sync_button`, `_cancel`, `test_upload_no_scan`, `_changed_only`, `_model_down`, `test_force_push_resync`, `test_prune_cascade_cleanup`, `test_git_log_stream_dates`, `test_folder_summary_voice`, `test_incremental_folder_summaries`, `test_local_directory_sources`, `test_source_ignore_paths`). Items 2–4 integration files re-run isolated: PASS.
**Notable decisions / notes**
- Real-run `phase:"scan"` frames are not observable on a 5-file fixture (single-digit-ms window vs a 100 ms poll), so the real arm pins the machine's *deterministic evidence* while the deterministic `Scanning… (n/total)` frames ride test 2's served statuses and the choreographed server sequence stays in `tests/integration/test_sync_scan_phase.py` (documented in the module docstring).
- One pin deliberately relaxed with a reason: the prelude `0/0` arm is skipped for run 2 (`prelude_zero=False`) because `_run_sync` sets `state="running"` one synchronous statement before resetting the scan pair while `GET /api/sync/status` answers from a threadpool — a pre-existing (phase-64) reset shape, not changed here.
- Two integration suites failed transiently when I ran them concurrently with an E2E batch (shared Postgres + per-test `TRUNCATE`); re-run in isolation they pass — E2E and integration must not overlap on this DB. Three orphaned uvicorns on ports 8997–8999 belong to another session's `dev` container; left untouched.
**Next pending task**: `.agents/phases/todo/194_sync_pending_count/05_e2e_and_gates.md` (the phase's own `tests/e2e/test_sync_pending_count.py`, the docs pass, the atomic commit).
.agents/reports/194_sync_pending_count/194_sync_pending_count__04_retarget_pinned_suites.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__04_retarget_pinned_suites.a1.validate
+151
View File
@@ -0,0 +1,151 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 242 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7240 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK
.agents/reports/194_sync_pending_count/194_sync_pending_count__05_e2e_and_gates.a1.err
View File
.agents/reports/194_sync_pending_count/194_sync_pending_count__05_e2e_and_gates.a1.md
+15
View File
@@ -0,0 +1,15 @@
**Phase 194 / task 05 — complete.**
`tests/e2e/test_sync_pending_count.py` (new, 1,999 lines, 8 arms, own ports 8212/8213 + slow-proxy 8915, two live module apps over the `file://` git transport and a delay-injecting mock model):
1. fresh KB — `null → scan → import → terminal`, scan pair completes and survives the terminal, `N remaining` = phase-193 walk arithmetic (pending == walk), real-run peak clause == payload `rss_peak_mb`;
2. incremental (8 indexed, 2 changed) — `files_total == 2` every running tick, count **holds** at `2 remaining` while `current_file` names unchanged files, absent on the final predicted file;
3. exact scan/null/hold payloads served to the real `sources.js` — `Scanning… (n/total)` on main/count/title/`#sync-progress` only, one `aria-live` box, no `N remaining`/file/chunk in the scan arm, no peak clause on any running label, terminal `0 added · 2 updated · 6 unchanged · peak 1234 MB`;
4. `BOR_SYNC_PENDING_COUNT=0` app on the same fixture — `"7 remaining"` (1 predicted), no `scan` phase, payload carries `scan_done/scan_total` at `0/0`;
5. no-change re-sync — `files_total == 0` on every tick, no count part anywhere, machine still clean;
6. empty source — scan reached, import never, bare `Scanning…`, terminal `0 added · 0 unchanged`;
7. prod-shaped 1,000-file KB — scan position caught **climbing through the running server** (`scan_total == 1000`, `current_file` null), every import tick at `2`, no tick and no label ever walk-sized;
8. upload — bare phase-64 key set (no `phase`/`scan_*`/chunk/image/summary/memory keys), bare label.
Docs: added the browser pin to the README phase-194 bullet and AGENTS.md rule 18 (incl. the terminal-only peak-clause reading of task 05's wording). `.env.example`/README settings row already correct — no other inconsistency found.
Gates: `pytest tests/e2e/test_sync_pending_count.py -v --no-cov` → **8 passed in 146.8s**; sync-lineage sweep of all 15 E2E suites in isolation → **15/15 exit 0**; full `pytest --cov=app` → **exit 0, app coverage 99%** (>90%); `ruff check .` → all checks passed; `pyright` → **0 errors**. No app defect surfaced this task; no git index/history touched.
.agents/reports/194_sync_pending_count/194_sync_pending_count__05_e2e_and_gates.a1.md.status
+1
View File
@@ -0,0 +1 @@
ok
.agents/reports/194_sync_pending_count/194_sync_pending_count__05_e2e_and_gates.a1.validate
+151
View File
@@ -0,0 +1,151 @@
........................................................................ [ 1%]
........................................................................ [ 2%]
........................................................................ [ 4%]
........................................................................ [ 5%]
........................................................................ [ 7%]
........................................................................ [ 8%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 16%]
........................................................................ [ 17%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 34%]
........................................................................ [ 35%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 92%]
........................................................................ [ 93%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........... [100%]
=============================== warnings summary ===============================
.venv/lib/python3.13/site-packages/fastapi/testclient.py:1
/var/home/ducoterra/Projects/Personal/brain_of_reese/.venv/lib/python3.13/site-packages/fastapi/testclient.py:1: StarletteDeprecationWarning: Using `httpx` with `starlette.testclient` is deprecated; install `httpx2` instead.
from starlette.testclient import TestClient as TestClient # noqa
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.13.13-final-0 _______________
Name Stmts Miss Cover
--------------------------------------------------
app/__init__.py 1 0 100%
app/api/__init__.py 0 0 100%
app/api/admin_settings.py 114 0 100%
app/api/auth.py 52 0 100%
app/api/chat.py 301 1 99%
app/api/chat_images.py 52 0 100%
app/api/chats.py 111 0 100%
app/api/config.py 13 0 100%
app/api/doc_drafts.py 99 0 100%
app/api/docs.py 239 1 99%
app/api/git_sources.py 259 0 100%
app/api/health.py 10 0 100%
app/api/source_labels.py 52 0 100%
app/api/steering.py 42 0 100%
app/api/suggestions.py 33 0 100%
app/api/sync.py 242 0 100%
app/api/tokens.py 40 0 100%
app/api/ui_settings.py 39 0 100%
app/config.py 339 0 100%
app/core/__init__.py 0 0 100%
app/core/auth.py 45 0 100%
app/core/caching.py 124 0 100%
app/core/compression.py 59 0 100%
app/core/debugging.py 29 2 93%
app/core/docs_push.py 43 0 100%
app/core/errors.py 5 0 100%
app/core/http_cache.py 22 0 100%
app/core/logging.py 13 0 100%
app/core/memory.py 32 0 100%
app/core/rate_limit.py 44 0 100%
app/core/response_budget.py 63 0 100%
app/core/security_headers.py 20 0 100%
app/core/settings_catalog.py 18 0 100%
app/core/settings_migrate.py 32 0 100%
app/core/settings_overlay.py 117 0 100%
app/core/theming.py 45 0 100%
app/core/tokens.py 44 0 100%
app/db.py 22 0 100%
app/main.py 107 0 100%
app/models.py 148 0 100%
app/rag/__init__.py 0 0 100%
app/rag/agent.py 501 1 99%
app/rag/archive_upload.py 134 0 100%
app/rag/chunker.py 220 4 98%
app/rag/doc_dates.py 18 0 100%
app/rag/folder_summaries.py 134 0 100%
app/rag/git_sources.py 77 0 100%
app/rag/importer.py 906 3 99%
app/rag/llm.py 253 1 99%
app/rag/md_images.py 63 0 100%
app/rag/overview.py 82 0 100%
app/rag/pdf.py 192 0 100%
app/rag/prompts.py 178 0 100%
app/rag/retriever.py 270 3 99%
app/rag/scaffolding.py 55 0 100%
app/rag/source_labels.py 87 0 100%
app/rag/source_removal.py 41 0 100%
app/rag/sources_meta.py 16 0 100%
app/rag/suggestions.py 27 0 100%
app/rag/summarizer.py 58 0 100%
app/rag/svg_text.py 111 0 100%
app/rag/verbatim.py 160 0 100%
app/schemas.py 587 0 100%
--------------------------------------------------
TOTAL 7240 16 99%
coverage gate: app/ 99% (>90%) OK
All checks passed!
0 errors, 0 warnings, 0 informations
WARNING: there is a new pyright version available (v1.1.411 -> v1.1.414).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`
validation OK