Experimental/brain-of-reese
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

phase: 191_source_name_labels_tags
Build and Push Containers / build-and-push-app (push) Successful in 7m24s
Details
Build and Push Containers / build-and-push-db (push) Successful in 36s
Details

Browse Source 
All gates re-verified against a stable tree. Report:

**Verified (all 6 tasks in `complete/`; this was the final pass)**
- `SourceLabel`+alembic `0030` (dev DB `alembic current` = `0030 (head)`, table present, 0 rows); admin-only `GET/PUT/DELETE /api/source-labels` (`require_admin` router-wide, `resolve_source_name`, partial-update caps/404s); Sources-page "Names, labels and tags" card (read-only derived-name `<code>`, per-row `saveInFlight`, 404 note that survives the settle, chips reuse `.git-source-hidden-count`); `question_name_hits` D2 rule; ONE `all_labels` read per turn in `chat.py` feeding both the `source_name_hit` flag and the prompt surfaces; `source_name_hit_bonus=0.005` in `_selection_order`; ` name=`/` label=` in `build_high_prompt` + `build_overview_prompt(names=…)`.
- **Completion criteria:** E2E `tests/e2e/test_source_name_labels.py` ✅ 7 passed (arms a–e: editor all kinds, control/named question, rename-never-breaks-identity, 404 note, reset) · full suite ✅ · coverage ✅ · README/AGENTS ✅ · phase dir in `complete/` ✅ · commit left to the harness.
- **Gates:** `uv run pytest --cov=app --cov-report=term-missing` → **exit 0, TOTAL 7160 stmts / 16 missed = 99%** (>90%) · `uv run pytest tests/e2e/test_source_name_labels.py -v --no-cov` → **7 passed, exit 0** · regression E2E isolated, exit 0: `test_document_viewer`, `test_admin_auth`, `test_api_tokens`, `test_git_sources_admin`, `test_kb_tree`, `test_local_directory_sources` · `uv run ruff check .` → All checks passed · `uv run pyright` → 0 errors.
- **Battery (task 04 criterion):** `uv run python -m scripts.eval_retrieval --from-file tests/fixtures/retrieval_battery.txt` at default vs `BOR_SOURCE_NAME_HIT_BONUS=0` → **`diff` empty (line-for-line identical)**, matching the recorded arms (dev KB is empty, so every question deflects; the observable boost is unit/E2E-pinned).
- **Documentation:** README — fixed the `/git-sources.html` rows (nav label is **Sources**) and cross-linked the phase-191 card from that section; full audit otherwise clean: all 86 `BOR_*` settings documented with defaults matching `app/config.py`, 0 broken file/dir references and 0 broken anchors in README+AGENTS, AGENTS rule 17 re-checked line-by-line against the code.
- **Notable / deviations:** (1) a **concurrent pipeline process** (`phased-execution/progress.mjs` for `191_…_00_phase`) was mutating this same working tree mid-pass — it rewrote `tests/e2e/test_source_name_labels.py` (11→7 tests), `frontend/assets/git-sources.js`, `tests/unit/test_source_labels_editor_ui.py` and `AGENTS.md` at 05:27; every gate above was (re-)run after those edits with the tree no source file newer than the green full run. (2) The task-06 report/phase text claims regression suites `test_chat_token_gate.py`, `test_git_sources_page.py`, `test_rag_browser.py` are green — **those files have never existed in git history** (those runs exit 4); I ran the real equivalents listed above instead (phase files are not mine to edit). (3) **No live changes**: the VM DB already has `source_labels` (0 rows) but the deployed app predates this uncommitted code, so deploying `/api/source-labels` to the live service is the owner's post-commit step, not part of this phase's criteria.
- **Next pending phase:** none — `191_source_name_labels_tags` is the only entry in `.agents/phases/todo/`.
This commit is contained in:
ducoterra
2026-10-19 06:13:32 -04:00
parent c6ffa050a6
commit 4f20c256d7
70 changed files with 7378 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+80 -7
View File
@@ -64,7 +64,7 @@ client-side router. Direct URLs deep-link to the matching view.
|------|-----|----------------|
| **Chat** | `/` | Everyone (token gate for non-shared) |
| **RAG / Knowledge base** | `/sources.html` | Admin only |
| **Git sources** | `/git-sources.html` | Admin only |
| **Git sources** (nav label **Sources**) | `/git-sources.html` | Admin only |
| **Tuning** | `/tuning.html` | Admin only |
| **Saved chats** | `/history.html` | Admin only |
| **Access tokens** | `/tokens.html` | Admin only |
@@ -269,12 +269,16 @@ just looked at costs a bodiless 304 (see *Caching & deploys*).
### Git sources (`/git-sources.html`)
The admin-managed source registry: add or remove git repositories, upload
source archives (`.tar`, `.tar.gz`, `.tgz`, `.zip`), and register local
directories — all in one table with a `kind` discriminator. No `.env`
editing, no restart. The **Sync sources** button — which lives on the
**RAG** page (`/sources.html`), not here — clones/pulls/walks every
source and re-imports in one click.
The admin-managed source registry (labelled **Sources** in the nav): add or
remove git repositories, upload source archives (`.tar`, `.tar.gz`, `.tgz`,
`.zip`), and register local directories — all in one table with a `kind`
discriminator. No `.env` editing, no restart. Below the table sits the
**Names, labels and tags** card, where each source gets your own name,
label and focus tags (phase 191 — see
[Source names, labels and tags](#source-names-labels-and-tags-phase-191)).
The **Sync sources** button — which lives on the **RAG** page
(`/sources.html`), not here — clones/pulls/walks every source and re-imports
in one click.
### Tuning (`/tuning.html`)
@@ -635,6 +639,74 @@ discriminator: `git` | `local`), **one admin page**, and **one Sync button**.
- The directory is re-verified to exist at sync time; a missing directory
fails the run loudly.
### Source names, labels and tags (phase 191)
Every source — git, local or uploaded — can carry **your** words for it: a
name ("the git server"), a one-line label ("self-hosted forge + Gitea") and
up to 10 focus tags. The storage is one row per source (`source_labels`,
migration 0030, created empty — an unnamed source behaves exactly as before).
**Edit them on the Sources page** (`/git-sources.html`, admin only), in the
**Names, labels and tags** card below the sources table. Its row list is the
catalogue `GET /api/source-labels` — the union of every registered source and
every indexed document source — so git repos, local directories **and**
uploads all get an editor without a second API. Each row leads with its
read-only mono source name (the identity), expands into the three fields
(Name ≤ 120, Label ≤ 200, Tags: type + **Enter** or a **comma** adds a chip,
**Backspace** drops the last, ten chips of ≤ 32 chars) plus **Save** /
**Reset** and its own status line — "unsaved changes" → "Saving…" → "saved",
or a one-line note on failure (a 404 says *that source no longer exists*, so
you know to reload rather than retry). A failure note **stays** on the row —
the row never repaints it back to "unsaved changes" over the reason; only
editing the fields again or collapsing the editor clears it. One save per row
at a time; the save response is what the row re-reads, so nothing on the page
claims more than the server has. A row with no stored label shows no **Reset**, and collapsing
an editor discards the un-saved draft instead of silently half-saving.
The API behind it:
| Route | What it does |
| --- | --- |
| `GET /api/source-labels` | The full catalogue: every registered source (git **and** local rows, or the `BOR_GIT_SOURCES` names while the table is empty) plus every indexed source no row derives (the uploads), de-duplicated and sorted by name, each with its label or `null`/`[]`. |
| `PUT /api/source-labels/{source}` | The upsert, partial-update style: an omitted field keeps its value, an explicit `null` clears it, `tags: []` clears the tags. Values are trimmed, the caps are 120 / 200 / ten tags of 32, tags de-duplicate case-insensitively (first case kept). A source that matches no row and no document is **404** — no label can be created for a source that does not exist; an over-cap or malformed field is the framework's **422** and writes nothing. |
| `DELETE /api/source-labels/{source}` | The reset — the row goes, and the **200** body is the entry's cleared state. No row to remove is **404**. |
Admin-only (all three, like the Theme and Settings surfaces): anonymous
callers and token users get `403 admin only`.
**A name is an alias, never a rename.** The identity of a source stays the
derived name the importer stores in `Document.source` — the clone directory,
the local directory, the upload file stem. A label row never rewrites a
document, a `source/path` locator in a saved or shared chat, a citation, a
verbatim panel caption or the RAG tree, so renaming a source cannot break
anything that was already written. What the name is *for* — focusing
retrieval and the prompt on the words you use — is the rest of the phase.
**The name focuses retrieval.** When a question token equals (case- and
accent-insensitively, 4-char minimum on both sides) a token of a source's
name, label or any tag, that source is a *source-name hit* for the turn:
every retrieved document from it gets a small, bounded head start in the
selection walk that picks the suggested tier — `source_name_hit_bonus`
(default `0.005`, env `BOR_SOURCE_NAME_HIT_BONUS`, `0` = off, a negative
value refuses startup). The bonus applies **once per document**, at most
two bonuses stacked (the phase-119 path-name bonus plus this one = 0.010
on the fused scale), and it lives in the selection layer only — chunk
scores, cosines and the honesty-gate inputs are untouched, so a deflected
answer stays deflected. With no label rows anywhere the selection walk is
byte-identical to the pre-phase one (the kill switch), which is also what
the retrieval battery records at `0`.
**The name reaches the model, too.** One label read per chat turn feeds both
the boost above and the prompt: each suggested document's `<document>` block
carries ` name="the git server" label="self-hosted forge"` beside its
`source` / `path`, so the model knows which documents your words refer to —
and the KB outline is generated from a document list that names your sources
the way you do (the display name replaces the raw source name in that one
model input; the stored outline and its deterministic `Sources: …` line keep
the derived name). Nothing here changes what the model can cite: it still
reads, greps and cites `source/path`, and a source without a label row is
byte-for-byte the pre-phase prompt.
### Sync from the UI
The **Sync sources** button on the **RAG** page (`/sources.html`, admin
@@ -1806,6 +1878,7 @@ startup (the DB rows shadow the env, `.env` is never rewritten).
| `BOR_RECENCY_BOOST` | `0.0007` | Max additive fused-score bonus a zero-age doc gets (phase 106); `0` = off |
| `BOR_RECENCY_HALF_LIFE_DAYS` | `365` | Recency decay timescale, days (phase 106) — the boost decays by e^-1 (~0.37x) per timescale of age |
| `BOR_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a name-hit doc's best fused score gets (phase 119); `0` = off |
| `BOR_SOURCE_NAME_HIT_BONUS` | `0.005` | Selection-time bonus a **source-name**-hit doc's best fused score gets — the question names the source's label-row name/label/tag (phase 191); independent knob, `0` = off |
### Agent tools