phase: 128_sync_label_mobile
Everything verified and 100% green. The earlier rejection was a report-format issue, not a test failure — all work is in the working tree. Final report: **Phase 128 final pass — verified (all 3 tasks complete; no code defects found)** - Verified: `#sync-label` parts markup, `setSyncLabel` writer, part builders (A4 untruncated title/aria-live), flex-row CSS + mobile media query — all match design 1:1. - `uv run pytest --cov=app --cov-report=term`: 3024 passed; TOTAL coverage **99%** (>90% gate). - `uv run pytest tests/e2e/test_sync_label_mobile.py -v --no-cov`: 4 passed (119s, isolated). - `uv run ruff check .`: clean; `uv run pyright`: 0 errors. - Criteria: (1) mobile 390×844 label in-viewport, count visible in import+summaries — ✅ E2E; (2) desktop strings/cap/title unchanged — ✅ E2E+unit; (3) E2E isolated green — ✅; (4) full suite green — ✅. - Docs: README.md fixed 5 pre-existing gaps (PDF phase-126 section + `pdf` token, hidden-folders toggle ph.105, document dates ph.106, private git-source masked tokens ph.121, doc-draft Discard ph.115); AGENTS.md verified consistent — no changes. - Notable: double-`-q` (addopts `-q` + CLI `-q`) suppresses pytest's summary line — standard `-qq` behavior, not a bug; exit codes confirm green. - Next pending phase: `129_sync_chunk_progress`.
This commit is contained in:
@@ -83,8 +83,13 @@ fresh deployment they seed from `BOR_SUGGESTIONS`.
|
||||
### RAG / Knowledge base (`/sources.html`)
|
||||
|
||||
The indexed document list. The *Path* column opens each document in the same
|
||||
**almost-fullscreen modal**. Admin-only — anonymous visitors see a sign-in
|
||||
gate instead.
|
||||
**almost-fullscreen modal**. Document rows carry a *Created* date and folder
|
||||
rows an *Updated* date (phase 106) — sourced at sync time (git commit dates
|
||||
for git sources, restored metadata for archive uploads, file mtime otherwise;
|
||||
unresolvable or future dates fall back to today) — and the admin can correct a
|
||||
document's date from the document viewer. The date also feeds the small
|
||||
recency boost in retrieval ranking (`BOR_RECENCY_BOOST`). Admin-only —
|
||||
anonymous visitors see a sign-in gate instead.
|
||||
|
||||
### Git sources (`/git-sources.html`)
|
||||
|
||||
@@ -118,7 +123,8 @@ everyone — no login or token required.
|
||||
### Edit doc (`/doc-edit.html`)
|
||||
|
||||
Review and adjust an AI-generated answer before committing it to the docs
|
||||
repository. Admin-only flow page.
|
||||
repository. A draft that should never be pushed can be **Discard**ed from
|
||||
the same page (permanent; admin-only). Admin-only flow page.
|
||||
|
||||
---
|
||||
|
||||
@@ -200,9 +206,15 @@ reflect the new files.
|
||||
so it is safe to run from cron or after every commit.
|
||||
- Indexed formats: **md, markdown, txt, yaml, yml, json, py**, the Podman
|
||||
quadlet family (**container, network, volume, image, pod, kube, swap,
|
||||
os, endpoint**), and **j2** Jinja templates (case-insensitive).
|
||||
os, endpoint**), **j2** Jinja templates — and, **opt-in**, **pdf**
|
||||
(add the `pdf` token to `BOR_IMPORT_EXTENSIONS`; see
|
||||
[PDF documents](#pdf-documents-phase-126)).
|
||||
- Hidden files and common cache dirs (`.venv`, `node_modules`, `.git`,
|
||||
`__pycache__`, etc.) are skipped.
|
||||
`__pycache__`, etc.) are skipped — unless the per-source **Index hidden
|
||||
folders** checkbox on the Git sources page is on for that source
|
||||
(phase 105): then dot-prefixed folders/files with an importable
|
||||
extension are indexed like visible ones, and switching it off prunes
|
||||
them on the next sync.
|
||||
- Non-markdown files get format-aware chunking (YAML keys, JSON keys,
|
||||
Python classes/defs).
|
||||
- With `BOR_IMAGES=1`, standalone images (`png, jpg, jpeg, webp, gif,
|
||||
@@ -227,8 +239,14 @@ BOR_GIT_SOURCES=https://git.reeseapps.com/reese/homelab.git,git@github.com:reese
|
||||
BOR_SOURCES_DIR=~/bor-sources # default; each repo lands in <dir>/<repo-name>/
|
||||
```
|
||||
|
||||
- Auth is whatever the machine supplies — HTTPS via the OS credential
|
||||
helper, or SSH via your key; no credentials are stored in the app.
|
||||
- Public repos need no setup: auth is whatever the machine supplies —
|
||||
HTTPS via the OS credential helper, or SSH via your key.
|
||||
- Private repos: the add/edit form on the Git sources page takes an
|
||||
optional **masked token** field (phase 121). The token is stored in a
|
||||
dedicated DB column and injected into the clone URL only at
|
||||
sync/clone time — it is never returned by any API response or shown
|
||||
in the UI (legacy rows that already embed `user:token@` in the stored
|
||||
URL keep working — the token is sanitized on output).
|
||||
- Every run **clones** (first time, shallow `--depth 1`) or **pulls**
|
||||
(`git pull --ff-only`) each repo, then indexes the checkout.
|
||||
- A failed sync aborts the run — no partial junk.
|
||||
@@ -494,6 +512,33 @@ of its answer:
|
||||
|
||||
---
|
||||
|
||||
## PDF documents (phase 126)
|
||||
|
||||
PDFs are **opt-in**: add the `pdf` token to `BOR_IMPORT_EXTENSIONS`
|
||||
(a stock `.env` never indexes them). Once admitted, a PDF is a
|
||||
first-class document through every entry point (git/local walks and
|
||||
uploaded archives):
|
||||
|
||||
- **Text, page-aware.** Text is extracted page by page (`pymupdf`); one
|
||||
document per PDF, its text-bearing pages joined with `---PAGE-N---`
|
||||
separators. Chunking is page-aware — every chunk keeps a `Page N:`
|
||||
anchor, so a retrieval hit stays readable in context.
|
||||
- **Summarized + dated like any document.** The `lite` model summarizes
|
||||
it, sync sources its date (git commit / file metadata), and it appears
|
||||
in the catalog, the KB tree, and the agent's `ls` / `read` / `grep`
|
||||
tools.
|
||||
- **Embedded images join the image pipeline.** With `BOR_IMAGES=1`,
|
||||
images embedded in the PDF are extracted and run through the same
|
||||
image-document flow as standalone images (chat-model description,
|
||||
persistent copy under `BOR_IMAGE_DIR`, retrievable + displayable);
|
||||
with the toggle off the pass is skipped and the PDF text indexes
|
||||
normally. Embedded images in a format outside `BOR_IMAGE_EXTENSIONS`
|
||||
are skipped with a warning.
|
||||
- **Served as text.** The document viewer renders the extracted text —
|
||||
never the binary PDF.
|
||||
|
||||
---
|
||||
|
||||
## Caching & deploys
|
||||
|
||||
A deploy is a commit — and the browser must see it **without a hard refresh**.
|
||||
@@ -611,7 +656,7 @@ to `.env` and adjust.
|
||||
|
||||
| Env | Default | Meaning |
|
||||
|-----|---------|---------|
|
||||
| `BOR_IMPORT_EXTENSIONS` | A9 family | Importable text formats — the A9 default is `md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2`; any well-formed extension is allowed. Images are **not** added here — see `BOR_IMAGES` |
|
||||
| `BOR_IMPORT_EXTENSIONS` | A9 family | Importable text formats — the A9 default is `md,markdown,txt,yaml,yml,json,py,container,network,volume,image,pod,kube,swap,os,endpoint,j2`; any well-formed extension is allowed. Add the `pdf` token to make PDFs first-class, retrievable documents (phase 126, opt-in). Images are **not** added here — see `BOR_IMAGES` |
|
||||
| `BOR_GIT_SOURCES` | — (empty) | CSV of git repo URLs — **fallback while the admin Git sources page's list is empty** |
|
||||
| `BOR_SOURCES_DIR` | `~/bor-sources` | Where git repos are cloned/pulled |
|
||||
| `BOR_UPLOAD_DIR` | `~/bor-sources/uploads` | Where uploaded source archives are unpacked |
|
||||
|
||||
Reference in New Issue
Block a user