# Story: Import Documents

**Phase:** `02_story_import_documents.md` · **E2E:** `tests/e2e/test_import_documents.py`

## Narrative

As **Reese** (the owner), I want to point the importer at one or more
directories of markdown files and have them chunked, embedded, and stored in
Postgres — so that Brain's answers always reflect my *current* documentation.

- **Given** the `~/Homelab` and `~/Deployments` trees (or any `--source` dirs)
- **When** I run `uv run python -m scripts.import_docs`
- **Then** every `*.md` file (after the exclusion list) is present in the
  `documents` table with its full content, a sha256 hash, and chunk rows with
  768-dim embeddings; unchanged files are skipped on re-runs; and the
  Sources page in the browser shows the indexed documents.

## Acceptance criteria
1. `scripts/import_docs.py` accepts repeatable `--source PATH` (default
   `~/Homelab` `~/Deployments`), `--prune`, and `--limit N` (debug).
2. Only `*.md` files are imported; excluded dirs: `.venv`, `node_modules`,
   `.git`, `__pycache__`, `.pytest_cache`, `dist`, `build` (PLAN A9).
3. Delta detection by sha256 on `(source, path)`: unchanged → skipped
   (no re-embedding); changed → re-chunked + re-embedded, old chunks
   replaced atomically.
4. Embeddings are batched (`BOR_EMBED_BATCH_SIZE`) against `aipi /v1/embeddings`
   (`embed`); a dimension mismatch fails loudly with an actionable message.
5. Rich per-file logging (`added|updated|unchanged|pruned`) + summary.
6. `GET /api/docs` returns the document list; the Sources page renders it
   (stat cards + table) or the designed empty state when none exist.
7. The whole flow works against the **mock LLM** in E2E (deterministic),
   and against real aipi for manual runs.

## UI Visualization & Structure
- **Sources page (`/sources.html`), desktop:** header row (h1 + sub), then
  stat cards in `repeat(auto-fit, minmax(170px,1fr))` (documents / chunks /
  last indexed), then a **full-width table** inside a scroll wrapper
  (min-width 640px → horizontal scroll, never a squeezed hairline list).
  Columns: Source · Path (mono, ellipsized w/ `title`) · Title · Chunks ·
  Indexed. Uses ≥85% of the 72rem container width.
- **Empty state (no docs):** centered card with 📂, "Nothing indexed yet",
  and the exact import command in a `<code>` pill. No dead links, no
  placeholder tables.
- **Accessibility:** `<caption class="visually-hidden">` on the table,
  `scope="col"` on headers, `role="region"` + `tabindex="0"` on the scroll
  wrapper (keyboard scrollable), stat values have visible labels.
- **Mobile:** stat cards stack (auto-fit), table scrolls horizontally,
  no content below the fold is unreachable.

## Playwright Mapping Rule
**Test Scenario → `tests/e2e/test_import_documents.py`** (one isolated
Playwright suite for this story):
1. *Seeding:* run the import function in-process against
   `tests/fixtures/docs/` (mock embeddings, temp DB state) — a fixture, not
   the test's subject.
2. `test_sources_page_lists_indexed_docs` — goto `/sources.html`, assert stat
   cards show the fixture counts and the table rows include
   `homelab/kubernetes.md`, `homelab/backups.md`, `deployments/new-service.md`.
3. `test_sources_table_layout` — table wrapper width ≥80% of container;
   `caption` present; on a 375px viewport the wrapper scrolls horizontally.
4. `test_empty_state_when_no_docs` (fresh/truncated DB) — empty state visible
   with the import command; table hidden.