phase: 102_extensionless_filenames
Build and Push Containers / build-and-push-app (push) Successful in 1m38s
Build and Push Containers / build-and-push-db (push) Successful in 13s

All verification complete — every gate green, no defects found in previously completed work.

**Phase 102 final verification pass — report**

Verified (all three task files present in `complete/`; working-tree implementation matches D1–D5 design):
- `match_extension` choke point in `app/rag/importer.py` (walk + `formats` counter), `doc_format` name-token badge in `app/api/docs.py`, config/`.env.example` docs, fixture `tests/fixtures/extensionless_kb/`, integration + E2E suites — all present and correct
- Completion criteria: end-to-end sync (✓ integration + E2E), case matrix incl. `mydockerfile`/`Dockerfile.dev`/`.dockerfile` exclusions (✓ unit), `formats=dockerfile:1` not `unknown` (✓ log-line assertion), badge `dockerfile`/`containerfile` + `text` fallback + suffixed unchanged (✓ unit/integration/E2E), prune-on-token-removal (✓ `pruned==2`), suffixed-path rule byte-identical (✓ single-line swap, existing cases untouched)

Test / lint results (exact commands):
- `uv run pytest --cov=app --cov-report=term-missing` → 2084 passed, **99%** coverage (>90% gate)
- `uv run pytest tests/e2e/test_extensionless_import.py -v --no-cov` → 2 passed, isolated, DB up
- Regressions isolated: `test_import_documents` 3✓, `test_import_extensions_env` 2✓, `test_quadlet_jinja_import` 4✓, `test_document_viewer` 7✓, `test_kb_tree` 8✓
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings

Notable: commit intentionally not made (harness commits the phase); 102's task files already sit in `complete/`, overview stays in `todo/` for the harness.
Next pending phases: 98, 99, 103, 104, 105 (numeric next after 102: `103_suggestions_session_openers`).
This commit is contained in:
2026-09-12 15:56:43 -04:00
parent 4dbac1660a
commit 3b2dea5685
29 changed files with 1276 additions and 11 deletions
@@ -0,0 +1,61 @@
# Phase 102 — Extensionless filenames: sync `Dockerfile` / `Containerfile` when their name is in `BOR_IMPORT_EXTENSIONS`
**Source:** owner report (chat): "files without extensions never get synced, so 'Dockerfile' and 'Containerfile' get skipped even if I put them in BOR_IMPORT_EXTENSIONS."
**Story:** n/a (defect — the import pipeline of `38_local_directory_sources` / `56_import_extensions_env`; the badge display rides on the phase-10 viewer).
**Context:** the importer's format filter is `path.suffix.lower() in extensions` (`app/rag/importer.py`, `iter_importable_files`) and `Settings.import_extension_set` (the dotted set, `app/config.py`) feeds it. `Path("Dockerfile").suffix` is `""`, so **extensionless files never match — no value of `BOR_IMPORT_EXTENSIONS` can import them**. The validator already accepts `dockerfile` as a well-formed token (lowercase alphanumerics, 1-16 chars, phase 56 — "any extension"), so today the user can *type* the format and it still silently imports nothing. The viewer's format badge (`doc_format`, `app/api/docs.py`, rendered by `frontend/assets/document.js`) falls back to `text` for any suffix-less path; the import `formats=` counter (`ImportSummary.formats`) keys extensionless files under `unknown`.
## Objective
An extensionless file is imported **iff its lowercased full filename is a token in `BOR_IMPORT_EXTENSIONS`** — exact name, case-insensitive. With `dockerfile,containerfile` in the env (the owner's current config), `Dockerfile` and `Containerfile` sync like any other A9 file: chunks, `lite` summary, Sources tree, viewer with a truthful format badge.
## Dependencies
- `56_import_extensions_env` (the env-driven extension scope this phase extends) — complete
- `38_local_directory_sources` / `97_kb_tree_catalog` (import pipeline + drill-down tree the E2E asserts on) — complete
## Design (shared by all tasks — the executor reads this, not the chat)
### D1 — the matching rule (task 01)
A file under a source root is in scope **iff**:
1. its lowercased dotted suffix is a member of `import_extension_set` (the existing rule — `kubernetes.md` → `.md`), **or**
2. it has **no suffix** and its lowercased **full filename** equals a bare token of `import_extension_set` (`Dockerfile` → `dockerfile`).
Consequences (pinned by tests): `DOCKERFILE` matches (case-insensitive name); `mydockerfile` and `dockerfile`-suffixed lookalikes do **not** (exact name only — the list is user-controlled and exact is the only predictable rule); `Dockerfile.dev` is governed by its `.dev` suffix as today; dot-prefixed files (`.dockerfile`) stay skipped by the pre-existing hidden-component rule; the rule applies at any depth (`services/api/Dockerfile` matches).
### D2 — one choke point (task 01)
`app/rag/importer.py` gains `match_extension(path: Path, extensions: frozenset[str]) -> str | None` — returns the matched **bare lowercased token** (``md``, ``dockerfile``) or `None` when out of scope; `extensions` keeps the existing dotted form (``{'.md'}``), bare tokens are derived by stripping the leading dot. `iter_importable_files` filters with `match_extension(path, extensions) is not None` (the walk and the phase-64 progress pre-walk both call this function — one change covers both, `total` stays honest). The `summary.formats` counter in `import_sources` uses `match_extension(path, …) or "unknown"`, so an imported `Dockerfile` counts under `dockerfile:1` in the PLAN §9 line, not `unknown:1`.
### D3 — the badge (task 02)
`doc_format(path, extensions: frozenset[str] = frozenset())` in `app/api/docs.py`: a **non-empty suffix still returns the suffix token unconditionally** (display must never depend on the import list — an out-of-scope `readme.rst` still badges `rst`); a suffix-less path returns the name token when it matches per D1, else `text` (today's fallback, byte-identical for every existing row). The content endpoint (`GET /api/documents/content`) passes `get_settings().import_extension_set` (`app.config`, lru-cached). `frontend/assets/document.js` needs no change: only `md`/`markdown` trigger markdown rendering, so a `dockerfile` badge renders the raw `<pre>` view — correct for a build file.
### D4 — no changes (pinned)
Title stays `full_path.stem` (`Dockerfile` — already correct); the phase-30 `lite` summary applies (non-markdown); prune semantics unchanged — a file that stops matching (file renamed, or token removed from the env) leaves `seen` and is deleted by the next `prune=True` run (the A9 junk precedent).
### D5 — docs
`app/config.py` `import_extensions` docstring + the `.env.example` "Import scope" comment gain the extensionless rule, with `dockerfile,containerfile` as the example (no default-list change — the built-in A9 default stays exactly as phase 56 locked it).
## Tasks
1. `01_name_token_matching.md` — `match_extension` in the importer + the walk + the `formats` counter (unit-pinned).
2. `02_format_badge_and_docs.md` — the viewer badge via `doc_format` + config/`.env.example` docs.
3. `03_integration_and_e2e.md` — the dedicated integration suite + the Playwright story suite + regressions + atomic commit.
## Testing & Quality
- Unit: `tests/unit/test_importer.py` — the matching matrix (D1), walk inclusion/exclusion, the `formats` counter; `tests/unit/test_document_viewer.py` — `doc_format` (with/without the token set, suffix precedence, `text` fallback).
- Integration: `tests/integration/test_import_extensionless.py` (new) — in-process `import_sources` against a story-dedicated fixture dir with the deterministic mock LLM (the phase-02/56 seeding pattern): positive import (counts, per-format `dockerfile:1,containerfile:1,md:1`, lite summary generated for the extensionless files), negative import (no token → not walked), prune-on-token-removal; `tests/integration/test_document_content.py` — the `format` field carries `dockerfile` for an extensionless row and `text` for an unknown-name row.
- E2E (mandatory, house rule 4): `tests/e2e/test_extensionless_import.py`, run in isolation (DB up) — admin Signs in, the drill-down tree lists `Dockerfile`/`Containerfile`, the viewer modal shows badge `dockerfile` + title + summary, `Makefile` (no token) is absent, rows cleaned up in `finally`.
- Coverage: **>90%** on `app/` (`uv run pytest --cov=app --cov-report=term-missing`) — the phase is "Complete" only when the full suite passes and the gate holds.
## Completion Criteria
- [ ] `BOR_IMPORT_EXTENSIONS` containing `dockerfile,containerfile` (plus the usual formats) makes a `Dockerfile` and a `Containerfile` under a source sync end-to-end: imported, chunked, `lite`-summarized, listed in the Sources drill-down tree, openable in the viewer.
- [ ] Case-insensitivity: `DOCKERFILE` imports under the `dockerfile` token; `mydockerfile` does **not**; `Dockerfile.dev` follows its `.dev` suffix; `.dockerfile` stays hidden-skipped.
- [ ] The import `formats=` counter reports `dockerfile:1` (not `unknown:1`) for an imported `Dockerfile`.
- [ ] The viewer format badge shows `dockerfile`/`containerfile`; a suffix-less file whose name is not in the env still badges `text`; every suffixed row's badge is unchanged.
- [ ] Removing a token from the env prunes the previously-imported extensionless documents on the next `prune=True` run.
- [ ] The existing suites stay green in isolation: `test_import_documents.py`, `test_import_extensions_env.py`, `test_quadlet_jinja_import.py`, `test_document_viewer.py`, `test_kb_tree.py`.
- [ ] `uv run pytest` green; `app/` coverage >90%; `uv run ruff check . && uv run pyright` clean.
- [ ] `uv run pytest tests/e2e/test_extensionless_import.py -v --no-cov` green in isolation (DB up).
- [ ] No behavior change to completed phases for any suffixed path (the existing walk rule is untouched — only the suffix-less branch is new).
- [ ] One atomic Conventional Commit, `--no-gpg-sign` (e.g. `fix(import): sync extensionless files (Dockerfile, Containerfile) named in BOR_IMPORT_EXTENSIONS`), `.agents/` phase files moved to `complete/` with the phase.
## Commit
```bash
git add app/ tests/ .env.example .agents/phases/ && git commit --no-gpg-sign -m "fix(import): sync extensionless files (Dockerfile, Containerfile) named in BOR_IMPORT_EXTENSIONS"
```