phase: 102_extensionless_filenames
Build and Push Containers / build-and-push-app (push) Successful in 1m38s
Build and Push Containers / build-and-push-db (push) Successful in 13s

All verification complete — every gate green, no defects found in previously completed work.

**Phase 102 final verification pass — report**

Verified (all three task files present in `complete/`; working-tree implementation matches D1–D5 design):
- `match_extension` choke point in `app/rag/importer.py` (walk + `formats` counter), `doc_format` name-token badge in `app/api/docs.py`, config/`.env.example` docs, fixture `tests/fixtures/extensionless_kb/`, integration + E2E suites — all present and correct
- Completion criteria: end-to-end sync (✓ integration + E2E), case matrix incl. `mydockerfile`/`Dockerfile.dev`/`.dockerfile` exclusions (✓ unit), `formats=dockerfile:1` not `unknown` (✓ log-line assertion), badge `dockerfile`/`containerfile` + `text` fallback + suffixed unchanged (✓ unit/integration/E2E), prune-on-token-removal (✓ `pruned==2`), suffixed-path rule byte-identical (✓ single-line swap, existing cases untouched)

Test / lint results (exact commands):
- `uv run pytest --cov=app --cov-report=term-missing` → 2084 passed, **99%** coverage (>90% gate)
- `uv run pytest tests/e2e/test_extensionless_import.py -v --no-cov` → 2 passed, isolated, DB up
- Regressions isolated: `test_import_documents` 3✓, `test_import_extensions_env` 2✓, `test_quadlet_jinja_import` 4✓, `test_document_viewer` 7✓, `test_kb_tree` 8✓
- `uv run ruff check .` → clean; `uv run pyright` → 0 errors, 0 warnings

Notable: commit intentionally not made (harness commits the phase); 102's task files already sit in `complete/`, overview stays in `todo/` for the harness.
Next pending phases: 98, 99, 103, 104, 105 (numeric next after 102: `103_suggestions_session_openers`).
This commit is contained in:
2026-09-12 15:56:43 -04:00
parent 4dbac1660a
commit 3b2dea5685
29 changed files with 1276 additions and 11 deletions
+20 -5
View File
@@ -35,11 +35,13 @@ from sqlalchemy import func, select
from sqlalchemy.orm import Session
from app.api.sync import _sanitize_error
from app.config import get_settings
from app.core.auth import require_admin, require_user
from app.db import get_db
from app.models import Chunk, Document, FolderSummary
from app.rag.agent import list_source_names
from app.rag.folder_summaries import folder_of
from app.rag.importer import match_extension
from app.rag.llm import EmbeddingError, LLMClient
from app.schemas import (
DocContent,
@@ -58,11 +60,24 @@ from app.schemas import (
router = APIRouter(tags=["kb"])
def doc_format(path: str) -> str:
def doc_format(path: str, extensions: frozenset[str] = frozenset()) -> str:
"""Lowercased path suffix without its dot (``kubernetes.md`` → ``md``,
``notes/deep.Markdown`` → ``markdown``); ``text`` when the path has no
suffix — the value shown in the viewer's format badge."""
return Path(path).suffix.lower().lstrip(".") or "text"
``notes/deep.Markdown`` → ``markdown``) — returned **unconditionally**
for a non-empty suffix (display never depends on the import list — an
out-of-scope ``readme.rst`` still badges ``rst``); ``text`` when the
path has no suffix **and its name is not a configured token** (phase
102: a suffix-less ``Dockerfile`` badges ``dockerfile`` when its
lowercased full filename is one of *extensions* — the importer's
:func:`app.rag.importer.match_extension` rule). The value shown in
the viewer's format badge; the default empty *extensions* keeps the
pre-phase-102 suffix-only result for every path (byte-identical).
"""
p = Path(path)
suffix = p.suffix.lower().lstrip(".")
if suffix:
return suffix
matched = match_extension(p, extensions)
return matched if matched is not None else "text"
@router.get("/docs", response_model=DocList)
@@ -140,7 +155,7 @@ def get_document_content(
source=doc.source,
path=doc.path,
title=doc.title,
format=doc_format(doc.path),
format=doc_format(doc.path, get_settings().import_extension_set),
summary=doc.summary,
content=doc.content,
indexed_at=doc.indexed_at.isoformat(),
+4 -1
View File
@@ -207,7 +207,10 @@ class Settings(BaseSettings):
# doubles as the typo guard); the value below is the built-in default
# (the A9 family, incl. the quadlet family + ``j2``) and the documented
# example in ``.env.example``. Hidden (dot) path components are always
# skipped, plus the importer's exclusion list.
# skipped, plus the importer's exclusion list. A token also matches
# extensionless files whose lowercased full filename equals it exactly
# (``dockerfile`` → ``Dockerfile``), case-insensitive, no partial
# names (phase 102).
# Stored as a raw CSV string (env-native — no JSON) and parsed on demand
# via :py:meth:`import_extension_set`. The validator rejects an empty
# list and malformed tokens so a typo fails loudly at startup (it can
+29 -3
View File
@@ -19,7 +19,9 @@ the two-phase upsert:
Scope (A9, revised 2026-08-21): any path containing a dot-prefixed
component (hidden dirs — vendored caches like ``.esphome/.espressif/**`` —
or hidden files) is skipped, plus the well-known exclusion list.
or hidden files) is skipped, plus the well-known exclusion list. A token
may also name extensionless files by their exact lowercased full filename
(``Dockerfile`` under the ``dockerfile`` token — phase 102).
``prune=True`` deletes documents (of the imported sources only) whose files
no longer exist **or no longer match the format filter** — this is how
@@ -158,6 +160,24 @@ def _ignore_for_root(
return tuple(p for p in (normalize_ignore_path(e) for e in raw) if p)
def match_extension(path: Path, extensions: frozenset[str]) -> str | None:
"""The bare lowercased token *path* imports under, or ``None``.
1. Non-empty lowercased dotted suffix in *extensions* (the A9 rule —
``kubernetes.md`` → ``md``).
2. No suffix: the lowercased FULL filename equals a bare token of
*extensions* (``Dockerfile`` → ``dockerfile``) — the phase-102
extensionless rule. Exact name only: ``mydockerfile`` never
matches the ``dockerfile`` token.
"""
suffix = path.suffix.lower()
if suffix and suffix in extensions:
return suffix.lstrip(".")
if not suffix and path.name.lower() in {e.lstrip(".") for e in extensions}:
return path.name.lower()
return None
def iter_importable_files(
root: Path,
extensions: frozenset[str],
@@ -187,7 +207,7 @@ def iter_importable_files(
continue
if ignore and is_ignored(rel.as_posix(), ignore):
continue
if path.suffix.lower() not in extensions:
if match_extension(path, extensions) is None:
continue
files.append(path)
return files
@@ -274,7 +294,13 @@ async def import_sources(
rel = path.relative_to(root).as_posix()
seen.add((source, rel))
summary.files += 1
ext = path.suffix.lower().lstrip(".") or "unknown"
# Phase 102: the matched bare token (``dockerfile`` for an
# extensionless ``Dockerfile``), never ``unknown`` — the
# file is in scope, so the walk matched it.
ext = (
match_extension(path, llm.settings.import_extension_set)
or "unknown"
)
summary.formats[ext] = summary.formats.get(ext, 0) + 1
if progress is not None:
# phase 64: report the file *before* indexing it — a