Phase 49 (owner request, chat 2026-08-28: "The git sources page should
remove local directory and should instead accept a tarball or zipfile
upload which it will unpack and scan … reuploading the same tarball
should not create a new folder, but should unpack and overwrite the
previously unpacked content" — design confirmed in the same
conversation):
* POST /api/git-sources/upload (admin-only, require_admin): accepts
.tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap
(bounds BOTH the compressed upload and the total extracted bytes —
zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink
escape and device/FIFO members rejected), and atomically swaps the
content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the
archive suffix — no missing window, a failed upload never touches the
existing folder/row/KB). The git_sources row is upserted by path
(kind='local', no duplicates, added_at preserved), the models are
checked fail-fast (503 sanitized when down — the folder/row stay
committed and the next sync/re-upload retries idempotently), and the
source is scanned synchronously in the request (single-source
import_sources prune=True + change-gated KB overview), answering 200
with the sync-style counts. One upload at a time (409); the request
session is released before the scan so a concurrent TRUNCATE cannot
deadlock against it.
* app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES,
archive_source_name (safe-name derivation), unpack_archive (guarded
zip/tar extraction with the extracted-byte cap, no partial state),
swap_in (atomic replace with restore-on-failure) — fully unit-tested.
* app/config.py + .env.example: BOR_UPLOAD_DIR (default
~/bor-sources/uploads, deliberately separate from the git checkouts)
and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at
startup on <= 0).
* python-multipart added to the dependencies — FastAPI's required
multipart parser (an A2 implementation detail, phase locked decision).
* The Sources page: the phase-38 "Add a local directory" form is
removed; #archive-upload-form takes its place (labeled file input,
"Upload & scan" button, the §7.4 never-stale lifecycle, inline
role=alert error, role=status count line); hint + table caption
updated. The POST /api/git-sources kind=local API contract is
UNCHANGED — a plain directory is still registrable via the API, and
existing Local rows list/remove/sync exactly as before.
* The phase-38 story E2E (test_local_directory_sources.py) is rewritten
API-driven — the form it drove is gone; its acceptance stands.
* The story E2E (test_archive_upload_sources.py): the swap,
upload→scan→list (the deterministic "Uploading…" in-flight state, the
Local row, /api/docs + the RAG catalog), same-filename re-upload
(in-place replace, prune, no duplicate row, v2-only folder), the
422 inline error + recovery (the form is not wedged), and the
anonymous gate + 403.
* README: the archive-upload section (formats, naming rule, in-place
replace, both new settings), the local-directory form removal noted,
config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB.
Gates: unit+integration green, app/ coverage 99%, the story E2E green
in isolation, the regression suites (git sources admin, local
directory sources, sync button, import documents, nav rename, smoke,
shared header) green in isolation, ruff + pyright clean.
Note: per this phase's file-level staging, frontend/assets/styles.css
also carries the small same-day in-flight owner rework already in the
working tree (the .sign-in-mobile companion rule for the phase-48
mobile sign-in copy); the phase-49 change is the upload form's block.
An existing, non-git directory is now a first-class source alongside
the git repos: one table (git_sources + kind discriminator — A13
reversible migration), one admin page, one Sync button (phase locked
decisions; the phase-35 table is extended, not duplicated). The DB is
the local-source registry — no env var for local paths;
BOR_GIT_SOURCES stays a git-only empty-table fallback.
Migration 0007 (reversible, up/down integration-tested):
git_sources.kind TEXT NOT NULL DEFAULT 'git' + ck_git_sources_kind
(kind IN ('git','local')); git_sources.path TEXT NULL +
uq_git_sources_path (mirrors 0006's uq_git_sources_url). Existing rows
read kind='git', path=NULL.
API (phase-35 contract extended, git byte-identical): POST kind=local
requires path — trimmed, ~-expanded, absolute + an existing server
directory, else 422 naming the path (fail loud at add-time); duplicate
path 409 (named); wrong field combos 422. GET rows carry kind + path
(git and env rows: path null); anonymous still 403 on every route (A10).
Sync + import_docs resolve DB git + local rows together: git →
clone_or_pull (unchanged); local → re-verified .is_dir() AT SYNC TIME
(it may have moved/deleted since add-time) — a missing dir raises
"local source missing: <path>" (sanitized) before anything imports;
one import_sources(..., prune=True) over the single combined list
(pruning covers the union). Both-empty fails loudly ("no sources
configured (git or local)"); --source still wins; the env fallback
stays git-only.
Page: second "Add a local directory" form (the same §7.4 never-stale
button + inline-error lifecycle as the git form; 422/409 details name
the path), Git/Local badges on rows (text + color, never color alone —
WCAG), updated hint (git + local together, union prune); the
anonymous sign-in gate is unchanged.
Tests: 0007 up/down; the API local-kind matrix (403/201/422/409) with
the git-kind suite green unchanged; the sync pipeline local/git/
mixed/missing against a host temp dir (the KB actually updated);
import_docs DB resolution + --source precedence. Story E2E (isolated,
deterministic across runs): add (Local badge) → missing path inline
422 naming it / duplicate 409 → the real Sync button imports the
fixture file (GET /api/docs + sentinel in its content) → file deleted
+ sync prunes it (union prune) → row removed; anonymous gate + 403s
(phase-35 regression). test_git_sources_admin.py (phase 35) green
UNCHANGED — no selector collision with the new form;
test_sync_button.py green.
Docs: README — the two managed kinds (git = clone/pull mirror; local =
direct in-place walk), add-time validation, union pruning, "the DB is
the local-source registry (no env var for local paths)";
.env.example — the env fallback is git-only.
ducoterra