feat(sources): upload tarball/zipfile archives as sources — unpack, scan, and replace in place
Phase 49 (owner request, chat 2026-08-28: "The git sources page should remove local directory and should instead accept a tarball or zipfile upload which it will unpack and scan … reuploading the same tarball should not create a new folder, but should unpack and overwrite the previously unpacked content" — design confirmed in the same conversation): * POST /api/git-sources/upload (admin-only, require_admin): accepts .tar/.tar.gz/.tgz/.zip, streams it with the BOR_UPLOAD_MAX_MB cap (bounds BOTH the compressed upload and the total extracted bytes — zip-bomb guard), safely unpacks (absolute/traversal/symlink/hardlink escape and device/FIFO members rejected), and atomically swaps the content in over BOR_UPLOAD_DIR/<name>/ (name = filename minus the archive suffix — no missing window, a failed upload never touches the existing folder/row/KB). The git_sources row is upserted by path (kind='local', no duplicates, added_at preserved), the models are checked fail-fast (503 sanitized when down — the folder/row stay committed and the next sync/re-upload retries idempotently), and the source is scanned synchronously in the request (single-source import_sources prune=True + change-gated KB overview), answering 200 with the sync-style counts. One upload at a time (409); the request session is released before the scan so a concurrent TRUNCATE cannot deadlock against it. * app/rag/archive_upload.py: ArchiveUploadError, ARCHIVE_SUFFIXES, archive_source_name (safe-name derivation), unpack_archive (guarded zip/tar extraction with the extracted-byte cap, no partial state), swap_in (atomic replace with restore-on-failure) — fully unit-tested. * app/config.py + .env.example: BOR_UPLOAD_DIR (default ~/bor-sources/uploads, deliberately separate from the git checkouts) and BOR_UPLOAD_MAX_MB (default 512; a validator fails loud at startup on <= 0). * python-multipart added to the dependencies — FastAPI's required multipart parser (an A2 implementation detail, phase locked decision). * The Sources page: the phase-38 "Add a local directory" form is removed; #archive-upload-form takes its place (labeled file input, "Upload & scan" button, the §7.4 never-stale lifecycle, inline role=alert error, role=status count line); hint + table caption updated. The POST /api/git-sources kind=local API contract is UNCHANGED — a plain directory is still registrable via the API, and existing Local rows list/remove/sync exactly as before. * The phase-38 story E2E (test_local_directory_sources.py) is rewritten API-driven — the form it drove is gone; its acceptance stands. * The story E2E (test_archive_upload_sources.py): the swap, upload→scan→list (the deterministic "Uploading…" in-flight state, the Local row, /api/docs + the RAG catalog), same-filename re-upload (in-place replace, prune, no duplicate row, v2-only folder), the 422 inline error + recovery (the form is not wedged), and the anonymous gate + 403. * README: the archive-upload section (formats, naming rule, in-place replace, both new settings), the local-directory form removal noted, config reference rows for BOR_UPLOAD_DIR / BOR_UPLOAD_MAX_MB. Gates: unit+integration green, app/ coverage 99%, the story E2E green in isolation, the regression suites (git sources admin, local directory sources, sync button, import documents, nav rename, smoke, shared header) green in isolation, ruff + pyright clean. Note: per this phase's file-level staging, frontend/assets/styles.css also carries the small same-day in-flight owner rework already in the working tree (the .sign-in-mobile companion rule for the phase-48 mobile sign-in copy); the phase-49 change is the upload form's block.
This commit is contained in:
+108
-32
@@ -335,9 +335,10 @@ html::after {
|
||||
.auth-link:disabled { opacity: 0.6; cursor: wait; }
|
||||
.auth-link svg { width: 16px; height: 16px; display: none; }
|
||||
|
||||
/* Mobile-only sign-out copy: hidden on desktop, revealed inside
|
||||
the hamburger dropdown on mobile (phase 46 UX revision). */
|
||||
.sign-out-mobile {
|
||||
/* Mobile-only sign-out / sign-in copies: hidden on desktop,
|
||||
revealed inside the hamburger dropdown on mobile (phase 46 UX revision). */
|
||||
.sign-out-mobile,
|
||||
.sign-in-mobile {
|
||||
display: none;
|
||||
}
|
||||
|
||||
@@ -1384,14 +1385,14 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
}
|
||||
|
||||
/* Add forms — the tuning form's surface as a single row: visible
|
||||
label + mono location input (git URLs may embed credentials, local
|
||||
paths may contain anything, so both inputs are mono) + the brand
|
||||
button; wraps to a column at narrow widths (the <=640px block
|
||||
below). Phase 38: the "Local directory" form (#local-source-form)
|
||||
reuses the git form's rules VERBATIM — one form language for both
|
||||
kinds. */
|
||||
label + mono location input (git URLs may embed credentials, so the
|
||||
input is mono) + the brand button; wraps to a column at narrow
|
||||
widths (the <=640px block below). Phase 49: the archive upload form
|
||||
(#archive-upload-form, replacing the phase-38 local-directory form)
|
||||
reuses the git form's card + button rules VERBATIM — one form
|
||||
language for both; its file input carries its own rules below. */
|
||||
#git-source-form,
|
||||
#local-source-form {
|
||||
#archive-upload-form {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
@@ -1403,11 +1404,10 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
padding: 0.9rem 1rem 1rem;
|
||||
}
|
||||
#git-source-form:focus-within,
|
||||
#local-source-form:focus-within { border-color: var(--brand); box-shadow: 0 0 0 3px var(--brand-soft), var(--shadow); }
|
||||
#archive-upload-form:focus-within { border-color: var(--brand); box-shadow: 0 0 0 3px var(--brand-soft), var(--shadow); }
|
||||
#git-source-form > label,
|
||||
#local-source-form > label { color: var(--ink); font-weight: 600; white-space: nowrap; }
|
||||
#git-source-url,
|
||||
#local-source-path {
|
||||
#archive-upload-form > label { color: var(--ink); font-weight: 600; white-space: nowrap; }
|
||||
#git-source-url {
|
||||
flex: 1;
|
||||
min-width: 14rem;
|
||||
min-height: 44px;
|
||||
@@ -1419,12 +1419,10 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
border-radius: var(--radius-sm);
|
||||
padding: 0.45rem 0.7rem;
|
||||
}
|
||||
#git-source-url::placeholder,
|
||||
#local-source-path::placeholder { color: var(--ink-soft); }
|
||||
#git-source-url:focus-visible,
|
||||
#local-source-path:focus-visible { outline-offset: 0; border-color: var(--brand); }
|
||||
#git-source-url::placeholder { color: var(--ink-soft); }
|
||||
#git-source-url:focus-visible { outline-offset: 0; border-color: var(--brand); }
|
||||
#git-source-add,
|
||||
#local-source-add {
|
||||
#archive-upload-btn {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
justify-content: center;
|
||||
@@ -1439,9 +1437,58 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
cursor: pointer;
|
||||
}
|
||||
#git-source-add:hover:not(:disabled),
|
||||
#local-source-add:hover:not(:disabled) { background: #7d88f5; }
|
||||
#archive-upload-btn:hover:not(:disabled) { background: #7d88f5; }
|
||||
#git-source-add:disabled,
|
||||
#local-source-add:disabled { opacity: 0.6; cursor: wait; }
|
||||
#archive-upload-btn:disabled { opacity: 0.6; cursor: wait; }
|
||||
|
||||
/* Phase 49: the upload form's file control — mono-ish, on-surface, a
|
||||
>=44px touch target, :focus-visible via the global 3px outline rule
|
||||
(offset zeroed + brand border, exactly like the git URL input). The
|
||||
chosen filename renders in mono; the selector button keeps a plain
|
||||
surface chip. */
|
||||
#archive-upload-file {
|
||||
flex: 1;
|
||||
min-width: 14rem;
|
||||
min-height: 44px;
|
||||
font-family: var(--mono);
|
||||
font-size: 0.88rem;
|
||||
color: var(--ink);
|
||||
background: var(--bg);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
padding: 0.3rem 0.5rem;
|
||||
}
|
||||
#archive-upload-file:focus-visible { outline-offset: 0; border-color: var(--brand); }
|
||||
#archive-upload-file::file-selector-button {
|
||||
min-height: 34px;
|
||||
margin-right: 0.6rem;
|
||||
padding: 0.3rem 0.9rem;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
background: var(--surface);
|
||||
color: var(--ink); /* 13.8:1 on surface */
|
||||
font: inherit;
|
||||
font-size: 0.85rem;
|
||||
font-weight: 600;
|
||||
cursor: pointer;
|
||||
}
|
||||
#archive-upload-file::file-selector-button:hover { border-color: var(--brand); color: var(--brand-ink); }
|
||||
|
||||
/* The upload's success line (role=status): the sync-result shape
|
||||
("2 added · 1 pruned") — ink-soft on surface (>=4.5:1), the dashed
|
||||
hint-box border marks it as a result, not an error; it drops onto
|
||||
its own row under the input like the error line. */
|
||||
#archive-upload-result {
|
||||
flex-basis: 100%;
|
||||
margin: 0;
|
||||
color: var(--ink-soft);
|
||||
background: var(--surface);
|
||||
border: 1px dashed var(--line);
|
||||
border-radius: var(--radius-sm);
|
||||
padding: 0.45rem 0.8rem;
|
||||
font-size: 0.85rem;
|
||||
font-weight: 600;
|
||||
}
|
||||
|
||||
/* The add form's inline error (role=alert): the err pair (9.1:1);
|
||||
flex-basis 100% drops it onto its own row under the input. */
|
||||
@@ -1829,7 +1876,7 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
font-weight: 700;
|
||||
letter-spacing: 0.08em;
|
||||
text-transform: uppercase;
|
||||
color: var(--brand-ink); /* #a5b4fc on --surface ≈8.7:1 */
|
||||
color: var(--brand-ink); /* #fca5a5 on --surface ≈9.0:1 */
|
||||
}
|
||||
.doc-summary-text {
|
||||
margin: 0;
|
||||
@@ -2187,10 +2234,12 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
.auth-link { padding: 0.4rem 0.3rem; }
|
||||
.auth-label { display: none; }
|
||||
.auth-link svg { display: block; }
|
||||
/* Phase 46 UX revision: sign-out moves into the hamburger dropdown
|
||||
on mobile — hide the bar copy, show the dropdown copy. */
|
||||
/* Phase 46 UX revision: sign-out & sign-in move into the hamburger
|
||||
dropdown on mobile — hide the bar copies, show the dropdown copies. */
|
||||
.sign-out-mobile { display: inline-flex; }
|
||||
#sign-out-btn { display: none !important; }
|
||||
.sign-in-mobile { display: inline-flex; }
|
||||
#sign-in-link { display: none !important; }
|
||||
.nav-toggle { margin-left: auto; }
|
||||
/* Dropdown-style sign-out: full-width row, error palette, label visible. */
|
||||
#app-nav .sign-out-btn {
|
||||
@@ -2217,6 +2266,33 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
flex: 0 0 auto;
|
||||
}
|
||||
#app-nav .sign-out-btn .auth-label { display: inline; }
|
||||
/* Dropdown-style sign-in: matches sign-out row styling. */
|
||||
#app-nav .sign-in-mobile {
|
||||
display: flex;
|
||||
width: 100%;
|
||||
align-items: center;
|
||||
gap: 0.6rem;
|
||||
text-align: left;
|
||||
padding: 0.75rem 1.25rem;
|
||||
font-size: 1rem;
|
||||
border: 0;
|
||||
border-radius: 0;
|
||||
background: transparent;
|
||||
color: var(--err-ink);
|
||||
font-weight: 600;
|
||||
text-decoration: none;
|
||||
min-height: 48px;
|
||||
}
|
||||
#app-nav .sign-in-mobile:hover {
|
||||
background: var(--err-bg);
|
||||
color: var(--err-ink);
|
||||
}
|
||||
#app-nav .sign-in-mobile svg {
|
||||
width: 18px;
|
||||
height: 18px;
|
||||
flex: 0 0 auto;
|
||||
}
|
||||
#app-nav .sign-in-mobile .auth-label { display: inline; }
|
||||
.steering-note { padding: 0.3rem 0.3rem 0.3rem 0.7rem; }
|
||||
.app-main > .steering-panel { width: calc(100% - 1.8rem); }
|
||||
.tune-btn { min-height: 44px; }
|
||||
@@ -2248,18 +2324,18 @@ details.thinking .thinking-text ul { margin: 0 0 0.5rem; }
|
||||
.doc-modal-meta { padding-inline: 0.9rem; }
|
||||
.doc-modal-content { padding: 0.75rem 0.9rem 1.25rem; }
|
||||
.composer { padding: 0.5rem; }
|
||||
/* Phase 35 (phase 38: + the local directory form): the add forms
|
||||
stack like the other cards — label, full-width mono input,
|
||||
full-width button; the table wrapper's horizontal scroll already
|
||||
covers long URLs/paths. */
|
||||
/* Phase 35 (phase 49: + the archive upload form): the add forms
|
||||
stack like the other cards — label, full-width input, full-width
|
||||
button; the table wrapper's horizontal scroll already covers long
|
||||
URLs/paths. */
|
||||
#git-source-form,
|
||||
#local-source-form { flex-direction: column; align-items: stretch; }
|
||||
#archive-upload-form { flex-direction: column; align-items: stretch; }
|
||||
#git-source-form > label,
|
||||
#local-source-form > label { white-space: normal; }
|
||||
#archive-upload-form > label { white-space: normal; }
|
||||
#git-source-url,
|
||||
#local-source-path { min-width: 0; }
|
||||
#archive-upload-file { min-width: 0; }
|
||||
#git-source-add,
|
||||
#local-source-add { width: 100%; }
|
||||
#archive-upload-btn { width: 100%; }
|
||||
.footer-inner { flex-direction: column; gap: 0.2rem; text-align: center; }
|
||||
main { padding-bottom: env(safe-area-inset-bottom, 0); }
|
||||
/* Sync button goes icon-only on mobile; the label hides, aria-label
|
||||
|
||||
Reference in New Issue
Block a user