phase: 121_git_source_tokens
**Phase 121 final verification pass — all green** (all 4 tasks already in `complete/`; verified, no defects found, no changes needed) - Verified implementation vs phase design: migration `0021` (reversible, round-tripped via `alembic downgrade base` + `upgrade head` → head `0021`), `GitSource.token` column, `normalize_credential`/`clone_url_for`/`sanitize_url`, clone callers switched (`sync.py`, `import_docs.py`), masked token fields in add form + editor, `extra="forbid"` output shapes - Tests: `uv run pytest` → 2662 passed, 0 failed (exit 0); `uv run pytest --cov=app --cov-report=term-missing` → TOTAL **99%** (≥90% gate) - Lint/types: `uv run ruff check .` → All checks passed; `uv run pyright` → 0 errors, 0 warnings - E2E in isolation: `uv run pytest tests/e2e/test_git_source_tokens.py -v --no-cov` → **4 passed** Completion criteria: 1. Private repo (UI add or pasted embedded-token URL) clones with injected token; token absent from every API response, page text, title attr, and full HTML — **PASS** (integration raw-JSON assertions + E2E `_assert_token_nowhere`) 2. Legacy embedded-token rows still clone from stored URL; output sanitized — **PASS** (`test_sync_legacy_row_clones_with_original_stored_url`, `test_get_masks_legacy_embedded_token_row`, env-fallback masking) 3. Public/local sources byte-identical — **PASS** (verbatim-URL + no-userinfo-unchanged tests) 4. pytest / coverage / ruff / pyright — **PASS** (see above) 5. Commit + phase move — harness responsibility; task files already in `complete/`, changes left in working tree (no commit made, per protocol) Notable: no deviations; DB left at head, functional. Next pending phase: **122_image_documents** (then 123_chat_image_questions).
This commit is contained in:
+14
-4
@@ -27,8 +27,14 @@ decisions):
|
||||
URLs) fails loudly (``no sources configured (git or local)``)
|
||||
instead of silently importing the legacy local directories;
|
||||
3. per resolved row: ``kind=git`` → :func:`scripts.git_sync.clone_or_pull`
|
||||
into ``BOR_SOURCES_DIR/<repo-name>/`` (phase 28 — reused, not
|
||||
re-implemented); ``kind=local`` → the stored directory, re-verified
|
||||
with the phase-121 clone URL (:func:`app.rag.git_sources.clone_url_for`
|
||||
— the row's ``token`` column injected as
|
||||
``https://x-access-token:<token>@…`` only for https? rows; NULL
|
||||
token → the bare stored URL verbatim, so public repos and legacy
|
||||
embedded-token rows clone exactly as before) into
|
||||
``BOR_SOURCES_DIR/<repo-name>/`` (phase 28 — reused, not
|
||||
re-implemented; the checkout name stays on the bare URL —
|
||||
credential-free); ``kind=local`` → the stored directory, re-verified
|
||||
``.is_dir()`` **at sync time** (it may have moved/deleted since
|
||||
add-time) — a missing directory raises ``local source missing:
|
||||
<path>``; a failing clone or a missing local dir aborts before any
|
||||
@@ -116,7 +122,7 @@ from app.core.auth import require_admin
|
||||
from app.core.errors import sanitize_error as _sanitize_error
|
||||
from app.db import SessionLocal
|
||||
from app.rag.folder_summaries import generate_folder_summaries, missing_folder_summaries
|
||||
from app.rag.git_sources import effective_sources
|
||||
from app.rag.git_sources import clone_url_for, effective_sources
|
||||
from app.rag.importer import ImportSummary, import_sources
|
||||
from app.rag.llm import LLMClient, check_models
|
||||
from app.rag.overview import regenerate_overview
|
||||
@@ -293,7 +299,11 @@ async def _run_sync() -> None:
|
||||
doc_dates_by_root: dict[str, dict[str, datetime]] = {}
|
||||
for row in rows:
|
||||
if row.kind == "git":
|
||||
root = clone_or_pull(row.url, sources_root / repo_name(row.url))
|
||||
# Phase 121: the token column is injected into the clone
|
||||
# URL ONLY here (clone_url_for — NULL token → the bare
|
||||
# stored URL verbatim); repo_name stays on the bare URL
|
||||
# so the checkout directory name is credential-free.
|
||||
root = clone_or_pull(clone_url_for(row), sources_root / repo_name(row.url))
|
||||
# Phase 106 (D2): the checkout's per-file last-commit
|
||||
# dates, keyed by the SAME root string the importer
|
||||
# sees (full-history checkouts → true per-file
|
||||
|
||||
Reference in New Issue
Block a user