phase: 02_level_content_and_validation

Phase `02_level_content_and_validation` — final verification **green**.

- **Gate:** `bash .agents/validate.sh` → ruff ✅ · pyright 0 errors ✅ · `undercooked-levels validate` OK (2 levels) ✅ · **668 passed** (incl. 10 Playwright e2e) ✅ · coverage **100%** ✅ · repo hygiene ✅
- **Criteria:** `validate` exit 0, one line per level; `list` = `01_first_kitchen`, `02_double_shift` in campaign order; live `:8000` `/healthz` 200, `/api/levels` (+detail) carries both levels with numeric `target_score`/`time_limit` and no grid/station leakage, `/api/config` correct, unknown id → 404, startup log `content loaded: 2 level(s), 4 recipe(s)`; `validate --dir tests/fixtures/levels/_bad/` → exit 1 with named `RECIPE_STATION_MISSING` / `SERVE_REACHABLE` / `TARGET_IMPLAUSIBLE`; A8 lock test proves a YAML-only level reaches loader+API+CLI.
- **Defect fixed (pre-existing path, phase-02-triggered):** `undercooked` with unusable content printed the report then a raw traceback and exited `1`. `src/undercooked/__main__.py` now catches `ContentError` → one closing line, exit `2` (like a bad setting); 2 new tests in `tests/unit/test_main.py` (incl. non-content failures still propagate).
- **Docs pass (full, project-wide):** README (content gate + exit `2`, config section), AGENTS.md §3, `docs/LEVELS.md` (startup refusal + exact CLI exit codes), `docs/DEPLOYMENT.md` (crash-loop diagnosis bullet); DESIGN/TESTING re-read, consistent.
- **Container re-verified (no phase-01 regression):** `podman build -f Containerfile` succeeds; the image ships `levels/` and serves `/healthz` 200, `/api/levels` (2 levels), `/` 200 as non-root.
- **Deviation to note:** the phase overview names `tests/fixtures/levels/bad/`; the actual broken-fixture dir is `tests/fixtures/levels/_bad/` (the `_` prefix is what keeps it out of normal discovery). Overview files are read-only to me — README/LEVELS.md document the real path.
- No git index/history commands run; all changes left unstaged for the harness. ntfy notification sent.
This commit is contained in:
2026-10-06 11:03:28 -04:00
parent 6fd83446ec
commit 3426264ffb
48 changed files with 8094 additions and 54 deletions
+12 -2
View File
@@ -1,6 +1,6 @@
# TESTING.md — how this project is verified
**Status:** current as of phase 01 · **Filled in by phase 14**, which re-runs every layer
**Status:** current as of phase 02 · **Filled in by phase 14**, which re-runs every layer
below against the finished game and records the release-time suite inventory.
## Purpose
@@ -20,7 +20,7 @@ layer live in `tests/conftest.py`, and layer-specific fixtures live in that laye
| Directory | Contains | I/O allowed |
|---|---|---|
| `tests/unit/` | pure logic with no I/O: config parsing, header values, CLI precedence, protocol/validator helpers, and the doc/content guards that keep README, `.env.example`, `PLAN.md` and the code agreeing | none (in-process `TestClient` requests are fine) |
| `tests/sim/` | headless determinism and replay tests against the pure kernel (`tests/sim/test_replay_*.py`, golden state hashes) | none — the sim must stay callable without a server |
| `tests/sim/` | tests of the pure kernel that need no server: grid geometry, the entity model, the A5 import-purity guard, and the level validator (phase 02) — plus the headless determinism/replay suites with golden state hashes (`tests/sim/test_replay_*.py`) that arrive with the tick in phase 03 | none — the sim must stay callable without a server |
| `tests/integration/` | multi-module, in-process tests: app boot through `create_app()`, HTTP routes, static serving, room/protocol behaviour with real WebSocket clients (room + protocol integration is PLAN §9 layer 4, and lives here) | in-process app, loopback sockets |
| `tests/e2e/` | one Playwright suite per user story, named `test_usNN_*.py`; multiplayer stories drive **two browser contexts** against one room | real browser, real socket |
| `tests/fixtures/` | fixture data only. `tests/fixtures/levels/` is the content dir every test sees — an autouse session fixture points `UNDERCOOKED_LEVELS_DIR` at it so no test can silently read the shipped `levels/` | none |
@@ -59,6 +59,16 @@ uv run pyright # types
bash .agents/validate.sh # the pipeline gate, in that order + hygiene
```
The gate runs one check that is not a pytest command: `uv run undercooked-levels
validate` between pyright and pytest (docs/LEVELS.md **Authoring workflow**). It is
unconditional — a campaign that breaks a PLAN §8 rule stops the pipeline even when no
test file changed — and it is the same `load_content` + `validate_bundle` pair
`create_app()` runs at startup, so it can disagree with the server about nothing.
Content tests stay hermetic the same way: the autouse session fixture points
`UNDERCOOKED_LEVELS_DIR` at `tests/fixtures/levels/` (a recipe book and zero levels),
and a test that wants real levels builds one under `tmp_path` or names `levels/`
explicitly. `tests/unit/test_content_not_hardcoded.py` is the A8 lock on top of that.
Coverage is configured in `pyproject.toml`: `source = ["src/undercooked"]`, branch
coverage on, `--cov-fail-under=90` (AGENTS.md A20 asks for > 90 % on new/changed code; the
floor is the mechanical half of that).