phase: 02_level_content_and_validation
Phase `02_level_content_and_validation` — final verification **green**. - **Gate:** `bash .agents/validate.sh` → ruff ✅ · pyright 0 errors ✅ · `undercooked-levels validate` OK (2 levels) ✅ · **668 passed** (incl. 10 Playwright e2e) ✅ · coverage **100%** ✅ · repo hygiene ✅ - **Criteria:** `validate` exit 0, one line per level; `list` = `01_first_kitchen`, `02_double_shift` in campaign order; live `:8000` `/healthz` 200, `/api/levels` (+detail) carries both levels with numeric `target_score`/`time_limit` and no grid/station leakage, `/api/config` correct, unknown id → 404, startup log `content loaded: 2 level(s), 4 recipe(s)`; `validate --dir tests/fixtures/levels/_bad/` → exit 1 with named `RECIPE_STATION_MISSING` / `SERVE_REACHABLE` / `TARGET_IMPLAUSIBLE`; A8 lock test proves a YAML-only level reaches loader+API+CLI. - **Defect fixed (pre-existing path, phase-02-triggered):** `undercooked` with unusable content printed the report then a raw traceback and exited `1`. `src/undercooked/__main__.py` now catches `ContentError` → one closing line, exit `2` (like a bad setting); 2 new tests in `tests/unit/test_main.py` (incl. non-content failures still propagate). - **Docs pass (full, project-wide):** README (content gate + exit `2`, config section), AGENTS.md §3, `docs/LEVELS.md` (startup refusal + exact CLI exit codes), `docs/DEPLOYMENT.md` (crash-loop diagnosis bullet); DESIGN/TESTING re-read, consistent. - **Container re-verified (no phase-01 regression):** `podman build -f Containerfile` succeeds; the image ships `levels/` and serves `/healthz` 200, `/api/levels` (2 levels), `/` 200 as non-root. - **Deviation to note:** the phase overview names `tests/fixtures/levels/bad/`; the actual broken-fixture dir is `tests/fixtures/levels/_bad/` (the `_` prefix is what keeps it out of normal discovery). Overview files are read-only to me — README/LEVELS.md document the real path. - No git index/history commands run; all changes left unstaged for the harness. ntfy notification sent.
This commit is contained in:
+12
-2
@@ -1,6 +1,6 @@
|
||||
# TESTING.md — how this project is verified
|
||||
|
||||
**Status:** current as of phase 01 · **Filled in by phase 14**, which re-runs every layer
|
||||
**Status:** current as of phase 02 · **Filled in by phase 14**, which re-runs every layer
|
||||
below against the finished game and records the release-time suite inventory.
|
||||
|
||||
## Purpose
|
||||
@@ -20,7 +20,7 @@ layer live in `tests/conftest.py`, and layer-specific fixtures live in that laye
|
||||
| Directory | Contains | I/O allowed |
|
||||
|---|---|---|
|
||||
| `tests/unit/` | pure logic with no I/O: config parsing, header values, CLI precedence, protocol/validator helpers, and the doc/content guards that keep README, `.env.example`, `PLAN.md` and the code agreeing | none (in-process `TestClient` requests are fine) |
|
||||
| `tests/sim/` | headless determinism and replay tests against the pure kernel (`tests/sim/test_replay_*.py`, golden state hashes) | none — the sim must stay callable without a server |
|
||||
| `tests/sim/` | tests of the pure kernel that need no server: grid geometry, the entity model, the A5 import-purity guard, and the level validator (phase 02) — plus the headless determinism/replay suites with golden state hashes (`tests/sim/test_replay_*.py`) that arrive with the tick in phase 03 | none — the sim must stay callable without a server |
|
||||
| `tests/integration/` | multi-module, in-process tests: app boot through `create_app()`, HTTP routes, static serving, room/protocol behaviour with real WebSocket clients (room + protocol integration is PLAN §9 layer 4, and lives here) | in-process app, loopback sockets |
|
||||
| `tests/e2e/` | one Playwright suite per user story, named `test_usNN_*.py`; multiplayer stories drive **two browser contexts** against one room | real browser, real socket |
|
||||
| `tests/fixtures/` | fixture data only. `tests/fixtures/levels/` is the content dir every test sees — an autouse session fixture points `UNDERCOOKED_LEVELS_DIR` at it so no test can silently read the shipped `levels/` | none |
|
||||
@@ -59,6 +59,16 @@ uv run pyright # types
|
||||
bash .agents/validate.sh # the pipeline gate, in that order + hygiene
|
||||
```
|
||||
|
||||
The gate runs one check that is not a pytest command: `uv run undercooked-levels
|
||||
validate` between pyright and pytest (docs/LEVELS.md **Authoring workflow**). It is
|
||||
unconditional — a campaign that breaks a PLAN §8 rule stops the pipeline even when no
|
||||
test file changed — and it is the same `load_content` + `validate_bundle` pair
|
||||
`create_app()` runs at startup, so it can disagree with the server about nothing.
|
||||
Content tests stay hermetic the same way: the autouse session fixture points
|
||||
`UNDERCOOKED_LEVELS_DIR` at `tests/fixtures/levels/` (a recipe book and zero levels),
|
||||
and a test that wants real levels builds one under `tmp_path` or names `levels/`
|
||||
explicitly. `tests/unit/test_content_not_hardcoded.py` is the A8 lock on top of that.
|
||||
|
||||
Coverage is configured in `pyproject.toml`: `source = ["src/undercooked"]`, branch
|
||||
coverage on, `--cov-fail-under=90` (AGENTS.md A20 asks for > 90 % on new/changed code; the
|
||||
floor is the mechanical half of that).
|
||||
|
||||
Reference in New Issue
Block a user