undercooked/docs/TESTING.md

TESTING.md — how this project is verified

Status: current as of phase 02 · Filled in by phase 14, which re-runs every layer below against the finished game and records the release-time suite inventory.

Purpose

This document is the single reference for how we know the game is right: the test tree, what belongs in each directory, how to run each layer without waiting for the others, the marker set, the coverage floor, and the artifact policy. The layered verification workflow behind it is .agents/PLAN.md §9; the gate that enforces all of it before any phase completes is .agents/validate.sh.

Layout

tests/ is deliberately not a package (no __init__.py); fixtures shared by every layer live in tests/conftest.py, and layer-specific fixtures live in that layer's own conftest.py.

Directory	Contains	I/O allowed
tests/unit/	pure logic with no I/O: config parsing, header values, CLI precedence, protocol/validator helpers, and the doc/content guards that keep README, .env.example, PLAN.md and the code agreeing	none (in-process TestClient requests are fine)
tests/sim/	tests of the pure kernel that need no server: grid geometry, the entity model, the A5 import-purity guard, and the level validator (phase 02), plus the tick itself (phase 03) — the seeded RNG's published golden vectors (test_rng.py), the kernel's clamping/accumulation/purity/determinism suite (test_step_kernel.py), and the movement stage: hand-computed geometry per rule (test_movement.py), a scripted 2000-substep four-cook walk on the sim's own RNG that re-checks containment, non-overlap, bounded displacement and hash equality after every substep (test_movement_stress.py), and the same walk through the whole step() on the shipped levels (test_movement_kernel.py), and the action button (phase 03 task 03): the reach-zone geometry and one-press-one-event press edge (test_targeting.py), and the table-driven verb ladder that covers every Verb and every DeniedReason plus the gate's contested arbitration (test_verbs.py); the golden-state-hash replay suites — test_replay_carry.py (task 04's two-player pass and contest), test_replay.py (the committed goldens against sim/replay.py's input-script format, the record/check CLI and the perf seam) — and the slow-marked perf smoke (test_perf_smoke.py)	none — the sim must stay callable without a server
tests/integration/	multi-module, in-process tests: app boot through create_app(), HTTP routes, static serving, room/protocol behaviour with real WebSocket clients (room + protocol integration is PLAN §9 layer 4, and lives here)	in-process app, loopback sockets
tests/e2e/	one Playwright suite per user story, named test_usNN_*.py; multiplayer stories drive two browser contexts against one room	real browser, real socket
tests/fixtures/	fixture data only. tests/fixtures/levels/ is the content dir every test sees — an autouse session fixture points UNDERCOOKED_LEVELS_DIR at it so no test can silently read the shipped levels/ (_bad/ holds the deliberately-broken campaign the _ discovery rule ignores; _replay/ holds the pick_drop_pass fixture kitchen, ignored the same way). tests/fixtures/levels_playable/ is the same recipe book plus one playable level — the only fixture campaign a live-server room can start a round on, which the WebSocket input suite (task 03) uses — the live-server suites against the default fixture set cannot start a round at all (it has no levels). tests/fixtures/replays/ holds the committed input scripts and tests/goldens/ the committed golden hashes they are gated on (sim/replay.py)	none

There is no tests/server/: PLAN §9 counts server/route layers of coverage, not directories. A route asserted as a pure function (header values, status codes, CLI precedence) is a tests/unit/ test; a route exercised against a process that actually binds a socket is tests/integration/. Adding a fifth directory would only hide which of those two a test is.

Markers

Registered in pyproject.toml and enforced with --strict-markers:

Marker	Meaning	Default run
unit	pure, no I/O, fast	yes
integration	multi-module, in-process server/room tests	yes
e2e	Playwright browser suites (needs uv run playwright install chromium)	yes
slow	replay/stress suites that exceed the quick loop (the 200-room stress run lives here from phase 12)	yes, excluded explicitly

The everyday loop is uv run pytest -m "not e2e" (no browser launches). Before a phase completes, and whenever .agents/validate.sh runs, the full suite runs — coverage gate included — because the gate is "the entire project is green", not "my files are green".

Running everything

uv sync --extra dev                            # dependencies + test toolchain
uv run playwright install chromium             # once, before the first browser run
uv run pytest -q -m "not e2e"                  # quick loop
uv run pytest -q                               # everything, with the coverage floor
uv run pytest tests/unit -q -p no:cacheprovider # one file, no nested coverage writes
uv run ruff check .                            # lint (E,F,W,I,N,UP,B,C4,SIM,ASYNC,S,RUF,ARG,RET,PL,PERF)
uv run pyright                                 # types
bash .agents/validate.sh                       # the pipeline gate, in that order + hygiene

The gate runs one check that is not a pytest command: uv run undercooked-levels validate between pyright and pytest (docs/LEVELS.md Authoring workflow). It is unconditional — a campaign that breaks a PLAN §8 rule stops the pipeline even when no test file changed — and it is the same load_content + validate_bundle pair create_app() runs at startup, so it can disagree with the server about nothing. Content tests stay hermetic the same way: the autouse session fixture points UNDERCOOKED_LEVELS_DIR at tests/fixtures/levels/ (a recipe book and zero levels), and a test that wants real levels builds one under tmp_path or names levels/ explicitly. tests/unit/test_content_not_hardcoded.py is the A8 lock on top of that.

Coverage is configured in pyproject.toml: source = ["src/undercooked"], branch coverage on, --cov-fail-under=90 (AGENTS.md A20 asks for > 90 % on new/changed code; the floor is the mechanical half of that).

Test processes never listen on a routable interface. The only fixture that binds a socket is live_server (session-scoped, tests/conftest.py), which asks uvicorn for an ephemeral 127.0.0.1 port and shuts the lifespan down cleanly.

Artifact policy

• Test output is disposable and never committed: test-results/, playwright-report/, blob-report/, .playwright/, .pytest_cache/, .coverage, htmlcov/, coverage.xml, coverage.json are all gitignored.
• No test writes into app/static/, levels/, src/, or docs/; anything a test needs is created under tmp_path.
• Nothing is downloaded during a test run: browsers are installed explicitly by a human (or CI step), and no test reaches the network except against 127.0.0.1.
• A test that cannot pass deterministically is a bug in the test — sleeps are bounded by conditions, and the sim-facing suites are seeded through the random_seed fixture.