phase: 01_runnable_service_scaffold
**Phase 01 final verification pass — all criteria green**
- Verified (nothing left to implement; all 4 task files already in `complete/`): toolchain lock, typed `Settings`+`.env.example`, debugpy bootstrap, app factory, CLI (`--host/--port/--reload/--levels-dir`, `python -m undercooked`), static shell + `/static`, security headers, no-CDN/doc guards, `validate.sh`.
- `uv sync --extra dev` → exit 0; `uv.lock` present (untracked → harness commits it with the phase).
- `uv run undercooked --host 127.0.0.1 --port 8000` → `curl /healthz` = 200 `{"status":"ok","version":"0.1.0","rooms":0}` (superset of the criterion's body — now documented in README); `curl /` = 200 shell HTML; static css/js 200; CSP/nosniff/no-referrer/DENY on 200, 404 and 405 too.
- `DEBUGPY=1`: port 5678 is held on this machine by the user's own podman dev-container debugpy adapter — **no live change made/killed**; verified via `DEBUGPY=1 DEBUGPY_PORT=5679` → listener opens, `/healthz` still 200; the occupied-port case degraded gracefully as designed (warning, server serves). Bad config → one line + exit 2 ✓.
- `bash .agents/validate.sh` (root **and** `.agents/`) → exit 0: `ruff check .` 0 errors, `pyright` 0 errors, levels-CLI notice-skip (matches README prose), **284 passed** full suite incl. 10 Playwright e2e, coverage **100.00%** (floor 90), repo hygiene clean. `uv run pytest -m e2e` → 10 passed.
- Docs fixed: README Status → phase 01 complete + next pending 02 (was pointing at `todo/01…`), documented real `/healthz` body, precedence now "CLI flags > env > .env > default", added the PLAN §13-promised **How to play** section (honest about shell-only today), Layout rows marked what exists vs phases 02/07. AGENTS.md: recorded the `data:`-URL exemption in the no-CDN rule (test allows it) and matched the session-token rule to the browser's WebCrypto/base64url implementation. Also corrected a stale "session-wide chromium" docstring in `tests/e2e/conftest.py` (fixtures are function-scoped).
- Deviations: no container build run (needs an image pull; not a phase criterion — `Containerfile`/`compose.yaml` reviewed and consistent with `docs/DEPLOYMENT.md`); `HEAD /` is 405 (routes are GET-only by design; health checks use GET). No git index/history commands run.
- Next pending phase: `02_level_content_and_validation`.
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
# TESTING.md — how this project is verified
|
||||
|
||||
**Status:** current as of phase 01 · **Filled in by phase 14**, which re-runs every layer
|
||||
below against the finished game and records the release-time suite inventory.
|
||||
|
||||
## Purpose
|
||||
|
||||
This document is the single reference for *how we know the game is right*: the test tree,
|
||||
what belongs in each directory, how to run each layer without waiting for the others, the
|
||||
marker set, the coverage floor, and the artifact policy. The layered verification workflow
|
||||
behind it is `.agents/PLAN.md` §9; the gate that enforces all of it before any phase
|
||||
completes is `.agents/validate.sh`.
|
||||
|
||||
## Layout
|
||||
|
||||
`tests/` is deliberately **not** a package (no `__init__.py`); fixtures shared by every
|
||||
layer live in `tests/conftest.py`, and layer-specific fixtures live in that layer's own
|
||||
`conftest.py`.
|
||||
|
||||
| Directory | Contains | I/O allowed |
|
||||
|---|---|---|
|
||||
| `tests/unit/` | pure logic with no I/O: config parsing, header values, CLI precedence, protocol/validator helpers, and the doc/content guards that keep README, `.env.example`, `PLAN.md` and the code agreeing | none (in-process `TestClient` requests are fine) |
|
||||
| `tests/sim/` | headless determinism and replay tests against the pure kernel (`tests/sim/test_replay_*.py`, golden state hashes) | none — the sim must stay callable without a server |
|
||||
| `tests/integration/` | multi-module, in-process tests: app boot through `create_app()`, HTTP routes, static serving, room/protocol behaviour with real WebSocket clients (room + protocol integration is PLAN §9 layer 4, and lives here) | in-process app, loopback sockets |
|
||||
| `tests/e2e/` | one Playwright suite per user story, named `test_usNN_*.py`; multiplayer stories drive **two browser contexts** against one room | real browser, real socket |
|
||||
| `tests/fixtures/` | fixture data only. `tests/fixtures/levels/` is the content dir every test sees — an autouse session fixture points `UNDERCOOKED_LEVELS_DIR` at it so no test can silently read the shipped `levels/` | none |
|
||||
|
||||
There is **no `tests/server/`**: PLAN §9 counts server/route layers of coverage, not
|
||||
directories. A route asserted as a pure function (header values, status codes, CLI
|
||||
precedence) is a `tests/unit/` test; a route exercised against a process that actually
|
||||
binds a socket is `tests/integration/`. Adding a fifth directory would only hide which of
|
||||
those two a test is.
|
||||
|
||||
## Markers
|
||||
|
||||
Registered in `pyproject.toml` and enforced with `--strict-markers`:
|
||||
|
||||
| Marker | Meaning | Default run |
|
||||
|---|---|---|
|
||||
| `unit` | pure, no I/O, fast | yes |
|
||||
| `integration` | multi-module, in-process server/room tests | yes |
|
||||
| `e2e` | Playwright browser suites (needs `uv run playwright install chromium`) | yes |
|
||||
| `slow` | replay/stress suites that exceed the quick loop (the 200-room stress run lives here from phase 12) | yes, excluded explicitly |
|
||||
|
||||
The everyday loop is `uv run pytest -m "not e2e"` (no browser launches). Before a phase
|
||||
completes, and whenever `.agents/validate.sh` runs, the **full** suite runs — coverage gate
|
||||
included — because the gate is "the entire project is green", not "my files are green".
|
||||
|
||||
## Running everything
|
||||
|
||||
```bash
|
||||
uv sync --extra dev # dependencies + test toolchain
|
||||
uv run playwright install chromium # once, before the first browser run
|
||||
uv run pytest -q -m "not e2e" # quick loop
|
||||
uv run pytest -q # everything, with the coverage floor
|
||||
uv run pytest tests/unit -q -p no:cacheprovider # one file, no nested coverage writes
|
||||
uv run ruff check . # lint (E,F,W,I,N,UP,B,C4,SIM,ASYNC,S,RUF,ARG,RET,PL,PERF)
|
||||
uv run pyright # types
|
||||
bash .agents/validate.sh # the pipeline gate, in that order + hygiene
|
||||
```
|
||||
|
||||
Coverage is configured in `pyproject.toml`: `source = ["src/undercooked"]`, branch
|
||||
coverage on, `--cov-fail-under=90` (AGENTS.md A20 asks for > 90 % on new/changed code; the
|
||||
floor is the mechanical half of that).
|
||||
|
||||
Test processes never listen on a routable interface. The only fixture that binds a socket
|
||||
is `live_server` (session-scoped, `tests/conftest.py`), which asks uvicorn for an
|
||||
ephemeral `127.0.0.1` port and shuts the lifespan down cleanly.
|
||||
|
||||
## Artifact policy
|
||||
|
||||
- Test output is disposable and **never committed**: `test-results/`, `playwright-report/`,
|
||||
`blob-report/`, `.playwright/`, `.pytest_cache/`, `.coverage`, `htmlcov/`,
|
||||
`coverage.xml`, `coverage.json` are all gitignored.
|
||||
- No test writes into `app/static/`, `levels/`, `src/`, or `docs/`; anything a test needs
|
||||
is created under `tmp_path`.
|
||||
- Nothing is downloaded during a test run: browsers are installed explicitly by a human
|
||||
(or CI step), and no test reaches the network except against `127.0.0.1`.
|
||||
- A test that cannot pass deterministically is a bug in the test — sleeps are bounded by
|
||||
conditions, and the sim-facing suites are seeded through the `random_seed` fixture.
|
||||
Reference in New Issue
Block a user