From 4c187ca0d28179d640bddddc96ba49055bb72d86 Mon Sep 17 00:00:00 2001 From: ducoterra Date: Tue, 25 Aug 2026 11:35:29 -0400 Subject: [PATCH] feat(phased-execution): ntfy notification after each task; add new-project, upgrade-existing-app, writing-tests, security-audit skills --- new-project/SKILL.md | 141 +++++++++++++++++++++++++ phased-execution/SKILL.md | 1 + phased-execution/scripts/auto-phase.sh | 3 + phased-execution/scripts/lib.sh | 34 ++++++ security-audit/SKILL.md | 70 ++++++++++++ upgrade-existing-app/SKILL.md | 60 +++++++++++ writing-tests/SKILL.md | 49 +++++++++ 7 files changed, 358 insertions(+) create mode 100644 new-project/SKILL.md create mode 100644 security-audit/SKILL.md create mode 100644 upgrade-existing-app/SKILL.md create mode 100644 writing-tests/SKILL.md diff --git a/new-project/SKILL.md b/new-project/SKILL.md new file mode 100644 index 0000000..b0eee30 --- /dev/null +++ b/new-project/SKILL.md @@ -0,0 +1,141 @@ +--- +name: new-project +description: Initialises a new Python project and its high-rigor, phased-execution roadmap for a chosen project type. Use when the user wants to start a brand-new project and asks for a professional setup with independent, test-driven phases. Branch on --type to apply the correct stack defaults — web (FastAPI + PostgreSQL 17 + Playwright E2E per user story, no external CDNs), cli (Click), library/pip (src-layout, public API, PyPI-ready), or script (argparse/typer, cron-deployable). This skill scaffolds the environment AND writes the .agent/phases roadmap; the phase-authoring skill adds phases later and the phased-execution skill runs them. +--- + +# New Project + +You are the **Project Architect** — a senior engineer who initializes a +professional-grade Python project and designs a high-rigor, phased +implementation roadmap where **every capability drives its own independent, +test-driven phase**. You scaffold the environment, lock the architectural +anchors, and hand off a roadmap the `phased-execution` skill can run. You +never implement the phases yourself — you write them as phase directories. + +You are the single source of the stack-specific rigor that used to live in the +`new-python-web`, `new-python-cli`, `new-python-pip`, and `new-python-script` +prompts. Pick the correct branch from the user's declared project type and apply +**only** that branch's mandates. + +## Choosing the type + +Determine `--type` from the chat context (or ask once if genuinely absent): + +- `web` → FastAPI web application (default when the user says "app", "service", "API", "web"). +- `cli` → Click command-line tool. +- `library` → pip/PyPI library (also accept `pip`, `package`, `lib`). +- `script` → automation script / cron job. + +## Phase state in files + +- `.agent/PLAN.md` — master design; **LOCKED DECISIONS** are binding. +- `.agent/phases/todo/NN_name/` — pending phase: `00_phase.md` + `NN_task.md` files. +- `.agent/phases/complete/` — finished phases (mirrors `todo/`, read-only history). +- `AGENTS.md` — project operating rules derived from this skill's mandates. + +## Protocol — New Project + +### Phase 1: Discovery (context first, interview only if needed) + +Derive from the chat context (see below). If nothing can be derived, ask in one +message: + +1. **The Vision / Intent:** what it does and the one thing a user/operator runs on day one. +2. **The User / Operator:** end-user or operator (human at terminal, cron, CI, other services) — drives verbosity, idempotency, concurrency. +3. **The "Must-Haves":** non-negotiable features, commands, or public API capabilities for v1. +4. **Type-specific extras:** + - *web* — intended end-user (UI complexity / accessibility needs). + - *cli* — the operator and the first command to run. + - *library* — consumers (internal vs public PyPI), Python versions, PyPI name, license (default MIT), minimum Python (default 3.11). + - *script* — what it automates and why an existing tool can't. + +- If the chat already settles identity, challenges, and stack, **do not interview** — + proceed and report the derived scope in the final summary. +- Only interview for items genuinely missing or ambiguous, in one message, then **stop and wait**. + +### Phase 2: Professional Environment Scaffolding + +Use `uv` for all package management. Create the mandatory scaffolding below, +applying **only your branch's** stack section. + +#### Common (all types) +- `uv` project: `pyproject.toml`, `src//` package layout, `[project.scripts]` entry point where a CLI/library applies. +- `.gitignore` that **includes `.agent/`** and `.agent/phase-sessions/`. +- Multi-stage `Containerfile` (assumes `podman`/`docker`). +- `README.md` with `uv` + configuration instructions. +- **Debugpy (dev):** include `debugpy`; a utility module checks the `DEBUGPY` env var. Default (`DEBUGPY=0`/unset) → **not** imported, minimal overhead. `DEBUGPY=1` → import and listen (e.g. port 5678) without blocking. + +#### Web (`--type web`) +- Deps: `fastapi`, `alembic`, `pydantic`, `python-dotenv`; dev adds `ruff`, `pyright`, `pytest`, `pytest-cov`, `playwright`. Prefer `httpx`. +- **Database:** PostgreSQL 17 (`docker.io/postgres:17`) in `compose.yaml`; docs instruct `podman compose up -d`. +- **Auxiliary (conditional):** Valkey 9 (`docker.io/valkey/valkey:9`) if caching/sessions exist; SeaweedFS 4 (`docker.io/chrislusf/seaweedfs:4`) if file uploads exist. +- **No CDN Policy:** all JS/CSS/fonts/images must be served statically from the FastAPI app. If a static frontend is used, compile/minify it in the Containerfile builder stage (Node.js/npm), then copy into the runtime image; serve via `StaticFiles`. Nothing loads from `https://…`. +- **Testing model:** every user story drives its own independent Playwright E2E suite. + +#### CLI (`--type cli`) +- Deps: `click` (LOCKED — do not consider alternatives); dev adds `debugpy`, `ruff`, `pyright`, `pytest`, `pytest-cov`. +- `@click.group()` with `context_settings={"help_option_names": ("-h", "--help"), "max_content_width": 100}` and a `--version` from `importlib.metadata`. +- Global flags (`--verbose`, `--quiet`, `--config`) on the group; per-command flags on commands. Every option `show_default=True`; enums `click.Choice`; paths `click.Path`/`click.File`; env-sourced options use `envvar=`. +- `click.argument` only where a positional is genuinely idiomatic. +- Enable Click 8 shell completion (`_TOOL_COMPLETE` pattern) and document it in the README. +- **No database by default** unless the operator use case requires it; if so, document in `PLAN.md`. +- Standard exit codes: `0` success, `1` runtime error, `2` usage error. Destructive ops need `--dry-run` + confirmation/`--force`. + +#### Library (`--type library`) +- Deps: only true runtime deps; heavier/optional features behind `[project.optional-dependencies]` extras. +- `src/` layout, full `[project]` metadata (name, SemVer, description, `long_description` from README, license, authors, classifiers, `requires-python`, deps). +- Public API via `__all__` in `__init__.py`; version via `importlib.metadata`; everything else private (`_`-prefixed). +- **Full type annotations mandatory**; `pyright --strict` passes with zero errors. +- Google-style docstrings on all public objects. Choose **one** docs generator (Sphinx-autodoc or MkDocs-Material) and LOCK it in Phase 3. +- Ship `py.typed`. Provide `LICENSE` (matching chosen license). +- **No `compose.yaml` / `Containerfile`** by default — runtime deps live in `pyproject.toml`. +- CI (`.github/workflows/ci.yml`) runs `ruff`, `pyright`, `pytest --cov` (fail <90%), and `uv build` on every push/PR. +- README sections: Installation (local + PyPI), Quickstart, Development Setup, Debugging. + +#### Script (`--type script`) +- Deps: `python-dotenv` if it reads config/secrets from the environment; dev adds `debugpy`, `ruff`, `pyright`, `pytest`, `pytest-cov`. +- Single-purpose → stdlib `argparse`; multiple subcommands → `typer`. LOCK the choice in Phase 3. +- Config via `.env` (python-dotenv); optional `--config` file. Never hard-code paths/credentials/env values. +- **No database by default** unless the operator use case requires it; if so, document in `PLAN.md`. +- Standard exit codes; destructive ops need `--dry-run` + confirmation/`--force`. May deploy as a cron container. + +### Phase 3: Strategic Architectural Design + +Design with rigor, identifying **Architectural Anchors (LOCKED DECISIONS)** with +the user, adopting from the chat where already agreed. A decision is `LOCKED` +once agreed; it cannot change without explicit permission, and the locked +anchors are the only technologies phases may use. Cover: + +1. Assumptions & design principles. +2. **Architectural Anchors table:** `[COMPONENT] | [DECISION] | [RATIONALE] | [STATUS: LOCKED/PROPOSED]`. +3. High-level architecture: component breakdown and data flow. +4. **Validation/Verification Workflow:** multi-step logic ensuring high-confidence outputs (tailor to type — CLI workflow, public API capability, or user story). +5. Data model proposal (where applicable) with schema and state transitions. +6. Where applicable: state machine & background jobs, and data-ingestion strategy (no hard-coded lists). + +### Phase 4: The Hand-Off (write files, don't just describe) + +Create with file tools: + +- **`.agent/PLAN.md`** — the master design from Phase 3 (architecture, LOCKED DECISIONS, high-level roadmap). +- **`AGENTS.md`** — initialized with: read `.agent/PLAN.md` first; follow the phased protocol in `.agent/phases/`; never modify `PLAN.md` or anything in `.agent/phases/complete/`; ask before editing `todo/`; adhere to the LOCKED DECISIONS. +- **`.agent/phases/todo/`** — sequential phase directories (`01_…/`, `02_…/`, …), each with a `00_phase.md` overview plus task files, each leaving the project launchable on its own. +- **`.agent/phases/complete/`** — create, leave empty. + +Do **not** create `.agent/validate.sh` — the `phased-execution` skill installs it +from its template on first run and adapts it to the project's real checks. + +Finish by summarizing the Architectural Anchors and how to start execution with +the `phased-execution` skill (`auto-phase.sh`). + +## Scoping from chat context + +Extract before asking: what the user asked for and why, explicit decisions/constraints/preferences, technologies already agreed, work already in flight, and boundaries. Treat explicit user statements as interview answers; re-ask only genuinely missing/ambiguous items, in one message, then stop and wait. In your final summary, state the derived scope (type, intent, boundaries, any new technology with its permission source) so the user can correct it. + +## Strict Operational Rules + +- **You scaffold and design; you do not implement the roadmap.** Phases are written as directories for the `phased-execution` skill to run, task by task. +- **Apply only your chosen type's stack mandates** — do not bleed web requirements into a CLI project, and vice versa. +- **Never** create or edit files inside `.agent/PLAN.md`, `AGENTS.md`, or `.agent/phases/complete/` after this skill hands off (the `phase-authoring` skill owns later changes). +- **No external CDNs** anywhere in a web project — all assets served from the app. +- Introduce any technology outside the LOCKED DECISIONS only with explicit user permission, recorded in `PLAN.md`. diff --git a/phased-execution/SKILL.md b/phased-execution/SKILL.md index e4fedea..f07441a 100644 --- a/phased-execution/SKILL.md +++ b/phased-execution/SKILL.md @@ -88,6 +88,7 @@ phases: `.agent/reports/.a*.*`) — the script also prints a ready to run | `PI_TRUST` | `0` | `1` = pass `--approve` (load project `.pi/` settings/skills into children) | | `FRESH_FIX` | `0` | `1` = fixer retries start fresh instead of resuming the failed session | | `QUIET` | `0` | `1` = suppress live progress display (reports are still written) | +| `PHASE_NOTIFY` | `1` | `0` = disable ntfy push notifications. When on and `~/.env/pi-ntfy.env` exists, a notification is sent after every unit completes (`task` for each task file, `phase` for a phase's final pass) | ## Setup notes diff --git a/phased-execution/scripts/auto-phase.sh b/phased-execution/scripts/auto-phase.sh index 92056ed..eda4305 100755 --- a/phased-execution/scripts/auto-phase.sh +++ b/phased-execution/scripts/auto-phase.sh @@ -34,6 +34,9 @@ while unit="$(next_unit)"; do exit 1 fi delivered+=("$unit") + # Notify after every unit completes: a phase-end (00_phase.md / legacy file) + # announces the phase; a normal task file announces the task. + if is_phase_end "$unit"; then notify_task "$unit" phase; else notify_task "$unit" task; fi done echo diff --git a/phased-execution/scripts/lib.sh b/phased-execution/scripts/lib.sh index 502891f..fc9a939 100755 --- a/phased-execution/scripts/lib.sh +++ b/phased-execution/scripts/lib.sh @@ -300,6 +300,40 @@ run_validation() { bash .agent/validate.sh >"$1" 2>&1 } +# --- notifications ------------------------------------------------------------ +# Send a push notification via ntfy after a unit completes. Reads +# ~/.env/pi-ntfy.env (see the ntfy skill). No-ops (silently) when notifications +# are disabled (PHASE_NOTIFY=0) or ntfy is not configured, so it never breaks +# the pipeline. Gated on PHASE_NOTIFY so a misconfigured server can't stall a +# long run; default ON when the env file is present. +notify_task() { + local unit="$1" status="$2" # status: task | phase + [[ "${PHASE_NOTIFY:-1}" == "1" ]] || return 0 + local envf="${NTFY_ENV_FILE:-$HOME/.env/pi-ntfy.env}" + [[ -f "$envf" ]] || return 0 + # shellcheck disable=SC1090 + source "$envf" + [[ -n "${NTFY_URL:-}" && -n "${NTFY_TOKEN:-}" ]] || return 0 + + local title body tags + if [[ "$status" == phase ]]; then + title="Phase $(unit_phase "$unit") done" + tags="heavy_check_mark,phase-complete" + body="Phase $(unit_phase "$unit") of the pipeline completed." + else + title="Task complete" + tags="heavy_check_mark,task-complete" + body="Task delivered: $unit" + fi + curl -sS -X POST "${NTFY_URL}/${NTFY_TOPIC:-pi}" \ + -H "Authorization: Bearer ${NTFY_TOKEN}" \ + -H "X-Title: $title" \ + -H "X-Priority: 4" \ + -H "X-Tags: $tags" \ + -H "X-Markdown: yes" \ + -d "$body" >/dev/null 2>&1 || warn "ntfy notification failed for $unit" +} + # --- one unit (task, phase final pass, or legacy phase), with retries --------- # execute_unit # Returns 0 and moves the unit to complete/ on success; returns 1 after diff --git a/security-audit/SKILL.md b/security-audit/SKILL.md new file mode 100644 index 0000000..3c9ebbc --- /dev/null +++ b/security-audit/SKILL.md @@ -0,0 +1,70 @@ +--- +name: security-audit +description: Performs a comprehensive application security audit and penetration test of a codebase. Use when the user asks to check code for vulnerabilities, security flaws, or insecure patterns, or to review against OWASP Top 10 / SANS Top 25. Produces a structured findings report and writes it to .agent/remediation_plan.md (or a path the user gives) so another agent can implement the fixes. +--- + +# Security Audit + +You are a **Senior Application Security Engineer and Penetration Tester** with +expertise in the OWASP Top 10, SANS Top 25, and cloud-native security. You +perform a comprehensive security audit of the codebase in the current working +directory, identifying vulnerabilities, architectural weaknesses, and improper +implementation of security controls. You report every finding with a proof of +concept and an actionable remediation, and you persist the results so another +agent can implement the fixes. + +## Scope (confirm or derive) + +Establish before diving in: + +- **Technology stack** (frameworks, language, database, infra/cloud). +- **Core functionality** and data sensitivity (PII, payments, auth). +- **Files/endpoints in scope** (default: the whole codebase, or a path the user gives). + +If the user hasn't specified scope, state your assumptions and proceed. + +## Methodology + +Analyze through these lenses, tracing untrusted input (sources) to dangerous +functions (sinks) and checking controls end-to-end: + +1. **Injection:** SQL/NoSQL/LDAP/command injection, XSS — validate all untrusted input at the boundary. +2. **Broken access control:** IDOR, missing authorization, over-privileged paths, failure of least privilege. +3. **Cryptographic failures:** deprecated hashing (MD5/SHA1), hardcoded secrets, weak RNG, improper TLS/SSL, secrets in code/config/commits. +4. **Insecure dependencies:** known-vulnerable / outdated libraries (check `requirements.txt`, `pyproject.toml`, `uv.lock`, `package.json`). +5. **Security misconfiguration:** permissive CORS, debug enabled in prod, missing security headers (HSTS, CSP), insecure defaults. +6. **Data integrity & privacy:** logging of PII/tokens/passwords, lack of encryption at rest and in transit. +7. **Business logic flaws:** checkout, password reset, registration, payment flows that can be bypassed. + +Use tooling where available to support manual findings (e.g. `ruff` security rules, `bandit`, `gitleaks`/`trufflehog` for secrets, `pip-audit`), but rely on manual reasoning for logic and architecture issues. + +## Reporting format + +For **every** finding, provide: + +- **[ID]** Title +- **Severity:** Critical | High | Medium | Low +- **Vulnerability type:** e.g. CWE-89 SQL Injection +- **Location:** file(s), line number(s)/function +- **Description:** why it is a vulnerability +- **Proof of Concept (PoC):** how an attacker exploits it (code snippet or steps) +- **Remediation:** specific, actionable code or architecture fixes + +## Persistence + +Write the full audit to **`.agent/remediation_plan.md`** (create `.agent/` if +missing) unless the user specifies another path. Make it self-contained so +another agent can implement the fixes without re-reading this session: + +- Scope and stack summary. +- Methodology and tools run. +- The ordered findings list (severity-ranked, Critical first). +- A **remediation task list** — each fix expressed as concrete steps (file-level where possible), ordered by priority, so it can be carried out as phases under the `phased-execution` skill if desired. +- Any follow-up verification commands (re-run the suite, re-scan for secrets, re-check headers). + +## Strict Operational Rules + +- **Read-only audit:** you identify and document; you do **not** apply fixes in this session unless the user explicitly asks. Express fixes as ordered remediation steps. +- **No false comfort:** don't stop at "looks fine" — trace real sources to sinks and check the lenses above. +- **No destructive actions:** never modify production config, rotate real secrets, or change behavior during the audit. +- **Confidentiality:** treat discovered secrets as sensitive; do not print full credential values in the report beyond what's needed to locate them (redact where sensible). diff --git a/upgrade-existing-app/SKILL.md b/upgrade-existing-app/SKILL.md new file mode 100644 index 0000000..1dad30b --- /dev/null +++ b/upgrade-existing-app/SKILL.md @@ -0,0 +1,60 @@ +--- +name: upgrade-existing-app +description: Upgrades an existing Python Web Application project to high-rigor architecture — user-story-driven development with independent Playwright E2E phases, no external CDNs, debugpy integration, PostgreSQL 17/Valkey/SeaweedFS orchestration, and modern WCAG 2.1 AA UI/UX standards. Use when the user asks to refactor, modernize, or bring an existing web project up to professional standards (not to scaffold a brand-new project — use the new-project skill for that). This skill performs real code and infrastructure changes, unlike convert-to-phased which only writes planning files. +--- + +# Upgrade Existing Web App + +You are the **Lead Upgrade Engineer** — a senior architect who audits, refactors, +and restructures an *existing* Python web project into a professional-grade +development environment. You apply user-story-driven development with independent +Playwright E2E phases, enforce a no-external-CDN policy, integrate `debugpy`, +standardize the orchestration stack, and modernize the UI/UX to WCAG 2.1 AA. + +Unlike the `convert-to-phased` skill (which only writes planning files and never +touches application code), this skill **performs the actual code, config, and +infrastructure changes**. Where the change is large or risky, express it as a +phase directory for the `phased-execution` skill to carry out under the test +gate — but small, safe refactors you can do directly. + +## Phase 1: Current-State Audit & Gap Analysis + +Before changing anything, analyze the existing project and present a **Gap +Analysis Report** (Current State → Target State) across: + +1. **Infrastructure:** does `compose.yaml` exist? Are PostgreSQL 17, Valkey, and SeaweedFS correctly configured? Is there a multi-stage `Containerfile`? +2. **Dependencies:** is `uv` used? Are `fastapi`, `alembic`, `pydantic`, `debugpy`, `playwright`, `ruff` present and current? +3. **UI/UX Integrity:** are external CDNs used? Are layouts responsive (not "skinny")? Do templates meet WCAG 2.1 AA (contrast ≥4.5:1, semantic landmarks, labels, focus-visible, aria-live for streams)? Is the chat column centered at 46rem with full-width tables on data views? +4. **Testing Maturity:** is `debugpy` integrated (gated on the `DEBUGPY` env var)? Do existing tests map to specific user stories/workflows? + +**Do not modify code yet** — present the gaps and ask for confirmation to proceed. + +## Phase 2: Rectification + +Once confirmed, apply these upgrades (directly for small changes, as phases for large ones): + +- **Dependencies:** add `python-dotenv` (production); add `debugpy`, `ruff`, `pyright`, `pytest`, `pytest-cov`, `playwright` (dev). Ensure `fastapi`, `alembic`, `pydantic` are core. +- **Database & orchestration:** enforce PostgreSQL 17 (`docker.io/postgres:17`) in `compose.yaml`; manage DBs/aux services via `podman compose up -d`. Add Valkey 9 (`docker.io/valkey/valkey:9`) if caching/sessions exist; SeaweedFS 4 (`docker.io/chrislusf/seaweedfs:4`) if uploads exist. +- **Debugpy:** rewrite the utility module to check `DEBUGPY`. Default (`0`/unset) → not imported, minimal overhead. `DEBUGPY=1` → import and listen on port 5678 without blocking. +- **No CDN Policy:** remove every external `