diff --git a/convert-to-phased/SKILL.md b/convert-to-phased/SKILL.md index c508516..2aaca1d 100644 --- a/convert-to-phased/SKILL.md +++ b/convert-to-phased/SKILL.md @@ -88,7 +88,7 @@ Create (or merge into what already exists) with file tools: - **`AGENTS.md`** — the five base rules below, plus the domain additions. - **`.agent/phases/todo/`** — the roadmap from Phase 5. - **`.agent/phases/complete/`** — create the directory, leave it empty (it mirrors `todo/` as phases complete). -- **`.gitignore`** — add `.agent/` and `.agent/phase-sessions/` if missing. +- **`.gitignore`** — add `.agent/phase-sessions/` and `.agent/pipeline.log` if missing (never `.agent/` itself — the planning tree is tracked and committed); if an existing `.gitignore` has a `.agent/` ignore line, remove it. **AGENTS.md base rules:** @@ -189,9 +189,9 @@ final pass is where they are verified end-to-end): ## Phase 6 — Version Control & Hand-Off - Git is mandatory: `git init` if the project is not a repository. -- Commit the conversion — `AGENTS.md`, `.gitignore`, and any other modified - non-`.agent/` files (the `.agent/` tree itself is git-ignored by protocol) — - with a Conventional Commits message (e.g. `chore(agent): adopt phased +- Commit the conversion — `AGENTS.md`, `.gitignore`, the whole `.agent/` + tree (it is tracked, not git-ignored), and any other modified files — with + a Conventional Commits message (e.g. `chore(agent): adopt phased execution strategy with NN-phase roadmap`), always with `--no-gpg-sign`. - Record in `AGENTS.md`: atomic commit at the conclusion of every completed phase, Conventional Commits messages, and `--no-gpg-sign` on every diff --git a/interactive-browser/SKILL.md b/interactive-browser/SKILL.md index 6d805e3..15a199b 100644 --- a/interactive-browser/SKILL.md +++ b/interactive-browser/SKILL.md @@ -255,6 +255,7 @@ done - **Secrets** — when possible, ask the user to type passwords themselves in the visible window rather than passing them through chat/commands - **Screenshots are saved** to the temp directory with timestamps — you can review them later - **Interaction timeouts are 8s** — failures return a JSON error fast; check the error's `details` for Playwright's call log +- **Trust the screenshot image, not a timing theory.** After `/screenshot`, always actually `read` the saved PNG before concluding anything. Never declare a page "blank" or "not rendered" based on an assumption about render timing or a navigation race — verify by opening the file. If it looks blank, re-capture once; if it still looks blank, run `evaluate` on `document.body.scrollHeight` to confirm rather than assuming. This skill's screenshots are rendered correctly; false "blank" reports come from not looking at the actual image. - **Always close the browser** when done to free resources - **The server stays running** until explicitly closed — you can launch, do other work, then come back and continue interacting @@ -269,6 +270,7 @@ done | Click succeeds but nothing happens | Element is covered or JS-gated | Retry with `force=true`, then screenshot to verify | | `Missing required parameter: X` | Param in URL but not decoded properly | Move it to the body with `--data-urlencode "X=..."` | | Stale page content after navigation | Page still loading / lazy content | `reload`, or scroll before extracting | +| Screenshot looks blank (but page is fine) | Agent assumed a render race without reading the image | `read` the actual PNG; re-capture once; check `document.body.scrollHeight` — don't assume blank | | Server port busy | Old server instance | `kill $(lsof -ti:9876)` before starting | ## Example Session diff --git a/new-project/SKILL.md b/new-project/SKILL.md index b0eee30..38f6a3b 100644 --- a/new-project/SKILL.md +++ b/new-project/SKILL.md @@ -60,7 +60,7 @@ applying **only your branch's** stack section. #### Common (all types) - `uv` project: `pyproject.toml`, `src//` package layout, `[project.scripts]` entry point where a CLI/library applies. -- `.gitignore` that **includes `.agent/`** and `.agent/phase-sessions/`. +- `.gitignore` that **includes `.agent/phase-sessions/` and `.agent/pipeline.log`** — but never `.agent/` itself (the phase roadmap is tracked and committed). - Multi-stage `Containerfile` (assumes `podman`/`docker`). - `README.md` with `uv` + configuration instructions. - **Debugpy (dev):** include `debugpy`; a utility module checks the `DEBUGPY` env var. Default (`DEBUGPY=0`/unset) → **not** imported, minimal overhead. `DEBUGPY=1` → import and listen (e.g. port 5678) without blocking. @@ -125,6 +125,11 @@ Create with file tools: Do **not** create `.agent/validate.sh` — the `phased-execution` skill installs it from its template on first run and adapts it to the project's real checks. +- **Version control:** `git init` if the project is not a repository, then + commit the scaffold — including the whole `.agent/` tree (tracked, never + git-ignored; only `.agent/phase-sessions/` and `.agent/pipeline.log` are + ignored) — with a Conventional Commits message, always `--no-gpg-sign`. + Finish by summarizing the Architectural Anchors and how to start execution with the `phased-execution` skill (`auto-phase.sh`). diff --git a/new-service/SKILL.md b/new-service/SKILL.md new file mode 100644 index 0000000..6ca35d4 --- /dev/null +++ b/new-service/SKILL.md @@ -0,0 +1,206 @@ +--- +name: new-service +description: "Creates a new service in the deployment repo from the template. Use when the user wants to add a new service (e.g. 'add a new service', 'create a service', 'new service for X'). Asks for the service name, domain, and target group, then scaffolds the full directory structure with certbot, nginx, app (postgres 17 + valkey + app container), route53, and a README with deployment instructions. Includes Requires= directives, health checks, and proper env templates." +--- + +# New Service + +You are the **Infrastructure Scaffolder** — a senior DevOps engineer who creates +new services in the Ansible deployment repo. You scaffold the complete directory +structure from the template, substituting the user's choices for all placeholders. +You always include Postgres 17 and Valkey by default, and you produce a README +with exact deployment commands. + +## Context + +The deployment repo lives at `/var/home/ducoterra/Deployments`. The template +resides at `/var/home/ducoterra/Deployments/template/` and contains skeleton +playbooks for the four-stage deploy: `init`, `certbot`, `nginx`, `app`. + +### Template structure + +``` +template/ +├── app/ # Quadlet containers + env files + app playbook +│ ├── playbook.yaml +│ ├── {foobar}.network +│ ├── {foobar}-app.container +│ ├── {foobar}-postgres.container +│ ├── {foobar}-valkey.container +│ ├── {foobar}.env.j2 +│ └── {foobar}-postgres.env.j2 +│ +# Note: Container naming varies by service (e.g. litellm uses litellm-db, +# immich uses immich-database). The template uses {foobar}-postgres as the +# default. Adjust names consistently across all files if changing. +├── certbot/playbook.yaml +├── nginx/playbook.yaml +├── route53/records.json +└── README.md +``` + +### Variables + +| Placeholder | Meaning | Example | +|-------------|---------|---------| +| `{foobar}` | Service name (kebab-case) | `litellm`, `openwebui`, `my-app` | +| `{domain}` | Top-level domain | `reeseapps`, `stackexpected`, `emmaleaf`, `wfc` | + +### Targets + +| Target | Location | Notes | +|--------|----------|-------| +| `stackexpected` | `stackexpected/{foobar}/` | Own services | +| `reeseapps` | `reeseapps/{foobar}/` | Personal/family services | +| `emmaleaf` | `emmaleaf/{foobar}/` | Third-party staging on KVM VM | +| `wfc` | `wfc/{foobar}/` | Third-party staging on KVM VM | + +### Deploy order + +1. `init` — system setup (dnf packages, svc user, SSH keys) +2. `certbot` — TLS certificates (DNS-01 challenge, requires AWS creds) +3. `nginx` — reverse proxy with SSL termination +4. `app` — deploy Quadlet containers (postgres, valkey, app) + +### Certbot playbook pattern + +The certbot playbook uses `install_certbot` role. The `domains` list contains +the service domain. For services with wildcard or multiple subdomains, add all +of them. The `ntfy_*` vars are optional — set to `""` to disable notifications. + +### Nginx playbook pattern + +The nginx playbook uses `install_nginx` role. The `nginx_http_conf` contains +the full server block. The `proxy_pass` port defaults to `8080` but should be +adjusted based on the app's actual listening port. + +### Route53 pattern + +The `records.json` contains a CNAME record pointing to `home.reeselink.com.` +This is a default — adjust if the service uses a different target (e.g. an +AWS ALB, a different VM, etc.). + +## Protocol + +### Phase 1: Gather Requirements + +Extract from chat context first. Ask only what's genuinely missing, in one +message: + +1. **Service name** (`{foobar}`) — kebab-case, unique within the target. +2. **Target** (`{domain}`) — `stackexpected`, `reeseapps`, `emmaleaf`, or `wfc`. +3. **Domain suffix** — usually matches the target (e.g. `reeseapps` → `reeseapps.com`), but ask if the user wants something different. +4. **App listening port** — what port does the app listen on internally? (default: `8080`) +5. **Extra domains** — any additional domains/subdomains that need certificates? (e.g. `api.{service}.{domain}.com`) +6. **Route53 target** — what should the CNAME point to? Default: `home.reeselink.com.` +7. **VM RAM** — how much memory? Default: 2048MB. Heavier services (immich) need 8192MB. + +Rules: +- Treat explicit user statements as answers. Do not re-ask. +- If the user says "add a service called litellm for reeseapps", you already + have the name and target — only ask for port, extra domains, and route53 target. +- If nothing can be derived, ask all six items in one message, then stop. + +### Phase 2: Scaffold the Directory + +Create the full directory tree under the chosen target: + +``` +{target}/{foobar}/ +├── app/ +│ ├── playbook.yaml +│ ├── {foobar}.network +│ ├── {foobar}-app.container +│ ├── {foobar}-postgres.container +│ ├── {foobar}-valkey.container +│ ├── {foobar}.env.j2 +│ └── {foobar}-postgres.env.j2 +├── certbot/ +│ └── playbook.yaml +├── nginx/ +│ └── playbook.yaml +├── route53/ +│ └── records.json +└── README.md +``` + +Use the template files as sources and substitute `{foobar}` and `{domain}` +throughout. Key customizations: + +#### `app/playbook.yaml` +- Use the template's `app/playbook.yaml` (which already includes postgres, valkey, app containers). +- If the user has extra quadlet files to copy (e.g. custom service timers), add them to `copy_svc_quadlet_files`. +- If the service exposes a port directly (not via nginx), add `expose_ports` with the port number and include the `configure_firewalld` role (see immich playbook). +- **Service names** in `restart_svc_services`: use the container name without `.service` suffix (consistent with most services; litellm uses `.service` suffix but that's an anomaly). + +#### `certbot/playbook.yaml` +- Set `domains` to include `{foobar}.{domain}.com` plus any extra domains. +- Set `account_name` to `{domain}`. +- Include `ntfy_url`, `ntfy_topic`, `ntfy_tags` for notifications (optional — default to empty/disabled). + +#### `nginx/playbook.yaml` +- Set `server_name` to `{foobar}.{domain}.com`. +- Set `proxy_pass` port to the user's specified port (default `8080`). +- Include the full server block with SSL, proxy headers, and client_max_body_size. +- **Timeouts**: Services like litellm/immich use `send_timeout`, `proxy_read_timeout`, etc. with high values (1800s). Only add these if the user requests them. +- **client_max_body_size**: Default to `100m`. Services that handle large uploads (immich) use `1000m`. + +#### `route53/records.json` +- Set `Name` to `{foobar}.{domain}.com`. +- Set `ResourceRecords.Value` to the user's route53 target (default `home.reeselink.com.`). + +#### `app/{foobar}.env.j2` +- Include `DATABASE_URL` using `postgresql://` scheme (not `postgresql+psycopg://`) pointing to `{foobar}-postgres:5432/{foobar}`. +- Include `VALKEY_URL` using `redis://` scheme pointing to `{foobar}-valkey:6379/0`, OR separate `REDIS_HOST`/`REDIS_PORT` vars (litellm style). +- Include placeholder secrets using `{{ lookup('env', 'VAR_NAME') }}`. +- Keep `APP_ENVIRONMENT=development` as a default. + +#### `app/{foobar}-postgres.env.j2` +- Set `POSTGRES_USER={foobar}`, `POSTGRES_DB={foobar}`. +- Set `POSTGRES_PASSWORD` from `{{ lookup('env', 'APP_DB_PASSWORD') }}`. +- **Note**: Some services (e.g. litellm) hardcode credentials inline in the container file using `Environment=` instead of an env file. The template uses the env file approach (more flexible, matches immich). + +#### `README.md` +- Generate a README with: + - Service name and description (use a meaningful title, e.g. "reeseapps-litellm") + - Deployment commands (exact `ansible-playbook` invocations) + - Secret locations (pass paths — suggest `pass {target}/{foobar}/...`) + - Inventory entry example + - SSH config example + - **VM RAM**: Default to 2048MB. Heavier services (immich) need 8192MB — ask the user if unsure. + +### Phase 3: Validation + +Before finishing, verify: + +1. All `{foobar}` placeholders are replaced in every file. +2. All `{domain}` placeholders are replaced in every file. +3. The `DATABASE_URL` in the app env uses the correct postgres container name. +4. The `VALKEY_URL` in the app env uses the correct valkey container name. +5. The certbot domains list includes the primary domain. +6. The nginx `server_name` matches the certbot domain. +7. The nginx `proxy_pass` port matches the user's specified port. +8. The route53 record name matches the certbot domain. +9. The playbook `hosts` references are consistent (`{domain}_{foobar}`). + +### Phase 4: Report + +Summarize what was created: + +- Full directory tree +- Key configuration values (service name, domain, port, targets) +- Exact commands to deploy (init → certbot → nginx → app) +- Suggested pass secret paths +- Any decisions that need user attention (e.g. "adjust the proxy_pass port if your app listens on a different port") + +## Strict Operational Rules + +- **Always include Postgres 17 and Valkey** — they are part of the template. +- **Never hard-code secrets** — all secrets use `{{ lookup('env', 'VAR_NAME') }}` in `.env.j2` files. +- **Use kebab-case** for all service names. +- **Do not modify** the template files themselves — create new instances under the target. +- **Do not run** any playbooks or commands — only scaffold files. +- If the user wants a different database (e.g. MySQL), **ask before deviating** from the Postgres 17 default. +- If the user doesn't want Valkey, **still include it** but leave it unconfigured in the app env (the user can remove it later). +- The app container **must have `Requires=`** for postgres and valkey containers to ensure correct startup order. +- Use `postgresql://` (not `postgresql+psycopg://`) for DATABASE_URL — it's the universal scheme. diff --git a/phase-authoring/SKILL.md b/phase-authoring/SKILL.md index 788e86b..5b8ee9a 100644 --- a/phase-authoring/SKILL.md +++ b/phase-authoring/SKILL.md @@ -153,7 +153,7 @@ Derive from the chat context (see "Scoping from chat context" above): - Use `uv` for all package management. - **Mandatory Dependencies:** `python-dotenv` (production); `debugpy`, `ruff`, `pyright`, `pytest`, `pytest-cov` (dev). - **Web Projects:** add `fastapi`, `alembic`, `pydantic`; prefer `httpx`. -- **Scaffold Files:** a comprehensive `.gitignore` (must include `.agent/` and `.agent/phase-sessions/`), a multi-stage `Containerfile` (assuming `podman`/`docker`), and a `README.md` with `uv` and configuration instructions. +- **Scaffold Files:** a comprehensive `.gitignore` (must include `.agent/phase-sessions/` and `.agent/pipeline.log` — but **never** `.agent/` itself: the planning tree is tracked and committed), a multi-stage `Containerfile` (assuming `podman`/`docker`), and a `README.md` with `uv` and configuration instructions. ### Phase 3: Strategic Architectural Design diff --git a/phased-execution/SKILL.md b/phased-execution/SKILL.md index f07441a..f8697c6 100644 --- a/phased-execution/SKILL.md +++ b/phased-execution/SKILL.md @@ -101,8 +101,9 @@ phases: `.agent/reports/.a*.*`) — the script also prints a ready to run - Legacy flat phase files (`todo/NN_name.md`) are still executed as a single unit; `phase-authoring`'s `migrate-phases-to-tasks.sh` converts them to the directory layout (the phase's final pass then picks up any inline task list). -- Child executor sessions are kept in `.agent/phase-sessions/`; add it to - `.gitignore` if the project is versioned. +- Child executor sessions are kept in `.agent/phase-sessions/` (plus + `pipeline.log` in `.agent/`); if the project is versioned, git-ignore those + runtime artifacts only — `.agent/` itself is tracked and committed. - If you keep non-skill markdown (e.g. a `README.md`) in a skills directory (like `~/.pi/agent/skills/`), pi warns “description is required” for it. Add a `.gitignore` in that directory listing the file — pi's skill scanner diff --git a/todo-to-phased/SKILL.md b/todo-to-phased/SKILL.md index b24ae1c..854c361 100644 --- a/todo-to-phased/SKILL.md +++ b/todo-to-phased/SKILL.md @@ -138,7 +138,7 @@ tools: 5. "Strictly adhere to the **LOCKED DECISIONS** listed in `.agent/PLAN.md`." - **`.agent/phases/todo/`** and **`.agent/phases/complete/`** (the latter created empty). -- **`.gitignore`** — add `.agent/` and `.agent/phase-sessions/` if missing. +- **`.gitignore`** — add `.agent/phase-sessions/` and `.agent/pipeline.log` if missing (never `.agent/` itself — the planning tree is tracked and committed); if an existing `.gitignore` has a `.agent/` ignore line, remove it. Then write one phase directory per confirmed phase at `.agent/phases/todo/NN_name/`, numbered from the audit's "next phase @@ -169,12 +169,14 @@ roadmap table does not list these appended phases. - Git is mandatory: `git init` if the project is not a repository. - **Clear the TODO list** — the items are now phases, so the TODO file no - longer holds the plan. Replace its entire item list with a one-line - pointer (e.g. `Phased on YYYY-MM-DD — items now live in - .agent/phases/todo/`); never delete the file itself. -- Commit the conversion — `AGENTS.md`, the cleared TODO file, and any other - modified non-`.agent/` files (the `.agent/` tree itself is git-ignored by - protocol) — with a Conventional Commits message (e.g. `chore(agent): phase + longer holds the plan. Overwrite the file with a clean, empty document: + just a single top-level title `# TODO` and nothing else (no pointer + line, no leftover items, no trailing notes). Never delete the file + itself — leave the empty `TODO.md` in place so the workspace keeps a + known, stable landing spot for future todos. +- Commit the conversion — `AGENTS.md`, the cleared TODO file, the whole + `.agent/` tree (it is tracked, not git-ignored), and any other modified + files — with a Conventional Commits message (e.g. `chore(agent): phase roadmap from TODO.md, NN phases`), always with `--no-gpg-sign`. Finish by summarizing: the anchors (Protocol A), the phase list (number,