stackpilot/ROADMAP.md

# StackPilot Roadmap — Phases 21–23

Planned 2026-06-09. Status keys: ☐ not started · ◐ in progress · ☑ done.
Each phase ships independently following the standing release checklist
(bump `backend/main.py` + `backend/agent_app.py` AGENT_VERSION +
`frontend/package.json` → build backend→agent→frontend `:VERSION`+`:latest` →
py_compile + `tsc -b && vite build` → route smoke-test → push all 3 → README →
git commit + push → update memory). Current released version: **0.26.0**.

Order: **21 → 22 → 23** (Terminal is highest-value and self-contained; Secrets
is most design-ambiguous, left last).

---

## Phase 21 — Container terminal (web exec)   ☑ DONE — shipped 0.27.0

**Result:** All backend paths live-verified via TestClient against real
containers — local exec (echo round-trip, exit code 0, unmanaged-guard,
non-admin→4403, no-token→4401), agent exec (echo, bad-token→4401), and the
central proxy (bidirectional: keystrokes browser→proxy→agent→container, output
back). docker-py 7.1.0 socket is `socket.SocketIO` with the real fd at
`._sock` (as predicted). Frontend `tsc -b && vite build` green with xterm.
Not click-tested in a real browser (the WS/exec plumbing is what's verified).

Interactive shell into a running, compose-managed container over WebSocket +
xterm.js, like Portainer's Console. Local + agent. **Admin-only** (exec is
root-equivalent).

### Backend
- ☐ `services/exec_service.py` (new). Reuse `container_service._get_managed()`
  to refuse non-compose containers. Functions:
  - `create_exec(container_id, cmd, tty=True)` → low-level
    `client.api.exec_create(container_id, cmd, stdin=True, tty=True,
    stdout=True, stderr=True)` returns `exec_id`.
  - `start_exec(exec_id)` → `client.api.exec_start(exec_id, socket=True,
    tty=True, demux=False)` returns the raw socket (a `SocketIO`/`socket`;
    on some docker-py versions the real fd is `sock._sock` — handle both;
    **verify live**, this is the main risk).
  - `resize_exec(exec_id, h, w)` → `client.api.exec_resize(exec_id, height=h,
    width=w)`.
  - default cmd = `["/bin/sh"]` (frontend may request `/bin/bash`).
- ☐ `routers/ws.py`: add `_authorize_admin(websocket, token)` (decode JWT like
  `_authorize` but also require `role == "admin"`; close 4403 if not admin).
  Then `@router.websocket("/ws/exec/{container_id}")`:
  - accept, `_authorize_admin`, read `?cmd=` (default /bin/sh), create+start exec.
  - **Bidirectional pump** with two asyncio tasks:
    - reader: `loop.run_in_executor(None, sock.recv, 4096)` → `send_bytes`/
      `send_text` to browser (decode utf-8, errors="replace").
    - writer: `await websocket.receive_text()`; JSON control msgs
      `{type:"resize",cols,rows}` → `resize_exec`; otherwise raw keystrokes →
      `sock.sendall(data.encode())` (also in executor).
    - on first task to finish, cancel the other; close sock + ws.
  - audit `container.exec`.
- ☐ `agent_app.py`: `@app.websocket("/agent/ws/exec/{container_id}")` — token
  via `?token=` vs AGENT_TOKEN, same pump (import exec_service).
- ☐ `routers/ws.py` proxy: `@router.websocket("/ws/agent-exec/{agent_id}/{container_id}")`
  — mirror `ws_agent_deploy` BUT forward **both directions** (agent-logs/deploy
  only pump upstream→browser; exec also needs browser→upstream). Use
  `urllib.parse.quote(token, safe='')`, InvalidStatus 404 → "update agent" hint.

### Frontend
- ☐ Add deps to `frontend/package.json`: `xterm` (`@xterm/xterm`) +
  `@xterm/addon-fit`. (rebuild installs them).
- ☐ `components/stacks/ContainerTerminal.tsx` — modal: xterm.js Terminal +
  FitAddon, connects to `/ws/exec/{id}` or `/ws/agent-exec/{aid}/{id}`; shell
  `<select>` (/bin/sh | /bin/bash); `term.onData` → ws.send; ws.onmessage →
  `term.write`; ResizeObserver/fit → send `{type:"resize",cols,rows}`; status
  line (connected/closed/error 4403 → "admin only").
- ☐ `ContainerCard.tsx`: add a **Terminal** button (admin + running only) next
  to start/stop/restart → opens `ContainerTerminal` with the same `agentId`.

### Verify
- ☐ Live: exec `/bin/sh` into a real throwaway compose container, run `ls`/`pwd`,
  type interactively, resize, Ctrl-C, exit. Bad/non-admin token → 4403.
- ☐ Agent path: same against a local agent container w/ docker.sock.
- ☐ py_compile + tsc/vite build + route smoke-test (expect main +2 WS, agent +1).

### Open risks
- docker-py `exec_start(socket=True)` socket object shape varies by version —
  verify `.recv`/`.sendall` vs needing `sock._sock`. This is THE thing to nail live.
- TTY mode (tty=True) gives a raw single stream (no 8-byte demux header) — right
  for xterm. If we ever do non-tty, we'd need demux handling.

---

## Phase 22 — Auto-Update (Watchtower-style)   ☑ DONE — shipped 0.28.0

**Result:** All four orchestration paths live-verified against real compose in
the 0.28.0 backend image (seeding `update_service._CACHE`): redeploy→`updated`
(pull+up, stack stays running), notify-only→`update-available`,
no-update→`up-to-date`, stopped-stack→`skipped`. Routes present (local +
agent + `/agent/stacks/{id}/updates`). Hooked into `update_service.background_loop`
(lazy import, no circular). Remote path reuses the proven `agent_service.call`;
not separately e2e'd against a live agent this round.

Per-stack policy: when an image used by the stack has a newer registry digest,
either auto pull+redeploy or just notify. Builds on the existing
`update_service` (digest check) + `compose_service.pull/up` + `notify_service`.

### Backend
- ☐ `models/auto_update.py` — `AutoUpdate(id, stack_id, agent_id nullable,
  enabled bool, redeploy bool [True=pull+up / False=notify-only], last_run,
  last_status, last_result)`. (agent_id None = local stack.)
- ☐ `services/auto_update_service.py`:
  - `run_due()` — for each enabled policy: resolve the stack's images
    (local: `update_service` images for that compose project; remote: via agent),
    `check_image` each; if any `update_available`:
    - redeploy=True local → `compose_service.pull(stack_id)` then
      `compose_service.up(stack_id)`; remote → `agent_service` POST
      `/agent/stacks/{id}/update`.
    - redeploy=False → notify only.
    - record last_run/last_status (`updated`/`up-to-date`/`error`), notify
      `EVENT_STACK_AUTO_UPDATED` (add to `models/setting.py` ALL_EVENTS +
      frontend EVENT_LABELS) on redeploy, reuse `pull_failed`/`stack_error` on
      failure.
  - Hook into the existing **`update_service.background_loop`** (it already polls
    on the settings interval) — after `check_all()`, call
    `auto_update_service.run_due()`. (Or the scheduler_loop; pick background_loop
    since it already has fresh digest data.)
- ☐ `routers/stacks.py`: `GET /api/stacks/{id}/auto-update` +
  `PUT /api/stacks/{id}/auto-update` (admin). Agent stacks: add to
  `routers/agents.py` `GET/PUT /api/agents/{id}/stacks/{sid}/auto-update`
  (policy stored centrally keyed by agent_id+stack_id; the agent itself stays
  stateless — central drives it).

### Frontend
- ☐ `StackDetail.tsx` + `RemoteStackDetail.tsx`: an **Auto-update** toggle in the
  action bar / a small card (enabled + mode redeploy|notify-only). `api/stacks.ts`
  + `api/agents.ts` get/set.
- ☐ Optional: Settings → **Auto-updates** section listing all policies w/ last
  run + status (like Scheduled backups).

### Verify
- ☐ Enable on a throwaway stack pinned to an old tag, point its tag at a newer
  digest (or use a 2-tag trick), confirm `run_due` pulls + redeploys + records
  status + fires the notification. Notify-only mode notifies without redeploy.
- ☐ py_compile + build + route smoke-test.

### Open risks
- Mapping a stack → its images reliably (compose `image:` refs vs built images;
  built/untagged images can't be digest-checked — skip them).
- Don't redeploy a stack the user has manually stopped (check status first?
  decide at impl: probably only auto-update stacks currently `running`).

---

## Phase 23 — Docker Secrets & Configs   ☐ NOT STARTED → target 0.29.0

**DESIGN DECISION TO MAKE FIRST (ask user / decide at phase start):** Docker
`secret`/`config` objects are a **Swarm** feature. Two interpretations:
- (A) **Compose file-based secrets** (`secrets:` top-level with `file:` +
  per-service `secrets:`) — works in plain compose, the relevant one for a
  compose manager. **Recommended default.**
- (B) **Swarm secrets/configs** via `client.secrets`/`client.configs` — only if
  swarm mode is active (detect `client.info()["Swarm"]["LocalNodeState"]=="active"`).

Plan assumes **(A)**, with (B) surfaced only when swarm is detected.

### Backend
- ☐ `models/secret.py` — `ManagedSecret(id, name, scope [global|stack], stack_id
  nullable, kind [secret|config], created, agent_id nullable)`. Content NOT in
  DB — stored on disk.
- ☐ `services/secret_service.py`:
  - store secret files under a sandboxed dir `<STACKS_DIR>/.stackpilot-secrets/`
    (chmod 700 dir, 600 files); `create(name, content)`, `update(name, content)`,
    `delete(name)`, `list()` (metadata only — never return content; mask), and
    `path_for(name)` for compose `file:` refs.
  - if swarm active: also expose `client.secrets.list/create/remove` +
    `client.configs.*` (interpretation B), behind a `swarm_active()` guard.
- ☐ `compose_edit_service`: `add_secret(yaml, service, secret_name)` — injects
  top-level `secrets: {<name>: {file: <path>}}` + per-service `secrets: [<name>]`;
  `remove_secret(...)`. Same for configs.
- ☐ `routers/secrets.py` (prefix `/api/secrets`): CRUD (admin, audit
  `secret.*`), content write-only. Agent `/agent/secrets/*` + proxy
  `/api/agents/{id}/secrets/*` for multi-host (reuse the patterns).

### Frontend
- ☐ Settings → **Secrets & Configs** section (or a dedicated page): list (name,
  scope, kind, created), create (name + content textarea, content masked after),
  delete. Multi-host host-switcher like Files.
- ☐ Editor helper panel: a **Secrets** wizard tab to attach an existing secret/
  config to a service (writes the compose `secrets:` block via
  `compose_edit_service`).

### Verify
- ☐ Create a file-based secret, attach to a service via the wizard, deploy, exec
  in and confirm `/run/secrets/<name>` is present with the content.
- ☐ Sandbox: secret files can't escape the secrets dir; content never returned by list.
- ☐ If swarm active on the build host, smoke-test the swarm path too (likely
  NOT active here → note as hardware-verify debt).
- ☐ py_compile + build + route smoke-test.

### Open risks
- Swarm-vs-compose decision (above). Confirm with user before coding.
- Secret file ownership/permissions inside the container vs on host (file-based
  secrets mount the host file → uid/gid must be readable by the service user).
- HOST_ROOT_PREFIX / multi-host: the secret file must live where that host's
  Docker daemon can read it (agent stores on its own host).

---

## After 23
Remaining un-built ideas from the gap analysis (not chosen this round):
Health-monitoring & alerting (Docker-events → notify; note `/ws/events` already
exists), per-container live-stats + historical graphs, GitOps/deploy-from-Git,
image-management expansion (pull/remove/build arbitrary images + registry creds
UI), 2FA/TOTP + OIDC SSO + API tokens.