stackpilot/README.md

# StackPilot

A self-hosted Docker Compose manager for power users and homelab enthusiasts —
as intuitive as Dockge, as capable as Portainer for Compose workflows.

> **Status:** Phase 1 (Core) + Phase 2 (Volumes & GPU) + Phase 3 (Quality of
> Life) + Phase 4 (Operations) + Phase 5 (Multi-host) + Phase 6 (Backup
> destinations) + Phase 7 (Scheduled backups) + Phase 8 (Remote-stack backups)
> + Phase 9 (Networks) + Phase 10 (iGPU passthrough) + Phase 11 (Remote UX &
> network attach) + Phase 12 (File browser) + Phase 13 (Multi-host networks &
> images) + Phase 14 (Multi-host file browser) + Phase 15 (Dashboard stack
> resource usage) + Phase 16 (Volumes page, multi-host) + Phase 17 (Multi-host
> dashboard) + Phase 18 (Image prune) + Phase 19 (Compose validate & diff) +
> Phase 20 (Container management) + Phase 21 (Container terminal) + Phase 22
> (Auto-update) + Phase 23 (Secrets & configs) + Phase 24 (Design System v2)
> complete.

## Upgrading to 0.46.0 — everyone is signed out once

Sessions now hold their access token in memory and the refresh token in an
httpOnly cookie, so **the upgrade signs everybody out exactly once**. Sign back
in and it behaves as before, including staying signed in across restarts.

What changed and why:

- **Tokens can be revoked.** Each account carries a `token_version` that every
  token is minted with and every request checks. Resetting a password, changing
  a role or disabling an account now bumps it, which cuts off the tokens that
  account already holds — previously a password reset was cosmetic and whoever
  had the old refresh token kept full access for up to 30 days.
- **The refresh token left `localStorage`.** It is an httpOnly cookie
  (`SameSite=Lax`, scoped to `/api/auth`), so a successful XSS can act inside the
  open page but cannot walk off with 30 days of access. The cookie is marked
  `Secure` only when the request arrived over HTTPS, so a plain-HTTP homelab
  keeps working. Any refresh token left in `localStorage` by an older build is
  deleted on first load.
- **"Sign out everywhere"** in the user menu revokes every token the account
  holds, on every device. Plain "Sign out" only ends the session on that device.
- **WebSockets re-check the database.** Log streams and the container terminal
  read the role from the live user instead of the token's claim, so a demotion
  or a disabled account takes effect immediately — the terminal is
  root-equivalent on the host.

Scripted clients that cannot hold a cookie can still get the refresh token in
the response body with `?in_body=true` on login and refresh.

## Upgrading to 0.44.0 / 0.45.0 — two defaults changed

0.44.0 closes a privilege-escalation hole and tightens two defaults. Both
changes can affect an existing install:

1. **The `user` role loses read access to secrets.** The file browser (page and
   `/api/files/*`), the host-path picker, the audit log, `GET /api/stacks/{id}/export`,
   a stack's `.env` and the template *detail* route (0.45.0 — "save stack as
   template" snapshots the stack's real `.env`) are now admin-only. The template
   *listing* stays open. Previously any logged-in account
   could download `stackpilot.db`, every `.env` and every `.secrets/*` file —
   and none of it was audit-logged. If you gave someone a `user` account so they
   could look at stacks, they still can; they just no longer get the
   credentials. Nothing changes for admins.
2. **`/` is no longer a default browse root.** The new default is
   `/mnt,/media,/srv,/opt,/home`. A `/` entry makes the sandbox allow every
   path, which is why it is gone — if you relied on it, set
   `ALLOWED_BROWSE_ROOTS` explicitly in your `.env`. StackPilot's own `DATA_DIR`
   is refused either way.

Two things also get fixed without any action on your part: the backend now runs
uvicorn with `--proxy-headers`, so the login rate limit works per client IP
instead of globally and the audit log records real IPs; and backup-destination
credentials are encrypted at rest, with existing rows migrated on first start.
That encryption is keyed off `SECRET_KEY`, which is now persisted to
`${DATA_DIR}/secret_key` when you have not set one — so restarts no longer log
everyone out. **If you have never set `SECRET_KEY`, do not delete that file**;
it is what your saved destination credentials are encrypted with.

## What works today (Phase 1)

- **File-first stacks** — every stack is a plain `compose.yaml` (+ optional `.env`)
  on disk. The DB only stores metadata; nothing is locked in.
- **Auth** — JWT access/refresh tokens, bcrypt hashing, admin/user roles, and a
  first-launch setup wizard that creates the initial admin account. The access
  token is held in memory; the refresh token is an httpOnly cookie. Every token
  carries the account's `token_version`, so a password reset, role change or
  disable revokes the tokens that account already holds — on every device.
- **Stack lifecycle** — create, edit, clone, delete, and `up / down / start /
  stop / restart / pull / update` via `docker compose`.
- **Live status** — running / partial / stopped / error / updating, computed from
  Docker container labels.
- **Real-time logs** — streamed over WebSocket, color-coded per service.
- **Live deploy console** — deploying from the editor streams `compose up`
  output (image pulls, container creation) over a WebSocket in real time instead
  of a blind spinner; the deploy keeps running server-side if the modal is closed.
  Works for **remote** stacks too — the central app proxies the agent's deploy
  stream through to the browser. Compose runs with `--progress json`, so the
  console shows a **real progress bar** (download bytes per layer, weighted by
  layer size, then container create/start) plus a per-image bar; the raw output
  is kept below it and updates one line per layer instead of scrolling past.
- **Inline update progress (0.42.0)** — hitting Update on a stack streams
  `compose pull && up -d` over `/ws/update/{stack_id}` and folds it into a
  **progress bar inside that stack's row** (same `--progress json` weighting as
  the deploy console: "Pulling images · 3/7 layers · 88 MB / 190 MB"). Several
  stacks can update at once — busy state and progress are tracked per stack, so
  the rows advance independently. The REST `POST /api/stacks/{id}/update` stays
  for non-interactive callers and as the fallback when no token is available.
- **Monaco editor** — YAML editing with an `.env` tab and a **`docker run` →
  compose** converter.
- **Dashboard** — system resource bar, stack grid with quick actions, and a
  recent-activity audit feed.
- **Auto-discovery** — stacks created outside the UI (any folder under the stacks
  dir containing a compose file) are picked up automatically.
- **Dark / light theme.**

### Phase 2 — Volumes & GPU

- **Volume Wizard** in the editor (right-hand helper panel): Bind / Named / NFS /
  SMB-CIFS / tmpfs, with a sandboxed **host path browser** for bind mounts and a
  live YAML preview. NFS/SMB `driver_opts` are generated for you.
- **GPU assignment** per service: auto-detects NVIDIA (`nvidia-smi`) and AMD/Intel
  (`/dev/dri` + sysfs), injects the right YAML (NVIDIA `deploy.reservations`, or
  `/dev/dri` passthrough + render/video groups + `LIBVA_DRIVER_NAME=iHD` for Intel).
- **Device passthrough**: lists host USB / serial-TTY / DRI nodes, add per service,
  plus a guarded `privileged` toggle.
- All wizard edits are merged into the compose YAML **server-side** (robust,
  validated) and returned to the editor for review before saving.

> GPU/device detection needs host visibility. The bundled compose bind-mounts
> `/dev:/dev:ro`; NVIDIA additionally requires the NVIDIA container runtime on the host.

### Phase 3 — Quality of Life

- **Env editor**: table mode with sensitive-value masking (`PASS`/`SECRET`/`KEY`/…
  auto-detected) + raw mode, plus PUID/PGID/TZ quick-insert.
- **Image update checker**: background task compares the local manifest digest with
  the registry (Docker Hub / ghcr / lscr / private v2 with token auth); update
  badges on the Images page + an "updates available" banner on the dashboard.
- **Port conflict detector**: pre-deploy check against host-bound ports
  (`/proc/net/tcp[6]`) and running container bindings, with a confirm dialog.
- **Resource limits**: CPU/memory sliders in the editor → `deploy.resources.limits`.
- **Template library**: each template is a ready-to-run **stack folder** in git
  (`backend/templates/<slug>/` — `compose.yaml` + `.env.example` + `template.json`,
  plus any extra config the app needs, e.g. `prometheus.yml` or a `Caddyfile`).
  83 bundled homelab apps across media, *arr automation, networking, reverse
  proxies, VPN, SSO, monitoring, dashboards, files/backup, notes and wikis, home
  automation, dev tooling, databases, local AI and more — searchable and
  filterable by tag on the Templates page. "Pull" copies the whole folder into a
  new stack (`.env.example` → `.env`) which you then edit and deploy. Save any
  stack back as a custom template (stored under `${DATA_DIR}/templates/`). Add
  your own by dropping a folder into the templates dir.
- **Healthcheck status** surfaced per container in the stack overview.

### Phase 4 — Operations

- **Self-update (0.32.0)**: the top-bar version badge checks the registry for a
  newer StackPilot release on page load (`GET /api/system/update`, anonymous v2
  token flow, 10 min cache) and shows an amber update pill. One click
  (`POST /api/system/update`, admin) spawns a detached helper container that runs
  `docker compose pull && up -d` on StackPilot's own compose project (resolved
  from its container labels) — the helper survives the backend being recreated;
  the UI polls `/api/health` and reloads when the new version answers. Installs
  not managed by compose get a clear "update manually" error instead.
- **Backup & restore**: per-stack `.tar.gz` backups covering the whole stack —
  the stack folder, every **bind-mounted data directory** (`./config`, `/mnt/appdata/…`)
  and every named volume. Bind sources and volumes are read through a throwaway
  helper container, i.e. by **host path**, so data that StackPilot itself cannot
  see is captured too (that is the case whenever `STACKS_HOST_DIR` differs from
  the container's `STACKS_DIR` — compose then creates the data directories at the
  container path *on the host*, and a naive backup would only find the compose
  file). The Backup dialog shows the full inventory with sizes and lets you pick
  what goes in; **NFS/CIFS-backed volumes are unchecked by default** because they
  live on a NAS and restoring one would overwrite the share. Restore puts bind
  folders back at their host paths and preserves permissions, ownership and
  symlinks (PUID/PGID-based images such as the *arr suite need this), with
  optional rename and overwrite/conflict detection.
- **Notification webhooks**: ntfy, Discord, Slack, Gotify, or generic JSON, each
  subscribed to chosen events (image update available, stack start/stop/error,
  pull failed). Managed in **Settings → Notifications**; env `NOTIFY_WEBHOOKS`
  still supported for generic endpoints.
- **Settings page**: tune the update-check interval, manage webhooks, and manage
  users (create/disable/delete, promote/demote, with last-admin safeguards).
- **Audit log page** (admin): searchable, paginated view of all recorded actions.
- **Mobile-responsive layout**: off-canvas sidebar + adaptive spacing.

### Phase 5 — Multi-host

- **Remote agents**: deploy `stackpilot-agent` (same image, different CMD) on any
  host — it needs only the Docker socket and a shared `AGENT_TOKEN`, and exposes a
  slim, token-guarded stack/system API (no UI, no DB).
- **Central management**: add hosts under **Settings → Remote hosts** (name, agent
  URL, token) with a live connectivity dot. The Stacks page groups stacks by host
  ("This host" + one section per agent); remote stacks have their own detail view
  with full lifecycle (start/stop/restart/pull/update/down), live logs, and
  compose/.env editing — all proxied to the agent.

### Phase 6 — Backup destinations

- **Off-box backups**: define **SFTP**, **S3-compatible** (MinIO, Backblaze B2,
  AWS S3, …) or **NFS share** (0.33.0) destinations under **Settings → Backup
  destinations** (with a Test button; secrets are masked in API responses).
  NFS needs no privileges in the StackPilot container: the Docker daemon mounts
  the export as a named volume (`stackpilot-nfs-dest-<id>`, recreated when the
  config changes) and file I/O runs through throwaway helper containers.
- **Push & restore**: the stack Backup dialog can push straight to a destination
  instead of downloading; the Restore dialog can browse a destination's backups
  and restore (volumes included) directly from it. Remote backups can also be
  deleted from the UI.

### Phase 7 — Scheduled backups

- **Recurring backups**: schedule a stack to back up to a destination **hourly,
  daily, or weekly** (UTC) under **Settings → Scheduled backups**. A background
  scheduler runs due jobs every minute and records last/next run + status.
- **Retention**: keep the newest *N* backups per stack on the destination; older
  ones are pruned automatically.
- **Run now** for an on-demand run, plus a `backup_failed` notification event
  wired into the webhook system.

### Phase 8 — Remote-stack backups

- **Back up agent stacks**: the agent exposes its own backup/restore endpoints,
  and the main app streams a remote stack's backup through to a destination
  (credentials stay central — agents never see them). Remote stack detail has a
  **Backup** button; each host section on the Stacks page has a **Restore** button.
- **Schedule remote stacks**: a backup schedule can target a remote host; backups
  are namespaced per host (`backup-<host>-<stack>-…`) so retention never prunes
  across hosts sharing a destination.

### Phase 9 — Networks

- **Network management**: the Networks page lists Docker networks (driver, scope,
  subnet, attached containers / in-use, owning stack), with **create** (bridge /
  macvlan / ipvlan / overlay, optional subnet+gateway, internal/attachable),
  **delete** (default networks protected; in-use guarded by Docker), and **prune
  unused**.
- **Stack delete**: local stacks can now be deleted from the UI (stack detail and
  the stack card), with a confirm dialog and an optional "keep files on disk".

### Phase 10 — iGPU passthrough

- **Render/video group detection**: for a passed-through Intel/AMD iGPU, StackPilot
  reads the host group ownership of the `/dev/dri` nodes (render node → `render`
  GID, paired `card` node → `video` GID) and injects them as **numeric**
  `group_add` entries (e.g. `group_add: ["991", "44"]`). Group names rarely
  resolve inside images, so the numeric GID is what actually grants access. The
  GPU selector shows the detected GIDs; removal cleans them (and `LIBVA_DRIVER_NAME`).

### Phase 11 — Remote UX & network attach

- **Live remote logs**: remote-stack logs now stream over a WebSocket proxied
  through the central app to the agent (`/ws/agent-logs/{agent}/{stack}`), instead
  of polling — same live viewer as local stacks.
- **Deploy to a remote host from the UI**: the New Stack editor and the template
  dialog gained a *host* selector. Pick an online agent and the stack is created
  (and optionally started) on that host; you land on its remote detail page.
- **Network attach/detach**: each network row on the Networks page expands to an
  inspect view listing connected containers, with admin controls to disconnect a
  container or connect any container on the host (`POST /api/networks/{id}/connect`
  / `/disconnect`).

### Phase 24 — Design System v2 (analytics-style UI)

- **New shell**: the sidebar is gone — a fixed 60px top bar carries a pill
  navigation (active route = dark pill), the logo mark, a version badge, a
  remote-host online indicator, theme toggle and an avatar menu. Narrow
  screens get an off-canvas drawer.
- **Design tokens** (`frontend/src/styles/tokens.css`): one CSS-variable set
  for surfaces, borders, text tiers, brand colours, radii and type scales,
  with class-based dark-mode overrides. Existing Tailwind aliases
  (`bg`/`card`/`accent`) are remapped onto the tokens so all pages reskin
  consistently. Typeface: Schibsted Grotesk (bundled, offline-friendly).
- **Stack Health funnel** — the dashboard centrepiece. Five stages
  (`discovered → running → healthy → updated → auto-managed`) from
  `GET /api/dashboard/funnel` (30 s server cache, `?refresh=true` to bust;
  the last stage — API key `monitored` — counts stacks with an enabled
  auto-update policy). SVG waterfall with alternating gradient /
  diagonal-hatch bars, value chips and hover conversion/drop-off tooltips.
- **Summary widgets** from `GET /api/dashboard/summary`: compose-containers
  card with per-host breakdown (agents report a compose-only count) and an
  "Insights" chip (healthy-rate %), a 30-day uptime line chart (sampled every
  5 min into `DATA_DIR/uptime.jsonl`, charted as daily averages), and an ops
  contribution grid from audit-log activity with the peak weekday.
  A 30/7-day range selector slices both series client-side.
- **Explore prompt bar** under the funnel: typed queries or `/running`,
  `/stopped`, `/attention` tags deep-link to the Stacks page, which now
  honours `?q=` and `?filter=` (plus a new status-filter select).

### Phase 23 — Secrets & configs (compose file-based)

- A **Secrets** tab on the stack detail page manages per-stack Docker
  **secrets** and **configs**: create one by name + content, list them (name,
  kind, size — content is **never** returned by the API), and delete. Content is
  write-only: once saved it is cleared from the form and cannot be read back.
- Files are stored inside the stack's own directory (`<stack_dir>/.secrets/<name>`
  / `.configs/<name>`, dir `0700` / file `0600`) and referenced from the compose
  file with a **relative** `file:` path, so the Docker daemon reads them with no
  `HOST_ROOT_PREFIX` dependency — exactly as if dropped next to `compose.yaml`.
- **Attach/detach** wires a stored secret/config into a chosen service: secrets
  appear at `/run/secrets/<name>`, configs mount at a target path you specify. The
  compose file is rewritten in place (top-level `secrets:`/`configs:` defs are
  pruned when no service still uses them); **redeploy the stack to apply**.
- **Admin-only** (secrets are sensitive); every write/delete/attach/detach is
  audited (`secret.*`). Works for **remote stacks** too — the agent stores the
  files on its own host (`/agent/stacks/{id}/secrets/*`, proxied via
  `/api/agents/{id}/stacks/{id}/secrets/*`). Names are validated against path
  traversal (single component, no `..`, no leading dot); content capped at 1 MiB.

### Phase 22 — Auto-update (Watchtower-style)

- A per-stack **Auto-update** policy (on the stack Overview tab): when the
  background image-update check finds a newer registry digest for one of the
  stack's images, the stack is either **pulled + redeployed** or merely
  **flagged** ("notify only"), with a **Check now** button for an on-demand run.
- Runs inside the existing image-update-check cycle (reuses the freshly-computed
  digest cache, no extra registry calls). Only **running** stacks are
  auto-redeployed — a stopped stack is never silently started ("skipped").
- New `stack_auto_updated` notification event. Works for **remote stacks** too
  (policy stored centrally; the agent answers `/agent/stacks/{id}/updates` and
  performs the redeploy). Last run + status (updated / up-to-date /
  update-available / skipped / error) are shown inline.

### Phase 21 — Container terminal (web exec)

- An **interactive terminal** into any running, compose-managed container,
  opened from the terminal button on its container card (stack Overview tab).
  Streams an exec session over WebSocket into xterm.js — pick `/bin/sh`,
  `/bin/bash`, or `/bin/ash`; full TTY with resize.
- **Admin-only** (exec is root-equivalent): a non-admin token is rejected at the
  WebSocket handshake (`4403`). Only containers with the
  `com.docker.compose.project` label can be reached.
- Works for **remote stacks** too: the same terminal proxies through
  `/ws/agent-exec/{agent_id}/{container_id}` to the agent's new
  `/agent/ws/exec/{container_id}` (bidirectional — keystrokes in, output out).

### Phase 20 — Container management

- The stack **Overview** tab now renders each service as an expandable
  **container card** instead of a static row. Expanding it fetches a curated
  single-container inspect view (image, state + exit code, restart count,
  started-at, networks, mounts, and environment) via
  `GET /api/containers/{id}`.
- Admins get **per-container start / stop / restart** buttons directly on the
  card (`POST /api/containers/{id}/{action}`), so a single misbehaving service
  can be bounced without touching the rest of the stack.
- Works for **remote stacks** too — the same card is used on the remote stack
  detail page, proxied through `/api/agents/{id}/containers/*` to the agent's
  new `/agent/containers/*` endpoints.
- Only containers carrying the `com.docker.compose.project` label are exposed,
  so this never becomes a generic "control any container on the host" backdoor.

### Phase 19 — Compose validate & diff

- The editor can **validate** a compose file (`docker compose config`) before
  deploying and show a **diff against the currently deployed** definition, so
  you can see exactly what a re-deploy will change.

### Phase 18 — Image prune

- **Prune images** (dangling, or all unused) from the Images page, on the local
  host and on each agent.

### Phase 17 — Multi-host dashboard

- The dashboard now shows, **per host** (local + each registered agent, online
  dot / offline notice), a **resource overview bar** (CPU cores, memory
  used/total, disk used/total, Docker volumes total, containers, Docker version)
  and a **stacks-with-usage table** with CPU/memory meters and inline
  start/stop/restart. The volumes total reuses the cached `/volumes/sizes`
  lookup (`docker system df`), polled gently (~60s).
- New agent endpoint `/agent/stacks/stats` (proxied at
  `/api/agents/{id}/stacks/stats`); `/agent/system` now also reports `cpu_cores`,
  `mem_total`, `mem_used`, and `disk_total`/`disk_used` (disk of the host volume
  backing the stacks dir) so the remote resource bar and meters have a host
  reference.

### Phase 16 — Volumes page (multi-host)

- **New Volumes page** (sidebar) with per-host sections (local + each online
  agent, like Networks/Images). Lists Docker volumes with driver, owning stack,
  in-use containers and mountpoint.
- Admin actions: delete a volume (with an in-use warning + force option) and
  **Prune unused**; an *Only unused* filter. New agent endpoints
  `/agent/volumes` (list/delete/prune), proxied at `/api/agents/{id}/volumes/*`.
- **Volume sizes** are loaded on demand via a *Compute sizes* button (runs
  `docker system df`, which walks volume contents and can take a few seconds);
  results are cached ~60s. Endpoint `GET /api/volumes/sizes` (+ per-agent).
- The Volume **Wizard** in the stack editor (bind/named/NFS/SMB/tmpfs YAML
  generation) is unchanged — the new page is for managing/cleaning up volumes.

### Phase 15 — Dashboard stack resource usage

- **The dashboard now lists stacks in a table** (status, services) with live
  **CPU** and **memory** usage per stack, sampled from `docker stats` and
  aggregated by compose project.
- When a stack has `deploy.resources.limits` assigned, the bar fills toward that
  limit and shows usage vs the limit (e.g. `0.42 / 1 cores`, `310 MB / 512 MB`);
  otherwise it shows absolute usage against the host total. Inline start/stop/
  restart actions per row for admins. New endpoint `GET /api/stacks/stats`.

### Phase 14 — Multi-host file browser

- **The Files page now has a host switcher.** When agents are registered, a
  *Host* dropdown at the top switches the whole browser between the local host
  and any online agent; switching resets the path and clipboard.
- All file operations (browse, view/edit, create, rename, copy/move, delete,
  upload files & folders, download) work against the selected agent, sandboxed
  by *that agent's* `ALLOWED_BROWSE_ROOTS`/`HOST_ROOT_PREFIX`.
- New agent endpoints `/agent/files/*`, proxied at `/api/agents/{id}/files/*`.

### Phase 13 — Multi-host networks & images

- **Networks and Images are now per-host.** Both pages render a section for the
  local host plus one for every registered agent (online dot included), exactly
  like the Stacks page. Each agent section talks to that host's Docker daemon.
- **Remote network management**: list, inspect, create, delete, prune, and
  connect/disconnect containers on an agent host — including a *Prune unused*
  button, which resolves the common "all predefined address pools have been
  fully subnetted" deploy error without SSH.
- **Remote images**: list image tags (with using-stacks) and run on-demand update
  checks per host.
- New agent endpoints `/agent/networks/*` and `/agent/images/*`, proxied through
  the central app at `/api/agents/{id}/networks/*` and `/api/agents/{id}/images/*`.

### Phase 12 — File browser

- **Files page (sidebar)**: a full host filesystem browser with breadcrumb
  navigation, clickable browse-root chips, an *Up* control, and a show/hide
  hidden-files toggle. Listings show size, permissions and modified time.
- **View & edit**: clicking a text file opens it in a Monaco editor (with syntax
  highlighting picked from the extension). Binary and oversized files are
  detected and offered as a download instead. Admins can edit and **Save**.
- **Admin only**: the whole page, including listing, viewing and downloading.
  Reads are not less sensitive than writes here — the browser reaches whatever
  the backend container can see, which includes every stack's `.env` and
  `.secrets/*`. Reading and downloading a file are audit-logged (`file.read`,
  `file.download`); directory listing is not, because the page polls it.
- **Manage** (admin): create folders/files, rename, delete (recursive for
  folders), upload files **or whole folders** (the directory tree is recreated
  server-side), and download any file. **Copy/cut & paste** moves files and
  folders between directories (clipboard bar + per-row copy/cut, with an
  overwrite prompt on conflict). Every mutation is audit-logged.
- **Sandboxed**: all access is confined to `ALLOWED_BROWSE_ROOTS`; path
  traversal and deleting a browse root are refused. StackPilot's own `DATA_DIR`
  is refused regardless of the setting — it holds `stackpilot.db` with password
  hashes, agent tokens and backup-destination credentials, none of which the API
  itself ever hands out. Note that a single `/` entry in `ALLOWED_BROWSE_ROOTS`
  switches the sandbox off entirely; it is no longer part of the default. To reach the real host
  filesystem, mount it into the backend and set `HOST_ROOT_PREFIX` (see the
  commented `/:/host_root` volume in `docker-compose.yml`). Endpoints live under
  `/api/files/*` (`list`, `read`, `write`, `mkdir`, `touch`, `rename`, `copy`,
  `move`, `upload` — with optional `rel_path` for folder uploads —, `download`,
  `DELETE`).

## Deploying an agent on another host

```bash
cd agent
cp .env.example .env          # set a strong AGENT_TOKEN
docker compose up -d          # exposes the agent on :5010
```

Then in the central UI: **Settings → Remote hosts → Add host** with
`http://<that-host>:5010` and the same `AGENT_TOKEN`.

## Architecture

```
frontend (React + Vite + Tailwind, served by nginx)
    │  proxies /api and /ws
    ▼
backend (FastAPI + docker-py + SQLite)
    │  docker-py + `docker compose` CLI
    ▼
Docker Engine (via /var/run/docker.sock — never exposed to the browser)
```

## Quick start

```bash
cd stackpilot
cp .env.example .env
# edit .env and set a strong SECRET_KEY:  openssl rand -base64 48
docker compose up -d --build
```

Open <http://localhost:5009> and complete the first-launch setup wizard to create
your admin account.

### Configuration

All backend settings are environment variables (see `backend/config.py`). The
most important ones:

| Variable        | Default            | Purpose                                   |
|-----------------|--------------------|-------------------------------------------|
| `SECRET_KEY`    | _(auto, persisted)_| JWT + at-rest encryption key (see below)  |
| `STACKS_DIR`    | `/opt/stacks`      | Where stack folders live (in-container)   |
| `DATA_DIR`      | `/data`            | SQLite DB + app data                      |
| `CORS_ORIGINS`  | localhost          | Allowed API origins (comma separated)     |

`SECRET_KEY` signs JWTs **and** derives the key that encrypts backup-destination
credentials in the database. Leave it unset and one is generated and written to
`${DATA_DIR}/secret_key` (mode 0600) on first start, so sessions and stored
credentials survive restarts — that file is then part of your backup. Setting it
explicitly always wins and nothing is written.

The host path for stacks is set via `STACKS_HOST_DIR` in `.env`, and it should
be **the same path as `STACKS_DIR`** (`/opt/stacks` by default). Compose runs
inside the backend container, so a stack's relative bind mounts (`./config`) are
resolved against the container path and the daemon creates those directories at
that path *on the host*. Point `STACKS_HOST_DIR` somewhere else and every stack's
data lives at `/opt/stacks/<stack>/…` on the host while StackPilot looks at a
different folder — the file browser and editor then show only the compose file.
Backups cover the data either way (they read bind sources by host path through a
helper container) and the Backup dialog warns when the two paths diverge.

## Local development

Backend:

```bash
cd backend
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
SECRET_KEY=dev STACKS_DIR=../data/stacks DATA_DIR=../data uvicorn main:app --reload --port 5008
```

Frontend (proxies to the backend on :5008):

```bash
cd frontend
npm install
npm run dev   # http://localhost:5173
```

### Tests & linting

Same three commands the CI runs — `build-and-push` only starts once they pass.

```bash
cd backend
pip install -r requirements-dev.txt
pytest                            # 698 tests, no Docker daemon needed
ruff check .
cd ../frontend && npx tsc --noEmit -p tsconfig.json
```

The suite drives the app through `TestClient` **without** the lifespan, so it
never opens a Docker socket and never starts the background loops; `conftest.py`
points `DATA_DIR`/`STACKS_DIR` at a temp directory before anything is imported.

The load-bearing one is `tests/test_route_authorization.py`. Authorization lives
in the routers — each of 171 routes independently picks `require_admin` or
`get_current_user`, and nothing checked that the choice was right, which is how
0.43.0 shipped a read-only role that could download the auth database. That file
states the policy once — *every route requires admin unless it is listed* — and
fails on any route that disagrees. Adding a route the `user` role may reach means
adding it to `USER_READABLE` with a note on why it cannot return a credential.
`tests/test_agent_authorization.py` does the same for the agent, where a single
forgotten `Depends(verify_token)` would expose a whole host.

`tests/test_token_revocation.py` covers the revoke switch — that each of the
three authority changes kills the account's tokens, that a cosmetic re-save does
not, and that the refresh cookie is httpOnly and not marked `Secure` over plain
HTTP. `tests/test_schema_migration.py` builds a database with the *old* user
table and asserts the added column is backfilled rather than left NULL, which is
what would otherwise have signed out every user on every install.

`tests/test_bundled_templates.py` covers the 83 shipped templates: each must
parse, name an image per service, keep `.env.example` in sync with the variables
compose actually reads, ship every file it bind-mounts, and never come with a
working default password.

## API surface (Phase 1)

```
POST   /api/auth/setup | login | refresh        GET /api/auth/me | needs-setup
POST   /api/auth/logout | logout-everywhere
GET    /api/stacks                              POST /api/stacks
GET    /api/stacks/{id}                          PUT /api/stacks/{id}   DELETE /api/stacks/{id}
POST   /api/stacks/{id}/{start|stop|restart|pull|update|down|clone}
GET    /api/stacks/{id}/logs                     GET /api/stacks/{id}/export
POST   /api/stacks/convert                       (docker run → compose)
GET    /api/system/info | gpus | devices         GET /api/audit
GET    /api/system/update                         POST /api/system/update (self-update)
WS     /ws/logs/{stack_id}[/{service}]           WS  /ws/events
WS     /ws/deploy/{stack_id}                     WS  /ws/update/{stack_id}
```

### Phase 2 endpoints

```
GET    /api/volumes | /orphaned                  DELETE /api/volumes/{name}
POST   /api/volumes/prune                        POST /api/volumes/generate-yaml
GET    /api/host/paths?path=&show_hidden=        (sandboxed browser)
POST   /api/editor/services | add-volume | set-gpu | add-device | remove-device | set-privileged
```

### Phase 3 endpoints

```
GET    /api/images | /updates                    POST /api/images/check
POST   /api/ports/conflicts                       POST /api/editor/set-resources
GET    /api/templates | /{id}                     POST /api/templates/{id}/instantiate
POST   /api/templates | /from-stack               DELETE /api/templates/custom/{slug}
```

### Phase 4 endpoints

```
GET    /api/stacks/{id}/backup?include_volumes=&stop_first=   POST /api/stacks/restore
GET    /api/settings                              PUT  /api/settings
GET    /api/settings/webhooks                     POST /api/settings/webhooks
PUT    /api/settings/webhooks/{id}                DELETE /api/settings/webhooks/{id}
POST   /api/settings/webhooks/{id}/test
GET    /api/auth/users                            POST /api/auth/users
PATCH  /api/auth/users/{id}                        DELETE /api/auth/users/{id}
```

### Phase 5 endpoints

```
GET    /api/agents                                POST /api/agents
PUT    /api/agents/{id}                            DELETE /api/agents/{id}
POST   /api/agents/{id}/ping                       GET  /api/agents/{id}/system
GET    /api/agents/{id}/stacks | /{sid}            GET  /api/agents/{id}/stacks/{sid}/logs
POST   /api/agents/{id}/stacks                     PUT  /api/agents/{id}/stacks/{sid}
DELETE /api/agents/{id}/stacks/{sid}               POST /api/agents/{id}/stacks/{sid}/{action}

agent (on the remote host, Bearer AGENT_TOKEN):
GET    /agent/ping | /system | /stacks | /stacks/{id} | /stacks/{id}/logs
GET    /agent/stacks/{id}/backup                   POST /agent/stacks/restore
POST   /agent/stacks | /stacks/{id}/{action}       PUT/DELETE /agent/stacks/{id}
```

### Phase 6 endpoints

```
GET    /api/backups/destinations                  POST /api/backups/destinations
PUT    /api/backups/destinations/{id}              DELETE /api/backups/destinations/{id}
POST   /api/backups/destinations/{id}/test         GET  /api/backups/destinations/{id}/backups
DELETE /api/backups/destinations/{id}/backups/{name}
POST   /api/stacks/{id}/backup/push                POST /api/stacks/restore-from
```

### Phase 7 endpoints

```
GET    /api/backups/schedules                     POST /api/backups/schedules
PUT    /api/backups/schedules/{id}                 DELETE /api/backups/schedules/{id}
POST   /api/backups/schedules/{id}/run
```

### Phase 8 endpoints (remote-stack backups)

```
GET    /api/agents/{id}/stacks/{sid}/backup        POST /api/agents/{id}/stacks/{sid}/backup/push
POST   /api/agents/{id}/stacks/restore             POST /api/agents/{id}/stacks/restore-from
backup schedules accept an optional agent_id to target a remote host.
```

### Phase 9 endpoints

```
GET    /api/networks | /{id}                       POST /api/networks
DELETE /api/networks/{id}                           POST /api/networks/prune
DELETE /api/stacks/{id}?delete_files=             (stack delete, now surfaced in the UI)
```

### Phase 11 endpoints

```
WS   /ws/agent-logs/{agent_id}/{stack_id}         (live remote logs, proxied to the agent)
GET  /api/networks/{id}/containers                 POST /api/networks/{id}/connect | /disconnect
POST /api/agents/{id}/stacks                       (create a stack on a remote host — now in the UI)
POST /api/templates/{id}/instantiate {agent_id}    (instantiate a template onto a remote host)
```

### Phase 24 endpoints

```
GET  /api/dashboard/funnel[?refresh=true]   (stack-health funnel, 30s TTL cache)
GET  /api/dashboard/summary                 (containers, uptime series, ops activity)
```

## Security notes

- The Docker socket is only ever touched by the backend process; it is never
  proxied to the browser.
- Login is rate-limited (10/min/IP).
- Compose files are backed up to `*.bak` before every overwrite.
- Generated YAML never includes the obsolete `version:` field and uses Compose v2
  (`docker compose`) syntax.