menzeljandClaude Opus 5
09bed274eb
F7 — Nothing stopped two compose operations landing on the same stack. There was a busy flag, but is_busy() was only ever read to colour the status column; no lifecycle handler consulted it before acting. Two tabs, or auto-update picking up a stack somebody had just clicked, both ran pull + up -d against the same project and raced over recreating containers. Lifecycle calls, the two deploy WebSockets and the auto-update pass now take a real lock; a second caller gets 409 (or an error frame and close 4409) and auto-update skips and retries next cycle. The lock is a row rather than a set in one worker's memory, so it holds across workers and across a restart, and it carries an expiry — a worker killed mid-deploy would otherwise strand the stack with no fix short of editing the database. F10 — /api/stacks/stats sampled every running container on every call, one blocking daemon request each, and both the dashboard and the stacks list poll it every five seconds. Two tabs on a 40-container host meant a sustained ~16 samples a second. Cached for 4s behind a lock so concurrent callers share one sweep, the same shape dashboard_service already used for its fleet aggregate. F11 — Three module dicts assumed exactly one uvicorn worker without saying so and were lost on restart. The busy set is the lock above. The image update cache is now mirrored to SQLite, so a restart shows the badges immediately instead of blanking them for up to an hour, and the already-notified marks come back with them rather than re-announcing the same updates. The login rate limiter is a table, so it cannot be cleared by getting the process to restart and no longer multiplies by the worker count. The constraint that shaped this: compose_service and update_service are shared with the agent, which has no database. Neither may import one. So the lock is a separate service the central app enforces at its own entry points, and update persistence is an opt-in callback the central app registers in its lifespan — the agent registers nothing and behaves exactly as before. A test asserts update_service never imports the database, since that is the kind of thing a later change breaks silently. Both new nets were checked by reverting the fix: dropping the lock from _lifecycle fails six tests, removing the stats cache fails the one that names the behaviour. Also wires up cache pruning in the same sweep — without it both the dict and the table grew one entry per image tag ever run, for the life of the install. 31 new tests (729 total). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
725 lines
39 KiB
Markdown
725 lines
39 KiB
Markdown
# StackPilot
|
|
|
|
A self-hosted Docker Compose manager for power users and homelab enthusiasts —
|
|
as intuitive as Dockge, as capable as Portainer for Compose workflows.
|
|
|
|
> **Status:** Phase 1 (Core) + Phase 2 (Volumes & GPU) + Phase 3 (Quality of
|
|
> Life) + Phase 4 (Operations) + Phase 5 (Multi-host) + Phase 6 (Backup
|
|
> destinations) + Phase 7 (Scheduled backups) + Phase 8 (Remote-stack backups)
|
|
> + Phase 9 (Networks) + Phase 10 (iGPU passthrough) + Phase 11 (Remote UX &
|
|
> network attach) + Phase 12 (File browser) + Phase 13 (Multi-host networks &
|
|
> images) + Phase 14 (Multi-host file browser) + Phase 15 (Dashboard stack
|
|
> resource usage) + Phase 16 (Volumes page, multi-host) + Phase 17 (Multi-host
|
|
> dashboard) + Phase 18 (Image prune) + Phase 19 (Compose validate & diff) +
|
|
> Phase 20 (Container management) + Phase 21 (Container terminal) + Phase 22
|
|
> (Auto-update) + Phase 23 (Secrets & configs) + Phase 24 (Design System v2)
|
|
> complete.
|
|
|
|
## Upgrading to 0.47.0 — nothing to do
|
|
|
|
Three pieces of state moved out of process memory and into the database, and
|
|
the live-stats endpoint got a cache. No configuration changes, no migration
|
|
steps; the new tables are created on first start.
|
|
|
|
- **Stacks can only run one compose operation at a time.** A second `start` /
|
|
`update` / `down` on a busy stack answers `409` instead of racing the first
|
|
one over the same containers. Auto-update skips a stack you are already
|
|
deploying and picks it up next cycle.
|
|
- **`GET /api/stacks/stats` is cached for four seconds.** It sampled every
|
|
running container on every call, and the dashboard and the stacks list both
|
|
poll it every five seconds — two open tabs on a 40-container host meant a
|
|
sustained ~16 daemon calls a second.
|
|
- **The update cache and the login rate limiter persist.** Both used to reset on
|
|
restart; the rate limiter also used to multiply by the worker count, so
|
|
neither behaved as documented with `--workers` set.
|
|
|
|
## Upgrading to 0.46.0 — everyone is signed out once
|
|
|
|
Sessions now hold their access token in memory and the refresh token in an
|
|
httpOnly cookie, so **the upgrade signs everybody out exactly once**. Sign back
|
|
in and it behaves as before, including staying signed in across restarts.
|
|
|
|
What changed and why:
|
|
|
|
- **Tokens can be revoked.** Each account carries a `token_version` that every
|
|
token is minted with and every request checks. Resetting a password, changing
|
|
a role or disabling an account now bumps it, which cuts off the tokens that
|
|
account already holds — previously a password reset was cosmetic and whoever
|
|
had the old refresh token kept full access for up to 30 days.
|
|
- **The refresh token left `localStorage`.** It is an httpOnly cookie
|
|
(`SameSite=Lax`, scoped to `/api/auth`), so a successful XSS can act inside the
|
|
open page but cannot walk off with 30 days of access. The cookie is marked
|
|
`Secure` only when the request arrived over HTTPS, so a plain-HTTP homelab
|
|
keeps working. Any refresh token left in `localStorage` by an older build is
|
|
deleted on first load.
|
|
- **"Sign out everywhere"** in the user menu revokes every token the account
|
|
holds, on every device. Plain "Sign out" only ends the session on that device.
|
|
- **WebSockets re-check the database.** Log streams and the container terminal
|
|
read the role from the live user instead of the token's claim, so a demotion
|
|
or a disabled account takes effect immediately — the terminal is
|
|
root-equivalent on the host.
|
|
|
|
Scripted clients that cannot hold a cookie can still get the refresh token in
|
|
the response body with `?in_body=true` on login and refresh.
|
|
|
|
## Upgrading to 0.44.0 / 0.45.0 — two defaults changed
|
|
|
|
0.44.0 closes a privilege-escalation hole and tightens two defaults. Both
|
|
changes can affect an existing install:
|
|
|
|
1. **The `user` role loses read access to secrets.** The file browser (page and
|
|
`/api/files/*`), the host-path picker, the audit log, `GET /api/stacks/{id}/export`,
|
|
a stack's `.env` and the template *detail* route (0.45.0 — "save stack as
|
|
template" snapshots the stack's real `.env`) are now admin-only. The template
|
|
*listing* stays open. Previously any logged-in account
|
|
could download `stackpilot.db`, every `.env` and every `.secrets/*` file —
|
|
and none of it was audit-logged. If you gave someone a `user` account so they
|
|
could look at stacks, they still can; they just no longer get the
|
|
credentials. Nothing changes for admins.
|
|
2. **`/` is no longer a default browse root.** The new default is
|
|
`/mnt,/media,/srv,/opt,/home`. A `/` entry makes the sandbox allow every
|
|
path, which is why it is gone — if you relied on it, set
|
|
`ALLOWED_BROWSE_ROOTS` explicitly in your `.env`. StackPilot's own `DATA_DIR`
|
|
is refused either way.
|
|
|
|
Two things also get fixed without any action on your part: the backend now runs
|
|
uvicorn with `--proxy-headers`, so the login rate limit works per client IP
|
|
instead of globally and the audit log records real IPs; and backup-destination
|
|
credentials are encrypted at rest, with existing rows migrated on first start.
|
|
That encryption is keyed off `SECRET_KEY`, which is now persisted to
|
|
`${DATA_DIR}/secret_key` when you have not set one — so restarts no longer log
|
|
everyone out. **If you have never set `SECRET_KEY`, do not delete that file**;
|
|
it is what your saved destination credentials are encrypted with.
|
|
|
|
## What works today (Phase 1)
|
|
|
|
- **File-first stacks** — every stack is a plain `compose.yaml` (+ optional `.env`)
|
|
on disk. The DB only stores metadata; nothing is locked in.
|
|
- **Auth** — JWT access/refresh tokens, bcrypt hashing, admin/user roles, and a
|
|
first-launch setup wizard that creates the initial admin account. The access
|
|
token is held in memory; the refresh token is an httpOnly cookie. Every token
|
|
carries the account's `token_version`, so a password reset, role change or
|
|
disable revokes the tokens that account already holds — on every device.
|
|
- **Stack lifecycle** — create, edit, clone, delete, and `up / down / start /
|
|
stop / restart / pull / update` via `docker compose`.
|
|
- **Live status** — running / partial / stopped / error / updating, computed from
|
|
Docker container labels.
|
|
- **One operation per stack** — a lifecycle call takes a lock (a row, so it
|
|
holds across workers and across a restart) and a second one gets `409` while
|
|
it is held; auto-update skips a stack somebody is already deploying. Locks
|
|
carry an expiry, so a worker killed mid-deploy does not strand a stack.
|
|
- **Real-time logs** — streamed over WebSocket, color-coded per service.
|
|
- **Live deploy console** — deploying from the editor streams `compose up`
|
|
output (image pulls, container creation) over a WebSocket in real time instead
|
|
of a blind spinner; the deploy keeps running server-side if the modal is closed.
|
|
Works for **remote** stacks too — the central app proxies the agent's deploy
|
|
stream through to the browser. Compose runs with `--progress json`, so the
|
|
console shows a **real progress bar** (download bytes per layer, weighted by
|
|
layer size, then container create/start) plus a per-image bar; the raw output
|
|
is kept below it and updates one line per layer instead of scrolling past.
|
|
- **Inline update progress (0.42.0)** — hitting Update on a stack streams
|
|
`compose pull && up -d` over `/ws/update/{stack_id}` and folds it into a
|
|
**progress bar inside that stack's row** (same `--progress json` weighting as
|
|
the deploy console: "Pulling images · 3/7 layers · 88 MB / 190 MB"). Several
|
|
stacks can update at once — busy state and progress are tracked per stack, so
|
|
the rows advance independently. The REST `POST /api/stacks/{id}/update` stays
|
|
for non-interactive callers and as the fallback when no token is available.
|
|
- **Monaco editor** — YAML editing with an `.env` tab and a **`docker run` →
|
|
compose** converter.
|
|
- **Dashboard** — system resource bar, stack grid with quick actions, and a
|
|
recent-activity audit feed.
|
|
- **Auto-discovery** — stacks created outside the UI (any folder under the stacks
|
|
dir containing a compose file) are picked up automatically.
|
|
- **Dark / light theme.**
|
|
|
|
### Phase 2 — Volumes & GPU
|
|
|
|
- **Volume Wizard** in the editor (right-hand helper panel): Bind / Named / NFS /
|
|
SMB-CIFS / tmpfs, with a sandboxed **host path browser** for bind mounts and a
|
|
live YAML preview. NFS/SMB `driver_opts` are generated for you.
|
|
- **GPU assignment** per service: auto-detects NVIDIA (`nvidia-smi`) and AMD/Intel
|
|
(`/dev/dri` + sysfs), injects the right YAML (NVIDIA `deploy.reservations`, or
|
|
`/dev/dri` passthrough + render/video groups + `LIBVA_DRIVER_NAME=iHD` for Intel).
|
|
- **Device passthrough**: lists host USB / serial-TTY / DRI nodes, add per service,
|
|
plus a guarded `privileged` toggle.
|
|
- All wizard edits are merged into the compose YAML **server-side** (robust,
|
|
validated) and returned to the editor for review before saving.
|
|
|
|
> GPU/device detection needs host visibility. The bundled compose bind-mounts
|
|
> `/dev:/dev:ro`; NVIDIA additionally requires the NVIDIA container runtime on the host.
|
|
|
|
### Phase 3 — Quality of Life
|
|
|
|
- **Env editor**: table mode with sensitive-value masking (`PASS`/`SECRET`/`KEY`/…
|
|
auto-detected) + raw mode, plus PUID/PGID/TZ quick-insert.
|
|
- **Image update checker**: background task compares the local manifest digest with
|
|
the registry (Docker Hub / ghcr / lscr / private v2 with token auth); update
|
|
badges on the Images page + an "updates available" banner on the dashboard.
|
|
Results are cached in the database, so a restart shows the badges immediately
|
|
instead of blanking them until the next sweep — and does not re-announce
|
|
updates it already notified about.
|
|
- **Port conflict detector**: pre-deploy check against host-bound ports
|
|
(`/proc/net/tcp[6]`) and running container bindings, with a confirm dialog.
|
|
- **Resource limits**: CPU/memory sliders in the editor → `deploy.resources.limits`.
|
|
- **Template library**: each template is a ready-to-run **stack folder** in git
|
|
(`backend/templates/<slug>/` — `compose.yaml` + `.env.example` + `template.json`,
|
|
plus any extra config the app needs, e.g. `prometheus.yml` or a `Caddyfile`).
|
|
83 bundled homelab apps across media, *arr automation, networking, reverse
|
|
proxies, VPN, SSO, monitoring, dashboards, files/backup, notes and wikis, home
|
|
automation, dev tooling, databases, local AI and more — searchable and
|
|
filterable by tag on the Templates page. "Pull" copies the whole folder into a
|
|
new stack (`.env.example` → `.env`) which you then edit and deploy. Save any
|
|
stack back as a custom template (stored under `${DATA_DIR}/templates/`). Add
|
|
your own by dropping a folder into the templates dir.
|
|
- **Healthcheck status** surfaced per container in the stack overview.
|
|
|
|
### Phase 4 — Operations
|
|
|
|
- **Self-update (0.32.0)**: the top-bar version badge checks the registry for a
|
|
newer StackPilot release on page load (`GET /api/system/update`, anonymous v2
|
|
token flow, 10 min cache) and shows an amber update pill. One click
|
|
(`POST /api/system/update`, admin) spawns a detached helper container that runs
|
|
`docker compose pull && up -d` on StackPilot's own compose project (resolved
|
|
from its container labels) — the helper survives the backend being recreated;
|
|
the UI polls `/api/health` and reloads when the new version answers. Installs
|
|
not managed by compose get a clear "update manually" error instead.
|
|
- **Backup & restore**: per-stack `.tar.gz` backups covering the whole stack —
|
|
the stack folder, every **bind-mounted data directory** (`./config`, `/mnt/appdata/…`)
|
|
and every named volume. Bind sources and volumes are read through a throwaway
|
|
helper container, i.e. by **host path**, so data that StackPilot itself cannot
|
|
see is captured too (that is the case whenever `STACKS_HOST_DIR` differs from
|
|
the container's `STACKS_DIR` — compose then creates the data directories at the
|
|
container path *on the host*, and a naive backup would only find the compose
|
|
file). The Backup dialog shows the full inventory with sizes and lets you pick
|
|
what goes in; **NFS/CIFS-backed volumes are unchecked by default** because they
|
|
live on a NAS and restoring one would overwrite the share. Restore puts bind
|
|
folders back at their host paths and preserves permissions, ownership and
|
|
symlinks (PUID/PGID-based images such as the *arr suite need this), with
|
|
optional rename and overwrite/conflict detection.
|
|
- **Notification webhooks**: ntfy, Discord, Slack, Gotify, or generic JSON, each
|
|
subscribed to chosen events (image update available, stack start/stop/error,
|
|
pull failed). Managed in **Settings → Notifications**; env `NOTIFY_WEBHOOKS`
|
|
still supported for generic endpoints.
|
|
- **Settings page**: tune the update-check interval, manage webhooks, and manage
|
|
users (create/disable/delete, promote/demote, with last-admin safeguards).
|
|
- **Audit log page** (admin): searchable, paginated view of all recorded actions.
|
|
- **Mobile-responsive layout**: off-canvas sidebar + adaptive spacing.
|
|
|
|
### Phase 5 — Multi-host
|
|
|
|
- **Remote agents**: deploy `stackpilot-agent` (same image, different CMD) on any
|
|
host — it needs only the Docker socket and a shared `AGENT_TOKEN`, and exposes a
|
|
slim, token-guarded stack/system API (no UI, no DB).
|
|
- **Central management**: add hosts under **Settings → Remote hosts** (name, agent
|
|
URL, token) with a live connectivity dot. The Stacks page groups stacks by host
|
|
("This host" + one section per agent); remote stacks have their own detail view
|
|
with full lifecycle (start/stop/restart/pull/update/down), live logs, and
|
|
compose/.env editing — all proxied to the agent.
|
|
|
|
### Phase 6 — Backup destinations
|
|
|
|
- **Off-box backups**: define **SFTP**, **S3-compatible** (MinIO, Backblaze B2,
|
|
AWS S3, …) or **NFS share** (0.33.0) destinations under **Settings → Backup
|
|
destinations** (with a Test button; secrets are masked in API responses).
|
|
NFS needs no privileges in the StackPilot container: the Docker daemon mounts
|
|
the export as a named volume (`stackpilot-nfs-dest-<id>`, recreated when the
|
|
config changes) and file I/O runs through throwaway helper containers.
|
|
- **Push & restore**: the stack Backup dialog can push straight to a destination
|
|
instead of downloading; the Restore dialog can browse a destination's backups
|
|
and restore (volumes included) directly from it. Remote backups can also be
|
|
deleted from the UI.
|
|
|
|
### Phase 7 — Scheduled backups
|
|
|
|
- **Recurring backups**: schedule a stack to back up to a destination **hourly,
|
|
daily, or weekly** (UTC) under **Settings → Scheduled backups**. A background
|
|
scheduler runs due jobs every minute and records last/next run + status.
|
|
- **Retention**: keep the newest *N* backups per stack on the destination; older
|
|
ones are pruned automatically.
|
|
- **Run now** for an on-demand run, plus a `backup_failed` notification event
|
|
wired into the webhook system.
|
|
|
|
### Phase 8 — Remote-stack backups
|
|
|
|
- **Back up agent stacks**: the agent exposes its own backup/restore endpoints,
|
|
and the main app streams a remote stack's backup through to a destination
|
|
(credentials stay central — agents never see them). Remote stack detail has a
|
|
**Backup** button; each host section on the Stacks page has a **Restore** button.
|
|
- **Schedule remote stacks**: a backup schedule can target a remote host; backups
|
|
are namespaced per host (`backup-<host>-<stack>-…`) so retention never prunes
|
|
across hosts sharing a destination.
|
|
|
|
### Phase 9 — Networks
|
|
|
|
- **Network management**: the Networks page lists Docker networks (driver, scope,
|
|
subnet, attached containers / in-use, owning stack), with **create** (bridge /
|
|
macvlan / ipvlan / overlay, optional subnet+gateway, internal/attachable),
|
|
**delete** (default networks protected; in-use guarded by Docker), and **prune
|
|
unused**.
|
|
- **Stack delete**: local stacks can now be deleted from the UI (stack detail and
|
|
the stack card), with a confirm dialog and an optional "keep files on disk".
|
|
|
|
### Phase 10 — iGPU passthrough
|
|
|
|
- **Render/video group detection**: for a passed-through Intel/AMD iGPU, StackPilot
|
|
reads the host group ownership of the `/dev/dri` nodes (render node → `render`
|
|
GID, paired `card` node → `video` GID) and injects them as **numeric**
|
|
`group_add` entries (e.g. `group_add: ["991", "44"]`). Group names rarely
|
|
resolve inside images, so the numeric GID is what actually grants access. The
|
|
GPU selector shows the detected GIDs; removal cleans them (and `LIBVA_DRIVER_NAME`).
|
|
|
|
### Phase 11 — Remote UX & network attach
|
|
|
|
- **Live remote logs**: remote-stack logs now stream over a WebSocket proxied
|
|
through the central app to the agent (`/ws/agent-logs/{agent}/{stack}`), instead
|
|
of polling — same live viewer as local stacks.
|
|
- **Deploy to a remote host from the UI**: the New Stack editor and the template
|
|
dialog gained a *host* selector. Pick an online agent and the stack is created
|
|
(and optionally started) on that host; you land on its remote detail page.
|
|
- **Network attach/detach**: each network row on the Networks page expands to an
|
|
inspect view listing connected containers, with admin controls to disconnect a
|
|
container or connect any container on the host (`POST /api/networks/{id}/connect`
|
|
/ `/disconnect`).
|
|
|
|
### Phase 24 — Design System v2 (analytics-style UI)
|
|
|
|
- **New shell**: the sidebar is gone — a fixed 60px top bar carries a pill
|
|
navigation (active route = dark pill), the logo mark, a version badge, a
|
|
remote-host online indicator, theme toggle and an avatar menu. Narrow
|
|
screens get an off-canvas drawer.
|
|
- **Design tokens** (`frontend/src/styles/tokens.css`): one CSS-variable set
|
|
for surfaces, borders, text tiers, brand colours, radii and type scales,
|
|
with class-based dark-mode overrides. Existing Tailwind aliases
|
|
(`bg`/`card`/`accent`) are remapped onto the tokens so all pages reskin
|
|
consistently. Typeface: Schibsted Grotesk (bundled, offline-friendly).
|
|
- **Stack Health funnel** — the dashboard centrepiece. Five stages
|
|
(`discovered → running → healthy → updated → auto-managed`) from
|
|
`GET /api/dashboard/funnel` (30 s server cache, `?refresh=true` to bust;
|
|
the last stage — API key `monitored` — counts stacks with an enabled
|
|
auto-update policy). SVG waterfall with alternating gradient /
|
|
diagonal-hatch bars, value chips and hover conversion/drop-off tooltips.
|
|
- **Summary widgets** from `GET /api/dashboard/summary`: compose-containers
|
|
card with per-host breakdown (agents report a compose-only count) and an
|
|
"Insights" chip (healthy-rate %), a 30-day uptime line chart (sampled every
|
|
5 min into `DATA_DIR/uptime.jsonl`, charted as daily averages), and an ops
|
|
contribution grid from audit-log activity with the peak weekday.
|
|
A 30/7-day range selector slices both series client-side.
|
|
- **Explore prompt bar** under the funnel: typed queries or `/running`,
|
|
`/stopped`, `/attention` tags deep-link to the Stacks page, which now
|
|
honours `?q=` and `?filter=` (plus a new status-filter select).
|
|
|
|
### Phase 23 — Secrets & configs (compose file-based)
|
|
|
|
- A **Secrets** tab on the stack detail page manages per-stack Docker
|
|
**secrets** and **configs**: create one by name + content, list them (name,
|
|
kind, size — content is **never** returned by the API), and delete. Content is
|
|
write-only: once saved it is cleared from the form and cannot be read back.
|
|
- Files are stored inside the stack's own directory (`<stack_dir>/.secrets/<name>`
|
|
/ `.configs/<name>`, dir `0700` / file `0600`) and referenced from the compose
|
|
file with a **relative** `file:` path, so the Docker daemon reads them with no
|
|
`HOST_ROOT_PREFIX` dependency — exactly as if dropped next to `compose.yaml`.
|
|
- **Attach/detach** wires a stored secret/config into a chosen service: secrets
|
|
appear at `/run/secrets/<name>`, configs mount at a target path you specify. The
|
|
compose file is rewritten in place (top-level `secrets:`/`configs:` defs are
|
|
pruned when no service still uses them); **redeploy the stack to apply**.
|
|
- **Admin-only** (secrets are sensitive); every write/delete/attach/detach is
|
|
audited (`secret.*`). Works for **remote stacks** too — the agent stores the
|
|
files on its own host (`/agent/stacks/{id}/secrets/*`, proxied via
|
|
`/api/agents/{id}/stacks/{id}/secrets/*`). Names are validated against path
|
|
traversal (single component, no `..`, no leading dot); content capped at 1 MiB.
|
|
|
|
### Phase 22 — Auto-update (Watchtower-style)
|
|
|
|
- A per-stack **Auto-update** policy (on the stack Overview tab): when the
|
|
background image-update check finds a newer registry digest for one of the
|
|
stack's images, the stack is either **pulled + redeployed** or merely
|
|
**flagged** ("notify only"), with a **Check now** button for an on-demand run.
|
|
- Runs inside the existing image-update-check cycle (reuses the freshly-computed
|
|
digest cache, no extra registry calls). Only **running** stacks are
|
|
auto-redeployed — a stopped stack is never silently started ("skipped").
|
|
- New `stack_auto_updated` notification event. Works for **remote stacks** too
|
|
(policy stored centrally; the agent answers `/agent/stacks/{id}/updates` and
|
|
performs the redeploy). Last run + status (updated / up-to-date /
|
|
update-available / skipped / error) are shown inline.
|
|
|
|
### Phase 21 — Container terminal (web exec)
|
|
|
|
- An **interactive terminal** into any running, compose-managed container,
|
|
opened from the terminal button on its container card (stack Overview tab).
|
|
Streams an exec session over WebSocket into xterm.js — pick `/bin/sh`,
|
|
`/bin/bash`, or `/bin/ash`; full TTY with resize.
|
|
- **Admin-only** (exec is root-equivalent): a non-admin token is rejected at the
|
|
WebSocket handshake (`4403`). Only containers with the
|
|
`com.docker.compose.project` label can be reached.
|
|
- Works for **remote stacks** too: the same terminal proxies through
|
|
`/ws/agent-exec/{agent_id}/{container_id}` to the agent's new
|
|
`/agent/ws/exec/{container_id}` (bidirectional — keystrokes in, output out).
|
|
|
|
### Phase 20 — Container management
|
|
|
|
- The stack **Overview** tab now renders each service as an expandable
|
|
**container card** instead of a static row. Expanding it fetches a curated
|
|
single-container inspect view (image, state + exit code, restart count,
|
|
started-at, networks, mounts, and environment) via
|
|
`GET /api/containers/{id}`.
|
|
- Admins get **per-container start / stop / restart** buttons directly on the
|
|
card (`POST /api/containers/{id}/{action}`), so a single misbehaving service
|
|
can be bounced without touching the rest of the stack.
|
|
- Works for **remote stacks** too — the same card is used on the remote stack
|
|
detail page, proxied through `/api/agents/{id}/containers/*` to the agent's
|
|
new `/agent/containers/*` endpoints.
|
|
- Only containers carrying the `com.docker.compose.project` label are exposed,
|
|
so this never becomes a generic "control any container on the host" backdoor.
|
|
|
|
### Phase 19 — Compose validate & diff
|
|
|
|
- The editor can **validate** a compose file (`docker compose config`) before
|
|
deploying and show a **diff against the currently deployed** definition, so
|
|
you can see exactly what a re-deploy will change.
|
|
|
|
### Phase 18 — Image prune
|
|
|
|
- **Prune images** (dangling, or all unused) from the Images page, on the local
|
|
host and on each agent.
|
|
|
|
### Phase 17 — Multi-host dashboard
|
|
|
|
- The dashboard now shows, **per host** (local + each registered agent, online
|
|
dot / offline notice), a **resource overview bar** (CPU cores, memory
|
|
used/total, disk used/total, Docker volumes total, containers, Docker version)
|
|
and a **stacks-with-usage table** with CPU/memory meters and inline
|
|
start/stop/restart. The volumes total reuses the cached `/volumes/sizes`
|
|
lookup (`docker system df`), polled gently (~60s).
|
|
- New agent endpoint `/agent/stacks/stats` (proxied at
|
|
`/api/agents/{id}/stacks/stats`); `/agent/system` now also reports `cpu_cores`,
|
|
`mem_total`, `mem_used`, and `disk_total`/`disk_used` (disk of the host volume
|
|
backing the stacks dir) so the remote resource bar and meters have a host
|
|
reference.
|
|
|
|
### Phase 16 — Volumes page (multi-host)
|
|
|
|
- **New Volumes page** (sidebar) with per-host sections (local + each online
|
|
agent, like Networks/Images). Lists Docker volumes with driver, owning stack,
|
|
in-use containers and mountpoint.
|
|
- Admin actions: delete a volume (with an in-use warning + force option) and
|
|
**Prune unused**; an *Only unused* filter. New agent endpoints
|
|
`/agent/volumes` (list/delete/prune), proxied at `/api/agents/{id}/volumes/*`.
|
|
- **Volume sizes** are loaded on demand via a *Compute sizes* button (runs
|
|
`docker system df`, which walks volume contents and can take a few seconds);
|
|
results are cached ~60s. Endpoint `GET /api/volumes/sizes` (+ per-agent).
|
|
- The Volume **Wizard** in the stack editor (bind/named/NFS/SMB/tmpfs YAML
|
|
generation) is unchanged — the new page is for managing/cleaning up volumes.
|
|
|
|
### Phase 15 — Dashboard stack resource usage
|
|
|
|
- **The dashboard now lists stacks in a table** (status, services) with live
|
|
**CPU** and **memory** usage per stack, sampled from `docker stats` and
|
|
aggregated by compose project.
|
|
- When a stack has `deploy.resources.limits` assigned, the bar fills toward that
|
|
limit and shows usage vs the limit (e.g. `0.42 / 1 cores`, `310 MB / 512 MB`);
|
|
otherwise it shows absolute usage against the host total. Inline start/stop/
|
|
restart actions per row for admins. New endpoint `GET /api/stacks/stats`.
|
|
|
|
### Phase 14 — Multi-host file browser
|
|
|
|
- **The Files page now has a host switcher.** When agents are registered, a
|
|
*Host* dropdown at the top switches the whole browser between the local host
|
|
and any online agent; switching resets the path and clipboard.
|
|
- All file operations (browse, view/edit, create, rename, copy/move, delete,
|
|
upload files & folders, download) work against the selected agent, sandboxed
|
|
by *that agent's* `ALLOWED_BROWSE_ROOTS`/`HOST_ROOT_PREFIX`.
|
|
- New agent endpoints `/agent/files/*`, proxied at `/api/agents/{id}/files/*`.
|
|
|
|
### Phase 13 — Multi-host networks & images
|
|
|
|
- **Networks and Images are now per-host.** Both pages render a section for the
|
|
local host plus one for every registered agent (online dot included), exactly
|
|
like the Stacks page. Each agent section talks to that host's Docker daemon.
|
|
- **Remote network management**: list, inspect, create, delete, prune, and
|
|
connect/disconnect containers on an agent host — including a *Prune unused*
|
|
button, which resolves the common "all predefined address pools have been
|
|
fully subnetted" deploy error without SSH.
|
|
- **Remote images**: list image tags (with using-stacks) and run on-demand update
|
|
checks per host.
|
|
- New agent endpoints `/agent/networks/*` and `/agent/images/*`, proxied through
|
|
the central app at `/api/agents/{id}/networks/*` and `/api/agents/{id}/images/*`.
|
|
|
|
### Phase 12 — File browser
|
|
|
|
- **Files page (sidebar)**: a full host filesystem browser with breadcrumb
|
|
navigation, clickable browse-root chips, an *Up* control, and a show/hide
|
|
hidden-files toggle. Listings show size, permissions and modified time.
|
|
- **View & edit**: clicking a text file opens it in a Monaco editor (with syntax
|
|
highlighting picked from the extension). Binary and oversized files are
|
|
detected and offered as a download instead. Admins can edit and **Save**.
|
|
- **Admin only**: the whole page, including listing, viewing and downloading.
|
|
Reads are not less sensitive than writes here — the browser reaches whatever
|
|
the backend container can see, which includes every stack's `.env` and
|
|
`.secrets/*`. Reading and downloading a file are audit-logged (`file.read`,
|
|
`file.download`); directory listing is not, because the page polls it.
|
|
- **Manage** (admin): create folders/files, rename, delete (recursive for
|
|
folders), upload files **or whole folders** (the directory tree is recreated
|
|
server-side), and download any file. **Copy/cut & paste** moves files and
|
|
folders between directories (clipboard bar + per-row copy/cut, with an
|
|
overwrite prompt on conflict). Every mutation is audit-logged.
|
|
- **Sandboxed**: all access is confined to `ALLOWED_BROWSE_ROOTS`; path
|
|
traversal and deleting a browse root are refused. StackPilot's own `DATA_DIR`
|
|
is refused regardless of the setting — it holds `stackpilot.db` with password
|
|
hashes, agent tokens and backup-destination credentials, none of which the API
|
|
itself ever hands out. Note that a single `/` entry in `ALLOWED_BROWSE_ROOTS`
|
|
switches the sandbox off entirely; it is no longer part of the default. To reach the real host
|
|
filesystem, mount it into the backend and set `HOST_ROOT_PREFIX` (see the
|
|
commented `/:/host_root` volume in `docker-compose.yml`). Endpoints live under
|
|
`/api/files/*` (`list`, `read`, `write`, `mkdir`, `touch`, `rename`, `copy`,
|
|
`move`, `upload` — with optional `rel_path` for folder uploads —, `download`,
|
|
`DELETE`).
|
|
|
|
## Deploying an agent on another host
|
|
|
|
```bash
|
|
cd agent
|
|
cp .env.example .env # set a strong AGENT_TOKEN
|
|
docker compose up -d # exposes the agent on :5010
|
|
```
|
|
|
|
Then in the central UI: **Settings → Remote hosts → Add host** with
|
|
`http://<that-host>:5010` and the same `AGENT_TOKEN`.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
frontend (React + Vite + Tailwind, served by nginx)
|
|
│ proxies /api and /ws
|
|
▼
|
|
backend (FastAPI + docker-py + SQLite)
|
|
│ docker-py + `docker compose` CLI
|
|
▼
|
|
Docker Engine (via /var/run/docker.sock — never exposed to the browser)
|
|
```
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
cd stackpilot
|
|
cp .env.example .env
|
|
# edit .env and set a strong SECRET_KEY: openssl rand -base64 48
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Open <http://localhost:5009> and complete the first-launch setup wizard to create
|
|
your admin account.
|
|
|
|
### Configuration
|
|
|
|
All backend settings are environment variables (see `backend/config.py`). The
|
|
most important ones:
|
|
|
|
| Variable | Default | Purpose |
|
|
|-----------------|--------------------|-------------------------------------------|
|
|
| `SECRET_KEY` | _(auto, persisted)_| JWT + at-rest encryption key (see below) |
|
|
| `STACKS_DIR` | `/opt/stacks` | Where stack folders live (in-container) |
|
|
| `DATA_DIR` | `/data` | SQLite DB + app data |
|
|
| `CORS_ORIGINS` | localhost | Allowed API origins (comma separated) |
|
|
|
|
`SECRET_KEY` signs JWTs **and** derives the key that encrypts backup-destination
|
|
credentials in the database. Leave it unset and one is generated and written to
|
|
`${DATA_DIR}/secret_key` (mode 0600) on first start, so sessions and stored
|
|
credentials survive restarts — that file is then part of your backup. Setting it
|
|
explicitly always wins and nothing is written.
|
|
|
|
The host path for stacks is set via `STACKS_HOST_DIR` in `.env`, and it should
|
|
be **the same path as `STACKS_DIR`** (`/opt/stacks` by default). Compose runs
|
|
inside the backend container, so a stack's relative bind mounts (`./config`) are
|
|
resolved against the container path and the daemon creates those directories at
|
|
that path *on the host*. Point `STACKS_HOST_DIR` somewhere else and every stack's
|
|
data lives at `/opt/stacks/<stack>/…` on the host while StackPilot looks at a
|
|
different folder — the file browser and editor then show only the compose file.
|
|
Backups cover the data either way (they read bind sources by host path through a
|
|
helper container) and the Backup dialog warns when the two paths diverge.
|
|
|
|
## Local development
|
|
|
|
Backend:
|
|
|
|
```bash
|
|
cd backend
|
|
python -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements.txt
|
|
SECRET_KEY=dev STACKS_DIR=../data/stacks DATA_DIR=../data uvicorn main:app --reload --port 5008
|
|
```
|
|
|
|
Frontend (proxies to the backend on :5008):
|
|
|
|
```bash
|
|
cd frontend
|
|
npm install
|
|
npm run dev # http://localhost:5173
|
|
```
|
|
|
|
### Tests & linting
|
|
|
|
Same three commands the CI runs — `build-and-push` only starts once they pass.
|
|
|
|
```bash
|
|
cd backend
|
|
pip install -r requirements-dev.txt
|
|
pytest # 729 tests, no Docker daemon needed
|
|
ruff check .
|
|
cd ../frontend && npx tsc --noEmit -p tsconfig.json
|
|
```
|
|
|
|
The suite drives the app through `TestClient` **without** the lifespan, so it
|
|
never opens a Docker socket and never starts the background loops; `conftest.py`
|
|
points `DATA_DIR`/`STACKS_DIR` at a temp directory before anything is imported.
|
|
|
|
The load-bearing one is `tests/test_route_authorization.py`. Authorization lives
|
|
in the routers — each of 171 routes independently picks `require_admin` or
|
|
`get_current_user`, and nothing checked that the choice was right, which is how
|
|
0.43.0 shipped a read-only role that could download the auth database. That file
|
|
states the policy once — *every route requires admin unless it is listed* — and
|
|
fails on any route that disagrees. Adding a route the `user` role may reach means
|
|
adding it to `USER_READABLE` with a note on why it cannot return a credential.
|
|
`tests/test_agent_authorization.py` does the same for the agent, where a single
|
|
forgotten `Depends(verify_token)` would expose a whole host.
|
|
|
|
`tests/test_token_revocation.py` covers the revoke switch — that each of the
|
|
three authority changes kills the account's tokens, that a cosmetic re-save does
|
|
not, and that the refresh cookie is httpOnly and not marked `Secure` over plain
|
|
HTTP. `tests/test_schema_migration.py` builds a database with the *old* user
|
|
table and asserts the added column is backfilled rather than left NULL, which is
|
|
what would otherwise have signed out every user on every install.
|
|
|
|
`tests/test_stack_locking.py` and `tests/test_runtime_state.py` cover the state
|
|
that moved into the database: that a busy stack answers 409 without ever
|
|
reaching Docker, that an expired lock is taken over rather than stranding the
|
|
stack, that the stats cache serves repeat callers from one sweep, and that the
|
|
update cache and rate limiter survive a restart. One of them asserts that
|
|
`update_service` never imports the database — it is shared with the agent,
|
|
which has none, so persistence has to stay opt-in.
|
|
|
|
`tests/test_bundled_templates.py` covers the 83 shipped templates: each must
|
|
parse, name an image per service, keep `.env.example` in sync with the variables
|
|
compose actually reads, ship every file it bind-mounts, and never come with a
|
|
working default password.
|
|
|
|
## API surface (Phase 1)
|
|
|
|
```
|
|
POST /api/auth/setup | login | refresh GET /api/auth/me | needs-setup
|
|
POST /api/auth/logout | logout-everywhere
|
|
GET /api/stacks POST /api/stacks
|
|
GET /api/stacks/{id} PUT /api/stacks/{id} DELETE /api/stacks/{id}
|
|
POST /api/stacks/{id}/{start|stop|restart|pull|update|down|clone}
|
|
GET /api/stacks/{id}/logs GET /api/stacks/{id}/export
|
|
POST /api/stacks/convert (docker run → compose)
|
|
GET /api/system/info | gpus | devices GET /api/audit
|
|
GET /api/system/update POST /api/system/update (self-update)
|
|
WS /ws/logs/{stack_id}[/{service}] WS /ws/events
|
|
WS /ws/deploy/{stack_id} WS /ws/update/{stack_id}
|
|
```
|
|
|
|
### Phase 2 endpoints
|
|
|
|
```
|
|
GET /api/volumes | /orphaned DELETE /api/volumes/{name}
|
|
POST /api/volumes/prune POST /api/volumes/generate-yaml
|
|
GET /api/host/paths?path=&show_hidden= (sandboxed browser)
|
|
POST /api/editor/services | add-volume | set-gpu | add-device | remove-device | set-privileged
|
|
```
|
|
|
|
### Phase 3 endpoints
|
|
|
|
```
|
|
GET /api/images | /updates POST /api/images/check
|
|
POST /api/ports/conflicts POST /api/editor/set-resources
|
|
GET /api/templates | /{id} POST /api/templates/{id}/instantiate
|
|
POST /api/templates | /from-stack DELETE /api/templates/custom/{slug}
|
|
```
|
|
|
|
### Phase 4 endpoints
|
|
|
|
```
|
|
GET /api/stacks/{id}/backup?include_volumes=&stop_first= POST /api/stacks/restore
|
|
GET /api/settings PUT /api/settings
|
|
GET /api/settings/webhooks POST /api/settings/webhooks
|
|
PUT /api/settings/webhooks/{id} DELETE /api/settings/webhooks/{id}
|
|
POST /api/settings/webhooks/{id}/test
|
|
GET /api/auth/users POST /api/auth/users
|
|
PATCH /api/auth/users/{id} DELETE /api/auth/users/{id}
|
|
```
|
|
|
|
### Phase 5 endpoints
|
|
|
|
```
|
|
GET /api/agents POST /api/agents
|
|
PUT /api/agents/{id} DELETE /api/agents/{id}
|
|
POST /api/agents/{id}/ping GET /api/agents/{id}/system
|
|
GET /api/agents/{id}/stacks | /{sid} GET /api/agents/{id}/stacks/{sid}/logs
|
|
POST /api/agents/{id}/stacks PUT /api/agents/{id}/stacks/{sid}
|
|
DELETE /api/agents/{id}/stacks/{sid} POST /api/agents/{id}/stacks/{sid}/{action}
|
|
|
|
agent (on the remote host, Bearer AGENT_TOKEN):
|
|
GET /agent/ping | /system | /stacks | /stacks/{id} | /stacks/{id}/logs
|
|
GET /agent/stacks/{id}/backup POST /agent/stacks/restore
|
|
POST /agent/stacks | /stacks/{id}/{action} PUT/DELETE /agent/stacks/{id}
|
|
```
|
|
|
|
### Phase 6 endpoints
|
|
|
|
```
|
|
GET /api/backups/destinations POST /api/backups/destinations
|
|
PUT /api/backups/destinations/{id} DELETE /api/backups/destinations/{id}
|
|
POST /api/backups/destinations/{id}/test GET /api/backups/destinations/{id}/backups
|
|
DELETE /api/backups/destinations/{id}/backups/{name}
|
|
POST /api/stacks/{id}/backup/push POST /api/stacks/restore-from
|
|
```
|
|
|
|
### Phase 7 endpoints
|
|
|
|
```
|
|
GET /api/backups/schedules POST /api/backups/schedules
|
|
PUT /api/backups/schedules/{id} DELETE /api/backups/schedules/{id}
|
|
POST /api/backups/schedules/{id}/run
|
|
```
|
|
|
|
### Phase 8 endpoints (remote-stack backups)
|
|
|
|
```
|
|
GET /api/agents/{id}/stacks/{sid}/backup POST /api/agents/{id}/stacks/{sid}/backup/push
|
|
POST /api/agents/{id}/stacks/restore POST /api/agents/{id}/stacks/restore-from
|
|
backup schedules accept an optional agent_id to target a remote host.
|
|
```
|
|
|
|
### Phase 9 endpoints
|
|
|
|
```
|
|
GET /api/networks | /{id} POST /api/networks
|
|
DELETE /api/networks/{id} POST /api/networks/prune
|
|
DELETE /api/stacks/{id}?delete_files= (stack delete, now surfaced in the UI)
|
|
```
|
|
|
|
### Phase 11 endpoints
|
|
|
|
```
|
|
WS /ws/agent-logs/{agent_id}/{stack_id} (live remote logs, proxied to the agent)
|
|
GET /api/networks/{id}/containers POST /api/networks/{id}/connect | /disconnect
|
|
POST /api/agents/{id}/stacks (create a stack on a remote host — now in the UI)
|
|
POST /api/templates/{id}/instantiate {agent_id} (instantiate a template onto a remote host)
|
|
```
|
|
|
|
### Phase 24 endpoints
|
|
|
|
```
|
|
GET /api/dashboard/funnel[?refresh=true] (stack-health funnel, 30s TTL cache)
|
|
GET /api/dashboard/summary (containers, uptime series, ops activity)
|
|
```
|
|
|
|
## Security notes
|
|
|
|
- The Docker socket is only ever touched by the backend process; it is never
|
|
proxied to the browser.
|
|
- Login is rate-limited (10/min/IP).
|
|
- Compose files are backed up to `*.bak` before every overwrite.
|
|
- Generated YAML never includes the obsolete `version:` field and uses Compose v2
|
|
(`docker compose`) syntax.
|