This was on the gap list as a missing feature, but it was a bug first. The
update checker asks the registry for a tag's digest over HTTP itself, and could
only do it anonymously. A private repository answers 401, remote_digest returned
None, and None already meant "could not reach registry" — so a private image was
indistinguishable from a network blip. The Images page showed nothing and a
stack pinned to a six-month-old image looked up to date indefinitely.
So AuthRequired is now its own exception, separate from unreachable, and the
error names the registry and which of the two problems it is: "ghcr.io needs
credentials" when there are none, "ghcr.io rejected the stored credentials" when
there are and they are wrong. Those are different fixes, and the message should
say which one you need. The plain unreachable message survives unchanged, with a
test pinning it, because not every failure is an auth failure.
Two consumers need the credentials and they need them in completely different
shapes, which is why this is its own service rather than a field on something
else. StackPilot's own checker wants (user, password) inside async code that has
no database session, so the rows are mirrored into an in-memory cache that
reload() refills on startup and after every write. The Docker CLI wants a
config.json, so reload() writes one into ${DATA_DIR}/docker and compose runs with
DOCKER_CONFIG pointed at it. Generating it from the database every time is what
makes deletion real: removing a registry in the UI revokes the CLI's login
instead of leaving a stale one in ~/.docker.
Host normalization is the join that makes any of it work, and it is easy to
underestimate. parse_ref only ever produces registry-1.docker.io, nobody types
that, and the CLI wants the whole thing under https://index.docker.io/v1/ — three
spellings of one registry across three layers. canonical_host settles on what
parse_ref produces, the config writer translates on the way out, and a bare
nginx:alpine finds credentials entered as "docker.io". Verified end to end:
typed as the v1 URL, stored as registry-1.docker.io, written as the v1 URL.
The password is encrypted at rest with the same key as backup destinations and
never leaves the server, not even masked — the API returns has_password, which
is all the form needs to offer "leave blank to keep". A row that cannot be
decrypted after a SECRET_KEY change is skipped with a warning rather than taking
every other registry down with it. Everything here is admin-only including the
reads, because even masked the rows say which registries this install talks to
and under what account.
The Test button asks the registry rather than validating a string, following the
Bearer challenge with credentials attached the way a real client does. Only an
outright 401 counts as wrong credentials; anything else means reachable and
talking, which is as much as a credentials check can honestly claim. Checked
against the live Docker Hub token endpoint with deliberately wrong credentials.
33 tests: the normalization table, the cache, the generated config.json down to
its 0600 mode and the Docker Hub key, encryption at rest, that no password field
appears in any response, and the 401-is-reported behaviour that started this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stacks were a name and a coloured dot. The dot carried the status but nothing
carried identity, so a list of twenty stacks read as twenty identical rows.
This gives each one an icon in front of its name and moves the status onto that
icon as a halo in the status colour, which is the thing the eye lands on anyway.
The constraint that shaped the design: people already have stacks. Asking them
to pick an icon for each one before the feature does anything would mean it
never gets used, so the icon is *derived* from the stack's name and the column
stays empty until somebody overrides it. ~700 keywords in 79 groups cover the
self-hosted long tail (jellyfin -> clapperboard, vaultwarden -> key,
home-assistant -> house) plus generic English and German terms; the longest
match wins, so photoprism beats a bare photo, and short keywords like "tv" only
match as whole words. No backfill, no migration, and a rename moves the icon
with it.
That is also why the catalog and the matcher live in the frontend. It is the
only place that can render an icon, so a copy in the backend would be a list to
keep in sync and nothing else. The server validates the shape of the stored
value and stores uploads; it never needs to know what "lucide:database" looks
like. An icon name that later leaves the catalog falls back to the derived one
rather than blanking the row.
Overriding happens in two places, because there are two moments: the editor
(holding a chosen file until the stack exists, since uploading needs an id) and
a click on the icon on the detail page, which is how a stack that has existed
for a year gets one without a trip through the editor.
Uploads are classified by their bytes, not by the filename or Content-Type the
browser claims, and land in ${DATA_DIR}/stack-icons/ under the stack id. SVG is
allowed — <img> does not execute it — but the endpoint serves every icon as an
attachment so one can never be opened as a document in the API's own origin. A
client-supplied "custom:" value is refused: the server mints those, so a stack
cannot be pointed at a file it does not own. Files follow the stack: replaced on
re-upload (including across formats, or the old one orphans), copied on clone,
removed on delete.
The one piece of plumbing worth knowing about: the icon endpoint needs the
bearer token like everything else, and an <img src> would not carry it. So
StackIcon fetches the bytes through the API client and renders the blob, keyed
on the stored value — which carries an upload timestamp precisely so a re-upload
changes the key and retires the cached image.
Covered by 22 backend tests (the value rules, byte-sniffing, the file lifecycle,
the API round-trip, and that the read-only role cannot change an icon) and 29
frontend ones for the matcher. The schema change was verified against a
hand-built pre-0.51 database: the column is added on start and existing rows
come back NULL, i.e. automatic. Not click-tested in a browser — no Docker in
this environment — so the row height the taller icon produces is unverified.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
StackPilot now manages exactly one Docker host: the one it runs on. The
stackpilot-agent sidecar and everything that proxied to it are gone — 4721
lines deleted against 657 added.
Deleted outright: agent/ (image, compose, env), agent_app.py, models/agent.py,
routers/agents.py (1200 lines), services/agent_service.py, the agent API client,
RemoteStackDetail, the host components and AgentStacksSection. That removes 57
API routes and the three /ws/agent-* proxies.
Threaded out everywhere else, which was the bulk of the work. Every API module
carried an optional agentId that switched the base path; every page that listed
Docker objects rendered one section per host behind a HostHeader; Files had a
host switcher; the New Stack editor and the template dialog had host selectors;
schedules, auto-update policies and stack summaries carried agent_id. All of it
is gone, and the typechecker drove the sweep — 85 files touched, tsc and the
build clean.
Two things the removal exposed as dead weight rather than merely unused:
compose_service kept an in-process busy set purely because the agent needed a
lock and has no database. With the agent gone that was a second source of truth
next to the real DB lock, so it is deleted; compute_status now reports only what
the containers say and the two callers that want "updating" overlay the lock.
StacksTable's linkBase prop only ever existed to point at /hosts/{id}/stacks.
The dashboard's "Hosts 1/1 online" KPI can no longer say anything else, so the
tile and the KPIs behind it are gone and the row is five wide.
Upgrading matters here. An existing install still has an agent table holding
each remote host's URL and bearer token — full Docker control of that host,
sitting in the database with nothing left to use it. _drop_removed_schema drops
it on first start, and drops the agent_id columns where the SQLite build
supports DROP COLUMN. Each statement runs in its own transaction on purpose: a
failed DDL poisons the transaction it is in, so sharing one would let an
unsupported column drop take the table drop down with it. test_agent_removal
covers both branches plus the fresh-install and idempotent cases, and an
end-to-end run against a seeded pre-0.48 database confirms the table is gone and
every /api/agents route answers 404.
Docstrings that justified a design by "shared with the agent, which has no
database" were rewritten rather than left lying: update_service's persistence
callback and image_status_store are still the right split (registry logic stays
testable without a database), but for that reason now, not the old one. The
README's multi-host sections are removed and an upgrade note explains what to do
with running agent containers; ROADMAP keeps its history behind a note saying
the feature it describes no longer exists.
CI no longer builds or pushes stackpilot-agent.
735 tests pass, ruff and tsc clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
F7 — Nothing stopped two compose operations landing on the same stack. There
was a busy flag, but is_busy() was only ever read to colour the status column;
no lifecycle handler consulted it before acting. Two tabs, or auto-update
picking up a stack somebody had just clicked, both ran pull + up -d against the
same project and raced over recreating containers.
Lifecycle calls, the two deploy WebSockets and the auto-update pass now take a
real lock; a second caller gets 409 (or an error frame and close 4409) and
auto-update skips and retries next cycle. The lock is a row rather than a set
in one worker's memory, so it holds across workers and across a restart, and it
carries an expiry — a worker killed mid-deploy would otherwise strand the stack
with no fix short of editing the database.
F10 — /api/stacks/stats sampled every running container on every call, one
blocking daemon request each, and both the dashboard and the stacks list poll
it every five seconds. Two tabs on a 40-container host meant a sustained ~16
samples a second. Cached for 4s behind a lock so concurrent callers share one
sweep, the same shape dashboard_service already used for its fleet aggregate.
F11 — Three module dicts assumed exactly one uvicorn worker without saying so
and were lost on restart. The busy set is the lock above. The image update
cache is now mirrored to SQLite, so a restart shows the badges immediately
instead of blanking them for up to an hour, and the already-notified marks come
back with them rather than re-announcing the same updates. The login rate
limiter is a table, so it cannot be cleared by getting the process to restart
and no longer multiplies by the worker count.
The constraint that shaped this: compose_service and update_service are shared
with the agent, which has no database. Neither may import one. So the lock is a
separate service the central app enforces at its own entry points, and update
persistence is an opt-in callback the central app registers in its lifespan —
the agent registers nothing and behaves exactly as before. A test asserts
update_service never imports the database, since that is the kind of thing a
later change breaks silently.
Both new nets were checked by reverting the fix: dropping the lock from
_lifecycle fails six tests, removing the stats cache fails the one that names
the behaviour.
Also wires up cache pruning in the same sweep — without it both the dict and
the table grew one entry per image tag ever run, for the life of the install.
31 new tests (729 total).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
F5 — A token was valid until it expired, full stop. Resetting a compromised
account's password changed nothing for whoever held its tokens (up to 30 days
for a refresh token), demoting or disabling an account only took effect once
the same clock ran out, and logout was purely client-side.
Every account now has a token_version, every token is minted carrying it, and
every request compares the two. Bumping it is the revoke switch, pulled on the
three changes that alter what an account may do: password, role, active flag.
"Sign out everywhere" in the user menu bumps your own. Plain "Sign out" only
drops the cookie, because signing out on your phone should not kill your
desktop session.
The refresh token left localStorage for an httpOnly cookie (SameSite=Lax,
scoped to /api/auth), and the access token is now held in memory only. A
successful XSS can still act inside the open page but can no longer walk off
with 30 days of access. The cookie is marked Secure only when the request
arrived over HTTPS — request.url.scheme is trustworthy since the F4 fix — so a
plain-HTTP homelab keeps working. Any refresh token an older build left in
localStorage is deleted on first load. Scripted clients that cannot hold a
cookie can still ask for it in the body with ?in_body=true.
F9 comes with it, as predicted: the WebSocket helpers read the role off the
live user instead of the token's claim. /ws/exec is root-equivalent on the
host, and a token minted while the account was an admin stayed syntactically
valid after a demotion.
The sharp edge was the migration, not the feature. _ensure_model_columns emits
ADD COLUMN without a DEFAULT, so SQLite would have filled token_version with
NULL on every existing install, every version check would have failed against
it, and the upgrade would have locked out every user everywhere. The helper now
renders NOT NULL DEFAULT <literal> for scalar defaults; test_schema_migration
builds a genuinely old-shaped user table and asserts the backfill. The version
comparison also tolerates NULL as 1, so a database migrated by some other route
still works.
Writing that test surfaced an undocumented precondition: _ensure_model_columns
does nothing unless `models` has been imported, since SQLModel.metadata is
empty until then. It holds in production because init_db imports first; now it
says so.
The authorization matrix did its job — adding two auth routes failed the suite
until both were classified, which is exactly the review moment it exists for.
30 new tests (698 total). Upgrading signs everyone out once.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
New destination type 'nfs' alongside SFTP and S3. The StackPilot container
needs no mount privileges: the Docker daemon mounts the export as a named
volume (stackpilot-nfs-dest-<id>, driver local/type nfs, recreated whenever
server/path/options change) and all file I/O runs through throwaway helper
containers (BACKUP_HELPER_IMAGE) — upload via put_archive, list via stat,
download via get_archive, delete/test via short-lived runs. Config: server,
export path, mount options (default rw), optional subdirectory (sanitized;
shell-safe charset). Mount failures surface as clean destination errors.
Settings UI gains the NFS form + summary; works everywhere destinations are
used (push, restore-from, scheduled backups incl. retention).
Verified live against a real kernel NFS server: test, push (file on the
export), list, restore-from incl. volume data, remote delete, config change
recreates the mount volume, unreachable server fails cleanly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Templates are now stack-shaped folders (compose.yaml + .env.example +
template.json) instead of DB rows + manifest.json + {{VAR}} rendering.
Pull copies the folder into a new stack; custom templates persist under
DATA_DIR/templates. Adds POST /api/templates/from-stack and a one-time
startup migration for pre-0.31 DB templates (drops the template table).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Per-stack auto-update policy on the stack Overview tab. When the background
image-update check finds a newer registry digest for one of a stack's images,
the stack is pulled + redeployed (or just flagged, "notify only"). Only running
stacks are auto-redeployed; a stopped stack is skipped, never silently started.
- models/auto_update.py: AutoUpdate(stack_id, agent_id, enabled, redeploy,
last_run/status/result) + schemas; registered in models/__init__.py.
- update_service: DB-free stack_images/stack_updates helpers (agent reuses
them); agent GET /agent/stacks/{id}/updates.
- services/auto_update_service.py: run_due/run_policy (local pull+up via
compose_service, remote via agent_service POST /agent/stacks/{id}/update,
notify-only with per-transition dedup); lazy-called from
update_service.background_loop. New stack_auto_updated notify event.
- routers: GET/PUT/run /api/stacks/{id}/auto-update and the
/api/agents/{id}/stacks/{sid}/auto-update variants (policy stored centrally).
- frontend: api/autoUpdate.ts + AutoUpdatePanel (enable, redeploy|notify-only,
Check now, last-run status) on StackDetail + RemoteStackDetail; EVENT_LABELS
gains stack_auto_updated + backup_failed.
Live-verified all four paths (updated / update-available / up-to-date /
skipped) against real compose.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Live remote-stack logs over a WebSocket proxied through the central app to
the agent (/ws/agent-logs/{agent}/{stack}); agent gains a WS log endpoint.
- Deploy to a remote host from the UI: host selector in the New Stack editor
and template dialog; templates instantiate onto an agent via the proxy.
- Network attach/detach: expandable inspect view per network with
connect/disconnect + container picker; GET /{id}/containers, POST connect/disconnect.
- Remove dead pages/Placeholder.tsx.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- BackupSchedule model + schedule_service: next-run calc (hourly/daily/weekly,
UTC), background scheduler loop (lifespan), run-one with retention pruning
(keep newest N per stack on the destination), backup_failed notify event.
- routers/schedules.py: schedules CRUD + run-now; registered in main.py.
- Frontend: api/schedules.ts + Settings → Scheduled backups (list with next/last
run + status, enable/disable, run-now, delete; add form with stack/destination/
frequency/time/weekday/retention/volumes).
Rough-verified only (per request): py_compile, frontend tsc build, app import
(95 routes), next-run math sanity. Full live run to be tested after deploy.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- BackupDestination model + backup_destination_service (SFTP via paramiko,
S3-compatible via boto3): upload/list/download/delete/test.
- routers/destinations.py: destinations CRUD (secrets masked, merge-on-update),
test, list/delete remote backups. backups.py: POST /{id}/backup/push and
POST /restore-from (download from a destination + restore, volumes included).
- Frontend: Settings → Backup destinations (SFTP/S3 forms + test); Backup dialog
can push to a destination; Restore dialog can pick a destination + backup.
- deps: paramiko 3.5.0, boto3 1.35.99.
Verified end-to-end against live MinIO + atmoz/sftp: create/test destinations,
push (incl. volumes), list, restore-from to a fresh stack (volume data intact),
delete remote backup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
