a1cd14a1cddbb45fb635cd885b8d98dbc5b498a1
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 menzeljandClaude Opus 5
|
a1cd14a1cd |
Scan images for known vulnerabilities (0.59.0)
The Images page knew what was running and whether it was current. It could not say whether any of it was exploitable, which is the question people actually have about a homelab full of images they pulled once and forgot. Trivy runs as a throwaway container rather than being installed into StackPilot's image, reusing the helper-container pattern backups already use for volume contents. Three reasons: a 100 MB security tool and a vulnerability database that changes weekly have no business in a release artifact, pinning SCANNER_IMAGE is then a real version control, and the scanner updates itself by pulling a newer tag. It gets the socket read-only so it inspects images the daemon already has instead of pulling them again, and a named volume for its database so the ~50 MB download happens once rather than per scan. The number the UI leads with is "fixable", not the total. A base image with 300 unfixable low-severity CVEs is not a task and a page that shows 300 in red teaches people to ignore it; three findings with a fixed version available are something to do this afternoon. Counts are stored per severity, findings are sorted worst-first and capped at 200 — every finding is counted, only the list is trimmed, so the cap can never hide the severity distribution. The failure mode this had to avoid is a security feature that reads as clean when it is broken. A scanner that cannot run stores the error and *keeps the previous counts* rather than resetting to zero, so a transient daemon problem does not silently turn a bad image green. There is a test for exactly that, and another for unparseable output. Staleness is handled the same way: the local image id is recorded with the scan, and pulling the image marks the result stale instead of presenting yesterday's numbers for today's bytes. Sweeps are deliberately serial and singly-locked. Scanning is CPU- and IO-heavy, and running eight at once on a homelab box would starve the very containers the scan is meant to protect. docker-py is synchronous, so the scan itself goes to a thread — otherwise a ten-minute scan blocks every other request on the loop. Reading results is allowed for the read-only role, which the authorization matrix made me justify in writing: CVE ids and package versions for images whose tags and compose files that role can already see, and polling them is the monitoring use case a read-only API token exists for. Running a scan stays admin-only because it spends real CPU. 20 tests against a report shaped like Trivy's real output, covering the counting, the fixable number, worst-first ordering, the cap, both failure paths, staleness, and that two sweeps cannot overlap. Verified end to end through the API as well, including that a failed rescan keeps its previous counts and shows the error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 5
|
9247ff9621 |
Deploy stacks from a Git repository (0.58.0)
StackPilot's stacks were already plain folders on disk, which makes GitOps less
of an architectural change than it would be elsewhere: a sync is "make these
files match that repo, then compose up". Almost all of the design effort went
into the word "these", because getting it wrong destroys data.
A stack folder is not just the compose file. Compose creates bind-mount
directories in it — ./config, ./data — and those hold the live state of whatever
is running. So the obvious implementation, clone into the stack folder and
git reset --hard, is a data-loss bug waiting for its first `git clean`. Instead
the clone lives in a cache under ${DATA_DIR}/git/<stack> where reset and clean
are safe, and the configured subtree is copied across. No .git ends up in the
stack folder, so backups and the file browser are unaffected too.
Deletion is the other half. Making a folder "match" a repo naively means
removing what the repo does not have, which is exactly the application data
above. So each sync records the paths it wrote, and the next sync may delete
only those — a file the repository never provided cannot be touched by any code
path here. Tested directly: a database file and a hand-written .env survive a
sync that replaces the compose file and removes a file the repo dropped.
What the repo does provide is overwritten, hand edits included. That is the
point of GitOps rather than a wart, but it is a surprise if you attach a repo to
a stack you have been editing, so the connect form says it before the first sync
and the first sync is never automatic.
The webhook is the only route in StackPilot with no bearer token, because a Git
forge has none to present. It authenticates with an HMAC over the body —
X-Hub-Signature-256 for GitHub/Gitea/Forgejo, X-Gitlab-Token for GitLab, both
compared in constant time — and answers 404, not 403, to anything unsigned. A
403 would confirm that a given stack exists and is connected to a repository,
which an unauthenticated caller has not earned. The authorization matrix test
caught this route being public and made me write that reasoning down in it,
which is exactly what that test is for.
Credentials never reach a command line: ps is readable by every process on the
host, and this runs in a container next to everything else. The HTTPS token goes
to git through GIT_ASKPASS and the environment, the SSH key through a 0600 file
kept outside the working tree, and everything git prints is scrubbed of both —
plus any credential-carrying URL — before it is stored in last_error or shown.
Auto-deploy takes the same per-stack lock as every other lifecycle action, so a
webhook firing mid-deploy reports "files synced, stack busy" instead of racing a
second compose run at the same project.
The image needed git and openssh-client, which is the only reason this release
touches the Dockerfile.
26 tests against real repositories created with the real git binary, none of
them touching the network — mocking git would mostly test the mock. Verified end
to end as well: connect, sync, a push that changes one file and deletes another,
a wrongly signed webhook, a correctly signed one, and the live data still there
afterwards.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 5
|
e650aa6833 |
Add API tokens for scripts and CI (0.57.0)
A session token is the wrong credential for automation. It expires in an hour, it is minted by typing a password, and revoking it signs every one of that person's devices out. So automation gets its own credential, revocable on its own, and showing up in the audit log as itself. Three decisions worth recording, because each one is a place this could have been built wrong. **Only a hash is stored.** This is the opposite call from registry passwords one release ago, and for a concrete reason: a registry password has to be handed back to the registry, so it must be recoverable and is encrypted. A token is only ever compared against, so it does not need to be — and not keeping it is the difference between leaking the database and leaking everything the database protects. It is shown once and cannot be recovered; a readable prefix is kept so rows are still identifiable in the UI and the audit log. The hash is SHA-256, deliberately not bcrypt: bcrypt is slow to make guessing low-entropy human passwords expensive, and a token is 256 bits of secrets output, so the cost would buy nothing and would land on every single API request. **The scope is not folded into the User object.** get_current_user returns a session-attached row; downgrading its role in place to represent a read-only token would be written back to the database the next time anything committed that user — logout-everywhere does exactly that. So the token row is stashed on request.state and require_admin consults it, leaving the User untouched. The same lookup caps a token at its owner's authority rather than trusting the scope alone, so a demoted admin's token drops to read-only with them and a disabled account's tokens stop working. **A token cannot make itself permanent.** Creating tokens and creating users now require a signed-in session, via a require_session dependency that rejects token-authenticated requests. Without it, a leaked CI credential could mint a second one and survive its own revocation — the failure mode where revoking the leak does nothing. This is the one behaviour change for existing installs: scripted user creation now needs a login. The WebSocket routes still take JWTs only. They carry logs, the terminal and the deploy console, which a CI job has no use for, and leaving them alone keeps the token surface to the REST API. 19 tests, covering what is stored, that a read token really is read-only while its owner is an admin, that demoting and disabling the owner both take effect, expiry, tampering, the throttle on last-used writes, and that a token can neither mint another nor create a user. Verified end to end against a running app: two tokens, both scopes, revocation, and no plaintext anywhere in the database or the list response. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 5
|
95e03f031f |
Add private registry credentials, and stop update checks lying (0.56.0)
This was on the gap list as a missing feature, but it was a bug first. The
update checker asks the registry for a tag's digest over HTTP itself, and could
only do it anonymously. A private repository answers 401, remote_digest returned
None, and None already meant "could not reach registry" — so a private image was
indistinguishable from a network blip. The Images page showed nothing and a
stack pinned to a six-month-old image looked up to date indefinitely.
So AuthRequired is now its own exception, separate from unreachable, and the
error names the registry and which of the two problems it is: "ghcr.io needs
credentials" when there are none, "ghcr.io rejected the stored credentials" when
there are and they are wrong. Those are different fixes, and the message should
say which one you need. The plain unreachable message survives unchanged, with a
test pinning it, because not every failure is an auth failure.
Two consumers need the credentials and they need them in completely different
shapes, which is why this is its own service rather than a field on something
else. StackPilot's own checker wants (user, password) inside async code that has
no database session, so the rows are mirrored into an in-memory cache that
reload() refills on startup and after every write. The Docker CLI wants a
config.json, so reload() writes one into ${DATA_DIR}/docker and compose runs with
DOCKER_CONFIG pointed at it. Generating it from the database every time is what
makes deletion real: removing a registry in the UI revokes the CLI's login
instead of leaving a stale one in ~/.docker.
Host normalization is the join that makes any of it work, and it is easy to
underestimate. parse_ref only ever produces registry-1.docker.io, nobody types
that, and the CLI wants the whole thing under https://index.docker.io/v1/ — three
spellings of one registry across three layers. canonical_host settles on what
parse_ref produces, the config writer translates on the way out, and a bare
nginx:alpine finds credentials entered as "docker.io". Verified end to end:
typed as the v1 URL, stored as registry-1.docker.io, written as the v1 URL.
The password is encrypted at rest with the same key as backup destinations and
never leaves the server, not even masked — the API returns has_password, which
is all the form needs to offer "leave blank to keep". A row that cannot be
decrypted after a SECRET_KEY change is skipped with a warning rather than taking
every other registry down with it. Everything here is admin-only including the
reads, because even masked the rows say which registries this install talks to
and under what account.
The Test button asks the registry rather than validating a string, following the
Bearer challenge with credentials attached the way a real client does. Only an
outright 401 counts as wrong credentials; anything else means reachable and
talking, which is as much as a credentials check can honestly claim. Checked
against the live Docker Hub token endpoint with deliberately wrong credentials.
33 tests: the normalization table, the cache, the generated config.json down to
its 0600 mode and the Docker Hub key, encryption at rest, that no password field
appears in any response, and the 401-is-reported behaviour that started this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 5
|
51d1998307 |
Remove the remote-host (agent) integration (0.48.0)
StackPilot now manages exactly one Docker host: the one it runs on. The
stackpilot-agent sidecar and everything that proxied to it are gone — 4721
lines deleted against 657 added.
Deleted outright: agent/ (image, compose, env), agent_app.py, models/agent.py,
routers/agents.py (1200 lines), services/agent_service.py, the agent API client,
RemoteStackDetail, the host components and AgentStacksSection. That removes 57
API routes and the three /ws/agent-* proxies.
Threaded out everywhere else, which was the bulk of the work. Every API module
carried an optional agentId that switched the base path; every page that listed
Docker objects rendered one section per host behind a HostHeader; Files had a
host switcher; the New Stack editor and the template dialog had host selectors;
schedules, auto-update policies and stack summaries carried agent_id. All of it
is gone, and the typechecker drove the sweep — 85 files touched, tsc and the
build clean.
Two things the removal exposed as dead weight rather than merely unused:
compose_service kept an in-process busy set purely because the agent needed a
lock and has no database. With the agent gone that was a second source of truth
next to the real DB lock, so it is deleted; compute_status now reports only what
the containers say and the two callers that want "updating" overlay the lock.
StacksTable's linkBase prop only ever existed to point at /hosts/{id}/stacks.
The dashboard's "Hosts 1/1 online" KPI can no longer say anything else, so the
tile and the KPIs behind it are gone and the row is five wide.
Upgrading matters here. An existing install still has an agent table holding
each remote host's URL and bearer token — full Docker control of that host,
sitting in the database with nothing left to use it. _drop_removed_schema drops
it on first start, and drops the agent_id columns where the SQLite build
supports DROP COLUMN. Each statement runs in its own transaction on purpose: a
failed DDL poisons the transaction it is in, so sharing one would let an
unsupported column drop take the table drop down with it. test_agent_removal
covers both branches plus the fresh-install and idempotent cases, and an
end-to-end run against a seeded pre-0.48 database confirms the table is gone and
every /api/agents route answers 404.
Docstrings that justified a design by "shared with the agent, which has no
database" were rewritten rather than left lying: update_service's persistence
callback and image_status_store are still the right split (registry logic stays
testable without a database), but for that reason now, not the old one. The
README's multi-host sections are removed and an upgrade note explains what to do
with running agent containers; ROADMAP keeps its history behind a note saying
the feature it describes no longer exists.
CI no longer builds or pushes stackpilot-agent.
735 tests pass, ruff and tsc clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
|
||
|
menzeljandClaude Opus 5
|
09bed274eb |
Lock stacks during compose runs, cache stats, persist runtime state (0.47.0)
F7 — Nothing stopped two compose operations landing on the same stack. There was a busy flag, but is_busy() was only ever read to colour the status column; no lifecycle handler consulted it before acting. Two tabs, or auto-update picking up a stack somebody had just clicked, both ran pull + up -d against the same project and raced over recreating containers. Lifecycle calls, the two deploy WebSockets and the auto-update pass now take a real lock; a second caller gets 409 (or an error frame and close 4409) and auto-update skips and retries next cycle. The lock is a row rather than a set in one worker's memory, so it holds across workers and across a restart, and it carries an expiry — a worker killed mid-deploy would otherwise strand the stack with no fix short of editing the database. F10 — /api/stacks/stats sampled every running container on every call, one blocking daemon request each, and both the dashboard and the stacks list poll it every five seconds. Two tabs on a 40-container host meant a sustained ~16 samples a second. Cached for 4s behind a lock so concurrent callers share one sweep, the same shape dashboard_service already used for its fleet aggregate. F11 — Three module dicts assumed exactly one uvicorn worker without saying so and were lost on restart. The busy set is the lock above. The image update cache is now mirrored to SQLite, so a restart shows the badges immediately instead of blanking them for up to an hour, and the already-notified marks come back with them rather than re-announcing the same updates. The login rate limiter is a table, so it cannot be cleared by getting the process to restart and no longer multiplies by the worker count. The constraint that shaped this: compose_service and update_service are shared with the agent, which has no database. Neither may import one. So the lock is a separate service the central app enforces at its own entry points, and update persistence is an opt-in callback the central app registers in its lifespan — the agent registers nothing and behaves exactly as before. A test asserts update_service never imports the database, since that is the kind of thing a later change breaks silently. Both new nets were checked by reverting the fix: dropping the lock from _lifecycle fails six tests, removing the stats cache fails the one that names the behaviour. Also wires up cache pruning in the same sweep — without it both the dict and the table grew one entry per image tag ever run, for the life of the install. 31 new tests (729 total). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG |
||
|
menzeljandClaude Fable 5
|
1609b8bcc3 |
Phase 25: templates as stack folders (0.31.0)
Templates are now stack-shaped folders (compose.yaml + .env.example +
template.json) instead of DB rows + manifest.json + {{VAR}} rendering.
Pull copies the folder into a new stack; custom templates persist under
DATA_DIR/templates. Adds POST /api/templates/from-stack and a one-time
startup migration for pre-0.31 DB templates (drops the template table).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
255c8441c6 |
Phase 22: auto-update (Watchtower-style), local + agent (0.28.0)
Per-stack auto-update policy on the stack Overview tab. When the background
image-update check finds a newer registry digest for one of a stack's images,
the stack is pulled + redeployed (or just flagged, "notify only"). Only running
stacks are auto-redeployed; a stopped stack is skipped, never silently started.
- models/auto_update.py: AutoUpdate(stack_id, agent_id, enabled, redeploy,
last_run/status/result) + schemas; registered in models/__init__.py.
- update_service: DB-free stack_images/stack_updates helpers (agent reuses
them); agent GET /agent/stacks/{id}/updates.
- services/auto_update_service.py: run_due/run_policy (local pull+up via
compose_service, remote via agent_service POST /agent/stacks/{id}/update,
notify-only with per-transition dedup); lazy-called from
update_service.background_loop. New stack_auto_updated notify event.
- routers: GET/PUT/run /api/stacks/{id}/auto-update and the
/api/agents/{id}/stacks/{sid}/auto-update variants (policy stored centrally).
- frontend: api/autoUpdate.ts + AutoUpdatePanel (enable, redeploy|notify-only,
Check now, last-run status) on StackDetail + RemoteStackDetail; EVENT_LABELS
gains stack_auto_updated + backup_failed.
Live-verified all four paths (updated / update-available / up-to-date /
skipped) against real compose.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
84ef3df59e |
Phase 7: scheduled (recurring) backups (0.7.0)
- BackupSchedule model + schedule_service: next-run calc (hourly/daily/weekly, UTC), background scheduler loop (lifespan), run-one with retention pruning (keep newest N per stack on the destination), backup_failed notify event. - routers/schedules.py: schedules CRUD + run-now; registered in main.py. - Frontend: api/schedules.ts + Settings → Scheduled backups (list with next/last run + status, enable/disable, run-now, delete; add form with stack/destination/ frequency/time/weekday/retention/volumes). Rough-verified only (per request): py_compile, frontend tsc build, app import (95 routes), next-run math sanity. Full live run to be tested after deploy. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 4.8
|
7bd449101d |
Phase 6: remote backup destinations — SFTP & S3 (0.6.0)
- BackupDestination model + backup_destination_service (SFTP via paramiko,
S3-compatible via boto3): upload/list/download/delete/test.
- routers/destinations.py: destinations CRUD (secrets masked, merge-on-update),
test, list/delete remote backups. backups.py: POST /{id}/backup/push and
POST /restore-from (download from a destination + restore, volumes included).
- Frontend: Settings → Backup destinations (SFTP/S3 forms + test); Backup dialog
can push to a destination; Restore dialog can pick a destination + backup.
- deps: paramiko 3.5.0, boto3 1.35.99.
Verified end-to-end against live MinIO + atmoz/sftp: create/test destinations,
push (incl. volumes), list, restore-from to a fresh stack (volume data intact),
delete remote backup.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
59037f4287 |
Phase 5: multi-host agents (0.5.0)
- stackpilot-agent: slim token-guarded FastAPI (reuses compose_service) exposing stack CRUD/lifecycle/logs + system info; same image, different CMD. agent/ Dockerfile + compose + .env.example. - Central proxy: Agent model, agent_service (httpx ping/proxy + live status: online/offline/unauthorized + hostname/last_seen), routers/agents.py (CRUD + ping + proxied stacks/lifecycle/logs/system). - Frontend: Settings → Remote hosts (add/check/remove, connectivity dot); Stacks grouped by host; remote stack detail with lifecycle, live logs, compose/.env edit. Verified end-to-end: agent+main on a shared network — register (good/bad token), list/create/start/logs/delete remote stacks, offline detection (502). Remote backup destinations (SFTP/S3) deferred. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 4.8
|
8d19b09abd |
Phase 4: backups w/ volumes, notifications, settings & users, audit page (0.4.0)
- Backup/restore: per-stack tar.gz incl. named-volume snapshots (helper container), upload restore with rename/overwrite/conflict detection. - Notifications: ntfy/Discord/Slack/Gotify/generic webhooks, per-event subscriptions; wired into the update checker and stack lifecycle. - Settings page: update-check interval, webhook CRUD + test, user management (with last-admin safeguards). - Audit log page (searchable, paginated). - Mobile-responsive sidebar/layout. Multi-host agents and remote backup destinations (SFTP/S3) deferred. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 4.8
|
22d9864436 |
Phase 3: env masking, image updates, port conflicts, resources, templates (0.3.0)
Backend:
- update_service: registry manifest digest check (Docker Hub/ghcr/lscr/private
v2 token auth) vs local RepoDigests; in-memory cache + background loop
- port_service: parse compose ports, check /proc/net/tcp[6] + docker bindings
- template_service + bundled templates (jellyfin/vaultwarden/uptime-kuma/
paperless-ngx/gitea) with {{VAR}} placeholders; custom templates in DB
- compose_edit set_resources (deploy.resources.limits/reservations)
- routers: images, ports, templates, editor/set-resources
- Template model; background update task wired into lifespan
Frontend:
- EnvEditor (table + raw, sensitive masking, quick-insert)
- Images page + UpdateBadge + dashboard 'updates available' banner
- PortConflictDialog pre-deploy check on Deploy
- ResourcePanel (CPU/RAM sliders) as editor Limits tab
- Templates page with per-variable instantiate form
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
f732cb080b |
Initial commit: StackPilot Phase 1 (Core)
Self-hosted Docker Compose manager. - Backend: FastAPI + docker-py + SQLite (JWT auth, file-first stacks, lifecycle, live status, WebSocket logs, docker-run converter, audit log) - Frontend: React + Vite + Tailwind (login/setup, dashboard, stacks, stack detail, Monaco editor, dark/light theme) - Deployment: docker-compose.yml, Dockerfiles, nginx reverse proxy Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |