d6cebb337109616dab2d42955f43633c02479073
10
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 menzeljandClaude Opus 5.5
|
d6cebb3371 |
Fix what a review of the whole app found, end to end (0.67.1)
A pass over every feature against a real Docker daemon, then clicking through the UI as admin and read-only user, on a desktop and a phone. Security. /ws/deploy ran compose up for the read-only role; it is admin-only now. A container's inspect view, a stack's compose file (POSTGRES_PASSWORD: hunter2) and the viewer's .env tab no longer show credentials to that role. Git branch and URL values that git would read as options (-x, ext::) are refused. SFTP host keys are checked: pinned on first use or entered by hand (key line or SHA256 fingerprint), a different key is refused before the password is sent. A backup name from the API can no longer reach outside the destination folder. The file browser refuses to move or rename a browse root, as delete did. Docker lifecycle. Closing the deploy console released the stack lock while compose was still running and logged rc=None as success; deploys and updates are now background runs that hold the lock to the end, report the real exit code and can be re-attached to. Every closed log view left a `compose logs -f` behind. A dozen open tabs exhausted the default thread pool and hung the API; event and exec streams have their own threads. Auto-update and the update endpoint ignored failed pulls. A deleted git stack was recreated and started by the next webhook or poll; deleting a stack now drops its git source, auto-update policy and schedule entries. Deleting no longer walks into mounted shares, is refused while a backup runs, and removes the containers even when the compose file is broken. Stack folders compose has to normalise (Nextcloud -> nextcloud) work everywhere and group correctly on the volumes, networks and images pages. Detection. Host port conflicts read the container's own network namespace; /proc/1/net is read instead. Images referenced as docker.io/… were never update-checked; local builds no longer report "needs credentials". NVIDIA GPUs are read from the driver's procfs. The backup dialog's "path mismatch" warning came from mountinfo's filesystem-relative root and fired for any stacks folder on its own disk, ZFS dataset or btrfs subvolume; the host path now comes from Docker's mounts. Backups and destinations. The download ignored the folder/volume selection (axios sent binds[]). Restoring under a new name kept the old one. A report header past nginx's buffer turned a good backup into a 502. SFTP/S3 errors were bare 500s and "Test" did not try to write. A destination used by a schedule could be deleted. Backup file names are in the schedule's or the browser's time zone instead of UTC. Editor and converter. `docker run` conversion produced invalid compose for --network host, named volumes and -dit; --gpus and --mount were dropped; a multi-line command pasted into the one-line field became `image: ' '` (now a textarea, and flattened continuations are read). Validation failed for env_file stacks. The editor overwrote unsaved edits on every container event, lost them on a stray click (now asks), threw when closing the diff, and its helper panel listed the starter service "app" for every existing stack. Monaco is bundled instead of loaded from jsdelivr, so the editor works offline (and monaco-editor 0.57 drops the dompurify advisory; axios and react-router updated). UI. All times were two hours off: offset-less UTC from SQLite was read as local time, and "next run" always said "just now". A token refresh mid-session duplicated deploys, reset terminals and doubled logs. A backend restart (update, self-update) signed everyone out on reload; the page now waits for it. The editor was unusable on a phone, tables and tabs overflowed, dialog text was black on the dark theme, and the file browser opened on an error with the default roots. Read-only users get "Admin only" pages instead of 403s; in-use networks offer no delete; "Scan all" covers StackPilot's own stacks only and runs in the background instead of timing out at 600s. Deployment. nginx passes the port in Host (the OIDC redirect URI lost :5009) and keeps an outer proxy's X-Forwarded-Proto. The frontend waits for a healthy backend (healthcheck probes every 2s while starting). The frontend image builds with npm ci. Secrets are 0644 in a 0700 folder so non-root containers can read them; existing files are fixed at startup. The test suite no longer starts real containers on a host with Docker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fx7gcuy8d6nPd95g43rGHu |
||
|
menzeljandClaude Opus 5
|
95e03f031f |
Add private registry credentials, and stop update checks lying (0.56.0)
This was on the gap list as a missing feature, but it was a bug first. The
update checker asks the registry for a tag's digest over HTTP itself, and could
only do it anonymously. A private repository answers 401, remote_digest returned
None, and None already meant "could not reach registry" — so a private image was
indistinguishable from a network blip. The Images page showed nothing and a
stack pinned to a six-month-old image looked up to date indefinitely.
So AuthRequired is now its own exception, separate from unreachable, and the
error names the registry and which of the two problems it is: "ghcr.io needs
credentials" when there are none, "ghcr.io rejected the stored credentials" when
there are and they are wrong. Those are different fixes, and the message should
say which one you need. The plain unreachable message survives unchanged, with a
test pinning it, because not every failure is an auth failure.
Two consumers need the credentials and they need them in completely different
shapes, which is why this is its own service rather than a field on something
else. StackPilot's own checker wants (user, password) inside async code that has
no database session, so the rows are mirrored into an in-memory cache that
reload() refills on startup and after every write. The Docker CLI wants a
config.json, so reload() writes one into ${DATA_DIR}/docker and compose runs with
DOCKER_CONFIG pointed at it. Generating it from the database every time is what
makes deletion real: removing a registry in the UI revokes the CLI's login
instead of leaving a stale one in ~/.docker.
Host normalization is the join that makes any of it work, and it is easy to
underestimate. parse_ref only ever produces registry-1.docker.io, nobody types
that, and the CLI wants the whole thing under https://index.docker.io/v1/ — three
spellings of one registry across three layers. canonical_host settles on what
parse_ref produces, the config writer translates on the way out, and a bare
nginx:alpine finds credentials entered as "docker.io". Verified end to end:
typed as the v1 URL, stored as registry-1.docker.io, written as the v1 URL.
The password is encrypted at rest with the same key as backup destinations and
never leaves the server, not even masked — the API returns has_password, which
is all the form needs to offer "leave blank to keep". A row that cannot be
decrypted after a SECRET_KEY change is skipped with a warning rather than taking
every other registry down with it. Everything here is admin-only including the
reads, because even masked the rows say which registries this install talks to
and under what account.
The Test button asks the registry rather than validating a string, following the
Bearer challenge with credentials attached the way a real client does. Only an
outright 401 counts as wrong credentials; anything else means reachable and
talking, which is as much as a credentials check can honestly claim. Checked
against the live Docker Hub token endpoint with deliberately wrong credentials.
33 tests: the normalization table, the cache, the generated config.json down to
its 0600 mode and the Docker Hub key, encryption at rest, that no password field
appears in any response, and the 401-is-reported behaviour that started this.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 5
|
51d1998307 |
Remove the remote-host (agent) integration (0.48.0)
StackPilot now manages exactly one Docker host: the one it runs on. The
stackpilot-agent sidecar and everything that proxied to it are gone — 4721
lines deleted against 657 added.
Deleted outright: agent/ (image, compose, env), agent_app.py, models/agent.py,
routers/agents.py (1200 lines), services/agent_service.py, the agent API client,
RemoteStackDetail, the host components and AgentStacksSection. That removes 57
API routes and the three /ws/agent-* proxies.
Threaded out everywhere else, which was the bulk of the work. Every API module
carried an optional agentId that switched the base path; every page that listed
Docker objects rendered one section per host behind a HostHeader; Files had a
host switcher; the New Stack editor and the template dialog had host selectors;
schedules, auto-update policies and stack summaries carried agent_id. All of it
is gone, and the typechecker drove the sweep — 85 files touched, tsc and the
build clean.
Two things the removal exposed as dead weight rather than merely unused:
compose_service kept an in-process busy set purely because the agent needed a
lock and has no database. With the agent gone that was a second source of truth
next to the real DB lock, so it is deleted; compute_status now reports only what
the containers say and the two callers that want "updating" overlay the lock.
StacksTable's linkBase prop only ever existed to point at /hosts/{id}/stacks.
The dashboard's "Hosts 1/1 online" KPI can no longer say anything else, so the
tile and the KPIs behind it are gone and the row is five wide.
Upgrading matters here. An existing install still has an agent table holding
each remote host's URL and bearer token — full Docker control of that host,
sitting in the database with nothing left to use it. _drop_removed_schema drops
it on first start, and drops the agent_id columns where the SQLite build
supports DROP COLUMN. Each statement runs in its own transaction on purpose: a
failed DDL poisons the transaction it is in, so sharing one would let an
unsupported column drop take the table drop down with it. test_agent_removal
covers both branches plus the fresh-install and idempotent cases, and an
end-to-end run against a seeded pre-0.48 database confirms the table is gone and
every /api/agents route answers 404.
Docstrings that justified a design by "shared with the agent, which has no
database" were rewritten rather than left lying: update_service's persistence
callback and image_status_store are still the right split (registry logic stays
testable without a database), but for that reason now, not the old one. The
README's multi-host sections are removed and an upgrade note explains what to do
with running agent containers; ROADMAP keeps its history behind a note saying
the feature it describes no longer exists.
CI no longer builds or pushes stackpilot-agent.
735 tests pass, ruff and tsc clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
|
||
|
menzeljandClaude Opus 5
|
09bed274eb |
Lock stacks during compose runs, cache stats, persist runtime state (0.47.0)
F7 — Nothing stopped two compose operations landing on the same stack. There was a busy flag, but is_busy() was only ever read to colour the status column; no lifecycle handler consulted it before acting. Two tabs, or auto-update picking up a stack somebody had just clicked, both ran pull + up -d against the same project and raced over recreating containers. Lifecycle calls, the two deploy WebSockets and the auto-update pass now take a real lock; a second caller gets 409 (or an error frame and close 4409) and auto-update skips and retries next cycle. The lock is a row rather than a set in one worker's memory, so it holds across workers and across a restart, and it carries an expiry — a worker killed mid-deploy would otherwise strand the stack with no fix short of editing the database. F10 — /api/stacks/stats sampled every running container on every call, one blocking daemon request each, and both the dashboard and the stacks list poll it every five seconds. Two tabs on a 40-container host meant a sustained ~16 samples a second. Cached for 4s behind a lock so concurrent callers share one sweep, the same shape dashboard_service already used for its fleet aggregate. F11 — Three module dicts assumed exactly one uvicorn worker without saying so and were lost on restart. The busy set is the lock above. The image update cache is now mirrored to SQLite, so a restart shows the badges immediately instead of blanking them for up to an hour, and the already-notified marks come back with them rather than re-announcing the same updates. The login rate limiter is a table, so it cannot be cleared by getting the process to restart and no longer multiplies by the worker count. The constraint that shaped this: compose_service and update_service are shared with the agent, which has no database. Neither may import one. So the lock is a separate service the central app enforces at its own entry points, and update persistence is an opt-in callback the central app registers in its lifespan — the agent registers nothing and behaves exactly as before. A test asserts update_service never imports the database, since that is the kind of thing a later change breaks silently. Both new nets were checked by reverting the fix: dropping the lock from _lifecycle fails six tests, removing the stats cache fails the one that names the behaviour. Also wires up cache pruning in the same sweep — without it both the dict and the table grew one entry per image tag ever run, for the life of the install. 31 new tests (729 total). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG |
||
|
menzeljandClaude Opus 5
|
60a7ccff93 |
Add a test suite, a linter and a CI gate in front of the build (0.45.0)
The repo had no tests, no lint config, and a CI that went straight from push
to docker push. That is the reason F1 could ship: authorization lives in the
routers, each of 171 routes independently picks require_admin or
get_current_user, and nothing checked the choice was right.
670 tests, no Docker daemon needed. The app is driven through TestClient
without entering it as a context manager, which skips the lifespan — no
background loops, no socket — and conftest points DATA_DIR/STACKS_DIR at a
temp directory before anything is imported.
test_route_authorization.py is the load-bearing one. Rather than 171 implied
decisions it states the policy once — every route requires admin unless it is
listed in USER_READABLE or PUBLIC — and fails on any route that disagrees. A
new route defaults to admin, which is the safe direction; what it catches is a
route written with get_current_user that nobody weighed against "can this
return a credential". Writing the allowlist meant auditing all 53 user-readable
routes, which turned up one more leak: GET /api/templates/{id} returns a
template's env, and "save stack as template" snapshots the stack's real .env
into it. Now admin-only; the listing stays open.
test_agent_authorization.py pins the same invariant on the agent, where the
whole access model is one shared token declared per route and a single
forgotten Depends(verify_token) would hand over the host.
Both were checked by reintroducing the bug: re-opening /api/files/read fails
three tests with actionable messages, dropping a token guard fails two.
test_bundled_templates.py covers the 83 templates — parse, image per service,
.env.example in sync with what compose reads, every bind-mounted file actually
shipped, and no working default password. It found one on its first run:
authentik shipped PG_PASS=change-me and AUTHENTIK_SECRET_KEY=change-me against
a compose that marks both required, so the stack would have come up with a
known password instead of refusing to start. Fixed.
The rest ports the ad-hoc harnesses from 0.44.0 into permanent tests (crypto
round-trip incl. plaintext passthrough and key-loss handling, the browse
sandbox) and covers compose_service's slug/status/file handling and
secret_service's name validation.
ruff is configured as a floor, not a style bar: F, E9 and B only. Import
sorting is deliberately out — it is style, and enabling it would rewrite the
imports of nine files that have nothing else wrong. The 12 findings it did have
are fixed here (unused imports, an unused local, four raise-without-from that
were swallowing exception context).
CI now runs check (ruff, pytest, tsc) and only builds if it passes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
|
||
|
menzeljandClaude Fable 5
|
9119f94536 |
Clear the stack update pill immediately after a manual/auto update (0.38.4)
The amber image-update indicator is fed from update_service._CACHE, which only the background loop refreshed — after a per-stack Update/Pull the stale digests kept the pill on until the next pass. Now the local digests are reconciled with the cached remote digests right after a successful pull/update (local backend, agent lifecycle, auto-update pass), and the frontend invalidates the stack-updates queries after actions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 4.8
|
cd15cdc75e |
0.36.0: per-stack image-update indicator on the Stacks overview
Shows an amber "Update" pill next to a stack's status (and highlights the
inline Update button) when any of the stack's images has a newer digest in
the registry. Reuses the existing background image-update check — a new
update_service.stacks_update_summary() reads the cached digests in a single
container sweep (no extra registry calls), exposed as GET /api/stacks/updates
and proxied per agent at GET /api/agents/{id}/stacks/updates. The Stacks page
and each remote-host section poll it every 60s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
255c8441c6 |
Phase 22: auto-update (Watchtower-style), local + agent (0.28.0)
Per-stack auto-update policy on the stack Overview tab. When the background
image-update check finds a newer registry digest for one of a stack's images,
the stack is pulled + redeployed (or just flagged, "notify only"). Only running
stacks are auto-redeployed; a stopped stack is skipped, never silently started.
- models/auto_update.py: AutoUpdate(stack_id, agent_id, enabled, redeploy,
last_run/status/result) + schemas; registered in models/__init__.py.
- update_service: DB-free stack_images/stack_updates helpers (agent reuses
them); agent GET /agent/stacks/{id}/updates.
- services/auto_update_service.py: run_due/run_policy (local pull+up via
compose_service, remote via agent_service POST /agent/stacks/{id}/update,
notify-only with per-transition dedup); lazy-called from
update_service.background_loop. New stack_auto_updated notify event.
- routers: GET/PUT/run /api/stacks/{id}/auto-update and the
/api/agents/{id}/stacks/{sid}/auto-update variants (policy stored centrally).
- frontend: api/autoUpdate.ts + AutoUpdatePanel (enable, redeploy|notify-only,
Check now, last-run status) on StackDetail + RemoteStackDetail; EVENT_LABELS
gains stack_auto_updated + backup_failed.
Live-verified all four paths (updated / update-available / up-to-date /
skipped) against real compose.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
menzeljandClaude Opus 4.8
|
8d19b09abd |
Phase 4: backups w/ volumes, notifications, settings & users, audit page (0.4.0)
- Backup/restore: per-stack tar.gz incl. named-volume snapshots (helper container), upload restore with rename/overwrite/conflict detection. - Notifications: ntfy/Discord/Slack/Gotify/generic webhooks, per-event subscriptions; wired into the update checker and stack lifecycle. - Settings page: update-check interval, webhook CRUD + test, user management (with last-admin safeguards). - Audit log page (searchable, paginated). - Mobile-responsive sidebar/layout. Multi-host agents and remote backup destinations (SFTP/S3) deferred. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
menzeljandClaude Opus 4.8
|
22d9864436 |
Phase 3: env masking, image updates, port conflicts, resources, templates (0.3.0)
Backend:
- update_service: registry manifest digest check (Docker Hub/ghcr/lscr/private
v2 token auth) vs local RepoDigests; in-memory cache + background loop
- port_service: parse compose ports, check /proc/net/tcp[6] + docker bindings
- template_service + bundled templates (jellyfin/vaultwarden/uptime-kuma/
paperless-ngx/gitea) with {{VAR}} placeholders; custom templates in DB
- compose_edit set_resources (deploy.resources.limits/reservations)
- routers: images, ports, templates, editor/set-resources
- Template model; background update task wired into lifespan
Frontend:
- EnvEditor (table + raw, sensitive masking, quick-insert)
- Images page + UpdateBadge + dashboard 'updates available' banner
- PortConflictDialog pre-deploy check on Deploy
- ResourcePanel (CPU/RAM sliders) as editor Limits tab
- Templates page with per-variable instantiate form
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|