Commit Graph
6 Commits
Author SHA1 Message Date
menzeljandClaude Opus 5.5 d6cebb3371 Fix what a review of the whole app found, end to end (0.67.1)
CI / check (push) Successful in 4m9s
CI / build-and-push (push) Failing after 16m10s
A pass over every feature against a real Docker daemon, then clicking
through the UI as admin and read-only user, on a desktop and a phone.

Security. /ws/deploy ran compose up for the read-only role; it is
admin-only now. A container's inspect view, a stack's compose file
(POSTGRES_PASSWORD: hunter2) and the viewer's .env tab no longer show
credentials to that role. Git branch and URL values that git would read
as options (-x, ext::) are refused. SFTP host keys are checked: pinned
on first use or entered by hand (key line or SHA256 fingerprint), a
different key is refused before the password is sent. A backup name
from the API can no longer reach outside the destination folder. The
file browser refuses to move or rename a browse root, as delete did.

Docker lifecycle. Closing the deploy console released the stack lock
while compose was still running and logged rc=None as success; deploys
and updates are now background runs that hold the lock to the end,
report the real exit code and can be re-attached to. Every closed log
view left a `compose logs -f` behind. A dozen open tabs exhausted the
default thread pool and hung the API; event and exec streams have their
own threads. Auto-update and the update endpoint ignored failed pulls.
A deleted git stack was recreated and started by the next webhook or
poll; deleting a stack now drops its git source, auto-update policy and
schedule entries. Deleting no longer walks into mounted shares, is
refused while a backup runs, and removes the containers even when the
compose file is broken. Stack folders compose has to normalise
(Nextcloud -> nextcloud) work everywhere and group correctly on the
volumes, networks and images pages.

Detection. Host port conflicts read the container's own network
namespace; /proc/1/net is read instead. Images referenced as docker.io/…
were never update-checked; local builds no longer report "needs
credentials". NVIDIA GPUs are read from the driver's procfs. The backup
dialog's "path mismatch" warning came from mountinfo's filesystem-relative
root and fired for any stacks folder on its own disk, ZFS dataset or
btrfs subvolume; the host path now comes from Docker's mounts.

Backups and destinations. The download ignored the folder/volume
selection (axios sent binds[]). Restoring under a new name kept the old
one. A report header past nginx's buffer turned a good backup into a
502. SFTP/S3 errors were bare 500s and "Test" did not try to write.
A destination used by a schedule could be deleted. Backup file names
are in the schedule's or the browser's time zone instead of UTC.

Editor and converter. `docker run` conversion produced invalid compose
for --network host, named volumes and -dit; --gpus and --mount were
dropped; a multi-line command pasted into the one-line field became
`image: ' '` (now a textarea, and flattened continuations are read).
Validation failed for env_file stacks. The editor overwrote unsaved
edits on every container event, lost them on a stray click (now asks),
threw when closing the diff, and its helper panel listed the starter
service "app" for every existing stack. Monaco is bundled instead of
loaded from jsdelivr, so the editor works offline (and monaco-editor
0.57 drops the dompurify advisory; axios and react-router updated).

UI. All times were two hours off: offset-less UTC from SQLite was read
as local time, and "next run" always said "just now". A token refresh
mid-session duplicated deploys, reset terminals and doubled logs. A
backend restart (update, self-update) signed everyone out on reload; the
page now waits for it. The editor was unusable on a phone, tables and
tabs overflowed, dialog text was black on the dark theme, and the file
browser opened on an error with the default roots. Read-only users get
"Admin only" pages instead of 403s; in-use networks offer no delete;
"Scan all" covers StackPilot's own stacks only and runs in the
background instead of timing out at 600s.

Deployment. nginx passes the port in Host (the OIDC redirect URI lost
:5009) and keeps an outer proxy's X-Forwarded-Proto. The frontend waits
for a healthy backend (healthcheck probes every 2s while starting). The
frontend image builds with npm ci. Secrets are 0644 in a 0700 folder so
non-root containers can read them; existing files are fixed at startup.
The test suite no longer starts real containers on a host with Docker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fx7gcuy8d6nPd95g43rGHu
2026-10-15 13:51:26 +02:00
menzeljandClaude Opus 5 54c835b032 Close the read-side privilege escalation and fix proxy-aware IPs (0.44.0)
CI / build-and-push (push) Successful in 3m53s
F1 — Any authenticated user could read any file the backend could see.
/api/files/read and /download hung on get_current_user, and the sandbox that
should have caught that was open by default: ALLOWED_BROWSE_ROOTS contained
"/", for which _is_allowed() waves through every path. So the `user` role could
download stackpilot.db (password hashes, agent tokens, backup credentials),
every stack's .env and every .secrets/* file — with no audit trail, because
only mutations were logged.

Implementing that turned up three more doors into the same room, all fixed
here since closing only the first would have made the fix cosmetic:
GET /api/stacks/{id} handed the .env to any user, /export tarred the whole
stack dir including .secrets/*, and both the agent file proxies and
/api/agents/{id}/stacks/{id} repeated the leak for every remote host. All 24
filesystem-touching routes are now admin-only; reads and downloads are audited
(listing is not — the Files page polls it). DATA_DIR is refused outright, since
the API deliberately masks agent tokens and destination secrets and the browser
would otherwise be the way around that. "/" is out of the default browse roots.

F2 — Backup destination credentials were plaintext JSON in the DB, which is
what made F1 worth exploiting. They are now Fernet-encrypted at rest behind
parse_config/dump_config, with existing rows migrated at startup.

This needed a prerequisite from F6: the key is derived from SECRET_KEY, which
was regenerated on every boot when unset. Encrypting against a key that changes
per restart would be worse than plaintext, so an auto-generated SECRET_KEY is
now persisted to ${DATA_DIR}/secret_key at mode 0600. Sessions surviving a
restart is a welcome side effect.

F3 — /api/audit is admin-only. Also hidden from the dashboard and the nav for
non-admins, so nobody polls into a 403.

F4 — uvicorn now runs with --proxy-headers, so nginx's X-Forwarded-For is
honoured. Without it request.client.host was the frontend container's IP for
every request, which made the login rate limit global instead of per-IP (10
failures locked out everyone) and filled the audit log's IP column with one
useless value.

Verified: encrypt/decrypt round-trip incl. plaintext passthrough, idempotent
re-encryption and wrong-key handling; sandbox denial for DATA_DIR, traversal
into it, and paths outside the roots, with the allowed roots still reachable.
Both against stubbed settings — there is no Docker here, so nothing was run
end to end.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
2026-08-31 13:01:53 +02:00
menzeljandClaude Opus 4.8 5ac9f15de4 0.37.2: stream folder zip-downloads to fix 504 on large folders
A big folder hit a 504 Gateway Timeout: the zip was built into a temp
file *before* any response was sent, so for large folders the backend
stayed silent past nginx's proxy_read_timeout.

Now the zip is streamed as it's built, end to end:
- file_service.open_archive() returns (filename, byte iterator); _iter_zip
  walks the dir and yields zip bytes incrementally via a small drain
  buffer, writing each file in 1 MiB chunks (bounded memory, valid CRCs).
  Same hardening as before — only real regular files; FIFOs/sockets/
  devices/symlinks skipped without open(); per-file read errors skipped.
- /api/files/download and /agent/files/download return a StreamingResponse
  (no temp file). The agent proxy streams the agent response straight
  through (agent_service.stream_download), pulling the first chunk eagerly
  so an offline/bad-token agent still yields a clean status before 200.
- Files page: streamed downloads have no Content-Length, so the progress
  bar shows the running downloaded byte count ("Downloading … 12.3 MB")
  instead of a percentage, after the initial "Preparing …".

Verified end to end via TestClient (200, application/zip, valid zip,
2 MiB file intact, FIFO skipped, no hang).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 20:28:49 +00:00
menzeljandClaude Opus 4.8 f1782eca0e 0.37.0: download whole folders (recursive) as a .zip from the file browser
The file browser/editor could only download individual files. Add a
recursive directory download that streams the folder as a zip archive,
on the local host and on every remote agent.

- file_service.archive_dir(): zip a directory recursively into a temp
  file, preserving the folder name as the archive root and empty
  subdirectories; symlinks are skipped (no sandbox escape / loops).
- /api/files/download and /agent/files/download branch on directories
  and return application/zip, cleaning up the temp file afterwards.
- Files page: show the download button for folders too (as <name>.zip).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-21 19:54:50 +00:00
menzeljandClaude Opus 4.8 25bba1cf2c File browser: folder upload + copy/move (0.13.0)
Folder upload: the Files page gained an "Upload folder" picker
(webkitdirectory); each file is sent with its webkitRelativePath and the
backend recreates the directory tree. upload_target now accepts an optional
rel_path, creating intermediate dirs (mkdir -p) inside the sandbox with each
component validated against traversal.

Copy/move: new file_service.copy/move + POST /api/files/{copy,move}
(admin, audit-logged). The UI adds per-row copy/cut actions, a clipboard bar
to paste into the current directory, and an overwrite prompt on conflict.
Both refuse to move/copy a folder into itself or its own subtree and are
sandbox-checked on source and destination.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 10:53:57 +00:00
menzeljandClaude Opus 4.8 e3313fb4ac Phase 12: file browser (0.12.0)
Add a full host filesystem browser reachable from the sidebar (/files):
breadcrumb navigation, browse-root chips, show-hidden toggle, and a table
with size/permissions/mtime. Text files open in a Monaco editor (language by
extension); binary/oversized files fall back to download. Admins can create
folders/files, rename, delete (recursive for dirs), upload, and save edits;
download is available to all users. Every mutation is audit-logged.

Backend: new services/file_service.py reuses device_service's sandbox helpers
(confined to ALLOWED_BROWSE_ROOTS, mapped via HOST_ROOT_PREFIX) and rejects
path traversal and deleting a browse root. routers/files.py exposes
/api/files/{list,read,download,write,mkdir,touch,rename,upload,DELETE}
(reads: any user; mutations: admin). device_service.browse entries gained
mtime + symlink (non-breaking).

Deployment: ALLOWED_BROWSE_ROOTS + HOST_ROOT_PREFIX are now env-wired in
docker-compose.yml and .env.example, with a commented /:/host_root mount to
browse/manage the real host filesystem.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 10:33:38 +00:00