Commit Graph
3 Commits
Author SHA1 Message Date
menzeljandClaude Opus 5 6c62803dc7 Stop leaking helper containers that fail to start (0.66.1)
CI / check (push) Successful in 13m17s
CI / build-and-push (push) Successful in 2m20s
docker-py's containers.run(remove=True) only removes a container after it
ran. A helper whose start failed - an NFS destination refusing the mount -
stayed behind in "Created", holding the destination's volume. Every Test
click and scheduled backup added one (a user found 18), the volume could
not be deleted even with everything stopped, and changed mount options
never took effect because the volume could not be recreated with them.

- docker_client.run_container: create, start, wait, collect output and
  always remove, also when the start fails; pulls a missing image. Used
  for NFS destination I/O, bind-path inspection, the volume wipe before a
  restore and the Trivy scanner.
- Every helper container is labelled stackpilot.helper; on startup the
  stopped ones are removed, together with anything left on an NFS
  destination volume by older versions.
- When an NFS destination's options change, stopped containers holding
  its volume are removed before it is recreated; a running one is now a
  clear error instead of silently keeping the old options.
- A refused mount explains itself: NFS enabled and reachable from the
  Docker host, NFSv3 needing rpcbind and mountd, and nfsvers=4 needing
  only port 2049.
- The scanner waits for Trivy's own 10-minute timeout instead of giving
  up after the default client's 60 seconds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 14:14:32 +02:00
menzeljandClaude Opus 5 8a0d673798 Back up several stacks, or all of them, from one schedule (0.65.0)
CI / check (push) Successful in 12m46s
CI / build-and-push (push) Successful in 2m4s
A schedule covered exactly one stack, so backing up a host meant one
schedule per stack, each edited separately, and new stacks were only
backed up once someone remembered to add a schedule.

A schedule now takes a list of stacks, or all_stacks, which includes
stacks added later. They are backed up one after another, each with its
own file, upload, retention and status, so one failing stack does not
cost the others their backup:

- stack_status records each stack's result; last_status summarises
  ("error: 2 of 5 stacks - a: ...; c: ...") and one notification per run
  names the stacks with a problem. The dashboard names them too and shows
  an incomplete backup as a warning rather than an error.
- A busy stack (deploy, update) is retried after the others, up to three
  times two minutes apart, replacing the 10-minute postponement of the
  whole schedule.
- Run now starts the run in the background (202) instead of holding the
  request, which a multi-stack run would outlast; the list reports
  running and polls, and the row toasts the outcome. A schedule never
  runs twice at once.
- Settings: "All stacks" or a checklist with select all/none; stacks can
  be changed when editing; deleted stacks stay listed to be unticked.

Schedules from before keep working without a data migration: stack_ids,
all_stacks and stack_status are added with defaults on startup and a row
with only stack_id reads as that one stack. The API still accepts
stack_id and reports it for single-stack schedules.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 12:44:28 +02:00
menzeljandClaude Opus 5 8f0a01c4f5 Make backup and restore reliable, and keep NFS out of them (0.64.0)
CI / check (push) Successful in 14m50s
CI / build-and-push (push) Successful in 2m20s
A review of the backup path found several ways a backup was not one, and a
restore could destroy data it did not contain.

Data loss:
- Restore with overwrite rmtree'd the whole stack directory, including
  folders the backup had left out (over 2 GiB, deselected, an NFS share
  mounted inside). It now clears only what the backup holds, keeps the
  excluded paths (recorded in the manifest) and never crosses into another
  filesystem.
- The stack was only stopped when named volumes were selected, so data
  folders inside the stack directory were archived while in use. Any
  selected data now stops it.
- Retention matched by prefix: pruning "app" deleted "app-db"'s backups.
  It now matches backup-<id>-(full|config)-<date>.tar.gz exactly.
- A restore under a new name wrote into the original's volumes when they
  had an explicit name, and into its folders outside the stack directory.
  Both are skipped now; a volume owned by another stack or mounted by a
  running container is never overwritten.
- A truncated archive was only noticed after the old stack was gone. The
  whole archive is verified first, and the stack directory is unpacked
  next to the old one and swapped in.

Remote storage:
- Bind folders on a mounted share (fstab/autofs), bind-type volumes
  pointing at one and shares mounted inside the stack folder are detected
  by filesystem type and not selected; a folder du cannot measure in 20s
  is no longer selected by default either.

Reliability:
- After a backup exactly the containers that ran are started again with
  compose start (falling back to the Docker API), instead of up -d, which
  started services stopped on purpose and applied undeployed edits. A
  stack that cannot be stopped is not backed up.
- Backup and restore hold the stack lock; a scheduled backup that finds
  the stack busy retries after 10 minutes. Restore stops the stack while
  replacing its data and brings it back up.
- external: true volumes are backed up; anonymous ones are listed, not
  selected. Restored volumes get their original driver and options.
- A backup with uncaptured data or a stack that did not restart is
  reported as incomplete: in the dialog, and for schedules as a warning
  with a notification and without rotating older backups out.
- Archive writing and restore run off the event loop; downloads delete
  their temp file; helper containers use the long-timeout client; S3
  listing is paginated; a file changing mid-read no longer corrupts the
  archive.

Manifest format 3 records compose_binds/excluded and volume
driver/options/external; format 1/2 backups still restore.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-18 11:33:54 +02:00