A review of the backup path found several ways a backup was not one, and a
restore could destroy data it did not contain.
Data loss:
- Restore with overwrite rmtree'd the whole stack directory, including
folders the backup had left out (over 2 GiB, deselected, an NFS share
mounted inside). It now clears only what the backup holds, keeps the
excluded paths (recorded in the manifest) and never crosses into another
filesystem.
- The stack was only stopped when named volumes were selected, so data
folders inside the stack directory were archived while in use. Any
selected data now stops it.
- Retention matched by prefix: pruning "app" deleted "app-db"'s backups.
It now matches backup-<id>-(full|config)-<date>.tar.gz exactly.
- A restore under a new name wrote into the original's volumes when they
had an explicit name, and into its folders outside the stack directory.
Both are skipped now; a volume owned by another stack or mounted by a
running container is never overwritten.
- A truncated archive was only noticed after the old stack was gone. The
whole archive is verified first, and the stack directory is unpacked
next to the old one and swapped in.
Remote storage:
- Bind folders on a mounted share (fstab/autofs), bind-type volumes
pointing at one and shares mounted inside the stack folder are detected
by filesystem type and not selected; a folder du cannot measure in 20s
is no longer selected by default either.
Reliability:
- After a backup exactly the containers that ran are started again with
compose start (falling back to the Docker API), instead of up -d, which
started services stopped on purpose and applied undeployed edits. A
stack that cannot be stopped is not backed up.
- Backup and restore hold the stack lock; a scheduled backup that finds
the stack busy retries after 10 minutes. Restore stops the stack while
replacing its data and brings it back up.
- external: true volumes are backed up; anonymous ones are listed, not
selected. Restored volumes get their original driver and options.
- A backup with uncaptured data or a stack that did not restart is
reported as incomplete: in the dialog, and for schedules as a warning
with a notification and without rotating older backups out.
- Archive writing and restore run off the event loop; downloads delete
their temp file; helper containers use the long-timeout client; S3
listing is paginated; a file changing mid-read no longer corrupts the
archive.
Manifest format 3 records compose_binds/excluded and volume
driver/options/external; format 1/2 backups still restore.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A stack's real state lives in its bind-mounted config directories, and those
were never captured: the backup only tarred the stack folder as this container
sees it. When STACKS_HOST_DIR differs from the container's STACKS_DIR, compose
resolves ./config against the container path and the daemon creates it at that
path on the *host* — invisible here, so the archive held little more than
compose.yaml and .env.
New services/stack_assets_service.py inventories a stack's data (bind sources
merged from container mounts + the compose file, named volumes) and does all
data I/O through a throwaway helper container, i.e. by host path, so unseen
directories are captured anyway. It also detects the host/container stacks-path
mismatch and reports it.
- manifest v2: full inventory, per-asset capture result, skip reasons (v1 still
restores)
- NFS/CIFS-backed volumes are skipped by default and never wiped on restore
- deselected data inside the stack folder no longer sneaks in via compose/
- volume/bind archives stream through temp files instead of RAM
- restore preserves mode, ownership, mtime and symlinks, and writes bind folders
back to their host paths (rewritten when the stack is renamed)
- backup dialog shows the inventory with sizes and per-item checkboxes; restore
gained a "restore bind folders" toggle
- new GET /api/stacks/{id}/backup/inventory (+ agent + proxy), backup endpoints
take include_binds/binds/volumes, restore takes restore_binds
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- backup_filename() crashed with NameError (bare now()) since 0.8.0 —
broke every scheduled backup at the upload step, agent backup download
and the central remote-backup/push endpoints. The local manual path
worked only because the router had its own copy (now an alias).
- restore: the manifest stack_id from an uploaded backup is now slugified
too — a crafted '../../...' id could previously escape STACKS_DIR.
- create_backup no longer starts a previously-stopped stack (stop/restart
only when the stack was actually running).
- overwrite-restore wipes the existing volume contents before extracting,
so files created since the backup no longer survive underneath it.
Verified end-to-end: full/config backup contents (compose, .env, .secrets,
bind dirs, extras, volume tars), delete→restore round-trip incl. volume
data, rename restore with volume re-prefixing, 409 conflict + overwrite,
traversal guard, scheduled run + retention prune + restore-from against
real MinIO, and the complete remote-agent cycle (download/push/restore).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
