Back up several stacks, or all of them, from one schedule (0.65.0)
A schedule covered exactly one stack, so backing up a host meant one
schedule per stack, each edited separately, and new stacks were only
backed up once someone remembered to add a schedule.
A schedule now takes a list of stacks, or all_stacks, which includes
stacks added later. They are backed up one after another, each with its
own file, upload, retention and status, so one failing stack does not
cost the others their backup:
- stack_status records each stack's result; last_status summarises
("error: 2 of 5 stacks - a: ...; c: ...") and one notification per run
names the stacks with a problem. The dashboard names them too and shows
an incomplete backup as a warning rather than an error.
- A busy stack (deploy, update) is retried after the others, up to three
times two minutes apart, replacing the 10-minute postponement of the
whole schedule.
- Run now starts the run in the background (202) instead of holding the
request, which a multi-stack run would outlast; the list reports
running and polls, and the row toasts the outcome. A schedule never
runs twice at once.
- Settings: "All stacks" or a checklist with select all/none; stacks can
be changed when editing; deleted stacks stay listed to be unticked.
Schedules from before keep working without a data migration: stack_ids,
all_stacks and stack_status are added with defaults on startup and a row
with only stack_id reads as that one stack. The API still accepts
stack_id and reports it for single-stack schedules.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3c187e2e80
commit
8a0d673798
11 files changed
+754
-220
No files matched your search
@@ -15,7 +15,7 @@ from datetime import datetime, timedelta, timezone
|
||||
from sqlmodel import Session, select
|
||||
|
||||
from docker_client import DockerError, get_client, safe_call
|
||||
from models.backup_schedule import BackupSchedule
|
||||
from models.backup_schedule import BackupSchedule, describe_stacks, status_of
|
||||
from services import compose_service, update_service
|
||||
|
||||
COMPOSE_LABEL = compose_service.COMPOSE_LABEL
|
||||
@@ -191,13 +191,20 @@ def _backup_attention(session: Session) -> list[dict]:
|
||||
for sch in session.exec(select(BackupSchedule).where(BackupSchedule.enabled == True)).all(): # noqa: E712
|
||||
host = "local"
|
||||
status = (sch.last_status or "").lower()
|
||||
# Name the stacks that actually had a problem, not the whole schedule.
|
||||
failing = sorted(
|
||||
sid for sid, entry in status_of(sch).items()
|
||||
if str((entry or {}).get("status", "")) != "ok"
|
||||
)
|
||||
what = ", ".join(failing) if failing else describe_stacks(sch)
|
||||
if status and not status.startswith("ok"):
|
||||
items.append(_attn("error", "backup_failed", host,
|
||||
f"Backup of {sch.stack_id} failed", sch.last_status or "",
|
||||
"/settings"))
|
||||
incomplete = status.startswith("warning")
|
||||
items.append(_attn("warn" if incomplete else "error", "backup_failed", host,
|
||||
f"Backup of {what} {'is incomplete' if incomplete else 'failed'}",
|
||||
sch.last_status or "", "/settings"))
|
||||
elif sch.next_run and _aware(sch.next_run) < now - timedelta(hours=1):
|
||||
items.append(_attn("warn", "backup_overdue", host,
|
||||
f"Backup of {sch.stack_id} is overdue",
|
||||
f"Backup of {describe_stacks(sch)} is overdue",
|
||||
"Scheduled run did not happen.", "/settings"))
|
||||
return items
|
||||
|
||||
|
||||
Reference in new issue
Block a user