Let a stuck backup be stopped instead of stranding the stack (0.67.0)
CI / check (push) Successful in 13m33s
CI / build-and-push (push) Successful in 1m51s

A backup holds the stack lock for up to twelve hours — a large volume
legitimately takes that long — and stops the stack's containers for a
consistent snapshot. One that died without unwinding therefore left the
stack stopped and refusing every start, stop, restart and deploy with
"Stack 'npmplus' is busy: backup in progress", with nothing to click
until the TTL ran out.

Backups and restores now register themselves in stack_job_service while
they run, with their phase and the helper containers they stream
through. Cancelling sets a flag and removes those containers, which is
what actually ends the long read: the backup then unwinds through its
own finally — the half-written archive goes, the containers it stopped
are started again, the lock is released. Checks sit between two files
and two volumes, never inside one. A restore is not cancellable at all;
it is halfway through writing a stack's data back.

POST /api/stacks/{id}/unlock does that and waits for it. A backup lock
with no job behind it is a leftover, so it is released on one click; a
deploy lock takes an explicit force, because compose keeps running past
its request. GET /api/stacks/{id}/lock and the stack detail response say
what holds the stack and since when, and the stack page turns that into
"Stop backup" / "Release lock".

Startup now clears every lock rather than only expired ones: after a
restart nothing of ours is in flight, the same reasoning the leftover
helper containers already follow.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
menzeljandClaude Opus 5 committed 2026-09-21 13:39:17 +02:00
1 parent 052a85067b
commit 270ddda51d
19 files changed
+1018 -22

No files matched your search

@@ -0,0 +1,139 @@
import { useState } from "react";
import { useMutation, useQueryClient } from "@tanstack/react-query";
import { Loader2, Lock, Unlock } from "lucide-react";
import { toast } from "sonner";
import { Button } from "@/components/ui";
import { ConfirmDialog } from "@/components/ui/ConfirmDialog";
import { stacksApi } from "@/api/stacks";
import { apiErrorMessage } from "@/api/client";
import type { StackLock } from "@/types";
/** "14 minutes", "3 hours" — how long this has been going on. */
function since(seconds: number): string {
if (seconds < 90) return `${Math.max(seconds, 0)} seconds`;
const minutes = Math.round(seconds / 60);
if (minutes < 90) return `${minutes} minutes`;
return `${Math.round(minutes / 60)} hours`;
}
const LABEL: Record<string, string> = {
backup: "A backup",
restore: "A restore",
update: "An update",
start: "A deploy",
"git-deploy": "A Git deploy",
"auto-update": "An automatic update",
};
function describe(lock: StackLock): string {
return LABEL[lock.action] ?? `A ${lock.action}`;
}
/**
* Shown while an operation holds the stack — and, more to the point, offers a
* way out of it.
*
* One operation runs per stack at a time, and a backup may hold that lock for
* twelve hours (a large volume legitimately takes that long). So a backup that
* dies without unwinding leaves every start, stop and restart answering
* "Stack 'x' is busy: backup in progress" with nothing to click. This is the
* button that was missing: stop the backup (it throws its half-written archive
* away and starts the containers it stopped), or — when there is nothing left
* running to stop — drop the lock.
*/
export function StackLockBanner({
stackId,
lock,
}: {
stackId: string;
lock: StackLock;
}) {
const qc = useQueryClient();
const [confirmForce, setConfirmForce] = useState(false);
const unlock = useMutation({
mutationFn: (force: boolean) => stacksApi.unlock(stackId, force),
onSuccess: (result) => {
toast.success(
result.cancelled
? `${describe(lock)} of ${stackId} was stopped`
: `The lock on ${stackId} was released`
);
setConfirmForce(false);
qc.invalidateQueries({ queryKey: ["stack", stackId] });
qc.invalidateQueries({ queryKey: ["stacks"] });
},
onError: (err) => {
toast.error(apiErrorMessage(err));
// It would not stop: offer the blunt instrument rather than leaving the
// user with an error and no next step.
setConfirmForce(true);
},
});
const running = unlock.isPending;
const canStop = lock.cancellable && !lock.cancelling;
return (
<div className="rounded-card border border-amber-300 bg-amber-50 px-3 py-2 text-sm text-amber-900 dark:border-amber-900/60 dark:bg-amber-950/40 dark:text-amber-200">
<div className="flex flex-wrap items-center gap-2">
{lock.stale ? (
<Lock className="h-4 w-4 shrink-0" />
) : (
<Loader2 className="h-4 w-4 shrink-0 animate-spin" />
)}
<span className="flex-1">
{lock.stale ? (
<>
{describe(lock)} left this stack locked, but it is no longer
running. Until the lock goes, start, stop and restart are refused.
</>
) : (
<>
{describe(lock)} is running — started {since(lock.held_for)} ago
{lock.phase ? ` (${lock.phase})` : ""}. The stack cannot be
started, stopped or deployed while it does.
{lock.cancelling && " Stopping it…"}
</>
)}
</span>
{canStop && (
<Button
variant="outline"
onClick={() => unlock.mutate(false)}
loading={running}
>
Stop {lock.action}
</Button>
)}
<Button
variant={canStop ? "ghost" : "outline"}
onClick={() =>
lock.stale ? unlock.mutate(false) : setConfirmForce(true)
}
loading={running && !canStop}
>
<Unlock className="h-4 w-4" /> Release lock
</Button>
</div>
{confirmForce && (
<ConfirmDialog
title={`Release the lock on ${stackId}?`}
message={
`This only removes the guard — it does not stop ${describe(lock)
.charAt(0)
.toLowerCase() + describe(lock).slice(1)}. ` +
"If it is in fact still running, it and your next deploy can then " +
"work on the same stack at the same time."
}
confirmLabel="Release anyway"
danger
busy={running}
onConfirm={() => unlock.mutate(true)}
onCancel={() => setConfirmForce(false)}
/>
)}
</div>
);
}