Fix what a review of the whole app found, end to end (0.67.1)

A pass over every feature against a real Docker daemon, then clicking
through the UI as admin and read-only user, on a desktop and a phone.

Security. /ws/deploy ran compose up for the read-only role; it is
admin-only now. A container's inspect view, a stack's compose file
(POSTGRES_PASSWORD: hunter2) and the viewer's .env tab no longer show
credentials to that role. Git branch and URL values that git would read
as options (-x, ext::) are refused. SFTP host keys are checked: pinned
on first use or entered by hand (key line or SHA256 fingerprint), a
different key is refused before the password is sent. A backup name
from the API can no longer reach outside the destination folder. The
file browser refuses to move or rename a browse root, as delete did.

Docker lifecycle. Closing the deploy console released the stack lock
while compose was still running and logged rc=None as success; deploys
and updates are now background runs that hold the lock to the end,
report the real exit code and can be re-attached to. Every closed log
view left a `compose logs -f` behind. A dozen open tabs exhausted the
default thread pool and hung the API; event and exec streams have their
own threads. Auto-update and the update endpoint ignored failed pulls.
A deleted git stack was recreated and started by the next webhook or
poll; deleting a stack now drops its git source, auto-update policy and
schedule entries. Deleting no longer walks into mounted shares, is
refused while a backup runs, and removes the containers even when the
compose file is broken. Stack folders compose has to normalise
(Nextcloud -> nextcloud) work everywhere and group correctly on the
volumes, networks and images pages.

Detection. Host port conflicts read the container's own network
namespace; /proc/1/net is read instead. Images referenced as docker.io/…
were never update-checked; local builds no longer report "needs
credentials". NVIDIA GPUs are read from the driver's procfs. The backup
dialog's "path mismatch" warning came from mountinfo's filesystem-relative
root and fired for any stacks folder on its own disk, ZFS dataset or
btrfs subvolume; the host path now comes from Docker's mounts.

Backups and destinations. The download ignored the folder/volume
selection (axios sent binds[]). Restoring under a new name kept the old
one. A report header past nginx's buffer turned a good backup into a
502. SFTP/S3 errors were bare 500s and "Test" did not try to write.
A destination used by a schedule could be deleted. Backup file names
are in the schedule's or the browser's time zone instead of UTC.

Editor and converter. `docker run` conversion produced invalid compose
for --network host, named volumes and -dit; --gpus and --mount were
dropped; a multi-line command pasted into the one-line field became
`image: ' '` (now a textarea, and flattened continuations are read).
Validation failed for env_file stacks. The editor overwrote unsaved
edits on every container event, lost them on a stray click (now asks),
threw when closing the diff, and its helper panel listed the starter
service "app" for every existing stack. Monaco is bundled instead of
loaded from jsdelivr, so the editor works offline (and monaco-editor
0.57 drops the dompurify advisory; axios and react-router updated).

UI. All times were two hours off: offset-less UTC from SQLite was read
as local time, and "next run" always said "just now". A token refresh
mid-session duplicated deploys, reset terminals and doubled logs. A
backend restart (update, self-update) signed everyone out on reload; the
page now waits for it. The editor was unusable on a phone, tables and
tabs overflowed, dialog text was black on the dark theme, and the file
browser opened on an error with the default roots. Read-only users get
"Admin only" pages instead of 403s; in-use networks offer no delete;
"Scan all" covers StackPilot's own stacks only and runs in the
background instead of timing out at 600s.

Deployment. nginx passes the port in Host (the OIDC redirect URI lost
:5009) and keeps an outer proxy's X-Forwarded-Proto. The frontend waits
for a healthy backend (healthcheck probes every 2s while starting). The
frontend image builds with npm ci. Secrets are 0644 in a 0700 folder so
non-root containers can read them; existing files are fixed at startup.
The test suite no longer starts real containers on a host with Docker.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fx7gcuy8d6nPd95g43rGHu

backend/services/auto_update_service.py

@@ -103,8 +103,12 @@ async def _run_local(session: Session, policy: AutoUpdate) -> None:
             # this stack, skip and pick it up next cycle rather than racing
             # them over the same containers.
             with stack_lock_service.hold(session, stack_id, "auto-update", "auto-update"):
-                await compose_service.pull(stack_id)
-                await compose_service.up(stack_id)
+                # pull + up, stopping at a failed pull. The exit codes used to
+                # be ignored entirely: a registry outage or a failing `up` was
+                # recorded as "updated" and announced as a success.
+                result = await compose_service.update(stack_id)
+            if result.get("returncode") != 0:
+                raise RuntimeError(_failure(result))
         except stack_lock_service.StackBusy as exc:
             _record(session, policy, "skipped", str(exc))
             return
@@ -127,6 +131,13 @@ async def _run_local(session: Session, policy: AutoUpdate) -> None:
             )
 
 
+def _failure(result: dict) -> str:
+    """The last meaningful line of a failed compose run."""
+    text = (result.get("stderr") or result.get("stdout") or "").strip()
+    tail = text.splitlines()[-1] if text else ""
+    return f"compose exited with {result.get('returncode')}" + (f": {tail}" if tail else "")
+
+
 async def _safe_notify(event: str, title: str, message: str, session: Session) -> None:
     try:
         await notify_service.notify(event, title, message, session)

backend/services/backup_destination_service.py

@@ -76,24 +76,169 @@ def migrate_plaintext_configs(session) -> int:
 # --------------------------------------------------------------------------- #
 
 
-def _sftp_connect(cfg: dict):
+#: Seconds to wait for the SFTP server to accept the TCP connection.
+SFTP_CONNECT_TIMEOUT = 20
+
+
+def _check_file_name(name: str) -> None:
+    """A backup is addressed by its bare file name, never a path.
+
+    The name comes from the API (restore from / delete on a destination); with
+    ``../`` in it a delete reached files outside the backup folder — on SFTP
+    anything the account may write. NFS has always checked this.
+    """
+    if (
+        not name
+        or name in (".", "..")
+        or "/" in name
+        or "\\" in name
+        or "\0" in name
+    ):
+        raise DestinationError(f"Invalid backup file name '{name}'")
+
+
+def host_key_fingerprint(key) -> str:
+    """OpenSSH's form: ``SHA256:`` + unpadded base64 of the key's SHA-256."""
+    import base64
+    import hashlib
+
+    digest = hashlib.sha256(key.asbytes()).digest()
+    return "SHA256:" + base64.b64encode(digest).decode().rstrip("=")
+
+
+def fingerprint_of(pinned: str) -> str:
+    """The ``SHA256:`` fingerprint of a stored host key (as given, if it is
+    one already) — what the UI shows and ``ssh-keygen -lf`` prints."""
+    import base64
+    import hashlib
+
+    pinned = pinned.strip()
+    if pinned.startswith("SHA256:"):
+        return pinned
+    parts = pinned.split()
+    for i, part in enumerate(parts[:-1]):
+        if part.startswith(("ssh-", "ecdsa-", "sk-")):
+            try:
+                blob = base64.b64decode(parts[i + 1])
+            except ValueError:
+                break
+            return "SHA256:" + base64.b64encode(hashlib.sha256(blob).digest()).decode().rstrip("=")
+    return pinned
+
+
+def _pinned_type(pinned: str) -> Optional[str]:
+    """The key type of a pinned ``<type> <base64>`` line (a known_hosts line
+    with the host in front works too); None for a bare fingerprint."""
+    parts = pinned.split()
+    for part in parts[:-1]:
+        if part.startswith(("ssh-", "ecdsa-", "sk-")):
+            return part
+    return None
+
+
+#: What paramiko negotiates for a key of each type. An RSA key is offered
+#: under the SHA-2 signature names, not under its own.
+_HOST_KEY_ALGORITHMS = {
+    "ssh-rsa": ("rsa-sha2-512", "rsa-sha2-256", "ssh-rsa"),
+}
+
+
+def _host_key_matches(pinned: str, key) -> bool:
+    pinned = pinned.strip()
+    if pinned.startswith("SHA256:"):
+        return pinned.rstrip("=") == host_key_fingerprint(key)
+    parts = pinned.split()
+    for i, part in enumerate(parts[:-1]):
+        if part == key.get_name() and parts[i + 1] == key.get_base64():
+            return True
+    return False
+
+
+def _sftp_connect(cfg: dict, pin=None):
+    """Connect and authenticate, checking the server's host key.
+
+    Before 0.67.1 any host key was accepted, so whoever could answer on the
+    server's address (a spoofed DNS entry, a box that took its IP) received
+    the password and every backup. Now the key is pinned in the destination's
+    ``host_key``: entered by hand (a key line or its ``SHA256:`` fingerprint),
+    or — when empty — the one seen on the first connection, handed to ``pin``
+    to be stored. After that a different key refuses the connection.
+    """
+    import socket
+
     import paramiko
 
     host = cfg.get("host")
     if not host:
         raise DestinationError("SFTP host is required")
-    port = int(cfg.get("port") or 22)
-    username = cfg.get("username")
-    transport = paramiko.Transport((host, port))
     try:
+        port = int(cfg.get("port") or 22)
+    except (TypeError, ValueError) as exc:
+        raise DestinationError(f"Invalid SFTP port '{cfg.get('port')}'") from exc
+    username = cfg.get("username")
+    pinned = (cfg.get("host_key") or "").strip()
+    # Connected here, with a timeout, rather than by Transport((host, port)):
+    # that connects inside the constructor, outside any handler, so an
+    # unreachable host was a bare 500 instead of a message — and with no
+    # timeout it hung for the kernel's two minutes first.
+    try:
+        sock = socket.create_connection((host, port), timeout=SFTP_CONNECT_TIMEOUT)
+    except OSError as exc:
+        raise DestinationError(f"SFTP connection to {host}:{port} failed: {exc}") from exc
+    transport = paramiko.Transport(sock)
+    try:
+        key_type = _pinned_type(pinned) if pinned else None
+        if key_type:
+            # Ask for the pinned kind of key: a server has several, and the
+            # one negotiated by default need not be the one that was pinned.
+            wanted = _HOST_KEY_ALGORITHMS.get(key_type, (key_type,))
+            options = transport.get_security_options()
+            usable = tuple(k for k in wanted if k in options.key_types)
+            if usable:
+                options.key_types = usable
+        transport.start_client(timeout=SFTP_CONNECT_TIMEOUT)
+        server_key = transport.get_remote_server_key()
+        if pinned:
+            if not _host_key_matches(pinned, server_key):
+                raise DestinationError(
+                    f"SFTP host key of {host}:{port} does not match the stored one: the "
+                    f"server now presents {server_key.get_name()} "
+                    f"{host_key_fingerprint(server_key)}. If the server was reinstalled or "
+                    "its keys were rotated, clear the destination's host key to trust the "
+                    "new one — otherwise something else is answering on that address."
+                )
+        elif pin is not None:
+            pin(f"{server_key.get_name()} {server_key.get_base64()}")
         pkey = None
         if cfg.get("private_key"):
             pkey = _load_key(cfg["private_key"])
-        transport.connect(username=username, password=cfg.get("password") or None, pkey=pkey)
+        # What Transport.connect did: the key when there is one, else the password.
+        if pkey is not None:
+            transport.auth_publickey(username, pkey)
+        elif cfg.get("password"):
+            transport.auth_password(username, cfg["password"])
+        return paramiko.SFTPClient.from_transport(transport), transport
     except Exception as exc:  # noqa: BLE001
         transport.close()
+        if isinstance(exc, DestinationError):
+            raise
         raise DestinationError(f"SFTP connection failed: {exc}") from exc
-    return paramiko.SFTPClient.from_transport(transport), transport
+
+
+def _sftp_io(what: str):
+    """Turn what paramiko raises mid-transfer into a DestinationError."""
+    import contextlib
+
+    @contextlib.contextmanager
+    def wrapper():
+        try:
+            yield
+        except DestinationError:
+            raise
+        except Exception as exc:  # noqa: BLE001 - IOError, SSHException, EOFError …
+            raise DestinationError(f"SFTP {what} failed: {exc}") from exc
+
+    return wrapper()
 
 
 def _load_key(key_str: str):
@@ -120,22 +265,24 @@ def _sftp_makedirs(sftp, path: str) -> None:
             sftp.mkdir(cur)
 
 
-def _sftp_upload(cfg: dict, local_path: str, filename: str) -> str:
-    sftp, transport = _sftp_connect(cfg)
+def _sftp_upload(cfg: dict, local_path: str, filename: str, pin=None) -> str:
+    _check_file_name(filename)
+    sftp, transport = _sftp_connect(cfg, pin)
     try:
-        base = cfg.get("path") or "."
-        if base not in (".", ""):
-            _sftp_makedirs(sftp, base)
-        remote = posixpath.join(base, filename) if base not in (".", "") else filename
-        sftp.put(local_path, remote)
-        return remote
+        with _sftp_io("upload"):
+            base = cfg.get("path") or "."
+            if base not in (".", ""):
+                _sftp_makedirs(sftp, base)
+            remote = posixpath.join(base, filename) if base not in (".", "") else filename
+            sftp.put(local_path, remote)
+            return remote
     finally:
         sftp.close()
         transport.close()
 
 
-def _sftp_list(cfg: dict) -> list[dict]:
-    sftp, transport = _sftp_connect(cfg)
+def _sftp_list(cfg: dict, pin=None) -> list[dict]:
+    sftp, transport = _sftp_connect(cfg, pin)
     try:
         base = cfg.get("path") or "."
         out = []
@@ -155,23 +302,51 @@ def _sftp_list(cfg: dict) -> list[dict]:
         transport.close()
 
 
-def _sftp_download(cfg: dict, name: str, local_path: str) -> None:
-    sftp, transport = _sftp_connect(cfg)
+def _sftp_download(cfg: dict, name: str, local_path: str, pin=None) -> None:
+    _check_file_name(name)
+    sftp, transport = _sftp_connect(cfg, pin)
     try:
-        base = cfg.get("path") or "."
-        remote = posixpath.join(base, name) if base not in (".", "") else name
-        sftp.get(remote, local_path)
+        with _sftp_io(f"download of '{name}'"):
+            base = cfg.get("path") or "."
+            remote = posixpath.join(base, name) if base not in (".", "") else name
+            sftp.get(remote, local_path)
     finally:
         sftp.close()
         transport.close()
 
 
-def _sftp_delete(cfg: dict, name: str) -> None:
-    sftp, transport = _sftp_connect(cfg)
+def _sftp_delete(cfg: dict, name: str, pin=None) -> None:
+    _check_file_name(name)
+    sftp, transport = _sftp_connect(cfg, pin)
     try:
-        base = cfg.get("path") or "."
-        remote = posixpath.join(base, name) if base not in (".", "") else name
-        sftp.remove(remote)
+        with _sftp_io(f"delete of '{name}'"):
+            base = cfg.get("path") or "."
+            remote = posixpath.join(base, name) if base not in (".", "") else name
+            sftp.remove(remote)
+    finally:
+        sftp.close()
+        transport.close()
+
+
+_TEST_FILE = ".stackpilot-test"
+
+
+def _sftp_test(cfg: dict, pin=None) -> None:
+    """Connect, create the folder if needed, write and remove a file.
+
+    Listing alone said "OK" for a folder that does not exist or cannot be
+    written to — the first scheduled backup was then the one to find out.
+    """
+    sftp, transport = _sftp_connect(cfg, pin)
+    try:
+        with _sftp_io("write test"):
+            base = cfg.get("path") or "."
+            if base not in (".", ""):
+                _sftp_makedirs(sftp, base)
+            remote = posixpath.join(base, _TEST_FILE) if base not in (".", "") else _TEST_FILE
+            with sftp.open(remote, "w") as fh:
+                fh.write(b"ok")
+            sftp.remove(remote)
     finally:
         sftp.close()
         transport.close()
@@ -188,13 +363,16 @@ def _s3_client(cfg: dict):
     bucket = cfg.get("bucket")
     if not bucket:
         raise DestinationError("S3 bucket is required")
-    return boto3.client(
-        "s3",
-        endpoint_url=cfg.get("endpoint_url") or None,
-        region_name=cfg.get("region") or None,
-        aws_access_key_id=cfg.get("access_key") or None,
-        aws_secret_access_key=cfg.get("secret_key") or None,
-    )
+    try:
+        return boto3.client(
+            "s3",
+            endpoint_url=cfg.get("endpoint_url") or None,
+            region_name=cfg.get("region") or None,
+            aws_access_key_id=cfg.get("access_key") or None,
+            aws_secret_access_key=cfg.get("secret_key") or None,
+        )
+    except Exception as exc:  # noqa: BLE001 - e.g. a malformed endpoint URL
+        raise DestinationError(f"Invalid S3 settings: {exc}") from exc
 
 
 def _s3_key(cfg: dict, filename: str) -> str:
@@ -203,6 +381,7 @@ def _s3_key(cfg: dict, filename: str) -> str:
 
 
 def _s3_upload(cfg: dict, local_path: str, filename: str) -> str:
+    _check_file_name(filename)
     client = _s3_client(cfg)
     key = _s3_key(cfg, filename)
     try:
@@ -241,6 +420,7 @@ def _s3_list(cfg: dict) -> list[dict]:
 
 
 def _s3_download(cfg: dict, name: str, local_path: str) -> None:
+    _check_file_name(name)
     client = _s3_client(cfg)
     try:
         client.download_file(cfg["bucket"], _s3_key(cfg, name), local_path)
@@ -249,8 +429,24 @@ def _s3_download(cfg: dict, name: str, local_path: str) -> None:
 
 
 def _s3_delete(cfg: dict, name: str) -> None:
+    _check_file_name(name)
     client = _s3_client(cfg)
-    client.delete_object(Bucket=cfg["bucket"], Key=_s3_key(cfg, name))
+    try:
+        client.delete_object(Bucket=cfg["bucket"], Key=_s3_key(cfg, name))
+    except Exception as exc:  # noqa: BLE001
+        raise DestinationError(f"S3 delete failed: {exc}") from exc
+
+
+def _s3_test(cfg: dict) -> None:
+    """List, then write and delete a small object: proves the keys may write."""
+    _s3_list(cfg)
+    client = _s3_client(cfg)
+    key = _s3_key(cfg, _TEST_FILE)
+    try:
+        client.put_object(Bucket=cfg["bucket"], Key=key, Body=b"ok")
+        client.delete_object(Bucket=cfg["bucket"], Key=key)
+    except Exception as exc:  # noqa: BLE001
+        raise DestinationError(f"S3 write test failed: {exc}") from exc
 
 
 # --------------------------------------------------------------------------- #
@@ -354,6 +550,23 @@ def _nfs_volume(dest: BackupDestination, cfg: dict) -> str:
     return name
 
 
+def forget_nfs_volume(dest_id: int) -> None:
+    """Remove the Docker volume of a deleted NFS destination (best effort).
+
+    Left behind, it kept the old mount options under a name the next
+    destination with the same id would pick up again.
+    """
+    from docker_client import DockerError, get_client, safe_call
+
+    name = f"{_NFS_VOLUME_PREFIX}{dest_id}"
+    try:
+        client = get_client()
+        clear_nfs_leftovers(client, name)
+        safe_call(safe_call(client.volumes.get, name).remove)
+    except DockerError as exc:
+        logger.info("NFS volume %s not removed: %s", name, exc.detail or exc.error)
+
+
 def clear_nfs_leftovers(client, volume: Optional[str] = None) -> int:
     """Remove stopped containers holding an NFS destination volume.
 
@@ -566,10 +779,38 @@ def _nfs_test(dest: BackupDestination, cfg: dict) -> bool:
 # --------------------------------------------------------------------------- #
 
 
+def _pinner(dest: BackupDestination):
+    """Store the first host key an SFTP destination presents (trust on first
+    use). Written straight to the row — the callers are a scheduled run and a
+    "Test" click, and neither holds a session that will commit."""
+
+    def pin(host_key: str) -> None:
+        if dest.id is None:
+            return  # not saved yet: nothing to pin it to
+        from sqlmodel import Session
+
+        from database import engine
+
+        with Session(engine) as session:
+            row = session.get(BackupDestination, dest.id)
+            if row is None:
+                return
+            cfg = parse_config(row)
+            if (cfg.get("host_key") or "").strip():
+                return  # pinned meanwhile; the next connect compares against it
+            cfg["host_key"] = host_key
+            row.config = dump_config(cfg)
+            session.add(row)
+            session.commit()
+        logger.info("Destination '%s': pinned SFTP host key %s", dest.name, host_key.split()[0])
+
+    return pin
+
+
 def upload(dest: BackupDestination, local_path: str, filename: str) -> str:
     cfg = parse_config(dest)
     if dest.type == "sftp":
-        return _sftp_upload(cfg, local_path, filename)
+        return _sftp_upload(cfg, local_path, filename, _pinner(dest))
     if dest.type == "s3":
         return _s3_upload(cfg, local_path, filename)
     if dest.type == "nfs":
@@ -580,7 +821,7 @@ def upload(dest: BackupDestination, local_path: str, filename: str) -> str:
 def list_backups(dest: BackupDestination) -> list[dict]:
     cfg = parse_config(dest)
     if dest.type == "sftp":
-        return _sftp_list(cfg)
+        return _sftp_list(cfg, _pinner(dest))
     if dest.type == "s3":
         return _s3_list(cfg)
     if dest.type == "nfs":
@@ -591,7 +832,7 @@ def list_backups(dest: BackupDestination) -> list[dict]:
 def download(dest: BackupDestination, name: str, local_path: str) -> None:
     cfg = parse_config(dest)
     if dest.type == "sftp":
-        _sftp_download(cfg, name, local_path)
+        _sftp_download(cfg, name, local_path, _pinner(dest))
     elif dest.type == "s3":
         _s3_download(cfg, name, local_path)
     elif dest.type == "nfs":
@@ -603,7 +844,7 @@ def download(dest: BackupDestination, name: str, local_path: str) -> None:
 def delete(dest: BackupDestination, name: str) -> None:
     cfg = parse_config(dest)
     if dest.type == "sftp":
-        _sftp_delete(cfg, name)
+        _sftp_delete(cfg, name, _pinner(dest))
     elif dest.type == "s3":
         _s3_delete(cfg, name)
     elif dest.type == "nfs":
@@ -614,7 +855,13 @@ def delete(dest: BackupDestination, name: str) -> None:
 
 def test(dest: BackupDestination) -> bool:
     """Connectivity check — validates reachability, auth and write access."""
+    cfg = parse_config(dest)
     if dest.type == "nfs":
-        return _nfs_test(dest, parse_config(dest))
-    list_backups(dest)
-    return True
+        return _nfs_test(dest, cfg)
+    if dest.type == "sftp":
+        _sftp_test(cfg, _pinner(dest))
+        return True
+    if dest.type == "s3":
+        _s3_test(cfg)
+        return True
+    raise DestinationError(f"Unknown destination type '{dest.type}'")

backend/services/backup_service.py

@@ -104,8 +104,20 @@ def backup_basename(stack_id: str, prefix: Optional[str] = None) -> str:
     return f"backup-{prefix}-{stack_id}" if prefix else f"backup-{stack_id}"
 
 
-def backup_filename(stack_id: str, include_volumes: bool, prefix: Optional[str] = None) -> str:
-    date = compose_service.now().strftime("%Y%m%d-%H%M%S")
+def backup_filename(
+    stack_id: str,
+    include_volumes: bool,
+    prefix: Optional[str] = None,
+    tz: Optional[str] = None,
+) -> str:
+    """``backup-<stack>-<full|config>-YYYYmmdd-HHMMSS.tar.gz``, the time on the
+    wall clock of ``tz`` — the schedule's zone, or the browser's for a backup
+    made by hand. It used to be UTC, so a backup taken at 12:21 in Berlin was
+    called …-102154 and seemed to be from two hours earlier. The format is
+    unchanged, so retention still matches older files (it orders by mtime)."""
+    from services.schedule_service import zone
+
+    date = compose_service.now().astimezone(zone(tz)).strftime("%Y%m%d-%H%M%S")
     suffix = "full" if include_volumes else "config"
     return f"{backup_basename(stack_id, prefix)}-{suffix}-{date}.tar.gz"
 
@@ -907,6 +919,10 @@ def restore_backup(
     stack_id = _target_id(manifest, target_id)
     old_id = compose_service.slugify(manifest.get("stack_id") or stack_id)
     renamed = stack_id != old_id
+    # Volumes are named after the compose *project*, which differs from the id
+    # for a folder compose had to normalise ("my.app" -> "myapp_data").
+    old_project = compose_service.project_name(manifest.get("stack_id") or stack_id)
+    new_project = compose_service.project_name(stack_id)
     old_dir = manifest.get("stack_dir") or ""
 
     directory = compose_service.stack_dir(stack_id)
@@ -969,14 +985,14 @@ def restore_backup(
                 skipped.append({"kind": "volume", "name": full, "reason": "not requested"})
                 continue
             retargeted = False
-            if renamed and full.startswith(old_id + "_"):
-                full = stack_id + full[len(old_id) :]
+            if old_project != new_project and full.startswith(old_project + "_"):
+                full = new_project + full[len(old_project) :]
                 retargeted = True
             labels = dict(v.get("labels") or {})
             if not v.get("external"):
-                # Re-target compose's labels to the (possibly new) stack id.
-                labels[COMPOSE_PROJECT_LABEL] = stack_id
-            reason = _volume_conflict(full, stack_id, old_id, renamed and not retargeted)
+                # Re-target compose's labels to the (possibly new) project.
+                labels[COMPOSE_PROJECT_LABEL] = new_project
+            reason = _volume_conflict(full, new_project, old_id, renamed and not retargeted)
             if reason:
                 skipped.append({"kind": "volume", "name": full, "reason": reason})
                 continue
@@ -1012,7 +1028,10 @@ def restore_backup(
     skipped += manifest.get("skipped", [])
     return {
         "stack_id": stack_id,
-        "name": manifest.get("name", stack_id),
+        # Restored under a new id, the copy must not carry the original's
+        # display name — two stacks called the same in the list is how the
+        # wrong one gets stopped.
+        "name": stack_id if renamed else manifest.get("name", stack_id),
         "volumes_restored": volumes_restored,
         "binds_restored": binds_restored,
         "skipped": skipped,
@@ -1021,13 +1040,13 @@ def restore_backup(
     }
 
 
-def _volume_conflict(full: str, stack_id: str, old_id: str, shared_with_original: bool) -> Optional[str]:
+def _volume_conflict(full: str, project: str, old_id: str, shared_with_original: bool) -> Optional[str]:
     """Why a volume must not be overwritten, or None if it may be."""
     attrs = assets.volume_attrs(full)
     if attrs is None:
         return None  # created fresh
     owner = (attrs.get("Labels") or {}).get(COMPOSE_PROJECT_LABEL)
-    if owner and owner != stack_id:
+    if owner and owner != project:
         return f"belongs to stack '{owner}'; not overwritten"
     if shared_with_original:
         return (

backend/services/compose_run_service.py

@@ -0,0 +1,174 @@
+"""Streamed compose runs (deploy console, update progress) that outlive their socket.
+
+The deploy console and the stacks list's update bar watch ``compose up`` /
+``compose pull && up`` over a WebSocket. That run used to *be* the socket
+handler, which went wrong the moment somebody closed the modal or navigated
+away halfway through a deploy:
+
+* the stack lock was released in the handler's ``finally`` at once, while
+  compose was still running — so the next deploy, update or backup could start
+  on top of it;
+* nobody read compose's output any more, so a chatty pull filled the pipe and
+  compose blocked on it;
+* the audit entry and the "stack started" notification went out with
+  ``rc=None`` before anything had finished, as a success.
+
+Here the run is its own task. It holds the lock and drains the output until
+compose exits, then records the real exit code. A socket only *watches*: it can
+leave at any time, and one that connects while the same operation is already in
+flight on that stack is attached to it (backlog first, then live) instead of
+being turned away or starting a second compose.
+"""
+from __future__ import annotations
+
+import asyncio
+import collections
+import logging
+from collections.abc import AsyncIterator, Awaitable, Callable
+from dataclasses import dataclass, field
+from datetime import timedelta
+from typing import Optional
+
+from sqlmodel import Session
+
+from database import engine
+from services import stack_lock_service
+
+logger = logging.getLogger("stackpilot.compose_run")
+
+#: Lines kept for a socket that attaches to a run already in progress.
+BACKLOG_LINES = 2000
+
+#: A streamed run has no timeout of its own (a large pull legitimately takes a
+#: while), so its lock may live longer than a REST call's. The unlock endpoint
+#: is the way out of one that hangs.
+LOCK_TTL = timedelta(hours=2)
+
+#: (kind, payload): ("log", line) or ("done", returncode).
+Item = tuple[str, object]
+
+
+@dataclass
+class ComposeRun:
+    stack_id: str
+    action: str
+    owner: str
+    returncode: Optional[int] = None
+    finished: bool = False
+    backlog: collections.deque = field(
+        default_factory=lambda: collections.deque(maxlen=BACKLOG_LINES)
+    )
+    _subscribers: list[asyncio.Queue] = field(default_factory=list)
+
+    def subscribe(self) -> asyncio.Queue:
+        """A queue that gets the backlog, every later item, then ``None``."""
+        queue: asyncio.Queue = asyncio.Queue()
+        for line in self.backlog:
+            queue.put_nowait(("log", line))
+        if self.finished:
+            queue.put_nowait(("done", self.returncode))
+            queue.put_nowait(None)
+        else:
+            self._subscribers.append(queue)
+        return queue
+
+    def unsubscribe(self, queue: asyncio.Queue) -> None:
+        if queue in self._subscribers:
+            self._subscribers.remove(queue)
+
+    def _publish(self, item: Optional[Item]) -> None:
+        for queue in list(self._subscribers):
+            queue.put_nowait(item)
+
+
+_runs: dict[str, ComposeRun] = {}
+#: asyncio keeps only weak references to tasks; these must not be collected
+#: while compose is still running.
+_tasks: set[asyncio.Task] = set()
+
+
+class RunBusy(Exception):
+    """The stack is held by a different operation."""
+
+    def __init__(self, stack_id: str, action: str):
+        self.stack_id = stack_id
+        self.action = action
+        super().__init__(f"Stack '{stack_id}' is busy: {action} in progress")
+
+
+def get(stack_id: str) -> Optional[ComposeRun]:
+    return _runs.get(stack_id)
+
+
+def start(
+    stack_id: str,
+    action: str,
+    owner: str,
+    stream: Callable[[], AsyncIterator[Item]],
+    on_finish: Optional[Callable[[Optional[int]], Awaitable[None]]] = None,
+) -> tuple[ComposeRun, asyncio.Queue, bool]:
+    """Start ``stream`` under the stack lock, or attach to the same run in flight.
+
+    Returns ``(run, queue, attached)``. Raises :class:`RunBusy` when another
+    operation holds the stack. Must be called from the event loop; the queue is
+    subscribed before the task gets to run, so not a line is missed.
+    """
+    current = _runs.get(stack_id)
+    if current is not None and not current.finished:
+        if current.action != action:
+            raise RunBusy(stack_id, current.action)
+        return current, current.subscribe(), True
+
+    with Session(engine) as session:
+        try:
+            stack_lock_service.acquire(session, stack_id, action, owner, ttl=LOCK_TTL)
+        except stack_lock_service.StackBusy as exc:
+            raise RunBusy(stack_id, exc.action) from exc
+
+    run = ComposeRun(stack_id=stack_id, action=action, owner=owner)
+    _runs[stack_id] = run
+    queue = run.subscribe()
+    task = asyncio.create_task(_drive(run, stream, on_finish), name=f"compose-{action}-{stack_id}")
+    _tasks.add(task)
+    task.add_done_callback(_tasks.discard)
+    return run, queue, False
+
+
+async def _drive(
+    run: ComposeRun,
+    stream: Callable[[], AsyncIterator[Item]],
+    on_finish: Optional[Callable[[Optional[int]], Awaitable[None]]],
+) -> None:
+    rc: Optional[int] = None
+    try:
+        async for kind, payload in stream():
+            if kind == "log":
+                run.backlog.append(payload)
+                run._publish(("log", payload))
+            else:
+                rc = payload  # type: ignore[assignment]
+    except asyncio.CancelledError:
+        raise
+    except Exception as exc:  # noqa: BLE001 - reported to whoever is watching
+        logger.warning("compose %s of '%s' failed: %s", run.action, run.stack_id, exc)
+        line = f"error: {exc}"
+        run.backlog.append(line)
+        run._publish(("log", line))
+        rc = rc if rc is not None else -1
+    finally:
+        run.returncode = rc
+        run.finished = True
+        try:
+            with Session(engine) as session:
+                stack_lock_service.release(session, run.stack_id)
+        except Exception:  # noqa: BLE001 - never mask the outcome
+            logger.exception("Failed to release the lock on '%s'", run.stack_id)
+        run._publish(("done", rc))
+        run._publish(None)
+        if _runs.get(run.stack_id) is run:
+            del _runs[run.stack_id]
+    if on_finish is not None:
+        try:
+            await on_finish(rc)
+        except Exception:  # noqa: BLE001 - audit/notify are best-effort
+            logger.exception("Post-run hook for '%s' failed", run.stack_id)

backend/services/compose_service.py

@@ -9,6 +9,7 @@ import asyncio
 import os
 import re
 import shutil
+import stat
 import tempfile
 from dataclasses import dataclass, field
 from datetime import datetime, timezone
@@ -33,6 +34,42 @@ class StackFileError(Exception):
 # --------------------------------------------------------------------------- #
 
 
+_PROJECT_INVALID_RE = re.compile(r"[^a-z0-9_-]+")
+
+
+def project_name(stack_id: str) -> str:
+    """The compose project name for a stack.
+
+    The same normalisation compose applies to a directory name when it picks
+    the project name itself (lowercase, only ``a-z0-9_-``, no leading ``-``/``_``).
+    For every stack StackPilot creates that is the id unchanged. A folder found
+    on disk as ``Nextcloud`` or ``my.app`` used to be passed as ``-p Nextcloud``,
+    which compose refuses — so every action on such a stack failed, while
+    ``docker compose up`` in that folder worked.
+    """
+    return _PROJECT_INVALID_RE.sub("", stack_id.lower()).lstrip("_-") or "stack"
+
+
+def stack_ids_by_project(stack_ids) -> dict[str, str]:
+    """``{project: stack_id}``, to key per-project Docker data by stack id."""
+    return {project_name(sid): sid for sid in stack_ids}
+
+
+def stack_ids_by_project_in(session) -> dict[str, str]:
+    """:func:`stack_ids_by_project` for every stack in the database."""
+    from sqlmodel import select
+
+    from models.stack import Stack
+
+    return stack_ids_by_project(s.id for s in session.exec(select(Stack)).all())
+
+
+def owner_stack(project: Optional[str], ids: dict[str, str]) -> Optional[str]:
+    """The stack id behind a ``com.docker.compose.project`` label, else the
+    label itself (a project StackPilot does not manage)."""
+    return ids.get(project, project) if project else project
+
+
 def slugify(name: str) -> str:
     slug = re.sub(r"[^a-z0-9_-]+", "-", name.strip().lower())
     slug = re.sub(r"-{2,}", "-", slug).strip("-_")
@@ -85,6 +122,57 @@ def read_env(stack_id: str, override: Optional[str] = None) -> str:
         return fh.read()
 
 
+#: Name parts that mark a value as a credential (DB_PASSWORD, JWT_SECRET,
+#: CF_API_KEY, TS_AUTHKEY …), matched per ``_``/``.``/``-`` separated part so
+#: that KEYCLOAK_ADMIN or a "monkey" service stay readable.
+_SECRET_PARTS = {
+    "pass", "passwd", "password", "passphrase", "pwd", "secret", "token", "key",
+    "apikey", "authkey", "credential", "credentials", "private",
+}
+#: ``KEY: value`` and ``- KEY=value``, keeping indentation and quotes.
+_MAPPING_LINE = re.compile(r"^(\s*)([\"']?)([A-Za-z0-9_.-]+)\2(\s*:\s+)(.+?)\s*$")
+_LIST_LINE = re.compile(r"^(\s*-\s*)([\"']?)([A-Za-z0-9_.-]+)=(.*?)\2\s*$")
+#: ``${VAR}``, ``$VAR`` or ``${VAR:?message}`` — but not ``${VAR:-default}``,
+#: whose default is a value like any other.
+_ONLY_A_VARIABLE = re.compile(
+    r"^[\"']?\$(?:\{[A-Za-z_][A-Za-z0-9_]*(?::?\?[^}]*)?\}|[A-Za-z_][A-Za-z0-9_]*)[\"']?$"
+)
+MASK = "••••••"
+
+
+def _secret_name(name: str) -> bool:
+    return any(part in _SECRET_PARTS for part in re.split(r"[_.-]", name.lower()))
+
+
+def mask_secrets(compose: str) -> str:
+    """The compose file with credential values replaced by ``••••••``.
+
+    For the read-only role, which is not given the ``.env`` — but plenty of
+    compose files carry the password inline (``POSTGRES_PASSWORD: hunter2``),
+    and the Compose tab showed it. A reference to a variable (``${DB_PASS}``)
+    is not a secret and stays. Line-based on purpose: the file keeps its
+    layout and comments, and a file that does not parse is masked all the same.
+    """
+    out = []
+    for line in compose.splitlines(keepends=True):
+        body = line.rstrip("\r\n")
+        ending = line[len(body):]
+        m = _LIST_LINE.match(body)
+        if m and _secret_name(m.group(3)) and m.group(4) and not _ONLY_A_VARIABLE.match(m.group(4)):
+            body = f"{m.group(1)}{m.group(2)}{m.group(3)}={MASK}{m.group(2)}"
+        else:
+            m = _MAPPING_LINE.match(body)
+            if (
+                m
+                and _secret_name(m.group(3))
+                and not m.group(5).startswith(("|", ">", "{", "[", "&", "*", "#"))
+                and not _ONLY_A_VARIABLE.match(m.group(5))
+            ):
+                body = f"{m.group(1)}{m.group(2)}{m.group(3)}{m.group(2)}{m.group(4)}{MASK}"
+        out.append(body + ending)
+    return "".join(out)
+
+
 def write_compose(stack_id: str, content: str, override: Optional[str] = None) -> None:
     directory = stack_dir(stack_id, override)
     os.makedirs(directory, exist_ok=True)
@@ -106,10 +194,71 @@ def write_env(stack_id: str, content: str, override: Optional[str] = None) -> No
         fh.write(content)
 
 
-def delete_stack_files(stack_id: str, override: Optional[str] = None) -> None:
+def _mount_points() -> set[str]:
+    """Every mount point this process sees (from /proc/self/mountinfo)."""
+    points: set[str] = set()
+    try:
+        with open("/proc/self/mountinfo", "r", encoding="utf-8") as fh:
+            for line in fh:
+                fields = line.split()
+                if len(fields) >= 5:
+                    # Spaces etc. are octal-escaped in mountinfo.
+                    points.add(fields[4].encode().decode("unicode_escape"))
+    except OSError:
+        pass
+    return points
+
+
+def delete_stack_files(stack_id: str, override: Optional[str] = None) -> list[str]:
+    """Remove the stack folder, but never what is mounted inside it.
+
+    A stack folder can have a NAS share or a second disk mounted in it (a media
+    library at ``./media``). ``rmtree`` would have walked straight into it and
+    deleted the share's contents along with the compose file. Directories on
+    another filesystem, or that are mount points of their own (a bind mount of
+    the same disk), are left exactly as they are. Returns those paths.
+    """
     directory = stack_dir(stack_id, override)
-    if os.path.isdir(directory):
-        shutil.rmtree(directory)
+    if not os.path.isdir(directory):
+        return []
+    mounts = _mount_points()
+    root_dev = os.lstat(directory).st_dev
+    kept: list[str] = []
+
+    def remove(path: str) -> None:
+        with os.scandir(path) as it:
+            entries = list(it)
+        for entry in entries:
+            if entry.is_dir(follow_symlinks=False):
+                if (
+                    entry.stat(follow_symlinks=False).st_dev != root_dev
+                    or entry.path in mounts
+                ):
+                    kept.append(entry.path)
+                    continue
+                remove(entry.path)
+            else:
+                os.unlink(entry.path)
+        try:
+            os.rmdir(path)
+        except OSError:
+            pass  # still holds something that was kept
+
+    remove(directory)
+    return kept
+
+
+def _skip_special_files(directory: str, names: list[str]) -> list[str]:
+    """copytree ignore hook: sockets, FIFOs and devices cannot be copied."""
+    skipped = []
+    for name in names:
+        try:
+            mode = os.lstat(os.path.join(directory, name)).st_mode
+        except OSError:
+            continue
+        if not (stat.S_ISDIR(mode) or stat.S_ISREG(mode) or stat.S_ISLNK(mode)):
+            skipped.append(name)
+    return skipped
 
 
 def clone_stack_files(src_id: str, dst_id: str, override: Optional[str] = None) -> None:
@@ -117,7 +266,42 @@ def clone_stack_files(src_id: str, dst_id: str, override: Optional[str] = None)
     dst = stack_dir(dst_id, override)
     if os.path.isdir(dst):
         raise StackFileError(f"Target stack '{dst_id}' already exists")
-    shutil.copytree(src, dst)
+    try:
+        # symlinks=True copies links as links: following them would duplicate
+        # whatever they point at (or fail on a dangling one). A container's
+        # runtime socket in a data folder would fail the whole copy.
+        shutil.copytree(src, dst, symlinks=True, ignore=_skip_special_files)
+    except (OSError, shutil.Error) as exc:
+        shutil.rmtree(dst, ignore_errors=True)
+        raise StackFileError(f"Could not copy the stack folder: {exc}") from exc
+
+
+def force_remove_project(stack_id: str) -> int:
+    """Remove a project's containers and networks through the Docker API.
+
+    The fallback for deleting a stack whose ``compose down`` failed — typically
+    because the compose file no longer parses. Without it the containers kept
+    running with nothing left in StackPilot to stop them. Returns how many
+    containers went.
+    """
+    client = get_client()
+    label = f"{COMPOSE_LABEL}={project_name(stack_id)}"
+    removed = 0
+    for container in safe_call(client.containers.list, all=True, filters={"label": label}):
+        try:
+            container.remove(force=True)
+            removed += 1
+        except Exception:  # noqa: BLE001 - best effort, report what went
+            pass
+    try:
+        for network in safe_call(client.networks.list, filters={"label": label}):
+            try:
+                network.remove()
+            except Exception:  # noqa: BLE001
+                pass
+    except DockerError:
+        pass
+    return removed
 
 
 def discover_stacks(override: Optional[str] = None) -> list[str]:
@@ -178,7 +362,7 @@ def containers_for_stack(stack_id: str) -> list[ContainerInfo]:
     raw = safe_call(
         client.containers.list,
         all=True,
-        filters={"label": f"{COMPOSE_LABEL}={stack_id}"},
+        filters={"label": f"{COMPOSE_LABEL}={project_name(stack_id)}"},
     )
     result = []
     for c in raw:
@@ -272,7 +456,7 @@ def _compose_base_cmd(stack_id: str, override: Optional[str] = None) -> list[str
         "docker",
         "compose",
         "-p",
-        stack_id,
+        project_name(stack_id),
         "--project-directory",
         directory,
         "-f",
@@ -298,6 +482,7 @@ async def run_compose(
         stdout_b, stderr_b = await asyncio.wait_for(proc.communicate(), timeout=timeout)
     except asyncio.TimeoutError as exc:
         proc.kill()
+        await proc.wait()
         raise StackFileError(f"compose command timed out: {' '.join(args)}") from exc
     return {
         "returncode": proc.returncode,
@@ -307,42 +492,69 @@ async def run_compose(
     }
 
 
-async def validate_yaml(content: str, env_content: str = "") -> dict:
-    """Validate compose YAML by running ``docker compose config -q`` against it
-    in a throwaway directory. Returns ``{"ok": bool, "errors": str}`` — the
-    errors string carries compose's own diagnostics (parse errors, unknown
-    keys, bad interpolation) so the editor can surface them before saving."""
+async def validate_yaml(
+    content: str, env_content: str = "", project_dir: Optional[str] = None
+) -> dict:
+    """Validate compose YAML by running ``docker compose config -q`` against it.
+    Returns ``{"ok": bool, "errors": str}`` — the errors string carries
+    compose's own diagnostics (parse errors, unknown keys, bad interpolation)
+    so the editor can surface them before saving.
+
+    With ``project_dir`` (an existing stack's folder) relative paths resolve
+    against that folder, as they will on deploy. Validated from a bare temp
+    directory, a stack using ``env_file: ./app.env`` — or an ``include``, a
+    ``secrets: file:`` — was reported broken although it deploys fine. The
+    edited ``.env`` is still the one used for interpolation.
+    """
     with tempfile.TemporaryDirectory(prefix="sp-validate-") as tmp:
         compose_file = os.path.join(tmp, DEFAULT_COMPOSE_NAME)
         with open(compose_file, "w", encoding="utf-8") as fh:
             fh.write(content)
-        if env_content:
-            with open(os.path.join(tmp, ".env"), "w", encoding="utf-8") as fh:
-                fh.write(env_content)
+        env_file = os.path.join(tmp, ".env")
+        with open(env_file, "w", encoding="utf-8") as fh:
+            fh.write(env_content or "")
+        directory = project_dir if project_dir and os.path.isdir(project_dir) else tmp
         cmd = [
             "docker", "compose",
-            "--project-directory", tmp,
+            "--project-directory", directory,
             "-f", compose_file,
+            "--env-file", env_file,
             "config", "-q",
         ]
+        if directory != tmp:
+            # The folder's name may not be a valid project name; config does
+            # not care what the project is called.
+            cmd[2:2] = ["-p", "sp-validate"]
+        proc = await asyncio.create_subprocess_exec(
+            *cmd,
+            env=registry_service.cli_env(),
+            stdout=asyncio.subprocess.PIPE,
+            stderr=asyncio.subprocess.PIPE,
+        )
         try:
-            proc = await asyncio.create_subprocess_exec(
-                *cmd,
-                env=registry_service.cli_env(),
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            )
             _out, err = await asyncio.wait_for(proc.communicate(), timeout=30.0)
         except asyncio.TimeoutError as exc:
+            proc.kill()
+            await proc.wait()
             raise StackFileError("compose config validation timed out") from exc
         ok = proc.returncode == 0
-        return {"ok": ok, "errors": "" if ok else err.decode("utf-8", "replace").strip()}
+        errors = "" if ok else err.decode("utf-8", "replace").strip()
+        # Name the file the user is editing, not our scratch copy of it.
+        errors = errors.replace(compose_file, DEFAULT_COMPOSE_NAME).replace(env_file, ".env")
+        errors = errors.replace(tmp + os.sep, "./")
+        return {"ok": ok, "errors": errors}
 
 
 async def stream_compose(
     stack_id: str, args: list[str], override: Optional[str] = None
 ):
-    """Yield lines from a `docker compose` subcommand as they are produced."""
+    """Yield lines from a `docker compose` subcommand as they are produced.
+
+    The subprocess is killed when the consumer stops early (the generator is
+    closed or its task cancelled). For ``logs -f`` that is the only way it ever
+    ends: without this every log view that was closed left a ``compose logs``
+    process behind for as long as the containers lived.
+    """
     cmd = _compose_base_cmd(stack_id, override) + args
     proc = await asyncio.create_subprocess_exec(
         *cmd,
@@ -351,9 +563,22 @@ async def stream_compose(
         stderr=asyncio.subprocess.STDOUT,
     )
     assert proc.stdout is not None
-    async for raw in proc.stdout:
-        yield raw.decode("utf-8", "replace").rstrip("\n")
-    await proc.wait()
+    try:
+        async for raw in proc.stdout:
+            yield raw.decode("utf-8", "replace").rstrip("\n")
+        await proc.wait()
+    finally:
+        if proc.returncode is None:
+            try:
+                proc.kill()
+            except ProcessLookupError:
+                pass
+            # Reap it, so it does not linger as a zombie. Shielded: this runs
+            # while the consuming task is being cancelled.
+            try:
+                await asyncio.shield(proc.wait())
+            except (asyncio.CancelledError, ProcessLookupError):
+                pass
 
 
 _json_progress: Optional[bool] = None
@@ -468,8 +693,15 @@ async def pull(stack_id: str, override: Optional[str] = None) -> dict:
 
 
 async def update(stack_id: str, override: Optional[str] = None) -> dict:
-    """Pull then up -d."""
+    """Pull then up -d.
+
+    A failed pull stops here, like the streamed update does. Running ``up``
+    anyway recreated nothing new — the old images were still there — and its
+    exit code 0 then reported the update as a success.
+    """
     pull_res = await run_compose(stack_id, ["pull"], override)
+    if pull_res["returncode"] != 0:
+        return {**pull_res, "command": "pull + up -d"}
     up_res = await run_compose(stack_id, ["up", "-d", "--remove-orphans"], override)
     return {
         "returncode": up_res["returncode"],
@@ -487,7 +719,8 @@ async def logs(
 ) -> dict:
     args = ["logs", "--no-color", "--tail", str(tail), "--timestamps"]
     if service:
-        args.append(service)
+        # "--" so a service name can never be read as a compose option.
+        args += ["--", service]
     return await run_compose(stack_id, args, override, timeout=60.0)
 
 

backend/services/convert_service.py

@@ -1,135 +1,357 @@
-"""Convert a `docker run ...` command string into a Compose YAML fragment."""
+"""Convert a `docker run ...` command string into a Compose YAML fragment.
+
+The result has to be a compose file that ``docker compose config`` accepts and
+that means what the command meant. Three things got that wrong before:
+
+* every flag the converter did not know was assumed to take a value, so a
+  boolean like ``--read-only`` or ``-P`` swallowed the image name, and
+  ``-dit ubuntu bash`` came out as a service running the image ``bash``;
+* ``--network host`` became ``networks: [host]`` and ``-v data:/data`` a
+  reference to a volume nobody declared — both of which compose rejects;
+* flags with real effect (``--gpus``, ``--mount``, ``--entrypoint``,
+  ``--cap-drop``, ``--shm-size`` …) were dropped without a word.
+
+``docker run``'s boolean flags are a short, closed list; everything else takes a
+value. That is what the parser goes by now.
+"""
 from __future__ import annotations
 
+import re
 import shlex
 
 import yaml
 
+#: docker run flags that take no value.
+_BOOL_LONG = {
+    "--detach", "--interactive", "--tty", "--rm", "--init", "--privileged",
+    "--read-only", "--publish-all", "--oom-kill-disable", "--no-healthcheck",
+    "--disable-content-trust", "--quiet", "--sig-proxy", "--use-api-socket",
+}
+#: Single-letter booleans, which may be bundled ("-dit").
+_BOOL_SHORT = set("ditPq")
+#: Single-letter flags that take a value ("-p 80:80", "-p80:80").
+_VALUE_SHORT = {
+    "p": "--publish", "v": "--volume", "e": "--env", "l": "--label",
+    "u": "--user", "w": "--workdir", "h": "--hostname", "m": "--memory",
+    "a": "--attach", "c": "--cpu-shares",
+}
+_SHORT_TO_LONG = {"d": "--detach", "i": "--interactive", "t": "--tty",
+                  "P": "--publish-all", "q": "--quiet"}
+
+#: network_mode values, as opposed to networks a service joins.
+_NETWORK_MODES = {"host", "none", "bridge"}
+
+_SERVICE_NAME_RE = re.compile(r"[^a-zA-Z0-9._-]+")
+
 
 def _service_name_from_image(image: str) -> str:
-    name = image.split("/")[-1].split(":")[0]
-    return name or "app"
+    name = image.split("@", 1)[0].split("/")[-1].split(":")[0]
+    return _SERVICE_NAME_RE.sub("-", name).strip("-.") or "app"
+
+
+#: A line continuation whose newline is already gone. Pasted into a one-line
+#: field, backslash-newline arrives as a backslash followed by the next line's
+#: indentation, which shlex read as an escaped space: a lone " " token that
+#: became the image. A backslash followed by an option, or by more than one
+#: blank, is such a continuation; a single escaped space inside a word
+#: (``My\ Files``) is left alone.
+_FLATTENED_CONTINUATION = re.compile(r"\\[ \t]+(?=-)|\\[ \t]{2,}")
+
+
+def _tokens(command: str) -> list[str]:
+    # Shell line continuations, as pasted from a README.
+    command = command.replace("\\\r\n", " ").replace("\\\n", " ")
+    tokens = shlex.split(_FLATTENED_CONTINUATION.sub(" ", command))
+    # Drop leading "sudo docker run" / "docker container run".
+    while tokens and tokens[0] in ("docker", "run", "container", "sudo", "podman"):
+        tokens.pop(0)
+    return tokens
+
+
+def _parse(tokens: list[str]) -> tuple[list[tuple[str, str | None]], str | None, list[str]]:
+    """Split into ([(long_flag, value)], image, command)."""
+    flags: list[tuple[str, str | None]] = []
+    i, n = 0, len(tokens)
+    while i < n:
+        tok = tokens[i]
+        if tok == "--":
+            i += 1
+            break
+        if not tok.startswith("-") or tok == "-":
+            break  # the image
+        if tok.startswith("--"):
+            name, eq, value = tok.partition("=")
+            if eq:
+                flags.append((name, value))
+                i += 1
+            elif name in _BOOL_LONG:
+                flags.append((name, None))
+                i += 1
+            else:
+                flags.append((name, tokens[i + 1] if i + 1 < n else ""))
+                i += 2
+            continue
+        # Short form: bundled booleans ("-dit"), or one value flag with its
+        # value attached ("-p8080:80") or following ("-p 8080:80").
+        body = tok[1:]
+        j = 0
+        while j < len(body) and body[j] in _BOOL_SHORT:
+            flags.append((_SHORT_TO_LONG[body[j]], None))
+            j += 1
+        if j == len(body):
+            i += 1
+            continue
+        letter = body[j]
+        long_name = _VALUE_SHORT.get(letter, f"-{letter}")
+        attached = body[j + 1 :]
+        if attached.startswith("="):
+            attached = attached[1:]
+        if attached:
+            flags.append((long_name, attached))
+            i += 1
+        else:
+            flags.append((long_name, tokens[i + 1] if i + 1 < n else ""))
+            i += 2
+    image = tokens[i] if i < n else None
+    command = tokens[i + 1 :] if image is not None else []
+    return flags, image, command
+
+
+def _is_named_volume(source: str) -> bool:
+    return bool(source) and not source.startswith(("/", ".", "~", "$"))
+
+
+def _mount(value: str) -> dict:
+    """``--mount type=bind,source=/a,target=/b,readonly`` as compose long syntax."""
+    spec: dict = {}
+    for part in value.split(","):
+        key, eq, val = part.partition("=")
+        key = key.strip()
+        if key in ("source", "src"):
+            spec["source"] = val
+        elif key in ("target", "destination", "dst"):
+            spec["target"] = val
+        elif key == "type":
+            spec["type"] = val
+        elif key in ("readonly", "ro"):
+            spec["read_only"] = (not eq) or val.lower() in ("1", "true")
+        elif key == "bind-propagation":
+            spec.setdefault("bind", {})["propagation"] = val
+        elif key == "volume-nocopy":
+            spec.setdefault("volume", {})["nocopy"] = (not eq) or val.lower() in ("1", "true")
+        elif key == "tmpfs-size":
+            spec.setdefault("tmpfs", {})["size"] = val
+    spec.setdefault("type", "volume")
+    return spec
+
+
+def _gpus(value: str) -> dict:
+    """``--gpus all`` / ``--gpus 2`` / ``--gpus device=0,1`` as a device reservation."""
+    reservation: dict = {"driver": "nvidia", "capabilities": ["gpu"]}
+    value = value.strip().strip('"').strip("'")
+    if value.startswith("device="):
+        reservation["device_ids"] = [d for d in value[len("device="):].split(",") if d]
+    elif value.isdigit():
+        reservation["count"] = int(value)
+    else:
+        reservation["count"] = "all"
+    return reservation
+
+
+def _ulimit(value: str, ulimits: dict) -> None:
+    name, _, limits = value.partition("=")
+    soft, _, hard = limits.partition(":")
+    try:
+        ulimits[name] = (
+            {"soft": int(soft), "hard": int(hard)} if hard else int(soft)
+        )
+    except ValueError:
+        pass
 
 
 def convert_docker_run(command: str) -> str:
-    tokens = shlex.split(command)
-    # Drop leading "docker run" / "docker container run".
-    while tokens and tokens[0] in ("docker", "run", "container", "sudo"):
-        tokens.pop(0)
-
-    service: dict = {}
-    name = None
-    image = None
-    ports: list[str] = []
-    volumes: list[str] = []
-    environment: list[str] = []
-    env_file: list[str] = []
-    devices: list[str] = []
-    cap_add: list[str] = []
-    labels: list[str] = []
-    networks: list[str] = []
-    command_args: list[str] = []
-
-    i = 0
-    n = len(tokens)
-
-    def take_value(idx: int, tok: str):
-        if "=" in tok and tok.startswith("--") and not tok.endswith("="):
-            return tok.split("=", 1)[1], idx + 1
-        return tokens[idx + 1], idx + 2
-
-    while i < n:
-        tok = tokens[i]
-        if image is not None:
-            # Everything after the image is the container command.
-            command_args = tokens[i:]
-            break
-        if not tok.startswith("-"):
-            image = tok
-            i += 1
-            continue
-
-        flag = tok.split("=", 1)[0]
-        if flag in ("-d", "--detach", "-i", "--interactive", "-t", "--tty", "--rm", "--init"):
-            i += 1
-        elif flag in ("--name",):
-            name, i = take_value(i, tok)
-        elif flag in ("-p", "--publish"):
-            val, i = take_value(i, tok)
-            ports.append(val)
-        elif flag in ("-v", "--volume", "--mount"):
-            val, i = take_value(i, tok)
-            volumes.append(val)
-        elif flag in ("-e", "--env"):
-            val, i = take_value(i, tok)
-            environment.append(val)
-        elif flag in ("--env-file",):
-            val, i = take_value(i, tok)
-            env_file.append(val)
-        elif flag in ("--device",):
-            val, i = take_value(i, tok)
-            devices.append(val)
-        elif flag in ("--cap-add",):
-            val, i = take_value(i, tok)
-            cap_add.append(val)
-        elif flag in ("-l", "--label"):
-            val, i = take_value(i, tok)
-            labels.append(val)
-        elif flag in ("--network", "--net"):
-            val, i = take_value(i, tok)
-            networks.append(val)
-        elif flag in ("--restart",):
-            val, i = take_value(i, tok)
-            service["restart"] = val
-        elif flag in ("--privileged",):
-            service["privileged"] = True
-            i += 1
-        elif flag in ("--hostname", "-h"):
-            val, i = take_value(i, tok)
-            service["hostname"] = val
-        elif flag in ("-u", "--user"):
-            val, i = take_value(i, tok)
-            service["user"] = val
-        elif flag in ("-w", "--workdir"):
-            val, i = take_value(i, tok)
-            service["working_dir"] = val
-        else:
-            # Unknown flag — try to consume a value if it looks like it takes one.
-            if "=" in tok:
-                i += 1
-            elif i + 1 < n and not tokens[i + 1].startswith("-"):
-                i += 2
-            else:
-                i += 1
-
+    tokens = _tokens(command)
+    flags, image, command_args = _parse(tokens)
     if not image:
         raise ValueError("Could not find an image in the docker run command")
 
-    service["image"] = image
-    if "restart" not in service:
-        service["restart"] = "unless-stopped"
-    if ports:
-        service["ports"] = ports
-    if volumes:
-        service["volumes"] = volumes
-    if environment:
-        service["environment"] = environment
-    if env_file:
-        service["env_file"] = env_file
-    if devices:
-        service["devices"] = devices
-    if cap_add:
-        service["cap_add"] = cap_add
-    if labels:
-        service["labels"] = labels
-    if networks:
-        service["networks"] = networks
+    service: dict = {"image": image}
+    top_volumes: dict = {}
+    top_networks: dict = {}
+    name = None
+    lists: dict[str, list] = {}
+    healthcheck: dict = {}
+    logging: dict = {}
+    ulimits: dict = {}
+    gpus: list[dict] = []
+
+    def add(key: str, value) -> None:
+        lists.setdefault(key, []).append(value)
+
+    for flag, value in flags:
+        value = value if value is not None else ""
+        if flag == "--name":
+            name = value
+        elif flag == "--publish":
+            add("ports", value)
+        elif flag == "--volume":
+            add("volumes", value)
+            source = value.split(":", 1)[0] if ":" in value else ""
+            if _is_named_volume(source):
+                top_volumes[source] = None
+        elif flag == "--mount":
+            spec = _mount(value)
+            add("volumes", spec)
+            if spec.get("type") == "volume" and _is_named_volume(spec.get("source", "")):
+                top_volumes[spec["source"]] = None
+        elif flag == "--tmpfs":
+            add("tmpfs", value)
+        elif flag == "--env":
+            add("environment", value)
+        elif flag == "--env-file":
+            add("env_file", value)
+        elif flag == "--device":
+            add("devices", value)
+        elif flag == "--cap-add":
+            add("cap_add", value)
+        elif flag == "--cap-drop":
+            add("cap_drop", value)
+        elif flag == "--label":
+            add("labels", value)
+        elif flag in ("--network", "--net"):
+            if value in _NETWORK_MODES or value.startswith(("container:", "service:")):
+                service["network_mode"] = value
+            elif value:
+                add("networks", value)
+                # `docker run --network x` needs x to exist already.
+                top_networks[value] = {"external": True}
+        elif flag == "--network-alias":
+            add("_aliases", value)
+        elif flag == "--add-host":
+            add("extra_hosts", value)
+        elif flag == "--dns":
+            add("dns", value)
+        elif flag == "--dns-search":
+            add("dns_search", value)
+        elif flag == "--security-opt":
+            add("security_opt", value)
+        elif flag == "--sysctl":
+            add("sysctls", value)
+        elif flag == "--group-add":
+            add("group_add", value)
+        elif flag == "--expose":
+            add("expose", value)
+        elif flag == "--gpus":
+            gpus.append(_gpus(value))
+        elif flag == "--ulimit":
+            _ulimit(value, ulimits)
+        elif flag == "--restart":
+            service["restart"] = value
+        elif flag == "--privileged":
+            service["privileged"] = True
+        elif flag == "--read-only":
+            service["read_only"] = True
+        elif flag == "--init":
+            service["init"] = True
+        elif flag == "--interactive":
+            service["stdin_open"] = True
+        elif flag == "--tty":
+            service["tty"] = True
+        elif flag == "--hostname":
+            service["hostname"] = value
+        elif flag == "--user":
+            service["user"] = value
+        elif flag == "--workdir":
+            service["working_dir"] = value
+        elif flag == "--entrypoint":
+            service["entrypoint"] = value
+        elif flag == "--shm-size":
+            service["shm_size"] = value
+        elif flag == "--memory":
+            service["mem_limit"] = value
+        elif flag == "--memory-reservation":
+            service["mem_reservation"] = value
+        elif flag == "--cpus":
+            try:
+                service["cpus"] = float(value)
+            except ValueError:
+                pass
+        elif flag == "--cpu-shares":
+            try:
+                service["cpu_shares"] = int(value)
+            except ValueError:
+                pass
+        elif flag == "--pids-limit":
+            try:
+                service["pids_limit"] = int(value)
+            except ValueError:
+                pass
+        elif flag == "--pid":
+            service["pid"] = value
+        elif flag == "--ipc":
+            service["ipc"] = value
+        elif flag == "--stop-signal":
+            service["stop_signal"] = value
+        elif flag in ("--stop-timeout", "--time"):
+            service["stop_grace_period"] = f"{value}s" if value.isdigit() else value
+        elif flag == "--platform":
+            service["platform"] = value
+        elif flag == "--pull":
+            service["pull_policy"] = value
+        elif flag == "--log-driver":
+            logging["driver"] = value
+        elif flag == "--log-opt":
+            key, _, val = value.partition("=")
+            logging.setdefault("options", {})[key] = val
+        elif flag == "--health-cmd":
+            healthcheck["test"] = ["CMD-SHELL", value]
+        elif flag == "--health-interval":
+            healthcheck["interval"] = value
+        elif flag == "--health-timeout":
+            healthcheck["timeout"] = value
+        elif flag == "--health-start-period":
+            healthcheck["start_period"] = value
+        elif flag == "--health-retries":
+            try:
+                healthcheck["retries"] = int(value)
+            except ValueError:
+                pass
+        elif flag == "--no-healthcheck":
+            healthcheck = {"disable": True}
+        # Everything else (--detach, --rm, --publish-all, --cidfile, …) has no
+        # compose equivalent worth writing down.
+
+    aliases = lists.pop("_aliases", [])
+    if aliases:
+        nets = lists.get("networks") or []
+        if nets:
+            # Aliases apply to the networks given on the command line.
+            lists["networks"] = {net: {"aliases": aliases} for net in nets}
+    service.setdefault("restart", "unless-stopped")
+    for key in (
+        "ports", "expose", "volumes", "tmpfs", "environment", "env_file", "devices",
+        "cap_add", "cap_drop", "security_opt", "sysctls", "group_add", "extra_hosts",
+        "dns", "dns_search", "labels", "networks",
+    ):
+        if key in lists:
+            service[key] = lists[key]
+    if ulimits:
+        service["ulimits"] = ulimits
+    if logging:
+        service["logging"] = logging
+    if healthcheck:
+        service["healthcheck"] = healthcheck
+    if gpus:
+        service["deploy"] = {"resources": {"reservations": {"devices": gpus}}}
     if command_args:
         service["command"] = command_args
-
-    svc_name = name or _service_name_from_image(image)
     if name:
         service["container_name"] = name
 
-    doc = {"services": {svc_name: service}}
+    svc_name = _SERVICE_NAME_RE.sub("-", name).strip("-.") if name else ""
+    doc: dict = {"services": {svc_name or _service_name_from_image(image): service}}
+    if top_volumes:
+        doc["volumes"] = top_volumes
+    if top_networks:
+        doc["networks"] = top_networks
     return yaml.safe_dump(doc, sort_keys=False, default_flow_style=False)

backend/services/dashboard_service.py

@@ -132,7 +132,7 @@ def _local_host() -> tuple[dict, list[dict]]:
     unhealthy: list[str] = []
     updates = 0
     for stack_id in discovered:
-        containers = by_project.get(stack_id, [])
+        containers = by_project.get(compose_service.project_name(stack_id), [])
         status = compose_service._status_from_states([c.get("State", "") for c in containers])
         statuses.append(status)
         if status == "running" and not _is_healthy(containers):

backend/services/device_service.py

@@ -118,12 +118,61 @@ def _is_allowed(path: str) -> bool:
     return False
 
 
+def _toward_roots(path: str) -> list[dict]:
+    """For a path *above* the allowed roots: the folders below it that lead to
+    one (``/`` -> ``mnt``, ``opt``; ``/srv`` -> ``data`` for ``/srv/data``).
+
+    The browser opens at ``/``, which the default roots do not include, and the
+    breadcrumbs and "up" lead there too — so the page used to open on an error
+    with no way into any root at all.
+    """
+    prefix = "/" if path == "/" else path + os.sep
+    names: set[str] = set()
+    for root in settings.ALLOWED_BROWSE_ROOTS:
+        root = os.path.normpath(root)
+        if not root.startswith(prefix):
+            continue
+        try:
+            if not os.path.isdir(_real_root(root)):
+                continue
+        except BrowseError:
+            continue
+        names.add(root[len(prefix):].split(os.sep, 1)[0])
+    entries = []
+    for name in sorted(names, key=str.lower):
+        try:
+            st = os.stat(_real_root(os.path.join(path, name)))
+        except (OSError, BrowseError):
+            continue
+        entries.append(
+            {
+                "name": name,
+                "type": "dir",
+                "size": None,
+                "permissions": oct(st.st_mode & 0o777),
+                "mtime": st.st_mtime,
+                "symlink": False,
+            }
+        )
+    return entries
+
+
 def browse(path: str = "/", show_hidden: bool = False) -> dict:
     path = os.path.normpath(path or "/")
     if not path.startswith("/"):
         raise BrowseError("Path must be absolute")
     if not _is_allowed(path):
-        raise BrowseError("Path is outside the allowed browse roots")
+        entries = _toward_roots(path)
+        if not entries:
+            raise BrowseError("Path is outside the allowed browse roots")
+        return {
+            "path": path,
+            "parent": os.path.dirname(path) if path != "/" else None,
+            "roots": settings.ALLOWED_BROWSE_ROOTS,
+            "entries": entries,
+            # Only a way through: nothing can be created or changed here.
+            "virtual": True,
+        }
 
     real = _real_root(path)
     if not os.path.isdir(real):

backend/services/exec_service.py

@@ -8,9 +8,12 @@ stdout/stderr stream), suitable for piping straight to xterm.js.
 from __future__ import annotations
 
 import asyncio
+import codecs
 import contextlib
 import json
+import queue
 import socket as _socket
+import threading
 
 from starlette.websockets import WebSocketDisconnect
 
@@ -71,22 +74,64 @@ async def pump_exec(websocket, exec_id: str, holder, raw) -> None:
     accepted as keystrokes). Container -> browser: ``{"type":"data","data":...}``
     then a final ``{"type":"exit","code":...}``.
 
-    The blocking ``recv`` runs in the default executor; on teardown we shut the
-    socket down so that orphaned recv thread unblocks and exits.
+    The blocking ``recv`` and ``sendall`` run on two threads of this session's
+    own. They used to run on the event loop's default executor, where an open
+    terminal held a worker for its whole lifetime — shared with every
+    ``asyncio.to_thread`` in the app. On teardown the socket is shut down, which
+    is what unblocks the reader.
     """
     raw.setblocking(True)
-    loop = asyncio.get_event_loop()
+    loop = asyncio.get_running_loop()
     closed = asyncio.Event()
+    incoming: asyncio.Queue = asyncio.Queue()
+    outgoing: queue.Queue = queue.Queue()
 
-    async def to_browser() -> None:
+    def post(item) -> None:
+        try:
+            loop.call_soon_threadsafe(incoming.put_nowait, item)
+        except RuntimeError:  # loop closed
+            pass
+
+    def reader() -> None:
         try:
             while True:
-                data = await loop.run_in_executor(None, raw.recv, 4096)
+                data = raw.recv(4096)
                 if not data:
                     break
-                await websocket.send_text(
-                    json.dumps({"type": "data", "data": data.decode("utf-8", "replace")})
-                )
+                post(data)
+        except Exception:  # noqa: BLE001 - shutdown on teardown lands here
+            pass
+        finally:
+            post(None)
+
+    def writer() -> None:
+        while True:
+            data = outgoing.get()
+            if data is None:
+                return
+            try:
+                raw.sendall(data)
+            except Exception:  # noqa: BLE001
+                return
+
+    name = exec_id[:12]
+    threading.Thread(target=reader, name=f"exec-read-{name}", daemon=True).start()
+    threading.Thread(target=writer, name=f"exec-write-{name}", daemon=True).start()
+
+    async def to_browser() -> None:
+        # Incremental: a multi-byte character (an umlaut, a box-drawing glyph in
+        # htop) split across two reads would otherwise turn into two U+FFFD.
+        decoder = codecs.getincrementaldecoder("utf-8")(errors="replace")
+        try:
+            while True:
+                data = await incoming.get()
+                if data is None:
+                    break
+                text = decoder.decode(data)
+                if text:
+                    await websocket.send_text(json.dumps({"type": "data", "data": text}))
+            if tail := decoder.decode(b"", final=True):
+                await websocket.send_text(json.dumps({"type": "data", "data": tail}))
         except Exception:  # noqa: BLE001
             pass
         finally:
@@ -105,9 +150,9 @@ async def pump_exec(websocket, exec_id: str, holder, raw) -> None:
                     with contextlib.suppress(Exception):
                         resize_exec(exec_id, int(obj.get("rows", 24)), int(obj.get("cols", 80)))
                 elif isinstance(obj, dict) and "data" in obj:
-                    await loop.run_in_executor(None, raw.sendall, str(obj["data"]).encode())
+                    outgoing.put(str(obj["data"]).encode())
                 else:
-                    await loop.run_in_executor(None, raw.sendall, msg.encode())
+                    outgoing.put(msg.encode())
         except WebSocketDisconnect:
             pass
         except Exception:  # noqa: BLE001
@@ -119,6 +164,7 @@ async def pump_exec(websocket, exec_id: str, holder, raw) -> None:
     in_task = asyncio.create_task(from_browser())
     await closed.wait()
 
+    outgoing.put(None)
     with contextlib.suppress(Exception):
         raw.shutdown(_socket.SHUT_RDWR)
     with contextlib.suppress(Exception):

backend/services/file_service.py

@@ -124,6 +124,8 @@ def create_file(path: str, name: str) -> dict:
 
 def rename(path: str, new_name: str) -> dict:
     real = _safe_real(path)
+    if _is_root(path):
+        raise BrowseError("Refusing to rename a browse root")
     if not os.path.lexists(real):
         raise BrowseError(f"No such path: {path}")
     parent = os.path.dirname(path) or "/"
@@ -138,10 +140,14 @@ def rename(path: str, new_name: str) -> dict:
     return {"path": dest}
 
 
+def _is_root(path: str) -> bool:
+    norm = os.path.normpath(path)
+    return norm == "/" or norm in {os.path.normpath(r) for r in _root_paths()}
+
+
 def delete(path: str, recursive: bool = False) -> dict:
     real = _safe_real(path)
-    norm = os.path.normpath(path)
-    if norm == "/" or norm in {os.path.normpath(r) for r in _root_paths()}:
+    if _is_root(path):
         raise BrowseError("Refusing to delete a browse root")
     if not os.path.lexists(real):
         raise BrowseError(f"No such path: {path}")
@@ -324,6 +330,10 @@ def _transfer_dest(src: str, dest_dir: str, overwrite: bool) -> tuple[str, str,
     name = os.path.basename(src.rstrip("/"))
     dest_logical = _child(dest_dir, name)
     dest_real = _safe_real(dest_logical)
+    # Overwriting removes the target first — which, for nested roots
+    # (/mnt and /mnt/user), could be a root. Same rule as delete.
+    if overwrite and _is_root(dest_logical):
+        raise BrowseError("Refusing to replace a browse root")
 
     # Refuse to copy/move a directory into itself or its own subtree.
     src_norm = os.path.normpath(src_real)
@@ -350,6 +360,10 @@ def copy(src: str, dest_dir: str, overwrite: bool = False) -> dict:
 
 
 def move(src: str, dest_dir: str, overwrite: bool = False) -> dict:
+    # Moving a root away takes the whole tree with it — a delete by another
+    # name, and delete refuses roots.
+    if _is_root(src):
+        raise BrowseError("Refusing to move a browse root")
     src_real, dest_real, dest_logical = _transfer_dest(src, dest_dir, overwrite)
     try:
         if os.path.exists(dest_real) and overwrite:

backend/services/git_service.py

@@ -53,6 +53,29 @@ class GitError(Exception):
     """A repository that cannot be reached, or a sync that cannot be completed."""
 
 
+#: What a branch name may look like here. git itself allows more, but not a
+#: leading "-": as the refspec of `git fetch origin <branch>` that would be read
+#: as an option (`--upload-pack=…` runs a command).
+_BRANCH_RE = re.compile(r"^[A-Za-z0-9._/][A-Za-z0-9._/+-]{0,199}$")
+
+
+def validate_branch(branch: str) -> str:
+    branch = (branch or "").strip()
+    if not _BRANCH_RE.match(branch) or ".." in branch or branch.endswith((".lock", "/")):
+        raise GitError(f"Invalid branch name '{branch}'")
+    return branch
+
+
+def validate_url(url: str) -> str:
+    url = (url or "").strip()
+    if not url or url.startswith("-") or any(c.isspace() for c in url):
+        raise GitError(f"Invalid repository URL '{url}'")
+    # git's ext:: and fd:: transports run commands or read file descriptors.
+    if url.lower().startswith(("ext::", "fd::")):
+        raise GitError("ext:: and fd:: repository URLs are not supported")
+    return url
+
+
 # --------------------------------------------------------------------------- #
 # Paths
 # --------------------------------------------------------------------------- #
@@ -179,7 +202,8 @@ def _cleanup_auth(source: GitSource) -> None:
 async def _fetch(source: GitSource, env: dict, secret: Optional[str]) -> str:
     """Bring the cached clone to the tip of the configured branch. Returns the commit."""
     directory = repo_dir(source.stack_id)
-    branch = source.branch or "main"
+    branch = validate_branch(source.branch or "main")
+    validate_url(source.url)
     os.makedirs(cache_root(), mode=0o700, exist_ok=True)
 
     if os.path.isdir(os.path.join(directory, ".git")):
@@ -193,8 +217,10 @@ async def _fetch(source: GitSource, env: dict, secret: Optional[str]) -> str:
             shutil.rmtree(directory, ignore_errors=True)
 
     if not os.path.isdir(os.path.join(directory, ".git")):
+        # "--" ends the options: a URL that starts with "-" is a URL.
         await _git(
-            ["clone", "--depth", "1", "--branch", branch, source.url, directory], env, secret
+            ["clone", "--depth", "1", "--branch", branch, "--", source.url, directory],
+            env, secret,
         )
     else:
         await _git(["-C", directory, "fetch", "--depth", "1", "origin", branch], env, secret)
@@ -294,6 +320,12 @@ def _prune_empty_dirs(stack_dir: str, removed: list[str]) -> None:
 
 async def sync(session: Session, source: GitSource, actor: str = "system") -> SyncResult:
     """Fetch, copy into the stack, and deploy when something changed."""
+    from models.stack import Stack
+
+    if session.get(Stack, source.stack_id) is None:
+        # A source left behind by a deleted stack: syncing it would recreate
+        # the folder and, with auto deploy, start the stack again.
+        raise GitError(f"Stack '{source.stack_id}' no longer exists")
     env, secret = _auth_env(source)
     try:
         commit = await _fetch(source, env, secret)

backend/services/gpu_service.py

@@ -8,6 +8,7 @@ from __future__ import annotations
 
 import glob
 import os
+import re
 import subprocess
 from dataclasses import asdict, dataclass
 from typing import Literal, Optional
@@ -83,6 +84,49 @@ def _detect_nvidia() -> list[GPUInfo]:
     return gpus
 
 
+def _detect_nvidia_procfs() -> list[GPUInfo]:
+    """NVIDIA GPUs from the kernel driver's /proc entries.
+
+    StackPilot's image has no ``nvidia-smi`` (and the container runs without
+    the NVIDIA runtime), and the DRI scan below skips NVIDIA on purpose — so on
+    a normal deployment an NVIDIA card was never detected at all. The driver
+    publishes every GPU under ``/proc/driver/nvidia/gpus/<bus>/information``,
+    which the host /proc mount (``HOST_PROC_PATH``) makes readable here.
+    """
+    from config import settings
+
+    for proc in (settings.HOST_PROC_PATH, "/proc"):
+        base = os.path.join(proc, "driver", "nvidia", "gpus")
+        if not os.path.isdir(base):
+            continue
+        version = _read(os.path.join(proc, "driver", "nvidia", "version"))
+        match = re.search(r"Kernel Module\s+([\d.]+)", version)
+        driver = match.group(1) if match else ""
+        gpus: list[GPUInfo] = []
+        for position, bus in enumerate(sorted(os.listdir(base))):
+            info: dict[str, str] = {}
+            for line in _read(os.path.join(base, bus, "information")).splitlines():
+                key, sep, value = line.partition(":")
+                if sep:
+                    info[key.strip()] = value.strip()
+            try:
+                index = int(info.get("Device Minor", position))
+            except ValueError:
+                index = position
+            gpus.append(
+                GPUInfo(
+                    vendor="nvidia",
+                    index=index,
+                    name=info.get("Model") or "NVIDIA GPU",
+                    uuid=info.get("GPU UUID") or None,
+                    driver=driver,
+                )
+            )
+        if gpus:
+            return sorted(gpus, key=lambda g: g.index)
+    return []
+
+
 def _read(path: str) -> str:
     try:
         with open(path, "r", encoding="utf-8") as fh:
@@ -151,7 +195,7 @@ def _detect_dri() -> list[GPUInfo]:
 
 
 def detect_gpus() -> list[GPUInfo]:
-    return _detect_nvidia() + _detect_dri()
+    return (_detect_nvidia() or _detect_nvidia_procfs()) + _detect_dri()
 
 
 def dri_group_gids() -> set[int]:

backend/services/image_service.py

@@ -14,23 +14,33 @@ COMPOSE_PROJECT_LABEL = "com.docker.compose.project"
 
 
 def list_images() -> list[dict]:
-    """Return one row per image tag, annotated with using-stacks + update status."""
+    """Return one row per image tag, annotated with using-stacks + update status.
+
+    Usage is matched on the image *id* the containers run, not on the name:
+    a compose file saying ``image: nginx`` or ``docker.io/library/nginx:1``
+    produces containers whose configured image never equals the tag Docker
+    lists (``nginx:latest``, ``nginx:1``), so those images showed up as used
+    by no stack — and were left out of "scan all".
+    """
     try:
         client = get_client()
         images = safe_call(client.images.list)
-        containers = safe_call(client.containers.list, all=True)
+        containers = safe_call(client.api.containers, all=True)
     except DockerError:
         return []
 
-    # Map image ref -> stacks using it.
-    usage: dict[str, set[str]] = {}
+    # image id -> stacks using it, and the references they were created from
+    # (which is what the update checker caches its results under).
+    stacks_by_id: dict[str, set[str]] = {}
+    refs_by_id: dict[str, set[str]] = {}
     for c in containers:
-        ref = c.attrs.get("Config", {}).get("Image")
-        stack = c.labels.get(COMPOSE_PROJECT_LABEL)
-        if ref:
-            usage.setdefault(ref, set())
-            if stack:
-                usage[ref].add(stack)
+        image_id = c.get("ImageID") or ""
+        stack = (c.get("Labels") or {}).get(COMPOSE_PROJECT_LABEL)
+        stacks_by_id.setdefault(image_id, set())
+        if stack:
+            stacks_by_id[image_id].add(stack)
+        if (ref := c.get("Image")) and not ref.startswith("sha256:"):
+            refs_by_id.setdefault(image_id, set()).add(ref)
 
     cache = update_service.get_cache()
     result = []
@@ -38,15 +48,20 @@ def list_images() -> list[dict]:
         tags = img.tags or []
         if not tags:
             continue
+        refs = refs_by_id.get(img.id, set())
         for tag in tags:
+            update = cache.get(tag)
+            if update is None:
+                # Cached under the name the compose file used for it.
+                update = next((cache[r] for r in sorted(refs) if r in cache), None)
             result.append(
                 {
                     "id": img.short_id,
                     "tag": tag,
                     "size": img.attrs.get("Size", 0),
                     "created": img.attrs.get("Created"),
-                    "stacks": sorted(usage.get(tag, set())),
-                    "update": cache.get(tag),
+                    "stacks": sorted(stacks_by_id.get(img.id, set())),
+                    "update": update,
                 }
             )
     result.sort(key=lambda r: r["tag"])

backend/services/image_status_store.py

@@ -31,6 +31,7 @@ def _to_status(row: ImageStatus) -> update_service.UpdateStatus:
         remote_digest=row.remote_digest,
         checked_at=row.checked_at,
         error=row.error,
+        local=row.local,
     )
 
 
@@ -46,6 +47,7 @@ def save(status: update_service.UpdateStatus, notified: bool) -> None:
         row.remote_digest = status.remote_digest
         row.checked_at = status.checked_at
         row.error = status.error
+        row.local = status.local
         row.notified = notified
         session.add(row)
         session.commit()

backend/services/notify_service.py

@@ -11,6 +11,7 @@ All delivery is best-effort: failures are logged, never raised to the caller.
 """
 from __future__ import annotations
 
+import base64
 import logging
 from typing import Optional
 
@@ -37,14 +38,14 @@ def _build_request(wtype: str, url: str, event: str, title: str, message: str):
         return {
             "url": url,
             "content": message.encode("utf-8"),
-            "headers": {"Title": title, "Tags": _ntfy_tag(event)},
+            "headers": {"Title": _header_value(title), "Tags": _ntfy_tag(event)},
         }
     if wtype == "discord":
         return {"url": url, "json": {"content": f"**{title}**\n{message}"}}
     if wtype == "slack":
         return {"url": url, "json": {"text": f"*{title}*\n{message}"}}
     if wtype == "gotify":
-        priority = 8 if event in ("stack_error", "pull_failed") else 5
+        priority = 8 if event in _URGENT_EVENTS else 5
         return {
             "url": url,
             "json": {"title": title, "message": message, "priority": priority},
@@ -56,6 +57,24 @@ def _build_request(wtype: str, url: str, event: str, title: str, message: str):
     }
 
 
+_URGENT_EVENTS = {"stack_error", "pull_failed", "backup_failed"}
+
+
+def _header_value(text: str) -> str:
+    """A header value httpx can send, keeping non-ASCII readable.
+
+    HTTP headers are ASCII; httpx raises UnicodeEncodeError on anything else,
+    so a title with an umlaut or a dash ("Backup to NAS Büro failed") never
+    reached ntfy at all. RFC 2047 encoded-words are what ntfy decodes.
+    """
+    try:
+        text.encode("ascii")
+        return text
+    except UnicodeEncodeError:
+        encoded = base64.b64encode(text.encode("utf-8")).decode("ascii")
+        return f"=?UTF-8?B?{encoded}?="
+
+
 def _ntfy_tag(event: str) -> str:
     return {
         "update_available": "arrow_up",
@@ -63,17 +82,21 @@ def _ntfy_tag(event: str) -> str:
         "stack_stop": "stop_button",
         "stack_error": "rotating_light",
         "pull_failed": "warning",
+        "backup_failed": "floppy_disk",
+        "stack_auto_updated": "arrows_counterclockwise",
     }.get(event, "bell")
 
 
 async def _deliver(client: httpx.AsyncClient, wtype: str, url: str, event: str, title: str, message: str) -> bool:
-    kwargs = _build_request(wtype, url, event, title, message)
-    target = kwargs.pop("url")
     try:
+        kwargs = _build_request(wtype, url, event, title, message)
+        target = kwargs.pop("url")
         resp = await client.post(target, timeout=_TIMEOUT, **kwargs)
         resp.raise_for_status()
         return True
-    except httpx.HTTPError as exc:
+    except Exception as exc:  # noqa: BLE001 - one bad webhook must not stop the rest
+        # Not just httpx.HTTPError: an invalid URL or an unencodable header
+        # raised past this and aborted delivery to every webhook after it.
         logger.warning("Webhook delivery failed (%s): %s", wtype, exc)
         return False
 

backend/services/port_service.py

@@ -8,6 +8,7 @@ import yaml
 
 from config import settings
 from docker_client import DockerError, get_client, safe_call
+from services.compose_service import project_name
 
 
 # --------------------------------------------------------------------------- #
@@ -21,30 +22,47 @@ def parse_compose_ports(yaml_str: str) -> list[dict]:
         data = yaml.safe_load(yaml_str) or {}
     except yaml.YAMLError:
         return []
+    if not isinstance(data, dict):
+        return []
     services = data.get("services") or {}
+    if not isinstance(services, dict):
+        return []
     out: list[dict] = []
     for svc_name, svc in services.items():
         if not isinstance(svc, dict):
             continue
         for entry in svc.get("ports", []) or []:
-            parsed = _parse_port_entry(entry)
-            if parsed:
+            for parsed in _parse_port_entry(entry):
                 parsed["service"] = svc_name
                 out.append(parsed)
     return out
 
 
-def _parse_port_entry(entry) -> Optional[dict]:
+#: A published range is checked port by port, up to this many.
+_MAX_RANGE = 1024
+
+
+def _host_ports(spec: str) -> list[int]:
+    """"8080" -> [8080]; "8080-8082" -> [8080, 8081, 8082]; junk -> []."""
+    try:
+        if "-" in spec:
+            low, high = (int(p) for p in spec.split("-", 1))
+            if low > high or high - low >= _MAX_RANGE:
+                return [low]
+            return list(range(low, high + 1))
+        return [int(spec)]
+    except ValueError:
+        return []
+
+
+def _parse_port_entry(entry) -> list[dict]:
     # Long form: {target, published, protocol}
     if isinstance(entry, dict):
         published = entry.get("published")
         if published is None:
-            return None
-        try:
-            host_port = int(str(published).split("-")[0])
-        except ValueError:
-            return None
-        return {"host_port": host_port, "protocol": entry.get("protocol", "tcp")}
+            return []
+        proto = str(entry.get("protocol") or "tcp")
+        return [{"host_port": p, "protocol": proto} for p in _host_ports(str(published))]
 
     # Short form string: "[ip:]host:container[/proto]" or "container"
     s = str(entry)
@@ -54,14 +72,10 @@ def _parse_port_entry(entry) -> Optional[dict]:
     parts = s.split(":")
     # No host mapping (only container port) -> random host port, no conflict.
     if len(parts) == 1:
-        return None
-    # host:container  or  ip:host:container
-    host = parts[-2]
-    try:
-        host_port = int(host.split("-")[0])
-    except ValueError:
-        return None
-    return {"host_port": host_port, "protocol": proto}
+        return []
+    # host:container  or  ip:host:container (an IPv6 address has more colons;
+    # the host port is always the second to last part)
+    return [{"host_port": p, "protocol": proto} for p in _host_ports(parts[-2])]
 
 
 # --------------------------------------------------------------------------- #
@@ -69,11 +83,27 @@ def _parse_port_entry(entry) -> Optional[dict]:
 # --------------------------------------------------------------------------- #
 
 
+def _proc_net_path(name: str) -> str:
+    """The host's socket table, not this container's.
+
+    ``/proc/net`` is a link to ``self/net`` — the network namespace of whoever
+    reads it — so ``${HOST_PROC_PATH}/net/tcp`` read from inside the StackPilot
+    container lists the container's own sockets (just its API port), and every
+    port a host process listens on went unnoticed. PID 1 of the host's /proc
+    lives in the host's namespace.
+    """
+    for candidate in (
+        os.path.join(settings.HOST_PROC_PATH, "1", "net", name),
+        os.path.join(settings.HOST_PROC_PATH, "net", name),
+    ):
+        if os.path.isfile(candidate):
+            return candidate
+    return os.path.join("/proc/net", name)
+
+
 def _read_proc_net(name: str) -> set[int]:
     ports: set[int] = set()
-    path = os.path.join(settings.HOST_PROC_PATH, "net", name)
-    if not os.path.isfile(path):
-        path = os.path.join("/proc/net", name)
+    path = _proc_net_path(name)
     try:
         with open(path, "r", encoding="utf-8") as fh:
             lines = fh.readlines()[1:]
@@ -154,7 +184,7 @@ def detect_conflicts(yaml_str: str, ignore_stack: Optional[str] = None) -> list[
             # Match on the compose project label, falling back to the legacy
             # name-prefix check for containers without the label.
             if ignore_stack and (
-                owner_project == ignore_stack
+                owner_project in (ignore_stack, project_name(ignore_stack))
                 or owner_name.startswith(f"{ignore_stack}-")
             ):
                 continue

backend/services/scan_service.py

@@ -43,7 +43,7 @@ SCAN_TIMEOUT = 900.0
 #: One sweep at a time. Scanning is CPU- and IO-heavy, and two passes over the
 #: same images would just fight each other for the daemon.
 _sweep_lock = threading.Lock()
-_sweeping = {"active": False, "done": 0, "total": 0}
+_sweeping = {"active": False, "done": 0, "total": 0, "scanned": 0, "failed": 0, "run": 0}
 
 
 class ScanError(Exception):
@@ -241,15 +241,16 @@ def sweep_status() -> dict:
     return dict(_sweeping)
 
 
-async def sweep(session: Session, images: list[str]) -> dict:
-    """Scan a list of images one after another.
-
-    Deliberately serial: each scan is already CPU-hungry, and running several at
-    once on a homelab box would starve the very containers being protected.
-    """
+def _begin(total: int) -> None:
     if not _sweep_lock.acquire(blocking=False):
         raise ScanError("A scan is already running")
-    _sweeping.update({"active": True, "done": 0, "total": len(images)})
+    _sweeping.update(
+        {"active": True, "done": 0, "total": total, "scanned": 0, "failed": 0,
+         "run": _sweeping.get("run", 0) + 1}
+    )
+
+
+async def _run(session: Session, images: list[str]) -> dict:
     failed = 0
     try:
         for image in images:
@@ -262,6 +263,46 @@ async def sweep(session: Session, images: list[str]) -> dict:
                 failed += 1
             _sweeping["done"] += 1
     finally:
-        _sweeping.update({"active": False})
+        _sweeping.update({"active": False, "scanned": len(images), "failed": failed})
         _sweep_lock.release()
     return {"scanned": len(images), "failed": failed}
+
+
+async def sweep(session: Session, images: list[str]) -> dict:
+    """Scan a list of images one after another.
+
+    Deliberately serial: each scan is already CPU-hungry, and running several at
+    once on a homelab box would starve the very containers being protected.
+    """
+    _begin(len(images))
+    return await _run(session, images)
+
+
+#: Background sweeps, kept referenced until they finish.
+_tasks: set[asyncio.Task] = set()
+
+
+def start_sweep(images: list[str], on_done=None) -> dict:
+    """Start a sweep in the background and return its status at once.
+
+    A sweep over a dozen images takes far longer than the proxy waits for a
+    response (600s), so the button used to report a 504 while the scans went
+    on. Marked active before this returns, so a poll right after sees it.
+    """
+    from database import engine
+
+    _begin(len(images))
+
+    async def work() -> None:
+        with Session(engine) as session:
+            result = await _run(session, images)
+            if on_done is not None:
+                try:
+                    on_done(session, result)
+                except Exception:  # noqa: BLE001 - the sweep itself is done
+                    logger.exception("Post-sweep hook failed")
+
+    task = asyncio.create_task(work(), name="image-scan-sweep")
+    _tasks.add(task)
+    task.add_done_callback(_tasks.discard)
+    return sweep_status()

backend/services/schedule_service.py

@@ -159,7 +159,9 @@ async def _backup_stack(
             stack_id, stack.name,
             include_volumes=schedule.include_volumes, stop_first=schedule.stop_first,
         )
-        filename = backup_service.backup_filename(stack_id, schedule.include_volumes)
+        filename = backup_service.backup_filename(
+            stack_id, schedule.include_volumes, tz=schedule.timezone
+        )
         await asyncio.to_thread(dest_service.upload, dest, path, filename)
         problems = _problems(report)
         if problems:

backend/services/secret_service.py

@@ -19,6 +19,17 @@ from typing import Optional
 from services import compose_edit_service, compose_service
 
 KIND_DIR = {"secret": ".secrets", "config": ".configs"}
+
+#: Who can read a secret is decided by the directory, not the file.
+#:
+#: Outside swarm, compose hands a file-based secret to the container as a bind
+#: mount of the file itself, with the file's own owner and mode. At 0600 root a
+#: container running as anything but root (postgres, every PUID/PGID image) got
+#: "permission denied" on /run/secrets/<name>. The bind mount reaches the file
+#: without passing through its directory, so a 0700 directory keeps other users
+#: on the host out while the file stays readable inside the container.
+DIR_MODE = 0o700
+FILE_MODE = 0o644
 _NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$")
 MAX_SECRET_BYTES = 1 * 1024 * 1024  # 1 MiB
 
@@ -83,11 +94,11 @@ def write_secret(stack_id: str, kind: str, name: str, content: str, override: Op
         raise SecretError("content exceeds 1 MiB limit")
     directory = kind_dir(stack_id, kind, override)
     os.makedirs(directory, exist_ok=True)
-    os.chmod(directory, 0o700)
+    os.chmod(directory, DIR_MODE)
     path = _path(stack_id, kind, name, override)
     with open(path, "wb") as fh:
         fh.write(data)
-    os.chmod(path, 0o600)
+    os.chmod(path, FILE_MODE)
     return {"name": name, "kind": kind, "size": len(data)}
 
 
@@ -107,6 +118,9 @@ def attach(stack_id: str, kind: str, name: str, service: str,
     file (relative ``file:`` path) and persist it. Returns the new YAML."""
     _check_kind(kind)
     _check_name(name)
+    stored = _path(stack_id, kind, name, override)
+    if os.path.isfile(stored):
+        os.chmod(stored, FILE_MODE)  # one written by an older version
     yaml_str = compose_service.read_compose(stack_id, override)
     file_path = rel_path(kind, name)
     if kind == "config":
@@ -126,3 +140,33 @@ def detach(stack_id: str, kind: str, name: str, service: str, override: Optional
         new_yaml = compose_edit_service.remove_secret(yaml_str, service, name)
     compose_service.write_compose(stack_id, new_yaml, override)
     return new_yaml
+
+
+def fix_permissions(root: Optional[str] = None) -> int:
+    """Make secrets written by older versions (0600) readable by containers.
+
+    Runs at startup over every stack's ``.secrets`` and ``.configs`` — folders
+    only StackPilot writes. Returns how many files were changed.
+    """
+    base = compose_service.stacks_root(root)
+    changed = 0
+    try:
+        stacks = os.listdir(base)
+    except OSError:
+        return 0
+    for stack in stacks:
+        for folder in KIND_DIR.values():
+            directory = os.path.join(base, stack, folder)
+            if not os.path.isdir(directory) or os.path.islink(directory):
+                continue
+            try:
+                os.chmod(directory, DIR_MODE)
+                for name in os.listdir(directory):
+                    path = os.path.join(directory, name)
+                    if os.path.isfile(path) and not os.path.islink(path):
+                        if os.stat(path).st_mode & 0o777 == 0o600:
+                            os.chmod(path, FILE_MODE)
+                            changed += 1
+            except OSError:
+                continue
+    return changed

backend/services/stack_assets_service.py

@@ -371,24 +371,48 @@ def import_volume(
 # --------------------------------------------------------------------------- #
 
 
+#: Found once and kept: a running container's mounts do not change.
+_HOST_STACKS_DIR: Optional[str] = None
+
+
 def host_stacks_dir() -> Optional[str]:
     """Host path backing ``STACKS_DIR`` inside this container, if detectable.
 
-    Read from /proc/self/mountinfo (field 4 is the source subtree on the host
-    filesystem). Returns None when not running in a container / not bind-mounted.
+    Taken from Docker's record of this container's mounts. Not from
+    /proc/self/mountinfo, as it once was: its 4th field is the path *inside the
+    mounted filesystem*, which is the host path only when that filesystem is the
+    host's root. A stacks folder on its own disk, a ZFS dataset or a btrfs
+    subvolume read as ``/stacks`` or ``/@/opt/stacks``, and every backup dialog
+    warned about a path mismatch that was not there. Returns None when not
+    running in a container or when the mount cannot be found.
     """
-    target = settings.STACKS_DIR.rstrip("/") or "/"
+    global _HOST_STACKS_DIR
+    if _HOST_STACKS_DIR is not None:
+        return _HOST_STACKS_DIR
+    from services.self_update_service import _own_container
+
+    target = os.path.normpath(settings.STACKS_DIR)
     try:
-        with open("/proc/self/mountinfo", "r", encoding="utf-8") as fh:
-            for line in fh:
-                parts = line.split()
-                if len(parts) < 5:
-                    continue
-                if parts[4].rstrip("/") == target:
-                    return parts[3]
-    except OSError:
+        container = _own_container()
+    except Exception:  # noqa: BLE001 - no Docker, no answer
         return None
-    return None
+    if container is None:
+        return None
+    # The mount whose destination is STACKS_DIR or the closest folder above it
+    # (a whole /opt mounted, STACKS_DIR=/opt/stacks).
+    best = None
+    for mount in container.attrs.get("Mounts") or []:
+        dest = os.path.normpath(mount.get("Destination") or "")
+        if mount.get("Type") != "bind" or not mount.get("Source"):
+            continue
+        if target == dest or target.startswith(dest.rstrip("/") + "/"):
+            if best is None or len(dest) > len(best[0]):
+                best = (dest, mount["Source"])
+    if best is None:
+        return None
+    dest, source = best
+    _HOST_STACKS_DIR = os.path.normpath(source + target[len(dest):])
+    return _HOST_STACKS_DIR
 
 
 def stacks_path_mismatch() -> Optional[dict]:
@@ -483,7 +507,7 @@ def _bind_specs_from_containers(stack_id: str) -> list[dict]:
         containers = safe_call(
             client.containers.list,
             all=True,
-            filters={"label": f"{COMPOSE_PROJECT_LABEL}={stack_id}"},
+            filters={"label": _project_label(stack_id)},
         )
     except DockerError:
         return []
@@ -503,6 +527,10 @@ def _bind_specs_from_containers(stack_id: str) -> list[dict]:
     return out
 
 
+def _project_label(stack_id: str) -> str:
+    return f"{COMPOSE_PROJECT_LABEL}={compose_service.project_name(stack_id)}"
+
+
 def is_system_path(path: str) -> bool:
     if path in SYSTEM_PATHS:
         return True
@@ -555,7 +583,7 @@ def _external_volume_names(stack_id: str) -> set[str]:
         containers = safe_call(
             client.containers.list,
             all=True,
-            filters={"label": f"{COMPOSE_PROJECT_LABEL}={stack_id}"},
+            filters={"label": _project_label(stack_id)},
         )
         for c in containers:
             for mount in c.attrs.get("Mounts") or []:
@@ -589,7 +617,7 @@ def compose_volumes(stack_id: str) -> list[dict]:
         client = get_client()
         vols = safe_call(
             client.volumes.list,
-            filters={"label": f"{COMPOSE_PROJECT_LABEL}={stack_id}"},
+            filters={"label": _project_label(stack_id)},
         )
     except DockerError:
         return []

backend/services/stack_cleanup_service.py

@@ -0,0 +1,86 @@
+"""What has to go with a deleted stack, besides its containers and files.
+
+Several features keep rows keyed by stack id, and deleting the stack used to
+leave all of them behind:
+
+* a **Git source**: the poll loop and the webhook kept syncing it — which
+  recreated the deleted stack's folder from the repository and, with auto
+  deploy on, started it again;
+* an **auto-update policy**, which a new stack of the same name then silently
+  inherited;
+* **backup schedules** naming it, which from then on failed on every run with
+  "stack not found" and paged somebody each time.
+"""
+from __future__ import annotations
+
+import logging
+
+from sqlmodel import Session, select
+
+from models.auto_update import AutoUpdate
+from models.backup_schedule import BackupSchedule, set_stacks, stacks_of
+from models.git_source import GitSource
+from services import git_service
+
+logger = logging.getLogger("stackpilot.stack_cleanup")
+
+
+def forget_stack(session: Session, stack_id: str) -> dict:
+    """Drop everything that refers to ``stack_id``. The caller deletes the Stack.
+
+    A schedule that covered only this stack is disabled rather than deleted:
+    it may carry a destination and retention somebody wants to point at
+    another stack. All-stacks schedules need nothing — they resolve the stack
+    list when they run.
+    """
+    report = {"git": False, "auto_update": 0, "schedules_changed": [], "schedules_disabled": []}
+
+    source = session.exec(select(GitSource).where(GitSource.stack_id == stack_id)).first()
+    if source is not None:
+        session.delete(source)
+        report["git"] = True
+
+    for policy in session.exec(select(AutoUpdate).where(AutoUpdate.stack_id == stack_id)).all():
+        session.delete(policy)
+        report["auto_update"] += 1
+
+    for schedule in session.exec(select(BackupSchedule)).all():
+        if schedule.all_stacks:
+            continue
+        ids = stacks_of(schedule)
+        if stack_id not in ids:
+            continue
+        remaining = [sid for sid in ids if sid != stack_id]
+        set_stacks(schedule, remaining)
+        if remaining:
+            report["schedules_changed"].append(schedule.id)
+        else:
+            schedule.enabled = False
+            schedule.last_status = f"disabled: its stack '{stack_id}' was deleted"
+            report["schedules_disabled"].append(schedule.id)
+        session.add(schedule)
+
+    session.commit()
+    if report["git"]:
+        git_service.forget(stack_id)
+    if any(report[k] for k in report):
+        logger.info("Cleaned up after deleted stack '%s': %s", stack_id, report)
+    return report
+
+
+def prune_orphans(session: Session) -> list[str]:
+    """Forget every stack that has rows here but no Stack any more.
+
+    Run at startup, after on-disk stacks have been registered: installs that
+    deleted stacks before this existed still carry their Git sources and
+    policies. Returns the stack ids cleaned up.
+    """
+    from models.stack import Stack
+
+    known = {s.id for s in session.exec(select(Stack)).all()}
+    referenced = {g.stack_id for g in session.exec(select(GitSource)).all()}
+    referenced |= {p.stack_id for p in session.exec(select(AutoUpdate)).all()}
+    orphans = sorted(referenced - known)
+    for stack_id in orphans:
+        forget_stack(session, stack_id)
+    return orphans

backend/services/update_service.py

@@ -39,6 +39,11 @@ class UpdateStatus:
     remote_digest: Optional[str]
     checked_at: float
     error: Optional[str] = None
+    #: Built on this host (``docker build``, compose ``build:``) rather than
+    #: pulled: there is no registry copy to compare against, so it is never
+    #: "out of date" — and asking Docker Hub about it only got a 401 that read
+    #: as "needs credentials".
+    local: bool = False
 
     def to_dict(self) -> dict:
         return asdict(self)
@@ -100,6 +105,14 @@ def _store(status: UpdateStatus, notified: bool) -> None:
 # --------------------------------------------------------------------------- #
 
 
+#: Docker Hub under the names people write in compose files. Only
+#: registry-1.docker.io serves the v2 API; https://docker.io/v2/ redirects to
+#: the marketing site, so a `docker.io/library/postgres` image could never be
+#: checked and sat at "could not reach registry" forever.
+_DOCKER_HUB = "registry-1.docker.io"
+_DOCKER_HUB_ALIASES = {"docker.io", "index.docker.io", "registry.docker.io", _DOCKER_HUB}
+
+
 def parse_ref(image: str) -> tuple[str, str, str]:
     """Return (registry_host, repository, tag/digest)."""
     ref = image
@@ -107,37 +120,68 @@ def parse_ref(image: str) -> tuple[str, str, str]:
     # split tag (but not the registry port colon)
     if "@" in ref:
         ref, tag = ref.split("@", 1)
-    else:
-        # find last colon after last slash
-        slash = ref.rfind("/")
-        colon = ref.rfind(":")
-        if colon > slash:
+    # find last colon after last slash. With a digest present this is the tag
+    # in "name:tag@sha256:…", which the digest outranks — but it must not stay
+    # glued to the repository name.
+    slash = ref.rfind("/")
+    colon = ref.rfind(":")
+    if colon > slash:
+        if "@" not in image:
             tag = ref[colon + 1 :]
-            ref = ref[:colon]
+        ref = ref[:colon]
 
     parts = ref.split("/", 1)
     if len(parts) == 2 and ("." in parts[0] or ":" in parts[0] or parts[0] == "localhost"):
         registry = parts[0]
         repo = parts[1]
+        if registry.lower() in _DOCKER_HUB_ALIASES:
+            registry = _DOCKER_HUB
     else:
-        registry = "registry-1.docker.io"
+        registry = _DOCKER_HUB
         repo = ref
-        if "/" not in repo:
-            repo = "library/" + repo
+    if registry == _DOCKER_HUB and "/" not in repo:
+        repo = "library/" + repo
     return registry, repo, tag
 
 
-def _local_digest(image: str) -> Optional[str]:
+def _local_attrs(image: str) -> Optional[dict]:
     try:
-        client = get_client()
-        img = safe_call(client.images.get, image)
+        return safe_call(get_client().images.get, image).attrs
     except DockerError:
         return None
-    repo_digests = img.attrs.get("RepoDigests") or []
+
+
+def _built_locally(attrs: Optional[dict]) -> bool:
+    """Whether the image came from a build on this host, not from a registry.
+
+    Docker 29+ records where an image came from (``Identity.Pull`` /
+    ``Identity.Build``). Older daemons do not; there a built image is the one
+    without RepoDigests. Unknown means "pulled" — skipping the check for a
+    pulled image would hide its updates for good.
+    """
+    if attrs is None:
+        return False
+    identity = attrs.get("Identity")
+    if isinstance(identity, dict) and identity:
+        return bool(identity.get("Build")) and not identity.get("Pull")
+    return not (attrs.get("RepoDigests") or [])
+
+
+def _local_digest(image: str, attrs: Optional[dict] = None) -> Optional[str]:
+    if attrs is None:
+        attrs = _local_attrs(image)
+    if attrs is None:
+        return None
+    repo_digests = [rd for rd in attrs.get("RepoDigests") or [] if "@" in rd]
+    # An image can carry digests for several repositories (the same image
+    # pulled as lscr.io/... and as docker.io/...); compare against the one
+    # this reference is checked against, not whichever comes first.
+    wanted = parse_ref(image)[:2]
     for rd in repo_digests:
-        if "@" in rd:
-            return rd.split("@", 1)[1]
-    return None
+        name, digest = rd.split("@", 1)
+        if parse_ref(name)[:2] == wanted:
+            return digest
+    return repo_digests[0].split("@", 1)[1] if repo_digests else None
 
 
 # --------------------------------------------------------------------------- #
@@ -236,7 +280,20 @@ async def remote_digest(image: str) -> Optional[str]:
 
 
 async def check_image(image: str) -> UpdateStatus:
-    local = _local_digest(image)
+    attrs = _local_attrs(image)
+    if _built_locally(attrs):
+        status = UpdateStatus(
+            image=image,
+            update_available=False,
+            current_digest=None,
+            remote_digest=None,
+            checked_at=time.time(),
+            local=True,
+        )
+        _NOTIFIED.discard(image)
+        _store(status, False)
+        return status
+    local = _local_digest(image, attrs)
     error = None
     try:
         remote = await remote_digest(image)
@@ -294,12 +351,15 @@ def _all_running_images() -> set[str]:
 
 def stack_images(stack_id: str) -> set[str]:
     """Images used by the containers of one compose project (== stack id)."""
+    from services.compose_service import project_name
+
+    project = project_name(stack_id)
     images: set[str] = set()
     try:
         client = get_client()
         for c in safe_call(client.containers.list, all=True):
             labels = c.labels or {}
-            if labels.get("com.docker.compose.project") != stack_id:
+            if labels.get("com.docker.compose.project") != project:
                 continue
             cfg_image = c.attrs.get("Config", {}).get("Image")
             if cfg_image:
@@ -380,6 +440,9 @@ def refresh_stack_local(stack_id: str) -> None:
         status.checked_at = time.time()
         if not status.update_available:
             _NOTIFIED.discard(image)
+        # Persisted too, or a restart brought the cleared badge back until the
+        # next background check.
+        _store(status, image in _NOTIFIED)
 
 
 async def check_all() -> dict[str, dict]: