Fix what a review of the whole app found, end to end (0.67.1)
A pass over every feature against a real Docker daemon, then clicking through the UI as admin and read-only user, on a desktop and a phone. Security. /ws/deploy ran compose up for the read-only role; it is admin-only now. A container's inspect view, a stack's compose file (POSTGRES_PASSWORD: hunter2) and the viewer's .env tab no longer show credentials to that role. Git branch and URL values that git would read as options (-x, ext::) are refused. SFTP host keys are checked: pinned on first use or entered by hand (key line or SHA256 fingerprint), a different key is refused before the password is sent. A backup name from the API can no longer reach outside the destination folder. The file browser refuses to move or rename a browse root, as delete did. Docker lifecycle. Closing the deploy console released the stack lock while compose was still running and logged rc=None as success; deploys and updates are now background runs that hold the lock to the end, report the real exit code and can be re-attached to. Every closed log view left a `compose logs -f` behind. A dozen open tabs exhausted the default thread pool and hung the API; event and exec streams have their own threads. Auto-update and the update endpoint ignored failed pulls. A deleted git stack was recreated and started by the next webhook or poll; deleting a stack now drops its git source, auto-update policy and schedule entries. Deleting no longer walks into mounted shares, is refused while a backup runs, and removes the containers even when the compose file is broken. Stack folders compose has to normalise (Nextcloud -> nextcloud) work everywhere and group correctly on the volumes, networks and images pages. Detection. Host port conflicts read the container's own network namespace; /proc/1/net is read instead. Images referenced as docker.io/… were never update-checked; local builds no longer report "needs credentials". NVIDIA GPUs are read from the driver's procfs. The backup dialog's "path mismatch" warning came from mountinfo's filesystem-relative root and fired for any stacks folder on its own disk, ZFS dataset or btrfs subvolume; the host path now comes from Docker's mounts. Backups and destinations. The download ignored the folder/volume selection (axios sent binds[]). Restoring under a new name kept the old one. A report header past nginx's buffer turned a good backup into a 502. SFTP/S3 errors were bare 500s and "Test" did not try to write. A destination used by a schedule could be deleted. Backup file names are in the schedule's or the browser's time zone instead of UTC. Editor and converter. `docker run` conversion produced invalid compose for --network host, named volumes and -dit; --gpus and --mount were dropped; a multi-line command pasted into the one-line field became `image: ' '` (now a textarea, and flattened continuations are read). Validation failed for env_file stacks. The editor overwrote unsaved edits on every container event, lost them on a stray click (now asks), threw when closing the diff, and its helper panel listed the starter service "app" for every existing stack. Monaco is bundled instead of loaded from jsdelivr, so the editor works offline (and monaco-editor 0.57 drops the dompurify advisory; axios and react-router updated). UI. All times were two hours off: offset-less UTC from SQLite was read as local time, and "next run" always said "just now". A token refresh mid-session duplicated deploys, reset terminals and doubled logs. A backend restart (update, self-update) signed everyone out on reload; the page now waits for it. The editor was unusable on a phone, tables and tabs overflowed, dialog text was black on the dark theme, and the file browser opened on an error with the default roots. Read-only users get "Admin only" pages instead of 403s; in-use networks offer no delete; "Scan all" covers StackPilot's own stacks only and runs in the background instead of timing out at 600s. Deployment. nginx passes the port in Host (the OIDC redirect URI lost :5009) and keeps an outer proxy's X-Forwarded-Proto. The frontend waits for a healthy backend (healthcheck probes every 2s while starting). The frontend image builds with npm ci. Secrets are 0644 in a 0700 folder so non-root containers can read them; existing files are fixed at startup. The test suite no longer starts real containers on a host with Docker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fx7gcuy8d6nPd95g43rGHu
This commit is contained in:
menzeljandClaude Opus 5.5 committed
1 parent
270ddda51d
commit
d6cebb3371
88 files changed
+4752
-925
No files matched your search
@@ -30,6 +30,14 @@ os.environ["STACKS_DIR"] = os.path.join(_TMP, "stacks")
|
||||
os.environ["SECRET_KEY"] = "test-secret-key-not-used-anywhere-real"
|
||||
os.environ["ALLOWED_BROWSE_ROOTS"] = "/mnt,/media,/srv,/opt,/home"
|
||||
os.environ["HOST_ROOT_PREFIX"] = ""
|
||||
# No Docker, even where there is one. The suite is written for a host without a
|
||||
# daemon, and where one *is* reachable — a dev box, or the CI runner, which gets
|
||||
# the host's socket — tests that go "far enough in" really ran `compose up` and
|
||||
# left containers running on that host after every run. docker-py reads
|
||||
# DOCKER_SOCKET, the docker CLI (compose) reads DOCKER_HOST; both point nowhere.
|
||||
_NO_DOCKER = os.path.join(_TMP, "no-docker.sock")
|
||||
os.environ["DOCKER_SOCKET"] = _NO_DOCKER
|
||||
os.environ["DOCKER_HOST"] = f"unix://{_NO_DOCKER}"
|
||||
|
||||
# The backend runs from its own root at runtime (`uvicorn main:app` with
|
||||
# /app as the workdir), so make the same layout importable here.
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
"""Streamed compose runs: the deploy console and the update bar.
|
||||
|
||||
Until 0.67.1 the run *was* the WebSocket handler, so closing the console in the
|
||||
middle of a deploy released the stack lock at once — while compose went on —
|
||||
and logged the deploy as a success with ``rc=None``. The run is its own task
|
||||
now; a socket only watches it. And the deploy socket is admin-only, like the
|
||||
REST ``/start`` it mirrors: before, the read-only role could run ``compose up``
|
||||
through it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
|
||||
import pytest
|
||||
from sqlmodel import Session
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_locks(db):
|
||||
from sqlmodel import delete
|
||||
|
||||
from database import engine
|
||||
from models.runtime_state import StackLock
|
||||
|
||||
with Session(engine) as session:
|
||||
session.exec(delete(StackLock))
|
||||
session.commit()
|
||||
yield
|
||||
|
||||
|
||||
def _locked(stack_id: str) -> bool:
|
||||
from database import engine
|
||||
from services import stack_lock_service
|
||||
|
||||
with Session(engine) as session:
|
||||
return stack_lock_service.is_busy(session, stack_id)
|
||||
|
||||
|
||||
def test_the_lock_is_held_until_compose_finishes(db):
|
||||
from services import compose_run_service as runs
|
||||
|
||||
async def scenario():
|
||||
release = asyncio.Event()
|
||||
finished: list = []
|
||||
|
||||
async def stream():
|
||||
yield ("log", "pulling")
|
||||
await release.wait()
|
||||
yield ("log", "started")
|
||||
yield ("done", 0)
|
||||
|
||||
async def on_finish(rc):
|
||||
finished.append(rc)
|
||||
|
||||
run, queue, attached = runs.start("rx", "start", "alice", stream, on_finish)
|
||||
assert not attached
|
||||
assert await queue.get() == ("log", "pulling")
|
||||
# Nobody is watching any more — the console was closed.
|
||||
run.unsubscribe(queue)
|
||||
await asyncio.sleep(0.05)
|
||||
assert _locked("rx"), "the lock must outlive the socket"
|
||||
assert finished == []
|
||||
|
||||
release.set()
|
||||
for _ in range(50):
|
||||
if finished:
|
||||
break
|
||||
await asyncio.sleep(0.02)
|
||||
assert finished == [0], "the real exit code, after the fact"
|
||||
assert not _locked("rx")
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_second_socket_attaches_to_the_run_in_flight(db):
|
||||
from services import compose_run_service as runs
|
||||
|
||||
async def scenario():
|
||||
release = asyncio.Event()
|
||||
|
||||
async def stream():
|
||||
yield ("log", "one")
|
||||
await release.wait()
|
||||
yield ("done", 0)
|
||||
|
||||
run, first, _ = runs.start("ry", "start", "alice", stream)
|
||||
assert await first.get() == ("log", "one")
|
||||
|
||||
again, second, attached = runs.start("ry", "start", "bob", stream)
|
||||
assert attached and again is run
|
||||
assert await second.get() == ("log", "one"), "backlog first"
|
||||
|
||||
with pytest.raises(runs.RunBusy):
|
||||
runs.start("ry", "update", "carol", stream)
|
||||
|
||||
release.set()
|
||||
items = []
|
||||
while (item := await second.get()) is not None:
|
||||
items.append(item)
|
||||
assert items == [("done", 0)]
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_run_that_cannot_get_the_lock_is_busy(db):
|
||||
from database import engine
|
||||
from services import compose_run_service as runs
|
||||
from services import stack_lock_service
|
||||
|
||||
with Session(engine) as session:
|
||||
stack_lock_service.acquire(session, "rz", "backup", "scheduled")
|
||||
|
||||
async def stream():
|
||||
yield ("done", 0)
|
||||
|
||||
async def scenario():
|
||||
with pytest.raises(runs.RunBusy) as excinfo:
|
||||
runs.start("rz", "start", "alice", stream)
|
||||
assert excinfo.value.action == "backup"
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_a_failing_stream_still_releases_the_lock(db):
|
||||
from services import compose_run_service as runs
|
||||
|
||||
async def scenario():
|
||||
async def stream():
|
||||
yield ("log", "about to break")
|
||||
raise RuntimeError("compose binary missing")
|
||||
|
||||
finished = []
|
||||
|
||||
async def on_finish(rc):
|
||||
finished.append(rc)
|
||||
|
||||
_run, queue, _ = runs.start("rf", "update", "alice", stream, on_finish)
|
||||
items = []
|
||||
while (item := await queue.get()) is not None:
|
||||
items.append(item)
|
||||
await asyncio.sleep(0.05)
|
||||
assert items[-1][0] == "done" and items[-1][1] != 0
|
||||
assert finished and finished[0] != 0
|
||||
assert not _locked("rf")
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_the_deploy_socket_is_admin_only(client, user_token):
|
||||
from starlette.websockets import WebSocketDisconnect
|
||||
|
||||
with pytest.raises(WebSocketDisconnect) as excinfo:
|
||||
with client.websocket_connect(f"/ws/deploy/anything?token={user_token}") as socket:
|
||||
socket.receive_text()
|
||||
assert excinfo.value.code == 4403
|
||||
|
||||
|
||||
def test_deploying_an_unknown_stack_is_refused(client, admin_token):
|
||||
from starlette.websockets import WebSocketDisconnect
|
||||
|
||||
with pytest.raises(WebSocketDisconnect) as excinfo:
|
||||
with client.websocket_connect(f"/ws/deploy/no-such-stack?token={admin_token}") as socket:
|
||||
frame = json.loads(socket.receive_text())
|
||||
assert frame["type"] == "error"
|
||||
socket.receive_text()
|
||||
assert excinfo.value.code == 4404
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# The REST paths
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def test_update_stops_at_a_failed_pull(monkeypatch):
|
||||
"""`up` after a failed pull recreated nothing and reported success."""
|
||||
from services import compose_service
|
||||
|
||||
calls = []
|
||||
|
||||
async def fake(stack_id, args, override=None, timeout=600.0):
|
||||
calls.append(args[0])
|
||||
rc = 1 if args[0] == "pull" else 0
|
||||
return {"returncode": rc, "stdout": "", "stderr": "denied", "command": args[0]}
|
||||
|
||||
monkeypatch.setattr(compose_service, "run_compose", fake)
|
||||
result = asyncio.run(compose_service.update("s"))
|
||||
assert calls == ["pull"]
|
||||
assert result["returncode"] == 1
|
||||
|
||||
|
||||
def test_auto_update_reports_a_failed_compose_run(db, monkeypatch):
|
||||
"""The exit codes used to be ignored: a failure was recorded as "updated"."""
|
||||
from database import engine
|
||||
from models.auto_update import AutoUpdate
|
||||
from services import auto_update_service, compose_service, update_service
|
||||
|
||||
monkeypatch.setattr(compose_service, "compute_status", lambda stack_id: "running")
|
||||
|
||||
async def stale(stack_id, refresh=False):
|
||||
return {"update_available": True, "stale_images": ["nginx:latest"]}
|
||||
|
||||
async def failing_update(stack_id, override=None):
|
||||
return {"returncode": 1, "stdout": "", "stderr": "Error: pull access denied"}
|
||||
|
||||
notified = []
|
||||
|
||||
async def notify(event, title, message, session=None):
|
||||
notified.append(title)
|
||||
|
||||
monkeypatch.setattr(update_service, "stack_updates", stale)
|
||||
monkeypatch.setattr(compose_service, "update", failing_update)
|
||||
monkeypatch.setattr(auto_update_service.notify_service, "notify", notify)
|
||||
|
||||
with Session(engine) as session:
|
||||
policy = AutoUpdate(stack_id="au-fail", enabled=True, redeploy=True)
|
||||
session.add(policy)
|
||||
session.commit()
|
||||
session.refresh(policy)
|
||||
asyncio.run(auto_update_service.run_policy(session, policy))
|
||||
session.refresh(policy)
|
||||
assert policy.last_status == "error"
|
||||
assert "pull access denied" in (policy.last_result or "")
|
||||
session.delete(policy)
|
||||
session.commit()
|
||||
assert notified and "failed" in notified[0]
|
||||
@@ -137,16 +137,38 @@ def test_secret_names_reject_traversal_and_hidden_files(name):
|
||||
secret_service._check_name(name)
|
||||
|
||||
|
||||
def test_secret_files_are_written_owner_only(tmp_path):
|
||||
def test_secret_files_are_guarded_by_their_directory(tmp_path):
|
||||
"""The 0700 directory keeps other host users out; the file itself has to be
|
||||
world-readable, because compose bind-mounts it into the container as it is
|
||||
and a container running as postgres or PUID 1000 could not open a 0600
|
||||
root file."""
|
||||
from services import secret_service
|
||||
|
||||
secret_service.write_secret("s", "secret", "db_password", "hunter2", override=str(tmp_path))
|
||||
path = tmp_path / "s" / ".secrets" / "db_password"
|
||||
assert path.read_text() == "hunter2"
|
||||
assert oct(path.stat().st_mode)[-3:] == "600"
|
||||
assert oct(path.stat().st_mode)[-3:] == "644"
|
||||
assert oct(path.parent.stat().st_mode)[-3:] == "700"
|
||||
|
||||
|
||||
def test_secrets_from_older_versions_are_made_readable(tmp_path):
|
||||
import os
|
||||
|
||||
from services import secret_service
|
||||
|
||||
folder = tmp_path / "old" / ".secrets"
|
||||
folder.mkdir(parents=True)
|
||||
old = folder / "pw"
|
||||
old.write_text("x")
|
||||
os.chmod(old, 0o600)
|
||||
os.chmod(folder, 0o755)
|
||||
|
||||
assert secret_service.fix_permissions(str(tmp_path)) == 1
|
||||
assert oct(old.stat().st_mode)[-3:] == "644"
|
||||
assert oct(folder.stat().st_mode)[-3:] == "700"
|
||||
assert secret_service.fix_permissions(str(tmp_path)) == 0, "idempotent"
|
||||
|
||||
|
||||
def test_listing_secrets_never_returns_their_content(tmp_path):
|
||||
from services import secret_service
|
||||
|
||||
|
||||
@@ -0,0 +1,167 @@
|
||||
"""Converting `docker run` into compose.
|
||||
|
||||
The output has to be a compose file compose accepts and that means what the
|
||||
command meant. Before 0.67.1 a boolean flag the converter did not know swallowed
|
||||
the image name (``-dit ubuntu bash`` became a service running ``bash``),
|
||||
``--network host`` and ``-v data:/data`` produced files compose rejects, and
|
||||
``--gpus`` or ``--mount`` were dropped without a word.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
|
||||
def _service(command: str) -> tuple[str, dict, dict]:
|
||||
from services.convert_service import convert_docker_run
|
||||
|
||||
doc = yaml.safe_load(convert_docker_run(command))
|
||||
((name, service),) = doc["services"].items()
|
||||
return name, service, doc
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"command, image",
|
||||
[
|
||||
("docker run -dit ubuntu bash", "ubuntu"),
|
||||
("docker run -d --read-only nginx", "nginx"),
|
||||
("docker run -P nginx", "nginx"),
|
||||
("docker run --rm --init --privileged alpine", "alpine"),
|
||||
("docker run -d --oom-kill-disable --no-healthcheck redis:7", "redis:7"),
|
||||
],
|
||||
)
|
||||
def test_boolean_flags_never_swallow_the_image(command, image):
|
||||
_name, service, _doc = _service(command)
|
||||
assert service["image"] == image
|
||||
|
||||
|
||||
def test_the_command_after_the_image_is_kept():
|
||||
name, service, _doc = _service("docker run -it --rm ubuntu bash -lc 'echo hi'")
|
||||
assert name == "ubuntu"
|
||||
assert service["command"] == ["bash", "-lc", "echo hi"]
|
||||
assert service["stdin_open"] is True and service["tty"] is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mode", ["host", "none", "bridge", "container:vpn"])
|
||||
def test_special_networks_become_network_mode(mode):
|
||||
_name, service, doc = _service(f"docker run -d --network {mode} alpine")
|
||||
assert service["network_mode"] == mode
|
||||
assert "networks" not in service and "networks" not in doc
|
||||
|
||||
|
||||
def test_a_named_network_is_declared_external():
|
||||
_name, service, doc = _service("docker run -d --net=proxy --network-alias web alpine")
|
||||
assert service["networks"] == {"proxy": {"aliases": ["web"]}}
|
||||
assert doc["networks"] == {"proxy": {"external": True}}
|
||||
|
||||
|
||||
def test_named_volumes_are_declared_and_paths_are_not():
|
||||
_name, service, doc = _service(
|
||||
"docker run -d -v data:/data -v /srv/media:/media:ro -v ./conf:/conf alpine"
|
||||
)
|
||||
assert service["volumes"] == ["data:/data", "/srv/media:/media:ro", "./conf:/conf"]
|
||||
assert doc["volumes"] == {"data": None}
|
||||
|
||||
|
||||
def test_mount_becomes_long_syntax():
|
||||
_name, service, doc = _service(
|
||||
"docker run --mount type=bind,source=/srv,target=/srv,readonly "
|
||||
"--mount type=volume,src=cache,dst=/cache alpine"
|
||||
)
|
||||
assert service["volumes"][0] == {
|
||||
"type": "bind", "source": "/srv", "target": "/srv", "read_only": True,
|
||||
}
|
||||
assert service["volumes"][1] == {"type": "volume", "source": "cache", "target": "/cache"}
|
||||
assert doc["volumes"] == {"cache": None}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value, expected",
|
||||
[
|
||||
("all", {"count": "all"}),
|
||||
("2", {"count": 2}),
|
||||
('"device=0,1"', {"device_ids": ["0", "1"]}),
|
||||
],
|
||||
)
|
||||
def test_gpus_become_a_device_reservation(value, expected):
|
||||
_name, service, _doc = _service(f"docker run -d --gpus {value} ollama/ollama")
|
||||
(device,) = service["deploy"]["resources"]["reservations"]["devices"]
|
||||
assert device["driver"] == "nvidia"
|
||||
assert device["capabilities"] == ["gpu"]
|
||||
for key, val in expected.items():
|
||||
assert device[key] == val
|
||||
|
||||
|
||||
def test_attached_short_values_and_equals_forms():
|
||||
_name, service, _doc = _service(
|
||||
"docker run -p8080:80 -eTZ=Europe/Berlin --restart=always --name=web nginx"
|
||||
)
|
||||
assert service["ports"] == ["8080:80"]
|
||||
assert service["environment"] == ["TZ=Europe/Berlin"]
|
||||
assert service["restart"] == "always"
|
||||
assert service["container_name"] == "web"
|
||||
|
||||
|
||||
def test_common_runtime_flags_are_kept():
|
||||
_name, service, _doc = _service(
|
||||
"sudo docker run -d --name plex --entrypoint /init --shm-size 1g -m 2g --cpus 1.5 "
|
||||
"--cap-add NET_ADMIN --cap-drop ALL --security-opt no-new-privileges "
|
||||
"--sysctl net.ipv4.ip_forward=1 --add-host nas:10.0.0.2 --dns 1.1.1.1 "
|
||||
"--ulimit nofile=1024:2048 --log-opt max-size=10m --tmpfs /run "
|
||||
"--health-cmd 'curl -f http://localhost' --health-interval 30s --stop-timeout 30 "
|
||||
"lscr.io/linuxserver/plex:latest"
|
||||
)
|
||||
assert service["entrypoint"] == "/init"
|
||||
assert service["shm_size"] == "1g"
|
||||
assert service["mem_limit"] == "2g"
|
||||
assert service["cpus"] == 1.5
|
||||
assert service["cap_add"] == ["NET_ADMIN"] and service["cap_drop"] == ["ALL"]
|
||||
assert service["security_opt"] == ["no-new-privileges"]
|
||||
assert service["sysctls"] == ["net.ipv4.ip_forward=1"]
|
||||
assert service["extra_hosts"] == ["nas:10.0.0.2"]
|
||||
assert service["dns"] == ["1.1.1.1"]
|
||||
assert service["ulimits"] == {"nofile": {"soft": 1024, "hard": 2048}}
|
||||
assert service["logging"] == {"options": {"max-size": "10m"}}
|
||||
assert service["tmpfs"] == ["/run"]
|
||||
assert service["healthcheck"] == {
|
||||
"test": ["CMD-SHELL", "curl -f http://localhost"], "interval": "30s",
|
||||
}
|
||||
assert service["stop_grace_period"] == "30s"
|
||||
|
||||
|
||||
def test_line_continuations_from_a_readme():
|
||||
name, service, _doc = _service("docker run -d \\\n --name x \\\r\n -p 1:1 \\\n alpine")
|
||||
assert name == "x"
|
||||
assert service["ports"] == ["1:1"]
|
||||
|
||||
|
||||
def test_the_service_name_comes_from_the_image_without_registry_or_tag():
|
||||
name, _service_, _doc = _service("docker run -d ghcr.io/home-assistant/home-assistant:stable")
|
||||
assert name == "home-assistant"
|
||||
|
||||
|
||||
def test_no_image_is_an_error():
|
||||
from services.convert_service import convert_docker_run
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
convert_docker_run("docker run -d --name x")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"command",
|
||||
[
|
||||
# Pasted into a one-line field: the newlines are gone, the backslashes
|
||||
# and indentation are not.
|
||||
"docker run -d \\ --name web \\ -p 8081:80 \\ -v data:/data \\ nginx:alpine",
|
||||
"docker run -d \\ --name web \\ -p 8081:80 \\\n nginx:alpine",
|
||||
],
|
||||
)
|
||||
def test_continuations_that_lost_their_newline(command):
|
||||
name, web, _ = _service(command)
|
||||
assert name == "web" and web["image"] == "nginx:alpine"
|
||||
assert web["ports"] == ["8081:80"]
|
||||
|
||||
|
||||
def test_an_escaped_space_in_a_path_is_still_one():
|
||||
_, service, _ = _service("docker run -v /mnt/My\\ Files:/data nginx")
|
||||
assert service["volumes"] == ["/mnt/My Files:/data"]
|
||||
@@ -0,0 +1,276 @@
|
||||
"""Backup destinations: errors that say what happened, and names that stay names.
|
||||
|
||||
An SFTP host that refused the connection was a bare 500 (paramiko connects in
|
||||
the Transport constructor, outside every handler), so the "Test" button showed
|
||||
nothing useful. And a backup file name from the API went straight into
|
||||
``posixpath.join`` — ``../`` let a delete reach outside the backup folder.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import socket
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["../etc/passwd", "a/b", "..", "", "x\\y", "a\0b"])
|
||||
def test_backup_names_must_be_bare_file_names(name):
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
with pytest.raises(dest.DestinationError):
|
||||
dest._check_file_name(name)
|
||||
|
||||
|
||||
def test_a_real_backup_name_passes():
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
dest._check_file_name("backup-immich-full-20260101-030000.tar.gz")
|
||||
|
||||
|
||||
def _free_port() -> int:
|
||||
with socket.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
return s.getsockname()[1]
|
||||
|
||||
|
||||
def _destination(kind: str, config: dict):
|
||||
from models.backup_destination import BackupDestination
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
return BackupDestination(id=99, name="t", type=kind, config=dest.dump_config(config))
|
||||
|
||||
|
||||
def test_an_unreachable_sftp_host_is_a_destination_error():
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
d = _destination(
|
||||
"sftp", {"host": "127.0.0.1", "port": _free_port(), "username": "u", "password": "p"}
|
||||
)
|
||||
with pytest.raises(dest.DestinationError, match="SFTP connection"):
|
||||
dest.test(d)
|
||||
|
||||
|
||||
def test_sftp_delete_refuses_a_path_before_connecting():
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
d = _destination("sftp", {"host": "198.51.100.1", "username": "u"})
|
||||
with pytest.raises(dest.DestinationError, match="Invalid backup file name"):
|
||||
dest.delete(d, "../../important")
|
||||
|
||||
|
||||
def test_a_malformed_s3_endpoint_is_a_destination_error():
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
d = _destination("s3", {"bucket": "b", "endpoint_url": "not a url"})
|
||||
with pytest.raises(dest.DestinationError, match="Invalid S3 settings"):
|
||||
dest.test(d)
|
||||
|
||||
|
||||
def test_the_test_endpoint_reports_instead_of_failing(as_admin, db):
|
||||
from sqlmodel import Session
|
||||
|
||||
from database import engine
|
||||
from models.backup_destination import BackupDestination
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
with Session(engine) as session:
|
||||
row = BackupDestination(
|
||||
name="refusing",
|
||||
type="sftp",
|
||||
config=dest.dump_config(
|
||||
{"host": "127.0.0.1", "port": _free_port(), "username": "u", "password": "p"}
|
||||
),
|
||||
)
|
||||
session.add(row)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
dest_id = row.id
|
||||
try:
|
||||
response = as_admin.post(f"/api/backups/destinations/{dest_id}/test")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["ok"] is False
|
||||
assert "SFTP connection" in response.json()["error"]
|
||||
finally:
|
||||
with Session(engine) as session:
|
||||
session.delete(session.get(BackupDestination, dest_id))
|
||||
session.commit()
|
||||
|
||||
|
||||
def test_a_destination_in_use_by_a_schedule_cannot_be_deleted(as_admin, db):
|
||||
"""Deleted from under it, the schedule failed on every run from then on."""
|
||||
from sqlmodel import Session
|
||||
|
||||
from database import engine
|
||||
from models.backup_destination import BackupDestination
|
||||
from models.backup_schedule import BackupSchedule
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
with Session(engine) as session:
|
||||
row = BackupDestination(name="nas", type="sftp", config=dest.dump_config({"host": "x"}))
|
||||
session.add(row)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
schedule = BackupSchedule(destination_id=row.id, all_stacks=True)
|
||||
session.add(schedule)
|
||||
session.commit()
|
||||
session.refresh(schedule)
|
||||
dest_id, schedule_id = row.id, schedule.id
|
||||
try:
|
||||
response = as_admin.delete(f"/api/backups/destinations/{dest_id}")
|
||||
assert response.status_code == 409
|
||||
assert "schedule" in response.json()["detail"]
|
||||
finally:
|
||||
with Session(engine) as session:
|
||||
session.delete(session.get(BackupSchedule, schedule_id))
|
||||
session.commit()
|
||||
assert as_admin.delete(f"/api/backups/destinations/{dest_id}").status_code == 200
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# SFTP host keys: a real SSH handshake against an in-process paramiko server.
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
class _SshServer:
|
||||
"""One-connection SSH server with an SFTP subsystem and a known host key."""
|
||||
|
||||
def __init__(self, host_key):
|
||||
import threading
|
||||
|
||||
import paramiko
|
||||
|
||||
self.host_key = host_key
|
||||
self.passwords: list[str] = []
|
||||
outer = self
|
||||
|
||||
class Iface(paramiko.ServerInterface):
|
||||
def get_allowed_auths(self, username):
|
||||
return "password"
|
||||
|
||||
def check_auth_password(self, username, password):
|
||||
outer.passwords.append(password)
|
||||
return paramiko.AUTH_SUCCESSFUL
|
||||
|
||||
def check_channel_request(self, kind, chanid):
|
||||
return paramiko.OPEN_SUCCEEDED
|
||||
|
||||
self._iface = Iface
|
||||
self.sock = socket.socket()
|
||||
self.sock.bind(("127.0.0.1", 0))
|
||||
self.sock.listen(1)
|
||||
self.port = self.sock.getsockname()[1]
|
||||
threading.Thread(target=self._run, daemon=True).start()
|
||||
|
||||
def _run(self):
|
||||
import paramiko
|
||||
|
||||
conn, _ = self.sock.accept()
|
||||
transport = paramiko.Transport(conn)
|
||||
transport.add_server_key(self.host_key)
|
||||
transport.set_subsystem_handler("sftp", paramiko.SFTPServer, paramiko.SFTPServerInterface)
|
||||
try:
|
||||
transport.start_server(server=self._iface())
|
||||
# Kept: a channel that is garbage collected closes itself, and the
|
||||
# client saw "Channel closed" in the middle of the SFTP handshake.
|
||||
self.channel = transport.accept(10)
|
||||
while transport.is_active():
|
||||
transport.join(0.1)
|
||||
except Exception: # noqa: BLE001 - the client hung up on purpose
|
||||
pass
|
||||
finally:
|
||||
self.sock.close()
|
||||
|
||||
def cfg(self, **extra):
|
||||
return {"host": "127.0.0.1", "port": self.port, "username": "u", "password": "pw", **extra}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def host_key():
|
||||
import paramiko
|
||||
|
||||
return paramiko.ECDSAKey.generate()
|
||||
|
||||
|
||||
def _connect(cfg, pin=None):
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
sftp, transport = dest._sftp_connect(cfg, pin)
|
||||
sftp.close()
|
||||
transport.close()
|
||||
|
||||
|
||||
def test_the_first_host_key_is_pinned(host_key):
|
||||
server = _SshServer(host_key)
|
||||
pinned = []
|
||||
_connect(server.cfg(), pinned.append)
|
||||
assert pinned == [f"{host_key.get_name()} {host_key.get_base64()}"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("form", ["line", "known_hosts", "fingerprint"])
|
||||
def test_a_pinned_key_lets_its_server_in(host_key, form):
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
line = f"{host_key.get_name()} {host_key.get_base64()}"
|
||||
stored = {
|
||||
"line": line,
|
||||
"known_hosts": f"[nas]:2222 {line}",
|
||||
"fingerprint": dest.host_key_fingerprint(host_key),
|
||||
}[form]
|
||||
server = _SshServer(host_key)
|
||||
pinned = []
|
||||
_connect(server.cfg(host_key=stored), pinned.append)
|
||||
assert pinned == [] # nothing new to pin
|
||||
assert server.passwords == ["pw"]
|
||||
|
||||
|
||||
def test_another_key_is_refused_before_the_password_is_sent(host_key):
|
||||
import paramiko
|
||||
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
impostor = _SshServer(host_key)
|
||||
real = paramiko.ECDSAKey.generate()
|
||||
with pytest.raises(dest.DestinationError, match="does not match"):
|
||||
_connect(impostor.cfg(host_key=f"{real.get_name()} {real.get_base64()}"))
|
||||
assert impostor.passwords == []
|
||||
|
||||
|
||||
def test_the_fingerprint_of_a_stored_key_is_the_one_ssh_keygen_prints(host_key):
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
line = f"{host_key.get_name()} {host_key.get_base64()}"
|
||||
assert dest.fingerprint_of(line) == dest.host_key_fingerprint(host_key)
|
||||
assert dest.fingerprint_of("SHA256:abc") == "SHA256:abc"
|
||||
|
||||
|
||||
def test_moving_a_destination_to_another_host_forgets_its_key(as_admin, db):
|
||||
from sqlmodel import Session
|
||||
|
||||
from database import engine
|
||||
from models.backup_destination import BackupDestination
|
||||
from services import backup_destination_service as dest
|
||||
|
||||
with Session(engine) as session:
|
||||
row = BackupDestination(
|
||||
name="moving",
|
||||
type="sftp",
|
||||
config=dest.dump_config({"host": "old", "username": "u", "host_key": "ssh-ed25519 AAAA"}),
|
||||
)
|
||||
session.add(row)
|
||||
session.commit()
|
||||
session.refresh(row)
|
||||
dest_id = row.id
|
||||
try:
|
||||
# The form sends every field back, the shown host key included.
|
||||
same = {"host": "old", "username": "u", "host_key": "ssh-ed25519 AAAA", "path": "/b"}
|
||||
as_admin.put(f"/api/backups/destinations/{dest_id}", json={"config": same})
|
||||
moved = {**same, "host": "new"}
|
||||
assert as_admin.put(
|
||||
f"/api/backups/destinations/{dest_id}", json={"config": moved}
|
||||
).status_code == 200
|
||||
with Session(engine) as session:
|
||||
cfg = dest.parse_config(session.get(BackupDestination, dest_id))
|
||||
assert cfg["host"] == "new" and "host_key" not in cfg
|
||||
finally:
|
||||
with Session(engine) as session:
|
||||
session.delete(session.get(BackupDestination, dest_id))
|
||||
session.commit()
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Image references, compose project names and published ports.
|
||||
|
||||
Small parsers with large consequences:
|
||||
|
||||
* ``docker.io/library/postgres`` is how many compose files name an image, and
|
||||
https://docker.io/v2/ is not a registry — so every such image sat at "could
|
||||
not reach registry" forever and an update never showed up.
|
||||
* compose refuses ``-p`` with anything but ``[a-z0-9_-]``, so a stack folder
|
||||
found on disk as ``Nextcloud`` could not be started, stopped or deleted.
|
||||
* the host's listening ports live in the host's network namespace, which
|
||||
``/proc/net`` read from inside the container is not.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"image, expected",
|
||||
[
|
||||
("nginx", ("registry-1.docker.io", "library/nginx", "latest")),
|
||||
("docker.io/nginx:1", ("registry-1.docker.io", "library/nginx", "1")),
|
||||
("docker.io/library/postgres:16", ("registry-1.docker.io", "library/postgres", "16")),
|
||||
("index.docker.io/foo/bar:2", ("registry-1.docker.io", "foo/bar", "2")),
|
||||
("ghcr.io/a/b:1@sha256:abc", ("ghcr.io", "a/b", "sha256:abc")),
|
||||
("localhost:5000/x", ("localhost:5000", "x", "latest")),
|
||||
("registry.example:5000/a/b:tag", ("registry.example:5000", "a/b", "tag")),
|
||||
],
|
||||
)
|
||||
def test_parse_ref(image, expected):
|
||||
from services.update_service import parse_ref
|
||||
|
||||
assert parse_ref(image) == expected
|
||||
|
||||
|
||||
def test_the_local_digest_is_the_one_for_this_repository(monkeypatch):
|
||||
"""An image pulled under two names carries a digest per repository."""
|
||||
from services import update_service
|
||||
|
||||
class Image:
|
||||
attrs = {"RepoDigests": [
|
||||
"lscr.io/linuxserver/radarr@sha256:aaa",
|
||||
"linuxserver/radarr@sha256:bbb",
|
||||
]}
|
||||
|
||||
class Images:
|
||||
def get(self, name):
|
||||
return Image()
|
||||
|
||||
class Client:
|
||||
images = Images()
|
||||
|
||||
monkeypatch.setattr(update_service, "get_client", lambda: Client())
|
||||
assert update_service._local_digest("linuxserver/radarr:latest") == "sha256:bbb"
|
||||
assert update_service._local_digest("lscr.io/linuxserver/radarr:latest") == "sha256:aaa"
|
||||
assert update_service._local_digest("docker.io/linuxserver/radarr") == "sha256:bbb"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"stack_id, project",
|
||||
[
|
||||
("immich", "immich"),
|
||||
("arr_stack-2", "arr_stack-2"),
|
||||
("Nextcloud", "nextcloud"),
|
||||
("my.app", "myapp"),
|
||||
("_hidden-ish", "hidden-ish"),
|
||||
("...", "stack"),
|
||||
],
|
||||
)
|
||||
def test_project_name_matches_compose(stack_id, project):
|
||||
from services.compose_service import project_name
|
||||
|
||||
assert project_name(stack_id) == project
|
||||
|
||||
|
||||
def test_compose_is_called_with_the_normalised_project():
|
||||
from services import compose_service
|
||||
|
||||
cmd = compose_service._compose_base_cmd("Nextcloud")
|
||||
assert cmd[cmd.index("-p") + 1] == "nextcloud"
|
||||
|
||||
|
||||
def test_published_ports_including_ranges_and_long_syntax():
|
||||
from services.port_service import parse_compose_ports
|
||||
|
||||
ports = parse_compose_ports(
|
||||
"services:\n a:\n ports:\n"
|
||||
' - "8080:80"\n'
|
||||
' - "127.0.0.1:9000:9000/udp"\n'
|
||||
' - "7000-7002:7000-7002"\n'
|
||||
' - "80"\n'
|
||||
' - {target: 80, published: "8443"}\n'
|
||||
' - "[::1]:5353:53/udp"\n'
|
||||
)
|
||||
assert [(p["host_port"], p["protocol"]) for p in ports] == [
|
||||
(8080, "tcp"), (9000, "udp"), (7000, "tcp"), (7001, "tcp"), (7002, "tcp"),
|
||||
(8443, "tcp"), (5353, "udp"),
|
||||
]
|
||||
|
||||
|
||||
def test_garbage_compose_yields_no_ports():
|
||||
from services.port_service import parse_compose_ports
|
||||
|
||||
assert parse_compose_ports("just a string") == []
|
||||
assert parse_compose_ports("services: [1, 2]") == []
|
||||
assert parse_compose_ports(": : :") == []
|
||||
|
||||
|
||||
def test_host_ports_are_read_from_the_hosts_namespace(tmp_path, monkeypatch):
|
||||
"""/proc/net is self/net; PID 1 of the host /proc is the host's namespace."""
|
||||
from config import settings
|
||||
from services import port_service
|
||||
|
||||
(tmp_path / "net").mkdir()
|
||||
(tmp_path / "1" / "net").mkdir(parents=True)
|
||||
header = " sl local_address rem_address st\n"
|
||||
# The container's own table: just its API port.
|
||||
(tmp_path / "net" / "tcp").write_text(header + " 0: 00000000:1390 00000000:0000 0A\n")
|
||||
# The host's: ssh on 22 (0x16).
|
||||
(tmp_path / "1" / "net" / "tcp").write_text(header + " 0: 00000000:0016 00000000:0000 0A\n")
|
||||
monkeypatch.setattr(settings, "HOST_PROC_PATH", str(tmp_path))
|
||||
|
||||
assert port_service._read_proc_net("tcp") == {22}
|
||||
@@ -0,0 +1,58 @@
|
||||
"""Webhook delivery.
|
||||
|
||||
Two ways a notification silently went nowhere before 0.67.1: an ntfy title with
|
||||
a non-ASCII character (a destination called "NAS Büro", an em dash) raised
|
||||
UnicodeEncodeError inside httpx, and any error that was not an
|
||||
``httpx.HTTPError`` escaped the per-webhook handler and aborted delivery to
|
||||
every webhook after it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import email.header
|
||||
|
||||
import httpx
|
||||
|
||||
|
||||
def test_a_non_ascii_ntfy_title_is_rfc2047_encoded():
|
||||
from services.notify_service import _build_request
|
||||
|
||||
kwargs = _build_request("ntfy", "http://ntfy/t", "backup_failed", "Backup to NAS Büro — failed", "m")
|
||||
request = httpx.Request("POST", kwargs["url"], headers=kwargs["headers"], content=kwargs["content"])
|
||||
(decoded, charset), = email.header.decode_header(request.headers["title"])
|
||||
assert decoded.decode(charset) == "Backup to NAS Büro — failed"
|
||||
|
||||
|
||||
def test_an_ascii_title_is_sent_as_is():
|
||||
from services.notify_service import _header_value
|
||||
|
||||
assert _header_value("Stack 'x' started") == "Stack 'x' started"
|
||||
|
||||
|
||||
def test_one_broken_webhook_does_not_stop_the_others(monkeypatch):
|
||||
from services import notify_service
|
||||
|
||||
delivered = []
|
||||
|
||||
class Client:
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *exc):
|
||||
return False
|
||||
|
||||
async def post(self, url, timeout=None, **kwargs):
|
||||
if "broken" in url:
|
||||
raise ValueError("not an httpx error")
|
||||
delivered.append(url)
|
||||
return httpx.Response(200, request=httpx.Request("POST", url))
|
||||
|
||||
monkeypatch.setattr(notify_service.httpx, "AsyncClient", lambda **kw: Client())
|
||||
monkeypatch.setattr(
|
||||
notify_service,
|
||||
"_targets_for_event",
|
||||
lambda session, event: [("generic", "http://broken/"), ("generic", "http://fine/")],
|
||||
)
|
||||
count = asyncio.run(notify_service.notify("stack_start", "t", "m", session=object()))
|
||||
assert count == 1
|
||||
assert delivered == ["http://fine/"]
|
||||
@@ -331,7 +331,7 @@ def _refusing(monkeypatch):
|
||||
raise update_service.AuthRequired("ghcr.io")
|
||||
|
||||
monkeypatch.setattr(update_service, "remote_digest", refuse)
|
||||
monkeypatch.setattr(update_service, "_local_digest", lambda image: "sha256:local")
|
||||
monkeypatch.setattr(update_service, "_local_digest", lambda image, attrs=None: "sha256:local")
|
||||
return update_service
|
||||
|
||||
|
||||
@@ -370,6 +370,6 @@ def test_a_registry_we_simply_cannot_reach_still_says_that(monkeypatch):
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(update_service, "remote_digest", unreachable)
|
||||
monkeypatch.setattr(update_service, "_local_digest", lambda image: "sha256:local")
|
||||
monkeypatch.setattr(update_service, "_local_digest", lambda image, attrs=None: "sha256:local")
|
||||
result = asyncio.run(update_service.check_image("ghcr.io/acme/app:1.0"))
|
||||
assert result.error == "could not reach registry"
|
||||
@@ -0,0 +1,137 @@
|
||||
"""Smaller regressions found in the 0.67 review, one test each.
|
||||
|
||||
* NVIDIA cards were never detected: the image has no nvidia-smi and the DRI
|
||||
scan skips NVIDIA on purpose. The driver's /proc entries are read instead.
|
||||
* A container's inspect view handed its environment — credentials, by
|
||||
convention — to the read-only role, which does not get the stack's .env.
|
||||
* Git branch names and URLs go to git's command line; a leading "-" made them
|
||||
options.
|
||||
* "Scan all" ran for longer than the proxy waits; it now runs in the
|
||||
background and reports through /scan-status.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def test_nvidia_gpus_are_read_from_the_driver_procfs(tmp_path, monkeypatch):
|
||||
from config import settings
|
||||
from services import gpu_service
|
||||
|
||||
gpus = tmp_path / "driver" / "nvidia" / "gpus"
|
||||
(gpus / "0000:01:00.0").mkdir(parents=True)
|
||||
(gpus / "0000:01:00.0" / "information").write_text(
|
||||
"Model: \t\t NVIDIA GeForce RTX 3060\n"
|
||||
"IRQ: \t\t 150\n"
|
||||
"GPU UUID: \t GPU-1234-abcd\n"
|
||||
"Device Minor: \t 0\n"
|
||||
)
|
||||
(tmp_path / "driver" / "nvidia" / "version").write_text(
|
||||
"NVRM version: NVIDIA UNIX x86_64 Kernel Module 550.120 Fri Sep 13 2024\n"
|
||||
)
|
||||
monkeypatch.setattr(settings, "HOST_PROC_PATH", str(tmp_path))
|
||||
monkeypatch.setattr(gpu_service, "_detect_nvidia", lambda: [])
|
||||
monkeypatch.setattr(gpu_service, "_detect_dri", lambda: [])
|
||||
|
||||
(gpu,) = gpu_service.detect_gpus()
|
||||
assert gpu.vendor == "nvidia"
|
||||
assert gpu.name == "NVIDIA GeForce RTX 3060"
|
||||
assert gpu.uuid == "GPU-1234-abcd"
|
||||
assert gpu.index == 0
|
||||
assert gpu.driver == "550.120"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_container(monkeypatch):
|
||||
from services import container_service
|
||||
|
||||
monkeypatch.setattr(
|
||||
container_service,
|
||||
"inspect_container",
|
||||
lambda container_id: {"id": container_id, "env": ["DB_PASSWORD=hunter2", "NOVALUE"]},
|
||||
)
|
||||
|
||||
|
||||
def test_container_env_values_are_hidden_from_the_read_only_role(as_user, fake_container):
|
||||
env = as_user.get("/api/containers/abc").json()["env"]
|
||||
assert env == ["DB_PASSWORD=••••••", "NOVALUE"]
|
||||
|
||||
|
||||
def test_admins_still_see_the_environment(as_admin, fake_container):
|
||||
assert as_admin.get("/api/containers/abc").json()["env"][0] == "DB_PASSWORD=hunter2"
|
||||
|
||||
|
||||
def test_a_missing_container_is_a_404(as_admin, monkeypatch):
|
||||
from docker_client import DockerError
|
||||
from services import container_service
|
||||
|
||||
def missing(container_id):
|
||||
raise DockerError("not_found", "No such container")
|
||||
|
||||
monkeypatch.setattr(container_service, "inspect_container", missing)
|
||||
assert as_admin.get("/api/containers/nope").status_code == 404
|
||||
|
||||
|
||||
@pytest.mark.parametrize("branch", ["-x", "--upload-pack=sh", "a..b", "x.lock", ""])
|
||||
def test_branch_names_that_git_would_read_as_options_are_refused(branch):
|
||||
from services import git_service
|
||||
|
||||
with pytest.raises(git_service.GitError):
|
||||
git_service.validate_branch(branch)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("url", ["-oProxyCommand=x", "ext::sh -c id", "fd::3", ""])
|
||||
def test_repository_urls_that_run_commands_are_refused(url):
|
||||
from services import git_service
|
||||
|
||||
with pytest.raises(git_service.GitError):
|
||||
git_service.validate_url(url)
|
||||
|
||||
|
||||
def test_ordinary_branches_and_urls_pass():
|
||||
from services import git_service
|
||||
|
||||
assert git_service.validate_branch("release/1.2") == "release/1.2"
|
||||
assert git_service.validate_url("git@forge:me/stack.git") == "git@forge:me/stack.git"
|
||||
|
||||
|
||||
def test_scan_all_in_the_background_is_active_before_the_request_returns(monkeypatch, db):
|
||||
from services import scan_service
|
||||
|
||||
release = asyncio.Event()
|
||||
|
||||
async def slow_scan(session, image):
|
||||
await release.wait()
|
||||
|
||||
class Row:
|
||||
error = None
|
||||
|
||||
return Row()
|
||||
|
||||
monkeypatch.setattr(scan_service, "scan", slow_scan)
|
||||
|
||||
async def scenario():
|
||||
status = scan_service.start_sweep(["a", "b"])
|
||||
assert status["active"] is True and status["total"] == 2
|
||||
with pytest.raises(scan_service.ScanError):
|
||||
scan_service.start_sweep(["c"])
|
||||
release.set()
|
||||
for _ in range(100):
|
||||
if not scan_service.sweep_status()["active"]:
|
||||
break
|
||||
await asyncio.sleep(0.01)
|
||||
final = scan_service.sweep_status()
|
||||
assert final["active"] is False
|
||||
assert final["scanned"] == 2 and final["failed"] == 0
|
||||
|
||||
asyncio.run(scenario())
|
||||
|
||||
|
||||
def test_setup_refuses_an_empty_account(client, db, monkeypatch):
|
||||
import auth as auth_mod
|
||||
|
||||
monkeypatch.setattr(auth_mod, "users_exist", lambda session: False)
|
||||
response = client.post("/api/auth/setup", json={"username": " ", "password": ""})
|
||||
assert response.status_code == 400
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Deleting a stack: what goes with it, and what must not.
|
||||
|
||||
Before 0.67.1 a deleted stack left its Git source behind — and the poll loop or
|
||||
the next push webhook recreated the folder from the repository and started the
|
||||
stack again. Its auto-update policy was inherited by the next stack of the same
|
||||
name, and every backup schedule naming it failed from then on. Deleting the
|
||||
folder also walked into whatever was mounted inside it (a NAS share at
|
||||
``./media``) and deleted that too.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
|
||||
import pytest
|
||||
from sqlmodel import Session, delete, select
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean(db):
|
||||
from database import engine
|
||||
from models.auto_update import AutoUpdate
|
||||
from models.backup_schedule import BackupSchedule
|
||||
from models.git_source import GitSource
|
||||
from models.runtime_state import StackLock
|
||||
|
||||
def wipe():
|
||||
with Session(engine) as session:
|
||||
for model in (GitSource, AutoUpdate, BackupSchedule, StackLock):
|
||||
session.exec(delete(model))
|
||||
session.commit()
|
||||
|
||||
wipe()
|
||||
yield
|
||||
wipe()
|
||||
|
||||
|
||||
def _schedule(session, stack_ids, all_stacks=False):
|
||||
from models.backup_schedule import BackupSchedule, set_stacks
|
||||
|
||||
schedule = BackupSchedule(destination_id=1, all_stacks=all_stacks)
|
||||
set_stacks(schedule, stack_ids)
|
||||
session.add(schedule)
|
||||
session.commit()
|
||||
session.refresh(schedule)
|
||||
return schedule.id
|
||||
|
||||
|
||||
def test_forget_stack_drops_what_refers_to_it(db):
|
||||
from database import engine
|
||||
from models.auto_update import AutoUpdate
|
||||
from models.backup_schedule import BackupSchedule, stacks_of
|
||||
from models.git_source import GitSource
|
||||
from services import stack_cleanup_service
|
||||
|
||||
with Session(engine) as session:
|
||||
session.add(GitSource(stack_id="gone", url="https://example.invalid/x.git"))
|
||||
session.add(AutoUpdate(stack_id="gone"))
|
||||
session.add(AutoUpdate(stack_id="stays"))
|
||||
both = _schedule(session, ["gone", "stays"])
|
||||
only = _schedule(session, ["gone"])
|
||||
everything = _schedule(session, [], all_stacks=True)
|
||||
|
||||
report = stack_cleanup_service.forget_stack(session, "gone")
|
||||
|
||||
assert report["git"] is True and report["auto_update"] == 1
|
||||
assert session.exec(select(GitSource)).all() == []
|
||||
assert [p.stack_id for p in session.exec(select(AutoUpdate)).all()] == ["stays"]
|
||||
assert stacks_of(session.get(BackupSchedule, both)) == ["stays"]
|
||||
lonely = session.get(BackupSchedule, only)
|
||||
assert stacks_of(lonely) == [] and lonely.enabled is False
|
||||
assert "was deleted" in lonely.last_status
|
||||
assert session.get(BackupSchedule, everything).enabled is True
|
||||
|
||||
|
||||
def test_orphans_from_older_versions_are_forgotten_at_startup(db):
|
||||
from database import engine
|
||||
from models.git_source import GitSource
|
||||
from models.stack import Stack
|
||||
from services import stack_cleanup_service
|
||||
|
||||
with Session(engine) as session:
|
||||
if not session.get(Stack, "alive"):
|
||||
session.add(Stack(id="alive", name="alive"))
|
||||
session.add(GitSource(stack_id="alive", url="https://example.invalid/a.git"))
|
||||
session.add(GitSource(stack_id="long-deleted", url="https://example.invalid/b.git"))
|
||||
session.commit()
|
||||
|
||||
assert stack_cleanup_service.prune_orphans(session) == ["long-deleted"]
|
||||
assert [g.stack_id for g in session.exec(select(GitSource)).all()] == ["alive"]
|
||||
|
||||
|
||||
def test_git_never_syncs_a_stack_that_no_longer_exists(db):
|
||||
from database import engine
|
||||
from models.git_source import GitSource
|
||||
from services import git_service
|
||||
|
||||
with Session(engine) as session:
|
||||
source = GitSource(stack_id="ghost", url="https://example.invalid/g.git")
|
||||
session.add(source)
|
||||
session.commit()
|
||||
session.refresh(source)
|
||||
with pytest.raises(git_service.GitError, match="no longer exists"):
|
||||
asyncio.run(git_service.sync(session, source))
|
||||
|
||||
|
||||
def test_deleting_a_busy_stack_is_refused(as_admin, db):
|
||||
from database import engine
|
||||
from models.stack import Stack
|
||||
from services import stack_lock_service
|
||||
|
||||
with Session(engine) as session:
|
||||
if not session.get(Stack, "busy-del"):
|
||||
session.add(Stack(id="busy-del", name="busy"))
|
||||
session.commit()
|
||||
stack_lock_service.acquire(session, "busy-del", "backup", "scheduled")
|
||||
|
||||
response = as_admin.delete("/api/stacks/busy-del")
|
||||
assert response.status_code == 409
|
||||
assert "backup" in response.json()["detail"]
|
||||
|
||||
|
||||
def test_folder_removal_stops_at_mount_points(tmp_path, monkeypatch):
|
||||
from services import compose_service
|
||||
|
||||
stack = tmp_path / "media-stack"
|
||||
(stack / "config").mkdir(parents=True)
|
||||
(stack / "config" / "app.db").write_text("x")
|
||||
(stack / "compose.yaml").write_text("services: {}\n")
|
||||
share = stack / "media"
|
||||
share.mkdir()
|
||||
(share / "movie.mkv").write_text("precious")
|
||||
|
||||
monkeypatch.setattr(compose_service, "_mount_points", lambda: {str(share)})
|
||||
kept = compose_service.delete_stack_files("media-stack", override=str(tmp_path))
|
||||
|
||||
assert kept == [str(share)]
|
||||
assert (share / "movie.mkv").read_text() == "precious"
|
||||
assert not (stack / "config").exists() and not (stack / "compose.yaml").exists()
|
||||
|
||||
|
||||
def test_cloning_copies_links_as_links_and_skips_sockets(tmp_path):
|
||||
import socket
|
||||
|
||||
from services import compose_service
|
||||
|
||||
src = tmp_path / "orig"
|
||||
src.mkdir()
|
||||
(src / "compose.yaml").write_text("services: {}\n")
|
||||
os.symlink("/definitely/not/there", src / "dangling")
|
||||
sock = socket.socket(socket.AF_UNIX)
|
||||
sock.bind(str(src / "app.sock"))
|
||||
try:
|
||||
compose_service.clone_stack_files("orig", "copy", override=str(tmp_path))
|
||||
finally:
|
||||
sock.close()
|
||||
|
||||
dst = tmp_path / "copy"
|
||||
assert (dst / "compose.yaml").exists()
|
||||
assert os.readlink(dst / "dangling") == "/definitely/not/there"
|
||||
assert not os.path.lexists(dst / "app.sock")
|
||||
|
||||
|
||||
def test_a_deleted_stack_name_does_not_inherit_its_schedules(as_admin, db, monkeypatch):
|
||||
"""End to end through the API, with Docker stubbed out."""
|
||||
from database import engine
|
||||
from models.backup_schedule import BackupSchedule, stacks_of
|
||||
from services import compose_service
|
||||
|
||||
async def down(stack_id, override=None):
|
||||
return {"returncode": 0, "stdout": "", "stderr": ""}
|
||||
|
||||
monkeypatch.setattr(compose_service, "down", down)
|
||||
assert as_admin.post("/api/stacks", json={"name": "del-me"}).status_code == 201
|
||||
with Session(engine) as session:
|
||||
schedule_id = _schedule(session, ["del-me"])
|
||||
|
||||
response = as_admin.delete("/api/stacks/del-me")
|
||||
assert response.status_code == 200, response.text
|
||||
assert response.json()["kept"] == []
|
||||
with Session(engine) as session:
|
||||
schedule = session.get(BackupSchedule, schedule_id)
|
||||
assert stacks_of(schedule) == [] and not schedule.enabled
|
||||
assert not os.path.isdir(compose_service.stack_dir("del-me"))
|
||||
assert json.loads(as_admin.get("/api/stacks").text) is not None
|
||||
@@ -0,0 +1,242 @@
|
||||
"""Found clicking through the UI for 0.67.1.
|
||||
|
||||
* The file browser opens at ``/``, which the default roots (/mnt, /opt, …)
|
||||
do not include: the page opened on "outside the allowed browse roots" and
|
||||
offered no way into any root. Paths above the roots now list the way to them.
|
||||
* A root could not be deleted, but it could be moved — the whole tree with it.
|
||||
* Locally built images (compose ``build:``) were checked against Docker Hub,
|
||||
which answers 401 for a repository that does not exist: "needs credentials".
|
||||
* Volumes, networks and images of a stack whose folder name compose had to
|
||||
normalise ("Nextcloud" -> project "nextcloud") were grouped as leftovers of a
|
||||
removed stack.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def roots(tmp_path, monkeypatch):
|
||||
from config import settings
|
||||
|
||||
(tmp_path / "srv" / "data").mkdir(parents=True)
|
||||
(tmp_path / "opt").mkdir()
|
||||
monkeypatch.setattr(settings, "HOST_ROOT_PREFIX", str(tmp_path), raising=False)
|
||||
monkeypatch.setattr(
|
||||
settings, "ALLOWED_BROWSE_ROOTS", ["/opt", "/srv/data", "/media"], raising=False
|
||||
)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def test_the_top_of_the_tree_lists_the_way_to_the_roots(roots):
|
||||
from services import device_service
|
||||
|
||||
top = device_service.browse("/")
|
||||
assert top["virtual"] is True
|
||||
# /media does not exist on this host, so it is not offered.
|
||||
assert [e["name"] for e in top["entries"]] == ["opt", "srv"]
|
||||
assert [e["name"] for e in device_service.browse("/srv")["entries"]] == ["data"]
|
||||
inside = device_service.browse("/srv/data")
|
||||
assert "virtual" not in inside and inside["parent"] == "/srv"
|
||||
|
||||
|
||||
def test_a_path_leading_nowhere_is_still_refused(roots):
|
||||
from services import device_service
|
||||
|
||||
with pytest.raises(device_service.BrowseError):
|
||||
device_service.browse("/etc")
|
||||
|
||||
|
||||
def test_a_root_cannot_be_moved_or_renamed(roots):
|
||||
from services import file_service
|
||||
|
||||
with pytest.raises(file_service.BrowseError, match="browse root"):
|
||||
file_service.move("/srv/data", "/opt")
|
||||
with pytest.raises(file_service.BrowseError, match="browse root"):
|
||||
file_service.rename("/opt", "opt2")
|
||||
assert (roots / "srv" / "data").is_dir()
|
||||
|
||||
|
||||
def test_moving_inside_a_root_still_works(roots):
|
||||
from services import file_service
|
||||
|
||||
(roots / "opt" / "a.txt").write_text("x")
|
||||
file_service.move("/opt/a.txt", "/srv/data")
|
||||
assert (roots / "srv" / "data" / "a.txt").read_text() == "x"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"attrs, local",
|
||||
[
|
||||
({"Identity": {"Build": [{"Ref": "x"}]}, "RepoDigests": ["app@sha256:1"]}, True),
|
||||
({"Identity": {"Pull": [{"Repository": "docker.io/library/nginx"}]}}, False),
|
||||
({"RepoDigests": []}, True), # older daemon, never pulled
|
||||
({"RepoDigests": ["nginx@sha256:1"]}, False),
|
||||
({"Identity": {}, "RepoDigests": ["nginx@sha256:1"]}, False),
|
||||
(None, False),
|
||||
],
|
||||
)
|
||||
def test_where_an_image_came_from(attrs, local):
|
||||
from services import update_service
|
||||
|
||||
assert update_service._built_locally(attrs) is local
|
||||
|
||||
|
||||
def test_a_local_build_is_not_asked_about_at_the_registry(monkeypatch):
|
||||
from services import update_service
|
||||
|
||||
monkeypatch.setattr(
|
||||
update_service, "_local_attrs", lambda image: {"Identity": {"Build": [{"Ref": "x"}]}}
|
||||
)
|
||||
|
||||
async def no_registry(image):
|
||||
raise AssertionError("registry contacted for a local build")
|
||||
|
||||
monkeypatch.setattr(update_service, "remote_digest", no_registry)
|
||||
status = asyncio.run(update_service.check_image("myapp-web:latest"))
|
||||
assert status.local is True and status.error is None and not status.update_available
|
||||
|
||||
|
||||
def test_resources_of_a_normalised_stack_are_grouped_under_it(as_admin, db, monkeypatch):
|
||||
from sqlmodel import Session
|
||||
|
||||
from database import engine
|
||||
from models.stack import Stack
|
||||
from services import network_service, volume_service
|
||||
|
||||
with Session(engine) as session:
|
||||
if not session.get(Stack, "Nextcloud"):
|
||||
session.add(Stack(id="Nextcloud", name="Nextcloud"))
|
||||
session.commit()
|
||||
monkeypatch.setattr(
|
||||
volume_service, "list_volumes", lambda: [{"name": "nextcloud_db", "stack": "nextcloud"}]
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
network_service,
|
||||
"list_networks",
|
||||
lambda: [{"name": "x", "stack": "elsewhere"}, {"name": "bridge", "stack": None}],
|
||||
)
|
||||
assert as_admin.get("/api/volumes").json()[0]["stack"] == "Nextcloud"
|
||||
assert [n["stack"] for n in as_admin.get("/api/networks").json()] == ["elsewhere", None]
|
||||
|
||||
|
||||
class _Container:
|
||||
def __init__(self, mounts):
|
||||
self.attrs = {"Mounts": mounts}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"mounts, host",
|
||||
[
|
||||
# The usual: same path on both sides — no warning.
|
||||
([{"Type": "bind", "Source": "/opt/stacks", "Destination": "/opt/stacks"}], None),
|
||||
# A real mismatch.
|
||||
(
|
||||
[{"Type": "bind", "Source": "/srv/compose", "Destination": "/opt/stacks"}],
|
||||
"/srv/compose",
|
||||
),
|
||||
# The folder above STACKS_DIR is what is mounted.
|
||||
([{"Type": "bind", "Source": "/srv", "Destination": "/opt"}], "/srv/stacks"),
|
||||
# Not mounted at all.
|
||||
([{"Type": "volume", "Source": "/var/lib/docker/volumes/x", "Destination": "/data"}], None),
|
||||
],
|
||||
)
|
||||
def test_the_host_stacks_path_comes_from_docker_not_mountinfo(monkeypatch, mounts, host):
|
||||
"""mountinfo's root field is relative to the filesystem: a stacks folder on
|
||||
its own disk or btrfs subvolume showed a mismatch that was not there."""
|
||||
from config import settings
|
||||
from services import self_update_service, stack_assets_service
|
||||
|
||||
monkeypatch.setattr(settings, "STACKS_DIR", "/opt/stacks")
|
||||
monkeypatch.setattr(stack_assets_service, "_HOST_STACKS_DIR", None)
|
||||
monkeypatch.setattr(self_update_service, "_own_container", lambda: _Container(mounts))
|
||||
mismatch = stack_assets_service.stacks_path_mismatch()
|
||||
assert (mismatch["host"] if mismatch else None) == host
|
||||
|
||||
|
||||
def test_backup_names_carry_the_time_on_the_users_clock(monkeypatch):
|
||||
"""Named in UTC, a backup taken at 12:21 in Berlin read as …-102154."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from services import backup_service, compose_service
|
||||
|
||||
monkeypatch.setattr(
|
||||
compose_service, "now", lambda: datetime(2026, 10, 15, 10, 21, 54, tzinfo=timezone.utc)
|
||||
)
|
||||
berlin = backup_service.backup_filename("app", True, tz="Europe/Berlin")
|
||||
assert berlin == "backup-app-full-20261015-122154.tar.gz"
|
||||
# No zone, or one that does not exist: UTC, as before.
|
||||
assert backup_service.backup_filename("app", False).endswith("-20261015-102154.tar.gz")
|
||||
assert backup_service.backup_filename("app", False, tz="Mars/Olympus").endswith("-102154.tar.gz")
|
||||
# Retention still recognises the files.
|
||||
assert backup_service.backup_name_pattern("app").match(berlin)
|
||||
|
||||
|
||||
COMPOSE_WITH_SECRETS = """services:
|
||||
db:
|
||||
image: postgres:16
|
||||
environment:
|
||||
POSTGRES_PASSWORD: hunter2
|
||||
POSTGRES_USER: app
|
||||
API_KEY: ${API_KEY:-changeme}
|
||||
DB_PASS: ${DB_PASS}
|
||||
KEYCLOAK_ADMIN: admin
|
||||
app:
|
||||
environment:
|
||||
- "TS_AUTHKEY=tskey-123"
|
||||
- TZ=Europe/Berlin
|
||||
"""
|
||||
|
||||
|
||||
def test_inline_credentials_are_masked_and_the_rest_left_alone():
|
||||
from services.compose_service import MASK, mask_secrets
|
||||
|
||||
masked = mask_secrets(COMPOSE_WITH_SECRETS)
|
||||
assert "hunter2" not in masked and "tskey-123" not in masked and "changeme" not in masked
|
||||
assert f"POSTGRES_PASSWORD: {MASK}" in masked
|
||||
assert f'- "TS_AUTHKEY={MASK}"' in masked
|
||||
# A bare variable is no secret; names that merely contain "key" are not either.
|
||||
for kept in ("DB_PASS: ${DB_PASS}", "KEYCLOAK_ADMIN: admin", "POSTGRES_USER: app", "TZ=Europe/Berlin"):
|
||||
assert kept in masked
|
||||
assert masked.count("\n") == COMPOSE_WITH_SECRETS.count("\n")
|
||||
|
||||
|
||||
def test_the_read_only_role_gets_the_masked_compose_file(as_admin, as_user, db):
|
||||
from services import compose_service
|
||||
|
||||
assert as_admin.post("/api/stacks", json={"name": "masked", "yaml": COMPOSE_WITH_SECRETS}).status_code == 201
|
||||
try:
|
||||
assert "hunter2" in as_admin.get("/api/stacks/masked").json()["yaml"]
|
||||
assert "hunter2" not in as_user.get("/api/stacks/masked").json()["yaml"]
|
||||
finally:
|
||||
compose_service.delete_stack_files("masked")
|
||||
|
||||
|
||||
def test_scan_all_covers_only_stackpilots_own_stacks(as_admin, db, monkeypatch):
|
||||
from sqlmodel import Session
|
||||
|
||||
from database import engine
|
||||
from models.stack import Stack
|
||||
from services import image_service, scan_service
|
||||
|
||||
with Session(engine) as session:
|
||||
if not session.get(Stack, "Mine"):
|
||||
session.add(Stack(id="Mine", name="Mine"))
|
||||
session.commit()
|
||||
monkeypatch.setattr(
|
||||
image_service,
|
||||
"list_images",
|
||||
lambda: [
|
||||
{"tag": "mine:1", "stacks": ["mine"]},
|
||||
{"tag": "theirs:1", "stacks": ["some-other-project"]},
|
||||
{"tag": "loose:1", "stacks": []},
|
||||
],
|
||||
)
|
||||
swept = []
|
||||
monkeypatch.setattr(
|
||||
scan_service, "start_sweep", lambda images, on_done=None: swept.append(images) or {"active": True}
|
||||
)
|
||||
assert as_admin.post("/api/images/scan-all?background=true").status_code == 202
|
||||
assert swept == [["mine:1"]]
|
||||
Reference in new issue
Block a user