Fix what a review of the whole app found, end to end (0.67.1)
A pass over every feature against a real Docker daemon, then clicking through the UI as admin and read-only user, on a desktop and a phone. Security. /ws/deploy ran compose up for the read-only role; it is admin-only now. A container's inspect view, a stack's compose file (POSTGRES_PASSWORD: hunter2) and the viewer's .env tab no longer show credentials to that role. Git branch and URL values that git would read as options (-x, ext::) are refused. SFTP host keys are checked: pinned on first use or entered by hand (key line or SHA256 fingerprint), a different key is refused before the password is sent. A backup name from the API can no longer reach outside the destination folder. The file browser refuses to move or rename a browse root, as delete did. Docker lifecycle. Closing the deploy console released the stack lock while compose was still running and logged rc=None as success; deploys and updates are now background runs that hold the lock to the end, report the real exit code and can be re-attached to. Every closed log view left a `compose logs -f` behind. A dozen open tabs exhausted the default thread pool and hung the API; event and exec streams have their own threads. Auto-update and the update endpoint ignored failed pulls. A deleted git stack was recreated and started by the next webhook or poll; deleting a stack now drops its git source, auto-update policy and schedule entries. Deleting no longer walks into mounted shares, is refused while a backup runs, and removes the containers even when the compose file is broken. Stack folders compose has to normalise (Nextcloud -> nextcloud) work everywhere and group correctly on the volumes, networks and images pages. Detection. Host port conflicts read the container's own network namespace; /proc/1/net is read instead. Images referenced as docker.io/… were never update-checked; local builds no longer report "needs credentials". NVIDIA GPUs are read from the driver's procfs. The backup dialog's "path mismatch" warning came from mountinfo's filesystem-relative root and fired for any stacks folder on its own disk, ZFS dataset or btrfs subvolume; the host path now comes from Docker's mounts. Backups and destinations. The download ignored the folder/volume selection (axios sent binds[]). Restoring under a new name kept the old one. A report header past nginx's buffer turned a good backup into a 502. SFTP/S3 errors were bare 500s and "Test" did not try to write. A destination used by a schedule could be deleted. Backup file names are in the schedule's or the browser's time zone instead of UTC. Editor and converter. `docker run` conversion produced invalid compose for --network host, named volumes and -dit; --gpus and --mount were dropped; a multi-line command pasted into the one-line field became `image: ' '` (now a textarea, and flattened continuations are read). Validation failed for env_file stacks. The editor overwrote unsaved edits on every container event, lost them on a stray click (now asks), threw when closing the diff, and its helper panel listed the starter service "app" for every existing stack. Monaco is bundled instead of loaded from jsdelivr, so the editor works offline (and monaco-editor 0.57 drops the dompurify advisory; axios and react-router updated). UI. All times were two hours off: offset-less UTC from SQLite was read as local time, and "next run" always said "just now". A token refresh mid-session duplicated deploys, reset terminals and doubled logs. A backend restart (update, self-update) signed everyone out on reload; the page now waits for it. The editor was unusable on a phone, tables and tabs overflowed, dialog text was black on the dark theme, and the file browser opened on an error with the default roots. Read-only users get "Admin only" pages instead of 403s; in-use networks offer no delete; "Scan all" covers StackPilot's own stacks only and runs in the background instead of timing out at 600s. Deployment. nginx passes the port in Host (the OIDC redirect URI lost :5009) and keeps an outer proxy's X-Forwarded-Proto. The frontend waits for a healthy backend (healthcheck probes every 2s while starting). The frontend image builds with npm ci. Secrets are 0644 in a 0700 folder so non-root containers can read them; existing files are fixed at startup. The test suite no longer starts real containers on a host with Docker. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Fx7gcuy8d6nPd95g43rGHu
This commit is contained in:
1 parent
270ddda51d
commit
d6cebb3371
88 files changed
+4752
-925
No files matched your search
@@ -114,8 +114,10 @@ def setup(
|
||||
) -> TokenPair:
|
||||
if auth_mod.users_exist(session):
|
||||
raise HTTPException(status_code=400, detail="Setup already completed")
|
||||
if not body.username.strip() or not body.password:
|
||||
raise HTTPException(status_code=400, detail="Username and password required")
|
||||
user = User(
|
||||
username=body.username,
|
||||
username=body.username.strip(),
|
||||
hashed_password=auth_mod.hash_password(body.password),
|
||||
role="admin",
|
||||
)
|
||||
|
||||
@@ -9,7 +9,7 @@ import tempfile
|
||||
from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Request, UploadFile
|
||||
from fastapi.responses import FileResponse
|
||||
from starlette.background import BackgroundTask
|
||||
from pydantic import BaseModel
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlmodel import Session
|
||||
|
||||
from auth import require_admin
|
||||
@@ -51,6 +51,23 @@ def _compact(report: dict) -> dict:
|
||||
}
|
||||
|
||||
|
||||
#: The report rides in a response header, which a proxy refuses past its
|
||||
#: header buffer (nginx: 4–8 KiB by default, then a 502 for a backup that
|
||||
#: worked). Lists are cut until it fits; the counts stay in the audit log.
|
||||
MAX_REPORT_HEADER = 6 * 1024
|
||||
|
||||
|
||||
def _report_header(report: dict) -> str:
|
||||
compact = _compact(report)
|
||||
text = json.dumps(compact)
|
||||
for key in ("warnings", "skipped", "binds", "volumes", "errors"):
|
||||
while len(text) > MAX_REPORT_HEADER and compact.get(key):
|
||||
compact[key] = compact[key][: len(compact[key]) // 2]
|
||||
compact["truncated"] = True
|
||||
text = json.dumps(compact)
|
||||
return text
|
||||
|
||||
|
||||
def _summary(report: dict) -> str:
|
||||
text = (
|
||||
f"binds={len(report.get('binds', []))} "
|
||||
@@ -96,6 +113,8 @@ async def backup_stack(
|
||||
stop_first: bool = Query(True),
|
||||
binds: list[str] | None = Query(None),
|
||||
volumes: list[str] | None = Query(None),
|
||||
# The browser's zone, for the time in the file name.
|
||||
tz: str | None = Query(None, max_length=64),
|
||||
session: Session = Depends(get_session),
|
||||
user: User = Depends(require_admin),
|
||||
):
|
||||
@@ -117,10 +136,10 @@ async def backup_stack(
|
||||
return FileResponse(
|
||||
path,
|
||||
media_type="application/gzip",
|
||||
filename=_backup_filename(stack_id, include_volumes),
|
||||
filename=_backup_filename(stack_id, include_volumes, tz=tz),
|
||||
# The browser downloads a blob, so the summary of what actually made it
|
||||
# into the archive rides along in a header.
|
||||
headers={"X-Stackpilot-Backup": json.dumps(_compact(report))},
|
||||
headers={"X-Stackpilot-Backup": _report_header(report)},
|
||||
# The archive is a temp file; without this every download stayed in
|
||||
# the container's /tmp until it filled the disk.
|
||||
background=BackgroundTask(os.unlink, path),
|
||||
@@ -184,6 +203,8 @@ class PushBody(BaseModel):
|
||||
stop_first: bool = True
|
||||
binds: list[str] | None = None
|
||||
volumes: list[str] | None = None
|
||||
#: The browser's zone, for the time in the file name.
|
||||
tz: str | None = Field(None, max_length=64)
|
||||
|
||||
|
||||
class RestoreFromBody(BaseModel):
|
||||
@@ -223,7 +244,7 @@ async def push_backup(
|
||||
except backup_service.BackupError as exc:
|
||||
raise _backup_http_error(exc) from exc
|
||||
|
||||
filename = _backup_filename(stack_id, body.include_volumes)
|
||||
filename = _backup_filename(stack_id, body.include_volumes, tz=body.tz)
|
||||
try:
|
||||
remote = await asyncio.to_thread(dest_service.upload, dest, path, filename)
|
||||
except dest_service.DestinationError as exc:
|
||||
|
||||
@@ -1,24 +1,46 @@
|
||||
"""Single-container inspect + lifecycle for compose-managed containers."""
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from sqlmodel import Session
|
||||
|
||||
from auth import get_current_user, require_admin
|
||||
from database import get_session
|
||||
from docker_client import DockerError
|
||||
from models.user import User
|
||||
from services import audit_service, container_service
|
||||
|
||||
router = APIRouter(prefix="/api/containers", tags=["containers"])
|
||||
|
||||
#: Docker errors that are the caller's to fix, not a daemon failure (the global
|
||||
#: handler turns every DockerError into a 502).
|
||||
_STATUS = {"not_found": 404, "not_managed": 404, "bad_action": 400}
|
||||
|
||||
|
||||
def _ip(request: Request) -> str:
|
||||
return request.client.host if request.client else "unknown"
|
||||
|
||||
|
||||
def _raise(exc: DockerError):
|
||||
status = _STATUS.get(exc.error)
|
||||
if status:
|
||||
raise HTTPException(status_code=status, detail=exc.detail or exc.error) from exc
|
||||
raise exc
|
||||
|
||||
|
||||
@router.get("/{container_id}")
|
||||
def inspect(container_id: str, _user: User = Depends(get_current_user)) -> dict:
|
||||
return container_service.inspect_container(container_id)
|
||||
def inspect(container_id: str, user: User = Depends(get_current_user)) -> dict:
|
||||
try:
|
||||
data = container_service.inspect_container(container_id)
|
||||
except DockerError as exc:
|
||||
_raise(exc)
|
||||
if user.role != "admin":
|
||||
# The environment is where a container's credentials live — the same
|
||||
# values the stack's .env holds, which the read-only role does not get
|
||||
# either. Names stay visible, values do not.
|
||||
data["env"] = [entry.split("=", 1)[0] + "=••••••" if "=" in entry else entry
|
||||
for entry in data.get("env") or []]
|
||||
return data
|
||||
|
||||
|
||||
@router.post("/{container_id}/{action}")
|
||||
@@ -29,7 +51,10 @@ def action(
|
||||
session: Session = Depends(get_session),
|
||||
user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
result = container_service.container_action(container_id, action)
|
||||
try:
|
||||
result = container_service.container_action(container_id, action)
|
||||
except DockerError as exc:
|
||||
_raise(exc)
|
||||
audit_service.record(
|
||||
session, user=user.username, action=f"container.{action}",
|
||||
target=container_id[:12], ip=_ip(request),
|
||||
|
||||
@@ -16,6 +16,7 @@ from models.backup_destination import (
|
||||
DestinationRead,
|
||||
DestinationUpdate,
|
||||
)
|
||||
from models.backup_schedule import BackupSchedule
|
||||
from models.user import User
|
||||
from services import audit_service, backup_destination_service as dest_service
|
||||
|
||||
@@ -92,10 +93,16 @@ def update_destination(
|
||||
if body.config is not None:
|
||||
# Merge so masked/blank secrets don't wipe stored ones.
|
||||
existing = dest_service.parse_config(d)
|
||||
before = dict(existing)
|
||||
for k, v in body.config.items():
|
||||
if k in SECRET_KEYS and (v == "" or v == "••••••"):
|
||||
continue # keep existing secret
|
||||
existing[k] = v
|
||||
# A host key pinned for the old address says nothing about a new one;
|
||||
# kept, the first connection to the new server would be refused.
|
||||
moved = any(str(existing.get(k) or "") != str(before.get(k) or "") for k in ("host", "port"))
|
||||
if moved and existing.get("host_key") == before.get("host_key"):
|
||||
existing.pop("host_key", None)
|
||||
d.config = dest_service.dump_config(existing)
|
||||
session.add(d)
|
||||
session.commit()
|
||||
@@ -115,9 +122,24 @@ def delete_destination(
|
||||
user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
d = _get_or_404(session, dest_id)
|
||||
name = d.name
|
||||
users = session.exec(
|
||||
select(BackupSchedule).where(BackupSchedule.destination_id == dest_id)
|
||||
).all()
|
||||
if users:
|
||||
# Deleted from under them, every run of these failed with
|
||||
# "destination not found" and paged somebody about it.
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=(
|
||||
f"Destination '{d.name}' is used by {len(users)} backup schedule(s) — "
|
||||
"point them at another destination or delete them first"
|
||||
),
|
||||
)
|
||||
name, kind = d.name, d.type
|
||||
session.delete(d)
|
||||
session.commit()
|
||||
if kind == "nfs":
|
||||
dest_service.forget_nfs_volume(dest_id)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="destination.delete", target=name,
|
||||
ip=_ip(request),
|
||||
@@ -134,9 +156,16 @@ async def test_destination(
|
||||
d = _get_or_404(session, dest_id)
|
||||
try:
|
||||
await asyncio.to_thread(dest_service.test, d)
|
||||
return {"ok": True}
|
||||
except dest_service.DestinationError as exc:
|
||||
return {"ok": False, "error": str(exc)}
|
||||
result: dict = {"ok": True}
|
||||
if d.type == "sftp":
|
||||
# Re-read: a first connection has just pinned the server's key.
|
||||
session.refresh(d)
|
||||
pinned = (dest_service.parse_config(d).get("host_key") or "").strip()
|
||||
if pinned:
|
||||
result["host_key"] = dest_service.fingerprint_of(pinned)
|
||||
return result
|
||||
|
||||
|
||||
@router.get("/{dest_id}/backups")
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
"""Editor helper endpoints — merge wizard fragments into compose YAML."""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -15,6 +17,8 @@ router = APIRouter(prefix="/api/editor", tags=["editor"])
|
||||
class ValidateBody(BaseModel):
|
||||
yaml: str
|
||||
env: str = ""
|
||||
#: An existing stack: relative paths are then resolved in its folder.
|
||||
stack_id: str | None = None
|
||||
|
||||
|
||||
class AddVolumeBody(BaseModel):
|
||||
@@ -61,7 +65,18 @@ def _run(fn, *args) -> dict:
|
||||
@router.post("/validate")
|
||||
async def validate(body: ValidateBody, _user: User = Depends(get_current_user)) -> dict:
|
||||
"""Run `docker compose config -q` on the supplied YAML (+ optional .env)."""
|
||||
return await compose_service.validate_yaml(body.yaml, body.env)
|
||||
project_dir = None
|
||||
if body.stack_id:
|
||||
candidate = compose_service.stack_dir(body.stack_id)
|
||||
# Only a real stack folder under STACKS_DIR, never a path out of it.
|
||||
if os.path.dirname(os.path.normpath(candidate)) == os.path.normpath(
|
||||
compose_service.stacks_root()
|
||||
):
|
||||
project_dir = candidate
|
||||
try:
|
||||
return await compose_service.validate_yaml(body.yaml, body.env, project_dir)
|
||||
except compose_service.StackFileError as exc:
|
||||
raise HTTPException(status_code=504, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/services")
|
||||
|
||||
@@ -10,7 +10,9 @@ All paths are sandboxed by :mod:`services.file_service`.
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import stat
|
||||
import tempfile
|
||||
from urllib.parse import quote
|
||||
|
||||
from fastapi import (
|
||||
APIRouter,
|
||||
@@ -35,9 +37,19 @@ router = APIRouter(prefix="/api/files", tags=["files"])
|
||||
|
||||
|
||||
def _attachment(filename: str) -> str:
|
||||
"""A safe ``Content-Disposition`` value for an arbitrary filename."""
|
||||
"""A safe ``Content-Disposition`` value for an arbitrary filename.
|
||||
|
||||
Headers are latin-1; a folder called "Fotos – 2024" (en dash) raised
|
||||
UnicodeEncodeError and the download died with a 500. Non-ASCII names go
|
||||
out as RFC 5987 ``filename*``, with an ASCII fallback beside it.
|
||||
"""
|
||||
safe = filename.replace("\\", "_").replace('"', "_")
|
||||
return f'attachment; filename="{safe}"'
|
||||
try:
|
||||
safe.encode("ascii")
|
||||
return f'attachment; filename="{safe}"'
|
||||
except UnicodeEncodeError:
|
||||
fallback = safe.encode("ascii", "replace").decode("ascii").replace("?", "_")
|
||||
return f"attachment; filename=\"{fallback}\"; filename*=UTF-8''{quote(filename)}"
|
||||
|
||||
|
||||
def _ip(request: Request) -> str:
|
||||
@@ -230,6 +242,27 @@ def delete(
|
||||
return result
|
||||
|
||||
|
||||
def _settle_mode(tmp: str, target: str) -> None:
|
||||
"""Give an upload the permissions a normal file there would have.
|
||||
|
||||
A temp file is created 0600, and ``os.replace`` kept that — so a config
|
||||
uploaded into a stack folder was unreadable for any container not running
|
||||
as root. An overwritten file keeps its mode and owner; a new one gets the
|
||||
usual 0644 (minus umask).
|
||||
"""
|
||||
try:
|
||||
st = os.stat(target)
|
||||
os.chmod(tmp, stat.S_IMODE(st.st_mode))
|
||||
try:
|
||||
os.chown(tmp, st.st_uid, st.st_gid)
|
||||
except OSError:
|
||||
pass
|
||||
except FileNotFoundError:
|
||||
umask = os.umask(0)
|
||||
os.umask(umask)
|
||||
os.chmod(tmp, 0o666 & ~umask)
|
||||
|
||||
|
||||
@router.post("/upload")
|
||||
async def upload(
|
||||
request: Request,
|
||||
@@ -249,6 +282,7 @@ async def upload(
|
||||
while chunk := await file.read(1024 * 1024):
|
||||
tmp.write(chunk)
|
||||
tmp.close()
|
||||
_settle_mode(tmp.name, real)
|
||||
os.replace(tmp.name, real)
|
||||
except OSError as exc:
|
||||
if os.path.exists(tmp.name):
|
||||
|
||||
@@ -97,6 +97,11 @@ def connect(
|
||||
raise HTTPException(status_code=400, detail=f"Unknown auth type '{body.auth_type}'")
|
||||
if not (body.url or "").strip():
|
||||
raise HTTPException(status_code=400, detail="A repository URL is required")
|
||||
try:
|
||||
git_service.validate_url(body.url)
|
||||
git_service.validate_branch(body.branch or "main")
|
||||
except git_service.GitError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
row = session.exec(select(GitSource).where(GitSource.stack_id == stack_id)).first()
|
||||
if row is None:
|
||||
@@ -221,6 +226,9 @@ async def webhook(
|
||||
body = await request.body()
|
||||
if not row or not git_service.verify_webhook(row, body, request.headers):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
if not session.get(Stack, stack_id):
|
||||
# Signed, but for a stack that was deleted: never bring it back.
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
|
||||
try:
|
||||
result = await git_service.sync(session, row, actor="webhook")
|
||||
|
||||
+40
-10
@@ -10,7 +10,7 @@ from auth import get_current_user, require_admin
|
||||
from database import engine, get_session
|
||||
from models.image_scan import ImageScan, ScanDetail, ScanRequest, ScanSummary
|
||||
from models.user import User
|
||||
from services import audit_service, image_service, scan_service, update_service
|
||||
from services import audit_service, compose_service, image_service, scan_service, update_service
|
||||
from services.prune_job import PruneBusy
|
||||
|
||||
router = APIRouter(prefix="/api/images", tags=["images"])
|
||||
@@ -21,8 +21,18 @@ def _ip(request: Request) -> str:
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_images(_user: User = Depends(get_current_user)) -> list[dict]:
|
||||
return image_service.list_images()
|
||||
def list_images(
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> list[dict]:
|
||||
rows = image_service.list_images()
|
||||
# Keyed by stack id like the stack list, or a stack whose folder name
|
||||
# compose had to normalise ("Nextcloud" -> "nextcloud") showed up as a
|
||||
# stranger whose stack had been removed.
|
||||
ids = compose_service.stack_ids_by_project_in(session)
|
||||
for row in rows:
|
||||
row["stacks"] = sorted({compose_service.owner_stack(p, ids) for p in row["stacks"]})
|
||||
return rows
|
||||
|
||||
|
||||
@router.get("/updates")
|
||||
@@ -160,20 +170,40 @@ async def scan_image(
|
||||
@router.post("/scan-all")
|
||||
async def scan_all(
|
||||
request: Request,
|
||||
response: Response,
|
||||
background: bool = Query(False),
|
||||
session: Session = Depends(get_session),
|
||||
user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
"""Scan every image a stack is using. Serial, and one sweep at a time."""
|
||||
images = sorted({row["tag"] for row in image_service.list_images() if row["stacks"]})
|
||||
"""Scan every image a stack is using. Serial, and one sweep at a time.
|
||||
|
||||
``background=true`` returns 202 with the sweep's status at once; poll
|
||||
``/scan-status`` for progress and the result. That is what the UI does —
|
||||
waiting on the response outlived the proxy's timeout.
|
||||
"""
|
||||
# Images of StackPilot's own stacks — not of every compose project on the
|
||||
# host, which "a stack is using" used to include (other tools' projects,
|
||||
# leftovers of deleted stacks).
|
||||
ids = compose_service.stack_ids_by_project_in(session)
|
||||
images = sorted(
|
||||
{row["tag"] for row in image_service.list_images() if set(row["stacks"]) & ids.keys()}
|
||||
)
|
||||
username, ip = user.username, _ip(request)
|
||||
|
||||
def audit(own: Session, result: dict) -> None:
|
||||
audit_service.record(
|
||||
own, user=username, action="image.scan-all", target="*",
|
||||
detail=f"{result['scanned']} scanned, {result['failed']} failed", ip=ip,
|
||||
)
|
||||
|
||||
try:
|
||||
if background:
|
||||
response.status_code = 202
|
||||
return scan_service.start_sweep(images, on_done=audit)
|
||||
result = await scan_service.sweep(session, images)
|
||||
except scan_service.ScanError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
audit_service.record(
|
||||
session, user=user.username, action="image.scan-all", target="*",
|
||||
detail=f"{result['scanned']} scanned, {result['failed']} failed",
|
||||
ip=_ip(request),
|
||||
)
|
||||
audit(session, result)
|
||||
return result
|
||||
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ from auth import get_current_user, require_admin
|
||||
from database import engine, get_session
|
||||
from docker_client import DockerError
|
||||
from models.user import User
|
||||
from services import audit_service, network_service
|
||||
from services import audit_service, compose_service, network_service
|
||||
from services.prune_job import PruneBusy
|
||||
|
||||
router = APIRouter(prefix="/api/networks", tags=["networks"])
|
||||
@@ -44,8 +44,15 @@ class ContainerRef(BaseModel):
|
||||
|
||||
|
||||
@router.get("")
|
||||
def list_networks(_user: User = Depends(get_current_user)) -> list[dict]:
|
||||
return network_service.list_networks()
|
||||
def list_networks(
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> list[dict]:
|
||||
rows = network_service.list_networks()
|
||||
ids = compose_service.stack_ids_by_project_in(session)
|
||||
for row in rows:
|
||||
row["stack"] = compose_service.owner_stack(row["stack"], ids)
|
||||
return rows
|
||||
|
||||
|
||||
# Before /{network_id}, which would otherwise take "prune-status" for an id.
|
||||
|
||||
@@ -13,6 +13,7 @@ from sqlmodel import Session
|
||||
|
||||
from auth import require_admin
|
||||
from database import get_session
|
||||
from models.stack import Stack
|
||||
from models.user import User
|
||||
from services import audit_service, secret_service
|
||||
from services.compose_edit_service import EditError
|
||||
@@ -51,6 +52,16 @@ def _guard(fn, *args, **kwargs):
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
def _stack(session: Session, stack_id: str) -> None:
|
||||
"""404 for a stack that does not exist.
|
||||
|
||||
Without it a typo in the URL created ``<STACKS_DIR>/<typo>/.secrets`` on
|
||||
disk, and ``..`` reached the directory above the stacks.
|
||||
"""
|
||||
if not session.get(Stack, stack_id):
|
||||
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
|
||||
|
||||
|
||||
|
||||
|
||||
def list_secrets(stack_id: str) -> list[dict]:
|
||||
@@ -82,7 +93,12 @@ def detach_secret(stack_id: str, body: DetachBody) -> dict:
|
||||
|
||||
|
||||
@router.get("")
|
||||
def http_list(stack_id: str, _user: User = Depends(require_admin)) -> list[dict]:
|
||||
def http_list(
|
||||
stack_id: str,
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(require_admin),
|
||||
) -> list[dict]:
|
||||
_stack(session, stack_id)
|
||||
return list_secrets(stack_id)
|
||||
|
||||
|
||||
@@ -91,6 +107,7 @@ def http_write(
|
||||
stack_id: str, body: SecretWrite, request: Request,
|
||||
session: Session = Depends(get_session), user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
_stack(session, stack_id)
|
||||
result = write_secret(stack_id, body)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="secret.write",
|
||||
@@ -104,6 +121,7 @@ def http_delete(
|
||||
stack_id: str, kind: str, name: str, request: Request,
|
||||
session: Session = Depends(get_session), user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
_stack(session, stack_id)
|
||||
result = delete_secret(stack_id, kind, name)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="secret.delete",
|
||||
@@ -117,6 +135,7 @@ def http_attach(
|
||||
stack_id: str, body: AttachBody, request: Request,
|
||||
session: Session = Depends(get_session), user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
_stack(session, stack_id)
|
||||
result = attach_secret(stack_id, body)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="secret.attach",
|
||||
@@ -130,6 +149,7 @@ def http_detach(
|
||||
stack_id: str, body: DetachBody, request: Request,
|
||||
session: Session = Depends(get_session), user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
_stack(session, stack_id)
|
||||
result = detach_secret(stack_id, body)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="secret.detach",
|
||||
|
||||
+79
-17
@@ -9,6 +9,7 @@ from dataclasses import asdict
|
||||
from fastapi import APIRouter, Depends, File, HTTPException, Query, Request, UploadFile
|
||||
from fastapi.responses import FileResponse
|
||||
from sqlmodel import Session, select
|
||||
from starlette.background import BackgroundTask
|
||||
|
||||
from auth import get_current_user, require_admin
|
||||
from database import get_session
|
||||
@@ -36,6 +37,7 @@ from services import (
|
||||
icon_service,
|
||||
logo_service,
|
||||
notify_service,
|
||||
stack_cleanup_service,
|
||||
stack_job_service,
|
||||
stack_lock_service,
|
||||
stats_service,
|
||||
@@ -131,7 +133,7 @@ def _stack_summary(
|
||||
total = running = 0
|
||||
status = "unknown"
|
||||
else:
|
||||
info = summaries.get(stack.id)
|
||||
info = summaries.get(compose_service.project_name(stack.id))
|
||||
total = info["total"] if info else 0
|
||||
running = info["running"] if info else 0
|
||||
if stack.id in busy:
|
||||
@@ -204,17 +206,30 @@ def create_stack(
|
||||
return _stack_summary(stack)
|
||||
|
||||
|
||||
def _by_stack_id(session: Session, per_project: dict) -> dict:
|
||||
"""Re-key per-compose-project data by stack id (they differ only for a
|
||||
folder whose name compose had to normalise, see project_name)."""
|
||||
ids = compose_service.stack_ids_by_project_in(session)
|
||||
return {ids.get(project, project): value for project, value in per_project.items()}
|
||||
|
||||
|
||||
@router.get("/stats")
|
||||
def stacks_stats(_user: User = Depends(get_current_user)) -> dict:
|
||||
def stacks_stats(
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
"""Live CPU (cores) and memory usage per stack, with assigned limits."""
|
||||
return stats_service.stack_stats()
|
||||
return _by_stack_id(session, stats_service.stack_stats())
|
||||
|
||||
|
||||
@router.get("/updates")
|
||||
def stacks_updates(_user: User = Depends(get_current_user)) -> dict:
|
||||
def stacks_updates(
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
"""Per-stack image-update availability, read from the cached registry
|
||||
digests (no live registry calls — safe for the list to poll)."""
|
||||
return update_service.stacks_update_summary()
|
||||
return _by_stack_id(session, update_service.stacks_update_summary())
|
||||
|
||||
|
||||
@router.get("/icons/search")
|
||||
@@ -275,7 +290,12 @@ def get_stack(
|
||||
# What is holding the stack, so the page can offer to stop it instead
|
||||
# of only showing a spinner that never ends.
|
||||
"lock": _lock_view(lock, stack_id),
|
||||
"yaml": compose_service.read_compose(stack_id),
|
||||
# Inline credentials masked for the read-only role, like the .env.
|
||||
"yaml": (
|
||||
compose_service.read_compose(stack_id)
|
||||
if user.role == "admin"
|
||||
else compose_service.mask_secrets(compose_service.read_compose(stack_id))
|
||||
),
|
||||
# The .env is where credentials live by convention, so it is withheld
|
||||
# from the read-only role — same reasoning as the admin-only file
|
||||
# browser. Non-admins still get status, services and the compose file.
|
||||
@@ -333,21 +353,49 @@ async def delete_stack(
|
||||
user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
stack = _get_stack_or_404(session, stack_id)
|
||||
# Deleting a stack in the middle of a backup, restore or deploy pulls the
|
||||
# folder out from under it — and a backup then starts the containers it
|
||||
# stopped again, for a stack that no longer exists.
|
||||
lock = stack_lock_service.info(session, stack_id)
|
||||
if lock:
|
||||
raise HTTPException(
|
||||
status_code=409,
|
||||
detail=f"Stack '{stack_id}' is busy: {lock['action']} in progress",
|
||||
)
|
||||
try:
|
||||
await compose_service.down(stack_id)
|
||||
except Exception: # noqa: BLE001 - best-effort teardown
|
||||
pass
|
||||
result = await compose_service.down(stack_id)
|
||||
down_ok = result.get("returncode") == 0
|
||||
except Exception: # noqa: BLE001 - fall back to the API below
|
||||
down_ok = False
|
||||
if not down_ok:
|
||||
# A compose file that no longer parses makes `down` fail, and the
|
||||
# containers would keep running with nothing left here to stop them.
|
||||
try:
|
||||
removed = await asyncio.to_thread(compose_service.force_remove_project, stack_id)
|
||||
logger.info("compose down failed for '%s'; removed %d container(s) directly",
|
||||
stack_id, removed)
|
||||
except DockerError as exc:
|
||||
logger.warning("Could not remove the containers of '%s': %s", stack_id, exc)
|
||||
kept: list[str] = []
|
||||
if delete_files:
|
||||
compose_service.delete_stack_files(stack_id)
|
||||
try:
|
||||
kept = compose_service.delete_stack_files(stack_id)
|
||||
except OSError as exc:
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"The stack was stopped, but its folder could not be deleted: {exc}",
|
||||
) from exc
|
||||
icon_service.remove(stack_id)
|
||||
logo_service.forget(stack_id)
|
||||
stack_cleanup_service.forget_stack(session, stack_id)
|
||||
session.delete(stack)
|
||||
session.commit()
|
||||
audit_service.record(
|
||||
session, user=user.username, action="stack.delete", target=stack_id,
|
||||
detail=f"delete_files={delete_files}", ip=_client_ip(request),
|
||||
detail=f"delete_files={delete_files}" + (f" kept={len(kept)} mount(s)" if kept else ""),
|
||||
ip=_client_ip(request),
|
||||
)
|
||||
return {"ok": True}
|
||||
return {"ok": True, "kept": kept}
|
||||
|
||||
|
||||
@router.post("/{stack_id}/clone")
|
||||
@@ -365,7 +413,8 @@ def clone_stack(
|
||||
try:
|
||||
compose_service.clone_stack_files(stack_id, new_id)
|
||||
except compose_service.StackFileError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
status = 409 if "already exists" in str(exc) else 500
|
||||
raise HTTPException(status_code=status, detail=str(exc)) from exc
|
||||
stack = Stack(
|
||||
id=new_id,
|
||||
name=body.name,
|
||||
@@ -412,6 +461,8 @@ async def get_stack_icon(
|
||||
|
||||
|
||||
def _icon_response(path: str, media_type: str, filename: str) -> FileResponse:
|
||||
# A folder found on disk can be named anything; headers are latin-1.
|
||||
filename = filename.encode("ascii", "replace").decode("ascii").replace("?", "_")
|
||||
return FileResponse(
|
||||
path,
|
||||
media_type=media_type,
|
||||
@@ -516,6 +567,9 @@ async def _lifecycle(action_fn, action_name, stack_id, request, session, user):
|
||||
status_code=409,
|
||||
detail=f"Stack '{stack_id}' is busy: {exc.action} in progress",
|
||||
) from exc
|
||||
except compose_service.StackFileError as exc:
|
||||
# The compose call timed out; it may still be finishing on the host.
|
||||
raise HTTPException(status_code=504, detail=str(exc)) from exc
|
||||
audit_service.record(
|
||||
session, user=user.username, action=f"stack.{action_name}", target=stack_id,
|
||||
detail=f"rc={result.get('returncode')}", ip=_client_ip(request),
|
||||
@@ -683,7 +737,7 @@ async def unlock_stack(
|
||||
@router.get("/{stack_id}/logs")
|
||||
async def stack_logs(
|
||||
stack_id: str,
|
||||
tail: int = Query(200, le=2000),
|
||||
tail: int = Query(200, ge=0, le=2000),
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
@@ -696,7 +750,7 @@ async def stack_logs(
|
||||
async def service_logs(
|
||||
stack_id: str,
|
||||
service: str,
|
||||
tail: int = Query(200, le=2000),
|
||||
tail: int = Query(200, ge=0, le=2000),
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
@@ -721,13 +775,20 @@ def export_stack(
|
||||
if not os.path.isdir(directory):
|
||||
raise HTTPException(status_code=404, detail="Stack directory missing")
|
||||
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".tar.gz")
|
||||
with tarfile.open(tmp.name, "w:gz") as tar:
|
||||
tar.add(directory, arcname=stack_id)
|
||||
tmp.close()
|
||||
try:
|
||||
with tarfile.open(tmp.name, "w:gz") as tar:
|
||||
tar.add(directory, arcname=stack_id)
|
||||
except BaseException:
|
||||
os.unlink(tmp.name)
|
||||
raise
|
||||
date = compose_service.now().strftime("%Y%m%d")
|
||||
return FileResponse(
|
||||
tmp.name,
|
||||
media_type="application/gzip",
|
||||
filename=f"stack-{stack_id}-{date}.tar.gz",
|
||||
# The archive is a temp file; without this every export stayed in /tmp.
|
||||
background=BackgroundTask(os.unlink, tmp.name),
|
||||
)
|
||||
|
||||
|
||||
@@ -765,6 +826,7 @@ def set_auto_update(
|
||||
session: Session = Depends(get_session),
|
||||
user: User = Depends(require_admin),
|
||||
) -> dict:
|
||||
_get_stack_or_404(session, stack_id)
|
||||
policy = auto_update_service.upsert_policy(session, stack_id, body.enabled, body.redeploy)
|
||||
audit_service.record(
|
||||
session, user=user.username, action="stack.auto_update",
|
||||
|
||||
@@ -7,7 +7,7 @@ from sqlmodel import Session
|
||||
from auth import get_current_user, require_admin
|
||||
from database import engine, get_session
|
||||
from models.user import User
|
||||
from services import audit_service, device_service, volume_service
|
||||
from services import audit_service, compose_service, device_service, volume_service
|
||||
from services.prune_job import PruneBusy
|
||||
|
||||
router = APIRouter(tags=["volumes"])
|
||||
@@ -23,8 +23,15 @@ def _ip(request: Request) -> str:
|
||||
|
||||
|
||||
@router.get("/api/volumes")
|
||||
def list_volumes(_user: User = Depends(get_current_user)) -> list[dict]:
|
||||
return volume_service.list_volumes()
|
||||
def list_volumes(
|
||||
session: Session = Depends(get_session),
|
||||
_user: User = Depends(get_current_user),
|
||||
) -> list[dict]:
|
||||
rows = volume_service.list_volumes()
|
||||
ids = compose_service.stack_ids_by_project_in(session)
|
||||
for row in rows:
|
||||
row["stack"] = compose_service.owner_stack(row["stack"], ids)
|
||||
return rows
|
||||
|
||||
|
||||
@router.get("/api/volumes/orphaned")
|
||||
|
||||
+206
-143
@@ -2,10 +2,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
|
||||
import contextlib
|
||||
import threading
|
||||
from collections.abc import Awaitable, Callable
|
||||
|
||||
from fastapi import APIRouter, Query, WebSocket, WebSocketDisconnect
|
||||
from jose import JWTError
|
||||
@@ -14,12 +15,13 @@ from sqlmodel import Session
|
||||
from auth import decode_token, resolve_token_user
|
||||
from database import engine
|
||||
from models.setting import EVENT_PULL_FAILED, EVENT_STACK_ERROR, EVENT_STACK_START
|
||||
from models.stack import Stack
|
||||
from services import (
|
||||
audit_service,
|
||||
compose_run_service,
|
||||
compose_service,
|
||||
exec_service,
|
||||
notify_service,
|
||||
stack_lock_service,
|
||||
update_service,
|
||||
)
|
||||
|
||||
@@ -49,52 +51,93 @@ def _user_for(token: str | None):
|
||||
return user
|
||||
|
||||
|
||||
async def _authorize(websocket: WebSocket, token: str | None) -> bool:
|
||||
"""Validate the JWT supplied as a query param. Closes socket on failure."""
|
||||
if _user_for(token) is None:
|
||||
await websocket.close(code=4401)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
async def _authorize_admin(websocket: WebSocket, token: str | None) -> bool:
|
||||
"""Like _authorize but also requires the admin role (exec is root-equivalent).
|
||||
|
||||
Closes 4401 on a missing/invalid token, 4403 on a valid non-admin token."""
|
||||
async def _authorize(websocket: WebSocket, token: str | None):
|
||||
"""The user behind the token, or None after closing the socket with 4401."""
|
||||
user = _user_for(token)
|
||||
if user is None:
|
||||
await websocket.close(code=4401)
|
||||
return False
|
||||
return user
|
||||
|
||||
|
||||
async def _authorize_admin(websocket: WebSocket, token: str | None):
|
||||
"""Like _authorize but also requires the admin role.
|
||||
|
||||
Anything that changes what runs on the host is admin-only over HTTP, so it
|
||||
has to be over a socket too — the deploy console runs the same
|
||||
``compose up`` the REST ``/start`` does. Closes 4401 on a missing/invalid
|
||||
token, 4403 on a valid non-admin one."""
|
||||
user = _user_for(token)
|
||||
if user is None:
|
||||
await websocket.close(code=4401)
|
||||
return None
|
||||
if user.role != "admin":
|
||||
await websocket.close(code=4403)
|
||||
return False
|
||||
return True
|
||||
return None
|
||||
return user
|
||||
|
||||
|
||||
async def _wait_disconnect(websocket: WebSocket) -> None:
|
||||
"""Return once the client has gone. Whatever it sends is ignored.
|
||||
|
||||
A stream that only ever *sends* notices a closed socket on its next send —
|
||||
which for a quiet log or an idle Docker host can be hours away, and until
|
||||
then everything behind the socket (a ``compose logs -f``, an event reader
|
||||
thread) stays alive for nobody.
|
||||
"""
|
||||
try:
|
||||
while True:
|
||||
message = await websocket.receive()
|
||||
if message.get("type") == "websocket.disconnect":
|
||||
return
|
||||
except Exception: # noqa: BLE001 - a broken socket is a gone client
|
||||
return
|
||||
|
||||
|
||||
async def _until_gone(websocket: WebSocket, work: Awaitable[None]) -> None:
|
||||
"""Run ``work`` until it finishes or the client disconnects, whichever is first.
|
||||
|
||||
Cancelling ``work`` is what stops whatever it was reading from; see
|
||||
:func:`compose_service.stream_compose`.
|
||||
"""
|
||||
task = asyncio.ensure_future(work)
|
||||
gone = asyncio.create_task(_wait_disconnect(websocket))
|
||||
try:
|
||||
await asyncio.wait({task, gone}, return_when=asyncio.FIRST_COMPLETED)
|
||||
finally:
|
||||
for pending in (task, gone):
|
||||
pending.cancel()
|
||||
await asyncio.gather(task, gone, return_exceptions=True)
|
||||
if task.done() and not task.cancelled() and task.exception() is not None:
|
||||
exc = task.exception()
|
||||
if not isinstance(exc, WebSocketDisconnect):
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
|
||||
|
||||
async def _stream_logs(websocket: WebSocket, stack_id: str, service: str | None):
|
||||
"""Stream `docker compose logs -f` output to the client."""
|
||||
args = ["logs", "--no-color", "--tail", "200", "--timestamps", "-f"]
|
||||
if service:
|
||||
args.append(service)
|
||||
try:
|
||||
async for line in compose_service.stream_compose(stack_id, args):
|
||||
await websocket.send_text(
|
||||
json.dumps(
|
||||
{
|
||||
"type": "log",
|
||||
"stack_id": stack_id,
|
||||
"service": service,
|
||||
"line": line,
|
||||
}
|
||||
)
|
||||
)
|
||||
except WebSocketDisconnect:
|
||||
raise
|
||||
except Exception as exc: # noqa: BLE001
|
||||
# "--" so a service name can never be read as a compose option.
|
||||
args += ["--", service]
|
||||
|
||||
async def pump() -> None:
|
||||
# aclosing: cancelled while *sending*, the loop is left by an exception
|
||||
# outside the generator, which would then only be closed — and its
|
||||
# `compose logs -f` killed — whenever garbage collection got to it.
|
||||
async with contextlib.aclosing(
|
||||
compose_service.stream_compose(stack_id, args)
|
||||
) as lines:
|
||||
async for line in lines:
|
||||
await _send_log(line)
|
||||
|
||||
async def _send_log(line: str) -> None:
|
||||
await websocket.send_text(
|
||||
json.dumps({"type": "error", "detail": str(exc)})
|
||||
json.dumps({"type": "log", "stack_id": stack_id, "service": service, "line": line})
|
||||
)
|
||||
|
||||
await _until_gone(websocket, pump())
|
||||
|
||||
|
||||
@router.websocket("/ws/logs/{stack_id}")
|
||||
async def ws_stack_logs(
|
||||
@@ -105,10 +148,7 @@ async def ws_stack_logs(
|
||||
await websocket.accept()
|
||||
if not await _authorize(websocket, token):
|
||||
return
|
||||
try:
|
||||
await _stream_logs(websocket, stack_id, None)
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
await _stream_logs(websocket, stack_id, None)
|
||||
|
||||
|
||||
@router.websocket("/ws/logs/{stack_id}/{service}")
|
||||
@@ -121,10 +161,67 @@ async def ws_service_logs(
|
||||
await websocket.accept()
|
||||
if not await _authorize(websocket, token):
|
||||
return
|
||||
await _stream_logs(websocket, stack_id, service)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Deploy / update: a compose run the socket only watches
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _stack_known(stack_id: str) -> bool:
|
||||
with Session(engine) as session:
|
||||
return session.get(Stack, stack_id) is not None
|
||||
|
||||
|
||||
async def _watch_run(
|
||||
websocket: WebSocket,
|
||||
stack_id: str,
|
||||
action: str,
|
||||
username: str,
|
||||
stream: Callable,
|
||||
on_finish: Callable[[int | None], Awaitable[None]],
|
||||
) -> None:
|
||||
"""Start (or attach to) the compose run and relay it until it ends.
|
||||
|
||||
The run belongs to :mod:`services.compose_run_service`, not to this socket:
|
||||
closing the deploy console no longer releases the stack lock under a
|
||||
running compose, and the audit entry carries the real exit code.
|
||||
"""
|
||||
if not _stack_known(stack_id):
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(
|
||||
json.dumps({"type": "error", "detail": f"Stack '{stack_id}' not found"})
|
||||
)
|
||||
await websocket.close(code=4404)
|
||||
return
|
||||
try:
|
||||
await _stream_logs(websocket, stack_id, service)
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
run, queue, _attached = compose_run_service.start(
|
||||
stack_id, action, username, stream, on_finish
|
||||
)
|
||||
except compose_run_service.RunBusy as exc:
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
await websocket.close(code=4409)
|
||||
return
|
||||
|
||||
async def relay() -> None:
|
||||
while True:
|
||||
item = await queue.get()
|
||||
if item is None:
|
||||
return
|
||||
kind, payload = item
|
||||
if kind == "log":
|
||||
await websocket.send_text(json.dumps({"type": "log", "line": payload}))
|
||||
else:
|
||||
await websocket.send_text(json.dumps({"type": "done", "returncode": payload}))
|
||||
|
||||
try:
|
||||
await _until_gone(websocket, relay())
|
||||
finally:
|
||||
run.unsubscribe(queue)
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.close()
|
||||
|
||||
|
||||
@router.websocket("/ws/deploy/{stack_id}")
|
||||
@@ -135,53 +232,21 @@ async def ws_deploy(
|
||||
):
|
||||
"""Run `docker compose up -d` and stream its output (image pulls, container
|
||||
creation) to the browser so the user sees deploy progress live. Records the
|
||||
same audit entry and notification as the REST `/start` endpoint."""
|
||||
same audit entry and notification as the REST `/start` endpoint. Admin-only,
|
||||
like that endpoint."""
|
||||
await websocket.accept()
|
||||
if not await _authorize(websocket, token):
|
||||
user = await _authorize_admin(websocket, token)
|
||||
if not user:
|
||||
return
|
||||
username = decode_token(token, "access").get("sub", "unknown") if token else "unknown"
|
||||
username = user.username
|
||||
|
||||
rc: int | None = None
|
||||
disconnected = False
|
||||
# Same guard the REST lifecycle uses — the deploy console runs the very
|
||||
# same `compose up`, so it has to queue behind an in-flight operation
|
||||
# rather than race it.
|
||||
lock_session = Session(engine)
|
||||
try:
|
||||
stack_lock_service.acquire(lock_session, stack_id, "start", username)
|
||||
except stack_lock_service.StackBusy as exc:
|
||||
lock_session.close()
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
await websocket.close(code=4409)
|
||||
return
|
||||
try:
|
||||
async for kind, payload in compose_service.stream_up(stack_id):
|
||||
if kind == "log":
|
||||
await websocket.send_text(json.dumps({"type": "log", "line": payload}))
|
||||
else:
|
||||
rc = payload
|
||||
await websocket.send_text(json.dumps({"type": "done", "returncode": rc}))
|
||||
except WebSocketDisconnect:
|
||||
# Client navigated away; the compose subprocess keeps running so the
|
||||
# deploy still completes in the background.
|
||||
disconnected = True
|
||||
except Exception as exc: # noqa: BLE001
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
finally:
|
||||
with contextlib.suppress(Exception):
|
||||
stack_lock_service.release(lock_session, stack_id)
|
||||
lock_session.close()
|
||||
|
||||
ok = rc in (0, None)
|
||||
try:
|
||||
async def finish(rc: int | None) -> None:
|
||||
with Session(engine) as session:
|
||||
audit_service.record(
|
||||
session, user=username, action="stack.start", target=stack_id,
|
||||
detail=f"rc={rc} (deploy console)", ip="ws",
|
||||
)
|
||||
if ok:
|
||||
if rc == 0:
|
||||
await notify_service.notify(
|
||||
EVENT_STACK_START, f"Stack '{stack_id}' started",
|
||||
"compose up completed successfully.", session,
|
||||
@@ -191,11 +256,11 @@ async def ws_deploy(
|
||||
EVENT_STACK_ERROR, f"Stack '{stack_id}' start failed",
|
||||
"compose up returned a non-zero exit code.", session,
|
||||
)
|
||||
except Exception: # noqa: BLE001 - audit/notify are best-effort
|
||||
pass
|
||||
if not disconnected:
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.close()
|
||||
|
||||
await _watch_run(
|
||||
websocket, stack_id, "start", username,
|
||||
lambda: compose_service.stream_up(stack_id), finish,
|
||||
)
|
||||
|
||||
|
||||
@router.websocket("/ws/update/{stack_id}")
|
||||
@@ -208,47 +273,18 @@ async def ws_update(
|
||||
list can render real update progress. Same audit/notify contract as the
|
||||
REST `/update` endpoint, which stays for non-interactive callers."""
|
||||
await websocket.accept()
|
||||
if not await _authorize_admin(websocket, token):
|
||||
user = await _authorize_admin(websocket, token)
|
||||
if not user:
|
||||
return
|
||||
username = decode_token(token, "access").get("sub", "unknown") if token else "unknown"
|
||||
username = user.username
|
||||
|
||||
rc: int | None = None
|
||||
disconnected = False
|
||||
lock_session = Session(engine)
|
||||
try:
|
||||
stack_lock_service.acquire(lock_session, stack_id, "update", username)
|
||||
except stack_lock_service.StackBusy as exc:
|
||||
lock_session.close()
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
await websocket.close(code=4409)
|
||||
return
|
||||
try:
|
||||
async for kind, payload in compose_service.stream_update(stack_id):
|
||||
if kind == "log":
|
||||
await websocket.send_text(json.dumps({"type": "log", "line": payload}))
|
||||
else:
|
||||
rc = payload
|
||||
await websocket.send_text(json.dumps({"type": "done", "returncode": rc}))
|
||||
except WebSocketDisconnect:
|
||||
# Client navigated away; compose keeps running so the update finishes.
|
||||
disconnected = True
|
||||
except Exception as exc: # noqa: BLE001
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
finally:
|
||||
with contextlib.suppress(Exception):
|
||||
stack_lock_service.release(lock_session, stack_id)
|
||||
lock_session.close()
|
||||
|
||||
ok = rc in (0, None)
|
||||
try:
|
||||
async def finish(rc: int | None) -> None:
|
||||
with Session(engine) as session:
|
||||
audit_service.record(
|
||||
session, user=username, action="stack.update", target=stack_id,
|
||||
detail=f"rc={rc} (update stream)", ip="ws",
|
||||
)
|
||||
if ok:
|
||||
if rc == 0:
|
||||
await notify_service.notify(
|
||||
EVENT_STACK_START, f"Stack '{stack_id}' updated",
|
||||
"compose pull + up completed successfully.", session,
|
||||
@@ -258,16 +294,19 @@ async def ws_update(
|
||||
EVENT_PULL_FAILED, f"Stack '{stack_id}' update failed",
|
||||
"compose pull/up returned a non-zero exit code.", session,
|
||||
)
|
||||
except Exception: # noqa: BLE001 - audit/notify are best-effort
|
||||
pass
|
||||
if not disconnected:
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.close()
|
||||
if rc == 0:
|
||||
await asyncio.to_thread(update_service.refresh_stack_local, stack_id)
|
||||
|
||||
if ok:
|
||||
update_service.refresh_stack_local(stack_id)
|
||||
await _watch_run(
|
||||
websocket, stack_id, "update", username,
|
||||
lambda: compose_service.stream_update(stack_id), finish,
|
||||
)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Docker events
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
#: Docker event types worth telling the UI about. Filtered daemon-side, so the
|
||||
#: bulk of the firehose never crosses the socket.
|
||||
_EVENT_TYPES = ["container", "image", "network", "volume"]
|
||||
@@ -300,33 +339,58 @@ async def ws_events(
|
||||
"container": "jellyfin", "stack": "jellyfin"}
|
||||
|
||||
``resource`` is what the client needs to decide which queries to drop.
|
||||
|
||||
The blocking read runs on a thread of its own, not on the event loop's
|
||||
shared executor: every open tab holds one reader for as long as it is open,
|
||||
and on the executor a handful of tabs used up every worker — after which
|
||||
each ``asyncio.to_thread`` (backups, the dashboard, NFS destinations) waited
|
||||
forever.
|
||||
"""
|
||||
await websocket.accept()
|
||||
if not await _authorize(websocket, token):
|
||||
return
|
||||
from docker_client import get_client
|
||||
|
||||
loop = asyncio.get_event_loop()
|
||||
loop = asyncio.get_running_loop()
|
||||
queue: asyncio.Queue = asyncio.Queue()
|
||||
stream = None
|
||||
guard = threading.Lock()
|
||||
state: dict = {"stream": None, "stop": False}
|
||||
|
||||
def reader():
|
||||
def post(item) -> None:
|
||||
try:
|
||||
loop.call_soon_threadsafe(queue.put_nowait, item)
|
||||
except RuntimeError: # the loop is gone (shutdown)
|
||||
pass
|
||||
|
||||
def reader() -> None:
|
||||
"""Blocking read of the event stream, handed to the loop thread-safely."""
|
||||
nonlocal stream
|
||||
try:
|
||||
stream = get_client().events(decode=True, filters={"type": _EVENT_TYPES})
|
||||
with guard:
|
||||
if state["stop"]:
|
||||
# The client left while we were connecting.
|
||||
with contextlib.suppress(Exception):
|
||||
stream.close()
|
||||
return
|
||||
state["stream"] = stream
|
||||
for event in stream:
|
||||
loop.call_soon_threadsafe(queue.put_nowait, event)
|
||||
post(event)
|
||||
except Exception: # noqa: BLE001 - a closed stream lands here on teardown
|
||||
pass
|
||||
finally:
|
||||
loop.call_soon_threadsafe(queue.put_nowait, None)
|
||||
post(None)
|
||||
|
||||
task = loop.run_in_executor(None, reader)
|
||||
threading.Thread(target=reader, name="docker-events", daemon=True).start()
|
||||
gone = asyncio.create_task(_wait_disconnect(websocket))
|
||||
try:
|
||||
await websocket.send_text(json.dumps({"type": "ready"}))
|
||||
while True:
|
||||
event = await queue.get()
|
||||
getter = asyncio.ensure_future(queue.get())
|
||||
done, _ = await asyncio.wait({getter, gone}, return_when=asyncio.FIRST_COMPLETED)
|
||||
if getter not in done:
|
||||
getter.cancel()
|
||||
break # the client disconnected
|
||||
event = getter.result()
|
||||
if event is None: # reader finished — daemon gone or stream closed
|
||||
await websocket.send_text(
|
||||
json.dumps({"type": "error", "detail": "Docker event stream ended"})
|
||||
@@ -354,14 +418,14 @@ async def ws_events(
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
|
||||
finally:
|
||||
gone.cancel()
|
||||
# Closing the stream is what actually unblocks the reader thread.
|
||||
# Cancelling the executor future does not: a thread already inside a
|
||||
# blocking read keeps that read, and the thread leaks for the life of
|
||||
# the process — once per page load, with the socket held open.
|
||||
with guard:
|
||||
state["stop"] = True
|
||||
stream = state["stream"]
|
||||
if stream is not None:
|
||||
with contextlib.suppress(Exception):
|
||||
stream.close()
|
||||
task.cancel()
|
||||
|
||||
|
||||
@router.websocket("/ws/exec/{container_id}")
|
||||
@@ -373,9 +437,10 @@ async def ws_exec(
|
||||
):
|
||||
"""Interactive shell into a compose-managed container (admin only)."""
|
||||
await websocket.accept()
|
||||
if not await _authorize_admin(websocket, token):
|
||||
user = await _authorize_admin(websocket, token)
|
||||
if not user:
|
||||
return
|
||||
username = decode_token(token, "access").get("sub", "unknown") if token else "unknown"
|
||||
username = user.username
|
||||
shell = cmd or exec_service.DEFAULT_SHELL
|
||||
|
||||
try:
|
||||
@@ -402,5 +467,3 @@ async def ws_exec(
|
||||
finally:
|
||||
with contextlib.suppress(Exception):
|
||||
await websocket.close()
|
||||
|
||||
|
||||
Reference in new issue
Block a user