menzeljonas/stackpilot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove the remote-host (agent) integration (0.48.0)
CI / check (push) Successful in 7m17s
Details
CI / build-and-push (push) Successful in 1m45s
Details

Browse Source 
StackPilot now manages exactly one Docker host: the one it runs on. The
stackpilot-agent sidecar and everything that proxied to it are gone — 4721
lines deleted against 657 added.

Deleted outright: agent/ (image, compose, env), agent_app.py, models/agent.py,
routers/agents.py (1200 lines), services/agent_service.py, the agent API client,
RemoteStackDetail, the host components and AgentStacksSection. That removes 57
API routes and the three /ws/agent-* proxies.

Threaded out everywhere else, which was the bulk of the work. Every API module
carried an optional agentId that switched the base path; every page that listed
Docker objects rendered one section per host behind a HostHeader; Files had a
host switcher; the New Stack editor and the template dialog had host selectors;
schedules, auto-update policies and stack summaries carried agent_id. All of it
is gone, and the typechecker drove the sweep — 85 files touched, tsc and the
build clean.

Two things the removal exposed as dead weight rather than merely unused:

compose_service kept an in-process busy set purely because the agent needed a
lock and has no database. With the agent gone that was a second source of truth
next to the real DB lock, so it is deleted; compute_status now reports only what
the containers say and the two callers that want "updating" overlay the lock.
StacksTable's linkBase prop only ever existed to point at /hosts/{id}/stacks.

The dashboard's "Hosts 1/1 online" KPI can no longer say anything else, so the
tile and the KPIs behind it are gone and the row is five wide.

Upgrading matters here. An existing install still has an agent table holding
each remote host's URL and bearer token — full Docker control of that host,
sitting in the database with nothing left to use it. _drop_removed_schema drops
it on first start, and drops the agent_id columns where the SQLite build
supports DROP COLUMN. Each statement runs in its own transaction on purpose: a
failed DDL poisons the transaction it is in, so sharing one would let an
unsupported column drop take the table drop down with it. test_agent_removal
covers both branches plus the fresh-install and idempotent cases, and an
end-to-end run against a seeded pre-0.48 database confirms the table is gone and
every /api/agents route answers 404.

Docstrings that justified a design by "shared with the agent, which has no
database" were rewritten rather than left lying: update_service's persistence
callback and image_status_store are still the right split (registry logic stays
testable without a database), but for that reason now, not the old one. The
README's multi-host sections are removed and an upgrade note explains what to do
with running agent containers; ROADMAP keeps its history behind a note saying
the feature it describes no longer exists.

CI no longer builds or pushes stackpilot-agent.

735 tests pass, ruff and tsc clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG
This commit is contained in:
menzelj
2026-08-31 14:11:54 +02:00
co-authored by Claude Opus 5
parent 09bed274eb
commit 51d1998307
85 changed files with 653 additions and 4717 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/agent_app.py
-857
View File
@@ -1,857 +0,0 @@
"""StackPilot agent — a slim, token-guarded Docker Compose API for one host.
The agent runs on each remote host (same image as the backend, different CMD).
It has no users, no database and no UI: it exposes just enough of the stack /
system surface for a central StackPilot to manage this host's compose stacks,
authenticated by a single shared bearer token (``AGENT_TOKEN``).
All compose/Docker logic is reused from the backend's ``compose_service`` and
``docker_client`` so behaviour matches the local host exactly.
"""
from __future__ import annotations
import asyncio
import logging
import os
import shutil
from dataclasses import asdict
import tempfile
import json
from fastapi import (
Depends,
FastAPI,
File,
Form,
Header,
HTTPException,
Query,
Request,
UploadFile,
WebSocket,
WebSocketDisconnect,
)
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
from pydantic import BaseModel
from config import settings
from version import APP_VERSION
from docker_client import DockerError, get_client, safe_call
from services import (
backup_service,
compose_edit_service,
compose_service,
container_service,
device_service,
exec_service,
file_service,
image_service,
network_service,
secret_service,
stats_service,
update_service,
volume_service,
)
logger = logging.getLogger("stackpilot.agent")
# Map network_service's DockerError codes to HTTP status. forbidden is mapped to
# 400 (not 403) so the central proxy doesn't misread it as a token failure.
_DOCKER_STATUS = {"invalid_request": 400, "forbidden": 400, "not_found": 404}
def _map_docker(exc: DockerError):
code = _DOCKER_STATUS.get(exc.error)
if code:
raise HTTPException(status_code=code, detail=exc.detail or exc.error)
raise exc # falls through to the global 502 DockerError handler
AGENT_VERSION = APP_VERSION
# --------------------------------------------------------------------------- #
# Auth
# --------------------------------------------------------------------------- #
def verify_token(authorization: str = Header(default="")) -> None:
expected = settings.AGENT_TOKEN
if not expected:
raise HTTPException(status_code=503, detail="Agent token not configured")
if authorization != f"Bearer {expected}":
raise HTTPException(status_code=401, detail="Invalid agent token")
# --------------------------------------------------------------------------- #
# Schemas
# --------------------------------------------------------------------------- #
class StackBody(BaseModel):
name: str | None = None
yaml: str | None = None
env: str | None = None
class NetworkCreateBody(BaseModel):
name: str
driver: str = "bridge"
subnet: str | None = None
gateway: str | None = None
internal: bool = False
attachable: bool = True
class ContainerRefBody(BaseModel):
container: str
aliases: list[str] | None = None
force: bool = False
class FileWriteBody(BaseModel):
path: str
content: str
class FileNameBody(BaseModel):
path: str
name: str
class FileRenameBody(BaseModel):
path: str
new_name: str
class FileTransferBody(BaseModel):
src: str
dest_dir: str
overwrite: bool = False
def _file_guard(fn, *args, **kwargs):
try:
return fn(*args, **kwargs)
except file_service.BrowseError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
# --------------------------------------------------------------------------- #
# Helpers
# --------------------------------------------------------------------------- #
def _summary(stack_id: str, summaries: dict | None = None) -> dict:
if summaries is None:
try:
containers = compose_service.containers_for_stack(stack_id)
total = len(containers)
running = sum(1 for c in containers if c.state == "running")
status = compose_service.compute_status(stack_id, containers)
except DockerError:
total = running = 0
status = "unknown"
else:
info = summaries.get(stack_id)
total = info["total"] if info else 0
running = info["running"] if info else 0
if compose_service.is_busy(stack_id):
status = "updating"
else:
status = info["status"] if info else "stopped"
return {
"id": stack_id,
"name": stack_id,
"description": None,
"status": status,
"service_count": total,
"running_count": running,
"created_at": None,
"updated_at": None,
}
def _hostname() -> str:
return os.uname().nodename
def _mem_info() -> tuple[int, int]:
"""Return (total_bytes, used_bytes) from meminfo (used = total - available)."""
for base in (settings.HOST_PROC_PATH, "/proc"):
try:
vals: dict[str, int] = {}
with open(os.path.join(base, "meminfo"), "r", encoding="utf-8") as fh:
for line in fh:
parts = line.split(":")
if len(parts) == 2 and parts[0] in ("MemTotal", "MemAvailable", "MemFree"):
try:
vals[parts[0]] = int(parts[1].split()[0]) * 1024 # kB -> bytes
except ValueError:
pass
total = vals.get("MemTotal", 0)
available = vals.get("MemAvailable", vals.get("MemFree", 0))
return total, max(total - available, 0)
except OSError:
continue
return 0, 0
def _disk_info() -> tuple[int, int]:
"""Return (total_bytes, used_bytes) for the host disk backing the stacks dir."""
for path in (settings.STACKS_DIR, "/"):
try:
usage = shutil.disk_usage(path)
return usage.total, usage.used
except OSError:
continue
return 0, 0
def _system_info() -> dict:
docker_version = ""
host_os = ""
running = total = compose_running = 0
try:
client = get_client()
docker_version = safe_call(client.version).get("Version", "")
info = safe_call(client.info)
host_os = info.get("OperatingSystem", "")
running = info.get("ContainersRunning", 0)
total = info.get("Containers", 0)
# Running compose-managed containers — the dashboard's container card
# compares this across hosts; counting everything would include this
# agent itself and skew the bars.
compose_running = len(
safe_call(client.api.containers, filters={"label": compose_service.COMPOSE_LABEL})
)
except DockerError as exc:
docker_version = f"unavailable ({exc.error})"
mem_total, mem_used = _mem_info()
disk_total, disk_used = _disk_info()
return {
"hostname": _hostname(),
"docker_version": docker_version,
"host_os": host_os,
"cpu_cores": os.cpu_count() or 0,
"mem_total": mem_total,
"mem_used": mem_used,
"disk_total": disk_total,
"disk_used": disk_used,
"containers_running": running,
"containers_total": total,
"compose_running": compose_running,
}
# --------------------------------------------------------------------------- #
# App
# --------------------------------------------------------------------------- #
app = FastAPI(title="StackPilot Agent", version=AGENT_VERSION)
@app.exception_handler(DockerError)
async def _docker_error(_request: Request, exc: DockerError):
return JSONResponse(status_code=502, content={"error": exc.error, "detail": exc.detail})
@app.get("/agent/ping", dependencies=[Depends(verify_token)])
def ping() -> dict:
return {"ok": True, "hostname": _hostname(), "version": AGENT_VERSION}
@app.get("/agent/system", dependencies=[Depends(verify_token)])
def system() -> dict:
return _system_info()
@app.get("/agent/stacks", dependencies=[Depends(verify_token)])
def list_stacks() -> list[dict]:
try:
summaries = compose_service.stack_status_summaries()
except DockerError:
summaries = {}
return [_summary(sid, summaries) for sid in compose_service.discover_stacks()]
@app.get("/agent/stacks/stats", dependencies=[Depends(verify_token)])
def stacks_stats() -> dict:
return stats_service.stack_stats()
@app.get("/agent/stacks/updates", dependencies=[Depends(verify_token)])
def stacks_updates() -> dict:
"""Per-stack image-update availability from the cached digests."""
return update_service.stacks_update_summary()
@app.get("/agent/stacks/{stack_id}", dependencies=[Depends(verify_token)])
def get_stack(stack_id: str) -> dict:
directory = compose_service.stack_dir(stack_id)
if not os.path.isdir(directory):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
try:
raw = compose_service.containers_for_stack(stack_id)
containers = [asdict(c) for c in raw]
status = compose_service.compute_status(stack_id, raw)
except DockerError:
containers = []
status = "unknown"
return {
"id": stack_id,
"name": stack_id,
"description": None,
"status": status,
"yaml": compose_service.read_compose(stack_id),
"env": compose_service.read_env(stack_id),
"containers": containers,
"created_at": None,
"updated_at": None,
}
@app.post("/agent/stacks", dependencies=[Depends(verify_token)], status_code=201)
def create_stack(body: StackBody) -> dict:
if not body.name:
raise HTTPException(status_code=400, detail="name is required")
stack_id = compose_service.slugify(body.name)
if os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=409, detail=f"Stack '{stack_id}' already exists")
compose_service.write_compose(stack_id, body.yaml or "services:\n")
if body.env:
compose_service.write_env(stack_id, body.env)
return _summary(stack_id)
@app.put("/agent/stacks/{stack_id}", dependencies=[Depends(verify_token)])
def update_stack(stack_id: str, body: StackBody) -> dict:
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
if body.yaml is not None:
compose_service.write_compose(stack_id, body.yaml)
if body.env is not None:
compose_service.write_env(stack_id, body.env)
return _summary(stack_id)
@app.delete("/agent/stacks/{stack_id}", dependencies=[Depends(verify_token)])
async def delete_stack(stack_id: str, delete_files: bool = Query(True)) -> dict:
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
try:
await compose_service.down(stack_id)
except Exception: # noqa: BLE001 - best-effort teardown
pass
if delete_files:
compose_service.delete_stack_files(stack_id)
return {"ok": True}
_ACTIONS = {
"start": compose_service.up,
"stop": compose_service.stop,
"restart": compose_service.restart,
"pull": compose_service.pull,
"update": compose_service.update,
"down": compose_service.down,
}
@app.post("/agent/stacks/{stack_id}/{action}", dependencies=[Depends(verify_token)])
async def lifecycle(stack_id: str, action: str) -> dict:
fn = _ACTIONS.get(action)
if not fn:
raise HTTPException(status_code=400, detail=f"Unknown action '{action}'")
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
result = await fn(stack_id)
if result.get("returncode") not in (0, None):
raise HTTPException(
status_code=500,
detail={
"error": f"compose {action} failed",
"detail": result.get("stderr", "").strip()[-2000:],
},
)
if action in ("pull", "update"):
update_service.refresh_stack_local(stack_id)
return result
@app.get("/agent/stacks/{stack_id}/logs", dependencies=[Depends(verify_token)])
async def stack_logs(stack_id: str, tail: int = Query(200, le=2000)) -> dict:
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
result = await compose_service.logs(stack_id, tail=tail)
return {"logs": result.get("stdout", "") + result.get("stderr", "")}
@app.get("/agent/stacks/{stack_id}/updates", dependencies=[Depends(verify_token)])
async def stack_updates(stack_id: str, refresh: bool = Query(True)) -> dict:
"""Update status for this stack's images (used by central auto-update)."""
return await update_service.stack_updates(stack_id, refresh=refresh)
# --------------------------------------------------------------------------- #
# Secrets & configs (per-stack, file-based)
# --------------------------------------------------------------------------- #
class SecretWriteBody(BaseModel):
kind: str = "secret"
name: str
content: str
class SecretAttachBody(BaseModel):
kind: str = "secret"
name: str
service: str
target: str | None = None
class SecretDetachBody(BaseModel):
kind: str = "secret"
name: str
service: str
def _ensure_stack(stack_id: str) -> None:
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
def _secret_guard(fn, *args, **kwargs):
try:
return fn(*args, **kwargs)
except (secret_service.SecretError, compose_edit_service.EditError) as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.get("/agent/stacks/{stack_id}/secrets", dependencies=[Depends(verify_token)])
def agent_list_secrets(stack_id: str) -> list:
_ensure_stack(stack_id)
return secret_service.list_all(stack_id)
@app.put("/agent/stacks/{stack_id}/secrets", dependencies=[Depends(verify_token)])
def agent_write_secret(stack_id: str, body: SecretWriteBody) -> dict:
_ensure_stack(stack_id)
return _secret_guard(secret_service.write_secret, stack_id, body.kind, body.name, body.content)
@app.delete("/agent/stacks/{stack_id}/secrets/{kind}/{name}", dependencies=[Depends(verify_token)])
def agent_delete_secret(stack_id: str, kind: str, name: str) -> dict:
_ensure_stack(stack_id)
_secret_guard(secret_service.delete_secret, stack_id, kind, name)
return {"ok": True}
@app.post("/agent/stacks/{stack_id}/secrets/attach", dependencies=[Depends(verify_token)])
def agent_attach_secret(stack_id: str, body: SecretAttachBody) -> dict:
_ensure_stack(stack_id)
if not secret_service.exists(stack_id, body.kind, body.name):
raise HTTPException(status_code=404, detail="secret not found")
new_yaml = _secret_guard(secret_service.attach, stack_id, body.kind, body.name, body.service, body.target)
return {"ok": True, "yaml": new_yaml}
@app.post("/agent/stacks/{stack_id}/secrets/detach", dependencies=[Depends(verify_token)])
def agent_detach_secret(stack_id: str, body: SecretDetachBody) -> dict:
_ensure_stack(stack_id)
new_yaml = _secret_guard(secret_service.detach, stack_id, body.kind, body.name, body.service)
return {"ok": True, "yaml": new_yaml}
@app.get("/agent/stacks/{stack_id}/backup/inventory", dependencies=[Depends(verify_token)])
async def backup_inventory(stack_id: str) -> dict:
"""What a backup of this stack would capture (see routers/backups.py)."""
_ensure_stack(stack_id)
return await asyncio.to_thread(backup_service.plan, stack_id)
@app.get("/agent/stacks/{stack_id}/backup", dependencies=[Depends(verify_token)])
async def backup_stack(
stack_id: str,
include_volumes: bool = Query(True),
include_binds: bool = Query(True),
stop_first: bool = Query(True),
binds: list[str] | None = Query(None),
volumes: list[str] | None = Query(None),
):
if not os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
try:
path, report = await backup_service.create_backup_ex(
stack_id, stack_id, include_volumes=include_volumes, stop_first=stop_first,
include_binds=include_binds, binds=binds, volumes=volumes,
)
except backup_service.BackupError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return FileResponse(
path,
media_type="application/gzip",
filename=backup_service.backup_filename(stack_id, include_volumes),
headers={"X-Stackpilot-Backup": json.dumps({
"size": report.get("size"),
"binds": report.get("binds", []),
"volumes": report.get("volumes", []),
"skipped": report.get("skipped", []),
"path_mismatch": report.get("path_mismatch"),
})},
)
@app.post("/agent/stacks/restore", dependencies=[Depends(verify_token)])
async def restore_stack(
file: UploadFile = File(...),
target_id: str | None = Form(None),
overwrite: bool = Form(False),
restore_volumes: bool = Form(True),
restore_binds: bool = Form(True),
) -> dict:
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".tar.gz")
try:
while chunk := await file.read(1024 * 1024):
tmp.write(chunk)
tmp.close()
target = compose_service.slugify(target_id) if target_id else None
try:
return backup_service.restore_backup(
tmp.name, target_id=target, overwrite=overwrite,
restore_volumes=restore_volumes, restore_binds=restore_binds,
)
except backup_service.BackupError as exc:
code = 409 if "already exists" in str(exc) else 400
raise HTTPException(status_code=code, detail=str(exc)) from exc
finally:
if os.path.exists(tmp.name):
os.unlink(tmp.name)
# --------------------------------------------------------------------------- #
# Networks
# --------------------------------------------------------------------------- #
@app.get("/agent/networks", dependencies=[Depends(verify_token)])
def list_networks() -> list[dict]:
return network_service.list_networks()
@app.get("/agent/networks/{network_id}", dependencies=[Depends(verify_token)])
def inspect_network(network_id: str) -> dict:
try:
return network_service.inspect_network(network_id)
except DockerError as exc:
_map_docker(exc)
@app.get("/agent/networks/{network_id}/containers", dependencies=[Depends(verify_token)])
def network_containers(network_id: str) -> list[dict]:
try:
return network_service.connectable_containers(network_id)
except DockerError as exc:
_map_docker(exc)
@app.post("/agent/networks/{network_id}/connect", dependencies=[Depends(verify_token)])
def connect_container(network_id: str, body: ContainerRefBody) -> dict:
try:
network_service.connect_container(network_id, body.container, body.aliases)
except DockerError as exc:
_map_docker(exc)
return {"ok": True}
@app.post("/agent/networks/{network_id}/disconnect", dependencies=[Depends(verify_token)])
def disconnect_container(network_id: str, body: ContainerRefBody) -> dict:
try:
network_service.disconnect_container(network_id, body.container, body.force)
except DockerError as exc:
_map_docker(exc)
return {"ok": True}
@app.post("/agent/networks", dependencies=[Depends(verify_token)], status_code=201)
def create_network(body: NetworkCreateBody) -> dict:
try:
return network_service.create_network(body.model_dump())
except DockerError as exc:
_map_docker(exc)
@app.delete("/agent/networks/{network_id}", dependencies=[Depends(verify_token)])
def delete_network(network_id: str) -> dict:
try:
network_service.delete_network(network_id)
except DockerError as exc:
_map_docker(exc)
return {"ok": True}
@app.post("/agent/networks/prune", dependencies=[Depends(verify_token)])
def prune_networks() -> dict:
return network_service.prune_networks()
# --------------------------------------------------------------------------- #
# Images
# --------------------------------------------------------------------------- #
@app.get("/agent/images", dependencies=[Depends(verify_token)])
def list_images() -> list[dict]:
return image_service.list_images()
@app.get("/agent/images/updates", dependencies=[Depends(verify_token)])
def image_updates() -> dict:
return update_service.get_cache()
@app.post("/agent/images/check", dependencies=[Depends(verify_token)])
async def image_check() -> dict:
return await update_service.check_all()
@app.post("/agent/images/prune", dependencies=[Depends(verify_token)])
def image_prune(all_unused: bool = Query(False, alias="all")) -> dict:
return image_service.prune_images(all_unused)
# --------------------------------------------------------------------------- #
# Containers (single-container inspect + lifecycle)
# --------------------------------------------------------------------------- #
@app.get("/agent/containers/{container_id}", dependencies=[Depends(verify_token)])
def inspect_container(container_id: str) -> dict:
return container_service.inspect_container(container_id)
@app.post("/agent/containers/{container_id}/{action}", dependencies=[Depends(verify_token)])
def container_action(container_id: str, action: str) -> dict:
return container_service.container_action(container_id, action)
# --------------------------------------------------------------------------- #
# Volumes
# --------------------------------------------------------------------------- #
@app.get("/agent/volumes", dependencies=[Depends(verify_token)])
def list_volumes() -> list[dict]:
return volume_service.list_volumes()
@app.get("/agent/volumes/sizes", dependencies=[Depends(verify_token)])
def volume_sizes(force: bool = Query(False)) -> dict:
return volume_service.volume_sizes(force=force)
@app.delete("/agent/volumes/{name}", dependencies=[Depends(verify_token)])
def delete_volume(name: str, force: bool = Query(False)) -> dict:
vols = {v["name"]: v for v in volume_service.list_volumes()}
if name in vols and vols[name]["in_use"] and not force:
raise HTTPException(
status_code=409,
detail={
"error": "volume_in_use",
"detail": f"Volume '{name}' is used by: {', '.join(vols[name]['used_by'])}",
},
)
volume_service.remove_volume(name, force=force)
return {"ok": True}
@app.post("/agent/volumes/prune", dependencies=[Depends(verify_token)])
def prune_volumes() -> dict:
return volume_service.prune_volumes()
# --------------------------------------------------------------------------- #
# File browser (sandboxed by this agent's ALLOWED_BROWSE_ROOTS/HOST_ROOT_PREFIX)
# --------------------------------------------------------------------------- #
@app.get("/agent/files/list", dependencies=[Depends(verify_token)])
def files_list(path: str = Query("/"), show_hidden: bool = Query(False)) -> dict:
return _file_guard(device_service.browse, path, show_hidden)
@app.get("/agent/files/read", dependencies=[Depends(verify_token)])
def files_read(path: str = Query(...)) -> dict:
return _file_guard(file_service.read_file, path)
@app.get("/agent/files/download", dependencies=[Depends(verify_token)])
def files_download(path: str = Query(...)):
if _file_guard(file_service.is_dir, path):
filename, chunks = _file_guard(file_service.open_archive, path)
# Stream the zip as it's built (no temp file, starts immediately).
return StreamingResponse(
chunks, media_type="application/zip",
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
)
real, filename = _file_guard(file_service.resolve_download, path)
return FileResponse(real, filename=filename, media_type="application/octet-stream")
@app.put("/agent/files/write", dependencies=[Depends(verify_token)])
def files_write(body: FileWriteBody) -> dict:
return _file_guard(file_service.write_file, body.path, body.content)
@app.post("/agent/files/mkdir", dependencies=[Depends(verify_token)])
def files_mkdir(body: FileNameBody) -> dict:
return _file_guard(file_service.create_dir, body.path, body.name)
@app.post("/agent/files/touch", dependencies=[Depends(verify_token)])
def files_touch(body: FileNameBody) -> dict:
return _file_guard(file_service.create_file, body.path, body.name)
@app.post("/agent/files/rename", dependencies=[Depends(verify_token)])
def files_rename(body: FileRenameBody) -> dict:
return _file_guard(file_service.rename, body.path, body.new_name)
@app.post("/agent/files/copy", dependencies=[Depends(verify_token)])
def files_copy(body: FileTransferBody) -> dict:
return _file_guard(file_service.copy, body.src, body.dest_dir, body.overwrite)
@app.post("/agent/files/move", dependencies=[Depends(verify_token)])
def files_move(body: FileTransferBody) -> dict:
return _file_guard(file_service.move, body.src, body.dest_dir, body.overwrite)
@app.delete("/agent/files", dependencies=[Depends(verify_token)])
def files_delete(path: str = Query(...), recursive: bool = Query(False)) -> dict:
return _file_guard(file_service.delete, path, recursive)
@app.post("/agent/files/upload", dependencies=[Depends(verify_token)])
async def files_upload(
path: str = Form(...),
overwrite: bool = Form(False),
rel_path: str = Form(""),
file: UploadFile = File(...),
) -> dict:
real = _file_guard(
file_service.upload_target, path, file.filename or "", overwrite, rel_path or None
)
tmp = tempfile.NamedTemporaryFile(delete=False, dir=os.path.dirname(real))
try:
while chunk := await file.read(1024 * 1024):
tmp.write(chunk)
tmp.close()
os.replace(tmp.name, real)
except OSError as exc:
if os.path.exists(tmp.name):
os.unlink(tmp.name)
raise HTTPException(status_code=400, detail=f"Upload failed: {exc}") from exc
return {"ok": True, "name": rel_path or file.filename}
@app.websocket("/agent/ws/logs/{stack_id}")
async def ws_logs(websocket: WebSocket, stack_id: str, token: str | None = Query(default=None)):
"""Stream `docker compose logs -f` to the central app (token via query param)."""
await websocket.accept()
expected = settings.AGENT_TOKEN
if not expected or token != expected:
await websocket.close(code=4401)
return
if not os.path.isdir(compose_service.stack_dir(stack_id)):
await websocket.send_text(json.dumps({"type": "error", "detail": "stack not found"}))
await websocket.close()
return
args = ["logs", "--no-color", "--tail", "200", "--timestamps", "-f"]
try:
async for line in compose_service.stream_compose(stack_id, args):
await websocket.send_text(
json.dumps({"type": "log", "stack_id": stack_id, "service": None, "line": line})
)
except WebSocketDisconnect:
pass
except Exception as exc: # noqa: BLE001
try:
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
except Exception: # noqa: BLE001
pass
@app.websocket("/agent/ws/deploy/{stack_id}")
async def ws_deploy(websocket: WebSocket, stack_id: str, token: str | None = Query(default=None)):
"""Run `docker compose up -d` and stream its output to the central app so the
browser sees deploy progress live (token via query param)."""
await websocket.accept()
expected = settings.AGENT_TOKEN
if not expected or token != expected:
await websocket.close(code=4401)
return
if not os.path.isdir(compose_service.stack_dir(stack_id)):
await websocket.send_text(json.dumps({"type": "error", "detail": "stack not found"}))
await websocket.close()
return
compose_service.mark_busy(stack_id)
try:
async for kind, payload in compose_service.stream_up(stack_id):
if kind == "log":
await websocket.send_text(json.dumps({"type": "log", "line": payload}))
else:
await websocket.send_text(json.dumps({"type": "done", "returncode": payload}))
except WebSocketDisconnect:
# Browser navigated away; the compose subprocess keeps running.
pass
except Exception as exc: # noqa: BLE001
try:
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
except Exception: # noqa: BLE001
pass
finally:
compose_service.clear_busy(stack_id)
@app.websocket("/agent/ws/exec/{container_id}")
async def ws_exec(
websocket: WebSocket,
container_id: str,
token: str | None = Query(default=None),
cmd: str | None = Query(default=None),
):
"""Interactive shell into a compose-managed container (token via query)."""
await websocket.accept()
expected = settings.AGENT_TOKEN
if not expected or token != expected:
await websocket.close(code=4401)
return
shell = cmd or exec_service.DEFAULT_SHELL
try:
exec_id = exec_service.create_exec(container_id, [shell])
holder, raw = exec_service.start_exec(exec_id)
except Exception as exc: # noqa: BLE001
try:
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
except Exception: # noqa: BLE001
pass
await websocket.close()
return
try:
await exec_service.pump_exec(websocket, exec_id, holder, raw)
except WebSocketDisconnect:
pass
finally:
try:
await websocket.close()
except Exception: # noqa: BLE001
pass
@app.get("/agent/health")
def health() -> dict:
return {"status": "ok"}
backend/config.py
-4
View File
@@ -38,10 +38,6 @@ class Settings(BaseSettings):
# Throwaway image used to read/write named-volume contents during backup.
BACKUP_HELPER_IMAGE: str = "alpine:latest"
# Multi-host agent: shared bearer token the agent requires on every request.
# Only used when running the agent app (agent_app:app).
AGENT_TOKEN: str = ""
# Host browser sandbox roots. Deliberately does NOT contain "/": that entry
# makes _is_allowed() wave through every path, i.e. it switches the sandbox
# off. Add it back explicitly if you really want the whole filesystem.
backend/database.py
+49
View File
@@ -7,6 +7,7 @@ from collections.abc import Generator
from typing import Optional
from sqlalchemy import inspect, text
from sqlalchemy.exc import OperationalError
from sqlmodel import Session, SQLModel, create_engine
from config import settings
@@ -93,12 +94,60 @@ def _ensure_model_columns() -> None:
logger.info("Schema migration: added column %s.%s", table_name, col.name)
#: Tables and columns left behind when the remote-host (agent) integration was
#: removed in 0.48.0. SQLite before 3.35 cannot DROP COLUMN, and the rows are
#: harmless dead weight either way — so the table goes and the columns are only
#: dropped where the SQLite build supports it.
_REMOVED_TABLES = ("agent",)
_REMOVED_COLUMNS = (("autoupdate", "agent_id"), ("backupschedule", "agent_id"))
def _drop_removed_schema() -> None:
"""Clean up schema left over from features that no longer exist.
Without this an upgraded install keeps an ``agent`` table full of host URLs
and bearer tokens for a feature that is gone — credentials sitting in the
database with nothing to use them.
Each statement runs in its own transaction on purpose: a failed DDL poisons
the transaction it is in, so sharing one would mean a single unsupported
DROP COLUMN takes the table drop down with it.
"""
insp = inspect(engine)
live = set(insp.get_table_names())
for table in _REMOVED_TABLES:
if table not in live:
continue
with engine.begin() as conn:
conn.execute(text(f'DROP TABLE "{table}"'))
logger.info("Schema migration: dropped obsolete table %s", table)
for table, column in _REMOVED_COLUMNS:
if table not in live:
continue
if column not in {c["name"] for c in insp.get_columns(table)}:
continue
try:
with engine.begin() as conn:
conn.execute(text(f'ALTER TABLE "{table}" DROP COLUMN "{column}"'))
logger.info("Schema migration: dropped obsolete column %s.%s", table, column)
except OperationalError:
# SQLite < 3.35 has no DROP COLUMN. The column is nullable and
# nothing reads it any more, so leaving it is harmless.
logger.info(
"Leaving obsolete column %s.%s in place (this SQLite cannot "
"drop columns); it is unused", table, column
)
def init_db() -> None:
# Import models so they are registered on SQLModel.metadata.
import models # noqa: F401
SQLModel.metadata.create_all(engine)
_ensure_model_columns()
_drop_removed_schema()
def get_session() -> Generator[Session, None, None]:
backend/main.py
-2
View File
@@ -15,7 +15,6 @@ from version import APP_VERSION
from database import engine, init_db
from docker_client import DockerError
from routers import (
agents,
audit,
auth,
backups,
@@ -132,7 +131,6 @@ app.include_router(backups.router)
app.include_router(destinations.router)
app.include_router(schedules.router)
app.include_router(networks.router)
app.include_router(agents.router)
app.include_router(ws.router)
backend/models/__init__.py
+1 -2
View File
@@ -1,5 +1,4 @@
"""SQLModel table models. Importing this package registers all tables."""
from models.agent import Agent
from models.audit import AuditLog
from models.auto_update import AutoUpdate
from models.backup_destination import BackupDestination
@@ -10,7 +9,7 @@ from models.stack import Stack
from models.user import User
__all__ = [
"User", "Stack", "AuditLog", "Setting", "Webhook", "Agent",
"User", "Stack", "AuditLog", "Setting", "Webhook",
"BackupDestination", "BackupSchedule", "AutoUpdate",
"StackLock", "ImageStatus", "LoginAttempt",
]
backend/models/agent.py
-49
View File
@@ -1,49 +0,0 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Optional
from sqlmodel import Field, SQLModel
def _now() -> datetime:
return datetime.now(timezone.utc)
class Agent(SQLModel, table=True):
"""A remote host running stackpilot-agent."""
id: Optional[int] = Field(default=None, primary_key=True)
name: str
url: str # e.g. http://10.0.0.5:5010
token: str # shared AGENT_TOKEN of that host
status: str = "unknown" # online | offline | unauthorized | unknown
hostname: Optional[str] = None # reported by the agent on ping
last_seen: Optional[datetime] = None
created_at: datetime = Field(default_factory=_now)
# --- API schemas ---
class AgentCreate(SQLModel):
name: str
url: str
token: str
class AgentUpdate(SQLModel):
name: Optional[str] = None
url: Optional[str] = None
token: Optional[str] = None
class AgentRead(SQLModel):
id: int
name: str
url: str
status: str
hostname: Optional[str]
last_seen: Optional[datetime]
created_at: datetime
token_set: bool
backend/models/auto_update.py
-4
View File
@@ -16,12 +16,10 @@ class AutoUpdate(SQLModel, table=True):
When the background image-update check finds a newer registry digest for one
of the stack's images, the stack is either pulled + redeployed
(``redeploy=True``) or merely notified about (``redeploy=False``).
``agent_id`` None = local host, otherwise a remote agent's stack.
"""
id: Optional[int] = Field(default=None, primary_key=True)
stack_id: str
agent_id: Optional[int] = None
enabled: bool = True
redeploy: bool = True # True = pull + up -d; False = notify only
last_run: Optional[datetime] = None
@@ -41,8 +39,6 @@ class AutoUpdateWrite(SQLModel):
class AutoUpdateRead(SQLModel):
id: Optional[int]
stack_id: str
agent_id: Optional[int]
agent_name: Optional[str] = None
enabled: bool
redeploy: bool
last_run: Optional[datetime]
backend/models/backup_schedule.py
-4
View File
@@ -19,7 +19,6 @@ class BackupSchedule(SQLModel, table=True):
id: Optional[int] = Field(default=None, primary_key=True)
stack_id: str
destination_id: int
agent_id: Optional[int] = None # None = local host; otherwise a remote agent
frequency: str = "daily" # one of FREQUENCIES
hour: int = 3 # UTC, used for daily/weekly
minute: int = 0
@@ -40,7 +39,6 @@ class BackupSchedule(SQLModel, table=True):
class ScheduleCreate(SQLModel):
stack_id: str
destination_id: int
agent_id: Optional[int] = None
frequency: str = "daily"
hour: int = 3
minute: int = 0
@@ -68,8 +66,6 @@ class ScheduleRead(SQLModel):
stack_id: str
destination_id: int
destination_name: Optional[str]
agent_id: Optional[int]
agent_name: Optional[str]
frequency: str
hour: int
minute: int
backend/models/runtime_state.py
-3
View File
@@ -8,9 +8,6 @@ and all three were lost on restart.
They are tables now. SQLite is already here; this needs no new dependency.
Note that ``services/compose_service.py`` and ``services/update_service.py``
are shared with the agent, which has no database at all — so these tables are
only ever touched from the central app's own routers and background loops.
"""
from __future__ import annotations
backend/models/template.py
-1
View File
@@ -40,4 +40,3 @@ class TemplateFromStackRequest(SQLModel):
class TemplateInstantiateRequest(SQLModel):
name: str # new stack name
agent_id: int | None = None # None = local host; otherwise deploy to a remote agent
backend/routers/agents.py
-1200
View File
File diff suppressed because it is too large Load Diff
backend/routers/dashboard.py
+1 -1
View File
@@ -24,7 +24,7 @@ async def fleet(
_user: User = Depends(get_current_user),
) -> dict:
"""Fleet-wide 'needs attention' list, KPIs and per-host rollup across the
local host and every agent — the data behind the operator cockpit."""
the host — the data behind the operator cockpit."""
try:
return await dashboard_service.compute_fleet(session, refresh=refresh)
except Exception as exc: # noqa: BLE001 — surface the real cause for diagnosis
backend/routers/schedules.py
+1 -9
View File
@@ -6,7 +6,6 @@ from sqlmodel import Session, select
from auth import require_admin
from database import get_session
from models.agent import Agent
from models.backup_destination import BackupDestination
from models.backup_schedule import (
FREQUENCIES,
@@ -28,14 +27,11 @@ def _ip(request: Request) -> str:
def _to_read(session: Session, s: BackupSchedule) -> ScheduleRead:
dest = session.get(BackupDestination, s.destination_id)
agent = session.get(Agent, s.agent_id) if s.agent_id is not None else None
return ScheduleRead(
id=s.id,
stack_id=s.stack_id,
destination_id=s.destination_id,
destination_name=dest.name if dest else None,
agent_id=s.agent_id,
agent_name=agent.name if agent else None,
frequency=s.frequency,
hour=s.hour,
minute=s.minute,
@@ -63,11 +59,7 @@ def _validate(session: Session, schedule: BackupSchedule) -> None:
raise HTTPException(status_code=400, detail=f"Unknown frequency '{schedule.frequency}'")
if not session.get(BackupDestination, schedule.destination_id):
raise HTTPException(status_code=404, detail=f"Destination {schedule.destination_id} not found")
if schedule.agent_id is not None:
# Remote stack: validate the agent exists; the stack is checked at run time.
if not session.get(Agent, schedule.agent_id):
raise HTTPException(status_code=404, detail=f"Agent {schedule.agent_id} not found")
elif not session.get(Stack, schedule.stack_id):
if not session.get(Stack, schedule.stack_id):
raise HTTPException(status_code=404, detail=f"Stack '{schedule.stack_id}' not found")
backend/routers/secrets.py
-1
View File
@@ -51,7 +51,6 @@ def _guard(fn, *args, **kwargs):
raise HTTPException(status_code=400, detail=str(exc)) from exc
# --- These functions are shared verbatim by the agent (see agent_app.py). ---
def list_secrets(stack_id: str) -> list[dict]:
backend/routers/stacks.py
+6 -3
View File
@@ -178,6 +178,9 @@ def get_stack(
except DockerError:
containers = []
status = "unknown"
# An operation in flight outranks whatever the containers currently say.
if stack_lock_service.is_busy(session, stack_id):
status = "updating"
return {
"id": stack.id,
"name": stack.name,
@@ -451,7 +454,7 @@ def get_auto_update(
_user: User = Depends(get_current_user),
) -> dict:
policy = auto_update_service.get_policy(session, stack_id)
return auto_update_service.to_read(session, policy, stack_id)
return auto_update_service.to_read(policy, stack_id)
@router.put("/{stack_id}/auto-update", response_model=AutoUpdateRead)
@@ -468,7 +471,7 @@ def set_auto_update(
target=stack_id, detail=f"enabled={body.enabled} redeploy={body.redeploy}",
ip=_client_ip(request),
)
return auto_update_service.to_read(session, policy, stack_id)
return auto_update_service.to_read(policy, stack_id)
@router.post("/{stack_id}/auto-update/run", response_model=AutoUpdateRead)
@@ -482,4 +485,4 @@ async def run_auto_update(
raise HTTPException(status_code=404, detail="No auto-update policy for this stack")
await auto_update_service.run_policy(session, policy)
session.refresh(policy)
return auto_update_service.to_read(session, policy, stack_id)
return auto_update_service.to_read(policy, stack_id)
backend/routers/templates.py
+4 -27
View File
@@ -12,7 +12,6 @@ from sqlmodel import Session
from auth import get_current_user, require_admin
from database import get_session
from models.agent import Agent
from models.stack import Stack
from models.template import (
TemplateFromStackRequest,
@@ -20,8 +19,7 @@ from models.template import (
TemplateSaveRequest,
)
from models.user import User
from services import agent_service, audit_service, compose_service, template_service
from services.agent_service import AgentError
from services import audit_service, compose_service, template_service
router = APIRouter(prefix="/api/templates", tags=["templates"])
@@ -103,7 +101,7 @@ def delete_template(
@router.post("/{template_id}/instantiate", status_code=201)
async def instantiate(
def instantiate(
template_id: str,
body: TemplateInstantiateRequest,
request: Request,
@@ -114,28 +112,7 @@ async def instantiate(
if not tpl:
raise HTTPException(status_code=404, detail="Template not found")
# Remote host: agents don't share our filesystem, so ship compose + env.
if body.agent_id is not None:
agent = session.get(Agent, body.agent_id)
if not agent:
raise HTTPException(status_code=404, detail=f"Agent {body.agent_id} not found")
try:
result = await agent_service.call(
session, agent, "POST", "/agent/stacks",
json={"name": body.name, "yaml": tpl["compose"], "env": tpl["env"] or None},
)
except AgentError as exc:
raise HTTPException(
status_code=exc.status if exc.status >= 400 else 502,
detail={"error": exc.error, "detail": exc.detail},
) from exc
audit_service.record(
session, user=user.username, action="template.instantiate",
target=f"{agent.name}/{result.get('id')}", detail=template_id, ip=_ip(request),
)
return {"id": result.get("id"), "name": body.name, "agent_id": agent.id}
# Local host: copy the whole template folder into a new stack.
# Copy the whole template folder into a new stack.
stack_id = compose_service.slugify(body.name)
if session.get(Stack, stack_id) or os.path.isdir(compose_service.stack_dir(stack_id)):
raise HTTPException(status_code=409, detail=f"Stack '{stack_id}' already exists")
@@ -156,4 +133,4 @@ async def instantiate(
session, user=user.username, action="template.instantiate",
target=stack_id, detail=template_id, ip=_ip(request),
)
return {"id": stack_id, "name": body.name, "agent_id": None}
return {"id": stack_id, "name": body.name}
backend/routers/ws.py
-246
View File
@@ -4,18 +4,15 @@ from __future__ import annotations
import asyncio
import json
import logging
import urllib.parse
import contextlib
import websockets
from fastapi import APIRouter, Query, WebSocket, WebSocketDisconnect
from jose import JWTError
from sqlmodel import Session
from auth import decode_token, resolve_token_user
from database import engine
from models.agent import Agent
from models.setting import EVENT_PULL_FAILED, EVENT_STACK_ERROR, EVENT_STACK_START
from services import (
audit_service,
@@ -158,7 +155,6 @@ async def ws_deploy(
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
await websocket.close(code=4409)
return
compose_service.mark_busy(stack_id)
try:
async for kind, payload in compose_service.stream_up(stack_id):
if kind == "log":
@@ -174,7 +170,6 @@ async def ws_deploy(
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
finally:
compose_service.clear_busy(stack_id)
with contextlib.suppress(Exception):
stack_lock_service.release(lock_session, stack_id)
lock_session.close()
@@ -203,78 +198,6 @@ async def ws_deploy(
await websocket.close()
@router.websocket("/ws/agent-logs/{agent_id}/{stack_id}")
async def ws_agent_logs(
websocket: WebSocket,
agent_id: int,
stack_id: str,
token: str | None = Query(default=None),
):
"""Proxy live compose logs from a remote agent through to the browser."""
await websocket.accept()
if not await _authorize(websocket, token):
return
with Session(engine) as session:
agent = session.get(Agent, agent_id)
if not agent:
await websocket.send_text(json.dumps({"type": "error", "detail": "agent not found"}))
await websocket.close()
return
base = agent.url.rstrip("/")
ws_url = ("wss://" + base[8:] if base.startswith("https://")
else "ws://" + base[7:] if base.startswith("http://")
else "ws://" + base)
# URL-encode the token: agent tokens may contain base64 chars (+ / =) that
# would otherwise be mangled in the query string and rejected as 4401.
ws_url += f"/agent/ws/logs/{stack_id}?token={urllib.parse.quote(agent.token, safe='')}"
async def _err(detail: str) -> None:
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"type": "error", "detail": detail}))
# Connect to the agent. Surface connection problems (agent down, wrong URL,
# an outdated agent that lacks /agent/ws/logs, TLS issues) instead of
# silently dropping the socket.
try:
upstream = await websockets.connect(ws_url, open_timeout=10, ping_interval=20)
except websockets.InvalidStatus as exc:
code = getattr(getattr(exc, "response", None), "status_code", None)
hint = " — the agent may be running an old version without live-log support; update it." if code == 404 else ""
logger.warning("Agent log proxy: handshake to %s failed (%s)", agent.name, code)
await _err(f"Agent '{agent.name}' rejected the log stream (HTTP {code}){hint}")
with contextlib.suppress(Exception):
await websocket.close()
return
except Exception as exc: # noqa: BLE001
logger.warning("Agent log proxy: cannot reach %s at %s: %s", agent.name, agent.url, exc)
await _err(f"Could not connect to agent '{agent.name}' at {agent.url}: {exc}")
with contextlib.suppress(Exception):
await websocket.close()
return
try:
async for message in upstream:
await websocket.send_text(
message if isinstance(message, str) else message.decode("utf-8", "replace")
)
except WebSocketDisconnect:
pass
except websockets.ConnectionClosed as exc:
# Abnormal upstream close (e.g. 4401 bad token, or agent-side error).
if exc.code not in (1000, 1001):
await _err(f"Agent log stream closed unexpectedly (code {exc.code}).")
except Exception as exc: # noqa: BLE001
logger.warning("Agent log proxy: stream error from %s: %s", agent.name, exc)
await _err(str(exc))
finally:
with contextlib.suppress(Exception):
await upstream.close()
with contextlib.suppress(Exception):
await websocket.close()
@router.websocket("/ws/update/{stack_id}")
async def ws_update(
websocket: WebSocket,
@@ -300,7 +223,6 @@ async def ws_update(
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
await websocket.close(code=4409)
return
compose_service.mark_busy(stack_id)
try:
async for kind, payload in compose_service.stream_update(stack_id):
if kind == "log":
@@ -315,7 +237,6 @@ async def ws_update(
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
finally:
compose_service.clear_busy(stack_id)
with contextlib.suppress(Exception):
stack_lock_service.release(lock_session, stack_id)
lock_session.close()
@@ -347,85 +268,6 @@ async def ws_update(
update_service.refresh_stack_local(stack_id)
@router.websocket("/ws/agent-deploy/{agent_id}/{stack_id}")
async def ws_agent_deploy(
websocket: WebSocket,
agent_id: int,
stack_id: str,
token: str | None = Query(default=None),
):
"""Proxy a remote agent's `compose up` deploy stream through to the browser,
then record the same audit entry as the REST agent lifecycle endpoint."""
await websocket.accept()
if not await _authorize(websocket, token):
return
username = decode_token(token, "access").get("sub", "unknown") if token else "unknown"
with Session(engine) as session:
agent = session.get(Agent, agent_id)
if not agent:
await websocket.send_text(json.dumps({"type": "error", "detail": "agent not found"}))
await websocket.close()
return
base = agent.url.rstrip("/")
ws_url = ("wss://" + base[8:] if base.startswith("https://")
else "ws://" + base[7:] if base.startswith("http://")
else "ws://" + base)
ws_url += f"/agent/ws/deploy/{stack_id}?token={urllib.parse.quote(agent.token, safe='')}"
async def _err(detail: str) -> None:
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"type": "error", "detail": detail}))
try:
upstream = await websockets.connect(ws_url, open_timeout=10, ping_interval=20)
except websockets.InvalidStatus as exc:
code = getattr(getattr(exc, "response", None), "status_code", None)
hint = " — the agent may be running an old version without deploy-console support; update it." if code == 404 else ""
logger.warning("Agent deploy proxy: handshake to %s failed (%s)", agent.name, code)
await _err(f"Agent '{agent.name}' rejected the deploy stream (HTTP {code}){hint}")
with contextlib.suppress(Exception):
await websocket.close()
return
except Exception as exc: # noqa: BLE001
logger.warning("Agent deploy proxy: cannot reach %s at %s: %s", agent.name, agent.url, exc)
await _err(f"Could not connect to agent '{agent.name}' at {agent.url}: {exc}")
with contextlib.suppress(Exception):
await websocket.close()
return
rc: int | None = None
try:
async for message in upstream:
text = message if isinstance(message, str) else message.decode("utf-8", "replace")
with contextlib.suppress(Exception):
msg = json.loads(text)
if msg.get("type") == "done":
rc = msg.get("returncode")
await websocket.send_text(text)
except WebSocketDisconnect:
pass
except websockets.ConnectionClosed as exc:
if exc.code not in (1000, 1001):
await _err(f"Agent deploy stream closed unexpectedly (code {exc.code}).")
except Exception as exc: # noqa: BLE001
logger.warning("Agent deploy proxy: stream error from %s: %s", agent.name, exc)
await _err(str(exc))
finally:
with contextlib.suppress(Exception):
await upstream.close()
with contextlib.suppress(Exception):
await websocket.close()
with contextlib.suppress(Exception):
with Session(engine) as session:
audit_service.record(
session, user=username, action="agent.stack.start",
target=f"{agent.name}/{stack_id}", detail=f"rc={rc} (deploy console)", ip="ws",
)
@router.websocket("/ws/events")
async def ws_events(
websocket: WebSocket,
@@ -514,91 +356,3 @@ async def ws_exec(
await websocket.close()
@router.websocket("/ws/agent-exec/{agent_id}/{container_id}")
async def ws_agent_exec(
websocket: WebSocket,
agent_id: int,
container_id: str,
token: str | None = Query(default=None),
cmd: str | None = Query(default=None),
):
"""Proxy an interactive exec session to a remote agent (admin only).
Unlike the log/deploy proxies this forwards in BOTH directions so keystrokes
reach the container and its output streams back."""
await websocket.accept()
if not await _authorize_admin(websocket, token):
return
with Session(engine) as session:
agent = session.get(Agent, agent_id)
if not agent:
await websocket.send_text(json.dumps({"type": "error", "detail": "agent not found"}))
await websocket.close()
return
base = agent.url.rstrip("/")
ws_url = ("wss://" + base[8:] if base.startswith("https://")
else "ws://" + base[7:] if base.startswith("http://")
else "ws://" + base)
ws_url += f"/agent/ws/exec/{container_id}?token={urllib.parse.quote(agent.token, safe='')}"
if cmd:
ws_url += f"&cmd={urllib.parse.quote(cmd, safe='')}"
async def _err(detail: str) -> None:
with contextlib.suppress(Exception):
await websocket.send_text(json.dumps({"type": "error", "detail": detail}))
try:
upstream = await websockets.connect(ws_url, open_timeout=10, ping_interval=20)
except websockets.InvalidStatus as exc:
code = getattr(getattr(exc, "response", None), "status_code", None)
hint = " — the agent may be running an old version without terminal support; update it." if code == 404 else ""
logger.warning("Agent exec proxy: handshake to %s failed (%s)", agent.name, code)
await _err(f"Agent '{agent.name}' rejected the terminal (HTTP {code}){hint}")
with contextlib.suppress(Exception):
await websocket.close()
return
except Exception as exc: # noqa: BLE001
logger.warning("Agent exec proxy: cannot reach %s at %s: %s", agent.name, agent.url, exc)
await _err(f"Could not connect to agent '{agent.name}' at {agent.url}: {exc}")
with contextlib.suppress(Exception):
await websocket.close()
return
with contextlib.suppress(Exception):
with Session(engine) as session:
username = decode_token(token, "access").get("sub", "unknown") if token else "unknown"
audit_service.record(
session, user=username, action="agent.container.exec",
target=f"{agent.name}/{container_id[:12]}", ip="ws",
)
async def browser_to_agent() -> None:
try:
while True:
msg = await websocket.receive_text()
await upstream.send(msg)
except (WebSocketDisconnect, websockets.ConnectionClosed):
pass
async def agent_to_browser() -> None:
try:
async for message in upstream:
await websocket.send_text(
message if isinstance(message, str) else message.decode("utf-8", "replace")
)
except (WebSocketDisconnect, websockets.ConnectionClosed):
pass
b2a = asyncio.create_task(browser_to_agent())
a2b = asyncio.create_task(agent_to_browser())
done, pending = await asyncio.wait({b2a, a2b}, return_when=asyncio.FIRST_COMPLETED)
for task in pending:
task.cancel()
with contextlib.suppress(Exception):
await asyncio.gather(*pending, return_exceptions=True)
with contextlib.suppress(Exception):
await upstream.close()
with contextlib.suppress(Exception):
await websocket.close()
backend/services/agent_service.py
-211
View File
@@ -1,211 +0,0 @@
"""Talk to remote stackpilot-agent hosts over HTTP.
The central app stores an ``Agent`` row per remote host and proxies stack /
system calls to it using the agent's shared token. Connectivity state
(``status``, ``hostname``, ``last_seen``) is refreshed on every successful or
failed call so the UI can show a live dot per host.
"""
from __future__ import annotations
import logging
from datetime import datetime, timezone
from typing import Any, Optional
import httpx
from sqlmodel import Session
from models.agent import Agent
logger = logging.getLogger("stackpilot.agent_proxy")
_TIMEOUT = 30.0
class AgentError(Exception):
def __init__(self, status: int, error: str, detail: str = ""):
self.status = status
self.error = error
self.detail = detail
super().__init__(f"{error}: {detail}" if detail else error)
def _now() -> datetime:
return datetime.now(timezone.utc)
def _mark(session: Session, agent: Agent, status: str, hostname: Optional[str] = None) -> None:
agent.status = status
if status == "online":
agent.last_seen = _now()
if hostname:
agent.hostname = hostname
session.add(agent)
session.commit()
session.refresh(agent)
async def _request(
agent: Agent,
method: str,
path: str,
*,
params: Optional[dict] = None,
json: Any = None,
) -> httpx.Response:
url = agent.url.rstrip("/") + path
headers = {"Authorization": f"Bearer {agent.token}"}
async with httpx.AsyncClient(follow_redirects=True) as client:
return await client.request(
method, url, headers=headers, params=params, json=json, timeout=_TIMEOUT
)
async def call(
session: Session,
agent: Agent,
method: str,
path: str,
*,
params: Optional[dict] = None,
json: Any = None,
) -> Any:
"""Proxy a request to the agent, updating its status, returning parsed JSON."""
try:
resp = await _request(agent, method, path, params=params, json=json)
except httpx.HTTPError as exc:
_mark(session, agent, "offline")
raise AgentError(502, "agent_unreachable", str(exc)) from exc
if resp.status_code in (401, 403):
_mark(session, agent, "unauthorized")
raise AgentError(resp.status_code, "agent_unauthorized", "Invalid agent token")
_mark(session, agent, "online")
if resp.status_code >= 400:
detail = ""
try:
body = resp.json()
detail = body.get("detail") if isinstance(body, dict) else str(body)
if isinstance(detail, dict):
detail = detail.get("detail") or detail.get("error") or str(detail)
except ValueError:
detail = resp.text[:500]
raise AgentError(resp.status_code, "agent_error", str(detail))
if resp.content:
try:
return resp.json()
except ValueError:
return resp.text
return None
def _handle_status(session: Session, agent: Agent, status_code: int, body_text: str = "") -> None:
"""Update agent status from a response code; raise AgentError on failure."""
if status_code in (401, 403):
_mark(session, agent, "unauthorized")
raise AgentError(status_code, "agent_unauthorized", "Invalid agent token")
_mark(session, agent, "online")
if status_code >= 400:
raise AgentError(status_code, "agent_error", body_text[:500])
async def download_to_file(
session: Session,
agent: Agent,
path: str,
dest_path: str,
*,
params: Optional[dict] = None,
) -> None:
"""Stream a GET from the agent into ``dest_path``."""
url = agent.url.rstrip("/") + path
headers = {"Authorization": f"Bearer {agent.token}"}
try:
async with httpx.AsyncClient(follow_redirects=True) as client:
async with client.stream("GET", url, headers=headers, params=params, timeout=None) as resp:
if resp.status_code >= 400:
text = (await resp.aread()).decode("utf-8", "replace")
_handle_status(session, agent, resp.status_code, text)
_handle_status(session, agent, resp.status_code)
with open(dest_path, "wb") as fh:
async for chunk in resp.aiter_bytes(1024 * 256):
fh.write(chunk)
except httpx.HTTPError as exc:
_mark(session, agent, "offline")
raise AgentError(502, "agent_unreachable", str(exc)) from exc
async def stream_download(
session: Session,
agent: Agent,
path: str,
*,
params: Optional[dict] = None,
):
"""Stream a GET from the agent straight through, yielding chunks.
Unlike :func:`download_to_file` this never buffers to disk, so a large
response (e.g. a folder zip the agent builds on the fly) starts flowing to
the browser immediately instead of being staged first.
"""
url = agent.url.rstrip("/") + path
headers = {"Authorization": f"Bearer {agent.token}"}
try:
async with httpx.AsyncClient(follow_redirects=True) as client:
async with client.stream("GET", url, headers=headers, params=params, timeout=None) as resp:
if resp.status_code >= 400:
text = (await resp.aread()).decode("utf-8", "replace")
_handle_status(session, agent, resp.status_code, text)
_handle_status(session, agent, resp.status_code)
async for chunk in resp.aiter_bytes(1024 * 256):
yield chunk
except httpx.HTTPError as exc:
_mark(session, agent, "offline")
raise AgentError(502, "agent_unreachable", str(exc)) from exc
async def upload_file(
session: Session,
agent: Agent,
path: str,
file_path: str,
filename: str,
data: dict,
) -> Any:
"""Stream a multipart POST (file + form fields) to the agent, returning JSON."""
url = agent.url.rstrip("/") + path
headers = {"Authorization": f"Bearer {agent.token}"}
try:
async with httpx.AsyncClient(follow_redirects=True) as client:
with open(file_path, "rb") as fh:
files = {"file": (filename, fh, "application/gzip")}
resp = await client.post(url, headers=headers, files=files, data=data, timeout=None)
except httpx.HTTPError as exc:
_mark(session, agent, "offline")
raise AgentError(502, "agent_unreachable", str(exc)) from exc
detail = ""
if resp.status_code >= 400:
try:
body = resp.json()
detail = body.get("detail") if isinstance(body, dict) else str(body)
except ValueError:
detail = resp.text[:500]
_handle_status(session, agent, resp.status_code, str(detail))
return resp.json() if resp.content else None
async def ping(session: Session, agent: Agent) -> dict:
"""Health-check an agent and refresh its status + hostname. Never raises."""
try:
data = await call(session, agent, "GET", "/agent/ping")
if isinstance(data, dict) and data.get("hostname"):
agent.hostname = data["hostname"]
session.add(agent)
session.commit()
session.refresh(agent)
return {"status": agent.status, "hostname": agent.hostname, "data": data}
except AgentError:
return {"status": agent.status, "hostname": agent.hostname, "data": None}
backend/services/auto_update_service.py
+15 -71
View File
@@ -5,9 +5,8 @@ Runs once per image-update-check cycle (called from
cache). For each enabled policy whose stack has a newer image available, either
pulls + redeploys the stack or just notifies, recording the outcome.
Central-only / DB-aware. Image resolution + digest comparison live in the
DB-free ``update_service`` so the agent can answer ``/agent/stacks/{id}/updates``
with the same logic.
Image resolution and digest comparison live in ``update_service``; this module
adds the policy layer on top.
"""
from __future__ import annotations
@@ -17,11 +16,9 @@ from datetime import datetime, timezone
from sqlmodel import Session, select
from database import engine
from models.agent import Agent
from models.auto_update import AutoUpdate
from models.setting import EVENT_PULL_FAILED, EVENT_STACK_AUTO_UPDATED
from services import (
agent_service,
compose_service,
notify_service,
stack_lock_service,
@@ -39,23 +36,18 @@ def _now() -> datetime:
# --------------------------------------------------------------------------- #
# Policy CRUD helpers (shared by the stacks + agents routers)
# Policy CRUD helpers
# --------------------------------------------------------------------------- #
def get_policy(session: Session, stack_id: str, agent_id: int | None = None) -> AutoUpdate | None:
stmt = select(AutoUpdate).where(AutoUpdate.stack_id == stack_id)
stmt = stmt.where(AutoUpdate.agent_id == agent_id) if agent_id is not None \
else stmt.where(AutoUpdate.agent_id.is_(None))
return session.exec(stmt).first()
def get_policy(session: Session, stack_id: str) -> AutoUpdate | None:
return session.exec(select(AutoUpdate).where(AutoUpdate.stack_id == stack_id)).first()
def upsert_policy(
session: Session, stack_id: str, enabled: bool, redeploy: bool, agent_id: int | None = None
) -> AutoUpdate:
policy = get_policy(session, stack_id, agent_id)
def upsert_policy(session: Session, stack_id: str, enabled: bool, redeploy: bool) -> AutoUpdate:
policy = get_policy(session, stack_id)
if policy is None:
policy = AutoUpdate(stack_id=stack_id, agent_id=agent_id)
policy = AutoUpdate(stack_id=stack_id)
policy.enabled = enabled
policy.redeploy = redeploy
session.add(policy)
@@ -64,22 +56,19 @@ def upsert_policy(
return policy
def to_read(session: Session, policy: AutoUpdate | None, stack_id: str, agent_id: int | None = None) -> dict:
def to_read(policy: AutoUpdate | None, stack_id: str) -> dict:
"""Build an AutoUpdateRead-shaped dict, defaulting to disabled when absent."""
agent_name = None
if agent_id is not None:
agent = session.get(Agent, agent_id)
agent_name = agent.name if agent else None
if policy is None:
return {
"id": None, "stack_id": stack_id, "agent_id": agent_id, "agent_name": agent_name,
"id": None, "stack_id": stack_id,
"enabled": False, "redeploy": True,
"last_run": None, "last_status": None, "last_result": None,
}
return {
"id": policy.id, "stack_id": policy.stack_id, "agent_id": policy.agent_id,
"agent_name": agent_name, "enabled": policy.enabled, "redeploy": policy.redeploy,
"last_run": policy.last_run, "last_status": policy.last_status, "last_result": policy.last_result,
"id": policy.id, "stack_id": policy.stack_id,
"enabled": policy.enabled, "redeploy": policy.redeploy,
"last_run": policy.last_run, "last_status": policy.last_status,
"last_result": policy.last_result,
}
@@ -138,48 +127,6 @@ async def _run_local(session: Session, policy: AutoUpdate) -> None:
)
async def _run_remote(session: Session, policy: AutoUpdate) -> None:
agent = session.get(Agent, policy.agent_id)
if not agent:
_record(session, policy, "error", "agent not found")
return
stack_id = policy.stack_id
try:
summary = await agent_service.call(
session, agent, "GET", f"/agent/stacks/{stack_id}/updates",
params={"refresh": "true"},
)
except Exception as exc: # noqa: BLE001
_record(session, policy, "error", f"agent check failed: {exc}")
return
if not summary or not summary.get("update_available"):
_record(session, policy, "up-to-date")
return
stale = ", ".join(summary.get("stale_images", []))
label = f"{agent.name}/{stack_id}"
prev = policy.last_status
if policy.redeploy:
try:
await agent_service.call(session, agent, "POST", f"/agent/stacks/{stack_id}/update")
except Exception as exc: # noqa: BLE001
_record(session, policy, "error", str(exc))
await _safe_notify(EVENT_PULL_FAILED, f"Auto-update of '{label}' failed", str(exc), session)
return
_record(session, policy, "updated", stale)
await _safe_notify(
EVENT_STACK_AUTO_UPDATED, f"Stack '{label}' auto-updated",
f"Pulled and redeployed: {stale}.", session,
)
else:
_record(session, policy, "update-available", stale)
if prev != "update-available":
await _safe_notify(
EVENT_STACK_AUTO_UPDATED, f"Update available for '{label}'",
f"Newer images: {stale} (auto-redeploy is off).", session,
)
async def _safe_notify(event: str, title: str, message: str, session: Session) -> None:
try:
await notify_service.notify(event, title, message, session)
@@ -188,10 +135,7 @@ async def _safe_notify(event: str, title: str, message: str, session: Session) -
async def run_policy(session: Session, policy: AutoUpdate) -> None:
if policy.agent_id is None:
await _run_local(session, policy)
else:
await _run_remote(session, policy)
await _run_local(session, policy)
async def run_due() -> None:
backend/services/compose_service.py
+17 -42
View File
@@ -199,22 +199,6 @@ def containers_for_stack(stack_id: str) -> list[ContainerInfo]:
return result
# in-memory set of stacks currently performing a pull/up
_BUSY: set[str] = set()
def mark_busy(stack_id: str) -> None:
_BUSY.add(stack_id)
def clear_busy(stack_id: str) -> None:
_BUSY.discard(stack_id)
def is_busy(stack_id: str) -> bool:
return stack_id in _BUSY
def _status_from_states(states: list[str]) -> str:
if not states:
return "stopped"
@@ -229,10 +213,13 @@ def _status_from_states(states: list[str]) -> str:
def compute_status(stack_id: str, containers: Optional[list[ContainerInfo]] = None) -> str:
"""Status for one stack. Pass already-fetched ``containers`` to avoid a
redundant Docker round-trip (the detail view already has them)."""
if stack_id in _BUSY:
return "updating"
"""Status for one stack, from its containers alone.
"updating" is not derived here: whether an operation is in flight lives in
``stack_lock_service``, and callers that want to show it overlay the lock on
top of this. Pass already-fetched ``containers`` to avoid a redundant Docker
round-trip (the detail view already has them).
"""
try:
if containers is None:
containers = containers_for_stack(stack_id)
@@ -447,11 +434,7 @@ async def stream_update(stack_id: str, override: Optional[str] = None):
async def up(stack_id: str, override: Optional[str] = None) -> dict:
mark_busy(stack_id)
try:
return await run_compose(stack_id, ["up", "-d", "--remove-orphans"], override)
finally:
clear_busy(stack_id)
return await run_compose(stack_id, ["up", "-d", "--remove-orphans"], override)
async def down(stack_id: str, override: Optional[str] = None) -> dict:
@@ -471,27 +454,19 @@ async def restart(stack_id: str, override: Optional[str] = None) -> dict:
async def pull(stack_id: str, override: Optional[str] = None) -> dict:
mark_busy(stack_id)
try:
return await run_compose(stack_id, ["pull"], override)
finally:
clear_busy(stack_id)
return await run_compose(stack_id, ["pull"], override)
async def update(stack_id: str, override: Optional[str] = None) -> dict:
"""Pull then up -d."""
mark_busy(stack_id)
try:
pull_res = await run_compose(stack_id, ["pull"], override)
up_res = await run_compose(stack_id, ["up", "-d", "--remove-orphans"], override)
return {
"returncode": up_res["returncode"],
"stdout": pull_res["stdout"] + "\n" + up_res["stdout"],
"stderr": pull_res["stderr"] + "\n" + up_res["stderr"],
"command": "pull + up -d",
}
finally:
clear_busy(stack_id)
pull_res = await run_compose(stack_id, ["pull"], override)
up_res = await run_compose(stack_id, ["up", "-d", "--remove-orphans"], override)
return {
"returncode": up_res["returncode"],
"stdout": pull_res["stdout"] + "\n" + up_res["stdout"],
"stderr": pull_res["stderr"] + "\n" + up_res["stderr"],
"command": "pull + up -d",
}
async def logs(
backend/services/container_service.py
+1 -1
View File
@@ -1,4 +1,4 @@
"""Single-container inspect + lifecycle — shared by the central app and agent.
"""Single-container inspect + lifecycle.
Only containers that belong to a compose-managed stack (i.e. carry the
``com.docker.compose.project`` label) are exposed, so this never becomes a
backend/services/crypto_service.py
+2 -2
View File
@@ -2,8 +2,8 @@
Most of StackPilot's secrets are files on disk (``.env``, ``.secrets/*``) where
filesystem permissions are the right control. A few can't be: backup
destination credentials and agent tokens are needed by background jobs, so they
sit in ``stackpilot.db``. This module encrypts those at rest.
destination credentials are needed by background jobs, so they sit in
``stackpilot.db``. This module encrypts those at rest.
The key is derived from ``SECRET_KEY`` rather than being a second thing to
configure — which is exactly why ``SECRET_KEY`` is now persisted (see
backend/services/dashboard_service.py
+16 -118
View File
@@ -1,11 +1,10 @@
"""Fleet aggregate for the dashboard cockpit.
"""Host aggregate for the dashboard cockpit.
One read-only call rolls up every host (local + agents) into a "needs
attention" list, headline KPIs and a per-host resource view. It is cheap by
construction: a single container *summary* list (no per-container inspect)
drives the local figures, image freshness comes from the cache the
update-service background loop already maintains, and per agent it makes a
small bounded set of HTTP fetches that degrade gracefully on failure.
One read-only call rolls the host up into a "needs attention" list, headline
KPIs and a resource view. It is cheap by construction: a single container
*summary* list (no per-container inspect) drives the figures, and image
freshness comes from the cache the update-service background loop already
maintains.
"""
from __future__ import annotations
@@ -15,15 +14,13 @@ from datetime import datetime, timedelta, timezone
from sqlmodel import Session, select
from database import engine
from docker_client import DockerError, get_client, safe_call
from models.agent import Agent
from models.backup_schedule import BackupSchedule
from services import agent_service, compose_service, update_service
from services import compose_service, update_service
COMPOSE_LABEL = compose_service.COMPOSE_LABEL
# Fleet aggregate: cached briefly because each call may fan out to every agent.
# Cached briefly: the dashboard polls this and the rollup is not free.
FLEET_TTL = 25.0
DISK_PRESSURE = 0.85 # disk used fraction above which a host needs attention
MEM_PRESSURE = 0.90 # memory used fraction above which a host needs attention
@@ -73,14 +70,12 @@ def _is_updated(containers: list[dict], cache: dict[str, dict]) -> bool:
# --------------------------------------------------------------------------- #
# Fleet aggregate (one call → "needs attention" + KPIs across every host)
# Host aggregate (one call → "needs attention" + KPIs)
#
# The dashboard used to poll each agent individually and recombine the numbers
# client-side. ``compute_fleet`` does the fan-out server-side instead: one local
# Docker pass plus, per online agent, a small set of system/stacks/updates
# fetches — all wrapped so a slow or broken agent degrades to "offline" rather
# than stalling the whole view. Cached for FLEET_TTL because the fan-out is not
# free.
# The dashboard used to fetch these numbers per stack and recombine them
# client-side. ``compute_fleet`` rolls them up server-side instead, off one
# Docker pass plus the update cache, and holds the result for FLEET_TTL because
# the dashboard polls it.
# --------------------------------------------------------------------------- #
# Stack-status buckets the status bar / KPIs are built from. Anything reporting
@@ -189,98 +184,12 @@ def _local_host() -> tuple[dict, list[dict]]:
return host, attention
def _offline_host(agent_id: int, name: str, status: str) -> dict:
return {"id": agent_id, "name": name, "online": False, "status": status,
"cpu_cores": 0, "mem_used": 0, "mem_total": 0, "disk_used": 0, "disk_total": 0,
"stacks": _bucket_statuses([]), "containers_running": 0, "containers_total": 0,
"unhealthy": 0, "updates_available": 0}
async def _agent_host(agent_id: int) -> tuple[dict, list[dict]]:
"""One agent's host card + attention items, tolerant of partial failure.
Opens its own Session so the fan-out across agents stays concurrency-safe
(a shared SQLModel session is not), and fetches sequentially within the
agent because :func:`agent_service.call` commits a status update each time.
"""
with Session(engine) as session:
agent = session.get(Agent, agent_id)
if agent is None:
return _offline_host(agent_id, str(agent_id), "unknown"), []
name = agent.name
# Agent stacks live in the host section of the dashboard, not a route
# of their own, so aggregate agent items deep-link back to it.
link = "/"
if agent.status != "online":
return _offline_host(agent_id, name, agent.status), [
_attn("error", "agent_offline", name,
f"{name} is {agent.status}", "Check it under Settings → Remote hosts.",
"/settings")]
async def _fetch(path: str):
try:
return await agent_service.call(session, agent, "GET", path)
except Exception: # AgentError or transport — degrade gracefully
return None
sys_data = await _fetch("/agent/system")
stacks = await _fetch("/agent/stacks")
updates = await _fetch("/agent/stacks/updates")
if sys_data is None and stacks is None:
# Couldn't reach it at all — agent_service.call already marked it offline.
return _offline_host(agent_id, name, "offline"), [
_attn("error", "agent_offline", name,
f"{name} is unreachable", "Check it under Settings → Remote hosts.",
"/settings")]
sys_data = sys_data or {}
statuses = [s.get("status", "") for s in (stacks or [])]
buckets = _bucket_statuses(statuses)
update_count = sum(1 for v in (updates or {}).values()
if isinstance(v, dict) and v.get("update_available"))
host = {
"id": agent_id,
"name": name,
"online": True,
"status": "online",
"cpu_cores": sys_data.get("cpu_cores", 0),
"mem_used": sys_data.get("mem_used", 0), "mem_total": sys_data.get("mem_total", 0),
"disk_used": sys_data.get("disk_used", 0), "disk_total": sys_data.get("disk_total", 0),
"stacks": buckets,
"containers_running": sys_data.get("compose_running", sys_data.get("containers_running", 0)),
"containers_total": sys_data.get("containers_total", 0),
"unhealthy": buckets["error"], # agents expose no healthcheck rollup; error is the proxy
"updates_available": update_count,
}
attention: list[dict] = []
if buckets["error"]:
attention.append(_attn("error", "stack_error", name,
f"{name}: {buckets['error']} stack(s) in error",
"Containers are dead.", link))
if buckets["partial"]:
attention.append(_attn("warn", "stack_partial", name,
f"{name}: {buckets['partial']} stack(s) partially running",
"Some services are down.", link))
if update_count:
attention.append(_attn("warn", "updates", name,
f"{name}: {update_count} stack(s) have image updates",
"Pull the newer images.", link))
attention += _resource_attention(name, link,
host["mem_used"], host["mem_total"],
host["disk_used"], host["disk_total"])
return host, attention
def _backup_attention(session: Session, agent_names: dict[int, str]) -> list[dict]:
def _backup_attention(session: Session) -> list[dict]:
"""Flag enabled backup schedules whose last run failed or is overdue."""
now = datetime.now(timezone.utc)
items: list[dict] = []
for sch in session.exec(select(BackupSchedule).where(BackupSchedule.enabled == True)).all(): # noqa: E712
host = agent_names.get(sch.agent_id, "local") if sch.agent_id else "local"
host = "local"
status = (sch.last_status or "").lower()
if status and not status.startswith("ok"):
items.append(_attn("error", "backup_failed", host,
@@ -307,23 +216,12 @@ async def compute_fleet(session: Session, refresh: bool = False) -> dict:
return _fleet_cache["data"]
local_host, attention = await asyncio.to_thread(_local_host)
agents = session.exec(select(Agent)).all()
agent_names = {a.id: a.name for a in agents}
agent_ids = [a.id for a in agents]
agent_results = await asyncio.gather(*[_agent_host(aid) for aid in agent_ids])
hosts = [local_host]
for host, items in agent_results:
hosts.append(host)
attention += items
attention += _backup_attention(session, agent_names)
attention += _backup_attention(session)
attention.sort(key=lambda a: _SEVERITY_ORDER.get(a["severity"], 9))
kpis = {
"hosts_online": sum(1 for h in hosts if h["online"]),
"hosts_total": len(hosts),
"stacks_running": sum(h["stacks"]["running"] for h in hosts),
"stacks_partial": sum(h["stacks"]["partial"] for h in hosts),
"stacks_total": sum(h["stacks"]["total"] for h in hosts),
backend/services/device_service.py
+3 -4
View File
@@ -94,11 +94,10 @@ def _real_root(path: str) -> str:
"""Map a logical host path into the container view (HOST_ROOT_PREFIX).
Refuses anything that resolves inside StackPilot's own ``DATA_DIR``. That
directory holds ``stackpilot.db`` — users, password hashes, agent tokens and
directory holds ``stackpilot.db`` — users, password hashes and
backup-destination credentials — and the API deliberately never hands those
out (``AgentRead.token_set`` is a bool, destination secrets come back
masked). Without this the file browser would be a way around that, for
admins too. Note this only bites when ``HOST_ROOT_PREFIX`` is empty: with a
out (destination secrets come back masked). Without this the file browser
would be a way around that, for admins too. Note this only bites when ``HOST_ROOT_PREFIX`` is empty: with a
prefix set, no logical path can reach the container's own ``/data`` at all.
"""
prefix = settings.HOST_ROOT_PREFIX.rstrip("/")
backend/services/exec_service.py
-1
View File
@@ -66,7 +66,6 @@ def exec_exit_code(exec_id: str):
async def pump_exec(websocket, exec_id: str, holder, raw) -> None:
"""Bidirectionally pump an exec socket <-> a WebSocket.
Shared by the central app and the agent (both pass a Starlette WebSocket).
Browser -> container: JSON ``{"type":"data","data":...}`` keystrokes and
``{"type":"resize","rows","cols"}`` control frames (raw text is also
accepted as keystrokes). Container -> browser: ``{"type":"data","data":...}``
backend/services/image_service.py
+1 -1
View File
@@ -1,4 +1,4 @@
"""Image listing — shared by the central images router and the agent."""
"""Image listing for the images router."""
from __future__ import annotations
from docker_client import DockerError, get_client, safe_call
backend/services/image_status_store.py
+4 -4
View File
@@ -1,9 +1,9 @@
"""Persistence for the image update cache.
``update_service`` holds the registry-digest results in a module dict because
it is shared with the agent, which has no database. This module is the central
app's half: it seeds that dict at startup and mirrors every write back into
SQLite, wired up in ``main.lifespan``.
``update_service`` holds the registry-digest results in a module dict and knows
nothing about storage — it is pure registry logic and stays unit-testable
without a database. This module is its persistence half: it seeds that dict at
startup and mirrors every write back into SQLite, wired up in ``main.lifespan``.
What it buys: after a restart the update badges are there immediately instead
of blank until the next background sweep (up to an hour), and the "already
backend/services/schedule_service.py
+9 -28
View File
@@ -11,22 +11,18 @@ from __future__ import annotations
import asyncio
import logging
import os
import tempfile
from datetime import datetime, timedelta, timezone
from sqlmodel import Session, select
from database import engine
from models.agent import Agent
from models.backup_destination import BackupDestination
from models.backup_schedule import BackupSchedule
from models.setting import EVENT_BACKUP_FAILED
from models.stack import Stack
from services import (
agent_service,
backup_destination_service as dest_service,
backup_service,
compose_service,
notify_service,
)
@@ -87,31 +83,16 @@ async def run_schedule(session: Session, schedule: BackupSchedule) -> dict:
if not dest:
raise RuntimeError(f"destination {schedule.destination_id} not found")
# Produce the backup archive — locally or by streaming it from an agent.
if schedule.agent_id is not None:
agent = session.get(Agent, schedule.agent_id)
if not agent:
raise RuntimeError(f"agent {schedule.agent_id} not found")
prefix = compose_service.slugify(agent.name)
tmp = tempfile.NamedTemporaryFile(delete=False, suffix=".tar.gz")
tmp.close()
path = tmp.name
await agent_service.download_to_file(
session, agent, f"/agent/stacks/{schedule.stack_id}/backup", path,
params={"include_volumes": schedule.include_volumes, "stop_first": schedule.stop_first},
)
else:
stack = session.get(Stack, schedule.stack_id)
if not stack:
raise RuntimeError(f"stack '{schedule.stack_id}' not found")
prefix = None
path = await backup_service.create_backup(
schedule.stack_id, stack.name,
include_volumes=schedule.include_volumes, stop_first=schedule.stop_first,
)
stack = session.get(Stack, schedule.stack_id)
if not stack:
raise RuntimeError(f"stack '{schedule.stack_id}' not found")
path = await backup_service.create_backup(
schedule.stack_id, stack.name,
include_volumes=schedule.include_volumes, stop_first=schedule.stop_first,
)
filename = backup_service.backup_filename(schedule.stack_id, schedule.include_volumes, prefix=prefix)
basename = backup_service.backup_basename(schedule.stack_id, prefix)
filename = backup_service.backup_filename(schedule.stack_id, schedule.include_volumes)
basename = backup_service.backup_basename(schedule.stack_id)
await asyncio.to_thread(dest_service.upload, dest, path, filename)
pruned = await asyncio.to_thread(_prune, dest, basename, schedule.keep)
schedule.last_status = "ok"
backend/services/stack_lock_service.py
+5 -9
View File
@@ -5,15 +5,11 @@ project — two open browser tabs, or the auto-update pass landing on a stack
somebody just clicked — both run ``pull`` and then ``up -d``, and race each
other recreating the same containers.
There *was* a busy flag (``compose_service._BUSY``), but it only ever fed the
status column: no lifecycle handler consulted it before acting. This module is
the actual guard, and it lives in the database so it holds across workers and
across a restart.
``compose_service`` keeps its in-process set because it is shared with the
agent, which has no database. The agent is a single process managing one host,
and the central app holds this lock before calling it, so the two do not
conflict.
There *was* a busy flag in ``compose_service``, but it only ever fed the status
column: no lifecycle handler consulted it before acting. This module is the
actual guard, and it lives in the database so it holds across workers and
across a restart. ``compose_service.compute_status`` therefore reports only
what the containers say; callers overlay the lock to show "updating".
"""
from __future__ import annotations
backend/services/update_service.py
+8 -8
View File
@@ -53,13 +53,13 @@ _NOTIFIED: set[str] = set()
#: Optional sink for cache writes.
#:
#: This module is shared with the agent, which has no database, so persistence
#: cannot live here. The central app registers a callback that mirrors each
#: entry into SQLite (see ``services/image_status_store.py``) and seeds the
#: cache from it at startup; the agent registers nothing and behaves exactly as
#: before. Without it a restart blanked every update badge until the next
#: background sweep — up to an hour — and re-announced updates it had already
#: notified about.
#: This module is pure registry logic and knows nothing about storage, which
#: keeps it unit-testable without a database. ``main.lifespan`` registers a
#: callback that mirrors each entry into SQLite (see
#: ``services/image_status_store.py``) and seeds the cache from it at startup.
#: Without it a restart blanked every update badge until the next background
#: sweep — up to an hour — and re-announced updates it had already notified
#: about.
_persist_cb: Optional[Callable[[UpdateStatus, bool], None]] = None
@@ -299,7 +299,7 @@ async def stack_updates(stack_id: str, refresh: bool = True) -> dict:
``refresh=True`` queries the registry now; ``False`` reads the cache the
background loop already populated (so the auto-update pass adds no extra
registry round-trips). DB-free, so the agent can reuse it verbatim.
registry round-trips).
"""
images = stack_images(stack_id)
result: dict[str, dict] = {}
backend/tests/test_agent_authorization.py
-97
View File
@@ -1,97 +0,0 @@
"""The agent's token guard.
The agent has no users and no roles: one shared ``AGENT_TOKEN`` is the whole
access-control model, declared per route as
``dependencies=[Depends(verify_token)]``. That makes a forgotten decorator
argument the entire failure mode — one route without it hands anonymous full
Docker control of that host, and nothing in review would show it.
So the invariant is asserted here rather than assumed across 50 decorators.
"""
from __future__ import annotations
import pytest
#: The only agent routes that may answer without a token — the liveness probe
#: the container's HEALTHCHECK calls, which returns nothing but a version.
UNAUTHENTICATED = {"GET /agent/health"}
@pytest.fixture(scope="module")
def agent_app():
import agent_app as module
return module.app
def _routes(app):
from fastapi.routing import APIRoute
out = []
for route in app.routes:
if not isinstance(route, APIRoute):
continue
for method in sorted(route.methods - {"HEAD", "OPTIONS"}):
out.append((f"{method} {route.path}", route))
return sorted(out, key=lambda r: r[0])
def _has_token_guard(route) -> bool:
from agent_app import verify_token
found = [False]
def walk(dependant):
for sub in dependant.dependencies:
if sub.call is verify_token:
found[0] = True
walk(sub)
walk(route.dependant)
return found[0]
def test_every_agent_route_requires_the_token(agent_app):
unguarded = {
key
for key, route in _routes(agent_app)
if not _has_token_guard(route) and not key.startswith("GET /agent/health")
}
assert not unguarded, (
"These agent routes answer without AGENT_TOKEN, which is full Docker "
f"access to the host: {sorted(unguarded)}"
)
def test_only_the_health_probe_is_unauthenticated(agent_app):
open_routes = {key for key, route in _routes(agent_app) if not _has_token_guard(route)}
assert open_routes == UNAUTHENTICATED
def test_a_wrong_token_is_rejected(agent_app, monkeypatch):
from starlette.testclient import TestClient
from config import settings
monkeypatch.setattr(settings, "AGENT_TOKEN", "the-real-token", raising=False)
client = TestClient(agent_app, raise_server_exceptions=False)
assert client.get("/agent/ping").status_code == 401
assert client.get(
"/agent/ping", headers={"Authorization": "Bearer wrong"}
).status_code == 401
# The health probe stays open so the container's HEALTHCHECK works.
assert client.get("/agent/health").status_code == 200
def test_an_unset_token_refuses_everything(agent_app, monkeypatch):
"""An agent started without AGENT_TOKEN must not be wide open."""
from starlette.testclient import TestClient
from config import settings
monkeypatch.setattr(settings, "AGENT_TOKEN", "", raising=False)
client = TestClient(agent_app, raise_server_exceptions=False)
response = client.get("/agent/ping", headers={"Authorization": "Bearer anything"})
assert response.status_code == 503
backend/tests/test_agent_removal.py
+201
View File
@@ -0,0 +1,201 @@
"""Cleaning up after the removed multi-host integration.
The agent sidecar and everything that proxied to it were removed in 0.48.0.
Upgrading installs still carry the schema it left behind, and one part of that
is not merely dead weight: the ``agent`` table held each remote host's URL and
its bearer token, which is full Docker control of that host. Leaving those in
the database for a feature that no longer exists would be worse than dropping
them, so ``_drop_removed_schema`` drops the table outright.
The ``agent_id`` columns are only dropped where the SQLite build supports it
(DROP COLUMN needs 3.35+); they are nullable and unread, so leaving them is
harmless.
"""
from __future__ import annotations
import pytest
from sqlalchemy import create_engine, inspect, text
@pytest.fixture
def pre_048_db(tmp_path, monkeypatch):
"""A database as an install running 0.47.0 would have it."""
import database
import models # noqa: F401 — populates SQLModel.metadata
engine = create_engine(f"sqlite:///{tmp_path / 'legacy.db'}")
with engine.begin() as conn:
conn.execute(
text(
"""
CREATE TABLE agent (
id INTEGER NOT NULL PRIMARY KEY,
name VARCHAR NOT NULL,
url VARCHAR NOT NULL,
token VARCHAR NOT NULL,
status VARCHAR NOT NULL,
hostname VARCHAR,
last_seen DATETIME,
created_at DATETIME NOT NULL
)
"""
)
)
conn.execute(
text(
"INSERT INTO agent (id, name, url, token, status, created_at) VALUES"
" (1, 'nas', 'http://10.0.0.5:5010', 'super-secret-agent-token',"
" 'online', '2026-01-01')"
)
)
conn.execute(
text(
"""
CREATE TABLE autoupdate (
id INTEGER NOT NULL PRIMARY KEY,
stack_id VARCHAR NOT NULL,
agent_id INTEGER,
enabled BOOLEAN NOT NULL,
redeploy BOOLEAN NOT NULL,
last_run DATETIME,
last_status VARCHAR,
last_result VARCHAR,
created_at DATETIME NOT NULL
)
"""
)
)
conn.execute(
text(
"INSERT INTO autoupdate (id, stack_id, agent_id, enabled, redeploy,"
" created_at) VALUES (1, 'jellyfin', NULL, 1, 1, '2026-01-01')"
)
)
monkeypatch.setattr(database, "engine", engine)
return engine
def test_the_agent_table_and_its_tokens_are_dropped(pre_048_db):
import database
assert "agent" in inspect(pre_048_db).get_table_names()
database._drop_removed_schema()
assert "agent" not in inspect(pre_048_db).get_table_names(), (
"the agent table holds host URLs and bearer tokens for a feature that no "
"longer exists — it must not survive the upgrade"
)
def test_surviving_rows_are_untouched(pre_048_db):
"""Dropping the agent schema must not disturb the policies that remain."""
import database
database._drop_removed_schema()
with pre_048_db.begin() as conn:
row = conn.execute(
text("SELECT stack_id, enabled FROM autoupdate WHERE id = 1")
).one()
assert row.stack_id == "jellyfin"
assert row.enabled
def test_the_agent_id_column_is_dropped_where_sqlite_allows_it(pre_048_db):
import sqlite3
import database
database._drop_removed_schema()
columns = {c["name"] for c in inspect(pre_048_db).get_columns("autoupdate")}
supports_drop = tuple(int(p) for p in sqlite3.sqlite_version.split(".")) >= (3, 35, 0)
if supports_drop:
assert "agent_id" not in columns
else: # pragma: no cover - depends on the host's SQLite
assert "agent_id" in columns, "the fallback must leave the column alone"
def test_an_unsupported_drop_column_does_not_block_the_table_drop(
pre_048_db, monkeypatch
):
"""A failed DDL poisons its transaction.
If the table drop and the column drops shared one, a SQLite too old for
DROP COLUMN would take the agent table — tokens and all — down with it.
"""
import database
from sqlalchemy.exc import OperationalError
real_execute = database.text
def exploding_text(sql):
if "DROP COLUMN" in sql:
raise OperationalError("DROP COLUMN", {}, Exception("near \"DROP\""))
return real_execute(sql)
monkeypatch.setattr(database, "text", exploding_text)
database._drop_removed_schema()
assert "agent" not in inspect(pre_048_db).get_table_names(), (
"the table drop must not be collateral damage of an unsupported column drop"
)
def test_running_it_twice_is_harmless(pre_048_db):
"""It runs on every start, not just the first one after the upgrade."""
import database
database._drop_removed_schema()
database._drop_removed_schema()
assert "agent" not in inspect(pre_048_db).get_table_names()
def test_a_fresh_install_is_unaffected(db):
"""Nothing to drop, and no error for trying."""
import database
database._drop_removed_schema()
assert "agent" not in inspect(database.engine).get_table_names()
# --------------------------------------------------------------------------- #
# Nothing agent-shaped is left in the running application
# --------------------------------------------------------------------------- #
def test_no_route_mentions_agents(app):
from fastapi.routing import APIRoute
leftovers = [
r.path
for r in app.routes
if "agent" in getattr(r, "path", "").lower()
]
assert not leftovers, f"remote-host routes still registered: {leftovers}"
assert not any(
isinstance(r, APIRoute) and "/api/agents" in r.path for r in app.routes
)
def test_no_agent_model_is_registered():
from sqlmodel import SQLModel
import models # noqa: F401
assert "agent" not in SQLModel.metadata.tables
def test_the_agent_modules_are_gone():
"""Import guards, so a stray file cannot quietly come back."""
import importlib
for name in ("agent_app", "models.agent", "routers.agents", "services.agent_service"):
with pytest.raises(ModuleNotFoundError):
importlib.import_module(name)
def test_no_agent_token_setting():
from config import settings
assert not hasattr(settings, "AGENT_TOKEN")
backend/tests/test_browse_sandbox.py
+1 -1
View File
@@ -7,7 +7,7 @@ Two independent gates, both in :mod:`services.device_service`:
anything landing inside StackPilot's own ``DATA_DIR``.
The second gate exists because the API deliberately never hands out what lives
there (agent tokens come back as a bool, destination secrets come back masked),
there (destination secrets come back masked),
so the file browser must not be the way around that — for admins either.
"""
from __future__ import annotations
backend/tests/test_compose_service.py
+4 -9
View File
@@ -78,15 +78,10 @@ def test_status_from_states(svc, states, expected):
assert svc._status_from_states(states) == expected
def test_busy_stacks_report_as_updating(svc):
"""The busy flag has to win over the container states, or a stack shows
'stopped' for the moment between `down` and `up` during an update."""
svc.mark_busy("busy-stack")
try:
assert svc.compute_status("busy-stack", containers=[]) == "updating"
finally:
svc.clear_busy("busy-stack")
assert svc.compute_status("busy-stack", containers=[]) == "stopped"
def test_status_comes_from_the_containers_alone(svc):
""""updating" is not derived here — that lives in ``stack_lock_service``,
and callers overlay it. Keeping both would be two sources of truth."""
assert svc.compute_status("no-such-stack", containers=[]) == "stopped"
# --------------------------------------------------------------------------- #
backend/tests/test_route_authorization.py
+1 -20
View File
@@ -44,10 +44,9 @@ PUBLIC = {
#: Reachable by the read-only ``user`` role. Everything here has been checked
#: for whether it can return a credential:
#:
#: * ``GET /api/agents`` returns ``AgentRead``, whose ``token_set`` is a bool.
#: * ``GET /api/settings`` returns the interval and counts — webhook URLs (which
#: carry tokens) come from the admin-only ``/api/settings/webhooks``.
#: * ``GET /api/stacks/{id}`` and its agent twin blank out ``env`` for non-admins.
#: * ``GET /api/stacks/{id}`` blanks out ``env`` for non-admins.
#: * ``GET /api/templates`` is metadata only; the detail route, which returns a
#: template's env, is admin-only.
#: * The ``/api/editor/*`` routes are pure YAML transformations — they take YAML
@@ -94,24 +93,6 @@ USER_READABLE = {
"GET /api/system/devices",
"GET /api/system/update",
"GET /api/templates",
# Remote hosts: the same read-only surface, proxied.
"GET /api/agents",
"POST /api/agents/{agent_id}/ping",
"GET /api/agents/{agent_id}/system",
"GET /api/agents/{agent_id}/stacks",
"GET /api/agents/{agent_id}/stacks/stats",
"GET /api/agents/{agent_id}/stacks/updates",
"GET /api/agents/{agent_id}/stacks/{stack_id}",
"GET /api/agents/{agent_id}/stacks/{stack_id}/auto-update",
"GET /api/agents/{agent_id}/stacks/{stack_id}/logs",
"GET /api/agents/{agent_id}/containers/{container_id}",
"GET /api/agents/{agent_id}/images",
"GET /api/agents/{agent_id}/images/updates",
"GET /api/agents/{agent_id}/networks",
"GET /api/agents/{agent_id}/networks/{network_id}",
"GET /api/agents/{agent_id}/networks/{network_id}/containers",
"GET /api/agents/{agent_id}/volumes",
"GET /api/agents/{agent_id}/volumes/sizes",
}
backend/tests/test_runtime_state.py
+5 -4
View File
@@ -184,11 +184,12 @@ def test_pruning_drops_images_no_stack_uses_any_more(clean_image_status):
assert [r.image for r in session.exec(select(ImageStatus)).all()] == ["kept:1"]
def test_the_agent_has_no_persistence_wired_up(clean_image_status):
"""``update_service`` is shared with the agent, which has no database.
def test_the_registry_layer_stays_free_of_storage(clean_image_status):
"""``update_service`` is pure registry logic.
Persistence therefore has to be opt-in, registered by the central app —
if this module ever imports the database directly, the agent breaks.
Persistence is opt-in, registered by ``main.lifespan``, which keeps the
module unit-testable without a database and keeps the storage decision in
one place. This is the kind of boundary a later change erases silently.
"""
import inspect
backend/version.py
+1 -1
View File
@@ -1,3 +1,3 @@
"""Single source of truth for the StackPilot release version."""
APP_VERSION = "0.47.0"
APP_VERSION = "0.48.0"