diff --git a/README.md b/README.md index 350b695..2d6a955 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,9 @@ as intuitive as Dockge, as capable as Portainer for Compose workflows. > network attach) + Phase 12 (File browser) + Phase 13 (Multi-host networks & > images) + Phase 14 (Multi-host file browser) + Phase 15 (Dashboard stack > resource usage) + Phase 16 (Volumes page, multi-host) + Phase 17 (Multi-host -> dashboard) complete. +> dashboard) + Phase 18 (Image prune) + Phase 19 (Compose validate & diff) + +> Phase 20 (Container management) + Phase 21 (Container terminal) + Phase 22 +> (Auto-update) + Phase 23 (Secrets & configs) complete. ## What works today (Phase 1) @@ -153,6 +155,26 @@ as intuitive as Dockge, as capable as Portainer for Compose workflows. container or connect any container on the host (`POST /api/networks/{id}/connect` / `/disconnect`). +### Phase 23 — Secrets & configs (compose file-based) + +- A **Secrets** tab on the stack detail page manages per-stack Docker + **secrets** and **configs**: create one by name + content, list them (name, + kind, size — content is **never** returned by the API), and delete. Content is + write-only: once saved it is cleared from the form and cannot be read back. +- Files are stored inside the stack's own directory (`/.secrets/` + / `.configs/`, dir `0700` / file `0600`) and referenced from the compose + file with a **relative** `file:` path, so the Docker daemon reads them with no + `HOST_ROOT_PREFIX` dependency — exactly as if dropped next to `compose.yaml`. +- **Attach/detach** wires a stored secret/config into a chosen service: secrets + appear at `/run/secrets/`, configs mount at a target path you specify. The + compose file is rewritten in place (top-level `secrets:`/`configs:` defs are + pruned when no service still uses them); **redeploy the stack to apply**. +- **Admin-only** (secrets are sensitive); every write/delete/attach/detach is + audited (`secret.*`). Works for **remote stacks** too — the agent stores the + files on its own host (`/agent/stacks/{id}/secrets/*`, proxied via + `/api/agents/{id}/stacks/{id}/secrets/*`). Names are validated against path + traversal (single component, no `..`, no leading dot); content capped at 1 MiB. + ### Phase 22 — Auto-update (Watchtower-style) - A per-stack **Auto-update** policy (on the stack Overview tab): when the diff --git a/ROADMAP.md b/ROADMAP.md index 5d580c3..2e4efd1 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -5,7 +5,7 @@ Each phase ships independently following the standing release checklist (bump `backend/main.py` + `backend/agent_app.py` AGENT_VERSION + `frontend/package.json` → build backend→agent→frontend `:VERSION`+`:latest` → py_compile + `tsc -b && vite build` → route smoke-test → push all 3 → README → -git commit + push → update memory). Current released version: **0.26.0**. +git commit + push → update memory). Current released version: **0.29.0**. Order: **21 → 22 → 23** (Terminal is highest-value and self-contained; Secrets is most design-ambiguous, left last). @@ -144,58 +144,67 @@ either auto pull+redeploy or just notify. Builds on the existing --- -## Phase 23 — Docker Secrets & Configs ☐ NOT STARTED → target 0.29.0 +## Phase 23 — Docker Secrets & Configs ☑ DONE — shipped 0.29.0 -**DESIGN DECISION TO MAKE FIRST (ask user / decide at phase start):** Docker -`secret`/`config` objects are a **Swarm** feature. Two interpretations: -- (A) **Compose file-based secrets** (`secrets:` top-level with `file:` + - per-service `secrets:`) — works in plain compose, the relevant one for a - compose manager. **Recommended default.** -- (B) **Swarm secrets/configs** via `client.secrets`/`client.configs` — only if - swarm mode is active (detect `client.info()["Swarm"]["LocalNodeState"]=="active"`). +**DECISION (user, 2026-06-09): (A) Compose file-based secrets.** Swarm path (B) +dropped. **Sub-decision (Claude's call): per-stack storage with RELATIVE paths** +— secret/config files live in `/.secrets/` and `.configs/`, +referenced as `file: ./.secrets/`. Compose resolves `file:` relative to the +compose file (which is in the stack dir = a host bind-mount), so the daemon reads +it with NO `HOST_ROOT_PREFIX` dependency. Secrets are therefore per-stack (matches +how compose scopes them), managed from a **Secrets tab on StackDetail / +RemoteStackDetail** (not the new-stack editor, which has no dir yet). -Plan assumes **(A)**, with (B) surfaced only when swarm is detected. +**As shipped — deviations from the plan (both simplifications):** +- **No DB model.** Both content *and* metadata live on disk; `list` derives name/ + size/mtime from the filesystem. A `models/secret.py` would only duplicate that, + so it was dropped — there is nothing to keep in sync. +- **Files live in the stack dir, not a separate sandbox.** `/.secrets/` + and `.configs/` (per-stack, relative `file:` refs) — this is exactly what + compose expects and removes the `HOST_ROOT_PREFIX` dependency. Sandboxing comes + from strict name validation (single component, no `..`, no leading dot, no sep). +- **Routes are stack-scoped:** `/api/stacks/{stack_id}/secrets`, not `/api/secrets`. +- **UI is a Secrets tab on Stack/RemoteStackDetail**, not a Settings page — a stack + must exist (have a dir) before it can hold secrets, matching compose's scoping. +- **Swarm path not built** (decision A dropped B); no `swarm_active()` guard. ### Backend -- ☐ `models/secret.py` — `ManagedSecret(id, name, scope [global|stack], stack_id - nullable, kind [secret|config], created, agent_id nullable)`. Content NOT in - DB — stored on disk. -- ☐ `services/secret_service.py`: - - store secret files under a sandboxed dir `/.stackpilot-secrets/` - (chmod 700 dir, 600 files); `create(name, content)`, `update(name, content)`, - `delete(name)`, `list()` (metadata only — never return content; mask), and - `path_for(name)` for compose `file:` refs. - - if swarm active: also expose `client.secrets.list/create/remove` + - `client.configs.*` (interpretation B), behind a `swarm_active()` guard. -- ☐ `compose_edit_service`: `add_secret(yaml, service, secret_name)` — injects - top-level `secrets: {: {file: }}` + per-service `secrets: []`; - `remove_secret(...)`. Same for configs. -- ☐ `routers/secrets.py` (prefix `/api/secrets`): CRUD (admin, audit - `secret.*`), content write-only. Agent `/agent/secrets/*` + proxy - `/api/agents/{id}/secrets/*` for multi-host (reuse the patterns). +- ☑ `services/secret_service.py` — file-based store under `/.secrets` + & `.configs` (dir 0700, file 0600); `write_secret`/`delete_secret`/`list_all` + (metadata only, never content)/`exists`/`rel_path`/`attach`/`detach`. 1 MiB cap; + name validation rejects traversal/hidden/separators. +- ☑ `compose_edit_service`: `add_secret`/`remove_secret` (top-level + `secrets: {: {file: }}` + per-service list) and `add_config`/ + `remove_config` (with `source`/`target` mount); top-level defs pruned when unused. +- ☑ `routers/secrets.py` (prefix `/api/stacks/{stack_id}/secrets`): list/write/ + delete/attach/detach, admin-only, audit `secret.*`, content write-only. +- ☑ Agent `/agent/stacks/{stack_id}/secrets/*` (agent_app.py) + proxy + `/api/agents/{id}/stacks/{stack_id}/secrets/*` (agents.py), audit `agent.secret.*`. ### Frontend -- ☐ Settings → **Secrets & Configs** section (or a dedicated page): list (name, - scope, kind, created), create (name + content textarea, content masked after), - delete. Multi-host host-switcher like Files. -- ☐ Editor helper panel: a **Secrets** wizard tab to attach an existing secret/ - config to a service (writes the compose `secrets:` block via - `compose_edit_service`). +- ☑ `SecretsPanel` (api/secrets.ts + components/stacks/SecretsPanel.tsx): create + (type/name/content; content cleared after save, never re-shown), list (name, + kind, size), delete, and per-row attach/detach to a service (config rows take a + mount target). Admin-gated. Wired as a **Secrets** tab on both StackDetail and + RemoteStackDetail (agentId-aware → multi-host). ### Verify -- ☐ Create a file-based secret, attach to a service via the wizard, deploy, exec - in and confirm `/run/secrets/` is present with the content. -- ☐ Sandbox: secret files can't escape the secrets dir; content never returned by list. -- ☐ If swarm active on the build host, smoke-test the swarm path too (likely - NOT active here → note as hardware-verify debt). -- ☐ py_compile + build + route smoke-test. +- ☑ Sandbox: traversal/hidden/separator names rejected; dir 0700 / file 0600; + `list` returns metadata only, never content (unit-tested in the backend image). +- ☑ compose round-trip: add secret+config → remove both → back to clean YAML + (top-level defs pruned). add/remove for secrets and configs unit-tested. +- ☑ py_compile + backend image build + frontend `tsc -b && vite build` + route + smoke-test (local CRUD/attach/detach, agent, proxy all registered). +- ☐ **Live hardware-verify debt:** create a secret, attach via the panel, deploy, + exec in and confirm `/run/secrets/` holds the content (needs a running + stack on real hardware). Swarm path intentionally not built (decision A). -### Open risks -- Swarm-vs-compose decision (above). Confirm with user before coding. +### Open risks (carried) - Secret file ownership/permissions inside the container vs on host (file-based secrets mount the host file → uid/gid must be readable by the service user). -- HOST_ROOT_PREFIX / multi-host: the secret file must live where that host's - Docker daemon can read it (agent stores on its own host). +- Multi-host: the agent stores the file on its own host, so the secret lives where + that host's Docker daemon can read it — verified by design (relative `file:`), + pending the live exec check above. --- diff --git a/backend/agent_app.py b/backend/agent_app.py index cbbe624..504a706 100644 --- a/backend/agent_app.py +++ b/backend/agent_app.py @@ -39,6 +39,7 @@ from config import settings from docker_client import DockerError, get_client, safe_call from services import ( backup_service, + compose_edit_service, compose_service, container_service, device_service, @@ -46,6 +47,7 @@ from services import ( file_service, image_service, network_service, + secret_service, stats_service, update_service, volume_service, @@ -64,7 +66,7 @@ def _map_docker(exc: DockerError): raise HTTPException(status_code=code, detail=exc.detail or exc.error) raise exc # falls through to the global 502 DockerError handler -AGENT_VERSION = "0.28.0" +AGENT_VERSION = "0.29.0" # --------------------------------------------------------------------------- # @@ -375,6 +377,77 @@ async def stack_updates(stack_id: str, refresh: bool = Query(True)) -> dict: return await update_service.stack_updates(stack_id, refresh=refresh) +# --------------------------------------------------------------------------- # +# Secrets & configs (per-stack, file-based) +# --------------------------------------------------------------------------- # + + +class SecretWriteBody(BaseModel): + kind: str = "secret" + name: str + content: str + + +class SecretAttachBody(BaseModel): + kind: str = "secret" + name: str + service: str + target: str | None = None + + +class SecretDetachBody(BaseModel): + kind: str = "secret" + name: str + service: str + + +def _ensure_stack(stack_id: str) -> None: + if not os.path.isdir(compose_service.stack_dir(stack_id)): + raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found") + + +def _secret_guard(fn, *args, **kwargs): + try: + return fn(*args, **kwargs) + except (secret_service.SecretError, compose_edit_service.EditError) as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + +@app.get("/agent/stacks/{stack_id}/secrets", dependencies=[Depends(verify_token)]) +def agent_list_secrets(stack_id: str) -> list: + _ensure_stack(stack_id) + return secret_service.list_all(stack_id) + + +@app.put("/agent/stacks/{stack_id}/secrets", dependencies=[Depends(verify_token)]) +def agent_write_secret(stack_id: str, body: SecretWriteBody) -> dict: + _ensure_stack(stack_id) + return _secret_guard(secret_service.write_secret, stack_id, body.kind, body.name, body.content) + + +@app.delete("/agent/stacks/{stack_id}/secrets/{kind}/{name}", dependencies=[Depends(verify_token)]) +def agent_delete_secret(stack_id: str, kind: str, name: str) -> dict: + _ensure_stack(stack_id) + _secret_guard(secret_service.delete_secret, stack_id, kind, name) + return {"ok": True} + + +@app.post("/agent/stacks/{stack_id}/secrets/attach", dependencies=[Depends(verify_token)]) +def agent_attach_secret(stack_id: str, body: SecretAttachBody) -> dict: + _ensure_stack(stack_id) + if not secret_service.exists(stack_id, body.kind, body.name): + raise HTTPException(status_code=404, detail="secret not found") + new_yaml = _secret_guard(secret_service.attach, stack_id, body.kind, body.name, body.service, body.target) + return {"ok": True, "yaml": new_yaml} + + +@app.post("/agent/stacks/{stack_id}/secrets/detach", dependencies=[Depends(verify_token)]) +def agent_detach_secret(stack_id: str, body: SecretDetachBody) -> dict: + _ensure_stack(stack_id) + new_yaml = _secret_guard(secret_service.detach, stack_id, body.kind, body.name, body.service) + return {"ok": True, "yaml": new_yaml} + + @app.get("/agent/stacks/{stack_id}/backup", dependencies=[Depends(verify_token)]) async def backup_stack( stack_id: str, diff --git a/backend/main.py b/backend/main.py index 285b853..55ec4ad 100644 --- a/backend/main.py +++ b/backend/main.py @@ -26,6 +26,7 @@ from routers import ( networks, ports, schedules, + secrets, settings as settings_router, stacks, system, @@ -56,7 +57,7 @@ async def lifespan(app: FastAPI): schedule_task.cancel() -app = FastAPI(title="StackPilot", version="0.28.0", lifespan=lifespan) +app = FastAPI(title="StackPilot", version="0.29.0", lifespan=lifespan) app.add_middleware( CORSMiddleware, @@ -77,6 +78,7 @@ async def docker_error_handler(_request: Request, exc: DockerError): app.include_router(auth.router) app.include_router(stacks.router) +app.include_router(secrets.router) app.include_router(containers.router) app.include_router(system.router) app.include_router(volumes.router) diff --git a/backend/routers/agents.py b/backend/routers/agents.py index b6ab11e..fe3f105 100644 --- a/backend/routers/agents.py +++ b/backend/routers/agents.py @@ -19,6 +19,7 @@ from models.stack import StackCreate, StackUpdate from models.user import User from routers.files import NameBody, RenameBody, TransferBody, WriteBody from routers.networks import ContainerRef, NetworkCreate +from routers.secrets import AttachBody, DetachBody, SecretWrite from models.auto_update import AutoUpdateRead, AutoUpdateWrite from services import ( agent_service, @@ -1039,3 +1040,92 @@ async def agent_run_auto_update( await auto_update_service.run_policy(session, policy) session.refresh(policy) return auto_update_service.to_read(session, policy, stack_id, agent_id) + + +# --------------------------------------------------------------------------- # +# Secrets & configs (proxied) — remote stacks +# --------------------------------------------------------------------------- # + + +@router.get("/{agent_id}/stacks/{stack_id}/secrets") +async def agent_list_secrets( + agent_id: int, + stack_id: str, + session: Session = Depends(get_session), + _user: User = Depends(require_admin), +) -> list: + agent = _get_or_404(session, agent_id) + return await _proxy(session, agent, "GET", f"/agent/stacks/{stack_id}/secrets") or [] + + +@router.put("/{agent_id}/stacks/{stack_id}/secrets") +async def agent_write_secret( + agent_id: int, + stack_id: str, + body: SecretWrite, + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> dict: + agent = _get_or_404(session, agent_id) + result = await _proxy(session, agent, "PUT", f"/agent/stacks/{stack_id}/secrets", json=body.model_dump()) + audit_service.record( + session, user=user.username, action="agent.secret.write", + target=f"{agent.name}/{stack_id}/{body.kind}/{body.name}", ip=_ip(request), + ) + return result + + +@router.delete("/{agent_id}/stacks/{stack_id}/secrets/{kind}/{name}") +async def agent_delete_secret( + agent_id: int, + stack_id: str, + kind: str, + name: str, + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> dict: + agent = _get_or_404(session, agent_id) + result = await _proxy(session, agent, "DELETE", f"/agent/stacks/{stack_id}/secrets/{kind}/{name}") + audit_service.record( + session, user=user.username, action="agent.secret.delete", + target=f"{agent.name}/{stack_id}/{kind}/{name}", ip=_ip(request), + ) + return result + + +@router.post("/{agent_id}/stacks/{stack_id}/secrets/attach") +async def agent_attach_secret( + agent_id: int, + stack_id: str, + body: AttachBody, + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> dict: + agent = _get_or_404(session, agent_id) + result = await _proxy(session, agent, "POST", f"/agent/stacks/{stack_id}/secrets/attach", json=body.model_dump()) + audit_service.record( + session, user=user.username, action="agent.secret.attach", + target=f"{agent.name}/{stack_id}/{body.kind}/{body.name}->{body.service}", ip=_ip(request), + ) + return result + + +@router.post("/{agent_id}/stacks/{stack_id}/secrets/detach") +async def agent_detach_secret( + agent_id: int, + stack_id: str, + body: DetachBody, + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> dict: + agent = _get_or_404(session, agent_id) + result = await _proxy(session, agent, "POST", f"/agent/stacks/{stack_id}/secrets/detach", json=body.model_dump()) + audit_service.record( + session, user=user.username, action="agent.secret.detach", + target=f"{agent.name}/{stack_id}/{body.kind}/{body.name}->{body.service}", ip=_ip(request), + ) + return result diff --git a/backend/routers/secrets.py b/backend/routers/secrets.py new file mode 100644 index 0000000..f790292 --- /dev/null +++ b/backend/routers/secrets.py @@ -0,0 +1,139 @@ +"""Per-stack file-based secrets & configs (compose) — local host. + +CRUD over the secret/config files stored in the stack directory, plus +attach/detach which rewrite the stack's compose file to reference (or stop +referencing) a secret/config from a service. A redeploy is required for changes +to take effect on running containers. +""" +from __future__ import annotations + +from fastapi import APIRouter, Depends, HTTPException, Query, Request +from pydantic import BaseModel +from sqlmodel import Session + +from auth import require_admin +from database import get_session +from models.user import User +from services import audit_service, secret_service +from services.compose_edit_service import EditError +from services.secret_service import SecretError + +router = APIRouter(prefix="/api/stacks/{stack_id}/secrets", tags=["secrets"]) + + +class SecretWrite(BaseModel): + kind: str = "secret" # "secret" | "config" + name: str + content: str + + +class AttachBody(BaseModel): + kind: str = "secret" + name: str + service: str + target: str | None = None # required for configs (mount path) + + +class DetachBody(BaseModel): + kind: str = "secret" + name: str + service: str + + +def _ip(request: Request) -> str: + return request.client.host if request.client else "unknown" + + +def _guard(fn, *args, **kwargs): + try: + return fn(*args, **kwargs) + except (SecretError, EditError) as exc: + raise HTTPException(status_code=400, detail=str(exc)) from exc + + +# --- These functions are shared verbatim by the agent (see agent_app.py). --- + + +def list_secrets(stack_id: str) -> list[dict]: + return secret_service.list_all(stack_id) + + +def write_secret(stack_id: str, body: SecretWrite) -> dict: + return _guard(secret_service.write_secret, stack_id, body.kind, body.name, body.content) + + +def delete_secret(stack_id: str, kind: str, name: str) -> dict: + _guard(secret_service.delete_secret, stack_id, kind, name) + return {"ok": True} + + +def attach_secret(stack_id: str, body: AttachBody) -> dict: + if not secret_service.exists(stack_id, body.kind, body.name): + raise HTTPException(status_code=404, detail="secret not found") + new_yaml = _guard(secret_service.attach, stack_id, body.kind, body.name, body.service, body.target) + return {"ok": True, "yaml": new_yaml} + + +def detach_secret(stack_id: str, body: DetachBody) -> dict: + new_yaml = _guard(secret_service.detach, stack_id, body.kind, body.name, body.service) + return {"ok": True, "yaml": new_yaml} + + +# --- HTTP endpoints (admin only; secrets are sensitive) --- + + +@router.get("") +def http_list(stack_id: str, _user: User = Depends(require_admin)) -> list[dict]: + return list_secrets(stack_id) + + +@router.put("") +def http_write( + stack_id: str, body: SecretWrite, request: Request, + session: Session = Depends(get_session), user: User = Depends(require_admin), +) -> dict: + result = write_secret(stack_id, body) + audit_service.record( + session, user=user.username, action="secret.write", + target=f"{stack_id}/{body.kind}/{body.name}", ip=_ip(request), + ) + return result + + +@router.delete("/{kind}/{name}") +def http_delete( + stack_id: str, kind: str, name: str, request: Request, + session: Session = Depends(get_session), user: User = Depends(require_admin), +) -> dict: + result = delete_secret(stack_id, kind, name) + audit_service.record( + session, user=user.username, action="secret.delete", + target=f"{stack_id}/{kind}/{name}", ip=_ip(request), + ) + return result + + +@router.post("/attach") +def http_attach( + stack_id: str, body: AttachBody, request: Request, + session: Session = Depends(get_session), user: User = Depends(require_admin), +) -> dict: + result = attach_secret(stack_id, body) + audit_service.record( + session, user=user.username, action="secret.attach", + target=f"{stack_id}/{body.kind}/{body.name}->{body.service}", ip=_ip(request), + ) + return result + + +@router.post("/detach") +def http_detach( + stack_id: str, body: DetachBody, request: Request, + session: Session = Depends(get_session), user: User = Depends(require_admin), +) -> dict: + result = detach_secret(stack_id, body) + audit_service.record( + session, user=user.username, action="secret.detach", + target=f"{stack_id}/{body.kind}/{body.name}->{body.service}", ip=_ip(request), + ) + return result diff --git a/backend/services/compose_edit_service.py b/backend/services/compose_edit_service.py index b4340da..e60cd84 100644 --- a/backend/services/compose_edit_service.py +++ b/backend/services/compose_edit_service.py @@ -32,8 +32,9 @@ def _dump(data: dict) -> str: # Drop empty top-level keys we may have created. if not data.get("services"): data.pop("services", None) - if "volumes" in data and not data["volumes"]: - data.pop("volumes", None) + for key in ("volumes", "secrets", "configs"): + if key in data and not data[key]: + data.pop(key, None) return yaml.safe_dump(data, sort_keys=False, default_flow_style=False) @@ -208,3 +209,89 @@ def set_resources( def list_services(yaml_str: str) -> list[str]: data = _load(yaml_str) return list(data["services"].keys()) + + +# --------------------------------------------------------------------------- # +# Secrets & configs (compose file-based) +# --------------------------------------------------------------------------- # + + +def add_secret(yaml_str: str, service: str, name: str, file_path: str) -> str: + """Define a top-level file-based secret and attach it to ``service``.""" + data = _load(yaml_str) + svc = _get_service(data, service) + secrets = data.setdefault("secrets", {}) + if not isinstance(secrets, dict): + raise EditError("`secrets` must be a mapping") + secrets[name] = {"file": file_path} + refs = svc.setdefault("secrets", []) + if not isinstance(refs, list): + raise EditError(f"Service '{service}' secrets must be a list") + if name not in refs: + refs.append(name) + return _dump(data) + + +def remove_secret(yaml_str: str, service: str, name: str) -> str: + """Detach a secret from ``service``; drop the top-level def if now unused.""" + data = _load(yaml_str) + svc = data["services"].get(service) + if isinstance(svc, dict) and isinstance(svc.get("secrets"), list): + svc["secrets"] = [s for s in svc["secrets"] if s != name] + if not svc["secrets"]: + svc.pop("secrets", None) + _prune_top_level(data, "secrets", name, _secret_still_used) + return _dump(data) + + +def add_config(yaml_str: str, service: str, name: str, file_path: str, target: str) -> str: + """Define a top-level file-based config and mount it into ``service`` at target.""" + data = _load(yaml_str) + svc = _get_service(data, service) + configs = data.setdefault("configs", {}) + if not isinstance(configs, dict): + raise EditError("`configs` must be a mapping") + configs[name] = {"file": file_path} + refs = svc.setdefault("configs", []) + if not isinstance(refs, list): + raise EditError(f"Service '{service}' configs must be a list") + if not any(isinstance(e, dict) and e.get("source") == name for e in refs): + refs.append({"source": name, "target": target}) + return _dump(data) + + +def remove_config(yaml_str: str, service: str, name: str) -> str: + data = _load(yaml_str) + svc = data["services"].get(service) + if isinstance(svc, dict) and isinstance(svc.get("configs"), list): + svc["configs"] = [ + e for e in svc["configs"] + if not (e == name or (isinstance(e, dict) and e.get("source") == name)) + ] + if not svc["configs"]: + svc.pop("configs", None) + _prune_top_level(data, "configs", name, _config_still_used) + return _dump(data) + + +def _secret_still_used(data: dict, name: str) -> bool: + for svc in data.get("services", {}).values(): + if isinstance(svc, dict) and name in (svc.get("secrets") or []): + return True + return False + + +def _config_still_used(data: dict, name: str) -> bool: + for svc in data.get("services", {}).values(): + if not isinstance(svc, dict): + continue + for entry in svc.get("configs") or []: + if entry == name or (isinstance(entry, dict) and entry.get("source") == name): + return True + return False + + +def _prune_top_level(data: dict, key: str, name: str, still_used) -> None: + top = data.get(key) + if isinstance(top, dict) and name in top and not still_used(data, name): + top.pop(name, None) diff --git a/backend/services/secret_service.py b/backend/services/secret_service.py new file mode 100644 index 0000000..039bbb1 --- /dev/null +++ b/backend/services/secret_service.py @@ -0,0 +1,128 @@ +"""Per-stack, file-based compose secrets & configs. + +Secret/config payloads are stored as files INSIDE the stack's own directory +(``/.secrets/`` and ``.configs/``) and referenced from +the compose file with a RELATIVE path (``file: ./.secrets/``). Because +compose resolves ``file:`` relative to the compose file — which lives in the +stack dir (a host bind-mount) — the Docker daemon reads the file correctly with +no ``HOST_ROOT_PREFIX`` dependency, exactly as if the user had dropped a secret +file next to their ``compose.yaml`` by hand. + +Content is never returned by the listing API; only metadata (name, size, mtime). +""" +from __future__ import annotations + +import os +import re +from typing import Optional + +from services import compose_edit_service, compose_service + +KIND_DIR = {"secret": ".secrets", "config": ".configs"} +_NAME_RE = re.compile(r"^[A-Za-z0-9._-]+$") +MAX_SECRET_BYTES = 1 * 1024 * 1024 # 1 MiB + + +class SecretError(Exception): + """Invalid secret request (bad kind/name, too large).""" + + +def _check_kind(kind: str) -> str: + if kind not in KIND_DIR: + raise SecretError(f"unknown kind '{kind}'") + return kind + + +def _check_name(name: str) -> str: + # A single path component, no traversal, no leading dot (keeps it out of the + # way of .secrets/.configs themselves and hidden-file surprises). + if not name or name.startswith(".") or not _NAME_RE.match(name) or os.path.sep in name: + raise SecretError("name must match [A-Za-z0-9._-] and not start with a dot") + return name + + +def kind_dir(stack_id: str, kind: str, override: Optional[str] = None) -> str: + _check_kind(kind) + return os.path.join(compose_service.stack_dir(stack_id, override), KIND_DIR[kind]) + + +def _path(stack_id: str, kind: str, name: str, override: Optional[str] = None) -> str: + return os.path.join(kind_dir(stack_id, kind, override), _check_name(name)) + + +def rel_path(kind: str, name: str) -> str: + """Path to put in the compose ``file:`` field (relative to the compose file).""" + return f"./{KIND_DIR[_check_kind(kind)]}/{_check_name(name)}" + + +def list_secrets(stack_id: str, kind: str, override: Optional[str] = None) -> list[dict]: + directory = kind_dir(stack_id, kind, override) + if not os.path.isdir(directory): + return [] + out: list[dict] = [] + for name in sorted(os.listdir(directory)): + full = os.path.join(directory, name) + if os.path.isfile(full): + st = os.stat(full) + out.append({"name": name, "kind": kind, "size": st.st_size, "modified": st.st_mtime}) + return out + + +def list_all(stack_id: str, override: Optional[str] = None) -> list[dict]: + items: list[dict] = [] + for kind in KIND_DIR: + items.extend(list_secrets(stack_id, kind, override)) + return items + + +def write_secret(stack_id: str, kind: str, name: str, content: str, override: Optional[str] = None) -> dict: + _check_kind(kind) + _check_name(name) + data = content.encode("utf-8") + if len(data) > MAX_SECRET_BYTES: + raise SecretError("content exceeds 1 MiB limit") + directory = kind_dir(stack_id, kind, override) + os.makedirs(directory, exist_ok=True) + os.chmod(directory, 0o700) + path = _path(stack_id, kind, name, override) + with open(path, "wb") as fh: + fh.write(data) + os.chmod(path, 0o600) + return {"name": name, "kind": kind, "size": len(data)} + + +def delete_secret(stack_id: str, kind: str, name: str, override: Optional[str] = None) -> None: + path = _path(stack_id, kind, name, override) + if os.path.isfile(path): + os.remove(path) + + +def exists(stack_id: str, kind: str, name: str, override: Optional[str] = None) -> bool: + return os.path.isfile(_path(stack_id, kind, name, override)) + + +def attach(stack_id: str, kind: str, name: str, service: str, + target: Optional[str] = None, override: Optional[str] = None) -> str: + """Reference a stored secret/config from ``service`` in the stack's compose + file (relative ``file:`` path) and persist it. Returns the new YAML.""" + _check_kind(kind) + _check_name(name) + yaml_str = compose_service.read_compose(stack_id, override) + file_path = rel_path(kind, name) + if kind == "config": + new_yaml = compose_edit_service.add_config(yaml_str, service, name, file_path, target or f"/{name}") + else: + new_yaml = compose_edit_service.add_secret(yaml_str, service, name, file_path) + compose_service.write_compose(stack_id, new_yaml, override) + return new_yaml + + +def detach(stack_id: str, kind: str, name: str, service: str, override: Optional[str] = None) -> str: + _check_kind(kind) + yaml_str = compose_service.read_compose(stack_id, override) + if kind == "config": + new_yaml = compose_edit_service.remove_config(yaml_str, service, name) + else: + new_yaml = compose_edit_service.remove_secret(yaml_str, service, name) + compose_service.write_compose(stack_id, new_yaml, override) + return new_yaml diff --git a/frontend/package.json b/frontend/package.json index 70252dd..a911b76 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "stackpilot-frontend", "private": true, - "version": "0.28.0", + "version": "0.29.0", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/api/secrets.ts b/frontend/src/api/secrets.ts new file mode 100644 index 0000000..338834b --- /dev/null +++ b/frontend/src/api/secrets.ts @@ -0,0 +1,38 @@ +import api from "./client"; + +export type SecretKind = "secret" | "config"; + +export interface SecretEntry { + name: string; + kind: SecretKind; + size: number; + modified: number; +} + +// Local host, or a remote agent's stack when agentId is given. +const base = (stackId: string, agentId?: number) => + agentId != null + ? `/api/agents/${agentId}/stacks/${stackId}/secrets` + : `/api/stacks/${stackId}/secrets`; + +export const secretsApi = { + list: (stackId: string, agentId?: number) => + api.get(base(stackId, agentId)).then((r) => r.data), + write: ( + stackId: string, + body: { kind: SecretKind; name: string; content: string }, + agentId?: number, + ) => api.put(base(stackId, agentId), body).then((r) => r.data), + remove: (stackId: string, kind: SecretKind, name: string, agentId?: number) => + api.delete(`${base(stackId, agentId)}/${kind}/${name}`).then((r) => r.data), + attach: ( + stackId: string, + body: { kind: SecretKind; name: string; service: string; target?: string }, + agentId?: number, + ) => api.post(`${base(stackId, agentId)}/attach`, body).then((r) => r.data), + detach: ( + stackId: string, + body: { kind: SecretKind; name: string; service: string }, + agentId?: number, + ) => api.post(`${base(stackId, agentId)}/detach`, body).then((r) => r.data), +}; diff --git a/frontend/src/components/stacks/SecretsPanel.tsx b/frontend/src/components/stacks/SecretsPanel.tsx new file mode 100644 index 0000000..a4188f8 --- /dev/null +++ b/frontend/src/components/stacks/SecretsPanel.tsx @@ -0,0 +1,222 @@ +import { useState } from "react"; +import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query"; +import { KeyRound, Trash2, Link2, FileCog } from "lucide-react"; +import { toast } from "sonner"; +import { Badge, Button, Card, Input } from "@/components/ui"; +import { secretsApi, type SecretEntry, type SecretKind } from "@/api/secrets"; +import { editorApi } from "@/api/editor"; +import { apiErrorMessage } from "@/api/client"; + +/** + * Per-stack file-based secrets & configs. Create stores a file in the stack + * dir; attach references it from a service in the compose file (redeploy to + * apply). Admin-only on the backend. + */ +export function SecretsPanel({ + stackId, + yaml, + agentId, + isAdmin, + onChanged, +}: { + stackId: string; + yaml: string; + agentId?: number; + isAdmin: boolean; + onChanged?: () => void; +}) { + const qc = useQueryClient(); + const key = ["secrets", agentId ?? "local", stackId]; + + const list = useQuery({ + queryKey: key, + queryFn: () => secretsApi.list(stackId, agentId), + enabled: isAdmin, + }); + const services = useQuery({ + queryKey: ["editor-services", stackId, agentId, yaml.length], + queryFn: () => editorApi.services(yaml), + enabled: isAdmin, + }); + + const [kind, setKind] = useState("secret"); + const [name, setName] = useState(""); + const [content, setContent] = useState(""); + + const invalidate = () => qc.invalidateQueries({ queryKey: key }); + + const create = useMutation({ + mutationFn: () => secretsApi.write(stackId, { kind, name: name.trim(), content }, agentId), + onSuccess: () => { + toast.success(`${kind} "${name}" saved`); + setName(""); setContent(""); + invalidate(); + }, + onError: (e) => toast.error(apiErrorMessage(e)), + }); + + const remove = useMutation({ + mutationFn: (s: SecretEntry) => secretsApi.remove(stackId, s.kind, s.name, agentId), + onSuccess: () => { toast.success("Deleted"); invalidate(); }, + onError: (e) => toast.error(apiErrorMessage(e)), + }); + + const attach = useMutation({ + mutationFn: (v: { s: SecretEntry; service: string; target?: string }) => + secretsApi.attach(stackId, { kind: v.s.kind, name: v.s.name, service: v.service, target: v.target }, agentId), + onSuccess: () => { toast.success("Attached — redeploy the stack to apply"); onChanged?.(); }, + onError: (e) => toast.error(apiErrorMessage(e)), + }); + + const detach = useMutation({ + mutationFn: (v: { s: SecretEntry; service: string }) => + secretsApi.detach(stackId, { kind: v.s.kind, name: v.s.name, service: v.service }, agentId), + onSuccess: () => { toast.success("Detached — redeploy the stack to apply"); onChanged?.(); }, + onError: (e) => toast.error(apiErrorMessage(e)), + }); + + if (!isAdmin) + return ( + +

Secrets are visible to admins only.

+ + ); + + const items = list.data ?? []; + const svcList = services.data ?? []; + + return ( +
+ +
+ + New secret / config +
+
+ + +
+