Deploy stacks from a Git repository (0.58.0)

StackPilot's stacks were already plain folders on disk, which makes GitOps less
of an architectural change than it would be elsewhere: a sync is "make these
files match that repo, then compose up". Almost all of the design effort went
into the word "these", because getting it wrong destroys data.

A stack folder is not just the compose file. Compose creates bind-mount
directories in it — ./config, ./data — and those hold the live state of whatever
is running. So the obvious implementation, clone into the stack folder and
git reset --hard, is a data-loss bug waiting for its first `git clean`. Instead
the clone lives in a cache under ${DATA_DIR}/git/<stack> where reset and clean
are safe, and the configured subtree is copied across. No .git ends up in the
stack folder, so backups and the file browser are unaffected too.

Deletion is the other half. Making a folder "match" a repo naively means
removing what the repo does not have, which is exactly the application data
above. So each sync records the paths it wrote, and the next sync may delete
only those — a file the repository never provided cannot be touched by any code
path here. Tested directly: a database file and a hand-written .env survive a
sync that replaces the compose file and removes a file the repo dropped.

What the repo does provide is overwritten, hand edits included. That is the
point of GitOps rather than a wart, but it is a surprise if you attach a repo to
a stack you have been editing, so the connect form says it before the first sync
and the first sync is never automatic.

The webhook is the only route in StackPilot with no bearer token, because a Git
forge has none to present. It authenticates with an HMAC over the body —
X-Hub-Signature-256 for GitHub/Gitea/Forgejo, X-Gitlab-Token for GitLab, both
compared in constant time — and answers 404, not 403, to anything unsigned. A
403 would confirm that a given stack exists and is connected to a repository,
which an unauthenticated caller has not earned. The authorization matrix test
caught this route being public and made me write that reasoning down in it,
which is exactly what that test is for.

Credentials never reach a command line: ps is readable by every process on the
host, and this runs in a container next to everything else. The HTTPS token goes
to git through GIT_ASKPASS and the environment, the SSH key through a 0600 file
kept outside the working tree, and everything git prints is scrubbed of both —
plus any credential-carrying URL — before it is stored in last_error or shown.

Auto-deploy takes the same per-stack lock as every other lifecycle action, so a
webhook firing mid-deploy reports "files synced, stack busy" instead of racing a
second compose run at the same project.

The image needed git and openssh-client, which is the only reason this release
touches the Dockerfile.

26 tests against real repositories created with the real git binary, none of
them touching the network — mocking git would mostly test the mock. Verified end
to end as well: connect, sync, a push that changes one file and deletes another,
a wrongly signed webhook, a correctly signed one, and the live data still there
afterwards.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

backend/Dockerfile

@@ -1,8 +1,10 @@
 FROM python:3.12-slim
 
 # Docker CLI + compose plugin are required for lifecycle commands.
+# git + openssh-client are required for deploying stacks from a Git repository
+# (services/git_service.py); ssh only for repositories reached over SSH.
 RUN apt-get update \
-    && apt-get install -y --no-install-recommends ca-certificates curl gnupg \
+    && apt-get install -y --no-install-recommends ca-certificates curl gnupg git openssh-client \
     && install -m 0755 -d /etc/apt/keyrings \
     && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
     && chmod a+r /etc/apt/keyrings/docker.asc \

backend/main.py

@@ -23,6 +23,7 @@ from routers import (
     destinations,
     editor,
     files,
+    git,
     images,
     networks,
     ports,
@@ -39,6 +40,7 @@ from routers import (
 )
 from services import (
     backup_destination_service,
+    git_service,
     image_status_store,
     logo_service,
     registry_service,
@@ -105,11 +107,14 @@ async def lifespan(app: FastAPI):
     # download, and a box with no outbound internet must still start instantly
     # (it just keeps the built-in glyphs).
     logo_task = asyncio.create_task(logo_service.catalog_loop())
+    git_service.ensure_cache_root()
+    git_task = asyncio.create_task(git_service.poll_loop())
     logger.info("StackPilot backend ready on port %s", settings.PORT)
     yield
     update_task.cancel()
     schedule_task.cancel()
     logo_task.cancel()
+    git_task.cancel()
 
 
 app = FastAPI(title="StackPilot", version=APP_VERSION, lifespan=lifespan)
@@ -133,6 +138,8 @@ async def docker_error_handler(_request: Request, exc: DockerError):
 
 app.include_router(auth.router)
 app.include_router(stacks.router)
+app.include_router(git.router)
+app.include_router(git.hook_router)
 app.include_router(tokens.router)
 app.include_router(registries.router)
 app.include_router(secrets.router)

backend/models/__init__.py

@@ -4,6 +4,7 @@ from models.audit import AuditLog
 from models.auto_update import AutoUpdate
 from models.backup_destination import BackupDestination
 from models.backup_schedule import BackupSchedule
+from models.git_source import GitSource
 from models.registry import Registry
 from models.runtime_state import ImageStatus, LoginAttempt, StackLock
 from models.setting import Setting, Webhook
@@ -13,5 +14,5 @@ from models.user import User
 __all__ = [
     "User", "Stack", "AuditLog", "Setting", "Webhook",
     "BackupDestination", "BackupSchedule", "AutoUpdate",
-    "StackLock", "ImageStatus", "LoginAttempt", "Registry", "ApiToken",
+    "StackLock", "ImageStatus", "LoginAttempt", "Registry", "ApiToken", "GitSource",
 ]

backend/models/git_source.py

@@ -0,0 +1,93 @@
+from __future__ import annotations
+
+from datetime import datetime, timezone
+from typing import Optional
+
+from sqlmodel import Field, SQLModel
+
+
+def _now() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+#: How to reach a private repository. "token" is an HTTPS username + personal
+#: access token; "ssh" is a private key.
+AUTH_TYPES = ["none", "token", "ssh"]
+
+
+class GitSource(SQLModel, table=True):
+    """A Git repository that a stack's files are deployed from.
+
+    The repository is the source of truth: a sync overwrites the stack's files
+    with what the repo says, which is the whole point of GitOps and also the
+    thing to be careful about. Only files the repo has ever provided are touched
+    — see ``services/git_service.py`` — so the data directories compose creates
+    inside a stack folder are never at risk.
+    """
+
+    id: Optional[int] = Field(default=None, primary_key=True)
+    stack_id: str = Field(index=True, unique=True)
+    url: str
+    branch: str = "main"
+    #: Subdirectory inside the repository holding the compose file. Empty means
+    #: the repository root, which is the common case for one-stack repos.
+    subdir: str = ""
+    auth_type: str = "none"
+    username: Optional[str] = None
+    #: Encrypted: the access token, or the SSH private key.
+    secret: Optional[str] = None
+    #: Run `compose up -d` after a sync that actually changed something.
+    auto_deploy: bool = True
+    #: Poll the repository this often. None means only manual syncs and webhooks.
+    poll_interval_minutes: Optional[int] = None
+    #: Shared secret for the webhook endpoint (HMAC, or GitLab's token header).
+    webhook_secret: str = ""
+    #: JSON list of the paths the last sync wrote, relative to the stack folder.
+    #: The only files a later sync is allowed to delete.
+    managed_files: str = "[]"
+    last_commit: Optional[str] = None
+    last_synced_at: Optional[datetime] = None
+    last_error: Optional[str] = None
+    created_at: datetime = Field(default_factory=_now)
+    updated_at: datetime = Field(default_factory=_now)
+
+
+# --- API schemas ---
+
+
+class GitSourceWrite(SQLModel):
+    url: str
+    branch: str = "main"
+    subdir: str = ""
+    auth_type: str = "none"
+    username: Optional[str] = None
+    #: Omitted on update keeps the stored one.
+    secret: Optional[str] = None
+    auto_deploy: bool = True
+    poll_interval_minutes: Optional[int] = None
+
+
+class GitSourceRead(SQLModel):
+    stack_id: str
+    url: str
+    branch: str
+    subdir: str
+    auth_type: str
+    username: Optional[str]
+    has_secret: bool
+    auto_deploy: bool
+    poll_interval_minutes: Optional[int]
+    webhook_url: str
+    last_commit: Optional[str]
+    last_synced_at: Optional[datetime]
+    last_error: Optional[str]
+    managed_file_count: int
+
+
+class SyncResult(SQLModel):
+    changed: bool
+    commit: Optional[str] = None
+    written: list[str] = []
+    removed: list[str] = []
+    deployed: bool = False
+    detail: Optional[str] = None

backend/routers/git.py

@@ -0,0 +1,239 @@
+"""Deploying stacks from Git.
+
+Everything here is admin-only except the webhook, which cannot be: a Git forge
+has no StackPilot credentials to present. It authenticates with an HMAC over the
+request body instead, against a secret generated per stack — see
+``git_service.verify_webhook``.
+"""
+from __future__ import annotations
+
+from datetime import datetime, timezone
+
+from fastapi import APIRouter, Depends, HTTPException, Request
+from sqlmodel import Session, select
+
+from auth import require_admin
+from database import get_session
+from models.git_source import (
+    AUTH_TYPES,
+    GitSource,
+    GitSourceRead,
+    GitSourceWrite,
+    SyncResult,
+)
+from models.stack import Stack
+from models.user import User
+from services import audit_service, crypto_service, git_service
+
+router = APIRouter(prefix="/api/stacks/{stack_id}/git", tags=["git"])
+
+
+def _ip(request: Request) -> str:
+    return request.client.host if request.client else "unknown"
+
+
+def _stack_or_404(session: Session, stack_id: str) -> Stack:
+    stack = session.get(Stack, stack_id)
+    if not stack:
+        raise HTTPException(status_code=404, detail=f"Stack '{stack_id}' not found")
+    return stack
+
+
+def _source(session: Session, stack_id: str) -> GitSource:
+    row = session.exec(select(GitSource).where(GitSource.stack_id == stack_id)).first()
+    if not row:
+        raise HTTPException(
+            status_code=404, detail=f"Stack '{stack_id}' is not connected to a repository"
+        )
+    return row
+
+
+def _to_read(row: GitSource) -> GitSourceRead:
+    return GitSourceRead(
+        stack_id=row.stack_id,
+        url=row.url,
+        branch=row.branch,
+        subdir=row.subdir,
+        auth_type=row.auth_type,
+        username=row.username,
+        has_secret=bool(row.secret),
+        auto_deploy=row.auto_deploy,
+        poll_interval_minutes=row.poll_interval_minutes,
+        # Relative on purpose: StackPilot does not know its own external URL,
+        # and guessing one into a forge's webhook settings would be worse than
+        # letting the UI prefix the address the admin is already looking at.
+        webhook_url=f"/api/git/webhook/{row.stack_id}",
+        last_commit=row.last_commit,
+        last_synced_at=row.last_synced_at,
+        last_error=row.last_error,
+        managed_file_count=len(git_service._managed(row)),
+    )
+
+
+@router.get("", response_model=GitSourceRead)
+def get_source(
+    stack_id: str,
+    session: Session = Depends(get_session),
+    _user: User = Depends(require_admin),
+) -> GitSourceRead:
+    return _to_read(_source(session, stack_id))
+
+
+@router.put("", response_model=GitSourceRead)
+def connect(
+    stack_id: str,
+    body: GitSourceWrite,
+    request: Request,
+    session: Session = Depends(get_session),
+    user: User = Depends(require_admin),
+) -> GitSourceRead:
+    """Connect a stack to a repository, or change how it is connected.
+
+    Does not sync — the caller decides when, because the first sync overwrites
+    the stack's compose file with whatever the repository says.
+    """
+    _stack_or_404(session, stack_id)
+    if body.auth_type not in AUTH_TYPES:
+        raise HTTPException(status_code=400, detail=f"Unknown auth type '{body.auth_type}'")
+    if not (body.url or "").strip():
+        raise HTTPException(status_code=400, detail="A repository URL is required")
+
+    row = session.exec(select(GitSource).where(GitSource.stack_id == stack_id)).first()
+    if row is None:
+        row = GitSource(stack_id=stack_id, url="", webhook_secret=git_service.new_webhook_secret())
+
+    row.url = body.url.strip()
+    row.branch = (body.branch or "main").strip() or "main"
+    row.subdir = (body.subdir or "").strip().strip("/")
+    row.auth_type = body.auth_type
+    row.username = body.username
+    if body.secret:
+        row.secret = crypto_service.encrypt(body.secret)
+    elif body.auth_type == "none":
+        row.secret = None
+    row.auto_deploy = body.auto_deploy
+    row.poll_interval_minutes = body.poll_interval_minutes or None
+    row.updated_at = datetime.now(timezone.utc)
+    session.add(row)
+    session.commit()
+    session.refresh(row)
+    audit_service.record(
+        session, user=user.username, action="stack.git-connect", target=stack_id,
+        detail=f"{row.url}#{row.branch}", ip=_ip(request),
+    )
+    return _to_read(row)
+
+
+@router.delete("")
+def disconnect(
+    stack_id: str,
+    request: Request,
+    session: Session = Depends(get_session),
+    user: User = Depends(require_admin),
+) -> dict:
+    """Stop tracking the repository. The stack's files are left exactly as they are."""
+    row = _source(session, stack_id)
+    session.delete(row)
+    session.commit()
+    git_service.forget(stack_id)
+    audit_service.record(
+        session, user=user.username, action="stack.git-disconnect", target=stack_id,
+        ip=_ip(request),
+    )
+    return {"ok": True}
+
+
+@router.post("/sync", response_model=SyncResult)
+async def sync_now(
+    stack_id: str,
+    request: Request,
+    session: Session = Depends(get_session),
+    user: User = Depends(require_admin),
+) -> SyncResult:
+    row = _source(session, stack_id)
+    try:
+        result = await git_service.sync(session, row, actor=user.username)
+    except git_service.GitError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+    audit_service.record(
+        session, user=user.username, action="stack.git-sync", target=stack_id,
+        detail=f"{(result.commit or '')[:8]} {'changed' if result.changed else 'no change'}",
+        ip=_ip(request),
+    )
+    return result
+
+
+@router.get("/webhook-secret")
+def reveal_webhook_secret(
+    stack_id: str,
+    session: Session = Depends(get_session),
+    _user: User = Depends(require_admin),
+) -> dict:
+    """The secret to paste into the forge's webhook settings.
+
+    Readable rather than shown-once: it lives in the forge's configuration too,
+    so hiding it here would only mean re-pointing the webhook to see it again.
+    """
+    return {"secret": _source(session, stack_id).webhook_secret}
+
+
+@router.post("/webhook-secret")
+def rotate_webhook_secret(
+    stack_id: str,
+    request: Request,
+    session: Session = Depends(get_session),
+    user: User = Depends(require_admin),
+) -> dict:
+    row = _source(session, stack_id)
+    row.webhook_secret = git_service.new_webhook_secret()
+    row.updated_at = datetime.now(timezone.utc)
+    session.add(row)
+    session.commit()
+    audit_service.record(
+        session, user=user.username, action="stack.git-rotate-secret", target=stack_id,
+        ip=_ip(request),
+    )
+    return {"secret": row.webhook_secret}
+
+
+# --------------------------------------------------------------------------- #
+# The webhook
+# --------------------------------------------------------------------------- #
+
+hook_router = APIRouter(prefix="/api/git", tags=["git"])
+
+
+@hook_router.post("/webhook/{stack_id}")
+async def webhook(
+    stack_id: str,
+    request: Request,
+    session: Session = Depends(get_session),
+) -> dict:
+    """Push webhook from a Git forge.
+
+    Unauthenticated in the usual sense — a forge holds no StackPilot session —
+    and authorized by an HMAC over the body instead. An unsigned or wrongly
+    signed call is a 404, not a 403: without credentials to present, telling a
+    caller that a given stack *is* connected to a repository is information it
+    has not earned.
+    """
+    row = session.exec(select(GitSource).where(GitSource.stack_id == stack_id)).first()
+    body = await request.body()
+    if not row or not git_service.verify_webhook(row, body, request.headers):
+        raise HTTPException(status_code=404, detail="Not found")
+
+    try:
+        result = await git_service.sync(session, row, actor="webhook")
+    except git_service.GitError as exc:
+        raise HTTPException(status_code=502, detail=str(exc)) from exc
+    audit_service.record(
+        session, user="webhook", action="stack.git-sync", target=stack_id,
+        detail=f"{(result.commit or '')[:8]} {'changed' if result.changed else 'no change'}",
+        ip=_ip(request),
+    )
+    return {
+        "ok": True,
+        "changed": result.changed,
+        "commit": result.commit,
+        "deployed": result.deployed,
+    }

backend/services/git_service.py

@@ -0,0 +1,447 @@
+"""Deploying a stack from a Git repository.
+
+The repository is the source of truth: a sync makes the stack's files match what
+the repo says, and optionally runs ``compose up -d`` when that changed anything.
+Two things about *how* are worth stating up front, because both are places this
+could quietly destroy data.
+
+**The clone does not live in the stack folder.** It is cached under
+``${DATA_DIR}/git/<stack_id>`` and the relevant subtree is copied across. A
+stack folder holds more than the repo's files — compose creates bind-mount
+directories like ``./config`` right there, full of live application data — so a
+``git reset --hard`` or ``git clean`` in that folder would be catastrophic. In
+the cache directory both are safe, and the copy step is where the care goes.
+
+**Only files the repo has provided are ever deleted.** Each sync records the
+paths it wrote (``GitSource.managed_files``); the next sync removes the ones the
+repo no longer has, and nothing else. A file that was never in the repository
+cannot be touched, no matter what happened to it.
+
+Credentials never reach a command line. A token is passed to git through
+``GIT_ASKPASS`` and an environment variable, an SSH key through a 0600 file
+outside the working tree — so neither shows up in ``ps``, in the repo's own
+config, or in an error message this module passes on.
+"""
+from __future__ import annotations
+
+import asyncio
+import filecmp
+import hmac
+import json
+import logging
+import os
+import re
+import secrets
+import shutil
+import stat
+from datetime import datetime, timezone
+from typing import Optional
+
+from sqlmodel import Session
+
+from config import settings
+from models.git_source import GitSource, SyncResult
+from services import compose_service, crypto_service, stack_lock_service
+
+logger = logging.getLogger("stackpilot.git")
+
+GIT_TIMEOUT = 300.0
+_SAFE_ID_RE = re.compile(r"^[A-Za-z0-9._-]{1,128}$")
+
+
+class GitError(Exception):
+    """A repository that cannot be reached, or a sync that cannot be completed."""
+
+
+# --------------------------------------------------------------------------- #
+# Paths
+# --------------------------------------------------------------------------- #
+
+
+def cache_root() -> str:
+    return os.path.join(settings.DATA_DIR, "git")
+
+
+def repo_dir(stack_id: str) -> str:
+    if not _SAFE_ID_RE.match(stack_id or "") or stack_id in (".", ".."):
+        raise GitError(f"Invalid stack id '{stack_id}'")
+    return os.path.join(cache_root(), stack_id)
+
+
+def _key_path(stack_id: str) -> str:
+    # Beside the clone, never inside it — a working tree gets reset and cleaned.
+    return os.path.join(cache_root(), f"{stack_id}.key")
+
+
+def _askpass_path(stack_id: str) -> str:
+    return os.path.join(cache_root(), f"{stack_id}.askpass")
+
+
+def new_webhook_secret() -> str:
+    return secrets.token_urlsafe(24)
+
+
+# --------------------------------------------------------------------------- #
+# Running git
+# --------------------------------------------------------------------------- #
+
+
+def _redact(text: str, *secrets_: Optional[str]) -> str:
+    """Strip anything secret out of git's output before it is shown or stored."""
+    for value in secrets_:
+        if value:
+            text = text.replace(value, "••••••")
+    # A URL that carries credentials, in case one ever reaches git's output.
+    return re.sub(r"(https?://)[^/\s:@]+:[^/\s@]+@", r"\1••••••@", text)
+
+
+async def _git(args: list[str], env: dict, secret: Optional[str] = None) -> str:
+    """Run git, returning stdout. Raises GitError with a redacted message."""
+    proc = await asyncio.create_subprocess_exec(
+        "git",
+        *args,
+        env=env,
+        stdout=asyncio.subprocess.PIPE,
+        stderr=asyncio.subprocess.PIPE,
+    )
+    try:
+        out_b, err_b = await asyncio.wait_for(proc.communicate(), timeout=GIT_TIMEOUT)
+    except asyncio.TimeoutError as exc:
+        proc.kill()
+        raise GitError("git timed out") from exc
+    out = out_b.decode("utf-8", "replace")
+    if proc.returncode != 0:
+        err = _redact(err_b.decode("utf-8", "replace").strip(), secret)
+        raise GitError(err or f"git {args[0]} failed (exit {proc.returncode})")
+    return out
+
+
+def _auth_env(source: GitSource) -> tuple[dict, Optional[str]]:
+    """Environment for git, plus the plaintext secret so output can be redacted.
+
+    Credentials go in the environment, never in argv: ``ps`` is readable by
+    every process on the host, and StackPilot runs in a container people share
+    with their whole stack.
+    """
+    env = {
+        **os.environ,
+        # No interactive prompting: a private repo without credentials must fail
+        # fast rather than hang forever waiting on a terminal that is not there.
+        "GIT_TERMINAL_PROMPT": "0",
+        "GIT_CONFIG_NOSYSTEM": "1",
+        "HOME": cache_root(),
+    }
+    if source.auth_type == "none" or not source.secret:
+        return env, None
+
+    plaintext = crypto_service.decrypt(source.secret)
+    os.makedirs(cache_root(), mode=0o700, exist_ok=True)
+
+    if source.auth_type == "ssh":
+        key_file = _key_path(source.stack_id)
+        fd = os.open(key_file, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
+        with os.fdopen(fd, "w", encoding="utf-8") as fh:
+            fh.write(plaintext.rstrip("\n") + "\n")
+        env["GIT_SSH_COMMAND"] = (
+            f"ssh -i {key_file} -o IdentitiesOnly=yes "
+            # accept-new pins the host key on first contact and refuses it if it
+            # ever changes, which is the strongest option that does not require
+            # the operator to paste a fingerprint by hand.
+            "-o StrictHostKeyChecking=accept-new "
+            f"-o UserKnownHostsFile={os.path.join(cache_root(), 'known_hosts')}"
+        )
+        return env, plaintext
+
+    # token: HTTPS basic auth, handed over through an askpass helper.
+    askpass = _askpass_path(source.stack_id)
+    fd = os.open(askpass, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o700)
+    with os.fdopen(fd, "w", encoding="utf-8") as fh:
+        fh.write('#!/bin/sh\ncase "$1" in Username*) echo "$GIT_USER";; *) echo "$GIT_TOKEN";; esac\n')
+    env["GIT_ASKPASS"] = askpass
+    env["GIT_USER"] = source.username or "git"
+    env["GIT_TOKEN"] = plaintext
+    return env, plaintext
+
+
+def _cleanup_auth(source: GitSource) -> None:
+    for path in (_key_path(source.stack_id), _askpass_path(source.stack_id)):
+        try:
+            os.remove(path)
+        except OSError:
+            pass
+
+
+# --------------------------------------------------------------------------- #
+# Fetching
+# --------------------------------------------------------------------------- #
+
+
+async def _fetch(source: GitSource, env: dict, secret: Optional[str]) -> str:
+    """Bring the cached clone to the tip of the configured branch. Returns the commit."""
+    directory = repo_dir(source.stack_id)
+    branch = source.branch or "main"
+    os.makedirs(cache_root(), mode=0o700, exist_ok=True)
+
+    if os.path.isdir(os.path.join(directory, ".git")):
+        try:
+            remote = (await _git(["-C", directory, "remote", "get-url", "origin"], env, secret)).strip()
+        except GitError:
+            remote = ""
+        if remote != source.url:
+            # Repointed at a different repository: start clean rather than try
+            # to reconcile two unrelated histories.
+            shutil.rmtree(directory, ignore_errors=True)
+
+    if not os.path.isdir(os.path.join(directory, ".git")):
+        await _git(
+            ["clone", "--depth", "1", "--branch", branch, source.url, directory], env, secret
+        )
+    else:
+        await _git(["-C", directory, "fetch", "--depth", "1", "origin", branch], env, secret)
+        await _git(["-C", directory, "checkout", "-B", branch, "FETCH_HEAD"], env, secret)
+        await _git(["-C", directory, "reset", "--hard", "FETCH_HEAD"], env, secret)
+        # Safe here and only here: this directory holds nothing but the clone.
+        await _git(["-C", directory, "clean", "-fdx"], env, secret)
+
+    return (await _git(["-C", directory, "rev-parse", "HEAD"], env, secret)).strip()
+
+
+# --------------------------------------------------------------------------- #
+# Copying the repo's files into the stack
+# --------------------------------------------------------------------------- #
+
+
+def _inside(path: str, parent: str) -> bool:
+    return os.path.realpath(path).startswith(os.path.realpath(parent).rstrip("/") + "/")
+
+
+def _source_tree(source: GitSource) -> str:
+    directory = repo_dir(source.stack_id)
+    subdir = (source.subdir or "").strip().strip("/")
+    if not subdir:
+        return directory
+    tree = os.path.join(directory, subdir)
+    # The subdirectory comes from user input and is about to be walked.
+    if not _inside(tree, directory):
+        raise GitError(f"Subdirectory '{source.subdir}' leaves the repository")
+    if not os.path.isdir(tree):
+        raise GitError(f"'{source.subdir}' does not exist in the repository")
+    return tree
+
+
+def _materialise(source: GitSource, previous: list[str]) -> tuple[list[str], list[str], list[str]]:
+    """Copy the repo subtree into the stack folder.
+
+    Returns (current, written, removed): everything the repo provides, the
+    subset that actually changed on disk, and the files dropped because the repo
+    no longer has them.
+    """
+    tree = _source_tree(source)
+    stack_dir = compose_service.stack_dir(source.stack_id)
+    os.makedirs(stack_dir, exist_ok=True)
+
+    current: list[str] = []
+    written: list[str] = []
+    for root, dirs, files in os.walk(tree):
+        dirs[:] = [d for d in dirs if d != ".git"]
+        for name in files:
+            src = os.path.join(root, name)
+            rel = os.path.relpath(src, tree)
+            dest = os.path.join(stack_dir, rel)
+            if not _inside(dest, stack_dir):
+                continue  # a symlinked path trying to escape the stack folder
+            current.append(rel)
+            # shallow=False: compare contents, not just size and mtime, or a
+            # revert to a same-sized earlier version would look like no change.
+            if os.path.isfile(dest) and filecmp.cmp(src, dest, shallow=False):
+                continue
+            os.makedirs(os.path.dirname(dest), exist_ok=True)
+            shutil.copy2(src, dest)
+            written.append(rel)
+
+    removed: list[str] = []
+    for rel in previous:
+        if rel in current:
+            continue
+        dest = os.path.join(stack_dir, rel)
+        if not _inside(dest, stack_dir) or not os.path.isfile(dest):
+            continue
+        try:
+            os.remove(dest)
+            removed.append(rel)
+        except OSError as exc:
+            logger.warning("Could not remove %s: %s", dest, exc)
+    _prune_empty_dirs(stack_dir, removed)
+    return sorted(current), sorted(written), sorted(removed)
+
+
+def _prune_empty_dirs(stack_dir: str, removed: list[str]) -> None:
+    """Drop directories left empty by removed files, never the stack folder."""
+    for rel in removed:
+        directory = os.path.dirname(os.path.join(stack_dir, rel))
+        while _inside(directory, stack_dir):
+            try:
+                os.rmdir(directory)  # fails unless empty, which is what we want
+            except OSError:
+                break
+            directory = os.path.dirname(directory)
+
+
+# --------------------------------------------------------------------------- #
+# Syncing
+# --------------------------------------------------------------------------- #
+
+
+async def sync(session: Session, source: GitSource, actor: str = "system") -> SyncResult:
+    """Fetch, copy into the stack, and deploy when something changed."""
+    env, secret = _auth_env(source)
+    try:
+        commit = await _fetch(source, env, secret)
+        previous = _managed(source)
+        current, written, removed = _materialise(source, previous)
+    except GitError as exc:
+        source.last_error = _redact(str(exc), secret)[:1000]
+        source.updated_at = datetime.now(timezone.utc)
+        session.add(source)
+        session.commit()
+        raise
+    finally:
+        _cleanup_auth(source)
+
+    changed = bool(written or removed)
+    source.managed_files = json.dumps(current)
+    source.last_commit = commit
+    source.last_synced_at = datetime.now(timezone.utc)
+    source.updated_at = source.last_synced_at
+    source.last_error = None
+    session.add(source)
+    session.commit()
+
+    result = SyncResult(changed=changed, commit=commit, written=written, removed=removed)
+    if changed and source.auto_deploy:
+        result.deployed, result.detail = await _deploy(session, source.stack_id, actor)
+    return result
+
+
+async def _deploy(session: Session, stack_id: str, actor: str) -> tuple[bool, Optional[str]]:
+    """`compose up -d`, under the same lock every other lifecycle action takes."""
+    from services import audit_service
+
+    try:
+        with stack_lock_service.hold(session, stack_id, "git-deploy", actor):
+            outcome = await compose_service.up(stack_id)
+    except stack_lock_service.StackBusy as exc:
+        # Somebody is already deploying. The files are updated; say so rather
+        # than queue a second compose run at the same project.
+        return False, f"stack is busy ({exc.action}); files synced but not deployed"
+    except Exception as exc:  # noqa: BLE001 - a failed deploy must not lose the sync
+        return False, str(exc)[:500]
+
+    ok = outcome.get("returncode") in (0, None)
+    audit_service.record(
+        session, user=actor, action="stack.git-deploy", target=stack_id,
+        detail=f"rc={outcome.get('returncode')}",
+    )
+    return ok, (outcome.get("stderr") or "").strip()[-1000:] or None
+
+
+def _managed(source: GitSource) -> list[str]:
+    try:
+        value = json.loads(source.managed_files or "[]")
+    except json.JSONDecodeError:
+        return []
+    return [str(v) for v in value] if isinstance(value, list) else []
+
+
+def forget(stack_id: str) -> None:
+    """Drop the cached clone and any credential files for a stack."""
+    try:
+        shutil.rmtree(repo_dir(stack_id), ignore_errors=True)
+    except GitError:
+        return
+    for path in (_key_path(stack_id), _askpass_path(stack_id)):
+        try:
+            os.remove(path)
+        except OSError:
+            pass
+
+
+# --------------------------------------------------------------------------- #
+# Webhooks
+# --------------------------------------------------------------------------- #
+
+
+def verify_webhook(source: GitSource, body: bytes, headers) -> bool:
+    """Is this webhook really from the forge that holds our secret?
+
+    Supports the two schemes between them covered by GitHub, Gitea, Forgejo and
+    GitLab. Both comparisons are constant-time.
+    """
+    expected = source.webhook_secret or ""
+    if not expected:
+        return False
+
+    signature = headers.get("X-Hub-Signature-256") or ""
+    if signature.startswith("sha256="):
+        digest = hmac.new(expected.encode(), body, "sha256").hexdigest()
+        return hmac.compare_digest(signature[len("sha256=") :], digest)
+
+    gitlab = headers.get("X-Gitlab-Token") or ""
+    if gitlab:
+        return hmac.compare_digest(gitlab, expected)
+
+    return False
+
+
+# --------------------------------------------------------------------------- #
+# Polling
+# --------------------------------------------------------------------------- #
+
+
+def due(source: GitSource, now: Optional[datetime] = None) -> bool:
+    """Is this source's polling interval up?"""
+    if not source.poll_interval_minutes or source.poll_interval_minutes < 1:
+        return False
+    if source.last_synced_at is None:
+        return True
+    last = source.last_synced_at
+    if last.tzinfo is None:
+        last = last.replace(tzinfo=timezone.utc)
+    elapsed = (now or datetime.now(timezone.utc)) - last
+    return elapsed.total_seconds() >= source.poll_interval_minutes * 60
+
+
+async def poll_loop(interval: float = 60.0) -> None:
+    """Sync every repository whose interval is up. Never dies on one failure."""
+    from sqlmodel import select
+
+    from database import engine
+
+    while True:
+        await asyncio.sleep(interval)
+        try:
+            with Session(engine) as session:
+                sources = session.exec(select(GitSource)).all()
+                for source in sources:
+                    if not due(source):
+                        continue
+                    try:
+                        result = await sync(session, source, actor="poll")
+                        if result.changed:
+                            logger.info(
+                                "Git sync updated %s to %s", source.stack_id,
+                                (result.commit or "")[:8],
+                            )
+                    except Exception as exc:  # noqa: BLE001 - one repo, not all
+                        logger.warning("Git sync failed for %s: %s", source.stack_id, exc)
+        except Exception as exc:  # noqa: BLE001 - the loop outlives everything
+            logger.warning("Git polling pass failed: %s", exc)
+
+
+def ensure_cache_root() -> None:
+    """The cache doubles as git's HOME, so it must exist and stay private."""
+    try:
+        os.makedirs(cache_root(), mode=0o700, exist_ok=True)
+        os.chmod(cache_root(), stat.S_IRWXU)
+    except OSError as exc:
+        logger.warning("Could not create the Git cache directory: %s", exc)

backend/tests/test_git_source.py

@@ -0,0 +1,413 @@
+"""Deploying a stack from Git.
+
+Real repositories, created locally with the real git binary — mocking git would
+mostly test the mock. Nothing here reaches the network.
+
+The test that matters most is the one about deletion. A stack folder holds live
+application data next to the compose file: compose creates bind-mount
+directories like ``./config`` right there. A sync that "makes the folder match
+the repo" by clearing what the repo does not have would destroy exactly that, so
+the rule is that only files the repository has itself provided may ever be
+removed.
+"""
+from __future__ import annotations
+
+import asyncio
+import json
+import os
+import subprocess
+
+import pytest
+from sqlmodel import Session, delete
+
+
+@pytest.fixture
+def svc(db):
+    from services import git_service
+
+    return git_service
+
+
+@pytest.fixture(autouse=True)
+def clean_sources(db):
+    from database import engine
+    from models.git_source import GitSource
+
+    def wipe():
+        with Session(engine) as session:
+            session.exec(delete(GitSource))
+            session.commit()
+
+    wipe()
+    yield
+    wipe()
+
+
+def _run(*args, cwd):
+    subprocess.run(
+        args,
+        cwd=cwd,
+        check=True,
+        capture_output=True,
+        env={
+            **os.environ,
+            "GIT_AUTHOR_NAME": "t",
+            "GIT_AUTHOR_EMAIL": "t@example.com",
+            "GIT_COMMITTER_NAME": "t",
+            "GIT_COMMITTER_EMAIL": "t@example.com",
+        },
+    )
+
+
+class _Origin:
+    """A real repository on disk. ``str()`` is its path, so it can be a clone URL."""
+
+    def __init__(self, path):
+        self.path = path
+
+    def __str__(self) -> str:
+        return str(self.path)
+
+    def commit(self, files: dict, message: str = "change") -> None:
+        for name, content in files.items():
+            path = self.path / name
+            if content is None:
+                path.unlink(missing_ok=True)
+                continue
+            path.parent.mkdir(parents=True, exist_ok=True)
+            path.write_text(content)
+        _run("git", "add", "-A", cwd=self.path)
+        _run("git", "commit", "-q", "-m", message, cwd=self.path)
+
+
+@pytest.fixture
+def origin(tmp_path):
+    repo = tmp_path / "origin"
+    repo.mkdir()
+    _run("git", "init", "-q", "-b", "main", cwd=repo)
+    made = _Origin(repo)
+    made.commit({"compose.yaml": "services:\n  app:\n    image: nginx:alpine\n"}, "init")
+    return made
+
+
+@pytest.fixture
+def source(db, origin, tmp_path, monkeypatch):
+    """A stack wired to that repository, with both roots inside tmp_path."""
+    from config import settings
+    from database import engine
+    from models.git_source import GitSource
+    from models.stack import Stack
+    from services import git_service
+
+    monkeypatch.setattr(settings, "DATA_DIR", str(tmp_path / "data"))
+    monkeypatch.setattr(settings, "STACKS_DIR", str(tmp_path / "stacks"))
+    os.makedirs(settings.STACKS_DIR, exist_ok=True)
+    git_service.ensure_cache_root()
+
+    with Session(engine) as session:
+        if (old := session.get(Stack, "gitstack")) is not None:
+            session.delete(old)
+            session.commit()
+        session.add(Stack(id="gitstack", name="Git Stack"))
+        row = GitSource(
+            stack_id="gitstack",
+            url=str(origin),
+            branch="main",
+            webhook_secret="s3cret",
+        )
+        session.add(row)
+        session.commit()
+        session.refresh(row)
+        row_id = row.id
+
+    with Session(engine) as session:
+        yield session, session.get(GitSource, row_id)
+
+
+def _sync(session, row, svc):
+    return asyncio.run(svc.sync(session, row, actor="test"))
+
+
+def _stack_dir() -> str:
+    from services import compose_service
+
+    return compose_service.stack_dir("gitstack")
+
+
+# --------------------------------------------------------------------------- #
+# Syncing
+# --------------------------------------------------------------------------- #
+
+
+def test_the_first_sync_brings_the_repo_into_the_stack(svc, source):
+    session, row = source
+    result = _sync(session, row, svc)
+
+    assert result.changed is True
+    assert result.written == ["compose.yaml"]
+    assert "nginx:alpine" in open(os.path.join(_stack_dir(), "compose.yaml")).read()
+    assert row.last_commit and row.last_error is None
+
+
+def test_syncing_again_with_no_new_commit_changes_nothing(svc, source):
+    session, row = source
+    _sync(session, row, svc)
+    again = _sync(session, row, svc)
+    assert again.changed is False
+    assert again.written == [] and again.removed == []
+
+
+def test_a_new_commit_is_picked_up(svc, source, origin):
+    session, row = source
+    _sync(session, row, svc)
+    origin.commit({"compose.yaml": "services:\n  app:\n    image: nginx:1.27\n"})
+
+    result = _sync(session, row, svc)
+    assert result.changed is True
+    assert "nginx:1.27" in open(os.path.join(_stack_dir(), "compose.yaml")).read()
+
+
+def test_a_hand_edited_file_is_put_back(svc, source):
+    """The repository is the source of truth; that is the whole point."""
+    session, row = source
+    _sync(session, row, svc)
+    path = os.path.join(_stack_dir(), "compose.yaml")
+    with open(path, "w") as fh:
+        fh.write("services: {}\n")
+
+    result = _sync(session, row, svc)
+    assert result.changed is True
+    assert "nginx:alpine" in open(path).read()
+
+
+def test_only_the_configured_subdirectory_is_deployed(svc, source, origin):
+    session, row = source
+    origin.commit({"stacks/web/compose.yaml": "services:\n  web:\n    image: caddy\n"})
+    row.subdir = "stacks/web"
+
+    result = _sync(session, row, svc)
+    assert result.written == ["compose.yaml"]
+    assert "caddy" in open(os.path.join(_stack_dir(), "compose.yaml")).read()
+
+
+def test_a_subdirectory_that_escapes_the_repo_is_refused(svc, source):
+    session, row = source
+    row.subdir = "../../../etc"
+    with pytest.raises(svc.GitError):
+        _sync(session, row, svc)
+
+
+def test_a_missing_subdirectory_is_reported(svc, source):
+    session, row = source
+    row.subdir = "nope"
+    with pytest.raises(svc.GitError):
+        _sync(session, row, svc)
+    assert row.last_error and "nope" in row.last_error
+
+
+def test_an_unreachable_repository_is_recorded_not_raised_away(svc, source, tmp_path):
+    session, row = source
+    row.url = str(tmp_path / "does-not-exist")
+    with pytest.raises(svc.GitError):
+        _sync(session, row, svc)
+    # Stored, so the UI can show why the last attempt failed.
+    assert row.last_error
+
+
+# --------------------------------------------------------------------------- #
+# What may be deleted — the dangerous part
+# --------------------------------------------------------------------------- #
+
+
+def test_a_file_dropped_from_the_repo_is_removed_from_the_stack(svc, source, origin):
+    session, row = source
+    origin.commit({"extra.env": "A=1\n"})
+    _sync(session, row, svc)
+    assert os.path.isfile(os.path.join(_stack_dir(), "extra.env"))
+
+    origin.commit({"extra.env": None}, "drop it")
+    result = _sync(session, row, svc)
+    assert result.removed == ["extra.env"]
+    assert not os.path.exists(os.path.join(_stack_dir(), "extra.env"))
+
+
+def test_files_the_repo_never_had_are_never_touched(svc, source, origin):
+    """Compose puts live application data in the stack folder. It must survive."""
+    session, row = source
+    _sync(session, row, svc)
+
+    data_dir = os.path.join(_stack_dir(), "config")
+    os.makedirs(data_dir, exist_ok=True)
+    with open(os.path.join(data_dir, "app.db"), "w") as fh:
+        fh.write("precious")
+    with open(os.path.join(_stack_dir(), ".env"), "w") as fh:
+        fh.write("SECRET=hunter2\n")
+
+    origin.commit({"compose.yaml": "services:\n  app:\n    image: nginx:1.27\n"})
+    _sync(session, row, svc)
+
+    assert open(os.path.join(data_dir, "app.db")).read() == "precious"
+    assert open(os.path.join(_stack_dir(), ".env")).read() == "SECRET=hunter2\n"
+
+
+def test_a_file_the_repo_stops_providing_is_only_removed_if_it_provided_it(svc, source, origin):
+    """A file the stack folder already had before Git ever touched it."""
+    session, row = source
+    os.makedirs(_stack_dir(), exist_ok=True)
+    with open(os.path.join(_stack_dir(), "notes.txt"), "w") as fh:
+        fh.write("mine")
+    _sync(session, row, svc)
+    # The repo never provided notes.txt, so the first sync left it alone.
+    assert os.path.isfile(os.path.join(_stack_dir(), "notes.txt"))
+    assert "notes.txt" not in json.loads(row.managed_files)
+
+
+def test_managed_files_are_recorded_for_the_next_sync(svc, source, origin):
+    session, row = source
+    origin.commit({"a.yaml": "a", "sub/b.yaml": "b"})
+    _sync(session, row, svc)
+    assert sorted(json.loads(row.managed_files)) == [
+        "a.yaml",
+        "compose.yaml",
+        os.path.join("sub", "b.yaml"),
+    ]
+
+
+def test_a_directory_left_empty_by_a_removal_is_pruned(svc, source, origin):
+    session, row = source
+    origin.commit({"sub/b.yaml": "b"})
+    _sync(session, row, svc)
+    assert os.path.isdir(os.path.join(_stack_dir(), "sub"))
+
+    origin.commit({"sub/b.yaml": None}, "drop")
+    _sync(session, row, svc)
+    assert not os.path.exists(os.path.join(_stack_dir(), "sub"))
+    # But never the stack folder itself.
+    assert os.path.isdir(_stack_dir())
+
+
+# --------------------------------------------------------------------------- #
+# Webhook authorization
+# --------------------------------------------------------------------------- #
+
+
+def _sig(secret: str, body: bytes) -> str:
+    import hmac
+
+    return "sha256=" + hmac.new(secret.encode(), body, "sha256").hexdigest()
+
+
+def test_a_correctly_signed_webhook_is_accepted(svc, source):
+    _session, row = source
+    body = b'{"ref":"refs/heads/main"}'
+    assert svc.verify_webhook(row, body, {"X-Hub-Signature-256": _sig("s3cret", body)})
+
+
+def test_a_wrong_signature_is_rejected(svc, source):
+    _session, row = source
+    body = b'{"ref":"refs/heads/main"}'
+    assert not svc.verify_webhook(row, body, {"X-Hub-Signature-256": _sig("wrong", body)})
+    # A signature over different content must not carry over.
+    assert not svc.verify_webhook(row, b"tampered", {"X-Hub-Signature-256": _sig("s3cret", body)})
+
+
+def test_an_unsigned_webhook_is_rejected(svc, source):
+    _session, row = source
+    assert not svc.verify_webhook(row, b"{}", {})
+
+
+def test_the_gitlab_token_header_works_too(svc, source):
+    _session, row = source
+    assert svc.verify_webhook(row, b"{}", {"X-Gitlab-Token": "s3cret"})
+    assert not svc.verify_webhook(row, b"{}", {"X-Gitlab-Token": "nope"})
+
+
+def test_a_source_with_no_secret_accepts_nothing(svc, source):
+    _session, row = source
+    row.webhook_secret = ""
+    assert not svc.verify_webhook(row, b"{}", {"X-Gitlab-Token": ""})
+
+
+def test_the_webhook_endpoint_hides_whether_a_stack_is_connected(client, source):
+    """Unsigned calls get 404, so the endpoint cannot be used to enumerate."""
+    connected = client.post("/api/git/webhook/gitstack", content=b"{}")
+    unknown = client.post("/api/git/webhook/no-such-stack", content=b"{}")
+    assert connected.status_code == 404
+    assert unknown.status_code == 404
+    assert connected.json() == unknown.json()
+
+
+# --------------------------------------------------------------------------- #
+# Polling
+# --------------------------------------------------------------------------- #
+
+
+def test_polling_is_off_unless_an_interval_is_set(svc, source):
+    _session, row = source
+    row.poll_interval_minutes = None
+    assert svc.due(row) is False
+    row.poll_interval_minutes = 0
+    assert svc.due(row) is False
+
+
+def test_a_source_that_has_never_synced_is_due(svc, source):
+    _session, row = source
+    row.poll_interval_minutes = 15
+    row.last_synced_at = None
+    assert svc.due(row) is True
+
+
+def test_due_respects_the_interval(svc, source):
+    from datetime import datetime, timedelta, timezone
+
+    _session, row = source
+    row.poll_interval_minutes = 15
+    now = datetime.now(timezone.utc)
+    row.last_synced_at = now - timedelta(minutes=5)
+    assert svc.due(row, now) is False
+    row.last_synced_at = now - timedelta(minutes=16)
+    assert svc.due(row, now) is True
+
+
+# --------------------------------------------------------------------------- #
+# Secrets never leak
+# --------------------------------------------------------------------------- #
+
+
+def test_the_api_never_returns_the_stored_secret(as_admin, source):
+    from services import crypto_service
+
+    session, row = source
+    row.auth_type = "token"
+    row.username = "bob"
+    row.secret = crypto_service.encrypt("ghp_supersecret")
+    session.add(row)
+    session.commit()
+
+    body = as_admin.get("/api/stacks/gitstack/git").text
+    assert "ghp_supersecret" not in body
+    assert '"has_secret":true' in body.replace(" ", "")
+
+
+def test_a_token_is_scrubbed_from_error_output(svc):
+    message = "fatal: could not read from https://bob:ghp_supersecret@example.com/x.git"
+    assert "ghp_supersecret" not in svc._redact(message, "ghp_supersecret")
+    # Even without being told the value, a credential-carrying URL is masked.
+    assert "ghp_supersecret" not in svc._redact(message)
+
+
+def test_connecting_and_disconnecting_through_the_api(as_admin, source, origin):
+    connected = as_admin.put(
+        "/api/stacks/gitstack/git",
+        json={"url": str(origin), "branch": "main", "auto_deploy": False},
+    )
+    assert connected.status_code == 200, connected.text
+    assert connected.json()["webhook_url"] == "/api/git/webhook/gitstack"
+
+    assert as_admin.delete("/api/stacks/gitstack/git").status_code == 200
+    assert as_admin.get("/api/stacks/gitstack/git").status_code == 404
+
+
+def test_the_read_only_role_cannot_touch_git_settings(as_user, source):
+    assert as_user.get("/api/stacks/gitstack/git").status_code == 403
+    assert as_user.post("/api/stacks/gitstack/git/sync").status_code == 403

backend/tests/test_route_authorization.py

@@ -39,6 +39,12 @@ PUBLIC = {
     # Only drops the refresh cookie. Requiring a valid token would mean you
     # cannot sign out once the session has already gone stale.
     "POST /api/auth/logout",
+    # A Git forge has no StackPilot credentials to present, so this one cannot
+    # be behind a bearer token. It is authorized instead by an HMAC over the
+    # request body against a per-stack secret, and answers 404 — not 403 — to
+    # anything unsigned, so it cannot be used to discover which stacks exist or
+    # which are connected to a repository. See routers/git.py.
+    "POST /api/git/webhook/{stack_id}",
 }
 
 #: Reachable by the read-only ``user`` role. Everything here has been checked

backend/version.py

@@ -1,3 +1,3 @@
 """Single source of truth for the StackPilot release version."""
 
-APP_VERSION = "0.57.0"
+APP_VERSION = "0.58.0"