From a1cd14a1cddbb45fb635cd885b8d98dbc5b498a1 Mon Sep 17 00:00:00 2001 From: menzelj Date: Fri, 18 Sep 2026 01:16:45 +0200 Subject: [PATCH] Scan images for known vulnerabilities (0.59.0) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Images page knew what was running and whether it was current. It could not say whether any of it was exploitable, which is the question people actually have about a homelab full of images they pulled once and forgot. Trivy runs as a throwaway container rather than being installed into StackPilot's image, reusing the helper-container pattern backups already use for volume contents. Three reasons: a 100 MB security tool and a vulnerability database that changes weekly have no business in a release artifact, pinning SCANNER_IMAGE is then a real version control, and the scanner updates itself by pulling a newer tag. It gets the socket read-only so it inspects images the daemon already has instead of pulling them again, and a named volume for its database so the ~50 MB download happens once rather than per scan. The number the UI leads with is "fixable", not the total. A base image with 300 unfixable low-severity CVEs is not a task and a page that shows 300 in red teaches people to ignore it; three findings with a fixed version available are something to do this afternoon. Counts are stored per severity, findings are sorted worst-first and capped at 200 — every finding is counted, only the list is trimmed, so the cap can never hide the severity distribution. The failure mode this had to avoid is a security feature that reads as clean when it is broken. A scanner that cannot run stores the error and *keeps the previous counts* rather than resetting to zero, so a transient daemon problem does not silently turn a bad image green. There is a test for exactly that, and another for unparseable output. Staleness is handled the same way: the local image id is recorded with the scan, and pulling the image marks the result stale instead of presenting yesterday's numbers for today's bytes. Sweeps are deliberately serial and singly-locked. Scanning is CPU- and IO-heavy, and running eight at once on a homelab box would starve the very containers the scan is meant to protect. docker-py is synchronous, so the scan itself goes to a thread — otherwise a ten-minute scan blocks every other request on the loop. Reading results is allowed for the read-only role, which the authorization matrix made me justify in writing: CVE ids and package versions for images whose tags and compose files that role can already see, and polling them is the monitoring use case a read-only API token exists for. Running a scan stays admin-only because it spends real CPU. 20 tests against a report shaped like Trivy's real output, covering the counting, the fixable number, worst-first ordering, the cap, both failure paths, staleness, and that two sweeps cannot overlap. Verified end to end through the API as well, including that a failed rescan keeps its previous counts and shows the error. Co-Authored-By: Claude Opus 5 --- README.md | 45 +++ backend/config.py | 4 + backend/models/__init__.py | 3 +- backend/models/image_scan.py | 72 ++++ backend/routers/images.py | 110 +++++- backend/services/scan_service.py | 263 +++++++++++++++ backend/tests/test_route_authorization.py | 9 + backend/tests/test_scan_service.py | 316 ++++++++++++++++++ backend/version.py | 2 +- frontend/package.json | 2 +- frontend/src/api/images.ts | 45 +++ .../src/components/stacks/ScanFindings.tsx | 204 +++++++++++ frontend/src/pages/Images.tsx | 48 ++- 13 files changed, 1114 insertions(+), 9 deletions(-) create mode 100644 backend/models/image_scan.py create mode 100644 backend/services/scan_service.py create mode 100644 backend/tests/test_scan_service.py create mode 100644 frontend/src/components/stacks/ScanFindings.tsx diff --git a/README.md b/README.md index f23f177..399eb69 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,36 @@ as intuitive as Dockge, as capable as Portainer for Compose workflows. > (Auto-update) + Phase 23 (Secrets & configs) + Phase 24 (Design System v2) > complete. +## Upgrading to 0.59.0 — CVE scanning + +The Images page can tell you what is wrong with the images you are running. +**Scan for CVEs** scans every image a stack uses; each row then shows critical +and high counts, the total, and — the number that actually matters — how many +findings **have a fix available**. A base image with 300 unfixable low-severity +CVEs is not a task; three fixable ones are. Clicking a row opens the full list +with package, installed version, fixed version and a link to the advisory. + +**The scanner is a container, not a dependency.** Trivy runs as a throwaway +container (the same trick backups use for volume contents), so a 100 MB security +tool and its weekly-changing vulnerability database stay out of StackPilot's own +image, and the scanner updates itself by pulling a newer tag. It gets the Docker +socket **read-only** to inspect images the daemon already has, and a named +volume (`stackpilot-trivy-cache`) so its ~50 MB database downloads once instead +of per scan. The first scan is therefore slow; the rest are not. + +Set `SCANNER_IMAGE` to pin a version (default `aquasec/trivy:latest`). + +**A scan that fails never looks clean.** If the scanner cannot run, the row +keeps the counts from the last good scan and shows the error — silence would +otherwise read as "no vulnerabilities", which is the one failure mode a security +feature must not have. Results also go stale honestly: pull an image and its +scan is marked *stale* rather than presenting yesterday's numbers for today's +image. + +Reading results works with the read-only role and a read-only API token, so a +monitoring script can poll `GET /api/images/scans`. Running a scan is admin-only +— it spends real CPU and spawns a container. + ## Upgrading to 0.58.0 — deploy stacks from Git A stack can now be backed by a Git repository. **Stack detail → Git**: point it @@ -426,6 +456,11 @@ it is what your saved destination credentials are encrypted with. compose** converter. - **Dashboard** — system resource bar, stack grid with quick actions, and a recent-activity audit feed. +- **CVE scanning** — Trivy runs as a throwaway container against the images your + stacks use; the Images page shows critical/high counts, how many findings are + fixable, and the full advisory list per image. A failed scan reports the + failure rather than reading as "clean", and a scan goes stale when the image + is pulled again. - **GitOps** — a stack can be deployed from a Git repository (branch and subdirectory selectable, HTTPS token or SSH key for private repos), synced manually, on a poll interval or from a push webhook, with optional automatic @@ -986,6 +1021,16 @@ GET /api/dashboard/funnel[?refresh=true] (stack-health funnel, 30s TTL cache) GET /api/dashboard/summary (containers, uptime series, ops activity) ``` +### CVE scanning endpoints + +``` +GET /api/images/scans (cached results; read-only role may read) +GET /api/images/scan?image=… (one image, with its findings) +POST /api/images/scan ({"image": "…"}, admin — runs the scanner) +POST /api/images/scan-all (every image a stack uses, serial, admin) +GET /api/images/scan-status (progress of a running sweep) +``` + ### GitOps endpoints ``` diff --git a/backend/config.py b/backend/config.py index 3afa2cd..04567bf 100644 --- a/backend/config.py +++ b/backend/config.py @@ -38,6 +38,10 @@ class Settings(BaseSettings): # Throwaway image used to read/write named-volume contents during backup. BACKUP_HELPER_IMAGE: str = "alpine:latest" + # Vulnerability scanner, run as a throwaway container (services/scan_service). + # Pinning a tag here is the supported way to hold a scanner version still. + SCANNER_IMAGE: str = "aquasec/trivy:latest" + # Host browser sandbox roots. Deliberately does NOT contain "/": that entry # makes _is_allowed() wave through every path, i.e. it switches the sandbox # off. Add it back explicitly if you really want the whole filesystem. diff --git a/backend/models/__init__.py b/backend/models/__init__.py index 8665f4d..6a4dbdc 100644 --- a/backend/models/__init__.py +++ b/backend/models/__init__.py @@ -5,6 +5,7 @@ from models.auto_update import AutoUpdate from models.backup_destination import BackupDestination from models.backup_schedule import BackupSchedule from models.git_source import GitSource +from models.image_scan import ImageScan from models.registry import Registry from models.runtime_state import ImageStatus, LoginAttempt, StackLock from models.setting import Setting, Webhook @@ -14,5 +15,5 @@ from models.user import User __all__ = [ "User", "Stack", "AuditLog", "Setting", "Webhook", "BackupDestination", "BackupSchedule", "AutoUpdate", - "StackLock", "ImageStatus", "LoginAttempt", "Registry", "ApiToken", "GitSource", + "StackLock", "ImageStatus", "LoginAttempt", "Registry", "ApiToken", "GitSource", "ImageScan", ] diff --git a/backend/models/image_scan.py b/backend/models/image_scan.py new file mode 100644 index 0000000..c15de70 --- /dev/null +++ b/backend/models/image_scan.py @@ -0,0 +1,72 @@ +from __future__ import annotations + +from datetime import datetime, timezone +from typing import Optional + +from sqlmodel import Field, SQLModel + + +def _now() -> datetime: + return datetime.now(timezone.utc) + + +#: Worst first. The order is load-bearing: findings are sorted and truncated by +#: it, so what survives a cap is always the part that matters. +SEVERITIES = ["critical", "high", "medium", "low", "unknown"] + + +class ImageScan(SQLModel, table=True): + """The result of scanning one image tag for known vulnerabilities. + + One row per image reference, replaced on each scan. ``digest`` is the local + image's digest when it was scanned: if the image is pulled again the digest + moves and the result is stale, which the UI says rather than quietly + presenting yesterday's counts for today's image. + """ + + id: Optional[int] = Field(default=None, primary_key=True) + image: str = Field(index=True, unique=True) + digest: Optional[str] = None + scanner: str = "trivy" + critical: int = 0 + high: int = 0 + medium: int = 0 + low: int = 0 + unknown: int = 0 + #: How many findings have a fixed version available. The actionable number — + #: a hundred unfixable CVEs are noise, three fixable ones are a to-do. + fixable: int = 0 + #: JSON list of the findings, worst first and capped (see scan_service). + findings: str = "[]" + scanned_at: datetime = Field(default_factory=_now) + duration_ms: int = 0 + error: Optional[str] = None + + +# --- API schemas --- + + +class ScanRequest(SQLModel): + image: str + + +class ScanSummary(SQLModel): + image: str + digest: Optional[str] + scanner: str + critical: int + high: int + medium: int + low: int + unknown: int + fixable: int + total: int + scanned_at: datetime + duration_ms: int + error: Optional[str] + #: True when the image has been pulled since this scan ran. + stale: bool = False + + +class ScanDetail(ScanSummary): + findings: list[dict] = [] diff --git a/backend/routers/images.py b/backend/routers/images.py index deb600c..e2c6b8c 100644 --- a/backend/routers/images.py +++ b/backend/routers/images.py @@ -1,13 +1,16 @@ -"""Image listing + update-check endpoints.""" +"""Image listing, update checks and vulnerability scanning.""" from __future__ import annotations -from fastapi import APIRouter, Depends, Query, Request +import json + +from fastapi import APIRouter, Depends, HTTPException, Query, Request from sqlmodel import Session from auth import get_current_user, require_admin from database import get_session +from models.image_scan import ImageScan, ScanDetail, ScanRequest, ScanSummary from models.user import User -from services import audit_service, image_service, update_service +from services import audit_service, image_service, scan_service, update_service router = APIRouter(prefix="/api/images", tags=["images"]) @@ -45,3 +48,104 @@ def prune( ip=_ip(request), ) return result + + +# --------------------------------------------------------------------------- # +# Vulnerability scanning +# --------------------------------------------------------------------------- # + + +def _summary(row: ImageScan, stale: bool = False) -> ScanSummary: + return ScanSummary( + image=row.image, + digest=row.digest, + scanner=row.scanner, + critical=row.critical, + high=row.high, + medium=row.medium, + low=row.low, + unknown=row.unknown, + fixable=row.fixable, + total=scan_service.total(row), + scanned_at=row.scanned_at, + duration_ms=row.duration_ms, + error=row.error, + stale=stale, + ) + + +@router.get("/scans", response_model=list[ScanSummary]) +def list_scans( + session: Session = Depends(get_session), + _user: User = Depends(get_current_user), +) -> list[ScanSummary]: + """Every cached scan result. Cheap — no scanning happens here.""" + rows = scan_service.all_scans(session).values() + return [_summary(row, scan_service.is_stale(row)) for row in rows] + + +@router.get("/scan", response_model=ScanDetail) +def scan_detail( + image: str = Query(...), + session: Session = Depends(get_session), + _user: User = Depends(get_current_user), +) -> ScanDetail: + row = scan_service.all_scans(session).get(image) + if not row: + raise HTTPException(status_code=404, detail=f"'{image}' has not been scanned") + try: + findings = json.loads(row.findings or "[]") + except json.JSONDecodeError: + findings = [] + return ScanDetail( + **_summary(row, scan_service.is_stale(row)).model_dump(), findings=findings + ) + + +@router.post("/scan", response_model=ScanSummary) +async def scan_image( + body: ScanRequest, + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> ScanSummary: + """Scan one image. Takes a while — the scanner runs as a container.""" + image = (body.image or "").strip() + if not image: + raise HTTPException(status_code=400, detail="An image is required") + row = await scan_service.scan(session, image) + audit_service.record( + session, user=user.username, action="image.scan", target=image, + detail=row.error or f"{scan_service.total(row)} finding(s), {row.fixable} fixable", + ip=_ip(request), + ) + if row.error: + # The row is stored either way, so the UI can show why it failed. + raise HTTPException(status_code=502, detail=row.error) + return _summary(row) + + +@router.post("/scan-all") +async def scan_all( + request: Request, + session: Session = Depends(get_session), + user: User = Depends(require_admin), +) -> dict: + """Scan every image a stack is using. Serial, and one sweep at a time.""" + images = sorted({row["tag"] for row in image_service.list_images() if row["stacks"]}) + try: + result = await scan_service.sweep(session, images) + except scan_service.ScanError as exc: + raise HTTPException(status_code=409, detail=str(exc)) from exc + audit_service.record( + session, user=user.username, action="image.scan-all", target="*", + detail=f"{result['scanned']} scanned, {result['failed']} failed", + ip=_ip(request), + ) + return result + + +@router.get("/scan-status") +def scan_status(_user: User = Depends(get_current_user)) -> dict: + """Progress of a running sweep, so the button can show it.""" + return scan_service.sweep_status() diff --git a/backend/services/scan_service.py b/backend/services/scan_service.py new file mode 100644 index 0000000..6c089e5 --- /dev/null +++ b/backend/services/scan_service.py @@ -0,0 +1,263 @@ +"""Vulnerability scanning for the images stacks actually run. + +Scanning runs Trivy in a throwaway container — the same helper-container trick +backups already use — rather than adding a scanner to StackPilot's own image. +That keeps a 100 MB security tool and its weekly-changing vulnerability database +out of the release, and means the scanner updates itself by pulling a newer tag. + +The container gets the Docker socket read-only, so it inspects images the daemon +already has instead of pulling them again from a registry, and a named volume +for its database, so the ~50 MB download happens once rather than per scan. + +Findings are stored worst-first and capped. A base image with 400 known CVEs is +common and the tail is unreadable; what matters is how many there are, how many +have a fix, and what the worst ones are. +""" +from __future__ import annotations + +import asyncio +import json +import logging +import threading +import time +from datetime import datetime, timezone +from typing import Optional + +from sqlmodel import Session, select + +from config import settings +from docker_client import DockerError, get_client, safe_call +from models.image_scan import SEVERITIES, ImageScan + +logger = logging.getLogger("stackpilot.scan") + +#: Named volume for Trivy's vulnerability database. +CACHE_VOLUME = "stackpilot-trivy-cache" + +#: Kept per image. Enough to see the shape of the problem, not enough to turn a +#: database row into a megabyte of JSON nobody reads. +MAX_FINDINGS = 200 + +SCAN_TIMEOUT = 900.0 + +#: One sweep at a time. Scanning is CPU- and IO-heavy, and two passes over the +#: same images would just fight each other for the daemon. +_sweep_lock = threading.Lock() +_sweeping = {"active": False, "done": 0, "total": 0} + + +class ScanError(Exception): + """The scanner could not run, or produced something unusable.""" + + +# --------------------------------------------------------------------------- # +# Running the scanner +# --------------------------------------------------------------------------- # + + +def _run_trivy(image: str) -> str: + """Run Trivy against a local image and return its JSON report.""" + client = get_client() + socket_path = settings.DOCKER_SOCKET + command = [ + "image", + "--format", "json", + "--quiet", + # Vulnerabilities only: secret and misconfiguration scanning are a + # different feature with a very different false-positive profile. + "--scanners", "vuln", + "--timeout", "10m", + image, + ] + try: + output = safe_call( + client.containers.run, + settings.SCANNER_IMAGE, + command, + volumes={ + socket_path: {"bind": "/var/run/docker.sock", "mode": "ro"}, + CACHE_VOLUME: {"bind": "/root/.cache/trivy", "mode": "rw"}, + }, + remove=True, + stdout=True, + stderr=False, + ) + except DockerError as exc: + raise ScanError(str(exc)) from exc + except Exception as exc: # noqa: BLE001 - docker-py raises ContainerError etc. + raise ScanError(_readable(exc)) from exc + return (output or b"").decode("utf-8", "replace") + + +def _readable(exc: Exception) -> str: + """A scanner failure a person can act on, not a docker-py repr.""" + text = str(exc) + if "No such image" in text or "not found" in text.lower(): + return "The scanner image could not be pulled, or the image no longer exists locally" + return text[:500] + + +def _parse(report: str) -> tuple[dict[str, int], int, list[dict]]: + """Counts by severity, how many are fixable, and the findings themselves.""" + try: + data = json.loads(report or "{}") + except json.JSONDecodeError as exc: + raise ScanError("The scanner returned output that is not JSON") from exc + + counts = {name: 0 for name in SEVERITIES} + fixable = 0 + findings: list[dict] = [] + for result in data.get("Results") or []: + target = result.get("Target") or "" + for vuln in result.get("Vulnerabilities") or []: + severity = str(vuln.get("Severity") or "unknown").lower() + if severity not in counts: + severity = "unknown" + counts[severity] += 1 + fixed = vuln.get("FixedVersion") or "" + if fixed: + fixable += 1 + findings.append( + { + "id": vuln.get("VulnerabilityID"), + "severity": severity, + "package": vuln.get("PkgName"), + "installed": vuln.get("InstalledVersion"), + "fixed": fixed or None, + "title": (vuln.get("Title") or "")[:300] or None, + "url": vuln.get("PrimaryURL"), + "target": target, + } + ) + + findings.sort(key=lambda f: (SEVERITIES.index(f["severity"]), f["id"] or "")) + return counts, fixable, findings[:MAX_FINDINGS] + + +def local_digest(image: str) -> Optional[str]: + """The digest of the image the daemon currently has for this tag.""" + try: + client = get_client() + img = safe_call(client.images.get, image) + except DockerError: + return None + return img.id # the content-addressable local id, which changes on a pull + + +# --------------------------------------------------------------------------- # +# Scanning and storing +# --------------------------------------------------------------------------- # + + +async def scan(session: Session, image: str) -> ImageScan: + """Scan one image and store the result, replacing any previous one.""" + started = time.monotonic() + row = session.exec(select(ImageScan).where(ImageScan.image == image)).first() + if row is None: + row = ImageScan(image=image) + + try: + # docker-py is synchronous and a scan takes minutes; off the event loop + # it goes, or every other request waits behind it. + report = await asyncio.wait_for( + asyncio.to_thread(_run_trivy, image), timeout=SCAN_TIMEOUT + ) + counts, fixable, findings = _parse(report) + except asyncio.TimeoutError: + return _store_error(session, row, image, "The scan timed out", started) + except ScanError as exc: + return _store_error(session, row, image, str(exc), started) + + row.digest = local_digest(image) + row.scanner = "trivy" + for name in SEVERITIES: + setattr(row, name, counts[name]) + row.fixable = fixable + row.findings = json.dumps(findings) + row.scanned_at = datetime.now(timezone.utc) + row.duration_ms = int((time.monotonic() - started) * 1000) + row.error = None + session.add(row) + session.commit() + session.refresh(row) + return row + + +def _store_error( + session: Session, row: ImageScan, image: str, message: str, started: float +) -> ImageScan: + """Record a failed scan, keeping whatever counts the last good one found. + + A scanner that cannot run must not silently turn into "no vulnerabilities". + """ + row.image = image + row.error = message[:500] + row.scanned_at = datetime.now(timezone.utc) + row.duration_ms = int((time.monotonic() - started) * 1000) + session.add(row) + session.commit() + session.refresh(row) + return row + + +def total(row: ImageScan) -> int: + return sum(getattr(row, name) for name in SEVERITIES) + + +def is_stale(row: ImageScan) -> bool: + """Has the image been pulled since it was scanned?""" + if not row.digest or row.error: + return False + current = local_digest(row.image) + return bool(current and current != row.digest) + + +def all_scans(session: Session) -> dict[str, ImageScan]: + return {row.image: row for row in session.exec(select(ImageScan)).all()} + + +def forget(session: Session, images: set[str]) -> int: + """Drop scans for images that no longer exist. Returns how many went.""" + removed = 0 + for row in session.exec(select(ImageScan)).all(): + if row.image not in images: + session.delete(row) + removed += 1 + if removed: + session.commit() + return removed + + +# --------------------------------------------------------------------------- # +# Scanning everything +# --------------------------------------------------------------------------- # + + +def sweep_status() -> dict: + return dict(_sweeping) + + +async def sweep(session: Session, images: list[str]) -> dict: + """Scan a list of images one after another. + + Deliberately serial: each scan is already CPU-hungry, and running several at + once on a homelab box would starve the very containers being protected. + """ + if not _sweep_lock.acquire(blocking=False): + raise ScanError("A scan is already running") + _sweeping.update({"active": True, "done": 0, "total": len(images)}) + failed = 0 + try: + for image in images: + try: + row = await scan(session, image) + if row.error: + failed += 1 + except Exception as exc: # noqa: BLE001 - one image must not stop the sweep + logger.warning("Scan failed for %s: %s", image, exc) + failed += 1 + _sweeping["done"] += 1 + finally: + _sweeping.update({"active": False}) + _sweep_lock.release() + return {"scanned": len(images), "failed": failed} diff --git a/backend/tests/test_route_authorization.py b/backend/tests/test_route_authorization.py index 205cfda..252b593 100644 --- a/backend/tests/test_route_authorization.py +++ b/backend/tests/test_route_authorization.py @@ -92,6 +92,15 @@ USER_READABLE = { "GET /api/containers/{container_id}", "GET /api/images", "GET /api/images/updates", + # Vulnerability scan results: CVE ids and package versions for images this + # host runs. Not a credential, and the read-only role can already see every + # image tag and compose file, so the package list adds nothing it could not + # work out. Reading is also the point — a monitoring script wants a + # read-only token. Running a scan stays admin-only, because it spends real + # CPU and spawns a container. + "GET /api/images/scans", + "GET /api/images/scan", + "GET /api/images/scan-status", "GET /api/networks", "GET /api/networks/{network_id}", "GET /api/networks/{network_id}/containers", diff --git a/backend/tests/test_scan_service.py b/backend/tests/test_scan_service.py new file mode 100644 index 0000000..94b4ef8 --- /dev/null +++ b/backend/tests/test_scan_service.py @@ -0,0 +1,316 @@ +"""Vulnerability scanning. + +The scanner itself is a container, so what is testable here is everything +around it: that a real Trivy report is read correctly, that the numbers people +act on (fixable, worst-first) are right, and — most importantly — that a +scanner which fails to run never looks like a clean bill of health. + +The report below is shaped like Trivy's actual JSON output, trimmed. +""" +from __future__ import annotations + +import asyncio +import json + +import pytest +from sqlmodel import Session, delete, select + +REPORT = json.dumps( + { + "SchemaVersion": 2, + "ArtifactName": "nginx:alpine", + "Results": [ + { + "Target": "nginx:alpine (alpine 3.20.3)", + "Class": "os-pkgs", + "Type": "alpine", + "Vulnerabilities": [ + { + "VulnerabilityID": "CVE-2024-9143", + "PkgName": "libcrypto3", + "InstalledVersion": "3.3.2-r0", + "FixedVersion": "3.3.2-r1", + "Severity": "LOW", + "Title": "openssl: Low-level invalid GF(2^m) parameters", + "PrimaryURL": "https://avd.aquasec.com/nvd/cve-2024-9143", + }, + { + "VulnerabilityID": "CVE-2025-1111", + "PkgName": "busybox", + "InstalledVersion": "1.36.1-r29", + "FixedVersion": "", + "Severity": "CRITICAL", + "Title": "busybox: something dire", + "PrimaryURL": "https://example.test/cve-2025-1111", + }, + { + "VulnerabilityID": "CVE-2025-2222", + "PkgName": "zlib", + "InstalledVersion": "1.3.1-r1", + "FixedVersion": "1.3.1-r2", + "Severity": "HIGH", + "PrimaryURL": "https://example.test/cve-2025-2222", + }, + { + "VulnerabilityID": "CVE-2025-3333", + "PkgName": "mystery", + "InstalledVersion": "1.0", + "Severity": "SOMETHING-ODD", + }, + ], + }, + {"Target": "app/package-lock.json", "Class": "lang-pkgs"}, + ], + } +) + + +@pytest.fixture +def svc(db): + from services import scan_service + + return scan_service + + +@pytest.fixture(autouse=True) +def clean_scans(db): + from database import engine + from models.image_scan import ImageScan + + def wipe(): + with Session(engine) as session: + session.exec(delete(ImageScan)) + session.commit() + + wipe() + yield + wipe() + + +@pytest.fixture +def session(db): + from database import engine + + with Session(engine) as s: + yield s + + +def _scan(svc, session, image="nginx:alpine", report=REPORT, digest="sha256:aaa"): + """Run a scan with the container replaced by a canned report.""" + import pytest as _pytest + + monkeypatch = _pytest.MonkeyPatch() + monkeypatch.setattr(svc, "_run_trivy", lambda img: report) + monkeypatch.setattr(svc, "local_digest", lambda img: digest) + try: + return asyncio.run(svc.scan(session, image)) + finally: + monkeypatch.undo() + + +# --------------------------------------------------------------------------- # +# Reading the report +# --------------------------------------------------------------------------- # + + +def test_severities_are_counted(svc, session): + row = _scan(svc, session) + assert (row.critical, row.high, row.low) == (1, 1, 1) + assert svc.total(row) == 4 + + +def test_an_unrecognised_severity_becomes_unknown(svc, session): + """A scanner that invents a severity must not be silently dropped.""" + row = _scan(svc, session) + assert row.unknown == 1 + + +def test_fixable_counts_only_what_can_be_fixed(svc, session): + """The actionable number: 400 unfixable CVEs are noise, 2 fixable are a to-do.""" + row = _scan(svc, session) + assert row.fixable == 2 + + +def test_findings_are_stored_worst_first(svc, session): + row = _scan(svc, session) + findings = json.loads(row.findings) + assert [f["severity"] for f in findings] == ["critical", "high", "low", "unknown"] + assert findings[0]["id"] == "CVE-2025-1111" + assert findings[0]["fixed"] is None + assert findings[1]["fixed"] == "1.3.1-r2" + + +def test_a_result_with_no_vulnerabilities_is_handled(svc, session): + row = _scan(svc, session, report=json.dumps({"Results": [{"Target": "x"}]})) + assert svc.total(row) == 0 and row.error is None + + +def test_an_empty_report_is_a_clean_result_not_a_crash(svc, session): + row = _scan(svc, session, report="{}") + assert svc.total(row) == 0 and row.error is None + + +def test_findings_are_capped(svc, session, monkeypatch): + many = { + "Results": [ + { + "Target": "t", + "Vulnerabilities": [ + {"VulnerabilityID": f"CVE-{i}", "Severity": "LOW", "PkgName": "p"} + for i in range(500) + ], + } + ] + } + row = _scan(svc, session, report=json.dumps(many)) + # Every one is counted; only the list is trimmed. + assert row.low == 500 + assert len(json.loads(row.findings)) == svc.MAX_FINDINGS + + +# --------------------------------------------------------------------------- # +# When the scanner cannot run +# --------------------------------------------------------------------------- # + + +def test_a_scanner_failure_is_recorded_not_reported_as_clean(svc, session, monkeypatch): + """The dangerous failure: "no findings" must never mean "could not scan".""" + + def explode(image): + raise svc.ScanError("docker: no such image aquasec/trivy:latest") + + monkeypatch.setattr(svc, "_run_trivy", explode) + row = asyncio.run(svc.scan(session, "nginx:alpine")) + + assert row.error and "trivy" in row.error + assert svc.total(row) == 0 + + +def test_a_failed_rescan_keeps_the_previous_counts(svc, session, monkeypatch): + """Otherwise a transient failure silently zeroes a real result.""" + first = _scan(svc, session) + assert first.critical == 1 + + def explode(image): + raise svc.ScanError("daemon unreachable") + + monkeypatch.setattr(svc, "_run_trivy", explode) + row = asyncio.run(svc.scan(session, "nginx:alpine")) + assert row.error == "daemon unreachable" + assert row.critical == 1 + + +def test_unparseable_output_is_an_error(svc, session): + row = _scan(svc, session, report="not json at all") + assert row.error and "JSON" in row.error + + +# --------------------------------------------------------------------------- # +# Staleness +# --------------------------------------------------------------------------- # + + +def test_a_scan_is_stale_once_the_image_is_pulled_again(svc, session, monkeypatch): + _scan(svc, session, digest="sha256:old") + from models.image_scan import ImageScan + + row = session.exec(select(ImageScan)).one() + monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:old") + assert svc.is_stale(row) is False + monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:new") + assert svc.is_stale(row) is True + + +def test_a_failed_scan_is_not_called_stale(svc, session, monkeypatch): + """It has no result to be stale; the error is what needs saying.""" + from models.image_scan import ImageScan + + def explode(image): + raise svc.ScanError("nope") + + monkeypatch.setattr(svc, "_run_trivy", explode) + asyncio.run(svc.scan(session, "nginx:alpine")) + row = session.exec(select(ImageScan)).one() + assert svc.is_stale(row) is False + + +# --------------------------------------------------------------------------- # +# Housekeeping +# --------------------------------------------------------------------------- # + + +def test_rescanning_replaces_rather_than_accumulates(svc, session): + _scan(svc, session) + _scan(svc, session) + from models.image_scan import ImageScan + + assert len(session.exec(select(ImageScan)).all()) == 1 + + +def test_scans_for_images_that_are_gone_can_be_dropped(svc, session): + _scan(svc, session, image="nginx:alpine") + _scan(svc, session, image="redis:7") + assert svc.forget(session, {"nginx:alpine"}) == 1 + assert set(svc.all_scans(session)) == {"nginx:alpine"} + + +def test_two_sweeps_cannot_run_at_once(svc, session, monkeypatch): + monkeypatch.setattr(svc, "_run_trivy", lambda img: REPORT) + monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:a") + + async def both(): + first = asyncio.create_task(svc.sweep(session, ["a:1", "b:1"])) + await asyncio.sleep(0) + with pytest.raises(svc.ScanError): + await svc.sweep(session, ["c:1"]) + return await first + + result = asyncio.run(both()) + assert result == {"scanned": 2, "failed": 0} + + +def test_a_sweep_survives_one_image_failing(svc, session, monkeypatch): + def sometimes(image): + if image == "bad:1": + raise svc.ScanError("boom") + return REPORT + + monkeypatch.setattr(svc, "_run_trivy", sometimes) + monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:a") + result = asyncio.run(svc.sweep(session, ["good:1", "bad:1", "also-good:1"])) + assert result == {"scanned": 3, "failed": 1} + assert set(svc.all_scans(session)) == {"good:1", "bad:1", "also-good:1"} + + +# --------------------------------------------------------------------------- # +# Through the API +# --------------------------------------------------------------------------- # + + +def test_the_list_and_detail_endpoints(as_admin, svc, session): + _scan(svc, session) + + listed = as_admin.get("/api/images/scans").json() + assert len(listed) == 1 + assert listed[0]["image"] == "nginx:alpine" + assert listed[0]["fixable"] == 2 + assert listed[0]["total"] == 4 + # The list stays small: findings only come with the detail call. + assert "findings" not in listed[0] + + detail = as_admin.get("/api/images/scan", params={"image": "nginx:alpine"}).json() + assert detail["findings"][0]["id"] == "CVE-2025-1111" + + +def test_asking_about_an_unscanned_image_is_a_404(as_admin): + assert as_admin.get("/api/images/scan", params={"image": "never:1"}).status_code == 404 + + +def test_the_read_only_role_may_read_scans_but_not_run_them(as_user): + assert as_user.get("/api/images/scans").status_code == 200 + assert as_user.post("/api/images/scan", json={"image": "nginx:alpine"}).status_code == 403 + assert as_user.post("/api/images/scan-all").status_code == 403 + + +def test_scanning_without_an_image_is_a_400(as_admin): + assert as_admin.post("/api/images/scan", json={"image": " "}).status_code == 400 diff --git a/backend/version.py b/backend/version.py index d115d87..8470c5e 100644 --- a/backend/version.py +++ b/backend/version.py @@ -1,3 +1,3 @@ """Single source of truth for the StackPilot release version.""" -APP_VERSION = "0.58.0" +APP_VERSION = "0.59.0" diff --git a/frontend/package.json b/frontend/package.json index 3e6b297..252bf59 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "stackpilot-frontend", "private": true, - "version": "0.58.0", + "version": "0.59.0", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/api/images.ts b/frontend/src/api/images.ts index 3c26e65..ab3f261 100644 --- a/frontend/src/api/images.ts +++ b/frontend/src/api/images.ts @@ -18,12 +18,57 @@ export interface ImageRow { update: UpdateStatus | null; } +export interface ScanSummary { + image: string; + digest: string | null; + scanner: string; + critical: number; + high: number; + medium: number; + low: number; + unknown: number; + /** How many findings have a fixed version — the number worth acting on. */ + fixable: number; + total: number; + scanned_at: string; + duration_ms: number; + error: string | null; + /** The image has been pulled since this scan ran. */ + stale: boolean; +} + +export interface Finding { + id: string; + severity: "critical" | "high" | "medium" | "low" | "unknown"; + package: string | null; + installed: string | null; + fixed: string | null; + title: string | null; + url: string | null; + target: string; +} + +export interface ScanDetail extends ScanSummary { + findings: Finding[]; +} + const base = "/api/images"; export const imagesApi = { list: () => api.get(base).then((r) => r.data), updates: () => api.get>(`${base}/updates`).then((r) => r.data), check: () => api.post>(`${base}/check`).then((r) => r.data), + scans: () => api.get(`${base}/scans`).then((r) => r.data), + scanDetail: (image: string) => + api.get(`${base}/scan`, { params: { image } }).then((r) => r.data), + scan: (image: string) => + api.post(`${base}/scan`, { image }).then((r) => r.data), + scanAll: () => + api.post<{ scanned: number; failed: number }>(`${base}/scan-all`).then((r) => r.data), + scanStatus: () => + api + .get<{ active: boolean; done: number; total: number }>(`${base}/scan-status`) + .then((r) => r.data), prune: (allUnused: boolean) => api .post<{ ImagesDeleted: unknown[]; SpaceReclaimed: number }>(`${base}/prune`, null, { diff --git a/frontend/src/components/stacks/ScanFindings.tsx b/frontend/src/components/stacks/ScanFindings.tsx new file mode 100644 index 0000000..be840e3 --- /dev/null +++ b/frontend/src/components/stacks/ScanFindings.tsx @@ -0,0 +1,204 @@ +import { useQuery } from "@tanstack/react-query"; +import { createPortal } from "react-dom"; +import { ShieldCheck, X, ExternalLink } from "lucide-react"; +import { Badge, Button, Spinner } from "@/components/ui"; +import { imagesApi, type Finding, type ScanSummary } from "@/api/images"; +import { cn } from "@/lib/utils"; +import { relativeTime } from "@/lib/utils"; + +/** Worst first, and coloured so the eye lands on what matters. */ +const SEVERITY_TONE: Record = { + critical: "bg-red-100 text-red-700 dark:bg-red-900/40 dark:text-red-300", + high: "bg-orange-100 text-orange-700 dark:bg-orange-900/40 dark:text-orange-300", + medium: "bg-amber-100 text-amber-700 dark:bg-amber-900/40 dark:text-amber-300", + low: "bg-slate-100 text-slate-600 dark:bg-slate-700 dark:text-slate-300", + unknown: "bg-slate-100 text-slate-500 dark:bg-slate-700 dark:text-slate-400", +}; + +/** + * The compact summary that sits in the Images table. + * + * It leads with critical + high and with how many are *fixable*, because that + * is the number you can do something about — a base image with 300 unfixable + * low-severity CVEs is not a task, three fixable ones are. + */ +export function ScanBadge({ + scan, + onOpen, +}: { + scan?: ScanSummary; + onOpen: () => void; +}) { + if (!scan) return not scanned; + if (scan.error) { + return ( + + ); + } + if (scan.total === 0) { + return ( + + ); + } + + return ( + + ); +} + +/** The full finding list for one image. */ +export function ScanDetailDialog({ + image, + onClose, +}: { + image: string; + onClose: () => void; +}) { + const { data, isLoading, error } = useQuery({ + queryKey: ["image-scan", image], + queryFn: () => imagesApi.scanDetail(image), + retry: false, + }); + + return createPortal( +
+
e.stopPropagation()} + > +
+
+

Vulnerabilities

+

{image}

+ {data && !data.error && ( +

+ scanned {relativeTime(data.scanned_at)} with {data.scanner} ·{" "} + {data.fixable} of {data.total} fixable + {data.stale && " · the image has been pulled since"} +

+ )} +
+ +
+ +
+ {isLoading && } + {error && ( +

+ This image has not been scanned yet. +

+ )} + {data?.error && ( +
+ {data.error} +
+ )} + {data && !data.error && data.findings.length === 0 && ( +

+ No known vulnerabilities. +

+ )} + {data && data.findings.length > 0 && ( + + + + + + + + + + + {data.findings.map((f) => ( + + + + + + + ))} + +
SeverityCVEPackageFix
+ + {f.severity} + + + {f.url ? ( + + {f.id} + + ) : ( + f.id + )} + {f.title && ( + + {f.title} + + )} + + {f.package} + {f.installed && ( + {f.installed} + )} + + {f.fixed ? ( + {f.fixed} + ) : ( + no fix yet + )} +
+ )} +
+ +
+ +
+
+
, + document.body + ); +} diff --git a/frontend/src/pages/Images.tsx b/frontend/src/pages/Images.tsx index 20506bd..8443efd 100644 --- a/frontend/src/pages/Images.tsx +++ b/frontend/src/pages/Images.tsx @@ -1,10 +1,11 @@ import { Fragment, useMemo, useState } from "react"; import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; -import { RefreshCw, ArrowUpCircle, CheckCircle2, HelpCircle, Eraser } from "lucide-react"; +import { RefreshCw, ArrowUpCircle, CheckCircle2, HelpCircle, Eraser, ShieldAlert } from "lucide-react"; import { toast } from "sonner"; import { Button, Card, Spinner } from "@/components/ui"; import { ConfirmDialog } from "@/components/ui/ConfirmDialog"; import { imagesApi, type ImageRow } from "@/api/images"; +import { ScanBadge, ScanDetailDialog } from "@/components/stacks/ScanFindings"; import { stacksApi } from "@/api/stacks"; import { groupByStack, type StackGroup } from "@/lib/stackGroups"; import { StackGroupHeader } from "@/components/ui/StackGroupHeader"; @@ -37,6 +38,8 @@ export function Images() { function ImagesSection({ isAdmin }: { isAdmin: boolean }) { const qc = useQueryClient(); const [checking, setChecking] = useState(false); + const [scanning, setScanning] = useState(false); + const [scanFor, setScanFor] = useState(null); const [pruneOpen, setPruneOpen] = useState(false); const [pruneAll, setPruneAll] = useState(false); const { data, isLoading } = useQuery({ @@ -64,6 +67,33 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { [data, stackById] ); + // Cheap and cached; the scanning itself only happens on the buttons below. + const scans = useQuery({ + queryKey: ["image-scans"], + queryFn: imagesApi.scans, + }); + const scanByImage = useMemo( + () => new Map((scans.data ?? []).map((s) => [s.image, s])), + [scans.data] + ); + + const scanAll = async () => { + setScanning(true); + const t = toast.loading("Scanning images… this runs the scanner per image"); + try { + const r = await imagesApi.scanAll(); + await qc.invalidateQueries({ queryKey: ["image-scans"] }); + toast.success( + `Scanned ${r.scanned} image(s)${r.failed ? `, ${r.failed} failed` : ""}`, + { id: t } + ); + } catch (e) { + toast.error(apiErrorMessage(e), { id: t }); + } finally { + setScanning(false); + } + }; + const check = async () => { setChecking(true); const t = toast.loading("Checking for updates…"); @@ -101,6 +131,9 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { + @@ -119,6 +152,7 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { Size Created Status + Vulnerabilities @@ -126,7 +160,7 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { {group.items.map((row) => ( @@ -145,13 +179,19 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { + + setScanFor(row.tag)} + /> + ))} ))} {data?.length === 0 && ( - + No images. @@ -161,6 +201,8 @@ function ImagesSection({ isAdmin }: { isAdmin: boolean }) { )} + {scanFor && setScanFor(null)} />} + {pruneOpen && (