menzeljandClaude Opus 5
a1cd14a1cd
The Images page knew what was running and whether it was current. It could not say whether any of it was exploitable, which is the question people actually have about a homelab full of images they pulled once and forgot. Trivy runs as a throwaway container rather than being installed into StackPilot's image, reusing the helper-container pattern backups already use for volume contents. Three reasons: a 100 MB security tool and a vulnerability database that changes weekly have no business in a release artifact, pinning SCANNER_IMAGE is then a real version control, and the scanner updates itself by pulling a newer tag. It gets the socket read-only so it inspects images the daemon already has instead of pulling them again, and a named volume for its database so the ~50 MB download happens once rather than per scan. The number the UI leads with is "fixable", not the total. A base image with 300 unfixable low-severity CVEs is not a task and a page that shows 300 in red teaches people to ignore it; three findings with a fixed version available are something to do this afternoon. Counts are stored per severity, findings are sorted worst-first and capped at 200 — every finding is counted, only the list is trimmed, so the cap can never hide the severity distribution. The failure mode this had to avoid is a security feature that reads as clean when it is broken. A scanner that cannot run stores the error and *keeps the previous counts* rather than resetting to zero, so a transient daemon problem does not silently turn a bad image green. There is a test for exactly that, and another for unparseable output. Staleness is handled the same way: the local image id is recorded with the scan, and pulling the image marks the result stale instead of presenting yesterday's numbers for today's bytes. Sweeps are deliberately serial and singly-locked. Scanning is CPU- and IO-heavy, and running eight at once on a homelab box would starve the very containers the scan is meant to protect. docker-py is synchronous, so the scan itself goes to a thread — otherwise a ten-minute scan blocks every other request on the loop. Reading results is allowed for the read-only role, which the authorization matrix made me justify in writing: CVE ids and package versions for images whose tags and compose files that role can already see, and polling them is the monitoring use case a read-only API token exists for. Running a scan stays admin-only because it spends real CPU. 20 tests against a report shaped like Trivy's real output, covering the counting, the fixable number, worst-first ordering, the cap, both failure paths, staleness, and that two sweeps cannot overlap. Verified end to end through the API as well, including that a failed rescan keeps its previous counts and shows the error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
317 lines
11 KiB
Python
317 lines
11 KiB
Python
"""Vulnerability scanning.
|
|
|
|
The scanner itself is a container, so what is testable here is everything
|
|
around it: that a real Trivy report is read correctly, that the numbers people
|
|
act on (fixable, worst-first) are right, and — most importantly — that a
|
|
scanner which fails to run never looks like a clean bill of health.
|
|
|
|
The report below is shaped like Trivy's actual JSON output, trimmed.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import asyncio
|
|
import json
|
|
|
|
import pytest
|
|
from sqlmodel import Session, delete, select
|
|
|
|
REPORT = json.dumps(
|
|
{
|
|
"SchemaVersion": 2,
|
|
"ArtifactName": "nginx:alpine",
|
|
"Results": [
|
|
{
|
|
"Target": "nginx:alpine (alpine 3.20.3)",
|
|
"Class": "os-pkgs",
|
|
"Type": "alpine",
|
|
"Vulnerabilities": [
|
|
{
|
|
"VulnerabilityID": "CVE-2024-9143",
|
|
"PkgName": "libcrypto3",
|
|
"InstalledVersion": "3.3.2-r0",
|
|
"FixedVersion": "3.3.2-r1",
|
|
"Severity": "LOW",
|
|
"Title": "openssl: Low-level invalid GF(2^m) parameters",
|
|
"PrimaryURL": "https://avd.aquasec.com/nvd/cve-2024-9143",
|
|
},
|
|
{
|
|
"VulnerabilityID": "CVE-2025-1111",
|
|
"PkgName": "busybox",
|
|
"InstalledVersion": "1.36.1-r29",
|
|
"FixedVersion": "",
|
|
"Severity": "CRITICAL",
|
|
"Title": "busybox: something dire",
|
|
"PrimaryURL": "https://example.test/cve-2025-1111",
|
|
},
|
|
{
|
|
"VulnerabilityID": "CVE-2025-2222",
|
|
"PkgName": "zlib",
|
|
"InstalledVersion": "1.3.1-r1",
|
|
"FixedVersion": "1.3.1-r2",
|
|
"Severity": "HIGH",
|
|
"PrimaryURL": "https://example.test/cve-2025-2222",
|
|
},
|
|
{
|
|
"VulnerabilityID": "CVE-2025-3333",
|
|
"PkgName": "mystery",
|
|
"InstalledVersion": "1.0",
|
|
"Severity": "SOMETHING-ODD",
|
|
},
|
|
],
|
|
},
|
|
{"Target": "app/package-lock.json", "Class": "lang-pkgs"},
|
|
],
|
|
}
|
|
)
|
|
|
|
|
|
@pytest.fixture
|
|
def svc(db):
|
|
from services import scan_service
|
|
|
|
return scan_service
|
|
|
|
|
|
@pytest.fixture(autouse=True)
|
|
def clean_scans(db):
|
|
from database import engine
|
|
from models.image_scan import ImageScan
|
|
|
|
def wipe():
|
|
with Session(engine) as session:
|
|
session.exec(delete(ImageScan))
|
|
session.commit()
|
|
|
|
wipe()
|
|
yield
|
|
wipe()
|
|
|
|
|
|
@pytest.fixture
|
|
def session(db):
|
|
from database import engine
|
|
|
|
with Session(engine) as s:
|
|
yield s
|
|
|
|
|
|
def _scan(svc, session, image="nginx:alpine", report=REPORT, digest="sha256:aaa"):
|
|
"""Run a scan with the container replaced by a canned report."""
|
|
import pytest as _pytest
|
|
|
|
monkeypatch = _pytest.MonkeyPatch()
|
|
monkeypatch.setattr(svc, "_run_trivy", lambda img: report)
|
|
monkeypatch.setattr(svc, "local_digest", lambda img: digest)
|
|
try:
|
|
return asyncio.run(svc.scan(session, image))
|
|
finally:
|
|
monkeypatch.undo()
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Reading the report
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_severities_are_counted(svc, session):
|
|
row = _scan(svc, session)
|
|
assert (row.critical, row.high, row.low) == (1, 1, 1)
|
|
assert svc.total(row) == 4
|
|
|
|
|
|
def test_an_unrecognised_severity_becomes_unknown(svc, session):
|
|
"""A scanner that invents a severity must not be silently dropped."""
|
|
row = _scan(svc, session)
|
|
assert row.unknown == 1
|
|
|
|
|
|
def test_fixable_counts_only_what_can_be_fixed(svc, session):
|
|
"""The actionable number: 400 unfixable CVEs are noise, 2 fixable are a to-do."""
|
|
row = _scan(svc, session)
|
|
assert row.fixable == 2
|
|
|
|
|
|
def test_findings_are_stored_worst_first(svc, session):
|
|
row = _scan(svc, session)
|
|
findings = json.loads(row.findings)
|
|
assert [f["severity"] for f in findings] == ["critical", "high", "low", "unknown"]
|
|
assert findings[0]["id"] == "CVE-2025-1111"
|
|
assert findings[0]["fixed"] is None
|
|
assert findings[1]["fixed"] == "1.3.1-r2"
|
|
|
|
|
|
def test_a_result_with_no_vulnerabilities_is_handled(svc, session):
|
|
row = _scan(svc, session, report=json.dumps({"Results": [{"Target": "x"}]}))
|
|
assert svc.total(row) == 0 and row.error is None
|
|
|
|
|
|
def test_an_empty_report_is_a_clean_result_not_a_crash(svc, session):
|
|
row = _scan(svc, session, report="{}")
|
|
assert svc.total(row) == 0 and row.error is None
|
|
|
|
|
|
def test_findings_are_capped(svc, session, monkeypatch):
|
|
many = {
|
|
"Results": [
|
|
{
|
|
"Target": "t",
|
|
"Vulnerabilities": [
|
|
{"VulnerabilityID": f"CVE-{i}", "Severity": "LOW", "PkgName": "p"}
|
|
for i in range(500)
|
|
],
|
|
}
|
|
]
|
|
}
|
|
row = _scan(svc, session, report=json.dumps(many))
|
|
# Every one is counted; only the list is trimmed.
|
|
assert row.low == 500
|
|
assert len(json.loads(row.findings)) == svc.MAX_FINDINGS
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# When the scanner cannot run
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_a_scanner_failure_is_recorded_not_reported_as_clean(svc, session, monkeypatch):
|
|
"""The dangerous failure: "no findings" must never mean "could not scan"."""
|
|
|
|
def explode(image):
|
|
raise svc.ScanError("docker: no such image aquasec/trivy:latest")
|
|
|
|
monkeypatch.setattr(svc, "_run_trivy", explode)
|
|
row = asyncio.run(svc.scan(session, "nginx:alpine"))
|
|
|
|
assert row.error and "trivy" in row.error
|
|
assert svc.total(row) == 0
|
|
|
|
|
|
def test_a_failed_rescan_keeps_the_previous_counts(svc, session, monkeypatch):
|
|
"""Otherwise a transient failure silently zeroes a real result."""
|
|
first = _scan(svc, session)
|
|
assert first.critical == 1
|
|
|
|
def explode(image):
|
|
raise svc.ScanError("daemon unreachable")
|
|
|
|
monkeypatch.setattr(svc, "_run_trivy", explode)
|
|
row = asyncio.run(svc.scan(session, "nginx:alpine"))
|
|
assert row.error == "daemon unreachable"
|
|
assert row.critical == 1
|
|
|
|
|
|
def test_unparseable_output_is_an_error(svc, session):
|
|
row = _scan(svc, session, report="not json at all")
|
|
assert row.error and "JSON" in row.error
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Staleness
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_a_scan_is_stale_once_the_image_is_pulled_again(svc, session, monkeypatch):
|
|
_scan(svc, session, digest="sha256:old")
|
|
from models.image_scan import ImageScan
|
|
|
|
row = session.exec(select(ImageScan)).one()
|
|
monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:old")
|
|
assert svc.is_stale(row) is False
|
|
monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:new")
|
|
assert svc.is_stale(row) is True
|
|
|
|
|
|
def test_a_failed_scan_is_not_called_stale(svc, session, monkeypatch):
|
|
"""It has no result to be stale; the error is what needs saying."""
|
|
from models.image_scan import ImageScan
|
|
|
|
def explode(image):
|
|
raise svc.ScanError("nope")
|
|
|
|
monkeypatch.setattr(svc, "_run_trivy", explode)
|
|
asyncio.run(svc.scan(session, "nginx:alpine"))
|
|
row = session.exec(select(ImageScan)).one()
|
|
assert svc.is_stale(row) is False
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Housekeeping
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_rescanning_replaces_rather_than_accumulates(svc, session):
|
|
_scan(svc, session)
|
|
_scan(svc, session)
|
|
from models.image_scan import ImageScan
|
|
|
|
assert len(session.exec(select(ImageScan)).all()) == 1
|
|
|
|
|
|
def test_scans_for_images_that_are_gone_can_be_dropped(svc, session):
|
|
_scan(svc, session, image="nginx:alpine")
|
|
_scan(svc, session, image="redis:7")
|
|
assert svc.forget(session, {"nginx:alpine"}) == 1
|
|
assert set(svc.all_scans(session)) == {"nginx:alpine"}
|
|
|
|
|
|
def test_two_sweeps_cannot_run_at_once(svc, session, monkeypatch):
|
|
monkeypatch.setattr(svc, "_run_trivy", lambda img: REPORT)
|
|
monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:a")
|
|
|
|
async def both():
|
|
first = asyncio.create_task(svc.sweep(session, ["a:1", "b:1"]))
|
|
await asyncio.sleep(0)
|
|
with pytest.raises(svc.ScanError):
|
|
await svc.sweep(session, ["c:1"])
|
|
return await first
|
|
|
|
result = asyncio.run(both())
|
|
assert result == {"scanned": 2, "failed": 0}
|
|
|
|
|
|
def test_a_sweep_survives_one_image_failing(svc, session, monkeypatch):
|
|
def sometimes(image):
|
|
if image == "bad:1":
|
|
raise svc.ScanError("boom")
|
|
return REPORT
|
|
|
|
monkeypatch.setattr(svc, "_run_trivy", sometimes)
|
|
monkeypatch.setattr(svc, "local_digest", lambda img: "sha256:a")
|
|
result = asyncio.run(svc.sweep(session, ["good:1", "bad:1", "also-good:1"]))
|
|
assert result == {"scanned": 3, "failed": 1}
|
|
assert set(svc.all_scans(session)) == {"good:1", "bad:1", "also-good:1"}
|
|
|
|
|
|
# --------------------------------------------------------------------------- #
|
|
# Through the API
|
|
# --------------------------------------------------------------------------- #
|
|
|
|
|
|
def test_the_list_and_detail_endpoints(as_admin, svc, session):
|
|
_scan(svc, session)
|
|
|
|
listed = as_admin.get("/api/images/scans").json()
|
|
assert len(listed) == 1
|
|
assert listed[0]["image"] == "nginx:alpine"
|
|
assert listed[0]["fixable"] == 2
|
|
assert listed[0]["total"] == 4
|
|
# The list stays small: findings only come with the detail call.
|
|
assert "findings" not in listed[0]
|
|
|
|
detail = as_admin.get("/api/images/scan", params={"image": "nginx:alpine"}).json()
|
|
assert detail["findings"][0]["id"] == "CVE-2025-1111"
|
|
|
|
|
|
def test_asking_about_an_unscanned_image_is_a_404(as_admin):
|
|
assert as_admin.get("/api/images/scan", params={"image": "never:1"}).status_code == 404
|
|
|
|
|
|
def test_the_read_only_role_may_read_scans_but_not_run_them(as_user):
|
|
assert as_user.get("/api/images/scans").status_code == 200
|
|
assert as_user.post("/api/images/scan", json={"image": "nginx:alpine"}).status_code == 403
|
|
assert as_user.post("/api/images/scan-all").status_code == 403
|
|
|
|
|
|
def test_scanning_without_an_image_is_a_400(as_admin):
|
|
assert as_admin.post("/api/images/scan", json={"image": " "}).status_code == 400
|