Refresh the UI from Docker events instead of polling (0.49.0)

F16 — /ws/events was implemented and nothing consumed it, while thirty polling
intervals across the pages asked for state that only changes when Docker does
something. The endpoint was the answer; it just was not usable as it stood, so
this is three fixes and a client, not a wiring job.

The endpoint forwarded the whole firehose. Three exec_* events fire per web
terminal session and top/attach fire whenever anything inspects a container, so
a client invalidating on each would have been noisier than the polling it
replaces. Now the daemon filters by resource type and the handler drops the
actions that say nothing about rendered state — matching on the verb before the
colon, since Docker reports these as "exec_create: /bin/sh".

It never said *what* changed, so there was nothing to decide which caches to
drop. The payload now carries the resource type.

And it leaked its reader thread. Cancelling the executor future does not
interrupt a thread already inside a blocking read; closing the underlying
CancellableStream is what does. Every page load left one behind holding a socket
open. A test asserts the close, because this is invisible until the process has
been up for a week.

Client side, useDockerEvents holds one connection for the session and maps
resource types to query keys. Bursts are coalesced over 300ms — a ten-service
compose up emits dozens of events in a second, and refetching per event would
reintroduce exactly the load being removed. Reconnects back off to 30s, and any
close reconnects including 4401, since the access token is short-lived and gets
refreshed out from under the socket.

Intervals drop from the mechanism to the safety net: 5s becomes 30-60s. Two
deliberately stay fast. Live CPU/memory drifts continuously with no event to
announce it, and that one is served from the 4s server-side cache added in
0.47.0, so it costs one sample per interval regardless of how many tabs are
open. The audit feed polls because its entries come from people, not Docker.

Net effect is both cheaper and faster: no fixed floor of requests per second
against the daemon, and a stack that finishes starting shows up immediately
rather than up to five seconds later.

23 new tests (758 total), driven against a fake daemon. Both nets were checked
by reverting the fix: dropping the filter fails one, dropping the stream close
fails the leak test.

Not covered: the hook itself has no test — there is no frontend test runner yet.
Its contract with the backend is tested; its own behaviour is only typechecked.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dk43rmEeRfYi5wsLDbmfyG

backend/routers/ws.py

@@ -268,12 +268,39 @@ async def ws_update(
         update_service.refresh_stack_local(stack_id)
 
 
+#: Docker event types worth telling the UI about. Filtered daemon-side, so the
+#: bulk of the firehose never crosses the socket.
+_EVENT_TYPES = ["container", "image", "network", "volume"]
+
+#: Container actions that say nothing about state a page renders. exec_* alone
+#: is three events per web-terminal keystroke session, and `top`/`attach` fire
+#: whenever something inspects a container — invalidating queries on those would
+#: make the stream noisier than the polling it replaces.
+_IGNORED_ACTIONS = {
+    "exec_create", "exec_start", "exec_die", "exec_detach",
+    "attach", "top", "resize", "archive-path", "extract-to-dir",
+}
+
+
 @router.websocket("/ws/events")
 async def ws_events(
     websocket: WebSocket,
     token: str | None = Query(default=None),
 ):
-    """Stream global Docker events (decoded subset)."""
+    """Stream Docker events so the UI can refresh on change instead of polling.
+
+    Every page used to poll its own endpoint every few seconds. Almost all of
+    that state only changes when Docker does something, which is exactly what
+    this reports — so the client refreshes on an event and keeps a slow poll as
+    a safety net.
+
+    Payload per event::
+
+        {"type": "event", "resource": "container", "action": "start",
+         "container": "jellyfin", "stack": "jellyfin"}
+
+    ``resource`` is what the client needs to decide which queries to drop.
+    """
     await websocket.accept()
     if not await _authorize(websocket, token):
         return
@@ -281,28 +308,40 @@ async def ws_events(
 
     loop = asyncio.get_event_loop()
     queue: asyncio.Queue = asyncio.Queue()
-    stop = asyncio.Event()
+    stream = None
 
     def reader():
+        """Blocking read of the event stream, handed to the loop thread-safely."""
+        nonlocal stream
         try:
-            client = get_client()
-            for event in client.events(decode=True):
-                if stop.is_set():
-                    break
+            stream = get_client().events(decode=True, filters={"type": _EVENT_TYPES})
+            for event in stream:
                 loop.call_soon_threadsafe(queue.put_nowait, event)
-        except Exception:  # noqa: BLE001
+        except Exception:  # noqa: BLE001 - a closed stream lands here on teardown
             pass
+        finally:
+            loop.call_soon_threadsafe(queue.put_nowait, None)
 
     task = loop.run_in_executor(None, reader)
     try:
+        await websocket.send_text(json.dumps({"type": "ready"}))
         while True:
             event = await queue.get()
+            if event is None:  # reader finished — daemon gone or stream closed
+                await websocket.send_text(
+                    json.dumps({"type": "error", "detail": "Docker event stream ended"})
+                )
+                break
+            action = (event.get("Action") or "").split(":")[0]
+            if action in _IGNORED_ACTIONS:
+                continue
             actor = event.get("Actor", {}) or {}
             attrs = actor.get("Attributes", {}) or {}
             await websocket.send_text(
                 json.dumps(
                     {
                         "type": "event",
+                        "resource": event.get("Type"),
                         "action": event.get("Action"),
                         "container": attrs.get("name"),
                         "stack": attrs.get("com.docker.compose.project"),
@@ -311,8 +350,17 @@ async def ws_events(
             )
     except WebSocketDisconnect:
         pass
+    except Exception as exc:  # noqa: BLE001
+        with contextlib.suppress(Exception):
+            await websocket.send_text(json.dumps({"type": "error", "detail": str(exc)}))
     finally:
-        stop.set()
+        # Closing the stream is what actually unblocks the reader thread.
+        # Cancelling the executor future does not: a thread already inside a
+        # blocking read keeps that read, and the thread leaks for the life of
+        # the process — once per page load, with the socket held open.
+        if stream is not None:
+            with contextlib.suppress(Exception):
+                stream.close()
         task.cancel()
 
 

backend/tests/test_docker_events.py

@@ -0,0 +1,223 @@
+"""The Docker event stream behind the UI's refresh (F16).
+
+Every page used to poll its own endpoint every few seconds for state that only
+changes when Docker does something — which the daemon already announces. The
+endpoint existed but nothing consumed it, and it was not usable as it stood:
+
+* it forwarded the whole firehose, including three ``exec_*`` events per web
+  terminal session, so a client would have invalidated its cache more often than
+  the polling it replaces;
+* it never told the client *what* changed, so there was nothing to decide which
+  queries to drop;
+* and it leaked its reader thread — cancelling the executor future does not
+  interrupt a thread already inside a blocking read, so every page load left one
+  behind holding a socket open.
+
+These tests pin the shape of the payload, the filtering, and the teardown.
+"""
+from __future__ import annotations
+
+import pytest
+
+
+@pytest.fixture
+def ws_module():
+    from routers import ws
+
+    return ws
+
+
+# --------------------------------------------------------------------------- #
+# Filtering
+# --------------------------------------------------------------------------- #
+
+
+def test_only_ui_relevant_resource_types_are_requested(ws_module):
+    """Filtered daemon-side, so the bulk never crosses the socket."""
+    assert set(ws_module._EVENT_TYPES) == {"container", "image", "network", "volume"}
+
+
+@pytest.mark.parametrize(
+    "action", ["exec_create", "exec_start", "exec_die", "attach", "top", "resize"]
+)
+def test_noise_actions_are_ignored(ws_module, action):
+    assert action in ws_module._IGNORED_ACTIONS
+
+
+@pytest.mark.parametrize(
+    "action",
+    ["create", "start", "stop", "die", "destroy", "restart", "health_status", "pull"],
+)
+def test_state_changing_actions_are_not_ignored(ws_module, action):
+    assert action not in ws_module._IGNORED_ACTIONS
+
+
+def test_parameterised_actions_still_match_the_ignore_list(ws_module):
+    """Docker reports these as ``exec_create: /bin/sh``, not a bare verb.
+
+    Matching the whole string would let every one of them through.
+    """
+    raw = "exec_create: /bin/sh -c 'ls'"
+    assert raw.split(":")[0] in ws_module._IGNORED_ACTIONS
+
+
+# --------------------------------------------------------------------------- #
+# The stream, driven against a fake daemon
+# --------------------------------------------------------------------------- #
+
+
+class _FakeStream:
+    """Stands in for docker-py's CancellableStream.
+
+    ``close()`` is what actually unblocks the reader thread; this records that
+    it was called, which is the whole point of the teardown test.
+    """
+
+    def __init__(self, events):
+        self._events = list(events)
+        self.closed = False
+
+    def __iter__(self):
+        yield from self._events
+
+    def close(self):
+        self.closed = True
+
+
+@pytest.fixture
+def fake_docker(monkeypatch):
+    """Point the endpoint at a canned event stream instead of a daemon."""
+    import docker_client
+
+    holder = {}
+
+    def make(events):
+        stream = _FakeStream(events)
+        holder["stream"] = stream
+
+        class Client:
+            def events(self, decode=True, filters=None):
+                holder["filters"] = filters
+                return stream
+
+        monkeypatch.setattr(docker_client, "get_client", lambda: Client())
+        return holder
+
+    return make
+
+
+def _event(resource, action, name=None, project=None):
+    return {
+        "Type": resource,
+        "Action": action,
+        "Actor": {"Attributes": {"name": name, "com.docker.compose.project": project}},
+    }
+
+
+def _collect(client, token, expected):
+    """Read frames off the socket until ``expected`` events have arrived."""
+    import json
+
+    frames = []
+    with client.websocket_connect(f"/ws/events?token={token}") as socket:
+        ready = json.loads(socket.receive_text())
+        assert ready == {"type": "ready"}, "clients rely on this to know it is live"
+        for _ in range(expected):
+            frames.append(json.loads(socket.receive_text()))
+    return frames
+
+
+def test_an_event_carries_what_the_client_needs(client, admin_token, fake_docker):
+    fake_docker([_event("container", "start", "jellyfin", "jellyfin")])
+
+    (frame,) = _collect(client, admin_token, 1)
+    assert frame == {
+        "type": "event",
+        "resource": "container",
+        "action": "start",
+        "container": "jellyfin",
+        "stack": "jellyfin",
+    }
+
+
+def test_noise_is_dropped_before_it_reaches_the_client(client, admin_token, fake_docker):
+    fake_docker(
+        [
+            _event("container", "exec_create: /bin/sh", "jellyfin"),
+            _event("container", "exec_start: /bin/sh", "jellyfin"),
+            _event("container", "exec_die", "jellyfin"),
+            _event("container", "die", "jellyfin", "jellyfin"),
+        ]
+    )
+
+    (frame,) = _collect(client, admin_token, 1)
+    assert frame["action"] == "die", "only the state change should survive"
+
+
+def test_the_daemon_is_asked_to_filter(client, admin_token, fake_docker):
+    holder = fake_docker([_event("container", "start", "x")])
+    _collect(client, admin_token, 1)
+    assert holder["filters"] == {"type": ["container", "image", "network", "volume"]}
+
+
+def test_the_stream_is_closed_on_disconnect(client, admin_token, fake_docker):
+    """Otherwise the reader thread stays blocked for the life of the process.
+
+    Cancelling the executor future does not interrupt a thread already inside a
+    blocking read — closing the underlying stream is what does.
+    """
+    holder = fake_docker([_event("container", "start", "x")])
+    _collect(client, admin_token, 1)
+    assert holder["stream"].closed, "a disconnected client must not leak its reader"
+
+
+def test_an_ending_stream_is_reported_not_silently_dropped(
+    client, admin_token, fake_docker
+):
+    """A daemon restart ends the iterator; the client should hear about it so it
+    reconnects rather than sitting on a dead socket believing it is live."""
+    import json
+
+    fake_docker([])  # iterator finishes immediately
+    with client.websocket_connect(f"/ws/events?token={admin_token}") as socket:
+        assert json.loads(socket.receive_text())["type"] == "ready"
+        frame = json.loads(socket.receive_text())
+    assert frame["type"] == "error"
+
+
+def test_events_require_a_token(client):
+    from starlette.websockets import WebSocketDisconnect
+
+    with pytest.raises(WebSocketDisconnect) as excinfo:
+        with client.websocket_connect("/ws/events") as socket:
+            socket.receive_text()
+    assert excinfo.value.code == 4401
+
+
+def test_a_revoked_session_is_refused(client, user_token, db):
+    """The socket resolves against the live user, like every other endpoint."""
+    from sqlmodel import Session, select
+    from starlette.websockets import WebSocketDisconnect
+
+    import auth as auth_mod
+    from database import engine
+    from models.user import User
+
+    with Session(engine) as session:
+        user = session.exec(select(User).where(User.username == "test-user")).one()
+        auth_mod.bump_token_version(user)
+        session.add(user)
+        session.commit()
+
+    try:
+        with pytest.raises(WebSocketDisconnect) as excinfo:
+            with client.websocket_connect(f"/ws/events?token={user_token}") as socket:
+                socket.receive_text()
+        assert excinfo.value.code == 4401
+    finally:
+        # Leave the shared fixture user usable for the rest of the session.
+        with Session(engine) as session:
+            user = session.exec(select(User).where(User.username == "test-user")).one()
+            user.token_version = 1
+            session.add(user)
+            session.commit()

backend/version.py

@@ -1,3 +1,3 @@
 """Single source of truth for the StackPilot release version."""
 
-APP_VERSION = "0.48.0"
+APP_VERSION = "0.49.0"