0.37.2: stream folder zip-downloads to fix 504 on large folders
A big folder hit a 504 Gateway Timeout: the zip was built into a temp
file *before* any response was sent, so for large folders the backend
stayed silent past nginx's proxy_read_timeout.
Now the zip is streamed as it's built, end to end:
- file_service.open_archive() returns (filename, byte iterator); _iter_zip
walks the dir and yields zip bytes incrementally via a small drain
buffer, writing each file in 1 MiB chunks (bounded memory, valid CRCs).
Same hardening as before — only real regular files; FIFOs/sockets/
devices/symlinks skipped without open(); per-file read errors skipped.
- /api/files/download and /agent/files/download return a StreamingResponse
(no temp file). The agent proxy streams the agent response straight
through (agent_service.stream_download), pulling the first chunk eagerly
so an offline/bad-token agent still yields a clean status before 200.
- Files page: streamed downloads have no Content-Length, so the progress
bar shows the running downloaded byte count ("Downloading … 12.3 MB")
instead of a percentage, after the initial "Preparing …".
Verified end to end via TestClient (200, application/zip, valid zip,
2 MiB file intact, FIFO skipped, no hang).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
0dc430bb2a
commit
5ac9f15de4
@@ -32,8 +32,7 @@ from fastapi import (
|
||||
WebSocket,
|
||||
WebSocketDisconnect,
|
||||
)
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
from starlette.background import BackgroundTask
|
||||
from fastapi.responses import FileResponse, JSONResponse, StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
from config import settings
|
||||
@@ -668,10 +667,11 @@ def files_read(path: str = Query(...)) -> dict:
|
||||
@app.get("/agent/files/download", dependencies=[Depends(verify_token)])
|
||||
def files_download(path: str = Query(...)):
|
||||
if _file_guard(file_service.is_dir, path):
|
||||
tmp, filename = _file_guard(file_service.archive_dir, path)
|
||||
return FileResponse(
|
||||
tmp, filename=filename, media_type="application/zip",
|
||||
background=BackgroundTask(os.unlink, tmp),
|
||||
filename, chunks = _file_guard(file_service.open_archive, path)
|
||||
# Stream the zip as it's built (no temp file, starts immediately).
|
||||
return StreamingResponse(
|
||||
chunks, media_type="application/zip",
|
||||
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
|
||||
)
|
||||
real, filename = _file_guard(file_service.resolve_download, path)
|
||||
return FileResponse(real, filename=filename, media_type="application/octet-stream")
|
||||
|
||||
Reference in New Issue
Block a user