Refreshing had no way to say "scan only this drive": a user with three external
drives had to spin all of them up for every refresh, and switching a drive off
made the next refresh treat its whole library as deleted (rows pruned from the
memory cache and the SQLite cache, preview_url stripped on the next scroll).
Backend (py/services/model_scanner.py, py/config.py):
* ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the
scope reconcile normally, everything outside is neither re-read nor removed.
The folder half is plumbing for the sidebar entry in the next change.
* Path-level pruning guard: cached entries under a path this walk could not
read are kept and reported instead of removed. Sources: a configured root that
is not reachable (drive switched off while LM runs), a directory os.walk
failed to enter (permissions / I/O error / Windows junction to an offline
drive), and a known first-level symlink whose target is gone
(Config.iter_path_mappings()).
* The recorded folder list is unioned instead of replaced whenever the scan did
not verify every root, so a scoped scan cannot empty the sidebar.
* _reconcile_cache returns a summary (added / removed / repaired /
scanned_roots / skipped_roots / unavailable_paths / kept_unreachable),
exposed as ModelScanner.last_reconcile_summary, returned by
BaseModelService.scan_models() and broadcast in the completed WS payload.
* _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown
leftwards with real parent segments until unique, shared by the walk-progress
line and the roots API.
* GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with
full_rebuild=true); GET /roots gains root_details (label / reachable / cached
count) while `roots` stays a plain path list for existing callers.
* serve_preview: a 404 no longer clears the cached preview_url when the file's
own directory is unreachable - browsing the grid with a drive off used to
strip preview references from the persistent cache.
Frontend:
* Refresh ▾ gains a "Scan one folder" section listing the page's roots with
their cached counts; offline roots stay clickable and explain themselves; rows
are wired by delegation (new static/js/components/controls/ScanScopeMenu.js).
* A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept
entries reports "<N> models kept: <paths> not reachable".
* registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and
an argument-forwarding refreshModels) so a page facade cannot drop them: the
first version rendered an empty menu and would have run a full refresh.
* createToastElement whitelists toast types, so a wrong `type` argument degrades
to the info style instead of rendering an unstyled box.
Verified in a sandbox instance with three roots: a scoped scan walks only the
requested root (progress roots=0/1, 240 files); a full refresh with one root
offline reports kept_unreachable=60 and leaves all 420 models cached; /roots
reports the offline root with its cached count. 3699 passed, 7 skipped;
frontend 1488 passed (148 files); vue widgets 96 passed.
- Add onerror handler on <img> previews to fallback to no-preview.png
- Fire async cache cleanup when preview file returns 404
- Add ModelCache.clear_preview_by_path() for safe stale-url removal
- Downgrade /api/lm/previews 404 log from warning to debug
Chrome does not cache 206 Partial Content responses for <video> elements
without an explicit Cache-Control header. When VirtualScroller recycles
cards and creates new <video> elements with the same URL, Chrome
re-downloads the full video (several MB each) instead of using the cache.
Verified via Chrome DevTools: same .mp4 URL appears 2-3 times in network
trace as separate requests with no cache hit, each returning 206. With
Cache-Control: max-age=86400, the browser will reuse the cached response
for 24 hours across scroll cycles.
Video preview files are ~3.5MB while image previews are ~50-100KB (due
to WebP optimization), making caching especially impactful for videos.
The previous commit (a19ddc14) restored Linux sendfile but kept the
manual streaming path for Windows via sys.platform guard. A Windows
user reports performance is still worse than v1.0.5.
Switch back to web.FileResponse for all files on all platforms as the
default. The IOCP crash is an edge case (fast scrolling through many
video previews) that affects few users, while the Python chunked I/O
performance penalty affects everyone.
_stream_file() is kept as an unused fallback for a future compat
setting toggle.
- Restrict manual video streaming to Windows only (sys.platform == 'win32');
Linux/macOS now uses kernel sendfile (zero-copy DMA) via aiohttp FileResponse
- Add Cache-Control: public, max-age=86400 to streaming responses so browsers
cache video previews across scroll cycles
- Increase chunk size from 256KB to 1MB to reduce async iteration overhead on
Windows where streaming is still required
aiohttp's FileResponse uses _sendfile_native on Windows (IOCP-based), which crashes with ov.getresult() when the client disconnects mid-transfer. This happens constantly when users scroll through a gallery of animated previews (video files like .mp4/.webm).
Detect video extensions and stream manually via StreamResponse + chunked reads instead, gracefully handling ConnectionResetError. Images continue using FileResponse (small files, sendfile works fine).
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Add `_entry_is_symlink` method to detect symlinks and Windows junctions
- Include first-level symlinks in fingerprint for better cache invalidation
- Re-enable preview path validation for security
- Update tests to verify retargeted symlinks trigger rescan
Temporary workaround for issues #772 and #774 where valid previews
are rejected. Path validation is disabled until proper fix for
preview root path handling is implemented.
Will Miao