4 Commits

Include renames

Author	SHA1	Message	Date
Will Miao	12930ce78c	feat(sidebar): scan a single folder from the folder context menu (#1108) ... The Refresh menu can scope a scan to one model root, but the folder a user is looking at lives in the sidebar's unified tree, which merges every root into one relative-path namespace. "Scan this folder" therefore addresses the folder, not a root: the backend walks that relative path under every root that holds it, which is also what makes the action safe while another drive is switched off. Backend: * GET /scan accepts `folder=<rel>` (alone or with `roots=`), rejected together with full_rebuild=true like the roots parameter. Validation reuses normalize_relative_folder(), extracted to module scope from ModelMoveService so the folder operations and the scan endpoint reject the same input (absolute paths, drive letters, `..` climbing) instead of each carrying its own copy. * The reconcile summary carries `scope_label` (the folder) for a folder scope, so the result toast names the folder the user clicked instead of the roots it happens to live under; the completed WS payload carries it too. * `folder` is a scope prefix exactly like a root: only that subtree is re-read or pruned, and an unreachable root keeps the entries that fall inside it. Frontend: * The sidebar folder context menu gains "Scan this folder" above "Check for updates in this folder" (they share the refresh divider); the entry is gated by the same supportsFolderManagement flag as the other folder operations. * SidebarManager.scanFolder() resolves the node through the existing _resolveFolderCandidates() before doing anything: a folder no root holds any more explains itself ("no longer exists on disk") instead of scanning nothing, and an unresolvable multi-root node is refused rather than guessed. * PageControls.refreshModels() and BaseModelApi.refreshModels() forward the folder scope, and _showRefreshSummary() prefers scope_label over the walked roots. Verified live on the three-root sandbox with one drive switched off: GET /scan?folder=pack000 walks drive-G and drive-Y, reports scope_label=pack000, keeps drive-Z's 6 entries under that folder (kept_unreachable=6) and leaves all 420 models cached. 3704 passed, 7 skipped; frontend 1495 passed (150 files); vue widgets 96 passed. The 2 new sidebar keys are [TODO: Translate] placeholders pending the feature owner's go-ahead.	2026-10-07 15:15:35 +08:00
Will Miao	470d85cca6	feat(scanner): scan a single root and keep unreachable entries (#1108) ... Refreshing had no way to say "scan only this drive": a user with three external drives had to spin all of them up for every refresh, and switching a drive off made the next refresh treat its whole library as deleted (rows pruned from the memory cache and the SQLite cache, preview_url stripped on the next scroll). Backend (py/services/model_scanner.py, py/config.py): * ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the scope reconcile normally, everything outside is neither re-read nor removed. The folder half is plumbing for the sidebar entry in the next change. * Path-level pruning guard: cached entries under a path this walk could not read are kept and reported instead of removed. Sources: a configured root that is not reachable (drive switched off while LM runs), a directory os.walk failed to enter (permissions / I/O error / Windows junction to an offline drive), and a known first-level symlink whose target is gone (Config.iter_path_mappings()). * The recorded folder list is unioned instead of replaced whenever the scan did not verify every root, so a scoped scan cannot empty the sidebar. * _reconcile_cache returns a summary (added / removed / repaired / scanned_roots / skipped_roots / unavailable_paths / kept_unreachable), exposed as ModelScanner.last_reconcile_summary, returned by BaseModelService.scan_models() and broadcast in the completed WS payload. * _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown leftwards with real parent segments until unique, shared by the walk-progress line and the roots API. * GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with full_rebuild=true); GET /roots gains root_details (label / reachable / cached count) while `roots` stays a plain path list for existing callers. * serve_preview: a 404 no longer clears the cached preview_url when the file's own directory is unreachable - browsing the grid with a drive off used to strip preview references from the persistent cache. Frontend: * Refresh ▾ gains a "Scan one folder" section listing the page's roots with their cached counts; offline roots stay clickable and explain themselves; rows are wired by delegation (new static/js/components/controls/ScanScopeMenu.js). * A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept entries reports "<N> models kept: <paths> not reachable". * registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and an argument-forwarding refreshModels) so a page facade cannot drop them: the first version rendered an empty menu and would have run a full refresh. * createToastElement whitelists toast types, so a wrong `type` argument degrades to the info style instead of rendering an unstyled box. Verified in a sandbox instance with three roots: a scoped scan walks only the requested root (progress roots=0/1, 240 files); a full refresh with one root offline reports kept_unreachable=60 and leaves all 420 models cached; /roots reports the offline root with its cached count. 3699 passed, 7 skipped; frontend 1488 passed (148 files); vue widgets 96 passed.	2026-10-07 15:01:26 +08:00
Will Miao	14e660e876	feat(scanner): report and parallelize the reconcile walk ... A regular Refresh walked every configured root sequentially, so a full 25 TB drive delayed the roots behind it, and the dialog sat on "Checking for changes..." at 0 % for the whole walk with no way to tell it was working. On a cold external drive the walk itself dominates the cost, so the fix is to overlap the drives and to show what the walk is doing. Backend (py/services/model_scanner.py): * Extract the per-root walk into the synchronous _walk_root_for_reconcile() worker and merge its results on the event loop afterwards, in configured root order: which business path wins a file reachable through several roots must not depend on the order the workers happened to finish in. * Group roots by device (_root_device_key: drive letter on Windows, st_dev on POSIX) and run one worker per device. Roots sharing a device stay sequential, so directory claims and the overlap dedup (#871, #1041) keep their configured-order semantics; different devices run in parallel. * Track walk progress per root (_ReconcileWalkTracker), weighted by each root's cached entry count because the real file count is only known once the walk ends. The bar splits into walk (0-50 %) and new-file (50-99 %) phases so it never jumps backwards, and the walk broadcasts files seen, active roots and the ETA counters. * Replace the Windows case-insensitive fallback -- a scan of every cached path per miss, i.e. O(files x cached) -- with a lazily built lower-cased index (_CachedPathLookups, also now guarding the realpath alias index for worker threads), and lift the os.name == "nt" gate into the module-level _CASE_INSENSITIVE_PATHS so the branch is testable off Windows. * Excluded-model membership is a set lookup instead of a list scan. Frontend: * render the walk phase as "Checking for changes... <roots> (N files)" with the ETA, and reset the ETA tracker when the stage changes: the per-file rate of counting files says nothing about processing them. * add common.scanProgress.walkFiles; the other locales keep the sanctioned [TODO: Translate] placeholder until the translation pass. Verified: no-change reconcile over 10k files/500 dirs 101 ms and 10k/5000 dirs 230 ms (was 93/229 ms, within noise); a two-device sandbox walk runs both roots concurrently, names them in the progress messages and finishes with added=15, removed=0; 3665 passed, 7 skipped; frontend 1456 passed, vue widgets 96 passed.	2026-10-06 12:40:57 +08:00
Will Miao	14da8a6f17	feat(ui): show live scan progress and ETA for cache refresh ... Broadcast typed scan_progress messages over /ws/fetch-progress from the manual refresh/rebuild paths of ModelScanner and RecipeScanner, and render percent, processed/total, current file name and an EMA-smoothed ETA in the loading overlay. Hardcoded refresh strings move to i18n (common.scanProgress); WS connection failure falls back to the previous static loading behavior.	2026-09-03 11:38:27 +08:00