feat(scanner): scan a single root and keep unreachable entries (#1108)

Refreshing had no way to say "scan only this drive": a user with three external
drives had to spin all of them up for every refresh, and switching a drive off
made the next refresh treat its whole library as deleted (rows pruned from the
memory cache and the SQLite cache, preview_url stripped on the next scroll).

Backend (py/services/model_scanner.py, py/config.py):
* ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the
  scope reconcile normally, everything outside is neither re-read nor removed.
  The folder half is plumbing for the sidebar entry in the next change.
* Path-level pruning guard: cached entries under a path this walk could not
  read are kept and reported instead of removed. Sources: a configured root that
  is not reachable (drive switched off while LM runs), a directory os.walk
  failed to enter (permissions / I/O error / Windows junction to an offline
  drive), and a known first-level symlink whose target is gone
  (Config.iter_path_mappings()).
* The recorded folder list is unioned instead of replaced whenever the scan did
  not verify every root, so a scoped scan cannot empty the sidebar.
* _reconcile_cache returns a summary (added / removed / repaired /
  scanned_roots / skipped_roots / unavailable_paths / kept_unreachable),
  exposed as ModelScanner.last_reconcile_summary, returned by
  BaseModelService.scan_models() and broadcast in the completed WS payload.
* _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown
  leftwards with real parent segments until unique, shared by the walk-progress
  line and the roots API.
* GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with
  full_rebuild=true); GET /roots gains root_details (label / reachable / cached
  count) while `roots` stays a plain path list for existing callers.
* serve_preview: a 404 no longer clears the cached preview_url when the file's
  own directory is unreachable - browsing the grid with a drive off used to
  strip preview references from the persistent cache.

Frontend:
* Refresh ▾ gains a "Scan one folder" section listing the page's roots with
  their cached counts; offline roots stay clickable and explain themselves; rows
  are wired by delegation (new static/js/components/controls/ScanScopeMenu.js).
* A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept
  entries reports "<N> models kept: <paths> not reachable".
* registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and
  an argument-forwarding refreshModels) so a page facade cannot drop them: the
  first version rendered an empty menu and would have run a full refresh.
* createToastElement whitelists toast types, so a wrong `type` argument degrades
  to the info style instead of rendering an unstyled box.

Verified in a sandbox instance with three roots: a scoped scan walks only the
requested root (progress roots=0/1, 240 files); a full refresh with one root
offline reports kept_unreachable=60 and leaves all 420 models cached; /roots
reports the offline root with its cached count. 3699 passed, 7 skipped;
frontend 1488 passed (148 files); vue widgets 96 passed.

docs/plans/issue-1108-scoped-scan.md

@@ -0,0 +1,283 @@
+# Plan: Scoped Scan (one root / one folder at a time)
+
+**Issue:** [#1108](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1108) — scan a single
+folder/root instead of the whole library.
+**Status:** **P1 implemented** (2026-10-07, uncommitted working tree). P2 (folder scope, Wave 5)
+is not implemented. Supersedes the earlier draft's per-root *status panel* and persisted
+*unreachable-subtree* state (both dropped, see "Must NOT have").
+
+Implementation notes / deviations from the draft below:
+
+* The scope dataclass is public (`ReconcileScope`) because it crosses the scanner → service →
+  route boundary; the draft called it `_ReconcileScope`.
+* `_reconcile_cache()` now returns a summary dict, exposed as
+  `ModelScanner.last_reconcile_summary` and returned by `BaseModelService.scan_models()`, so the
+  scan HTTP response carries the counts (the WS `completed` payload carries the same fields).
+* Root labels are computed over the **whole configured set** and passed into the walk tracker, so
+  the progress line and the menu never disagree.
+* Discovery: `Config._dedupe_existing_paths()` drops roots that do not exist **at config load**, so
+  a drive that is already off when LM starts is not in `get_model_roots()` at all. Two consequences:
+  it cannot appear in the refresh menu (nothing to select), and its cached entries are kept because
+  they are outside every configured root prefix. The `root_unreachable` path therefore covers the
+  *mid-session* case (drive switched off while LM runs), which is the workflow that motivated the
+  issue; both paths keep the entries, which is what matters.
+* Known limitation to keep in mind: after a scoped scan (or one with unreadable paths) the recorded
+  empty-folder list is unioned instead of replaced, so an empty folder deleted on disk can linger in
+  the sidebar until the next full refresh. Models are unaffected.
+
+## TL;DR (For humans)
+
+**What you'll get:** the Refresh ▾ menu can scan **one model root** (one drive) and leave every
+other root untouched. Refreshing while a drive is switched off no longer deletes that drive's
+models from the cache/DB — they are kept and reported. Browsing the grid while a drive is off no
+longer silently strips `preview_url` from that drive's models.
+
+**Why this approach:** a scan scope is a **path scope**, not a device scope, so it also works for
+layouts where one root contains symlinks to other drives. The safety net collapses into one rule —
+*anything this walk could not read is left untouched* — and that rule is nearly free: the root
+existence check already exists, `os.walk(onerror=...)` only fires on failure, and the cached-entry
+prefix attribution is already computed for the walk-progress weights.
+
+**What it will NOT do:** no active probing of nested symlinks; no persisted "unreachable subtree"
+table; no per-card offline badges (separate change); no root enable/disable switch; the main
+Refresh button keeps meaning "scan everything"; no configuration-convention/documentation push to
+re-layout libraries that use cross-drive symlinks (dropped by request).
+
+**Effort:** Medium (~1.5–2 days for P1 including tests)
+**Risk:** Medium-low — the risk sits in the folder-tree (`all_folders`) merge under a scoped scan
+and in the exact pruning predicate; both get explicit tests.
+
+**Decisions to sanity-check:**
+1. A full Refresh now treats a *configured but unreachable* root as "keep + report" instead of
+   "everything under it was deleted". This is a behaviour change (release-note it).
+2. Keep the near-zero-cost **first-level symlink** reachability check (reuses
+   `config._path_mappings`); nested symlinks stay uncovered.
+3. Root labels become set-aware (`G: loras` / `usb/loras`, auto-deduped by parent segments).
+   This also changes the walk-progress line.
+
+## Scope
+
+### Must have (P1)
+
+- **Scope-aware reconcile**: `_reconcile_cache(scope=...)` where scope is
+  `{roots: [path] | None, folder: rel | None}`. Files inside the scope reconcile normally
+  (add / repair / remove); everything outside is neither re-read nor removed.
+- **Path-level pruning guard**: cached entries under a path this walk could not read are excluded
+  from `missing` and reported. Three sources:
+  - configured root that fails the reachability check (already filtered at
+    `model_scanner.py:1468`),
+  - directories `os.walk` failed to enter (`onerror` collector; covers Windows junctions to an
+    offline drive, permission errors, I/O errors),
+  - first-level symlink mappings whose target is not a directory (`config._path_mappings`).
+- **Folder tree correctness under scope**: `all_folders` becomes
+  `(old outside scope) ∪ (old under unreachable paths) ∪ (discovered)`, so a scoped scan cannot
+  collapse the sidebar tree and an offline root keeps its folders.
+- **Result payload** for the toast: `added`, `removed`, `repaired`, `scanned_roots` (labels),
+  `skipped_roots` (`{path, label, kept}`), `unavailable_paths` (`{path, kept}`, capped),
+  `kept_unreachable`.
+- **API**: `GET /api/lm/{prefix}/scan?full_rebuild=false&roots=<path>` (repeatable);
+  `GET /api/lm/{prefix}/roots` gains `root_details: [{path, label, reachable, models}]` while
+  keeping `roots: [str]` unchanged (other callers depend on it).
+- **Frontend**: Refresh ▾ menu lists the current page's roots (label + model count, offline rows
+  greyed out and non-clickable), main button unchanged, root items disabled while a scan runs;
+  scoped completion toast reports counts and any kept/skipped paths.
+- **Preview fix**: a preview 404 must not clear `preview_url` when the file's parent directory is
+  itself unreachable (`preview_handlers.py:57` + `_cleanup_stale_preview_url`).
+- **Set-aware root labels** (`_root_display_labels(roots)`), shared by the progress broadcasts and
+  `/roots`: last path segment, Windows drive prefix (`G: loras`), deduped by prepending real parent
+  segments (`usb/loras`, `a/models/loras`), `(2)` fallback by sorted path, 40-char cap, full path in
+  the tooltip.
+- i18n keys in `locales/en.json` + `python scripts/sync_translation_keys.py`.
+- pytest + vitest coverage for every item above.
+
+### Must have (P2, optional follow-up)
+
+- `folder=<rel>` parameter: walk `<root>/<rel>` for every reachable root that contains it (the
+  sidebar's unified tree has no root identity, so "this folder" means "this relative path in all
+  roots"), prune only inside that prefix.
+- Sidebar folder context menu entry "Scan this folder" (`templates/components/context_menu.html`
+  `#sidebarFolderContextMenu`, above `check-folder-updates`; update
+  `tests/frontend/regression/sidebarFolderContextMenu.test.js` expectations).
+
+### Must NOT have (guardrails)
+
+- NO per-entry symlink probing during the walk — `DirEntry.is_symlink()` costs a syscall per entry
+  on Windows, the exact cost the reconcile optimisation removed. Nested symlinks stay uncovered.
+- NO persisted `unavailable_paths` table, no DB schema change, no new cache state.
+- NO configuration-convention guidance for cross-drive symlinks (dropped by request).
+- NO per-card/modal offline badges in this plan (separate change; needs the L1 prefix list which
+  P1 produces, so it can build on this later).
+- NO root enable/disable setting, NO status panel in Settings.
+- NO change to `full_rebuild=true` (it still walks everything and replaces the cache);
+  `roots` + `full_rebuild=true` is rejected with 400.
+- NO new dependency, NO change to the extension-facing endpoints.
+- NO `os.path.realpath` for scope/prune routing — business paths only (AGENTS.md rule).
+
+## Todos
+
+### Wave 1 — backend semantics and safety net
+
+1. **Scope plumbing in the scanner.**
+   What to do: add `_ReconcileScope` (dataclass: `roots: Optional[List[str]]`, `folder: Optional[str]`)
+   and `_reconcile_cache(scope=None)`. Resolve the effective root list once: configured roots ∩
+   scope.roots, filtered by reachability; missing ones become `skipped_roots` (never pruned).
+   `_walk_roots_for_reconcile` / `_walk_root_group_sync` / `_walk_root_for_reconcile` take the scope
+   so the walk can start at `<root>/<folder>` while still computing `folder`/`file_path` relative to
+   the **root** (`_process_model_file(path, root_path)` must keep receiving the root).
+   References: `py/services/model_scanner.py:1453` (`_reconcile_cache`), `:1468` (root filter),
+   `:1832` (`_walk_root_group_sync`), `:365` (`_walk_root_for_reconcile`), `:134`
+   (`_count_cached_entries_per_root`), `:126` (`_normalized_root_prefix`).
+   Done when: a scoped reconcile over 2 roots leaves the other root's `raw_data`, hash index and
+   folder list byte-identical.
+
+2. **Pruning predicate + unreachable collection.**
+   What to do: `missing = {p for p in cached_paths - found_paths if in_scope(p) and not
+   under_unreachable(p)}`; collect unreachable prefixes from (a) skipped roots, (b) an `onerror`
+   callback on `os.walk` (record `oserror.filename` as a business path), (c) `config._path_mappings`
+   entries whose target fails `os.path.isdir` (mapping is keyed target→link; protect the **link**
+   prefix). Count the kept entries per prefix for reporting.
+   References: `py/services/model_scanner.py:1679` (`missing_files = cached_paths - found_paths`),
+   `:398` (`os.walk(..., followlinks=True)`), `py/config.py:167` (`_path_mappings`),
+   `:801` (`add_path_mapping`, target→link), `:750-775` (first-level-only symlink scan).
+   Done when: with a root removed from disk (or a directory replaced by an unreadable junction),
+   `raw_data` keeps those entries, `removed` is 0, and the counts land in the result payload.
+
+3. **Folder-tree merge under scope.**
+   What to do: replace the unconditional `sorted_discovered` assignment with the union rule from
+   "Must have"; keep the `folders_changed` comparison and the persist path unchanged. Unscoped
+   scans with nothing unreachable must produce exactly today's list.
+   References: `py/services/model_scanner.py` (walk loop records `discovered_folders`; `model_scanner.py:1738`
+   (`sorted_discovered`) and `:1739` (`folders_changed`) after the dedup pass).
+   Done when: a scoped scan keeps folders outside the scope, an offline root keeps its subtree, and
+   a full scan still drops folders deleted on disk.
+
+4. **Result payload + progress/cancel/complete messages.**
+   What to do: extend the `completed` broadcast with `scanned_roots`, `skipped_roots`,
+   `unavailable_paths`, `kept_unreachable`, `repaired`; the walk tracker already receives only the
+   scoped roots, so the progress line shows that root's label with `roots_total=1`.
+   References: `py/services/model_scanner.py` `_broadcast_scan_progress` (`:576`),
+   `_ReconcileWalkTracker` (`:266`), the completed broadcast at the end of `_reconcile_cache`.
+   Done when: a scoped scan emits `stage=reconcile_scan` with the single scoped label and a
+   `completed` payload carrying the new fields.
+
+5. **API surface.**
+   What to do: `scan_models` accepts repeated `roots` query params (400 on unknown root, 400 when
+   combined with `full_rebuild=true`); `get_model_roots` adds `root_details` (label, reachable,
+   model count via the shared prefix attribution) without touching `roots`.
+   References: `py/routes/handlers/model_handlers.py:1139` (`scan_models`), `:1165`
+   (`get_model_roots`), `py/routes/model_route_registrar.py:61-62`, `static/js/api/apiConfig.js:114`.
+   Done when: `tests/routes/test_lora_routes.py`-style coverage passes for the new params and the
+   backward-compatible `/roots` payload.
+
+6. **Preview 404 must not prune when the parent is unreachable.**
+   What to do: in `serve_preview`, before `_cleanup_stale_preview_url`, check
+   `os.path.isdir(os.path.dirname(resolved))`; if the directory itself is missing/unreachable,
+   return 404 **without** clearing caches. (Centralized sidecar mode puts the preview in the mirror
+   tree, so the check must use the preview file's own parent.)
+   References: `py/routes/handlers/preview_handlers.py:57` (404 branch), `:74-110`
+   (`_cleanup_stale_preview_url` → `clear_preview_by_path` + `_persist_current_cache`),
+   `tests/routes/test_preview_routes.py`.
+   Done when: a test asserts the cache keeps `preview_url` when the parent dir is absent and still
+   clears it when the file was really deleted from a reachable directory.
+
+### Wave 2 — labels
+
+7. **`_root_display_labels(roots)` (set-aware, order-independent).**
+   What to do: replace the single-root `_root_display_label`; compute over the sorted root list;
+   prepend real parent segments until unique; `(2)`/`(3)` fallback; 40-char middle-ellipsis; return
+   `{path: label}`. Use it in `_ReconcileWalkTracker.__init__` (progress) and in the `/roots`
+   handler so both always agree.
+   References: `py/services/model_scanner.py:117` (`_root_display_label`, current single-root
+   version), `:266` (`_ReconcileWalkTracker`).
+   Done when: unit tests cover `usb/loras` + `ssd/loras`, `a/models/loras` + `b/models/loras`,
+   Windows `G:\x\loras` + `H:\y\loras` → `G: loras` / `H: loras`, and the identical-path fallback.
+
+### Wave 3 — frontend
+
+8. **Refresh ▾ scope section.**
+   What to do: add an empty container to `templates/components/controls.html` inside
+   `.dropdown-menu` (`#refreshScopeMenu`) + a section title; on dropdown open, fetch
+   `endpoints.roots`, render one row per root (`data-action="scan-root"`, `data-root="<path>"`,
+   label + model count, `disabled` + "Offline" for unreachable), hide the section on the recipes
+   page (recipes have no roots); wire clicks by **event delegation** on the menu (the existing
+   `full-rebuild` item is bound with a direct `querySelector`, dynamic rows cannot be);
+   invalidate the cached list after a scan finishes; add CSS for the section + `max-height`/
+   `overflow-y` so 4+ roots stay usable.
+   References: `templates/components/controls.html:67-76`, `static/js/components/controls/PageControls.js:131`
+   (`data-action="refresh"`), `:220-260` (`initDropdowns` + `full-rebuild` wiring),
+   `static/js/api/apiConfig.js:118` (`roots` endpoint), `static/js/api/baseModelApi.js:1318`
+   (`fetchModelRoots`).
+   Done when: the menu lists roots with counts, an offline root is visibly disabled, and clicking a
+   row starts a scoped refresh.
+
+9. **Scoped request + toast.**
+   What to do: `refreshModels(fullRebuild, {roots})` appends repeated `roots` params; the
+   `completed` payload drives the toast (`added`/`removed`/`kept_unreachable`/`skipped_roots`);
+   offline rows toast `… is not reachable right now. Nothing was changed.` without a request.
+   References: `static/js/api/baseModelApi.js:514-602` (refreshModels, URL build at `:601`, toast
+   at `:618`), `static/js/components/controls/PageControls.js:485`.
+   Done when: the frontend test asserts the scoped URL and both toast variants.
+
+10. **i18n.**
+    What to do: add `loras.controls.refresh.scopeSection`, `.rootOffline`, `.rootModels`,
+    `toast.api.refreshCompleteScoped`, `toast.api.refreshKeptUnreachable`,
+    `toast.api.scanRootUnreachable` to `locales/en.json`, run
+    `python scripts/sync_translation_keys.py`, then stop (placeholders are the expected state until
+    translations are requested).
+    Done when: `pytest tests/i18n` passes and every locale has the keys.
+
+### Wave 4 — tests and verification
+
+11. **Backend tests** (`tests/services/test_model_scanner.py`, `tests/routes/test_lora_routes.py`,
+    `tests/routes/test_preview_routes.py`):
+    `test_reconcile_scoped_scan_leaves_other_roots_untouched`,
+    `test_reconcile_scoped_scan_removes_deleted_files_in_scope`,
+    `test_reconcile_scoped_scan_preserves_folder_tree_outside_scope`,
+    `test_reconcile_keeps_entries_of_unreachable_root`,
+    `test_reconcile_keeps_entries_under_unreadable_dir`,
+    `test_reconcile_keeps_entries_under_offline_first_level_symlink`,
+    `test_root_display_labels_dedupe_by_parent_segments`,
+    `test_scan_models_accepts_roots_param`, `test_scan_rejects_unknown_root`,
+    `test_scan_rejects_roots_with_full_rebuild`, `test_roots_endpoint_reports_details`,
+    `test_preview_404_keeps_cache_when_parent_dir_missing`.
+
+12. **Frontend tests**: extend `tests/frontend/api/baseModelApi.refresh.test.js` (scoped URL, both
+    toasts) and add `tests/frontend/components/controls/pageControls.scanRoots.test.js`
+    (rows rendered from `/roots`, offline row disabled, delegation wiring).
+
+13. **Full suites + sandbox eyeball**: `pytest -q`, `npx vitest run`, `npm run test:vue`; then a
+    sandboxed standalone instance with two roots and a slowed walk
+    (`LM_WALK_DELAY_S`-style `sitecustomize` hook) to eyeball the scoped progress line, the offline
+    row and the toast — the user verifies by eye.
+
+### Wave 5 — P2 (folder scope), only if approved
+
+14. `folder=<rel>` in the scan endpoint + `_ReconcileScope.folder`, the sidebar menu entry, the
+    context-menu regression test update, and the "folder in several roots" semantics (scan every
+    reachable root that has the relative path; report per root).
+
+## Known limitations (accepted)
+
+- **Nested symlinks** (a symlink *below* a root pointing at another drive) are only covered when
+  `os.walk` fails to enter the target (Windows junctions, permission errors). A broken symlink that
+  `os.walk` classifies as a non-directory (typical on POSIX) is not detected: its entries are pruned
+  as today and come back on the next scan once the target is reachable (sidecars carry the sha256,
+  so no re-hash). Verify the Windows junction behaviour before writing release notes.
+- Only **first-level** symlinks are known to `config` (by design, `config.py:750-775`), so the
+  symlink reachability check inherits that limit.
+- A scoped scan of a folder does not re-read metadata for unchanged files (unchanged behaviour).
+- Empty folders removed on disk can linger in the sidebar after a scoped scan (the folder list is
+  unioned rather than replaced whenever the scan did not verify every root); the next full refresh
+  drops them.
+- `root_details[].reachable` is a live `os.path.exists()` per root, so a root that disappears
+  mid-session shows as offline; a root that was already gone at startup is filtered out by
+  `Config` and therefore absent from the list.
+
+## Verification checklist
+
+- [ ] `pytest -q` green, `npx vitest run` green, `npm run test:vue` green.
+- [ ] `python scripts/sync_translation_keys.py --dry-run` reports no pending changes.
+- [ ] Sandbox: 2 roots, one offline → scoped scan of the online root reports `added`, keeps the
+      offline root's models, and the grid/sidebar still show them.
+- [ ] Sandbox: preview of an offline root's model returns 404 and the DB keeps `preview_url`.
+- [ ] Release note wording agreed for the behaviour change (decision 1).