feat(scanner): scan a single root and keep unreachable entries (#1108)

Refreshing had no way to say "scan only this drive": a user with three external
drives had to spin all of them up for every refresh, and switching a drive off
made the next refresh treat its whole library as deleted (rows pruned from the
memory cache and the SQLite cache, preview_url stripped on the next scroll).

Backend (py/services/model_scanner.py, py/config.py):
* ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the
  scope reconcile normally, everything outside is neither re-read nor removed.
  The folder half is plumbing for the sidebar entry in the next change.
* Path-level pruning guard: cached entries under a path this walk could not
  read are kept and reported instead of removed. Sources: a configured root that
  is not reachable (drive switched off while LM runs), a directory os.walk
  failed to enter (permissions / I/O error / Windows junction to an offline
  drive), and a known first-level symlink whose target is gone
  (Config.iter_path_mappings()).
* The recorded folder list is unioned instead of replaced whenever the scan did
  not verify every root, so a scoped scan cannot empty the sidebar.
* _reconcile_cache returns a summary (added / removed / repaired /
  scanned_roots / skipped_roots / unavailable_paths / kept_unreachable),
  exposed as ModelScanner.last_reconcile_summary, returned by
  BaseModelService.scan_models() and broadcast in the completed WS payload.
* _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown
  leftwards with real parent segments until unique, shared by the walk-progress
  line and the roots API.
* GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with
  full_rebuild=true); GET /roots gains root_details (label / reachable / cached
  count) while `roots` stays a plain path list for existing callers.
* serve_preview: a 404 no longer clears the cached preview_url when the file's
  own directory is unreachable - browsing the grid with a drive off used to
  strip preview references from the persistent cache.

Frontend:
* Refresh ▾ gains a "Scan one folder" section listing the page's roots with
  their cached counts; offline roots stay clickable and explain themselves; rows
  are wired by delegation (new static/js/components/controls/ScanScopeMenu.js).
* A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept
  entries reports "<N> models kept: <paths> not reachable".
* registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and
  an argument-forwarding refreshModels) so a page facade cannot drop them: the
  first version rendered an empty menu and would have run a full refresh.
* createToastElement whitelists toast types, so a wrong `type` argument degrades
  to the info style instead of rendering an unstyled box.

Verified in a sandbox instance with three roots: a scoped scan walks only the
requested root (progress roots=0/1, 240 files); a full refresh with one root
offline reports kept_unreachable=60 and leaves all 420 models cached; /roots
reports the offline root with its cached count. 3699 passed, 7 skipped;
frontend 1488 passed (148 files); vue widgets 96 passed.
This commit is contained in:
Will Miao
2026-10-07 15:01:26 +08:00
parent 2dcaf6a30e
commit 470d85cca6
34 changed files with 1981 additions and 106 deletions
+6
View File
@@ -151,6 +151,12 @@ Locales: `en`, `zh-CN`, `zh-TW`, `ja`, `ko`, `fr`, `de`, `es`, `ru`, `he` (RTL).
> nothing is tracked for alerts). Register follows each file's existing norm: 你 (zh-CN), > nothing is tracked for alerts). Register follows each file's existing norm: 你 (zh-CN),
> 您 (zh-TW), Sie (de), tú (es), вы (ru). No remaining `[TODO: Translate]` placeholders. > 您 (zh-TW), Sie (de), tú (es), вы (ru). No remaining `[TODO: Translate]` placeholders.
> **Status (2026-10, scoped scan):** the Refresh dropdown gained a per-root scope section
> (`loras.controls.refresh.scopeSection` / `.rootOffline` / `.rootModels`) and the scan result
> toasts (`toast.api.refreshCompleteScoped`, `.refreshKeptUnreachable`, `.scanRootUnreachable`),
> 6 keys in total. They are `[TODO: Translate]` placeholders in the other 9 locales for now —
> the feature owner has not asked for this pass yet.
> **Status (2026-10, reconcile walk progress):** a regular Refresh now reports the reconcile > **Status (2026-10, reconcile walk progress):** a regular Refresh now reports the reconcile
> walk per model root (which roots are being checked, how many model files have been seen, and > walk per model root (which roots are being checked, how many model files have been seen, and
> an ETA) and splits the progress bar into walk (0-50 %) and new-file (50-99 %) phases, which > an ETA) and splits the progress bar into walk (0-50 %) and new-file (50-99 %) phases, which
+283
View File
@@ -0,0 +1,283 @@
# Plan: Scoped Scan (one root / one folder at a time)
**Issue:** [#1108](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1108) — scan a single
folder/root instead of the whole library.
**Status:** **P1 implemented** (2026-10-07, uncommitted working tree). P2 (folder scope, Wave 5)
is not implemented. Supersedes the earlier draft's per-root *status panel* and persisted
*unreachable-subtree* state (both dropped, see "Must NOT have").
Implementation notes / deviations from the draft below:
* The scope dataclass is public (`ReconcileScope`) because it crosses the scanner → service →
route boundary; the draft called it `_ReconcileScope`.
* `_reconcile_cache()` now returns a summary dict, exposed as
`ModelScanner.last_reconcile_summary` and returned by `BaseModelService.scan_models()`, so the
scan HTTP response carries the counts (the WS `completed` payload carries the same fields).
* Root labels are computed over the **whole configured set** and passed into the walk tracker, so
the progress line and the menu never disagree.
* Discovery: `Config._dedupe_existing_paths()` drops roots that do not exist **at config load**, so
a drive that is already off when LM starts is not in `get_model_roots()` at all. Two consequences:
it cannot appear in the refresh menu (nothing to select), and its cached entries are kept because
they are outside every configured root prefix. The `root_unreachable` path therefore covers the
*mid-session* case (drive switched off while LM runs), which is the workflow that motivated the
issue; both paths keep the entries, which is what matters.
* Known limitation to keep in mind: after a scoped scan (or one with unreadable paths) the recorded
empty-folder list is unioned instead of replaced, so an empty folder deleted on disk can linger in
the sidebar until the next full refresh. Models are unaffected.
## TL;DR (For humans)
**What you'll get:** the Refresh ▾ menu can scan **one model root** (one drive) and leave every
other root untouched. Refreshing while a drive is switched off no longer deletes that drive's
models from the cache/DB — they are kept and reported. Browsing the grid while a drive is off no
longer silently strips `preview_url` from that drive's models.
**Why this approach:** a scan scope is a **path scope**, not a device scope, so it also works for
layouts where one root contains symlinks to other drives. The safety net collapses into one rule —
*anything this walk could not read is left untouched* — and that rule is nearly free: the root
existence check already exists, `os.walk(onerror=...)` only fires on failure, and the cached-entry
prefix attribution is already computed for the walk-progress weights.
**What it will NOT do:** no active probing of nested symlinks; no persisted "unreachable subtree"
table; no per-card offline badges (separate change); no root enable/disable switch; the main
Refresh button keeps meaning "scan everything"; no configuration-convention/documentation push to
re-layout libraries that use cross-drive symlinks (dropped by request).
**Effort:** Medium (~1.5–2 days for P1 including tests)
**Risk:** Medium-low — the risk sits in the folder-tree (`all_folders`) merge under a scoped scan
and in the exact pruning predicate; both get explicit tests.
**Decisions to sanity-check:**
1. A full Refresh now treats a *configured but unreachable* root as "keep + report" instead of
"everything under it was deleted". This is a behaviour change (release-note it).
2. Keep the near-zero-cost **first-level symlink** reachability check (reuses
`config._path_mappings`); nested symlinks stay uncovered.
3. Root labels become set-aware (`G: loras` / `usb/loras`, auto-deduped by parent segments).
This also changes the walk-progress line.
## Scope
### Must have (P1)
- **Scope-aware reconcile**: `_reconcile_cache(scope=...)` where scope is
`{roots: [path] | None, folder: rel | None}`. Files inside the scope reconcile normally
(add / repair / remove); everything outside is neither re-read nor removed.
- **Path-level pruning guard**: cached entries under a path this walk could not read are excluded
from `missing` and reported. Three sources:
- configured root that fails the reachability check (already filtered at
`model_scanner.py:1468`),
- directories `os.walk` failed to enter (`onerror` collector; covers Windows junctions to an
offline drive, permission errors, I/O errors),
- first-level symlink mappings whose target is not a directory (`config._path_mappings`).
- **Folder tree correctness under scope**: `all_folders` becomes
`(old outside scope) ∪ (old under unreachable paths) ∪ (discovered)`, so a scoped scan cannot
collapse the sidebar tree and an offline root keeps its folders.
- **Result payload** for the toast: `added`, `removed`, `repaired`, `scanned_roots` (labels),
`skipped_roots` (`{path, label, kept}`), `unavailable_paths` (`{path, kept}`, capped),
`kept_unreachable`.
- **API**: `GET /api/lm/{prefix}/scan?full_rebuild=false&roots=<path>` (repeatable);
`GET /api/lm/{prefix}/roots` gains `root_details: [{path, label, reachable, models}]` while
keeping `roots: [str]` unchanged (other callers depend on it).
- **Frontend**: Refresh ▾ menu lists the current page's roots (label + model count, offline rows
greyed out and non-clickable), main button unchanged, root items disabled while a scan runs;
scoped completion toast reports counts and any kept/skipped paths.
- **Preview fix**: a preview 404 must not clear `preview_url` when the file's parent directory is
itself unreachable (`preview_handlers.py:57` + `_cleanup_stale_preview_url`).
- **Set-aware root labels** (`_root_display_labels(roots)`), shared by the progress broadcasts and
`/roots`: last path segment, Windows drive prefix (`G: loras`), deduped by prepending real parent
segments (`usb/loras`, `a/models/loras`), `(2)` fallback by sorted path, 40-char cap, full path in
the tooltip.
- i18n keys in `locales/en.json` + `python scripts/sync_translation_keys.py`.
- pytest + vitest coverage for every item above.
### Must have (P2, optional follow-up)
- `folder=<rel>` parameter: walk `<root>/<rel>` for every reachable root that contains it (the
sidebar's unified tree has no root identity, so "this folder" means "this relative path in all
roots"), prune only inside that prefix.
- Sidebar folder context menu entry "Scan this folder" (`templates/components/context_menu.html`
`#sidebarFolderContextMenu`, above `check-folder-updates`; update
`tests/frontend/regression/sidebarFolderContextMenu.test.js` expectations).
### Must NOT have (guardrails)
- NO per-entry symlink probing during the walk — `DirEntry.is_symlink()` costs a syscall per entry
on Windows, the exact cost the reconcile optimisation removed. Nested symlinks stay uncovered.
- NO persisted `unavailable_paths` table, no DB schema change, no new cache state.
- NO configuration-convention guidance for cross-drive symlinks (dropped by request).
- NO per-card/modal offline badges in this plan (separate change; needs the L1 prefix list which
P1 produces, so it can build on this later).
- NO root enable/disable setting, NO status panel in Settings.
- NO change to `full_rebuild=true` (it still walks everything and replaces the cache);
`roots` + `full_rebuild=true` is rejected with 400.
- NO new dependency, NO change to the extension-facing endpoints.
- NO `os.path.realpath` for scope/prune routing — business paths only (AGENTS.md rule).
## Todos
### Wave 1 — backend semantics and safety net
1. **Scope plumbing in the scanner.**
What to do: add `_ReconcileScope` (dataclass: `roots: Optional[List[str]]`, `folder: Optional[str]`)
and `_reconcile_cache(scope=None)`. Resolve the effective root list once: configured roots ∩
scope.roots, filtered by reachability; missing ones become `skipped_roots` (never pruned).
`_walk_roots_for_reconcile` / `_walk_root_group_sync` / `_walk_root_for_reconcile` take the scope
so the walk can start at `<root>/<folder>` while still computing `folder`/`file_path` relative to
the **root** (`_process_model_file(path, root_path)` must keep receiving the root).
References: `py/services/model_scanner.py:1453` (`_reconcile_cache`), `:1468` (root filter),
`:1832` (`_walk_root_group_sync`), `:365` (`_walk_root_for_reconcile`), `:134`
(`_count_cached_entries_per_root`), `:126` (`_normalized_root_prefix`).
Done when: a scoped reconcile over 2 roots leaves the other root's `raw_data`, hash index and
folder list byte-identical.
2. **Pruning predicate + unreachable collection.**
What to do: `missing = {p for p in cached_paths - found_paths if in_scope(p) and not
under_unreachable(p)}`; collect unreachable prefixes from (a) skipped roots, (b) an `onerror`
callback on `os.walk` (record `oserror.filename` as a business path), (c) `config._path_mappings`
entries whose target fails `os.path.isdir` (mapping is keyed target→link; protect the **link**
prefix). Count the kept entries per prefix for reporting.
References: `py/services/model_scanner.py:1679` (`missing_files = cached_paths - found_paths`),
`:398` (`os.walk(..., followlinks=True)`), `py/config.py:167` (`_path_mappings`),
`:801` (`add_path_mapping`, target→link), `:750-775` (first-level-only symlink scan).
Done when: with a root removed from disk (or a directory replaced by an unreadable junction),
`raw_data` keeps those entries, `removed` is 0, and the counts land in the result payload.
3. **Folder-tree merge under scope.**
What to do: replace the unconditional `sorted_discovered` assignment with the union rule from
"Must have"; keep the `folders_changed` comparison and the persist path unchanged. Unscoped
scans with nothing unreachable must produce exactly today's list.
References: `py/services/model_scanner.py` (walk loop records `discovered_folders`; `model_scanner.py:1738`
(`sorted_discovered`) and `:1739` (`folders_changed`) after the dedup pass).
Done when: a scoped scan keeps folders outside the scope, an offline root keeps its subtree, and
a full scan still drops folders deleted on disk.
4. **Result payload + progress/cancel/complete messages.**
What to do: extend the `completed` broadcast with `scanned_roots`, `skipped_roots`,
`unavailable_paths`, `kept_unreachable`, `repaired`; the walk tracker already receives only the
scoped roots, so the progress line shows that root's label with `roots_total=1`.
References: `py/services/model_scanner.py` `_broadcast_scan_progress` (`:576`),
`_ReconcileWalkTracker` (`:266`), the completed broadcast at the end of `_reconcile_cache`.
Done when: a scoped scan emits `stage=reconcile_scan` with the single scoped label and a
`completed` payload carrying the new fields.
5. **API surface.**
What to do: `scan_models` accepts repeated `roots` query params (400 on unknown root, 400 when
combined with `full_rebuild=true`); `get_model_roots` adds `root_details` (label, reachable,
model count via the shared prefix attribution) without touching `roots`.
References: `py/routes/handlers/model_handlers.py:1139` (`scan_models`), `:1165`
(`get_model_roots`), `py/routes/model_route_registrar.py:61-62`, `static/js/api/apiConfig.js:114`.
Done when: `tests/routes/test_lora_routes.py`-style coverage passes for the new params and the
backward-compatible `/roots` payload.
6. **Preview 404 must not prune when the parent is unreachable.**
What to do: in `serve_preview`, before `_cleanup_stale_preview_url`, check
`os.path.isdir(os.path.dirname(resolved))`; if the directory itself is missing/unreachable,
return 404 **without** clearing caches. (Centralized sidecar mode puts the preview in the mirror
tree, so the check must use the preview file's own parent.)
References: `py/routes/handlers/preview_handlers.py:57` (404 branch), `:74-110`
(`_cleanup_stale_preview_url` → `clear_preview_by_path` + `_persist_current_cache`),
`tests/routes/test_preview_routes.py`.
Done when: a test asserts the cache keeps `preview_url` when the parent dir is absent and still
clears it when the file was really deleted from a reachable directory.
### Wave 2 — labels
7. **`_root_display_labels(roots)` (set-aware, order-independent).**
What to do: replace the single-root `_root_display_label`; compute over the sorted root list;
prepend real parent segments until unique; `(2)`/`(3)` fallback; 40-char middle-ellipsis; return
`{path: label}`. Use it in `_ReconcileWalkTracker.__init__` (progress) and in the `/roots`
handler so both always agree.
References: `py/services/model_scanner.py:117` (`_root_display_label`, current single-root
version), `:266` (`_ReconcileWalkTracker`).
Done when: unit tests cover `usb/loras` + `ssd/loras`, `a/models/loras` + `b/models/loras`,
Windows `G:\x\loras` + `H:\y\loras` → `G: loras` / `H: loras`, and the identical-path fallback.
### Wave 3 — frontend
8. **Refresh ▾ scope section.**
What to do: add an empty container to `templates/components/controls.html` inside
`.dropdown-menu` (`#refreshScopeMenu`) + a section title; on dropdown open, fetch
`endpoints.roots`, render one row per root (`data-action="scan-root"`, `data-root="<path>"`,
label + model count, `disabled` + "Offline" for unreachable), hide the section on the recipes
page (recipes have no roots); wire clicks by **event delegation** on the menu (the existing
`full-rebuild` item is bound with a direct `querySelector`, dynamic rows cannot be);
invalidate the cached list after a scan finishes; add CSS for the section + `max-height`/
`overflow-y` so 4+ roots stay usable.
References: `templates/components/controls.html:67-76`, `static/js/components/controls/PageControls.js:131`
(`data-action="refresh"`), `:220-260` (`initDropdowns` + `full-rebuild` wiring),
`static/js/api/apiConfig.js:118` (`roots` endpoint), `static/js/api/baseModelApi.js:1318`
(`fetchModelRoots`).
Done when: the menu lists roots with counts, an offline root is visibly disabled, and clicking a
row starts a scoped refresh.
9. **Scoped request + toast.**
What to do: `refreshModels(fullRebuild, {roots})` appends repeated `roots` params; the
`completed` payload drives the toast (`added`/`removed`/`kept_unreachable`/`skipped_roots`);
offline rows toast `… is not reachable right now. Nothing was changed.` without a request.
References: `static/js/api/baseModelApi.js:514-602` (refreshModels, URL build at `:601`, toast
at `:618`), `static/js/components/controls/PageControls.js:485`.
Done when: the frontend test asserts the scoped URL and both toast variants.
10. **i18n.**
What to do: add `loras.controls.refresh.scopeSection`, `.rootOffline`, `.rootModels`,
`toast.api.refreshCompleteScoped`, `toast.api.refreshKeptUnreachable`,
`toast.api.scanRootUnreachable` to `locales/en.json`, run
`python scripts/sync_translation_keys.py`, then stop (placeholders are the expected state until
translations are requested).
Done when: `pytest tests/i18n` passes and every locale has the keys.
### Wave 4 — tests and verification
11. **Backend tests** (`tests/services/test_model_scanner.py`, `tests/routes/test_lora_routes.py`,
`tests/routes/test_preview_routes.py`):
`test_reconcile_scoped_scan_leaves_other_roots_untouched`,
`test_reconcile_scoped_scan_removes_deleted_files_in_scope`,
`test_reconcile_scoped_scan_preserves_folder_tree_outside_scope`,
`test_reconcile_keeps_entries_of_unreachable_root`,
`test_reconcile_keeps_entries_under_unreadable_dir`,
`test_reconcile_keeps_entries_under_offline_first_level_symlink`,
`test_root_display_labels_dedupe_by_parent_segments`,
`test_scan_models_accepts_roots_param`, `test_scan_rejects_unknown_root`,
`test_scan_rejects_roots_with_full_rebuild`, `test_roots_endpoint_reports_details`,
`test_preview_404_keeps_cache_when_parent_dir_missing`.
12. **Frontend tests**: extend `tests/frontend/api/baseModelApi.refresh.test.js` (scoped URL, both
toasts) and add `tests/frontend/components/controls/pageControls.scanRoots.test.js`
(rows rendered from `/roots`, offline row disabled, delegation wiring).
13. **Full suites + sandbox eyeball**: `pytest -q`, `npx vitest run`, `npm run test:vue`; then a
sandboxed standalone instance with two roots and a slowed walk
(`LM_WALK_DELAY_S`-style `sitecustomize` hook) to eyeball the scoped progress line, the offline
row and the toast — the user verifies by eye.
### Wave 5 — P2 (folder scope), only if approved
14. `folder=<rel>` in the scan endpoint + `_ReconcileScope.folder`, the sidebar menu entry, the
context-menu regression test update, and the "folder in several roots" semantics (scan every
reachable root that has the relative path; report per root).
## Known limitations (accepted)
- **Nested symlinks** (a symlink *below* a root pointing at another drive) are only covered when
`os.walk` fails to enter the target (Windows junctions, permission errors). A broken symlink that
`os.walk` classifies as a non-directory (typical on POSIX) is not detected: its entries are pruned
as today and come back on the next scan once the target is reachable (sidecars carry the sha256,
so no re-hash). Verify the Windows junction behaviour before writing release notes.
- Only **first-level** symlinks are known to `config` (by design, `config.py:750-775`), so the
symlink reachability check inherits that limit.
- A scoped scan of a folder does not re-read metadata for unchanged files (unchanged behaviour).
- Empty folders removed on disk can linger in the sidebar after a scoped scan (the folder list is
unioned rather than replaced whenever the scan did not verify every root); the next full refresh
drops them.
- `root_details[].reachable` is a live `os.path.exists()` per root, so a root that disappears
mid-session shows as offline; a root that was already gone at startup is filtered out by
`Config` and therefore absent from the list.
## Verification checklist
- [ ] `pytest -q` green, `npx vitest run` green, `npm run test:vue` green.
- [ ] `python scripts/sync_translation_keys.py --dry-run` reports no pending changes.
- [ ] Sandbox: 2 roots, one offline → scoped scan of the online root reports `added`, keeps the
offline root's models, and the grid/sidebar still show them.
- [ ] Sandbox: preview of an offline root's model returns 404 and the DB keeps `preview_url`.
- [ ] Release note wording agreed for the behaviour change (decision 1).
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "Modelliste aktualisieren", "title": "Modelliste aktualisieren",
"full": "Cache neu aufbauen", "full": "Cache neu aufbauen",
"fullTooltip": "Alle Modelldetails aus Metadatendateien neu laden – nutzen, wenn die Bibliothek veraltet wirkt oder nach manuellen Änderungen." "fullTooltip": "Alle Modelldetails aus Metadatendateien neu laden – nutzen, wenn die Bibliothek veraltet wirkt oder nach manuellen Änderungen.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "Metadaten von CivitAI abrufen", "title": "Metadaten von CivitAI abrufen",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "Fehler beim Herunterladen der Beispielbilder: {message}", "exampleImagesDownloadFailed": "Fehler beim Herunterladen der Beispielbilder: {message}",
"moveFailed": "Element konnte nicht verschoben werden: {message}", "moveFailed": "Element konnte nicht verschoben werden: {message}",
"copiedToClipboard": "In die Zwischenablage kopiert", "copiedToClipboard": "In die Zwischenablage kopiert",
"downloadStarted": "Download gestartet" "downloadStarted": "Download gestartet",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "KI-Anbieter nicht konfiguriert. Aktivieren Sie ihn unter Einstellungen → KI-Anbieter.", "llmNotConfigured": "KI-Anbieter nicht konfiguriert. Aktivieren Sie ihn unter Einstellungen → KI-Anbieter.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "Refresh model list", "title": "Refresh model list",
"full": "Rebuild Cache", "full": "Rebuild Cache",
"fullTooltip": "Reload all model details from metadata files—use if the library looks out of date or after manual edits." "fullTooltip": "Reload all model details from metadata files—use if the library looks out of date or after manual edits.",
"scopeSection": "Scan one folder",
"rootOffline": "Offline",
"rootModels": "{count} models"
}, },
"fetch": { "fetch": {
"title": "Fetch metadata from CivitAI", "title": "Fetch metadata from CivitAI",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "Failed to download example images: {message}", "exampleImagesDownloadFailed": "Failed to download example images: {message}",
"moveFailed": "Failed to move item: {message}", "moveFailed": "Failed to move item: {message}",
"copiedToClipboard": "Copied to clipboard", "copiedToClipboard": "Copied to clipboard",
"downloadStarted": "Download started" "downloadStarted": "Download started",
"refreshCompleteScoped": "Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "{count} models kept: {paths} not reachable",
"scanRootUnreachable": "{scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "AI provider not configured. Enable it in Settings → AI Provider.", "llmNotConfigured": "AI provider not configured. Enable it in Settings → AI Provider.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "Actualizar lista de modelos", "title": "Actualizar lista de modelos",
"full": "Reconstruir caché", "full": "Reconstruir caché",
"fullTooltip": "Vuelve a cargar todos los detalles desde los archivos de metadatos; úsalo si la biblioteca parece desactualizada o tras ediciones manuales." "fullTooltip": "Vuelve a cargar todos los detalles desde los archivos de metadatos; úsalo si la biblioteca parece desactualizada o tras ediciones manuales.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "Obtener metadatos de CivitAI", "title": "Obtener metadatos de CivitAI",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "Error al descargar imágenes de ejemplo: {message}", "exampleImagesDownloadFailed": "Error al descargar imágenes de ejemplo: {message}",
"moveFailed": "No se pudo mover el elemento: {message}", "moveFailed": "No se pudo mover el elemento: {message}",
"copiedToClipboard": "Copiado al portapapeles", "copiedToClipboard": "Copiado al portapapeles",
"downloadStarted": "Descarga iniciada" "downloadStarted": "Descarga iniciada",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "Proveedor de IA no configurado. Actívelo en Configuración → Proveedor de IA.", "llmNotConfigured": "Proveedor de IA no configurado. Actívelo en Configuración → Proveedor de IA.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "Actualiser la liste des modèles", "title": "Actualiser la liste des modèles",
"full": "Reconstruire le cache", "full": "Reconstruire le cache",
"fullTooltip": "Recharge tous les détails des modèles depuis les fichiers de métadonnées — à utiliser si la bibliothèque paraît obsolète ou après des modifications manuelles." "fullTooltip": "Recharge tous les détails des modèles depuis les fichiers de métadonnées — à utiliser si la bibliothèque paraît obsolète ou après des modifications manuelles.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "Récupérer les métadonnées depuis CivitAI", "title": "Récupérer les métadonnées depuis CivitAI",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "Échec du téléchargement des images d'exemple : {message}", "exampleImagesDownloadFailed": "Échec du téléchargement des images d'exemple : {message}",
"moveFailed": "Échec du déplacement de l'élément : {message}", "moveFailed": "Échec du déplacement de l'élément : {message}",
"copiedToClipboard": "Copié dans le presse-papiers", "copiedToClipboard": "Copié dans le presse-papiers",
"downloadStarted": "Téléchargement démarré" "downloadStarted": "Téléchargement démarré",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "Fournisseur d'IA non configuré. Activez-le dans Paramètres → Fournisseur d'IA.", "llmNotConfigured": "Fournisseur d'IA non configuré. Activez-le dans Paramètres → Fournisseur d'IA.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "רענן רשימת מודלים", "title": "רענן רשימת מודלים",
"full": "בניית מטמון מחדש", "full": "בניית מטמון מחדש",
"fullTooltip": "טוען מחדש את כל פרטי המודלים מקבצי המטא-נתונים – לשימוש אם הספרייה נראית לא מעודכנת או לאחר עריכות ידניות." "fullTooltip": "טוען מחדש את כל פרטי המודלים מקבצי המטא-נתונים – לשימוש אם הספרייה נראית לא מעודכנת או לאחר עריכות ידניות.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "אחזר מטא-נתונים מ-CivitAI", "title": "אחזר מטא-נתונים מ-CivitAI",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "הורדת תמונות הדוגמה נכשלה: {message}", "exampleImagesDownloadFailed": "הורדת תמונות הדוגמה נכשלה: {message}",
"moveFailed": "לא ניתן היה להעביר את הפריט: {message}", "moveFailed": "לא ניתן היה להעביר את הפריט: {message}",
"copiedToClipboard": "הועתק ללוח", "copiedToClipboard": "הועתק ללוח",
"downloadStarted": "ההורדה החלה" "downloadStarted": "ההורדה החלה",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "ספק AI לא הוגדר. הפעל אותו בהגדרות → ספק AI.", "llmNotConfigured": "ספק AI לא הוגדר. הפעל אותו בהגדרות → ספק AI.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "モデルリストを更新", "title": "モデルリストを更新",
"full": "キャッシュを再構築", "full": "キャッシュを再構築",
"fullTooltip": "メタデータファイルから全モデル情報を再読み込みします。リストが古いと感じるときや手動編集後に使用してください。" "fullTooltip": "メタデータファイルから全モデル情報を再読み込みします。リストが古いと感じるときや手動編集後に使用してください。",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "CivitAIからメタデータを取得", "title": "CivitAIからメタデータを取得",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "例画像のダウンロードに失敗しました:{message}", "exampleImagesDownloadFailed": "例画像のダウンロードに失敗しました:{message}",
"moveFailed": "アイテムを移動できませんでした: {message}", "moveFailed": "アイテムを移動できませんでした: {message}",
"copiedToClipboard": "クリップボードにコピーしました", "copiedToClipboard": "クリップボードにコピーしました",
"downloadStarted": "ダウンロードを開始しました" "downloadStarted": "ダウンロードを開始しました",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "AIプロバイダーが設定されていません。設定 → AIプロバイダーで有効にしてください。", "llmNotConfigured": "AIプロバイダーが設定されていません。設定 → AIプロバイダーで有効にしてください。",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "모델 목록 새로고침", "title": "모델 목록 새로고침",
"full": "캐시 재구성", "full": "캐시 재구성",
"fullTooltip": "메타데이터 파일에서 모든 모델 정보를 다시 불러옵니다. 라이브러리가 오래되어 보이거나 수동 수정 후에 사용하세요." "fullTooltip": "메타데이터 파일에서 모든 모델 정보를 다시 불러옵니다. 라이브러리가 오래되어 보이거나 수동 수정 후에 사용하세요.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "CivitAI에서 메타데이터 가져오기", "title": "CivitAI에서 메타데이터 가져오기",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "예시 이미지 다운로드 실패: {message}", "exampleImagesDownloadFailed": "예시 이미지 다운로드 실패: {message}",
"moveFailed": "항목을 이동하지 못했습니다: {message}", "moveFailed": "항목을 이동하지 못했습니다: {message}",
"copiedToClipboard": "클립보드에 복사됨", "copiedToClipboard": "클립보드에 복사됨",
"downloadStarted": "다운로드 시작됨" "downloadStarted": "다운로드 시작됨",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "AI 제공자가 설정되지 않았습니다. 설정 → AI 제공자에서 활성화하세요.", "llmNotConfigured": "AI 제공자가 설정되지 않았습니다. 설정 → AI 제공자에서 활성화하세요.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "Обновить список моделей", "title": "Обновить список моделей",
"full": "Перестроить кэш", "full": "Перестроить кэш",
"fullTooltip": "Перечитывает все данные моделей из файлов метаданных — используйте, если библиотека выглядит устаревшей или после ручных правок." "fullTooltip": "Перечитывает все данные моделей из файлов метаданных — используйте, если библиотека выглядит устаревшей или после ручных правок.",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "Получить метаданные с CivitAI", "title": "Получить метаданные с CivitAI",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "Не удалось загрузить примеры изображений: {message}", "exampleImagesDownloadFailed": "Не удалось загрузить примеры изображений: {message}",
"moveFailed": "Не удалось переместить элемент: {message}", "moveFailed": "Не удалось переместить элемент: {message}",
"copiedToClipboard": "Скопировано в буфер обмена", "copiedToClipboard": "Скопировано в буфер обмена",
"downloadStarted": "Загрузка начата" "downloadStarted": "Загрузка начата",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "Поставщик ИИ не настроен. Включите его в Настройки → Поставщик ИИ.", "llmNotConfigured": "Поставщик ИИ не настроен. Включите его в Настройки → Поставщик ИИ.",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "刷新模型列表", "title": "刷新模型列表",
"full": "重建缓存", "full": "重建缓存",
"fullTooltip": "从元数据文件重新加载所有模型信息;用于列表过时或手动编辑后。" "fullTooltip": "从元数据文件重新加载所有模型信息;用于列表过时或手动编辑后。",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "从 CivitAI 获取元数据", "title": "从 CivitAI 获取元数据",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "示例图片下载失败:{message}", "exampleImagesDownloadFailed": "示例图片下载失败:{message}",
"moveFailed": "移动条目失败:{message}", "moveFailed": "移动条目失败:{message}",
"copiedToClipboard": "已复制到剪贴板", "copiedToClipboard": "已复制到剪贴板",
"downloadStarted": "下载已开始" "downloadStarted": "下载已开始",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "AI 提供商未配置。请在 设置 → AI 提供商 中进行配置。", "llmNotConfigured": "AI 提供商未配置。请在 设置 → AI 提供商 中进行配置。",
+8 -2
View File
@@ -918,7 +918,10 @@
"refresh": { "refresh": {
"title": "重新整理模型列表", "title": "重新整理模型列表",
"full": "重建快取", "full": "重建快取",
"fullTooltip": "從中繼資料檔重新載入所有模型資訊;適用於清單過時或手動編輯後。" "fullTooltip": "從中繼資料檔重新載入所有模型資訊;適用於清單過時或手動編輯後。",
"scopeSection": "[TODO: Translate] Scan one folder",
"rootOffline": "[TODO: Translate] Offline",
"rootModels": "[TODO: Translate] {count} models"
}, },
"fetch": { "fetch": {
"title": "從 CivitAI 取得 metadata", "title": "從 CivitAI 取得 metadata",
@@ -2804,7 +2807,10 @@
"exampleImagesDownloadFailed": "下載範例圖片失敗:{message}", "exampleImagesDownloadFailed": "下載範例圖片失敗:{message}",
"moveFailed": "移動項目失敗:{message}", "moveFailed": "移動項目失敗:{message}",
"copiedToClipboard": "已複製到剪貼簿", "copiedToClipboard": "已複製到剪貼簿",
"downloadStarted": "下載已開始" "downloadStarted": "下載已開始",
"refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
"refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
"scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
}, },
"agent": { "agent": {
"llmNotConfigured": "AI 提供者尚未設定。請在 設定 → AI 提供者 中進行設定。", "llmNotConfigured": "AI 提供者尚未設定。請在 設定 → AI 提供者 中進行設定。",
+10
View File
@@ -798,6 +798,16 @@ class Config:
except Exception as e: except Exception as e:
logger.error(f"Error scanning links in {root}: {e}") logger.error(f"Error scanning links in {root}: {e}")
def iter_path_mappings(self) -> List[Tuple[str, str]]:
"""Return the known ``(physical target, virtual link)`` symlink pairs.
Only symlinks directly under a model root are tracked (see
:meth:`_scan_symbolic_links`), so callers must treat this as a partial
view of the on-disk link layout — enough to notice that a linked drive
went away, not enough to resolve nested links.
"""
return list(self._path_mappings.items())
def add_path_mapping(self, link_path: str, target_path: str): def add_path_mapping(self, link_path: str, target_path: str):
"""Add a symbolic link path mapping """Add a symbolic link path mapping
target_path: actual target path target_path: actual target path
+40 -9
View File
@@ -26,6 +26,7 @@ from ...services.connectivity_guard import (
) )
from ...services.metadata_sync_service import MetadataSyncService from ...services.metadata_sync_service import MetadataSyncService
from ...services.model_file_service import ModelMoveService from ...services.model_file_service import ModelMoveService
from ...services.model_scanner import ReconcileScope
from ...services.preview_asset_service import PreviewAssetService from ...services.preview_asset_service import PreviewAssetService
from ...services.service_registry import ServiceRegistry from ...services.service_registry import ServiceRegistry
from ...services.settings_manager import SettingsManager, get_settings_manager from ...services.settings_manager import SettingsManager, get_settings_manager
@@ -1139,8 +1140,30 @@ class ModelQueryHandler:
async def scan_models(self, request: web.Request) -> web.Response: async def scan_models(self, request: web.Request) -> web.Response:
try: try:
full_rebuild = request.query.get("full_rebuild", "false").lower() == "true" full_rebuild = request.query.get("full_rebuild", "false").lower() == "true"
await self._service.scan_models( requested_roots = [
force_refresh=True, rebuild_cache=full_rebuild value for value in request.query.getall("roots", []) if value
]
if requested_roots and full_rebuild:
return web.json_response(
{
"error": "Scoped scans are not supported with "
"full_rebuild=true; a full rebuild always walks every root"
},
status=400,
)
scope = None
if requested_roots:
configured = self._service.get_model_roots()
unknown = [root for root in requested_roots if root not in configured]
if unknown:
return web.json_response(
{"error": "Unknown model root(s)", "roots": unknown}, status=400
)
scope = ReconcileScope(roots=tuple(requested_roots))
summary = await self._service.scan_models(
force_refresh=True, rebuild_cache=full_rebuild, scope=scope
) )
_broadcast_models_changed() _broadcast_models_changed()
if self._service.scanner.is_cancelled(): if self._service.scanner.is_cancelled():
@@ -1150,12 +1173,13 @@ class ModelQueryHandler:
"message": f"{self._service.model_type.capitalize()} scan cancelled", "message": f"{self._service.model_type.capitalize()} scan cancelled",
} }
) )
return web.json_response( payload: Dict[str, Any] = {
{ "status": "success",
"status": "success", "message": f"{self._service.model_type.capitalize()} scan completed",
"message": f"{self._service.model_type.capitalize()} scan completed", }
} if summary:
) payload.update(summary)
return web.json_response(payload)
except Exception as exc: except Exception as exc:
self._logger.error( self._logger.error(
"Error scanning %ss: %s", self._service.model_type, exc, exc_info=True "Error scanning %ss: %s", self._service.model_type, exc, exc_info=True
@@ -1165,7 +1189,14 @@ class ModelQueryHandler:
async def get_model_roots(self, request: web.Request) -> web.Response: async def get_model_roots(self, request: web.Request) -> web.Response:
try: try:
roots = self._service.get_model_roots() roots = self._service.get_model_roots()
return web.json_response({"success": True, "roots": roots}) try:
root_details = self._service.describe_model_roots()
except Exception as exc: # pragma: no cover - defensive
self._logger.debug("Root details unavailable: %s", exc)
root_details = []
return web.json_response(
{"success": True, "roots": roots, "root_details": root_details}
)
except Exception as exc: except Exception as exc:
self._logger.error( self._logger.error(
"Error getting %s roots: %s", "Error getting %s roots: %s",
+12 -1
View File
@@ -54,7 +54,18 @@ class PreviewHandler:
if not resolved.is_file(): if not resolved.is_file():
logger.debug("Preview file not found at %s", str(resolved)) logger.debug("Preview file not found at %s", str(resolved))
asyncio.create_task(self._cleanup_stale_preview_url(normalized)) if resolved.parent.is_dir():
# The file is really gone from a reachable directory, so the
# cached preview_url is stale and can be cleared.
asyncio.create_task(self._cleanup_stale_preview_url(normalized))
else:
# The directory itself is unreachable (drive switched off,
# unmounted share). Nothing was deleted: keep the cached
# preview_url so the card recovers when the drive is back.
logger.debug(
"Preview directory unreachable, keeping cached preview_url: %s",
str(resolved.parent),
)
raise web.HTTPNotFound(text="Preview file not found") raise web.HTTPNotFound(text="Preview file not found")
# aiohttp's FileResponse handles range requests, content headers, and # aiohttp's FileResponse handles range requests, content headers, and
+19 -5
View File
@@ -27,6 +27,7 @@ from ..utils.civitai_utils import build_civitai_model_page_url
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
if TYPE_CHECKING: if TYPE_CHECKING:
from .model_scanner import ReconcileScope
from .model_update_service import ModelUpdateService from .model_update_service import ModelUpdateService
@@ -936,12 +937,25 @@ class BaseModelService(ABC):
return self.scanner.get_hash_by_path(file_path) return self.scanner.get_hash_by_path(file_path)
async def scan_models( async def scan_models(
self, force_refresh: bool = False, rebuild_cache: bool = False self,
): force_refresh: bool = False,
"""Trigger model scanning""" rebuild_cache: bool = False,
return await self.scanner.get_cached_data( scope: Optional["ReconcileScope"] = None,
force_refresh=force_refresh, rebuild_cache=rebuild_cache ) -> Optional[Dict[str, Any]]:
"""Trigger model scanning, optionally restricted to a scope.
Returns the reconcile summary (added / removed / repaired / skipped
roots / kept entries) for incremental scans, ``None`` for a full
rebuild.
"""
await self.scanner.get_cached_data(
force_refresh=force_refresh, rebuild_cache=rebuild_cache, scope=scope
) )
return self.scanner.last_reconcile_summary
def describe_model_roots(self) -> List[Dict[str, Any]]:
"""Describe configured roots (label / reachability / cached count)."""
return self.scanner.describe_model_roots()
async def get_model_info_by_name(self, name: str): async def get_model_info_by_name(self, name: str):
"""Get model information by name""" """Get model information by name"""
+425 -42
View File
@@ -114,13 +114,84 @@ def _new_file_pass_progress(processed: int, total: int) -> int:
return _WALK_PROGRESS_SHARE + min(49, int(ratio * 49)) return _WALK_PROGRESS_SHARE + min(49, int(ratio * 49))
def _root_display_label(root: str) -> str: # Maximum length of a root label in progress messages and menus; the full path
"""Short label identifying a model root in progress messages.""" # always stays available in the tooltip.
drive, _tail = os.path.splitdrive(root) _ROOT_LABEL_MAX_LENGTH = 40
if drive:
return drive
normalized = root.rstrip("/\\") def _truncate_label(label: str, limit: int = _ROOT_LABEL_MAX_LENGTH) -> str:
return os.path.basename(normalized) or root """Middle-truncate an over-long label."""
if len(label) <= limit:
return label
keep = max(limit - 3, 1)
head = (keep + 1) // 2
tail = keep - head
if not tail:
return f"{label[:head]}..."
return f"{label[:head]}...{label[-tail:]}"
def _root_path_parts(root: str) -> Tuple[str, List[str]]:
"""Split a root into its drive prefix (Windows) and its path segments."""
drive, tail = os.path.splitdrive(root)
normalized = tail.replace("\\", "/").strip("/")
return drive, [part for part in normalized.split("/") if part]
def _root_display_labels(roots: Sequence[str]) -> Dict[str, str]:
"""Return a unique, human-friendly label for every root in the set.
The label starts as the last path segment (prefixed with the drive letter on
Windows, e.g. ``G: loras``) and grows leftwards with **real parent path
segments** until it is unique inside the set (``usb/loras`` vs
``ssd/loras``). Roots that still render identically (Windows case variants,
a duplicated config entry) get a deterministic numeric suffix.
Labels are a property of the whole set, so they are computed from the sorted
root list: the same configuration always yields the same labels, whatever
order the roots are walked in.
"""
if not roots:
return {}
parts_by_root = {root: _root_path_parts(root) for root in roots}
depth = {root: (1 if parts_by_root[root][1] else 0) for root in roots}
def _render(root: str) -> str:
drive, parts = parts_by_root[root]
if not parts:
return drive or root
take = max(min(depth[root], len(parts)), 1)
tail = "/".join(parts[len(parts) - take:])
return f"{drive} {tail}" if drive else tail
# Grow the depth of colliding labels until they can be told apart.
while True:
groups: Dict[str, List[str]] = {}
for root in roots:
groups.setdefault(_render(root), []).append(root)
colliding = [group for group in groups.values() if len(group) > 1]
if not colliding:
break
grew = False
for group in colliding:
for root in group:
if depth[root] < len(parts_by_root[root][1]):
depth[root] += 1
grew = True
if not grew:
break
labels = {root: _render(root) for root in roots}
fallback: Dict[str, List[str]] = {}
for root in sorted(roots):
fallback.setdefault(labels[root], []).append(root)
for label, group in fallback.items():
if len(group) > 1:
for index, root in enumerate(group, start=1):
labels[root] = f"{label} ({index})"
return {root: _truncate_label(label) for root, label in labels.items()}
def _normalized_root_prefix(root: str) -> str: def _normalized_root_prefix(root: str) -> str:
@@ -131,34 +202,136 @@ def _normalized_root_prefix(root: str) -> str:
return prefix.lower() if _CASE_INSENSITIVE_PATHS else prefix return prefix.lower() if _CASE_INSENSITIVE_PATHS else prefix
def _count_cached_entries_per_root( def _path_matches_prefix(path: str, prefix: str) -> bool:
cached_paths: Set[str], roots: Sequence[str] """Match a cached business path against a normalized scope prefix."""
) -> Dict[str, int]: candidate = path.lower() if _CASE_INSENSITIVE_PATHS else path
"""Attribute cached entries to model roots (longest prefix wins). return candidate.startswith(prefix)
def _scope_prefix(root: str, folder: Optional[str] = None) -> str:
"""Normalized prefix covering a root, or a folder inside that root."""
prefix = _normalized_root_prefix(root)
if folder:
relative = folder.replace("\\", "/").strip("/")
if relative:
if _CASE_INSENSITIVE_PATHS:
relative = relative.lower()
prefix = f"{prefix}{relative}/"
return prefix
def _scope_walk_path(root: str, folder: Optional[str] = None) -> str:
"""Filesystem path a scoped walk starts at (the root itself when unscoped)."""
if not folder:
return root
relative = folder.replace("\\", "/").strip("/")
if not relative:
return root
return os.path.join(root, *relative.split("/"))
@dataclass(frozen=True)
class ReconcileScope:
"""Restrict a reconcile to some of the configured roots and/or a folder.
``roots=None`` means "every configured root". ``folder`` is a root-relative
folder in forward-slash form and applies to each root in the scope.
"""
roots: Optional[Tuple[str, ...]] = None
folder: Optional[str] = None
class _UnreachablePaths:
"""Prefixes this reconcile could not read; their entries are never pruned.
Populated from three sources: a configured root that is not reachable, a
directory ``os.walk`` failed to enter (permissions, I/O error, a Windows
junction to an offline drive), and a known first-level symlink whose target
is not a directory. Cached entries under these prefixes are reported instead
of removed, so switching a drive off can no longer wipe its models.
"""
def __init__(self) -> None:
self._lock = threading.Lock()
self._prefixes: List[str] = []
self._details: Dict[str, Dict[str, Any]] = {}
def add(self, path: str, reason: str) -> None:
prefix = _normalized_root_prefix(path)
with self._lock:
if prefix in self._details:
return
self._details[prefix] = {
'path': path.replace(os.sep, '/'),
'reason': reason,
'kept': 0,
}
self._prefixes.append(prefix)
def match(self, cached_path: str) -> Optional[str]:
"""Return the prefix covering ``cached_path``, if any."""
for prefix in self._prefixes:
if _path_matches_prefix(cached_path, prefix):
return prefix
return None
def record_kept(self, prefix: str, amount: int = 1) -> None:
detail = self._details.get(prefix)
if detail is not None:
detail['kept'] += amount
def total_kept(self) -> int:
return sum(detail['kept'] for detail in self._details.values())
def count(self) -> int:
return len(self._details)
def has_entries(self) -> bool:
return bool(self._details)
def payload(self, limit: int = 5) -> List[Dict[str, Any]]:
"""Most affected prefixes first, capped for the progress payload."""
entries = sorted(
self._details.values(), key=lambda detail: detail['kept'], reverse=True
)
return [dict(entry) for entry in entries[:limit]]
def _count_cached_entries_for_prefixes(
cached_paths: Set[str], prefixes: Sequence[Tuple[str, str]]
) -> Dict[str, int]:
"""Count cached entries per ``(key, normalized prefix)`` pair.
Longest prefix wins, so a folder scope counts only the entries inside it.
Used as the walk-workload weight for progress reporting: the walk itself is Used as the walk-workload weight for progress reporting: the walk itself is
what discovers the real file count, so the cached entry count is the only what discovers the real file count, so the cached entry count is the only
estimate available up front. estimate available up front.
""" """
counts: Dict[str, int] = {root: 0 for root in roots} counts: Dict[str, int] = {key: 0 for key, _prefix in prefixes}
if not roots: if not prefixes:
return counts return counts
prefixes = sorted( ordered = sorted(prefixes, key=lambda item: len(item[1]), reverse=True)
((_normalized_root_prefix(root), root) for root in roots),
key=lambda item: len(item[0]),
reverse=True,
)
case_insensitive = _CASE_INSENSITIVE_PATHS case_insensitive = _CASE_INSENSITIVE_PATHS
for path in cached_paths: for path in cached_paths:
candidate = path.lower() if case_insensitive else path candidate = path.lower() if case_insensitive else path
for prefix, root in prefixes: for key, prefix in ordered:
if candidate.startswith(prefix): if candidate.startswith(prefix):
counts[root] += 1 counts[key] += 1
break break
return counts return counts
def _count_cached_entries_per_root(
cached_paths: Set[str], roots: Sequence[str]
) -> Dict[str, int]:
"""Attribute cached entries to model roots (longest prefix wins)."""
return _count_cached_entries_for_prefixes(
cached_paths, [(root, _normalized_root_prefix(root)) for root in roots]
)
def _root_device_key(root: str) -> str: def _root_device_key(root: str) -> str:
"""Group key identifying the storage device that holds a root. """Group key identifying the storage device that holds a root.
@@ -273,10 +446,19 @@ class _ReconcileWalkTracker:
they finish. they finish.
""" """
def __init__(self, roots: Sequence[str], expected: Mapping[str, int]) -> None: def __init__(
self,
roots: Sequence[str],
expected: Mapping[str, int],
labels: Optional[Mapping[str, str]] = None,
) -> None:
self._lock = threading.Lock() self._lock = threading.Lock()
self._roots: List[str] = list(roots) self._roots: List[str] = list(roots)
self._labels = {root: _root_display_label(root) for root in self._roots} # Callers that already labelled the full configured root set pass those
# labels in, so the progress line and the refresh menu never disagree on
# how a root is named.
resolved = dict(labels) if labels else _root_display_labels(self._roots)
self._labels = {root: resolved.get(root) or root for root in self._roots}
self._expected = { self._expected = {
root: max(int(expected.get(root, 0) or 0), 0) for root in self._roots root: max(int(expected.get(root, 0) or 0), 0) for root in self._roots
} }
@@ -373,15 +555,28 @@ def _walk_root_for_reconcile(
excluded_models: Set[str], excluded_models: Set[str],
is_cancelled: Callable[[], bool], is_cancelled: Callable[[], bool],
report_progress: Callable[[int], None], report_progress: Callable[[int], None],
walk_path: Optional[str] = None,
unreachable: Optional[_UnreachablePaths] = None,
) -> _RootWalkResult: ) -> _RootWalkResult:
"""Walk one model root and classify every model file found. """Walk one model root and classify every model file found.
``walk_path`` defaults to the root itself; a folder scope starts the walk
deeper but still resolves ``folder``/``file_path`` against ``root_path`` so
cache entries keep their library-relative shape.
Runs synchronously (worker thread) and never mutates scanner state: all Runs synchronously (worker thread) and never mutates scanner state: all
cache updates happen on the event loop once every root has been walked. cache updates happen on the event loop once every root has been walked.
""" """
result = _RootWalkResult(root_path=root_path) result = _RootWalkResult(root_path=root_path)
stale_seen: Set[str] = set() stale_seen: Set[str] = set()
files_since_report = 0 files_since_report = 0
walk_start = walk_path or root_path
def _on_walk_error(error: OSError) -> None:
"""Record a directory the walk could not enter (offline/denied)."""
filename = getattr(error, 'filename', None)
if filename and unreachable is not None:
unreachable.add(str(filename), 'unreadable_dir')
def mark_stale_if_needed(cached_path: str) -> None: def mark_stale_if_needed(cached_path: str) -> None:
"""Queue a cached path for file_name repair when it drifted.""" """Queue a cached path for file_name repair when it drifted."""
@@ -395,7 +590,9 @@ def _walk_root_for_reconcile(
stale_seen.add(cached_path) stale_seen.add(cached_path)
result.stale_paths.append(cached_path) result.stale_paths.append(cached_path)
for root, dirnames, files in os.walk(root_path, followlinks=True): for root, dirnames, files in os.walk(
walk_start, followlinks=True, onerror=_on_walk_error
):
dirnames[:] = [d for d in dirnames if not _is_excluded_dir(d)] dirnames[:] = [d for d in dirnames if not _is_excluded_dir(d)]
real_root = os.path.realpath(root) real_root = os.path.realpath(root)
@@ -554,6 +751,8 @@ class ModelScanner:
self._defer_persist_depth = 0 self._defer_persist_depth = 0
self._deferred_persist_pending = False self._deferred_persist_pending = False
self._autov3_backfill_scheduled = False # One-time AutoV3 backfill trigger per process self._autov3_backfill_scheduled = False # One-time AutoV3 backfill trigger per process
# Summary of the last incremental reconcile (see last_reconcile_summary)
self._last_reconcile_summary: Optional[Dict[str, Any]] = None
# Guard against concurrent all-folders backfill walks (cold fallback # Guard against concurrent all-folders backfill walks (cold fallback
# for persisted snapshots that predate folder recording). # for persisted snapshots that predate folder recording).
self._all_folders_backfill_running = False self._all_folders_backfill_running = False
@@ -1334,12 +1533,19 @@ class ModelScanner:
asyncio.set_event_loop(None) asyncio.set_event_loop(None)
loop.close() loop.close()
async def get_cached_data(self, force_refresh: bool = False, rebuild_cache: bool = False) -> ModelCache: async def get_cached_data(
self,
force_refresh: bool = False,
rebuild_cache: bool = False,
scope: Optional[ReconcileScope] = None,
) -> ModelCache:
"""Get cached model data, refresh if needed """Get cached model data, refresh if needed
Args: Args:
force_refresh: Whether to refresh the cache force_refresh: Whether to refresh the cache
rebuild_cache: Whether to completely rebuild the cache rebuild_cache: Whether to completely rebuild the cache
scope: Restrict an incremental reconcile to some roots / a folder
(ignored by a full rebuild, which always walks everything)
""" """
# If cache is not initialized, return an empty cache # If cache is not initialized, return an empty cache
# Actual initialization should be done via initialize_in_background # Actual initialization should be done via initialize_in_background
@@ -1353,12 +1559,55 @@ class ModelScanner:
# If force refresh is requested, initialize the cache directly # If force refresh is requested, initialize the cache directly
if force_refresh: if force_refresh:
if rebuild_cache: if rebuild_cache:
self._last_reconcile_summary = None
await self._initialize_cache() await self._initialize_cache()
else: else:
await self._reconcile_cache() self._last_reconcile_summary = await self._reconcile_cache(scope=scope)
return cast(ModelCache, self._cache) return cast(ModelCache, self._cache)
@property
def last_reconcile_summary(self) -> Optional[Dict[str, Any]]:
"""Summary of the most recent incremental reconcile (None otherwise).
Carries the counts the refresh toast reports (added / removed /
repaired) plus the scan scope and the paths that were left untouched
because they were unreachable.
"""
return self._last_reconcile_summary
def describe_model_roots(self) -> List[Dict[str, Any]]:
"""Describe the configured roots for the refresh scope menu.
``models`` is the cached entry count per root (the same attribution the
walk-progress weights use) and ``reachable`` is a live filesystem check,
so a switched-off drive shows as offline without touching the cache.
"""
roots: List[str] = []
seen: Set[str] = set()
for root in self.get_model_roots():
if not root or root in seen:
continue
seen.add(root)
roots.append(root)
labels = _root_display_labels(roots)
cached_paths = (
{item.get('file_path', '') for item in self._cache.raw_data}
if self._cache is not None
else set()
)
counts = _count_cached_entries_per_root(cached_paths, roots)
return [
{
'path': root.replace(os.sep, '/'),
'label': labels[root],
'reachable': os.path.exists(root),
'models': counts.get(root, 0),
}
for root in roots
]
async def _initialize_cache(self) -> None: async def _initialize_cache(self) -> None:
"""Initialize or refresh the cache""" """Initialize or refresh the cache"""
self._is_initializing = True # Set flag self._is_initializing = True # Set flag
@@ -1450,8 +1699,15 @@ class ModelScanner:
finally: finally:
self._is_initializing = False # Unset flag self._is_initializing = False # Unset flag
async def _reconcile_cache(self) -> None: async def _reconcile_cache(self, scope: Optional[ReconcileScope] = None) -> Optional[Dict[str, Any]]:
"""Fast cache reconciliation - only process differences between cache and filesystem""" """Fast cache reconciliation - only process differences between cache and filesystem.
``scope`` restricts the reconcile to some of the configured roots and/or
a folder inside them: entries outside the scope are neither re-read nor
removed, so scanning one drive can never touch another. Paths the walk
could not read are kept and reported instead of being treated as
deleted. Returns a summary of the run (also broadcast to the UI).
"""
self.reset_cancellation() self.reset_cancellation()
self._is_initializing = True # Set flag for reconciliation duration self._is_initializing = True # Set flag for reconciliation duration
try: try:
@@ -1465,30 +1721,69 @@ class ModelScanner:
cached_paths = {item['file_path'] for item in self._cache.raw_data} cached_paths = {item['file_path'] for item in self._cache.raw_data}
path_to_item = {item['file_path']: item for item in self._cache.raw_data} path_to_item = {item['file_path']: item for item in self._cache.raw_data}
# Every configured root that is currently reachable. A root that is # Snapshot the configured roots once for this phase: scope
# missing (drive switched off, unmounted share) is skipped, so its # resolution, reachability, symlink relevance and the new-file pass
# cached entries are reported as missing below. # all read from this list.
roots: List[str] = [] configured_roots: List[str] = []
seen_roots: Set[str] = set() seen_roots: Set[str] = set()
for root_path in self.get_model_roots(): for root_path in self.get_model_roots():
if not root_path or root_path in seen_roots: if not root_path or root_path in seen_roots:
continue continue
if not os.path.exists(root_path):
continue
seen_roots.add(root_path) seen_roots.add(root_path)
configured_roots.append(root_path)
all_labels = _root_display_labels(configured_roots)
scope_roots = [
root for root in configured_roots
if scope is None or scope.roots is None or root in scope.roots
]
scope_folder = scope.folder if scope is not None else None
# A configured root that is not reachable (drive switched off,
# unmounted share) is skipped instead of being treated as deleted.
unreachable = _UnreachablePaths()
roots: List[str] = []
skipped_roots: List[Dict[str, Any]] = []
for root_path in scope_roots:
if not os.path.exists(root_path):
unreachable.add(root_path, 'root_unreachable')
skipped_roots.append({
'path': root_path.replace(os.sep, '/'),
'label': all_labels.get(root_path, root_path),
'reason': 'root_unreachable',
})
continue
roots.append(root_path) roots.append(root_path)
self._collect_offline_symlink_prefixes(roots, unreachable)
# Scope prefixes come from every root the caller asked for (even the
# unreachable ones) so their entries are recognised as "in scope but
# unreadable" and reported instead of silently disappearing.
scope_prefixes = [
(root, _scope_prefix(root, scope_folder)) for root in scope_roots
]
walk_prefixes = [
(root, _scope_prefix(root, scope_folder)) for root in roots
]
# Roots on different devices are walked by parallel workers (a cold # Roots on different devices are walked by parallel workers (a cold
# or slow drive then no longer serializes the others); roots sharing # or slow drive then no longer serializes the others); roots sharing
# a device stay sequential so directory claims remain deterministic. # a device stay sequential so directory claims remain deterministic.
tracker = _ReconcileWalkTracker( tracker = _ReconcileWalkTracker(
roots, _count_cached_entries_per_root(cached_paths, roots) roots,
_count_cached_entries_for_prefixes(cached_paths, walk_prefixes),
labels={root: all_labels.get(root, root) for root in roots},
) )
walk_results = await self._walk_roots_for_reconcile( walk_results = await self._walk_roots_for_reconcile(
roots=roots, roots=roots,
walk_targets={
root: _scope_walk_path(root, scope_folder) for root in roots
},
tracker=tracker, tracker=tracker,
cached_paths=cached_paths, cached_paths=cached_paths,
path_to_item=path_to_item, path_to_item=path_to_item,
unreachable=unreachable,
) )
# Final walk snapshot: the bar reaches the walk share, then the # Final walk snapshot: the bar reaches the walk share, then the
@@ -1675,8 +1970,21 @@ class ModelScanner:
exc, exc,
) )
# Find missing files (in cache but not in filesystem) # Find missing files (in cache but not in filesystem). Only paths
missing_files = cached_paths - found_paths # inside this scan's scope are candidates, and paths the walk could
# not read are kept and reported instead of being removed.
missing_files: Set[str] = set()
for path in cached_paths - found_paths:
if not any(
_path_matches_prefix(path, prefix)
for _root, prefix in scope_prefixes
):
continue
unreachable_prefix = unreachable.match(path)
if unreachable_prefix is not None:
unreachable.record_kept(unreachable_prefix)
continue
missing_files.add(path)
total_removed = 0 total_removed = 0
if missing_files: if missing_files:
@@ -1732,13 +2040,28 @@ class ModelScanner:
self._cache.raw_data = list(reversed(deduped)) self._cache.raw_data = list(reversed(deduped))
total_removed += dedup_removed total_removed += dedup_removed
# The walk above visited every directory, so refresh the recorded # The walk above visited every directory *in scope*, so refresh the
# folder list (including empty folders) even when no model files # recorded folder list (including empty folders) when nothing was
# changed — e.g. an empty folder was created or removed externally. # left unverified — e.g. an empty folder was created or removed
# externally. A scoped scan, or one with unreadable paths, can only
# add to the list: dropping folders this walk never looked at would
# empty the sidebar for every drive that was not scanned.
sorted_discovered = sorted(discovered_folders, key=lambda x: x.lower()) sorted_discovered = sorted(discovered_folders, key=lambda x: x.lower())
folders_changed = self._cache.all_folders != sorted_discovered fully_verified = (
len(roots) == len(configured_roots)
and scope_folder is None
and not unreachable.has_entries()
)
if fully_verified:
merged_folders = sorted_discovered
else:
merged_folders = sorted(
set(self._cache.all_folders or []) | set(sorted_discovered),
key=lambda x: x.lower(),
)
folders_changed = self._cache.all_folders != merged_folders
if folders_changed: if folders_changed:
self._cache.all_folders = sorted_discovered self._cache.all_folders = merged_folders
# Resort cache if changes were made # Resort cache if changes were made
if total_added > 0 or total_removed > 0: if total_added > 0 or total_removed > 0:
@@ -1755,16 +2078,33 @@ class ModelScanner:
elif folders_changed: elif folders_changed:
await self._persist_current_cache() await self._persist_current_cache()
summary: Dict[str, Any] = {
'added': total_added,
'removed': total_removed,
'repaired': total_repaired,
'scanned_roots': [all_labels.get(root, root) for root in roots],
'skipped_roots': skipped_roots,
'unavailable_paths': unreachable.payload(),
'unavailable_paths_total': unreachable.count(),
'kept_unreachable': unreachable.total_kept(),
}
logger.info( logger.info(
f"{self.model_type.capitalize()} Scanner: Cache reconciliation completed in " f"{self.model_type.capitalize()} Scanner: Cache reconciliation completed in "
f"{time.time() - start_time:.2f} seconds. Added {total_added}, " f"{time.time() - start_time:.2f} seconds. Added {total_added}, "
f"removed {total_removed}, repaired {total_repaired} models." f"removed {total_removed}, repaired {total_repaired} models."
+ (
f" Skipped {len(skipped_roots)} unreachable root(s), kept "
f"{summary['kept_unreachable']} model(s) under unreadable paths."
if skipped_roots or unreachable.has_entries()
else ""
)
) )
await self._broadcast_scan_progress( await self._broadcast_scan_progress(
'completed', 'process_new', 100, False, 'completed', 'process_new', 100, False,
added=total_added, removed=total_removed,
elapsed_seconds=time.time() - start_time, elapsed_seconds=time.time() - start_time,
**summary,
) )
return summary
except Exception as e: except Exception as e:
logger.error(f"{self.model_type.capitalize()} Scanner: Error reconciling cache: {e}", exc_info=True) logger.error(f"{self.model_type.capitalize()} Scanner: Error reconciling cache: {e}", exc_info=True)
await self._broadcast_scan_progress( await self._broadcast_scan_progress(
@@ -1779,9 +2119,11 @@ class ModelScanner:
self, self,
*, *,
roots: List[str], roots: List[str],
walk_targets: Mapping[str, str],
tracker: _ReconcileWalkTracker, tracker: _ReconcileWalkTracker,
cached_paths: Set[str], cached_paths: Set[str],
path_to_item: Mapping[str, Dict[str, Any]], path_to_item: Mapping[str, Dict[str, Any]],
unreachable: _UnreachablePaths,
) -> List[_RootWalkResult]: ) -> List[_RootWalkResult]:
"""Walk every root off the event loop and return results in root order. """Walk every root off the event loop and return results in root order.
@@ -1810,6 +2152,8 @@ class ModelScanner:
dir_claims, dir_claims,
excluded_models, excluded_models,
path_to_item, path_to_item,
walk_targets,
unreachable,
) )
for group in _group_roots_by_device(roots) for group in _group_roots_by_device(roots)
] ]
@@ -1837,6 +2181,8 @@ class ModelScanner:
dir_claims: _RealDirClaims, dir_claims: _RealDirClaims,
excluded_models: Set[str], excluded_models: Set[str],
path_to_item: Mapping[str, Dict[str, Any]], path_to_item: Mapping[str, Dict[str, Any]],
walk_targets: Mapping[str, str],
unreachable: _UnreachablePaths,
) -> List[_RootWalkResult]: ) -> List[_RootWalkResult]:
"""Walk the roots of one device sequentially (worker-thread entry point).""" """Walk the roots of one device sequentially (worker-thread entry point)."""
results: List[_RootWalkResult] = [] results: List[_RootWalkResult] = []
@@ -1852,6 +2198,7 @@ class ModelScanner:
try: try:
result = _walk_root_for_reconcile( result = _walk_root_for_reconcile(
root_path=root_path, root_path=root_path,
walk_path=walk_targets.get(root_path, root_path),
file_extensions=self.file_extensions, file_extensions=self.file_extensions,
cached_paths=lookups.cached_paths, cached_paths=lookups.cached_paths,
path_to_item=path_to_item, path_to_item=path_to_item,
@@ -1860,6 +2207,7 @@ class ModelScanner:
excluded_models=excluded_models, excluded_models=excluded_models,
is_cancelled=self.is_cancelled, is_cancelled=self.is_cancelled,
report_progress=report_progress, report_progress=report_progress,
unreachable=unreachable,
) )
finally: finally:
tracker.finish_root(root_path) tracker.finish_root(root_path)
@@ -1869,6 +2217,41 @@ class ModelScanner:
break break
return results return results
def _collect_offline_symlink_prefixes(
self, roots: Sequence[str], unreachable: _UnreachablePaths
) -> None:
"""Flag known first-level symlinks whose target is not a directory.
``config`` deliberately tracks only symlinks directly under a root (see
``Config._scan_symbolic_links``), so this costs one ``isdir`` per known
mapping and covers "the linked drive is switched off" for the layouts the
scanner already knows about. Nested symlinks stay out of scope.
"""
try:
mappings = config.iter_path_mappings()
except Exception as exc: # pragma: no cover - defensive logging
logger.debug(
"%s Scanner: symlink map unavailable: %s",
self.model_type.capitalize(),
exc,
)
return
if not mappings or not roots:
return
root_prefixes = [_normalized_root_prefix(root) for root in roots]
for target, link in mappings:
link_prefix = _normalized_root_prefix(link)
if not any(
link_prefix == root_prefix
or _path_matches_prefix(link_prefix, root_prefix)
for root_prefix in root_prefixes
):
continue
if os.path.isdir(target):
continue
unreachable.add(link, 'symlink_target_offline')
async def _monitor_walk_progress(self, tracker: _ReconcileWalkTracker) -> None: async def _monitor_walk_progress(self, tracker: _ReconcileWalkTracker) -> None:
"""Broadcast walk progress while the root workers are running.""" """Broadcast walk progress while the root workers are running."""
while True: while True:
+45
View File
@@ -700,3 +700,48 @@ body.sticky-controls .sticky-topbar {
padding: 10px 0; padding: 10px 0;
} }
} }
/* Refresh dropdown: per-root scan scope (issue #1108). The list can hold one
entry per configured model root, so it scrolls instead of growing past the
viewport. Offline roots stay clickable (clicking explains the state) and are
only visually de-emphasised — unlike .dropdown-item.disabled, which swallows
the click via pointer-events: none. */
.dropdown-separator {
height: 1px;
margin: 5px 0;
background-color: var(--border-color);
}
.dropdown-section-title {
padding: 4px 15px 2px;
font-size: 0.85em;
font-weight: 600;
color: var(--text-color);
opacity: 0.6;
}
.dropdown-scope-list {
max-height: 45vh;
overflow-y: auto;
}
.scan-root-item.is-offline {
cursor: default;
opacity: 0.6;
}
.scan-root-label {
margin-right: 12px;
}
.scan-root-count,
.scan-root-offline {
float: right;
margin-left: 16px;
font-size: 0.9em;
opacity: 0.7;
}
.scan-root-item.is-offline .scan-root-offline {
opacity: 0.9;
}
+59 -2
View File
@@ -511,8 +511,9 @@ export class BaseModelApiClient {
} }
} }
async refreshModels(fullRebuild = false) { async refreshModels(fullRebuild = false, { roots = null } = {}) {
const abortController = new AbortController(); const abortController = new AbortController();
const scopeRoots = Array.isArray(roots) ? roots.filter(Boolean) : [];
const displayName = this.apiConfig.config.displayName; const displayName = this.apiConfig.config.displayName;
const singularName = this.apiConfig.config.singularName; const singularName = this.apiConfig.config.singularName;
const actionText = translate( const actionText = translate(
@@ -600,6 +601,9 @@ export class BaseModelApiClient {
const url = new URL(this.apiConfig.endpoints.scan, window.location.origin); const url = new URL(this.apiConfig.endpoints.scan, window.location.origin);
url.searchParams.append('full_rebuild', fullRebuild); url.searchParams.append('full_rebuild', fullRebuild);
for (const root of scopeRoots) {
url.searchParams.append('roots', root);
}
const response = await fetch(url, { signal: abortController.signal }); const response = await fetch(url, { signal: abortController.signal });
@@ -615,7 +619,7 @@ export class BaseModelApiClient {
resetAndReload(true); resetAndReload(true);
showToast('toast.api.refreshComplete', { action: actionText }, 'success'); this._showRefreshSummary(data, actionText, scopeRoots);
} catch (error) { } catch (error) {
if (error.name === 'AbortError') { if (error.name === 'AbortError') {
showToast('toast.api.operationCancelled', {}, 'info'); showToast('toast.api.operationCancelled', {}, 'info');
@@ -632,6 +636,59 @@ export class BaseModelApiClient {
} }
} }
/**
* Report what a finished scan did.
*
* A scoped scan names the folder it scanned and how many models changed; a
* scan that could not read part of its scope says so instead of silently
* looking like "nothing found" (the entries are kept, not deleted).
* @param {Object} summary - Scan response payload
* @param {string} actionText - Localized "Refresh" / "Full rebuild"
* @param {Array<string>} scopeRoots - Roots the scan was restricted to
*/
_showRefreshSummary(summary, actionText, scopeRoots = []) {
const payload = summary || {};
const scannedRoots = Array.isArray(payload.scanned_roots) ? payload.scanned_roots : [];
const scopeLabel = scannedRoots.length ? scannedRoots.join(', ') : '';
if (scopeRoots.length && scopeLabel) {
showToast(
'toast.api.refreshCompleteScoped',
{
scope: scopeLabel,
added: Number(payload.added || 0),
removed: Number(payload.removed || 0),
},
'success'
);
} else {
showToast('toast.api.refreshComplete', { action: actionText }, 'success');
}
const keptCount = Number(payload.kept_unreachable || 0);
if (keptCount > 0) {
const unavailable = Array.isArray(payload.unavailable_paths)
? payload.unavailable_paths
: [];
const skipped = Array.isArray(payload.skipped_roots) ? payload.skipped_roots : [];
// Skipped roots already carry a short label; the remaining
// unreadable folders only have a path.
const names = (
skipped.length
? skipped.map(entry => entry?.label || entry?.path)
: unavailable.map(entry => entry?.path)
).filter(Boolean);
showToast(
'toast.api.refreshKeptUnreachable',
{
count: keptCount.toLocaleString(),
paths: names.slice(0, 3).join(', ') || '—',
},
'info'
);
}
}
/** /**
* Connect to the shared fetch-progress WebSocket for scan progress updates. * Connect to the shared fetch-progress WebSocket for scan progress updates.
* Returns null when the connection cannot be established (silent fallback). * Returns null when the connection cannot be established (silent fallback).
@@ -33,9 +33,6 @@ export class CheckpointsControls extends PageControls {
return await resetAndReload(updateFolders); return await resetAndReload(updateFolders);
}, },
refreshModels: async (fullRebuild = false) => {
return await getModelApiClient().refreshModels(fullRebuild);
},
// Add fetch from Civitai functionality for checkpoints // Add fetch from Civitai functionality for checkpoints
fetchFromCivitai: async () => { fetchFromCivitai: async () => {
@@ -30,9 +30,6 @@ export class EmbeddingsControls extends PageControls {
return await resetAndReload(updateFolders); return await resetAndReload(updateFolders);
}, },
refreshModels: async (fullRebuild = false) => {
return await getModelApiClient().refreshModels(fullRebuild);
},
// Add fetch from Civitai functionality for embeddings // Add fetch from Civitai functionality for embeddings
fetchFromCivitai: async () => { fetchFromCivitai: async () => {
@@ -37,9 +37,6 @@ export class LorasControls extends PageControls {
return await resetAndReload(updateFolders); return await resetAndReload(updateFolders);
}, },
refreshModels: async (fullRebuild = false) => {
return await getModelApiClient().refreshModels(fullRebuild);
},
// LoRA-specific API functions // LoRA-specific API functions
fetchFromCivitai: async () => { fetchFromCivitai: async () => {
@@ -31,9 +31,6 @@ export class OtherControls extends PageControls {
return await resetAndReload(updateFolders); return await resetAndReload(updateFolders);
}, },
refreshModels: async (fullRebuild = false) => {
return await getModelApiClient().refreshModels(fullRebuild);
},
// Add fetch from Civitai functionality for other models // Add fetch from Civitai functionality for other models
fetchFromCivitai: async () => { fetchFromCivitai: async () => {
+93 -4
View File
@@ -5,7 +5,9 @@ import { showToast, openCivitaiByMetadata, isTypingContext } from '../../utils/u
import { eventManager } from '../../utils/EventManager.js'; import { eventManager } from '../../utils/EventManager.js';
import { performModelUpdateCheck } from '../../utils/updateCheckHelpers.js'; import { performModelUpdateCheck } from '../../utils/updateCheckHelpers.js';
import { sidebarManager } from '../SidebarManager.js'; import { sidebarManager } from '../SidebarManager.js';
import { getModelApiClient } from '../../api/modelApiFactory.js';
import { initSortDropdown, applySortToSelect, randomizeSortValue } from './SortDropdown.js'; import { initSortDropdown, applySortToSelect, randomizeSortValue } from './SortDropdown.js';
import { renderScanScopeMenu, resolveScanScopeTarget } from './ScanScopeMenu.js';
/** /**
* PageControls class - Unified control management for model pages * PageControls class - Unified control management for model pages
@@ -82,7 +84,18 @@ export class PageControls {
* @param {Object} api - API methods for the page * @param {Object} api - API methods for the page
*/ */
registerAPI(api) { registerAPI(api) {
this.api = api; this.api = {
...api,
// These two are identical for every page and must stay in sync with
// the scan API: the refresh scope menu reads the root list, and a
// scoped refresh has to pass `roots` through. They are defined here
// rather than in each page facade so a page cannot silently drop
// them (which is exactly how the scope menu ended up empty).
fetchModelRoots: async () => getModelApiClient().fetchModelRoots(),
// Forward the exact arguments: an unscoped refresh must keep calling
// the client with a single argument.
refreshModels: async (...args) => getModelApiClient().refreshModels(...args),
};
console.log(`API methods registered for ${this.pageType} page`); console.log(`API methods registered for ${this.pageType} page`);
// Initialize sidebar manager after API is registered // Initialize sidebar manager after API is registered
@@ -233,7 +246,10 @@ export class PageControls {
}); });
// Toggle current dropdown // Toggle current dropdown
dropdownGroup.classList.toggle('active'); const isOpen = dropdownGroup.classList.toggle('active');
if (isOpen) {
this.loadScanScopeMenu();
}
}); });
}); });
@@ -248,6 +264,21 @@ export class PageControls {
}); });
} }
// Per-root scan rows are rendered dynamically, so they are wired by
// delegation instead of a direct querySelector per item.
const scopeMenu = document.getElementById('refreshScopeMenu');
if (scopeMenu) {
scopeMenu.addEventListener('click', (e) => {
const item = e.target.closest('[data-action="scan-root"]');
if (!item) {
return;
}
e.stopPropagation();
this.handleScanRootSelection(item);
document.querySelector('.dropdown-group.active')?.classList.remove('active');
});
}
const checkUpdatesOption = document.getElementById('checkUpdatesMenuItem'); const checkUpdatesOption = document.getElementById('checkUpdatesMenuItem');
if (checkUpdatesOption) { if (checkUpdatesOption) {
checkUpdatesOption.addEventListener('click', async (e) => { checkUpdatesOption.addEventListener('click', async (e) => {
@@ -266,6 +297,55 @@ export class PageControls {
}); });
} }
/**
* Render the per-root scan entries of the refresh dropdown.
*
* Roots come from the backend (label, model count, reachability) so the menu
* always names a root exactly like the scan progress line does. The list is
* cached until the next scan finishes.
*/
async loadScanScopeMenu() {
const menu = document.getElementById('refreshScopeMenu');
if (!menu || typeof this.api?.fetchModelRoots !== 'function') {
return;
}
if (this._scanScopeDetails) {
this.renderScanScopeMenu(this._scanScopeDetails);
return;
}
if (this._scanScopeLoading) {
return;
}
this._scanScopeLoading = true;
try {
const payload = await this.api.fetchModelRoots();
const details = Array.isArray(payload?.root_details) ? payload.root_details : [];
this._scanScopeDetails = details;
renderScanScopeMenu(menu, details);
} catch (error) {
console.error('Error loading model roots:', error);
menu.innerHTML = '';
} finally {
this._scanScopeLoading = false;
}
}
handleScanRootSelection(item) {
const { rootPath, label, offline } = resolveScanScopeTarget(item);
if (!rootPath) {
return;
}
if (offline) {
// showToast(key, params, type): the sentence must NOT be passed here,
// it used to land in the `type` slot and produced an unstyled toast.
showToast('toast.api.scanRootUnreachable', { scope: label }, 'info');
return;
}
this.refreshModels(false, { roots: [rootPath] });
}
async handleCheckModelUpdates(menuItem) { async handleCheckModelUpdates(menuItem) {
if (this._updateCheckInProgress) { if (this._updateCheckInProgress) {
return; return;
@@ -482,14 +562,23 @@ export class PageControls {
* Refresh models list * Refresh models list
* @param {boolean} fullRebuild - Whether to perform a full rebuild * @param {boolean} fullRebuild - Whether to perform a full rebuild
*/ */
async refreshModels(fullRebuild = false) { async refreshModels(fullRebuild = false, { roots = null } = {}) {
if (!this.api) { if (!this.api) {
console.error('API methods not registered'); console.error('API methods not registered');
return; return;
} }
// Root labels / reachability may have changed with the scan, and the
// model counts definitely did.
this._scanScopeDetails = null;
const scopedRoots = Array.isArray(roots) ? roots.filter(Boolean) : [];
try { try {
await this.api.refreshModels(fullRebuild); if (scopedRoots.length) {
await this.api.refreshModels(fullRebuild, { roots: scopedRoots });
} else {
await this.api.refreshModels(fullRebuild);
}
// Refresh sidebar after rebuild // Refresh sidebar after rebuild
if (this.sidebarManager) { if (this.sidebarManager) {
@@ -0,0 +1,72 @@
// ScanScopeMenu.js - per-root entries of the Refresh dropdown (issue #1108)
import { translate } from '../../utils/i18nHelpers.js';
/**
* Render one row per configured model root.
*
* Rows carry the root path in `data-root` and are wired by delegation, because
* the list is rebuilt every time the root set changes. An offline root stays
* clickable (the click explains why nothing can be scanned) and is only
* de-emphasised visually.
*
* @param {HTMLElement|null} menu - Container element (#refreshScopeMenu)
* @param {Array<Object>} details - `root_details` from GET /api/lm/{type}/roots
*/
export function renderScanScopeMenu(menu, details = []) {
if (!menu) {
return;
}
menu.innerHTML = '';
details.forEach(detail => {
if (!detail || !detail.path) {
return;
}
const item = document.createElement('div');
item.className = `dropdown-item scan-root-item${detail.reachable ? '' : ' is-offline'}`;
item.dataset.action = 'scan-root';
item.dataset.root = detail.path;
item.title = detail.path;
const icon = document.createElement('i');
icon.className = `fas ${detail.reachable ? 'fa-folder-open' : 'fa-plug-circle-xmark'}`;
item.appendChild(icon);
const label = document.createElement('span');
label.className = 'scan-root-label';
label.textContent = detail.label || detail.path;
item.appendChild(label);
const meta = document.createElement('span');
if (detail.reachable) {
const count = Number(detail.models || 0).toLocaleString();
meta.className = 'scan-root-count';
meta.textContent = translate(
'loras.controls.refresh.rootModels',
{ count },
`${count} models`
);
} else {
meta.className = 'scan-root-offline';
meta.textContent = translate('loras.controls.refresh.rootOffline', {}, 'Offline');
}
item.appendChild(meta);
menu.appendChild(item);
});
}
/**
* Read what a clicked row refers to.
* @param {HTMLElement} item - Row rendered by renderScanScopeMenu
* @returns {{rootPath: string, label: string, offline: boolean}}
*/
export function resolveScanScopeTarget(item) {
const rootPath = item?.dataset?.root || '';
const label = item?.querySelector?.('.scan-root-label')?.textContent || rootPath;
return {
rootPath,
label,
offline: Boolean(item?.classList?.contains('is-offline')),
};
}
+6 -1
View File
@@ -139,9 +139,14 @@ export async function copyToClipboard(text, successMessage = null) {
* @param {string} type - Toast type (info/success/warning/error) * @param {string} type - Toast type (info/success/warning/error)
* @returns {HTMLElement} The toast element (not yet attached to the DOM) * @returns {HTMLElement} The toast element (not yet attached to the DOM)
*/ */
// Only these carry styling (icon + accent border). An unknown type is coerced
// to `info` instead of ending up as a bare `.toast` box — a wrong argument
// order at a call site used to leak the message itself into the class list.
const TOAST_TYPES = new Set(['info', 'success', 'warning', 'error', 'copy']);
function createToastElement(message, type) { function createToastElement(message, type) {
const toast = document.createElement('div'); const toast = document.createElement('div');
toast.className = `toast toast-${type}`; toast.className = `toast toast-${TOAST_TYPES.has(type) ? type : 'info'}`;
toast.textContent = message; toast.textContent = message;
return toast; return toast;
} }
+6
View File
@@ -73,6 +73,12 @@
<div class="dropdown-item" data-action="full-rebuild" title="{% if page_id == 'recipes' %}{{ t('recipes.controls.refresh.fullTooltip', default='Rebuild cache - full rescan of all recipe files') }}{% else %}{{ t('loras.controls.refresh.fullTooltip') }}{% endif %}"> <div class="dropdown-item" data-action="full-rebuild" title="{% if page_id == 'recipes' %}{{ t('recipes.controls.refresh.fullTooltip', default='Rebuild cache - full rescan of all recipe files') }}{% else %}{{ t('loras.controls.refresh.fullTooltip') }}{% endif %}">
<i class="fas fa-tools"></i> <span>{% if page_id == 'recipes' %}{{ t('loras.controls.refresh.full', default='Rebuild Cache') }}{% else %}{{ t('loras.controls.refresh.full') }}{% endif %}</span> <i class="fas fa-tools"></i> <span>{% if page_id == 'recipes' %}{{ t('loras.controls.refresh.full', default='Rebuild Cache') }}{% else %}{{ t('loras.controls.refresh.full') }}{% endif %}</span>
</div> </div>
{% if page_id != 'recipes' %}
<!-- Per-root scan scope, filled by PageControls.loadScanScopeMenu() -->
<div class="dropdown-separator"></div>
<div class="dropdown-section-title">{{ t('loras.controls.refresh.scopeSection') }}</div>
<div id="refreshScopeMenu" class="dropdown-scope-list"></div>
{% endif %}
</div> </div>
</div> </div>
@@ -153,8 +153,8 @@ describe('BaseModelApiClient.refreshModels scan progress', () => {
}; };
} }
async function startRefresh(client, fullRebuild = false) { async function startRefresh(client, fullRebuild = false, options = {}) {
const promise = client.refreshModels(fullRebuild); const promise = client.refreshModels(fullRebuild, options);
await vi.waitFor(() => { await vi.waitFor(() => {
expect(FakeWebSocket.instances.length).toBe(1); expect(FakeWebSocket.instances.length).toBe(1);
}); });
@@ -392,6 +392,75 @@ describe('BaseModelApiClient.refreshModels scan progress', () => {
await promise; await promise;
}); });
it('requests a scoped scan when roots are passed', async () => {
global.fetch = vi.fn().mockResolvedValue({
ok: true,
json: async () => ({ status: 'success' }),
});
const client = await createClient();
await client.refreshModels(false, { roots: ['/mnt/a/loras', '/mnt/b/loras'] });
const [url] = global.fetch.mock.calls[0];
expect(url.searchParams.getAll('roots')).toEqual(['/mnt/a/loras', '/mnt/b/loras']);
expect(url.searchParams.get('full_rebuild')).toBe('false');
});
it('reports the scoped scan summary and the entries kept unreachable', async () => {
const fetchControl = mockFetchPending();
const client = await createClient();
const { promise } = await startRefresh(client, false, { roots: ['/mnt/a/loras'] });
fetchControl.resolveOk({
status: 'success',
scanned_roots: ['a/loras'],
added: 2,
removed: 1,
kept_unreachable: 5,
unavailable_paths: [
{ path: '/mnt/g/loras', reason: 'root_unreachable', kept: 5 },
],
skipped_roots: [{ path: '/mnt/g/loras', label: 'g/loras' }],
});
await promise;
expect(showToastMock).toHaveBeenCalledWith(
'toast.api.refreshCompleteScoped',
{ scope: 'a/loras', added: 2, removed: 1 },
'success'
);
expect(showToastMock).toHaveBeenCalledWith(
'toast.api.refreshKeptUnreachable',
{ count: '5', paths: 'g/loras' },
'info'
);
});
it('keeps the generic completion toast for a full-library scan', async () => {
const fetchControl = mockFetchPending();
const client = await createClient();
const { promise } = await startRefresh(client);
fetchControl.resolveOk({
status: 'success',
scanned_roots: ['a/loras', 'b/loras'],
added: 0,
removed: 0,
});
await promise;
expect(showToastMock).toHaveBeenCalledWith(
'toast.api.refreshComplete',
{ action: 'Refresh' },
'success'
);
expect(showToastMock).not.toHaveBeenCalledWith(
'toast.api.refreshCompleteScoped',
expect.anything(),
expect.anything()
);
});
it('shows the cancelled toast when the server reports cancellation', async () => { it('shows the cancelled toast when the server reports cancellation', async () => {
const fetchControl = mockFetchPending(); const fetchControl = mockFetchPending();
const client = await createClient(); const client = await createClient();
@@ -0,0 +1,236 @@
import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest';
import { readFileSync } from 'fs';
import path from 'path';
const {
PAGE_CONTROLS_MODULE,
STATE_MODULE,
STORAGE_MODULE,
UI_HELPERS_MODULE,
EVENT_MANAGER_MODULE,
UPDATE_CHECK_MODULE,
SIDEBAR_MODULE,
SORT_DROPDOWN_MODULE,
API_FACTORY_MODULE,
I18N_MODULE,
} = vi.hoisted(() => ({
PAGE_CONTROLS_MODULE: new URL('../../../../static/js/components/controls/PageControls.js', import.meta.url).pathname,
STATE_MODULE: new URL('../../../../static/js/state/index.js', import.meta.url).pathname,
STORAGE_MODULE: new URL('../../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
UI_HELPERS_MODULE: new URL('../../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
EVENT_MANAGER_MODULE: new URL('../../../../static/js/utils/EventManager.js', import.meta.url).pathname,
UPDATE_CHECK_MODULE: new URL('../../../../static/js/utils/updateCheckHelpers.js', import.meta.url).pathname,
SIDEBAR_MODULE: new URL('../../../../static/js/components/SidebarManager.js', import.meta.url).pathname,
SORT_DROPDOWN_MODULE: new URL('../../../../static/js/components/controls/SortDropdown.js', import.meta.url).pathname,
API_FACTORY_MODULE: new URL('../../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
I18N_MODULE: new URL('../../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
}));
const CONTROLS_TEMPLATE = path.resolve(
__dirname,
'../../../../templates/components/controls.html'
);
const showToastMock = vi.fn();
const modelClient = {
fetchModelRoots: vi.fn(),
refreshModels: vi.fn(),
};
vi.mock(STATE_MODULE, () => ({
state: {},
getCurrentPageState: vi.fn(() => ({ filters: {} })),
setCurrentPageType: vi.fn(),
}));
vi.mock(STORAGE_MODULE, () => ({
getStorageItem: vi.fn(),
setStorageItem: vi.fn(),
removeStorageItem: vi.fn(),
getSessionItem: vi.fn(),
setSessionItem: vi.fn(),
removeSessionItem: vi.fn(),
}));
vi.mock(UI_HELPERS_MODULE, () => ({
showToast: showToastMock,
openCivitaiByMetadata: vi.fn(),
isTypingContext: vi.fn(() => false),
}));
vi.mock(EVENT_MANAGER_MODULE, () => ({
eventManager: { on: vi.fn(), off: vi.fn(), emit: vi.fn() },
}));
vi.mock(UPDATE_CHECK_MODULE, () => ({
performModelUpdateCheck: vi.fn(),
}));
vi.mock(SIDEBAR_MODULE, () => ({
sidebarManager: {
setHostPageControls: vi.fn(),
initialize: vi.fn(async () => {}),
refresh: vi.fn(async () => {}),
},
}));
vi.mock(SORT_DROPDOWN_MODULE, () => ({
initSortDropdown: vi.fn(),
applySortToSelect: vi.fn(),
randomizeSortValue: vi.fn(),
}));
vi.mock(API_FACTORY_MODULE, () => ({
getModelApiClient: () => modelClient,
}));
vi.mock(I18N_MODULE, () => ({
translate: vi.fn((key, params, fallback) => {
if (!fallback) {
return key;
}
return Object.entries(params || {}).reduce(
(text, [name, value]) => text.replaceAll(`{${name}}`, value),
fallback
);
}),
}));
const { PageControls } = await import(PAGE_CONTROLS_MODULE);
const ROOT_DETAILS = [
{ path: '/mnt/a/loras', label: 'a/loras', reachable: true, models: 12 },
{ path: '/mnt/b/loras', label: 'b/loras', reachable: false, models: 3 },
];
function buildControlsDom() {
document.body.innerHTML = `
<div class="dropdown-group">
<button data-action="refresh" class="dropdown-main"></button>
<button class="dropdown-toggle"></button>
<div class="dropdown-menu">
<div class="dropdown-item" data-action="full-rebuild"></div>
<div class="dropdown-separator"></div>
<div class="dropdown-section-title">Scan one folder</div>
<div id="refreshScopeMenu" class="dropdown-scope-list"></div>
</div>
</div>`;
}
function bareControls(api = {}) {
const controls = Object.create(PageControls.prototype);
controls.pageType = 'loras';
controls.api = api;
controls._scanScopeDetails = null;
controls._scanScopeLoading = false;
controls.refreshModels = vi.fn();
controls.sidebarManager = {
setHostPageControls: vi.fn(),
initialize: vi.fn(async () => {}),
};
return controls;
}
describe('PageControls refresh scope menu', () => {
beforeEach(() => {
showToastMock.mockReset();
modelClient.fetchModelRoots.mockReset();
modelClient.refreshModels.mockReset();
modelClient.fetchModelRoots.mockResolvedValue({ root_details: ROOT_DETAILS });
});
afterEach(() => {
document.body.innerHTML = '';
});
it('keeps the scope container in the controls template (non-recipes pages)', () => {
const html = readFileSync(CONTROLS_TEMPLATE, 'utf-8');
const rebuildIndex = html.indexOf('data-action="full-rebuild"');
const scopeIndex = html.indexOf('id="refreshScopeMenu"');
expect(rebuildIndex).toBeGreaterThan(-1);
// Same dropdown, after the rebuild entry.
expect(scopeIndex).toBeGreaterThan(rebuildIndex);
expect(html).toContain('loras.controls.refresh.scopeSection');
// The recipes page has no model roots, so the section is gated.
expect(html.slice(rebuildIndex, scopeIndex)).toContain("{% if page_id != 'recipes' %}");
});
it('registers the scan-scope API for every page through registerAPI', async () => {
const controls = bareControls();
controls.registerAPI({});
expect(typeof controls.api.fetchModelRoots).toBe('function');
expect(typeof controls.api.refreshModels).toBe('function');
await controls.api.fetchModelRoots();
expect(modelClient.fetchModelRoots).toHaveBeenCalled();
// The page facades used to drop the second argument, which silently turned
// a scoped scan into a full refresh.
await controls.api.refreshModels(false, { roots: ['/mnt/a/loras'] });
expect(modelClient.refreshModels).toHaveBeenCalledWith(false, { roots: ['/mnt/a/loras'] });
});
it('renders the root rows when the dropdown is opened', async () => {
buildControlsDom();
const controls = bareControls({
fetchModelRoots: vi.fn(async () => ({ root_details: ROOT_DETAILS })),
refreshModels: vi.fn(),
});
controls.initDropdowns();
document.querySelector('.dropdown-toggle').click();
await vi.waitFor(() => {
expect(document.querySelectorAll('.scan-root-item').length).toBe(2);
});
expect(document.querySelector('.scan-root-item').dataset.root).toBe('/mnt/a/loras');
expect(document.querySelector('.scan-root-count').textContent).toBe('12 models');
expect(document.querySelectorAll('.scan-root-item')[1].classList.contains('is-offline')).toBe(true);
});
it('scans the clicked root only', async () => {
buildControlsDom();
const controls = bareControls({
fetchModelRoots: vi.fn(async () => ({ root_details: ROOT_DETAILS })),
refreshModels: vi.fn(),
});
controls.initDropdowns();
document.querySelector('.dropdown-toggle').click();
await vi.waitFor(() => {
expect(document.querySelectorAll('.scan-root-item').length).toBe(2);
});
document.querySelectorAll('.scan-root-item')[0].click();
expect(controls.refreshModels).toHaveBeenCalledWith(false, { roots: ['/mnt/a/loras'] });
expect(showToastMock).not.toHaveBeenCalled();
});
it('explains an offline root instead of scanning it', async () => {
buildControlsDom();
const controls = bareControls({
fetchModelRoots: vi.fn(async () => ({ root_details: ROOT_DETAILS })),
refreshModels: vi.fn(),
});
controls.initDropdowns();
document.querySelector('.dropdown-toggle').click();
await vi.waitFor(() => {
expect(document.querySelectorAll('.scan-root-item').length).toBe(2);
});
document.querySelectorAll('.scan-root-item')[1].click();
expect(controls.refreshModels).not.toHaveBeenCalled();
// Exactly three arguments: key, params, type. A 4th "fallback" argument
// pushed the sentence into the type slot and rendered an unstyled toast.
expect(showToastMock).toHaveBeenCalledWith(
'toast.api.scanRootUnreachable',
{ scope: 'b/loras' },
'info'
);
});
});
@@ -0,0 +1,77 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
const { I18N_MODULE, MODULE } = vi.hoisted(() => ({
I18N_MODULE: new URL('../../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
MODULE: new URL('../../../../static/js/components/controls/ScanScopeMenu.js', import.meta.url).pathname,
}));
vi.mock(I18N_MODULE, () => ({
translate: vi.fn((key, params, fallback) => {
if (!fallback) {
return key;
}
return Object.entries(params || {}).reduce(
(text, [name, value]) => text.replaceAll(`{${name}}`, value),
fallback
);
}),
}));
const { renderScanScopeMenu, resolveScanScopeTarget } = await import(MODULE);
describe('ScanScopeMenu', () => {
let menu;
beforeEach(() => {
document.body.innerHTML = '<div id="refreshScopeMenu"></div>';
menu = document.getElementById('refreshScopeMenu');
});
it('renders one row per root with label, count and offline state', () => {
renderScanScopeMenu(menu, [
{ path: '/mnt/a/loras', label: 'a/loras', reachable: true, models: 7 },
{ path: '/mnt/b/loras', label: 'b/loras', reachable: false, models: 0 },
]);
const rows = menu.querySelectorAll('.scan-root-item');
expect(rows.length).toBe(2);
expect(rows[0].dataset.action).toBe('scan-root');
expect(rows[0].dataset.root).toBe('/mnt/a/loras');
expect(rows[0].title).toBe('/mnt/a/loras');
expect(rows[0].querySelector('.scan-root-label').textContent).toBe('a/loras');
expect(rows[0].querySelector('.scan-root-count').textContent).toBe('7 models');
expect(rows[0].classList.contains('is-offline')).toBe(false);
expect(rows[1].classList.contains('is-offline')).toBe(true);
expect(rows[1].querySelector('.scan-root-offline').textContent).toBe('Offline');
// Offline rows stay clickable so the click can explain the state.
expect(rows[1].dataset.action).toBe('scan-root');
});
it('resolveScanScopeTarget reads path, label and offline state', () => {
renderScanScopeMenu(menu, [
{ path: '/mnt/b/loras', label: 'b/loras', reachable: false, models: 0 },
]);
expect(resolveScanScopeTarget(menu.querySelector('.scan-root-item'))).toEqual({
rootPath: '/mnt/b/loras',
label: 'b/loras',
offline: true,
});
});
it('skips malformed entries and replaces the previous list', () => {
renderScanScopeMenu(menu, [{ path: '/mnt/a' }, null, {}]);
expect(menu.querySelectorAll('.scan-root-item').length).toBe(1);
renderScanScopeMenu(menu, []);
expect(menu.querySelectorAll('.scan-root-item').length).toBe(0);
});
it('falls back to the path when a root has no label', () => {
renderScanScopeMenu(menu, [{ path: '/mnt/a/loras', reachable: true, models: 0 }]);
expect(menu.querySelector('.scan-root-label').textContent).toBe('/mnt/a/loras');
});
});
+36
View File
@@ -0,0 +1,36 @@
import { describe, it, expect, vi, beforeEach } from 'vitest';
const { UI_HELPERS_MODULE, I18N_MODULE } = vi.hoisted(() => ({
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
I18N_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
}));
vi.mock(I18N_MODULE, () => ({
translate: vi.fn((key, params, fallback) => fallback || key),
}));
const { showToast } = await import(UI_HELPERS_MODULE);
describe('toast type styling', () => {
beforeEach(() => {
document.body.innerHTML = '';
});
it('applies the requested type class', () => {
showToast('plain message', {}, 'success');
expect(document.querySelector('.toast').className).toBe('toast toast-success');
});
it('coerces an unknown type to info instead of rendering an unstyled toast', () => {
// Regression guard: a call site that passed its fallback sentence as the
// `type` produced `toast toast-drive-Z is not reachable ...`, which matched
// no rule at all (no icon, no accent border).
showToast('plain message', {}, 'drive-Z is not reachable right now');
expect(document.querySelector('.toast').className).toBe('toast toast-info');
});
it('defaults to info when no type is given', () => {
showToast('plain message');
expect(document.querySelector('.toast').className).toBe('toast toast-info');
});
});
+123
View File
@@ -196,3 +196,126 @@ async def test_get_folder_tree_threads_include_empty():
assert payload["success"] is True assert payload["success"] is True
assert service.received_include_empty is True assert service.received_include_empty is True
class QueryParams(dict):
"""Minimal stand-in for aiohttp's MultiDict query (supports getall)."""
def getall(self, key, default=None):
value = self.get(key)
if value is None:
return list(default) if default else []
return value if isinstance(value, list) else [value]
class ScopedScanService:
"""Stub service recording the scan scope the handler resolves."""
model_type = "lora"
def __init__(self, roots=None, summary=None, cancelled=False):
self._roots = list(roots or [])
self._summary = summary
self.cancelled = cancelled
self.scanner = SimpleNamespace(is_cancelled=lambda: self.cancelled)
self.received_scope = "not-called"
self.received_rebuild = None
def get_model_roots(self):
return list(self._roots)
def describe_model_roots(self):
return [
{"path": root, "label": root.rsplit("/", 1)[-1], "reachable": True, "models": 7}
for root in self._roots
]
async def scan_models(self, force_refresh=False, rebuild_cache=False, scope=None):
self.received_scope = scope
self.received_rebuild = rebuild_cache
return self._summary
SUMMARY = {
"added": 3,
"removed": 0,
"repaired": 1,
"scanned_roots": ["a"],
"skipped_roots": [],
"unavailable_paths": [],
"unavailable_paths_total": 0,
"kept_unreachable": 0,
}
@pytest.mark.asyncio
async def test_scan_models_accepts_roots_param():
service = ScopedScanService(roots=["/mnt/a", "/mnt/b"], summary=SUMMARY)
handler = ModelQueryHandler(service=service, logger=logging.getLogger(__name__))
response = await handler.scan_models(
SimpleNamespace(query=QueryParams({"roots": ["/mnt/a"]}))
)
payload = json.loads(response.text)
assert response.status == 200
assert payload["status"] == "success"
assert payload["added"] == 3
assert payload["scanned_roots"] == ["a"]
assert service.received_scope is not None
assert service.received_scope.roots == ("/mnt/a",)
assert service.received_rebuild is False
@pytest.mark.asyncio
async def test_scan_models_without_roots_scans_every_root():
service = ScopedScanService(roots=["/mnt/a"], summary=SUMMARY)
handler = ModelQueryHandler(service=service, logger=logging.getLogger(__name__))
await handler.scan_models(SimpleNamespace(query=QueryParams({})))
assert service.received_scope is None
@pytest.mark.asyncio
async def test_scan_models_rejects_unknown_root():
service = ScopedScanService(roots=["/mnt/a"], summary=SUMMARY)
handler = ModelQueryHandler(service=service, logger=logging.getLogger(__name__))
response = await handler.scan_models(
SimpleNamespace(query=QueryParams({"roots": ["/mnt/nope"]}))
)
payload = json.loads(response.text)
assert response.status == 400
assert payload["roots"] == ["/mnt/nope"]
assert service.received_scope == "not-called"
@pytest.mark.asyncio
async def test_scan_models_rejects_roots_with_full_rebuild():
service = ScopedScanService(roots=["/mnt/a"], summary=SUMMARY)
handler = ModelQueryHandler(service=service, logger=logging.getLogger(__name__))
response = await handler.scan_models(
SimpleNamespace(
query=QueryParams({"roots": ["/mnt/a"], "full_rebuild": "true"})
)
)
assert response.status == 400
assert service.received_scope == "not-called"
@pytest.mark.asyncio
async def test_get_model_roots_reports_details():
service = ScopedScanService(roots=["/mnt/a", "/mnt/b"])
handler = ModelQueryHandler(service=service, logger=logging.getLogger(__name__))
response = await handler.get_model_roots(SimpleNamespace(query=QueryParams({})))
payload = json.loads(response.text)
# `roots` stays a plain list of paths for the existing callers.
assert payload["roots"] == ["/mnt/a", "/mnt/b"]
assert [detail["label"] for detail in payload["root_details"]] == ["a", "b"]
assert all(detail["models"] == 7 for detail in payload["root_details"])
+66 -1
View File
@@ -1,7 +1,8 @@
import asyncio
import os import os
import urllib.parse import urllib.parse
from pathlib import Path from pathlib import Path
from unittest.mock import patch from unittest.mock import AsyncMock, patch
import pytest import pytest
from aiohttp import web from aiohttp import web
@@ -348,3 +349,67 @@ async def test_deep_symlink_discovered_on_first_access(tmp_path):
# Deep symlink should now be in mappings # Deep symlink should now be in mappings
assert normalized_external in config._path_mappings assert normalized_external in config._path_mappings
async def test_preview_404_keeps_cache_when_parent_dir_missing(tmp_path):
"""A preview whose directory is unreachable (drive off) must not clear the
cached preview_url: nothing was deleted, the file just cannot be read."""
library_root = tmp_path / "library"
library_root.mkdir()
unreachable_parent = library_root / "gone"
preview_path = unreachable_parent / "model.webp"
config = Config()
config.apply_library_settings(
{
"folder_paths": {
"loras": [str(library_root)],
"checkpoints": [],
"unet": [],
"embeddings": [],
}
}
)
handler = PreviewHandler(config=config)
encoded_path = urllib.parse.quote(str(preview_path), safe="")
request = make_mocked_request("GET", f"/api/lm/previews?path={encoded_path}")
with patch.object(handler, "_cleanup_stale_preview_url", new=AsyncMock()) as cleanup:
with pytest.raises(web.HTTPNotFound):
await handler.serve_preview(request)
await asyncio.sleep(0)
cleanup.assert_not_called()
async def test_preview_404_clears_cache_when_file_really_deleted(tmp_path):
"""A preview deleted from a reachable directory is genuinely stale."""
library_root = tmp_path / "library"
library_root.mkdir()
preview_path = library_root / "model.webp"
preview_path.write_bytes(b"preview")
preview_path.unlink()
config = Config()
config.apply_library_settings(
{
"folder_paths": {
"loras": [str(library_root)],
"checkpoints": [],
"unet": [],
"embeddings": [],
}
}
)
handler = PreviewHandler(config=config)
encoded_path = urllib.parse.quote(str(preview_path), safe="")
request = make_mocked_request("GET", f"/api/lm/previews?path={encoded_path}")
with patch.object(handler, "_cleanup_stale_preview_url", new=AsyncMock()) as cleanup:
with pytest.raises(web.HTTPNotFound):
await handler.serve_preview(request)
await asyncio.sleep(0)
cleanup.assert_awaited_once()
+216 -7
View File
@@ -20,9 +20,11 @@ from py.services.model_hash_index import ModelHashIndex
from py.services.model_scanner import ( from py.services.model_scanner import (
CacheBuildResult, CacheBuildResult,
ModelScanner, ModelScanner,
ReconcileScope,
_ReconcileWalkTracker, _ReconcileWalkTracker,
_count_cached_entries_per_root, _count_cached_entries_per_root,
_group_roots_by_device, _group_roots_by_device,
_root_display_labels,
) )
from py.services.pending_delete_service import ( from py.services.pending_delete_service import (
PENDING_DELETE_DIR_NAME, PENDING_DELETE_DIR_NAME,
@@ -2479,10 +2481,9 @@ async def test_reconcile_walk_broadcasts_per_root_progress(tmp_path: Path, monke
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_reconcile_prunes_entries_of_an_offline_root(tmp_path: Path): async def test_reconcile_keeps_entries_of_an_offline_root(tmp_path: Path):
"""A root that is gone (drive switched off) is skipped by the walk, so its """A configured root that is gone (drive switched off) must not be treated
cached entries are reported as missing. Locked in because it is the as deleted: its entries are kept and reported as unreachable instead."""
documented consequence of refreshing with a drive powered down."""
root_a = tmp_path / "a" root_a = tmp_path / "a"
root_b = tmp_path / "b" root_b = tmp_path / "b"
root_a.mkdir() root_a.mkdir()
@@ -2498,9 +2499,217 @@ async def test_reconcile_prunes_entries_of_an_offline_root(tmp_path: Path):
root_b.rename(tmp_path / "b_offline") root_b.rename(tmp_path / "b_offline")
await scanner._reconcile_cache() summary = await scanner._reconcile_cache()
remaining = {item["file_path"] for item in scanner._cache.raw_data} remaining = {item["file_path"] for item in scanner._cache.raw_data}
assert remaining == {_normalize_path(file_a)} assert remaining == {_normalize_path(file_a), _normalize_path(file_b)}
assert scanner._hash_index.get_path("hash-b0") is None assert scanner._hash_index.get_path("hash-b0") == _normalize_path(file_b)
assert scanner._hash_index.get_path("hash-a0") == _normalize_path(file_a) assert scanner._hash_index.get_path("hash-a0") == _normalize_path(file_a)
assert summary is not None
assert summary["removed"] == 0
assert summary["kept_unreachable"] == 1
assert summary["unavailable_paths_total"] == 1
assert [entry["path"] for entry in summary["unavailable_paths"]] == [
_normalize_path(root_b)
]
assert summary["unavailable_paths"][0]["reason"] == "root_unreachable"
assert [entry["path"] for entry in summary["skipped_roots"]] == [
_normalize_path(root_b)
]
# --- scoped scans (issue #1108) --------------------------------------------
@pytest.mark.asyncio
async def test_reconcile_scoped_scan_leaves_other_roots_untouched(tmp_path: Path):
"""Scanning one root must not add, remove or re-read anything in another."""
root_a = tmp_path / "a"
root_b = tmp_path / "b"
root_a.mkdir()
root_b.mkdir()
(root_a / "a0.txt").write_text("a", encoding="utf-8")
(root_b / "b0.txt").write_text("b", encoding="utf-8")
scanner = MultiRootDummyScanner([root_a, root_b])
await scanner._initialize_cache()
# Changes in B stay invisible while only A is scanned.
(root_b / "b0.txt").unlink()
(root_b / "b_new.txt").write_text("new", encoding="utf-8")
new_a = root_a / "a_new.txt"
new_a.write_text("new", encoding="utf-8")
summary = await scanner._reconcile_cache(scope=ReconcileScope(roots=(str(root_a),)))
assert summary is not None
assert summary["added"] == 1
assert summary["removed"] == 0
assert summary["scanned_roots"] == ["a"]
cached_paths = {item["file_path"] for item in scanner._cache.raw_data}
assert cached_paths == {
_normalize_path(root_a / "a0.txt"),
_normalize_path(new_a),
_normalize_path(root_b / "b0.txt"),
}
@pytest.mark.asyncio
async def test_reconcile_scoped_scan_removes_deleted_files_in_scope(tmp_path: Path):
"""Deletions inside the scanned root still reconcile normally."""
root_a = tmp_path / "a"
root_b = tmp_path / "b"
root_a.mkdir()
root_b.mkdir()
deleted_a = root_a / "a0.txt"
deleted_a.write_text("a", encoding="utf-8")
(root_b / "b0.txt").write_text("b", encoding="utf-8")
scanner = MultiRootDummyScanner([root_a, root_b])
await scanner._initialize_cache()
deleted_a.unlink()
summary = await scanner._reconcile_cache(scope=ReconcileScope(roots=(str(root_a),)))
assert summary is not None
assert summary["removed"] == 1
assert scanner._hash_index.get_path("hash-a0") is None
assert {item["file_path"] for item in scanner._cache.raw_data} == {
_normalize_path(root_b / "b0.txt")
}
@pytest.mark.asyncio
async def test_reconcile_folder_scope_keeps_folder_tree_outside_scope(tmp_path: Path):
"""A folder-scoped scan must not collapse the recorded folder list to the
scanned subtree (the sidebar would lose every other folder)."""
root = tmp_path / "loras"
(root / "alpha").mkdir(parents=True)
(root / "beta").mkdir()
(root / "alpha" / "m.txt").write_text("m", encoding="utf-8")
scanner = DummyScanner(root)
await scanner._initialize_cache()
folders_before = set(scanner._cache.all_folders or [])
assert "beta" in folders_before
(root / "alpha" / "nested").mkdir()
(root / "alpha" / "m.txt").unlink()
summary = await scanner._reconcile_cache(scope=ReconcileScope(folder="alpha"))
assert summary is not None
assert summary["removed"] == 1
folders_after = set(scanner._cache.all_folders or [])
assert "beta" in folders_after
assert "alpha/nested" in folders_after
assert {item["file_path"] for item in scanner._cache.raw_data} == set()
@pytest.mark.asyncio
async def test_reconcile_keeps_entries_under_unreadable_dir(tmp_path: Path, monkeypatch):
"""A directory os.walk cannot enter (permissions, I/O error, offline
junction) keeps its cached entries instead of losing them."""
root = tmp_path / "loras"
sub = root / "sub"
sub.mkdir(parents=True)
(sub / "m.txt").write_text("m", encoding="utf-8")
(root / "top.txt").write_text("t", encoding="utf-8")
scanner = DummyScanner(root)
await scanner._initialize_cache()
real_scandir = os.scandir
def _failing_scandir(path=".", *args, **kwargs):
if os.path.normpath(str(path)) == os.path.normpath(str(sub)):
raise PermissionError(13, "Permission denied", str(sub))
return real_scandir(path, *args, **kwargs)
monkeypatch.setattr(model_scanner.os, "scandir", _failing_scandir)
summary = await scanner._reconcile_cache()
assert summary is not None
assert summary["removed"] == 0
assert summary["kept_unreachable"] == 1
assert summary["unavailable_paths"][0]["reason"] == "unreadable_dir"
assert _normalize_path(sub / "m.txt") in {
item["file_path"] for item in scanner._cache.raw_data
}
@pytest.mark.asyncio
async def test_reconcile_keeps_entries_under_offline_first_level_symlink(
tmp_path: Path, monkeypatch
):
"""A known first-level symlink whose target is gone protects the entries
that live under the link path (the linked drive is switched off)."""
root = tmp_path / "loras"
linked = root / "linked"
linked.mkdir(parents=True)
(linked / "m.txt").write_text("m", encoding="utf-8")
scanner = DummyScanner(root)
await scanner._initialize_cache()
# The linked folder goes away with its drive; config still knows the link.
linked.rename(tmp_path / "linked_offline")
monkeypatch.setattr(
model_scanner.config,
"iter_path_mappings",
lambda: [(str(tmp_path / "gone_target"), str(linked))],
)
summary = await scanner._reconcile_cache()
assert summary is not None
assert summary["removed"] == 0
assert summary["kept_unreachable"] == 1
assert summary["unavailable_paths"][0]["reason"] == "symlink_target_offline"
assert _normalize_path(linked / "m.txt") in {
item["file_path"] for item in scanner._cache.raw_data
}
def test_root_display_labels_dedupe_by_parent_segments(monkeypatch):
labels = _root_display_labels(["/mnt/usb/loras", "/mnt/ssd/loras"])
assert labels == {"/mnt/usb/loras": "usb/loras", "/mnt/ssd/loras": "ssd/loras"}
# Still colliding after one parent segment: keep growing leftwards.
labels = _root_display_labels(["/mnt/a/models/loras", "/mnt/b/models/loras"])
assert labels == {
"/mnt/a/models/loras": "a/models/loras",
"/mnt/b/models/loras": "b/models/loras",
}
# A single root keeps the short form.
assert _root_display_labels(["/mnt/usb/loras"]) == {"/mnt/usb/loras": "loras"}
# Windows drive prefixes disambiguate on their own.
def _fake_splitdrive(path):
for drive in ("G:", "H:"):
if path.startswith(drive):
return drive, path[len(drive):]
return "", path
monkeypatch.setattr(model_scanner.os.path, "splitdrive", _fake_splitdrive)
labels = _root_display_labels(["G:\\x\\loras", "H:\\y\\loras"])
assert labels == {"G:\\x\\loras": "G: loras", "H:\\y\\loras": "H: loras"}
# Identical renderings (trailing separator / duplicated config entry) grow to
# the full path and then fall back to a deterministic suffix instead of
# silently sharing a label.
labels = _root_display_labels(["/mnt/x/loras", "/mnt/x/loras/"])
assert labels == {
"/mnt/x/loras": "mnt/x/loras (1)",
"/mnt/x/loras/": "mnt/x/loras (2)",
}
long_root = "/mnt/" + "d" * 60 + "/loras"
label = _root_display_labels([long_root])[long_root]
assert len(label) <= 40
assert label.endswith("loras")