feat(scanner): scan a single root and keep unreachable entries (#1108)

Refreshing had no way to say "scan only this drive": a user with three external drives had to spin all of them up for every refresh, and switching a drive off made the next refresh treat its whole library as deleted (rows pruned from the memory cache and the SQLite cache, preview_url stripped on the next scroll). Backend (py/services/model_scanner.py, py/config.py): * ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the scope reconcile normally, everything outside is neither re-read nor removed. The folder half is plumbing for the sidebar entry in the next change. * Path-level pruning guard: cached entries under a path this walk could not read are kept and reported instead of removed. Sources: a configured root that is not reachable (drive switched off while LM runs), a directory os.walk failed to enter (permissions / I/O error / Windows junction to an offline drive), and a known first-level symlink whose target is gone (Config.iter_path_mappings()). * The recorded folder list is unioned instead of replaced whenever the scan did not verify every root, so a scoped scan cannot empty the sidebar. * _reconcile_cache returns a summary (added / removed / repaired / scanned_roots / skipped_roots / unavailable_paths / kept_unreachable), exposed as ModelScanner.last_reconcile_summary, returned by BaseModelService.scan_models() and broadcast in the completed WS payload. * _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown leftwards with real parent segments until unique, shared by the walk-progress line and the roots API. * GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with full_rebuild=true); GET /roots gains root_details (label / reachable / cached count) while `roots` stays a plain path list for existing callers. * serve_preview: a 404 no longer clears the cached preview_url when the file's own directory is unreachable - browsing the grid with a drive off used to strip preview references from the persistent cache. Frontend: * Refresh ▾ gains a "Scan one folder" section listing the page's roots with their cached counts; offline roots stay clickable and explain themselves; rows are wired by delegation (new static/js/components/controls/ScanScopeMenu.js). * A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept entries reports "<N> models kept: <paths> not reachable". * registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and an argument-forwarding refreshModels) so a page facade cannot drop them: the first version rendered an empty menu and would have run a full refresh. * createToastElement whitelists toast types, so a wrong `type` argument degrades to the info style instead of rendering an unstyled box. Verified in a sandbox instance with three roots: a scoped scan walks only the requested root (progress roots=0/1, 240 files); a full refresh with one root offline reports kept_unreachable=60 and leaves all 420 models cached; /roots reports the offline root with its cached count. 3699 passed, 7 skipped; frontend 1488 passed (148 files); vue widgets 96 passed.
2026-10-07 18:12:12 -03:00 · 2026-10-07 15:01:26 +08:00
parent 2dcaf6a30e
commit 470d85cca6
34 changed files with 1981 additions and 106 deletions
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "Modelliste aktualisieren",
                "full": "Cache neu aufbauen",
-                "fullTooltip": "Alle Modelldetails aus Metadatendateien neu laden – nutzen, wenn die Bibliothek veraltet wirkt oder nach manuellen Änderungen."
+                "fullTooltip": "Alle Modelldetails aus Metadatendateien neu laden – nutzen, wenn die Bibliothek veraltet wirkt oder nach manuellen Änderungen.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "Metadaten von CivitAI abrufen",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "Fehler beim Herunterladen der Beispielbilder: {message}",
            "moveFailed": "Element konnte nicht verschoben werden: {message}",
            "copiedToClipboard": "In die Zwischenablage kopiert",
-            "downloadStarted": "Download gestartet"
+            "downloadStarted": "Download gestartet",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "KI-Anbieter nicht konfiguriert. Aktivieren Sie ihn unter Einstellungen → KI-Anbieter.",
@@ -918,7 +918,10 @@
      "refresh": {
        "title": "Refresh model list",
        "full": "Rebuild Cache",
-        "fullTooltip": "Reload all model details from metadata files—use if the library looks out of date or after manual edits."
+        "fullTooltip": "Reload all model details from metadata files—use if the library looks out of date or after manual edits.",
+        "scopeSection": "Scan one folder",
+        "rootOffline": "Offline",
+        "rootModels": "{count} models"
      },
      "fetch": {
        "title": "Fetch metadata from CivitAI",
@@ -2804,7 +2807,10 @@
      "exampleImagesDownloadFailed": "Failed to download example images: {message}",
      "moveFailed": "Failed to move item: {message}",
      "copiedToClipboard": "Copied to clipboard",
-      "downloadStarted": "Download started"
+      "downloadStarted": "Download started",
+      "refreshCompleteScoped": "Scanned {scope}: {added} new, {removed} removed",
+      "refreshKeptUnreachable": "{count} models kept: {paths} not reachable",
+      "scanRootUnreachable": "{scope} is not reachable right now. Nothing was changed."
    },
    "agent": {
      "llmNotConfigured": "AI provider not configured. Enable it in Settings → AI Provider.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "Actualizar lista de modelos",
                "full": "Reconstruir caché",
-                "fullTooltip": "Vuelve a cargar todos los detalles desde los archivos de metadatos; úsalo si la biblioteca parece desactualizada o tras ediciones manuales."
+                "fullTooltip": "Vuelve a cargar todos los detalles desde los archivos de metadatos; úsalo si la biblioteca parece desactualizada o tras ediciones manuales.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "Obtener metadatos de CivitAI",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "Error al descargar imágenes de ejemplo: {message}",
            "moveFailed": "No se pudo mover el elemento: {message}",
            "copiedToClipboard": "Copiado al portapapeles",
-            "downloadStarted": "Descarga iniciada"
+            "downloadStarted": "Descarga iniciada",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "Proveedor de IA no configurado. Actívelo en Configuración → Proveedor de IA.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "Actualiser la liste des modèles",
                "full": "Reconstruire le cache",
-                "fullTooltip": "Recharge tous les détails des modèles depuis les fichiers de métadonnées — à utiliser si la bibliothèque paraît obsolète ou après des modifications manuelles."
+                "fullTooltip": "Recharge tous les détails des modèles depuis les fichiers de métadonnées — à utiliser si la bibliothèque paraît obsolète ou après des modifications manuelles.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "Récupérer les métadonnées depuis CivitAI",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "Échec du téléchargement des images d'exemple : {message}",
            "moveFailed": "Échec du déplacement de l'élément : {message}",
            "copiedToClipboard": "Copié dans le presse-papiers",
-            "downloadStarted": "Téléchargement démarré"
+            "downloadStarted": "Téléchargement démarré",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "Fournisseur d'IA non configuré. Activez-le dans Paramètres → Fournisseur d'IA.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "רענן רשימת מודלים",
                "full": "בניית מטמון מחדש",
-                "fullTooltip": "טוען מחדש את כל פרטי המודלים מקבצי המטא-נתונים – לשימוש אם הספרייה נראית לא מעודכנת או לאחר עריכות ידניות."
+                "fullTooltip": "טוען מחדש את כל פרטי המודלים מקבצי המטא-נתונים – לשימוש אם הספרייה נראית לא מעודכנת או לאחר עריכות ידניות.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "אחזר מטא-נתונים מ-CivitAI",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "הורדת תמונות הדוגמה נכשלה: {message}",
            "moveFailed": "לא ניתן היה להעביר את הפריט: {message}",
            "copiedToClipboard": "הועתק ללוח",
-            "downloadStarted": "ההורדה החלה"
+            "downloadStarted": "ההורדה החלה",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "ספק AI לא הוגדר. הפעל אותו בהגדרות → ספק AI.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "モデルリストを更新",
                "full": "キャッシュを再構築",
-                "fullTooltip": "メタデータファイルから全モデル情報を再読み込みします。リストが古いと感じるときや手動編集後に使用してください。"
+                "fullTooltip": "メタデータファイルから全モデル情報を再読み込みします。リストが古いと感じるときや手動編集後に使用してください。",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "CivitAIからメタデータを取得",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "例画像のダウンロードに失敗しました：{message}",
            "moveFailed": "アイテムを移動できませんでした: {message}",
            "copiedToClipboard": "クリップボードにコピーしました",
-            "downloadStarted": "ダウンロードを開始しました"
+            "downloadStarted": "ダウンロードを開始しました",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "AIプロバイダーが設定されていません。設定 → AIプロバイダーで有効にしてください。",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "모델 목록 새로고침",
                "full": "캐시 재구성",
-                "fullTooltip": "메타데이터 파일에서 모든 모델 정보를 다시 불러옵니다. 라이브러리가 오래되어 보이거나 수동 수정 후에 사용하세요."
+                "fullTooltip": "메타데이터 파일에서 모든 모델 정보를 다시 불러옵니다. 라이브러리가 오래되어 보이거나 수동 수정 후에 사용하세요.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "CivitAI에서 메타데이터 가져오기",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "예시 이미지 다운로드 실패: {message}",
            "moveFailed": "항목을 이동하지 못했습니다: {message}",
            "copiedToClipboard": "클립보드에 복사됨",
-            "downloadStarted": "다운로드 시작됨"
+            "downloadStarted": "다운로드 시작됨",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "AI 제공자가 설정되지 않았습니다. 설정 → AI 제공자에서 활성화하세요.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "Обновить список моделей",
                "full": "Перестроить кэш",
-                "fullTooltip": "Перечитывает все данные моделей из файлов метаданных — используйте, если библиотека выглядит устаревшей или после ручных правок."
+                "fullTooltip": "Перечитывает все данные моделей из файлов метаданных — используйте, если библиотека выглядит устаревшей или после ручных правок.",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "Получить метаданные с CivitAI",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "Не удалось загрузить примеры изображений: {message}",
            "moveFailed": "Не удалось переместить элемент: {message}",
            "copiedToClipboard": "Скопировано в буфер обмена",
-            "downloadStarted": "Загрузка начата"
+            "downloadStarted": "Загрузка начата",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "Поставщик ИИ не настроен. Включите его в Настройки → Поставщик ИИ.",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "刷新模型列表",
                "full": "重建缓存",
-                "fullTooltip": "从元数据文件重新加载所有模型信息；用于列表过时或手动编辑后。"
+                "fullTooltip": "从元数据文件重新加载所有模型信息；用于列表过时或手动编辑后。",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "从 CivitAI 获取元数据",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "示例图片下载失败：{message}",
            "moveFailed": "移动条目失败：{message}",
            "copiedToClipboard": "已复制到剪贴板",
-            "downloadStarted": "下载已开始"
+            "downloadStarted": "下载已开始",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "AI 提供商未配置。请在 设置 → AI 提供商 中进行配置。",
@@ -918,7 +918,10 @@
            "refresh": {
                "title": "重新整理模型列表",
                "full": "重建快取",
-                "fullTooltip": "從中繼資料檔重新載入所有模型資訊；適用於清單過時或手動編輯後。"
+                "fullTooltip": "從中繼資料檔重新載入所有模型資訊；適用於清單過時或手動編輯後。",
+                "scopeSection": "[TODO: Translate] Scan one folder",
+                "rootOffline": "[TODO: Translate] Offline",
+                "rootModels": "[TODO: Translate] {count} models"
            },
            "fetch": {
                "title": "從 CivitAI 取得 metadata",
@@ -2804,7 +2807,10 @@
            "exampleImagesDownloadFailed": "下載範例圖片失敗：{message}",
            "moveFailed": "移動項目失敗：{message}",
            "copiedToClipboard": "已複製到剪貼簿",
-            "downloadStarted": "下載已開始"
+            "downloadStarted": "下載已開始",
+            "refreshCompleteScoped": "[TODO: Translate] Scanned {scope}: {added} new, {removed} removed",
+            "refreshKeptUnreachable": "[TODO: Translate] {count} models kept: {paths} not reachable",
+            "scanRootUnreachable": "[TODO: Translate] {scope} is not reachable right now. Nothing was changed."
        },
        "agent": {
            "llmNotConfigured": "AI 提供者尚未設定。請在 設定 → AI 提供者 中進行設定。",