feat(scanner): report and re-admit model roots that are unavailable (#1108)

`Config._dedupe_existing_paths()` drops roots whose directory does not exist
when the root list is built, so a drive that is switched off while LM starts was
not "an offline root" — it was not a root at all. The Refresh ▾ menu had no row
for it in either state, a full refresh reported nothing (its cached entries
survived only because they fall outside every configured prefix), and plugging
the drive back in changed nothing until the next restart.

* `Config` now records the configured (existence-unfiltered) paths per model
  type — from ComfyUI's list in plugin mode, from the active library snapshot in
  standalone mode, where the host mock filters non-existent paths itself — and
  exposes them through `configured_roots_for()`.
* `ModelScanner.describe_model_roots()` appends configured-but-unavailable roots
  with a live `reachable`, an `available` flag and their cached entry count, so
  `/roots` (and therefore the menu) offers a row in both states: greyed while the
  directory is missing, normal and clickable once it is back.
* `_reconcile_cache()` reports them as `skipped_roots` / `unavailable_paths` with
  reason `root_unavailable` and counts their cached entries in `kept_unreachable`,
  which is what makes the "N models kept" toast appear in the startup-offline
  case. Report-only: they stay outside the scan scope, so nothing about pruning
  changes. A root-scoped scan stays quiet about roots it was not asked about, and
  a folder-scoped scan only mentions a missing root that holds cached entries of
  that folder.
* `Config.admit_configured_roots()` re-runs the per-type prepare helpers against
  the configured paths and APPENDS what is readable now (plus the
  checkpoint/unet/other side maps), then refreshes the preview allowlist, so a
  drive plugged in mid-session can be scanned without restarting. Append-only and
  order-preserving by design: removing a root mid-session would let a later
  settings save persist the loss, and re-sorting would move `*_roots[0]`, which
  derives the recipes directory and the usage-stats file location. `_roots[0]`
  never moves.
* `/roots` and `/scan` admit first, so `describe_model_roots()` stays a pure read
  and a caller that never opens the menu (the browser extension) still gets a
  drive that came back.

Verified on the sandbox with drive-Z switched off before startup: `/roots`
reports `drive-Z reachable=false available=false models=60`, a full refresh
returns `skipped_roots=[drive-Z root_unavailable]`,
`unavailable_paths=[{... kept: 60}]`, `kept_unreachable=60`, and all 432 models
stay cached. Renaming the drive back while the server runs admits it
(`available=true`) and `GET /scan?roots=<drive-Z>` walks it with no restart.
3714 passed, 7 skipped; frontend 1495 passed (149 files); vue widgets 96 passed.

py/config.py

@@ -1679,6 +1679,105 @@ class Config:
         """
         return list((getattr(self, "_configured_root_paths", None) or {}).get(model_type, []))
 
+    def admit_configured_roots(self) -> List[str]:
+        """Append configured roots that became readable again, never removing any.
+
+        The live lists are built once at startup (and when a library snapshot is
+        applied), so a drive plugged in later is invisible until the process is
+        restarted. Re-running the per-type prepare helpers against the configured
+        paths and appending only the new entries is what makes "plug the drive
+        in, scan it" work without a restart. Nothing is ever dropped or
+        re-ordered — see ``_append_new_paths`` for why order matters.
+
+        Returns the paths that were admitted by this call.
+        """
+        admitted: List[str] = []
+
+        def _extend(current, candidates) -> List[str]:
+            merged = _append_new_paths(list(current or []), candidates)
+            admitted.extend(
+                path for path in merged if path not in (current or [])
+            )
+            return merged
+
+        try:
+            lora_configured = self.configured_roots_for("lora")
+            if lora_configured:
+                self.loras_roots = _extend(
+                    self.loras_roots, self._prepare_lora_paths(lora_configured)
+                )
+
+            checkpoint_configured = self.configured_roots_for("checkpoint")
+            unet_configured = self.configured_roots_for("unet")
+            if checkpoint_configured or unet_configured:
+                (
+                    all_roots,
+                    checkpoint_roots,
+                    unet_roots,
+                ) = self._prepare_checkpoint_paths(
+                    checkpoint_configured, unet_configured
+                )
+                self.base_models_roots = _extend(self.base_models_roots, all_roots)
+                self.checkpoints_roots = _extend(self.checkpoints_roots, checkpoint_roots)
+                self.unet_roots = _extend(self.unet_roots, unet_roots)
+
+            embedding_configured = self.configured_roots_for("embedding")
+            if embedding_configured:
+                self.embeddings_roots = _extend(
+                    self.embeddings_roots,
+                    self._prepare_embedding_paths(embedding_configured),
+                )
+
+            other_configured = self.configured_roots_for("other")
+            if other_configured:
+                (
+                    other_roots,
+                    other_subtypes,
+                    other_per_key,
+                ) = self._prepare_other_paths(
+                    {
+                        key: [
+                            path
+                            for path in self._configured_other_paths_for_key(key)
+                        ]
+                        for key in self._get_enabled_other_folder_keys()
+                    }
+                )
+                self.other_roots = _extend(self.other_roots, other_roots)
+                for root, sub_type in other_subtypes.items():
+                    self.other_root_subtypes.setdefault(root, sub_type)
+                for key, roots in other_per_key.items():
+                    self.other_folder_roots[key] = _append_new_paths(
+                        self.other_folder_roots.get(key, []), roots
+                    )
+        except Exception as exc:  # pragma: no cover - defensive logging
+            logger.warning("Failed to admit configured roots: %s", exc)
+            return admitted
+
+        if admitted:
+            logger.info(
+                "Admitted %d previously unavailable model root(s): %s",
+                len(admitted),
+                ", ".join(admitted),
+            )
+            try:
+                # Previews of the newly admitted drive must be servable again.
+                self.refresh_preview_roots()
+            except Exception as exc:  # pragma: no cover - defensive logging
+                logger.debug("Failed to refresh preview roots: %s", exc)
+        return admitted
+
+    def _configured_other_paths_for_key(self, key: str) -> List[str]:
+        """Configured other-model paths for one folder_paths key (may not exist)."""
+        try:
+            return [
+                path.strip()
+                for path in folder_paths.get_folder_paths(key) or []
+                if isinstance(path, str) and path.strip()
+            ]
+        except Exception:
+            return []
+
     def refresh_other_roots(self) -> None:
         """Rebuild other-model roots after the management toggles changed.
 

py/routes/handlers/model_handlers.py

@@ -1163,6 +1163,11 @@ class ModelQueryHandler:
                 except ValueError as exc:
                     return web.json_response({"error": str(exc)}, status=400)
 
+            # A drive that was plugged in after startup must be scannable without
+            # a restart, and callers that never open the Refresh menu (the browser
+            # extension) have no other chance to admit it.
+            self._service.refresh_model_roots()
+
             if requested_roots:
                 configured = self._service.get_model_roots()
                 unknown = [root for root in requested_roots if root not in configured]
@@ -1206,6 +1211,9 @@ class ModelQueryHandler:
 
     async def get_model_roots(self, request: web.Request) -> web.Response:
         try:
+            # A drive plugged in after startup is admitted here, so the menu shows
+            # it as a normal row instead of "unavailable" until the next restart.
+            self._service.refresh_model_roots()
             roots = self._service.get_model_roots()
             try:
                 root_details = self._service.describe_model_roots()

py/services/base_model_service.py

@@ -954,9 +954,17 @@ class BaseModelService(ABC):
         return self.scanner.last_reconcile_summary
 
     def describe_model_roots(self) -> List[Dict[str, Any]]:
-        """Describe configured roots (label / reachability / cached count)."""
+        """Describe configured roots (label / reachability / cached count).
+
+        Pure read: callers that want a drive which was plugged in after startup to
+        appear as a normal row call :meth:`refresh_model_roots` first.
+        """
         return self.scanner.describe_model_roots()
 
+    def refresh_model_roots(self) -> List[str]:
+        """Admit configured roots that became readable again (append-only)."""
+        return self.scanner.refresh_model_roots()
+
     async def get_model_info_by_name(self, name: str):
         """Get model information by name"""
         return await self.scanner.get_model_info_by_name(name)

py/services/model_scanner.py

@@ -257,7 +257,13 @@ class _UnreachablePaths:
         self._prefixes: List[str] = []
         self._details: Dict[str, Dict[str, Any]] = {}
 
-    def add(self, path: str, reason: str) -> None:
+    def add(self, path: str, reason: str, kept: int = 0) -> None:
+        """Register an unreadable prefix.
+
+        ``kept`` seeds the count for a root whose entries never reach the prune
+        phase because they are outside the scan scope entirely (a configured root
+        that was not even part of the live root list).
+        """
         prefix = _normalized_root_prefix(path)
         with self._lock:
             if prefix in self._details:
@@ -265,7 +271,7 @@ class _UnreachablePaths:
             self._details[prefix] = {
                 'path': path.replace(os.sep, '/'),
                 'reason': reason,
-                'kept': 0,
+                'kept': kept,
             }
             self._prefixes.append(prefix)
 
@@ -1576,6 +1582,44 @@ class ModelScanner:
         """
         return self._last_reconcile_summary
 
+    def get_configured_model_roots(self) -> List[str]:
+        """Roots the user configured for this type, including unavailable ones.
+
+        ``get_model_roots()` answers "what can be walked right now"; this answers
+        "what did the user configure", which is what the refresh menu reports and
+        what must survive a switched-off drive.
+        """
+        try:
+            from ..config import config
+        except Exception:  # pragma: no cover - import guard
+            return []
+
+        # The checkpoint page scans two configured keys.
+        keys = ('checkpoint', 'unet') if self.model_type == 'checkpoint' else (self.model_type,)
+        roots: List[str] = []
+        seen: Set[str] = set()
+        for key in keys:
+            for root in config.configured_roots_for(key):
+                if root and root not in seen:
+                    seen.add(root)
+                    roots.append(root)
+        return roots
+
+    def refresh_model_roots(self) -> List[str]:
+        """Admit configured roots whose directory is readable again.
+
+        Append-only and idempotent: a drive plugged in while the process runs
+        becomes scannable without a restart. Called before describing the roots
+        (`/roots`) and before a scan, so both see the same list.
+        """
+        try:
+            from ..config import config
+
+            return config.admit_configured_roots()
+        except Exception as exc:  # pragma: no cover - defensive logging
+            logger.debug("Failed to refresh model roots: %s", exc)
+            return []
+
     def describe_model_roots(self) -> List[Dict[str, Any]]:
         """Describe the configured roots for the refresh scope menu.
 
@@ -1591,21 +1635,30 @@ class ModelScanner:
             seen.add(root)
             roots.append(root)
 
-        labels = _root_display_labels(roots)
+        # Configured roots that are missing from the live list: their directory
+        # did not exist when it was built (a drive switched off at startup).
+        # They are reported so the menu offers a row in both states instead of
+        # hiding the root until the process is restarted.
+        unavailable = [
+            root for root in self.get_configured_model_roots() if root not in seen
+        ]
+
+        labels = _root_display_labels(roots + unavailable)
         cached_paths = (
             {item.get('file_path', '') for item in self._cache.raw_data}
             if self._cache is not None
             else set()
         )
-        counts = _count_cached_entries_per_root(cached_paths, roots)
+        counts = _count_cached_entries_per_root(cached_paths, roots + unavailable)
         return [
             {
                 'path': root.replace(os.sep, '/'),
                 'label': labels[root],
                 'reachable': os.path.exists(root),
+                'available': root in seen,
                 'models': counts.get(root, 0),
             }
-            for root in roots
+            for root in roots + unavailable
         ]
 
     async def _initialize_cache(self) -> None:
@@ -1732,7 +1785,17 @@ class ModelScanner:
                 seen_roots.add(root_path)
                 configured_roots.append(root_path)
 
-            all_labels = _root_display_labels(configured_roots)
+            # Configured roots that never made it into the live list (their
+            # directory was missing when it was built). They are outside the scan
+            # scope entirely — which is exactly why their cached entries survive
+            # — so they only need reporting, and their labels must not collide
+            # with the live ones.
+            configured_only = [
+                root for root in self.get_configured_model_roots()
+                if root and root not in seen_roots
+            ]
+
+            all_labels = _root_display_labels(configured_roots + configured_only)
             scope_roots = [
                 root for root in configured_roots
                 if scope is None or scope.roots is None or root in scope.roots
@@ -1755,6 +1818,29 @@ class ModelScanner:
                     continue
                 roots.append(root_path)
 
+            # Report the roots that were filtered out of the live list, so a
+            # switched-off drive no longer keeps its models in silence. A scoped
+            # scan stays quiet about them: with explicit roots the caller asked
+            # for specific ones, and for a folder only the roots that actually
+            # hold cached entries of that folder are relevant.
+            if configured_only and (scope is None or scope.roots is None):
+                folder_candidates = [
+                    (root, _scope_prefix(root, scope_folder)) for root in configured_only
+                ]
+                folder_counts = _count_cached_entries_for_prefixes(
+                    cached_paths, folder_candidates
+                )
+                for root_path, _prefix in folder_candidates:
+                    kept = folder_counts.get(root_path, 0)
+                    if scope_folder is not None and not kept:
+                        continue
+                    unreachable.add(root_path, 'root_unavailable', kept=kept)
+                    skipped_roots.append({
+                        'path': root_path.replace(os.sep, '/'),
+                        'label': all_labels.get(root_path, root_path),
+                        'reason': 'root_unavailable',
+                    })
+
             self._collect_offline_symlink_prefixes(roots, unreachable)
 
             # Scope prefixes come from every root the caller asked for (even the