fix(scanner): compare model roots by identity, not by spelling (#1146)

ComfyUI hands its model folders over in the host's own spelling --
`utils/extra_config.py` runs every configured path through `os.path.normpath()`,
which is all backslashes on Windows, and `main.py` registers `<output>/loras`
unconditionally for its save nodes. `Config` kept that spelling verbatim while
`_prepare_*_paths()` normalized the live root lists to forward slashes, and the
scanner compared the two as raw strings. On Windows every configured root
therefore looked like a root that was not in the live list: the Refresh menu
listed each directory twice (the twin showing "0 models"), the completion toast
claimed reachable roots were "currently inaccessible" and kept their whole cache
count as `kept_unreachable`, and the settings sync persisted host-registered
directories that were never created.

The duplication was not only cosmetic. `_UnreachablePaths.add()` normalizes the
prefix it is given, so the configured twin of a live root registered exactly the
live root's prefix as unreadable: in the prune phase every cached-but-missing
entry under it matched and was "kept", so models deleted from disk were never
removed from the cache again.

* `py/utils/root_paths.py` holds the single `normalize_root_identity()`
  (separator style, redundant/trailing separators, and case where the filesystem
  folds it: whole paths on Windows, drive letters everywhere). `config.py` and
  `settings_manager.py` each carried a private copy and `model_scanner.py`
  compared raw strings.
* `ModelScanner` compares root identities in `get_configured_model_roots()`,
  `describe_model_roots()` (dedupe + `available`), the reconcile's
  `configured_only` list and the `ReconcileScope.roots` membership test, so one
  directory is one root whatever the host called it. `_normalized_root_prefix()`
  is built on the same identity, and an empty prefix no longer matches every
  path.
* A configured root that never made it into the live list is reported only while
  it still holds cached entries -- in the menu and in `skipped_roots` /
  `unavailable_paths` alike. A directory that neither exists nor was ever scanned
  has nothing to keep, so `<output>/loras` stops being an unavailable root.
* The startup sync drops saved roots that neither exist nor hold cached entries
  (`PersistentModelCache.roots_with_cached_entries()`, one scan per model type,
  separator-agnostic and wildcard-escaped). A root named by the library's
  `default_*_root` is kept even when empty: dropping it would let the next boot
  "repair" the default onto another root. When the cache cannot be consulted,
  every root is kept.
* `GET /scan?roots=` validates requested roots by identity too, so the browser
  extension or a script passing the spelling settings.json holds selects the root
  instead of answering 400 "Unknown model root(s)".

Verified: 8 of the new tests fail against the pre-fix source (checked with the
fix stashed); 3798 passed, 7 skipped, 11 deselected. The probe costs 71ms for a
100k-model library with seven missing roots (357ms when probing root by root).
This commit is contained in:
Will Miao
2026-10-12 08:43:06 +08:00
parent e056142669
commit 6d0d7cb15b
11 changed files with 822 additions and 57 deletions
+6 -15
View File
@@ -2,7 +2,6 @@ import asyncio
import copy
import json
import os
import posixpath
import shutil
import tempfile
import logging
@@ -36,6 +35,7 @@ from ..utils.constants import (
normalize_other_sub_types,
)
from ..utils.preview_selection import VALID_MATURE_BLUR_LEVELS
from ..utils.root_paths import normalize_root_identity
from ..utils.settings_paths import (
APP_NAME,
_portable_env_override,
@@ -156,15 +156,6 @@ DEFAULT_SETTINGS: Dict[str, Any] = {
}
def _normalize_root_identity(path: str) -> str:
"""Normalize a root path for equality checks across slash styles."""
normalized = posixpath.normpath(path.strip().replace("\\", "/"))
if len(normalized) >= 2 and normalized[1] == ":":
return normalized.lower()
return normalized
class SettingsManager:
def __init__(self):
self.settings_file = ensure_settings_file(logger)
@@ -1058,7 +1049,7 @@ class SettingsManager:
return False
allowed_roots = self._get_allowed_roots(key)
if current and _normalize_root_identity(current) in allowed_roots:
if current and normalize_root_identity(current) in allowed_roots:
return False
self.settings[setting_key] = primary_candidates[0]
@@ -1094,7 +1085,7 @@ class SettingsManager:
candidate_identities: set[str] = set()
for folder_key in OTHER_SUB_TYPE_FOLDER_KEYS.get(sub_type, []):
for candidate in self._get_valid_root_candidates(folder_key):
identity = _normalize_root_identity(candidate)
identity = normalize_root_identity(candidate)
if identity in candidate_identities:
continue
candidate_identities.add(identity)
@@ -1102,7 +1093,7 @@ class SettingsManager:
if not candidates:
continue
current = other_roots.get(sub_type, "")
if current and _normalize_root_identity(current) in candidate_identities:
if current and normalize_root_identity(current) in candidate_identities:
continue
other_roots[sub_type] = candidates[0]
if current:
@@ -1152,7 +1143,7 @@ class SettingsManager:
normalized = value.strip()
if not normalized:
continue
identity = _normalize_root_identity(normalized)
identity = normalize_root_identity(normalized)
if identity in seen:
continue
seen.add(identity)
@@ -1162,7 +1153,7 @@ class SettingsManager:
def _get_allowed_roots(self, key: str) -> set[str]:
"""Return all valid roots for a model type, including extra roots."""
return {_normalize_root_identity(path) for path in self._get_valid_root_candidates(key)}
return {normalize_root_identity(path) for path in self._get_valid_root_candidates(key)}
def _check_environment_variables(self) -> None:
"""Check for environment variables and update settings if needed"""