Commit Graph

25 Commits

Author SHA1 Message Date
Will Miao 4d87ae7637 fix(other-models): stop warning about legacy folder keys that alias
Enabling Other Models logged two warnings on a stock ComfyUI install:

  Detected the same folder '.../clip' under multiple other-model categories
  ('.../clip' is already mapped). Keeping the first category; please fix
  your path configuration.

Nothing was wrong with the configuration. ComfyUI's folder_paths rewrites
legacy names before every access (map_legacy: clip -> text_encoders,
unet -> diffusion_models) and registers both legacy directories under the
canonical key, so get_folder_paths("clip") returns exactly the same list as
get_folder_paths("text_encoders"). Both keys are in the enabled allow-list,
so the second pass hit the overlap guard for every text-encoder folder and
printed advice the user cannot act on. The path list itself was correct
(deduped), only the message was wrong.

- Config._collapse_legacy_folder_keys() drops a key when the host exposes
  map_legacy and resolves it to another queried key. That is provably
  lossless: an empty canonical list implies an empty alias list. The
  standalone MockFolderPaths has no map_legacy and its keys are independent
  settings.json entries, so every key is still queried there.
- _prepare_other_paths() now tracks the claiming sub_type alongside the
  business path and downgrades a same-sub_type duplicate to debug, keeping
  the warning for a genuine cross-category collision (and naming the other
  category in the message).

Regression tests cover the aliased-key layout (no warning, no redundant
query, both folders still managed) and the same-sub_type duplicate, and the
opt-in test is parametrized over controlnet and clip_vision.
2026-09-13 20:12:45 +08:00
Will Miao 28fbb86dce feat(backend): gate Other Models behind opt-in management toggles
Other Models management is now opt-in: enable_other_models (default false)
plus the enabled_other_sub_types allow-list replace the unreleased additive
enabled_other_folders key.

- config._get_enabled_other_folder_keys() is the single scan gate; a new
  refresh_other_roots() rebuilds roots and preview roots on toggle.
- ModelScanner gains a _should_keep_cached_entry() hydration hook and
  on_library_changed(reconcile=...) so switching a sub_type off drops its
  entries (and hash/autov3 rows) at load time and switching it on rescans.
- OtherScanner filters location-derived entries accordingly.
- Other routes reject every other type while off (or a disabled sub_type) and
  expose an "other_disabled" page flag; download routing returns a disabled
  marker instead of guessing; the download manager refuses other-type
  downloads and default-path routing for switched-off sub_types.
- Doctor / init-status / refresh-all skip the other scanner while off; the
  scanner stays registered so staged pending-deletes still merge.
- Tests updated with explicit opt-in fixtures plus new gating coverage.
2026-09-13 07:59:28 +08:00
Will Miao 27da7b3ca3 feat(backend): add Other model type (VAE/upscaler/text encoder) scanner, service and routes 2026-09-12 11:25:51 +08:00
Will Miao d2f955266d fix(types): resolve pre-existing basedpyright errors in tests
Fix ~790 basedpyright errors across the test suite:
- Type stub subclasses of real production classes with super().__init__()
- Add missing generic type arguments and Dict[str, Any] annotations
- Add None guards before subscript/member access
- Adapt tests to production API changes (removed dead handlers,
  PersistentModelCache.get_default, _i18n_filter_added location)
2026-08-08 20:12:59 +08:00
Will Miao 7ee2361e87 fix(config): remove stale 'default' library entry and consolidate example images on startup 2026-07-18 17:25:36 +08:00
Will Miao ef4923fd94 fix(settings): normalize default root path comparisons 2026-04-21 09:43:37 +08:00
Will Miao 34f03d6495 fix(settings): preserve extra default roots in comfyui sync 2026-04-20 15:48:30 +08:00
Will Miao 9bdb337962 fix(settings): enforce valid default model roots 2026-04-01 20:36:37 +08:00
Will Miao 76ee59cdb9 fix(paths): deduplicate LoRA path overlap (#871) 2026-03-27 17:35:24 +08:00
Will Miao 2dae4c1291 fix: isolate extra unet paths from checkpoints to prevent type misclassification
Refactor _prepare_checkpoint_paths() to return a tuple instead of having
side effects on instance variables. This prevents extra unet paths from
being incorrectly classified as checkpoints when processing extra paths.

- Changed return type from List[str] to Tuple[List[str], List[str], List[str]]
  (all_paths, checkpoint_roots, unet_roots)
- Updated _init_checkpoint_paths() and _apply_library_paths() callers
- Fixed extra paths processing to properly isolate main and extra roots
- Updated test_checkpoint_path_overlap.py tests for new API

This ensures models in extra unet paths are correctly identified as
diffusion_model type and don't appear in checkpoints list.
2026-03-17 22:03:57 +08:00
Will Miao 87b462192b feat: Add extra folder paths support for LoRA Manager
Introduce extra_folder_paths feature to allow users to add additional
model roots that are managed by LoRA Manager but not shared with ComfyUI.

Changes:
- Add extra_folder_paths support in SettingsManager (stored per library)
- Add extra path attributes in Config class (extra_loras_roots, etc.)
- Merge folder_paths with extra_folder_paths when applying library settings
- Update LoraScanner, CheckpointScanner, EmbeddingScanner to include
  extra paths in their model roots
- Add comprehensive tests for the new functionality

This enables users to manage models from additional directories without
modifying ComfyUI's model folder configuration.
2026-02-25 18:16:17 +08:00
Will Miao 7bcf4e4491 feat(config): discover deep symlinks dynamically when accessing previews 2026-02-04 00:16:59 +08:00
Will Miao 655d3cab71 fix(config): prioritize checkpoints over unet when paths overlap, #799
When checkpoints and unet folders point to the same physical location
(via symlinks), prioritize checkpoints for backward compatibility.

This prevents the 'Failed to load Checkpoint root' error that users
experience when they have incorrectly configured their ComfyUI paths.

Changes:
- Detect overlapping real paths between checkpoints and unet
- Log warning to inform users of the configuration issue
- Remove overlapping paths from unet_map, keeping checkpoints

Fixes #<issue-number>
2026-02-03 18:27:42 +08:00
Will Miao 553e1868e1 perf(config): limit symlink scan to first level for faster startup
Replace recursive directory traversal with first-level-only symlink scanning
to fix severe performance issues on large model collections (220K+ files).

- Rename _scan_directory_links to _scan_first_level_symlinks
- Only scan symlinks directly under each root directory
- Skip traversal of normal subdirectories entirely
- Update tests to reflect first-level behavior
- Add test_deep_symlink_not_scanned to document intentional limitation

Startup time reduced from 15+ minutes to seconds for affected users.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 12:42:35 +08:00
pixelpaws 93cd0b54dc Merge branch 'main' into fix-symlink 2026-01-26 17:29:31 +08:00
Will Miao 7b0c6c8bab refactor(cache): reorganize cache directory structure with automatic legacy cleanup
- Centralize cache path resolution in new py/utils/cache_paths.py module
- Migrate legacy cache files to organized structure: {settings_dir}/cache/{model|recipe|fts|symlink}/
- Automatically clean up legacy files after successful migration with integrity verification
- Update Config symlink cache to use new path and migrate from old location
- Simplify service classes (PersistentModelCache, PersistentRecipeCache, RecipeFTSIndex, TagFTSIndex) to use centralized migration logic
- Add comprehensive test coverage for cache paths and automatic cleanup
2026-01-26 16:12:08 +08:00
Will Miao d063d48417 feat(symlink): add deep validation for symlink cache invalidation
Detects symlink changes at any depth, not just at root level. Uses two-tier validation:
- Fingerprint check for new symlinks
- Deep mapping validation for removed/retargeted symlinks
2026-01-26 09:30:10 +08:00
Will Miao c9e305397c feat: enhance symlink detection and cache invalidation
- Add `_entry_is_symlink` method to detect symlinks and Windows junctions
- Include first-level symlinks in fingerprint for better cache invalidation
- Re-enable preview path validation for security
- Update tests to verify retargeted symlinks trigger rescan
2026-01-25 19:14:16 +08:00
Will Miao edbcca9bbd refactor: simplify symlink cache invalidation by removing background rescan and noise_mtime in favor of a root-path-only fingerprint. 2026-01-03 19:29:53 +08:00
Will Miao 099a71b2cc feat(config): seed root symlink mappings before deep scanning
Add `_seed_root_symlink_mappings` method to ensure symlinked root folders are recorded before deep scanning, preventing them from being missed during directory traversal. This ensures that root symlinks are properly captured in the path mappings.

Additionally, normalize separators in relative paths for cross-platform consistency in `BaseModelService`, and update tests to verify root symlinks are preserved in the cache.
2025-12-16 22:05:40 +08:00
Will Miao 2494fa19a6 feat(config): add background symlink rescan and simplify cache validation
- Added threading import and optional `_rescan_thread` for background operations
- Simplified `_load_symlink_cache` to only validate path mappings, removing fingerprint checks
- Updated `_initialize_symlink_mappings` to rebuild preview roots and schedule rescan when cache is loaded
- Added `_schedule_symlink_rescan` method to perform background validation of symlinks
- Cleared `_path_mappings` at start of `_scan_symbolic_links` to prevent stale entries
- Background rescan improves performance by deferring symlink validation after cache load
2025-12-15 18:46:23 +08:00
Will Miao 3fc72d6bc1 feat(config): replace symlink scanning with cached mapping system
- Add `get_settings_dir` import for cache directory resolution
- Replace `_scan_symbolic_links` and `_rebuild_preview_roots` with unified `_initialize_symlink_mappings` method
- Implement fingerprint-based cache validation using root mtimes, inodes, and noise-aware timestamps
- Add helper methods for path normalization, cache location, and symlink root aggregation
- Improve performance by avoiding redundant symlink traversal when directory structure is unchanged
2025-12-09 19:51:01 +08:00
pixelpaws ef0bcc6cf1 fix(config): remove template default library before saving paths 2025-11-02 09:09:36 +08:00
Will Miao 7757f72859 Enhance test for saving paths to ensure cross-platform compatibility in folder paths 2025-10-05 22:40:43 +08:00
pixelpaws 50f2c2dfe6 test(config): cover save path migration flows 2025-10-05 22:19:36 +08:00