Commit Graph

923 Commits

Author SHA1 Message Date
Will Miao
70398ed985 feat(lora-loader): Load LoRAs using lower-level API to bypass folder_paths validation
- Add get_lora_info_absolute() function to return absolute file paths
- Replace LoraLoader().load_lora() with comfy.utils.load_torch_file() +
  comfy.sd.load_lora_for_models() to enable loading LoRAs from any path
- This allows LoRA Manager to load LoRAs from non-standard paths (multi-library support)
- Fixes #805
2026-02-23 18:06:15 +08:00
pixelpaws
7a04cec08d Merge pull request #825 from RanKaze/main
feat: filter node with mode:0
2026-02-23 16:39:45 +08:00
Will Miao
26b139884c perf(usage-stats): prevent unnecessary writes when idle
- Add is_dirty flag to track if statistics have changed
- Only write stats file when data actually changes
- Add enable_usage_statistics setting in ComfyUI settings
- Skip backend requests when usage statistics is disabled
- Fix standalone mode compatibility for MetadataRegistry

Fixes #826
2026-02-23 14:00:00 +08:00
K1einB1ue
60324c1299 feat: filter node with mode:0 2026-02-22 07:19:08 +08:00
Will Miao
773adb27c9 feat(model_download): add file_params JSON parsing to download handler
- Parse optional file_params query parameter as JSON in ModelDownloadHandler
- Add error handling for invalid JSON with warning log
- Maintain backward compatibility with existing download parameters
2026-02-22 04:26:38 +08:00
Will Miao
9117ee60dd feat(download): add file_params support for precise file selection 2026-02-20 15:32:48 +08:00
Will Miao
879588e252 refactor(settings): invert sync logic from whitelist to blacklist
Replace _SYNC_KEYS (37 keys) with _NO_SYNC_KEYS (5 keys) in SettingsHandler.
New settings automatically sync to frontend unless explicitly excluded.

Changes:
- SettingsHandler now syncs all settings except those in _NO_SYNC_KEYS
- Added keys() method to SettingsManager for iteration
- Updated tests to use new behavior

Benefits:
- No more missing keys when adding new settings
- Reduced maintenance burden
- Explicit exclusions for sensitive/internal settings only

Fixes: #86
2026-02-20 12:14:50 +08:00
Will Miao
67869f19ff feat(early-access): implement EA filtering and UI improvements
Add Early Access version support with filtering and improved UI:

Backend:
- Add is_early_access and early_access_ends_at fields to ModelVersionRecord
- Implement two-phase EA detection (bulk API + single API enrichment)
- Add hide_early_access_updates setting to filter EA updates
- Update has_update() and has_updates_bulk() to respect EA filter setting
- Add _enrich_early_access_details() for precise EA time fetching
- Fix setting propagation through base_model_service and model_update_service

Frontend:
- Add smart relative time display for EA (in Xh, in Xd, or date)
- Replace EA label with clock icon in metadata (fa-clock)
- Show Download button with bolt icon for EA versions (fa-bolt)
- Change EA badge color to #F59F00 (CivitAI Buzz theme)
- Fix toggle UI for hide_early_access_updates setting
- Add translation keys for EA time formatting

Tests:
- Update all tests to pass with new EA functionality
- Add test coverage for EA filtering logic

Closes #815
2026-02-20 10:32:51 +08:00
Will Miao
e8b37365a6 fix: Show all tags in LoRA Pool without limit (#819)
- Backend: Support limit=0 to return all tags in top-tags API
- Frontend: Remove tags limit setting and fetch all tags by default
- UI: Implement virtual scrolling in TagsModal for performance
  - Initial display 200 tags, load more on scroll
  - Show all results when searching
- Remove lora_pool_tags_limit setting to simplify UX

Fixes #819
2026-02-19 09:59:08 +08:00
Will Miao
b9516c6b62 fix: Handle missing Civitai API response fields gracefully
Fix KeyError when 'hashes', 'name', or 'model' fields are missing from
Civitai API responses. Use .get() with defaults instead of direct dict
access in:

- LoraMetadata.from_civitai_info()
- CheckpointMetadata.from_civitai_info()
- EmbeddingMetadata.from_civitai_info()
- RecipeScanner._get_hash_from_civitai()
- DownloadManager._process_download()

Fixes #820
2026-02-18 12:02:48 +08:00
Will Miao
16c52877ad feat: add dynamic trigger_words inputs to PromptLM node
- Backend: Add _AllContainer for dynamic input validation bypass
- Backend: Modify INPUT_TYPES to support trigger_words1, trigger_words2, etc.
- Backend: Update encode() to collect all trigger_words* from kwargs
- Frontend: Create prompt_dynamic_inputs.js extension
- Frontend: Implement onConnectionsChange to auto-add/remove input slots
- Frontend: Renumber inputs sequentially on connect/disconnect

Based on Impact Pack's Switch (Any) node dynamic input pattern.
2026-02-18 07:18:12 +08:00
Will Miao
6b1e3f06ed refactor(example-images): minimize async lock contention by moving I/O outside critical sections
- Extract progress file loading to async methods to run in executor
- Refactor start_download to reduce lock time by pre-loading data before entering lock
- Improve check_pending_models efficiency with single-pass model collection and async loading
- Add type hints to get_status method
- Add tests for download task callback execution and error handling
2026-02-11 09:24:00 +08:00
Will Miao
94edde7744 feat(settings): add metadata_refresh_skip_paths to sync keys for UI update 2026-02-09 10:09:53 +08:00
Will Miao
024dfff021 feat: add metadata refresh skip paths setting, #790 2026-02-09 09:56:19 +08:00
Will Miao
765c1c42a9 feat: enhance skip metadata refresh with smart UI and subtle badges, #790 2026-02-09 09:56:18 +08:00
Will Miao
b4ad03c9bf fix: improve example image upload reliability and error handling, #804
- Sequential per-file upload to avoid client_max_size limits
- Add backend exception handler with proper 500 responses
- Increase standalone server upload limit to 256MB
- Add partial success localization support
2026-02-08 09:17:19 +08:00
Will Miao
fa3625ff72 feat(filter): add tag logic toggle (OR/AND) for include tags filtering
Add a segmented toggle in the Filter Panel to switch between 'Any' (OR)
and 'All' (AND) logic when filtering by multiple include tags.

Changes:
- Backend: Add tag_logic field to FilterCriteria and ModelFilterSet
- Backend: Parse tag_logic parameter in model handlers
- Frontend: Add segmented toggle UI in filter panel header
- Frontend: Add interaction logic and state management for tag logic
- Add translations for all supported languages
- Add comprehensive tests for the new feature

Closes #802
2026-02-05 22:36:30 +08:00
Will Miao
895d13dc96 feat(settings): clean up default values from settings.json
Add automatic cleanup of default values from settings.json to keep configuration files minimal and focused on user customizations. Introduces a threshold-based cleanup that only removes default values when the file contains a significant number of them (10+), preserving small template-based configurations while cleaning up legacy bloated files.

Key changes:
- Add DEFAULT_KEYS_CLEANUP_THRESHOLD constant to control cleanup aggressiveness
- Implement _cleanup_default_values_from_disk() method that removes default values from disk while keeping them available in memory
- Modify _ensure_default_settings() to only save when existing values are updated, not when defaults are inserted
- Update _serialize_settings_for_disk() to only persist settings that differ from defaults
- Add cleanup call during initialization for existing settings files

This reduces file size and noise in settings.json while maintaining full functionality at runtime.
2026-02-05 08:40:27 +08:00
Will Miao
b7e0821f66 feat(duplicates): add filter support for duplicate model finding, #783 2026-02-04 20:47:30 +08:00
Will Miao
7bcf4e4491 feat(config): discover deep symlinks dynamically when accessing previews 2026-02-04 00:16:59 +08:00
Will Miao
c12aefa82a fix(recipes): detect duplicates for remote imports using modelVersionId and Civitai URL, #750
- Use modelVersionId as fallback for all loras in fingerprint calculation (not just deleted)
- Add URL-based duplicate detection using source_path field
- Combine both fingerprint and URL-based duplicate detection in API response
- Fix _download_remote_media return type and unbound variable issue
2026-02-03 21:32:15 +08:00
Will Miao
655d3cab71 fix(config): prioritize checkpoints over unet when paths overlap, #799
When checkpoints and unet folders point to the same physical location
(via symlinks), prioritize checkpoints for backward compatibility.

This prevents the 'Failed to load Checkpoint root' error that users
experience when they have incorrectly configured their ComfyUI paths.

Changes:
- Detect overlapping real paths between checkpoints and unet
- Log warning to inform users of the configuration issue
- Remove overlapping paths from unet_map, keeping checkpoints

Fixes #<issue-number>
2026-02-03 18:27:42 +08:00
Will Miao
358e658459 fix(trigger_word_toggle): add trigger word normalization method
Introduce a new private method `_normalize_trigger_words` to handle consistent splitting and cleaning of trigger word strings. This method splits input by both single and double commas, strips whitespace, and filters out empty strings, returning a set of normalized words. It is now used in `process_trigger_words` to compare trigger word overrides, ensuring accurate detection of changes by comparing normalized sets instead of raw strings.
2026-02-03 15:42:09 +08:00
Will Miao
f5dbd6b8e8 fix(metadata): auto-disable archive_db setting when database file is missing 2026-02-03 08:36:27 +08:00
Will Miao
2c026a2646 fix(metadata-sync): persist db_checked flag for deleted models
When a deleted model is checked against the SQLite archive and not found, the `db_checked` flag was set in memory but never saved to disk. This occurred because the save operation was only triggered when `civitai_api_not_found` was True, which is not the case for deleted models (since the CivitAI API is not attempted). As a result, deleted models would be rechecked on every refresh instead of being skipped.

Changes:
- Introduce a `needs_save` flag to track when metadata state is updated
- Save metadata whenever `db_checked` is set to True, regardless of API status
- Ensure `last_checked_at` is set for SQLite-only attempts
- Add regression test to verify the fix
2026-02-03 07:34:41 +08:00
Will Miao
820afe9319 feat(recipe_scanner): ensure cache initialization and improve type safety
- Initialize RecipeCache in scan_recipes to prevent None reference errors
- Import PersistedRecipeData directly instead of using string annotation
- Remove redundant import inside _reconcile_recipe_cache method
2026-02-02 21:57:44 +08:00
Will Miao
5a97f4bc75 feat(recipe_scanner): optimize recipe lookup performance
Refactor recipe lookup logic to improve efficiency from O(n²) to O(n + m):
- Build recipe_by_id dictionary for O(1) recipe ID lookups
- Simplify persisted_by_path construction using recipe_id extraction
- Add fallback lookup by recipe_id when path lookup fails
- Maintain same functionality while reducing computational complexity
2026-02-02 19:37:06 +08:00
Will Miao
94da404cc5 fix: skip confirmed not-found models in bulk metadata refresh
When enable_metadata_archive_db=True, the previous filter logic would
repeatedly try to fetch metadata for models that were already confirmed
to not exist on CivitAI (from_civitai=False, civitai_deleted=True).

The fix adds a skip condition to exclude models that:
1. Are confirmed not from CivitAI (from_civitai=False)
2. Are marked as deleted/not found on CivitAI (civitai_deleted=True)
3. Either have no archive DB enabled, or have already been checked (db_checked=True)

This prevents unnecessary API calls to CivArchive for user-trained models
or models from non-CivitAI sources.

Fixes repeated "Error fetching version of CivArchive model by hash" logs
for models that will never be found on CivitAI/CivArchive.
2026-02-02 13:27:18 +08:00
Will Miao
1da476d858 feat(example-images): add check pending models endpoint and improve async handling
- Add /api/example-images/check-pending endpoint to quickly check models needing downloads
- Improve DownloadManager.start_download() to return immediately without blocking
- Add _handle_download_task_done callback for proper error handling and progress saving
- Add check_pending_models() method for lightweight pre-download validation
- Update frontend ExampleImagesManager to use new check-pending endpoint
- Add comprehensive tests for new functionality
2026-02-02 12:31:07 +08:00
Will Miao
e252e44403 refactor(logging): replace print statements with logger for consistency 2026-02-02 10:47:17 +08:00
Will Miao
778ad8abd2 feat(cache): add cache health monitoring and validation system, see #730
- Add cache entry validator service for data integrity checks
- Add cache health monitor service for periodic health checks
- Enhance model cache and scanner with validation support
- Update websocket manager for health status broadcasting
- Add initialization banner service for cache health alerts
- Add comprehensive test coverage for new services
- Update translations across all locales
- Refactor sync translation keys script
2026-02-02 08:30:59 +08:00
Will Miao
db1aec94e5 refactor(logging): replace print statements with logger in metadata_collector 2026-02-01 15:41:41 +08:00
Will Miao
553e1868e1 perf(config): limit symlink scan to first level for faster startup
Replace recursive directory traversal with first-level-only symlink scanning
to fix severe performance issues on large model collections (220K+ files).

- Rename _scan_directory_links to _scan_first_level_symlinks
- Only scan symlinks directly under each root directory
- Skip traversal of normal subdirectories entirely
- Update tests to reflect first-level behavior
- Add test_deep_symlink_not_scanned to document intentional limitation

Startup time reduced from 15+ minutes to seconds for affected users.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-01 12:42:35 +08:00
Will Miao
c0f03b79a8 feat(settings): change model card footer action default to replace_preview 2026-02-01 07:38:04 +08:00
Will Miao
e17d6c8ebf feat(testing): enhance test configuration and add Vue component tests
- Update package.json test script to run both JS and Vue tests
- Simplify LoraCyclerLM output by removing redundant lora name fallback
- Extend Vitest config to include TypeScript test files
- Add Vue testing dependencies and setup for component testing
- Implement comprehensive test suite for BatchQueueSimulator component
- Add test setup file with global mocks for ComfyUI modules
2026-02-01 00:59:50 +08:00
Will Miao
ffcfe5ea3e fix(metadata): rename model_type to sub_type and add embedding subtype, see #797
- Change `model_type` field to `sub_type` for checkpoint models to improve naming consistency
- Add `sub_type="embedding"` for embedding models to properly categorize model subtypes
- Maintain backward compatibility with existing metadata structure
2026-01-31 22:54:53 +08:00
Will Miao
719e18adb6 feat(media): add media type hint support for file extension detection, fixes #795 and fixes #751
- Add optional `media_type_hint` parameter to `_get_file_extension_from_content_or_headers` method
- When `media_type_hint` is "video" and no extension can be determined from content/headers/URL, default to `.mp4`
- Pass image metadata type as hint in both `process_example_images` and `process_example_images_batch` methods
- Add unit tests to verify media type hint behavior and priority
2026-01-31 19:39:37 +08:00
Will Miao
84c62f2954 refactor(model-type): complete phase 5 cleanup by removing deprecated model_type field
- Remove backward compatibility code for `model_type` in `ModelScanner._build_cache_entry()`
- Update `CheckpointScanner` to only handle `sub_type` in `adjust_metadata()` and `adjust_cached_entry()`
- Delete deprecated aliases `resolve_civitai_model_type` and `normalize_civitai_model_type` from `model_query.py`
- Update frontend components (`RecipeModal.js`, `ModelCard.js`, etc.) to use `sub_type` instead of `model_type`
- Update API response format to return only `sub_type`, removing `model_type` from service responses
- Revise technical documentation to mark Phase 5 as completed and remove outdated TODO items

All cleanup tasks for the model type refactoring are now complete, ensuring consistent use of `sub_type` across the codebase.
2026-01-30 07:48:31 +08:00
Will Miao
5e91073476 refactor: unify model_type semantics by introducing sub_type field
This commit resolves the semantic confusion around the model_type field by
clearly distinguishing between:
- scanner_type: architecture-level (lora/checkpoint/embedding)
- sub_type: business-level subtype (lora/locon/dora/checkpoint/diffusion_model/embedding)

Backend Changes:
- Rename model_type to sub_type in CheckpointMetadata and EmbeddingMetadata
- Add resolve_sub_type() and normalize_sub_type() in model_query.py
- Update checkpoint_scanner to use _resolve_sub_type()
- Update service format_response to include both sub_type and model_type
- Add VALID_*_SUB_TYPES constants with backward compatible aliases

Frontend Changes:
- Add MODEL_SUBTYPE_DISPLAY_NAMES constants
- Keep MODEL_TYPE_DISPLAY_NAMES as backward compatible alias

Testing:
- Add 43 new tests covering sub_type resolution and API response

Documentation:
- Add refactoring todo document to docs/technical/

BREAKING CHANGE: None - full backward compatibility maintained
2026-01-30 06:56:10 +08:00
Will Miao
08267cdb48 refactor(filter): extract preset management logic into FilterPresetManager
Move filter preset creation, deletion, application, and storage logic
from FilterManager into a dedicated FilterPresetManager class to
improve separation of concerns and maintainability.

- Add FilterPresetManager with preset CRUD operations
- Update FilterManager to use preset manager via composition
- Handle EMPTY_WILDCARD_MARKER for wildcard base model filters
- Add preset-related translations to all locale files
- Update filter preset UI styling and interactions
2026-01-29 16:32:41 +08:00
Will Miao
2eea92abdf fix: allow STRING input connections for AUTOCOMPLETE_TEXT_PROMPT widgets
Use union type "AUTOCOMPLETE_TEXT_PROMPT,STRING" to enable input mode
compatibility with STRING outputs while preserving autocomplete widget
functionality via widgetType option.

Fixes issue where text inputs could not receive connections from
STRING-type outputs after changing from built-in STRING to custom
AUTOCOMPLETE_TEXT_PROMPT type.

Affected nodes:
- Prompt (LoraManager)
- Text (LoraManager)
2026-01-29 09:07:22 +08:00
Will Miao
58ae6b9de6 fix: persist onboarding and banner dismiss state to backend
Moves onboarding_completed and dismissed_banners from localStorage
to backend settings (settings.json) to survive incognito/private
browser modes.

Fixes #786
2026-01-29 08:48:04 +08:00
Will Miao
565b61d1c2 feat: add Text node with autocomplete support
Introduce a new TextLM node to the Lora Manager extension, providing a simple text input with autocomplete functionality for tags and styles. The node is integrated into the module's import system and node class mappings, enabling users to utilize autocomplete features for efficient prompt creation.
2026-01-28 11:39:05 +08:00
Will Miao
5dc5fd5971 feat(tag-search): add alias support to FTS index
- Add aliases column to tags table to store comma-separated alias lists
- Update FTS schema to version 2 with searchable_text field containing tag names and aliases
- Implement schema migration to rebuild index when upgrading from old schema
- Modify search logic to match aliases and return canonical tag with matched alias info
- Update index building to include aliases in searchable text for FTS matching

This enables users to search for tag aliases (e.g., "miku") and get results for the canonical tag (e.g., "hatsune_miku") with indication of which alias was matched.
2026-01-27 00:36:06 +08:00
Will Miao
7249c9fd4b refactor(autocomplete): remove old CSV fallback, use TagFTSIndex exclusively
Remove all autocomplete.txt parsing logic and fallback code, simplifying
the service to use only TagFTSIndex for Danbooru/e621 tag search
with category filtering.

- Remove WordEntry dataclass and _words_cache, _file_path attributes
- Remove _determine_file_path(), get_file_path(), load_words(), save_words(),
  get_content(), _parse_csv_content() methods
- Simplify search_words() to only use TagFTSIndex, always returning
  enriched results with {tag_name, category, post_count}
- Remove GET/POST /api/lm/custom-words endpoints (unused)
- Keep GET /api/lm/custom-words/search for frontend autocomplete
- Rewrite tests to focus on TagFTSIndex integration

This reduces code by 446 lines and removes untested pysssss plugin
integration. Feature is unreleased so no backward compatibility needed.
2026-01-26 20:36:00 +08:00
pixelpaws
93cd0b54dc Merge branch 'main' into fix-symlink 2026-01-26 17:29:31 +08:00
Will Miao
7b0c6c8bab refactor(cache): reorganize cache directory structure with automatic legacy cleanup
- Centralize cache path resolution in new py/utils/cache_paths.py module
- Migrate legacy cache files to organized structure: {settings_dir}/cache/{model|recipe|fts|symlink}/
- Automatically clean up legacy files after successful migration with integrity verification
- Update Config symlink cache to use new path and migrate from old location
- Simplify service classes (PersistentModelCache, PersistentRecipeCache, RecipeFTSIndex, TagFTSIndex) to use centralized migration logic
- Add comprehensive test coverage for cache paths and automatic cleanup
2026-01-26 16:12:08 +08:00
Will Miao
4b36d60e46 feat(prompt): enhance placeholder with quick tag search instructions
Update the placeholder text in the PromptLM class to include guidance for quick tag search functionality. The new placeholder now reads "Enter prompt... /char, /artist for quick tag search", providing users with immediate cues on how to utilize tag search features directly within the input field. This improves usability by making advanced functionality more discoverable.
2026-01-26 14:42:47 +08:00
Will Miao
42f35be9d3 feat(autocomplete): add Danbooru/e621 tag search with category filtering
- Add TagFTSIndex service for fast SQLite FTS5-based tag search (221k+ tags)
- Implement command-mode autocomplete: /char, /artist, /general, /meta, etc.
- Support category filtering via category IDs or names
- Return enriched results with post counts and category badges
- Add UI styling for category badges and command list dropdown
2026-01-26 13:51:45 +08:00
Will Miao
d063d48417 feat(symlink): add deep validation for symlink cache invalidation
Detects symlink changes at any depth, not just at root level. Uses two-tier validation:
- Fingerprint check for new symlinks
- Deep mapping validation for removed/retargeted symlinks
2026-01-26 09:30:10 +08:00