Commit Graph

881 Commits

Author SHA1 Message Date
Will Miao
565b61d1c2 feat: add Text node with autocomplete support
Introduce a new TextLM node to the Lora Manager extension, providing a simple text input with autocomplete functionality for tags and styles. The node is integrated into the module's import system and node class mappings, enabling users to utilize autocomplete features for efficient prompt creation.
2026-01-28 11:39:05 +08:00
Will Miao
5dc5fd5971 feat(tag-search): add alias support to FTS index
- Add aliases column to tags table to store comma-separated alias lists
- Update FTS schema to version 2 with searchable_text field containing tag names and aliases
- Implement schema migration to rebuild index when upgrading from old schema
- Modify search logic to match aliases and return canonical tag with matched alias info
- Update index building to include aliases in searchable text for FTS matching

This enables users to search for tag aliases (e.g., "miku") and get results for the canonical tag (e.g., "hatsune_miku") with indication of which alias was matched.
2026-01-27 00:36:06 +08:00
Will Miao
7249c9fd4b refactor(autocomplete): remove old CSV fallback, use TagFTSIndex exclusively
Remove all autocomplete.txt parsing logic and fallback code, simplifying
the service to use only TagFTSIndex for Danbooru/e621 tag search
with category filtering.

- Remove WordEntry dataclass and _words_cache, _file_path attributes
- Remove _determine_file_path(), get_file_path(), load_words(), save_words(),
  get_content(), _parse_csv_content() methods
- Simplify search_words() to only use TagFTSIndex, always returning
  enriched results with {tag_name, category, post_count}
- Remove GET/POST /api/lm/custom-words endpoints (unused)
- Keep GET /api/lm/custom-words/search for frontend autocomplete
- Rewrite tests to focus on TagFTSIndex integration

This reduces code by 446 lines and removes untested pysssss plugin
integration. Feature is unreleased so no backward compatibility needed.
2026-01-26 20:36:00 +08:00
pixelpaws
93cd0b54dc Merge branch 'main' into fix-symlink 2026-01-26 17:29:31 +08:00
Will Miao
7b0c6c8bab refactor(cache): reorganize cache directory structure with automatic legacy cleanup
- Centralize cache path resolution in new py/utils/cache_paths.py module
- Migrate legacy cache files to organized structure: {settings_dir}/cache/{model|recipe|fts|symlink}/
- Automatically clean up legacy files after successful migration with integrity verification
- Update Config symlink cache to use new path and migrate from old location
- Simplify service classes (PersistentModelCache, PersistentRecipeCache, RecipeFTSIndex, TagFTSIndex) to use centralized migration logic
- Add comprehensive test coverage for cache paths and automatic cleanup
2026-01-26 16:12:08 +08:00
Will Miao
4b36d60e46 feat(prompt): enhance placeholder with quick tag search instructions
Update the placeholder text in the PromptLM class to include guidance for quick tag search functionality. The new placeholder now reads "Enter prompt... /char, /artist for quick tag search", providing users with immediate cues on how to utilize tag search features directly within the input field. This improves usability by making advanced functionality more discoverable.
2026-01-26 14:42:47 +08:00
Will Miao
42f35be9d3 feat(autocomplete): add Danbooru/e621 tag search with category filtering
- Add TagFTSIndex service for fast SQLite FTS5-based tag search (221k+ tags)
- Implement command-mode autocomplete: /char, /artist, /general, /meta, etc.
- Support category filtering via category IDs or names
- Return enriched results with post counts and category badges
- Add UI styling for category badges and command list dropdown
2026-01-26 13:51:45 +08:00
Will Miao
d063d48417 feat(symlink): add deep validation for symlink cache invalidation
Detects symlink changes at any depth, not just at root level. Uses two-tier validation:
- Fingerprint check for new symlinks
- Deep mapping validation for removed/retargeted symlinks
2026-01-26 09:30:10 +08:00
Will Miao
c9e305397c feat: enhance symlink detection and cache invalidation
- Add `_entry_is_symlink` method to detect symlinks and Windows junctions
- Include first-level symlinks in fingerprint for better cache invalidation
- Re-enable preview path validation for security
- Update tests to verify retargeted symlinks trigger rescan
2026-01-25 19:14:16 +08:00
Will Miao
d5a2bd1e24 feat: add custom words autocomplete support for Prompt node
Adds custom words autocomplete functionality similar to comfyui-custom-scripts,
with the following features:

Backend (Python):
- Create CustomWordsService for CSV parsing and priority-based search
- Add API endpoints: GET/POST /api/lm/custom-words and
  GET /api/lm/custom-words/search
- Share storage with pysssss plugin (checks for their user/autocomplete.txt first)
- Fallback to Lora Manager's user directory for storage

Frontend (JavaScript/Vue):
- Add 'custom_words' and 'prompt' model types to autocomplete system
- Prompt node now supports dual-mode autocomplete:
  * Type 'emb:' prefix → search embeddings
  * Type normally → search custom words (no prefix required)
- Add AUTOCOMPLETE_TEXT_PROMPT widget type
- Update Vue component and composable types

Key Features:
- CSV format: word[,priority] compatible with danbooru-tags.txt
- Priority-based sorting: 20% top priority + prefix + include matches
- Preview tooltip for embeddings (not for custom words)
- Dynamic endpoint switching based on prefix detection

Breaking Changes:
- Prompt (LoraManager) node widget type changed from
  AUTOCOMPLETE_TEXT_EMBEDDINGS to AUTOCOMPLETE_TEXT_PROMPT
- Removed standalone web/comfyui/prompt.js (integrated into main widgets)

Fixes comfy_dir path calculation by prioritizing folder_paths.base_path
from ComfyUI when available, with fallback to computed path.
2026-01-25 12:24:32 +08:00
Will Miao
41101ad5c6 refactor(nodes): standardize node class names with LM suffix
Rename all node classes to use consistent 'LM' suffix pattern:
- LoraCyclerNode → LoraCyclerLM
- LoraManagerLoader → LoraLoaderLM
- LoraManagerTextLoader → LoraTextLoaderLM
- LoraStacker → LoraStackerLM
- LoraRandomizerNode → LoraRandomizerLM
- LoraPoolNode → LoraPoolLM
- WanVideoLoraSelectFromText → WanVideoLoraTextSelectLM
- DebugMetadata → DebugMetadataLM
- TriggerWordToggle → TriggerWordToggleLM
- PromptLoraManager → PromptLM

Updated:
- Core node class definitions (9 files)
- NODE_CLASS_MAPPINGS in __init__.py
- Node type mappings in node_extractors.py
- All related test imports and references
- Logger prefixes for consistency

Frontend extension names remain unchanged (LoraManager.LoraStacker, etc.)
2026-01-25 10:38:10 +08:00
Will Miao
d655fb8008 feat(nodes): improve placeholder text for LoRA autocomplete input 2026-01-25 09:10:16 +08:00
Will Miao
194f2f702c refactor: replace comfy built-in text widget with custome autocomplete text widget for better event handler binding
- Change `STRING` input type to `AUTOCOMPLETE_TEXT_LORAS` in LoraManagerLoader, LoraStacker, and WanVideoLoraSelectLM nodes for LoRA syntax input
- Change `STRING` input type to `AUTOCOMPLETE_TEXT_EMBEDDINGS` in PromptLoraManager node for prompt input
- Remove manual multiline, autocomplete, and dynamicPrompts configurations in favor of built-in autocomplete types
- Update placeholder text for consistency across nodes
- Remove unused `setupInputWidgetWithAutocomplete` mock from frontend tests
- Add Vue app cleanup logic to prevent memory leaks in widget management
2026-01-25 08:30:06 +08:00
Will Miao
eb2af454cc feat: add SQLite-based persistent recipe cache for faster startup
Introduce a new PersistentRecipeCache service that stores recipe metadata in an SQLite database to significantly reduce application startup time. The cache eliminates the need to walk directories and parse JSON files on each launch by persisting recipe data between sessions.

Key features:
- Thread-safe singleton implementation with library-specific instances
- Automatic schema initialization and migration support
- JSON serialization for complex recipe fields (LoRAs, checkpoints, generation parameters, tags)
- File system monitoring with mtime/size validation for cache invalidation
- Environment variable toggle (LORA_MANAGER_DISABLE_PERSISTENT_CACHE) for debugging
- Comprehensive test suite covering save/load cycles, cache invalidation, and edge cases

The cache improves user experience by enabling near-instantaneous recipe loading after the initial cache population, while maintaining data consistency through file change detection.
2026-01-23 22:56:38 +08:00
Will Miao
7bba24c19f feat(update-modal): display last 5 release notes instead of single
- Modified backend to fetch last 5 releases from GitHub API
- Updated frontend to iterate through and display multiple releases
- Added latest badge and publish date styling
- Added update.latestBadge translation key to all locales
- Maintains backward compatibility for single changelog display
2026-01-23 22:22:48 +08:00
Will Miao
0bb75fdf77 feat(trigger-word-toggle): use trigger_words directly when it differs from original message 2026-01-23 09:50:53 +08:00
Will Miao
2121054cb9 feat(lora-cycler): implement batch queue synchronization with dual-index mechanism
- Add execution_index and next_index fields to CyclerConfig interface
- Introduce beforeQueued hook in widget to handle index shifting for batch executions
- Use execution_index when provided, fall back to current_index for single executions
- Track execution state with Symbol to differentiate first vs subsequent executions
- Update state management to handle dual-index logic for proper LoRA cycling in batch queues
2026-01-22 21:22:52 +08:00
Will Miao
b0f852cc6c refactor(lora-cycler): remove sort by control, always use filename
Removed the sort by selection UI from the Lora Cycler widget and
hardcoded the sorting to always use filename. This simplifies the
interface while maintaining all sorting functionality.

Changes:
- Removed sort_by prop/emit from LoraCyclerSettingsView
- Removed sort tabs UI and associated styles
- Hardcoded sort_by = "filename" in backend node
- Removed sort by handling logic from LoraCyclerWidget
- Updated widget height to accommodate removal
2026-01-22 19:58:51 +08:00
Will Miao
6fbea77137 feat(lora-cycler): add sequential LoRA cycling through filtered pool
Add Lora Cycler node that cycles through LoRAs sequentially from a filtered pool. Supports configurable sort order, strength settings, and persists cycle progress across workflow save/load.

Backend:
- New LoraCyclerNode with cycle() method
- New /api/lm/loras/cycler-list endpoint
- LoraService.get_cycler_list() for filtered/sorted list

Frontend:
- LoraCyclerWidget with Vue.js component
- useLoraCyclerState composable
- LoraCyclerSettingsView for UI display
2026-01-22 15:36:32 +08:00
Will Miao
17c5583297 fix(fts): fix multi-word field-restricted search query building
Fixes a critical bug in FTS query building where multi-word searches
with field restrictions incorrectly used OR between all word+field
combinations instead of requiring ALL words to match within at least
one field.

Example: searching "cute cat" in {title, tags} previously produced:
  title:cute* OR title:cat* OR tags:cute* OR tags:cat*
Which matched recipes with ANY word in ANY field.

Now produces:
  (title:cute* title:cat*) OR (tags:cute* tags:cat*)
Which requires ALL words to match within at least one field.

Also adds fallback to fuzzy search when FTS returns empty results,
improving search reliability.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-01-22 10:25:03 +08:00
Will Miao
50abd85fae fix(previews): temporarily bypass path validation to restore preview functionality
Temporary workaround for issues #772 and #774 where valid previews
are rejected. Path validation is disabled until proper fix for
preview root path handling is implemented.
2026-01-21 11:33:42 +08:00
Will Miao
6f74186498 feat(config): add debug logging for preview root operations, see #772 and #774
- Log preview root rebuilding with counts of different root types
- Add detailed debug output when preview paths are rejected
- Improve visibility into path mapping and validation processes
2026-01-21 09:22:42 +08:00
Will Miao
eb8b95176b fix(config): return normalized path in link mapping methods
Previously, `map_path_to_link` and `map_link_to_path` returned the original input path when no mapping was found, instead of the normalized version. This could cause inconsistencies when paths with different representations (e.g., trailing slashes) were used. Now both methods consistently return the normalized path, ensuring uniform path handling throughout the application.
2026-01-21 09:09:02 +08:00
Will Miao
379e3ce2f6 feat(config): normalize paths for case-insensitive comparison on Windows, see #774 and #772
Use os.path.normcase to ensure case-insensitive path matching on Windows, addressing issues where drive letter case mismatches (e.g., 'a:/folder' vs 'A:/folder') prevented correct detection of paths under preview roots. Replace Path.relative_to() with string-based comparison for consistent behavior across platforms.
2026-01-21 08:32:22 +08:00
Will Miao
fd06086a05 feat(lora_randomizer): implement dual seed mechanism for batch queue synchronization, fixes #773
- Add execution_seed and next_seed parameters to support deterministic randomization across batch executions
- Separate UI display generation from execution stack generation to maintain consistency in batch queues
- Update LoraService to accept optional seed parameter for reproducible randomization
- Ensure each execution with a different seed produces unique results without affecting global random state
2026-01-21 00:52:08 +08:00
Will Miao
1dee7f5cf9 feat(constants): standardize formatting and expand diffusion model list
- Normalize string quotes to double quotes across all constants for consistency
- Add trailing commas in dictionaries and lists to improve diff readability
- Expand DIFFUSION_MODEL_BASE_MODELS with additional Wan Video and Qwen models
- Fix comment spacing in NSFW_LEVELS dictionary
- Maintain all existing functionality while improving code style
2026-01-19 01:20:46 +08:00
Will Miao
b0f0158f98 feat(example-images): add force parameter to retry failed downloads
When force=true is passed via API, models in failed_models set are
re-downloaded instead of being skipped. On successful download, model is
removed from failed_models set.

This provides a manual batch repair mechanism for users when CivitAI
media server is temporarily down and causes empty folders.

Changes:
- Backend: Add force parameter to start_download(), _download_all_example_images(), _process_model()
- Backend: Skip failed_models check when force=true
- Backend: Remove model from failed_models on successful force retry
- Frontend: GlobalContextMenu now calls API with force=true directly
- Tests: Update mock to accept force parameter
2026-01-18 21:58:12 +08:00
Will Miao
7f2e8a0afb feat(search): add SQLite FTS5 full-text search index for recipes
Introduce a new RecipeFTSIndex class that provides fast prefix-based search across recipe fields (title, tags, LoRA names/models, prompts) using SQLite's FTS5 extension. The implementation supports sub-100ms search times for large datasets (20k+ recipes) and includes asynchronous indexing, incremental updates, and comprehensive unit tests.
2026-01-18 20:44:22 +08:00
Will Miao
f0c852ef23 fix(randomizer): convert numeric config values to proper types to prevent string subtraction errors 2026-01-18 12:40:58 +08:00
Will Miao
839bcbd37f fix(settings): add default_unet_root to SYNC_KEYS for proper frontend sync
The default_unet_root setting was not being synced from backend to frontend
because it was missing from the _SYNC_KEYS tuple in misc_handlers.py. This
caused the "Default Diffusion Model Root" setting to always display "No Default"
even when a valid path was configured in settings.json.
2026-01-18 12:38:46 +08:00
Will Miao
dad549f65f feat(download): auto-route diffusion models to unet folder based on baseModel, see #770
CivitAI does not distinguish between checkpoint and diffusion model types -
both are labeled as "checkpoint". For certain base model types like
"ZImageTurbo", all models are actually diffusion models and should be
saved to the unet/diffusion model folder instead of the checkpoint folder.

- Add DIFFUSION_MODEL_BASE_MODELS constant for known diffusion model types
- Add default_unet_root setting with auto-set logic
- Route downloads to unet folder when baseModel matches known diffusion types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 11:58:20 +08:00
Will Miao
aab1797269 Revert "feat: add automatic retry for failed example image downloads"
This reverts commit cb460fcdb0.
2026-01-18 10:55:30 +08:00
Will Miao
cb460fcdb0 feat: add automatic retry for failed example image downloads
- Add failed_model_timestamps to track when models fail
- Retry failed models after 24-hour cooldown period
- Skip retry if example folder already has files
- Skip retry if failure was less than 24 hours ago
- Log count of failed models with retry message
- Fix unbound snapshot variable in exception path
- Remove duplicate/unreachable directory check code
- Update string quotes to double quotes (PEP 8)

This fixes the issue where failed models were permanently skipped in
auto-download mode, even when their example folders were empty.
2026-01-18 08:55:49 +08:00
Will Miao
07d599810d feat(debug): replace websocket with ComfyUI UI system for metadata display
- Update DebugMetadata node to return metadata via ComfyUI's UI system instead of websocket
- Add new JsonDisplayWidget Vue component for displaying metadata in the UI
- Remove dependency on PromptServer and websocket communication
- Improve error handling with proper UI feedback
- Maintain backward compatibility with existing metadata collection system
2026-01-16 21:29:53 +08:00
Will Miao
ad7d372887 fix: use CivArchive provider when source is 'civarchive' (#769)
When users paste CivArchive URLs, the system now fetches metadata from
CivArchive API first instead of Civitai. This prevents download failures
when a model has been deleted from Civitai but remains available on
CivArchive with alternative mirrors.

Changes:
- Source-aware metadata fetching: Uses CivArchive API when source='civarchive'
- URL prioritization: Prefers non-Civitai mirrors for CivArchive downloads
- Fallback mechanism: Falls back to default provider if CivArchive fails

Fixes #769
2026-01-16 10:57:22 +08:00
Will Miao
428a2ce420 fix: support multiple include folders in LoRA pool widget
- Add folder_include parameter support in backend API handlers
- Add folder_include to FilterCriteria and implement multi-folder filtering logic
- Update frontend to send all include folders instead of only the first
- Add tests for single/multiple include folders, include with exclude, and non-recursive filtering
2026-01-15 15:17:33 +08:00
Will Miao
9ed5319ad2 refactor: remove Lora Demo node
Remove the Lora Demo Node (LoraDemoNode) and all related imports and mappings
from the codebase.
2026-01-14 22:44:53 +08:00
Will Miao
fc8240e99e feat: add "Respect Recommended Strength" feature to LoRA Randomizer
Add support for respecting recommended strength values from LoRA usage_tips
when randomizing LoRA selection.

Features:
- New toggle setting to enable/disable recommended strength respect (default off)
- Scale range slider (0-2, default 0.5-1.0) to adjust recommended values
- Uses recommended strength × random(scale) when feature enabled
- Fallbacks to original Model/Clip Strength range when no recommendation exists
- Clip strength recommendations only apply when using Custom Range mode

Backend changes:
- Parse usage_tips JSON string to extract strength/clipStrength
- Apply scale factor to recommended values during randomization
- Pass new parameters through API route and node

Frontend changes:
- Update RandomizerConfig type with new properties
- Add new UI section with toggle and dual-range slider
- Wire up state management and event handlers
- No layout shift (removed description text)

Tests:
- Add tests for enabled/disabled recommended strength in API routes
- Add test verifying config passed to service
- All existing tests pass

Build: Include compiled Vue widgets
2026-01-14 16:34:24 +08:00
Will Miao
4951ff358e feat: add WSL and Docker support for file location opening
- Add WSL detection and Windows path conversion using wslpath
- Add Docker/Kubernetes detection via /.dockerenv and /proc/1/cgroup
- Implement clipboard fallback for containerized environments
- Update open_file_location handler to detect WSL/Docker before POSIX
- Update open_settings_location handler with same detection logic
- Add clipboard API integration with graceful fallback in frontend
- Add translations for clipboard feature across all 10 languages
- Add unit tests for _is_wsl(), _is_docker(), and _wsl_to_windows_path()

Fixes file manager opening failures in WSL and Docker environments.
2026-01-14 15:49:35 +08:00
Will Miao
bc08a45214 feat: improve code formatting and readability in model handlers
- Add blank line after module docstring for better PEP 8 compliance
- Reformat long lines to adhere to 88-character limit using Black-style formatting
- Improve string consistency by using double quotes consistently
- Enhance readability of complex list comprehensions and method calls
- Maintain all existing functionality while improving code structure
2026-01-13 22:57:15 +08:00
Will Miao
514846cd4a feat(lora-randomizer): refactor randomization logic and add input preprocessing
- Add `_preprocess_loras_input` method to handle different widget input formats
- Move core randomization logic to `LoraService` for better separation of concerns
- Update `_select_loras` method to use new service-based approach
- Add comprehensive test fixtures for license filtering scenarios
- Include debug print statement for pool config inspection during development

This refactor improves code organization by centralizing business logic in the service layer while maintaining backward compatibility with existing widget inputs.
2026-01-13 15:47:59 +08:00
Will Miao
1ebd2c93a0 feat: add .opencode to gitignore and refactor lora routes
- Add .opencode directory to gitignore for agent-related files
- Refactor lora_routes.py with consistent string formatting and improved route registration
- Add DualRangeSlider Vue component for enhanced UI controls
2026-01-13 13:59:36 +08:00
Will Miao
6a17e75782 docs: add frontend UI architecture and ComfyUI widget guidelines
- Document dual UI systems: standalone web UI and ComfyUI custom node widgets
- Add ComfyUI widget development guidelines including styling and constraints
- Update terminology in LoraRandomizerNode from 'frontend/backend' to 'fixed/always' for clarity
- Include UI constraints for ComfyUI widgets: minimize vertical space, avoid dynamic height changes, keep UI simple
2026-01-13 11:20:50 +08:00
Will Miao
bce6b0e610 feat(randomizer): add LoRA locking and roll modes
- Implement LoRA locking to prevent specific LoRAs from being changed during randomization
- Add visual styling for locked state with amber accents and distinct backgrounds
- Introduce `roll_mode` configuration with 'backend' (execute current selection while generating new) and 'frontend' (execute newly generated selection) behaviors
- Move LoraPoolNode to 'Lora Manager/randomizer' category and remove standalone class mappings
- Standardize RETURN_NAMES in LoraRandomizerNode for consistency
2026-01-12 21:53:47 +08:00
Will Miao
177b20263d feat: add LoraDemoNode and LoraRandomizerNode with documentation
- Import and register two new nodes: LoraDemoNode and LoraRandomizerNode
- Update import exception handling for better readability with multi-line formatting
- Add comprehensive documentation file `docs/custom-node-ui-output.md` for UI output usage in custom nodes
- Ensure proper node registration in NODE_CLASS_MAPPINGS for ComfyUI integration
- Maintain backward compatibility with existing node structure and import fallbacks
2026-01-12 15:06:38 +08:00
Will Miao
647728b2e1 feat: rename demo widget to lora-manager-widgets and remove demo node
- Update documentation to reflect new widget filename `lora-manager-widgets.js`
- Remove `LoraManagerDemoNode` import and registration from `__init__.py`
- Translate development guide from Chinese to English for broader accessibility
- Clean up obsolete demo references to align with actual widget implementation
2026-01-11 19:08:55 +08:00
Will Miao
3d348900ac feat(randomizer): add lora pool Vue widget 2026-01-11 16:26:38 +08:00
Will Miao
32249d1886 feat: add Vue widget demo node and development support
- Add LoraManagerDemoNode to node mappings for Vue widget demonstration
- Update .gitignore to exclude Vue widget development artifacts (node_modules, .vite, dist)
- Implement automatic Vue widget build check in development mode with fallback handling
- Maintain pytest compatibility with proper import error handling
2026-01-10 17:45:26 +08:00
Will Miao
f842ea990e feat(metadata): prevent overwriting high-quality Civitai API metadata with archive data. See #764
- Update `is_civitai_api_metadata` to exclude both "archive_db" and "civarchive" sources
- Skip Civitai metadata updates when existing metadata is higher quality than incoming archive data
- Add test to verify API metadata is preserved when CivArchive provides lower-quality data
2026-01-09 19:47:32 +08:00
Will Miao
f62b3f62be feat(recipe_scanner): prioritize local sibling images and persist repairs
Updated image path resolution logic to prioritize local sibling images in the same directory as recipes. When a stored image path differs from a local sibling, the system now automatically updates the recipe file to use the local path and persists this repair. This improves reliability when recipe assets are moved or reorganized, ensuring images remain accessible even if original paths become invalid.
2026-01-08 15:53:01 +08:00