Compare commits

...

903 Commits

Author SHA1 Message Date
Will Miao
9150718edb feat: bump version to 0.9.13
Update the project version in pyproject.toml from 0.9.12 to 0.9.13 to reflect the latest changes and prepare for a new release.
2026-01-21 21:20:34 +08:00
Will Miao
50abd85fae fix(previews): temporarily bypass path validation to restore preview functionality
Temporary workaround for issues #772 and #774 where valid previews
are rejected. Path validation is disabled until proper fix for
preview root path handling is implemented.
2026-01-21 11:33:42 +08:00
Will Miao
7b4607bed7 feat(standalone): add --verbose flag for DEBUG logging
Add --verbose command line argument that enables DEBUG level logging, equivalent to --log-level DEBUG
2026-01-21 09:35:28 +08:00
Will Miao
6f74186498 feat(config): add debug logging for preview root operations, see #772 and #774
- Log preview root rebuilding with counts of different root types
- Add detailed debug output when preview paths are rejected
- Improve visibility into path mapping and validation processes
2026-01-21 09:22:42 +08:00
Will Miao
eb8b95176b fix(config): return normalized path in link mapping methods
Previously, `map_path_to_link` and `map_link_to_path` returned the original input path when no mapping was found, instead of the normalized version. This could cause inconsistencies when paths with different representations (e.g., trailing slashes) were used. Now both methods consistently return the normalized path, ensuring uniform path handling throughout the application.
2026-01-21 09:09:02 +08:00
Will Miao
091d8aba39 feat(tests): add case-insensitive path validation tests for Windows
Add two new test cases to verify preview path validation behavior on Windows:

1. `test_is_preview_path_allowed_case_insensitive_on_windows`: Ensures path validation is case-insensitive on Windows, addressing issues where drive letters and paths with different cases should match. This resolves GitHub issues #772 and #774.

2. `test_is_preview_path_allowed_rejects_prefix_without_separator`: Prevents false positives by ensuring paths are only allowed when they match the root path exactly followed by a separator, not just sharing a common prefix.
2026-01-21 08:49:41 +08:00
Will Miao
379e3ce2f6 feat(config): normalize paths for case-insensitive comparison on Windows, see #774 and #772
Use os.path.normcase to ensure case-insensitive path matching on Windows, addressing issues where drive letter case mismatches (e.g., 'a:/folder' vs 'A:/folder') prevented correct detection of paths under preview roots. Replace Path.relative_to() with string-based comparison for consistent behavior across platforms.
2026-01-21 08:32:22 +08:00
Will Miao
1b7b598f7a feat(sliders): adjust value label positioning and line height
- Move slider handle value labels 6px upward in both DualRangeSlider and SingleSlider components
- Add consistent line-height of 14px to ensure proper text alignment
- Improves visual spacing and readability of value labels during slider interaction
2026-01-21 01:05:15 +08:00
Will Miao
fd06086a05 feat(lora_randomizer): implement dual seed mechanism for batch queue synchronization, fixes #773
- Add execution_seed and next_seed parameters to support deterministic randomization across batch executions
- Separate UI display generation from execution stack generation to maintain consistency in batch queues
- Update LoraService to accept optional seed parameter for reproducible randomization
- Ensure each execution with a different seed produces unique results without affecting global random state
2026-01-21 00:52:08 +08:00
Will Miao
50c012ae33 fix(ui): unify Lora Randomizer widget styles with Loras widget
Align visual design of Lora Randomizer widget with Loras widget for
consistent UI/UX across the node interface.

Changes:
- Unified border-radius system (4px→6px for containers, 6px for inputs)
- Standardized padding (12px→6px for widget container)
- Reduced slider height (32px→24px) following desktop tool best practices
- Aligned font sizes (12px→13px for labels, 11px→12px for buttons)
- Unified spacing system (16px→6px for sections, 8px→6px for gaps)
- Adjusted widget minimum height (510px→448px) to reflect layout changes
2026-01-20 20:38:24 +08:00
Will Miao
796acba764 chore: bump version from 0.9.11 to 0.9.12
Update the project version in pyproject.toml to prepare for the next release.
2026-01-19 17:37:19 +08:00
Will Miao
3aab0cc916 feat: add v0.9.12 release notes and update LoRA Randomizer workflow example
- Introduce LoRA Randomizer system with LoRA Pool and Randomizer nodes
- Add recipe folders, bulk operations, search, sorting, and favorites
- Enable video recipe support and ComfyUI Nodes 2.0 compatibility
- Include performance improvements for faster startup and loading
- Update example workflow for LoRA Randomizer template reference
2026-01-19 16:23:49 +08:00
Will Miao
4c2c8c2bc8 feat(randomizer): add mode change listener to update downstream trigger words
Add LoraRandomizer extension that monitors node mode changes and triggers
updates to connected downstream trigger word toggle nodes, matching the
behavior implemented for Lora Stacker nodes.
2026-01-19 14:39:44 +08:00
Will Miao
e44180b832 feat(ui): exclude lock button from drag init in LoRA widget
Add `.lm-lora-lock-button` to the list of elements that should not trigger drag initialization in the LoRA widget event handler. This prevents unintended drag actions when interacting with the lock button, improving user experience and interaction clarity.
2026-01-19 12:53:59 +08:00
Will Miao
4ff397e9c1 fix(modals): preserve model type during navigation (#771)
Move cleanupNavigationShortcuts() call before setting navigationModelType
to ensure the correct model type is preserved when using left/right arrow
keys to navigate between models. Previously, the cleanup would immediately
nullify navigationModelType, causing type-specific modal sections (like
trigger words for embeddings and usage tips for loras) to disappear.
2026-01-19 09:17:05 +08:00
Will Miao
633ad2d386 fix(test): add fetch polyfill and update context menu test for new API implementation
- Add fetch polyfill to test setup for jsdom environment
- Update context menu test to match new implementation that uses fetch API
- Remove deprecated handleDownloadButton expectation
- Fix mock indices for multiple fetch calls

Resolves test failures from commit b0f0158 which refactored GlobalContextMenu
to use fetch API directly instead of calling exampleImagesManager.
2026-01-19 08:34:31 +08:00
Will Miao
1dee7f5cf9 feat(constants): standardize formatting and expand diffusion model list
- Normalize string quotes to double quotes across all constants for consistency
- Add trailing commas in dictionaries and lists to improve diff readability
- Expand DIFFUSION_MODEL_BASE_MODELS with additional Wan Video and Qwen models
- Fix comment spacing in NSFW_LEVELS dictionary
- Maintain all existing functionality while improving code style
2026-01-19 01:20:46 +08:00
Will Miao
b0f0158f98 feat(example-images): add force parameter to retry failed downloads
When force=true is passed via API, models in failed_models set are
re-downloaded instead of being skipped. On successful download, model is
removed from failed_models set.

This provides a manual batch repair mechanism for users when CivitAI
media server is temporarily down and causes empty folders.

Changes:
- Backend: Add force parameter to start_download(), _download_all_example_images(), _process_model()
- Backend: Skip failed_models check when force=true
- Backend: Remove model from failed_models on successful force retry
- Frontend: GlobalContextMenu now calls API with force=true directly
- Tests: Update mock to accept force parameter
2026-01-18 21:58:12 +08:00
Will Miao
7f2e8a0afb feat(search): add SQLite FTS5 full-text search index for recipes
Introduce a new RecipeFTSIndex class that provides fast prefix-based search across recipe fields (title, tags, LoRA names/models, prompts) using SQLite's FTS5 extension. The implementation supports sub-100ms search times for large datasets (20k+ recipes) and includes asynchronous indexing, incremental updates, and comprehensive unit tests.
2026-01-18 20:44:22 +08:00
Will Miao
7a7517cfb6 fix(test): add PointerEvent polyfills and update drag interaction test to match implementation 2026-01-18 16:32:01 +08:00
Will Miao
f0c852ef23 fix(randomizer): convert numeric config values to proper types to prevent string subtraction errors 2026-01-18 12:40:58 +08:00
Will Miao
839bcbd37f fix(settings): add default_unet_root to SYNC_KEYS for proper frontend sync
The default_unet_root setting was not being synced from backend to frontend
because it was missing from the _SYNC_KEYS tuple in misc_handlers.py. This
caused the "Default Diffusion Model Root" setting to always display "No Default"
even when a valid path was configured in settings.json.
2026-01-18 12:38:46 +08:00
Will Miao
ab6a4844f0 chore: remove unused md files 2026-01-18 11:59:50 +08:00
Will Miao
dad549f65f feat(download): auto-route diffusion models to unet folder based on baseModel, see #770
CivitAI does not distinguish between checkpoint and diffusion model types -
both are labeled as "checkpoint". For certain base model types like
"ZImageTurbo", all models are actually diffusion models and should be
saved to the unet/diffusion model folder instead of the checkpoint folder.

- Add DIFFUSION_MODEL_BASE_MODELS constant for known diffusion model types
- Add default_unet_root setting with auto-set logic
- Route downloads to unet folder when baseModel matches known diffusion types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 11:58:20 +08:00
Will Miao
aab1797269 Revert "feat: add automatic retry for failed example image downloads"
This reverts commit cb460fcdb0.
2026-01-18 10:55:30 +08:00
Will Miao
cb460fcdb0 feat: add automatic retry for failed example image downloads
- Add failed_model_timestamps to track when models fail
- Retry failed models after 24-hour cooldown period
- Skip retry if example folder already has files
- Skip retry if failure was less than 24 hours ago
- Log count of failed models with retry message
- Fix unbound snapshot variable in exception path
- Remove duplicate/unreachable directory check code
- Update string quotes to double quotes (PEP 8)

This fixes the issue where failed models were permanently skipped in
auto-download mode, even when their example folders were empty.
2026-01-18 08:55:49 +08:00
Will Miao
88e7f671d2 fix(autocomplete): resolve instability in Vue DOM mode and fix WanVideo node binding
- Fix infinite reinitialization loop by only validating stale widget.inputEl when it's actually in DOM
- Improve findWidgetInputElement to specifically search for textarea for text widgets, avoiding mismatches with checkbox inputs on nodes like WanVideo Lora Select that have toggle switches
- Add data-node-id based element search as primary strategy for better reliability across rendering modes
- Fix autocomplete initialization to properly handle element DOM state transitions

Fixes autocomplete failing after Canvas ↔ Vue DOM mode switches and WanVideo node always failing to trigger autocomplete.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
2026-01-17 14:19:20 +08:00
Will Miao
07d599810d feat(debug): replace websocket with ComfyUI UI system for metadata display
- Update DebugMetadata node to return metadata via ComfyUI's UI system instead of websocket
- Add new JsonDisplayWidget Vue component for displaying metadata in the UI
- Remove dependency on PromptServer and websocket communication
- Improve error handling with proper UI feedback
- Maintain backward compatibility with existing metadata collection system
2026-01-16 21:29:53 +08:00
Will Miao
4f3c91b307 feat: migrate LoRA Manager top menu to actionBarButtons API
- Replace custom button creation and attachment logic with built-in actionBarButtons API
- Remove manual DOM manipulation for button positioning and retry logic
- Add custom styling and icon replacement for better visual integration
- Maintain existing functionality for opening LoRA Manager in same/new window
- Simplify extension setup by leveraging ComfyUI's extension system
2026-01-16 21:15:56 +08:00
Will Miao
ad7d372887 fix: use CivArchive provider when source is 'civarchive' (#769)
When users paste CivArchive URLs, the system now fetches metadata from
CivArchive API first instead of Civitai. This prevents download failures
when a model has been deleted from Civitai but remains available on
CivArchive with alternative mirrors.

Changes:
- Source-aware metadata fetching: Uses CivArchive API when source='civarchive'
- URL prioritization: Prefers non-Civitai mirrors for CivArchive downloads
- Fallback mechanism: Falls back to default provider if CivArchive fails

Fixes #769
2026-01-16 10:57:22 +08:00
Will Miao
4e909f3008 fix: enable wheel event handling in tags widget for Vue DOM render mode 2026-01-15 20:15:18 +08:00
Will Miao
bd0dfd4ef5 fix: lora entry selection and strength display issues
- Fix lora entry click-to-select broken after pointer events refactoring
  - Only stopPropagation() after pointer moves beyond 3 pixel threshold
  - This allows click events to fire on lora entries for selection
  - Applied to all drag handlers: initDrag, initHeaderDrag, initReorderDrag

- Fix strength value display to always show 2 decimal places
  - Use toFixed(2) when updating strength input during drag
  - Ensures consistent display (e.g., "1.00" instead of "1", "1.40" instead of "1.4")
2026-01-15 19:04:56 +08:00
Will Miao
c5b597dc89 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2026-01-15 16:05:41 +08:00
Will Miao
bd4958edc3 fix: improve loras widget drag functionality in Vue DOM render mode
- Use pointer events (pointerdown/pointermove/pointerup/pointercancel) with proper capture
- Fix drag not updating strength values by avoiding re-renders during drag
- Fix cursor stuck in resize state by ensuring proper cleanup
- Fix cursor showing wrong icon on hover (should be pointer)
- Ensure strength values display fixed width with 2 decimal places
- Remove unnecessary data-capture-wheel attribute (no wheel adjustment in loras widget)
- Add font-variant-numeric: tabular-nums for consistent number display

This ensures loras widget works consistently in both Canvas and Vue DOM render modes.
2026-01-15 15:57:48 +08:00
Will Miao
428a2ce420 fix: support multiple include folders in LoRA pool widget
- Add folder_include parameter support in backend API handlers
- Add folder_include to FilterCriteria and implement multi-folder filtering logic
- Update frontend to send all include folders instead of only the first
- Add tests for single/multiple include folders, include with exclude, and non-recursive filtering
2026-01-15 15:17:33 +08:00
Will Miao
5636437df2 fix: enable autocomplete in Vue DOM render mode
In Vue DOM render mode, widget.inputEl is not in the DOM, causing autocomplete to fail. This commit:

- Adds findWidgetInputElement() helper to search DOM for actual input elements
- Checks if widget.inputEl is in document before using it
- Falls back to DOM search for Vue-rendered widgets using .lg-node-widget containers
- Implements async initialization with retry logic (20 attempts, 50ms interval)
- Adds debug logging for troubleshooting
- Prevents duplicate initialization with isInitializing flag

Fixes autocomplete functionality for Lora Loader nodes when ComfyUI uses Vue DOM rendering instead of canvas rendering.
2026-01-15 14:34:05 +08:00
Will Miao
10c0668b02 fix: enable prompt search functionality in recipes page
- Add prompt option to recipes default searchOptions state
- Update SearchManager to propagate prompt option to backend
2026-01-15 09:45:03 +08:00
Will Miao
0c67ff85ee build: rebuild Vue widgets with slider compatibility fixes 2026-01-15 07:03:17 +08:00
Will Miao
cde6151c71 fix: make sliders compatible with Vue DOM render mode
Add data-capture-wheel attribute to SingleSlider and DualRangeSlider
components to prevent wheel events from being intercepted by the canvas
in ComfyUI's new Vue DOM render mode. This allows mouse wheel to work
for adjusting slider values while still enabling workflow zoom on
non-interactive widget areas.

Also update event handling to use pointer events with proper stop
propagation and pointer capture for reliable drag operations in both
rendering modes.

Update development guide with Section 8 documenting Vue DOM render mode
event handling patterns and best practices.
2026-01-15 07:03:05 +08:00
Will Miao
9ed5319ad2 refactor: remove Lora Demo node
Remove the Lora Demo Node (LoraDemoNode) and all related imports and mappings
from the codebase.
2026-01-14 22:44:53 +08:00
Will Miao
40756b7dd3 feat: add clear button to search inputs in modals
Add a clear button (X icon) to the search bars in BaseModelModal and TagsModal. The button appears when there is search text, and clicking it clears the search input and refocuses the search field.
2026-01-14 21:38:42 +08:00
Will Miao
2a9ceb9e85 feat: auto-focus search bar in LoRA pool modals 2026-01-14 21:22:52 +08:00
Will Miao
30077099ec fix: improve LoRA Randomizer toggle UX and semantic clarity
- Fix toggle UX consistency: both toggles now follow 'enabled → slider enabled' pattern
- Rename useSameClipStrength to useCustomClipRange for semantic clarity
- Update 'Respect Recommended Strength' label to 'Preset Strength Scale'
- Add explicit conversion logic in composable for backend compatibility
- Add visual disabled state for clip strength slider container
2026-01-14 18:43:46 +08:00
Will Miao
fc8240e99e feat: add "Respect Recommended Strength" feature to LoRA Randomizer
Add support for respecting recommended strength values from LoRA usage_tips
when randomizing LoRA selection.

Features:
- New toggle setting to enable/disable recommended strength respect (default off)
- Scale range slider (0-2, default 0.5-1.0) to adjust recommended values
- Uses recommended strength × random(scale) when feature enabled
- Fallbacks to original Model/Clip Strength range when no recommendation exists
- Clip strength recommendations only apply when using Custom Range mode

Backend changes:
- Parse usage_tips JSON string to extract strength/clipStrength
- Apply scale factor to recommended values during randomization
- Pass new parameters through API route and node

Frontend changes:
- Update RandomizerConfig type with new properties
- Add new UI section with toggle and dual-range slider
- Wire up state management and event handlers
- No layout shift (removed description text)

Tests:
- Add tests for enabled/disabled recommended strength in API routes
- Add test verifying config passed to service
- All existing tests pass

Build: Include compiled Vue widgets
2026-01-14 16:34:24 +08:00
Will Miao
4951ff358e feat: add WSL and Docker support for file location opening
- Add WSL detection and Windows path conversion using wslpath
- Add Docker/Kubernetes detection via /.dockerenv and /proc/1/cgroup
- Implement clipboard fallback for containerized environments
- Update open_file_location handler to detect WSL/Docker before POSIX
- Update open_settings_location handler with same detection logic
- Add clipboard API integration with graceful fallback in frontend
- Add translations for clipboard feature across all 10 languages
- Add unit tests for _is_wsl(), _is_docker(), and _wsl_to_windows_path()

Fixes file manager opening failures in WSL and Docker environments.
2026-01-14 15:49:35 +08:00
Will Miao
73f2a34d08 fix: prevent cursor flickering when dragging slider handles
Fix issue where mouse cursor flickers between 'grabbing' and 'default'
while dragging slider handles. The cursor now remains 'grabbing'
throughout the entire drag operation regardless of mouse position.

Changes:
- Add dynamic 'is-dragging' class to SingleSlider and DualRangeSlider
- Apply cursor: grabbing to root component when dragging state is active
2026-01-14 11:47:47 +08:00
Will Miao
394eebe070 fix: avoid scanner.py false positives in test fixtures
Replace NODE_CLASS_MAPPINGS.update({...}) with direct assignment
to prevent ComfyUI Manager scanner from detecting test mock nodes
as actual plugin nodes.

The scanner.py pattern '_CLASS_MAPPINGS\.update\s*\(\s*{([^}]*)}\s*\)'
was matching test fixtures that use .update() to register mock nodes,
causing false positive conflict warnings.
2026-01-14 10:21:44 +08:00
Will Miao
bc08a45214 feat: improve code formatting and readability in model handlers
- Add blank line after module docstring for better PEP 8 compliance
- Reformat long lines to adhere to 88-character limit using Black-style formatting
- Improve string consistency by using double quotes consistently
- Enhance readability of complex list comprehensions and method calls
- Maintain all existing functionality while improving code structure
2026-01-13 22:57:15 +08:00
Will Miao
0c96e8d328 chore: rename example workflow files to use underscores 2026-01-13 20:03:18 +08:00
Will Miao
859277a7eb feat(ui): enhance tag hover states and adjust toggle switch alignment
- Improve tag chip hover states in TagsModal with contextual colors for include/exclude modes
- Adjust toggle switch thumb vertical alignment in LicenseSection and LoraRandomizerSettingsView
- Remove debug console.log from loras widget value update
2026-01-13 19:54:36 +08:00
Will Miao
9e510d64ec feat(lora-randomizer): prevent early watch triggers by tracking mount state
Add isMounted ref to LoraRandomizerWidget to avoid premature updates from the loras widget watch. The watch now only responds after the component is fully mounted, and the onMounted hook captures the initial loras widget value before enabling the watcher. This prevents the watch from overwriting valid initial data with empty values during component initialization.
2026-01-13 19:22:51 +08:00
Will Miao
430ba84cf7 feat(workflow): add lora randomizer template workflow 2026-01-13 19:17:28 +08:00
Will Miao
0ae2d084f4 feat(lora-randomizer): add segmented scale mode to strength sliders
- Add `scaleMode` and `segments` props to DualRangeSlider component
- Implement segmented scale visualization with configurable segment widths
- Define strength segments for model and clip strength sliders with expanded middle range
- Enable finer control in common value ranges via wheel step multipliers
2026-01-13 16:16:11 +08:00
Will Miao
514846cd4a feat(lora-randomizer): refactor randomization logic and add input preprocessing
- Add `_preprocess_loras_input` method to handle different widget input formats
- Move core randomization logic to `LoraService` for better separation of concerns
- Update `_select_loras` method to use new service-based approach
- Add comprehensive test fixtures for license filtering scenarios
- Include debug print statement for pool config inspection during development

This refactor improves code organization by centralizing business logic in the service layer while maintaining backward compatibility with existing widget inputs.
2026-01-13 15:47:59 +08:00
Will Miao
1ebd2c93a0 feat: add .opencode to gitignore and refactor lora routes
- Add .opencode directory to gitignore for agent-related files
- Refactor lora_routes.py with consistent string formatting and improved route registration
- Add DualRangeSlider Vue component for enhanced UI controls
2026-01-13 13:59:36 +08:00
Will Miao
688baef2f0 feat(dom-widgets): forward middle mouse events to canvas for panning
Add `forwardMiddleMouseToCanvas` utility to forward middle mouse button events from DOM widgets to the ComfyUI canvas, enabling workflow panning when the cursor is over a widget. The function is implemented in `vue-widgets/src/main.ts` and documented in the developer guide. Additionally, fix `getPoolConfigFromConnectedNode` to return null for inactive pool nodes.
2026-01-13 11:45:12 +08:00
Will Miao
6a17e75782 docs: add frontend UI architecture and ComfyUI widget guidelines
- Document dual UI systems: standalone web UI and ComfyUI custom node widgets
- Add ComfyUI widget development guidelines including styling and constraints
- Update terminology in LoraRandomizerNode from 'frontend/backend' to 'fixed/always' for clarity
- Include UI constraints for ComfyUI widgets: minimize vertical space, avoid dynamic height changes, keep UI simple
2026-01-13 11:20:50 +08:00
Will Miao
bce6b0e610 feat(randomizer): add LoRA locking and roll modes
- Implement LoRA locking to prevent specific LoRAs from being changed during randomization
- Add visual styling for locked state with amber accents and distinct backgrounds
- Introduce `roll_mode` configuration with 'backend' (execute current selection while generating new) and 'frontend' (execute newly generated selection) behaviors
- Move LoraPoolNode to 'Lora Manager/randomizer' category and remove standalone class mappings
- Standardize RETURN_NAMES in LoraRandomizerNode for consistency
2026-01-12 21:53:47 +08:00
Will Miao
177b20263d feat: add LoraDemoNode and LoraRandomizerNode with documentation
- Import and register two new nodes: LoraDemoNode and LoraRandomizerNode
- Update import exception handling for better readability with multi-line formatting
- Add comprehensive documentation file `docs/custom-node-ui-output.md` for UI output usage in custom nodes
- Ensure proper node registration in NODE_CLASS_MAPPINGS for ComfyUI integration
- Maintain backward compatibility with existing node structure and import fallbacks
2026-01-12 15:06:38 +08:00
Will Miao
65cede7335 feat(lora-pool): add folder filtering and preview tooltip enhancements
- Add include/exclude folder modals for advanced filtering
- Implement folder tree search with auto-expand functionality
- Add hover tooltip to preview header showing matching LoRA thumbnails
- Format match count with locale string for better readability
- Prevent event propagation on refresh button click
- Improve folder tree component with expand/collapse controls
2026-01-12 10:08:16 +08:00
Will Miao
9719dd4d07 docs(dom_widget_dev_guide): clarify dynamic resizing and add performance note
- Add performance note explaining that providing `getMinHeight` and `getHeight` via `options` avoids expensive DOM measurements
- Expand dynamic resizing section with detailed update sequence and common scenarios table
- Update LoraPoolSummaryView.vue with `min-height: 0` to allow flex shrinking
- Update main.ts to provide `getMinHeight` via options and adjust `computeLayoutSize` for performance
2026-01-12 09:22:18 +08:00
Will Miao
7a5f4514f3 feat(lora-pool): add external value handling and config update support
- Add `onSetValue` callback to handle external updates like workflow loading
- Implement `updateConfig` method for direct widget value updates
- Add value change detection in `restoreFromConfig` to prevent unnecessary updates
- Remove debug console log on component mount
- Extend widget value type to support legacy config format
2026-01-11 20:37:17 +08:00
Will Miao
b44ef9ceaa feat(ui): update LoRA pool widget color scheme and empty state styling
- Change primary accent color from green to blue across multiple components
- Update background colors for better visual consistency
- Improve empty state styling in TagsSection with better padding and background
- Add box-sizing to BaseModelSection for consistent layout
- Update CSS comments to reflect new color scheme
2026-01-11 19:54:44 +08:00
Will Miao
647728b2e1 feat: rename demo widget to lora-manager-widgets and remove demo node
- Update documentation to reflect new widget filename `lora-manager-widgets.js`
- Remove `LoraManagerDemoNode` import and registration from `__init__.py`
- Translate development guide from Chinese to English for broader accessibility
- Clean up obsolete demo references to align with actual widget implementation
2026-01-11 19:08:55 +08:00
Will Miao
3d348900ac feat(randomizer): add lora pool Vue widget 2026-01-11 16:26:38 +08:00
Will Miao
32249d1886 feat: add Vue widget demo node and development support
- Add LoraManagerDemoNode to node mappings for Vue widget demonstration
- Update .gitignore to exclude Vue widget development artifacts (node_modules, .vite, dist)
- Implement automatic Vue widget build check in development mode with fallback handling
- Maintain pytest compatibility with proper import error handling
2026-01-10 17:45:26 +08:00
Will Miao
f842ea990e feat(metadata): prevent overwriting high-quality Civitai API metadata with archive data. See #764
- Update `is_civitai_api_metadata` to exclude both "archive_db" and "civarchive" sources
- Skip Civitai metadata updates when existing metadata is higher quality than incoming archive data
- Add test to verify API metadata is preserved when CivArchive provides lower-quality data
2026-01-09 19:47:32 +08:00
Will Miao
f2e12c0fd3 feat: add "Strength Range" option to LoRA preset parameters dropdown, fixes #386 2026-01-08 22:19:19 +08:00
Will Miao
f62b3f62be feat(recipe_scanner): prioritize local sibling images and persist repairs
Updated image path resolution logic to prioritize local sibling images in the same directory as recipes. When a stored image path differs from a local sibling, the system now automatically updates the recipe file to use the local path and persists this repair. This improves reliability when recipe assets are moved or reorganized, ensuring images remain accessible even if original paths become invalid.
2026-01-08 15:53:01 +08:00
Will Miao
b57a317c82 feat(docs): add DOMWidget development guide for vanilla JavaScript 2026-01-08 13:56:53 +08:00
Will Miao
fa063ba1ce fix: Reprocess example images with missing folders, improve error handling, and add new tests. Fixes #760 2026-01-08 00:25:36 +08:00
Will Miao
eb30595d23 feat(header): improve CSS formatting and spacing 2026-01-07 16:20:46 +08:00
Will Miao
fd7cb3300d fix: Disable virtual scroll keyboard navigation when editing content or a modal is open. See #759 2026-01-07 00:01:09 +08:00
Will Miao
f199c9b591 feat: propagate version info to model update record creation. Fixes #756
- Pass `version_info` parameter through download manager to model update service
- Enhance `_create_record` to use version info when creating records for missing versions
- Add `_extract_single_version` helper method for consistent version extraction
- Improve handling of version metadata during library synchronization
2026-01-06 08:46:55 +08:00
Will Miao
255ca4fc93 fix: Reposition keyboard navigation tooltip and its arrow to the left. Fixes #753 2026-01-04 19:26:13 +08:00
Will Miao
09c1bd78cd feat: Add and hide loading indicators for tag operations. Fixes #755 2026-01-04 19:12:15 +08:00
Will Miao
edbcca9bbd refactor: simplify symlink cache invalidation by removing background rescan and noise_mtime in favor of a root-path-only fingerprint. 2026-01-03 19:29:53 +08:00
Will Miao
8c68298202 feat: rename WanVideoLoraSelect node's class name. 2026-01-03 17:12:21 +08:00
Will Miao
a80380d1f0 chore: reduce symlink cache and scan log verbosity from info to debug level. 2026-01-03 14:50:05 +08:00
Will Miao
f13f22c949 chore: Improve logging by adding SHA256 hash calculation timing and clarifying metadata creation messages. 2026-01-03 08:56:34 +08:00
Will Miao
07aeeb6c70 feat: Deselect moved items and no longer automatically exit bulk mode after a successful move. Fixes #749 2026-01-02 20:36:06 +08:00
Will Miao
4317b06049 fix: Prevent EXIF metadata operations on video files by adding type checks. 2026-01-02 20:18:03 +08:00
Will Miao
ab85ba54a9 feat: Implement recipe repair cancellation with UI support and refactor LoadingManager to a singleton. 2026-01-02 20:03:27 +08:00
Will Miao
837c32c42f feat: implement task cancellation for model scanning and bulk operations 2026-01-02 18:48:28 +08:00
Will Miao
953117efa1 feat: add logging setup and standalone mode detection to LoRA Manager
- Initialize logging configuration via `setup_logging()` when not in standalone mode
- Detect standalone mode using environment variables `LORA_MANAGER_STANDALONE` and `HF_HUB_DISABLE_TELEMETRY`
- Remove redundant `STANDALONE_MODE` variable that previously checked `sys.modules`
2025-12-31 18:52:44 +08:00
Will Miao
afa5533145 feat: reduce log verbosity by changing info logs to debug
Changed logging level from INFO to DEBUG for performance-related messages in model management service. This reduces noise in production logs while maintaining debugging capability for performance analysis.
2025-12-31 16:14:24 +08:00
Will Miao
102defe29c feat: add open settings location endpoint
- Add `open_settings_location` method to `FileSystemHandler` to open OS file explorer at settings file location
- Register new POST route `/api/lm/settings/open-location` for settings file access
- Inject `SettingsManager` dependency into `FileSystemHandler` constructor
- Add cross-platform support for Windows, macOS, and Linux file explorers
- Include error handling for missing settings files and system exceptions
2025-12-31 16:09:23 +08:00
Will Miao
8120716cd8 feat: enhance model move functionality with cache entry updates
- Return cache entry data from model move operations for immediate UI updates
- Add recalculate_type parameter to update_single_model_cache for proper type adjustment
- Propagate cache entry through API layer to frontend MoveManager
- Enable virtual scroller to update moved items with new cache data
2025-12-31 10:33:22 +08:00
Will Miao
2b239c3747 feat: add MoveManager for handling model movement, including UI, bulk operations, and path management. 2025-12-30 23:04:22 +08:00
Will Miao
a59c31bc06 fix: Synchronize aiohttp session creation and refresh with an asyncio lock to prevent race conditions. 2025-12-30 19:47:30 +08:00
Will Miao
d30c8e13df feat: implement various UI helpers including clipboard, toasts, theme toggling, and Civitai integration, and add RecipeModal component. 2025-12-29 16:14:55 +08:00
Will Miao
5d5a2a998a feat: Implement model move, import, and download functionalities with corresponding UI and API updates. 2025-12-28 21:18:27 +08:00
Will Miao
e5b557504e feat: Add context menu option to move checkpoint models between type folders and complete various UI translations. 2025-12-28 17:52:30 +08:00
Will Miao
e43aa5cae4 feat: Update recipe syntax API to accept a recipe ID and add a new test for the endpoint. 2025-12-27 22:09:15 +08:00
Will Miao
f5d5bffa61 feat: Improve Nunchaku LoRA loading with copy_with_ctx support and add unit tests. see #733 2025-12-27 21:46:14 +08:00
pixelpaws
7d6b717385 Merge pull request #743 from willmiao/sort-by-usage-count
Sort by usage count
2025-12-26 22:51:29 +08:00
Will Miao
d9ce2c56c0 feat: Add 'times used' translation keys and implement usage-based sorting in the model service. 2025-12-26 22:39:51 +08:00
pixelpaws
914d24b8bf Merge pull request #723 from stone9k/main
feat(usage_count): sorting by usage_count + usage_count on ModelCard
2025-12-26 22:18:06 +08:00
pixelpaws
1329294981 Merge branch 'sort-by-usage-count' into main 2025-12-26 22:17:03 +08:00
Will Miao
475906a25e perf: Add detailed performance logging to model retrieval, filtering, and sorting operations. see #711 2025-12-25 22:36:24 +08:00
Will Miao
84b68cff90 test: Add clone method to _DummyLoraLoader and import copy. 2025-12-25 21:48:58 +08:00
Will Miao
41759f5e67 refactor: use model.clone() instead of copy.deepcopy() for model duplication, see #733 2025-12-25 21:15:28 +08:00
Will Miao
91cd88f1df feat: Add recursive root folder scanning with API and UI updates. fixes #737 2025-12-25 21:07:52 +08:00
Will Miao
e5869648fb feat: Add keyboard navigation hints and a breadcrumb container to the recipes page, and refactor control layout. 2025-12-24 20:46:14 +08:00
Will Miao
7b139b9b1d refactor: Move base_model resolution to occur before checkpoint formatting and remove a gen_params checkpoint assertion. 2025-12-24 20:35:06 +08:00
Will Miao
a552f07448 feat: Refactor checkpoint metadata to use Civitai API naming conventions and remove gen_params checkpoint syncing. 2025-12-24 20:25:39 +08:00
Will Miao
6486107ca2 feat: Introduce recipe management with data models, scanning, enrichment, and repair for generation configurations. 2025-12-24 20:02:20 +08:00
Will Miao
6330c65d41 feat: Add recipe metadata repair functionality with UI, API, and progress tracking. 2025-12-23 21:50:58 +08:00
Will Miao
00e6904664 feat: Introduce "No tags" filter option for models and recipes. fixes #728 2025-12-23 18:48:35 +08:00
pixelpaws
39195aa529 Merge pull request #735 from willmiao/recipe-folder
Recipe folder
2025-12-23 18:21:48 +08:00
Will Miao
fc0a834beb feat: Introduce generation parameter merging from request, Civitai, and embedded image metadata, and enhance ComfyUI metadata parsing. 2025-12-23 15:31:04 +08:00
Will Miao
b044b329fc feat: Update recipes page with default descending date sort, refactor state properties for search/filters, and add new localization strings. 2025-12-23 11:57:25 +08:00
Will Miao
502c29c6bd test: add prompt filter option to recipes page tests 2025-12-23 11:00:08 +08:00
Will Miao
bc9dd317f7 feat: Add prompt search filter for recipes and fix 'Favorites' localization across multiple languages. 2025-12-23 10:52:12 +08:00
Will Miao
61816cf75d refactor: reposition bulk operations button in recipes UI 2025-12-23 10:09:04 +08:00
Will Miao
db7f09797b feat: Introduce recipe favoriting with star icon toggle and filter options. 2025-12-23 10:07:09 +08:00
Will Miao
6e64f97e2b feat: add bulk move recipes endpoint
Add new move_recipes_bulk endpoint to handle moving multiple recipes simultaneously. This improves efficiency when reorganizing recipe collections by allowing batch operations instead of individual moves.

- Add move_recipes_bulk handler method with proper error handling
- Register new POST /api/lm/recipes/move-bulk route
- Implement bulk move logic in persistence service
- Validate required parameters (recipe_ids and target_path)
- Handle common error cases including validation, not found, and server errors
2025-12-23 09:15:07 +08:00
Will Miao
3f646aa0c9 feat: add recipe root directory and move recipe endpoints
- Add GET /api/lm/recipes/roots endpoint to retrieve recipe root directories
- Add POST /api/lm/recipe/move endpoint to move recipes between directories
- Register new endpoints in route definitions
- Implement error handling for both new endpoints with proper status codes
- Enable recipe management operations for better file organization
2025-12-23 09:13:57 +08:00
Will Miao
67fb205b43 feat: add folder-based recipe organization and navigation
- Add new API endpoints for folder operations: get_folders, get_folder_tree, and get_unified_folder_tree
- Extend recipe listing handler to support folder and recursive filtering parameters
- Register new folder-related routes in route definitions
- Enable users to organize and browse recipes using folder structures
2025-12-23 09:12:27 +08:00
Will Miao
dd89aa49c1 feat: Add HTML and attribute escaping for trigger words and class tokens to prevent XSS vulnerabilities, along with new frontend tests. Fixes #732 2025-12-23 08:47:15 +08:00
Will Miao
3ba5c4c2ab refactor: improve update_lora_filename_by_hash logic and add a test to verify recipe updates. 2025-12-22 17:58:04 +08:00
Will Miao
7caca0163e feat: Add support for remote video analysis and preview for recipe imports. see #420 2025-12-21 21:42:28 +08:00
Will Miao
30fd0470de feat: Add support for video recipe previews by conditionally optimizing media during persistence and updating UI components to display videos. 2025-12-21 20:00:44 +08:00
Will Miao
63b087fc80 feat: Implement cache busting for static assets, remove client-side version mismatch banner, and add project overview documentation. 2025-12-19 22:40:36 +08:00
Will Miao
154ae82519 feat(metadata_processor): enhance primary sampler selection logic
- Add pre-processing step to populate missing parameters for candidate samplers, especially for SamplerCustomAdvanced requiring tracing
- Change sampler selection from most recent (closest to downstream) to first in execution order to prioritize base samplers over refine samplers
- Improve parameter handling by updating sampler parameters with traced values before ranking
- Maintain backward compatibility with fallback to first sampler if no criteria match
2025-12-19 01:30:08 +08:00
Will Miao
c8a179488a feat(metadata): enhance primary sampler detection and workflow tracing
- Add support for `basic_pipe` nodes in metadata processor to handle pipeline nodes like FromBasicPipe
- Optimize `find_primary_checkpoint` by accepting optional `primary_sampler_id` to avoid redundant calculations
- Update `get_workflow_trace` to pass known primary sampler ID for improved efficiency
2025-12-18 22:30:41 +08:00
Will Miao
ca6bb43406 feat: remove path separator normalization for cross-platform compatibility
Removed the forced normalization of path separators to forward slashes in BaseModelService to maintain platform-specific separators. Updated test cases to use os.sep for constructing expected paths, ensuring tests work correctly across different operating systems while preserving native path representations.
2025-12-17 19:07:08 +08:00
Will Miao
a07720a3bf feat: Add model path tracing to accurately identify the primary checkpoint in workflows and include new tests. 2025-12-17 12:52:52 +08:00
Will Miao
bdb4422cbc feat(ui): adjust modal header width and enhance close button z-index, fixes #729
- Decrease modal header width from 85% to 84% for better visual alignment
- Add z-index: 10 to close button to ensure it remains above other modal elements
2025-12-17 10:34:04 +08:00
Will Miao
099a71b2cc feat(config): seed root symlink mappings before deep scanning
Add `_seed_root_symlink_mappings` method to ensure symlinked root folders are recorded before deep scanning, preventing them from being missed during directory traversal. This ensures that root symlinks are properly captured in the path mappings.

Additionally, normalize separators in relative paths for cross-platform consistency in `BaseModelService`, and update tests to verify root symlinks are preserved in the cache.
2025-12-16 22:05:40 +08:00
Will Miao
3382d83aee feat: remove prewarm cache and improve recipe scanner initialization
- Remove prewarm_cache startup hook from BaseRecipeRoutes
- Add post-scan task management to RecipeScanner for proper cleanup
- Ensure LoRA scanner initialization completes before recipe enrichment
- Schedule post-scan enrichment after cache initialization
- Improve error handling and task cancellation during shutdown
2025-12-16 21:00:04 +08:00
Will Miao
7e133e4b9d feat: rename SaveImage class to SaveImageLM for clarity
The SaveImage class has been renamed to SaveImageLM to better reflect its purpose within the Lora Manager module. This change ensures consistent naming across import statements, class mappings, and the actual class definition, improving code readability and maintainability.
2025-12-15 22:09:26 +08:00
Will Miao
2494fa19a6 feat(config): add background symlink rescan and simplify cache validation
- Added threading import and optional `_rescan_thread` for background operations
- Simplified `_load_symlink_cache` to only validate path mappings, removing fingerprint checks
- Updated `_initialize_symlink_mappings` to rebuild preview roots and schedule rescan when cache is loaded
- Added `_schedule_symlink_rescan` method to perform background validation of symlinks
- Cleared `_path_mappings` at start of `_scan_symbolic_links` to prevent stale entries
- Background rescan improves performance by deferring symlink validation after cache load
2025-12-15 18:46:23 +08:00
Will Miao
5359129fad feat(config): improve symlink cache logging and add performance timing
- Add `time` import for performance measurement
- Change debug logs to info level for better visibility of cache operations
- Add detailed logging for cache validation failures and successes
- Include timing metrics for symlink initialization and scanning
- Log cache save/load operations with mapping counts
2025-12-14 15:58:58 +08:00
pixelpaws
4743b3c406 Merge pull request #724 from stone9k/patch-1
fix(trigger_word_toggle): missing consumeExistingState after refactor
2025-12-13 15:58:23 +08:00
stone9k
32d94be08a fix(trigger_word_toggle): missing consumeExistingState after refactor 2025-12-12 18:50:28 +01:00
stone9k
56143eb170 feat(usage_count): sorting by usage_count + usage_count on ModelCard 2025-12-12 16:39:24 +01:00
Will Miao
817de3a0ae test: improve vi.mock calls to preserve original module exports
Updated vi.mock calls in test files to use async importOriginal pattern, ensuring original module exports are preserved while mocking specific functions. This prevents unintended side effects and maintains better test isolation.
2025-12-11 18:27:55 +08:00
Will Miao
675d49e4ce feat(security): escape HTML attributes and content in model modal, fixes #720
- Import `escapeAttribute` and `escapeHtml` utilities from shared utils
- Remove duplicate `escapeAttribute` function from ModelModal.js
- Apply escaping to file path attributes in model modal and trigger words
- Escape folder path HTML content to prevent XSS vulnerabilities
- Ensure safe handling of user-controlled data in UI components
2025-12-11 18:08:35 +08:00
Will Miao
fbb95bc623 feat(context-menu): pass file path to NSFW level selector
- Add `cardPath` parameter to `show` method in NsfwLevelSelector component
- Include `filePath` from card dataset when calling selector in ModelContextMenuMixin
- Clear `cardPath` from dataset when hiding selector to prevent stale data

This enables the NSFW level selector to access the file path context, which may be needed for backend operations when changing NSFW levels.
2025-12-09 22:13:00 +08:00
Will Miao
6b3a11e01a fix(config): ensure symlink mappings are recorded before duplicate check
Update symlink traversal logic to always record path mappings before checking for visited directories. This prevents valid link->target pairs from being dropped when the target directory has already been visited via another path. Also correct path mapping lookup to properly replace link paths with their actual target paths.
2025-12-09 21:46:33 +08:00
Will Miao
40f7f14c1b ci: add symlink verification step to backend tests workflow
Add a Python script step to verify that the CI environment supports directory symlinks before running tests. This ensures that symlink-dependent tests will not fail due to environment limitations.
2025-12-09 21:32:50 +08:00
Will Miao
a6e23a7630 feat(example-images): add NSFW level setting endpoint
Add new POST endpoint `/api/lm/example-images/set-nsfw-level` to allow updating NSFW classification for individual example images. The endpoint supports both regular and custom images, validates required parameters, and updates the corresponding model metadata. This enables users to manually adjust NSFW ratings for better content filtering.
2025-12-09 20:37:16 +08:00
Will Miao
3fc72d6bc1 feat(config): replace symlink scanning with cached mapping system
- Add `get_settings_dir` import for cache directory resolution
- Replace `_scan_symbolic_links` and `_rebuild_preview_roots` with unified `_initialize_symlink_mappings` method
- Implement fingerprint-based cache validation using root mtimes, inodes, and noise-aware timestamps
- Add helper methods for path normalization, cache location, and symlink root aggregation
- Improve performance by avoiding redundant symlink traversal when directory structure is unchanged
2025-12-09 19:51:01 +08:00
Will Miao
a3a00bbeed feat(trigger_word_toggle): refactor trigger word filtering logic, fixes #718 and fixes #285
- Simplify and consolidate the logic for processing trigger words and groups
- Remove redundant code paths and improve maintainability
- Ensure consistent behavior between list and string trigger data inputs
- Preserve existing functionality for strength adjustment and group mode
2025-12-09 14:16:56 +08:00
Will Miao
74bfd397aa feat: add CSP middleware to allow remote media previews, fixes #710, see #715
Introduce `relax_csp_for_remote_media` middleware that modifies Content Security Policy headers to permit loading media from trusted external domains (Civitai and Genur). This is necessary for LoRA Manager UI previews when ComfyUI runs with `--disable-api-nodes`, which otherwise blocks remote images and videos. The middleware is inserted after ComfyUI's `block_external_middleware` to properly extend the restrictive CSP header.
2025-12-09 10:37:35 +08:00
Will Miao
5000478991 feat(download): support multiple model file extensions in archive extraction
- Add `_get_supported_extensions_for_type` method to return allowed extensions per model type
- Rename `_extract_safetensors_from_archive` to `_extract_model_files_from_archive` and extend to filter by allowed extensions
- Update error message to list supported extensions when archive contains no valid files
- Add test for extracting .pt embedding files from zip archives
2025-12-07 09:00:47 +08:00
Will Miao
40cd2e23ac feat(i18n): add model navigation translations for multiple languages
Add navigation section to locale files for model browsing functionality. Includes labels and tooltips for previous/next model navigation with keyboard shortcuts (←/→ arrows). Translations added for German, English, Spanish, French, Hebrew, Japanese, Korean, and Russian locales to support international users.
2025-12-06 10:01:09 +08:00
Will Miao
6efe59bd9e feat(model-modal): add dynamic update availability indicators, see #715
- Add update badge to versions tab button when model has updates
- Sync update status between modal and model cards in gallery
- Pass `onUpdateStatusChange` callback to versions tab for real-time updates
- Introduce `updateAvailabilityState` to track update status changes
- Improve user awareness of available model updates across UI components
2025-12-06 09:43:15 +08:00
Will Miao
83f379df33 feat(modal): add scrollbar-gutter to prevent layout shift 2025-12-05 22:27:42 +08:00
Will Miao
4d6f4fcf69 feat(model-modal): add keyboard navigation and UI controls for model browsing, fixes #714 and #350
- Add CSS for modal navigation buttons with hover and disabled states
- Implement keyboard shortcuts (arrow keys) for navigating between models
- Add navigation controls UI to modal header with previous/next buttons
- Store navigation state to enable sequential model browsing
- Clean up event handlers to prevent memory leaks when modal closes
2025-12-05 22:25:17 +08:00
Will Miao
22ee37b817 feat: parse aggregate commercial use values, see #708
Add support for parsing comma-separated and JSON-style commercial use permission values in both Python backend and JavaScript frontend. Implement helper functions to split aggregated values into individual permissions while preserving original values when no aggregation is detected.

Added comprehensive test coverage for the new parsing functionality to ensure correct handling of various input formats including strings, arrays, and iterable objects with aggregated commercial use values.
2025-11-30 17:18:28 +08:00
Will Miao
f09224152a feat: bump version to 0.9.11 2025-11-29 17:46:06 +08:00
Will Miao
df93670598 feat: add checkpoint metadata to EXIF recipe data
Add support for storing checkpoint information in image EXIF metadata. The checkpoint data is simplified and includes fields like model ID, version, name, hash, and base model. This allows for better tracking of AI model checkpoints used in image generation workflows.
2025-11-29 08:46:38 +08:00
Will Miao
073fb3a94a feat(recipe-parser): enhance LoRA metadata with local file matching
Add comprehensive local file matching for LoRA entries in recipe metadata:
- Add modelVersionId-based lookup via new _get_lora_from_version_index method
- Extend LoRA entry with additional fields: existsLocally, inLibrary, localPath, thumbnailUrl, size
- Improve local file detection by checking both SHA256 hash and modelVersionId
- Set default thumbnail URL and size values for missing LoRA files
- Add proper typing with Optional imports for better code clarity

This provides more accurate local file status and metadata for LoRA entries in recipes.
2025-11-29 08:29:05 +08:00
Will Miao
53c4165d82 feat(parser): enhance model metadata extraction in Automatic1111 parser
- Add MODEL_NAME_PATTERN regex to extract model names from parameters
- Extract model hash from parsed hashes when available in metadata
- Add checkpoint model hash and name extraction from parameters section
- Implement checkpoint resource processing from Civitai metadata
- Improve model information completeness for better recipe tracking
2025-11-29 08:13:55 +08:00
Will Miao
8cd4550189 feat: add Flux.2 D and ZImageTurbo model constants
Add new model constants for Flux.2 D and ZImageTurbo to the BASE_MODELS object,
along with their corresponding abbreviations in BASE_MODEL_ABBREVIATIONS. Also
include these new models in the appropriate categories within BASE_MODEL_CATEGORIES.

This update ensures the application can properly recognize and handle these
newly supported AI models in the system.
2025-11-28 11:42:46 +08:00
Will Miao
2b2e4fefab feat(tests): restructure test HTML to nest elements under model modal
Refactor the test HTML structure to properly nest all model metadata elements within the model modal container. This improves test accuracy by matching the actual DOM structure used in the application, ensuring that element selection and event handling work correctly during testing.
2025-11-27 20:44:05 +08:00
Will Miao
5f93648297 feat: scope DOM queries to modal element in ModelMetadata
Refactor updateModalFilePathReferences function to scope all DOM queries within the modal element. This prevents potential conflicts with other elements on the page that might have the same CSS selectors. Added helper functions scopedQuery and scopedQueryAll to limit element selection to the modal context, improving reliability and preventing unintended side effects.
2025-11-27 20:33:04 +08:00
pixelpaws
8a628f0bd0 Merge pull request #703 from willmiao/fix/showcase-listener-leaks
fix(showcase): tear down modal listeners
2025-11-27 20:09:45 +08:00
Will Miao
b67c8598d6 feat(metadata): clear stale cache entries when metadata is empty
Update metadata registry to remove cache entries when node metadata becomes empty instead of keeping stale data. This prevents accumulation of unused cache entries and ensures cache only contains valid metadata. Added test case to verify cache behavior when LoRA configurations are removed.
2025-11-27 20:04:38 +08:00
Will Miao
0254c9d0e9 fix(showcase): tear down modal listeners 2025-11-27 18:00:59 +08:00
Will Miao
ecb512995c feat(civitai): expand image metadata detection criteria, see #700
Add additional CivitAI image metadata fields to detection logic including generation parameters (prompt, steps, sampler, etc.) and model information. Also improve LoRA hash detection by checking both main metadata and nested meta objects. This ensures more comprehensive identification of CivitAI image metadata across different response formats.
2025-11-27 10:28:04 +08:00
Will Miao
f8b9fa9b20 fix(civitai): improve metadata parsing for nested structures, see #700
- Refactor metadata detection to handle nested "meta" objects
- Add support for lowercase "lora:" hash keys
- Extract metadata from nested "meta" field when present
- Update tests to verify nested metadata parsing
- Handle case-insensitive LORA hash detection

The changes ensure proper parsing of Civitai image metadata that may be wrapped in nested structures, improving compatibility with different API response formats.
2025-11-26 13:46:08 +08:00
Will Miao
5d4917c8d9 feat: add v0.9.10 release notes with new features and improvements
- Implement smarter update matching with base model grouping options
- Add flexible tag filtering with include/exclude functionality
- Display license icons and add license filtering controls
- Improve recipes with zero-LoRA imports and checkpoint references
- Enhance ZIP downloads with automatic model extraction
- Update template workflow with improved guidance
- Include various bug fixes and stability improvements
2025-11-24 11:15:05 +08:00
Will Miao
a50309c22e feat: update template workflow and image assets 2025-11-24 10:22:12 +08:00
Will Miao
f5020e081f feat(autocomplete): restrict embeddings autocomplete to explicit prefix
Only trigger autocomplete for embeddings when the current token starts with "emb:" prefix. This prevents interrupting normal prompt typing while maintaining quick manual access to embeddings suggestions.
2025-11-22 20:55:20 +08:00
Will Miao
3c0bfcb226 feat: add KSampler_inspire node extractor for comfyui-inspire-pack, fixes #693 2025-11-22 14:28:44 +08:00
Will Miao
9198a23ba9 feat: normalize and validate checkpoint entries before enrichment
Add _normalize_checkpoint_entry method to handle legacy checkpoint data formats (strings, tuples) by converting them to dictionaries. This prevents errors during enrichment when checkpoint data is not in the expected dictionary format. Invalid checkpoint entries are now removed instead of causing processing failures.

- Update get_paginated_data and get_recipe_by_id methods to use normalization
- Add test cases for legacy string and tuple checkpoint formats
- Ensure backward compatibility with existing checkpoint handling
2025-11-21 23:36:32 +08:00
Will Miao
02bac7edfb feat: normalize and validate checkpoint entries in recipes
Add _normalize_checkpoint_entry method to handle legacy and malformed checkpoint data by:
- Converting string entries to structured dict format
- Handling single-element lists/tuples recursively
- Dropping invalid entries with appropriate warnings
- Maintaining backward compatibility while improving data consistency

Add test case to verify string checkpoint conversion works correctly.
2025-11-21 23:00:02 +08:00
Will Miao
ea1d1a49c9 feat: enhance search with include/exclude tokens and improved sorting
- Add token parsing to support include/exclude search terms using "-" prefix
- Implement token-based matching logic for relative path searches
- Improve search result sorting by prioritizing prefix matches and match position
- Add frontend test for multi-token highlighting with exclusion support
2025-11-21 19:48:43 +08:00
Will Miao
9a789f8f08 feat: add checkpoint hash filtering and navigation
- Add checkpoint hash parameter parsing to backend routes
- Implement checkpoint hash filtering in frontend API client
- Add click navigation from recipe modal to checkpoints page
- Update checkpoint items to use pointer cursor for better UX

Checkpoint items in recipe modal are now clickable and will navigate to the checkpoints page with appropriate hash filtering applied. This improves user workflow when wanting to view checkpoint details from recipes.
2025-11-21 16:17:01 +08:00
Will Miao
1971881537 feat: add checkpoint scanner integration to recipe scanner
- Add CheckpointScanner dependency to RecipeScanner singleton
- Implement checkpoint enrichment in recipe data processing
- Add _enrich_checkpoint_entry method to enhance checkpoint metadata
- Update recipe formatting to include checkpoint information
- Extend get_instance, __new__, and __init__ methods to support checkpoint scanner
- Add _get_checkpoint_from_version_index method for cache lookup

This enables recipe scanner to handle checkpoint models alongside existing LoRA support, providing complete model metadata for recipes.
2025-11-21 15:36:54 +08:00
Will Miao
4eb46a8d3e feat: consolidate checkpoint metadata handling
- Extract checkpoint entry from multiple metadata locations using helper method
- Sanitize checkpoint metadata by removing transient/local-only fields
- Remove checkpoint duplication from generation parameters to store only at top level
- Update frontend to properly populate checkpoint metadata during import
- Add tests for new checkpoint handling functionality

This ensures consistent checkpoint metadata structure and prevents data duplication across different storage locations.
2025-11-21 14:55:45 +08:00
Will Miao
36f28b3c65 feat: normalize LoRA preview URLs for browser accessibility
Add _normalize_preview_url method to ensure preview URLs are properly formatted for browser access. The method handles absolute paths by converting them to static URLs via config.get_preview_static_url, while preserving API paths and other valid URLs. This ensures consistent preview image display across different URL formats.

Update _enrich_lora_entry to apply URL normalization to preview URLs obtained from both hash-based lookups and version entries. Add comprehensive test coverage for absolute path normalization scenarios.
2025-11-21 12:31:23 +08:00
Will Miao
2452cc4df1 feat(recipes): resolve base model from checkpoint metadata
Add metadata service integration to automatically resolve base model information from checkpoint metadata during recipe import. This replaces the previous approach of relying solely on request parameters and provides more accurate base model information.

- Add _resolve_base_model_from_checkpoint method to fetch base model from metadata provider
- Update recipe import logic to use resolved base model when available
- Add comprehensive tests for base model resolution with fallback behavior
- Remove debug print statement from import parameters
2025-11-21 12:12:27 +08:00
Will Miao
eda1ce9743 feat: improve base model display with abbreviations in RecipeCard
- Import getBaseModelAbbreviation utility function
- Add fallback handling for missing base model values
- Display abbreviated base model names while keeping full name in tooltip
- Maintain "Unknown" label for recipes without base model specification
- Improve user experience by showing cleaner, more readable model identifiers
2025-11-21 11:36:17 +08:00
Will Miao
e24621a0af feat(recipe-scanner): add version index fallback for LoRA enrichment
Add _get_lora_from_version_index method to fetch cached LoRA entries by modelVersionId when hash is unavailable. This improves LoRA enrichment by using version index as fallback when hash is missing, ensuring proper library status, file paths, and preview URLs are set even without hash values.

Update test suite to include version_index in stub cache and add test coverage for version-based lookup functionality.
2025-11-21 11:27:09 +08:00
Will Miao
7173a2b9d6 feat: add remote recipe import functionality
Add support for importing recipes from remote sources by:
- Adding import_remote_recipe endpoint to RecipeHandlerSet
- Injecting downloader_factory and civitai_client_getter dependencies
- Implementing image download and resource parsing logic
- Supporting Civitai resource payloads with checkpoints and LoRAs
- Adding required imports for regex and temporary file handling

This enables users to import recipes directly from external sources like Civitai without manual file downloads.
2025-11-21 11:12:58 +08:00
pixelpaws
d540b21aac Merge pull request #691 from willmiao/feat/zip-preview
feat(downloads): support safetensors zips and previews
2025-11-20 19:56:31 +08:00
Will Miao
9952721e76 feat(downloads): support safetensors zips and previews 2025-11-20 19:41:31 +08:00
Will Miao
26e4895807 feat(auto-organize): improve exclusion handling and progress reporting
- Add auto_organize_exclusions to settings handler proxy keys
- Refactor model file service to handle exclusions relative to model roots
- Improve auto-organize progress reporting for empty operations
- Fix exclusion pattern matching to consider relative paths within model roots
- Ensure proper validation when no model roots are configured
- Add comprehensive cleanup reporting for empty auto-organize operations
2025-11-20 18:33:48 +08:00
Will Miao
c533a8e7bf feat: enhance Civitai metadata handling and image URL processing
- Import rewrite_preview_url utility for optimized image URL handling
- Update thumbnail URL processing for both LoRA and checkpoint entries to use rewritten URLs
- Expand checkpoint metadata with modelId, file size, SHA256 hash, and file name
- Improve error handling and data validation for Civitai API responses
- Maintain backward compatibility with existing data structures
2025-11-20 16:31:48 +08:00
pixelpaws
dc820a456f Merge pull request #690 from willmiao/codex/add-auto-organize-exclusions-field
Add auto-organize exclusion settings
2025-11-20 16:24:29 +08:00
pixelpaws
07721af87c feat(settings): add auto-organize exclusions 2025-11-20 16:08:32 +08:00
Will Miao
5093c30c06 feat: add video support to model version delete preview
- Extend CSS to style video elements in delete previews
- Add video rendering logic for model version previews
- Use consistent placeholder image for missing previews
- Maintain existing image preview functionality while adding video support

This allows users to see video previews when deleting model versions, improving the user experience for video-based models.
2025-11-19 22:42:58 +08:00
Will Miao
8c77080ae6 feat: conditionally hide license filters on recipes page
Add shouldShowLicenseFilters method to check if current page is 'recipes' and skip license filter initialization and updates when on recipes page. Also conditionally render license filter section in header template based on current page.

This prevents license filters from appearing on the recipes page where they are not applicable.
2025-11-19 22:26:16 +08:00
pixelpaws
bcf72c6bcc Merge pull request #689 from willmiao/civitai-deletion-logic
feat(metadata): improve civitai deletion detection logic, see #670
2025-11-19 19:27:28 +08:00
Will Miao
3849f7eef9 feat(metadata): improve civitai deletion detection logic
- Track when Civitai API returns "Model not found" for default provider
- Use dedicated flag instead of error string comparison for deletion detection
- Ensure archive-sourced models don't get marked as deleted
- Add test coverage for archive source deletion flag behavior
- Fix deletion flag logic to properly handle provider fallback scenarios
2025-11-19 19:16:40 +08:00
pixelpaws
7eced1e3e9 Merge pull request #686 from willmiao/fix/model-extension-delete-rename
fix(model): preserve original extension on rename
2025-11-19 11:43:01 +08:00
Will Miao
51b5261f40 fix(model): align rename extension detection 2025-11-19 11:20:09 +08:00
Will Miao
963f6b1383 fix(model): preserve original extension on rename 2025-11-19 11:08:08 +08:00
Will Miao
b75baa1d1a fix: support GGUF model deletion in model lifecycle service
- Add optional main_extension parameter to delete_model_artifacts function
- Extract file extension from model filename to handle different file types
- Update model scanner to pass file extension when deleting models
- Add test case for GGUF file deletion to ensure proper cleanup
- Maintain backward compatibility with existing safetensors models

This change allows the model lifecycle service to properly delete GGUF model files along with their associated metadata and preview files, expanding support beyond just safetensors format.
2025-11-19 10:36:03 +08:00
Will Miao
6d95e93378 feat: simplify model ID parsing and loading manager usage 2025-11-19 10:26:07 +08:00
pixelpaws
7117e0c33e Merge pull request #684 from willmiao/codex/add-check-update-to-single-model-context-menu
Add single-model update checks to context menus
2025-11-19 00:09:59 +08:00
pixelpaws
d261474f3a feat(context-menu): add single model update checks 2025-11-19 00:01:50 +08:00
pixelpaws
c09d67d2e4 Merge pull request #683 from willmiao/deletion-sync, see #673
feat(model-lifecycle): integrate model update service for deletion sync
2025-11-18 23:28:17 +08:00
Will Miao
1427dc8e38 feat(model-lifecycle): integrate model update service for deletion sync
Add ModelUpdateService dependency to ModelLifecycleService to enable synchronization during model deletion. The service is now passed through BaseModelRoutes initialization and used in delete_model to trigger updates when a model is removed. This ensures external systems stay in sync with local model state changes.

Key changes:
- Inject update_service into ModelLifecycleService constructor
- Extract model ID from metadata during deletion
- Call update service sync method after successful deletion
- Add proper type hints and TYPE_CHECKING imports
2025-11-18 21:02:39 +08:00
pixelpaws
77a7b90dc7 Merge pull request #682 from willmiao/feature/model-type-filter
Feature/model type filter
2025-11-18 18:51:52 +08:00
Will Miao
e9d55fe146 feat(filters): add model type filter 2025-11-18 16:43:44 +08:00
Will Miao
57f369a6de feat(model): add model type filtering support
- Add model_types parameter to ModelListingHandler to support filtering by model type
- Implement get_model_types endpoint in ModelQueryHandler to retrieve available model types
- Register new /api/lm/{prefix}/model-types route for model type queries
- Extend BaseModelService to handle model type filtering in queries
- Support both model_type and civitai_model_type query parameters for backward compatibility

This enables users to filter models by specific types, improving model discovery and organization capabilities.
2025-11-18 15:36:01 +08:00
Will Miao
059ebeead7 feat: include Negative file type in primary file selection for embeddings 2025-11-18 14:16:22 +08:00
Will Miao
831a9da9d7 feat: update version badge logic for same-base update strategy, see #676
- Remove unused isNewer variable calculation
- Use dividerThresholdVersionId instead of latestLibraryVersionId for badge logic
- Add test case to verify newer version badge appears with same-base strategy
- Ensures correct badge display when filtering by same base model versions
2025-11-18 11:18:32 +08:00
Will Miao
6000e08640 feat(i18n): add new license restriction translations
Add four new license restriction keys to all locale files:
- noImageSell: "No selling generated content"
- noRentCivit: "No Civitai generation"
- noRent: "No generation services"
- noSell: "No selling models"

These additions provide comprehensive coverage for various commercial and generation restrictions in the licensing system, ensuring proper localization across all supported languages.
2025-11-18 09:17:04 +08:00
pixelpaws
3edc65c106 Merge pull request #681 from willmiao/update-strategy, see #676
Add update flag strategy
2025-11-18 08:44:46 +08:00
Will Miao
655157434e feat(versions): add base filter toggle UI and styling
Add CSS classes and JavaScript logic for the base filter toggle button in the versions toolbar. The filter allows users to switch between showing all versions or only versions matching the current base model. Includes styling for different states (active, hover, disabled) and accessibility features like screen reader support.
2025-11-18 06:47:07 +08:00
Will Miao
3661b11b70 feat(i18n): add update flag strategy settings
Add new "updateFlags" section to settings navigation and implement update flag strategy configuration. The strategy allows users to choose when update badges appear:
- Match updates by base model (only show when new release shares same base model)
- Flag any available update (show whenever newer version exists)

Includes translations for English, German, Spanish, and French locales.
2025-11-17 20:02:26 +08:00
Will Miao
0e73db0669 feat: implement same_base update strategy for model annotations
Add support for configurable update flag strategy with new "same_base" mode that considers base model versions when determining update availability. The strategy is controlled by the "update_flag_strategy" setting.

When strategy is set to "same_base":
- Uses get_records_bulk instead of has_updates_bulk
- Compares model versions against highest local versions per base model
- Provides more granular update detection based on base model relationships

Fallback to existing bulk or individual update checks when:
- Strategy is not "same_base"
- Bulk operations fail
- Records are unavailable

This enables more precise update flagging for models sharing common bases.
2025-11-17 19:26:41 +08:00
Will Miao
8158441a92 feat: add CheckpointLoaderKJ extractor and improve model filename handling, fixes #666
- Add CheckpointLoaderKJ to NODE_EXTRACTORS mapping for KJNodes support
- Enhance model filename generation in SaveImage to handle different data types
- Add proper type checking and fallback for model metadata values
- Improve robustness when processing checkpoint paths for filename generation
2025-11-17 08:52:51 +08:00
pixelpaws
5600471093 Merge pull request #675 from willmiao/fix/portable-mode-sync
fix(settings): sync portable mode toggle
2025-11-16 17:52:01 +08:00
Will Miao
354cf03bbc fix(settings): sync portable mode toggle 2025-11-16 17:36:52 +08:00
Will Miao
645b7c247d feat(i18n): increase trigger word length limit from 30 to 100 words
Update trigger word validation message across all language files to reflect increased character limit. The change allows users to create longer trigger words, providing more flexibility in trigger word creation while maintaining the existing maximum count of 30 trigger words.
2025-11-15 22:22:42 +08:00
Will Miao
5f25a29303 Revert "修复:在应用LoRA值到文本时仅包含激活的LoRA", see #669
This reverts commit 1cdbb9a851.
2025-11-15 16:26:31 +08:00
Will Miao
906d00106d feat(trigger-words): increase maximum word limit from 30 to 100, fixes #660 2025-11-15 08:19:53 +08:00
Will Miao
7850131969 feat: add metadata extractor for KJNodes model loaders, see #666
Add KJNodesModelLoaderExtractor to handle metadata extraction from KJNodes loaders that expose model_name. This supports GGUFLoaderKJ and DiffusionModelLoaderKJ nodes, ensuring consistent checkpoint metadata collection across different node types.
2025-11-14 15:46:11 +08:00
pixelpaws
3d5ec4a9f1 Merge pull request #668 from Aaalice233/main
修复:在应用LoRA值到文本时仅包含激活的LoRA
2025-11-14 15:19:32 +08:00
Luna_K
1cdbb9a851 修复:在应用LoRA值到文本时仅包含激活的LoRA
- 在applyLoraValuesToText函数中添加激活状态检查
- 如果LoRA的active属性为false,则跳过该LoRA
- 保持向后兼容性:当active属性未定义或为null时,默认视为激活状态
- 确保只有用户选中的LoRA会被应用到工作流文本中
2025-11-14 13:53:10 +08:00
pixelpaws
e224be4b88 Merge pull request #664 from willmiao/codex/remove-recipevalidationerror-on-empty-lora_matches
Allow widget recipe saves without LoRA matches
2025-11-13 16:22:57 +08:00
pixelpaws
b9d3a4afce Merge pull request #665 from willmiao/codex/refactor-imageprocessor-to-normalize-loras-array
fix: allow importing recipes without loras
2025-11-13 16:22:40 +08:00
pixelpaws
aa4aa1a613 fix(import): allow zero lora recipes 2025-11-13 15:53:54 +08:00
pixelpaws
cc8e1c5049 fix(recipes): allow widget save without lora matches 2025-11-13 15:52:31 +08:00
pixelpaws
41e649415a Merge pull request #658 from willmiao/feature/global-license-refresh
Feature/global license refresh
2025-11-11 14:54:37 +08:00
Will Miao
c8f770a86b feat: batch process model license data retrieval 2025-11-11 14:36:19 +08:00
Will Miao
29bb85359e feat(context-menu): refresh missing license metadata 2025-11-11 14:24:59 +08:00
Will Miao
4557da8b63 feat(metadata): return tuple with metadata and success flag
Change `load_metadata` method to return a tuple containing both the metadata object and a boolean success flag instead of just the metadata object. This provides clearer error handling and allows callers to distinguish between successful loads with null metadata versus actual load failures.
2025-11-11 11:18:33 +08:00
pixelpaws
09b75de25b Merge pull request #656 from willmiao/feat/hash-chunk-size-config
feat(settings): add configurable hash chunk size
2025-11-10 10:18:00 +08:00
Will Miao
415fc5720c feat(settings): add configurable hash chunk size 2025-11-10 10:15:01 +08:00
Will Miao
4dd8ce778e feat(trigger): add optional strength adjustment for trigger words
Add `allow_strength_adjustment` parameter to enable mouse wheel adjustment of trigger word strengths. When enabled, strength values are preserved and can be modified interactively. Also improves trigger word parsing by handling whitespace more consistently and adding debug logging for trigger data inspection.
2025-11-09 22:24:23 +08:00
Will Miao
f81ff2efe9 feat: remove strength-based styling from tags widget
Remove visual styling for tags with modified strength values. The gold border and gradient background were previously applied to tags with strength values other than 1.0, but this visual distinction is no longer needed. This simplifies the tag styling logic and maintains consistent appearance across all tags regardless of their strength values.
2025-11-09 18:02:57 +08:00
Will Miao
837bb17b08 feat(comfyui): fix trigger word toggle widget initialization
Change from loadedGraphNode to nodeCreated lifecycle method to ensure proper widget initialization timing. Wrap widget creation and highlight logic in requestAnimationFrame to prevent race conditions with node setup. This ensures the trigger word toggle widget functions correctly when nodes are created.
2025-11-08 19:39:25 +08:00
Will Miao
5ee93a27ee feat: add license flags display to model preview tooltip #613
- Add optional license_flags parameter to model preview API endpoint
- Include license flags in response when requested via query parameter
- Add CSS styles for license overlay and icons in tooltip
- Implement license flag parsing and icon mapping logic
- Display license restrictions as icons in preview tooltip overlay

This allows users to see model license restrictions directly in the preview tooltip without needing to navigate to detailed model information pages.
2025-11-08 19:09:06 +08:00
Will Miao
2e6aa5fe9f feat: replace nodeCreated with loadedGraphNode for LoraManager nodes
- Change lifecycle hook from nodeCreated to loadedGraphNode in Lora Loader, Lora Stacker, and TriggerWord Toggle nodes
- Remove requestAnimationFrame wrappers as loadedGraphNode ensures proper initialization timing
- Maintain same functionality for restoring saved values and widget initialization
- Improves reliability by using the appropriate node lifecycle event
2025-11-08 14:08:43 +08:00
pixelpaws
c14e066f8f Merge pull request #651 from willmiao/tag-filtering-with-include-exclude-states, see #622
feat: implement tag filtering with include/exclude states
2025-11-08 12:01:13 +08:00
Will Miao
c09100c22e feat: implement tag filtering with include/exclude states
- Update frontend tag filter to cycle through include/exclude/clear states
- Add backend support for tag_include and tag_exclude query parameters
- Maintain backward compatibility with legacy tag parameter
- Store tag states as dictionary with 'include'/'exclude' values
- Update test matrix documentation to reflect new tag behavior

The changes enable more granular tag filtering where users can now explicitly include or exclude specific tags, rather than just adding tags to a simple inclusion list. This provides better control over search results and improves the filtering user experience.
2025-11-08 11:45:31 +08:00
pixelpaws
839ed3bda3 Merge pull request #650 from willmiao/license-filter, see #548 and #613
License filter
2025-11-08 10:30:32 +08:00
Will Miao
1f627774c1 feat(i18n): add license and content usage filter labels
Add new translation keys for model filter interface:
- license
- noCreditRequired
- allowSellingGeneratedContent

These labels support new filtering options for model licensing and content usage permissions, enabling users to filter models based on their license requirements and commercial usage rights.
2025-11-08 10:20:28 +08:00
Will Miao
3b842355c2 feat: add license-based filtering for model listings
Add support for filtering models by license requirements:
- credit_required: filter models that require credits or allow free use
- allow_selling_generated_content: filter models based on commercial usage rights

These filters use license_flags bitmask to determine model permissions and enable users to find models that match their specific usage requirements and budget constraints.
2025-11-07 22:28:29 +08:00
Will Miao
dd27411ebf feat(trigger-word-toggle): add strength value support for trigger words
- Extract and preserve strength values from trigger words in format "(word:strength)"
- Maintain strength formatting when filtering active trigger words in both group and individual modes
- Update active state tracking to handle strength-modified words correctly
- Ensure backward compatibility with existing trigger word formats
2025-11-07 16:38:04 +08:00
Will Miao
388ff7f5b4 feat(ui): add trigger word highlighting for selected LoRAs
- Import applySelectionHighlight in lora_loader and lora_stacker
- Pass onSelectionChange callback to loras_widget to handle selection changes
- Implement selection tracking and payload building in loras_widget
- Emit selection changes when LoRA selection is modified
- Update tags_widget to support highlighted tag styling

This provides visual feedback when LoRAs are selected by highlighting associated trigger words in the interface.
2025-11-07 16:08:56 +08:00
Will Miao
f76343f389 feat(lora): add mode change listeners to update trigger words
Add property descriptor to listen for mode changes in Lora Loader and Lora Stacker nodes. When node mode changes, automatically update connected trigger word toggle nodes and downstream loader nodes to maintain synchronization between node modes and trigger word states.

- Lora Loader: Updates connected trigger words when mode changes
- Lora Stacker: Updates connected trigger words and downstream loaders when mode changes
- Both nodes log mode changes for debugging purposes
2025-11-07 15:11:59 +08:00
Will Miao
ce5a1ae3d0 feat(lora-stacker): conditionally update trigger words based on node mode
Add node mode checks to ensure trigger words are only updated when the stacker node is active (mode 0 for Always or mode 3 for On Trigger). This prevents unnecessary updates when the node is inactive (mode 2 for Never or mode 4 for Bypass), improving performance and ensuring trigger words reflect the actual active state of the node.

The changes include:
- Adding mode checks before updating active LoRA names in the stacker callback
- Modifying collectActiveLorasFromChain to only include active nodes
- Adding comments to clarify node mode behavior
2025-11-07 14:21:58 +08:00
pixelpaws
1d40d7400f Merge pull request #648 from willmiao/fix-rate-limit-retry, see #647
feat(metadata): add rate limit retry support to metadata providers
2025-11-07 10:57:48 +08:00
Will Miao
1bb5d0b072 feat(metadata): add rate limit retry support to metadata providers
Add RateLimitRetryingProvider and _RateLimitRetryHelper classes to handle rate limiting with exponential backoff retries. Update get_metadata_provider function to automatically wrap providers with rate limit handling. This improves reliability when external APIs return rate limit errors by implementing automatic retries with configurable delays and jitter.
2025-11-07 09:18:59 +08:00
Will Miao
c3932538e1 feat: add git reset and clean before nightly and release updates, fixes #646
Add hard reset and clean operations to ensure a clean working directory
before switching branches or checking out release tags. This prevents
local changes from interfering with the update process and ensures
consistent behavior across both nightly and release update paths.
2025-11-07 08:17:20 +08:00
Will Miao
a68141adf4 feat(i18n): add license restriction translations for multiple languages
Add license-related translation keys including credit requirements, derivative restrictions, and license sharing permissions. This supports displaying proper license information and restrictions in the UI across all supported languages (DE, EN, ES, FR, HE, JA, KO, RU).
2025-11-06 23:01:29 +08:00
Will Miao
fb8ba4c076 feat: update commercial icon configuration order 2025-11-06 22:55:08 +08:00
Will Miao
4ed3bd9039 feat: refactor model hash lookup to improve error handling and code clarity
- Simplify error handling logic by checking for "not found" message directly
- Extract model data fetching into separate _fetch_model_data method
- Extract version enrichment into separate _enrich_version_with_model_data method
- Improve logging consistency using %s formatting
- Rename variables for better clarity (result -> version, e -> exc)
2025-11-06 22:41:50 +08:00
Will Miao
ba6e2eadba feat: update license flag handling and default permissions
- Update DEFAULT_LICENSE_FLAGS from 57 to 127 to enable all commercial modes by default
- Replace CommercialUseLevel enum with bitwise commercial permission handling
- Simplify commercial value normalization and validation using allowed values set
- Adjust bit shifting in license flag construction to accommodate new commercial bits structure
- Remove CommercialUseLevel from exports and update tests accordingly
- Improve handling of empty commercial use values with proper type checking

The changes streamline commercial permission processing and align with CivitAI's default license configuration while maintaining backward compatibility.
2025-11-06 22:14:36 +08:00
Will Miao
1c16392367 feat: improve license restriction labels for clarity
Update license restriction labels in ModelModal component to be more descriptive and user-friendly. Changed fallback text and translation keys for various license restrictions including:
- Selling models
- Generation services
- Civitai generation
- Selling generated content
- Creator credit requirements
- Sharing merges
- Permission requirements

The changes make the license restrictions more clear and specific about what actions are prohibited or required.
2025-11-06 21:31:28 +08:00
Will Miao
035ad4b473 feat(ui): increase border opacity and adjust color for better visibility
Update the --lora-border CSS custom property to use a darker, more opaque color. The previous border color was too subtle and lacked sufficient contrast against the background. This change improves visual hierarchy and makes interface elements more distinguishable.
2025-11-06 21:17:29 +08:00
Will Miao
a7ee883227 feat(modal): add license restriction indicators to model modal
Add visual indicators for commercial license restrictions in the model modal. New CSS classes and JavaScript utilities handle the display of restriction icons for selling, renting, and image usage limitations. The modal header actions container has been restructured to accommodate the new license restriction section.

- Add `.modal-header-actions` and `.license-restrictions` CSS classes
- Implement commercial license icon configuration and rendering logic
- Normalize and sanitize commercial restriction values
- Update header layout to remove bottom margin for better visual alignment
2025-11-06 21:04:59 +08:00
Will Miao
ddf9e33961 feat: add license information handling for Civitai models
Add license resolution utilities and integrate license information into model metadata processing. The changes include:

- Add `resolve_license_payload` function to extract license data from Civitai model responses
- Integrate license information into model metadata in CivitaiClient and MetadataSyncService
- Add license flags support in model scanning and caching
- Implement CommercialUseLevel enum for standardized license classification
- Update model scanner to handle unknown fields when extracting metadata values

This ensures proper license attribution and compliance when working with Civitai models.
2025-11-06 17:05:54 +08:00
Will Miao
4301b3455f feat(civarchive_client): remove HTML scraping implementation and bs4 dependency
Remove legacy HTML scraping implementation of get_model_by_url method
and associated BeautifulSoup dependency. The functionality has been
replaced by API-based implementation in get_model_version method.

This simplifies the codebase and removes the optional bs4 dependency,
making the client more maintainable and reliable.
2025-11-05 22:31:39 +08:00
Will Miao
3d6bb432c4 feat: normalize tags to lowercase for Windows compatibility, see #637
Convert all tags to lowercase in tag processing logic to prevent case sensitivity issues on Windows filesystems. This ensures consistent tag matching and prevents duplicate tags with different cases from being created.

Changes include:
- TagUpdateService now converts tags to lowercase before comparison
- Utils function converts model tags to lowercase before priority resolution
- Test cases updated to reflect lowercase tag expectations
2025-11-04 12:54:09 +08:00
Will Miao
6c03aa1430 feat: add v0.9.9 release features and update version 2025-11-03 22:40:03 +08:00
pixelpaws
5376fd8724 Merge pull request #640 from willmiao/codex/optimize-confirmation-modal-message
feat(updates): improve check updates confirmation
2025-11-03 22:37:57 +08:00
pixelpaws
6dea9a76bc feat(updates): improve check updates confirmation 2025-11-03 22:36:57 +08:00
Will Miao
d73903e82e feat: prevent duplicate banner entries in recent history
Add duplicate detection to banner recording to prevent multiple entries
for the same banner ID in recent history. This prevents duplicate history
entries when pages refresh or banners are shown multiple times.

- Check if banner ID already exists in recentHistory before adding
- Return early if duplicate found to prevent adding same banner multiple times
- Add comprehensive tests for banner history functionality including:
  - Adding new banners to history
  - Preventing duplicate entries
  - Handling multiple different banners
- Clear history between tests to ensure test isolation
2025-11-03 20:13:48 +08:00
Will Miao
4862419b61 feat: refactor banner service and add comprehensive tests
- Remove legacy community support banner tracking variables and logic
- Simplify banner dismissal handling by checking dismissal state before marking
- Replace timer-based community support banner with immediate registration
- Clean up unused constants and legacy storage keys
- Add comprehensive test suite with mocked dependencies
- Improve code maintainability and test coverage
2025-11-03 19:50:35 +08:00
Will Miao
e6e7df7454 feat: add Chinese localization for community support banner
- Update zh-CN locale with Chinese text for community support section
- Add support for Afdian platform for Chinese users alongside existing Ko-fi
- Implement language-based URL routing for support links and tutorials
- Chinese users now see localized content with appropriate payment options (Alipay/WeChat)
- Maintains existing functionality for non-Chinese users
2025-11-03 18:00:25 +08:00
Will Miao
30f9e3e2ec feat(loras): add drag event callbacks and preview suppression
- Add onDragStart and onDragEnd callbacks to initDrag function
- Implement preview suppression during and briefly after strength dragging
- Clear preview timer on drag start/end to prevent tooltip conflicts
- Update tests to verify drag callbacks are properly triggered

This prevents tooltip previews from interfering with drag interactions and provides better control over drag lifecycle events.
2025-11-03 12:18:59 +08:00
Will Miao
707d0cb8a4 feat(lora-loader): add trigger word update on LoRA syntax edits
Add test coverage for trigger word refresh functionality when LoRA syntax is edited in the input widget. The test verifies that after modifying LoRA syntax in the input field, the connected trigger words are properly updated to reflect the active LoRAs.

Additionally, implement the actual trigger word update logic in lora_loader.js by calling updateConnectedTriggerWords after merging LoRAs, ensuring the UI stays synchronized with the current LoRA state.
2025-11-03 12:03:21 +08:00
Will Miao
56ea7594ce feat: simplify portable installation instructions
Remove redundant details about settings configuration to make installation steps clearer and more concise. The simplified instructions now focus on essential steps without unnecessary explanations about placeholder values and automatic registry generation.
2025-11-03 09:00:53 +08:00
Will Miao
389e46c251 feat(sidebar): add force initialization option and improve state management
- Add `forceInitialize` option to sidebar initialization to bypass disabled setting
- Refactor sidebar toggle logic to handle initialization promises more reliably
- Improve cleanup behavior when sidebar is disabled
- Ensure proper DOM updates when sidebar state changes
- Maintain container layout consistency during sidebar operations
2025-11-03 07:15:29 +08:00
pixelpaws
6db17e682a Merge pull request #639 from willmiao/codex/add-setting-to-toggle-folder-sidebar, fixes #630
feat: add setting to toggle folder sidebar visibility
2025-11-03 07:04:08 +08:00
pixelpaws
94e0308a12 feat(settings): allow hiding folder sidebar 2025-11-03 06:39:13 +08:00
pixelpaws
1f9f821576 Merge pull request #636 from willmiao/codex/reset-community-support-banner-logic
fix: reset community support banner timing
2025-11-02 22:40:25 +08:00
pixelpaws
57933dfba6 fix(banner): reset community support schedule 2025-11-02 22:30:33 +08:00
pixelpaws
c50bee7757 Merge pull request #635 from willmiao/codex/design-banner-message-review-feature
chore(ui): improve notification center accessibility
2025-11-02 21:04:07 +08:00
pixelpaws
4e3ee843f9 chore(ui): improve notification center accessibility 2025-11-02 20:59:00 +08:00
Will Miao
7e40f6fcb9 feat: add GGUF loader metadata extractor support, fixes #627
Add GGUFLoaderExtractor class to handle metadata extraction for GGUF model loaders. Register extractor for both LoaderGGUF and LoaderGGUFAdvanced node types to capture checkpoint metadata from gguf_name input parameter. This enables proper metadata tracking for GGUF model files used in the system.
2025-11-02 10:20:44 +08:00
Will Miao
7976956b6b feat: add model_cache to plugin folder cleanup skip list
Update the plugin cleanup process to preserve the model_cache folder
along with settings.json and civitai. This prevents accidental deletion
of cached model files during plugin updates, improving performance by
avoiding unnecessary model re-downloads.
2025-11-02 10:09:56 +08:00
Will Miao
adce5293d5 feat: add model_cache directory to gitignore 2025-11-02 10:01:15 +08:00
pixelpaws
c2db5eb6df Merge pull request #634 from willmiao/codex/refactor-config-to-handle-default-library
fix: drop template default library when saving ComfyUI paths
2025-11-02 09:12:26 +08:00
pixelpaws
f958ecdf18 Merge pull request #633 from willmiao/codex/update-settings-path-handling-for-portable-mode
Fix portable settings to use project root storage
2025-11-02 09:11:17 +08:00
pixelpaws
ef0bcc6cf1 fix(config): remove template default library before saving paths 2025-11-02 09:09:36 +08:00
pixelpaws
285428ad3a fix(portable): use project root for settings storage 2025-11-02 09:07:57 +08:00
pixelpaws
ee18cff3d9 Merge pull request #626 from willmiao/codex/update-model_update_versions-primary-key
fix: support per-model version ids in update service
2025-10-30 23:23:11 +08:00
pixelpaws
1be3235564 fix(model-updates): support per-model version ids 2025-10-30 23:15:23 +08:00
pixelpaws
a92883509a Merge pull request #624 from willmiao/codex/clarify-multi-libraries-support-for-comfyui, see #623
Add portable settings mode toggle
2025-10-30 14:18:41 +08:00
pixelpaws
ce42d83ce9 feat(settings): add portable settings toggle 2025-10-30 14:08:21 +08:00
Will Miao
077cf7b574 feat(metadata): Add extractors for NunchakuFluxDiTLoader and NunchakuQwenImageDiTLoader nodes, fixes #621 2025-10-29 23:19:11 +08:00
Will Miao
b99d78bda6 feat(nodes): enhance LoRA loading with path support and add tests
- Allow direct file paths in addition to registered LoRA names
- Add graceful handling for missing LoRA files with warning logs
- Add comprehensive unit tests for missing LoRA file handling
- Ensure backward compatibility with existing LoRA loading behavior
2025-10-29 22:39:08 +08:00
Will Miao
39586f4a20 feat: add LoRA syntax utilities and comprehensive test suite, fixes #600
- Implement core LoRA syntax manipulation functions including:
  - applyLoraValuesToText for updating LoRA strengths and clip values
  - normalizeStrengthValue for consistent numeric formatting
  - shouldIncludeClipStrength for clip strength inclusion logic
  - cleanupLoraSyntax for text normalization
  - debounce utility for input handling

- Add comprehensive test suite covering all utility functions
- Include edge cases for clip strength handling, numeric formatting, and syntax cleanup
- Support both basic and expanded LoRA syntax formats with proper value preservation
- Enable debounced input synchronization for better performance

The utilities provide robust handling of LoRA syntax patterns while maintaining compatibility with existing ComfyUI workflows.
2025-10-29 22:13:54 +08:00
Will Miao
4ef750b206 feat: improve dropdown menu responsiveness
Update dropdown menu CSS to use max-content and max() function for better responsive behavior. Replace fixed min-width with dynamic width calculation to ensure proper content fitting across different screen sizes while maintaining dropdown functionality.
2025-10-29 16:06:08 +08:00
pixelpaws
9d3d93823d Merge pull request #620 from willmiao/codex/update-refresh-button-titles
feat: clarify refresh menu copy
2025-10-29 15:47:08 +08:00
pixelpaws
45c1113b72 feat(ui): clarify refresh menu labels 2025-10-29 15:39:42 +08:00
Will Miao
e10717dcda feat(ui): improve update controls styling and error handling
- Add disabled and loading states for control group buttons with appropriate cursor and opacity styling
- Enhance dropdown toggle active state styling for update filter group
- Improve dropdown toggle layout with flex centering
- Add disabled state styling for dropdown items
- Refactor model update check to use shared helper function, removing redundant success handling and simplifying error flow
- Maintain existing functionality while improving user experience and code maintainability
2025-10-29 14:38:11 +08:00
Will Miao
315ab6f70b feat(ui): update filter button icon for better visual clarity
Changed the sync icon to an exclamation circle icon on the updates filter button to provide clearer visual indication of the filter's purpose and improve user interface consistency.
2025-10-29 09:45:31 +08:00
Will Miao
cf4d654c4b feat: improve file size extraction logic with primary model preference
Refactor `_extract_size_bytes` method to prioritize primary model files when calculating size. The new implementation:
- Extracts size parsing into separate `parse_size` function
- Adds logic to prefer files marked as both "model" type and "primary"
- Falls back to first valid size if no primary model file found
- Adds comprehensive tests for primary preference and fallback behavior

This ensures more accurate size reporting for model files, particularly when multiple file types are present in the response.
2025-10-29 09:16:29 +08:00
pixelpaws
569c829709 Merge pull request #619 from willmiao/codex/add-check-for-updates-in-bulk-context-menu
feat(bulk): add selected update check
2025-10-29 08:56:26 +08:00
pixelpaws
de05b59f29 test(routes): cover snake case model id payload 2025-10-29 07:33:58 +08:00
Will Miao
70a282a6c0 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-29 07:33:00 +08:00
Will Miao
b10bcf7e78 feat: add update availability filter to model list
Add a new filter option to show only models with available updates across all supported languages. This includes:

- Adding "updates" filter translations in all locale files (de, en, es, fr, he, ja, ko)
- Extending BaseModelApiClient to handle update_available_only query parameter
- Implementing update filter button in PageControls component with event listeners
- Adding corresponding CSS styles for active filter state

The feature allows users to quickly identify and focus on models that have updates available, improving the update management workflow.
2025-10-29 07:32:53 +08:00
pixelpaws
5fb10263f3 Merge pull request #618 from willmiao/codex/fix-database-schema-conflict-error, fixes #617
fix: prevent model update refresh failures on legacy schemas
2025-10-28 22:28:26 +08:00
pixelpaws
9e76c9783e fix(update-service): backfill unique constraint for status table 2025-10-28 22:19:21 +08:00
pixelpaws
7770976513 Merge pull request #616 from willmiao/codex/analyze-update-metadata-refresh-duration
fix: ignore removed civitai models during update refresh
2025-10-28 21:55:39 +08:00
pixelpaws
dc1f7ab6fe fix: handle civitai not found responses 2025-10-28 21:47:30 +08:00
pixelpaws
32b1d6c561 Merge pull request #614 from willmiao/codex/extend-modelcache-for-model_id-indexing
feat: cache versions by model id for faster lookup
2025-10-28 18:48:17 +08:00
pixelpaws
5264e49f2a feat(cache): index versions by model id 2025-10-28 18:39:37 +08:00
Will Miao
ce3adaf831 feat: disable automatic refresh in model versions fetch 2025-10-28 09:11:39 +08:00
Will Miao
e2f3e57f5c feat: replace window.confirm with modal for version deletion
Replace the native browser confirmation dialog with a custom modal when deleting model versions. This provides better UX with consistent styling, allows displaying version information (name, preview, metadata), and gives users more context before confirming deletion.

Key changes:
- Added modalManager import
- Created showDeleteVersionModal function to display deletion confirmation modal
- Enhanced performDeleteVersion function with better error handling and button state management
- Modal shows version preview, name, base model, and metadata
- Improved accessibility with proper modal interactions
- Maintains existing deletion functionality with enhanced user experience
2025-10-27 22:46:36 +08:00
Will Miao
5c2349ff42 feat: remove external links from model version names
Remove Civitai external links from model version names in the versions tab to improve UI consistency and prevent unintended navigation. Version names are now displayed as plain text spans instead of clickable links while maintaining the same visual styling.
2025-10-27 21:14:58 +08:00
Will Miao
50eee8c373 feat: add clickable version rows with Civitai links
- Add CSS class `is-clickable` to make version rows appear interactive
- Implement URL builder function for Civitai model version links
- Make version names clickable links that open Civitai pages in new tabs
- Add tooltips and data attributes for enhanced user experience
- Pass modelId to version rendering to support external linking

This improves user navigation by allowing direct access to model versions on Civitai without leaving the application.
2025-10-27 20:59:32 +08:00
pixelpaws
f89b792535 Merge pull request #609 from willmiao/codex/add-progress-logging-for-check-updates
feat: add progress logging to update refresh workflow
2025-10-27 20:57:14 +08:00
pixelpaws
6d0ea2841c feat(updates): add progress logging to refresh service 2025-10-27 20:41:07 +08:00
Will Miao
98678a8698 feat(loras): add backspace key handling for LoRA deletion with input focus check, fixes #601
Add keyboard navigation support for deleting selected LoRA entries using Backspace key while preventing accidental deletion when editing strength input values. The implementation includes:

- Backspace key now deletes selected LoRA when pressed outside strength inputs
- Backspace is ignored when focused on strength input fields to allow normal text editing
- Added corresponding test cases to verify both deletion and non-deletion scenarios

This prevents users from accidentally deleting LoRA entries while editing strength values and provides intuitive keyboard controls for LoRA management.
2025-10-27 19:39:49 +08:00
pixelpaws
5326fa2970 Merge pull request #607 from willmiao/codex/fix-404-errors-for-example-image-links, see #590
Preserve local previews when CivitAI images 404
2025-10-27 19:09:41 +08:00
pixelpaws
90547670a2 Merge pull request #608 from willmiao/codex/fix-example-images-download-path-issue
fix: allow legacy library folders when validating example image paths
2025-10-27 18:25:49 +08:00
pixelpaws
4753206c52 fix(example-images): accept legacy library folders 2025-10-27 18:21:43 +08:00
pixelpaws
613aa3b1c3 fix(example-images): preserve local previews on 404 2025-10-27 18:21:14 +08:00
Will Miao
a6b704d4b4 feat: remove version ID display from model versions tab
Remove the .version-id CSS class and corresponding HTML element that displayed version IDs in the model versions tab. This simplifies the UI by removing redundant information since version IDs are already available elsewhere in the interface and were causing visual clutter.
2025-10-27 12:58:15 +08:00
Will Miao
227d06c736 feat: adjust image cropping to prioritize face visibility in LoRA modal
Update CSS for version media images to bias cropping toward the upper region, ensuring faces remain visible when images are cropped. This improves user experience by maintaining important visual content within the constrained display area.
2025-10-27 12:42:25 +08:00
Will Miao
8508763831 feat: improve video URL detection to handle query parameters
Enhanced the `isVideoUrl` function to more accurately detect video URLs by:
- Adding support for URL query parameters and fragments
- Creating helper function `extractExtension` to handle URI decoding
- Checking multiple candidate values from different parts of the URL
- Maintaining backward compatibility with existing video detection

This improves reliability when detecting video URLs that contain query parameters or encoded characters.
2025-10-27 12:21:51 +08:00
Will Miao
136d3153fa feat: add modal file path resolution and synchronization
- Add getActiveModalFilePath function to resolve current file path from DOM state
- Add updateModalFilePathReferences function to synchronize file path across all modal controls
- Refactor existing code to use new path resolution functions
- Ensure metadata interactions remain in sync after file renames or moves
- Improve robustness by handling cases where DOM state hasn't been initialized yet
2025-10-27 12:04:48 +08:00
Will Miao
49bdf77040 feat: improve multipart file extension detection
Refactor _get_multipart_ext method to use known suffixes list for more reliable file extension detection. The new implementation handles compound file extensions like '.metadata.json.bak' and '.safetensors' by checking against predefined suffixes in order of length. Falls back to existing logic for unknown file types. This improves accuracy when working with model files that have complex naming conventions.
2025-10-27 11:15:16 +08:00
pixelpaws
f4dcd89835 Merge pull request #603 from willmiao/codex/analyze-lora-manager-recipe-detection-issue
fix(recipes): normalize relocated preview paths
2025-10-27 09:55:43 +08:00
pixelpaws
139e915711 fix(recipes): normalize relocated preview paths 2025-10-27 09:50:25 +08:00
pixelpaws
22eda58074 Merge pull request #599 from willmiao/codex/fix-settings.json-initialization-behavior
Fix settings template preservation on restart
2025-10-27 00:03:49 +08:00
pixelpaws
fb91cf4df2 fix(settings): preserve template settings file 2025-10-26 23:57:59 +08:00
Will Miao
e0332571da feat(settings): improve library bootstrap logic and path handling
- Normalize folder paths before library bootstrap to ensure consistent structure
- Add _has_configured_paths helper to detect valid folder configurations
- Enhance bootstrap logic to handle edge cases with single libraries and empty paths
- Update library payload construction to use normalized paths
- Add example settings file changes to demonstrate new path structure

The changes ensure more robust library initialization when folder paths are configured at the top level but not properly propagated to individual libraries.
2025-10-26 23:40:07 +08:00
pixelpaws
2d4bc47746 Merge pull request #597 from willmiao/codex/refactor-settings-manager-for-core-keys
Reduce core settings persistence to essential keys
2025-10-26 20:01:08 +08:00
pixelpaws
38e766484e fix(settings): restrict minimal persistence keys 2025-10-26 19:53:43 +08:00
Will Miao
b5ee4a6408 feat(settings): enhance settings handling and add startup messages, fixes #593 and fixes #594
- Add standalone mode detection via LORA_MANAGER_STANDALONE environment variable
- Improve error handling for settings file loading with specific JSON decode errors
- Add startup messages system to communicate configuration warnings and errors to users
- Include settings file path and startup messages in settings API response
- Automatically save settings when bootstrapping from defaults due to missing/invalid settings file
- Add configuration warnings collection for environment variables and other settings issues

The changes improve robustness of settings management and provide better user feedback when configuration issues occur.
2025-10-26 18:07:00 +08:00
Will Miao
7892df21ec feat: add dynamic loading overlay creation with accessibility
Add fallback DOM element creation in LoadingManager constructor to handle cases where loading overlay elements don't exist in the DOM. This ensures the loading functionality works even when the required HTML elements are missing.

- Create loading overlay, content container, progress bar, and status text elements dynamically
- Add ARIA attributes to progress bar for accessibility
- Move details container insertion to use the created loadingContent reference
- Maintain existing functionality while adding robustness for missing DOM elements
2025-10-26 10:52:24 +08:00
Will Miao
188fe407b6 feat(download): sync downloaded versions with update tracking
Add automatic synchronization of downloaded model versions with the update tracking system. After a successful download, the system now resolves model and version IDs from the download response and updates the update service with the newly downloaded version along with any existing local versions.

This ensures that:
- Update tracking accurately reflects which versions are available locally
- The system properly tracks both newly downloaded and existing versions
- Failed sync operations are gracefully handled with appropriate logging
- Support is included for LoRA, checkpoint, and embedding model types
2025-10-26 10:42:09 +08:00
Will Miao
600afdcd92 feat: add update badge to model modal versions tab
- Add CSS styling for tab badges with update indicator animation
- Include update_available flag in model data parsing
- Display animated badge on versions tab when updates are available
- Improve tab button layout with flexbox alignment and spacing
2025-10-26 10:11:04 +08:00
Will Miao
994fa4bd43 feat: enhance model version download with progress tracking
- Set refresh to true when fetching model update versions to ensure latest data
- Refactor handleDownloadVersion to be async and accept button parameter
- Add progress tracking and WebSocket integration for download operations
- Implement button state management during download process
- Add error handling and cleanup for download operations
- Update download action to await async download handler
2025-10-26 09:39:42 +08:00
Will Miao
51098f2829 feat: update model versions tab styling and refresh behavior
- Rename CSS class from 'version-name' to 'versions-tab-version-name' for better specificity
- Remove color-mix styling from version title for cleaner appearance
- Set refresh parameter to false in versions fetch to prevent unnecessary data reloads
- Maintains same functionality while improving performance and code organization
2025-10-26 09:13:59 +08:00
Will Miao
795b9e8418 feat: enhance model version context with file metadata
- Rename `preview_overrides` to `version_context` to better reflect expanded purpose
- Add file_path and file_name fields to version serialization
- Update method names and parameter signatures for consistency
- Include file metadata from cache in version context building
- Maintain backward compatibility with existing preview URL functionality

The changes provide more comprehensive version information including file details while maintaining existing preview override behavior.
2025-10-26 08:53:53 +08:00
Will Miao
9ca2b9dd56 feat: add model updates check to global context menu
Add a new "Check Model Updates" option to the global context menu that allows users to manually check for model updates. This includes:

- Adding refreshUpdates endpoint to API configuration
- Implementing checkModelUpdates method with proper loading states
- Adding internationalization support for update messages
- Handling success/error states with appropriate user feedback
- Automatically reloading models after update check completes

The feature provides users with manual control over update checks and improves visibility into model update availability.
2025-10-25 21:32:08 +08:00
Will Miao
d77b6d78b7 feat(model-updates): filter records without updates in refresh response
Add logic to only include model update records that have actual updates in the refresh response. This improves API efficiency by reducing payload size and only returning relevant data to clients.

The change:
- Adds filtering in ModelUpdateHandler.refresh_model_updates to check has_update method
- Only serializes records that have updates available
- Updates corresponding test to verify filtering behavior

This prevents returning unnecessary data for models that don't have updates available.
2025-10-25 21:31:36 +08:00
Will Miao
427e7a36d5 feat: improve model update detection logic
Update ModelUpdateRecord.has_update() to only detect updates when a newer remote version exists than the latest local version. Previously, any missing remote version would trigger an update, which could include older versions that shouldn't be considered updates.

- Add logic to find the maximum version ID in library
- Only return True for remote versions newer than the latest local version
- Add comprehensive unit tests for the new update detection behavior
- Update docstring to reflect the new logic
2025-10-25 21:31:01 +08:00
Will Miao
c90306cc9b feat: display abbreviated base model labels in model cards
- Add BASE_MODEL_ABBREVIATIONS mapping and getBaseModelAbbreviation utility function
- Replace full base model names with abbreviated versions in ModelCard component
- Implement fallback abbreviation generation for unknown base models
- Maintain full base model name in tooltip for accessibility
- Improve card layout by reducing label width while preserving information
2025-10-25 17:04:16 +08:00
Will Miao
5fe0660c64 feat: add update available indicator to model cards
- Add CSS custom properties for update badge styling in both light and dark themes
- Create new card header info layout with flexbox for better content organization
- Implement model-update-badge component with glow effects and proper spacing
- Add has-update class to cards when updates are available with visual border indicators
- Update ModelCard.js to conditionally render update badges based on model data
- Include internationalization support for update badge labels and tooltips

The changes provide users with clear visual indicators when model updates are available, improving the user experience by making update status immediately visible without requiring manual checks.
2025-10-25 16:41:35 +08:00
Will Miao
2abb5bf122 feat: add update_available flag to model services
Add update_available field to checkpoint, embedding, and LoRA service response formatting. The flag indicates whether a model update is available and defaults to false when not specified.

Include comprehensive tests to verify the update flag is properly included in formatted responses and defaults to false when not present in the payload.
2025-10-25 16:29:54 +08:00
Will Miao
bb65527469 feat: add database migration system for model update schema
Add migration support to handle schema changes without data loss. Instead of dropping and recreating tables, the system now:
- Uses CREATE TABLE IF NOT EXISTS for initial table creation
- Adds _apply_migrations method to handle incremental schema updates
- Adds _get_table_columns helper to inspect existing table structure
- Adds new columns to model_update_versions table (sort_index, name, base_model, released_at, size_bytes, preview_url, is_in_library, should_ignore)
- Adds should_ignore_model column to model_update_status table

This ensures existing databases are upgraded gracefully while preserving user data.
2025-10-25 16:05:39 +08:00
Will Miao
d9a6db3359 feat: optimize model update checking with bulk operations
- Refactor update filter logic to use bulk update checks when available
- Add annotation method to attach update flags to response items
- Improve performance by reducing API calls for update status checks
- Maintain backward compatibility with fallback to individual checks
- Handle edge cases and logging for failed update status resolutions
2025-10-25 15:31:33 +08:00
Will Miao
58cafdb713 feat: add model version management endpoints
- Add set_version_update_ignore endpoint to toggle ignore status for specific versions
- Add get_model_versions endpoint to retrieve version details with optional refresh
- Update serialization to include version-specific data and preview overrides
- Modify database schema to support version-level ignore tracking
- Improve error handling for rate limiting and missing models

These changes enable granular control over version updates and provide better visibility into model version status.
2025-10-25 14:54:23 +08:00
pixelpaws
0594e278b6 Merge pull request #592 from willmiao/codex/update-preview-download-logic-for-nsfw-settings
feat(preview): respect blur mature content setting
2025-10-25 06:49:51 +08:00
pixelpaws
807425f12a feat(preview): respect blur mature content setting 2025-10-25 06:43:03 +08:00
pixelpaws
aa4b1ccc25 Update FUNDING.yml 2025-10-24 22:23:32 +08:00
pixelpaws
58255ec28b Update FUNDING.yml 2025-10-24 22:12:36 +08:00
pixelpaws
d62b84693d Update FUNDING.yml 2025-10-24 22:10:33 +08:00
Will Miao
df75c7e68d feat: enhance Lora modal recipes section with new header and card components
- Add comprehensive recipes header with title, description, and view-all button
- Implement recipe card grid layout with responsive design
- Add recipe cards featuring titles, metadata badges, and copy functionality
- Include theme-aware styling for both light and dark modes
- Improve visual hierarchy and user interaction with hover states and transitions
2025-10-24 20:27:59 +08:00
pixelpaws
c5c7fdf54f Merge pull request #591 from willmiao/codex/investigate-share-button-network-error
Fix recipe share filename sanitization
2025-10-24 15:22:04 +08:00
pixelpaws
49e0deeff3 fix(recipes): sanitize shared recipe filenames 2025-10-24 15:17:12 +08:00
pixelpaws
0c20701bef Merge pull request #589 from willmiao/codex/fix-download-stalling-issues
Fix stalled downloads by adding stall detection and reconnect logic
2025-10-23 17:40:31 +08:00
pixelpaws
faa26651dd fix(download): recover stalled transfers automatically 2025-10-23 17:25:38 +08:00
pixelpaws
2eae8a7729 Merge pull request #587 from willmiao/codex/evaluate-file-validation-improvements-for-lora-manager
Ensure downloader rejects empty or truncated files
2025-10-23 12:17:13 +08:00
pixelpaws
dde2b2a960 fix(downloader): enforce file size integrity checks 2025-10-23 11:55:39 +08:00
pixelpaws
4a9089d3dd Merge pull request #586 from willmiao/codex/analyze-and-propose-fix-for-header-value-error
fix: sanitize aiohttp header limit overrides
2025-10-23 11:23:08 +08:00
pixelpaws
3244a5f1a1 fix(lora-manager): sanitize header limit overrides 2025-10-23 11:13:31 +08:00
Will Miao
449c1e9d10 feat(i18n): add new UI text for model management features
Add localization strings for new model management functionality:
- Copy checkpoint and embedding name actions
- Send checkpoint and embedding to ComfyUI workflow
- Error messages for missing model paths and workflow compatibility
- Node selection validation messages

These additions support upcoming features for better model handling and workflow integration.
2025-10-23 11:11:13 +08:00
Will Miao
d0aa916683 feat(node-registry): add support to send checkpoint/diffusion model to workflow
- Add capabilities parsing and validation for node registration
- Implement widget_names extraction from capabilities with type safety
- Add supports_lora boolean conversion in capabilities
- Include comfy_class fallback to node_type when missing
- Add new update_node_widget API endpoint for bulk widget updates
- Improve error handling and input validation for widget updates
- Remove unused parameters from node selector event setup function

These changes improve node metadata handling and enable dynamic widget management capabilities.
2025-10-23 10:44:48 +08:00
pixelpaws
13433f8cd2 Merge pull request #585 from willmiao/codex/fix-model_type-not-updating-for-checkpoints
fix: apply adjust_cached_entry during model reconciliation
2025-10-23 08:33:33 +08:00
pixelpaws
8d336320c0 fix(scanner): apply metadata adjustments during reconciliation 2025-10-23 07:34:35 +08:00
pixelpaws
d945c58d51 Merge pull request #583 from willmiao/codex/analyze-zstd-content-encoding-error
fix: disable compression in default downloader headers
2025-10-22 10:21:17 +08:00
pixelpaws
acaf122346 fix(downloader): request identity encoding by default 2025-10-22 10:17:39 +08:00
pixelpaws
713759b411 Merge pull request #582 from willmiao/codex/fix-model-type-adjustment-in-scanner
Fix checkpoint model type when hydrating persisted cache
2025-10-21 22:59:55 +08:00
pixelpaws
c5175bb870 fix(checkpoints): preserve model type on persisted load 2025-10-21 22:55:00 +08:00
pixelpaws
e63ef8d031 Merge pull request #581 from willmiao/codex/fix-typeerror-in-autocomplete.js
fix: clean up autocomplete event handlers
2025-10-21 19:33:02 +08:00
pixelpaws
e043537241 fix(autocomplete): detach listeners when dropdown removed 2025-10-21 19:28:22 +08:00
Will Miao
46126f9950 feat(extensions): add auto path correction toggle for LoRA Manager, fixes #410 2025-10-21 18:47:42 +08:00
Will Miao
f4eb916914 feat: standardize LoRA Manager frontend with CSS classes and improved styles
- Replace inline styles with CSS classes for better maintainability
- Update class names to use consistent 'lm-' prefix across components
- Add comprehensive CSS stylesheet with tooltip system and responsive layouts
- Improve accessibility with proper focus states and keyboard navigation
- Implement hover and active state transitions for enhanced UX
- Refactor expand button to use CSS classes instead of inline styles
- Update test files to reflect new class naming convention
2025-10-21 17:42:32 +08:00
Will Miao
49b9b7a5ea feat(lora): remove deprecated defaultInput, use only forceInput 2025-10-21 11:55:51 +08:00
Will Miao
9b1a9ee071 feat: refactor LoRA manager widget into top menu extension
- Rename ui_utils.js to top_menu_extension.js to better reflect functionality
- Replace custom button creation with ComfyUI Button and ButtonGroup components
- Implement proper top menu integration using ComfyUI's menu system
- Simplify window opening logic with shift-click support for new windows
- Add retry mechanism for attaching button to menu
- Improve code organization and maintainability by leveraging existing UI components
2025-10-21 11:54:50 +08:00
Will Miao
0b8f137a1b feat(i18n): update French translation for cleanup example images label 2025-10-19 22:33:32 +08:00
Will Miao
6148a12301 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-19 20:06:08 +08:00
Will Miao
fadbf21b4f fix(relink): keep sha untouched during relinking 2025-10-19 20:05:58 +08:00
pixelpaws
c38a06937d Merge pull request #578 from willmiao/codex/propose-solutions-for-missing-model_name, fixes #429
fix: guard model cache against missing metadata fields
2025-10-18 21:33:00 +08:00
pixelpaws
1a34403b0e fix(model-cache): avoid mutating raw entries without fields 2025-10-18 21:30:49 +08:00
pixelpaws
e4d58d0f60 fix(cache): harden metadata defaults 2025-10-18 21:19:09 +08:00
pixelpaws
4e4ea85cc3 Merge pull request #577 from willmiao/codex/add-sanitize_folder_name-utility-method, #552
fix: sanitize path template folder names
2025-10-18 16:48:43 +08:00
pixelpaws
f7a856349a fix(utils): sanitize path template folder names 2025-10-18 16:20:47 +08:00
Will Miao
15edd7a42c feat(settings): remove redundant option descriptions from layout settings 2025-10-18 09:52:47 +08:00
Will Miao
46243a236d feat(ui): refactor LoRA widget focus navigation
Refactor focus navigation logic in LoRA widget to separate focus queueing from execution. Added helper functions for finding focus entries and managing focus queue. This improves code organization and prevents focus issues when tabbing between strength and clip inputs.

Key changes:
- Extract focus navigation logic into reusable functions
- Separate focus queueing from focus execution
- Maintain same keyboard navigation behavior while improving code structure
- Fix potential focus loss when tabbing between inputs

The refactoring makes the focus navigation code more maintainable and reduces duplication while preserving the existing tab navigation functionality.
2025-10-18 09:26:33 +08:00
Will Miao
6f382e587a feat(loras): track pending focus target for strength inputs
Add pendingFocusTarget state to track which LoRA strength input is being interacted with. This ensures proper focus behavior when clicking on strength inputs, particularly when the widget is being re-rendered. The focus is now properly restored to the correct input after UI updates.
2025-10-18 08:49:12 +08:00
Will Miao
bf3d706bf4 feat(ui): add keyboard navigation for LoRA strength inputs, #432 2025-10-18 08:36:10 +08:00
Will Miao
cdf21e813c feat(settings): improve priority tags header layout and accessibility 2025-10-17 16:51:11 +08:00
Will Miao
10f5588e4a feat: add model_name and version_name placeholders to download paths, #552
Add support for {model_name} and {version_name} placeholders in download path templates. These new placeholders allow for more flexible and descriptive file organization by including the actual model name and version name in the download directory structure.

Changes include:
- Updated download_manager.py and utils.py to handle new placeholders
- Added placeholders to constants.js for UI reference
- Updated settings modal template to show available placeholders
- Added comprehensive tests to verify placeholder functionality

This enhancement provides users with more control over how downloaded models are organized on their file system.
2025-10-17 16:01:06 +08:00
Will Miao
0ecbdf6f39 feat(context-menu): prevent duplicate NSFW selector initialization
Add initialization tracking to prevent multiple event listener attachments
in context menu components. Use dataset.initialized flag to ensure NSFW
selector events are only set up once per component instance.

In ModelContextMenuMixin, replace DOM elements and reattach event listeners
to avoid duplicates when components are reinitialized. This fixes issues
where multiple click handlers could be attached to the same elements.
2025-10-17 10:52:02 +08:00
Will Miao
61101a7ad0 fix(recipe-scanner): honor SFW filtering option, fixes #576 2025-10-17 10:35:00 +08:00
Will Miao
6d9be814a5 feat(ui): add configurable model card footer action, fixes #249 2025-10-17 08:43:35 +08:00
pixelpaws
52bf93e430 Merge pull request #574 from willmiao/codex/add-model-name-display-setting
feat: respect model name display preference in model cache
2025-10-16 09:21:19 +08:00
pixelpaws
00fade756c fix(settings): dispatch name display updates on original loop 2025-10-16 09:02:35 +08:00
pixelpaws
3c0feb23ba feat(model-cache): respect model name display preference 2025-10-16 07:01:04 +08:00
Will Miao
3627840fe9 feat: update portable package download link to v0.9.8 2025-10-15 21:19:15 +08:00
pixelpaws
bbdc1bba87 Merge pull request #573 from willmiao/codex/add-batch-model-version-retrieval
feat: batch model update refresh using Civitai bulk API
2025-10-15 20:55:53 +08:00
pixelpaws
21a1bc1a01 feat(metadata): batch refresh model versions 2025-10-15 20:47:30 +08:00
Will Miao
0968698804 feat: add v0.9.8 release notes and update version 2025-10-15 19:47:13 +08:00
Will Miao
a5b2e9b0bf feat: add update service dependency and has_update filter
- Pass ModelUpdateService to CheckpointService, EmbeddingService, and LoraService constructors
- Add has_update query parameter filter to model listing handler
- Update BaseModelService to accept optional update_service parameter

These changes enable model update functionality across different model types and provide filtering capability for models with available updates.
2025-10-15 17:25:16 +08:00
pixelpaws
5a6ff444b9 Merge pull request #572 from willmiao/codex/design-ui-for-model-update-notifications
refactor: tighten civitai update endpoints
2025-10-15 16:01:20 +08:00
pixelpaws
3bb240d3c1 fix(updates): avoid caching failed civitai lookups 2025-10-15 16:00:23 +08:00
pixelpaws
ee0d241c75 refactor(routes): limit update endpoints to essentials 2025-10-15 15:37:35 +08:00
Will Miao
321ff72953 feat: remove delay from bulk auto-organize completion 2025-10-15 10:32:59 +08:00
Will Miao
412f1e62a1 feat(i18n): add model name display option and improve localization, fixes #440
- Add new model name display setting with options to show model name or file name
- Implement helper function to determine display name based on user preference
- Update model card footer to use dynamic display name
- Include model name display setting in settings modal and state management
- Remove redundant labels from display density descriptions in multiple locales
- Simplify card info display descriptions by removing duplicate text

The changes provide cleaner UI text and add flexibility for users to choose between displaying model names or file names in card footers.
2025-10-15 10:23:39 +08:00
Will Miao
8901b32a55 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-15 09:19:05 +08:00
Will Miao
8ab6cc72ad feat: add project documentation in IFLOW.md 2025-10-15 09:18:59 +08:00
pixelpaws
52e671638b feat(example-images): add stop control for download panel 2025-10-15 08:46:03 +08:00
Will Miao
a3070f8d82 feat: add rate limit error handling to CivArchive client
- Add RateLimitError import and exception handling in API methods
- Create _make_request wrapper to surface rate limit errors from downloader
- Add test case to verify rate limit error propagation
- Set default provider as "civarchive_api" for rate limit errors

This ensures rate limit errors are properly propagated and handled throughout the CivArchive client, improving error reporting and allowing callers to implement appropriate retry logic.
2025-10-14 21:38:24 +08:00
Will Miao
3fde474583 feat(civitai): add rate limiting support and error handling
- Add RateLimitError import and _make_request wrapper method to handle rate limiting
- Update API methods to use _make_request wrapper instead of direct downloader calls
- Add explicit RateLimitError handling in API methods to properly propagate rate limit errors
- Add _extract_retry_after method to parse Retry-After headers
- Improve error handling by surfacing rate limit information to callers

These changes ensure that rate limiting from the Civitai API is properly detected and handled, allowing callers to implement appropriate backoff strategies when rate limits are encountered.
2025-10-14 21:38:24 +08:00
Will Miao
1454991d6d feat(i18n): update bulk action labels to reflect selected items
Change bulk action labels from "All" to "Selected" in both English and Chinese locales to accurately reflect that these actions apply only to selected items rather than all items. This improves user interface clarity and prevents potential confusion about the scope of bulk operations.
2025-10-14 21:36:11 +08:00
Will Miao
4398851bb9 feat: reorder and update context menu items
- Remove 'clear' action from context menu
- Reorder menu items to prioritize common operations
- Move destructive operations (delete, move) to bottom section
- Add visual separation between action groups
- Maintain all existing functionality with improved organization
2025-10-14 21:29:09 +08:00
Will Miao
5173aa6c20 feat(model-scanner): add logging for file processing, fixes #566 2025-10-14 19:44:59 +08:00
Will Miao
3d98572a62 feat: improve civitai data handling and type safety, fixes #565
- Replace setdefault with get and explicit dict initialization in MetadataUpdater
- Change civitai field type from Optional[Dict] to Dict[str, Any] with default_factory
- Add None check and dict initialization in BaseModelMetadata.__post_init__
- Ensures civitai data is always a dictionary, preventing type errors and improving code reliability
2025-10-14 16:03:33 +08:00
Will Miao
c48095d9c6 feat: replace IO type imports with string literals
Remove direct imports of IO type constants from comfy.comfy_types and replace them with string literals "STRING" in input type definitions and return types. This improves code portability and reduces dependency on external type definitions.

Changes made across multiple files:
- Remove `from comfy.comfy_types import IO` imports
- Replace `IO.STRING` with "STRING" in INPUT_TYPES and RETURN_TYPES
- Move CLIPTextEncode import to function scope in prompt.py for better dependency management

This refactor maintains the same functionality while making the code more self-contained and reducing external dependencies.
2025-10-14 09:12:55 +08:00
Will Miao
1e4d1b8f15 feat(nodes): add Promp (LoraManager) node and autocomplete support 2025-10-13 23:23:32 +08:00
pixelpaws
8c037465ba Merge pull request #564 from willmiao/codex/design-apis-for-pause-and-resume-download
test: add coverage for download pause and resume controls
2025-10-13 21:39:47 +08:00
pixelpaws
055c1ca0d4 test(downloads): cover pause and resume flows 2025-10-13 21:30:23 +08:00
Will Miao
27370df93a feat(download): add support to download models from civarchive, fixes #381 2025-10-13 19:27:56 +08:00
Will Miao
60d23aa238 feat(download): enhance download progress ui with transfer stats 2025-10-13 19:06:51 +08:00
pixelpaws
5e441d9c4f Merge pull request #563 from willmiao/codex/add-download-speed-info-to-progress
feat(downloads): expose throughput metrics in progress APIs
2025-10-13 18:11:32 +08:00
pixelpaws
eb76468280 feat(downloads): expose throughput metrics in progress APIs 2025-10-13 14:39:31 +08:00
Will Miao
01bbaa31a8 fix(ModelTags): fix performance and UX issues in ModelTags 2025-10-12 22:31:10 +08:00
Will Miao
bddf023dc4 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-12 17:43:31 +08:00
Will Miao
8e69a247ed feat(i18n): Update priority tags translations for better localization 2025-10-12 17:43:26 +08:00
pixelpaws
97141b01e1 Merge pull request #562 from willmiao/incremental-cache, see #561
feat(model-scanner): add metadata tracking and improve cache management
2025-10-12 17:09:19 +08:00
Will Miao
acf610ddff feat(model-scanner): add metadata tracking and improve cache management
- Add metadata_source field to track origin of model metadata
- Define MODEL_COLUMNS constants for consistent column management
- Refactor SQL queries to use dynamic column selection
- Improve Civitai data detection to include creator_username and trained_words
- Update database operations to handle new metadata field and tag management
2025-10-12 16:54:39 +08:00
Will Miao
a9a6f66035 feat(api): enhance model API creation with validation and default fallback
Refactored `createModelApiClient` to pass the specific model type as a parameter to each client constructor. Introduced `isValidModelType` for validation and added logic to set a default model type if provided type is invalid or not specified. Updated `getModelApiClient` function to utilize these improvements, ensuring robust model API instantiation.
2025-10-12 15:28:30 +08:00
Will Miao
0040863a03 feat(tests): introduce ROUTE_CALLS_KEY for organizing route calls
Addressed the aiohttp warnings by aligning the test scaffolding with current best practices. Added an AppKey constant and stored the route tracking list through it to satisfy aiohttp’s NotAppKeyWarning expectations. Swapped the websocket lambdas for async no-op handlers so the registered routes now point to coroutine callables, clearing the deprecation warning about bare functions.
2025-10-12 09:12:57 +08:00
Will Miao
4ab86b4ae2 feat(locale): add drag drop error message in locales 2025-10-12 09:07:36 +08:00
Will Miao
b32b4b4042 feat: enhance model scanning to include creator username
Updated the `ModelScanner` class to extract and format the creator username from Civitai data. This enhancement ensures that the creator information is properly included in slim model data.
2025-10-12 08:51:42 +08:00
Will Miao
4e552dcf3e feat: Add drag-and-drop support with visual feedback for sidebar nodes
This commit implements drag-and-drop functionality for sidebar nodes,
adding visual feedback via highlight styling when dragging over
valid drop targets. The CSS introduces new classes to style
drop indicators using the lora-accent color scheme, while the JS
adds event handlers to manage drag operations and update the UI
state accordingly. This improves user interaction by providing
clear visual cues for valid drop areas during file operations.
2025-10-12 06:55:01 +08:00
Will Miao
8f4c02efdc Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-12 05:44:07 +08:00
Will Miao
b77c596f3a Fix error message to improve clarity in DownloadManager 2025-10-12 05:43:59 +08:00
pixelpaws
181f0b5626 Merge pull request #560 from willmiao/codex/analyze-coverage-for-backend-tests
test: add standalone bootstrap and model factory coverage
2025-10-11 22:48:35 +08:00
pixelpaws
480e5d966f test: add standalone bootstrap and model factory coverage 2025-10-11 22:42:24 +08:00
Will Miao
e8636b949d feat(ModelTags): Implemented drag-and-drop reordering for tag edit mode so users can rearrange tags directly in the UI, fixes #414 2025-10-11 20:56:22 +08:00
pixelpaws
8ea369db47 Merge pull request #559 from willmiao/codex/add-info-level-logging-in-fetch_and_update_model
feat: log metadata channel on metadata fetch
2025-10-11 20:37:08 +08:00
Will Miao
ec9b37eb53 feat: add model type context to tag suggestions
- Pass modelType parameter to setupTagEditMode function
- Implement model type aware priority tag suggestions
- Add model type normalization and resolution logic
- Handle suggestion state reset when model type changes
- Maintain backward compatibility with existing functionality

The changes enable context-aware tag suggestions based on model type, improving tag relevance and user experience when editing tags for different model types.
2025-10-11 20:36:38 +08:00
Will Miao
b0847f6b87 feat(doc): update priority tags configuration guide wiki 2025-10-11 20:08:42 +08:00
pixelpaws
84d10b1f3b feat(metadata): log metadata channel on fetch 2025-10-11 20:07:01 +08:00
Will Miao
4fdc97d062 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-11 19:43:41 +08:00
Will Miao
5fe5e7ea54 feat(ui): enhance settings modal styling and add priority tags tabs
- Rename `.settings-open-location-button` to `.settings-action-link` for better semantic meaning
- Add enhanced hover/focus states with accent colors and border transitions
- Implement tabbed interface for priority tags with LoRA, checkpoint, and embedding sections
- Improve input styling with consistent error states and example code formatting
- Remove deprecated grid layout in favor of tab-based organization
- Add responsive tab navigation with proper focus management and visual feedback
2025-10-11 19:43:22 +08:00
pixelpaws
7be1a2bd65 Merge pull request #557 from willmiao/civarc-api-support
CivArchive API support
2025-10-11 18:10:06 +08:00
pixelpaws
87842385c6 Merge pull request #558 from willmiao/codex/design-custom-priority-tags-format
feat: add customizable priority tags
2025-10-11 17:46:13 +08:00
Will Miao
1dc189eb39 feat(metadata): implement fallback provider strategy for deleted models
Refactor metadata sync service to use a prioritized provider fallback system when handling deleted CivitAI models. The new approach:

1. Attempts civarchive_api provider first for deleted models
2. Falls back to sqlite provider if archive DB is enabled
3. Maintains existing default provider behavior for non-deleted models
4. Tracks provider attempts and errors for better debugging

This improves reliability when fetching metadata for deleted models by trying multiple sources before giving up, and provides clearer error messages based on which providers were attempted.
2025-10-11 17:44:38 +08:00
pixelpaws
6120922204 chore(priority-tags): add newline terminator 2025-10-11 17:38:20 +08:00
Will Miao
ddb30dbb17 Revert "feat(civarchive_client): update get_model_version_info to resolve the real model/version IDs before fetching the target metadata."
This reverts commit c3a66ecf28.
2025-10-11 16:11:17 +08:00
Will Miao
1e8bd88e28 feat(metadata): improve model ID redirect logic and provider ordering
- Fix CivArchive model ID redirect logic to only follow redirects when context points to original model
- Rename CivitaiModelMetadataProvider to CivArchiveModelMetadataProvider for consistency
- Reorder fallback metadata providers to prioritize Civitai API over CivArchive API for better metadata quality
- Remove unused asyncio import and redundant logging from metadata sync service
2025-10-11 16:11:13 +08:00
Will Miao
c3a66ecf28 feat(civarchive_client): update get_model_version_info to resolve the real model/version IDs before fetching the target metadata. 2025-10-11 15:07:42 +08:00
Will Miao
1f60160e8b feat(civarchive_client): enhance request handling and context parsing
Introduce `_request_json` method for async JSON requests and improved error handling. Add static methods `_normalize_payload`, `_split_context`, `_ensure_list`, and `_build_model_info` to parse and normalize API responses. These changes improve the robustness of the CivArchiveClient by ensuring consistent data structures and handling potential API response issues gracefully.
2025-10-11 13:07:29 +08:00
Will Miao
7d560bf07a chore: add refs 2025-10-11 12:59:13 +08:00
Will Miao
47da9949d9 feat: update recipe download URL path to include /lm prefix
Update the download URL path in RecipeSharingService to include '/lm' prefix,
aligning with the new API route structure for recipe sharing endpoints.
2025-10-10 21:46:56 +08:00
scruffynerf
68c0a5ba71 Better Civ Archive support (adds API) (#549)
* add CivArchive API

* Oops, missed committing this part when I updated codebase to latest version

* Adjust API for version fetching and solve the broken API (hash gives only files, not models - likely to be fixed but in the meantime...)

* add asyncio import to allow timeout cooldown

---------

Co-authored-by: Scruffy Nerf <Scruffynerf@duck.com>
2025-10-10 20:04:01 +08:00
pixelpaws
1aa81c803b Merge pull request #551 from willmiao/codex/refactor-model-metadata-saving-logic
fix: hydrate cached metadata before persisting updates
2025-10-10 08:56:12 +08:00
pixelpaws
8f5e134d3e fix: skip redundant hydration in metadata sync service 2025-10-10 08:49:54 +08:00
pixelpaws
ef03a2a917 fix(metadata): hydrate cached records before saving 2025-10-10 08:30:51 +08:00
Will Miao
e275968553 feat(civitai): remove debug print statement from rewrite_preview_url function 2025-10-10 08:18:23 +08:00
Will Miao
76d3aa2b5b feat(version): bump version to 0.9.7 in pyproject.toml 2025-10-09 22:15:50 +08:00
Will Miao
c9a65c7347 feat(metadata): implement model data hydration and enhance metadata handling across services, fixes #547 2025-10-09 22:15:07 +08:00
Will Miao
f542ade628 feat(civitai): implement URL rewriting for Civitai previews and enhance download handling, fixes #499 2025-10-09 17:54:37 +08:00
Will Miao
d2c2bfbe6a feat(sidebar): add recursive search functionality and toggle button 2025-10-09 17:07:10 +08:00
Will Miao
2b6910bd55 feat(misc): mark model versions in library for Civitai user models 2025-10-09 15:23:42 +08:00
Will Miao
b1dd733493 feat(civitai): enhance model version handling with cache lookup 2025-10-09 14:10:00 +08:00
pixelpaws
5dcf0a1e48 Merge pull request #545 from willmiao/codex/evaluate-sqlite-cache-indexing-necessity
feat: index cached models by version id
2025-10-09 13:54:46 +08:00
pixelpaws
cf357b57fc feat(scanner): index cached models by version id 2025-10-09 13:50:44 +08:00
pixelpaws
4e1773833f Merge pull request #544 from willmiao/codex/add-endpoint-to-fetch-civitai-user-models
Add endpoint to fetch Civitai user models
2025-10-09 11:56:57 +08:00
pixelpaws
8cf762ffd3 feat(misc): add civitai user model lookup 2025-10-09 11:49:41 +08:00
pixelpaws
d997eaa429 Merge pull request #543 from willmiao/codex/refactor-get_model_version-logic-and-add-tests, fixes #540
fix: improve Civitai model version retrieval
2025-10-09 11:07:33 +08:00
pixelpaws
8e51f0f19f fix(civitai): improve model version retrieval 2025-10-09 10:56:25 +08:00
pixelpaws
f0e246b4ac Merge pull request #542 from willmiao/codex/investigate-backend-tests-modifying-settings.json
Refactor settings manager to lazy singleton
2025-10-08 16:02:11 +08:00
pixelpaws
a232997a79 fix(utils): respect metadata sync overrides 2025-10-08 15:52:15 +08:00
pixelpaws
08a449db99 fix(metadata): refresh metadata sync settings 2025-10-08 10:38:05 +08:00
pixelpaws
0c023c9888 fix(settings): lazily resolve module aliases 2025-10-08 10:10:23 +08:00
pixelpaws
0ad92d00b3 fix(settings): restore legacy settings aliases 2025-10-08 09:54:36 +08:00
pixelpaws
a726cbea1e fix(routes): pass resolved settings to metadata sync 2025-10-08 09:32:57 +08:00
pixelpaws
c53fa8692b refactor(settings): lazily initialize manager 2025-10-08 08:56:57 +08:00
Will Miao
3118f3b43c feat(graph): enhance node handling with graph identifiers and improve metadata updates, see #408, #538 2025-10-07 23:22:38 +08:00
Will Miao
9199950b74 feat(release): update version to 0.9.6 and add release notes for critical performance optimizations and new features 2025-10-07 17:41:58 +08:00
Will Miao
4c7e31687b feat(recipe_scanner): reset cached state on active library change 2025-10-07 08:07:44 +08:00
Will Miao
75e207b520 fix(banner): remove redundant community support banner registration 2025-10-06 22:39:49 +08:00
Will Miao
631289b75e feat(banner): add community support banner with Ko-fi integration and translations 2025-10-06 22:39:21 +08:00
Will Miao
1b958d0a5d feat(modal): use CSS variables for header height and improve recipe modal layout 2025-10-06 16:28:56 +08:00
Will Miao
35fdf9020d docs(README): update instructions for settings.json file creation in Portable Edition 2025-10-06 15:32:37 +08:00
Will Miao
45926b1dca feat(constants): add CHROMA model to BASE_MODELS and update categories 2025-10-06 08:48:15 +08:00
Will Miao
686ba5024d fix(tests): add loadingManager mocks in settingsManager tests 2025-10-06 08:24:58 +08:00
pixelpaws
cf375c7c86 Merge pull request #537 from willmiao/codex/implement-video-lazy-loading-with-queue
fix: throttle model card video lazy loading
2025-10-06 08:07:24 +08:00
pixelpaws
5e53d76f44 fix(model-card): throttle preview video loading 2025-10-06 07:45:51 +08:00
Will Miao
7757f72859 Enhance test for saving paths to ensure cross-platform compatibility in folder paths 2025-10-05 22:40:43 +08:00
pixelpaws
c8cc584049 Merge pull request #536 from willmiao/codex/add-unit-tests-for-config-saving-paths
Add regression tests for Config save path handling
2025-10-05 22:24:57 +08:00
pixelpaws
2cdd269bba Merge pull request #535 from willmiao/codex/add-tests-for-migration-utility-functions
test: cover example images migration flows
2025-10-05 22:24:42 +08:00
pixelpaws
d2d97ae5bb Merge pull request #534 from willmiao/codex/add-tests-for-cache-middleware
test: add cache control middleware coverage
2025-10-05 22:24:26 +08:00
pixelpaws
d08d77c555 Merge pull request #533 from willmiao/codex/add-async-test-module-for-lifecycle
test: add LoRA manager lifecycle coverage
2025-10-05 22:24:12 +08:00
pixelpaws
92f8d2139a Merge pull request #532 from willmiao/codex/create-tests-for-statsroutes
test: add coverage for stats routes endpoints
2025-10-05 22:23:59 +08:00
pixelpaws
50f2c2dfe6 test(config): cover save path migration flows 2025-10-05 22:19:36 +08:00
pixelpaws
3539c453d3 test(utils): cover example images migrations 2025-10-05 22:19:29 +08:00
pixelpaws
1631122f95 test(middleware): add cache control coverage 2025-10-05 22:19:20 +08:00
pixelpaws
8fcb979544 test(routes): cover lora manager lifecycle 2025-10-05 22:19:10 +08:00
pixelpaws
8a5af0b7f3 test(routes): add stats routes coverage 2025-10-05 22:18:59 +08:00
Will Miao
cb1f08d556 Fix low contrast on nav-item hover 2025-10-05 21:10:01 +08:00
pixelpaws
1150267765 Merge pull request #531 from willmiao/codex/-activelibrary
fix(locales): translate library selection strings
2025-10-05 21:04:10 +08:00
pixelpaws
5c1252548d fix(locales): translate library selection strings 2025-10-05 21:03:21 +08:00
Will Miao
3c7cdf5db8 feat(header): enhance navigation and search functionality with context-aware behavior 2025-10-05 20:58:14 +08:00
Will Miao
9ac4203b1c test(example-images): allow monkeypatching os.startfile on linux 2025-10-05 16:30:44 +08:00
pixelpaws
d0800510db Merge pull request #529 from willmiao/codex/update-file-url-formatting-methods
fix: route recipe previews through preview API
2025-10-05 15:53:26 +08:00
pixelpaws
f8ba551cc4 fix(recipes): use preview endpoint for recipe images 2025-10-05 15:49:18 +08:00
Will Miao
413444500e refactor(model_scanner): normalize path comparisons for model roots
fix(example_images_download_manager): re-raise specific exception on download error

refactor(usage_stats): define constants locally to avoid conditional imports

test(example_images_download_manager): update exception handling in download tests

test(example_images_file_manager): differentiate between os.startfile and subprocess.Popen in tests

test(example_images_paths): ensure valid example images root with single-library mode

test(usage_stats): use string literals for metadata payload to avoid conditional imports
2025-10-05 15:48:50 +08:00
pixelpaws
e21d5835ec Merge pull request #524 from willmiao/codex/add-async-tests-for-websocketmanager
Add websocket broadcast and usage stats tests
2025-10-05 15:19:40 +08:00
pixelpaws
f2f354e478 Merge pull request #528 from willmiao/codex/add-bulk-action-set-content-rating, fixes #428
Add bulk content rating update action
2025-10-05 15:19:19 +08:00
pixelpaws
b195d4569c test(usage-stats): allow metadata registry monkeypatch 2025-10-05 15:13:20 +08:00
pixelpaws
3b77fed72d feat(bulk): add bulk content rating action 2025-10-05 15:09:43 +08:00
pixelpaws
fc64e97f92 Merge pull request #525 from willmiao/codex/develop-tests-for-metadatasyncservice-and-modelscanner
Add coverage for metadata sync service and scanner reconciliation
2025-10-05 15:03:36 +08:00
pixelpaws
1da0434454 Merge pull request #526 from willmiao/codex/add-tests-for-utility-functions
test: add coverage for utility helpers
2025-10-05 15:03:20 +08:00
pixelpaws
cf2fe40612 Merge pull request #527 from willmiao/codex/add-unit-tests-for-metadata-components
Add metadata collector unit tests and fixtures
2025-10-05 15:03:04 +08:00
pixelpaws
8f46433ff7 Merge pull request #523 from willmiao/codex/add-tests-for-settingsmanager-migration
Add tests for settings migrations and service registry lazy loading
2025-10-05 15:02:44 +08:00
pixelpaws
f3be3ae269 Merge pull request #522 from willmiao/codex/add-tests-for-example-images-pipeline
test: add example images route and utility coverage
2025-10-05 15:02:07 +08:00
pixelpaws
cfec5447d3 test(metadata): add collector coverage 2025-10-05 14:44:17 +08:00
pixelpaws
2d36b461cf test(utils): add coverage for helper utilities 2025-10-05 14:43:21 +08:00
pixelpaws
5e23e4b13d test(metadata): cover sync service and scanner reconciliation 2025-10-05 14:42:13 +08:00
pixelpaws
badae2e8b3 test: cover websocket broadcasts and usage stats 2025-10-05 14:41:47 +08:00
pixelpaws
9e64531de6 test(settings): cover migrations and registry lazy loading 2025-10-05 14:41:24 +08:00
pixelpaws
fdec8d283c test(example-images): expand coverage for routes and utilities 2025-10-05 14:40:48 +08:00
Will Miao
9abedbf7cb fix(metadata-sync): improve error handling for deleted CivitAI models, fixes #497 2025-10-05 11:05:52 +08:00
pixelpaws
66004c1cdc Merge pull request #520 from willmiao/codex/adjust-example-images-download-to-use-library-name
fix: keep example image downloads in initial library
2025-10-05 10:08:26 +08:00
pixelpaws
5b564cd8a3 fix(example-images): pin downloads to start library 2025-10-05 09:10:25 +08:00
pixelpaws
2e79970e6e Merge pull request #519 from willmiao/codex/update-example-images-download-flow
fix: reuse migrated example image folders before download
2025-10-05 09:04:06 +08:00
pixelpaws
67c82ba6ea fix(example-images): reuse migrated folders during downloads 2025-10-05 08:37:11 +08:00
Will Miao
98425f37b8 fix(download-manager): improve handling of civitai payloads to avoid empty dictionaries 2025-10-05 07:29:11 +08:00
Will Miao
9d22dd3465 fix(model-library): update response structure to return model versions directly 2025-10-04 22:06:33 +08:00
pixelpaws
837138db49 Merge pull request #517 from willmiao/codex/determine-example_images_path-structure
feat: namespace example image storage by library
2025-10-04 20:42:15 +08:00
pixelpaws
d43d992362 feat(example-images): namespace storage by library 2025-10-04 20:29:49 +08:00
Will Miao
16b611cb7e Simplify settings file location and configuration 2025-10-04 18:42:53 +08:00
Will Miao
8dde2d5e0d feat(websocket-manager): implement caching for initialization progress and enhance broadcast functionality 2025-10-04 18:19:20 +08:00
Will Miao
22b0b2bd24 fix(model-card): correct query parameter handling in versioned preview URL 2025-10-04 17:32:16 +08:00
Will Miao
056f727bfd feat(model-scanner): enhance page type determination for model types 2025-10-04 17:08:02 +08:00
Will Miao
0aa6c53c1f feat(initialization): add support for embeddings page type and log progress updates 2025-10-04 14:11:27 +08:00
pixelpaws
d9b0660611 Merge pull request #516 from willmiao/codex/investigate-library-switching-functionality-issue
feat: serve dynamic preview assets after library switch
2025-10-04 10:51:52 +08:00
pixelpaws
d01666f4e2 feat(previews): serve dynamic library previews 2025-10-04 10:38:06 +08:00
Will Miao
51bee87cd0 fix(persistence): improve handling of lora_info attributes in recipe persistence 2025-10-04 09:52:53 +08:00
pixelpaws
3041b443e5 Merge pull request #515 from willmiao/codex/add-library-selection-in-settings-modal
Add tests for settings library switcher
2025-10-04 09:22:11 +08:00
pixelpaws
d95e6c939b test(settings): cover library switch workflow 2025-10-04 09:07:02 +08:00
pixelpaws
fd38c63b35 Merge pull request #514 from willmiao/codex/add-multi-library-endpoints-for-frontend
test: add coverage for settings library endpoints
2025-10-04 08:43:03 +08:00
pixelpaws
b69c24ae14 test(routes): cover settings library endpoints 2025-10-04 08:38:59 +08:00
pixelpaws
65a0c00e33 Merge pull request #513 from willmiao/codex/add-link-button-to-settings-modal-header
Adjust settings modal location shortcut styling
2025-10-04 07:57:43 +08:00
pixelpaws
b12a5ef133 style(settings): restyle settings location shortcut 2025-10-04 07:52:10 +08:00
pixelpaws
9e1b92c26e Merge pull request #512 from willmiao/codex/analyze-and-add-tests-for-misc-routes
test: improve misc routes coverage
2025-10-04 05:25:58 +08:00
pixelpaws
3922aec36e test(routes): extend misc routes coverage 2025-10-04 05:10:43 +08:00
pixelpaws
41cca8e56d Merge pull request #511 from willmiao/codex/refactor-misc_routes.py-and-add-tests
refactor: modularize misc route controller
2025-10-03 22:47:22 +08:00
pixelpaws
2d37a7341a fix(routes): await trained words extraction 2025-10-03 22:45:25 +08:00
pixelpaws
40e3c6134c refactor(routes): modularize misc route handling 2025-10-03 22:19:09 +08:00
pixelpaws
edddd47a1e Merge pull request #510 from willmiao/codex/update-default-library-folder-path-handling
fix: rename legacy default library to comfyui
2025-10-03 21:50:41 +08:00
pixelpaws
4ea6f38645 fix(config): rename legacy default library 2025-10-03 21:24:17 +08:00
Will Miao
40d998a026 fix(settings): use timezone-aware datetime for current timestamp
fix(tests): normalize stored paths in library upsert test
2025-10-03 20:59:47 +08:00
pixelpaws
3af8f151ac Merge pull request #509 from willmiao/codex/implement-features-from-multi-library-design
feat: add multi-library backend support
2025-10-03 20:37:59 +08:00
pixelpaws
e066fa6873 feat(settings): add multi-library backend support 2025-10-03 20:08:35 +08:00
Will Miao
6bd94269d4 refactor(model-scanner): remove unused model scanning methods 2025-10-03 18:58:02 +08:00
Will Miao
c90edec18a feat(multi-library): add design documentation for multi-library management in standalone mode 2025-10-03 18:34:52 +08:00
Will Miao
cbb302614c Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-03 15:23:30 +08:00
Will Miao
c54611a11b fix(metadata-service): change log level to debug for SQLite and fallback provider registration 2025-10-03 15:23:28 +08:00
pixelpaws
88f249649a Merge pull request #508 from willmiao/codex/sync-persistent-model-cache-metadata-updates
fix: sync persistent cache with metadata updates
2025-10-03 15:06:30 +08:00
pixelpaws
fe9fbdb93c fix(cache): sync persistent metadata updates 2025-10-03 14:57:44 +08:00
Will Miao
28bc966b76 feat(model-scanner): enhance cache loading with progress reporting and fallback to full scan 2025-10-03 14:31:08 +08:00
Will Miao
77bbf85b52 feat(persistent-cache): implement SQLite-based persistent model cache with loading and saving functionality 2025-10-03 11:00:51 +08:00
Will Miao
3b1990e97a feat(scanner): enhance model scanning with cache build result and progress reporting 2025-10-02 21:25:09 +08:00
Will Miao
375b5a49f3 fix(config): update standalone mode environment variable usage 2025-10-02 09:40:24 +08:00
pixelpaws
392c157cb5 Merge pull request #503 from willmiao/codex/add-hebrew-locale-to-tests
fix(i18n): add hebrew locale coverage
2025-09-30 22:07:52 +08:00
pixelpaws
6f5bf4b582 fix(i18n): add hebrew locale coverage 2025-09-30 22:03:44 +08:00
pixelpaws
2e3f48ebb7 Merge pull request #426 from start-life/main
Adding Hebrew language
2025-09-30 21:51:50 +08:00
Will Miao
e4a2c518bb fix(preset): update filePath retrieval method in removePreset function 2025-09-30 21:41:30 +08:00
pixelpaws
f19fb68b4c Merge pull request #501 from willmiao/codex/update-downloadmanager-to-handle-multiple-download-urls
feat: retry mirror downloads sequentially
2025-09-30 17:17:34 +08:00
pixelpaws
9121c12a2c feat(download): retry mirror urls sequentially 2025-09-30 17:14:59 +08:00
Will Miao
d0fe28cfe2 fix(recipe): validate modelVersionId before fetching hash from cache or Civitai 2025-09-28 09:18:59 +08:00
Will Miao
656e3e43be fix(imports): update import paths for ensure_settings_file to use relative imports 2025-09-28 08:40:09 +08:00
pixelpaws
c2c1772371 Merge pull request #496 from willmiao/codex/find-best-practices-for-settings-file-storage
feat(settings): persist settings in user config directory
2025-09-28 07:06:02 +08:00
pixelpaws
88d5caf642 feat(settings): migrate settings to user config dir 2025-09-27 22:22:15 +08:00
pixelpaws
1684978693 Merge pull request #491 from willmiao/codex/replace-spaces-in-embedding-paths
Fix embedding relative paths by replacing spaces
2025-09-26 09:06:56 +08:00
pixelpaws
8e4927600f fix(embeddings): replace spaces in relative paths 2025-09-26 09:02:46 +08:00
pixelpaws
4d72dc57e7 Merge pull request #490 from willmiao/codex/remove-comfy-field-from-images-metadata
fix: strip comfy metadata from civitai model images
2025-09-26 08:59:30 +08:00
pixelpaws
e7316b3389 fix(civitai): strip comfy metadata from images 2025-09-26 08:55:46 +08:00
start-life
e17b374606 Update zh-CN.json 2025-09-26 02:16:58 +03:00
Will Miao
141f83065f fix(locales): update language selection text in Chinese 2025-09-25 21:14:58 +08:00
pixelpaws
6381dbafc1 Merge pull request #488 from willmiao/codex/fix-bespoke-import_from-loader-issue
test: standardize backend package imports
2025-09-25 16:51:17 +08:00
pixelpaws
fc9db4510f test: fix duplicate pytest import 2025-09-25 16:44:43 +08:00
pixelpaws
66abf736c9 Merge pull request #487 from willmiao/codex/fix-lora-information-import-issue
fix: parse civitai image LoRAs from hash metadata
2025-09-25 16:00:48 +08:00
pixelpaws
af713470c1 fix(recipes): parse loras from civitai hashes 2025-09-25 15:59:44 +08:00
pixelpaws
93a51d2bcb Merge pull request #486 from willmiao/codex/enable-test-coverage-metrics-in-ci
ci: add pytest coverage reporting
2025-09-25 15:58:47 +08:00
pixelpaws
3f3e06de8a Fix pytest command in backend tests workflow 2025-09-25 15:57:59 +08:00
pixelpaws
7315aac9d8 ci: add pytest coverage reporting 2025-09-25 15:47:07 +08:00
pixelpaws
d933308a6f Merge pull request #485 from willmiao/codex/expand-vitest-coverage-for-frontend-components
test(frontend): extend coverage for comfyui widgets and helpers
2025-09-25 15:37:48 +08:00
pixelpaws
3baf93dcc5 test(frontend): extend coverage for comfyui widgets and helpers 2025-09-25 15:32:25 +08:00
pixelpaws
6ba14bd8fe Merge pull request #484 from willmiao/codex/fix-update-check-in-offline-mode
Fix offline update logging and respect update notification toggle
2025-09-25 14:54:28 +08:00
pixelpaws
7499570766 fix(updates): avoid network stack traces offline 2025-09-25 14:50:06 +08:00
start-life
003ee55a75 Merge branch 'main' into main 2025-09-25 09:28:30 +03:00
pixelpaws
b0cc42ef1f Merge pull request #483 from willmiao/codex/introduce-integration-and-contract-tests
test: add aiohttp integration coverage for ServiceRegistry
2025-09-25 14:23:53 +08:00
pixelpaws
23679ec3f5 chore(tests): clean integration route header 2025-09-25 14:17:45 +08:00
Will Miao
da52e5b9dd fix(settings): improve logic for auto-setting default root paths based on folder presence 2025-09-25 10:56:09 +08:00
Will Miao
c4e357793f Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-09-25 10:43:14 +08:00
Will Miao
6c3424029c fix(recipe image): optimize image saving and update PNG metadata handling, fixes #481 2025-09-25 10:43:10 +08:00
pixelpaws
dd9e6a5b69 Merge pull request #482 from willmiao/codex/add-pytest-modules-for-untested-services
Add backend service and route test coverage
2025-09-25 09:48:27 +08:00
pixelpaws
095320ef72 test(routes): tidy lora route test imports 2025-09-25 09:40:25 +08:00
pixelpaws
35f7674bcd Merge pull request #480 from willmiao/codex/modularize-i18n-tests-into-smaller-cases
test(i18n): modularize translation validation
2025-09-25 09:25:06 +08:00
pixelpaws
26b36c123d test(i18n): modularize translation validation 2025-09-25 09:21:08 +08:00
Will Miao
c85e694c1d docs: update repository guidelines for clarity and consistency 2025-09-25 08:36:15 +08:00
Will Miao
ec05282db6 fix(coverage): improve Vitest CLI path handling and error checking 2025-09-25 08:11:48 +08:00
pixelpaws
3d6f9b226f Merge pull request #479 from willmiao/codex/execute-phase-5-tasks-and-update-roadmap
Add frontend coverage workflow and reporting script
2025-09-25 07:00:04 +08:00
pixelpaws
eda6df4a5d chore(ci): add frontend coverage workflow 2025-09-24 23:22:32 +08:00
Will Miao
d504f89f6a Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-09-24 21:11:43 +08:00
Will Miao
14c468f2a2 feat(video): enhance video handling in model cards with lazy loading and autoplay settings, see #446 2025-09-24 21:11:36 +08:00
pixelpaws
2a99b0e46f Merge pull request #478 from willmiao/codex/execute-phase-4-tasks-from-roadmap
Add interaction-level frontend regression tests
2025-09-24 20:38:51 +08:00
pixelpaws
ae8914f5c8 test(frontend): add interaction regression suites 2025-09-24 20:33:41 +08:00
pixelpaws
0c9f8971ce Merge pull request #477 from willmiao/codex/execute-phase-3-tasks-and-update-roadmap
test: add embeddings and recipes manager suites
2025-09-24 20:18:37 +08:00
pixelpaws
d7a75ea4e5 test(frontend): cover embeddings and recipes managers 2025-09-24 20:15:38 +08:00
pixelpaws
3ad8d8b17c Merge pull request #476 from willmiao/codex/plan-next-tasks-based-on-roadmap-juork7
test(frontend): cover filtering flows for lora and checkpoints
2025-09-24 17:54:50 +08:00
pixelpaws
39225dc204 test(frontend): add filtering coverage for model pages 2025-09-24 17:50:04 +08:00
pixelpaws
4fb69f7d89 Merge pull request #475 from willmiao/codex/plan-next-tasks-based-on-roadmap-gpw3fe
Add checkpoints page smoke tests
2025-09-24 17:22:22 +08:00
pixelpaws
0890c6ad24 test(frontend): add checkpoints manager smoke tests 2025-09-24 17:18:20 +08:00
pixelpaws
dd81809589 Merge pull request #474 from willmiao/codex/plan-next-steps-for-roadmap-tasks
test(frontend): add loras page manager suite
2025-09-24 17:09:38 +08:00
pixelpaws
f0672beb46 test(frontend): add loras page manager suite 2025-09-24 16:22:17 +08:00
pixelpaws
cc5301e710 Merge pull request #473 from willmiao/codex/plan-next-tasks-based-on-roadmap
fix(frontend): validate AppCore initialization wiring
2025-09-24 16:15:05 +08:00
pixelpaws
9d5ec43c4e fix(frontend): correct AppCore example images initialization 2025-09-24 16:10:27 +08:00
pixelpaws
6d41211b07 Merge pull request #472 from willmiao/codex/plan-and-update-frontend-testing-roadmap
Add AppCore page orchestration tests
2025-09-24 15:56:17 +08:00
pixelpaws
d58b61eed5 test(frontend): cover appcore page features 2025-09-24 15:55:50 +08:00
pixelpaws
4b53d98bfc Merge pull request #471 from willmiao/codex/organize-frontend-tests-into-new-directories
test(frontend): document dom fixture workflow
2025-09-24 15:44:23 +08:00
pixelpaws
f51f354e48 test(frontend): add dom fixture helpers 2025-09-24 15:39:52 +08:00
Will Miao
59d027181d refactor: remove obsolete JSON files and add new version metadata for LORA model 2025-09-24 10:56:06 +08:00
Will Miao
0d0988c090 feat: add functionality to attach model files to version data in SQLiteModelMetadataProvider 2025-09-24 10:55:55 +08:00
Will Miao
dc2de50924 bump: update version to 0.9.5 in pyproject.toml 2025-09-24 09:16:50 +08:00
Will Miao
12c88835f2 refactor: enhance model version retrieval logic in CivitaiClient, fixes #460 2025-09-24 09:16:02 +08:00
Will Miao
6f4453aaf3 refactor: remove storage migration logic and associated tests 2025-09-24 06:04:08 +08:00
Will Miao
4b4b8fe3c1 refactor: remove unused ModelRouteUtils class and its methods 2025-09-24 05:41:30 +08:00
pixelpaws
49e7c2e9f5 Merge pull request #470 from willmiao/codex/add-tests-for-storagehelpers-and-appcore
test: add vitest coverage for storage helpers and core
2025-09-24 05:21:02 +08:00
pixelpaws
4653c273e3 test(frontend): add storage and core initialization specs 2025-09-24 05:20:39 +08:00
Will Miao
ae145de2f2 feat(tests): add frontend automated testing setup with Vitest and jsdom 2025-09-23 23:05:55 +08:00
Will Miao
dde7cf71c6 fix(locales): translate global context menu entries for downloading example images 2025-09-23 22:24:57 +08:00
pixelpaws
219cd242db Merge pull request #467 from willmiao/codex/migrate-frontend-settings-to-backend
feat(settings): centralize settings loading and snake_case keys
2025-09-23 22:16:10 +08:00
pixelpaws
e5b712c082 fix(i18n): sync language with state settings 2025-09-23 22:00:31 +08:00
pixelpaws
4d2c60d59b fix(settings): persist language preference 2025-09-23 22:00:20 +08:00
pixelpaws
1d2c1b114b Merge pull request #469 from willmiao/codex/add-download-example-images-to-global-context-menu
feat: add download example images to global context menu
2025-09-23 20:58:35 +08:00
pixelpaws
2bde936d05 Merge pull request #468 from willmiao/codex/migrate-i18n-tests-to-tests-framework
Migrate i18n test script into pytest suite
2025-09-23 20:58:14 +08:00
pixelpaws
cd3e32bf4b feat(context-menu): add example image download entry 2025-09-23 20:49:44 +08:00
pixelpaws
454536d631 test(i18n): migrate localization tests into pytest suite 2025-09-23 20:47:33 +08:00
Will Miao
656f1755fd feat: add cleanup example image folders functionality and UI integration 2025-09-23 20:35:35 +08:00
pixelpaws
8aa76ce5c1 feat(settings): centralize frontend settings on backend 2025-09-23 20:28:32 +08:00
Will Miao
49fa37f00d Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-09-23 19:31:19 +08:00
pixelpaws
9f83548cf3 Merge pull request #466 from willmiao/refactor
Refactor
2025-09-23 19:29:54 +08:00
pixelpaws
6054d95e85 Update py/services/model_query.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-23 19:25:12 +08:00
pixelpaws
8c9bb35824 Update tests/services/test_base_model_service.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-09-23 19:25:02 +08:00
Will Miao
3eacf9558a docs: remove outdated developer notes and add example image route architecture documentation 2025-09-23 15:39:56 +08:00
pixelpaws
fee37172b4 Merge pull request #465 from willmiao/codex/add-async-tests-for-concurrent-behavior
Add async tests for example image download concurrency
2025-09-23 15:00:31 +08:00
pixelpaws
e128c80eb1 test(services): add async example image download tests 2025-09-23 14:58:35 +08:00
pixelpaws
5cc735ed57 Merge pull request #464 from willmiao/codex/refactor-websocket-integration-for-downloading
refactor: align example image downloads with websocket manager
2025-09-23 14:43:37 +08:00
pixelpaws
43fcce6361 refactor(example-images): inject websocket manager 2025-09-23 14:40:43 +08:00
pixelpaws
49b7126278 Merge pull request #463 from willmiao/codex/refactor-downloadmanager-to-instance-based
refactor: convert example image download manager to service instance
2025-09-23 13:08:08 +08:00
pixelpaws
679cfb5c69 refactor(example-images): encapsulate download manager state 2025-09-23 13:07:11 +08:00
pixelpaws
50616bc680 Merge pull request #462 from willmiao/codex/wrap-long-running-flows-in-use-cases
feat(example-images): add use case orchestration
2025-09-23 11:55:06 +08:00
pixelpaws
aaad270822 feat(example-images): add use case orchestration 2025-09-23 11:47:12 +08:00
pixelpaws
bd10280736 Merge pull request #461 from willmiao/codex/refactor-example-images-routes
Add regression tests for example images routes
2025-09-23 11:31:22 +08:00
Will Miao
d477050239 feat: add global context menu with actions and integration 2025-09-23 11:19:36 +08:00
pixelpaws
85f79cd8d1 refactor(routes): introduce example images controller 2025-09-23 11:12:08 +08:00
pixelpaws
613cd81152 refactor(routes): add registrar for example images 2025-09-23 11:12:05 +08:00
pixelpaws
e0aba6c49a test(example-images): add route regression harness 2025-09-23 10:41:56 +08:00
pixelpaws
d78bcf2494 Merge pull request #459 from willmiao/codex/complete-refactor-with-tests-and-documentation
Add recipe route integration tests and update architecture docs
2025-09-22 14:19:41 +08:00
pixelpaws
f7cffd2eba test(recipes): add route smoke tests and docs 2025-09-22 14:15:24 +08:00
pixelpaws
0d0b91aa80 Merge pull request #458 from willmiao/codex/expose-first-class-operations-on-recipescanner
feat: expose recipe scanner mutation APIs
2025-09-22 13:54:49 +08:00
pixelpaws
42872e6d2d feat(recipes): expose recipe scanner mutation apis 2025-09-22 13:45:40 +08:00
Will Miao
b91f06405d feat: support clip strength in LoRA usage tips, fixes #401 2025-09-22 13:42:36 +08:00
pixelpaws
dac4c688d6 Merge pull request #456 from willmiao/codex/refactor-http-handlers-with-recipe-services
Refactor recipe handlers to use dedicated services
2025-09-22 13:30:05 +08:00
pixelpaws
097a68ad18 refactor(recipes): introduce dedicated services for handlers 2025-09-22 13:25:21 +08:00
pixelpaws
4a98710db0 Merge pull request #455 from willmiao/codex/refactor-reciperoutes-into-handler-objects
refactor: split recipe routes into dedicated handlers
2025-09-22 13:02:39 +08:00
pixelpaws
d033a374dd refactor(routes): split recipe handlers into dedicated classes 2025-09-22 12:57:37 +08:00
pixelpaws
6aa23fe36a Merge pull request #454 from willmiao/codex/migrate-recipe-http-layer-architecture
Refactor recipe routes to use registrar scaffolding
2025-09-22 12:42:37 +08:00
pixelpaws
3220cfb79c test(recipe-routes): add scaffolding baseline 2025-09-22 12:41:37 +08:00
pixelpaws
b92e7aa446 chore(routes): dedupe os import 2025-09-22 12:15:12 +08:00
Will Miao
c3b9c73541 refactor: remove ModelRouteUtils usage and implement filtering directly in services 2025-09-22 09:09:40 +08:00
pixelpaws
81c6672880 Merge pull request #453 from willmiao/codex/evaluate-need-for-further-refactoring
refactor: migrate model lifecycle handlers to dedicated service
2025-09-22 08:37:40 +08:00
pixelpaws
08baf884d3 refactor(routes): migrate lifecycle mutations to service 2025-09-22 08:28:30 +08:00
Will Miao
1c4096f3d5 test(routes): add tests for service readiness and error handling in download model 2025-09-22 06:28:30 +08:00
Will Miao
66a3f3f59a refactor(tests): enhance async test handling in pytest_pyfunc_call 2025-09-22 05:37:24 +08:00
pixelpaws
624df1328b Merge pull request #452 from willmiao/codex/create-application-level-use-case-services
feat(routes): extract orchestration use cases
2025-09-22 05:27:19 +08:00
pixelpaws
c063854b51 feat(routes): extract orchestration use cases 2025-09-22 05:25:27 +08:00
Will Miao
8cf99dd928 refactor(tests): remove deprecated test runner script 2025-09-21 23:39:21 +08:00
pixelpaws
c07e885725 Merge pull request #451 from willmiao/codex/refactor-routes_common.py-into-services
Refactor model route utilities into dedicated services
2025-09-21 23:38:38 +08:00
pixelpaws
21772feadd refactor(routes): extract route utilities into services 2025-09-21 23:34:46 +08:00
Will Miao
2d00cfdd31 refactor: enhance BaseModelService initialization and improve filtering logic 2025-09-21 23:13:30 +08:00
pixelpaws
49e03d658b Merge pull request #450 from willmiao/codex/refactor-basemodelroutes-for-better-separation
refactor(routes): extract registrar and handlers
2025-09-21 22:46:13 +08:00
Will Miao
fec85bcc08 refactor: unify standalone mode check using environment variable 2025-09-21 22:45:11 +08:00
pixelpaws
0e93a6bcb0 refactor(routes): extract registrar and handlers 2025-09-21 20:52:08 +08:00
pixelpaws
7e20f738fb Merge pull request #449 from willmiao/codex/document-and-map-basemodelroutes-contracts
docs(routes): map base model dependencies and contracts
2025-09-21 20:40:44 +08:00
pixelpaws
24090e6077 docs(routes): map base model dependencies and contracts 2025-09-21 20:34:45 +08:00
Will Miao
1022b07f64 feat: enhance model metadata provider with import error handling and mock setup for tests 2025-09-21 19:57:49 +08:00
pixelpaws
4faf912c6f Merge pull request #448 from willmiao/codex/create-documentation-and-tests-for-model-routes
test(routes): add base model route smoke coverage
2025-09-21 17:19:52 +08:00
pixelpaws
56e4b24b07 test(routes): clean smoke test module 2025-09-21 17:15:24 +08:00
Will Miao
12295d2fdc feat(docs): add comprehensive repository guidelines and project structure documentation 2025-09-21 16:40:07 +08:00
Will Miao
6261f7d18d Update LM extension wiki 2025-09-20 23:21:10 +08:00
Will Miao
9e1a2e3bb7 chore(pyproject): bump version to 0.9.4 2025-09-20 22:03:29 +08:00
Will Miao
40cbb2155c refactor(baseModelApi): comment out failure message handling in bulk metadata refresh 2025-09-20 21:43:00 +08:00
Will Miao
a8d7070832 feat(civitai): enhance metadata fetching with error handling and cache validation 2025-09-20 21:35:34 +08:00
Will Miao
ab7266f3a4 fix(download_manager): streamline output directory retrieval by using settings directly, fixes #443 2025-09-20 08:12:14 +08:00
Will Miao
3053b13fcb feat(metadata): enhance model processing with CivitAI metadata checks and new fields for archive DB status 2025-09-19 23:22:47 +08:00
Will Miao
f3544b3471 refactor(settings): replace getStorageItem with state.global.settings for default root retrieval 2025-09-19 22:57:05 +08:00
Will Miao
1610048974 refactor(metadata): update model fetching methods to return error messages alongside results 2025-09-19 16:36:34 +08:00
Will Miao
fc6f1bf95b fix(lora_loader): remove unnecessary string stripping from lora names in loaders, fixes #441 2025-09-19 11:17:19 +08:00
Will Miao
67b274c1b2 feat(settings): add 'show_only_sfw' setting to manage content visibility 2025-09-18 21:55:21 +08:00
Will Miao
fb0d6b5641 feat(docs): add comprehensive documentation for LoRA Manager Civitai Extension, including features, installation, privacy, and usage guidelines 2025-09-18 19:33:47 +08:00
Will Miao
d30fbeb286 feat(example_images): add dedicated folder check and update settings handling for example images path, see #431 2025-09-18 19:22:29 +08:00
Will Miao
46e430ebbb fix(utils): update API endpoint for fetching connected trigger words 2025-09-18 15:45:57 +08:00
Will Miao
bc4cd45fcb fix(lora_manager): rename invalid hash folder removal to orphaned folders and update logging 2025-09-18 15:09:32 +08:00
Will Miao
bdc86ddf15 Refactor API endpoints to use '/api/lm/' prefix
- Updated all relevant routes in `stats_routes.py` and `update_routes.py` to include the new '/api/lm/' prefix for consistency.
- Modified API endpoint configurations in `apiConfig.js` to reflect the new structure, ensuring all CRUD and bulk operations are correctly routed.
- Adjusted fetch calls in various components and managers to utilize the updated API paths, including recipe, model, and example image operations.
- Ensured all instances of the old API paths were replaced with the new '/api/lm/' prefix across the codebase for uniformity and to prevent broken links.
2025-09-18 14:50:40 +08:00
Will Miao
ded17c1479 feat(routes): add model versions status endpoint and enhance metadata retrieval 2025-09-17 22:06:59 +08:00
Will Miao
933e2fc01d feat(routes): integrate CivitAI model version retrieval for various model types 2025-09-17 15:47:30 +08:00
Will Miao
1cddeee264 style(autocomplete): remove font styles from dropdown for consistency 2025-09-17 11:04:51 +08:00
Will Miao
183c000080 Refactor ComfyUI: Remove legacy tags widget and related dynamic imports
- Deleted the legacy tags widget implementation from legacy_tags_widget.js.
- Updated trigger_word_toggle.js to directly import the new tags widget.
- Removed unused dynamic import functions and version checks from utils.js.
- Cleaned up lora_loader.js and lora_stacker.js by removing redundant node registration code.
2025-09-16 21:48:20 +08:00
Will Miao
adf7b6d4b2 chore(version): bump version to 0.9.3 in pyproject.toml 2025-09-16 18:55:59 +08:00
Will Miao
0566d50346 feat(middleware): add .mp4 to image extensions for cache control 2025-09-16 15:39:12 +08:00
Will Miao
4275dc3003 refactor(middleware): reorganize cache middleware into py directory and update import paths 2025-09-16 15:16:53 +08:00
Will Miao
30956aeefc feat(middleware): add cache control middleware to manage response caching for image files 2025-09-16 15:05:31 +08:00
Will Miao
64e1dd3dd6 chore(release): update release notes for v0.9.3 with new features and bug fixes 2025-09-15 21:35:24 +08:00
Will Miao
0dc4b6f728 refactor(showcase): improve custom image identification logic in renderMediaItem and findLocalFile functions 2025-09-15 20:18:39 +08:00
Will Miao
86074c87d7 refactor(downloader): update download_to_memory calls to include response headers 2025-09-15 19:24:09 +08:00
Will Miao
6f9245df01 refactor(downloader): enhance download_to_memory to return response headers and improve error handling 2025-09-15 18:53:04 +08:00
Will Miao
4540e47055 refactor(baseModelApi): update example images path retrieval to use state settings 2025-09-15 18:07:22 +08:00
Will Miao
4bb8981e78 refactor(routes): update API endpoints for settings to use '/api/lm/settings', see #435 2025-09-15 16:22:59 +08:00
Will Miao
c49be91aa0 refactor(update_routes): exclude civitai folder from plugin update process 2025-09-15 16:04:20 +08:00
Will Miao
2b847039d4 refactor(settings-modal): adjust font size for path template preview 2025-09-15 15:38:01 +08:00
Will Miao
1147725fd7 feat(settings): add base model, author, and first tag option to download path templates
refactor(constants): reorder preset tag suggestions for consistency
2025-09-15 12:23:46 +08:00
Will Miao
26891e12a4 refactor(ExampleImagesManager): enhance path input handling with Enter key and blur events 2025-09-15 11:34:39 +08:00
Will Miao
2f7e44a76f refactor(settings): Update synchronization logic 2025-09-15 10:30:06 +08:00
Will Miao
9366d3d2d0 feat: add API endpoint for fetching application settings and update frontend settings management 2025-09-14 22:57:17 +08:00
Will Miao
6b606a5cc8 refactor(CivArchiveModelMetadataProvider): remove session management and use downloader for HTTP requests 2025-09-13 20:04:41 +08:00
Will Miao
e5339c178a fix: increase max-height for expanded sidebar tree children to improve visibility, fixes #403 2025-09-13 16:36:01 +08:00
Will Miao
1a76f74482 refactor(BaseModelRoutes): temporary comment out model description and creator checks 2025-09-13 13:07:25 +08:00
Will Miao
13f13eb095 fix: update preview versions keys for consistency in state management, fixes #406 2025-09-13 09:20:55 +08:00
Will Miao
125fdecd61 fix: handle missing download URL for primary file in metadata 2025-09-13 09:03:34 +08:00
Will Miao
d05076d258 feat: add CivArchive metadata provider and support for optional source parameter in downloads 2025-09-12 21:13:15 +08:00
Will Miao
00b77581fc refactor(Downloader): change logger info statements to debug level for proxy usage 2025-09-12 15:20:34 +08:00
Will Miao
897787d17c refactor(AutoComplete): simplify search term extraction and insertion logic 2025-09-12 14:35:25 +08:00
Will Miao
d5a280cf2b fix: increase maxItems for autocomplete to improve user experience 2025-09-12 14:01:52 +08:00
Will Miao
a0c2d9b5ad refactor: change logger info statements to debug level for improved logging granularity 2025-09-12 11:48:59 +08:00
Will Miao
e713bd1ca2 feat: add app-level proxy settings with UI integration and session management, fixes #382 2025-09-12 11:22:45 +08:00
start-life
a3c28c1003 Update zh-TW.json 2025-09-12 03:33:34 +03:00
start-life
f4b7c9a138 Update zh-CN.json 2025-09-12 03:33:09 +03:00
start-life
6b860b5f29 Update ru.json 2025-09-12 03:32:15 +03:00
start-life
37dfcd6abd Update ko.json 2025-09-12 03:31:57 +03:00
start-life
bc2fca3a4f Update ja.json 2025-09-12 03:31:35 +03:00
start-life
f8ef159656 Update fr.json 2025-09-12 03:31:12 +03:00
start-life
b2b8a9d37e Update es.json 2025-09-12 03:30:56 +03:00
start-life
15ae4031b7 Update de.json 2025-09-12 03:30:39 +03:00
start-life
688976ce3b Update en.json 2025-09-12 03:30:14 +03:00
start-life
a548af01dc Update settings_modal.html
Adding Hebrew language
2025-09-12 03:28:45 +03:00
start-life
0dd52eceb3 Update index.js
Adding Hebrew language
2025-09-12 03:26:08 +03:00
start-life
b8c6cf4ac1 Add files via upload
Adding Hebrew language
2025-09-12 03:20:38 +03:00
Will Miao
beb8ff1dd1 refactor(ModelFileService): enhance auto-organize logic to track source directories for cleanup, see #407 2025-09-11 23:02:30 +08:00
Will Miao
6a8f0867d9 refactor: migrate auto_organize_models logic to service layer with dependency injection 2025-09-11 22:37:46 +08:00
Will Miao
51ad1c9a33 refactor(MetadataProcessor): comment out guidance parameter in generation params, fixes #425 2025-09-11 16:55:41 +08:00
pixelpaws
34872eb612 Merge pull request #411 from gaoqi125/main
__init__.py register WanVideoLoraSelectFromText
2025-09-11 16:31:57 +08:00
Will Miao
8b4e3128ff feat: add functionality to open file location from model modal and update translations, fixes #405 2025-09-11 15:54:32 +08:00
Will Miao
c66cbc800b refactor: remove clear cache functionality and associated modal from settings manager 2025-09-11 15:21:06 +08:00
Will Miao
21941521a0 fix(sidebar): increase max-height for expanded sidebar tree children, see #403 2025-09-11 12:53:58 +08:00
Will Miao
0d33884052 refactor(ModelScanner): remove unused metadata fetching logic from model processing 2025-09-11 12:33:34 +08:00
Will Miao
415df49377 fix(SearchManager): update search options handling to modify relevant fields instead of replacing the entire object, see #415 2025-09-11 12:30:01 +08:00
Will Miao
f5f45002c7 fix(routes): skip tag check in model validation to allow empty tags 2025-09-11 12:10:00 +08:00
Will Miao
1edf7126bb fix(routes): add support for metadata archive settings in model processing 2025-09-11 09:31:58 +08:00
Will Miao
a1a55a1002 feat(node_extractors): add PCTextEncode extractor to NODE_EXTRACTORS registry, fixes #424 2025-09-11 06:45:22 +08:00
Will Miao
45f5cb46bd fix(utils): update base model retrieval to use model_data for consistency, fixes #423 2025-09-10 23:44:42 +08:00
Will Miao
1b5e608a27 fix(routes): enhance model processing to include checks for missing tags, description, and creator 2025-09-10 23:30:08 +08:00
Will Miao
a7df8ae15c feat(civitai_client): enrich model version info with additional metadata 2025-09-10 23:28:19 +08:00
Will Miao
47ce0d0fe2 fix(model_scanner): comment out fetch missing metadata call to prevent potential issues 2025-09-10 22:08:44 +08:00
pixelpaws
b220e288d0 Merge pull request #422 from willmiao/ca
Civitai metadata archive db
2025-09-10 20:35:16 +08:00
Will Miao
1fc8b45b68 feat(dependencies): add GitPython and aiosqlite to project dependencies 2025-09-10 20:33:45 +08:00
Will Miao
62f06302f0 refactor(routes): replace ModelMetadataProviderManager with get_default_metadata_provider in checkpoint, embedding, and lora routes 2025-09-10 20:29:26 +08:00
Will Miao
3e5cb223f3 refactor(metadata): remove outdated metadata provider summary documentation 2025-09-10 20:09:05 +08:00
Will Miao
4ee5b7481c fix(downloader): set socket read timeout to 5 minutes for improved stability during large downloads 2025-09-10 18:49:35 +08:00
gaoqi125
e104b78c01 Merge branch 'willmiao:main' into main 2025-09-10 18:02:51 +08:00
Will Miao
ba1ac58721 feat(metadata): trigger metadata provider update when enabling metadata archive database 2025-09-10 16:18:04 +08:00
Will Miao
a4fbeb6295 feat(metadata): update metadata archive management and remove provider priority settings 2025-09-10 15:55:29 +08:00
Will Miao
68f8871403 feat(metadata): add source tracking for SQLite metadata and implement Civitai API metadata validation 2025-09-10 11:20:58 +08:00
Will Miao
6fd74952b7 Refactor metadata handling to use unified provider system
- Replaced direct usage of Civitai client with a fallback metadata provider across all recipe parsers.
- Updated metadata service to improve initialization and error handling.
- Enhanced download manager to utilize a downloader service for file operations.
- Improved recipe scanner to fetch model information through the new metadata provider.
- Updated utility functions to streamline image downloading and processing.
- Added comprehensive logging and error handling for better debugging and reliability.
- Introduced `get_default_metadata_provider()` for simplified access to the default provider.
- Ensured backward compatibility with existing APIs and workflows.
2025-09-09 20:57:45 +08:00
Will Miao
1ea468cfc4 feat(metadata): enhance metadata archive management with download progress and status updates 2025-09-09 15:24:28 +08:00
Will Miao
14721c265f Refactor download logic to use unified downloader service
- Introduced a new `Downloader` class to centralize HTTP/HTTPS download management.
- Replaced direct `aiohttp` session handling with the unified downloader in `MetadataArchiveManager`, `DownloadManager`, and `ExampleImagesProcessor`.
- Added support for resumable downloads, progress tracking, and error handling in the new downloader.
- Updated methods to utilize the downloader's capabilities for downloading files and images, improving code maintainability and readability.
2025-09-09 10:34:14 +08:00
Will Miao
821827a375 feat(metadata): implement metadata archive management and update settings for metadata providers 2025-09-08 22:41:17 +08:00
Will Miao
9ba3e2c204 feat(metadata): implement metadata providers and initialize metadata service
- Added ModelMetadataProvider and CivitaiModelMetadataProvider for handling model metadata.
- Introduced SQLiteModelMetadataProvider for SQLite database integration.
- Created metadata_service.py to initialize and configure metadata providers.
- Updated CivitaiClient to register as a metadata provider.
- Refactored download_manager to use the new download_file method.
- Added SQL schema for models, model_versions, and model_files.
- Updated requirements.txt to include aiosqlite.
2025-09-08 22:41:17 +08:00
Will Miao
d287883671 refactor(civitai): remove legacy get_model_description and _get_hash_from_civitai methods 2025-09-08 22:41:17 +08:00
Will Miao
ead34818db feat(utils): implement forwardMiddleMouseToCanvas function and integrate it into JSON, LoRA, and Tags widgets, see #417 2025-09-08 21:49:16 +08:00
Will Miao
a060010b96 feat(loras_widget): add delayed preview tooltip for LoRA names, see #416 2025-09-08 21:03:22 +08:00
gaoqi125
76a92ac847 Update wanvideo_lora_select_from_text.py 2025-09-07 23:21:33 +08:00
gaoqi125
74bc490383 Update __init__.py 2025-09-07 19:51:19 +08:00
Will Miao
510d476323 feat(civitai): enhance LoRA matching by extracting hashes from metadata 2025-09-07 10:05:30 +08:00
Will Miao
1e7257fd53 fix(download): temporarily disable delay to speed up downloads 2025-09-06 18:47:18 +08:00
Will Miao
4ff1f51b1c fix(docs): update portable package download link to version 0.9.2 2025-09-06 18:02:57 +08:00
Will Miao
74507cef05 feat(settings): add validation for settings.json to ensure required configuration is present
fix(usage_stats): handle initialization errors for usage statistics when no valid paths are configured, fixes #375
2025-09-06 17:39:51 +08:00
Will Miao
c23ab04d90 chore(release): update version to 0.9.2 and add release notes for bulk auto-organization feature 2025-09-06 14:38:00 +08:00
Will Miao
d50dde6cf6 refactor(i18n): remove legacy migration summary and transition to JSON format 2025-09-06 10:07:43 +08:00
Will Miao
fcb1fb39be feat(controls): add toggleBulkMode functionality for Checkpoints and Embeddings pages 2025-09-06 08:15:18 +08:00
Will Miao
b0ef74f802 feat(LoraManager): add example images cleanup functionality to remove invalid or empty folders, see #402 2025-09-06 07:59:33 +08:00
Will Miao
f332aef41d fix(BulkManager): prevent initialization on recipes page to avoid unnecessary processing 2025-09-05 22:45:23 +08:00
Will Miao
1f91a3da8e fix(BulkManager): streamline cleanupBulkBaseModelModal to clear base model select options 2025-09-05 21:00:54 +08:00
Will Miao
16840c321d feat(api): enhance fetchModelDescription to improve error handling and response parsing 2025-09-05 20:57:36 +08:00
Will Miao
c109e392ad feat(auto-organize): add auto-organize functionality for selected models and update context menu 2025-09-05 20:51:30 +08:00
pixelpaws
5e69671366 Merge pull request #398 from gaoqi125/gaoqi125-patch-1
Create wanvideo_lora_select_from_text.py
2025-09-05 19:55:40 +08:00
Will Miao
52d23d9b75 feat(constants): update model tags to include 'realistic', 'anime', 'toon', and 'furry' 2025-09-05 19:53:29 +08:00
Will Miao
4c4e6d7a7b feat(release-notes): update version to 0.9.1 and enhance bulk operations documentation 2025-09-05 18:15:08 +08:00
Will Miao
03b6e78705 feat(locales): add bulk base model functionality in multiple languages and update toast messages 2025-09-05 18:00:21 +08:00
pixelpaws
24c01141d7 Merge pull request #400 from willmiao/bulk-menu
Bulk menu
2025-09-05 17:44:40 +08:00
Will Miao
6dc2811af4 feat(bulk-modal): refactor bulk base model modal for improved UI and functionality, fixes 352 2025-09-05 17:36:54 +08:00
Will Miao
e6425dce32 feat(bulk-manager): enhance bulk mode handling by skipping actions when a modal is open 2025-09-05 17:07:57 +08:00
Will Miao
95e2ff5f1e Implement centralized event management system with priority handling and state tracking
- Enhanced EventManager class to support priority-based event handling, conditional execution, and automatic cleanup.
- Integrated event management into BulkManager for global keyboard shortcuts and marquee selection events.
- Migrated mouse tracking and node selector events to UIHelpers for better coordination.
- Established global event handlers for context menu interactions and modal state management.
- Added comprehensive documentation for event management implementation and usage.
- Implemented initialization logic for event management, including error handling and cleanup on page unload.
2025-09-05 16:56:26 +08:00
Will Miao
92ac487128 feat(bulk-base-model): implement bulk base model setting functionality with UI and context menu integration 2025-09-05 14:07:03 +08:00
Will Miao
3250fa89cb feat(selection): implement marquee selection for bulk operations 2025-09-05 11:24:48 +08:00
Will Miao
7475de366b feat(context-menu): enhance bulk workflow options with append and replace actions 2025-09-05 11:24:48 +08:00
Will Miao
affb507b37 feat(sync): enhance translation key synchronization to remove obsolete keys 2025-09-05 11:24:48 +08:00
pixelpaws
3320b80150 Merge pull request #399 from willmiao/bulk-menu
Bulk context menu
2025-09-05 09:31:48 +08:00
Will Miao
fb2b69b787 feat(tags): refactor preset tags to constants for better maintainability 2025-09-05 09:27:45 +08:00
Will Miao
29a05f6533 Move test_i18n.py to scripts folder 2025-09-05 08:48:20 +08:00
Will Miao
9fa3fac973 feat(locales): add bulk tag management translations for multiple languages 2025-09-05 08:43:01 +08:00
Will Miao
904b0d104a feat(sync): add translation key synchronization script for locale management 2025-09-05 08:35:20 +08:00
Will Miao
1d31dae110 feat(tags): implement bulk tag addition and replacement functionality 2025-09-05 07:18:24 +08:00
Will Miao
476ecb7423 fix(banner): ensure href attribute defaults to '#' for actions without a URL 2025-09-04 22:09:15 +08:00
Will Miao
4eb67cf6da feat(bulk-tags): add bulk tag management modal and context menu integration 2025-09-04 22:08:55 +08:00
Will Miao
a5a9f7ed83 fix(banner): ensure href attribute defaults to '#' for actions without a URL 2025-09-04 22:07:07 +08:00
Will Miao
c0b029e228 feat(context-menu): refactor context menu initialization and coordination for improved bulk operations 2025-09-04 16:34:05 +08:00
Will Miao
9bebcc9a4b feat(bulk): implement bulk context menu for model operations and remove bulk operations panel 2025-09-04 15:24:54 +08:00
Will Miao
ac7d23011c chore(release): update version to 0.9.0 and add release notes for UI overhaul and new features 2025-09-04 00:04:25 +08:00
pixelpaws
491e09b7b5 Merge pull request #395 from willmiao/ot
Onboarding Tutorial
2025-09-03 23:25:31 +08:00
Will Miao
192bc237bf fix(onboarding): update language selection button text and remove skip option from translations 2025-09-03 23:04:06 +08:00
Will Miao
f041f4a114 feat(onboarding): prevent onboarding from starting if version-mismatch banner is visible 2025-09-03 22:48:29 +08:00
Will Miao
2546580377 fix(localization): update French translations for "recipe" to ensure consistency in terminology 2025-09-03 22:23:35 +08:00
Will Miao
8fbf2ab56d feat(onboarding): add multilingual support for onboarding steps and language selection 2025-09-03 22:17:48 +08:00
Will Miao
ea727aad2e feat(onboarding): enhance target highlighting with mask and pulsing effect 2025-09-03 21:44:23 +08:00
Will Miao
5520aecbba fix(onboarding): adjust language selection logic to skip if already set and update prompt text 2025-09-03 19:22:53 +08:00
Will Miao
6b738a4769 fix(onboarding): update language handling and selection logic in onboarding process 2025-09-03 19:15:55 +08:00
Will Miao
903a8050b3 Add SVG flags for France, Hong Kong, Japan, South Korea, Russia, and the United States
- Added France flag (fr.svg) with three vertical stripes: blue, white, and red.
- Added Hong Kong flag (hk.svg) featuring a red background with a white flower emblem.
- Added Japan flag (jp.svg) with a white field and a red circle in the center.
- Added South Korea flag (kr.svg) showcasing a white background with a central yin-yang symbol and four black trigrams.
- Added Russia flag (ru.svg) with three horizontal stripes: white, blue, and red.
- Added United States flag (us.svg) with red and white stripes and a blue canton featuring stars.
2025-09-03 18:19:34 +08:00
Will Miao
31b032429d fix(sidebar): change default pinned state to true for sidebar restoration 2025-09-03 15:46:33 +08:00
Will Miao
2bcf341f04 feat(onboarding): implement onboarding tutorial with language selection and step guidance 2025-09-03 15:42:36 +08:00
Will Miao
ca6f45b359 fix(download-manager): temporarily disable delay to speed up downloads 2025-09-02 22:36:36 +08:00
Will Miao
2a67cec16b fix(sidebar): update tree selection logic and improve breadcrumb and header state handling 2025-09-02 18:19:01 +08:00
Will Miao
1800afe31b feat(sidebar): implement display mode toggle and update sidebar actions for improved navigation. See #389 2025-09-02 17:42:21 +08:00
gaoqi125
8c6311355d Create wanvideo_lora_select_from_text.py
Stacking new LoRA nodes via lora_syntax text input
2025-09-02 17:18:48 +08:00
Will Miao
91801dff85 feat(localization): add new workflow-related messages for LoRA and recipe actions in multiple languages 2025-09-02 11:50:20 +08:00
Will Miao
be594133f0 feat(localization): update app title from "oRA Manager" to "LoRA Manager" across all locale files 2025-09-02 10:29:29 +08:00
Will Miao
8a538d117e feat(localization): simplify language selection labels and update app title across all locale files 2025-09-02 10:11:55 +08:00
Will Miao
8d9118cbee feat(localization): update control labels and actions for improved clarity in multiple languages 2025-09-01 22:00:19 +08:00
Will Miao
b67464ea13 feat(trigger-word-toggle): update existing tags' active state based on default_active widget value 2025-09-01 20:55:50 +08:00
Will Miao
33334da0bb feat(i18n): add structural consistency tests for locale files and enhance existing tests 2025-09-01 19:29:50 +08:00
pixelpaws
40ce2baa7b Merge pull request #388 from willmiao/i18n
I18n
2025-09-01 08:57:39 +08:00
Will Miao
1134466cc0 feat(i18n): complete locale files for all languages 2025-09-01 08:48:34 +08:00
Will Miao
92341111ad feat(localization): enhance import modal and related components with new labels, descriptions, and error messages for improved user experience 2025-08-31 22:41:35 +08:00
Will Miao
4956d6781f feat(localization): enhance download modal with new labels and error messages for improved user experience 2025-08-31 22:06:59 +08:00
Will Miao
63562240c4 feat(localization): enhance English and Chinese translations for update notifications and support modal 2025-08-31 21:54:54 +08:00
Will Miao
84d801cf14 feat(localization): enhance settings modal with new sections and translations for improved user experience 2025-08-31 21:27:59 +08:00
Will Miao
b56fe4ca68 Implement code changes to enhance functionality and improve performance 2025-08-31 20:55:08 +08:00
Will Miao
6c83c65e02 feat(localization): add custom filter message and update toast keys for recipe actions 2025-08-31 20:32:37 +08:00
Will Miao
a83f020fcc feat(localization): add file size labels and enhance search placeholders in UI components 2025-08-31 20:26:13 +08:00
Will Miao
7f9a3bf272 feat(i18n): enhance translation key extraction to optionally include container nodes 2025-08-31 19:01:23 +08:00
Will Miao
f80e266d02 feat(localization): update toast messages for consistency and improved error handling across various components 2025-08-31 18:38:42 +08:00
Will Miao
7bef562541 feat(localization): update toast messages for improved user feedback and localization support across various components 2025-08-31 16:52:58 +08:00
Will Miao
b2428f607c feat(localization): add trigger words functionality with localization support for UI elements and messages 2025-08-31 15:13:12 +08:00
Will Miao
8303196b57 feat(localization): enhance toast messages for context menu actions, model tags, and download management with improved error handling and user feedback 2025-08-31 14:27:33 +08:00
Will Miao
987b8c8742 feat(localization): enhance toast messages for recipes and example images with improved error handling and success feedback 2025-08-31 13:51:37 +08:00
Will Miao
e60a579b85 feat(localization): enhance toast messages for API actions and model management with i18n support
refactor(localization): update toast messages in various components and managers for better user feedback
2025-08-31 12:25:08 +08:00
Will Miao
be8edafed0 feat(localization): enhance toast messages for better user feedback and localization support 2025-08-31 11:51:28 +08:00
Will Miao
a258a18fa4 refactor(preload): remove unnecessary preload blocks from multiple templates 2025-08-31 11:28:49 +08:00
Will Miao
59010ca431 Refactor localization handling and improve i18n support across the application
- Replaced `safeTranslate` with `translate` in various components for consistent translation handling.
- Updated Chinese (Simplified and Traditional) localization files to include new keys and improved translations for model card actions, metadata, and usage tips.
- Enhanced the ModelCard, ModelDescription, ModelMetadata, ModelModal, and ModelTags components to utilize the new translation functions.
- Improved user feedback messages for actions like copying to clipboard, saving notes, and updating tags with localized strings.
- Ensured all UI elements reflect the correct translations based on the user's language preference.
2025-08-31 11:19:06 +08:00
Will Miao
75f3764e6c refactor(i18n): optimize safeTranslate usage by removing unnecessary await calls 2025-08-31 10:32:15 +08:00
Will Miao
867ffd1163 feat(localization): add model description translations and enhance UI text across multiple languages 2025-08-31 10:12:54 +08:00
Will Miao
6acccbbb94 fix(localization): update language labels to use English and native scripts for consistency 2025-08-31 09:16:26 +08:00
Will Miao
b2c4efab45 refactor(i18n): streamline i18n initialization and update translation methods 2025-08-31 09:03:06 +08:00
Will Miao
408a435b71 Add copilot instructions to enforce English for comments 2025-08-31 09:02:51 +08:00
Will Miao
36d3cd93d5 Enhance localization and UI for model management features
- Added new localization keys for usage statistics, collection analysis, storage efficiency, and insights in English and Chinese.
- Updated modal templates to utilize localization for delete, exclude, and bulk delete confirmations.
- Improved download modal with localized labels and placeholders.
- Enhanced example access modal with localized titles and descriptions.
- Updated help modal to include localized content for update vlogs and documentation sections.
- Refactored move modal to use localization for labels and buttons.
- Implemented localization in relink Civitai modal for warnings and help text.
- Updated update modal to reflect localized text for actions and progress messages.
- Enhanced statistics template with localized titles for charts and lists.
2025-08-30 23:20:13 +08:00
Will Miao
b36fea002e Add localization support for new features and update existing translations
- Added "unknown" status to model states in English and Chinese locales.
- Introduced new actions for checking updates and support in both locales.
- Added settings for Civitai API key with help text in both locales.
- Updated context menus and control components to use localized strings.
- Enhanced help and support modals with localization.
- Updated update modal to reflect current and new version information in localized format.
- Refactored various templates to utilize the translation function for better internationalization.
2025-08-30 22:32:44 +08:00
Will Miao
52acbd954a Add Chinese (Simplified and Traditional) localization files and implement i18n tests
- Created zh-CN.json and zh-TW.json for Simplified and Traditional Chinese translations respectively.
- Added comprehensive test suite in test_i18n.py to validate JSON structure, server-side i18n functionality, and translation completeness across multiple languages.
2025-08-30 21:41:48 +08:00
Will Miao
f6709a55c3 refactor(i18n): Remove server_i18n references from routes and update translations in zh-CN and zh-TW locales 2025-08-30 19:02:37 +08:00
Will Miao
7b374d747b cleanup 2025-08-30 18:44:33 +08:00
Will Miao
fd480a9360 refactor(i18n): Remove language setting endpoints and related logic from MiscRoutes 2025-08-30 17:48:32 +08:00
Will Miao
ec8b228867 fix(statistics): Add margin-top to metrics grid for improved spacing 2025-08-30 17:30:49 +08:00
Will Miao
401200050b feat(i18n): Enhance internationalization support by updating storage retrieval and translation handling 2025-08-30 17:29:04 +08:00
Will Miao
29160bd6e5 feat(i18n): Implement server-side internationalization support
- Added ServerI18nManager to handle translations and locale settings on the server.
- Integrated server-side translations into templates, reducing language flashing on initial load.
- Created API endpoints for setting and getting user language preferences.
- Enhanced client-side i18n handling to work seamlessly with server-rendered content.
- Updated various templates to utilize the new translation system.
- Added mixed i18n handler to coordinate server and client translations, improving user experience.
- Expanded translation files to include initialization messages for various components.
2025-08-30 16:56:56 +08:00
Will Miao
3c9e402bc0 Add Korean, Russian, and Traditional Chinese translations for LoRA Manager 2025-08-30 11:32:39 +08:00
Will Miao
ff4d0f0208 feat: Update Simplified Chinese translations for LoRA Manager to improve clarity and consistency 2025-08-29 21:32:48 +08:00
Will Miao
f82908221c Implement internationalization (i18n) system for LoRA Manager
- Added i18n support with automatic language detection based on browser settings.
- Implemented translations for English (en) and Simplified Chinese (zh-CN).
- Created utility functions for text replacement in HTML templates and JavaScript.
- Developed a comprehensive translation key structure for various application components.
- Added formatting functions for numbers, dates, and file sizes according to locale.
- Included RTL language support and dynamic updates for DOM elements.
- Created tests to verify the functionality of the i18n system.
2025-08-29 21:32:48 +08:00
Will Miao
4246908f2e feat: Add updateContainerMargin method and integrate it into sidebar state management for improved layout handling 2025-08-29 21:28:19 +08:00
Will Miao
f64597afd2 feat: Update restoreSelectedFolder to ensure activeFolder is a string before assignment and reset selectedPath if not 2025-08-29 17:46:43 +08:00
Will Miao
975ff2672d feat: Add new Flux model 'FLUX_1_KREA' and update Video Models list for enhanced model support 2025-08-28 16:24:01 +08:00
Will Miao
e90ba31784 feat: Update filter_civitai_data to include 'id' and 'modelId' fields for improved data retrieval 2025-08-28 15:21:04 +08:00
Will Miao
a4074c93bc feat: Improve folder filtering logic to ensure exact matches and handle root folder case 2025-08-28 05:33:53 +08:00
Will Miao
7a8b7598c7 feat: Enhance deepMerge function to only update existing keys in target for improved merging logic 2025-08-27 20:42:57 +08:00
Will Miao
cd0d832f14 feat: Refactor showModelModal to fetch complete metadata and update related functions for improved data handling 2025-08-27 19:42:34 +08:00
Will Miao
5b0becaaf2 feat: Implement model description retrieval and update related API endpoints 2025-08-27 18:22:56 +08:00
Will Miao
9817bac2fe feat: Add metadata endpoint and implement model metadata retrieval functionality 2025-08-27 17:44:29 +08:00
Will Miao
f6bd48cfcd feat: Update box-shadow for header and adjust controls styling for improved layout 2025-08-27 15:43:44 +08:00
Will Miao
01843b8f2b feat: Update media query breakpoints from 2000px to 2150px for improved responsiveness across components 2025-08-27 09:54:08 +08:00
Will Miao
94ed81de5e feat: Update tooltip positioning comments for clarity and consistency 2025-08-27 09:11:19 +08:00
Will Miao
0700b8f399 feat: Adjust sidebar position to align with viewport edges for improved layout consistency 2025-08-27 09:11:05 +08:00
Will Miao
d62cff9841 feat: Refactor SidebarManager integration and cleanup methods for improved state management 2025-08-26 21:38:33 +08:00
Will Miao
083f4805b2 feat: Enhance get_preview_static_url to find the longest matching route for static URLs 2025-08-26 20:41:01 +08:00
Will Miao
8e5bfd379e feat: Add closeDropdown method to manage dropdown state in SidebarManager 2025-08-26 19:26:05 +08:00
pixelpaws
2366f143d8 Merge pull request #377 from willmiao/sidebar, See #257 #52
Sidebar
2025-08-26 19:10:30 +08:00
Will Miao
e997f5bc1b feat: Update activeFolder state initialization to load from localStorage for each model type 2025-08-26 19:04:23 +08:00
Will Miao
842beec7cc feat: Update recursive search option to default to true and remove related UI elements 2025-08-26 18:14:43 +08:00
Will Miao
d2268fc9e0 feat: Implement initial hidden state for sidebar and enhance visibility handling 2025-08-26 18:02:52 +08:00
Will Miao
a98e26139f feat: Implement auto-hide functionality for sidebar and update controls layout 2025-08-26 17:57:59 +08:00
Will Miao
522a3ea88b feat: Update sidebar breadcrumb styles and enhance dropdown functionality 2025-08-26 17:13:04 +08:00
Will Miao
d7949fbc30 feat: Enhance sidebar navigation with dropdowns and refactor breadcrumb structure 2025-08-26 16:44:01 +08:00
Will Miao
6df083a1d5 feat: Refactor sidebar components for improved structure and styling 2025-08-26 15:26:45 +08:00
Will Miao
4dc80e7f6e feat: Implement sidebar navigation with folder tree and controls 2025-08-26 10:33:46 +08:00
Will Miao
c2a8508513 feat: Add get_preview_extension function to retrieve complete preview file extensions 2025-08-26 10:19:17 +08:00
Will Miao
159193ef43 feat: Implement unique filename generation with conflict resolution using metadata hash 2025-08-25 15:33:46 +08:00
Will Miao
1f37ffb105 feat: Refactor unique filename generation to use a hash provider for improved flexibility 2025-08-25 14:52:44 +08:00
Will Miao
919fed05c5 feat: Enhance model moving functionality with improved error handling and unique filename generation 2025-08-25 13:08:35 +08:00
Will Miao
1814f83bee feat: Implement post-initialization tasks and backup file cleanup in LoraManager 2025-08-25 09:03:40 +08:00
Will Miao
1823840456 feat: Disable image optimization in find_preview_file function for future configuration 2025-08-25 09:03:28 +08:00
Will Miao
623c28bfc3 feat: Remove backup creation from metadata saving functions for streamlined operations 2025-08-24 22:30:53 +08:00
511 changed files with 121278 additions and 18530 deletions

4
.github/FUNDING.yml vendored
View File

@@ -1,5 +1,5 @@
# These are supported funding model platforms
patreon: PixelPawsAI
ko_fi: pixelpawsai
custom: ['paypal.me/pixelpawsai']
patreon: PixelPawsAI
custom: ['paypal.me/pixelpawsai', 'https://afdian.com/a/pixelpawsai']

1
.github/copilot-instructions.md vendored Normal file
View File

@@ -0,0 +1 @@
Always use English for comments.

93
.github/workflows/backend-tests.yml vendored Normal file
View File

@@ -0,0 +1,93 @@
name: Backend Tests
on:
push:
branches:
- main
- master
paths:
- 'py/**'
- 'standalone.py'
- 'tests/**'
- 'requirements.txt'
- 'requirements-dev.txt'
- 'pyproject.toml'
- 'pytest.ini'
- '.github/workflows/backend-tests.yml'
pull_request:
paths:
- 'py/**'
- 'standalone.py'
- 'tests/**'
- 'requirements.txt'
- 'requirements-dev.txt'
- 'pyproject.toml'
- 'pytest.ini'
- '.github/workflows/backend-tests.yml'
jobs:
pytest:
name: Run pytest with coverage
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
cache-dependency-path: |
requirements.txt
requirements-dev.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Verify symlink support
run: |
python - <<'PY'
import os
import pathlib
import tempfile
root = pathlib.Path(tempfile.mkdtemp(prefix="lm-symlink-check-"))
target = root / "target"
target.mkdir()
link = root / "link"
try:
link.symlink_to(target, target_is_directory=True)
except OSError as exc:
raise SystemExit(f"Failed to create directory symlink in CI: {exc}")
is_link = os.path.islink(link)
is_dir = os.path.isdir(link)
realpath = os.path.realpath(link)
print(f"islink={is_link} isdir={is_dir} realpath={realpath}")
if not (is_link and is_dir and realpath == str(target)):
raise SystemExit("Directory symlink is not functioning correctly in CI; aborting.")
PY
- name: Run pytest with coverage
env:
COVERAGE_FILE: coverage/backend/.coverage
run: |
mkdir -p coverage/backend
python -m pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=html:coverage/backend/html \
--cov-report=json:coverage/backend/coverage.json
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: backend-coverage
path: coverage/backend
if-no-files-found: warn

52
.github/workflows/frontend-tests.yml vendored Normal file
View File

@@ -0,0 +1,52 @@
name: Frontend Tests
on:
push:
branches:
- main
- master
paths:
- 'package.json'
- 'package-lock.json'
- 'vitest.config.js'
- 'tests/frontend/**'
- 'static/js/**'
- 'scripts/run_frontend_coverage.js'
- '.github/workflows/frontend-tests.yml'
pull_request:
paths:
- 'package.json'
- 'package-lock.json'
- 'vitest.config.js'
- 'tests/frontend/**'
- 'static/js/**'
- 'scripts/run_frontend_coverage.js'
- '.github/workflows/frontend-tests.yml'
jobs:
vitest:
name: Run Vitest with coverage
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Use Node.js 20
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run frontend tests with coverage
run: npm run test:coverage
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: frontend-coverage
path: coverage/frontend
if-no-files-found: warn

14
.gitignore vendored
View File

@@ -1,7 +1,21 @@
__pycache__/
.pytest_cache/
settings.json
path_mappings.yaml
output/*
py/run_test.py
.vscode/
cache/
civitai/
node_modules/
coverage/
.coverage
model_cache/
# agent
.opencode/
# Vue widgets development cache (but keep build output)
vue-widgets/node_modules/
vue-widgets/.vite/
vue-widgets/dist/

192
AGENTS.md Normal file
View File

@@ -0,0 +1,192 @@
# AGENTS.md
This file provides guidance for agentic coding assistants working in this repository.
## Development Commands
### Backend Development
```bash
# Install dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run standalone server (port 8188 by default)
python standalone.py --port 8188
# Run all backend tests
pytest
# Run specific test file
pytest tests/test_recipes.py
# Run specific test function
pytest tests/test_recipes.py::test_function_name
# Run backend tests with coverage
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
### Frontend Development
```bash
# Install frontend dependencies
npm install
# Run frontend tests
npm test
# Run frontend tests in watch mode
npm run test:watch
# Run frontend tests with coverage
npm run test:coverage
```
## Python Code Style
### Imports
- Use `from __future__ import annotations` for forward references in type hints
- Group imports: standard library, third-party, local (separated by blank lines)
- Use absolute imports within `py/` package: `from ..services import X`
- Mock ComfyUI dependencies in tests using `tests/conftest.py` patterns
### Formatting & Types
- PEP 8 with 4-space indentation
- Type hints required for function signatures and class attributes
- Use `TYPE_CHECKING` guard for type-checking-only imports
- Prefer dataclasses for simple data containers
- Use `Optional[T]` for nullable types, `Union[T, None]` only when necessary
### Naming Conventions
- Files: `snake_case.py` (e.g., `model_scanner.py`, `lora_service.py`)
- Classes: `PascalCase` (e.g., `ModelScanner`, `LoraService`)
- Functions/variables: `snake_case` (e.g., `get_instance`, `model_type`)
- Constants: `UPPER_SNAKE_CASE` (e.g., `VALID_LORA_TYPES`)
- Private members: `_single_underscore` (protected), `__double_underscore` (name-mangled)
### Error Handling
- Use `logging.getLogger(__name__)` for module-level loggers
- Define custom exceptions in `py/services/errors.py`
- Use `asyncio.Lock` for thread-safe singleton patterns
- Raise specific exceptions with descriptive messages
- Log errors at appropriate levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
### Async Patterns
- Use `async def` for I/O-bound operations
- Mark async tests with `@pytest.mark.asyncio`
- Use `async with` for context managers
- Singleton pattern with class-level locks: see `ModelScanner.get_instance()`
- Use `aiohttp.web.Response` for HTTP responses
### Testing Patterns
- Use `pytest` with `--import-mode=importlib`
- Fixtures in `tests/conftest.py` handle ComfyUI mocking
- Use `@pytest.mark.no_settings_dir_isolation` for tests needing real paths
- Test files: `tests/test_*.py`
- Use `tmp_path_factory` for temporary directory isolation
## JavaScript Code Style
### Imports & Modules
- ES modules with `import`/`export`
- Use `import { app } from "../../scripts/app.js"` for ComfyUI integration
- Export named functions/classes: `export function foo() {}`
- Widget files use `*_widget.js` suffix
### Naming & Formatting
- camelCase for functions, variables, object properties
- PascalCase for classes/constructors
- Constants: `UPPER_SNAKE_CASE` (e.g., `CONVERTED_TYPE`)
- Files: `snake_case.js` or `kebab-case.js`
- 2-space indentation preferred (follow existing file conventions)
### Widget Development
- Use `app.registerExtension()` to register ComfyUI extensions
- Use `node.addDOMWidget(name, type, element, options)` for custom widgets
- Event handlers attached via `addEventListener` or widget callbacks
- See `web/comfyui/utils.js` for shared utilities
## Architecture Patterns
### Service Layer
- Use `ServiceRegistry` singleton for dependency injection
- Services follow singleton pattern via `get_instance()` class method
- Separate scanners (discovery) from services (business logic)
- Handlers in `py/routes/handlers/` implement route logic
### Model Types
- BaseModelService is abstract base for LoRA, Checkpoint, Embedding services
- ModelScanner provides file discovery and hash-based deduplication
- Persistent cache in SQLite via `PersistentModelCache`
- Metadata sync from CivitAI/CivArchive via `MetadataSyncService`
### Routes & Handlers
- Route registrars organize endpoints by domain: `ModelRouteRegistrar`, etc.
- Handlers are pure functions taking dependencies as parameters
- Use `WebSocketManager` for real-time progress updates
- Return `aiohttp.web.json_response` or `web.Response`
### Recipe System
- Base metadata in `py/recipes/base.py`
- Enrichment adds model metadata: `RecipeEnrichmentService`
- Parsers for different formats in `py/recipes/parsers/`
## Important Notes
- Always use English for comments (per copilot-instructions.md)
- Dual mode: ComfyUI plugin (uses folder_paths) vs standalone (reads settings.json)
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
- Settings auto-saved in user directory or portable mode
- WebSocket broadcasts for real-time updates (downloads, scans)
- Symlink handling requires normalized paths
- API endpoints follow `/loras/*`, `/checkpoints/*`, `/embeddings/*` patterns
- Run `python scripts/sync_translation_keys.py` after UI string updates
## Frontend UI Architecture
This project has two distinct UI systems:
### 1. Standalone Lora Manager Web UI
- Location: `./static/` and `./templates/`
- Purpose: Full-featured web application for managing LoRA models
- Tech stack: Vanilla JS + CSS, served by the standalone server
- Development: Uses npm for frontend testing (`npm test`, `npm run test:watch`, etc.)
### 2. ComfyUI Custom Node Widgets
- Location: `./web/comfyui/`
- Purpose: Widgets and UI logic that ComfyUI loads as custom node extensions
- Tech stack: Vanilla JS + Vue.js widgets (in `./vue-widgets/` and built to `./web/comfyui/vue-widgets/`)
- Widget styling: Primary styles in `./web/comfyui/lm_styles.css` (NOT `./static/css/`)
- Development: No npm build step for these widgets (Vue widgets use build system)
### Widget Development Guidelines
- Use `app.registerExtension()` to register ComfyUI extensions (ComfyUI integration layer)
- Use `node.addDOMWidget()` for custom DOM widgets
- Widget styles should follow the patterns in `./web/comfyui/lm_styles.css`
- Selected state: `rgba(66, 153, 225, 0.3)` background, `rgba(66, 153, 225, 0.6)` border
- Hover state: `rgba(66, 153, 225, 0.2)` background
- Color palette matches the Lora Manager accent color (blue #4299e1)
- Use oklch() for color values when possible (defined in `./static/css/base.css`)
- Vue widget components are in `./vue-widgets/src/components/` and built to `./web/comfyui/vue-widgets/`
- When modifying widget styles, check `./web/comfyui/lm_styles.css` for consistency with other ComfyUI widgets

211
CLAUDE.md Normal file
View File

@@ -0,0 +1,211 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Overview
ComfyUI LoRA Manager is a comprehensive LoRA management system for ComfyUI that combines a Python backend with browser-based widgets. It provides model organization, downloading from CivitAI/CivArchive, recipe management, and one-click workflow integration.
## Development Commands
### Backend Development
```bash
# Install dependencies
pip install -r requirements.txt
# Install development dependencies (for testing)
pip install -r requirements-dev.txt
# Run standalone server (port 8188 by default)
python standalone.py --port 8188
# Run backend tests with coverage
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
# Run specific test file
pytest tests/test_recipes.py
```
### Frontend Development
```bash
# Install frontend dependencies
npm install
# Run frontend tests
npm test
# Run frontend tests in watch mode
npm run test:watch
# Run frontend tests with coverage
npm run test:coverage
```
### Localization
```bash
# Sync translation keys after UI string updates
python scripts/sync_translation_keys.py
```
## Architecture
### Backend Structure (Python)
**Core Entry Points:**
- `__init__.py` - ComfyUI plugin entry point, registers nodes and routes
- `standalone.py` - Standalone server that mocks ComfyUI dependencies
- `py/lora_manager.py` - Main LoraManager class that registers HTTP routes
**Service Layer** (`py/services/`):
- `ServiceRegistry` - Singleton service registry for dependency management
- `ModelServiceFactory` - Factory for creating model services (LoRA, Checkpoint, Embedding)
- Scanner services (`lora_scanner.py`, `checkpoint_scanner.py`, `embedding_scanner.py`) - Model file discovery and indexing
- `model_scanner.py` - Base scanner with hash-based deduplication and metadata extraction
- `persistent_model_cache.py` - SQLite-based cache for model metadata
- `metadata_sync_service.py` - Syncs metadata from CivitAI/CivArchive APIs
- `civitai_client.py` / `civarchive_client.py` - API clients for external services
- `downloader.py` / `download_manager.py` - Model download orchestration
- `recipe_scanner.py` - Recipe file management and image association
- `settings_manager.py` - Application settings with migration support
- `websocket_manager.py` - WebSocket broadcasting for real-time updates
- `use_cases/` - Business logic orchestration (auto-organize, bulk refresh, downloads)
**Routes Layer** (`py/routes/`):
- Route registrars organize endpoints by domain (models, recipes, previews, example images, updates)
- `handlers/` - Request handlers implementing business logic
- Routes use aiohttp and integrate with ComfyUI's PromptServer
**Recipe System** (`py/recipes/`):
- `base.py` - Base recipe metadata structure
- `enrichment.py` - Enriches recipes with model metadata
- `merger.py` - Merges recipe data from multiple sources
- `parsers/` - Parsers for different recipe formats (PNG, JSON, workflow)
**Custom Nodes** (`py/nodes/`):
- `lora_loader.py` - LoRA loader nodes with preset support
- `save_image.py` - Enhanced save image with pattern-based filenames
- `trigger_word_toggle.py` - Toggle trigger words in prompts
- `lora_stacker.py` - Stack multiple LoRAs
- `prompt.py` - Prompt node with autocomplete
- `wanvideo_lora_select.py` - WanVideo-specific LoRA selection
**Configuration** (`py/config.py`):
- Manages folder paths for models, checkpoints, embeddings
- Handles symlink mappings for complex directory structures
- Auto-saves paths to settings.json in ComfyUI mode
### Frontend Structure (JavaScript)
**ComfyUI Widgets** (`web/comfyui/`):
- Vanilla JavaScript ES modules extending ComfyUI's LiteGraph-based UI
- `loras_widget.js` - Main LoRA selection widget with preview
- `loras_widget_events.js` - Event handling for widget interactions
- `autocomplete.js` - Autocomplete for trigger words and embeddings
- `preview_tooltip.js` - Preview tooltip for model cards
- `top_menu_extension.js` - Adds "Launch LoRA Manager" menu item
- `trigger_word_highlight.js` - Syntax highlighting for trigger words
- `utils.js` - Shared utilities and API helpers
**Widget Development:**
- Widgets use `app.registerExtension` and `getCustomWidgets` hooks
- `node.addDOMWidget(name, type, element, options)` embeds HTML in nodes
- See `docs/dom_widget_dev_guide.md` for complete DOMWidget development guide
**Web Source** (`web-src/`):
- Modern frontend components (if migrating from static)
- `components/` - Reusable UI components
- `styles/` - CSS styling
### Key Patterns
**Dual Mode Operation:**
- ComfyUI plugin mode: Integrates with ComfyUI's PromptServer, uses folder_paths
- Standalone mode: Mocks ComfyUI dependencies via `standalone.py`, reads paths from settings.json
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
**Settings Management:**
- Settings stored in user directory (via `platformdirs`) or portable mode (in repo)
- Migration system tracks settings schema version
- Template in `settings.json.example` defines defaults
**Model Scanning Flow:**
1. Scanner walks folder paths, computes file hashes
2. Hash-based deduplication prevents duplicate processing
3. Metadata extracted from safetensors headers
4. Persistent cache stores results in SQLite
5. Background sync fetches CivitAI/CivArchive metadata
6. WebSocket broadcasts updates to connected clients
**Recipe System:**
- Recipes store LoRA combinations with parameters
- Supports import from workflow JSON, PNG metadata
- Images associated with recipes via sibling file detection
- Enrichment adds model metadata for display
**Frontend-Backend Communication:**
- REST API for CRUD operations
- WebSocket for real-time progress updates (downloads, scans)
- API endpoints follow `/loras/*` pattern
## Code Style
**Python:**
- PEP 8 with 4-space indentation
- snake_case for files, functions, variables
- PascalCase for classes
- Type hints preferred
- English comments only (per copilot-instructions.md)
- Loggers via `logging.getLogger(__name__)`
**JavaScript:**
- ES modules with camelCase
- Files use `*_widget.js` suffix for ComfyUI widgets
- Prefer vanilla JS, avoid framework dependencies
## Testing
**Backend Tests:**
- pytest with `--import-mode=importlib`
- Test files: `tests/test_*.py`
- Fixtures in `tests/conftest.py`
- Mock ComfyUI dependencies using standalone.py patterns
- Markers: `@pytest.mark.asyncio` for async tests, `@pytest.mark.no_settings_dir_isolation` for real paths
**Frontend Tests:**
- Vitest with jsdom environment
- Test files: `tests/frontend/**/*.test.js`
- Setup in `tests/frontend/setup.js`
- Coverage via `npm run test:coverage`
## Important Notes
**Settings Location:**
- ComfyUI mode: Auto-saves folder paths to user settings directory
- Standalone mode: Use `settings.json` (copy from `settings.json.example`)
- Portable mode: Set `"use_portable_settings": true` in settings.json
**API Integration:**
- CivitAI API key required for downloads (add to settings)
- CivArchive API used as fallback for deleted models
- Metadata archive database available for offline metadata
**Symlink Handling:**
- Config scans symlinks to map virtual paths to physical locations
- Preview validation uses normalized preview root paths
- Fingerprinting prevents redundant symlink rescans
**ComfyUI Node Development:**
- Nodes defined in `py/nodes/`, registered in `__init__.py`
- Frontend widgets in `web/comfyui/`, matched by node type
- Use `WEB_DIRECTORY = "./web/comfyui"` convention
**Recipe Image Association:**
- Recipes scan for sibling images in same directory
- Supports repair/migration of recipe image paths
- See `py/services/recipe_scanner.py` for implementation details

167
README.md
View File

@@ -34,88 +34,62 @@ Enhance your Civitai browsing experience with our companion browser extension! S
## Release Notes
### v0.8.30
* **Automatic Model Path Correction** - Added auto-correction for model paths in built-in nodes such as Load Checkpoint, Load Diffusion Model, Load LoRA, and other custom nodes with similar functionality. Workflows containing outdated or incorrect model paths will now be automatically updated to reflect the current location of your models.
* **Node UI Enhancements** - Improved node interface for a smoother and more intuitive user experience.
* **Bug Fixes** - Addressed various bugs to enhance stability and reliability.
### v0.9.12
* **LoRA Randomizer System** - Introduced a comprehensive LoRA randomization system featuring LoRA Pool and LoRA Randomizer nodes for flexible and dynamic generation workflows.
* **LoRA Randomizer Template** - Refer to the new "LoRA Randomizer" template workflow for detailed examples of flexible randomization modes, lock & reuse options, and other features.
* **Recipe Folders** - Introduced a folder system for the Recipes page, allowing users to freely organize recipes just like they do with models.
* **Recipe Bulk Operations** - Added bulk mode support for batch moving, deleting, and setting base models for selected recipes with intuitive controls like click-and-drag selection, drag-to-folder, and Ctrl+A (Select All).
* **Prompt Search & Sorting** - Search recipes by prompt content and sort by Recipe Name, Imported Date, or LoRA Count for better browsing.
* **Recipe Favorites** - Mark specific recipes as favorites for quick access.
* **Video Recipe Support** - Enabled support for video recipes (import via LM extension or URL; video file import not supported).
* **Performance Improvements** - Fixed performance issues for dramatically improved startup and loading speed. After first scan, subsequent loads are instant regardless of collection size.
* **ComfyUI Nodes 2.0 Support** - Basic support for ComfyUI Nodes 2.0.
### v0.8.29
* **Enhanced Recipe Imports** - Improved recipe importing with new target folder selection, featuring path input autocomplete and interactive folder tree navigation. Added a "Use Default Path" option when downloading missing LoRAs.
* **WanVideo Lora Select Node Update** - Updated the WanVideo Lora Select node with a 'merge_loras' option to match the counterpart node in the WanVideoWrapper node package.
* **Autocomplete Conflict Resolution** - Resolved an autocomplete feature conflict in LoRA nodes with pysssss autocomplete.
* **Improved Download Functionality** - Enhanced download functionality with resumable downloads and improved error handling.
* **Bug Fixes** - Addressed several bugs for improved stability and performance.
### v0.9.10
* **Smarter Update Matching** - Users can now choose to check and group updates by matching base model only or with no base-model constraint; version lists also support toggling between same-base versions or all versions.
* **Flexible Tag Filtering** - The filter panel now supports tag exclusion: click a tag to include, click again to exclude, and click a third time to clear, enabling stronger and more flexible tag filters.
* **License Visibility & Controls** - Model detail headers and ComfyUI preview popups now show Civitai license icons. The filter panel gains license include/exclude options, and a new global context menu action, "Refresh license metadata," fetches missing license data.
* **Recipe Improvements** - Recipes now allow importing with zero LoRAs, and recipe detail pages show the related checkpoint for easier reference.
* **Better ZIP Downloads** - When downloading models packaged in ZIPs, model files are extracted into the target model folder; ZIPs containing multiple model files (e.g., WanVideo high/low LoRA pairs) are added as separate models.
* **Template Workflow Update** - Refreshed the "Illustrious Pony Example" template workflow with usage guidance for each LoRA Manager node.
* **Bug Fixes & Stability** - General fixes and stability improvements.
### v0.8.28
* **Autocomplete for Node Inputs** - Instantly find and add LoRAs by filename directly in Lora Loader, Lora Stacker, and WanVideo Lora Select nodes. Autocomplete suggestions include preview tooltips and preset weights, allowing you to quickly select LoRAs without opening the LoRA Manager UI.
* **Duplicate Notification Control** - Added a switch to duplicates mode, enabling users to turn off duplicate model notifications for a more streamlined experience.
* **Download Example Images from Context Menu** - Introduced a new context menu option to download example images for individual models.
### v0.9.9
* **Check for Updates Feature** - Users can now check for updates for all models or selected models in bulk mode. Models with available updates will display an "update available" badge on their model card, and users can filter to show only models with updates.
* **Model Versions Management** - Added a new Versions tab in the model modal that centralizes all versions of a model, providing download, delete, and ignore update functions.
* **Send Checkpoint to ComfyUI** - Users can now click the send button on a checkpoint card to send the checkpoint directly to the current workflow's checkpoint or diffusion model loader node in ComfyUI.
* **Customizable Model Card Display** - Added a new setting that allows users to choose whether to display the model name or filename on model cards.
* **New Path Template Placeholders** - Added new path template placeholders: `{model_name}` and `{version_name}` for more flexible organization.
* **ComfyUI Auto Path Correction Setting** - Added a new setting within ComfyUI to enable or disable the auto path correction feature.
### v0.8.27
* **User Experience Enhancements** - Improved the model download target folder selection with path input autocomplete and interactive folder tree navigation, making it easier and faster to choose where models are saved.
* **Default Path Option for Downloads** - Added a "Use Default Path" option when downloading models. When enabled, models are automatically organized and stored according to your configured path template settings.
* **Advanced Download Path Templates** - Expanded path template settings, allowing users to set individual templates for LoRA, checkpoint, and embedding models for greater flexibility. Introduced the `{author}` placeholder, enabling automatic organization of model files by creator name.
* **Bug Fixes & Stability Improvements** - Addressed various bugs and improved overall stability for a smoother experience.
### v0.9.8
* **Full CivArchive API Support** - Added complete support for the CivArchive API as a fallback metadata source beyond Civitai API. Models deleted from Civitai can now still retrieve metadata through the CivArchive API.
* **Download Models from CivArchive** - Added support for downloading models directly from CivArchive, similar to downloading from Civitai. Simply click the Download button and paste the model URL to download the corresponding model.
* **Custom Priority Tags** - Introduced Custom Priority Tags feature, allowing users to define custom priority tags. These tags will appear as suggestions when editing tags or during auto organization/download using default paths, providing more precise and controlled folder organization. [Guide](https://github.com/willmiao/ComfyUI-Lora-Manager/wiki/Priority-Tags-Configuration-Guide)
* **Drag and Drop Tag Reordering** - Added drag and drop functionality to reorder tags in the tags edit mode for improved usability.
* **Download Control in Example Images Panel** - Added stop control in the Download Example Images Panel for better download management.
* **Prompt (LoraManager) Node with Autocomplete** - Added new Prompt (LoraManager) node with autocomplete feature for adding embeddings.
* **Lora Manager Nodes in Subgraphs** - Lora Manager nodes now support being placed within subgraphs for more flexible workflow organization.
### v0.8.26
* **Creator Search Option** - Added ability to search models by creator name, making it easier to find models from specific authors.
* **Enhanced Node Usability** - Improved user experience for Lora Loader, Lora Stacker, and WanVideo Lora Select nodes by fixing the maximum height of the text input area. Users can now freely and conveniently adjust the LoRA region within these nodes.
* **Compatibility Fixes** - Resolved compatibility issues with ComfyUI and certain custom nodes, including ComfyUI-Custom-Scripts, ensuring smoother integration and operation.
### v0.9.6
* **Metadata Archive Database Support** - Added the ability to download and utilize a metadata archive database, enabling access to metadata for models that have been deleted from CivitAI.
* **App-Level Proxy Settings** - Introduced support for configuring a global proxy within the application, making it easier to use the manager behind network restrictions.
* **Bug Fixes** - Various bug fixes for improved stability and reliability.
### v0.8.25
* **LoRA List Reordering**
- Drag & Drop: Easily rearrange LoRA entries using the drag handle.
- Keyboard Shortcuts:
- Arrow keys: Navigate between LoRAs
- Ctrl/Cmd + Arrow: Move selected LoRA up/down
- Ctrl/Cmd + Home/End: Move selected LoRA to top/bottom
- Delete/Backspace: Remove selected LoRA
- Context Menu: Right-click for quick actions like Move Up, Move Down, Move to Top, Move to Bottom.
* **Bulk Operations for Checkpoints & Embeddings**
- Bulk Mode: Select multiple checkpoints or embeddings for batch actions.
- Bulk Refresh: Update Civitai metadata for selected models.
- Bulk Delete: Remove multiple models at once.
- Bulk Move (Embeddings): Move selected embeddings to a different folder.
* **New Setting: Auto Download Example Images**
- Automatically fetch example images for models missing previews (requires download location to be set). Enabled by default.
* **General Improvements**
- Various user experience enhancements and stability fixes.
### v0.9.2
* **Bulk Auto-Organization Action** - Added a new bulk auto-organization feature. You can now select multiple models and automatically organize them according to your current path template settings for streamlined management.
* **Bug Fixes** - Addressed several bugs to improve stability and reliability.
### v0.8.22
* **Embeddings Management** - Added Embeddings page for comprehensive embedding model management.
* **Advanced Sorting Options** - Introduced flexible sorting controls, allowing sorting by name, added date, or file size in both ascending and descending order.
* **Custom Download Path Templates & Base Model Mapping** - Implemented UI settings for configuring download path templates and base model path mappings, allowing customized model organization and storage location when downloading models via LM Civitai Extension.
* **LM Civitai Extension Enhancements** - Improved concurrent download performance and stability, with new support for canceling active downloads directly from the extension interface.
* **Update Feature** - Added update functionality, allowing users to update LoRA Manager to the latest release version directly from the LoRA Manager UI.
* **Bulk Operations: Refresh All** - Added bulk refresh functionality, allowing users to update Civitai metadata across multiple LoRAs.
### v0.9.1
* **Enhanced Bulk Operations** - Improved bulk operations with Marquee Selection and a bulk operation context menu, providing a more intuitive, desktop-application-like user experience.
* **New Bulk Actions** - Added bulk operations for adding tags and setting base models to multiple models simultaneously.
### v0.8.20
* **LM Civitai Extension** - Released [browser extension through Chrome Web Store](https://chromewebstore.google.com/detail/lm-civitai-extension/capigligggeijgmocnaflanlbghnamgm?utm_source=item-share-cb) that works seamlessly with LoRA Manager to enhance Civitai browsing experience, showing which models are already in your local library, enabling one-click downloads, and providing queue and parallel download support
* **Enhanced Lora Loader** - Added support for nunchaku, improving convenience when working with ComfyUI-nunchaku workflows, plus new template workflows for quick onboarding
* **WanVideo Integration** - Introduced WanVideo Lora Select (LoraManager) node compatible with ComfyUI-WanVideoWrapper for streamlined lora usage in video workflows, including a template workflow to help you get started quickly
### v0.8.19
* **Analytics Dashboard** - Added new Statistics page providing comprehensive visual analysis of model collection and usage patterns for better library insights
* **Target Node Selection** - Enhanced workflow integration with intelligent target choosing when sending LoRAs/recipes to workflows with multiple loader/stacker nodes; a visual selector now appears showing node color, type, ID, and title for precise targeting
* **Enhanced NSFW Controls** - Added support for setting NSFW levels on recipes with automatic content blurring based on user preferences
* **Customizable Card Display** - New display settings allowing users to choose whether card information and action buttons are always visible or only revealed on hover
* **Expanded Compatibility** - Added support for efficiency-nodes-comfyui in Save Recipe and Save Image nodes, plus fixed compatibility with ComfyUI_Custom_Nodes_AlekPet
### v0.8.18
* **Custom Example Images** - Added ability to import your own example images for LoRAs and checkpoints with automatic metadata extraction from embedded information
* **Enhanced Example Management** - New action buttons to set specific examples as previews or delete custom examples
* **Improved Duplicate Detection** - Enhanced "Find Duplicates" with hash verification feature to eliminate false positives when identifying duplicate models
* **Tag Management** - Added tag editing functionality allowing users to customize and manage model tags
* **Advanced Selection Controls** - Implemented Ctrl+A shortcut for quickly selecting all filtered LoRAs, automatically entering bulk mode when needed
* **Note**: Cache file functionality temporarily disabled pending rework
### v0.8.17
* **Duplicate Model Detection** - Added "Find Duplicates" functionality for LoRAs and checkpoints using model file hash detection, enabling convenient viewing and batch deletion of duplicate models
* **Enhanced URL Recipe Imports** - Optimized import recipe via URL functionality using CivitAI API calls instead of web scraping, now supporting all rated images (including NSFW) for recipe imports
* **Improved TriggerWord Control** - Enhanced TriggerWord Toggle node with new default_active switch to set the initial state (active/inactive) when trigger words are added
* **Centralized Example Management** - Added "Migrate Existing Example Images" feature to consolidate downloaded example images from model folders into central storage with customizable naming patterns
* **Intelligent Word Suggestions** - Implemented smart trigger word suggestions by reading class tokens and tag frequency from safetensors files, displaying recommendations when editing trigger words
* **Model Version Management** - Added "Re-link to CivitAI" context menu option for connecting models to different CivitAI versions when needed
### v0.9.0
* **UI Overhaul for Enhanced Navigation** - Replaced the top flat folder tags with a new folder sidebar and breadcrumb navigation system for a more intuitive folder browsing and selection experience.
* **Dual-Mode Folder Sidebar** - The new folder sidebar offers two display modes: 'List Mode,' which mirrors the classic folder view, and 'Tree Mode,' which presents a hierarchical folder structure for effortless navigation through nested directories.
* **Internationalization Support** - Introduced multi-language support, now available in English, Simplified Chinese, Traditional Chinese, Spanish, Japanese, Korean, French, Russian, and German. Feedback from native speakers is welcome to improve the translations.
* **Automatic Filename Conflict Resolution** - Implemented automatic file renaming (`original name + short hash`) to prevent conflicts when downloading or moving models.
* **Performance Optimizations & Bug Fixes** - Various performance improvements and bug fixes for a more stable and responsive experience.
[View Update History](./update_logs.md)
@@ -174,9 +148,10 @@ Enhance your Civitai browsing experience with our companion browser extension! S
### Option 2: **Portable Standalone Edition** (No ComfyUI required)
1. Download the [Portable Package](https://github.com/willmiao/ComfyUI-Lora-Manager/releases/download/v0.8.26/lora_manager_portable.7z)
2. Copy the provided `settings.json.example` file to create a new file named `settings.json` in `comfyui-lora-manager` folder
3. Edit `settings.json` to include your correct model folder paths and CivitAI API key
1. Download the [Portable Package](https://github.com/willmiao/ComfyUI-Lora-Manager/releases/download/v0.9.8/lora_manager_portable.7z)
2. Copy the provided `settings.json.example` file to create a new file named `settings.json` in `comfyui-lora-manager` folder.
3. Edit the new `settings.json` to include your correct model folder paths and CivitAI API key
- Set `"use_portable_settings": true` if you want the configuration to remain inside the repository folder instead of your user settings directory.
4. Run run.bat
- To change the startup port, edit `run.bat` and modify the parameter (e.g. `--port 9001`)
@@ -244,7 +219,7 @@ You can combine multiple patterns to create detailed, organized filenames for yo
You can now run LoRA Manager independently from ComfyUI:
1. **For ComfyUI users**:
- Launch ComfyUI with LoRA Manager at least once to initialize the necessary path information in the `settings.json` file.
- Launch ComfyUI with LoRA Manager at least once to initialize the necessary path information in the `settings.json` file located in your user settings folder (see paths above).
- Make sure dependencies are installed: `pip install -r requirements.txt`
- From your ComfyUI root directory, run:
```bash
@@ -257,8 +232,9 @@ You can now run LoRA Manager independently from ComfyUI:
```
2. **For non-ComfyUI users**:
- Copy the provided `settings.json.example` file to create a new file named `settings.json`
- Edit `settings.json` to include your correct model folder paths and CivitAI API key
- Copy the provided `settings.json.example` file to create a new file named `settings.json`. Update the API key, optional language, and folder paths only—the library registry is created automatically when LoRA Manager starts.
- Edit `settings.json` to include your correct model folder paths and CivitAI API key (you can leave the defaults until ready to configure them)
- Enable portable mode by setting `"use_portable_settings": true` if you prefer LoRA Manager to read and write the `settings.json` located in the project directory.
- Install required dependencies: `pip install -r requirements.txt`
- Run standalone mode:
```bash
@@ -266,8 +242,37 @@ You can now run LoRA Manager independently from ComfyUI:
```
- Access the interface through your browser at: `http://localhost:8188/loras`
> **Note:** Existing installations automatically migrate the legacy `settings.json` from the plugin folder to the user settings directory the first time you launch this version.
This standalone mode provides a lightweight option for managing your model and recipe collection without needing to run the full ComfyUI environment, making it useful even for users who primarily use other stable diffusion interfaces.
## Testing & Coverage
### Backend
Install the development dependencies and run pytest with coverage reports:
```bash
pip install -r requirements-dev.txt
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
HTML, XML, and JSON artifacts are stored under `coverage/backend/` so you can inspect hot spots locally or from CI artifacts.
### Frontend
Run the Vitest coverage suite to analyze widget hot spots:
```bash
npm run test:coverage
```
---
## Contributing

View File

@@ -1,28 +1,91 @@
from .py.lora_manager import LoraManager
from .py.nodes.lora_loader import LoraManagerLoader, LoraManagerTextLoader
from .py.nodes.trigger_word_toggle import TriggerWordToggle
from .py.nodes.lora_stacker import LoraStacker
from .py.nodes.save_image import SaveImage
from .py.nodes.debug_metadata import DebugMetadata
from .py.nodes.wanvideo_lora_select import WanVideoLoraSelect
# Import metadata collector to install hooks on startup
from .py.metadata_collector import init as init_metadata_collector
try: # pragma: no cover - import fallback for pytest collection
from .py.lora_manager import LoraManager
from .py.nodes.lora_loader import LoraManagerLoader, LoraManagerTextLoader
from .py.nodes.trigger_word_toggle import TriggerWordToggle
from .py.nodes.prompt import PromptLoraManager
from .py.nodes.lora_stacker import LoraStacker
from .py.nodes.save_image import SaveImageLM
from .py.nodes.debug_metadata import DebugMetadata
from .py.nodes.wanvideo_lora_select import WanVideoLoraSelectLM
from .py.nodes.wanvideo_lora_select_from_text import WanVideoLoraSelectFromText
from .py.nodes.lora_pool import LoraPoolNode
from .py.nodes.lora_randomizer import LoraRandomizerNode
from .py.metadata_collector import init as init_metadata_collector
except (
ImportError
): # pragma: no cover - allows running under pytest without package install
import importlib
import pathlib
import sys
package_root = pathlib.Path(__file__).resolve().parent
if str(package_root) not in sys.path:
sys.path.append(str(package_root))
PromptLoraManager = importlib.import_module("py.nodes.prompt").PromptLoraManager
LoraManager = importlib.import_module("py.lora_manager").LoraManager
LoraManagerLoader = importlib.import_module(
"py.nodes.lora_loader"
).LoraManagerLoader
LoraManagerTextLoader = importlib.import_module(
"py.nodes.lora_loader"
).LoraManagerTextLoader
TriggerWordToggle = importlib.import_module(
"py.nodes.trigger_word_toggle"
).TriggerWordToggle
LoraStacker = importlib.import_module("py.nodes.lora_stacker").LoraStacker
SaveImageLM = importlib.import_module("py.nodes.save_image").SaveImageLM
DebugMetadata = importlib.import_module("py.nodes.debug_metadata").DebugMetadata
WanVideoLoraSelectLM = importlib.import_module(
"py.nodes.wanvideo_lora_select"
).WanVideoLoraSelectLM
WanVideoLoraSelectFromText = importlib.import_module(
"py.nodes.wanvideo_lora_select_from_text"
).WanVideoLoraSelectFromText
LoraPoolNode = importlib.import_module("py.nodes.lora_pool").LoraPoolNode
LoraRandomizerNode = importlib.import_module(
"py.nodes.lora_randomizer"
).LoraRandomizerNode
init_metadata_collector = importlib.import_module("py.metadata_collector").init
NODE_CLASS_MAPPINGS = {
PromptLoraManager.NAME: PromptLoraManager,
LoraManagerLoader.NAME: LoraManagerLoader,
LoraManagerTextLoader.NAME: LoraManagerTextLoader,
TriggerWordToggle.NAME: TriggerWordToggle,
LoraStacker.NAME: LoraStacker,
SaveImage.NAME: SaveImage,
SaveImageLM.NAME: SaveImageLM,
DebugMetadata.NAME: DebugMetadata,
WanVideoLoraSelect.NAME: WanVideoLoraSelect
WanVideoLoraSelectLM.NAME: WanVideoLoraSelectLM,
WanVideoLoraSelectFromText.NAME: WanVideoLoraSelectFromText,
LoraPoolNode.NAME: LoraPoolNode,
LoraRandomizerNode.NAME: LoraRandomizerNode,
}
WEB_DIRECTORY = "./web/comfyui"
# Check and build Vue widgets if needed (development mode)
try:
from .py.vue_widget_builder import check_and_build_vue_widgets
# Auto-build in development, warn only if fails
check_and_build_vue_widgets(auto_build=True, warn_only=True)
except ImportError:
# Fallback for pytest
import importlib
check_and_build_vue_widgets = importlib.import_module(
"py.vue_widget_builder"
).check_and_build_vue_widgets
check_and_build_vue_widgets(auto_build=True, warn_only=True)
except Exception as e:
import logging
logging.warning(f"[LoRA Manager] Vue widget build check skipped: {e}")
# Initialize metadata collector
init_metadata_collector()
# Register routes on import
LoraManager.add_routes()
__all__ = ['NODE_CLASS_MAPPINGS', 'WEB_DIRECTORY']
__all__ = ["NODE_CLASS_MAPPINGS", "WEB_DIRECTORY"]

180
docs/LM-Extension-Wiki.md Normal file
View File

@@ -0,0 +1,180 @@
## Overview
The **LoRA Manager Civitai Extension** is a Browser extension designed to work seamlessly with [LoRA Manager](https://github.com/willmiao/ComfyUI-Lora-Manager) to significantly enhance your browsing experience on [Civitai](https://civitai.com).
It also supports browsing on [CivArchive](https://civarchive.com/) (formerly CivitaiArchive).
With this extension, you can:
✅ Instantly see which models are already present in your local library
✅ Download new models with a single click
✅ Manage downloads efficiently with queue and parallel download support
✅ Keep your downloaded models automatically organized according to your custom settings
![Civitai Models page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-models-page.png)
![CivArchive Models page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civarchive-models-page.png)
---
## Why Are All Features for Supporters Only?
I love building tools for the Stable Diffusion and ComfyUI communities, and LoRA Manager is a passion project that I've poured countless hours into. When I created this companion extension, my hope was to offer its core features for free, as a thank-you to all of you.
Unfortunately, I've reached a point where I need to be realistic. The level of support from the free model has been far lower than what's needed to justify the continuous development and maintenance for both projects. It was a difficult decision, but I've chosen to make the extension's features exclusive to supporters.
This change is crucial for me to be able to continue dedicating my time to improving the free and open-source LoRA Manager, which I'm committed to keeping available for everyone.
Your support does more than just unlock a few features—it allows me to keep innovating and ensures the core LoRA Manager project thrives. I'm incredibly grateful for your understanding and any support you can offer. ❤️
(_For those who previously supported me on Ko-fi with a one-time donation, I'll be sending out license keys individually as a thank-you._)
---
## Installation
### Supported Browsers & Installation Methods
| Browser | Installation Method |
|--------------------|-------------------------------------------------------------------------------------|
| **Google Chrome** | [Chrome Web Store link](https://chromewebstore.google.com/detail/capigligggeijgmocnaflanlbghnamgm?utm_source=item-share-cb) |
| **Microsoft Edge** | Install via Chrome Web Store (compatible) |
| **Brave Browser** | Install via Chrome Web Store (compatible) |
| **Opera** | Install via Chrome Web Store (compatible) |
| **Firefox** | <div id="firefox-install" class="install-ok"><a href="https://github.com/willmiao/lm-civitai-extension-firefox/releases/latest/download/extension.xpi">📦 Install Firefox Extension (reviewed and verified by Mozilla)</a></div> |
For non-Chrome browsers (e.g., Microsoft Edge), you can typically install extensions from the Chrome Web Store by following these steps: open the extensions Chrome Web Store page, click 'Get extension', then click 'Allow' when prompted to enable installations from other stores, and finally click 'Add extension' to complete the installation.
---
## Privacy & Security
I understand concerns around browser extensions and privacy, and I want to be fully transparent about how the **LM Civitai Extension** works:
- **Reviewed and Verified**
This extension has been **manually reviewed and approved by the Chrome Web Store**. The Firefox version uses the **exact same code** (only the packaging format differs) and has passed **Mozillas Add-on review**.
- **Minimal Network Access**
The only external server this extension connects to is:
**`https://willmiao.shop`** — used solely for **license validation**.
It does **not collect, transmit, or store any personal or usage data**.
No browsing history, no user IDs, no analytics, no hidden trackers.
- **Local-Only Model Detection**
Model detection and LoRA Manager communication all happen **locally** within your browser, directly interacting with your local LoRA Manager backend.
I value your trust and are committed to keeping your local setup private and secure. If you have any questions, feel free to reach out!
---
## How to Use
After installing the extension, you'll automatically receive a **7-day trial** to explore all features.
When the extension is correctly installed and your license is valid:
- Open **Civitai**, and you'll see visual indicators added by the extension on model cards, showing:
- ✅ Models already present in your local library
- ⬇️ A download button for models not in your library
Clicking the download button adds the corresponding model version to the download queue, waiting to be downloaded. You can set up to **5 models to download simultaneously**.
### Visual Indicators Appear On:
- **Home Page** — Featured models
- **Models Page**
- **Creator Profiles** — If the creator has set their models to be visible
- **Recommended Resources** — On individual model pages
### Version Buttons on Model Pages
On a specific model page, visual indicators also appear on version buttons, showing which versions are already in your local library.
When switching to a specific version by clicking a version button:
- Clicking the download button will open a dropdown:
- Download via **LoRA Manager**
- Download via **Original Download** (browser download)
You can check **Remember my choice** to set your preferred default. You can change this setting anytime in the extension's settings.
![Civitai Model Page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-model-page.png)
### Resources on Image Pages (2025-08-05) — now shows in-library indicators for image resources. Import image as recipe coming soon!
![Civitai Image Page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-image-page.jpg)
---
## Model Download Location & LoRA Manager Settings
To use the **one-click download function**, you must first set:
- Your **Default LoRAs Root**
- Your **Default Checkpoints Root**
These are set within LoRA Manager's settings.
When everything is configured, downloaded model files will be placed in:
`<Default_Models_Root>/<Base_Model_of_the_Model>/<First_Tag_of_the_Model>`
### Update: Default Path Customization (2025-07-21)
A new setting to customize the default download path has been added in the nightly version. You can now personalize where models are saved when downloading via the LM Civitai Extension.
![Default Path Customization](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/default-path-customization.png)
The previous YAML path mapping file will be deprecated—settings will now be unified in settings.json to simplify configuration.
---
## Backend Port Configuration
If your **ComfyUI** or **LoRA Manager** backend is running on a port **other than the default 8188**, you must configure the backend port in the extension's settings.
After correctly setting and saving the port, you'll see in the extension's header area:
- A **Healthy** status with the tooltip: `Connected to LoRA Manager on port xxxx`
---
## Advanced Usage
### Connecting to a Remote LoRA Manager
If your LoRA Manager is running on another computer, you can still connect from your browser using port forwarding.
> **Why can't you set a remote IP directly?**
>
> For privacy and security, the extension only requests access to `http://127.0.0.1/*`. Supporting remote IPs would require much broader permissions, which may be rejected by browser stores and could raise user concerns.
**Solution: Port Forwarding with `socat`**
On your browser computer, run:
`socat TCP-LISTEN:8188,bind=127.0.0.1,fork TCP:REMOTE.IP.ADDRESS.HERE:8188`
- Replace `REMOTE.IP.ADDRESS.HERE` with the IP of the machine running LoRA Manager.
- Adjust the port if needed.
This lets the extension connect to `127.0.0.1:8188` as usual, with traffic forwarded to your remote server.
_Thanks to user **Temikus** for sharing this solution!_
---
## Roadmap
The extension will evolve alongside **LoRA Manager** improvements. Planned features include:
- [x] Support for **additional model types** (e.g., embeddings)
- [ ] One-click **Recipe Import**
- [x] Display of in-library status for all resources in the **Resources Used** section of the image page
- [x] One-click **Auto-organize Models**
**Stay tuned — and thank you for your support!**
---

View File

@@ -0,0 +1,93 @@
# Example image route architecture
The example image routing stack mirrors the layered model route stack described in
[`docs/architecture/model_routes.md`](model_routes.md). HTTP wiring, controller setup,
handler orchestration, and long-running workflows now live in clearly separated modules so
we can extend download/import behaviour without touching the entire feature surface.
```mermaid
graph TD
subgraph HTTP
A[ExampleImagesRouteRegistrar] -->|binds| B[ExampleImagesRoutes controller]
end
subgraph Application
B --> C[ExampleImagesHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Download manager / processor / file manager]
end
subgraph Side Effects
F1 --> G1[Filesystem]
F1 --> G2[Model metadata]
F1 --> G3[WebSocket progress]
end
```
## Layer responsibilities
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/example_images_route_registrar.py` | Declarative catalogue of every example image endpoint plus helpers that bind them to an `aiohttp` router. Keeps HTTP concerns symmetrical with the model registrar. |
| Controller | `py/routes/example_images_routes.py` | Lazily constructs `ExampleImagesHandlerSet`, injects defaults for the download manager, processor, and file manager, and exposes the registrar-ready mapping just like `BaseModelRoutes`. |
| Handler set | `py/routes/handlers/example_images_handlers.py` | Groups HTTP adapters by concern (downloads, imports/deletes, filesystem access). Each handler translates domain errors into HTTP responses and defers to a use case or utility service. |
| Use cases | `py/services/use_cases/example_images/*.py` | Encapsulate orchestration for downloads and imports. They validate input, translate concurrency/configuration errors, and keep handler logic declarative. |
| Supporting services | `py/utils/example_images_download_manager.py`, `py/utils/example_images_processor.py`, `py/utils/example_images_file_manager.py` | Execute long-running work: pull assets from Civitai, persist uploads, clean metadata, expose filesystem actions with guardrails, and broadcast progress snapshots. |
## Handler responsibilities & invariants
`ExampleImagesHandlerSet` flattens the handler objects into the `{"handler_name": coroutine}`
mapping consumed by the registrar. The table below outlines how each handler collaborates
with the use cases and utilities.
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `ExampleImagesDownloadHandler` | `/api/lm/download-example-images`, `/api/lm/example-images-status`, `/api/lm/pause-example-images`, `/api/lm/resume-example-images`, `/api/lm/force-download-example-images` | `DownloadExampleImagesUseCase`, `DownloadManager` | Delegates payload validation and concurrency checks to the use case; progress/status endpoints expose the same snapshot used for WebSocket broadcasts; pause/resume surface `DownloadNotRunningError` as HTTP 400 instead of 500. |
| `ExampleImagesManagementHandler` | `/api/lm/import-example-images`, `/api/lm/delete-example-image` | `ImportExampleImagesUseCase`, `ExampleImagesProcessor` | Multipart uploads are streamed to disk via the use case; validation failures return HTTP 400 with no filesystem side effects; deletion funnels through the processor to prune metadata and cached images consistently. |
| `ExampleImagesFileHandler` | `/api/lm/open-example-images-folder`, `/api/lm/example-image-files`, `/api/lm/has-example-images` | `ExampleImagesFileManager` | Centralises filesystem access, enforcing settings-based root paths and returning HTTP 400/404 for missing configuration or folders; responses always include `success`/`has_images` booleans for UI consumption. |
## Use case boundaries
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `DownloadExampleImagesUseCase` | `execute(payload)` | `DownloadManager.start_download`, download configuration errors | Raises `DownloadExampleImagesInProgressError` when the manager reports an active job, rewraps configuration errors into `DownloadExampleImagesConfigurationError`, and lets `ExampleImagesDownloadError` bubble as 500s so handlers do not duplicate logging. |
| `ImportExampleImagesUseCase` | `execute(request)` | `ExampleImagesProcessor.import_images`, temporary file helpers | Supports multipart or JSON payloads, normalises file paths into a single list, cleans up temp files even on failure, and maps validation issues to `ImportExampleImagesValidationError` for HTTP 400 responses. |
## Maintaining critical invariants
* **Shared progress snapshots** - The download handler returns the same snapshot built by
`DownloadManager`, guaranteeing parity between HTTP polling endpoints and WebSocket
progress events.
* **Safe filesystem access** - All folder/file actions flow through
`ExampleImagesFileManager`, which validates the configured example image root and ensures
responses never leak absolute paths outside the allowed directory.
* **Metadata hygiene** - Import/delete operations run through `ExampleImagesProcessor`,
which updates model metadata via `MetadataManager` and notifies the relevant scanners so
cache state stays in sync.
## Migration notes
The refactor brings the example image stack in line with the model/recipe stacks:
1. `ExampleImagesRouteRegistrar` now owns the declarative route list. Downstream projects
should rely on `ExampleImagesRoutes.to_route_mapping()` instead of manually wiring
handler callables.
2. `ExampleImagesRoutes` caches its `ExampleImagesHandlerSet` just like
`BaseModelRoutes`. If you previously instantiated handlers directly, inject custom
collaborators via the controller constructor (`download_manager`, `processor`,
`file_manager`) to keep test seams predictable.
3. Tests that mocked `ExampleImagesRoutes.setup_routes` should switch to patching
`DownloadExampleImagesUseCase`/`ImportExampleImagesUseCase` at import time. The handlers
expect those abstractions to surface validation/concurrency errors, and bypassing them
will skip the HTTP-friendly error mapping.
## Extending the stack
1. Add the endpoint to `ROUTE_DEFINITIONS` with a unique `handler_name`.
2. Expose the coroutine on an existing handler class (or create a new handler and extend
`ExampleImagesHandlerSet`).
3. Wire additional services or factories inside `_build_handler_set` on
`ExampleImagesRoutes`, mirroring how the model stack introduces new use cases.
`tests/routes/test_example_images_routes.py` exercises registrar binding, download pause
flows, and import validations. Use it as a template when introducing new handler
collaborators or error mappings.

View File

@@ -0,0 +1,100 @@
# Base model route architecture
The model routing stack now splits HTTP wiring, orchestration logic, and
business rules into discrete layers. The goal is to make it obvious where a
new collaborator should live and which contract it must honour. The diagram
below captures the end-to-end flow for a typical request:
```mermaid
graph TD
subgraph HTTP
A[ModelRouteRegistrar] -->|binds| B[BaseModelRoutes handler proxy]
end
subgraph Application
B --> C[ModelHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Services / scanners]
end
subgraph Side Effects
F1 --> G1[Cache & metadata]
F1 --> G2[Filesystem]
F1 --> G3[WebSocket state]
end
```
Every box maps to a concrete module:
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/model_route_registrar.py` | Declarative list of routes shared by every model type and helper methods for binding them to an `aiohttp` application. |
| Route controller | `py/routes/base_model_routes.py` | Constructs the handler graph, injects shared services, exposes proxies that surface `503 Service not ready` when the model service has not been attached. |
| Handler set | `py/routes/handlers/model_handlers.py` | Thin HTTP adapters grouped by concern (page rendering, listings, mutations, queries, downloads, CivitAI integration, move operations, auto-organize). |
| Use cases | `py/services/use_cases/*.py` | Encapsulate long-running flows (`DownloadModelUseCase`, `BulkMetadataRefreshUseCase`, `AutoOrganizeUseCase`). They normalise validation errors and concurrency constraints before returning control to the handlers. |
| Services | `py/services/*.py` | Existing services and scanners that mutate caches, write metadata, move files, and broadcast WebSocket updates. |
## Handler responsibilities & contracts
`ModelHandlerSet` flattens the handler objects into the exact callables used by
the registrar. The table below highlights the separation of concerns within
the set and the invariants that must hold after each handler returns.
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `ModelPageView` | `/{prefix}` | `SettingsManager`, `server_i18n`, Jinja environment, `service.scanner` | Template is rendered with `is_initializing` flag when caches are cold; i18n filter is registered exactly once per environment instance. |
| `ModelListingHandler` | `/api/lm/{prefix}/list` | `service.get_paginated_data`, `service.format_response` | Listings respect pagination query parameters and cap `page_size` at 100; every item is formatted before response. |
| `ModelManagementHandler` | Mutations (delete, exclude, metadata, preview, tags, rename, bulk delete, duplicate verification) | `ModelLifecycleService`, `MetadataSyncService`, `PreviewAssetService`, `TagUpdateService`, scanner cache/index | Cache state mirrors filesystem changes: deletes prune cache & hash index, preview replacements synchronise metadata and cache NSFW levels, metadata saves trigger cache resort when names change. |
| `ModelQueryHandler` | Read-only queries (top tags, folders, duplicates, metadata, URLs) | Service query helpers & scanner cache | Outputs always wrapped in `{"success": True}` when no error; duplicate/filename grouping omits empty entries; invalid parameters (e.g. missing `model_root`) return HTTP 400. |
| `ModelDownloadHandler` | `/api/lm/download-model`, `/download-model-get`, `/download-progress/{id}`, `/cancel-download-get` | `DownloadModelUseCase`, `DownloadCoordinator`, `WebSocketManager` | Payload validation errors become HTTP 400 without mutating download progress cache; early-access failures surface as HTTP 401; successful downloads cache progress snapshots that back both WebSocket broadcasts and polling endpoints. |
| `ModelCivitaiHandler` | CivitAI metadata routes | `MetadataSyncService`, metadata provider factory, `BulkMetadataRefreshUseCase` | `fetch_all_civitai` streams progress via `WebSocketBroadcastCallback`; version lookups validate model type before returning; local availability fields derive from hash lookups without mutating cache state. |
| `ModelMoveHandler` | `move_model`, `move_models_bulk` | `ModelMoveService` | Moves execute atomically per request; bulk operations aggregate success/failure per file set. |
| `ModelAutoOrganizeHandler` | `/api/lm/{prefix}/auto-organize` (GET/POST), `/auto-organize-progress` | `AutoOrganizeUseCase`, `WebSocketProgressCallback`, `WebSocketManager` | Enforces single-flight execution using the shared lock; progress broadcasts remain available to polling clients until explicitly cleared; conflicts return HTTP 409 with a descriptive error. |
## Use case boundaries
Each use case exposes a narrow asynchronous API that hides the underlying
services. Their error mapping is essential for predictable HTTP responses.
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `DownloadModelUseCase` | `execute(payload)` | `DownloadCoordinator.schedule_download` | Translates `ValueError` into `DownloadModelValidationError` for HTTP 400, recognises early-access errors (`"401"` in message) and surfaces them as `DownloadModelEarlyAccessError`, forwards success dictionaries untouched. |
| `AutoOrganizeUseCase` | `execute(file_paths, progress_callback)` | `ModelFileService.auto_organize_models`, `WebSocketManager` lock | Guarded by `ws_manager` lock + status checks; raises `AutoOrganizeInProgressError` before invoking the file service when another run is already active. |
| `BulkMetadataRefreshUseCase` | `execute_with_error_handling(progress_callback)` | `MetadataSyncService`, `SettingsManager`, `WebSocketBroadcastCallback` | Iterates through cached models, applies metadata sync, emits progress snapshots that handlers broadcast unchanged. |
## Maintaining legacy contracts
The refactor preserves the invariants called out in the previous architecture
notes. The most critical ones are reiterated here to emphasise the
collaboration points:
1. **Cache mutations** Delete, exclude, rename, and bulk delete operations are
channelled through `ModelManagementHandler`. The handler delegates to
`ModelLifecycleService` or `MetadataSyncService`, and the scanner cache is
mutated in-place before the handler returns. The accompanying tests assert
that `scanner._cache.raw_data` and `scanner._hash_index` stay in sync after
each mutation.
2. **Preview updates** `PreviewAssetService.replace_preview` writes the new
asset, `MetadataSyncService` persists the JSON metadata, and
`scanner.update_preview_in_cache` mirrors the change. The handler returns
the static URL produced by `config.get_preview_static_url`, keeping browser
clients in lockstep with disk state.
3. **Download progress** `DownloadCoordinator.schedule_download` generates the
download identifier, registers a WebSocket progress callback, and caches the
latest numeric progress via `WebSocketManager`. Both `download_model`
responses and `/download-progress/{id}` polling read from the same cache to
guarantee consistent progress reporting across transports.
## Extending the stack
To add a new shared route:
1. Declare it in `COMMON_ROUTE_DEFINITIONS` using a unique handler name.
2. Implement the corresponding coroutine on one of the handlers inside
`ModelHandlerSet` (or introduce a new handler class when the concern does not
fit existing ones).
3. Inject additional dependencies in `BaseModelRoutes._create_handler_set` by
wiring services or use cases through the constructor parameters.
Model-specific routes should continue to be registered inside the subclass
implementation of `setup_specific_routes`, reusing the shared registrar where
possible.

View File

@@ -0,0 +1,34 @@
# Multi-Library Management for Standalone Mode
## Requirements Summary
- **Independent libraries**: In standalone mode, users can maintain multiple libraries, where each library represents a distinct set of model folders (LoRAs, checkpoints, embeddings, etc.). Only one library is active at any given time, but users need a fast way to switch between them.
- **Library-specific settings**: The fields that vary per library are `folder_paths`, `default_lora_root`, `default_checkpoint_root`, and `default_embedding_root` inside `settings.json`.
- **Persistent caches**: Every library must have its own SQLite persistent model cache so that metadata generated for one library does not leak into another.
- **Backward compatibility**: Existing single-library setups should continue to work. When no multi-library configuration is provided, the application should behave exactly as before.
## Proposed Design
1. **Library registry**
- Extend the standalone configuration to hold a list of libraries, each identified by a unique name.
- Each entry stores the folder path configuration plus any library-scoped metadata (e.g. creation time, display name).
- The active library key is stored separately to allow quick switching without rewriting the full config.
2. **Settings management**
- Update `settings_manager` to load and persist the library registry. When a library is activated, hydrate the in-memory settings object with that library's folder configuration.
- Provide helper methods for creating, renaming, and deleting libraries, ensuring validation for duplicate names and path collisions.
- Continue writing the active library settings to `settings.json` for compatibility, while storing the registry in a new section such as `libraries`.
3. **Persistent model cache**
- Derive the SQLite file path from the active library, e.g. `model_cache_<library>.sqlite` or a nested directory structure like `model_cache/<library>/models.sqlite`.
- Update `PersistentModelCache` so it resolves the database path dynamically whenever the active library changes. Ensure connections are closed before switching to avoid locking issues.
- Migrate existing single cache files by treating them as the default library's cache.
4. **Model scanning workflow**
- Modify `ModelScanner` and related services to react to library switches by clearing in-memory caches, re-reading folder paths, and rehydrating metadata from the library-specific SQLite cache.
- Provide API endpoints in standalone mode to list libraries, activate one, and trigger a rescan.
5. **UI/UX considerations**
- In the standalone UI, introduce a library selector component that surfaces available libraries and offers quick switching.
- Offer feedback when switching libraries (e.g. spinner while rescanning) and guard destructive actions with confirmation prompts.
## Implementation Notes
- **Data migration**: On startup, detect if the old `settings.json` structure is present. If so, create a default library entry using the current folder paths and point the active library to it.
- **Thread safety**: Ensure that any long-running scans are cancelled or awaited before switching libraries to prevent race conditions in cache writes.
- **Testing**: Add unit tests for the settings manager to cover library CRUD operations and cache path resolution. Include integration tests that simulate switching libraries and verifying that the correct models are loaded.
- **Documentation**: Update user guides to explain how to define libraries, switch between them, and where the new cache files are stored.
- **Extensibility**: Keep the design open to future per-library settings (e.g. auto-refresh intervals, metadata overrides) by storing library data as objects instead of flat maps.

View File

@@ -0,0 +1,89 @@
# Recipe route architecture
The recipe routing stack now mirrors the modular model route design. HTTP
bindings, controller wiring, handler orchestration, and business rules live in
separate layers so new behaviours can be added without re-threading the entire
feature. The diagram below outlines the flow for a typical request:
```mermaid
graph TD
subgraph HTTP
A[RecipeRouteRegistrar] -->|binds| B[RecipeRoutes controller]
end
subgraph Application
B --> C[RecipeHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Services / scanners]
end
subgraph Side Effects
F1 --> G1[Cache & fingerprint index]
F1 --> G2[Metadata files]
F1 --> G3[Temporary shares]
end
```
## Layer responsibilities
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/recipe_route_registrar.py` | Declarative list of every recipe endpoint and helper methods that bind them to an `aiohttp` application. |
| Controller | `py/routes/base_recipe_routes.py`, `py/routes/recipe_routes.py` | Lazily resolves scanners/clients from the service registry, wires shared templates/i18n, instantiates `RecipeHandlerSet`, and exposes a `{handler_name: coroutine}` mapping for the registrar. |
| Handler set | `py/routes/handlers/recipe_handlers.py` | Thin HTTP adapters grouped by concern (page view, listings, queries, mutations, sharing). They normalise responses and translate service exceptions into HTTP status codes. |
| Services & scanners | `py/services/recipes/*.py`, `py/services/recipe_scanner.py`, `py/services/service_registry.py` | Concrete business logic: metadata parsing, persistence, sharing, fingerprint/index maintenance, and cache refresh. |
## Handler responsibilities & invariants
`RecipeHandlerSet` flattens purpose-built handler objects into the callables the
registrar binds. Each handler is responsible for a narrow concern and enforces a
set of invariants before returning:
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `RecipePageView` | `/loras/recipes` | `SettingsManager`, `server_i18n`, Jinja environment, recipe scanner getter | Template rendered with `is_initializing` flag when caches are still warming; i18n filter registered exactly once per environment instance. |
| `RecipeListingHandler` | `/api/lm/recipes`, `/api/lm/recipe/{id}` | `recipe_scanner.get_paginated_data`, `recipe_scanner.get_recipe_by_id` | Listings respect pagination and search filters; every item receives a `file_url` fallback even when metadata is incomplete; missing recipes become HTTP 404. |
| `RecipeQueryHandler` | Tag/base-model stats, syntax, LoRA lookups | Recipe scanner cache, `format_recipe_file_url` helper | Cache snapshots are reused without forcing refresh; duplicate lookups collapse groups by fingerprint; syntax lookups return helpful errors when LoRAs are absent. |
| `RecipeManagementHandler` | Save, update, reconnect, bulk delete, widget ingest | `RecipePersistenceService`, `RecipeAnalysisService`, recipe scanner | Persistence results propagate HTTP status codes; fingerprint/index updates flow through the scanner before returning; validation errors surface as HTTP 400 without touching disk. |
| `RecipeAnalysisHandler` | Uploaded/local/remote analysis | `RecipeAnalysisService`, `civitai_client`, recipe scanner | Unsupported content types map to HTTP 400; download errors (`RecipeDownloadError`) are not retried; every response includes a `loras` array for client compatibility. |
| `RecipeSharingHandler` | Share + download | `RecipeSharingService`, recipe scanner | Share responses provide a stable download URL and filename; expired shares surface as HTTP 404; downloads stream via `web.FileResponse` with attachment headers. |
## Use case boundaries
The dedicated services encapsulate long-running work so handlers stay thin.
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `RecipeAnalysisService` | `analyze_uploaded_image`, `analyze_remote_image`, `analyze_local_image`, `analyze_widget_metadata` | `ExifUtils`, `RecipeParserFactory`, downloader factory, optional metadata collector/processor | Normalises missing/invalid payloads into `RecipeValidationError`; generates consistent fingerprint data to keep duplicate detection stable; temporary files are cleaned up after every analysis path. |
| `RecipePersistenceService` | `save_recipe`, `delete_recipe`, `update_recipe`, `reconnect_lora`, `bulk_delete`, `save_recipe_from_widget` | `ExifUtils`, recipe scanner, card preview sizing constants | Writes images/JSON metadata atomically; updates scanner caches and hash indices before returning; recalculates fingerprints whenever LoRA assignments change. |
| `RecipeSharingService` | `share_recipe`, `prepare_download` | `tempfile`, recipe scanner | Copies originals to TTL-managed temp files; metadata lookups re-use the scanner; expired shares trigger cleanup and `RecipeNotFoundError`. |
## Maintaining critical invariants
* **Cache updates** Mutations (`save`, `delete`, `bulk_delete`, `update`) call
back into the recipe scanner to mutate the in-memory cache and fingerprint
index before returning a response. Tests assert that these methods are invoked
even when stubbing persistence.
* **Fingerprint management** `RecipePersistenceService` recomputes
fingerprints whenever LoRA metadata changes and duplicate lookups use those
fingerprints to group recipes. Handlers bubble the resulting IDs so clients
can merge duplicates without an extra fetch.
* **Metadata synchronisation** Saving or reconnecting a recipe updates the
JSON sidecar, refreshes embedded metadata via `ExifUtils`, and instructs the
scanner to resort its cache. Sharing relies on this metadata to generate
filenames and ensure downloads stay in sync with on-disk state.
## Extending the stack
1. Declare the new endpoint in `ROUTE_DEFINITIONS` with a unique handler name.
2. Implement the coroutine on an existing handler or introduce a new handler
class inside `py/routes/handlers/recipe_handlers.py` when the concern does
not fit existing ones.
3. Wire additional collaborators inside
`BaseRecipeRoutes._create_handler_set` (inject new services or factories) and
expose helper getters on the handler owner if the handler needs to share
utilities.
Integration tests in `tests/routes/test_recipe_routes.py` exercise the listing,
mutation, analysis-error, and sharing paths end-to-end, ensuring the controller
and handler wiring remains valid as new capabilities are added.

View File

@@ -0,0 +1,46 @@
# Custom Priority Tag Format Proposal
To support user-defined priority tags with flexible aliasing across different model types, the configuration will be stored as editable strings. The format balances readability with enough structure for parsing on both the backend and frontend.
## Format Overview
- Each model type is declared on its own line: `model_type: entries`.
- Entries are comma-separated and ordered by priority from highest to lowest.
- An entry may be a single canonical tag (e.g., `realistic`) or a canonical tag with aliases.
- Canonical tags define the final folder name that should be used when matching that entry.
- Aliases are enclosed in parentheses and separated by `|` (vertical bar).
- All matching is case-insensitive; stored canonical names preserve the user-specified casing for folder creation and UI suggestions.
### Grammar
```
priority-config := model-config { "\n" model-config }
model-config := model-type ":" entry-list
model-type := <identifier without spaces>
entry-list := entry { "," entry }
entry := canonical [ "(" alias { "|" alias } ")" ]
canonical := <tag text without parentheses or commas>
alias := <tag text without parentheses, commas, or pipes>
```
Examples:
```
lora: celebrity(celeb|celebrity), stylized, character(char)
checkpoint: realistic(realism|realistic), anime(anime-style|toon)
embedding: face, celeb(celebrity|celeb)
```
## Parsing Notes
- Whitespace around separators is ignored to make manual editing more forgiving.
- Duplicate canonical tags within the same model type collapse to a single entry; the first definition wins.
- Aliases map to their canonical tag. When generating folder names, the canonical form is used.
- Tags that do not match any alias or canonical entry fall back to the first tag in the model's tag list, preserving current behavior.
## Usage
- **Backend:** Convert each model type's string into an ordered list of canonical tags with alias sets. During path generation, iterate by priority order and match tags against both canonical names and their aliases.
- **Frontend:** Surface canonical tags as suggestions, optionally displaying aliases in tooltips or secondary text. Input validation should warn about duplicate aliases within the same model type.
This format allows users to customize priority tag handling per model type while keeping editing simple and avoiding proliferation of folder names through alias normalization.

View File

@@ -0,0 +1,544 @@
# DOMWidget Development Guide
This document provides a comprehensive guide for developing custom DOMWidgets in ComfyUI using Vanilla JavaScript. DOMWidgets allow you to embed standard HTML elements (div, video, canvas, input, etc.) into ComfyUI nodes while benefitting from the frontend's automatic layout and zoom management.
## 1. Core Concepts
In ComfyUI, a `DOMWidget` extends the default LiteGraph Canvas rendering logic. It maintains an HTML layer on top of the Canvas, making complex interactions and media displays significantly easier to implement than pure Canvas drawing.
### Key APIs
* **`app.registerExtension`**: The entry point for registering extensions.
* **`getCustomWidgets`**: A hook for defining new widget types associated with specific input types.
* **`node.addDOMWidget`**: The core method to add HTML elements to a node.
---
## 2. Basic Structure
A standard custom DOMWidget extension typically follows this structure:
```javascript
import { app } from "../../scripts/app.js";
app.registerExtension({
name: "My.Custom.Extension",
async getCustomWidgets() {
return {
// Define a new widget type named "MY_WIDGET_TYPE"
MY_WIDGET_TYPE(node, inputName, inputData, app) {
// 1. Create the HTML element
const container = document.createElement("div");
container.innerHTML = "Hello <b>DOMWidget</b>!";
// 2. Setup styles (Optional but recommended)
container.style.color = "white";
container.style.backgroundColor = "#222";
container.style.padding = "5px";
// 3. Add the DOMWidget and return the result
const widget = node.addDOMWidget(inputName, "MY_WIDGET_TYPE", container, {
// Configuration options
getValue() {
return container.innerText;
},
setValue(v) {
container.innerText = v;
}
});
// 4. Return in the standard format
return { widget };
}
};
}
});
```
---
## ComfyUI Dual Rendering Modes
ComfyUI frontend supports two rendering modes:
| Mode | Description | DOM Structure |
| :--- | :--- | :--- |
| **Canvas Mode** | Traditional rendering where widgets are rendered on top of canvas using absolute positioning | Uses `.dom-widget` class on containers |
| **Vue DOM Mode** | New rendering mode where nodes and widgets are rendered as Vue components | Uses `.lg-node-widget` class on containers with dynamic IDs (e.g., `v-1-0`) |
### Mode Switching
The frontend switches between modes via `LiteGraph.vueNodesMode` boolean:
- `LiteGraph.vueNodesMode = true` → Vue DOM Mode
- `LiteGraph.vueNodesMode = false` → Canvas Mode
**Key Behavior**: Mode switching triggers DOM re-rendering WITHOUT page reload. Widget elements are destroyed and recreated, so any event listeners or references to old DOM elements become invalid.
### Testing Mode Switches via Chrome DevTools MCP
```javascript
// Trigger render mode change
LiteGraph.vueNodesMode = !LiteGraph.vueNodesMode;
// Force canvas redraw (optional but helps trigger re-render)
if (app.canvas) {
app.canvas.draw(true, true);
}
```
### Development Notes
When implementing widgets that attach event listeners or maintain external references:
1. **Use `node.onRemoved`** to clean up when node is deleted
2. **Detect DOM changes** by checking if widget input element is still in document: `document.body.contains(inputElement)`
3. **Poll for mode changes** by watching `LiteGraph.vueNodesMode` and re-initializing when it changes
4. **Use `loadedGraphNode` hook** for initial setup (guarantees DOM is fully rendered)
---
## 3. The `addDOMWidget` API
```javascript
node.addDOMWidget(name, type, element, options)
```
### Parameters
1. **`name`**: The internal name of the widget (usually matches the input name).
2. **`type`**: The type identifier for the widget.
3. **`element`**: The actual HTMLElement to embed.
4. **`options`**: (Object) Configuration for lifecycle, sizing, and persistence.
### Common `options` Fields
| Field | Type | Description |
| :--- | :--- | :--- |
| `getValue` | `Function` | Defines how to retrieve the widget's value for serialization. |
| `setValue` | `Function` | Defines how to restore the widget's state from workflow data. |
| `getMinHeight` | `Function` | Returns the minimum height in pixels. |
| `getHeight` | `Function` | Returns the preferred height (supports numbers or percentage strings like `"50%"`). |
| `onResize` | `Function` | Callback triggered when the widget is resized. |
| `hideOnZoom`| `Boolean` | Whether to hide the DOM element when zoomed out to improve performance (default: `true`). |
| `selectOn` | `string[]` | Events on the element that should trigger node selection (default: `['focus', 'click']`). |
---
## 4. Size Control
Custom DOMWidgets must actively inform the parent Node of their size requirements to ensure the Node layout is calculated correctly and connection wires remain aligned.
### 4.1 Core Mechanism
Whether in Canvas Mode or Vue Mode, the underlying logic model (`LGraphNode`) calls the widget's `computeLayoutSize` method to determine dimensions. This logic is used to calculate the Node's total size and the position of input/output slots.
### 4.2 Controlling Height
It is recommended to use the `options` parameter to define height behavior.
**Performance Note:** providing `getMinHeight` and `getHeight` via `options` allows the system to skip expensive DOM measurements (`getComputedStyle`) during rendering loop. This significantly improves performance and prevents FPS drops during node resizing.
**Method 1: Using `options` (Recommended)**
```javascript
const widget = node.addDOMWidget("MyWidget", "custom", element, {
// Specify minimum height in pixels
getMinHeight: () => 150,
// Or specify preferred height (pixels or percentage string)
// getHeight: () => "50%",
});
```
**Method 2: Using CSS Variables**
You can also set specific CSS variables on the root element:
```javascript
element.style.setProperty("--comfy-widget-min-height", "150px");
// or --comfy-widget-height
```
### 4.3 Controlling Width
By default, a DOMWidget's width automatically stretches to fit the Node's width (which is determined by the Title or other Input Slots).
If you must **force the Node to be wider** to accommodate your widget, you need to override the widget instance's `computeLayoutSize` method:
```javascript
const widget = node.addDOMWidget("WideWidget", "custom", element);
// Override the default layout calculation
widget.computeLayoutSize = (targetNode) => {
return {
minHeight: 150, // Must return height
minWidth: 300 // Force the Node to be at least 300px wide
};
};
```
### 4.4 Dynamic Resizing
If your widget's content changes dynamically (e.g., expanding sections, loading images, or CSS changes), the DOM element will resize, but the Canvas-rendered Node background and Slots will not automatically follow. You must manually trigger a synchronization.
**The Update Sequence:**
Whenever the **actual rendering height** of your DOM element changes, execute the following "three-step combo":
```javascript
// 1. Calculate the new optimal size for the node based on current widget requirements
const newSize = node.computeSize();
// 2. Apply the new size to the node model (updates bounding box and slot positions)
node.setSize(newSize);
// 3. Mark the canvas as dirty to trigger a redraw in the next animation frame
node.setDirtyCanvas(true, true);
```
**Common Scenarios:**
| Scenario | Actual Height Change? | Update Required? |
| :--- | :--- | :--- |
| **Expand/Collapse content** | **Yes** | ✅ **Yes**. Prevents widget from overflowing node boundaries. |
| **Image/Video finished loading** | **Yes** | ✅ **Yes**. Initial height might be 0 until the media loads. |
| **Changing `minHeight`** | **Maybe** | ❓ **Only if** the change causes the element's actual height to shift. |
| **Changing font size/styles** | **Yes** | ✅ **Yes**. Text reflow often changes the total height. |
| **User dragging node corner** | **Yes** | ❌ **No**. LiteGraph handles this internally. |
---
## 5. State Persistence (Serialization)
### 5.1 Default Behavior
DOMWidgets have **serialization enabled** by default (`serialize` property is `true`).
* **Saving**: ComfyUI attempts to read the widget's value to save into the Workflow file.
* **Loading**: ComfyUI reads the value from the Workflow file and assigns it to the widget.
### 5.2 Custom Serialization
To make persistence work effectively (saving internal DOM state and restoring it), you must implement `getValue` and `setValue` in the `options`:
* **`getValue`**: Returns the state to be saved (Number, String, or Object).
* **`setValue`**: Receives the restored value and updates the DOM element.
**Example:**
```javascript
const inputEl = document.createElement("input");
const widget = node.addDOMWidget("MyInput", "custom", inputEl, {
// 1. Called during Save
getValue: () => {
return inputEl.value;
},
// 2. Called during Load or Copy/Paste
setValue: (value) => {
inputEl.value = value || "";
}
});
// Optional: Listen for changes to update widget.value immediately
inputEl.addEventListener("change", () => {
widget.value = inputEl.value; // Triggers callbacks
});
```
### 5.3 The Restoration Mechanism (`configure`)
* **`configure(data)`**: When a Workflow is loaded, `LGraphNode` calls its `configure(data)` method.
* **`setValue` Chain**: During `configure`, the Node iterates over the saved `widgets_values` array and assigns each value (`widget.value = savedValue`). For DOMWidgets, this assignment triggers the `setValue` callback defined in your options.
Therefore, `options.setValue` is the critical hook for restoring widget state.
### 5.4 Disabling Serialization
If your widget is purely for display (e.g., a real-time monitor or generated chart) and doesn't need to save state, disable serialization to reduce workflow file size.
**Note**: You cannot set this via `options`. You must modify the widget instance directly.
```javascript
const widget = node.addDOMWidget("DisplayOnly", "custom", element);
widget.serialize = false; // Explicitly disable
```
---
## 6. Lifecycle & Events
### 6.1 `onResize`
When the Node size changes (e.g., user drags the corner), the widget can receive a notification via `options`:
```javascript
const widget = node.addDOMWidget("ResizingWidget", "custom", element, {
onResize: (w) => {
// 'w' is the widget instance
// Adjust internal DOM layout here if necessary
console.log("Widget resized");
}
});
```
### 6.2 Construction & Mounting
* **Construction**: Occurs immediately when `addDOMWidget` is called.
* **Mounting**:
* **Canvas Mode**: Appended to `.dom-widget-container` via `DomWidget.vue`.
* **Vue Mode**: Appended inside the Node component via `WidgetDOM.vue`.
* **Caution**: When `addDOMWidget` returns, the element may not be in the `document.body` yet. If you need to access layout properties like `getBoundingClientRect`, use `setTimeout` or wait for the first `onResize`.
### 6.3 Cleanup
If you create external references (like `setInterval` or global event listeners), ensure you clean them up using `node.onRemoved`:
```javascript
node.onRemoved = function() {
clearInterval(myInterval);
// Call original onRemoved if it existed
};
```
---
## 7. Styling & Best Practices
### 7.1 Styling
Since DOMWidgets are placed in absolute positioned containers or managed by Vue, ensure your container handles sizing gracefully:
```javascript
container.style.width = "100%";
container.style.boxSizing = "border-box";
```
### 7.2 Path References
When importing `app`, adjust the path based on your extension's folder depth. Typically:
`import { app } from "../../scripts/app.js";`
### 7.3 Security
If setting `innerHTML` dynamically, ensure the content is sanitized or trusted to prevent XSS attacks.
### 7.4 UI Constraints for ComfyUI Custom Node Widgets
When developing DOMWidgets as internal UI widgets for ComfyUI custom nodes, keep the following constraints in mind:
#### 7.4.1 Minimize Vertical Space
ComfyUI nodes are often displayed in a compact graph view with many nodes visible simultaneously. Avoid excessive vertical spacing that could clutter the workspace.
- Keep layouts compact and efficient
- Use appropriate padding and margins (4-8px typically)
- Stack related controls vertically but avoid unnecessary spacing
#### 7.4.2 Avoid Dynamic Height Changes
Dynamic height changes (expand/collapse sections, showing/hiding content) can cause node layout recalculations and affect connection wire positioning.
- Prefer static layouts over expandable/collapsible sections
- Use tooltips or overlays for additional information instead
- If dynamic height is unavoidable, manually trigger layout updates (see Section 4.4)
#### 7.4.3 Keep UI Simple and Intuitive
As internal widgets for ComfyUI custom nodes, the UI should be accessible to users without technical implementation details.
- Use clear, user-friendly terminology (avoid "frontend/backend roll" in favor of "fixed/always randomize")
- Focus on user intent rather than implementation details
- Avoid complex interactions that may confuse users
#### 7.4.4 Forward Middle Mouse Events to Canvas
By default, when a DOM widget receives pointer events (e.g., mouse clicks, drags), these events are captured by the widget and not forwarded to the ComfyUI canvas. This prevents users from panning the workflow using the middle mouse button when the cursor is over a DOM widget.
To enable workflow panning over your widget, you should forward middle mouse events (button 1) to the canvas using the `forwardMiddleMouseToCanvas` utility function:
```javascript
import { forwardMiddleMouseToCanvas } from "./utils.js";
// In your widget creation function
const container = document.createElement("div");
container.style.width = "100%";
container.style.height = "100%";
// ... other styles ...
// Forward middle mouse events to canvas for panning
forwardMiddleMouseToCanvas(container);
const widget = node.addDOMWidget(name, type, container, { ... });
```
The `forwardMiddleMouseToCanvas` function:
- Forwards `pointerdown` events with button 1 (middle mouse button) to `app.canvas.processMouseDown`
- Forwards `pointermove` events while middle mouse button is pressed to `app.canvas.processMouseMove`
- Forwards `pointerup` events with button 1 to `app.canvas.processMouseUp`
This allows users to pan the workflow canvas even when their mouse cursor is hovering over your DOM widget.
---
## 8. Event Handling in Vue DOM Render Mode
ComfyUI frontend supports two rendering modes for nodes:
- **Legacy Canvas Mode**: Traditional rendering where widgets are rendered on top of the canvas using absolute positioning
- **Vue DOM Render Mode**: New rendering mode where nodes and widgets are rendered as Vue components
In Vue DOM render mode, event handling works differently. The frontend uses `useCanvasInteractions` composable to manage event forwarding to the canvas. This can cause custom event handlers in your widgets (e.g., mouse wheel for sliders, custom drag operations) to be intercepted by the canvas.
### 8.1 Wheel Event Handling
By default in Vue DOM render mode, wheel events on widgets may be forwarded to the canvas for workflow zoom, overriding your custom wheel handlers (e.g., adjusting slider values with mouse wheel).
To fix this, use the `data-capture-wheel="true"` attribute on elements that should capture wheel events:
```vue
<!-- Vue component template -->
<div class="my-slider" data-capture-wheel="true" @wheel="onWheel">
<!-- Slider content -->
</div>
<script setup lang="ts">
const onWheel = (event: WheelEvent) => {
event.preventDefault()
// Custom wheel handling logic here
}
</script>
```
**How it works:**
- ComfyUI's `useCanvasInteractions.ts` checks `target?.closest('[data-capture-wheel="true"]')` before forwarding wheel events
- If an element (or its ancestor) has this attribute, wheel events are not forwarded to canvas
- Your custom `@wheel` handler will work as expected
**Granular control:**
- Apply `data-capture-wheel="true"` to specific interactive elements (e.g., sliders, scrollable areas)
- Widget container without this attribute will allow workflow zoom when wheel is used elsewhere
- This allows users to both: adjust widget values with wheel, and zoom workflow with wheel in widget's non-interactive areas
**Example from DualRangeSlider.vue:**
```vue
<template>
<div
class="dual-range-slider"
:class="{ disabled, 'is-dragging': dragging !== null }"
data-capture-wheel="true"
@wheel="onWheel"
>
<!-- Slider tracks and handles -->
</div>
</template>
```
### 8.2 Pointer Event Handling
In Vue DOM render mode, pointer events (click, drag, etc.) may also be captured by the canvas system. For custom drag operations:
1. **Use event modifiers to stop propagation:**
```vue
<div
@pointerdown.stop="startDrag"
@pointermove.stop="onDrag"
@pointerup.stop="stopDrag"
>
```
2. **Use pointer capture for reliable drag tracking:**
```javascript
const startDrag = (event: PointerEvent) => {
const target = event.currentTarget as HTMLElement
target.setPointerCapture(event.pointerId)
// ... drag initialization
}
const stopDrag = (event: PointerEvent) => {
const target = event.currentTarget as HTMLElement
target.releasePointerCapture(event.pointerId)
// ... drag cleanup
}
```
3. **Use `touch-action: none` CSS for touch devices:**
```css
.my-draggable {
touch-action: none;
}
```
### 8.3 Compatibility Checklist
Ensure your widget works in both rendering modes:
| Feature | Canvas Mode | Vue DOM Mode | Solution |
|---------|-------------|--------------|----------|
| Mouse wheel on sliders | Works by default | Needs `data-capture-wheel` | Add `data-capture-wheel="true"` to slider elements |
| Custom drag operations | Works with `stopPropagation()` | Needs `stopPropagation()` | Use `.stop` modifier and pointer capture |
| Middle mouse panning | Manual forwarding required | Manual forwarding required | Use `forwardMiddleMouseToCanvas()` |
| Workflow zoom on widget edges | Works by default | Works by default | No action needed (works by default) |
### 8.4 Testing Recommendations
Test your widget in both rendering modes:
1. Toggle between Canvas Mode and Vue DOM Mode in ComfyUI settings
2. Verify custom interactions (wheel, drag, etc.) work in both modes
3. Verify canvas interactions (zoom, pan) still work when cursor is over non-interactive widget areas
4. Test with touch devices if applicable
---
## 9. Complete Example: Text Counter
This example implements a simple widget that displays the character count of another text widget in the same node.
```javascript
import { app } from "../../scripts/app.js";
app.registerExtension({
name: "Comfy.TextCounter",
getCustomWidgets() {
return {
TEXT_COUNTER(node, inputName) {
const el = document.createElement("div");
Object.assign(el.style, {
background: "#222",
border: "1px solid #444",
padding: "8px",
borderRadius: "4px",
fontSize: "12px",
color: "#eee"
});
const label = document.createElement("span");
label.innerText = "Characters: 0";
el.appendChild(label);
const widget = node.addDOMWidget(inputName, "TEXT_COUNTER", el, {
getValue() { return ""; }, // Nothing to save
setValue(v) { }, // Nothing to restore
getMinHeight() { return 40; }
});
// Disable serialization for this display-only widget
widget.serialize = false;
// Custom method to update UI
widget.updateCount = (text) => {
label.innerText = `Characters: ${text.length}`;
};
return { widget };
}
};
},
nodeCreated(node) {
// Logic to link widgets after the node is initialized
if (node.comfyClass === "MyTextNode") {
const counterWidget = node.widgets.find(w => w.type === "TEXT_COUNTER");
const textWidget = node.widgets.find(w => w.name === "text");
if (counterWidget && textWidget) {
// Hook into the text widget's callback
const oldCallback = textWidget.callback;
textWidget.callback = function(v) {
if (oldCallback) oldCallback.apply(this, arguments);
counterWidget.updateCount(v);
};
}
}
}
});
```

View File

@@ -0,0 +1,51 @@
# Frontend DOM Fixture Strategy
This guide outlines how to reproduce the markup emitted by the Django templates while running Vitest in jsdom. The aim is to make it straightforward to write integration-style unit tests for managers and UI helpers without having to duplicate template fragments inline.
## Loading Template Markup
Vitest executes inside Node, so we can read the same HTML templates that ship with the extension:
1. Use the helper utilities from `tests/frontend/utils/domFixtures.js` to read files under the `templates/` directory.
2. Mount the returned markup into `document.body` (or any custom container) before importing the module under test so its query selectors resolve correctly.
```js
import { renderTemplate } from '../utils/domFixtures.js'; // adjust the relative path to your spec
beforeEach(() => {
renderTemplate('loras.html', {
dataset: { page: 'loras' }
});
});
```
The helper ensures the dataset is applied to the container, which mirrors how Django sets `data-page` in production.
## Working with Partial Components
Many features are implemented as template partials located under `templates/components/`. When a test only needs a fragment (for example, the progress panel or context menu markup), load the component file directly:
```js
const container = renderTemplate('components/progress_panel.html');
const progressPanel = container.querySelector('#progress-panel');
```
This pattern avoids hand-written fixture strings and keeps the tests aligned with the actual markup.
## Resetting Between Tests
The shared Vitest setup clears `document.body` and storage APIs before each test. If a suite adds additional DOM nodes outside of the body or needs to reset custom attributes mid-test, use `resetDom()` exported from `domFixtures.js`.
```js
import { resetDom } from '../utils/domFixtures.js';
afterEach(() => {
resetDom();
});
```
## Future Enhancements
- Provide typed helpers for injecting mock script tags (e.g., replicating ComfyUI globals).
- Compose higher-level fixtures that mimic specific pages (loras, checkpoints, recipes) once those managers receive dedicated suites.

View File

@@ -0,0 +1,44 @@
# LoRA & Checkpoints Filtering/Sorting Test Matrix
This matrix captures the scenarios that Phase 3 frontend tests should cover for the LoRA and Checkpoint managers. It focuses on how search, filter, sort, and duplicate badge toggles interact so future specs can share fixtures and expectations.
## Scope
- **Components**: `PageControls`, `FilterManager`, `SearchManager`, and `ModelDuplicatesManager` wiring invoked through `CheckpointsPageManager` and `LorasPageManager`.
- **Templates**: `templates/loras.html` and `templates/checkpoints.html` along with shared filter panel and toolbar partials.
- **APIs**: Requests issued through `baseModelApi.fetchModels` (via `resetAndReload`/`refreshModels`) and duplicates badge updates.
## Shared Setup Considerations
1. Render full page templates using `renderLorasPage` / `renderCheckpointsPage` helpers before importing modules so DOM queries resolve.
2. Stub storage helpers (`getStorageItem`, `setStorageItem`, `getSessionItem`, `setSessionItem`) to observe persistence behavior without mutating real storage.
3. Mock `sidebarManager` to capture refresh calls triggered after sort/filter actions.
4. Provide fake API implementations exposing `resetAndReload`, `refreshModels`, `fetchFromCivitai`, `toggleBulkMode`, and `clearCustomFilter` so control events remain asynchronous but deterministic.
5. Supply a minimal `ModelDuplicatesManager` mock exposing `toggleDuplicateMode`, `checkDuplicatesCount`, and `updateDuplicatesBadgeAfterRefresh` to validate duplicate badge wiring.
## Scenario Matrix
| ID | Feature | Scenario | LoRAs Expectations | Checkpoints Expectations | Notes |
| --- | --- | --- | --- | --- | --- |
| F-01 | Search filter | Typing a query updates `pageState.filters.search`, persists to session, and triggers `resetAndReload` on submit | Validate `SearchManager` writes query and reloads via API stub; confirm LoRA cards pass query downstream | Same as LoRAs | Cover `enter` press and clicking search icon |
| F-02 | Tag filter | Selecting a tag chip cycles include ➜ exclude ➜ clear, updates storage, and reloads results | Tag state stored under `filters.tags[tagName] = 'include'|'exclude'`; `FilterManager.applyFilters` persists and triggers `resetAndReload(true)` | Same; ensure base model tag set is scoped to checkpoints dataset | Include removal path |
| F-03 | Base model filter | Toggling base model checkboxes updates `filters.baseModel`, persists, and reloads | Ensure only LoRA-supported models show; toggle multi-select | Ensure SDXL/Flux base models appear as expected | Capture UI state restored from storage on next init |
| F-04 | Favorites-only | Clicking favorites toggle updates session flag and calls `resetAndReload(true)` | Button gains `.active` class and API called | Same | Verify duplicates badge refresh when active |
| F-05 | Sort selection | Changing sort select saves preference (legacy + new format) and reloads | Confirm `PageControls.saveSortPreference` invoked with option and API called | Same with checkpoints-specific defaults | Cover `convertLegacySortFormat` branch |
| F-06 | Filter persistence | Re-initializing manager loads stored filters/sort and updates DOM | Filters pre-populate chips/checkboxes; favorites state restored | Same | Requires simulating repeated construction |
| F-07 | Combined filters | Applying search + tag + base model yields aggregated query params for fetch | Assert API receives merged filter payload | Same | Validate toast messaging for active filters |
| F-08 | Clearing filters | Using "Clear filters" resets state, storage, and reloads list | `FilterManager.clearFilters` empties `filters`, removes active class, shows toast | Same | Ensure favorites-only toggle unaffected |
| F-09 | Duplicate badge toggle | Pressing "Find duplicates" toggles duplicate mode and updates badge counts post-refresh | `ModelDuplicatesManager.toggleDuplicateMode` invoked and badge refresh called after API rebuild | Same plus checkpoint-specific duplicate badge dataset | Connects to future duplicate-specific specs |
| F-10 | Bulk actions menu | Opening bulk dropdown keeps filters intact and closes on outside click | Validate dropdown class toggling and no unintended reload | Same | Guard against regression when dropdown interacts with filters |
## Automation Coverage Status
- ✅ F-01 Search filter, F-02 Tag filter, F-03 Base model filter, F-04 Favorites-only toggle, F-05 Sort selection, and F-09 Duplicate badge toggle are covered by `tests/frontend/components/pageControls.filtering.test.js` for both LoRA and checkpoint pages.
- ⏳ F-06 Filter persistence, F-07 Combined filters, F-08 Clearing filters, and F-10 Bulk actions remain to be automated alongside upcoming bulk mode refinements.
## Coverage Gaps & Follow-Ups
- Write Vitest suites that exercise the matrix for both managers, sharing fixtures through page helpers to avoid duplication.
- Capture API parameter assertions by inspecting `baseModelApi.fetchModels` mocks rather than relying solely on state mutations.
- Add regression cases for legacy storage migrations (old filter keys) once fixtures exist for older payloads.
- Extend duplicate badge coverage with scenarios where `checkDuplicatesCount` signals zero duplicates versus pending calculations.

View File

@@ -0,0 +1,33 @@
# Frontend Automation Testing Roadmap
This roadmap tracks the planned rollout of automated testing for the ComfyUI LoRA Manager frontend. Each phase builds on the infrastructure introduced in this change set and records progress so future contributors can quickly identify the next tasks.
## Phase Overview
| Phase | Goal | Primary Focus | Status | Notes |
| --- | --- | --- | --- | --- |
| Phase 0 | Establish baseline tooling | Add Node test runner, jsdom environment, and seed smoke tests | ✅ Complete | Vitest + jsdom configured, example state tests committed |
| Phase 1 | Cover state management logic | Unit test selectors, derived data helpers, and storage utilities under `static/js/state` and `static/js/utils` | ✅ Complete | Storage helpers and state selectors now exercised via deterministic suites |
| Phase 2 | Test AppCore orchestration | Simulate page bootstrapping, infinite scroll hooks, and manager registration using JSDOM DOM fixtures | ✅ Complete | AppCore initialization + page feature suites now validate manager wiring, infinite scroll hooks, and onboarding gating |
| Phase 3 | Validate page-specific managers | Add focused suites for `loras`, `checkpoints`, `embeddings`, and `recipes` managers covering filtering, sorting, and bulk actions | ✅ Complete | LoRA/checkpoint suites expanded; embeddings + recipes managers now covered with initialization, filtering, and duplicate workflows |
| Phase 4 | Interaction-level regression tests | Exercise template fragments, modals, and menus to ensure UI wiring remains intact | ✅ Complete | Vitest DOM suites cover NSFW selector, recipe modal editing, and global context menus |
| Phase 5 | Continuous integration & coverage | Integrate frontend tests into CI workflow and track coverage metrics | ✅ Complete | CI workflow runs Vitest and aggregates V8 coverage into `coverage/frontend` via a dedicated script |
## Next Steps Checklist
- [x] Expand unit tests for `storageHelpers` covering migrations and namespace behavior.
- [x] Document DOM fixture strategy for reproducing template structures in tests.
- [x] Prototype AppCore initialization test that verifies manager bootstrapping with stubbed dependencies.
- [x] Add AppCore page feature suite exercising context menu creation and infinite scroll registration via DOM fixtures.
- [x] Extend AppCore orchestration tests to cover manager wiring, bulk menu setup, and onboarding gating scenarios.
- [x] Add interaction regression suites for context menus and recipe modals to complete Phase 4.
- [x] Evaluate integrating coverage reporting once test surface grows (> 20 specs).
- [x] Create shared fixtures for the loras and checkpoints pages once dedicated manager suites are added.
- [x] Draft focused test matrix for loras/checkpoints manager filtering and sorting paths ahead of Phase 3.
- [x] Implement LoRAs manager filtering/sorting specs for scenarios F-01F-05 & F-09; queue remaining edge cases after duplicate/bulk flows stabilize.
- [x] Implement checkpoints manager filtering/sorting specs for scenarios F-01F-05 & F-09; cover remaining paths alongside bulk action work.
- [x] Implement checkpoints page manager smoke tests covering initialization and duplicate badge wiring.
- [x] Outline focused checkpoints scenarios (filtering, sorting, duplicate badge toggles) to feed into the shared test matrix.
- [ ] Add duplicate badge regression coverage for zero/pending states after API refreshes.
Maintaining this roadmap alongside code changes will make it easier to append new automated test tasks and update their progress.

28
docs/library-switching.md Normal file
View File

@@ -0,0 +1,28 @@
# Library Switching and Preview Routes
Library switching no longer requires restarting the backend. The preview
thumbnails shown in the UI are now served through a dynamic endpoint that
resolves files against the folders registered for the active library at request
time. This allows the multi-library flow to update model roots without touching
the aiohttp router, so previews remain available immediately after a switch.
## How the dynamic preview endpoint works
* `config.get_preview_static_url()` now returns `/api/lm/previews?path=<encoded>`
for any preview path. The raw filesystem location is URL encoded so that it
can be passed through the query string without leaking directory structure in
the route itself.【F:py/config.py†L398-L404】
* `PreviewRoutes` exposes the `/api/lm/previews` handler which validates the
decoded path against the directories registered for the current library. The
request is rejected if it falls outside those roots or if the file does not
exist.【F:py/routes/preview_routes.py†L5-L21】【F:py/routes/handlers/preview_handlers.py†L9-L48】
* `Config` keeps an up-to-date cache of allowed preview roots. Every time a
library is applied the cache is rebuilt using the declared LoRA, checkpoint
and embedding directories (including symlink targets). The validation logic
checks preview requests against this cache.【F:py/config.py†L51-L68】【F:py/config.py†L180-L248】【F:py/config.py†L332-L346】
Both the ComfyUI runtime (`LoraManager.add_routes`) and the standalone launcher
(`StandaloneLoraManager.add_routes`) register the new preview routes instead of
mounting a static directory per root. Switching libraries therefore works
without restarting the application, and preview URLs generated before or after a
switch continue to resolve correctly.【F:py/lora_manager.py†L21-L82】【F:standalone.py†L302-L315】

View File

@@ -0,0 +1,71 @@
# Priority Tags Configuration Guide
This guide explains how to tailor the tag priority order that powers folder naming and tag suggestions in the LoRA Manager. You only need to edit the comma-separated list of entries shown in the **Priority Tags** field for each model type.
## 1. Pick the Model Type
In the **Priority Tags** dialog you will find one tab per model type (LoRA, Checkpoint, Embedding). Select the tab you want to update; changes on one tab do not affect the others.
## 2. Edit the Entry List
Inside the textarea you will see a line similar to:
```
character, concept, style(toon|toon_style)
```
This entire line is the **entry list**. Replace it with your own ordered list.
### Entry Rules
Each entry is separated by a comma, in order from highest to lowest priority:
- **Canonical tag only:** `realistic`
- **Canonical tag with aliases:** `character(char|chars)`
Aliases live inside `()` and are separated with `|`. The canonical name is what appears in folder names and UI suggestions when any of the aliases are detected. Matching is case-insensitive.
## Use `{first_tag}` in Path Templates
When your path template contains `{first_tag}`, the app picks a folder name based on your priority list and the models own tags:
- It checks the priority list from top to bottom. If a canonical tag or any of its aliases appear in the model tags, that canonical name becomes the folder name.
- If no priority tags are found but the model has tags, the very first model tag is used.
- If the model has no tags at all, the folder falls back to `no tags`.
### Example
With a template like `/{model_type}/{first_tag}` and the priority entry list `character(char|chars), style(anime|toon)`:
| Model Tags | Folder Name | Why |
| --- | --- | --- |
| `["chars", "female"]` | `character` | `chars` matches the `character` alias, so the canonical wins. |
| `["anime", "portrait"]` | `style` | `anime` hits the `style` entry, so its canonical label is used. |
| `["portrait", "bw"]` | `portrait` | No priority match, so the first model tag is used. |
| `[]` | `no tags` | Nothing to match, so the fallback is applied. |
## 3. Save the Settings
After editing the entry list, press **Enter** to save. Use **Shift+Enter** whenever you need a new line. Clicking outside the field also saves automatically. A success toast confirms the update.
## Examples
| Goal | Entry List |
| --- | --- |
| Prefer people over styles | `character, portraits, style(anime\|toon)` |
| Group sci-fi variants | `sci-fi(scifi\|science_fiction), cyberpunk(cyber\|punk)` |
| Alias shorthand tags | `realistic(real\|realisim), photorealistic(photo_real)` |
## Tips
- Keep canonical names short and meaningful—they become folder names.
- Place the most important categories first; the first match wins.
- Avoid duplicate canonical names within the same list; only the first instance is used.
## Troubleshooting
- **Unexpected folder name?** Check that the canonical name you want is placed before other matches.
- **Alias not working?** Ensure the alias is inside parentheses and separated with `|`, e.g. `character(char|chars)`.
- **Validation error?** Look for missing parentheses or stray commas. Each entry must follow the `canonical(alias|alias)` pattern or just `canonical`.
With these basics you can quickly adapt Priority Tags to match your librarys organization style.

View File

@@ -0,0 +1,26 @@
# Backend Test Coverage Notes
## Pytest Execution
- Command: `python -m pytest`
- Result: All 283 collected tests passed in the current environment.
- Coverage tooling (``pytest-cov``/``coverage``) is unavailable in the offline sandbox, so line-level metrics could not be generated. The earlier attempt to install ``pytest-cov`` failed because the package index cannot be reached from the container.
## High-Priority Gaps to Address
### 1. Standalone server bootstrapping
* **Source:** [`standalone.py`](../../standalone.py)
* **Why it matters:** The standalone entry point wires together the aiohttp application, static asset routes, model-route registration, and configuration validation. None of these behaviours are covered by automated tests, leaving regressions in bootstrapping logic undetected.
* **Suggested coverage:** Add integration-style tests that instantiate `StandaloneServer`/`StandaloneLoraManager` with temporary settings and assert that routes (HTTP + websocket) are registered, configuration warnings fire for missing paths, and the mock ComfyUI shims behave as expected.
### 2. Model service registration factory
* **Source:** [`py/services/model_service_factory.py`](../../py/services/model_service_factory.py)
* **Why it matters:** The factory coordinates which model services and routes the API exposes, including error handling when unknown model types are requested. No current tests verify registration, memoization of route instances, or the logging path on failures.
* **Suggested coverage:** Unit tests that exercise `register_model_type`, `get_route_instance`, error branches in `get_service_class`/`get_route_class`, and `setup_all_routes` when a route setup raises. Use lightweight fakes to confirm the logger is called and state is cleared via `clear_registrations`.
### 3. Server-side i18n helper
* **Source:** [`py/services/server_i18n.py`](../../py/services/server_i18n.py)
* **Why it matters:** Template rendering relies on the `ServerI18nManager` to load locale JSON, perform key lookups, and format parameters. The fallback logic (dot-notation lookup, English fallbacks, placeholder substitution) is untested, so malformed locale files or regressions in placeholder handling would slip through.
* **Suggested coverage:** Tests that load fixture locale dictionaries, assert `set_locale` fallbacks, verify nested key resolution and placeholder substitution, and ensure missing keys return the original identifier.
## Next Steps
Prioritize creating focused unit tests around these modules, then re-run pytest once coverage tooling is available to confirm the new tests close the identified gaps.

Binary file not shown.

Before

Width:  |  Height:  |  Size: 669 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

Before

Width:  |  Height:  |  Size: 669 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

After

Width:  |  Height:  |  Size: 668 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

After

Width:  |  Height:  |  Size: 739 KiB

File diff suppressed because one or more lines are too long

1541
locales/de.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/en.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/es.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/fr.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/he.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/ja.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/ko.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/ru.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/zh-CN.json Normal file

File diff suppressed because it is too large Load Diff

1541
locales/zh-TW.json Normal file

File diff suppressed because it is too large Load Diff

2575
package-lock.json generated Normal file

File diff suppressed because it is too large Load Diff

15
package.json Normal file
View File

@@ -0,0 +1,15 @@
{
"name": "comfyui-lora-manager-frontend",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "node scripts/run_frontend_coverage.js"
},
"devDependencies": {
"jsdom": "^24.0.0",
"vitest": "^1.6.0"
}
}

View File

@@ -0,0 +1,12 @@
"""Project namespace package."""
# pytest's internal compatibility layer still imports ``py.path.local`` from the
# historical ``py`` dependency. Because this project reuses the ``py`` package
# name, we expose a minimal shim so ``py.path.local`` resolves to ``pathlib.Path``
# during test runs without pulling in the external dependency.
from pathlib import Path
from types import SimpleNamespace
path = SimpleNamespace(local=Path)
__all__ = ["path"]

View File

@@ -1,27 +1,89 @@
import os
import platform
import threading
from pathlib import Path
import folder_paths # type: ignore
from typing import List
from typing import Any, Dict, Iterable, List, Mapping, Optional, Set, Tuple
import logging
import sys
import json
import urllib.parse
import time
# Check if running in standalone mode
standalone_mode = 'nodes' not in sys.modules
from .utils.settings_paths import ensure_settings_file, get_settings_dir, load_settings_template
# Use an environment variable to control standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
logger = logging.getLogger(__name__)
def _normalize_folder_paths_for_comparison(
folder_paths: Mapping[str, Iterable[str]]
) -> Dict[str, Set[str]]:
"""Normalize folder paths for comparison across libraries."""
normalized: Dict[str, Set[str]] = {}
for key, values in folder_paths.items():
if isinstance(values, str):
candidate_values: Iterable[str] = [values]
else:
try:
candidate_values = iter(values)
except TypeError:
continue
normalized_values: Set[str] = set()
for value in candidate_values:
if not isinstance(value, str):
continue
stripped = value.strip()
if not stripped:
continue
normalized_values.add(os.path.normcase(os.path.normpath(stripped)))
if normalized_values:
normalized[key] = normalized_values
return normalized
def _normalize_library_folder_paths(
library_payload: Mapping[str, Any]
) -> Dict[str, Set[str]]:
"""Return normalized folder paths extracted from a library payload."""
folder_paths = library_payload.get("folder_paths")
if isinstance(folder_paths, Mapping):
return _normalize_folder_paths_for_comparison(folder_paths)
return {}
def _get_template_folder_paths() -> Dict[str, Set[str]]:
"""Return normalized folder paths defined in the bundled template."""
template_payload = load_settings_template()
if not template_payload:
return {}
folder_paths = template_payload.get("folder_paths")
if isinstance(folder_paths, Mapping):
return _normalize_folder_paths_for_comparison(folder_paths)
return {}
class Config:
"""Global configuration for LoRA Manager"""
def __init__(self):
self.templates_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'templates')
self.static_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'static')
self.i18n_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'locales')
# Path mapping dictionary, target to link mapping
self._path_mappings = {}
# Static route mapping dictionary, target to route mapping
self._route_mappings = {}
self._path_mappings: Dict[str, str] = {}
# Normalized preview root directories used to validate preview access
self._preview_root_paths: Set[Path] = set()
# Fingerprint of the symlink layout from the last successful scan
self._cached_fingerprint: Optional[Dict[str, object]] = None
self.loras_roots = self._init_lora_paths()
self.checkpoints_roots = None
self.unet_roots = None
@@ -29,46 +91,118 @@ class Config:
self.base_models_roots = self._init_checkpoint_paths()
self.embeddings_roots = self._init_embedding_paths()
# Scan symbolic links during initialization
self._scan_symbolic_links()
self._initialize_symlink_mappings()
if not standalone_mode:
# Save the paths to settings.json when running in ComfyUI mode
self.save_folder_paths_to_settings()
def save_folder_paths_to_settings(self):
"""Save folder paths to settings.json for standalone mode to use later"""
"""Persist ComfyUI-derived folder paths to the multi-library settings."""
try:
# Check if we're running in ComfyUI mode (not standalone)
# Load existing settings
settings_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'settings.json')
settings = {}
if os.path.exists(settings_path):
with open(settings_path, 'r', encoding='utf-8') as f:
settings = json.load(f)
# Update settings with paths
settings['folder_paths'] = {
'loras': self.loras_roots,
'checkpoints': self.checkpoints_roots,
'unet': self.unet_roots,
'embeddings': self.embeddings_roots,
}
# Add default roots if there's only one item and key doesn't exist
if len(self.loras_roots) == 1 and "default_lora_root" not in settings:
settings["default_lora_root"] = self.loras_roots[0]
if self.checkpoints_roots and len(self.checkpoints_roots) == 1 and "default_checkpoint_root" not in settings:
settings["default_checkpoint_root"] = self.checkpoints_roots[0]
ensure_settings_file(logger)
from .services.settings_manager import get_settings_manager
if self.embeddings_roots and len(self.embeddings_roots) == 1 and "default_embedding_root" not in settings:
settings["default_embedding_root"] = self.embeddings_roots[0]
# Save settings
with open(settings_path, 'w', encoding='utf-8') as f:
json.dump(settings, f, indent=2)
logger.info("Saved folder paths to settings.json")
settings_service = get_settings_manager()
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
default_library = libraries.get("default", {})
template_folder_paths = _get_template_folder_paths()
default_library_paths: Dict[str, Set[str]] = {}
if isinstance(default_library, Mapping):
default_library_paths = _normalize_library_folder_paths(default_library)
libraries_changed = False
if (
isinstance(default_library, Mapping)
and template_folder_paths
and default_library_paths == template_folder_paths
):
if "comfyui" in libraries:
try:
settings_service.delete_library("default")
libraries_changed = True
logger.info("Removed template 'default' library entry")
except Exception as delete_error:
logger.debug(
"Failed to delete template 'default' library: %s",
delete_error,
)
else:
try:
settings_service.rename_library("default", "comfyui")
libraries_changed = True
logger.info("Renamed template 'default' library to 'comfyui'")
except Exception as rename_error:
logger.debug(
"Failed to rename template 'default' library: %s",
rename_error,
)
if libraries_changed:
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
default_library = libraries.get("default", {})
target_folder_paths = {
'loras': list(self.loras_roots),
'checkpoints': list(self.checkpoints_roots or []),
'unet': list(self.unet_roots or []),
'embeddings': list(self.embeddings_roots or []),
}
normalized_target_paths = _normalize_folder_paths_for_comparison(target_folder_paths)
normalized_default_paths: Optional[Dict[str, Set[str]]] = None
if isinstance(default_library, Mapping):
normalized_default_paths = _normalize_library_folder_paths(default_library)
if (
not comfy_library
and default_library
and normalized_target_paths
and normalized_default_paths == normalized_target_paths
):
try:
settings_service.rename_library("default", "comfyui")
logger.info("Renamed legacy 'default' library to 'comfyui'")
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
except Exception as rename_error:
logger.debug(
"Failed to rename legacy 'default' library: %s", rename_error
)
default_lora_root = comfy_library.get("default_lora_root", "")
if not default_lora_root and len(self.loras_roots) == 1:
default_lora_root = self.loras_roots[0]
default_checkpoint_root = comfy_library.get("default_checkpoint_root", "")
if (not default_checkpoint_root and self.checkpoints_roots and
len(self.checkpoints_roots) == 1):
default_checkpoint_root = self.checkpoints_roots[0]
default_embedding_root = comfy_library.get("default_embedding_root", "")
if (not default_embedding_root and self.embeddings_roots and
len(self.embeddings_roots) == 1):
default_embedding_root = self.embeddings_roots[0]
metadata = dict(comfy_library.get("metadata", {}))
metadata.setdefault("display_name", "ComfyUI")
metadata["source"] = "comfyui"
settings_service.upsert_library(
"comfyui",
folder_paths=target_folder_paths,
default_lora_root=default_lora_root,
default_checkpoint_root=default_checkpoint_root,
default_embedding_root=default_embedding_root,
metadata=metadata,
activate=True,
)
logger.info("Updated 'comfyui' library with current folder paths")
except Exception as e:
logger.warning(f"Failed to save folder paths: {e}")
@@ -89,96 +223,398 @@ class Config:
logger.error(f"Error checking link status for {path}: {e}")
return False
def _normalize_path(self, path: str) -> str:
return os.path.normpath(path).replace(os.sep, '/')
def _get_symlink_cache_path(self) -> Path:
cache_dir = Path(get_settings_dir(create=True)) / "cache"
cache_dir.mkdir(parents=True, exist_ok=True)
return cache_dir / "symlink_map.json"
def _symlink_roots(self) -> List[str]:
roots: List[str] = []
roots.extend(self.loras_roots or [])
roots.extend(self.base_models_roots or [])
roots.extend(self.embeddings_roots or [])
return roots
def _build_symlink_fingerprint(self) -> Dict[str, object]:
roots = [self._normalize_path(path) for path in self._symlink_roots() if path]
unique_roots = sorted(set(roots))
# Fingerprint now only contains the root paths to avoid sensitivity to folder content changes.
return {"roots": unique_roots}
def _initialize_symlink_mappings(self) -> None:
start = time.perf_counter()
cache_loaded = self._load_persisted_cache_into_mappings()
if cache_loaded:
logger.info(
"Symlink mappings restored from cache in %.2f ms",
(time.perf_counter() - start) * 1000,
)
self._rebuild_preview_roots()
# Only rescan if target roots have changed.
# This is stable across file additions/deletions.
current_fingerprint = self._build_symlink_fingerprint()
cached_fingerprint = self._cached_fingerprint
if cached_fingerprint and current_fingerprint == cached_fingerprint:
return
logger.info("Symlink root paths changed; rescanning symbolic links")
self.rebuild_symlink_cache()
logger.info(
"Symlink mappings rebuilt and cached in %.2f ms",
(time.perf_counter() - start) * 1000,
)
def rebuild_symlink_cache(self) -> None:
"""Force a fresh scan of all symbolic links and update the persistent cache."""
self._scan_symbolic_links()
self._save_symlink_cache()
self._rebuild_preview_roots()
def _load_persisted_cache_into_mappings(self) -> bool:
"""Load the symlink cache and store its fingerprint for comparison."""
cache_path = self._get_symlink_cache_path()
if not cache_path.exists():
return False
try:
with cache_path.open("r", encoding="utf-8") as handle:
payload = json.load(handle)
except Exception as exc:
logger.info("Failed to load symlink cache %s: %s", cache_path, exc)
return False
if not isinstance(payload, dict):
return False
cached_mappings = payload.get("path_mappings")
if not isinstance(cached_mappings, Mapping):
return False
# Store the cached fingerprint for comparison during initialization
self._cached_fingerprint = payload.get("fingerprint")
normalized_mappings: Dict[str, str] = {}
for target, link in cached_mappings.items():
if not isinstance(target, str) or not isinstance(link, str):
continue
normalized_mappings[self._normalize_path(target)] = self._normalize_path(link)
self._path_mappings = normalized_mappings
logger.info("Symlink cache loaded with %d mappings", len(self._path_mappings))
return True
def _save_symlink_cache(self) -> None:
cache_path = self._get_symlink_cache_path()
payload = {
"fingerprint": self._build_symlink_fingerprint(),
"path_mappings": self._path_mappings,
}
try:
with cache_path.open("w", encoding="utf-8") as handle:
json.dump(payload, handle, ensure_ascii=False, indent=2)
logger.debug("Symlink cache saved to %s with %d mappings", cache_path, len(self._path_mappings))
except Exception as exc:
logger.info("Failed to write symlink cache %s: %s", cache_path, exc)
def _scan_symbolic_links(self):
"""Scan all symbolic links in LoRA, Checkpoint, and Embedding root directories"""
for root in self.loras_roots:
self._scan_directory_links(root)
start = time.perf_counter()
for root in self.base_models_roots:
self._scan_directory_links(root)
for root in self.embeddings_roots:
self._scan_directory_links(root)
# Reset mappings before rescanning to avoid stale entries
self._path_mappings.clear()
self._seed_root_symlink_mappings()
visited_dirs: Set[str] = set()
for root in self._symlink_roots():
self._scan_directory_links(root, visited_dirs)
logger.debug(
"Symlink scan finished in %.2f ms with %d mappings",
(time.perf_counter() - start) * 1000,
len(self._path_mappings),
)
def _scan_directory_links(self, root: str):
"""Recursively scan symbolic links in a directory"""
def _scan_directory_links(self, root: str, visited_dirs: Set[str]):
"""Iteratively scan directory symlinks to avoid deep recursion."""
try:
with os.scandir(root) as it:
for entry in it:
if self._is_link(entry.path):
target_path = os.path.realpath(entry.path)
if os.path.isdir(target_path):
self.add_path_mapping(entry.path, target_path)
self._scan_directory_links(target_path)
elif entry.is_dir(follow_symlinks=False):
self._scan_directory_links(entry.path)
except Exception as e:
logger.error(f"Error scanning links in {root}: {e}")
# Note: We only use realpath for the initial root if it's not already resolved
# to ensure we have a valid entry point.
root_real = self._normalize_path(os.path.realpath(root))
except OSError:
root_real = self._normalize_path(root)
if root_real in visited_dirs:
return
visited_dirs.add(root_real)
# Stack entries: (display_path, real_resolved_path)
stack: List[Tuple[str, str]] = [(root, root_real)]
while stack:
current_display, current_real = stack.pop()
try:
with os.scandir(current_display) as it:
for entry in it:
try:
# 1. High speed detection using dirent data (is_symlink)
is_link = entry.is_symlink()
# On Windows, is_symlink handles reparse points
if is_link:
# Only resolve realpath when we actually find a link
target_path = os.path.realpath(entry.path)
if not os.path.isdir(target_path):
continue
normalized_target = self._normalize_path(target_path)
self.add_path_mapping(entry.path, target_path)
if normalized_target in visited_dirs:
continue
visited_dirs.add(normalized_target)
stack.append((target_path, normalized_target))
continue
# 2. Process normal directories
if not entry.is_dir(follow_symlinks=False):
continue
# For normal directories, we avoid realpath() call by
# incrementally building the real path relative to current_real.
# This is safe because 'entry' is NOT a symlink.
entry_real = self._normalize_path(os.path.join(current_real, entry.name))
if entry_real in visited_dirs:
continue
visited_dirs.add(entry_real)
stack.append((entry.path, entry_real))
except Exception as inner_exc:
logger.debug(
"Error processing directory entry %s: %s", entry.path, inner_exc
)
except Exception as e:
logger.error(f"Error scanning links in {current_display}: {e}")
def add_path_mapping(self, link_path: str, target_path: str):
"""Add a symbolic link path mapping
target_path: actual target path
link_path: symbolic link path
"""
normalized_link = os.path.normpath(link_path).replace(os.sep, '/')
normalized_target = os.path.normpath(target_path).replace(os.sep, '/')
normalized_link = self._normalize_path(link_path)
normalized_target = self._normalize_path(target_path)
# Keep the original mapping: target path -> link path
self._path_mappings[normalized_target] = normalized_link
logger.info(f"Added path mapping: {normalized_target} -> {normalized_link}")
self._preview_root_paths.update(self._expand_preview_root(normalized_target))
self._preview_root_paths.update(self._expand_preview_root(normalized_link))
def add_route_mapping(self, path: str, route: str):
"""Add a static route mapping"""
normalized_path = os.path.normpath(path).replace(os.sep, '/')
self._route_mappings[normalized_path] = route
# logger.info(f"Added route mapping: {normalized_path} -> {route}")
def _seed_root_symlink_mappings(self) -> None:
"""Ensure symlinked root folders are recorded before deep scanning."""
for root in self._symlink_roots():
if not root:
continue
try:
if not self._is_link(root):
continue
target_path = os.path.realpath(root)
if not os.path.isdir(target_path):
continue
self.add_path_mapping(root, target_path)
except Exception as exc:
logger.debug("Skipping root symlink %s: %s", root, exc)
def _expand_preview_root(self, path: str) -> Set[Path]:
"""Return normalized ``Path`` objects representing a preview root."""
roots: Set[Path] = set()
if not path:
return roots
try:
raw_path = Path(path).expanduser()
except Exception:
return roots
if raw_path.is_absolute():
roots.add(raw_path)
try:
resolved = raw_path.resolve(strict=False)
except RuntimeError:
resolved = raw_path.absolute()
roots.add(resolved)
try:
real_path = raw_path.resolve()
except (FileNotFoundError, RuntimeError):
real_path = resolved
roots.add(real_path)
normalized: Set[Path] = set()
for candidate in roots:
if candidate.is_absolute():
normalized.add(candidate)
else:
try:
normalized.add(candidate.resolve(strict=False))
except RuntimeError:
normalized.add(candidate.absolute())
return normalized
def _rebuild_preview_roots(self) -> None:
"""Recompute the cache of directories permitted for previews."""
preview_roots: Set[Path] = set()
for root in self.loras_roots or []:
preview_roots.update(self._expand_preview_root(root))
for root in self.base_models_roots or []:
preview_roots.update(self._expand_preview_root(root))
for root in self.embeddings_roots or []:
preview_roots.update(self._expand_preview_root(root))
for target, link in self._path_mappings.items():
preview_roots.update(self._expand_preview_root(target))
preview_roots.update(self._expand_preview_root(link))
self._preview_root_paths = {path for path in preview_roots if path.is_absolute()}
logger.debug(
"Preview roots rebuilt: %d paths from %d lora roots, %d checkpoint roots, %d embedding roots, %d symlink mappings",
len(self._preview_root_paths),
len(self.loras_roots or []),
len(self.base_models_roots or []),
len(self.embeddings_roots or []),
len(self._path_mappings),
)
def map_path_to_link(self, path: str) -> str:
"""Map a target path back to its symbolic link path"""
normalized_path = os.path.normpath(path).replace(os.sep, '/')
# Check if the path is contained in any mapped target path
for target_path, link_path in self._path_mappings.items():
if normalized_path.startswith(target_path):
# Match whole path components to avoid prefix collisions (e.g., /a/b vs /a/bc)
if normalized_path == target_path:
return link_path
if normalized_path.startswith(target_path + '/'):
# If the path starts with the target path, replace with link path
mapped_path = normalized_path.replace(target_path, link_path, 1)
return mapped_path
return path
return normalized_path
def map_link_to_path(self, link_path: str) -> str:
"""Map a symbolic link path back to the actual path"""
normalized_link = os.path.normpath(link_path).replace(os.sep, '/')
# Check if the path is contained in any mapped target path
for target_path, link_path in self._path_mappings.items():
if normalized_link.startswith(target_path):
# If the path starts with the target path, replace with actual path
mapped_path = normalized_link.replace(target_path, link_path, 1)
for target_path, link_path_mapped in self._path_mappings.items():
# Match whole path components
if normalized_link == link_path_mapped:
return target_path
if normalized_link.startswith(link_path_mapped + '/'):
# If the path starts with the link path, replace with actual path
mapped_path = normalized_link.replace(link_path_mapped, target_path, 1)
return mapped_path
return link_path
return normalized_link
def _dedupe_existing_paths(self, raw_paths: Iterable[str]) -> Dict[str, str]:
dedup: Dict[str, str] = {}
for path in raw_paths:
if not isinstance(path, str):
continue
if not os.path.exists(path):
continue
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
normalized = os.path.normpath(path).replace(os.sep, '/')
if real_path not in dedup:
dedup[real_path] = normalized
return dedup
def _prepare_lora_paths(self, raw_paths: Iterable[str]) -> List[str]:
path_map = self._dedupe_existing_paths(raw_paths)
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _prepare_checkpoint_paths(
self, checkpoint_paths: Iterable[str], unet_paths: Iterable[str]
) -> List[str]:
checkpoint_map = self._dedupe_existing_paths(checkpoint_paths)
unet_map = self._dedupe_existing_paths(unet_paths)
merged_map: Dict[str, str] = {}
for real_path, original in {**checkpoint_map, **unet_map}.items():
if real_path not in merged_map:
merged_map[real_path] = original
unique_paths = sorted(merged_map.values(), key=lambda p: p.lower())
checkpoint_values = set(checkpoint_map.values())
unet_values = set(unet_map.values())
self.checkpoints_roots = [p for p in unique_paths if p in checkpoint_values]
self.unet_roots = [p for p in unique_paths if p in unet_values]
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _prepare_embedding_paths(self, raw_paths: Iterable[str]) -> List[str]:
path_map = self._dedupe_existing_paths(raw_paths)
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _apply_library_paths(self, folder_paths: Mapping[str, Iterable[str]]) -> None:
self._path_mappings.clear()
self._preview_root_paths = set()
lora_paths = folder_paths.get('loras', []) or []
checkpoint_paths = folder_paths.get('checkpoints', []) or []
unet_paths = folder_paths.get('unet', []) or []
embedding_paths = folder_paths.get('embeddings', []) or []
self.loras_roots = self._prepare_lora_paths(lora_paths)
self.base_models_roots = self._prepare_checkpoint_paths(checkpoint_paths, unet_paths)
self.embeddings_roots = self._prepare_embedding_paths(embedding_paths)
self._initialize_symlink_mappings()
def _init_lora_paths(self) -> List[str]:
"""Initialize and validate LoRA paths from ComfyUI settings"""
try:
raw_paths = folder_paths.get_folder_paths("loras")
# Normalize and resolve symlinks, store mapping from resolved -> original
path_map = {}
for path in raw_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
path_map[real_path] = path_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Now sort and use only the deduplicated real paths
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
unique_paths = self._prepare_lora_paths(raw_paths)
logger.info("Found LoRA roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]"))
if not unique_paths:
logger.warning("No valid loras folders found in ComfyUI configuration")
return []
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
except Exception as e:
logger.warning(f"Error initializing LoRA paths: {e}")
@@ -187,52 +623,17 @@ class Config:
def _init_checkpoint_paths(self) -> List[str]:
"""Initialize and validate checkpoint paths from ComfyUI settings"""
try:
# Get checkpoint paths from folder_paths
raw_checkpoint_paths = folder_paths.get_folder_paths("checkpoints")
raw_unet_paths = folder_paths.get_folder_paths("unet")
# Normalize and resolve symlinks for checkpoints, store mapping from resolved -> original
checkpoint_map = {}
for path in raw_checkpoint_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
checkpoint_map[real_path] = checkpoint_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Normalize and resolve symlinks for unet, store mapping from resolved -> original
unet_map = {}
for path in raw_unet_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
unet_map[real_path] = unet_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Merge both maps and deduplicate by real path
merged_map = {}
for real_path, orig_path in {**checkpoint_map, **unet_map}.items():
if real_path not in merged_map:
merged_map[real_path] = orig_path
unique_paths = self._prepare_checkpoint_paths(raw_checkpoint_paths, raw_unet_paths)
# Now sort and use only the deduplicated real paths
unique_paths = sorted(merged_map.values(), key=lambda p: p.lower())
# Split back into checkpoints and unet roots for class properties
self.checkpoints_roots = [p for p in unique_paths if p in checkpoint_map.values()]
self.unet_roots = [p for p in unique_paths if p in unet_map.values()]
all_paths = unique_paths
logger.info("Found checkpoint roots:" + ("\n - " + "\n - ".join(all_paths) if all_paths else "[]"))
if not all_paths:
logger.info("Found checkpoint roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]"))
if not unique_paths:
logger.warning("No valid checkpoint folders found in ComfyUI configuration")
return []
# Initialize path mappings
for original_path in all_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return all_paths
return unique_paths
except Exception as e:
logger.warning(f"Error initializing checkpoint paths: {e}")
return []
@@ -241,47 +642,94 @@ class Config:
"""Initialize and validate embedding paths from ComfyUI settings"""
try:
raw_paths = folder_paths.get_folder_paths("embeddings")
# Normalize and resolve symlinks, store mapping from resolved -> original
path_map = {}
for path in raw_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
path_map[real_path] = path_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Now sort and use only the deduplicated real paths
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
unique_paths = self._prepare_embedding_paths(raw_paths)
logger.info("Found embedding roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]"))
if not unique_paths:
logger.warning("No valid embeddings folders found in ComfyUI configuration")
return []
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
except Exception as e:
logger.warning(f"Error initializing embedding paths: {e}")
return []
def get_preview_static_url(self, preview_path: str) -> str:
"""Convert local preview path to static URL"""
if not preview_path:
return ""
real_path = os.path.realpath(preview_path).replace(os.sep, '/')
for path, route in self._route_mappings.items():
if real_path.startswith(path):
relative_path = os.path.relpath(real_path, path).replace(os.sep, '/')
safe_parts = [urllib.parse.quote(part) for part in relative_path.split('/')]
safe_path = '/'.join(safe_parts)
return f'{route}/{safe_path}'
normalized = os.path.normpath(preview_path).replace(os.sep, '/')
encoded_path = urllib.parse.quote(normalized, safe='')
return f'/api/lm/previews?path={encoded_path}'
return ""
def is_preview_path_allowed(self, preview_path: str) -> bool:
"""Return ``True`` if ``preview_path`` is within an allowed directory."""
if not preview_path:
return False
try:
candidate = Path(preview_path).expanduser().resolve(strict=False)
except Exception:
return False
# Use os.path.normcase for case-insensitive comparison on Windows.
# On Windows, Path.relative_to() is case-sensitive for drive letters,
# causing paths like 'a:/folder' to not match 'A:/folder'.
candidate_str = os.path.normcase(str(candidate))
for root in self._preview_root_paths:
root_str = os.path.normcase(str(root))
# Check if candidate is equal to or under the root directory
if candidate_str == root_str or candidate_str.startswith(root_str + os.sep):
return True
if self._preview_root_paths:
logger.debug(
"Preview path rejected: %s (candidate=%s, num_roots=%d, first_root=%s)",
preview_path,
candidate_str,
len(self._preview_root_paths),
os.path.normcase(str(next(iter(self._preview_root_paths)))),
)
else:
logger.debug(
"Preview path rejected (no roots configured): %s",
preview_path,
)
return False
def apply_library_settings(self, library_config: Mapping[str, object]) -> None:
"""Update runtime paths to match the provided library configuration."""
folder_paths = library_config.get('folder_paths') if isinstance(library_config, Mapping) else {}
if not isinstance(folder_paths, Mapping):
folder_paths = {}
self._apply_library_paths(folder_paths)
logger.info(
"Applied library settings with %d lora roots, %d checkpoint roots, and %d embedding roots",
len(self.loras_roots or []),
len(self.base_models_roots or []),
len(self.embeddings_roots or []),
)
def get_library_registry_snapshot(self) -> Dict[str, object]:
"""Return the current library registry and active library name."""
try:
from .services.settings_manager import get_settings_manager
settings_service = get_settings_manager()
libraries = settings_service.get_libraries()
active_library = settings_service.get_active_library_name()
return {
"active_library": active_library,
"libraries": libraries,
}
except Exception as exc: # pragma: no cover - defensive logging
logger.debug("Failed to collect library registry snapshot: %s", exc)
return {"active_library": "", "libraries": {}}
# Global config instance
config = Config()

View File

@@ -2,7 +2,15 @@ import asyncio
import sys
import os
import logging
from pathlib import Path
from .utils.logging_config import setup_logging
# Check if we're in standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
# Only setup logging prefix if not in standalone mode
if not standalone_mode:
setup_logging()
from server import PromptServer # type: ignore
from .config import config
@@ -11,16 +19,47 @@ from .routes.recipe_routes import RecipeRoutes
from .routes.stats_routes import StatsRoutes
from .routes.update_routes import UpdateRoutes
from .routes.misc_routes import MiscRoutes
from .routes.preview_routes import PreviewRoutes
from .routes.example_images_routes import ExampleImagesRoutes
from .services.service_registry import ServiceRegistry
from .services.settings_manager import settings
from .services.settings_manager import get_settings_manager
from .utils.example_images_migration import ExampleImagesMigration
from .services.websocket_manager import ws_manager
from .services.example_images_cleanup_service import ExampleImagesCleanupService
from .middleware.csp_middleware import relax_csp_for_remote_media
logger = logging.getLogger(__name__)
# Check if we're in standalone mode
STANDALONE_MODE = 'nodes' not in sys.modules
HEADER_SIZE_LIMIT = 16384
def _sanitize_size_limit(value):
"""Return a non-negative integer size for ``handler_args`` comparisons."""
try:
coerced = int(value)
except (TypeError, ValueError):
return 0
return coerced if coerced >= 0 else 0
class _SettingsProxy:
def __init__(self):
self._manager = None
def _resolve(self):
if self._manager is None:
self._manager = get_settings_manager()
return self._manager
def get(self, *args, **kwargs):
return self._resolve().get(*args, **kwargs)
def __getattr__(self, item):
return getattr(self._resolve(), item)
settings = _SettingsProxy()
class LoraManager:
"""Main entry point for LoRA Manager plugin"""
@@ -30,6 +69,41 @@ class LoraManager:
"""Initialize and register all routes using the new refactored architecture"""
app = PromptServer.instance.app
if relax_csp_for_remote_media not in app.middlewares:
# Ensure CSP relaxer executes after ComfyUI's block_external_middleware so it can
# see and extend the restrictive header instead of being overwritten by it.
block_middleware_index = next(
(
idx
for idx, middleware in enumerate(app.middlewares)
if getattr(middleware, "__name__", "") == "block_external_middleware"
),
None,
)
if block_middleware_index is None:
app.middlewares.append(relax_csp_for_remote_media)
else:
app.middlewares.insert(block_middleware_index, relax_csp_for_remote_media)
# Increase allowed header sizes so browsers with large localhost cookie
# jars (multiple UIs on 127.0.0.1) don't trip aiohttp's 8KB default
# limits. Cookies for unrelated apps are still sent to the plugin and
# may otherwise raise LineTooLong errors when the request parser reads
# them. Preserve any previously configured handler arguments while
# ensuring our minimum sizes are applied.
handler_args = getattr(app, "_handler_args", {}) or {}
updated_handler_args = dict(handler_args)
updated_handler_args["max_field_size"] = max(
_sanitize_size_limit(handler_args.get("max_field_size", 0)),
HEADER_SIZE_LIMIT,
)
updated_handler_args["max_line_size"] = max(
_sanitize_size_limit(handler_args.get("max_line_size", 0)),
HEADER_SIZE_LIMIT,
)
app._handler_args = updated_handler_args
# Configure aiohttp access logger to be less verbose
logging.getLogger('aiohttp.access').setLevel(logging.WARNING)
@@ -49,103 +123,18 @@ class LoraManager:
asyncio_logger = logging.getLogger("asyncio")
asyncio_logger.addFilter(ConnectionResetFilter())
added_targets = set() # Track already added target paths
# Add static route for example images if the path exists in settings
example_images_path = settings.get('example_images_path')
logger.info(f"Example images path: {example_images_path}")
if example_images_path and os.path.exists(example_images_path):
app.router.add_static('/example_images_static', example_images_path)
logger.info(f"Added static route for example images: /example_images_static -> {example_images_path}")
# Add static routes for each lora root
for idx, root in enumerate(config.loras_roots, start=1):
preview_path = f'/loras_static/root{idx}/preview'
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for each checkpoint root
for idx, root in enumerate(config.base_models_roots, start=1):
preview_path = f'/checkpoints_static/root{idx}/preview'
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for each embedding root
for idx, root in enumerate(config.embeddings_roots, start=1):
preview_path = f'/embeddings_static/root{idx}/preview'
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for symlink target paths
link_idx = {
'lora': 1,
'checkpoint': 1,
'embedding': 1
}
for target_path, link_path in config._path_mappings.items():
if target_path not in added_targets:
# Determine if this is a checkpoint, lora, or embedding link based on path
is_checkpoint = any(cp_root in link_path for cp_root in config.base_models_roots)
is_checkpoint = is_checkpoint or any(cp_root in target_path for cp_root in config.base_models_roots)
is_embedding = any(emb_root in link_path for emb_root in config.embeddings_roots)
is_embedding = is_embedding or any(emb_root in target_path for emb_root in config.embeddings_roots)
if is_checkpoint:
route_path = f'/checkpoints_static/link_{link_idx["checkpoint"]}/preview'
link_idx["checkpoint"] += 1
elif is_embedding:
route_path = f'/embeddings_static/link_{link_idx["embedding"]}/preview'
link_idx["embedding"] += 1
else:
route_path = f'/loras_static/link_{link_idx["lora"]}/preview'
link_idx["lora"] += 1
try:
app.router.add_static(route_path, Path(target_path).resolve(strict=False))
logger.info(f"Added static route for link target {route_path} -> {target_path}")
config.add_route_mapping(target_path, route_path)
added_targets.add(target_path)
except Exception as e:
logger.warning(f"Failed to add static route on initialization for {target_path}: {e}")
continue
# Add static route for locales JSON files
if os.path.exists(config.i18n_path):
app.router.add_static('/locales', config.i18n_path)
logger.info(f"Added static route for locales: /locales -> {config.i18n_path}")
# Add static route for plugin assets
app.router.add_static('/loras_static', config.static_path)
@@ -161,7 +150,8 @@ class LoraManager:
RecipeRoutes.setup_routes(app)
UpdateRoutes.setup_routes(app)
MiscRoutes.setup_routes(app)
ExampleImagesRoutes.setup_routes(app)
ExampleImagesRoutes.setup_routes(app, ws_manager=ws_manager)
PreviewRoutes.setup_routes(app)
# Setup WebSocket routes that are shared across all model types
app.router.add_get('/ws/fetch-progress', ws_manager.handle_connection)
@@ -174,8 +164,6 @@ class LoraManager:
# Add cleanup
app.on_shutdown.append(cls._cleanup)
logger.info(f"LoRA Manager: Set up routes for {len(ModelServiceFactory.get_registered_types())} model types: {', '.join(ModelServiceFactory.get_registered_types())}")
@classmethod
async def _initialize_services(cls):
"""Initialize all services using the ServiceRegistry"""
@@ -185,6 +173,9 @@ class LoraManager:
# Register DownloadManager with ServiceRegistry
await ServiceRegistry.get_download_manager()
from .services.metadata_service import initialize_metadata_providers
await initialize_metadata_providers()
# Initialize WebSocket manager
await ServiceRegistry.get_websocket_manager()
@@ -198,29 +189,188 @@ class LoraManager:
recipe_scanner = await ServiceRegistry.get_recipe_scanner()
# Create low-priority initialization tasks
asyncio.create_task(lora_scanner.initialize_in_background(), name='lora_cache_init')
asyncio.create_task(checkpoint_scanner.initialize_in_background(), name='checkpoint_cache_init')
asyncio.create_task(embedding_scanner.initialize_in_background(), name='embedding_cache_init')
asyncio.create_task(recipe_scanner.initialize_in_background(), name='recipe_cache_init')
init_tasks = [
asyncio.create_task(lora_scanner.initialize_in_background(), name='lora_cache_init'),
asyncio.create_task(checkpoint_scanner.initialize_in_background(), name='checkpoint_cache_init'),
asyncio.create_task(embedding_scanner.initialize_in_background(), name='embedding_cache_init'),
asyncio.create_task(recipe_scanner.initialize_in_background(), name='recipe_cache_init')
]
await ExampleImagesMigration.check_and_run_migrations()
logger.info("LoRA Manager: All services initialized and background tasks scheduled")
# Schedule post-initialization tasks to run after scanners complete
asyncio.create_task(
cls._run_post_initialization_tasks(init_tasks),
name='post_init_tasks'
)
logger.debug("LoRA Manager: All services initialized and background tasks scheduled")
except Exception as e:
logger.error(f"LoRA Manager: Error initializing services: {e}", exc_info=True)
@classmethod
async def _run_post_initialization_tasks(cls, init_tasks):
"""Run post-initialization tasks after all scanners complete"""
try:
logger.debug("LoRA Manager: Waiting for scanner initialization to complete...")
# Wait for all scanner initialization tasks to complete
await asyncio.gather(*init_tasks, return_exceptions=True)
logger.debug("LoRA Manager: Scanner initialization completed, starting post-initialization tasks...")
# Run post-initialization tasks
post_tasks = [
asyncio.create_task(cls._cleanup_backup_files(), name='cleanup_bak_files'),
# Add more post-initialization tasks here as needed
# asyncio.create_task(cls._another_post_task(), name='another_task'),
]
# Run all post-initialization tasks
results = await asyncio.gather(*post_tasks, return_exceptions=True)
# Log results
for i, result in enumerate(results):
task_name = post_tasks[i].get_name()
if isinstance(result, Exception):
logger.error(f"Post-initialization task '{task_name}' failed: {result}")
else:
logger.debug(f"Post-initialization task '{task_name}' completed successfully")
logger.debug("LoRA Manager: All post-initialization tasks completed")
except Exception as e:
logger.error(f"LoRA Manager: Error in post-initialization tasks: {e}", exc_info=True)
@classmethod
async def _cleanup_backup_files(cls):
"""Clean up .bak files in all model roots"""
try:
logger.debug("Starting cleanup of .bak files in model directories...")
# Collect all model roots
all_roots = set()
all_roots.update(config.loras_roots)
all_roots.update(config.base_models_roots)
all_roots.update(config.embeddings_roots)
total_deleted = 0
total_size_freed = 0
for root_path in all_roots:
if not os.path.exists(root_path):
continue
try:
deleted_count, size_freed = await cls._cleanup_backup_files_in_directory(root_path)
total_deleted += deleted_count
total_size_freed += size_freed
if deleted_count > 0:
logger.debug(f"Cleaned up {deleted_count} .bak files in {root_path} (freed {size_freed / (1024*1024):.2f} MB)")
except Exception as e:
logger.error(f"Error cleaning up .bak files in {root_path}: {e}")
# Yield control periodically
await asyncio.sleep(0.01)
if total_deleted > 0:
logger.debug(f"Backup cleanup completed: removed {total_deleted} .bak files, freed {total_size_freed / (1024*1024):.2f} MB total")
else:
logger.debug("Backup cleanup completed: no .bak files found")
except Exception as e:
logger.error(f"Error during backup file cleanup: {e}", exc_info=True)
@classmethod
async def _cleanup_backup_files_in_directory(cls, directory_path: str):
"""Clean up .bak files in a specific directory recursively
Args:
directory_path: Path to the directory to clean
Returns:
Tuple[int, int]: (number of files deleted, total size freed in bytes)
"""
deleted_count = 0
size_freed = 0
visited_paths = set()
def cleanup_recursive(path):
nonlocal deleted_count, size_freed
try:
real_path = os.path.realpath(path)
if real_path in visited_paths:
return
visited_paths.add(real_path)
with os.scandir(path) as it:
for entry in it:
try:
if entry.is_file(follow_symlinks=True) and entry.name.endswith('.bak'):
file_size = entry.stat().st_size
os.remove(entry.path)
deleted_count += 1
size_freed += file_size
logger.debug(f"Deleted .bak file: {entry.path}")
elif entry.is_dir(follow_symlinks=True):
cleanup_recursive(entry.path)
except Exception as e:
logger.warning(f"Could not delete .bak file {entry.path}: {e}")
except Exception as e:
logger.error(f"Error scanning directory {path} for .bak files: {e}")
# Run the recursive cleanup in a thread pool to avoid blocking
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, cleanup_recursive, directory_path)
return deleted_count, size_freed
@classmethod
async def _cleanup_example_images_folders(cls):
"""Invoke the example images cleanup service for manual execution."""
try:
service = ExampleImagesCleanupService()
result = await service.cleanup_example_image_folders()
if result.get('success'):
logger.debug(
"Manual example images cleanup completed: moved=%s",
result.get('moved_total'),
)
elif result.get('partial_success'):
logger.warning(
"Manual example images cleanup partially succeeded: moved=%s failures=%s",
result.get('moved_total'),
result.get('move_failures'),
)
else:
logger.debug(
"Manual example images cleanup skipped or failed: %s",
result.get('error', 'no changes'),
)
return result
except Exception as e: # pragma: no cover - defensive guard
logger.error(f"Error during example images cleanup: {e}", exc_info=True)
return {
'success': False,
'error': str(e),
'error_code': 'unexpected_error',
}
@classmethod
async def _cleanup(cls, app):
"""Cleanup resources using ServiceRegistry"""
try:
logger.info("LoRA Manager: Cleaning up services")
# Close CivitaiClient gracefully
civitai_client = await ServiceRegistry.get_service("civitai_client")
if civitai_client:
await civitai_client.close()
logger.info("Closed CivitaiClient connection")
except Exception as e:
logger.error(f"Error during cleanup: {e}", exc_info=True)

View File

@@ -1,9 +1,7 @@
import os
import importlib
import sys
# Check if running in standalone mode
standalone_mode = 'nodes' not in sys.modules
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
if not standalone_mode:
from .metadata_hook import MetadataHook

View File

@@ -1,9 +1,9 @@
import json
import sys
import os
from .constants import IMAGES
# Check if running in standalone mode
standalone_mode = 'nodes' not in sys.modules
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER
@@ -39,8 +39,39 @@ class MetadataProcessor:
if node_id in metadata.get(SAMPLING, {}) and metadata[SAMPLING][node_id].get(IS_SAMPLER, False):
candidate_samplers[node_id] = metadata[SAMPLING][node_id]
# If we found candidate samplers, apply primary sampler logic to these candidates only
if candidate_samplers:
# If we found candidate samplers, apply primary sampler logic to these candidates only
# PRE-PROCESS: Ensure all candidate samplers have their parameters populated
# This is especially important for SamplerCustomAdvanced which needs tracing
prompt = metadata.get("current_prompt")
for node_id in candidate_samplers:
# If a sampler is missing common parameters like steps or denoise,
# try to populate them using tracing before ranking
sampler_info = candidate_samplers[node_id]
params = sampler_info.get("parameters", {})
if prompt and (params.get("steps") is None or params.get("denoise") is None):
# Create a temporary params dict to use the handler
temp_params = {
"steps": params.get("steps"),
"denoise": params.get("denoise"),
"sampler": params.get("sampler_name"),
"scheduler": params.get("scheduler")
}
# Check if it's SamplerCustomAdvanced
if prompt.original_prompt and node_id in prompt.original_prompt:
if prompt.original_prompt[node_id].get("class_type") == "SamplerCustomAdvanced":
MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, node_id, temp_params)
# Update the actual parameters with found values
params["steps"] = temp_params.get("steps")
params["denoise"] = temp_params.get("denoise")
if temp_params.get("sampler"):
params["sampler_name"] = temp_params.get("sampler")
if temp_params.get("scheduler"):
params["scheduler"] = temp_params.get("scheduler")
# Collect potential primary samplers based on different criteria
custom_advanced_samplers = []
advanced_add_noise_samplers = []
@@ -49,7 +80,6 @@ class MetadataProcessor:
high_denoise_id = None
# First, check for SamplerCustomAdvanced among candidates
prompt = metadata.get("current_prompt")
if prompt and prompt.original_prompt:
for node_id in candidate_samplers:
node_info = prompt.original_prompt.get(node_id, {})
@@ -77,15 +107,16 @@ class MetadataProcessor:
# Combine all potential primary samplers
potential_samplers = custom_advanced_samplers + advanced_add_noise_samplers + high_denoise_samplers
# Find the most recent potential primary sampler (closest to downstream node)
for i in range(downstream_index - 1, -1, -1):
# Find the first potential primary sampler (prefer base sampler over refine)
# Use forward search to prioritize the first one in execution order
for i in range(downstream_index):
node_id = execution_order[i]
if node_id in potential_samplers:
return node_id, candidate_samplers[node_id]
# If no potential sampler found from our criteria, return the most recent sampler
# If no potential sampler found from our criteria, return the first sampler
if candidate_samplers:
for i in range(downstream_index - 1, -1, -1):
for i in range(downstream_index):
node_id = execution_order[i]
if node_id in candidate_samplers:
return node_id, candidate_samplers[node_id]
@@ -176,8 +207,11 @@ class MetadataProcessor:
found_node_id = input_value[0] # Connected node_id
# If we're looking for a specific node class
if target_class and prompt.original_prompt[found_node_id].get("class_type") == target_class:
return found_node_id
if target_class:
if found_node_id not in prompt.original_prompt:
return None
if prompt.original_prompt[found_node_id].get("class_type") == target_class:
return found_node_id
# If we're not looking for a specific class, update the last valid node
if not target_class:
@@ -185,11 +219,19 @@ class MetadataProcessor:
# Continue tracing through intermediate nodes
current_node_id = found_node_id
# For most conditioning nodes, the input we want to follow is named "conditioning"
if "conditioning" in prompt.original_prompt[current_node_id].get("inputs", {}):
# Check if current source node exists
if current_node_id not in prompt.original_prompt:
return found_node_id if not target_class else None
# Determine which input to follow next on the source node
source_node_inputs = prompt.original_prompt[current_node_id].get("inputs", {})
if input_name in source_node_inputs:
current_input = input_name
elif "conditioning" in source_node_inputs:
current_input = "conditioning"
else:
# If there's no "conditioning" input, return the current node
# If there's no suitable input to follow, return the current node
# if we're not looking for a specific target_class
return found_node_id if not target_class else None
else:
@@ -202,12 +244,89 @@ class MetadataProcessor:
return last_valid_node if not target_class else None
@staticmethod
def find_primary_checkpoint(metadata):
"""Find the primary checkpoint model in the workflow"""
if not metadata.get(MODELS):
def trace_model_path(metadata, prompt, start_node_id):
"""
Trace the model connection path upstream to find the checkpoint
"""
if not prompt or not prompt.original_prompt:
return None
# In most workflows, there's only one checkpoint, so we can just take the first one
current_node_id = start_node_id
depth = 0
max_depth = 50
while depth < max_depth:
# Check if current node is a registered checkpoint in our metadata
# This handles cached nodes correctly because metadata contains info for all nodes in the graph
if current_node_id in metadata.get(MODELS, {}):
if metadata[MODELS][current_node_id].get("type") == "checkpoint":
return current_node_id
if current_node_id not in prompt.original_prompt:
return None
node = prompt.original_prompt[current_node_id]
inputs = node.get("inputs", {})
class_type = node.get("class_type", "")
# Determine which input to follow next
next_input_name = "model"
# Special handling for initial node
if depth == 0:
if class_type == "SamplerCustomAdvanced":
next_input_name = "guider"
# If the specific input doesn't exist, try generic 'model'
if next_input_name not in inputs:
if "model" in inputs:
next_input_name = "model"
elif "basic_pipe" in inputs:
# Handle pipe nodes like FromBasicPipe by following the pipeline
next_input_name = "basic_pipe"
else:
# Dead end - no model input to follow
return None
# Get connected node
input_val = inputs[next_input_name]
if isinstance(input_val, list) and len(input_val) > 0:
current_node_id = input_val[0]
else:
return None
depth += 1
return None
@staticmethod
def find_primary_checkpoint(metadata, downstream_id=None, primary_sampler_id=None):
"""
Find the primary checkpoint model in the workflow
Parameters:
- metadata: The workflow metadata
- downstream_id: Optional ID of a downstream node to help identify the specific primary sampler
- primary_sampler_id: Optional ID of the primary sampler if already known
"""
if not metadata.get(MODELS):
return None
# Method 1: Topology-based tracing (More accurate for complex workflows)
# First, find the primary sampler if not provided
if not primary_sampler_id:
primary_sampler_id, _ = MetadataProcessor.find_primary_sampler(metadata, downstream_id)
if primary_sampler_id:
prompt = metadata.get("current_prompt")
if prompt:
# Trace back from the sampler to find the checkpoint
checkpoint_id = MetadataProcessor.trace_model_path(metadata, prompt, primary_sampler_id)
if checkpoint_id and checkpoint_id in metadata.get(MODELS, {}):
return metadata[MODELS][checkpoint_id].get("name")
# Method 2: Fallback to the first available checkpoint (Original behavior)
# In most simple workflows, there's only one checkpoint, so we can just take the first one
for node_id, model_info in metadata.get(MODELS, {}).items():
if model_info.get("type") == "checkpoint":
return model_info.get("name")
@@ -295,7 +414,7 @@ class MetadataProcessor:
"seed": None,
"steps": None,
"cfg_scale": None,
"guidance": None, # Add guidance parameter
# "guidance": None, # Add guidance parameter
"sampler": None,
"scheduler": None,
"checkpoint": None,
@@ -311,7 +430,8 @@ class MetadataProcessor:
primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id)
# Directly get checkpoint from metadata instead of tracing
checkpoint = MetadataProcessor.find_primary_checkpoint(metadata)
# Pass primary_sampler_id to avoid redundant calculation
checkpoint = MetadataProcessor.find_primary_checkpoint(metadata, id, primary_sampler_id)
if checkpoint:
params["checkpoint"] = checkpoint
@@ -445,6 +565,7 @@ class MetadataProcessor:
scheduler_params = metadata[SAMPLING][scheduler_node_id].get("parameters", {})
params["steps"] = scheduler_params.get("steps")
params["scheduler"] = scheduler_params.get("scheduler")
params["denoise"] = scheduler_params.get("denoise")
# 2. Trace sampler input to find KSamplerSelect (only if sampler input exists)
if "sampler" in sampler_inputs:

View File

@@ -196,9 +196,11 @@ class MetadataRegistry:
node_metadata[category] = {}
node_metadata[category][node_id] = current_metadata[category][node_id]
# Save to cache if we have any metadata for this node
# Save new metadata or clear stale cache entries when metadata is empty
if any(node_metadata.values()):
self.node_cache[cache_key] = node_metadata
else:
self.node_cache.pop(cache_key, None)
def clear_unused_cache(self):
"""Clean up node_cache entries that are no longer in use"""

View File

@@ -3,6 +3,18 @@ import os
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER
def _store_checkpoint_metadata(metadata, node_id, model_name):
"""Store checkpoint model information when available."""
if not model_name:
return
metadata.setdefault(MODELS, {})
metadata[MODELS][node_id] = {
"name": model_name,
"type": "checkpoint",
"node_id": node_id
}
class NodeMetadataExtractor:
"""Base class for node-specific metadata extraction"""
@@ -29,12 +41,48 @@ class CheckpointLoaderExtractor(NodeMetadataExtractor):
return
model_name = inputs.get("ckpt_name")
if model_name:
metadata[MODELS][node_id] = {
"name": model_name,
"type": "checkpoint",
"node_id": node_id
}
_store_checkpoint_metadata(metadata, node_id, model_name)
class NunchakuFluxDiTLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "model_path" not in inputs:
return
model_name = inputs.get("model_path")
_store_checkpoint_metadata(metadata, node_id, model_name)
class NunchakuQwenImageDiTLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "model_name" not in inputs:
return
model_name = inputs.get("model_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class GGUFLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "gguf_name" not in inputs:
return
model_name = inputs.get("gguf_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class KJNodesModelLoaderExtractor(NodeMetadataExtractor):
"""Extract metadata from KJNodes loaders that expose `model_name`."""
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "model_name" not in inputs:
return
model_name = inputs.get("model_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class TSCCheckpointLoaderExtractor(NodeMetadataExtractor):
@staticmethod
@@ -43,12 +91,7 @@ class TSCCheckpointLoaderExtractor(NodeMetadataExtractor):
return
model_name = inputs.get("ckpt_name")
if model_name:
metadata[MODELS][node_id] = {
"name": model_name,
"type": "checkpoint",
"node_id": node_id
}
_store_checkpoint_metadata(metadata, node_id, model_name)
# For loader node has lora_stack input, like Efficient Loader from Efficient Nodes
active_loras = []
@@ -651,6 +694,7 @@ NODE_EXTRACTORS = {
"KSamplerAdvancedBasicPipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-impact-pack
"KSampler_inspire_pipe": KSamplerBasicPipeExtractor, # comfyui-inspire-pack
"KSamplerAdvanced_inspire_pipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-inspire-pack
"KSampler_inspire": SamplerExtractor, # comfyui-inspire-pack
# Sampling Selectors
"KSamplerSelect": KSamplerSelectExtractor, # Add KSamplerSelect
"BasicScheduler": BasicSchedulerExtractor, # Add BasicScheduler
@@ -660,17 +704,26 @@ NODE_EXTRACTORS = {
"comfyLoader": CheckpointLoaderExtractor, # easy comfyLoader
"CheckpointLoaderSimpleWithImages": CheckpointLoaderExtractor, # CheckpointLoader|pysssss
"TSC_EfficientLoader": TSCCheckpointLoaderExtractor, # Efficient Nodes
"NunchakuFluxDiTLoader": NunchakuFluxDiTLoaderExtractor, # ComfyUI-Nunchaku
"NunchakuQwenImageDiTLoader": NunchakuQwenImageDiTLoaderExtractor, # ComfyUI-Nunchaku
"LoaderGGUF": GGUFLoaderExtractor, # calcuis gguf
"LoaderGGUFAdvanced": GGUFLoaderExtractor, # calcuis gguf
"GGUFLoaderKJ": KJNodesModelLoaderExtractor, # KJNodes
"DiffusionModelLoaderKJ": KJNodesModelLoaderExtractor, # KJNodes
"CheckpointLoaderKJ": CheckpointLoaderExtractor, # KJNodes
"UNETLoader": UNETLoaderExtractor, # Updated to use dedicated extractor
"UnetLoaderGGUF": UNETLoaderExtractor, # Updated to use dedicated extractor
"LoraLoader": LoraLoaderExtractor,
"LoraManagerLoader": LoraLoaderManagerExtractor,
# Conditioning
"CLIPTextEncode": CLIPTextEncodeExtractor,
"PromptLoraManager": CLIPTextEncodeExtractor,
"CLIPTextEncodeFlux": CLIPTextEncodeFluxExtractor, # Add CLIPTextEncodeFlux
"WAS_Text_to_Conditioning": CLIPTextEncodeExtractor,
"AdvancedCLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb
"smZ_CLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/shiimizu/ComfyUI_smZNodes
"CR_ApplyControlNetStack": CR_ApplyControlNetStackExtractor, # Add CR_ApplyControlNetStack
"PCTextEncode": CLIPTextEncodeExtractor, # From https://github.com/asagi4/comfyui-prompt-control
# Latent
"EmptyLatentImage": ImageSizeExtractor,
# Flux

View File

@@ -0,0 +1 @@
"""Server middleware modules"""

View File

@@ -0,0 +1,53 @@
"""Cache control middleware for ComfyUI server"""
from aiohttp import web
from typing import Callable, Awaitable
# Time in seconds
ONE_HOUR: int = 3600
ONE_DAY: int = 86400
IMG_EXTENSIONS = (
".jpg",
".jpeg",
".png",
".ppm",
".bmp",
".pgm",
".tif",
".tiff",
".webp",
".mp4"
)
@web.middleware
async def cache_control(
request: web.Request, handler: Callable[[web.Request], Awaitable[web.Response]]
) -> web.Response:
"""Cache control middleware that sets appropriate cache headers based on file type and response status"""
response: web.Response = await handler(request)
if (
request.path.endswith(".js")
or request.path.endswith(".css")
or request.path.endswith("index.json")
):
response.headers.setdefault("Cache-Control", "no-cache")
return response
# Early return for non-image files - no cache headers needed
if not request.path.lower().endswith(IMG_EXTENSIONS):
return response
# Handle image files
if response.status == 404:
response.headers.setdefault("Cache-Control", f"public, max-age={ONE_HOUR}")
elif response.status in (200, 201, 202, 203, 204, 205, 206, 301, 308):
# Success responses and permanent redirects - cache for 1 day
response.headers.setdefault("Cache-Control", f"public, max-age={ONE_DAY}")
elif response.status in (302, 303, 307):
# Temporary redirects - no cache
response.headers.setdefault("Cache-Control", "no-cache")
# Note: 304 Not Modified falls through - no cache headers set
return response

View File

@@ -0,0 +1,65 @@
"""Middleware helpers for adjusting Content Security Policy headers."""
from typing import Awaitable, Callable, Dict, List
from aiohttp import web
REMOTE_MEDIA_SOURCES = (
"https://image.civitai.com",
"https://img.genur.art",
)
@web.middleware
async def relax_csp_for_remote_media(
request: web.Request, handler: Callable[[web.Request], Awaitable[web.StreamResponse]]
) -> web.StreamResponse:
"""Allow LoRA Manager media previews to load from trusted remote domains.
When ComfyUI is started with ``--disable-api-nodes`` it injects a restrictive
``Content-Security-Policy`` header that blocks remote images and videos. The
LoRA Manager UI legitimately needs to fetch previews from Civitai and Genur,
so this middleware augments the existing CSP to whitelist those hosts while
preserving all other directives.
"""
response: web.StreamResponse = await handler(request)
header_value = response.headers.get("Content-Security-Policy")
if not header_value:
return response
directive_order: List[str] = []
directives: Dict[str, List[str]] = {}
for raw_directive in header_value.split(";"):
directive = raw_directive.strip()
if not directive:
continue
parts = directive.split()
name, values = parts[0], parts[1:]
if name not in directive_order:
directive_order.append(name)
directives[name] = values
def merge_sources(name: str, sources: List[str], defaults: List[str] | None = None) -> None:
existing = directives.get(name, list(defaults or []))
for source in sources:
if source not in existing:
existing.append(source)
directives[name] = existing
if name not in directive_order:
directive_order.append(name)
merge_sources("img-src", list(REMOTE_MEDIA_SOURCES))
merge_sources("media-src", ["'self'", *REMOTE_MEDIA_SOURCES], defaults=["'self'"])
updated_header = "; ".join(
f"{name} {' '.join(directives[name])}".rstrip() for name in directive_order
)
response.headers["Content-Security-Policy"] = f"{updated_header};"
return response

View File

@@ -1,15 +1,15 @@
import logging
from server import PromptServer # type: ignore
from ..metadata_collector.metadata_processor import MetadataProcessor
logger = logging.getLogger(__name__)
class DebugMetadata:
NAME = "Debug Metadata (LoraManager)"
CATEGORY = "Lora Manager/utils"
DESCRIPTION = "Debug node to verify metadata_processor functionality"
OUTPUT_NODE = True
@classmethod
def INPUT_TYPES(cls):
return {
@@ -25,21 +25,37 @@ class DebugMetadata:
FUNCTION = "process_metadata"
def process_metadata(self, images, id):
"""
Process metadata from the execution context and return it for UI display.
The metadata is returned via the 'ui' key in the return dict, which triggers
node.onExecuted on the frontend to update the JsonDisplayWidget.
Args:
images: Input images (required for execution flow)
id: Node's unique ID (hidden)
Returns:
Dict with 'result' (empty tuple) and 'ui' (metadata dict for widget display)
"""
try:
# Get the current execution context's metadata
from ..metadata_collector import get_metadata
metadata = get_metadata()
# Use the MetadataProcessor to convert it to JSON string
metadata_json = MetadataProcessor.to_json(metadata, id)
# Send metadata to frontend for display
PromptServer.instance.send_sync("metadata_update", {
"id": id,
"metadata": metadata_json
})
# Use the MetadataProcessor to convert it to dict
metadata_dict = MetadataProcessor.to_dict(metadata, id)
return {
"result": (),
# ComfyUI expects ui values to be lists, wrap the dict in a list
"ui": {"metadata": [metadata_dict]},
}
except Exception as e:
logger.error(f"Error processing metadata: {e}")
return ()
return {
"result": (),
"ui": {"metadata": [{"error": str(e)}]},
}

View File

@@ -1,7 +1,6 @@
import logging
import re
from nodes import LoraLoader
from comfy.comfy_types import IO # type: ignore
from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list, nunchaku_load_lora
@@ -17,7 +16,7 @@ class LoraManagerLoader:
"required": {
"model": ("MODEL",),
# "clip": ("CLIP",),
"text": (IO.STRING, {
"text": ("STRING", {
"multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True,
@@ -28,7 +27,7 @@ class LoraManagerLoader:
"optional": FlexibleOptionalInputType(any_type),
}
RETURN_TYPES = ("MODEL", "CLIP", IO.STRING, IO.STRING)
RETURN_TYPES = ("MODEL", "CLIP", "STRING", "STRING")
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras"
@@ -115,7 +114,7 @@ class LoraManagerLoader:
formatted_loras = []
for item in loaded_loras:
parts = item.split(":")
lora_name = parts[0].strip()
lora_name = parts[0]
strength_parts = parts[1].strip().split(",")
if len(strength_parts) > 1:
@@ -141,8 +140,7 @@ class LoraManagerTextLoader:
return {
"required": {
"model": ("MODEL",),
"lora_syntax": (IO.STRING, {
"defaultInput": True,
"lora_syntax": ("STRING", {
"forceInput": True,
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation"
}),
@@ -153,7 +151,7 @@ class LoraManagerTextLoader:
}
}
RETURN_TYPES = ("MODEL", "CLIP", IO.STRING, IO.STRING)
RETURN_TYPES = ("MODEL", "CLIP", "STRING", "STRING")
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras_from_text"
@@ -165,7 +163,7 @@ class LoraManagerTextLoader:
loras = []
for match in matches:
lora_name = match[0].strip()
lora_name = match[0]
model_strength = float(match[1])
clip_strength = float(match[2]) if match[2] else model_strength

87
py/nodes/lora_pool.py Normal file
View File

@@ -0,0 +1,87 @@
"""
LoRA Pool Node - Defines filter configuration for LoRA selection.
This node provides a visual filter editor that generates a LORA_POOL_CONFIG
object for use by downstream nodes (like LoRA Randomizer).
"""
import logging
logger = logging.getLogger(__name__)
class LoraPoolNode:
"""
A node that defines LoRA filter criteria through a Vue-based widget.
Outputs a LORA_POOL_CONFIG that can be consumed by:
- Frontend: LoRA Randomizer widget reads connected pool's widget value
- Backend: LoRA Randomizer receives config during workflow execution
"""
NAME = "Lora Pool (LoraManager)"
CATEGORY = "Lora Manager/randomizer"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"pool_config": ("LORA_POOL_CONFIG", {}),
},
"hidden": {
# Hidden input to pass through unique node ID for frontend
"unique_id": "UNIQUE_ID",
},
}
RETURN_TYPES = ("POOL_CONFIG",)
RETURN_NAMES = ("POOL_CONFIG",)
FUNCTION = "process"
OUTPUT_NODE = False
def process(self, pool_config, unique_id=None):
"""
Pass through the pool configuration filters.
The config is generated entirely by the frontend widget.
This function validates and returns only the filters field.
Args:
pool_config: Dict containing filter criteria from widget
unique_id: Node's unique ID (hidden)
Returns:
Tuple containing the filters dict from pool_config
"""
# Validate required structure
if not isinstance(pool_config, dict):
logger.warning("Invalid pool_config type, using empty config")
pool_config = self._default_config()
# Ensure version field exists
if "version" not in pool_config:
pool_config["version"] = 1
# Extract filters field
filters = pool_config.get("filters", self._default_config()["filters"])
# Log for debugging
logger.debug(f"[LoraPoolNode] Processing filters: {filters}")
return (filters,)
@staticmethod
def _default_config():
"""Return default empty configuration."""
return {
"version": 1,
"filters": {
"baseModels": [],
"tags": {"include": [], "exclude": []},
"folders": {"include": [], "exclude": []},
"favoritesOnly": False,
"license": {"noCreditRequired": False, "allowSelling": False},
},
"preview": {"matchCount": 0, "lastUpdated": 0},
}

206
py/nodes/lora_randomizer.py Normal file
View File

@@ -0,0 +1,206 @@
"""
Lora Randomizer Node - Randomly selects LoRAs from a pool with configurable settings.
This node accepts optional pool_config input to filter available LoRAs, and outputs
a LORA_STACK with randomly selected LoRAs. Returns UI updates with new random LoRAs
and tracks the last used combination for reuse.
"""
import logging
import random
import os
from ..utils.utils import get_lora_info
from .utils import extract_lora_name
logger = logging.getLogger(__name__)
class LoraRandomizerNode:
"""Node that randomly selects LoRAs from a pool"""
NAME = "Lora Randomizer (LoraManager)"
CATEGORY = "Lora Manager/randomizer"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"randomizer_config": ("RANDOMIZER_CONFIG", {}),
"loras": ("LORAS", {}),
},
"optional": {
"pool_config": ("POOL_CONFIG", {}),
},
}
RETURN_TYPES = ("LORA_STACK",)
RETURN_NAMES = ("LORA_STACK",)
FUNCTION = "randomize"
OUTPUT_NODE = False
def _preprocess_loras_input(self, loras):
"""
Preprocess loras input to handle different widget formats.
Args:
loras: Input from widget, either:
- List of LoRA dicts (expected format)
- Dict with '__value__' key containing the list
Returns:
List of LoRA dicts
"""
if isinstance(loras, dict) and "__value__" in loras:
return loras["__value__"]
return loras
async def randomize(self, randomizer_config, loras, pool_config=None):
"""
Randomize LoRAs based on configuration and pool filters.
Args:
randomizer_config: Dict with randomizer settings (count, strength ranges, roll_mode)
loras: List of LoRA dicts from LORAS widget (includes locked state)
pool_config: Optional config from LoRA Pool node for filtering
Returns:
Dictionary with 'result' (LORA_STACK tuple) and 'ui' (for widget display)
"""
from ..services.service_registry import ServiceRegistry
loras = self._preprocess_loras_input(loras)
roll_mode = randomizer_config.get("roll_mode", "always")
logger.debug(f"[LoraRandomizerNode] roll_mode: {roll_mode}")
# Dual seed mechanism for batch queue synchronization
# execution_seed: seed for generating execution_stack (= previous next_seed)
# next_seed: seed for generating ui_loras (= what will be displayed after execution)
execution_seed = randomizer_config.get("execution_seed", None)
next_seed = randomizer_config.get("next_seed", None)
if roll_mode == "fixed":
ui_loras = loras
execution_loras = loras
else:
scanner = await ServiceRegistry.get_lora_scanner()
# Generate execution_loras from execution_seed (if available)
if execution_seed is not None:
# Use execution_seed to regenerate the same loras that were shown to user
execution_loras = await self._generate_random_loras_for_ui(
scanner, randomizer_config, loras, pool_config, seed=execution_seed
)
else:
# First execution: use loras input (what user sees in the widget)
execution_loras = loras
# Generate ui_loras from next_seed (for display after execution)
ui_loras = await self._generate_random_loras_for_ui(
scanner, randomizer_config, loras, pool_config, seed=next_seed
)
execution_stack = self._build_execution_stack_from_input(execution_loras)
return {
"result": (execution_stack,),
"ui": {"loras": ui_loras, "last_used": execution_loras},
}
def _build_execution_stack_from_input(self, loras):
"""
Build LORA_STACK tuple from input loras list for execution.
Args:
loras: List of LoRA dicts with name, strength, clipStrength, active
Returns:
List of tuples (lora_path, model_strength, clip_strength)
"""
lora_stack = []
for lora in loras:
if not lora.get("active", False):
continue
# Get file path
lora_path, trigger_words = get_lora_info(lora["name"])
if not lora_path:
logger.warning(
f"[LoraRandomizerNode] Could not find path for LoRA: {lora['name']}"
)
continue
# Normalize path separators
lora_path = lora_path.replace("/", os.sep)
# Extract strengths (convert to float to prevent string subtraction errors)
model_strength = float(lora.get("strength", 1.0))
clip_strength = float(lora.get("clipStrength", model_strength))
lora_stack.append((lora_path, model_strength, clip_strength))
return lora_stack
async def _generate_random_loras_for_ui(
self, scanner, randomizer_config, input_loras, pool_config=None, seed=None
):
"""
Generate new random loras for UI display.
Args:
scanner: LoraScanner instance
randomizer_config: Dict with randomizer settings
input_loras: Current input loras (for extracting locked loras)
pool_config: Optional pool filters
seed: Optional seed for deterministic randomization
Returns:
List of LoRA dicts for UI display
"""
from ..services.lora_service import LoraService
# Parse randomizer settings (convert numeric values to float to prevent type errors)
count_mode = randomizer_config.get("count_mode", "range")
count_fixed = int(randomizer_config.get("count_fixed", 5))
count_min = int(randomizer_config.get("count_min", 3))
count_max = int(randomizer_config.get("count_max", 7))
model_strength_min = float(randomizer_config.get("model_strength_min", 0.0))
model_strength_max = float(randomizer_config.get("model_strength_max", 1.0))
use_same_clip_strength = randomizer_config.get("use_same_clip_strength", True)
clip_strength_min = float(randomizer_config.get("clip_strength_min", 0.0))
clip_strength_max = float(randomizer_config.get("clip_strength_max", 1.0))
use_recommended_strength = randomizer_config.get(
"use_recommended_strength", False
)
recommended_strength_scale_min = float(
randomizer_config.get("recommended_strength_scale_min", 0.5)
)
recommended_strength_scale_max = float(
randomizer_config.get("recommended_strength_scale_max", 1.0)
)
# Extract locked LoRAs from input
locked_loras = [lora for lora in input_loras if lora.get("locked", False)]
# Use LoraService to generate random LoRAs
lora_service = LoraService(scanner)
result_loras = await lora_service.get_random_loras(
count=count_fixed,
model_strength_min=model_strength_min,
model_strength_max=model_strength_max,
use_same_clip_strength=use_same_clip_strength,
clip_strength_min=clip_strength_min,
clip_strength_max=clip_strength_max,
locked_loras=locked_loras,
pool_config=pool_config,
count_mode=count_mode,
count_min=count_min,
count_max=count_max,
use_recommended_strength=use_recommended_strength,
recommended_strength_scale_min=recommended_strength_scale_min,
recommended_strength_scale_max=recommended_strength_scale_max,
seed=seed,
)
return result_loras

View File

@@ -1,4 +1,3 @@
from comfy.comfy_types import IO # type: ignore
import os
from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list
@@ -15,7 +14,7 @@ class LoraStacker:
def INPUT_TYPES(cls):
return {
"required": {
"text": (IO.STRING, {
"text": ("STRING", {
"multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True,
@@ -26,7 +25,7 @@ class LoraStacker:
"optional": FlexibleOptionalInputType(any_type),
}
RETURN_TYPES = ("LORA_STACK", IO.STRING, IO.STRING)
RETURN_TYPES = ("LORA_STACK", "STRING", "STRING")
RETURN_NAMES = ("LORA_STACK", "trigger_words", "active_loras")
FUNCTION = "stack_loras"

59
py/nodes/prompt.py Normal file
View File

@@ -0,0 +1,59 @@
from typing import Any, Optional
class PromptLoraManager:
"""Encodes text (and optional trigger words) into CLIP conditioning."""
NAME = "Prompt (LoraManager)"
CATEGORY = "Lora Manager/conditioning"
DESCRIPTION = (
"Encodes a text prompt using a CLIP model into an embedding that can be used "
"to guide the diffusion model towards generating specific images."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"text": (
'STRING',
{
"multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True,
"tooltip": "The text to be encoded.",
},
),
"clip": (
'CLIP',
{"tooltip": "The CLIP model used for encoding the text."},
),
},
"optional": {
"trigger_words": (
'STRING',
{
"forceInput": True,
"tooltip": (
"Optional trigger words to prepend to the text before "
"encoding."
)
},
)
},
}
RETURN_TYPES = ('CONDITIONING', 'STRING',)
RETURN_NAMES = ('CONDITIONING', 'PROMPT',)
OUTPUT_TOOLTIPS = (
"A conditioning containing the embedded text used to guide the diffusion model.",
)
FUNCTION = "encode"
def encode(self, text: str, clip: Any, trigger_words: Optional[str] = None):
prompt = text
if trigger_words:
prompt = ", ".join([trigger_words, text])
from nodes import CLIPTextEncode # type: ignore
conditioning = CLIPTextEncode().encode(clip, prompt)[0]
return (conditioning, prompt,)

View File

@@ -9,7 +9,7 @@ from ..metadata_collector import get_metadata
from PIL import Image, PngImagePlugin
import piexif
class SaveImage:
class SaveImageLM:
NAME = "Save Image (LoraManager)"
CATEGORY = "Lora Manager/utils"
DESCRIPTION = "Save images with embedded generation metadata in compatible format"
@@ -273,9 +273,15 @@ class SaveImage:
length = int(parts[1])
prompt = prompt[:length]
filename = filename.replace(segment, prompt.strip())
elif key == "model" and 'checkpoint' in metadata_dict:
model = metadata_dict.get('checkpoint', '')
model = os.path.splitext(os.path.basename(model))[0]
elif key == "model":
model_value = metadata_dict.get('checkpoint')
if isinstance(model_value, (bytes, os.PathLike)):
model_value = str(model_value)
if not isinstance(model_value, str) or not model_value:
model = "model_unavailable"
else:
model = os.path.splitext(os.path.basename(model_value))[0]
if len(parts) >= 2:
length = int(parts[1])
model = model[:length]
@@ -442,4 +448,4 @@ class SaveImage:
add_counter_to_filename
)
return (images,)
return (images,)

View File

@@ -1,6 +1,5 @@
import json
import re
from server import PromptServer # type: ignore
from .utils import FlexibleOptionalInputType, any_type
import logging
@@ -24,6 +23,10 @@ class TriggerWordToggle:
"default": True,
"tooltip": "Sets the default initial state (active or inactive) when trigger words are added."
}),
"allow_strength_adjustment": ("BOOLEAN", {
"default": False,
"tooltip": "Enable mouse wheel adjustment of each trigger word's strength."
}),
},
"optional": FlexibleOptionalInputType(any_type),
"hidden": {
@@ -48,7 +51,14 @@ class TriggerWordToggle:
else:
return data
def process_trigger_words(self, id, group_mode, default_active, **kwargs):
def process_trigger_words(
self,
id,
group_mode,
default_active,
allow_strength_adjustment=False,
**kwargs,
):
# Handle both old and new formats for trigger_words
trigger_words_data = self._get_toggle_data(kwargs, 'orinalMessage')
trigger_words = trigger_words_data if isinstance(trigger_words_data, str) else ""
@@ -62,36 +72,81 @@ class TriggerWordToggle:
# Convert to list if it's a JSON string
if isinstance(trigger_data, str):
trigger_data = json.loads(trigger_data)
# Create dictionaries to track active state of words or groups
active_state = {item['text']: item.get('active', False) for item in trigger_data}
if group_mode:
# Split by two or more consecutive commas to get groups
groups = re.split(r',{2,}', trigger_words)
# Remove leading/trailing whitespace from each group
groups = [group.strip() for group in groups]
# Filter groups: keep those not in toggle_trigger_words or those that are active
filtered_groups = [group for group in groups if group not in active_state or active_state[group]]
if filtered_groups:
filtered_triggers = ', '.join(filtered_groups)
if isinstance(trigger_data, list):
if group_mode:
if allow_strength_adjustment:
parsed_items = [
self._parse_trigger_item(item, allow_strength_adjustment)
for item in trigger_data
]
filtered_groups = [
self._format_word_output(
item["text"],
item["strength"],
allow_strength_adjustment,
)
for item in parsed_items
if item["text"] and item["active"]
]
else:
filtered_groups = [
(item.get('text') or "").strip()
for item in trigger_data
if (item.get('text') or "").strip() and item.get('active', False)
]
filtered_triggers = ', '.join(filtered_groups) if filtered_groups else ""
else:
filtered_triggers = ""
parsed_items = [
self._parse_trigger_item(item, allow_strength_adjustment)
for item in trigger_data
]
filtered_words = [
self._format_word_output(
item["text"],
item["strength"],
allow_strength_adjustment,
)
for item in parsed_items
if item["text"] and item["active"]
]
filtered_triggers = ', '.join(filtered_words) if filtered_words else ""
else:
# Original behavior for individual words mode
original_words = [word.strip() for word in trigger_words.split(',')]
# Filter out empty strings
original_words = [word for word in original_words if word]
filtered_words = [word for word in original_words if word not in active_state or active_state[word]]
if filtered_words:
filtered_triggers = ', '.join(filtered_words)
# Fallback to original message parsing if data is not in the expected list format
if group_mode:
groups = re.split(r',{2,}', trigger_words)
groups = [group.strip() for group in groups if group.strip()]
filtered_triggers = ', '.join(groups)
else:
filtered_triggers = ""
words = [word.strip() for word in trigger_words.split(',') if word.strip()]
filtered_triggers = ', '.join(words)
except Exception as e:
logger.error(f"Error processing trigger words: {e}")
return (filtered_triggers,)
return (filtered_triggers,)
def _parse_trigger_item(self, item, allow_strength_adjustment):
text = (item.get('text') or "").strip()
active = bool(item.get('active', False))
strength = item.get('strength')
strength_match = re.match(r'^\((.+):([\d.]+)\)$', text)
if strength_match:
text = strength_match.group(1).strip()
if strength is None:
try:
strength = float(strength_match.group(2))
except ValueError:
strength = None
return {
"text": text,
"active": active,
"strength": strength if allow_strength_adjustment else None,
}
def _format_word_output(self, base_word, strength, allow_strength_adjustment):
if allow_strength_adjustment and strength is not None:
return f"({base_word}:{strength:.2f})"
return base_word

View File

@@ -36,6 +36,7 @@ any_type = AnyType("*")
import os
import logging
import copy
import sys
import folder_paths
logger = logging.getLogger(__name__)
@@ -98,22 +99,38 @@ def to_diffusers(input_lora):
def nunchaku_load_lora(model, lora_name, lora_strength):
"""Load a Flux LoRA for Nunchaku model"""
# Get full path to the LoRA file. Allow both direct paths and registered LoRA names.
lora_path = lora_name if os.path.isfile(lora_name) else folder_paths.get_full_path("loras", lora_name)
if not lora_path or not os.path.isfile(lora_path):
logger.warning("Skipping LoRA '%s' because it could not be found", lora_name)
return model
model_wrapper = model.model.diffusion_model
transformer = model_wrapper.model
# Save the transformer temporarily
model_wrapper.model = None
ret_model = copy.deepcopy(model) # copy everything except the model
ret_model_wrapper = ret_model.model.diffusion_model
# Restore the model and set it for the copy
model_wrapper.model = transformer
ret_model_wrapper.model = transformer
# Get full path to the LoRA file
lora_path = folder_paths.get_full_path("loras", lora_name)
ret_model_wrapper.loras.append((lora_path, lora_strength))
# Try to find copy_with_ctx in the same module as ComfyFluxWrapper
module_name = model_wrapper.__class__.__module__
module = sys.modules.get(module_name)
copy_with_ctx = getattr(module, "copy_with_ctx", None)
if copy_with_ctx is not None:
# New logic using copy_with_ctx from ComfyUI-nunchaku 1.1.0+
ret_model_wrapper, ret_model = copy_with_ctx(model_wrapper)
ret_model_wrapper.loras = [*model_wrapper.loras, (lora_path, lora_strength)]
else:
# Fallback to legacy logic
logger.warning("Please upgrade ComfyUI-nunchaku to 1.1.0 or above for better LoRA support. Falling back to legacy loading logic.")
transformer = model_wrapper.model
# Save the transformer temporarily
model_wrapper.model = None
ret_model = copy.deepcopy(model) # copy everything except the model
ret_model_wrapper = ret_model.model.diffusion_model
# Restore the model and set it for the copy
model_wrapper.model = transformer
ret_model_wrapper.model = transformer
ret_model_wrapper.loras.append((lora_path, lora_strength))
# Convert the LoRA to diffusers format
sd = to_diffusers(lora_path)

View File

@@ -1,4 +1,3 @@
from comfy.comfy_types import IO # type: ignore
import folder_paths # type: ignore
from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, get_loras_list
@@ -6,7 +5,7 @@ import logging
logger = logging.getLogger(__name__)
class WanVideoLoraSelect:
class WanVideoLoraSelectLM:
NAME = "WanVideo Lora Select (LoraManager)"
CATEGORY = "Lora Manager/stackers"
@@ -16,7 +15,7 @@ class WanVideoLoraSelect:
"required": {
"low_mem_load": ("BOOLEAN", {"default": False, "tooltip": "Load LORA models with less VRAM usage, slower loading. This affects ALL LoRAs, not just the current ones. No effect if merge_loras is False"}),
"merge_loras": ("BOOLEAN", {"default": True, "tooltip": "Merge LoRAs into the model, otherwise they are loaded on the fly. Always disabled for GGUF and scaled fp8 models. This affects ALL LoRAs, not just the current one"}),
"text": (IO.STRING, {
"text": ("STRING", {
"multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True,
@@ -27,7 +26,7 @@ class WanVideoLoraSelect:
"optional": FlexibleOptionalInputType(any_type),
}
RETURN_TYPES = ("WANVIDLORA", IO.STRING, IO.STRING)
RETURN_TYPES = ("WANVIDLORA", "STRING", "STRING")
RETURN_NAMES = ("lora", "trigger_words", "active_loras")
FUNCTION = "process_loras"

View File

@@ -0,0 +1,125 @@
import folder_paths # type: ignore
from ..utils.utils import get_lora_info
from .utils import any_type
import logging
# 初始化日志记录器
logger = logging.getLogger(__name__)
# 定义新节点的类
class WanVideoLoraSelectFromText:
# 节点在UI中显示的名称
NAME = "WanVideo Lora Select From Text (LoraManager)"
# 节点所属的分类
CATEGORY = "Lora Manager/stackers"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"low_mem_load": ("BOOLEAN", {"default": False, "tooltip": "Load LORA models with less VRAM usage, slower loading. This affects ALL LoRAs, not just the current ones. No effect if merge_loras is False"}),
"merge_lora": ("BOOLEAN", {"default": True, "tooltip": "Merge LoRAs into the model, otherwise they are loaded on the fly. Always disabled for GGUF and scaled fp8 models. This affects ALL LoRAs, not just the current one"}),
"lora_syntax": ("STRING", {
"multiline": True,
"forceInput": True,
"tooltip": "Connect a TEXT output for LoRA syntax: <lora:name:strength>"
}),
},
"optional": {
"prev_lora": ("WANVIDLORA",),
"blocks": ("BLOCKS",)
}
}
RETURN_TYPES = ("WANVIDLORA", "STRING", "STRING")
RETURN_NAMES = ("lora", "trigger_words", "active_loras")
FUNCTION = "process_loras_from_syntax"
def process_loras_from_syntax(self, lora_syntax, low_mem_load=False, merge_lora=True, **kwargs):
text_to_process = lora_syntax
blocks = kwargs.get('blocks', {})
selected_blocks = blocks.get("selected_blocks", {})
layer_filter = blocks.get("layer_filter", "")
loras_list = []
all_trigger_words = []
active_loras = []
prev_lora = kwargs.get('prev_lora', None)
if prev_lora is not None:
loras_list.extend(prev_lora)
if not merge_lora:
low_mem_load = False
parts = text_to_process.split('<lora:')
for part in parts[1:]:
end_index = part.find('>')
if end_index == -1:
continue
content = part[:end_index]
lora_parts = content.split(':')
lora_name_raw = ""
model_strength = 1.0
clip_strength = 1.0
if len(lora_parts) == 2:
lora_name_raw = lora_parts[0].strip()
try:
model_strength = float(lora_parts[1])
clip_strength = model_strength
except (ValueError, IndexError):
logger.warning(f"Invalid strength for LoRA '{lora_name_raw}'. Skipping.")
continue
elif len(lora_parts) >= 3:
lora_name_raw = lora_parts[0].strip()
try:
model_strength = float(lora_parts[1])
clip_strength = float(lora_parts[2])
except (ValueError, IndexError):
logger.warning(f"Invalid strengths for LoRA '{lora_name_raw}'. Skipping.")
continue
else:
continue
lora_path, trigger_words = get_lora_info(lora_name_raw)
lora_item = {
"path": folder_paths.get_full_path("loras", lora_path),
"strength": model_strength,
"name": lora_path.split(".")[0],
"blocks": selected_blocks,
"layer_filter": layer_filter,
"low_mem_load": low_mem_load,
"merge_loras": merge_lora,
}
loras_list.append(lora_item)
active_loras.append((lora_name_raw, model_strength, clip_strength))
all_trigger_words.extend(trigger_words)
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
formatted_loras = []
for name, model_strength, clip_strength in active_loras:
if abs(model_strength - clip_strength) > 0.001:
formatted_loras.append(f"<lora:{name}:{str(model_strength).strip()}:{str(clip_strength).strip()}>")
else:
formatted_loras.append(f"<lora:{name}:{str(model_strength).strip()}>")
active_loras_text = " ".join(formatted_loras)
return (loras_list, trigger_words_text, active_loras_text)
NODE_CLASS_MAPPINGS = {
"WanVideoLoraSelectFromText": WanVideoLoraSelectFromText
}
NODE_DISPLAY_NAME_MAPPINGS = {
"WanVideoLoraSelectFromText": "WanVideo Lora Select From Text (LoraManager)"
}

View File

@@ -8,6 +8,7 @@ from typing import Dict, List, Any, Optional, Tuple
from abc import ABC, abstractmethod
from ..config import config
from ..utils.constants import VALID_LORA_TYPES
from ..utils.civitai_utils import rewrite_preview_url
logger = logging.getLogger(__name__)
@@ -36,7 +37,8 @@ class RecipeMetadataParser(ABC):
"""
pass
async def populate_lora_from_civitai(self, lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any], Optional[str]],
@staticmethod
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any], Optional[str]],
recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]:
"""
Populate a lora entry with information from Civitai API response
@@ -55,7 +57,7 @@ class RecipeMetadataParser(ABC):
# Unpack the tuple to get the actual data
civitai_info, error_msg = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
if not civitai_info or civitai_info.get("error") == "Model not found":
if not civitai_info or error_msg == "Model not found":
# Model not found or deleted
lora_entry['isDeleted'] = True
lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
@@ -78,7 +80,7 @@ class RecipeMetadataParser(ABC):
# Update model name if available
if 'model' in civitai_info and 'name' in civitai_info['model']:
lora_entry['name'] = civitai_info['model']['name']
lora_entry['id'] = civitai_info.get('id')
lora_entry['modelId'] = civitai_info.get('modelId')
@@ -88,7 +90,10 @@ class RecipeMetadataParser(ABC):
# Get thumbnail URL from first image
if 'images' in civitai_info and civitai_info['images']:
lora_entry['thumbnailUrl'] = civitai_info['images'][0].get('url', '')
image_url = civitai_info['images'][0].get('url')
if image_url:
rewritten_image_url, _ = rewrite_preview_url(image_url, media_type='image')
lora_entry['thumbnailUrl'] = rewritten_image_url or image_url
# Get base model
current_base_model = civitai_info.get('baseModel', '')
@@ -144,40 +149,68 @@ class RecipeMetadataParser(ABC):
logger.error(f"Error populating lora from Civitai info: {e}")
return lora_entry
async def populate_checkpoint_from_civitai(self, checkpoint: Dict[str, Any], civitai_info: Dict[str, Any]) -> Dict[str, Any]:
@staticmethod
async def populate_checkpoint_from_civitai(checkpoint: Dict[str, Any], civitai_info: Dict[str, Any]) -> Dict[str, Any]:
"""
Populate checkpoint information from Civitai API response
Args:
checkpoint: The checkpoint entry to populate
civitai_info: The response from Civitai API
civitai_info: The response from Civitai API or a (data, error_msg) tuple
Returns:
The populated checkpoint dict
"""
try:
if civitai_info and civitai_info.get("error") != "Model not found":
# Update model name if available
if 'model' in civitai_info and 'name' in civitai_info['model']:
checkpoint['name'] = civitai_info['model']['name']
# Update version if available
if 'name' in civitai_info:
checkpoint['version'] = civitai_info.get('name', '')
# Get thumbnail URL from first image
if 'images' in civitai_info and civitai_info['images']:
checkpoint['thumbnailUrl'] = civitai_info['images'][0].get('url', '')
# Get base model
checkpoint['baseModel'] = civitai_info.get('baseModel', '')
# Get download URL
checkpoint['downloadUrl'] = civitai_info.get('downloadUrl', '')
else:
# Model not found or deleted
civitai_data, error_msg = (
(civitai_info, None)
if not isinstance(civitai_info, tuple)
else civitai_info
)
if not civitai_data or error_msg == "Model not found":
checkpoint['isDeleted'] = True
return checkpoint
if 'model' in civitai_data and 'name' in civitai_data['model']:
checkpoint['name'] = civitai_data['model']['name']
if 'name' in civitai_data:
checkpoint['version'] = civitai_data.get('name', '')
if 'images' in civitai_data and civitai_data['images']:
image_url = civitai_data['images'][0].get('url')
if image_url:
rewritten_image_url, _ = rewrite_preview_url(image_url, media_type='image')
checkpoint['thumbnailUrl'] = rewritten_image_url or image_url
checkpoint['baseModel'] = civitai_data.get('baseModel', '')
checkpoint['downloadUrl'] = civitai_data.get('downloadUrl', '')
checkpoint['modelId'] = civitai_data.get('modelId', checkpoint.get('modelId', 0))
checkpoint['id'] = civitai_data.get('id', 0)
if 'files' in civitai_data:
model_file = next(
(
file
for file in civitai_data.get('files', [])
if file.get('type') == 'Model'
),
None,
)
if model_file:
checkpoint['size'] = model_file.get('sizeKB', 0) * 1024
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
checkpoint['hash'] = sha256.lower()
file_name = model_file.get('name', '')
if file_name:
checkpoint['file_name'] = os.path.splitext(file_name)[0]
except Exception as e:
logger.error(f"Error populating checkpoint from Civitai info: {e}")

216
py/recipes/enrichment.py Normal file
View File

@@ -0,0 +1,216 @@
import logging
import json
import re
import os
from typing import Any, Dict, Optional
from .merger import GenParamsMerger
from .base import RecipeMetadataParser
from ..services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
class RecipeEnricher:
"""Service to enrich recipe metadata from multiple sources (Civitai, Embedded, User)."""
@staticmethod
async def enrich_recipe(
recipe: Dict[str, Any],
civitai_client: Any,
request_params: Optional[Dict[str, Any]] = None
) -> bool:
"""
Enrich a recipe dictionary in-place with metadata from Civitai and embedded params.
Args:
recipe: The recipe dictionary to enrich. Must have 'gen_params' initialized.
civitai_client: Authenticated Civitai client instance.
request_params: (Optional) Parameters from a user request (e.g. import).
Returns:
bool: True if the recipe was modified, False otherwise.
"""
updated = False
gen_params = recipe.get("gen_params", {})
# 1. Fetch Civitai Info if available
civitai_meta = None
model_version_id = None
source_url = recipe.get("source_url") or recipe.get("source_path", "")
# Check if it's a Civitai image URL
image_id_match = re.search(r'civitai\.com/images/(\d+)', str(source_url))
if image_id_match:
image_id = image_id_match.group(1)
try:
image_info = await civitai_client.get_image_info(image_id)
if image_info:
# Handle nested meta often found in Civitai API responses
raw_meta = image_info.get("meta")
if isinstance(raw_meta, dict):
if "meta" in raw_meta and isinstance(raw_meta["meta"], dict):
civitai_meta = raw_meta["meta"]
else:
civitai_meta = raw_meta
model_version_id = image_info.get("modelVersionId")
# If not at top level, check resources in meta
if not model_version_id and civitai_meta:
resources = civitai_meta.get("civitaiResources", [])
for res in resources:
if res.get("type") == "checkpoint":
model_version_id = res.get("modelVersionId")
break
except Exception as e:
logger.warning(f"Failed to fetch Civitai image info: {e}")
# 2. Merge Parameters
# Priority: request_params > civitai_meta > embedded (existing gen_params)
new_gen_params = GenParamsMerger.merge(
request_params=request_params,
civitai_meta=civitai_meta,
embedded_metadata=gen_params
)
if new_gen_params != gen_params:
recipe["gen_params"] = new_gen_params
updated = True
# 3. Checkpoint Enrichment
# If we have a checkpoint entry, or we can find one
# Use 'id' (from Civitai version) as a marker that it's been enriched
checkpoint_entry = recipe.get("checkpoint")
has_full_checkpoint = checkpoint_entry and checkpoint_entry.get("name") and checkpoint_entry.get("id")
if not has_full_checkpoint:
# Helper to look up values in priority order
def start_lookup(keys):
for source in [request_params, civitai_meta, gen_params]:
if source:
if isinstance(keys, list):
for k in keys:
if k in source: return source[k]
else:
if keys in source: return source[keys]
return None
target_version_id = model_version_id or start_lookup("modelVersionId")
# Also check existing checkpoint entry
if not target_version_id and checkpoint_entry:
target_version_id = checkpoint_entry.get("modelVersionId") or checkpoint_entry.get("id")
# Check for version ID in resources (which might be a string in gen_params)
if not target_version_id:
# Look in all sources for "Civitai resources"
resources_val = start_lookup(["Civitai resources", "civitai_resources", "resources"])
if resources_val:
target_version_id = RecipeEnricher._extract_version_id_from_resources({"Civitai resources": resources_val})
target_hash = start_lookup(["Model hash", "checkpoint_hash", "hashes"])
if not target_hash and checkpoint_entry:
target_hash = checkpoint_entry.get("hash") or checkpoint_entry.get("model_hash")
# Look for 'Model' which sometimes is the hash or name
model_val = start_lookup("Model")
# Look for Checkpoint name fallback
checkpoint_val = checkpoint_entry.get("name") if checkpoint_entry else None
if not checkpoint_val:
checkpoint_val = start_lookup(["Checkpoint", "checkpoint"])
checkpoint_updated = await RecipeEnricher._resolve_and_populate_checkpoint(
recipe, target_version_id, target_hash, model_val, checkpoint_val
)
if checkpoint_updated:
updated = True
else:
# Checkpoint exists, no need to sync to gen_params anymore.
pass
# base_model resolution moved to _resolve_and_populate_checkpoint to support strict formatting
return updated
@staticmethod
def _extract_version_id_from_resources(gen_params: Dict[str, Any]) -> Optional[Any]:
"""Try to find modelVersionId in Civitai resources parameter."""
civitai_resources_raw = gen_params.get("Civitai resources")
if not civitai_resources_raw:
return None
resources_list = None
if isinstance(civitai_resources_raw, str):
try:
resources_list = json.loads(civitai_resources_raw)
except Exception:
pass
elif isinstance(civitai_resources_raw, list):
resources_list = civitai_resources_raw
if isinstance(resources_list, list):
for res in resources_list:
if res.get("type") == "checkpoint":
return res.get("modelVersionId")
return None
@staticmethod
async def _resolve_and_populate_checkpoint(
recipe: Dict[str, Any],
target_version_id: Optional[Any],
target_hash: Optional[str],
model_val: Optional[str],
checkpoint_val: Optional[str]
) -> bool:
"""Find checkpoint metadata and populate it in the recipe."""
metadata_provider = await get_default_metadata_provider()
civitai_info = None
if target_version_id:
civitai_info = await metadata_provider.get_model_version_info(str(target_version_id))
elif target_hash:
civitai_info = await metadata_provider.get_model_by_hash(target_hash)
else:
# Look for 'Model' which sometimes is the hash or name
if model_val and len(model_val) == 10: # Likely a short hash
civitai_info = await metadata_provider.get_model_by_hash(model_val)
if civitai_info and not (isinstance(civitai_info, tuple) and civitai_info[1] == "Model not found"):
# If we already have a partial checkpoint, use it as base
existing_cp = recipe.get("checkpoint")
if existing_cp is None:
existing_cp = {}
checkpoint_data = await RecipeMetadataParser.populate_checkpoint_from_civitai(existing_cp, civitai_info)
# 1. First, resolve base_model using full data before we format it away
current_base_model = recipe.get("base_model")
resolved_base_model = checkpoint_data.get("baseModel")
if resolved_base_model:
# Update if empty OR if it matches our generic prefix but is less specific
is_generic = not current_base_model or current_base_model.lower() in ["flux", "sdxl", "sd15"]
if is_generic and resolved_base_model != current_base_model:
recipe["base_model"] = resolved_base_model
# 2. Format according to requirements: type, modelId, modelVersionId, modelName, modelVersionName
formatted_checkpoint = {
"type": "checkpoint",
"modelId": checkpoint_data.get("modelId"),
"modelVersionId": checkpoint_data.get("id") or checkpoint_data.get("modelVersionId"),
"modelName": checkpoint_data.get("name"), # In base.py, 'name' is populated from civitai_data['model']['name']
"modelVersionName": checkpoint_data.get("version") # In base.py, 'version' is populated from civitai_data['name']
}
# Remove None values
recipe["checkpoint"] = {k: v for k, v in formatted_checkpoint.items() if v is not None}
return True
else:
# Fallback to name extraction if we don't already have one
existing_cp = recipe.get("checkpoint")
if not existing_cp or not existing_cp.get("modelName"):
cp_name = checkpoint_val
if cp_name:
recipe["checkpoint"] = {
"type": "checkpoint",
"modelName": cp_name
}
return True
return False

98
py/recipes/merger.py Normal file
View File

@@ -0,0 +1,98 @@
from typing import Any, Dict, Optional
import logging
logger = logging.getLogger(__name__)
class GenParamsMerger:
"""Utility to merge generation parameters from multiple sources with priority."""
BLACKLISTED_KEYS = {
"id", "url", "userId", "username", "createdAt", "updatedAt", "hash", "meta",
"draft", "extra", "width", "height", "process", "quantity", "workflow",
"baseModel", "resources", "disablePoi", "aspectRatio", "Created Date",
"experimental", "civitaiResources", "civitai_resources", "Civitai resources",
"modelVersionId", "modelId", "hashes", "Model", "Model hash", "checkpoint_hash",
"checkpoint", "checksum", "model_checksum"
}
NORMALIZATION_MAPPING = {
# Civitai specific
"cfgScale": "cfg_scale",
"clipSkip": "clip_skip",
"negativePrompt": "negative_prompt",
# Case variations
"Sampler": "sampler",
"Steps": "steps",
"Seed": "seed",
"Size": "size",
"Prompt": "prompt",
"Negative prompt": "negative_prompt",
"Cfg scale": "cfg_scale",
"Clip skip": "clip_skip",
"Denoising strength": "denoising_strength",
}
@staticmethod
def merge(
request_params: Optional[Dict[str, Any]] = None,
civitai_meta: Optional[Dict[str, Any]] = None,
embedded_metadata: Optional[Dict[str, Any]] = None
) -> Dict[str, Any]:
"""
Merge generation parameters from three sources.
Priority: request_params > civitai_meta > embedded_metadata
Args:
request_params: Params provided directly in the import request
civitai_meta: Params from Civitai Image API 'meta' field
embedded_metadata: Params extracted from image EXIF/embedded metadata
Returns:
Merged parameters dictionary
"""
result = {}
# 1. Start with embedded metadata (lowest priority)
if embedded_metadata:
# If it's a full recipe metadata, we use its gen_params
if "gen_params" in embedded_metadata and isinstance(embedded_metadata["gen_params"], dict):
GenParamsMerger._update_normalized(result, embedded_metadata["gen_params"])
else:
# Otherwise assume the dict itself contains gen_params
GenParamsMerger._update_normalized(result, embedded_metadata)
# 2. Layer Civitai meta (medium priority)
if civitai_meta:
GenParamsMerger._update_normalized(result, civitai_meta)
# 3. Layer request params (highest priority)
if request_params:
GenParamsMerger._update_normalized(result, request_params)
# Filter out blacklisted keys and also the original camelCase keys if they were normalized
final_result = {}
for k, v in result.items():
if k in GenParamsMerger.BLACKLISTED_KEYS:
continue
if k in GenParamsMerger.NORMALIZATION_MAPPING:
continue
final_result[k] = v
return final_result
@staticmethod
def _update_normalized(target: Dict[str, Any], source: Dict[str, Any]) -> None:
"""Update target dict with normalized keys from source."""
for k, v in source.items():
normalized_key = GenParamsMerger.NORMALIZATION_MAPPING.get(k, k)
target[normalized_key] = v
# Also keep the original key for now if it's not the same,
# so we can filter at the end or avoid losing it if it wasn't supposed to be renamed?
# Actually, if we rename it, we should probably NOT keep both in 'target'
# because we want to filter them out at the end anyway.
if normalized_key != k:
# If we are overwriting an existing snake_case key with a camelCase one's value,
# that's fine because of the priority order of calls to _update_normalized.
pass
target[k] = v

View File

@@ -1,11 +1,13 @@
"""Parser for Automatic1111 metadata format."""
import re
import os
import json
import logging
from typing import Dict, Any
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
@@ -21,6 +23,7 @@ class AutomaticMetadataParser(RecipeMetadataParser):
CIVITAI_METADATA_REGEX = r', Civitai metadata:\s*(\{.*?\})'
EXTRANETS_REGEX = r'<(lora|hypernet):([^:]+):(-?[0-9.]+)>'
MODEL_HASH_PATTERN = r'Model hash: ([a-zA-Z0-9]+)'
MODEL_NAME_PATTERN = r'Model: ([^,]+)'
VAE_HASH_PATTERN = r'VAE hash: ([a-zA-Z0-9]+)'
def is_metadata_matching(self, user_comment: str) -> bool:
@@ -30,6 +33,9 @@ class AutomaticMetadataParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Automatic1111 format"""
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Split on Negative prompt if it exists
if "Negative prompt:" in user_comment:
parts = user_comment.split('Negative prompt:', 1)
@@ -111,6 +117,12 @@ class AutomaticMetadataParser(RecipeMetadataParser):
except json.JSONDecodeError:
logger.error("Error parsing hashes JSON")
# Pick up model hash from parsed hashes if available
if "hashes" in metadata and not metadata.get("model_hash"):
model_hash_from_hashes = metadata["hashes"].get("model")
if model_hash_from_hashes:
metadata["model_hash"] = model_hash_from_hashes
# Extract Lora hashes in alternative format
lora_hashes_match = re.search(self.LORA_HASHES_REGEX, params_section)
if not hashes_match and lora_hashes_match:
@@ -133,6 +145,17 @@ class AutomaticMetadataParser(RecipeMetadataParser):
params_section = params_section.replace(lora_hashes_match.group(0), '')
except Exception as e:
logger.error(f"Error parsing Lora hashes: {e}")
# Extract checkpoint model hash/name when provided outside Civitai resources
model_hash_match = re.search(self.MODEL_HASH_PATTERN, params_section)
if model_hash_match:
metadata["model_hash"] = model_hash_match.group(1).strip()
params_section = params_section.replace(model_hash_match.group(0), '')
model_name_match = re.search(self.MODEL_NAME_PATTERN, params_section)
if model_name_match:
metadata["model_name"] = model_name_match.group(1).strip()
params_section = params_section.replace(model_name_match.group(0), '')
# Extract basic parameters
param_pattern = r'([A-Za-z\s]+): ([^,]+)'
@@ -174,9 +197,10 @@ class AutomaticMetadataParser(RecipeMetadataParser):
metadata["gen_params"] = gen_params
# Extract LoRA information
# Extract LoRA and checkpoint information
loras = []
base_model_counts = {}
checkpoint = None
# First use Civitai resources if available (more reliable source)
if metadata.get("civitai_resources"):
@@ -198,6 +222,50 @@ class AutomaticMetadataParser(RecipeMetadataParser):
resource["modelVersionId"] = air_modelVersionId
# --- End added ---
if resource.get("type") == "checkpoint" and resource.get("modelVersionId"):
version_id = resource.get("modelVersionId")
version_id_str = str(version_id)
checkpoint_entry = {
'id': version_id,
'modelId': resource.get("modelId", 0),
'name': resource.get("modelName", "Unknown Checkpoint"),
'version': resource.get("modelVersionName", resource.get("versionName", "")),
'type': resource.get("type", "checkpoint"),
'existsLocally': False,
'localPath': None,
'file_name': resource.get("modelName", ""),
'hash': resource.get("hash", "") or "",
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = await metadata_provider.get_model_version_info(version_id_str)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(
"Error fetching Civitai info for checkpoint version %s: %s",
version_id,
e,
)
# Prefer the first checkpoint found
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
if checkpoint is None:
checkpoint = checkpoint_entry
continue
if resource.get("type") in ["lora", "lycoris", "hypernet"] and resource.get("modelVersionId"):
# Initialize lora entry
lora_entry = {
@@ -216,9 +284,9 @@ class AutomaticMetadataParser(RecipeMetadataParser):
}
# Get additional info from Civitai
if civitai_client:
if metadata_provider:
try:
civitai_info = await civitai_client.get_model_version_info(resource.get("modelVersionId"))
civitai_info = await metadata_provider.get_model_version_info(resource.get("modelVersionId"))
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
@@ -233,6 +301,52 @@ class AutomaticMetadataParser(RecipeMetadataParser):
loras.append(lora_entry)
# Fallback checkpoint parsing from generic "Model" and "Model hash" fields
if checkpoint is None:
model_hash = metadata.get("model_hash")
if not model_hash and metadata.get("hashes"):
model_hash = metadata["hashes"].get("model")
model_name = metadata.get("model_name")
file_name = ""
if model_name:
cleaned_name = re.split(r"[\\\\/]", model_name)[-1]
file_name = os.path.splitext(cleaned_name)[0]
if model_hash or model_name:
checkpoint_entry = {
'id': 0,
'modelId': 0,
'name': model_name or "Unknown Checkpoint",
'version': '',
'type': 'checkpoint',
'hash': model_hash or "",
'existsLocally': False,
'localPath': None,
'file_name': file_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider and model_hash:
try:
civitai_info = await metadata_provider.get_model_by_hash(model_hash)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint hash {model_hash}: {e}")
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
checkpoint = checkpoint_entry
# If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"]
if not loras or len(loras) == 0:
# Extract lora weights from extranet tags in prompt (for later use)
@@ -271,11 +385,11 @@ class AutomaticMetadataParser(RecipeMetadataParser):
}
# Try to get info from Civitai
if civitai_client:
if metadata_provider:
try:
if lora_hash:
# If we have hash, use it for lookup
civitai_info = await civitai_client.get_model_by_hash(lora_hash)
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
else:
civitai_info = None
@@ -296,7 +410,9 @@ class AutomaticMetadataParser(RecipeMetadataParser):
# Try to get base model from resources or make educated guess
base_model = None
if base_model_counts:
if checkpoint and checkpoint.get("baseModel"):
base_model = checkpoint.get("baseModel")
elif base_model_counts:
# Use the most common base model from the loras
base_model = max(base_model_counts.items(), key=lambda x: x[1])[0]
@@ -313,6 +429,10 @@ class AutomaticMetadataParser(RecipeMetadataParser):
'gen_params': filtered_gen_params,
'from_automatic_metadata': True
}
if checkpoint:
result['checkpoint'] = checkpoint
result['model'] = checkpoint
return result

View File

@@ -5,6 +5,7 @@ import logging
from typing import Dict, Any, Union
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
@@ -22,13 +23,48 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
"""
if not metadata or not isinstance(metadata, dict):
return False
# Check for key markers specific to Civitai image metadata
return any([
"resources" in metadata,
"civitaiResources" in metadata,
"additionalResources" in metadata
])
def has_markers(payload: Dict[str, Any]) -> bool:
# Check for common CivitAI image metadata fields
civitai_image_fields = (
"resources",
"civitaiResources",
"additionalResources",
"hashes",
"prompt",
"negativePrompt",
"steps",
"sampler",
"cfgScale",
"seed",
"width",
"height",
"Model",
"Model hash"
)
return any(key in payload for key in civitai_image_fields)
# Check the main metadata object
if has_markers(metadata):
return True
# Check for LoRA hash patterns
hashes = metadata.get("hashes")
if isinstance(hashes, dict) and any(str(key).lower().startswith("lora:") for key in hashes):
return True
# Check nested meta object (common in CivitAI image responses)
nested_meta = metadata.get("meta")
if isinstance(nested_meta, dict):
if has_markers(nested_meta):
return True
# Also check for LoRA hash patterns in nested meta
hashes = nested_meta.get("hashes")
if isinstance(hashes, dict) and any(str(key).lower().startswith("lora:") for key in hashes):
return True
return False
async def parse_metadata(self, metadata, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Civitai image format
@@ -36,16 +72,40 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
Args:
metadata: The metadata from the image (dict)
recipe_scanner: Optional recipe scanner service
civitai_client: Optional Civitai API client
civitai_client: Optional Civitai API client (deprecated, use metadata_provider instead)
Returns:
Dict containing parsed recipe data
"""
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Civitai image responses may wrap the actual metadata inside a "meta" key
if (
isinstance(metadata, dict)
and "meta" in metadata
and isinstance(metadata["meta"], dict)
):
inner_meta = metadata["meta"]
if any(
key in inner_meta
for key in (
"resources",
"civitaiResources",
"additionalResources",
"hashes",
"prompt",
"negativePrompt",
)
):
metadata = inner_meta
# Initialize result structure
result = {
'base_model': None,
'loras': [],
'model': None,
'gen_params': {},
'from_civitai_image': True
}
@@ -53,6 +113,15 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Track already added LoRAs to prevent duplicates
added_loras = {} # key: model_version_id or hash, value: index in result["loras"]
# Extract hash information from hashes field for LoRA matching
lora_hashes = {}
if "hashes" in metadata and isinstance(metadata["hashes"], dict):
for key, hash_value in metadata["hashes"].items():
key_str = str(key)
if key_str.lower().startswith("lora:"):
lora_name = key_str.split(":", 1)[1]
lora_hashes[lora_name] = hash_value
# Extract prompt and negative prompt
if "prompt" in metadata:
result["gen_params"]["prompt"] = metadata["prompt"]
@@ -77,9 +146,9 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Extract base model information - directly if available
if "baseModel" in metadata:
result["base_model"] = metadata["baseModel"]
elif "Model hash" in metadata and civitai_client:
elif "Model hash" in metadata and metadata_provider:
model_hash = metadata["Model hash"]
model_info = await civitai_client.get_model_by_hash(model_hash)
model_info, error = await metadata_provider.get_model_by_hash(model_hash)
if model_info:
result["base_model"] = model_info.get("baseModel", "")
elif "Model" in metadata and isinstance(metadata.get("resources"), list):
@@ -87,8 +156,8 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
for resource in metadata.get("resources", []):
if resource.get("type") == "model" and resource.get("name") == metadata.get("Model"):
# This is likely the checkpoint model
if civitai_client and resource.get("hash"):
model_info = await civitai_client.get_model_by_hash(resource.get("hash"))
if metadata_provider and resource.get("hash"):
model_info, error = await metadata_provider.get_model_by_hash(resource.get("hash"))
if model_info:
result["base_model"] = model_info.get("baseModel", "")
@@ -101,6 +170,10 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
if resource.get("type", "lora") == "lora":
lora_hash = resource.get("hash", "")
# Try to get hash from the hashes field if not present in resource
if not lora_hash and resource.get("name"):
lora_hash = lora_hashes.get(resource["name"], "")
# Skip LoRAs without proper identification (hash or modelVersionId)
if not lora_hash and not resource.get("modelVersionId"):
logger.debug(f"Skipping LoRA resource '{resource.get('name', 'Unknown')}' - no hash or modelVersionId")
@@ -126,9 +199,9 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
}
# Try to get info from Civitai if hash is available
if lora_entry['hash'] and civitai_client:
if lora_entry['hash'] and metadata_provider:
try:
civitai_info = await civitai_client.get_model_by_hash(lora_hash)
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
@@ -158,13 +231,48 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Process civitaiResources array
if "civitaiResources" in metadata and isinstance(metadata["civitaiResources"], list):
for resource in metadata["civitaiResources"]:
# Get unique identifier for deduplication
# Get resource type and identifier
resource_type = str(resource.get("type") or "").lower()
version_id = str(resource.get("modelVersionId", ""))
if resource_type == "checkpoint":
checkpoint_entry = {
'id': resource.get("modelVersionId", 0),
'modelId': resource.get("modelId", 0),
'name': resource.get("modelName", "Unknown Checkpoint"),
'version': resource.get("modelVersionName", ""),
'type': resource.get("type", "checkpoint"),
'existsLocally': False,
'localPath': None,
'file_name': resource.get("modelName", ""),
'hash': resource.get("hash", "") or "",
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if version_id and metadata_provider:
try:
civitai_info = await metadata_provider.get_model_version_info(version_id)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint version {version_id}: {e}")
if result["model"] is None:
result["model"] = checkpoint_entry
continue
# Skip if we've already added this LoRA
if version_id and version_id in added_loras:
continue
# Initialize lora entry
lora_entry = {
'id': resource.get("modelVersionId", 0),
@@ -180,35 +288,31 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
'downloadUrl': '',
'isDeleted': False
}
# Try to get info from Civitai if modelVersionId is available
if version_id and civitai_client:
if version_id and metadata_provider:
try:
# Use get_model_version_info instead of get_model_version
civitai_info, error = await civitai_client.get_model_version_info(version_id)
if error:
logger.warning(f"Error getting model version info: {error}")
continue
civitai_info = await metadata_provider.get_model_version_info(version_id)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for model version {version_id}: {e}")
# Track this LoRA in our deduplication dict
if version_id:
added_loras[version_id] = len(result["loras"])
result["loras"].append(lora_entry)
# Process additionalResources array
@@ -247,35 +351,84 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
'isDeleted': False
}
# If we have a version ID and civitai client, try to get more info
if version_id and civitai_client:
# If we have a version ID and metadata provider, try to get more info
if version_id and metadata_provider:
try:
# Use get_model_version_info with the version ID
civitai_info, error = await civitai_client.get_model_version_info(version_id)
civitai_info = await metadata_provider.get_model_version_info(version_id)
if error:
logger.warning(f"Error getting model version info: {error}")
else:
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts
)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts
)
if populated_entry is None:
continue # Skip invalid LoRA types
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
# Track this LoRA for deduplication
if version_id:
added_loras[version_id] = len(result["loras"])
lora_entry = populated_entry
# Track this LoRA for deduplication
if version_id:
added_loras[version_id] = len(result["loras"])
except Exception as e:
logger.error(f"Error fetching Civitai info for model ID {version_id}: {e}")
result["loras"].append(lora_entry)
# If we found LoRA hashes in the metadata but haven't already
# populated entries for them, fall back to creating LoRAs from
# the hashes section. Some Civitai image responses only include
# LoRA information here without explicit resources entries.
for lora_name, lora_hash in lora_hashes.items():
if not lora_hash:
continue
# Skip LoRAs we've already added via resources or other fields
if lora_hash in added_loras:
continue
lora_entry = {
'name': lora_name,
'type': "lora",
'weight': 1.0,
'hash': lora_hash,
'existsLocally': False,
'localPath': None,
'file_name': lora_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash
)
if populated_entry is None:
continue
lora_entry = populated_entry
if 'id' in lora_entry and lora_entry['id']:
added_loras[str(lora_entry['id'])] = len(result["loras"])
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA hash {lora_hash}: {e}")
added_loras[lora_hash] = len(result["loras"])
result["loras"].append(lora_entry)
# Check for LoRA info in the format "Lora_0 Model hash", "Lora_0 Model name", etc.
lora_index = 0
while f"Lora_{lora_index} Model hash" in metadata and f"Lora_{lora_index} Model name" in metadata:
@@ -304,9 +457,9 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
}
# Try to get info from Civitai if hash is available
if lora_entry['hash'] and civitai_client:
if lora_entry['hash'] and metadata_provider:
try:
civitai_info = await civitai_client.get_model_by_hash(lora_hash)
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,

View File

@@ -6,6 +6,7 @@ import logging
from typing import Dict, Any
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
@@ -26,15 +27,15 @@ class ComfyMetadataParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Civitai ComfyUI metadata format"""
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
data = json.loads(user_comment)
loras = []
# Find all LoraLoader nodes
lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'}
if not lora_nodes:
return {"error": "No LoRA information found in this ComfyUI workflow", "loras": []}
# Process each LoraLoader node
for node_id, node in lora_nodes.items():
if 'inputs' not in node or 'lora_name' not in node['inputs']:
@@ -73,10 +74,10 @@ class ComfyMetadataParser(RecipeMetadataParser):
'isDeleted': False
}
# Get additional info from Civitai if client is available
if civitai_client:
# Get additional info from Civitai if metadata provider is available
if metadata_provider:
try:
civitai_info_tuple = await civitai_client.get_model_version_info(model_version_id)
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
@@ -116,9 +117,9 @@ class ComfyMetadataParser(RecipeMetadataParser):
}
# Get additional checkpoint info from Civitai
if civitai_client:
if metadata_provider:
try:
civitai_info_tuple = await civitai_client.get_model_version_info(checkpoint_version_id)
civitai_info_tuple = await metadata_provider.get_model_version_info(checkpoint_version_id)
civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
# Populate checkpoint with Civitai info
checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info)

View File

@@ -1,10 +1,12 @@
"""Parser for meta format (Lora_N Model hash) metadata."""
import os
import re
import logging
from typing import Dict, Any
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
@@ -18,8 +20,11 @@ class MetaFormatParser(RecipeMetadataParser):
return re.search(self.METADATA_MARKER, user_comment, re.IGNORECASE | re.DOTALL) is not None
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from images with meta format metadata"""
"""Parse metadata from images with meta format metadata (Lora_N Model hash format)"""
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Extract prompt and negative prompt
parts = user_comment.split('Negative prompt:', 1)
prompt = parts[0].strip()
@@ -122,9 +127,9 @@ class MetaFormatParser(RecipeMetadataParser):
}
# Get info from Civitai by hash if available
if civitai_client and hash_value:
if metadata_provider and hash_value:
try:
civitai_info = await civitai_client.get_model_by_hash(hash_value)
civitai_info = await metadata_provider.get_model_by_hash(hash_value)
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
@@ -141,14 +146,53 @@ class MetaFormatParser(RecipeMetadataParser):
loras.append(lora_entry)
# Extract model information
model = None
if 'model' in metadata:
model = metadata['model']
# Extract checkpoint information from generic Model/Model hash fields
checkpoint = None
model_hash = metadata.get("model_hash")
model_name = metadata.get("model")
if model_hash or model_name:
cleaned_name = None
if model_name:
cleaned_name = re.split(r"[\\\\/]", model_name)[-1]
cleaned_name = os.path.splitext(cleaned_name)[0]
checkpoint_entry = {
'id': 0,
'modelId': 0,
'name': model_name or "Unknown Checkpoint",
'version': '',
'type': 'checkpoint',
'hash': model_hash or "",
'existsLocally': False,
'localPath': None,
'file_name': cleaned_name or (model_name or ""),
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider and model_hash:
try:
civitai_info = await metadata_provider.get_model_by_hash(model_hash)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint hash {model_hash}: {e}")
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
checkpoint = checkpoint_entry
# Set base_model to the most common one from civitai_info
base_model = None
if base_model_counts:
# Set base_model to the most common one from civitai_info or checkpoint
base_model = checkpoint["baseModel"] if checkpoint and checkpoint.get("baseModel") else None
if not base_model and base_model_counts:
base_model = max(base_model_counts.items(), key=lambda x: x[1])[0]
# Extract generation parameters for recipe metadata
@@ -166,7 +210,8 @@ class MetaFormatParser(RecipeMetadataParser):
'loras': loras,
'gen_params': gen_params,
'raw_metadata': metadata,
'from_meta_format': True
'from_meta_format': True,
**({'checkpoint': checkpoint, 'model': checkpoint} if checkpoint else {})
}
except Exception as e:

View File

@@ -3,10 +3,11 @@
import re
import json
import logging
from typing import Dict, Any
from typing import Dict, Any, Optional
from ...config import config
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
@@ -15,6 +16,28 @@ class RecipeFormatParser(RecipeMetadataParser):
# Regular expression pattern for extracting recipe metadata
METADATA_MARKER = r'Recipe metadata: (\{.*\})'
async def _get_lora_from_version_index(self, recipe_scanner, model_version_id: Any) -> Optional[Dict[str, Any]]:
"""Return a cached LoRA entry by modelVersionId if available."""
if not recipe_scanner or not getattr(recipe_scanner, "_lora_scanner", None):
return None
try:
normalized_id = int(model_version_id)
except (TypeError, ValueError):
return None
try:
cache = await recipe_scanner._lora_scanner.get_cached_data()
except Exception as exc: # pragma: no cover - defensive logging
logger.debug("Unable to load lora cache for version lookup: %s", exc)
return None
if not cache or not getattr(cache, "version_index", None):
return None
return cache.version_index.get(normalized_id)
def is_metadata_matching(self, user_comment: str) -> bool:
"""Check if the user comment matches the metadata format"""
@@ -23,6 +46,9 @@ class RecipeFormatParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from images with dedicated recipe metadata format"""
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Extract recipe metadata from user comment
try:
# Look for recipe metadata section
@@ -49,49 +75,110 @@ class RecipeFormatParser(RecipeMetadataParser):
'type': 'lora',
'weight': lora.get('strength', 1.0),
'file_name': lora.get('file_name', ''),
'hash': lora.get('hash', '')
'hash': lora.get('hash', ''),
'existsLocally': False,
'inLibrary': False,
'localPath': None,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'size': 0
}
# Check if this LoRA exists locally by SHA256 hash
if lora.get('hash') and recipe_scanner:
if recipe_scanner:
lora_scanner = recipe_scanner._lora_scanner
exists_locally = lora_scanner.has_hash(lora['hash'])
if exists_locally:
lora_cache = await lora_scanner.get_cached_data()
lora_item = next((item for item in lora_cache.raw_data if item['sha256'].lower() == lora['hash'].lower()), None)
if lora_item:
if lora.get('hash'):
exists_locally = lora_scanner.has_hash(lora['hash'])
if exists_locally:
lora_cache = await lora_scanner.get_cached_data()
lora_item = next((item for item in lora_cache.raw_data if item['sha256'].lower() == lora['hash'].lower()), None)
if lora_item:
lora_entry['existsLocally'] = True
lora_entry['inLibrary'] = True
lora_entry['localPath'] = lora_item['file_path']
lora_entry['file_name'] = lora_item['file_name']
lora_entry['size'] = lora_item['size']
lora_entry['thumbnailUrl'] = config.get_preview_static_url(lora_item['preview_url'])
else:
lora_entry['existsLocally'] = False
lora_entry['inLibrary'] = False
lora_entry['localPath'] = None
# If we still don't have a local match, try matching by modelVersionId
if not lora_entry['existsLocally'] and lora.get('modelVersionId') is not None:
cached_lora = await self._get_lora_from_version_index(recipe_scanner, lora.get('modelVersionId'))
if cached_lora:
lora_entry['existsLocally'] = True
lora_entry['localPath'] = lora_item['file_path']
lora_entry['file_name'] = lora_item['file_name']
lora_entry['size'] = lora_item['size']
lora_entry['thumbnailUrl'] = config.get_preview_static_url(lora_item['preview_url'])
else:
lora_entry['existsLocally'] = False
lora_entry['localPath'] = None
# Try to get additional info from Civitai if we have a model version ID
if lora.get('modelVersionId') and civitai_client:
try:
civitai_info_tuple = await civitai_client.get_model_version_info(lora['modelVersionId'])
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner,
None, # No need to track base model counts
lora['hash']
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
lora_entry['inLibrary'] = True
lora_entry['localPath'] = cached_lora.get('file_path')
lora_entry['file_name'] = cached_lora.get('file_name') or lora_entry['file_name']
lora_entry['size'] = cached_lora.get('size', lora_entry['size'])
if cached_lora.get('sha256'):
lora_entry['hash'] = cached_lora['sha256']
preview_url = cached_lora.get('preview_url')
if preview_url:
lora_entry['thumbnailUrl'] = config.get_preview_static_url(preview_url)
# Try to get additional info from Civitai if we have a model version ID and still missing locally
if not lora_entry['existsLocally'] and lora.get('modelVersionId') and metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(lora['modelVersionId'])
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner,
None, # No need to track base model counts
lora_entry.get('hash', '')
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
loras.append(lora_entry)
logger.info(f"Found {len(loras)} loras in recipe metadata")
# Process checkpoint information if present
checkpoint = None
checkpoint_data = recipe_metadata.get('checkpoint') or {}
if isinstance(checkpoint_data, dict) and checkpoint_data:
version_id = checkpoint_data.get('modelVersionId') or checkpoint_data.get('id')
checkpoint_entry = {
'id': version_id or 0,
'modelId': checkpoint_data.get('modelId', 0),
'name': checkpoint_data.get('name', 'Unknown Checkpoint'),
'version': checkpoint_data.get('version', ''),
'type': checkpoint_data.get('type', 'checkpoint'),
'hash': checkpoint_data.get('hash', ''),
'existsLocally': False,
'localPath': None,
'file_name': checkpoint_data.get('file_name', ''),
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = None
if version_id:
civitai_info = await metadata_provider.get_model_version_info(str(version_id))
elif checkpoint_entry.get('hash'):
civitai_info = await metadata_provider.get_model_by_hash(checkpoint_entry['hash'])
if civitai_info:
checkpoint_entry = await self.populate_checkpoint_from_civitai(checkpoint_entry, civitai_info)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint in recipe metadata: {e}")
checkpoint = checkpoint_entry
# Filter gen_params to only include recognized keys
filtered_gen_params = {}
@@ -101,12 +188,13 @@ class RecipeFormatParser(RecipeMetadataParser):
filtered_gen_params[key] = value
return {
'base_model': recipe_metadata.get('base_model', ''),
'base_model': checkpoint['baseModel'] if checkpoint and checkpoint.get('baseModel') else recipe_metadata.get('base_model', ''),
'loras': loras,
'gen_params': filtered_gen_params,
'tags': recipe_metadata.get('tags', []),
'title': recipe_metadata.get('title', ''),
'from_recipe_metadata': True
'from_recipe_metadata': True,
**({'checkpoint': checkpoint, 'model': checkpoint} if checkpoint else {})
}
except Exception as e:

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,200 @@
"""Base infrastructure shared across recipe routes."""
from __future__ import annotations
import logging
import os
from typing import Callable, Mapping
import jinja2
from aiohttp import web
from ..config import config
from ..recipes import RecipeParserFactory
from ..services.downloader import get_downloader
from ..services.recipes import (
RecipeAnalysisService,
RecipePersistenceService,
RecipeSharingService,
)
from ..services.server_i18n import server_i18n
from ..services.service_registry import ServiceRegistry
from ..services.settings_manager import get_settings_manager
from ..utils.constants import CARD_PREVIEW_WIDTH
from ..utils.exif_utils import ExifUtils
from .handlers.recipe_handlers import (
RecipeAnalysisHandler,
RecipeHandlerSet,
RecipeListingHandler,
RecipeManagementHandler,
RecipePageView,
RecipeQueryHandler,
RecipeSharingHandler,
)
from .recipe_route_registrar import ROUTE_DEFINITIONS
logger = logging.getLogger(__name__)
class BaseRecipeRoutes:
"""Common dependency and startup wiring for recipe routes."""
_HANDLER_NAMES: tuple[str, ...] = tuple(
definition.handler_name for definition in ROUTE_DEFINITIONS
)
template_name: str = "recipes.html"
def __init__(self) -> None:
self.recipe_scanner = None
self.lora_scanner = None
self.civitai_client = None
self.settings = get_settings_manager()
self.server_i18n = server_i18n
self.template_env = jinja2.Environment(
loader=jinja2.FileSystemLoader(config.templates_path),
autoescape=True,
)
self._i18n_registered = False
self._startup_hooks_registered = False
self._handler_set: RecipeHandlerSet | None = None
self._handler_mapping: dict[str, Callable] | None = None
async def attach_dependencies(self, app: web.Application | None = None) -> None:
"""Resolve shared services from the registry."""
await self._ensure_services()
self._ensure_i18n_filter()
async def ensure_dependencies_ready(self) -> None:
"""Ensure dependencies are available for request handlers."""
if self.recipe_scanner is None or self.civitai_client is None:
await self.attach_dependencies()
def register_startup_hooks(self, app: web.Application) -> None:
"""Register startup hooks once for dependency wiring."""
if self._startup_hooks_registered:
return
app.on_startup.append(self.attach_dependencies)
self._startup_hooks_registered = True
def to_route_mapping(self) -> Mapping[str, Callable]:
"""Return a mapping of handler name to coroutine for registrar binding."""
if self._handler_mapping is None:
handler_set = self._create_handler_set()
self._handler_set = handler_set
self._handler_mapping = handler_set.to_route_mapping()
return self._handler_mapping
# Internal helpers -------------------------------------------------
async def _ensure_services(self) -> None:
if self.recipe_scanner is None:
self.recipe_scanner = await ServiceRegistry.get_recipe_scanner()
self.lora_scanner = getattr(self.recipe_scanner, "_lora_scanner", None)
if self.civitai_client is None:
self.civitai_client = await ServiceRegistry.get_civitai_client()
def _ensure_i18n_filter(self) -> None:
if not self._i18n_registered:
self.template_env.filters["t"] = self.server_i18n.create_template_filter()
self._i18n_registered = True
def get_handler_owner(self):
"""Return the object supplying bound handler coroutines."""
if self._handler_set is None:
self._handler_set = self._create_handler_set()
return self._handler_set
def _create_handler_set(self) -> RecipeHandlerSet:
recipe_scanner_getter = lambda: self.recipe_scanner
civitai_client_getter = lambda: self.civitai_client
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
if not standalone_mode:
from ..metadata_collector import get_metadata # type: ignore[import-not-found]
from ..metadata_collector.metadata_processor import ( # type: ignore[import-not-found]
MetadataProcessor,
)
from ..metadata_collector.metadata_registry import ( # type: ignore[import-not-found]
MetadataRegistry,
)
else: # pragma: no cover - optional dependency path
get_metadata = None # type: ignore[assignment]
MetadataProcessor = None # type: ignore[assignment]
MetadataRegistry = None # type: ignore[assignment]
analysis_service = RecipeAnalysisService(
exif_utils=ExifUtils,
recipe_parser_factory=RecipeParserFactory,
downloader_factory=get_downloader,
metadata_collector=get_metadata,
metadata_processor_cls=MetadataProcessor,
metadata_registry_cls=MetadataRegistry,
standalone_mode=standalone_mode,
logger=logger,
)
persistence_service = RecipePersistenceService(
exif_utils=ExifUtils,
card_preview_width=CARD_PREVIEW_WIDTH,
logger=logger,
)
sharing_service = RecipeSharingService(logger=logger)
page_view = RecipePageView(
ensure_dependencies_ready=self.ensure_dependencies_ready,
settings_service=self.settings,
server_i18n=self.server_i18n,
template_env=self.template_env,
template_name=self.template_name,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
)
listing = RecipeListingHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
)
query = RecipeQueryHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
format_recipe_file_url=listing.format_recipe_file_url,
logger=logger,
)
management = RecipeManagementHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
persistence_service=persistence_service,
analysis_service=analysis_service,
downloader_factory=get_downloader,
civitai_client_getter=civitai_client_getter,
)
analysis = RecipeAnalysisHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
civitai_client_getter=civitai_client_getter,
logger=logger,
analysis_service=analysis_service,
)
sharing = RecipeSharingHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
sharing_service=sharing_service,
)
return RecipeHandlerSet(
page_view=page_view,
listing=listing,
query=query,
management=management,
analysis=analysis,
sharing=sharing,
)

View File

@@ -1,7 +1,9 @@
import logging
from typing import Dict
from aiohttp import web
from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.checkpoint_service import CheckpointService
from ..services.service_registry import ServiceRegistry
from ..config import config
@@ -13,19 +15,18 @@ class CheckpointRoutes(BaseModelRoutes):
def __init__(self):
"""Initialize Checkpoint routes with Checkpoint service"""
# Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
super().__init__()
self.template_name = "checkpoints.html"
async def initialize_services(self):
"""Initialize services from ServiceRegistry"""
checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
self.service = CheckpointService(checkpoint_scanner)
self.civitai_client = await ServiceRegistry.get_civitai_client()
# Initialize parent with the service
super().__init__(self.service)
update_service = await ServiceRegistry.get_model_update_service()
self.service = CheckpointService(checkpoint_scanner, update_service=update_service)
self.set_model_update_service(update_service)
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
"""Setup Checkpoint routes"""
@@ -35,17 +36,35 @@ class CheckpointRoutes(BaseModelRoutes):
# Setup common routes with 'checkpoints' prefix (includes page route)
super().setup_routes(app, 'checkpoints')
def setup_specific_routes(self, app: web.Application, prefix: str):
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup Checkpoint-specific routes"""
# Checkpoint-specific CivitAI integration
app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_checkpoint)
# Checkpoint info by name
app.router.add_get(f'/api/{prefix}/info/{{name}}', self.get_checkpoint_info)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/info/{name}', prefix, self.get_checkpoint_info)
# Checkpoint roots and Unet roots
app.router.add_get(f'/api/{prefix}/checkpoints_roots', self.get_checkpoints_roots)
app.router.add_get(f'/api/{prefix}/unet_roots', self.get_unet_roots)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/checkpoints_roots', prefix, self.get_checkpoints_roots)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/unet_roots', prefix, self.get_unet_roots)
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for Checkpoint"""
return model_type.lower() == 'checkpoint'
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "Checkpoint"
def _parse_specific_params(self, request: web.Request) -> Dict:
"""Parse Checkpoint-specific parameters"""
params: Dict = {}
if 'checkpoint_hash' in request.query:
params['hash_filters'] = {'single_hash': request.query['checkpoint_hash'].lower()}
elif 'checkpoint_hashes' in request.query:
params['hash_filters'] = {
'multiple_hashes': [h.lower() for h in request.query['checkpoint_hashes'].split(',')]
}
return params
async def get_checkpoint_info(self, request: web.Request) -> web.Response:
"""Get detailed information for a specific checkpoint by name"""
@@ -62,53 +81,6 @@ class CheckpointRoutes(BaseModelRoutes):
logger.error(f"Error in get_checkpoint_info: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500)
async def get_civitai_versions_checkpoint(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai checkpoint model with local availability info"""
try:
model_id = request.match_info['model_id']
response = await self.civitai_client.get_model_versions(model_id)
if not response or not response.get('modelVersions'):
return web.Response(status=404, text="Model not found")
versions = response.get('modelVersions', [])
model_type = response.get('type', '')
# Check model type - should be Checkpoint
if model_type.lower() != 'checkpoint':
return web.json_response({
'error': f"Model type mismatch. Expected Checkpoint, got {model_type}"
}, status=400)
# Check local availability for each version
for version in versions:
# Find the primary model file (type="Model" and primary=true) in the files list
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model' and file.get('primary') == True), None)
# If no primary file found, try to find any model file
if not model_file:
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model'), None)
if model_file:
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
# Set existsLocally and localPath at the version level
version['existsLocally'] = self.service.has_hash(sha256)
if version['existsLocally']:
version['localPath'] = self.service.get_path_by_hash(sha256)
# Also set the model file size at the version level for easier access
version['modelSizeKB'] = model_file.get('sizeKB')
else:
# No model file found in this version
version['existsLocally'] = False
return web.json_response(versions)
except Exception as e:
logger.error(f"Error fetching checkpoint model versions: {e}")
return web.Response(status=500, text=str(e))
async def get_checkpoints_roots(self, request: web.Request) -> web.Response:
"""Return the list of checkpoint roots from config"""
try:
@@ -137,4 +109,4 @@ class CheckpointRoutes(BaseModelRoutes):
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
}, status=500)

View File

@@ -2,6 +2,7 @@ import logging
from aiohttp import web
from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.embedding_service import EmbeddingService
from ..services.service_registry import ServiceRegistry
@@ -12,19 +13,18 @@ class EmbeddingRoutes(BaseModelRoutes):
def __init__(self):
"""Initialize Embedding routes with Embedding service"""
# Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
super().__init__()
self.template_name = "embeddings.html"
async def initialize_services(self):
"""Initialize services from ServiceRegistry"""
embedding_scanner = await ServiceRegistry.get_embedding_scanner()
self.service = EmbeddingService(embedding_scanner)
self.civitai_client = await ServiceRegistry.get_civitai_client()
# Initialize parent with the service
super().__init__(self.service)
update_service = await ServiceRegistry.get_model_update_service()
self.service = EmbeddingService(embedding_scanner, update_service=update_service)
self.set_model_update_service(update_service)
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
"""Setup Embedding routes"""
@@ -34,13 +34,18 @@ class EmbeddingRoutes(BaseModelRoutes):
# Setup common routes with 'embeddings' prefix (includes page route)
super().setup_routes(app, 'embeddings')
def setup_specific_routes(self, app: web.Application, prefix: str):
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup Embedding-specific routes"""
# Embedding-specific CivitAI integration
app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_embedding)
# Embedding info by name
app.router.add_get(f'/api/{prefix}/info/{{name}}', self.get_embedding_info)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/info/{name}', prefix, self.get_embedding_info)
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for Embedding"""
return model_type.lower() == 'textualinversion'
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "TextualInversion"
async def get_embedding_info(self, request: web.Request) -> web.Response:
"""Get detailed information for a specific embedding by name"""
@@ -56,50 +61,3 @@ class EmbeddingRoutes(BaseModelRoutes):
except Exception as e:
logger.error(f"Error in get_embedding_info: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500)
async def get_civitai_versions_embedding(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai embedding model with local availability info"""
try:
model_id = request.match_info['model_id']
response = await self.civitai_client.get_model_versions(model_id)
if not response or not response.get('modelVersions'):
return web.Response(status=404, text="Model not found")
versions = response.get('modelVersions', [])
model_type = response.get('type', '')
# Check model type - should be TextualInversion (Embedding)
if model_type.lower() not in ['textualinversion', 'embedding']:
return web.json_response({
'error': f"Model type mismatch. Expected TextualInversion/Embedding, got {model_type}"
}, status=400)
# Check local availability for each version
for version in versions:
# Find the primary model file (type="Model" and primary=true) in the files list
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model' and file.get('primary') == True), None)
# If no primary file found, try to find any model file
if not model_file:
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model'), None)
if model_file:
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
# Set existsLocally and localPath at the version level
version['existsLocally'] = self.service.has_hash(sha256)
if version['existsLocally']:
version['localPath'] = self.service.get_path_by_hash(sha256)
# Also set the model file size at the version level for easier access
version['modelSizeKB'] = model_file.get('sizeKB')
else:
# No model file found in this version
version['existsLocally'] = False
return web.json_response(versions)
except Exception as e:
logger.error(f"Error fetching embedding model versions: {e}")
return web.Response(status=500, text=str(e))

View File

@@ -0,0 +1,64 @@
"""Route registrar for example image endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative configuration for a HTTP route."""
method: str
path: str
handler_name: str
ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("POST", "/api/lm/download-example-images", "download_example_images"),
RouteDefinition("POST", "/api/lm/import-example-images", "import_example_images"),
RouteDefinition("GET", "/api/lm/example-images-status", "get_example_images_status"),
RouteDefinition("POST", "/api/lm/pause-example-images", "pause_example_images"),
RouteDefinition("POST", "/api/lm/resume-example-images", "resume_example_images"),
RouteDefinition("POST", "/api/lm/stop-example-images", "stop_example_images"),
RouteDefinition("POST", "/api/lm/open-example-images-folder", "open_example_images_folder"),
RouteDefinition("GET", "/api/lm/example-image-files", "get_example_image_files"),
RouteDefinition("GET", "/api/lm/has-example-images", "has_example_images"),
RouteDefinition("POST", "/api/lm/delete-example-image", "delete_example_image"),
RouteDefinition("POST", "/api/lm/force-download-example-images", "force_download_example_images"),
RouteDefinition("POST", "/api/lm/cleanup-example-image-folders", "cleanup_example_image_folders"),
RouteDefinition("POST", "/api/lm/example-images/set-nsfw-level", "set_example_image_nsfw_level"),
)
class ExampleImagesRouteRegistrar:
"""Bind declarative example image routes to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(
self,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = ROUTE_DEFINITIONS,
) -> None:
"""Register each route definition using the supplied handlers."""
for definition in definitions:
handler = handler_lookup[definition.handler_name]
self._bind_route(definition.method, definition.path, handler)
def _bind_route(self, method: str, path: str, handler: Callable[[web.Request], object]) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -1,74 +1,88 @@
from __future__ import annotations
import logging
from ..utils.example_images_download_manager import DownloadManager
from ..utils.example_images_processor import ExampleImagesProcessor
from typing import Callable, Mapping
from aiohttp import web
from .example_images_route_registrar import ExampleImagesRouteRegistrar
from .handlers.example_images_handlers import (
ExampleImagesDownloadHandler,
ExampleImagesFileHandler,
ExampleImagesHandlerSet,
ExampleImagesManagementHandler,
)
from ..services.use_cases.example_images import (
DownloadExampleImagesUseCase,
ImportExampleImagesUseCase,
)
from ..utils.example_images_download_manager import (
DownloadManager,
get_default_download_manager,
)
from ..utils.example_images_file_manager import ExampleImagesFileManager
from ..services.websocket_manager import ws_manager
from ..utils.example_images_processor import ExampleImagesProcessor
from ..services.example_images_cleanup_service import ExampleImagesCleanupService
logger = logging.getLogger(__name__)
class ExampleImagesRoutes:
"""Routes for example images related functionality"""
@staticmethod
def setup_routes(app):
"""Register example images routes"""
app.router.add_post('/api/download-example-images', ExampleImagesRoutes.download_example_images)
app.router.add_post('/api/import-example-images', ExampleImagesRoutes.import_example_images)
app.router.add_get('/api/example-images-status', ExampleImagesRoutes.get_example_images_status)
app.router.add_post('/api/pause-example-images', ExampleImagesRoutes.pause_example_images)
app.router.add_post('/api/resume-example-images', ExampleImagesRoutes.resume_example_images)
app.router.add_post('/api/open-example-images-folder', ExampleImagesRoutes.open_example_images_folder)
app.router.add_get('/api/example-image-files', ExampleImagesRoutes.get_example_image_files)
app.router.add_get('/api/has-example-images', ExampleImagesRoutes.has_example_images)
app.router.add_post('/api/delete-example-image', ExampleImagesRoutes.delete_example_image)
app.router.add_post('/api/force-download-example-images', ExampleImagesRoutes.force_download_example_images)
"""Route controller for example image endpoints."""
@staticmethod
async def download_example_images(request):
"""Download example images for models from Civitai"""
return await DownloadManager.start_download(request)
def __init__(
self,
*,
ws_manager,
download_manager: DownloadManager | None = None,
processor=ExampleImagesProcessor,
file_manager=ExampleImagesFileManager,
cleanup_service: ExampleImagesCleanupService | None = None,
) -> None:
if ws_manager is None:
raise ValueError("ws_manager is required")
self._download_manager = download_manager or get_default_download_manager(ws_manager)
self._processor = processor
self._file_manager = file_manager
self._cleanup_service = cleanup_service or ExampleImagesCleanupService()
self._handler_set: ExampleImagesHandlerSet | None = None
self._handler_mapping: Mapping[str, Callable[[web.Request], web.StreamResponse]] | None = None
@staticmethod
async def get_example_images_status(request):
"""Get the current status of example images download"""
return await DownloadManager.get_status(request)
@classmethod
def setup_routes(cls, app: web.Application, *, ws_manager) -> None:
"""Register routes on the given aiohttp application using default wiring."""
@staticmethod
async def pause_example_images(request):
"""Pause the example images download"""
return await DownloadManager.pause_download(request)
controller = cls(ws_manager=ws_manager)
controller.register(app)
@staticmethod
async def resume_example_images(request):
"""Resume the example images download"""
return await DownloadManager.resume_download(request)
@staticmethod
async def open_example_images_folder(request):
"""Open the example images folder for a specific model"""
return await ExampleImagesFileManager.open_folder(request)
def register(self, app: web.Application) -> None:
"""Bind the controller's handlers to the aiohttp router."""
@staticmethod
async def get_example_image_files(request):
"""Get list of example image files for a specific model"""
return await ExampleImagesFileManager.get_files(request)
registrar = ExampleImagesRouteRegistrar(app)
registrar.register_routes(self.to_route_mapping())
@staticmethod
async def import_example_images(request):
"""Import local example images for a model"""
return await ExampleImagesProcessor.import_images(request)
@staticmethod
async def has_example_images(request):
"""Check if example images folder exists and is not empty for a model"""
return await ExampleImagesFileManager.has_images(request)
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
"""Return the registrar-compatible mapping of handler names to callables."""
@staticmethod
async def delete_example_image(request):
"""Delete a custom example image for a model"""
return await ExampleImagesProcessor.delete_custom_image(request)
if self._handler_mapping is None:
handler_set = self._build_handler_set()
self._handler_set = handler_set
self._handler_mapping = handler_set.to_route_mapping()
return self._handler_mapping
@staticmethod
async def force_download_example_images(request):
"""Force download example images for specific models"""
return await DownloadManager.start_force_download(request)
def _build_handler_set(self) -> ExampleImagesHandlerSet:
logger.debug("Building ExampleImagesHandlerSet with %s, %s, %s", self._download_manager, self._processor, self._file_manager)
download_use_case = DownloadExampleImagesUseCase(download_manager=self._download_manager)
download_handler = ExampleImagesDownloadHandler(download_use_case, self._download_manager)
import_use_case = ImportExampleImagesUseCase(processor=self._processor)
management_handler = ExampleImagesManagementHandler(
import_use_case,
self._processor,
self._cleanup_service,
)
file_handler = ExampleImagesFileHandler(self._file_manager)
return ExampleImagesHandlerSet(
download=download_handler,
management=management_handler,
files=file_handler,
)

View File

@@ -0,0 +1,171 @@
"""Handler set for example image routes."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Mapping
from aiohttp import web
from ...services.use_cases.example_images import (
DownloadExampleImagesConfigurationError,
DownloadExampleImagesInProgressError,
DownloadExampleImagesUseCase,
ImportExampleImagesUseCase,
ImportExampleImagesValidationError,
)
from ...utils.example_images_download_manager import (
DownloadConfigurationError,
DownloadInProgressError,
DownloadNotRunningError,
ExampleImagesDownloadError,
)
from ...utils.example_images_processor import ExampleImagesImportError
class ExampleImagesDownloadHandler:
"""HTTP adapters for download-related example image endpoints."""
def __init__(
self,
download_use_case: DownloadExampleImagesUseCase,
download_manager,
) -> None:
self._download_use_case = download_use_case
self._download_manager = download_manager
async def download_example_images(self, request: web.Request) -> web.StreamResponse:
try:
payload = await request.json()
result = await self._download_use_case.execute(payload)
return web.json_response(result)
except DownloadExampleImagesInProgressError as exc:
response = {
'success': False,
'error': str(exc),
'status': exc.progress,
}
return web.json_response(response, status=400)
except DownloadExampleImagesConfigurationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesDownloadError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
async def get_example_images_status(self, request: web.Request) -> web.StreamResponse:
result = await self._download_manager.get_status(request)
return web.json_response(result)
async def pause_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.pause_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def resume_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.resume_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def stop_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.stop_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def force_download_example_images(self, request: web.Request) -> web.StreamResponse:
try:
payload = await request.json()
result = await self._download_manager.start_force_download(payload)
return web.json_response(result)
except DownloadInProgressError as exc:
response = {
'success': False,
'error': str(exc),
'status': exc.progress_snapshot,
}
return web.json_response(response, status=400)
except DownloadConfigurationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesDownloadError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
class ExampleImagesManagementHandler:
"""HTTP adapters for import/delete endpoints."""
def __init__(self, import_use_case: ImportExampleImagesUseCase, processor, cleanup_service) -> None:
self._import_use_case = import_use_case
self._processor = processor
self._cleanup_service = cleanup_service
async def import_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._import_use_case.execute(request)
return web.json_response(result)
except ImportExampleImagesValidationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesImportError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
async def delete_example_image(self, request: web.Request) -> web.StreamResponse:
return await self._processor.delete_custom_image(request)
async def set_example_image_nsfw_level(self, request: web.Request) -> web.StreamResponse:
return await self._processor.set_example_image_nsfw_level(request)
async def cleanup_example_image_folders(self, request: web.Request) -> web.StreamResponse:
result = await self._cleanup_service.cleanup_example_image_folders()
if result.get('success') or result.get('partial_success'):
return web.json_response(result, status=200)
error_code = result.get('error_code')
status = 400 if error_code in {'path_not_configured', 'path_not_found'} else 500
return web.json_response(result, status=status)
class ExampleImagesFileHandler:
"""HTTP adapters for filesystem-centric endpoints."""
def __init__(self, file_manager) -> None:
self._file_manager = file_manager
async def open_example_images_folder(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.open_folder(request)
async def get_example_image_files(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.get_files(request)
async def has_example_images(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.has_images(request)
@dataclass(frozen=True)
class ExampleImagesHandlerSet:
"""Aggregate of handlers exposed to the registrar."""
download: ExampleImagesDownloadHandler
management: ExampleImagesManagementHandler
files: ExampleImagesFileHandler
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
"""Flatten handler methods into the registrar mapping."""
return {
"download_example_images": self.download.download_example_images,
"get_example_images_status": self.download.get_example_images_status,
"pause_example_images": self.download.pause_example_images,
"resume_example_images": self.download.resume_example_images,
"stop_example_images": self.download.stop_example_images,
"force_download_example_images": self.download.force_download_example_images,
"import_example_images": self.management.import_example_images,
"delete_example_image": self.management.delete_example_image,
"set_example_image_nsfw_level": self.management.set_example_image_nsfw_level,
"cleanup_example_image_folders": self.management.cleanup_example_image_folders,
"open_example_images_folder": self.files.open_example_images_folder,
"get_example_image_files": self.files.get_example_image_files,
"has_example_images": self.files.has_example_images,
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,57 @@
"""Handlers responsible for serving preview assets dynamically."""
from __future__ import annotations
import logging
import urllib.parse
from pathlib import Path
from aiohttp import web
from ...config import config as global_config
logger = logging.getLogger(__name__)
class PreviewHandler:
"""Serve preview assets for the active library at request time."""
def __init__(self, *, config=global_config) -> None:
self._config = config
async def serve_preview(self, request: web.Request) -> web.StreamResponse:
"""Return the preview file referenced by the encoded ``path`` query."""
raw_path = request.query.get("path", "")
if not raw_path:
raise web.HTTPBadRequest(text="Missing 'path' query parameter")
try:
decoded_path = urllib.parse.unquote(raw_path)
except Exception as exc: # pragma: no cover - defensive guard
logger.debug("Failed to decode preview path %s: %s", raw_path, exc)
raise web.HTTPBadRequest(text="Invalid preview path encoding") from exc
normalized = decoded_path.replace("\\", "/")
candidate = Path(normalized)
try:
resolved = candidate.expanduser().resolve(strict=False)
except Exception as exc:
logger.debug("Failed to resolve preview path %s: %s", normalized, exc)
raise web.HTTPBadRequest(text="Unable to resolve preview path") from exc
resolved_str = str(resolved)
# TODO: Temporarily disabled path validation due to issues #772 and #774
# Re-enable after fixing preview root path handling
# if not self._config.is_preview_path_allowed(resolved_str):
# raise web.HTTPForbidden(text="Preview path is not within an allowed directory")
if not resolved.is_file():
logger.debug("Preview file not found at %s", resolved_str)
raise web.HTTPNotFound(text="Preview file not found")
# aiohttp's FileResponse handles range requests and content headers for us.
return web.FileResponse(path=resolved, chunk_size=256 * 1024)
__all__ = ["PreviewHandler"]

File diff suppressed because it is too large Load Diff

View File

@@ -5,394 +5,323 @@ from typing import Dict
from server import PromptServer # type: ignore
from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.lora_service import LoraService
from ..services.service_registry import ServiceRegistry
from ..utils.routes_common import ModelRouteUtils
from ..utils.utils import get_lora_info
logger = logging.getLogger(__name__)
class LoraRoutes(BaseModelRoutes):
"""LoRA-specific route controller"""
def __init__(self):
"""Initialize LoRA routes with LoRA service"""
# Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
super().__init__()
self.template_name = "loras.html"
async def initialize_services(self):
"""Initialize services from ServiceRegistry"""
lora_scanner = await ServiceRegistry.get_lora_scanner()
self.service = LoraService(lora_scanner)
self.civitai_client = await ServiceRegistry.get_civitai_client()
# Initialize parent with the service
super().__init__(self.service)
update_service = await ServiceRegistry.get_model_update_service()
self.service = LoraService(lora_scanner, update_service=update_service)
self.set_model_update_service(update_service)
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
"""Setup LoRA routes"""
# Schedule service initialization on app startup
app.on_startup.append(lambda _: self.initialize_services())
# Setup common routes with 'loras' prefix (includes page route)
super().setup_routes(app, 'loras')
def setup_specific_routes(self, app: web.Application, prefix: str):
super().setup_routes(app, "loras")
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup LoRA-specific routes"""
# LoRA-specific query routes
app.router.add_get(f'/api/{prefix}/letter-counts', self.get_letter_counts)
app.router.add_get(f'/api/{prefix}/get-trigger-words', self.get_lora_trigger_words)
app.router.add_get(f'/api/{prefix}/model-description', self.get_lora_model_description)
app.router.add_get(f'/api/{prefix}/usage-tips-by-path', self.get_lora_usage_tips_by_path)
# CivitAI integration with LoRA-specific validation
app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_lora)
app.router.add_get(f'/api/{prefix}/civitai/model/version/{{modelVersionId}}', self.get_civitai_model_by_version)
app.router.add_get(f'/api/{prefix}/civitai/model/hash/{{hash}}', self.get_civitai_model_by_hash)
registrar.add_prefixed_route(
"GET", "/api/lm/{prefix}/letter-counts", prefix, self.get_letter_counts
)
registrar.add_prefixed_route(
"GET",
"/api/lm/{prefix}/get-trigger-words",
prefix,
self.get_lora_trigger_words,
)
registrar.add_prefixed_route(
"GET",
"/api/lm/{prefix}/usage-tips-by-path",
prefix,
self.get_lora_usage_tips_by_path,
)
# Randomizer routes
registrar.add_prefixed_route(
"POST", "/api/lm/{prefix}/random-sample", prefix, self.get_random_loras
)
# ComfyUI integration
app.router.add_post(f'/api/{prefix}/get_trigger_words', self.get_trigger_words)
registrar.add_prefixed_route(
"POST", "/api/lm/{prefix}/get_trigger_words", prefix, self.get_trigger_words
)
def _parse_specific_params(self, request: web.Request) -> Dict:
"""Parse LoRA-specific parameters"""
params = {}
# LoRA-specific parameters
if 'first_letter' in request.query:
params['first_letter'] = request.query.get('first_letter')
if "first_letter" in request.query:
params["first_letter"] = request.query.get("first_letter")
# Handle fuzzy search parameter name variation
if request.query.get('fuzzy') == 'true':
params['fuzzy_search'] = True
if request.query.get("fuzzy") == "true":
params["fuzzy_search"] = True
# Handle additional filter parameters for LoRAs
if 'lora_hash' in request.query:
if not params.get('hash_filters'):
params['hash_filters'] = {}
params['hash_filters']['single_hash'] = request.query['lora_hash'].lower()
elif 'lora_hashes' in request.query:
if not params.get('hash_filters'):
params['hash_filters'] = {}
params['hash_filters']['multiple_hashes'] = [h.lower() for h in request.query['lora_hashes'].split(',')]
if "lora_hash" in request.query:
if not params.get("hash_filters"):
params["hash_filters"] = {}
params["hash_filters"]["single_hash"] = request.query["lora_hash"].lower()
elif "lora_hashes" in request.query:
if not params.get("hash_filters"):
params["hash_filters"] = {}
params["hash_filters"]["multiple_hashes"] = [
h.lower() for h in request.query["lora_hashes"].split(",")
]
return params
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for LoRA"""
from ..utils.constants import VALID_LORA_TYPES
return model_type.lower() in VALID_LORA_TYPES
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "LORA, LoCon, or DORA"
# LoRA-specific route handlers
async def get_letter_counts(self, request: web.Request) -> web.Response:
"""Get count of LoRAs for each letter of the alphabet"""
try:
letter_counts = await self.service.get_letter_counts()
return web.json_response({
'success': True,
'letter_counts': letter_counts
})
return web.json_response({"success": True, "letter_counts": letter_counts})
except Exception as e:
logger.error(f"Error getting letter counts: {e}")
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_notes(self, request: web.Request) -> web.Response:
"""Get notes for a specific LoRA file"""
try:
lora_name = request.query.get('name')
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text='Lora file name is required', status=400)
return web.Response(text="Lora file name is required", status=400)
notes = await self.service.get_lora_notes(lora_name)
if notes is not None:
return web.json_response({
'success': True,
'notes': notes
})
return web.json_response({"success": True, "notes": notes})
else:
return web.json_response({
'success': False,
'error': 'LoRA not found in cache'
}, status=404)
return web.json_response(
{"success": False, "error": "LoRA not found in cache"}, status=404
)
except Exception as e:
logger.error(f"Error getting lora notes: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_trigger_words(self, request: web.Request) -> web.Response:
"""Get trigger words for a specific LoRA file"""
try:
lora_name = request.query.get('name')
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text='Lora file name is required', status=400)
return web.Response(text="Lora file name is required", status=400)
trigger_words = await self.service.get_lora_trigger_words(lora_name)
return web.json_response({
'success': True,
'trigger_words': trigger_words
})
return web.json_response({"success": True, "trigger_words": trigger_words})
except Exception as e:
logger.error(f"Error getting lora trigger words: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_usage_tips_by_path(self, request: web.Request) -> web.Response:
"""Get usage tips for a LoRA by its relative path"""
try:
relative_path = request.query.get('relative_path')
relative_path = request.query.get("relative_path")
if not relative_path:
return web.Response(text='Relative path is required', status=400)
usage_tips = await self.service.get_lora_usage_tips_by_relative_path(relative_path)
return web.json_response({
'success': True,
'usage_tips': usage_tips or ''
})
return web.Response(text="Relative path is required", status=400)
usage_tips = await self.service.get_lora_usage_tips_by_relative_path(
relative_path
)
return web.json_response({"success": True, "usage_tips": usage_tips or ""})
except Exception as e:
logger.error(f"Error getting lora usage tips by path: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_preview_url(self, request: web.Request) -> web.Response:
"""Get the static preview URL for a LoRA file"""
try:
lora_name = request.query.get('name')
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text='Lora file name is required', status=400)
return web.Response(text="Lora file name is required", status=400)
preview_url = await self.service.get_lora_preview_url(lora_name)
if preview_url:
return web.json_response({
'success': True,
'preview_url': preview_url
})
return web.json_response({"success": True, "preview_url": preview_url})
else:
return web.json_response({
'success': False,
'error': 'No preview URL found for the specified lora'
}, status=404)
return web.json_response(
{
"success": False,
"error": "No preview URL found for the specified lora",
},
status=404,
)
except Exception as e:
logger.error(f"Error getting lora preview URL: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_civitai_url(self, request: web.Request) -> web.Response:
"""Get the Civitai URL for a LoRA file"""
try:
lora_name = request.query.get('name')
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text='Lora file name is required', status=400)
return web.Response(text="Lora file name is required", status=400)
result = await self.service.get_lora_civitai_url(lora_name)
if result['civitai_url']:
return web.json_response({
'success': True,
**result
})
if result["civitai_url"]:
return web.json_response({"success": True, **result})
else:
return web.json_response({
'success': False,
'error': 'No Civitai data found for the specified lora'
}, status=404)
return web.json_response(
{
"success": False,
"error": "No Civitai data found for the specified lora",
},
status=404,
)
except Exception as e:
logger.error(f"Error getting lora Civitai URL: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
# CivitAI integration methods
async def get_civitai_versions_lora(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai LoRA model with local availability info"""
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_random_loras(self, request: web.Request) -> web.Response:
"""Get random LoRAs based on filters and strength ranges"""
try:
model_id = request.match_info['model_id']
response = await self.civitai_client.get_model_versions(model_id)
if not response or not response.get('modelVersions'):
return web.Response(status=404, text="Model not found")
versions = response.get('modelVersions', [])
model_type = response.get('type', '')
# Check model type - should be LORA, LoCon, or DORA
from ..utils.constants import VALID_LORA_TYPES
if model_type.lower() not in VALID_LORA_TYPES:
return web.json_response({
'error': f"Model type mismatch. Expected LORA or LoCon, got {model_type}"
}, status=400)
# Check local availability for each version
for version in versions:
# Find the model file (type="Model") in the files list
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model'), None)
if model_file:
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
# Set existsLocally and localPath at the version level
version['existsLocally'] = self.service.has_hash(sha256)
if version['existsLocally']:
version['localPath'] = self.service.get_path_by_hash(sha256)
# Also set the model file size at the version level for easier access
version['modelSizeKB'] = model_file.get('sizeKB')
else:
# No model file found in this version
version['existsLocally'] = False
return web.json_response(versions)
json_data = await request.json()
# Parse parameters
count = json_data.get("count", 5)
count_min = json_data.get("count_min")
count_max = json_data.get("count_max")
model_strength_min = float(json_data.get("model_strength_min", 0.0))
model_strength_max = float(json_data.get("model_strength_max", 1.0))
use_same_clip_strength = json_data.get("use_same_clip_strength", True)
clip_strength_min = float(json_data.get("clip_strength_min", 0.0))
clip_strength_max = float(json_data.get("clip_strength_max", 1.0))
locked_loras = json_data.get("locked_loras", [])
pool_config = json_data.get("pool_config")
use_recommended_strength = json_data.get("use_recommended_strength", False)
recommended_strength_scale_min = float(
json_data.get("recommended_strength_scale_min", 0.5)
)
recommended_strength_scale_max = float(
json_data.get("recommended_strength_scale_max", 1.0)
)
# Determine target count
if count_min is not None and count_max is not None:
import random
target_count = random.randint(count_min, count_max)
else:
target_count = count
# Validate parameters
if target_count < 1 or target_count > 100:
return web.json_response(
{"success": False, "error": "Count must be between 1 and 100"},
status=400,
)
if model_strength_min < -10 or model_strength_max > 10:
return web.json_response(
{
"success": False,
"error": "Model strength must be between -10 and 10",
},
status=400,
)
# Get random LoRAs from service
result_loras = await self.service.get_random_loras(
count=target_count,
model_strength_min=model_strength_min,
model_strength_max=model_strength_max,
use_same_clip_strength=use_same_clip_strength,
clip_strength_min=clip_strength_min,
clip_strength_max=clip_strength_max,
locked_loras=locked_loras,
pool_config=pool_config,
use_recommended_strength=use_recommended_strength,
recommended_strength_scale_min=recommended_strength_scale_min,
recommended_strength_scale_max=recommended_strength_scale_max,
)
return web.json_response(
{"success": True, "loras": result_loras, "count": len(result_loras)}
)
except ValueError as e:
logger.error(f"Invalid parameter for random LoRAs: {e}")
return web.json_response({"success": False, "error": str(e)}, status=400)
except Exception as e:
logger.error(f"Error fetching LoRA model versions: {e}")
return web.Response(status=500, text=str(e))
async def get_civitai_model_by_version(self, request: web.Request) -> web.Response:
"""Get CivitAI model details by model version ID"""
try:
model_version_id = request.match_info.get('modelVersionId')
# Get model details from Civitai API
model, error_msg = await self.civitai_client.get_model_version_info(model_version_id)
if not model:
# Log warning for failed model retrieval
logger.warning(f"Failed to fetch model version {model_version_id}: {error_msg}")
# Determine status code based on error message
status_code = 404 if error_msg and "not found" in error_msg.lower() else 500
return web.json_response({
"success": False,
"error": error_msg or "Failed to fetch model information"
}, status=status_code)
return web.json_response(model)
except Exception as e:
logger.error(f"Error fetching model details: {e}")
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
async def get_civitai_model_by_hash(self, request: web.Request) -> web.Response:
"""Get CivitAI model details by hash"""
try:
hash = request.match_info.get('hash')
model = await self.civitai_client.get_model_by_hash(hash)
return web.json_response(model)
except Exception as e:
logger.error(f"Error fetching model details by hash: {e}")
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
async def get_lora_model_description(self, request: web.Request) -> web.Response:
"""Get model description for a Lora model"""
try:
# Get parameters
model_id = request.query.get('model_id')
file_path = request.query.get('file_path')
if not model_id:
return web.json_response({
'success': False,
'error': 'Model ID is required'
}, status=400)
# Check if we already have the description stored in metadata
description = None
tags = []
creator = {}
if file_path:
import os
from ..utils.metadata_manager import MetadataManager
metadata_path = os.path.splitext(file_path)[0] + '.metadata.json'
metadata = await ModelRouteUtils.load_local_metadata(metadata_path)
description = metadata.get('modelDescription')
tags = metadata.get('tags', [])
creator = metadata.get('creator', {})
# If description is not in metadata, fetch from CivitAI
if not description:
logger.info(f"Fetching model metadata for model ID: {model_id}")
model_metadata, _ = await self.civitai_client.get_model_metadata(model_id)
if model_metadata:
description = model_metadata.get('description')
tags = model_metadata.get('tags', [])
creator = model_metadata.get('creator', {})
# Save the metadata to file if we have a file path and got metadata
if file_path:
try:
metadata_path = os.path.splitext(file_path)[0] + '.metadata.json'
metadata = await ModelRouteUtils.load_local_metadata(metadata_path)
metadata['modelDescription'] = description
metadata['tags'] = tags
# Ensure the civitai dict exists
if 'civitai' not in metadata:
metadata['civitai'] = {}
# Store creator in the civitai nested structure
metadata['civitai']['creator'] = creator
await MetadataManager.save_metadata(file_path, metadata, True)
except Exception as e:
logger.error(f"Error saving model metadata: {e}")
return web.json_response({
'success': True,
'description': description or "<p>No model description available.</p>",
'tags': tags,
'creator': creator
})
except Exception as e:
logger.error(f"Error getting model metadata: {e}")
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
logger.error(f"Error getting random LoRAs: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_trigger_words(self, request: web.Request) -> web.Response:
"""Get trigger words for specified LoRA models"""
try:
json_data = await request.json()
lora_names = json_data.get("lora_names", [])
node_ids = json_data.get("node_ids", [])
all_trigger_words = []
for lora_name in lora_names:
_, trigger_words = get_lora_info(lora_name)
all_trigger_words.extend(trigger_words)
# Format the trigger words
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
trigger_words_text = (
",, ".join(all_trigger_words) if all_trigger_words else ""
)
# Send update to all connected trigger word toggle nodes
for node_id in node_ids:
PromptServer.instance.send_sync("trigger_word_update", {
"id": node_id,
"message": trigger_words_text
})
for entry in node_ids:
node_identifier = entry
graph_identifier = None
if isinstance(entry, dict):
node_identifier = entry.get("node_id")
graph_identifier = entry.get("graph_id")
try:
parsed_node_id = int(node_identifier)
except (TypeError, ValueError):
parsed_node_id = node_identifier
payload = {"id": parsed_node_id, "message": trigger_words_text}
if graph_identifier is not None:
payload["graph_id"] = str(graph_identifier)
PromptServer.instance.send_sync("trigger_word_update", payload)
return web.json_response({"success": True})
except Exception as e:
logger.error(f"Error getting trigger words: {e}")
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
return web.json_response({"success": False, "error": str(e)}, status=500)

View File

@@ -0,0 +1,73 @@
"""Route registrar for miscellaneous endpoints.
This module mirrors the model route registrar architecture so that
miscellaneous endpoints share a consistent registration flow.
"""
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a HTTP route."""
method: str
path: str
handler_name: str
MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/settings", "get_settings"),
RouteDefinition("POST", "/api/lm/settings", "update_settings"),
RouteDefinition("GET", "/api/lm/priority-tags", "get_priority_tags"),
RouteDefinition("GET", "/api/lm/settings/libraries", "get_settings_libraries"),
RouteDefinition("POST", "/api/lm/settings/libraries/activate", "activate_library"),
RouteDefinition("GET", "/api/lm/health-check", "health_check"),
RouteDefinition("POST", "/api/lm/open-file-location", "open_file_location"),
RouteDefinition("POST", "/api/lm/update-usage-stats", "update_usage_stats"),
RouteDefinition("GET", "/api/lm/get-usage-stats", "get_usage_stats"),
RouteDefinition("POST", "/api/lm/update-lora-code", "update_lora_code"),
RouteDefinition("GET", "/api/lm/trained-words", "get_trained_words"),
RouteDefinition("GET", "/api/lm/model-example-files", "get_model_example_files"),
RouteDefinition("POST", "/api/lm/register-nodes", "register_nodes"),
RouteDefinition("POST", "/api/lm/update-node-widget", "update_node_widget"),
RouteDefinition("GET", "/api/lm/get-registry", "get_registry"),
RouteDefinition("GET", "/api/lm/check-model-exists", "check_model_exists"),
RouteDefinition("GET", "/api/lm/civitai/user-models", "get_civitai_user_models"),
RouteDefinition("POST", "/api/lm/download-metadata-archive", "download_metadata_archive"),
RouteDefinition("POST", "/api/lm/remove-metadata-archive", "remove_metadata_archive"),
RouteDefinition("GET", "/api/lm/metadata-archive-status", "get_metadata_archive_status"),
RouteDefinition("GET", "/api/lm/model-versions-status", "get_model_versions_status"),
RouteDefinition("POST", "/api/lm/settings/open-location", "open_settings_location"),
)
class MiscRouteRegistrar:
"""Bind miscellaneous route definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(
self,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = MISC_ROUTE_DEFINITIONS,
) -> None:
for definition in definitions:
self._bind(definition.method, definition.path, handler_lookup[definition.handler_name])
def _bind(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -1,699 +1,135 @@
"""Route controller for miscellaneous endpoints."""
from __future__ import annotations
import logging
import os
import sys
import threading
import asyncio
from server import PromptServer # type: ignore
from typing import Awaitable, Callable, Mapping
from aiohttp import web
from ..services.settings_manager import settings
from server import PromptServer # type: ignore
from ..services.metadata_service import (
get_metadata_archive_manager,
get_metadata_provider,
update_metadata_providers,
)
from ..services.settings_manager import get_settings_manager
from ..services.downloader import get_downloader
from ..utils.usage_stats import UsageStats
from ..utils.lora_metadata import extract_trained_words
from ..config import config
from ..utils.constants import SUPPORTED_MEDIA_EXTENSIONS, NODE_TYPES, DEFAULT_NODE_COLOR
from ..services.service_registry import ServiceRegistry
import re
from .handlers.misc_handlers import (
FileSystemHandler,
HealthCheckHandler,
LoraCodeHandler,
MetadataArchiveHandler,
MiscHandlerSet,
ModelExampleFilesHandler,
ModelLibraryHandler,
NodeRegistry,
NodeRegistryHandler,
SettingsHandler,
TrainedWordsHandler,
UsageStatsHandler,
build_service_registry_adapter,
)
from .misc_route_registrar import MiscRouteRegistrar
logger = logging.getLogger(__name__)
standalone_mode = 'nodes' not in sys.modules
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get(
"HF_HUB_DISABLE_TELEMETRY", "0"
) == "0"
# Node registry for tracking active workflow nodes
class NodeRegistry:
"""Thread-safe registry for tracking Lora nodes in active workflows"""
def __init__(self):
self._lock = threading.RLock()
self._nodes = {} # node_id -> node_info
self._registry_updated = threading.Event()
def register_nodes(self, nodes):
"""Register multiple nodes at once, replacing existing registry"""
with self._lock:
# Clear existing registry
self._nodes.clear()
# Register all new nodes
for node in nodes:
node_id = node['node_id']
node_type = node.get('type', '')
# Convert node type name to integer
type_id = NODE_TYPES.get(node_type, 0) # 0 for unknown types
# Handle null bgcolor with default color
bgcolor = node.get('bgcolor')
if bgcolor is None:
bgcolor = DEFAULT_NODE_COLOR
self._nodes[node_id] = {
'id': node_id,
'bgcolor': bgcolor,
'title': node.get('title'),
'type': type_id,
'type_name': node_type
}
logger.debug(f"Registered {len(nodes)} nodes in registry")
# Signal that registry has been updated
self._registry_updated.set()
def get_registry(self):
"""Get current registry information"""
with self._lock:
return {
'nodes': dict(self._nodes), # Return a copy
'node_count': len(self._nodes)
}
def clear_registry(self):
"""Clear the entire registry"""
with self._lock:
self._nodes.clear()
logger.info("Node registry cleared")
def wait_for_update(self, timeout=1.0):
"""Wait for registry update with timeout"""
self._registry_updated.clear()
return self._registry_updated.wait(timeout)
# Global registry instance
node_registry = NodeRegistry()
class MiscRoutes:
"""Miscellaneous routes for various utility functions"""
@staticmethod
def setup_routes(app):
"""Register miscellaneous routes"""
app.router.add_post('/api/settings', MiscRoutes.update_settings)
# Add new route for clearing cache
app.router.add_post('/api/clear-cache', MiscRoutes.clear_cache)
"""Route controller that mirrors the model route architecture."""
app.router.add_get('/api/health-check', lambda request: web.json_response({'status': 'ok'}))
def __init__(
self,
*,
settings_service=None,
usage_stats_factory: Callable[[], UsageStats] = UsageStats,
prompt_server: type[PromptServer] = PromptServer,
service_registry_adapter=build_service_registry_adapter(),
metadata_provider_factory=get_metadata_provider,
metadata_archive_manager_factory=get_metadata_archive_manager,
metadata_provider_updater=update_metadata_providers,
downloader_factory=get_downloader,
registrar_factory=MiscRouteRegistrar,
handler_set_factory=MiscHandlerSet,
node_registry: NodeRegistry | None = None,
standalone_mode_flag: bool = standalone_mode,
) -> None:
self._settings = settings_service or get_settings_manager()
self._usage_stats_factory = usage_stats_factory
self._prompt_server = prompt_server
self._service_registry_adapter = service_registry_adapter
self._metadata_provider_factory = metadata_provider_factory
self._metadata_archive_manager_factory = metadata_archive_manager_factory
self._metadata_provider_updater = metadata_provider_updater
self._downloader_factory = downloader_factory
self._registrar_factory = registrar_factory
self._handler_set_factory = handler_set_factory
self._node_registry = node_registry or NodeRegistry()
self._standalone_mode = standalone_mode_flag
# Usage stats routes
app.router.add_post('/api/update-usage-stats', MiscRoutes.update_usage_stats)
app.router.add_get('/api/get-usage-stats', MiscRoutes.get_usage_stats)
# Lora code update endpoint
app.router.add_post('/api/update-lora-code', MiscRoutes.update_lora_code)
# Add new route for getting trained words
app.router.add_get('/api/trained-words', MiscRoutes.get_trained_words)
# Add new route for getting model example files
app.router.add_get('/api/model-example-files', MiscRoutes.get_model_example_files)
# Node registry endpoints
app.router.add_post('/api/register-nodes', MiscRoutes.register_nodes)
app.router.add_get('/api/get-registry', MiscRoutes.get_registry)
# Add new route for checking if a model exists in the library
app.router.add_get('/api/check-model-exists', MiscRoutes.check_model_exists)
self._handler_mapping: Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]] | None = None
@staticmethod
async def clear_cache(request):
"""Clear all cache files from the cache folder"""
try:
# Get the cache folder path (relative to project directory)
project_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
cache_folder = os.path.join(project_dir, 'cache')
# Check if cache folder exists
if not os.path.exists(cache_folder):
logger.info("Cache folder does not exist, nothing to clear")
return web.json_response({'success': True, 'message': 'No cache folder found'})
# Get list of cache files before deleting for reporting
cache_files = [f for f in os.listdir(cache_folder) if os.path.isfile(os.path.join(cache_folder, f))]
deleted_files = []
# Delete each .msgpack file in the cache folder
for filename in cache_files:
if filename.endswith('.msgpack'):
file_path = os.path.join(cache_folder, filename)
try:
os.remove(file_path)
deleted_files.append(filename)
logger.info(f"Deleted cache file: {filename}")
except Exception as e:
logger.error(f"Failed to delete {filename}: {e}")
return web.json_response({
'success': False,
'error': f"Failed to delete {filename}: {str(e)}"
}, status=500)
return web.json_response({
'success': True,
'message': f"Successfully cleared {len(deleted_files)} cache files",
'deleted_files': deleted_files
})
except Exception as e:
logger.error(f"Error clearing cache files: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def setup_routes(app: web.Application) -> None:
"""Entry point used by the application bootstrap."""
controller = MiscRoutes()
controller.bind(app)
@staticmethod
async def update_settings(request):
"""Update application settings"""
try:
data = await request.json()
# Validate and update settings
for key, value in data.items():
if value == settings.get(key):
# No change, skip
continue
# Special handling for example_images_path - verify path exists
if key == 'example_images_path' and value:
if not os.path.exists(value):
return web.json_response({
'success': False,
'error': f"Path does not exist: {value}"
})
# Path changed - server restart required for new path to take effect
old_path = settings.get('example_images_path')
if old_path != value:
logger.info(f"Example images path changed to {value} - server restart required")
# Save to settings
settings.set(key, value)
return web.json_response({'success': True})
except Exception as e:
logger.error(f"Error updating settings: {e}", exc_info=True)
return web.Response(status=500, text=str(e))
@staticmethod
async def update_usage_stats(request):
"""
Update usage statistics based on a prompt_id
Expects a JSON body with:
{
"prompt_id": "string"
}
"""
try:
# Parse the request body
data = await request.json()
prompt_id = data.get('prompt_id')
if not prompt_id:
return web.json_response({
'success': False,
'error': 'Missing prompt_id'
}, status=400)
# Call the UsageStats to process this prompt_id synchronously
usage_stats = UsageStats()
await usage_stats.process_execution(prompt_id)
return web.json_response({
'success': True
})
except Exception as e:
logger.error(f"Failed to update usage stats: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def get_usage_stats(request):
"""Get current usage statistics"""
try:
usage_stats = UsageStats()
stats = await usage_stats.get_stats()
# Add version information to help clients handle format changes
stats_response = {
'success': True,
'data': stats,
'format_version': 2 # Indicate this is the new format with history
}
return web.json_response(stats_response)
except Exception as e:
logger.error(f"Failed to get usage stats: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def update_lora_code(request):
"""
Update Lora code in ComfyUI nodes
Expects a JSON body with:
{
"node_ids": [123, 456], # Optional - List of node IDs to update (for browser mode)
"lora_code": "<lora:modelname:1.0>", # The Lora code to send
"mode": "append" # or "replace" - whether to append or replace existing code
}
"""
try:
# Parse the request body
data = await request.json()
node_ids = data.get('node_ids')
lora_code = data.get('lora_code', '')
mode = data.get('mode', 'append')
if not lora_code:
return web.json_response({
'success': False,
'error': 'Missing lora_code parameter'
}, status=400)
results = []
# Desktop mode: no specific node_ids provided
if node_ids is None:
try:
# Send broadcast message with id=-1 to all Lora Loader nodes
PromptServer.instance.send_sync("lora_code_update", {
"id": -1,
"lora_code": lora_code,
"mode": mode
})
results.append({
'node_id': 'broadcast',
'success': True
})
except Exception as e:
logger.error(f"Error broadcasting lora code: {e}")
results.append({
'node_id': 'broadcast',
'success': False,
'error': str(e)
})
else:
# Browser mode: send to specific nodes
for node_id in node_ids:
try:
# Send the message to the frontend
PromptServer.instance.send_sync("lora_code_update", {
"id": node_id,
"lora_code": lora_code,
"mode": mode
})
results.append({
'node_id': node_id,
'success': True
})
except Exception as e:
logger.error(f"Error sending lora code to node {node_id}: {e}")
results.append({
'node_id': node_id,
'success': False,
'error': str(e)
})
return web.json_response({
'success': True,
'results': results
})
except Exception as e:
logger.error(f"Failed to update lora code: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def bind(self, app: web.Application) -> None:
registrar = self._registrar_factory(app)
registrar.register_routes(self._ensure_handler_mapping())
@staticmethod
async def get_trained_words(request):
"""
Get trained words from a safetensors file, sorted by frequency
Expects a query parameter:
file_path: Path to the safetensors file
"""
try:
# Get file path from query parameters
file_path = request.query.get('file_path')
if not file_path:
return web.json_response({
'success': False,
'error': 'Missing file_path parameter'
}, status=400)
# Check if file exists and is a safetensors file
if not os.path.exists(file_path):
return web.json_response({
'success': False,
'error': f"File not found: {file_path}"
}, status=404)
if not file_path.lower().endswith('.safetensors'):
return web.json_response({
'success': False,
'error': 'File is not a safetensors file'
}, status=400)
# Extract trained words and class_tokens
trained_words, class_tokens = await extract_trained_words(file_path)
# Return result with both trained words and class tokens
return web.json_response({
'success': True,
'trained_words': trained_words,
'class_tokens': class_tokens
})
except Exception as e:
logger.error(f"Failed to get trained words: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def _ensure_handler_mapping(self) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
if self._handler_mapping is None:
handler_set = self._create_handler_set()
self._handler_mapping = handler_set.to_route_mapping()
return self._handler_mapping
@staticmethod
async def get_model_example_files(request):
"""
Get list of example image files for a specific model based on file path
Expects:
- file_path in query parameters
Returns:
- List of image files with their paths as static URLs
"""
try:
# Get the model file path from query parameters
file_path = request.query.get('file_path')
if not file_path:
return web.json_response({
'success': False,
'error': 'Missing file_path parameter'
}, status=400)
# Extract directory and base filename
model_dir = os.path.dirname(file_path)
model_filename = os.path.basename(file_path)
model_name = os.path.splitext(model_filename)[0]
# Check if the directory exists
if not os.path.exists(model_dir):
return web.json_response({
'success': False,
'error': 'Model directory not found',
'files': []
}, status=404)
# Look for files matching the pattern modelname.example.<index>.<ext>
files = []
pattern = f"{model_name}.example."
for file in os.listdir(model_dir):
file_lower = file.lower()
if file_lower.startswith(pattern.lower()):
file_full_path = os.path.join(model_dir, file)
if os.path.isfile(file_full_path):
# Check if the file is a supported media file
file_ext = os.path.splitext(file)[1].lower()
if (file_ext in SUPPORTED_MEDIA_EXTENSIONS['images'] or
file_ext in SUPPORTED_MEDIA_EXTENSIONS['videos']):
# Extract the index from the filename
try:
# Extract the part after '.example.' and before file extension
index_part = file[len(pattern):].split('.')[0]
# Try to parse it as an integer
index = int(index_part)
except (ValueError, IndexError):
# If we can't parse the index, use infinity to sort at the end
index = float('inf')
# Convert file path to static URL
static_url = config.get_preview_static_url(file_full_path)
files.append({
'name': file,
'path': static_url,
'extension': file_ext,
'is_video': file_ext in SUPPORTED_MEDIA_EXTENSIONS['videos'],
'index': index
})
# Sort files by their index for consistent ordering
files.sort(key=lambda x: x['index'])
# Remove the index field as it's only used for sorting
for file in files:
file.pop('index', None)
return web.json_response({
'success': True,
'files': files
})
except Exception as e:
logger.error(f"Failed to get model example files: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def _create_handler_set(self) -> MiscHandlerSet:
health = HealthCheckHandler()
settings_handler = SettingsHandler(
settings_service=self._settings,
metadata_provider_updater=self._metadata_provider_updater,
downloader_factory=self._downloader_factory,
)
usage_stats = UsageStatsHandler(usage_stats_factory=self._usage_stats_factory)
lora_code = LoraCodeHandler(prompt_server=self._prompt_server)
trained_words = TrainedWordsHandler()
model_examples = ModelExampleFilesHandler()
metadata_archive = MetadataArchiveHandler(
metadata_archive_manager_factory=self._metadata_archive_manager_factory,
settings_service=self._settings,
metadata_provider_updater=self._metadata_provider_updater,
)
filesystem = FileSystemHandler(settings_service=self._settings)
node_registry_handler = NodeRegistryHandler(
node_registry=self._node_registry,
prompt_server=self._prompt_server,
standalone_mode=self._standalone_mode,
)
model_library = ModelLibraryHandler(
service_registry=self._service_registry_adapter,
metadata_provider_factory=self._metadata_provider_factory,
)
@staticmethod
async def register_nodes(request):
"""
Register multiple Lora nodes at once
Expects a JSON body with:
{
"nodes": [
{
"node_id": 123,
"bgcolor": "#535",
"title": "Lora Loader (LoraManager)"
},
...
]
}
"""
try:
data = await request.json()
# Validate required fields
nodes = data.get('nodes', [])
if not isinstance(nodes, list):
return web.json_response({
'success': False,
'error': 'nodes must be a list'
}, status=400)
# Validate each node
for i, node in enumerate(nodes):
if not isinstance(node, dict):
return web.json_response({
'success': False,
'error': f'Node {i} must be an object'
}, status=400)
node_id = node.get('node_id')
if node_id is None:
return web.json_response({
'success': False,
'error': f'Node {i} missing node_id parameter'
}, status=400)
# Validate node_id is an integer
try:
node['node_id'] = int(node_id)
except (ValueError, TypeError):
return web.json_response({
'success': False,
'error': f'Node {i} node_id must be an integer'
}, status=400)
# Register all nodes
node_registry.register_nodes(nodes)
return web.json_response({
'success': True,
'message': f'{len(nodes)} nodes registered successfully'
})
except Exception as e:
logger.error(f"Failed to register nodes: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def get_registry(request):
"""Get current node registry information by refreshing from frontend"""
try:
# Check if running in standalone mode
if standalone_mode:
logger.warning("Registry refresh not available in standalone mode")
return web.json_response({
'success': False,
'error': 'Standalone Mode Active',
'message': 'Cannot interact with ComfyUI in standalone mode.'
}, status=503)
# Send message to frontend to refresh registry
try:
PromptServer.instance.send_sync("lora_registry_refresh", {})
logger.debug("Sent registry refresh request to frontend")
except Exception as e:
logger.error(f"Failed to send registry refresh message: {e}")
return web.json_response({
'success': False,
'error': 'Communication Error',
'message': f'Failed to communicate with ComfyUI frontend: {str(e)}'
}, status=500)
# Wait for registry update with timeout
def wait_for_registry():
return node_registry.wait_for_update(timeout=1.0)
# Run the wait in a thread to avoid blocking the event loop
loop = asyncio.get_event_loop()
registry_updated = await loop.run_in_executor(None, wait_for_registry)
if not registry_updated:
logger.warning("Registry refresh timeout after 1 second")
return web.json_response({
'success': False,
'error': 'Timeout Error',
'message': 'Registry refresh timeout - ComfyUI frontend may not be responsive'
}, status=408)
# Get updated registry
registry_info = node_registry.get_registry()
return web.json_response({
'success': True,
'data': registry_info
})
except Exception as e:
logger.error(f"Failed to get registry: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': 'Internal Error',
'message': str(e)
}, status=500)
return self._handler_set_factory(
health=health,
settings=settings_handler,
usage_stats=usage_stats,
lora_code=lora_code,
trained_words=trained_words,
model_examples=model_examples,
node_registry=node_registry_handler,
model_library=model_library,
metadata_archive=metadata_archive,
filesystem=filesystem,
)
@staticmethod
async def check_model_exists(request):
"""
Check if a model with specified modelId and optionally modelVersionId exists in the library
Expects query parameters:
- modelId: int - Civitai model ID (required)
- modelVersionId: int - Civitai model version ID (optional)
Returns:
- If modelVersionId is provided: JSON with a boolean 'exists' field
- If modelVersionId is not provided: JSON with a list of modelVersionIds that exist in the library
"""
try:
# Get the modelId and modelVersionId from query parameters
model_id_str = request.query.get('modelId')
model_version_id_str = request.query.get('modelVersionId')
# Validate modelId parameter (required)
if not model_id_str:
return web.json_response({
'success': False,
'error': 'Missing required parameter: modelId'
}, status=400)
try:
# Convert modelId to integer
model_id = int(model_id_str)
except ValueError:
return web.json_response({
'success': False,
'error': 'Parameter modelId must be an integer'
}, status=400)
# Get all scanners
lora_scanner = await ServiceRegistry.get_lora_scanner()
checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
embedding_scanner = await ServiceRegistry.get_embedding_scanner()
# If modelVersionId is provided, check for specific version
if model_version_id_str:
try:
model_version_id = int(model_version_id_str)
except ValueError:
return web.json_response({
'success': False,
'error': 'Parameter modelVersionId must be an integer'
}, status=400)
# Check lora scanner first
exists = False
model_type = None
if await lora_scanner.check_model_version_exists(model_version_id):
exists = True
model_type = 'lora'
elif checkpoint_scanner and await checkpoint_scanner.check_model_version_exists(model_version_id):
exists = True
model_type = 'checkpoint'
elif embedding_scanner and await embedding_scanner.check_model_version_exists(model_version_id):
exists = True
model_type = 'embedding'
return web.json_response({
'success': True,
'exists': exists,
'modelType': model_type if exists else None
})
# If modelVersionId is not provided, return all version IDs for the model
else:
lora_versions = await lora_scanner.get_model_versions_by_id(model_id)
checkpoint_versions = []
embedding_versions = []
# 优先lora其次checkpoint最后embedding
if not lora_versions:
checkpoint_versions = await checkpoint_scanner.get_model_versions_by_id(model_id)
if not lora_versions and not checkpoint_versions:
embedding_versions = await embedding_scanner.get_model_versions_by_id(model_id)
model_type = None
versions = []
if lora_versions:
model_type = 'lora'
versions = lora_versions
elif checkpoint_versions:
model_type = 'checkpoint'
versions = checkpoint_versions
elif embedding_versions:
model_type = 'embedding'
versions = embedding_versions
return web.json_response({
'success': True,
'modelId': model_id,
'modelType': model_type,
'versions': versions
})
except Exception as e:
logger.error(f"Failed to check model existence: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
__all__ = ["MiscRoutes"]

View File

@@ -0,0 +1,108 @@
"""Route registrar for model endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a HTTP route."""
method: str
path_template: str
handler_name: str
def build_path(self, prefix: str) -> str:
return self.path_template.replace("{prefix}", prefix)
COMMON_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/{prefix}/list", "get_models"),
RouteDefinition("POST", "/api/lm/{prefix}/delete", "delete_model"),
RouteDefinition("POST", "/api/lm/{prefix}/exclude", "exclude_model"),
RouteDefinition("POST", "/api/lm/{prefix}/fetch-civitai", "fetch_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/fetch-all-civitai", "fetch_all_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/relink-civitai", "relink_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/replace-preview", "replace_preview"),
RouteDefinition("POST", "/api/lm/{prefix}/save-metadata", "save_metadata"),
RouteDefinition("POST", "/api/lm/{prefix}/add-tags", "add_tags"),
RouteDefinition("POST", "/api/lm/{prefix}/rename", "rename_model"),
RouteDefinition("POST", "/api/lm/{prefix}/bulk-delete", "bulk_delete_models"),
RouteDefinition("POST", "/api/lm/{prefix}/verify-duplicates", "verify_duplicates"),
RouteDefinition("POST", "/api/lm/{prefix}/move_model", "move_model"),
RouteDefinition("POST", "/api/lm/{prefix}/move_models_bulk", "move_models_bulk"),
RouteDefinition("GET", "/api/lm/{prefix}/auto-organize", "auto_organize_models"),
RouteDefinition("POST", "/api/lm/{prefix}/auto-organize", "auto_organize_models"),
RouteDefinition("GET", "/api/lm/{prefix}/auto-organize-progress", "get_auto_organize_progress"),
RouteDefinition("GET", "/api/lm/{prefix}/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/{prefix}/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/{prefix}/model-types", "get_model_types"),
RouteDefinition("GET", "/api/lm/{prefix}/scan", "scan_models"),
RouteDefinition("GET", "/api/lm/{prefix}/roots", "get_model_roots"),
RouteDefinition("GET", "/api/lm/{prefix}/folders", "get_folders"),
RouteDefinition("GET", "/api/lm/{prefix}/folder-tree", "get_folder_tree"),
RouteDefinition("GET", "/api/lm/{prefix}/unified-folder-tree", "get_unified_folder_tree"),
RouteDefinition("GET", "/api/lm/{prefix}/find-duplicates", "find_duplicate_models"),
RouteDefinition("GET", "/api/lm/{prefix}/find-filename-conflicts", "find_filename_conflicts"),
RouteDefinition("GET", "/api/lm/{prefix}/get-notes", "get_model_notes"),
RouteDefinition("GET", "/api/lm/{prefix}/preview-url", "get_model_preview_url"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai-url", "get_model_civitai_url"),
RouteDefinition("GET", "/api/lm/{prefix}/metadata", "get_model_metadata"),
RouteDefinition("GET", "/api/lm/{prefix}/model-description", "get_model_description"),
RouteDefinition("GET", "/api/lm/{prefix}/relative-paths", "get_relative_paths"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/versions/{model_id}", "get_civitai_versions"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/model/version/{modelVersionId}", "get_civitai_model_by_version"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/model/hash/{hash}", "get_civitai_model_by_hash"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/refresh", "refresh_model_updates"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/fetch-missing-license", "fetch_missing_civitai_license_data"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/ignore", "set_model_update_ignore"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/ignore-version", "set_version_update_ignore"),
RouteDefinition("GET", "/api/lm/{prefix}/updates/status/{model_id}", "get_model_update_status"),
RouteDefinition("GET", "/api/lm/{prefix}/updates/versions/{model_id}", "get_model_versions"),
RouteDefinition("POST", "/api/lm/download-model", "download_model"),
RouteDefinition("GET", "/api/lm/download-model-get", "download_model_get"),
RouteDefinition("GET", "/api/lm/cancel-download-get", "cancel_download_get"),
RouteDefinition("GET", "/api/lm/pause-download", "pause_download_get"),
RouteDefinition("GET", "/api/lm/resume-download", "resume_download_get"),
RouteDefinition("GET", "/api/lm/download-progress/{download_id}", "get_download_progress"),
RouteDefinition("POST", "/api/lm/{prefix}/cancel-task", "cancel_task"),
RouteDefinition("GET", "/{prefix}", "handle_models_page"),
)
class ModelRouteRegistrar:
"""Bind declarative definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_common_routes(
self,
prefix: str,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = COMMON_ROUTE_DEFINITIONS,
) -> None:
for definition in definitions:
self._bind_route(definition.method, definition.build_path(prefix), handler_lookup[definition.handler_name])
def add_route(self, method: str, path: str, handler: Callable) -> None:
self._bind_route(method, path, handler)
def add_prefixed_route(self, method: str, path_template: str, prefix: str, handler: Callable) -> None:
self._bind_route(method, path_template.replace("{prefix}", prefix), handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -0,0 +1,25 @@
"""Route controller for preview asset delivery."""
from __future__ import annotations
from aiohttp import web
from .handlers.preview_handlers import PreviewHandler
class PreviewRoutes:
"""Register routes that expose preview assets."""
def __init__(self, *, handler: PreviewHandler | None = None) -> None:
self._handler = handler or PreviewHandler()
@classmethod
def setup_routes(cls, app: web.Application) -> None:
controller = cls()
controller.register(app)
def register(self, app: web.Application) -> None:
app.router.add_get('/api/lm/previews', self._handler.serve_preview)
__all__ = ["PreviewRoutes"]

View File

@@ -0,0 +1,74 @@
"""Route registrar for recipe endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a recipe HTTP route."""
method: str
path: str
handler_name: str
ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/loras/recipes", "render_page"),
RouteDefinition("GET", "/api/lm/recipes", "list_recipes"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}", "get_recipe"),
RouteDefinition("GET", "/api/lm/recipes/import-remote", "import_remote_recipe"),
RouteDefinition("POST", "/api/lm/recipes/analyze-image", "analyze_uploaded_image"),
RouteDefinition("POST", "/api/lm/recipes/analyze-local-image", "analyze_local_image"),
RouteDefinition("POST", "/api/lm/recipes/save", "save_recipe"),
RouteDefinition("DELETE", "/api/lm/recipe/{recipe_id}", "delete_recipe"),
RouteDefinition("GET", "/api/lm/recipes/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/recipes/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/recipes/roots", "get_roots"),
RouteDefinition("GET", "/api/lm/recipes/folders", "get_folders"),
RouteDefinition("GET", "/api/lm/recipes/folder-tree", "get_folder_tree"),
RouteDefinition("GET", "/api/lm/recipes/unified-folder-tree", "get_unified_folder_tree"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/share", "share_recipe"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/share/download", "download_shared_recipe"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/syntax", "get_recipe_syntax"),
RouteDefinition("PUT", "/api/lm/recipe/{recipe_id}/update", "update_recipe"),
RouteDefinition("POST", "/api/lm/recipe/move", "move_recipe"),
RouteDefinition("POST", "/api/lm/recipes/move-bulk", "move_recipes_bulk"),
RouteDefinition("POST", "/api/lm/recipe/lora/reconnect", "reconnect_lora"),
RouteDefinition("GET", "/api/lm/recipes/find-duplicates", "find_duplicates"),
RouteDefinition("POST", "/api/lm/recipes/bulk-delete", "bulk_delete"),
RouteDefinition("POST", "/api/lm/recipes/save-from-widget", "save_recipe_from_widget"),
RouteDefinition("GET", "/api/lm/recipes/for-lora", "get_recipes_for_lora"),
RouteDefinition("GET", "/api/lm/recipes/scan", "scan_recipes"),
RouteDefinition("POST", "/api/lm/recipes/repair", "repair_recipes"),
RouteDefinition("POST", "/api/lm/recipes/cancel-repair", "cancel_repair"),
RouteDefinition("POST", "/api/lm/recipe/{recipe_id}/repair", "repair_recipe"),
RouteDefinition("GET", "/api/lm/recipes/repair-progress", "get_repair_progress"),
)
class RecipeRouteRegistrar:
"""Bind declarative recipe definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(self, handler_lookup: Mapping[str, Callable[[web.Request], object]]) -> None:
for definition in ROUTE_DEFINITIONS:
handler = handler_lookup[definition.handler_name]
self._bind_route(definition.method, definition.path, handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

File diff suppressed because it is too large Load Diff

View File

@@ -8,12 +8,32 @@ from collections import defaultdict, Counter
from typing import Dict, List, Any
from ..config import config
from ..services.settings_manager import settings
from ..services.settings_manager import get_settings_manager
from ..services.server_i18n import server_i18n
from ..services.service_registry import ServiceRegistry
from ..utils.usage_stats import UsageStats
logger = logging.getLogger(__name__)
class _SettingsProxy:
def __init__(self):
self._manager = None
def _resolve(self):
if self._manager is None:
self._manager = get_settings_manager()
return self._manager
def get(self, *args, **kwargs):
return self._resolve().get(*args, **kwargs)
def __getattr__(self, item):
return getattr(self._resolve(), item)
settings = _SettingsProxy()
class StatsRoutes:
"""Route handlers for Statistics page and API endpoints"""
@@ -32,7 +52,13 @@ class StatsRoutes:
self.lora_scanner = await ServiceRegistry.get_lora_scanner()
self.checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
self.embedding_scanner = await ServiceRegistry.get_embedding_scanner()
self.usage_stats = UsageStats()
# Only initialize usage stats if we have valid paths configured
try:
self.usage_stats = UsageStats()
except RuntimeError as e:
logger.warning(f"Could not initialize usage statistics: {e}")
self.usage_stats = None
async def handle_stats_page(self, request: web.Request) -> web.Response:
"""Handle GET /statistics request"""
@@ -58,11 +84,25 @@ class StatsRoutes:
is_initializing = lora_initializing or checkpoint_initializing or embedding_initializing
# 获取用户语言设置
settings_object = settings
user_language = settings_object.get('language', 'en')
settings_manager = settings_object if not isinstance(settings_object, _SettingsProxy) else settings_object._resolve()
# 设置服务端i18n语言
server_i18n.set_locale(user_language)
# 为模板环境添加i18n过滤器
if not hasattr(self.template_env, '_i18n_filter_added'):
self.template_env.filters['t'] = server_i18n.create_template_filter()
self.template_env._i18n_filter_added = True
template = self.template_env.get_template('statistics.html')
rendered = template.render(
is_initializing=is_initializing,
settings=settings,
request=request
settings=settings_manager,
request=request,
t=server_i18n.get_translation,
)
return web.Response(
@@ -488,12 +528,12 @@ class StatsRoutes:
app.router.add_get('/statistics', self.handle_stats_page)
# Register API routes
app.router.add_get('/api/stats/collection-overview', self.get_collection_overview)
app.router.add_get('/api/stats/usage-analytics', self.get_usage_analytics)
app.router.add_get('/api/stats/base-model-distribution', self.get_base_model_distribution)
app.router.add_get('/api/stats/tag-analytics', self.get_tag_analytics)
app.router.add_get('/api/stats/storage-analytics', self.get_storage_analytics)
app.router.add_get('/api/stats/insights', self.get_insights)
app.router.add_get('/api/lm/stats/collection-overview', self.get_collection_overview)
app.router.add_get('/api/lm/stats/usage-analytics', self.get_usage_analytics)
app.router.add_get('/api/lm/stats/base-model-distribution', self.get_base_model_distribution)
app.router.add_get('/api/lm/stats/tag-analytics', self.get_tag_analytics)
app.router.add_get('/api/lm/stats/storage-analytics', self.get_storage_analytics)
app.router.add_get('/api/lm/stats/insights', self.get_insights)
async def _on_startup(self, app):
"""Initialize services when the app starts"""

View File

@@ -1,26 +1,31 @@
import os
import aiohttp
import logging
import toml
import git
import zipfile
import shutil
import tempfile
from aiohttp import web
import asyncio
from aiohttp import web, ClientError
from typing import Dict, List
from ..utils.settings_paths import ensure_settings_file
from ..services.downloader import get_downloader
logger = logging.getLogger(__name__)
NETWORK_EXCEPTIONS = (ClientError, OSError, asyncio.TimeoutError)
class UpdateRoutes:
"""Routes for handling plugin update checks"""
@staticmethod
def setup_routes(app):
"""Register update check routes"""
app.router.add_get('/api/check-updates', UpdateRoutes.check_updates)
app.router.add_get('/api/version-info', UpdateRoutes.get_version_info)
app.router.add_post('/api/perform-update', UpdateRoutes.perform_update)
app.router.add_get('/api/lm/check-updates', UpdateRoutes.check_updates)
app.router.add_get('/api/lm/version-info', UpdateRoutes.get_version_info)
app.router.add_post('/api/lm/perform-update', UpdateRoutes.perform_update)
@staticmethod
async def check_updates(request):
@@ -64,6 +69,12 @@ class UpdateRoutes:
'nightly': nightly
})
except NETWORK_EXCEPTIONS as e:
logger.warning("Network unavailable during update check: %s", e)
return web.json_response({
'success': False,
'error': 'Network unavailable for update check'
})
except Exception as e:
logger.error(f"Failed to check for updates: {e}", exc_info=True)
return web.json_response({
@@ -112,7 +123,7 @@ class UpdateRoutes:
current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir))
settings_path = os.path.join(plugin_root, 'settings.json')
settings_path = ensure_settings_file(logger)
settings_backup = None
if os.path.exists(settings_path):
with open(settings_path, 'r', encoding='utf-8') as f:
@@ -155,51 +166,66 @@ class UpdateRoutes:
async def _download_and_replace_zip(plugin_root: str) -> tuple[bool, str]:
"""
Download latest release ZIP from GitHub and replace plugin files.
Skips settings.json. Writes extracted file list to .tracking.
Skips settings.json and civitai folder. Writes extracted file list to .tracking.
"""
repo_owner = "willmiao"
repo_name = "ComfyUI-Lora-Manager"
github_api = f"https://api.github.com/repos/{repo_owner}/{repo_name}/releases/latest"
try:
async with aiohttp.ClientSession() as session:
async with session.get(github_api) as resp:
if resp.status != 200:
logger.error(f"Failed to fetch release info: {resp.status}")
return False, ""
data = await resp.json()
zip_url = data.get("zipball_url")
version = data.get("tag_name", "unknown")
downloader = await get_downloader()
# Get release info
success, data = await downloader.make_request(
'GET',
github_api,
use_auth=False
)
if not success:
logger.error(f"Failed to fetch release info: {data}")
return False, ""
zip_url = data.get("zipball_url")
version = data.get("tag_name", "unknown")
# Download ZIP
async with session.get(zip_url) as zip_resp:
if zip_resp.status != 200:
logger.error(f"Failed to download ZIP: {zip_resp.status}")
return False, ""
with tempfile.NamedTemporaryFile(delete=False, suffix=".zip") as tmp_zip:
tmp_zip.write(await zip_resp.read())
zip_path = tmp_zip.name
# Download ZIP to temporary file
with tempfile.NamedTemporaryFile(delete=False, suffix=".zip") as tmp_zip:
tmp_zip_path = tmp_zip.name
success, result = await downloader.download_file(
url=zip_url,
save_path=tmp_zip_path,
use_auth=False,
allow_resume=False
)
if not success:
logger.error(f"Failed to download ZIP: {result}")
return False, ""
UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json'])
zip_path = tmp_zip_path
# Extract ZIP to temp dir
with tempfile.TemporaryDirectory() as tmp_dir:
# Skip both settings.json, civitai and model cache folder
UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json', 'civitai', 'model_cache'])
# Extract ZIP to temp dir
with tempfile.TemporaryDirectory() as tmp_dir:
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
zip_ref.extractall(tmp_dir)
# Find extracted folder (GitHub ZIP contains a root folder)
extracted_root = next(os.scandir(tmp_dir)).path
# Copy files, skipping settings.json
# Copy files, skipping settings.json and civitai folder
for item in os.listdir(extracted_root):
if item == 'settings.json' or item == 'civitai':
continue
src = os.path.join(extracted_root, item)
dst = os.path.join(plugin_root, item)
if os.path.isdir(src):
if os.path.exists(dst):
shutil.rmtree(dst)
shutil.copytree(src, dst, ignore=shutil.ignore_patterns('settings.json'))
shutil.copytree(src, dst, ignore=shutil.ignore_patterns('settings.json', 'civitai'))
else:
if item == 'settings.json':
continue
shutil.copy2(src, dst)
# Write .tracking file: list all files under extracted_root, relative to extracted_root
@@ -207,15 +233,22 @@ class UpdateRoutes:
tracking_info_file = os.path.join(plugin_root, '.tracking')
tracking_files = []
for root, dirs, files in os.walk(extracted_root):
# Skip civitai folder and its contents
rel_root = os.path.relpath(root, extracted_root)
if rel_root == 'civitai' or rel_root.startswith('civitai' + os.sep):
continue
for file in files:
rel_path = os.path.relpath(os.path.join(root, file), extracted_root)
# Skip settings.json and any file under civitai
if rel_path == 'settings.json' or rel_path.startswith('civitai' + os.sep):
continue
tracking_files.append(rel_path.replace("\\", "/"))
with open(tracking_info_file, "w", encoding='utf-8') as file:
file.write('\n'.join(tracking_files))
os.remove(zip_path)
logger.info(f"Updated plugin via ZIP to {version}")
return True, version
os.remove(zip_path)
logger.info(f"Updated plugin via ZIP to {version}")
return True, version
except Exception as e:
logger.error(f"ZIP update failed: {e}", exc_info=True)
@@ -244,24 +277,27 @@ class UpdateRoutes:
github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/commits/main"
try:
async with aiohttp.ClientSession() as session:
async with session.get(github_url, headers={'Accept': 'application/vnd.github+json'}) as response:
if response.status != 200:
logger.warning(f"Failed to fetch GitHub commit: {response.status}")
return "main", []
data = await response.json()
commit_sha = data.get('sha', '')[:7] # Short hash
commit_message = data.get('commit', {}).get('message', '')
# Format as "main-{short_hash}"
version = f"main-{commit_sha}"
# Use commit message as changelog
changelog = [commit_message] if commit_message else []
return version, changelog
downloader = await get_downloader()
success, data = await downloader.make_request('GET', github_url, custom_headers={'Accept': 'application/vnd.github+json'})
if not success:
logger.warning(f"Failed to fetch GitHub commit: {data}")
return "main", []
commit_sha = data.get('sha', '')[:7] # Short hash
commit_message = data.get('commit', {}).get('message', '')
# Format as "main-{short_hash}"
version = f"main-{commit_sha}"
# Use commit message as changelog
changelog = [commit_message] if commit_message else []
return version, changelog
except NETWORK_EXCEPTIONS as e:
logger.warning("Unable to reach GitHub for nightly version: %s", e)
return "main", []
except Exception as e:
logger.error(f"Error fetching nightly version: {e}", exc_info=True)
return "main", []
@@ -308,6 +344,11 @@ class UpdateRoutes:
origin.fetch()
if nightly:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Switch to main branch and pull latest
main_branch = 'main'
if main_branch not in [branch.name for branch in repo.branches]:
@@ -321,6 +362,11 @@ class UpdateRoutes:
new_version = f"main-{repo.head.commit.hexsha[:7]}"
else:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Get latest release tag
tags = sorted(repo.tags, key=lambda t: t.commit.committed_datetime, reverse=True)
if not tags:
@@ -410,23 +456,26 @@ class UpdateRoutes:
github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/releases/latest"
try:
async with aiohttp.ClientSession() as session:
async with session.get(github_url, headers={'Accept': 'application/vnd.github+json'}) as response:
if response.status != 200:
logger.warning(f"Failed to fetch GitHub release: {response.status}")
return "v0.0.0", []
data = await response.json()
version = data.get('tag_name', '')
if not version.startswith('v'):
version = f"v{version}"
# Extract changelog from release notes
body = data.get('body', '')
changelog = UpdateRoutes._parse_changelog(body)
return version, changelog
downloader = await get_downloader()
success, data = await downloader.make_request('GET', github_url, custom_headers={'Accept': 'application/vnd.github+json'})
if not success:
logger.warning(f"Failed to fetch GitHub release: {data}")
return "v0.0.0", []
version = data.get('tag_name', '')
if not version.startswith('v'):
version = f"v{version}"
# Extract changelog from release notes
body = data.get('body', '')
changelog = UpdateRoutes._parse_changelog(body)
return version, changelog
except NETWORK_EXCEPTIONS as e:
logger.warning("Unable to reach GitHub for release info: %s", e)
return "v0.0.0", []
except Exception as e:
logger.error(f"Error fetching remote version: {e}", exc_info=True)
return "v0.0.0", []

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
import logging
from typing import List
from typing import Any, Dict, List, Optional
from ..utils.models import CheckpointMetadata
from ..config import config
@@ -21,14 +21,33 @@ class CheckpointScanner(ModelScanner):
hash_index=ModelHashIndex()
)
def _resolve_model_type(self, root_path: Optional[str]) -> Optional[str]:
if not root_path:
return None
if config.checkpoints_roots and root_path in config.checkpoints_roots:
return "checkpoint"
if config.unet_roots and root_path in config.unet_roots:
return "diffusion_model"
return None
def adjust_metadata(self, metadata, file_path, root_path):
if hasattr(metadata, "model_type"):
if root_path in config.checkpoints_roots:
metadata.model_type = "checkpoint"
elif root_path in config.unet_roots:
metadata.model_type = "diffusion_model"
model_type = self._resolve_model_type(root_path)
if model_type:
metadata.model_type = model_type
return metadata
def adjust_cached_entry(self, entry: Dict[str, Any]) -> Dict[str, Any]:
model_type = self._resolve_model_type(
self._find_root_for_file(entry.get("file_path"))
)
if model_type:
entry["model_type"] = model_type
return entry
def get_model_roots(self) -> List[str]:
"""Get checkpoint root directories"""
return config.base_models_roots
return config.base_models_roots

View File

@@ -1,24 +1,24 @@
import os
import logging
from typing import Dict, List, Optional
from typing import Dict
from .base_model_service import BaseModelService
from ..utils.models import CheckpointMetadata
from ..config import config
from ..utils.routes_common import ModelRouteUtils
logger = logging.getLogger(__name__)
class CheckpointService(BaseModelService):
"""Checkpoint-specific service implementation"""
def __init__(self, scanner):
def __init__(self, scanner, update_service=None):
"""Initialize Checkpoint service
Args:
scanner: Checkpoint scanner instance
update_service: Optional service for remote update tracking.
"""
super().__init__("checkpoint", scanner, CheckpointMetadata)
super().__init__("checkpoint", scanner, CheckpointMetadata, update_service=update_service)
async def format_response(self, checkpoint_data: Dict) -> Dict:
"""Format Checkpoint data for API response"""
@@ -34,12 +34,13 @@ class CheckpointService(BaseModelService):
"file_size": checkpoint_data.get("size", 0),
"modified": checkpoint_data.get("modified", ""),
"tags": checkpoint_data.get("tags", []),
"modelDescription": checkpoint_data.get("modelDescription", ""),
"from_civitai": checkpoint_data.get("from_civitai", True),
"usage_count": checkpoint_data.get("usage_count", 0),
"notes": checkpoint_data.get("notes", ""),
"model_type": checkpoint_data.get("model_type", "checkpoint"),
"favorite": checkpoint_data.get("favorite", False),
"civitai": ModelRouteUtils.filter_civitai_data(checkpoint_data.get("civitai", {}))
"update_available": bool(checkpoint_data.get("update_available", False)),
"civitai": self.filter_civitai_data(checkpoint_data.get("civitai", {}), minimal=True)
}
def find_duplicate_hashes(self) -> Dict:
@@ -48,4 +49,4 @@ class CheckpointService(BaseModelService):
def find_duplicate_filenames(self) -> Dict:
"""Find Checkpoints with conflicting filenames"""
return self.scanner._hash_index.get_duplicate_filenames()
return self.scanner._hash_index.get_duplicate_filenames()

View File

@@ -0,0 +1,431 @@
import json
import logging
import asyncio
from copy import deepcopy
from typing import Optional, Dict, Tuple, List
from .model_metadata_provider import CivArchiveModelMetadataProvider, ModelMetadataProviderManager
from .downloader import get_downloader
from .errors import RateLimitError
logger = logging.getLogger(__name__)
class CivArchiveClient:
_instance = None
_lock = asyncio.Lock()
@classmethod
async def get_instance(cls):
"""Get singleton instance of CivArchiveClient"""
async with cls._lock:
if cls._instance is None:
cls._instance = cls()
# Register this client as a metadata provider
provider_manager = await ModelMetadataProviderManager.get_instance()
provider_manager.register_provider('civarchive', CivArchiveModelMetadataProvider(cls._instance), False)
return cls._instance
def __init__(self):
# Check if already initialized for singleton pattern
if hasattr(self, '_initialized'):
return
self._initialized = True
self.base_url = "https://civarchive.com/api"
async def _request_json(
self,
path: str,
params: Optional[Dict[str, str]] = None
) -> Tuple[Optional[Dict], Optional[str]]:
"""Call CivArchive API and return JSON payload"""
success, payload = await self._make_request(path, params=params)
if not success:
error = payload if isinstance(payload, str) else "Request failed"
return None, error
if not isinstance(payload, dict):
return None, "Invalid response structure"
return payload, None
async def _make_request(
self,
path: str,
*,
params: Optional[Dict[str, str]] = None,
) -> Tuple[bool, Dict | str]:
"""Wrapper around downloader.make_request that surfaces rate limits."""
downloader = await get_downloader()
kwargs: Dict[str, Dict[str, str]] = {}
if params:
safe_params = {str(key): str(value) for key, value in params.items() if value is not None}
if safe_params:
kwargs["params"] = safe_params
success, payload = await downloader.make_request(
"GET",
f"{self.base_url}{path}",
use_auth=False,
**kwargs,
)
if not success and isinstance(payload, RateLimitError):
if payload.provider is None:
payload.provider = "civarchive_api"
raise payload
return success, payload
@staticmethod
def _normalize_payload(payload: Dict) -> Dict:
"""Unwrap CivArchive responses that wrap content under a data key"""
if not isinstance(payload, dict):
return {}
data = payload.get("data")
if isinstance(data, dict):
return data
return payload
@staticmethod
def _split_context(payload: Dict) -> Tuple[Dict, Dict, List[Dict]]:
"""Separate version payload from surrounding model context"""
data = CivArchiveClient._normalize_payload(payload)
context: Dict = {}
fallback_files: List[Dict] = []
version: Dict = {}
for key, value in data.items():
if key in {"version", "model"}:
continue
context[key] = value
if isinstance(data.get("version"), dict):
version = data["version"]
model_block = data.get("model")
if isinstance(model_block, dict):
for key, value in model_block.items():
if key == "version":
if not version and isinstance(value, dict):
version = value
continue
context.setdefault(key, value)
fallback_files = fallback_files or model_block.get("files") or []
fallback_files = fallback_files or data.get("files") or []
return context, version, fallback_files
@staticmethod
def _ensure_list(value) -> List:
if isinstance(value, list):
return value
if value is None:
return []
return [value]
@staticmethod
def _build_model_info(context: Dict) -> Dict:
tags = context.get("tags")
if not isinstance(tags, list):
tags = list(tags) if isinstance(tags, (set, tuple)) else ([] if tags is None else [tags])
return {
"name": context.get("name"),
"type": context.get("type"),
"nsfw": bool(context.get("is_nsfw", context.get("nsfw", False))),
"description": context.get("description"),
"tags": tags,
}
@staticmethod
def _build_creator_info(context: Dict) -> Dict:
username = context.get("creator_username") or context.get("username") or ""
image = context.get("creator_image") or context.get("creator_avatar") or ""
creator: Dict[str, Optional[str]] = {
"username": username,
"image": image,
}
if context.get("creator_name"):
creator["name"] = context["creator_name"]
if context.get("creator_url"):
creator["url"] = context["creator_url"]
return creator
@staticmethod
def _transform_file_entry(file_data: Dict) -> Dict:
mirrors = file_data.get("mirrors") or []
if not isinstance(mirrors, list):
mirrors = [mirrors]
available_mirror = next(
(mirror for mirror in mirrors if isinstance(mirror, dict) and mirror.get("deletedAt") is None),
None
)
download_url = file_data.get("downloadUrl")
if not download_url and available_mirror:
download_url = available_mirror.get("url")
name = file_data.get("name")
if not name and available_mirror:
name = available_mirror.get("filename")
transformed: Dict = {
"id": file_data.get("id"),
"sizeKB": file_data.get("sizeKB"),
"name": name,
"type": file_data.get("type"),
"downloadUrl": download_url,
"primary": True,
# TODO: for some reason is_primary is false in CivArchive response, need to figure this out,
# "primary": bool(file_data.get("is_primary", file_data.get("primary", False))),
"mirrors": mirrors,
}
sha256 = file_data.get("sha256")
if sha256:
transformed["hashes"] = {"SHA256": str(sha256).upper()}
elif isinstance(file_data.get("hashes"), dict):
transformed["hashes"] = file_data["hashes"]
if "metadata" in file_data:
transformed["metadata"] = file_data["metadata"]
if file_data.get("modelVersionId") is not None:
transformed["modelVersionId"] = file_data.get("modelVersionId")
elif file_data.get("model_version_id") is not None:
transformed["modelVersionId"] = file_data.get("model_version_id")
if file_data.get("modelId") is not None:
transformed["modelId"] = file_data.get("modelId")
elif file_data.get("model_id") is not None:
transformed["modelId"] = file_data.get("model_id")
return transformed
def _transform_files(
self,
files: Optional[List[Dict]],
fallback_files: Optional[List[Dict]] = None
) -> List[Dict]:
candidates: List[Dict] = []
if isinstance(files, list) and files:
candidates = files
elif isinstance(fallback_files, list):
candidates = fallback_files
transformed_files: List[Dict] = []
for file_data in candidates:
if isinstance(file_data, dict):
transformed_files.append(self._transform_file_entry(file_data))
return transformed_files
def _transform_version(
self,
context: Dict,
version: Dict,
fallback_files: Optional[List[Dict]] = None
) -> Optional[Dict]:
if not version:
return None
version_copy = deepcopy(version)
version_copy.pop("model", None)
version_copy.pop("creator", None)
if "trigger" in version_copy:
triggers = version_copy.pop("trigger")
if isinstance(triggers, list):
version_copy["trainedWords"] = triggers
elif triggers is None:
version_copy["trainedWords"] = []
else:
version_copy["trainedWords"] = [triggers]
if "trainedWords" in version_copy and isinstance(version_copy["trainedWords"], str):
version_copy["trainedWords"] = [version_copy["trainedWords"]]
if "nsfw_level" in version_copy:
version_copy["nsfwLevel"] = version_copy.pop("nsfw_level")
elif "nsfwLevel" not in version_copy and context.get("nsfw_level") is not None:
version_copy["nsfwLevel"] = context.get("nsfw_level")
stats_keys = ["downloadCount", "ratingCount", "rating"]
stats = {key: version_copy.pop(key) for key in stats_keys if key in version_copy}
if stats:
version_copy["stats"] = stats
version_copy["files"] = self._transform_files(version_copy.get("files"), fallback_files)
version_copy["images"] = self._ensure_list(version_copy.get("images"))
version_copy["model"] = self._build_model_info(context)
version_copy["creator"] = self._build_creator_info(context)
version_copy["source"] = "civarchive"
version_copy["is_deleted"] = bool(context.get("deletedAt")) or bool(version.get("deletedAt"))
return version_copy
async def _resolve_version_from_files(self, payload: Dict) -> Optional[Dict]:
"""Fallback to fetch version data when only file metadata is available"""
data = self._normalize_payload(payload)
files = data.get("files") or payload.get("files") or []
if not isinstance(files, list):
files = [files]
for file_data in files:
if not isinstance(file_data, dict):
continue
model_id = file_data.get("model_id") or file_data.get("modelId")
version_id = file_data.get("model_version_id") or file_data.get("modelVersionId")
if model_id is None or version_id is None:
continue
resolved = await self.get_model_version(model_id, version_id)
if resolved:
return resolved
return None
async def get_model_by_hash(self, model_hash: str) -> Tuple[Optional[Dict], Optional[str]]:
"""Find model by SHA256 hash value using CivArchive API"""
try:
payload, error = await self._request_json(f"/sha256/{model_hash.lower()}")
if error:
if "not found" in error.lower():
return None, "Model not found"
return None, error
context, version_data, fallback_files = self._split_context(payload)
transformed = self._transform_version(context, version_data, fallback_files)
if transformed:
return transformed, None
resolved = await self._resolve_version_from_files(payload)
if resolved:
return resolved, None
logger.error("Error fetching version of CivArchive model by hash %s", model_hash[:10])
return None, "No version data found"
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model by hash {model_hash[:10]}: {e}")
return None, str(e)
async def get_model_versions(self, model_id: str) -> Optional[Dict]:
"""Get all versions of a model using CivArchive API"""
try:
payload, error = await self._request_json(f"/models/{model_id}")
if error or payload is None:
if error and "not found" in error.lower():
return None
logger.error(f"Error fetching CivArchive model versions for {model_id}: {error}")
return None
data = self._normalize_payload(payload)
context, version_data, fallback_files = self._split_context(payload)
versions_meta = data.get("versions") or []
transformed_versions: List[Dict] = []
for meta in versions_meta:
if not isinstance(meta, dict):
continue
version_id = meta.get("id")
if version_id is None:
continue
target_model_id = meta.get("modelId") or model_id
version = await self.get_model_version(target_model_id, version_id)
if version:
transformed_versions.append(version)
# Ensure the primary version is included even if versions list was empty
primary_version = self._transform_version(context, version_data, fallback_files)
if primary_version:
transformed_versions.insert(0, primary_version)
ordered_versions: List[Dict] = []
seen_ids = set()
for version in transformed_versions:
version_id = version.get("id")
if version_id in seen_ids:
continue
seen_ids.add(version_id)
ordered_versions.append(version)
return {
"modelVersions": ordered_versions,
"type": context.get("type", ""),
"name": context.get("name", ""),
}
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model versions for {model_id}: {e}")
return None
async def get_model_version(self, model_id: int = None, version_id: int = None) -> Optional[Dict]:
"""Get specific model version using CivArchive API
Args:
model_id: The model ID (required)
version_id: Optional specific version ID to filter to
Returns:
Optional[Dict]: The model version data or None if not found
"""
if model_id is None:
return None
try:
params = {"modelVersionId": version_id} if version_id is not None else None
payload, error = await self._request_json(f"/models/{model_id}", params=params)
if error or payload is None:
if error and "not found" in error.lower():
return None
logger.error(f"Error fetching CivArchive model version via API {model_id}/{version_id}: {error}")
return None
context, version_data, fallback_files = self._split_context(payload)
if not version_data:
return await self._resolve_version_from_files(payload)
if version_id is not None:
raw_id = version_data.get("id")
if raw_id != version_id:
logger.warning(
"Requested version %s doesn't match default version %s for model %s",
version_id,
raw_id,
model_id,
)
return None
actual_model_id = version_data.get("modelId")
context_model_id = context.get("id")
# CivArchive can respond with data for a different model id while already
# returning the fully resolved model context. Only follow the redirect when
# the context itself still points to the original (wrong) model.
if (
actual_model_id is not None
and str(actual_model_id) != str(model_id)
and (context_model_id is None or str(context_model_id) != str(actual_model_id))
):
return await self.get_model_version(actual_model_id, version_id)
return self._transform_version(context, version_data, fallback_files)
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model version via API {model_id}/{version_id}: {e}")
return None
async def get_model_version_info(self, version_id: str) -> Tuple[Optional[Dict], Optional[str]]:
""" Fetch model version metadata using a known bogus model lookup
CivArchive lacks a direct version lookup API, this uses a workaround (which we handle in the main model request now)
Args:
version_id: The model version ID
Returns:
Tuple[Optional[Dict], Optional[str]]: (version_data, error_message)
"""
version = await self.get_model_version(1, version_id)
if version is None:
return None, "Model not found"
return version, None

View File

@@ -1,11 +1,12 @@
from datetime import datetime
import aiohttp
import os
import logging
import asyncio
from email.parser import Parser
from typing import Optional, Dict, Tuple, List
from urllib.parse import unquote
import copy
import logging
import os
from typing import Any, Optional, Dict, Tuple, List, Sequence
from .model_metadata_provider import CivitaiModelMetadataProvider, ModelMetadataProviderManager
from .downloader import get_downloader
from .errors import RateLimitError, ResourceNotFoundError
from ..utils.civitai_utils import resolve_license_payload
logger = logging.getLogger(__name__)
@@ -19,6 +20,11 @@ class CivitaiClient:
async with cls._lock:
if cls._instance is None:
cls._instance = cls()
# Register this client as a metadata provider
provider_manager = await ModelMetadataProviderManager.get_instance()
provider_manager.register_provider('civitai', CivitaiModelMetadataProvider(cls._instance), True)
return cls._instance
def __init__(self):
@@ -28,80 +34,49 @@ class CivitaiClient:
self._initialized = True
self.base_url = "https://civitai.com/api/v1"
self.headers = {
'User-Agent': 'ComfyUI-LoRA-Manager/1.0'
}
self._session = None
self._session_created_at = None
# Adjust chunk size based on storage type - consider making this configurable
self.chunk_size = 4 * 1024 * 1024 # 4MB chunks for better HDD throughput
async def _make_request(
self,
method: str,
url: str,
*,
use_auth: bool = False,
**kwargs,
) -> Tuple[bool, Dict | str]:
"""Wrapper around downloader.make_request that surfaces rate limits."""
downloader = await get_downloader()
success, result = await downloader.make_request(
method,
url,
use_auth=use_auth,
**kwargs,
)
if not success and isinstance(result, RateLimitError):
if result.provider is None:
result.provider = "civitai_api"
raise result
return success, result
@staticmethod
def _remove_comfy_metadata(model_version: Optional[Dict]) -> None:
"""Remove Comfy-specific metadata from model version images."""
if not isinstance(model_version, dict):
return
images = model_version.get("images")
if not isinstance(images, list):
return
for image in images:
if not isinstance(image, dict):
continue
meta = image.get("meta")
if isinstance(meta, dict) and "comfy" in meta:
meta.pop("comfy", None)
@property
async def session(self) -> aiohttp.ClientSession:
"""Lazy initialize the session"""
if self._session is None:
# Optimize TCP connection parameters
connector = aiohttp.TCPConnector(
ssl=True,
limit=8, # Increase from 3 to 8 for better parallelism
ttl_dns_cache=300, # Enable DNS caching with reasonable timeout
force_close=False, # Keep connections for reuse
enable_cleanup_closed=True
)
trust_env = True # Allow using system environment proxy settings
# Configure timeout parameters - increase read timeout for large files and remove sock_read timeout
timeout = aiohttp.ClientTimeout(total=None, connect=60, sock_read=None)
self._session = aiohttp.ClientSession(
connector=connector,
trust_env=trust_env,
timeout=timeout
)
self._session_created_at = datetime.now()
return self._session
async def _ensure_fresh_session(self):
"""Refresh session if it's been open too long"""
if self._session is not None:
if not hasattr(self, '_session_created_at') or \
(datetime.now() - self._session_created_at).total_seconds() > 300: # 5 minutes
await self.close()
self._session = None
return await self.session
def _parse_content_disposition(self, header: str) -> str:
"""Parse filename from content-disposition header"""
if not header:
return None
# Handle quoted filenames
if 'filename="' in header:
start = header.index('filename="') + 10
end = header.index('"', start)
return unquote(header[start:end])
# Fallback to original parsing
disposition = Parser().parsestr(f'Content-Disposition: {header}')
filename = disposition.get_param('filename')
if filename:
return unquote(filename)
return None
def _get_request_headers(self) -> dict:
"""Get request headers with optional API key"""
headers = {
'User-Agent': 'ComfyUI-LoRA-Manager/1.0',
'Content-Type': 'application/json'
}
from .settings_manager import settings
api_key = settings.get('civitai_api_key')
if (api_key):
headers['Authorization'] = f'Bearer {api_key}'
return headers
async def _download_file(self, url: str, save_dir: str, default_filename: str, progress_callback=None) -> Tuple[bool, str]:
async def download_file(self, url: str, save_dir: str, default_filename: str, progress_callback=None) -> Tuple[bool, str]:
"""Download file with resumable downloads and retry mechanism
Args:
@@ -113,316 +88,335 @@ class CivitaiClient:
Returns:
Tuple[bool, str]: (success, save_path or error message)
"""
max_retries = 5
retry_count = 0
base_delay = 2.0 # Base delay for exponential backoff
# Initial setup
session = await self._ensure_fresh_session()
downloader = await get_downloader()
save_path = os.path.join(save_dir, default_filename)
part_path = save_path + '.part'
# Get existing file size for resume
resume_offset = 0
if os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Resuming download from offset {resume_offset} bytes")
# Use unified downloader with CivitAI authentication
success, result = await downloader.download_file(
url=url,
save_path=save_path,
progress_callback=progress_callback,
use_auth=True, # Enable CivitAI authentication
allow_resume=True
)
total_size = 0
filename = default_filename
while retry_count <= max_retries:
try:
headers = self._get_request_headers()
# Add Range header for resume if we have partial data
if resume_offset > 0:
headers['Range'] = f'bytes={resume_offset}-'
# Add Range header to allow resumable downloads
headers['Accept-Encoding'] = 'identity' # Disable compression for better chunked downloads
logger.debug(f"Download attempt {retry_count + 1}/{max_retries + 1} from: {url}")
if resume_offset > 0:
logger.debug(f"Requesting range from byte {resume_offset}")
async with session.get(url, headers=headers, allow_redirects=True) as response:
# Handle different response codes
if response.status == 200:
# Full content response
if resume_offset > 0:
# Server doesn't support ranges, restart from beginning
logger.warning("Server doesn't support range requests, restarting download")
resume_offset = 0
if os.path.exists(part_path):
os.remove(part_path)
elif response.status == 206:
# Partial content response (resume successful)
content_range = response.headers.get('Content-Range')
if content_range:
# Parse total size from Content-Range header (e.g., "bytes 1024-2047/2048")
range_parts = content_range.split('/')
if len(range_parts) == 2:
total_size = int(range_parts[1])
logger.info(f"Successfully resumed download from byte {resume_offset}")
elif response.status == 416:
# Range not satisfiable - file might be complete or corrupted
if os.path.exists(part_path):
part_size = os.path.getsize(part_path)
logger.warning(f"Range not satisfiable. Part file size: {part_size}")
# Try to get actual file size
head_response = await session.head(url, headers=self._get_request_headers())
if head_response.status == 200:
actual_size = int(head_response.headers.get('content-length', 0))
if part_size == actual_size:
# File is complete, just rename it
os.rename(part_path, save_path)
if progress_callback:
await progress_callback(100)
return True, save_path
# Remove corrupted part file and restart
os.remove(part_path)
resume_offset = 0
continue
elif response.status == 401:
logger.warning(f"Unauthorized access to resource: {url} (Status 401)")
return False, "Invalid or missing CivitAI API key, or early access restriction."
elif response.status == 403:
logger.warning(f"Forbidden access to resource: {url} (Status 403)")
return False, "Access forbidden: You don't have permission to download this file."
else:
logger.error(f"Download failed for {url} with status {response.status}")
return False, f"Download failed with status {response.status}"
return success, result
# Get filename from content-disposition header (only on first attempt)
if retry_count == 0:
content_disposition = response.headers.get('Content-Disposition')
parsed_filename = self._parse_content_disposition(content_disposition)
if parsed_filename:
filename = parsed_filename
# Update paths with correct filename
save_path = os.path.join(save_dir, filename)
new_part_path = save_path + '.part'
# Rename existing part file if filename changed
if part_path != new_part_path and os.path.exists(part_path):
os.rename(part_path, new_part_path)
part_path = new_part_path
# Get total file size for progress calculation (if not set from Content-Range)
if total_size == 0:
total_size = int(response.headers.get('content-length', 0))
if response.status == 206:
# For partial content, add the offset to get total file size
total_size += resume_offset
current_size = resume_offset
last_progress_report_time = datetime.now()
# Stream download to file with progress updates using larger buffer
loop = asyncio.get_running_loop()
mode = 'ab' if resume_offset > 0 else 'wb'
with open(part_path, mode) as f:
async for chunk in response.content.iter_chunked(self.chunk_size):
if chunk:
# Run blocking file write in executor
await loop.run_in_executor(None, f.write, chunk)
current_size += len(chunk)
# Limit progress update frequency to reduce overhead
now = datetime.now()
time_diff = (now - last_progress_report_time).total_seconds()
if progress_callback and total_size and time_diff >= 1.0:
progress = (current_size / total_size) * 100
await progress_callback(progress)
last_progress_report_time = now
# Download completed successfully
# Verify file size if total_size was provided
final_size = os.path.getsize(part_path)
if total_size > 0 and final_size != total_size:
logger.warning(f"File size mismatch. Expected: {total_size}, Got: {final_size}")
# Don't treat this as fatal error, rename anyway
# Atomically rename .part to final file with retries
max_rename_attempts = 5
rename_attempt = 0
rename_success = False
while rename_attempt < max_rename_attempts and not rename_success:
try:
os.rename(part_path, save_path)
rename_success = True
except PermissionError as e:
rename_attempt += 1
if rename_attempt < max_rename_attempts:
logger.info(f"File still in use, retrying rename in 2 seconds (attempt {rename_attempt}/{max_rename_attempts})")
await asyncio.sleep(2) # Wait before retrying
else:
logger.error(f"Failed to rename file after {max_rename_attempts} attempts: {e}")
return False, f"Failed to finalize download: {str(e)}"
# Ensure 100% progress is reported
if progress_callback:
await progress_callback(100)
return True, save_path
except (aiohttp.ClientError, aiohttp.ClientPayloadError,
aiohttp.ServerDisconnectedError, asyncio.TimeoutError) as e:
retry_count += 1
logger.warning(f"Network error during download (attempt {retry_count}/{max_retries + 1}): {e}")
if retry_count <= max_retries:
# Calculate delay with exponential backoff
delay = base_delay * (2 ** (retry_count - 1))
logger.info(f"Retrying in {delay} seconds...")
await asyncio.sleep(delay)
# Update resume offset for next attempt
if os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Will resume from byte {resume_offset}")
# Refresh session to get new connection
await self.close()
session = await self._ensure_fresh_session()
continue
else:
logger.error(f"Max retries exceeded for download: {e}")
return False, f"Network error after {max_retries + 1} attempts: {str(e)}"
except Exception as e:
logger.error(f"Unexpected download error: {e}")
return False, str(e)
return False, f"Download failed after {max_retries + 1} attempts"
async def get_model_by_hash(self, model_hash: str) -> Optional[Dict]:
async def get_model_by_hash(self, model_hash: str) -> Tuple[Optional[Dict], Optional[str]]:
try:
session = await self._ensure_fresh_session()
async with session.get(f"{self.base_url}/model-versions/by-hash/{model_hash}") as response:
if response.status == 200:
return await response.json()
return None
except Exception as e:
logger.error(f"API Error: {str(e)}")
return None
success, version = await self._make_request(
'GET',
f"{self.base_url}/model-versions/by-hash/{model_hash}",
use_auth=True
)
if not success:
message = str(version)
if "not found" in message.lower():
return None, "Model not found"
logger.error("Failed to fetch model info for %s: %s", model_hash[:10], message)
return None, message
model_id = version.get('modelId')
if model_id:
model_data = await self._fetch_model_data(model_id)
if model_data:
self._enrich_version_with_model_data(version, model_data)
self._remove_comfy_metadata(version)
return version, None
except RateLimitError:
raise
except Exception as exc:
logger.error("API Error: %s", exc)
return None, str(exc)
async def download_preview_image(self, image_url: str, save_path: str):
try:
session = await self._ensure_fresh_session()
async with session.get(image_url) as response:
if response.status == 200:
content = await response.read()
with open(save_path, 'wb') as f:
f.write(content)
return True
return False
downloader = await get_downloader()
success, content, headers = await downloader.download_to_memory(
image_url,
use_auth=False # Preview images don't need auth
)
if success:
# Ensure directory exists
os.makedirs(os.path.dirname(save_path), exist_ok=True)
with open(save_path, 'wb') as f:
f.write(content)
return True
return False
except Exception as e:
print(f"Download Error: {str(e)}")
logger.error(f"Download Error: {str(e)}")
return False
async def get_model_versions(self, model_id: str) -> List[Dict]:
@staticmethod
def _extract_error_message(payload: Any) -> str:
"""Return a human-readable error message from an API payload."""
def _from_value(value: Any) -> str:
if isinstance(value, str):
return value
if isinstance(value, dict):
for key in ("message", "error", "detail", "details"):
if key in value:
candidate = _from_value(value[key])
if candidate:
return candidate
if isinstance(value, list):
for item in value:
candidate = _from_value(item)
if candidate:
return candidate
return ""
return _from_value(payload)
async def get_model_versions(self, model_id: str) -> Optional[Dict]:
"""Get all versions of a model with local availability info"""
try:
session = await self._ensure_fresh_session() # Use fresh session
async with session.get(f"{self.base_url}/models/{model_id}") as response:
if response.status != 200:
return None
data = await response.json()
success, result = await self._make_request(
'GET',
f"{self.base_url}/models/{model_id}",
use_auth=True
)
if success:
# Also return model type along with versions
return {
'modelVersions': data.get('modelVersions', []),
'type': data.get('type', '')
'modelVersions': result.get('modelVersions', []),
'type': result.get('type', ''),
'name': result.get('name', '')
}
message = self._extract_error_message(result)
if message and 'not found' in message.lower():
raise ResourceNotFoundError(f"Resource not found for model {model_id}")
if message:
raise RuntimeError(message)
return None
except RateLimitError:
raise
except ResourceNotFoundError as exc:
logger.info("Model %s is no longer available on Civitai: %s", model_id, exc)
raise
except Exception as e:
logger.error(f"Error fetching model versions: {e}")
logger.error("Error fetching model versions: %s", e, exc_info=True)
raise
async def get_model_versions_bulk(
self, model_ids: Sequence[int]
) -> Optional[Dict[int, Dict]]:
"""Fetch model metadata for multiple ids using the batch API."""
deduped: Dict[int, None] = {}
for raw_id in model_ids:
try:
normalized = int(raw_id)
except (TypeError, ValueError):
continue
deduped.setdefault(normalized, None)
normalized_ids = [str(model_id) for model_id in deduped.keys()]
if not normalized_ids:
return {}
try:
query = ",".join(normalized_ids)
success, result = await self._make_request(
'GET',
f"{self.base_url}/models",
use_auth=True,
params={'ids': query},
)
if not success:
return None
items = result.get('items') if isinstance(result, dict) else None
if not isinstance(items, list):
return {}
payload: Dict[int, Dict] = {}
for item in items:
if not isinstance(item, dict):
continue
model_id = item.get('id')
try:
normalized_id = int(model_id)
except (TypeError, ValueError):
continue
payload[normalized_id] = {
'modelVersions': item.get('modelVersions', []),
'type': item.get('type', ''),
'name': item.get('name', ''),
'allowNoCredit': item.get('allowNoCredit'),
'allowCommercialUse': item.get('allowCommercialUse'),
'allowDerivatives': item.get('allowDerivatives'),
'allowDifferentLicense': item.get('allowDifferentLicense'),
}
return payload
except RateLimitError:
raise
except Exception as exc:
logger.error(f"Error fetching model versions in bulk: {exc}")
return None
async def get_model_version(self, model_id: int = None, version_id: int = None) -> Optional[Dict]:
"""Get specific model version with additional metadata
Args:
model_id: The Civitai model ID (optional if version_id is provided)
version_id: Optional specific version ID to retrieve
Returns:
Optional[Dict]: The model version data with additional fields or None if not found
"""
"""Get specific model version with additional metadata."""
try:
session = await self._ensure_fresh_session()
headers = self._get_request_headers()
# Case 1: Only version_id is provided
if model_id is None and version_id is not None:
# First get the version info to extract model_id
async with session.get(f"{self.base_url}/model-versions/{version_id}", headers=headers) as response:
if response.status != 200:
return None
version = await response.json()
model_id = version.get('modelId')
if not model_id:
logger.error(f"No modelId found in version {version_id}")
return None
# Now get the model data for additional metadata
async with session.get(f"{self.base_url}/models/{model_id}") as response:
if response.status != 200:
return version # Return version without additional metadata
model_data = await response.json()
# Enrich version with model data
version['model']['description'] = model_data.get("description")
version['model']['tags'] = model_data.get("tags", [])
version['creator'] = model_data.get("creator")
return version
# Case 2: model_id is provided (with or without version_id)
elif model_id is not None:
# Step 1: Get model data to find version_id if not provided and get additional metadata
async with session.get(f"{self.base_url}/models/{model_id}") as response:
if response.status != 200:
return None
data = await response.json()
model_versions = data.get('modelVersions', [])
# Step 2: Determine the version_id to use
target_version_id = version_id
if target_version_id is None:
target_version_id = model_versions[0].get('id')
# Step 3: Get detailed version info using the version_id
async with session.get(f"{self.base_url}/model-versions/{target_version_id}", headers=headers) as response:
if response.status != 200:
return None
version = await response.json()
# Step 4: Enrich version_info with model data
# Add description and tags from model data
version['model']['description'] = data.get("description")
version['model']['tags'] = data.get("tags", [])
# Add creator from model data
version['creator'] = data.get("creator")
return version
# Case 3: Neither model_id nor version_id provided
else:
logger.error("Either model_id or version_id must be provided")
return None
return await self._get_version_by_id_only(version_id)
if model_id is not None:
return await self._get_version_with_model_id(model_id, version_id)
logger.error("Either model_id or version_id must be provided")
return None
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching model version: {e}")
return None
async def _get_version_by_id_only(self, version_id: int) -> Optional[Dict]:
version = await self._fetch_version_by_id(version_id)
if version is None:
return None
model_id = version.get('modelId')
if not model_id:
logger.error(f"No modelId found in version {version_id}")
return None
model_data = await self._fetch_model_data(model_id)
if model_data:
self._enrich_version_with_model_data(version, model_data)
self._remove_comfy_metadata(version)
return version
async def _get_version_with_model_id(self, model_id: int, version_id: Optional[int]) -> Optional[Dict]:
model_data = await self._fetch_model_data(model_id)
if not model_data:
return None
target_version = self._select_target_version(model_data, model_id, version_id)
if target_version is None:
return None
target_version_id = target_version.get('id')
version = await self._fetch_version_by_id(target_version_id) if target_version_id else None
if version is None:
model_hash = self._extract_primary_model_hash(target_version)
if model_hash:
version = await self._fetch_version_by_hash(model_hash)
else:
logger.warning(
f"No primary model hash found for model {model_id} version {target_version_id}"
)
if version is None:
version = self._build_version_from_model_data(target_version, model_id, model_data)
self._enrich_version_with_model_data(version, model_data)
self._remove_comfy_metadata(version)
return version
async def _fetch_model_data(self, model_id: int) -> Optional[Dict]:
success, data = await self._make_request(
'GET',
f"{self.base_url}/models/{model_id}",
use_auth=True
)
if success:
return data
logger.warning(f"Failed to fetch model data for model {model_id}")
return None
async def _fetch_version_by_id(self, version_id: Optional[int]) -> Optional[Dict]:
if version_id is None:
return None
success, version = await self._make_request(
'GET',
f"{self.base_url}/model-versions/{version_id}",
use_auth=True
)
if success:
return version
logger.warning(f"Failed to fetch version by id {version_id}")
return None
async def _fetch_version_by_hash(self, model_hash: Optional[str]) -> Optional[Dict]:
if not model_hash:
return None
success, version = await self._make_request(
'GET',
f"{self.base_url}/model-versions/by-hash/{model_hash}",
use_auth=True
)
if success:
return version
logger.warning(f"Failed to fetch version by hash {model_hash}")
return None
def _select_target_version(self, model_data: Dict, model_id: int, version_id: Optional[int]) -> Optional[Dict]:
model_versions = model_data.get('modelVersions', [])
if not model_versions:
logger.warning(f"No model versions found for model {model_id}")
return None
if version_id is not None:
target_version = next(
(item for item in model_versions if item.get('id') == version_id),
None
)
if target_version is None:
logger.warning(
f"Version {version_id} not found for model {model_id}, defaulting to first version"
)
return model_versions[0]
return target_version
return model_versions[0]
def _extract_primary_model_hash(self, version_entry: Dict) -> Optional[str]:
for file_info in version_entry.get('files', []):
if file_info.get('type') == 'Model' and file_info.get('primary'):
hashes = file_info.get('hashes', {})
model_hash = hashes.get('SHA256')
if model_hash:
return model_hash
return None
def _build_version_from_model_data(self, version_entry: Dict, model_id: int, model_data: Dict) -> Dict:
version = copy.deepcopy(version_entry)
version.pop('index', None)
version['modelId'] = model_id
version['model'] = {
'name': model_data.get('name'),
'type': model_data.get('type'),
'nsfw': model_data.get('nsfw'),
'poi': model_data.get('poi')
}
return version
def _enrich_version_with_model_data(self, version: Dict, model_data: Dict) -> None:
model_info = version.get('model')
if not isinstance(model_info, dict):
model_info = {}
version['model'] = model_info
model_info['description'] = model_data.get("description")
model_info['tags'] = model_data.get("tags", [])
version['creator'] = model_data.get("creator")
license_payload = resolve_license_payload(model_data)
for field, value in license_payload.items():
model_info[field] = value
async def get_model_version_info(self, version_id: str) -> Tuple[Optional[Dict], Optional[str]]:
"""Fetch model version metadata from Civitai
@@ -435,119 +429,39 @@ class CivitaiClient:
- An error message if there was an error, or None on success
"""
try:
session = await self._ensure_fresh_session()
url = f"{self.base_url}/model-versions/{version_id}"
headers = self._get_request_headers()
logger.debug(f"Resolving DNS for model version info: {url}")
async with session.get(url, headers=headers) as response:
if response.status == 200:
logger.debug(f"Successfully fetched model version info for: {version_id}")
return await response.json(), None
# Handle specific error cases
if response.status == 404:
# Try to parse the error message
try:
error_data = await response.json()
error_msg = error_data.get('error', f"Model not found (status 404)")
logger.warning(f"Model version not found: {version_id} - {error_msg}")
return None, error_msg
except:
return None, "Model not found (status 404)"
# Other error cases
logger.error(f"Failed to fetch model info for {version_id} (status {response.status})")
return None, f"Failed to fetch model info (status {response.status})"
success, result = await self._make_request(
'GET',
url,
use_auth=True
)
if success:
logger.debug(f"Successfully fetched model version info for: {version_id}")
self._remove_comfy_metadata(result)
return result, None
# Handle specific error cases
if "not found" in str(result):
error_msg = f"Model not found"
logger.warning(f"Model version not found: {version_id} - {error_msg}")
return None, error_msg
# Other error cases
logger.error(f"Failed to fetch model info for {version_id}: {result}")
return None, str(result)
except RateLimitError:
raise
except Exception as e:
error_msg = f"Error fetching model version info: {e}"
logger.error(error_msg)
return None, error_msg
async def get_model_metadata(self, model_id: str) -> Tuple[Optional[Dict], int]:
"""Fetch model metadata (description, tags, and creator info) from Civitai API
Args:
model_id: The Civitai model ID
Returns:
Tuple[Optional[Dict], int]: A tuple containing:
- A dictionary with model metadata or None if not found
- The HTTP status code from the request
"""
try:
session = await self._ensure_fresh_session()
headers = self._get_request_headers()
url = f"{self.base_url}/models/{model_id}"
async with session.get(url, headers=headers) as response:
status_code = response.status
if status_code != 200:
logger.warning(f"Failed to fetch model metadata: Status {status_code}")
return None, status_code
data = await response.json()
# Extract relevant metadata
metadata = {
"description": data.get("description") or "No model description available",
"tags": data.get("tags", []),
"creator": {
"username": data.get("creator", {}).get("username"),
"image": data.get("creator", {}).get("image")
}
}
if metadata["description"] or metadata["tags"] or metadata["creator"]["username"]:
return metadata, status_code
else:
logger.warning(f"No metadata found for model {model_id}")
return None, status_code
except Exception as e:
logger.error(f"Error fetching model metadata: {e}", exc_info=True)
return None, 0
# Keep old method for backward compatibility, delegating to the new one
async def get_model_description(self, model_id: str) -> Optional[str]:
"""Fetch the model description from Civitai API (Legacy method)"""
metadata, _ = await self.get_model_metadata(model_id)
return metadata.get("description") if metadata else None
async def close(self):
"""Close the session if it exists"""
if self._session is not None:
await self._session.close()
self._session = None
async def _get_hash_from_civitai(self, model_version_id: str) -> Optional[str]:
"""Get hash from Civitai API"""
try:
session = await self._ensure_fresh_session()
if not session:
return None
version_info = await session.get(f"{self.base_url}/model-versions/{model_version_id}")
if not version_info or not version_info.json().get('files'):
return None
# Get hash from the first file
for file_info in version_info.json().get('files', []):
if file_info.get('hashes', {}).get('SHA256'):
# Convert hash to lowercase to standardize
hash_value = file_info['hashes']['SHA256'].lower()
return hash_value
return None
except Exception as e:
logger.error(f"Error getting hash from Civitai: {e}")
return None
async def get_image_info(self, image_id: str) -> Optional[Dict]:
"""Fetch image information from Civitai API
Args:
image_id: The Civitai image ID
@@ -555,23 +469,62 @@ class CivitaiClient:
Optional[Dict]: The image data or None if not found
"""
try:
session = await self._ensure_fresh_session()
headers = self._get_request_headers()
url = f"{self.base_url}/images?imageId={image_id}&nsfw=X"
logger.debug(f"Fetching image info for ID: {image_id}")
async with session.get(url, headers=headers) as response:
if response.status == 200:
data = await response.json()
if data and "items" in data and len(data["items"]) > 0:
logger.debug(f"Successfully fetched image info for ID: {image_id}")
return data["items"][0]
logger.warning(f"No image found with ID: {image_id}")
return None
logger.error(f"Failed to fetch image info for ID: {image_id} (status {response.status})")
success, result = await self._make_request(
'GET',
url,
use_auth=True
)
if success:
if result and "items" in result and len(result["items"]) > 0:
logger.debug(f"Successfully fetched image info for ID: {image_id}")
return result["items"][0]
logger.warning(f"No image found with ID: {image_id}")
return None
logger.error(f"Failed to fetch image info for ID: {image_id}: {result}")
return None
except RateLimitError:
raise
except Exception as e:
error_msg = f"Error fetching image info: {e}"
logger.error(error_msg)
return None
async def get_user_models(self, username: str) -> Optional[List[Dict]]:
"""Fetch all models for a specific Civitai user."""
if not username:
return None
try:
url = f"{self.base_url}/models?username={username}"
success, result = await self._make_request(
'GET',
url,
use_auth=True
)
if not success:
logger.error("Failed to fetch models for %s: %s", username, result)
return None
items = result.get("items") if isinstance(result, dict) else None
if not isinstance(items, list):
return []
for model in items:
versions = model.get("modelVersions")
if not isinstance(versions, list):
continue
for version in versions:
self._remove_comfy_metadata(version)
return items
except RateLimitError:
raise
except Exception as exc: # pragma: no cover - defensive logging
logger.error("Error fetching models for %s: %s", username, exc)
return None

View File

@@ -0,0 +1,178 @@
"""Service wrapper for coordinating download lifecycle events."""
from __future__ import annotations
import logging
from typing import Any, Awaitable, Callable, Dict, Optional
from .downloader import DownloadProgress
logger = logging.getLogger(__name__)
class DownloadCoordinator:
"""Manage download scheduling, cancellation and introspection."""
def __init__(
self,
*,
ws_manager,
download_manager_factory: Callable[[], Awaitable],
) -> None:
self._ws_manager = ws_manager
self._download_manager_factory = download_manager_factory
async def schedule_download(self, payload: Dict[str, Any]) -> Dict[str, Any]:
"""Schedule a download using the provided payload."""
download_manager = await self._download_manager_factory()
download_id = payload.get("download_id") or self._ws_manager.generate_download_id()
payload.setdefault("download_id", download_id)
async def progress_callback(progress: Any, snapshot: Optional[DownloadProgress] = None) -> None:
percent = 0.0
metrics: Optional[DownloadProgress] = None
if isinstance(progress, DownloadProgress):
metrics = progress
percent = progress.percent_complete
elif isinstance(snapshot, DownloadProgress):
metrics = snapshot
percent = snapshot.percent_complete
else:
try:
percent = float(progress)
except (TypeError, ValueError):
percent = 0.0
payload: Dict[str, Any] = {
"status": "progress",
"progress": round(percent),
"download_id": download_id,
}
if metrics is not None:
payload.update(
{
"bytes_downloaded": metrics.bytes_downloaded,
"total_bytes": metrics.total_bytes,
"bytes_per_second": metrics.bytes_per_second,
}
)
await self._ws_manager.broadcast_download_progress(
download_id,
payload,
)
model_id = self._parse_optional_int(payload.get("model_id"), "model_id")
model_version_id = self._parse_optional_int(
payload.get("model_version_id"), "model_version_id"
)
if model_id is None and model_version_id is None:
raise ValueError(
"Missing required parameter: Please provide either 'model_id' or 'model_version_id'"
)
result = await download_manager.download_from_civitai(
model_id=model_id,
model_version_id=model_version_id,
save_dir=payload.get("model_root"),
relative_path=payload.get("relative_path", ""),
use_default_paths=payload.get("use_default_paths", False),
progress_callback=progress_callback,
download_id=download_id,
source=payload.get("source"),
)
result["download_id"] = download_id
return result
async def cancel_download(self, download_id: str) -> Dict[str, Any]:
"""Cancel an active download and emit a broadcast event."""
download_manager = await self._download_manager_factory()
result = await download_manager.cancel_download(download_id)
await self._ws_manager.broadcast_download_progress(
download_id,
{
"status": "cancelled",
"progress": 0,
"download_id": download_id,
"message": "Download cancelled by user",
},
)
return result
async def pause_download(self, download_id: str) -> Dict[str, Any]:
"""Pause an active download and notify listeners."""
download_manager = await self._download_manager_factory()
result = await download_manager.pause_download(download_id)
if result.get("success"):
cached_progress = self._ws_manager.get_download_progress(download_id) or {}
payload: Dict[str, Any] = {
"status": "paused",
"progress": cached_progress.get("progress", 0),
"download_id": download_id,
"message": "Download paused by user",
}
for field in ("bytes_downloaded", "total_bytes", "bytes_per_second"):
if field in cached_progress:
payload[field] = cached_progress[field]
payload["bytes_per_second"] = 0.0
await self._ws_manager.broadcast_download_progress(download_id, payload)
return result
async def resume_download(self, download_id: str) -> Dict[str, Any]:
"""Resume a paused download and notify listeners."""
download_manager = await self._download_manager_factory()
result = await download_manager.resume_download(download_id)
if result.get("success"):
cached_progress = self._ws_manager.get_download_progress(download_id) or {}
payload: Dict[str, Any] = {
"status": "downloading",
"progress": cached_progress.get("progress", 0),
"download_id": download_id,
"message": "Download resumed by user",
}
for field in ("bytes_downloaded", "total_bytes"):
if field in cached_progress:
payload[field] = cached_progress[field]
payload["bytes_per_second"] = cached_progress.get("bytes_per_second", 0.0)
await self._ws_manager.broadcast_download_progress(download_id, payload)
return result
async def list_active_downloads(self) -> Dict[str, Any]:
"""Return the active download map from the underlying manager."""
download_manager = await self._download_manager_factory()
return await download_manager.get_active_downloads()
def _parse_optional_int(self, value: Any, field: str) -> Optional[int]:
"""Parse an optional integer from user input."""
if value is None or value == "":
return None
try:
return int(value)
except (TypeError, ValueError) as exc:
raise ValueError(f"Invalid {field}: Must be an integer") from exc

File diff suppressed because it is too large Load Diff

863
py/services/downloader.py Normal file
View File

@@ -0,0 +1,863 @@
"""
Unified download manager for all HTTP/HTTPS downloads in the application.
This module provides a centralized download service with:
- Singleton pattern for global session management
- Support for authenticated downloads (e.g., CivitAI API key)
- Resumable downloads with automatic retry
- Progress tracking and callbacks
- Optimized connection pooling and timeouts
- Unified error handling and logging
"""
import os
import logging
import asyncio
import aiohttp
from collections import deque
from dataclasses import dataclass
from datetime import datetime, timedelta
from email.utils import parsedate_to_datetime
from typing import Optional, Dict, Tuple, Callable, Union, Awaitable
from ..services.settings_manager import get_settings_manager
from .errors import RateLimitError
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class DownloadProgress:
"""Snapshot of a download transfer at a moment in time."""
percent_complete: float
bytes_downloaded: int
total_bytes: Optional[int]
bytes_per_second: float
timestamp: float
class DownloadStreamControl:
"""Synchronize pause/resume requests and reconnect hints for a download."""
def __init__(self, *, stall_timeout: Optional[float] = None) -> None:
self._event = asyncio.Event()
self._event.set()
self._reconnect_requested = False
self.last_progress_timestamp: Optional[float] = None
self.stall_timeout: float = float(stall_timeout) if stall_timeout is not None else 120.0
def is_set(self) -> bool:
return self._event.is_set()
def is_paused(self) -> bool:
return not self._event.is_set()
def set(self) -> None:
self._event.set()
def clear(self) -> None:
self._event.clear()
async def wait(self) -> None:
await self._event.wait()
def pause(self) -> None:
self.clear()
def resume(self, *, force_reconnect: bool = False) -> None:
if force_reconnect:
self._reconnect_requested = True
self.set()
def request_reconnect(self) -> None:
self._reconnect_requested = True
self.set()
def has_reconnect_request(self) -> bool:
return self._reconnect_requested
def consume_reconnect_request(self) -> bool:
reconnect = self._reconnect_requested
self._reconnect_requested = False
return reconnect
def mark_progress(self, timestamp: Optional[float] = None) -> None:
self.last_progress_timestamp = timestamp or datetime.now().timestamp()
self._reconnect_requested = False
def time_since_last_progress(self, *, now: Optional[float] = None) -> Optional[float]:
if self.last_progress_timestamp is None:
return None
reference = now if now is not None else datetime.now().timestamp()
return max(0.0, reference - self.last_progress_timestamp)
def update_stall_timeout(self, stall_timeout: float) -> None:
self.stall_timeout = float(stall_timeout)
class DownloadRestartRequested(Exception):
"""Raised when a caller explicitly requests a fresh HTTP stream."""
class DownloadStalledError(Exception):
"""Raised when download progress stalls beyond the configured timeout."""
class Downloader:
"""Unified downloader for all HTTP/HTTPS downloads in the application."""
_instance = None
_lock = asyncio.Lock()
@classmethod
async def get_instance(cls):
"""Get singleton instance of Downloader"""
async with cls._lock:
if cls._instance is None:
cls._instance = cls()
return cls._instance
def __init__(self):
"""Initialize the downloader with optimal settings"""
# Check if already initialized for singleton pattern
if hasattr(self, '_initialized'):
return
self._initialized = True
# Session management
self._session = None
self._session_created_at = None
self._proxy_url = None # Store proxy URL for current session
self._session_lock = asyncio.Lock()
# Configuration
self.chunk_size = 4 * 1024 * 1024 # 4MB chunks for better throughput
self.max_retries = 5
self.base_delay = 2.0 # Base delay for exponential backoff
self.session_timeout = 300 # 5 minutes
self.stall_timeout = self._resolve_stall_timeout()
# Default headers
self.default_headers = {
'User-Agent': 'ComfyUI-LoRA-Manager/1.0',
# Explicitly request uncompressed payloads so aiohttp doesn't need optional
# decoders (e.g. zstandard) that may be missing in runtime environments.
'Accept-Encoding': 'identity',
}
@property
async def session(self) -> aiohttp.ClientSession:
"""Get or create the global aiohttp session with optimized settings"""
if self._session is None or self._should_refresh_session():
async with self._session_lock:
# Double check after acquiring lock
if self._session is None or self._should_refresh_session():
await self._create_session()
return self._session
@property
def proxy_url(self) -> Optional[str]:
"""Get the current proxy URL (initialize if needed)"""
if not hasattr(self, '_proxy_url'):
self._proxy_url = None
return self._proxy_url
def _resolve_stall_timeout(self) -> float:
"""Determine the stall timeout from settings or environment."""
default_timeout = 120.0
settings_timeout = None
try:
settings_manager = get_settings_manager()
settings_timeout = settings_manager.get('download_stall_timeout_seconds')
except Exception as exc: # pragma: no cover - defensive guard
logger.debug("Failed to read stall timeout from settings: %s", exc)
raw_value = (
settings_timeout
if settings_timeout not in (None, "")
else os.environ.get('COMFYUI_DOWNLOAD_STALL_TIMEOUT')
)
try:
timeout_value = float(raw_value)
except (TypeError, ValueError):
timeout_value = default_timeout
return max(30.0, timeout_value)
def _should_refresh_session(self) -> bool:
"""Check if session should be refreshed"""
if self._session is None:
return True
if not hasattr(self, '_session_created_at') or self._session_created_at is None:
return True
# Refresh if session is older than timeout
if (datetime.now() - self._session_created_at).total_seconds() > self.session_timeout:
return True
return False
async def _create_session(self):
"""Create a new aiohttp session with optimized settings.
Note: This is private and caller MUST hold self._session_lock.
"""
# Close existing session if any
if self._session is not None:
try:
await self._session.close()
except Exception as e: # pragma: no cover
logger.warning(f"Error closing previous session: {e}")
finally:
self._session = None
# Check for app-level proxy settings
proxy_url = None
settings_manager = get_settings_manager()
if settings_manager.get('proxy_enabled', False):
proxy_host = settings_manager.get('proxy_host', '').strip()
proxy_port = settings_manager.get('proxy_port', '').strip()
proxy_type = settings_manager.get('proxy_type', 'http').lower()
proxy_username = settings_manager.get('proxy_username', '').strip()
proxy_password = settings_manager.get('proxy_password', '').strip()
if proxy_host and proxy_port:
# Build proxy URL
if proxy_username and proxy_password:
proxy_url = f"{proxy_type}://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"
else:
proxy_url = f"{proxy_type}://{proxy_host}:{proxy_port}"
logger.debug(f"Using app-level proxy: {proxy_type}://{proxy_host}:{proxy_port}")
logger.debug("Proxy mode: app-level proxy is active.")
else:
logger.debug("Proxy mode: system-level proxy (trust_env) will be used if configured in environment.")
# Optimize TCP connection parameters
connector = aiohttp.TCPConnector(
ssl=True,
limit=8, # Concurrent connections
ttl_dns_cache=300, # DNS cache timeout
force_close=False, # Keep connections for reuse
enable_cleanup_closed=True
)
# Configure timeout parameters
timeout = aiohttp.ClientTimeout(
total=None, # No total timeout for large downloads
connect=60, # Connection timeout
sock_read=300 # 5 minute socket read timeout
)
self._session = aiohttp.ClientSession(
connector=connector,
trust_env=proxy_url is None, # Only use system proxy if no app-level proxy is set
timeout=timeout
)
# Store proxy URL for use in requests
self._proxy_url = proxy_url
self._session_created_at = datetime.now()
logger.debug("Created new HTTP session with proxy settings. App-level proxy: %s, System-level proxy (trust_env): %s", bool(proxy_url), proxy_url is None)
def _get_auth_headers(self, use_auth: bool = False) -> Dict[str, str]:
"""Get headers with optional authentication"""
headers = self.default_headers.copy()
if use_auth:
# Add CivitAI API key if available
settings_manager = get_settings_manager()
api_key = settings_manager.get('civitai_api_key')
if api_key:
headers['Authorization'] = f'Bearer {api_key}'
headers['Content-Type'] = 'application/json'
return headers
async def download_file(
self,
url: str,
save_path: str,
progress_callback: Optional[Callable[..., Awaitable[None]]] = None,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
allow_resume: bool = True,
pause_event: Optional[DownloadStreamControl] = None,
) -> Tuple[bool, str]:
"""
Download a file with resumable downloads and retry mechanism
Args:
url: Download URL
save_path: Full path where the file should be saved
progress_callback: Optional callback for progress updates (0-100)
use_auth: Whether to include authentication headers (e.g., CivitAI API key)
custom_headers: Additional headers to include in request
allow_resume: Whether to support resumable downloads
pause_event: Optional stream control used to pause/resume and request reconnects
Returns:
Tuple[bool, str]: (success, save_path or error message)
"""
retry_count = 0
part_path = save_path + '.part' if allow_resume else save_path
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
# Get existing file size for resume
resume_offset = 0
if allow_resume and os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Resuming download from offset {resume_offset} bytes")
total_size = 0
while retry_count <= self.max_retries:
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[download_file] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[download_file] Using system-level proxy (trust_env) if configured.")
# Add Range header for resume if we have partial data
request_headers = headers.copy()
if allow_resume and resume_offset > 0:
request_headers['Range'] = f'bytes={resume_offset}-'
# Disable compression for better chunked downloads
request_headers['Accept-Encoding'] = 'identity'
logger.debug(f"Download attempt {retry_count + 1}/{self.max_retries + 1} from: {url}")
if resume_offset > 0:
logger.debug(f"Requesting range from byte {resume_offset}")
async with session.get(url, headers=request_headers, allow_redirects=True, proxy=self.proxy_url) as response:
# Handle different response codes
if response.status == 200:
# Full content response
if resume_offset > 0:
# Server doesn't support ranges, restart from beginning
logger.warning("Server doesn't support range requests, restarting download")
resume_offset = 0
if os.path.exists(part_path):
os.remove(part_path)
elif response.status == 206:
# Partial content response (resume successful)
content_range = response.headers.get('Content-Range')
if content_range:
# Parse total size from Content-Range header (e.g., "bytes 1024-2047/2048")
range_parts = content_range.split('/')
if len(range_parts) == 2:
total_size = int(range_parts[1])
logger.info(f"Successfully resumed download from byte {resume_offset}")
elif response.status == 416:
# Range not satisfiable - file might be complete or corrupted
if allow_resume and os.path.exists(part_path):
part_size = os.path.getsize(part_path)
logger.warning(f"Range not satisfiable. Part file size: {part_size}")
# Try to get actual file size
head_response = await session.head(url, headers=headers, proxy=self.proxy_url)
if head_response.status == 200:
actual_size = int(head_response.headers.get('content-length', 0))
if part_size == actual_size:
# File is complete, just rename it
if allow_resume:
os.rename(part_path, save_path)
if progress_callback:
await self._dispatch_progress_callback(
progress_callback,
DownloadProgress(
percent_complete=100.0,
bytes_downloaded=part_size,
total_bytes=actual_size,
bytes_per_second=0.0,
timestamp=datetime.now().timestamp(),
),
)
return True, save_path
# Remove corrupted part file and restart
os.remove(part_path)
resume_offset = 0
continue
elif response.status == 401:
logger.warning(f"Unauthorized access to resource: {url} (Status 401)")
return False, "Invalid or missing API key, or early access restriction."
elif response.status == 403:
logger.warning(f"Forbidden access to resource: {url} (Status 403)")
return False, "Access forbidden: You don't have permission to download this file."
elif response.status == 404:
logger.warning(f"Resource not found: {url} (Status 404)")
return False, "File not found - the download link may be invalid or expired."
else:
logger.error(f"Download failed for {url} with status {response.status}")
return False, f"Download failed with status {response.status}"
# Get total file size for progress calculation (if not set from Content-Range)
if total_size == 0:
total_size = int(response.headers.get('content-length', 0))
if response.status == 206:
# For partial content, add the offset to get total file size
total_size += resume_offset
current_size = resume_offset
last_progress_report_time = datetime.now()
progress_samples: deque[tuple[datetime, int]] = deque()
progress_samples.append((last_progress_report_time, current_size))
# Ensure directory exists
os.makedirs(os.path.dirname(save_path), exist_ok=True)
# Stream download to file with progress updates
loop = asyncio.get_running_loop()
mode = 'ab' if (allow_resume and resume_offset > 0) else 'wb'
control = pause_event
if control is not None:
control.update_stall_timeout(self.stall_timeout)
with open(part_path, mode) as f:
while True:
active_stall_timeout = control.stall_timeout if control else self.stall_timeout
if control is not None:
if control.is_paused():
await control.wait()
resume_time = datetime.now()
last_progress_report_time = resume_time
if control.consume_reconnect_request():
raise DownloadRestartRequested(
"Reconnect requested after resume"
)
elif control.consume_reconnect_request():
raise DownloadRestartRequested("Reconnect requested")
try:
chunk = await asyncio.wait_for(
response.content.read(self.chunk_size),
timeout=active_stall_timeout,
)
except asyncio.TimeoutError as exc:
logger.warning(
"Download stalled for %.1f seconds without progress from %s",
active_stall_timeout,
url,
)
raise DownloadStalledError(
f"No data received for {active_stall_timeout:.1f} seconds"
) from exc
if not chunk:
break
# Run blocking file write in executor
await loop.run_in_executor(None, f.write, chunk)
current_size += len(chunk)
now = datetime.now()
if control is not None:
control.mark_progress(timestamp=now.timestamp())
# Limit progress update frequency to reduce overhead
time_diff = (now - last_progress_report_time).total_seconds()
if progress_callback and time_diff >= 1.0:
progress_samples.append((now, current_size))
cutoff = now - timedelta(seconds=5)
while progress_samples and progress_samples[0][0] < cutoff:
progress_samples.popleft()
percent = (current_size / total_size) * 100 if total_size else 0.0
bytes_per_second = 0.0
if len(progress_samples) >= 2:
first_time, first_bytes = progress_samples[0]
last_time, last_bytes = progress_samples[-1]
elapsed = (last_time - first_time).total_seconds()
if elapsed > 0:
bytes_per_second = (last_bytes - first_bytes) / elapsed
progress_snapshot = DownloadProgress(
percent_complete=percent,
bytes_downloaded=current_size,
total_bytes=total_size or None,
bytes_per_second=bytes_per_second,
timestamp=now.timestamp(),
)
await self._dispatch_progress_callback(progress_callback, progress_snapshot)
last_progress_report_time = now
# Download completed successfully
# Verify file size integrity before finalizing
final_size = os.path.getsize(part_path) if os.path.exists(part_path) else 0
expected_size = total_size if total_size > 0 else None
integrity_error: Optional[str] = None
if final_size <= 0:
integrity_error = "Downloaded file is empty"
elif expected_size is not None and final_size != expected_size:
integrity_error = (
f"File size mismatch. Expected: {expected_size}, Got: {final_size}"
)
if integrity_error is not None:
logger.error(
"Download integrity check failed for %s: %s",
save_path,
integrity_error,
)
# Remove the corrupted payload so future attempts start fresh
if os.path.exists(part_path):
try:
os.remove(part_path)
except OSError as remove_error:
logger.warning(
"Failed to delete corrupted download %s: %s",
part_path,
remove_error,
)
if part_path != save_path and os.path.exists(save_path):
try:
os.remove(save_path)
except OSError as remove_error:
logger.warning(
"Failed to delete target file %s after integrity error: %s",
save_path,
remove_error,
)
retry_count += 1
if retry_count <= self.max_retries:
delay = self.base_delay * (2 ** (retry_count - 1))
logger.info(
"Retrying download in %s seconds due to integrity check failure",
delay,
)
await asyncio.sleep(delay)
resume_offset = 0
total_size = 0
await self._create_session()
continue
return False, integrity_error
# Atomically rename .part to final file (only if using resume)
if allow_resume and part_path != save_path:
max_rename_attempts = 5
rename_attempt = 0
rename_success = False
while rename_attempt < max_rename_attempts and not rename_success:
try:
# If the destination file exists, remove it first (Windows safe)
if os.path.exists(save_path):
os.remove(save_path)
os.rename(part_path, save_path)
rename_success = True
except PermissionError as e:
rename_attempt += 1
if rename_attempt < max_rename_attempts:
logger.info(f"File still in use, retrying rename in 2 seconds (attempt {rename_attempt}/{max_rename_attempts})")
await asyncio.sleep(2)
else:
logger.error(f"Failed to rename file after {max_rename_attempts} attempts: {e}")
return False, f"Failed to finalize download: {str(e)}"
final_size = os.path.getsize(save_path)
# Ensure 100% progress is reported
if progress_callback:
final_snapshot = DownloadProgress(
percent_complete=100.0,
bytes_downloaded=final_size,
total_bytes=total_size or final_size,
bytes_per_second=0.0,
timestamp=datetime.now().timestamp(),
)
await self._dispatch_progress_callback(progress_callback, final_snapshot)
return True, save_path
except (
aiohttp.ClientError,
aiohttp.ClientPayloadError,
aiohttp.ServerDisconnectedError,
asyncio.TimeoutError,
DownloadStalledError,
DownloadRestartRequested,
) as e:
retry_count += 1
logger.warning(f"Network error during download (attempt {retry_count}/{self.max_retries + 1}): {e}")
if retry_count <= self.max_retries:
# Calculate delay with exponential backoff
delay = self.base_delay * (2 ** (retry_count - 1))
logger.info(f"Retrying in {delay} seconds...")
await asyncio.sleep(delay)
# Update resume offset for next attempt
if allow_resume and os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Will resume from byte {resume_offset}")
# Refresh session to get new connection
await self._create_session()
continue
else:
logger.error(f"Max retries exceeded for download: {e}")
return False, f"Network error after {self.max_retries + 1} attempts: {str(e)}"
except Exception as e:
logger.error(f"Unexpected download error: {e}")
return False, str(e)
return False, f"Download failed after {self.max_retries + 1} attempts"
async def _dispatch_progress_callback(
self,
progress_callback: Callable[..., Awaitable[None]],
snapshot: DownloadProgress,
) -> None:
"""Invoke a progress callback while preserving backward compatibility."""
try:
result = progress_callback(snapshot, snapshot)
except TypeError:
result = progress_callback(snapshot.percent_complete)
if asyncio.iscoroutine(result):
await result
elif hasattr(result, "__await__"):
await result
async def download_to_memory(
self,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
return_headers: bool = False
) -> Tuple[bool, Union[bytes, str], Optional[Dict]]:
"""
Download a file to memory (for small files like preview images)
Args:
url: Download URL
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
return_headers: Whether to return response headers along with content
Returns:
Tuple[bool, Union[bytes, str], Optional[Dict]]: (success, content or error message, response headers if requested)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[download_to_memory] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[download_to_memory] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
async with session.get(url, headers=headers, proxy=self.proxy_url) as response:
if response.status == 200:
content = await response.read()
if return_headers:
return True, content, dict(response.headers)
else:
return True, content, None
elif response.status == 401:
error_msg = "Unauthorized access - invalid or missing API key"
return False, error_msg, None
elif response.status == 403:
error_msg = "Access forbidden"
return False, error_msg, None
elif response.status == 404:
error_msg = "File not found"
return False, error_msg, None
else:
error_msg = f"Download failed with status {response.status}"
return False, error_msg, None
except Exception as e:
logger.error(f"Error downloading to memory from {url}: {e}")
return False, str(e), None
async def get_response_headers(
self,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None
) -> Tuple[bool, Union[Dict, str]]:
"""
Get response headers without downloading the full content
Args:
url: URL to check
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
Returns:
Tuple[bool, Union[Dict, str]]: (success, headers dict or error message)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[get_response_headers] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[get_response_headers] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
async with session.head(url, headers=headers, proxy=self.proxy_url) as response:
if response.status == 200:
return True, dict(response.headers)
else:
return False, f"Head request failed with status {response.status}"
except Exception as e:
logger.error(f"Error getting headers from {url}: {e}")
return False, str(e)
async def make_request(
self,
method: str,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
**kwargs
) -> Tuple[bool, Union[Dict, str]]:
"""
Make a generic HTTP request and return JSON response
Args:
method: HTTP method (GET, POST, etc.)
url: Request URL
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
**kwargs: Additional arguments for aiohttp request
Returns:
Tuple[bool, Union[Dict, str]]: (success, response data or error message)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[make_request] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[make_request] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
# Add proxy to kwargs if not already present
if 'proxy' not in kwargs:
kwargs['proxy'] = self.proxy_url
async with session.request(method, url, headers=headers, **kwargs) as response:
if response.status == 200:
# Try to parse as JSON, fall back to text
try:
data = await response.json()
return True, data
except:
text = await response.text()
return True, text
elif response.status == 401:
return False, "Unauthorized access - invalid or missing API key"
elif response.status == 403:
return False, "Access forbidden"
elif response.status == 404:
return False, "Resource not found"
elif response.status == 429:
retry_after = self._extract_retry_after(response.headers)
error_msg = "Request rate limited"
logger.warning(
"Rate limit encountered for %s %s; retry_after=%s",
method,
url,
retry_after,
)
return False, RateLimitError(
error_msg,
retry_after=retry_after,
)
else:
return False, f"Request failed with status {response.status}"
except Exception as e:
logger.error(f"Error making {method} request to {url}: {e}")
return False, str(e)
async def close(self):
"""Close the HTTP session"""
if self._session is not None:
await self._session.close()
self._session = None
self._session_created_at = None
self._proxy_url = None
logger.debug("Closed HTTP session")
async def refresh_session(self):
"""Force refresh the HTTP session (useful when proxy settings change)"""
async with self._session_lock:
await self._create_session()
logger.info("HTTP session refreshed due to settings change")
@staticmethod
def _extract_retry_after(headers) -> Optional[float]:
"""Parse the Retry-After header into seconds."""
if not headers:
return None
header_value = headers.get("Retry-After")
if not header_value:
return None
header_value = header_value.strip()
if not header_value:
return None
if header_value.isdigit():
try:
seconds = float(header_value)
except ValueError:
return None
return max(0.0, seconds)
try:
retry_datetime = parsedate_to_datetime(header_value)
except (TypeError, ValueError):
return None
if retry_datetime.tzinfo is None:
return None
delta = retry_datetime - datetime.now(tz=retry_datetime.tzinfo)
return max(0.0, delta.total_seconds())
# Global instance accessor
async def get_downloader() -> Downloader:
"""Get the global downloader instance"""
return await Downloader.get_instance()

View File

@@ -1,24 +1,24 @@
import os
import logging
from typing import Dict, List, Optional
from typing import Dict
from .base_model_service import BaseModelService
from ..utils.models import EmbeddingMetadata
from ..config import config
from ..utils.routes_common import ModelRouteUtils
logger = logging.getLogger(__name__)
class EmbeddingService(BaseModelService):
"""Embedding-specific service implementation"""
def __init__(self, scanner):
def __init__(self, scanner, update_service=None):
"""Initialize Embedding service
Args:
scanner: Embedding scanner instance
update_service: Optional service for remote update tracking.
"""
super().__init__("embedding", scanner, EmbeddingMetadata)
super().__init__("embedding", scanner, EmbeddingMetadata, update_service=update_service)
async def format_response(self, embedding_data: Dict) -> Dict:
"""Format Embedding data for API response"""
@@ -34,12 +34,13 @@ class EmbeddingService(BaseModelService):
"file_size": embedding_data.get("size", 0),
"modified": embedding_data.get("modified", ""),
"tags": embedding_data.get("tags", []),
"modelDescription": embedding_data.get("modelDescription", ""),
"from_civitai": embedding_data.get("from_civitai", True),
# "usage_count": embedding_data.get("usage_count", 0), # TODO: Enable when embedding usage tracking is implemented
"notes": embedding_data.get("notes", ""),
"model_type": embedding_data.get("model_type", "embedding"),
"favorite": embedding_data.get("favorite", False),
"civitai": ModelRouteUtils.filter_civitai_data(embedding_data.get("civitai", {}))
"update_available": bool(embedding_data.get("update_available", False)),
"civitai": self.filter_civitai_data(embedding_data.get("civitai", {}), minimal=True)
}
def find_duplicate_hashes(self) -> Dict:

Some files were not shown because too many files have changed in this diff Show More