Commit Graph

207 Commits

Author SHA1 Message Date
Will Miao
0254c9d0e9 fix(showcase): tear down modal listeners 2025-11-27 18:00:59 +08:00
Will Miao
f8b9fa9b20 fix(civitai): improve metadata parsing for nested structures, see #700
- Refactor metadata detection to handle nested "meta" objects
- Add support for lowercase "lora:" hash keys
- Extract metadata from nested "meta" field when present
- Update tests to verify nested metadata parsing
- Handle case-insensitive LORA hash detection

The changes ensure proper parsing of Civitai image metadata that may be wrapped in nested structures, improving compatibility with different API response formats.
2025-11-26 13:46:08 +08:00
Will Miao
9198a23ba9 feat: normalize and validate checkpoint entries before enrichment
Add _normalize_checkpoint_entry method to handle legacy checkpoint data formats (strings, tuples) by converting them to dictionaries. This prevents errors during enrichment when checkpoint data is not in the expected dictionary format. Invalid checkpoint entries are now removed instead of causing processing failures.

- Update get_paginated_data and get_recipe_by_id methods to use normalization
- Add test cases for legacy string and tuple checkpoint formats
- Ensure backward compatibility with existing checkpoint handling
2025-11-21 23:36:32 +08:00
Will Miao
02bac7edfb feat: normalize and validate checkpoint entries in recipes
Add _normalize_checkpoint_entry method to handle legacy and malformed checkpoint data by:
- Converting string entries to structured dict format
- Handling single-element lists/tuples recursively
- Dropping invalid entries with appropriate warnings
- Maintaining backward compatibility while improving data consistency

Add test case to verify string checkpoint conversion works correctly.
2025-11-21 23:00:02 +08:00
Will Miao
ea1d1a49c9 feat: enhance search with include/exclude tokens and improved sorting
- Add token parsing to support include/exclude search terms using "-" prefix
- Implement token-based matching logic for relative path searches
- Improve search result sorting by prioritizing prefix matches and match position
- Add frontend test for multi-token highlighting with exclusion support
2025-11-21 19:48:43 +08:00
Will Miao
9a789f8f08 feat: add checkpoint hash filtering and navigation
- Add checkpoint hash parameter parsing to backend routes
- Implement checkpoint hash filtering in frontend API client
- Add click navigation from recipe modal to checkpoints page
- Update checkpoint items to use pointer cursor for better UX

Checkpoint items in recipe modal are now clickable and will navigate to the checkpoints page with appropriate hash filtering applied. This improves user workflow when wanting to view checkpoint details from recipes.
2025-11-21 16:17:01 +08:00
Will Miao
4eb46a8d3e feat: consolidate checkpoint metadata handling
- Extract checkpoint entry from multiple metadata locations using helper method
- Sanitize checkpoint metadata by removing transient/local-only fields
- Remove checkpoint duplication from generation parameters to store only at top level
- Update frontend to properly populate checkpoint metadata during import
- Add tests for new checkpoint handling functionality

This ensures consistent checkpoint metadata structure and prevents data duplication across different storage locations.
2025-11-21 14:55:45 +08:00
Will Miao
36f28b3c65 feat: normalize LoRA preview URLs for browser accessibility
Add _normalize_preview_url method to ensure preview URLs are properly formatted for browser access. The method handles absolute paths by converting them to static URLs via config.get_preview_static_url, while preserving API paths and other valid URLs. This ensures consistent preview image display across different URL formats.

Update _enrich_lora_entry to apply URL normalization to preview URLs obtained from both hash-based lookups and version entries. Add comprehensive test coverage for absolute path normalization scenarios.
2025-11-21 12:31:23 +08:00
Will Miao
2452cc4df1 feat(recipes): resolve base model from checkpoint metadata
Add metadata service integration to automatically resolve base model information from checkpoint metadata during recipe import. This replaces the previous approach of relying solely on request parameters and provides more accurate base model information.

- Add _resolve_base_model_from_checkpoint method to fetch base model from metadata provider
- Update recipe import logic to use resolved base model when available
- Add comprehensive tests for base model resolution with fallback behavior
- Remove debug print statement from import parameters
2025-11-21 12:12:27 +08:00
Will Miao
e24621a0af feat(recipe-scanner): add version index fallback for LoRA enrichment
Add _get_lora_from_version_index method to fetch cached LoRA entries by modelVersionId when hash is unavailable. This improves LoRA enrichment by using version index as fallback when hash is missing, ensuring proper library status, file paths, and preview URLs are set even without hash values.

Update test suite to include version_index in stub cache and add test coverage for version-based lookup functionality.
2025-11-21 11:27:09 +08:00
Will Miao
7173a2b9d6 feat: add remote recipe import functionality
Add support for importing recipes from remote sources by:
- Adding import_remote_recipe endpoint to RecipeHandlerSet
- Injecting downloader_factory and civitai_client_getter dependencies
- Implementing image download and resource parsing logic
- Supporting Civitai resource payloads with checkpoints and LoRAs
- Adding required imports for regex and temporary file handling

This enables users to import recipes directly from external sources like Civitai without manual file downloads.
2025-11-21 11:12:58 +08:00
Will Miao
9952721e76 feat(downloads): support safetensors zips and previews 2025-11-20 19:41:31 +08:00
Will Miao
c533a8e7bf feat: enhance Civitai metadata handling and image URL processing
- Import rewrite_preview_url utility for optimized image URL handling
- Update thumbnail URL processing for both LoRA and checkpoint entries to use rewritten URLs
- Expand checkpoint metadata with modelId, file size, SHA256 hash, and file name
- Improve error handling and data validation for Civitai API responses
- Maintain backward compatibility with existing data structures
2025-11-20 16:31:48 +08:00
pixelpaws
07721af87c feat(settings): add auto-organize exclusions 2025-11-20 16:08:32 +08:00
Will Miao
3849f7eef9 feat(metadata): improve civitai deletion detection logic
- Track when Civitai API returns "Model not found" for default provider
- Use dedicated flag instead of error string comparison for deletion detection
- Ensure archive-sourced models don't get marked as deleted
- Add test coverage for archive source deletion flag behavior
- Fix deletion flag logic to properly handle provider fallback scenarios
2025-11-19 19:16:40 +08:00
Will Miao
51b5261f40 fix(model): align rename extension detection 2025-11-19 11:20:09 +08:00
Will Miao
963f6b1383 fix(model): preserve original extension on rename 2025-11-19 11:08:08 +08:00
Will Miao
b75baa1d1a fix: support GGUF model deletion in model lifecycle service
- Add optional main_extension parameter to delete_model_artifacts function
- Extract file extension from model filename to handle different file types
- Update model scanner to pass file extension when deleting models
- Add test case for GGUF file deletion to ensure proper cleanup
- Maintain backward compatibility with existing safetensors models

This change allows the model lifecycle service to properly delete GGUF model files along with their associated metadata and preview files, expanding support beyond just safetensors format.
2025-11-19 10:36:03 +08:00
Will Miao
1427dc8e38 feat(model-lifecycle): integrate model update service for deletion sync
Add ModelUpdateService dependency to ModelLifecycleService to enable synchronization during model deletion. The service is now passed through BaseModelRoutes initialization and used in delete_model to trigger updates when a model is removed. This ensures external systems stay in sync with local model state changes.

Key changes:
- Inject update_service into ModelLifecycleService constructor
- Extract model ID from metadata during deletion
- Call update service sync method after successful deletion
- Add proper type hints and TYPE_CHECKING imports
2025-11-18 21:02:39 +08:00
Will Miao
e9d55fe146 feat(filters): add model type filter 2025-11-18 16:43:44 +08:00
Will Miao
57f369a6de feat(model): add model type filtering support
- Add model_types parameter to ModelListingHandler to support filtering by model type
- Implement get_model_types endpoint in ModelQueryHandler to retrieve available model types
- Register new /api/lm/{prefix}/model-types route for model type queries
- Extend BaseModelService to handle model type filtering in queries
- Support both model_type and civitai_model_type query parameters for backward compatibility

This enables users to filter models by specific types, improving model discovery and organization capabilities.
2025-11-18 15:36:01 +08:00
Will Miao
831a9da9d7 feat: update version badge logic for same-base update strategy, see #676
- Remove unused isNewer variable calculation
- Use dividerThresholdVersionId instead of latestLibraryVersionId for badge logic
- Add test case to verify newer version badge appears with same-base strategy
- Ensures correct badge display when filtering by same base model versions
2025-11-18 11:18:32 +08:00
Will Miao
655157434e feat(versions): add base filter toggle UI and styling
Add CSS classes and JavaScript logic for the base filter toggle button in the versions toolbar. The filter allows users to switch between showing all versions or only versions matching the current base model. Includes styling for different states (active, hover, disabled) and accessibility features like screen reader support.
2025-11-18 06:47:07 +08:00
Will Miao
0e73db0669 feat: implement same_base update strategy for model annotations
Add support for configurable update flag strategy with new "same_base" mode that considers base model versions when determining update availability. The strategy is controlled by the "update_flag_strategy" setting.

When strategy is set to "same_base":
- Uses get_records_bulk instead of has_updates_bulk
- Compares model versions against highest local versions per base model
- Provides more granular update detection based on base model relationships

Fallback to existing bulk or individual update checks when:
- Strategy is not "same_base"
- Bulk operations fail
- Records are unavailable

This enables more precise update flagging for models sharing common bases.
2025-11-17 19:26:41 +08:00
Will Miao
354cf03bbc fix(settings): sync portable mode toggle 2025-11-16 17:36:52 +08:00
pixelpaws
cc8e1c5049 fix(recipes): allow widget save without lora matches 2025-11-13 15:52:31 +08:00
Will Miao
29bb85359e feat(context-menu): refresh missing license metadata 2025-11-11 14:24:59 +08:00
Will Miao
4dd8ce778e feat(trigger): add optional strength adjustment for trigger words
Add `allow_strength_adjustment` parameter to enable mouse wheel adjustment of trigger word strengths. When enabled, strength values are preserved and can be modified interactively. Also improves trigger word parsing by handling whitespace more consistently and adding debug logging for trigger data inspection.
2025-11-09 22:24:23 +08:00
Will Miao
c09100c22e feat: implement tag filtering with include/exclude states
- Update frontend tag filter to cycle through include/exclude/clear states
- Add backend support for tag_include and tag_exclude query parameters
- Maintain backward compatibility with legacy tag parameter
- Store tag states as dictionary with 'include'/'exclude' values
- Update test matrix documentation to reflect new tag behavior

The changes enable more granular tag filtering where users can now explicitly include or exclude specific tags, rather than just adding tags to a simple inclusion list. This provides better control over search results and improves the filtering user experience.
2025-11-08 11:45:31 +08:00
Will Miao
3b842355c2 feat: add license-based filtering for model listings
Add support for filtering models by license requirements:
- credit_required: filter models that require credits or allow free use
- allow_selling_generated_content: filter models based on commercial usage rights

These filters use license_flags bitmask to determine model permissions and enable users to find models that match their specific usage requirements and budget constraints.
2025-11-07 22:28:29 +08:00
Will Miao
f76343f389 feat(lora): add mode change listeners to update trigger words
Add property descriptor to listen for mode changes in Lora Loader and Lora Stacker nodes. When node mode changes, automatically update connected trigger word toggle nodes and downstream loader nodes to maintain synchronization between node modes and trigger word states.

- Lora Loader: Updates connected trigger words when mode changes
- Lora Stacker: Updates connected trigger words and downstream loaders when mode changes
- Both nodes log mode changes for debugging purposes
2025-11-07 15:11:59 +08:00
Will Miao
1bb5d0b072 feat(metadata): add rate limit retry support to metadata providers
Add RateLimitRetryingProvider and _RateLimitRetryHelper classes to handle rate limiting with exponential backoff retries. Update get_metadata_provider function to automatically wrap providers with rate limit handling. This improves reliability when external APIs return rate limit errors by implementing automatic retries with configurable delays and jitter.
2025-11-07 09:18:59 +08:00
Will Miao
ba6e2eadba feat: update license flag handling and default permissions
- Update DEFAULT_LICENSE_FLAGS from 57 to 127 to enable all commercial modes by default
- Replace CommercialUseLevel enum with bitwise commercial permission handling
- Simplify commercial value normalization and validation using allowed values set
- Adjust bit shifting in license flag construction to accommodate new commercial bits structure
- Remove CommercialUseLevel from exports and update tests accordingly
- Improve handling of empty commercial use values with proper type checking

The changes streamline commercial permission processing and align with CivitAI's default license configuration while maintaining backward compatibility.
2025-11-06 22:14:36 +08:00
Will Miao
ddf9e33961 feat: add license information handling for Civitai models
Add license resolution utilities and integrate license information into model metadata processing. The changes include:

- Add `resolve_license_payload` function to extract license data from Civitai model responses
- Integrate license information into model metadata in CivitaiClient and MetadataSyncService
- Add license flags support in model scanning and caching
- Implement CommercialUseLevel enum for standardized license classification
- Update model scanner to handle unknown fields when extracting metadata values

This ensures proper license attribution and compliance when working with Civitai models.
2025-11-06 17:05:54 +08:00
Will Miao
3d6bb432c4 feat: normalize tags to lowercase for Windows compatibility, see #637
Convert all tags to lowercase in tag processing logic to prevent case sensitivity issues on Windows filesystems. This ensures consistent tag matching and prevents duplicate tags with different cases from being created.

Changes include:
- TagUpdateService now converts tags to lowercase before comparison
- Utils function converts model tags to lowercase before priority resolution
- Test cases updated to reflect lowercase tag expectations
2025-11-04 12:54:09 +08:00
pixelpaws
6dea9a76bc feat(updates): improve check updates confirmation 2025-11-03 22:36:57 +08:00
Will Miao
d73903e82e feat: prevent duplicate banner entries in recent history
Add duplicate detection to banner recording to prevent multiple entries
for the same banner ID in recent history. This prevents duplicate history
entries when pages refresh or banners are shown multiple times.

- Check if banner ID already exists in recentHistory before adding
- Return early if duplicate found to prevent adding same banner multiple times
- Add comprehensive tests for banner history functionality including:
  - Adding new banners to history
  - Preventing duplicate entries
  - Handling multiple different banners
- Clear history between tests to ensure test isolation
2025-11-03 20:13:48 +08:00
Will Miao
4862419b61 feat: refactor banner service and add comprehensive tests
- Remove legacy community support banner tracking variables and logic
- Simplify banner dismissal handling by checking dismissal state before marking
- Replace timer-based community support banner with immediate registration
- Clean up unused constants and legacy storage keys
- Add comprehensive test suite with mocked dependencies
- Improve code maintainability and test coverage
2025-11-03 19:50:35 +08:00
Will Miao
30f9e3e2ec feat(loras): add drag event callbacks and preview suppression
- Add onDragStart and onDragEnd callbacks to initDrag function
- Implement preview suppression during and briefly after strength dragging
- Clear preview timer on drag start/end to prevent tooltip conflicts
- Update tests to verify drag callbacks are properly triggered

This prevents tooltip previews from interfering with drag interactions and provides better control over drag lifecycle events.
2025-11-03 12:18:59 +08:00
Will Miao
707d0cb8a4 feat(lora-loader): add trigger word update on LoRA syntax edits
Add test coverage for trigger word refresh functionality when LoRA syntax is edited in the input widget. The test verifies that after modifying LoRA syntax in the input field, the connected trigger words are properly updated to reflect the active LoRAs.

Additionally, implement the actual trigger word update logic in lora_loader.js by calling updateConnectedTriggerWords after merging LoRAs, ensuring the UI stays synchronized with the current LoRA state.
2025-11-03 12:03:21 +08:00
pixelpaws
94e0308a12 feat(settings): allow hiding folder sidebar 2025-11-03 06:39:13 +08:00
pixelpaws
c2db5eb6df Merge pull request #634 from willmiao/codex/refactor-config-to-handle-default-library
fix: drop template default library when saving ComfyUI paths
2025-11-02 09:12:26 +08:00
pixelpaws
ef0bcc6cf1 fix(config): remove template default library before saving paths 2025-11-02 09:09:36 +08:00
pixelpaws
285428ad3a fix(portable): use project root for settings storage 2025-11-02 09:07:57 +08:00
pixelpaws
1be3235564 fix(model-updates): support per-model version ids 2025-10-30 23:15:23 +08:00
pixelpaws
ce42d83ce9 feat(settings): add portable settings toggle 2025-10-30 14:08:21 +08:00
Will Miao
b99d78bda6 feat(nodes): enhance LoRA loading with path support and add tests
- Allow direct file paths in addition to registered LoRA names
- Add graceful handling for missing LoRA files with warning logs
- Add comprehensive unit tests for missing LoRA file handling
- Ensure backward compatibility with existing LoRA loading behavior
2025-10-29 22:39:08 +08:00
Will Miao
39586f4a20 feat: add LoRA syntax utilities and comprehensive test suite, fixes #600
- Implement core LoRA syntax manipulation functions including:
  - applyLoraValuesToText for updating LoRA strengths and clip values
  - normalizeStrengthValue for consistent numeric formatting
  - shouldIncludeClipStrength for clip strength inclusion logic
  - cleanupLoraSyntax for text normalization
  - debounce utility for input handling

- Add comprehensive test suite covering all utility functions
- Include edge cases for clip strength handling, numeric formatting, and syntax cleanup
- Support both basic and expanded LoRA syntax formats with proper value preservation
- Enable debounced input synchronization for better performance

The utilities provide robust handling of LoRA syntax patterns while maintaining compatibility with existing ComfyUI workflows.
2025-10-29 22:13:54 +08:00
Will Miao
e10717dcda feat(ui): improve update controls styling and error handling
- Add disabled and loading states for control group buttons with appropriate cursor and opacity styling
- Enhance dropdown toggle active state styling for update filter group
- Improve dropdown toggle layout with flex centering
- Add disabled state styling for dropdown items
- Refactor model update check to use shared helper function, removing redundant success handling and simplifying error flow
- Maintain existing functionality while improving user experience and code maintainability
2025-10-29 14:38:11 +08:00
Will Miao
cf4d654c4b feat: improve file size extraction logic with primary model preference
Refactor `_extract_size_bytes` method to prioritize primary model files when calculating size. The new implementation:
- Extracts size parsing into separate `parse_size` function
- Adds logic to prefer files marked as both "model" type and "primary"
- Falls back to first valid size if no primary model file found
- Adds comprehensive tests for primary preference and fallback behavior

This ensures more accurate size reporting for model files, particularly when multiple file types are present in the response.
2025-10-29 09:16:29 +08:00