When users paste CivArchive URLs, the system now fetches metadata from
CivArchive API first instead of Civitai. This prevents download failures
when a model has been deleted from Civitai but remains available on
CivArchive with alternative mirrors.
Changes:
- Source-aware metadata fetching: Uses CivArchive API when source='civarchive'
- URL prioritization: Prefers non-Civitai mirrors for CivArchive downloads
- Fallback mechanism: Falls back to default provider if CivArchive fails
Fixes#769
- Add folder_include parameter support in backend API handlers
- Add folder_include to FilterCriteria and implement multi-folder filtering logic
- Update frontend to send all include folders instead of only the first
- Add tests for single/multiple include folders, include with exclude, and non-recursive filtering
- Add blank line after module docstring for better PEP 8 compliance
- Reformat long lines to adhere to 88-character limit using Black-style formatting
- Improve string consistency by using double quotes consistently
- Enhance readability of complex list comprehensions and method calls
- Maintain all existing functionality while improving code structure
- Add `_preprocess_loras_input` method to handle different widget input formats
- Move core randomization logic to `LoraService` for better separation of concerns
- Update `_select_loras` method to use new service-based approach
- Add comprehensive test fixtures for license filtering scenarios
- Include debug print statement for pool config inspection during development
This refactor improves code organization by centralizing business logic in the service layer while maintaining backward compatibility with existing widget inputs.
- Update `is_civitai_api_metadata` to exclude both "archive_db" and "civarchive" sources
- Skip Civitai metadata updates when existing metadata is higher quality than incoming archive data
- Add test to verify API metadata is preserved when CivArchive provides lower-quality data
- Pass `version_info` parameter through download manager to model update service
- Enhance `_create_record` to use version info when creating records for missing versions
- Add `_extract_single_version` helper method for consistent version extraction
- Improve handling of version metadata during library synchronization
- Add GET /api/lm/recipes/roots endpoint to retrieve recipe root directories
- Add POST /api/lm/recipe/move endpoint to move recipes between directories
- Register new endpoints in route definitions
- Implement error handling for both new endpoints with proper status codes
- Enable recipe management operations for better file organization
Removed the forced normalization of path separators to forward slashes in BaseModelService to maintain platform-specific separators. Updated test cases to use os.sep for constructing expected paths, ensuring tests work correctly across different operating systems while preserving native path representations.
- Add `_get_supported_extensions_for_type` method to return allowed extensions per model type
- Rename `_extract_safetensors_from_archive` to `_extract_model_files_from_archive` and extend to filter by allowed extensions
- Update error message to list supported extensions when archive contains no valid files
- Add test for extracting .pt embedding files from zip archives
Add comprehensive local file matching for LoRA entries in recipe metadata:
- Add modelVersionId-based lookup via new _get_lora_from_version_index method
- Extend LoRA entry with additional fields: existsLocally, inLibrary, localPath, thumbnailUrl, size
- Improve local file detection by checking both SHA256 hash and modelVersionId
- Set default thumbnail URL and size values for missing LoRA files
- Add proper typing with Optional imports for better code clarity
This provides more accurate local file status and metadata for LoRA entries in recipes.
- Add MODEL_NAME_PATTERN regex to extract model names from parameters
- Extract model hash from parsed hashes when available in metadata
- Add checkpoint model hash and name extraction from parameters section
- Implement checkpoint resource processing from Civitai metadata
- Improve model information completeness for better recipe tracking
- Refactor metadata detection to handle nested "meta" objects
- Add support for lowercase "lora:" hash keys
- Extract metadata from nested "meta" field when present
- Update tests to verify nested metadata parsing
- Handle case-insensitive LORA hash detection
The changes ensure proper parsing of Civitai image metadata that may be wrapped in nested structures, improving compatibility with different API response formats.
Add _normalize_checkpoint_entry method to handle legacy checkpoint data formats (strings, tuples) by converting them to dictionaries. This prevents errors during enrichment when checkpoint data is not in the expected dictionary format. Invalid checkpoint entries are now removed instead of causing processing failures.
- Update get_paginated_data and get_recipe_by_id methods to use normalization
- Add test cases for legacy string and tuple checkpoint formats
- Ensure backward compatibility with existing checkpoint handling
Add _normalize_checkpoint_entry method to handle legacy and malformed checkpoint data by:
- Converting string entries to structured dict format
- Handling single-element lists/tuples recursively
- Dropping invalid entries with appropriate warnings
- Maintaining backward compatibility while improving data consistency
Add test case to verify string checkpoint conversion works correctly.
- Add token parsing to support include/exclude search terms using "-" prefix
- Implement token-based matching logic for relative path searches
- Improve search result sorting by prioritizing prefix matches and match position
- Add frontend test for multi-token highlighting with exclusion support
- Extract checkpoint entry from multiple metadata locations using helper method
- Sanitize checkpoint metadata by removing transient/local-only fields
- Remove checkpoint duplication from generation parameters to store only at top level
- Update frontend to properly populate checkpoint metadata during import
- Add tests for new checkpoint handling functionality
This ensures consistent checkpoint metadata structure and prevents data duplication across different storage locations.
Add _normalize_preview_url method to ensure preview URLs are properly formatted for browser access. The method handles absolute paths by converting them to static URLs via config.get_preview_static_url, while preserving API paths and other valid URLs. This ensures consistent preview image display across different URL formats.
Update _enrich_lora_entry to apply URL normalization to preview URLs obtained from both hash-based lookups and version entries. Add comprehensive test coverage for absolute path normalization scenarios.
Add _get_lora_from_version_index method to fetch cached LoRA entries by modelVersionId when hash is unavailable. This improves LoRA enrichment by using version index as fallback when hash is missing, ensuring proper library status, file paths, and preview URLs are set even without hash values.
Update test suite to include version_index in stub cache and add test coverage for version-based lookup functionality.
Add support for importing recipes from remote sources by:
- Adding import_remote_recipe endpoint to RecipeHandlerSet
- Injecting downloader_factory and civitai_client_getter dependencies
- Implementing image download and resource parsing logic
- Supporting Civitai resource payloads with checkpoints and LoRAs
- Adding required imports for regex and temporary file handling
This enables users to import recipes directly from external sources like Civitai without manual file downloads.
- Import rewrite_preview_url utility for optimized image URL handling
- Update thumbnail URL processing for both LoRA and checkpoint entries to use rewritten URLs
- Expand checkpoint metadata with modelId, file size, SHA256 hash, and file name
- Improve error handling and data validation for Civitai API responses
- Maintain backward compatibility with existing data structures
- Track when Civitai API returns "Model not found" for default provider
- Use dedicated flag instead of error string comparison for deletion detection
- Ensure archive-sourced models don't get marked as deleted
- Add test coverage for archive source deletion flag behavior
- Fix deletion flag logic to properly handle provider fallback scenarios
- Add optional main_extension parameter to delete_model_artifacts function
- Extract file extension from model filename to handle different file types
- Update model scanner to pass file extension when deleting models
- Add test case for GGUF file deletion to ensure proper cleanup
- Maintain backward compatibility with existing safetensors models
This change allows the model lifecycle service to properly delete GGUF model files along with their associated metadata and preview files, expanding support beyond just safetensors format.
Add ModelUpdateService dependency to ModelLifecycleService to enable synchronization during model deletion. The service is now passed through BaseModelRoutes initialization and used in delete_model to trigger updates when a model is removed. This ensures external systems stay in sync with local model state changes.
Key changes:
- Inject update_service into ModelLifecycleService constructor
- Extract model ID from metadata during deletion
- Call update service sync method after successful deletion
- Add proper type hints and TYPE_CHECKING imports
- Add model_types parameter to ModelListingHandler to support filtering by model type
- Implement get_model_types endpoint in ModelQueryHandler to retrieve available model types
- Register new /api/lm/{prefix}/model-types route for model type queries
- Extend BaseModelService to handle model type filtering in queries
- Support both model_type and civitai_model_type query parameters for backward compatibility
This enables users to filter models by specific types, improving model discovery and organization capabilities.
Add support for configurable update flag strategy with new "same_base" mode that considers base model versions when determining update availability. The strategy is controlled by the "update_flag_strategy" setting.
When strategy is set to "same_base":
- Uses get_records_bulk instead of has_updates_bulk
- Compares model versions against highest local versions per base model
- Provides more granular update detection based on base model relationships
Fallback to existing bulk or individual update checks when:
- Strategy is not "same_base"
- Bulk operations fail
- Records are unavailable
This enables more precise update flagging for models sharing common bases.
- Update frontend tag filter to cycle through include/exclude/clear states
- Add backend support for tag_include and tag_exclude query parameters
- Maintain backward compatibility with legacy tag parameter
- Store tag states as dictionary with 'include'/'exclude' values
- Update test matrix documentation to reflect new tag behavior
The changes enable more granular tag filtering where users can now explicitly include or exclude specific tags, rather than just adding tags to a simple inclusion list. This provides better control over search results and improves the filtering user experience.
Add support for filtering models by license requirements:
- credit_required: filter models that require credits or allow free use
- allow_selling_generated_content: filter models based on commercial usage rights
These filters use license_flags bitmask to determine model permissions and enable users to find models that match their specific usage requirements and budget constraints.
