Commit Graph

71 Commits

Author SHA1 Message Date
Will Miao
dad549f65f feat(download): auto-route diffusion models to unet folder based on baseModel, see #770
CivitAI does not distinguish between checkpoint and diffusion model types -
both are labeled as "checkpoint". For certain base model types like
"ZImageTurbo", all models are actually diffusion models and should be
saved to the unet/diffusion model folder instead of the checkpoint folder.

- Add DIFFUSION_MODEL_BASE_MODELS constant for known diffusion model types
- Add default_unet_root setting with auto-set logic
- Route downloads to unet folder when baseModel matches known diffusion types

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-18 11:58:20 +08:00
Will Miao
ad7d372887 fix: use CivArchive provider when source is 'civarchive' (#769)
When users paste CivArchive URLs, the system now fetches metadata from
CivArchive API first instead of Civitai. This prevents download failures
when a model has been deleted from Civitai but remains available on
CivArchive with alternative mirrors.

Changes:
- Source-aware metadata fetching: Uses CivArchive API when source='civarchive'
- URL prioritization: Prefers non-Civitai mirrors for CivArchive downloads
- Fallback mechanism: Falls back to default provider if CivArchive fails

Fixes #769
2026-01-16 10:57:22 +08:00
Will Miao
f199c9b591 feat: propagate version info to model update record creation. Fixes #756
- Pass `version_info` parameter through download manager to model update service
- Enhance `_create_record` to use version info when creating records for missing versions
- Add `_extract_single_version` helper method for consistent version extraction
- Improve handling of version metadata during library synchronization
2026-01-06 08:46:55 +08:00
Will Miao
5000478991 feat(download): support multiple model file extensions in archive extraction
- Add `_get_supported_extensions_for_type` method to return allowed extensions per model type
- Rename `_extract_safetensors_from_archive` to `_extract_model_files_from_archive` and extend to filter by allowed extensions
- Update error message to list supported extensions when archive contains no valid files
- Add test for extracting .pt embedding files from zip archives
2025-12-07 09:00:47 +08:00
Will Miao
9952721e76 feat(downloads): support safetensors zips and previews 2025-11-20 19:41:31 +08:00
Will Miao
059ebeead7 feat: include Negative file type in primary file selection for embeddings 2025-11-18 14:16:22 +08:00
Will Miao
188fe407b6 feat(download): sync downloaded versions with update tracking
Add automatic synchronization of downloaded model versions with the update tracking system. After a successful download, the system now resolves model and version IDs from the download response and updates the update service with the newly downloaded version along with any existing local versions.

This ensures that:
- Update tracking accurately reflects which versions are available locally
- The system properly tracks both newly downloaded and existing versions
- Failed sync operations are gracefully handled with appropriate logging
- Support is included for LoRA, checkpoint, and embedding model types
2025-10-26 10:42:09 +08:00
pixelpaws
807425f12a feat(preview): respect blur mature content setting 2025-10-25 06:43:03 +08:00
pixelpaws
faa26651dd fix(download): recover stalled transfers automatically 2025-10-23 17:25:38 +08:00
pixelpaws
8d336320c0 fix(scanner): apply metadata adjustments during reconciliation 2025-10-23 07:34:35 +08:00
pixelpaws
f7a856349a fix(utils): sanitize path template folder names 2025-10-18 16:20:47 +08:00
Will Miao
10f5588e4a feat: add model_name and version_name placeholders to download paths, #552
Add support for {model_name} and {version_name} placeholders in download path templates. These new placeholders allow for more flexible and descriptive file organization by including the actual model name and version name in the download directory structure.

Changes include:
- Updated download_manager.py and utils.py to handle new placeholders
- Added placeholders to constants.js for UI reference
- Updated settings modal template to show available placeholders
- Added comprehensive tests to verify placeholder functionality

This enhancement provides users with more control over how downloaded models are organized on their file system.
2025-10-17 16:01:06 +08:00
pixelpaws
055c1ca0d4 test(downloads): cover pause and resume flows 2025-10-13 21:30:23 +08:00
Will Miao
27370df93a feat(download): add support to download models from civarchive, fixes #381 2025-10-13 19:27:56 +08:00
pixelpaws
eb76468280 feat(downloads): expose throughput metrics in progress APIs 2025-10-13 14:39:31 +08:00
Will Miao
b77c596f3a Fix error message to improve clarity in DownloadManager 2025-10-12 05:43:59 +08:00
pixelpaws
7be1a2bd65 Merge pull request #557 from willmiao/civarc-api-support
CivArchive API support
2025-10-11 18:10:06 +08:00
pixelpaws
6120922204 chore(priority-tags): add newline terminator 2025-10-11 17:38:20 +08:00
Will Miao
1f60160e8b feat(civarchive_client): enhance request handling and context parsing
Introduce `_request_json` method for async JSON requests and improved error handling. Add static methods `_normalize_payload`, `_split_context`, `_ensure_list`, and `_build_model_info` to parse and normalize API responses. These changes improve the robustness of the CivArchiveClient by ensuring consistent data structures and handling potential API response issues gracefully.
2025-10-11 13:07:29 +08:00
Will Miao
f542ade628 feat(civitai): implement URL rewriting for Civitai previews and enhance download handling, fixes #499 2025-10-09 17:54:37 +08:00
pixelpaws
c53fa8692b refactor(settings): lazily initialize manager 2025-10-08 08:56:57 +08:00
pixelpaws
9121c12a2c feat(download): retry mirror urls sequentially 2025-09-30 17:14:59 +08:00
pixelpaws
8e4927600f fix(embeddings): replace spaces in relative paths 2025-09-26 09:02:46 +08:00
Will Miao
86074c87d7 refactor(downloader): update download_to_memory calls to include response headers 2025-09-15 19:24:09 +08:00
Will Miao
125fdecd61 fix: handle missing download URL for primary file in metadata 2025-09-13 09:03:34 +08:00
Will Miao
d05076d258 feat: add CivArchive metadata provider and support for optional source parameter in downloads 2025-09-12 21:13:15 +08:00
Will Miao
6fd74952b7 Refactor metadata handling to use unified provider system
- Replaced direct usage of Civitai client with a fallback metadata provider across all recipe parsers.
- Updated metadata service to improve initialization and error handling.
- Enhanced download manager to utilize a downloader service for file operations.
- Improved recipe scanner to fetch model information through the new metadata provider.
- Updated utility functions to streamline image downloading and processing.
- Added comprehensive logging and error handling for better debugging and reliability.
- Introduced `get_default_metadata_provider()` for simplified access to the default provider.
- Ensured backward compatibility with existing APIs and workflows.
2025-09-09 20:57:45 +08:00
Will Miao
9ba3e2c204 feat(metadata): implement metadata providers and initialize metadata service
- Added ModelMetadataProvider and CivitaiModelMetadataProvider for handling model metadata.
- Introduced SQLiteModelMetadataProvider for SQLite database integration.
- Created metadata_service.py to initialize and configure metadata providers.
- Updated CivitaiClient to register as a metadata provider.
- Refactored download_manager to use the new download_file method.
- Added SQL schema for models, model_versions, and model_files.
- Updated requirements.txt to include aiosqlite.
2025-09-08 22:41:17 +08:00
Will Miao
159193ef43 feat: Implement unique filename generation with conflict resolution using metadata hash 2025-08-25 15:33:46 +08:00
Will Miao
623c28bfc3 feat: Remove backup creation from metadata saving functions for streamlined operations 2025-08-24 22:30:53 +08:00
Will Miao
141f72963a fix: Enhance download functionality with resumable downloads and improved error handling 2025-08-20 16:40:22 +08:00
Will Miao
2356662492 fix: Improve author retrieval logic in DownloadManager to handle non-dictionary creator data 2025-08-16 21:10:57 +08:00
Will Miao
1624a45093 fix: Update author retrieval to handle missing username gracefully in DownloadManager and utils 2025-08-16 16:11:56 +08:00
Will Miao
b8aa7184bd feat: update download path template handling for model types and migrate old settings 2025-08-13 19:23:37 +08:00
Will Miao
96517cbdef fix: update model_id and model_version_id handling across various services for improved flexibility 2025-08-11 15:31:49 +08:00
Will Miao
3cf9121a8c refactor: enhance scanner handling and add embedding support in download manager and misc routes 2025-07-25 23:59:27 +08:00
Will Miao
de06c6b2f6 feat: Add download cancellation and tracking features in DownloadManager and API routes 2025-07-21 15:38:20 +08:00
Will Miao
124002a472 feat: Add JSON parsing for base_model_path_mappings and refactor path handling in DownloadManager 2025-07-21 07:37:34 +08:00
Will Miao
f749dd0d52 feat: Add YAML configuration for path mappings to customize model download directories 2025-07-16 17:07:13 +08:00
Will Miao
7e0df4d718 feat: Add Civitai model tags for prioritized subfolder organization in download manager 2025-07-12 10:32:15 +08:00
Will Miao
79011bd257 refactor: Update model_id and model_version_id types to integers and add validation in routes 2025-07-09 14:21:49 +08:00
Will Miao
c692713ffb refactor: Simplify model version existence checks and enhance version retrieval methods in scanners 2025-07-09 10:26:03 +08:00
Will Miao
0653f59473 feat: Enhance relative path handling in download manager to include base model 2025-07-03 10:28:52 +08:00
Will Miao
7a4b5a4667 feat: Implement download progress WebSocket and enhance download manager with unique IDs 2025-07-02 23:48:35 +08:00
Will Miao
ab22d16bad feat: Rename download endpoint from /api/download-lora to /api/download-model and update related logic 2025-07-02 19:21:25 +08:00
Will Miao
d7cb546c5f refactor: Simplify model download handling by consolidating download logic and updating parameter usage 2025-07-02 18:25:42 +08:00
Will Miao
2ee057e19b feat: update metadata saving to ensure backup creation and support nested civitai structure 2025-06-25 11:50:10 +08:00
Will Miao
afa5a42f5a Refactor metadata handling by introducing MetadataManager for centralized operations and improving error handling 2025-06-17 21:01:48 +08:00
Will Miao
a7207084ef Remove unused monitor cleanup logic from LoraManager and DownloadManager 2025-06-13 05:52:52 +08:00
Will Miao
68fdb14cd6 Remove unused lora monitor retrieval and ignore path logic from ApiRoutes, DownloadManager, and ModelScanner. Fixes #226 2025-06-13 05:46:22 +08:00