Commit Graph

315 Commits

Author SHA1 Message Date
pixelpaws
38e766484e fix(settings): restrict minimal persistence keys 2025-10-26 19:53:43 +08:00
Will Miao
b5ee4a6408 feat(settings): enhance settings handling and add startup messages, fixes #593 and fixes #594
- Add standalone mode detection via LORA_MANAGER_STANDALONE environment variable
- Improve error handling for settings file loading with specific JSON decode errors
- Add startup messages system to communicate configuration warnings and errors to users
- Include settings file path and startup messages in settings API response
- Automatically save settings when bootstrapping from defaults due to missing/invalid settings file
- Add configuration warnings collection for environment variables and other settings issues

The changes improve robustness of settings management and provide better user feedback when configuration issues occur.
2025-10-26 18:07:00 +08:00
Will Miao
188fe407b6 feat(download): sync downloaded versions with update tracking
Add automatic synchronization of downloaded model versions with the update tracking system. After a successful download, the system now resolves model and version IDs from the download response and updates the update service with the newly downloaded version along with any existing local versions.

This ensures that:
- Update tracking accurately reflects which versions are available locally
- The system properly tracks both newly downloaded and existing versions
- Failed sync operations are gracefully handled with appropriate logging
- Support is included for LoRA, checkpoint, and embedding model types
2025-10-26 10:42:09 +08:00
Will Miao
427e7a36d5 feat: improve model update detection logic
Update ModelUpdateRecord.has_update() to only detect updates when a newer remote version exists than the latest local version. Previously, any missing remote version would trigger an update, which could include older versions that shouldn't be considered updates.

- Add logic to find the maximum version ID in library
- Only return True for remote versions newer than the latest local version
- Add comprehensive unit tests for the new update detection behavior
- Update docstring to reflect the new logic
2025-10-25 21:31:01 +08:00
Will Miao
2abb5bf122 feat: add update_available flag to model services
Add update_available field to checkpoint, embedding, and LoRA service response formatting. The flag indicates whether a model update is available and defaults to false when not specified.

Include comprehensive tests to verify the update flag is properly included in formatted responses and defaults to false when not present in the payload.
2025-10-25 16:29:54 +08:00
Will Miao
bb65527469 feat: add database migration system for model update schema
Add migration support to handle schema changes without data loss. Instead of dropping and recreating tables, the system now:
- Uses CREATE TABLE IF NOT EXISTS for initial table creation
- Adds _apply_migrations method to handle incremental schema updates
- Adds _get_table_columns helper to inspect existing table structure
- Adds new columns to model_update_versions table (sort_index, name, base_model, released_at, size_bytes, preview_url, is_in_library, should_ignore)
- Adds should_ignore_model column to model_update_status table

This ensures existing databases are upgraded gracefully while preserving user data.
2025-10-25 16:05:39 +08:00
Will Miao
d9a6db3359 feat: optimize model update checking with bulk operations
- Refactor update filter logic to use bulk update checks when available
- Add annotation method to attach update flags to response items
- Improve performance by reducing API calls for update status checks
- Maintain backward compatibility with fallback to individual checks
- Handle edge cases and logging for failed update status resolutions
2025-10-25 15:31:33 +08:00
Will Miao
58cafdb713 feat: add model version management endpoints
- Add set_version_update_ignore endpoint to toggle ignore status for specific versions
- Add get_model_versions endpoint to retrieve version details with optional refresh
- Update serialization to include version-specific data and preview overrides
- Modify database schema to support version-level ignore tracking
- Improve error handling for rate limiting and missing models

These changes enable granular control over version updates and provide better visibility into model version status.
2025-10-25 14:54:23 +08:00
pixelpaws
807425f12a feat(preview): respect blur mature content setting 2025-10-25 06:43:03 +08:00
pixelpaws
49e0deeff3 fix(recipes): sanitize shared recipe filenames 2025-10-24 15:17:12 +08:00
pixelpaws
faa26651dd fix(download): recover stalled transfers automatically 2025-10-23 17:25:38 +08:00
pixelpaws
dde2b2a960 fix(downloader): enforce file size integrity checks 2025-10-23 11:55:39 +08:00
pixelpaws
8d336320c0 fix(scanner): apply metadata adjustments during reconciliation 2025-10-23 07:34:35 +08:00
pixelpaws
acaf122346 fix(downloader): request identity encoding by default 2025-10-22 10:17:39 +08:00
pixelpaws
c5175bb870 fix(checkpoints): preserve model type on persisted load 2025-10-21 22:55:00 +08:00
Will Miao
6148a12301 Merge branch 'main' of https://github.com/willmiao/ComfyUI-Lora-Manager 2025-10-19 20:06:08 +08:00
Will Miao
fadbf21b4f fix(relink): keep sha untouched during relinking 2025-10-19 20:05:58 +08:00
pixelpaws
1a34403b0e fix(model-cache): avoid mutating raw entries without fields 2025-10-18 21:30:49 +08:00
pixelpaws
e4d58d0f60 fix(cache): harden metadata defaults 2025-10-18 21:19:09 +08:00
pixelpaws
f7a856349a fix(utils): sanitize path template folder names 2025-10-18 16:20:47 +08:00
Will Miao
10f5588e4a feat: add model_name and version_name placeholders to download paths, #552
Add support for {model_name} and {version_name} placeholders in download path templates. These new placeholders allow for more flexible and descriptive file organization by including the actual model name and version name in the download directory structure.

Changes include:
- Updated download_manager.py and utils.py to handle new placeholders
- Added placeholders to constants.js for UI reference
- Updated settings modal template to show available placeholders
- Added comprehensive tests to verify placeholder functionality

This enhancement provides users with more control over how downloaded models are organized on their file system.
2025-10-17 16:01:06 +08:00
Will Miao
61101a7ad0 fix(recipe-scanner): honor SFW filtering option, fixes #576 2025-10-17 10:35:00 +08:00
Will Miao
6d9be814a5 feat(ui): add configurable model card footer action, fixes #249 2025-10-17 08:43:35 +08:00
pixelpaws
00fade756c fix(settings): dispatch name display updates on original loop 2025-10-16 09:02:35 +08:00
pixelpaws
3c0feb23ba feat(model-cache): respect model name display preference 2025-10-16 07:01:04 +08:00
pixelpaws
21a1bc1a01 feat(metadata): batch refresh model versions 2025-10-15 20:47:30 +08:00
Will Miao
a5b2e9b0bf feat: add update service dependency and has_update filter
- Pass ModelUpdateService to CheckpointService, EmbeddingService, and LoraService constructors
- Add has_update query parameter filter to model listing handler
- Update BaseModelService to accept optional update_service parameter

These changes enable model update functionality across different model types and provide filtering capability for models with available updates.
2025-10-15 17:25:16 +08:00
pixelpaws
3bb240d3c1 fix(updates): avoid caching failed civitai lookups 2025-10-15 16:00:23 +08:00
pixelpaws
ee0d241c75 refactor(routes): limit update endpoints to essentials 2025-10-15 15:37:35 +08:00
Will Miao
412f1e62a1 feat(i18n): add model name display option and improve localization, fixes #440
- Add new model name display setting with options to show model name or file name
- Implement helper function to determine display name based on user preference
- Update model card footer to use dynamic display name
- Include model name display setting in settings modal and state management
- Remove redundant labels from display density descriptions in multiple locales
- Simplify card info display descriptions by removing duplicate text

The changes provide cleaner UI text and add flexibility for users to choose between displaying model names or file names in card footers.
2025-10-15 10:23:39 +08:00
Will Miao
a3070f8d82 feat: add rate limit error handling to CivArchive client
- Add RateLimitError import and exception handling in API methods
- Create _make_request wrapper to surface rate limit errors from downloader
- Add test case to verify rate limit error propagation
- Set default provider as "civarchive_api" for rate limit errors

This ensures rate limit errors are properly propagated and handled throughout the CivArchive client, improving error reporting and allowing callers to implement appropriate retry logic.
2025-10-14 21:38:24 +08:00
Will Miao
3fde474583 feat(civitai): add rate limiting support and error handling
- Add RateLimitError import and _make_request wrapper method to handle rate limiting
- Update API methods to use _make_request wrapper instead of direct downloader calls
- Add explicit RateLimitError handling in API methods to properly propagate rate limit errors
- Add _extract_retry_after method to parse Retry-After headers
- Improve error handling by surfacing rate limit information to callers

These changes ensure that rate limiting from the Civitai API is properly detected and handled, allowing callers to implement appropriate backoff strategies when rate limits are encountered.
2025-10-14 21:38:24 +08:00
Will Miao
5173aa6c20 feat(model-scanner): add logging for file processing, fixes #566 2025-10-14 19:44:59 +08:00
pixelpaws
055c1ca0d4 test(downloads): cover pause and resume flows 2025-10-13 21:30:23 +08:00
Will Miao
27370df93a feat(download): add support to download models from civarchive, fixes #381 2025-10-13 19:27:56 +08:00
pixelpaws
eb76468280 feat(downloads): expose throughput metrics in progress APIs 2025-10-13 14:39:31 +08:00
Will Miao
acf610ddff feat(model-scanner): add metadata tracking and improve cache management
- Add metadata_source field to track origin of model metadata
- Define MODEL_COLUMNS constants for consistent column management
- Refactor SQL queries to use dynamic column selection
- Improve Civitai data detection to include creator_username and trained_words
- Update database operations to handle new metadata field and tag management
2025-10-12 16:54:39 +08:00
Will Miao
b32b4b4042 feat: enhance model scanning to include creator username
Updated the `ModelScanner` class to extract and format the creator username from Civitai data. This enhancement ensures that the creator information is properly included in slim model data.
2025-10-12 08:51:42 +08:00
Will Miao
b77c596f3a Fix error message to improve clarity in DownloadManager 2025-10-12 05:43:59 +08:00
pixelpaws
84d10b1f3b feat(metadata): log metadata channel on fetch 2025-10-11 20:07:01 +08:00
pixelpaws
7be1a2bd65 Merge pull request #557 from willmiao/civarc-api-support
CivArchive API support
2025-10-11 18:10:06 +08:00
Will Miao
1dc189eb39 feat(metadata): implement fallback provider strategy for deleted models
Refactor metadata sync service to use a prioritized provider fallback system when handling deleted CivitAI models. The new approach:

1. Attempts civarchive_api provider first for deleted models
2. Falls back to sqlite provider if archive DB is enabled
3. Maintains existing default provider behavior for non-deleted models
4. Tracks provider attempts and errors for better debugging

This improves reliability when fetching metadata for deleted models by trying multiple sources before giving up, and provides clearer error messages based on which providers were attempted.
2025-10-11 17:44:38 +08:00
pixelpaws
6120922204 chore(priority-tags): add newline terminator 2025-10-11 17:38:20 +08:00
Will Miao
ddb30dbb17 Revert "feat(civarchive_client): update get_model_version_info to resolve the real model/version IDs before fetching the target metadata."
This reverts commit c3a66ecf28.
2025-10-11 16:11:17 +08:00
Will Miao
1e8bd88e28 feat(metadata): improve model ID redirect logic and provider ordering
- Fix CivArchive model ID redirect logic to only follow redirects when context points to original model
- Rename CivitaiModelMetadataProvider to CivArchiveModelMetadataProvider for consistency
- Reorder fallback metadata providers to prioritize Civitai API over CivArchive API for better metadata quality
- Remove unused asyncio import and redundant logging from metadata sync service
2025-10-11 16:11:13 +08:00
Will Miao
c3a66ecf28 feat(civarchive_client): update get_model_version_info to resolve the real model/version IDs before fetching the target metadata. 2025-10-11 15:07:42 +08:00
Will Miao
1f60160e8b feat(civarchive_client): enhance request handling and context parsing
Introduce `_request_json` method for async JSON requests and improved error handling. Add static methods `_normalize_payload`, `_split_context`, `_ensure_list`, and `_build_model_info` to parse and normalize API responses. These changes improve the robustness of the CivArchiveClient by ensuring consistent data structures and handling potential API response issues gracefully.
2025-10-11 13:07:29 +08:00
Will Miao
47da9949d9 feat: update recipe download URL path to include /lm prefix
Update the download URL path in RecipeSharingService to include '/lm' prefix,
aligning with the new API route structure for recipe sharing endpoints.
2025-10-10 21:46:56 +08:00
scruffynerf
68c0a5ba71 Better Civ Archive support (adds API) (#549)
* add CivArchive API

* Oops, missed committing this part when I updated codebase to latest version

* Adjust API for version fetching and solve the broken API (hash gives only files, not models - likely to be fixed but in the meantime...)

* add asyncio import to allow timeout cooldown

---------

Co-authored-by: Scruffy Nerf <Scruffynerf@duck.com>
2025-10-10 20:04:01 +08:00
pixelpaws
8f5e134d3e fix: skip redundant hydration in metadata sync service 2025-10-10 08:49:54 +08:00