When a deleted model is checked against the SQLite archive and not found, the `db_checked` flag was set in memory but never saved to disk. This occurred because the save operation was only triggered when `civitai_api_not_found` was True, which is not the case for deleted models (since the CivitAI API is not attempted). As a result, deleted models would be rechecked on every refresh instead of being skipped.
Changes:
- Introduce a `needs_save` flag to track when metadata state is updated
- Save metadata whenever `db_checked` is set to True, regardless of API status
- Ensure `last_checked_at` is set for SQLite-only attempts
- Add regression test to verify the fix
- Update `is_civitai_api_metadata` to exclude both "archive_db" and "civarchive" sources
- Skip Civitai metadata updates when existing metadata is higher quality than incoming archive data
- Add test to verify API metadata is preserved when CivArchive provides lower-quality data
- Track when Civitai API returns "Model not found" for default provider
- Use dedicated flag instead of error string comparison for deletion detection
- Ensure archive-sourced models don't get marked as deleted
- Add test coverage for archive source deletion flag behavior
- Fix deletion flag logic to properly handle provider fallback scenarios
Add license resolution utilities and integrate license information into model metadata processing. The changes include:
- Add `resolve_license_payload` function to extract license data from Civitai model responses
- Integrate license information into model metadata in CivitaiClient and MetadataSyncService
- Add license flags support in model scanning and caching
- Implement CommercialUseLevel enum for standardized license classification
- Update model scanner to handle unknown fields when extracting metadata values
This ensures proper license attribution and compliance when working with Civitai models.
- Add RateLimitError import and _make_request wrapper method to handle rate limiting
- Update API methods to use _make_request wrapper instead of direct downloader calls
- Add explicit RateLimitError handling in API methods to properly propagate rate limit errors
- Add _extract_retry_after method to parse Retry-After headers
- Improve error handling by surfacing rate limit information to callers
These changes ensure that rate limiting from the Civitai API is properly detected and handled, allowing callers to implement appropriate backoff strategies when rate limits are encountered.
Refactor metadata sync service to use a prioritized provider fallback system when handling deleted CivitAI models. The new approach:
1. Attempts civarchive_api provider first for deleted models
2. Falls back to sqlite provider if archive DB is enabled
3. Maintains existing default provider behavior for non-deleted models
4. Tracks provider attempts and errors for better debugging
This improves reliability when fetching metadata for deleted models by trying multiple sources before giving up, and provides clearer error messages based on which providers were attempted.
- Fix CivArchive model ID redirect logic to only follow redirects when context points to original model
- Rename CivitaiModelMetadataProvider to CivArchiveModelMetadataProvider for consistency
- Reorder fallback metadata providers to prioritize Civitai API over CivArchive API for better metadata quality
- Remove unused asyncio import and redundant logging from metadata sync service
* add CivArchive API
* Oops, missed committing this part when I updated codebase to latest version
* Adjust API for version fetching and solve the broken API (hash gives only files, not models - likely to be fixed but in the meantime...)
* add asyncio import to allow timeout cooldown
---------
Co-authored-by: Scruffy Nerf <Scruffynerf@duck.com>
