Address the rate-limit flood and secondary errors seen during large
recipe ingestion (example-images directory import):
- batch import: share one adaptive-concurrency semaphore across the whole
batch (previously each item got a fresh semaphore, so the min/max
concurrency bounds never applied and every item ran concurrently);
synchronize the shared semaphore capacity after each completed item.
- comfy parser: guard ckpt_name against list/None values so re.search no
longer raises TypeError and fails the whole image import.
- civarchive client: normalize empty-string failure payloads to
"Request failed" and treat a missing payload as an error, fixing the
"'NoneType' object has no attribute 'get'" crash.
- civarchive client: log connectivity-guard offline-cooldown
short-circuits at DEBUG instead of one ERROR per request.
Fix ~790 basedpyright errors across the test suite:
- Type stub subclasses of real production classes with super().__init__()
- Add missing generic type arguments and Dict[str, Any] annotations
- Add None guards before subscript/member access
- Adapt tests to production API changes (removed dead handlers,
PersistentModelCache.get_default, _i18n_filter_added location)
- Add RateLimitError import and exception handling in API methods
- Create _make_request wrapper to surface rate limit errors from downloader
- Add test case to verify rate limit error propagation
- Set default provider as "civarchive_api" for rate limit errors
This ensures rate limit errors are properly propagated and handled throughout the CivArchive client, improving error reporting and allowing callers to implement appropriate retry logic.
- Fix CivArchive model ID redirect logic to only follow redirects when context points to original model
- Rename CivitaiModelMetadataProvider to CivArchiveModelMetadataProvider for consistency
- Reorder fallback metadata providers to prioritize Civitai API over CivArchive API for better metadata quality
- Remove unused asyncio import and redundant logging from metadata sync service
Introduce `_request_json` method for async JSON requests and improved error handling. Add static methods `_normalize_payload`, `_split_context`, `_ensure_list`, and `_build_model_info` to parse and normalize API responses. These changes improve the robustness of the CivArchiveClient by ensuring consistent data structures and handling potential API response issues gracefully.
Will Miao