mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-08-28 08:21:27 -03:00
feat(services): add per-destination rate-limit gate for API traffic (#1085)
Implement Phase 1 of docs/plans/issue-1085-rate-limit-design.md: - New RateLimitCoordinator: per-host shared Retry-After gate with exponential backoff (30s base, 1800s cap), minimum inter-request pacing (default 0.75s), herd-free waiter serialization via per-destination locks, and a bounded wait (default 300s) that raises instead of parking. - Downloader.make_request: connectivity-guard fail-fast first, then gate pacing; on 429 register the cooldown and wait-and-resend (bounded); errors that passed through the gate are marked gate_handled. - FallbackMetadataProvider / MetadataSyncService: a network provider 429 no longer fails over to other network providers (stops the CivArchive flood); sqlite stays as local last resort. Rate-limited lookups now report "Rate limited" instead of "Model not found", so transient 429s no longer mark models civitai_deleted. - _RateLimitRetryHelper skips its own sleep for gate_handled errors, removing the double wait. - New settings: rate_limit_gate_enabled, rate_limit_max_wait_seconds, rate_limit_min_interval_seconds.
This commit is contained in:
@@ -245,16 +245,23 @@ class MetadataSyncService:
|
||||
civitai_api_not_found = False
|
||||
any_rate_limited = False
|
||||
|
||||
skip_network_providers = False
|
||||
for provider_name, provider in provider_attempts:
|
||||
if skip_network_providers and provider_name != "sqlite":
|
||||
# A network provider was already rate-limited; failing
|
||||
# over to another network provider just spreads the flood
|
||||
# (#1085). The local sqlite archive stays as last resort.
|
||||
continue
|
||||
try:
|
||||
civitai_metadata_candidate, error = await provider.get_model_by_hash(sha256)
|
||||
except RateLimitError as exc:
|
||||
logger.warning(
|
||||
"Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
|
||||
"Provider %s is rate-limited (retry_after=%.0fs); not failing over to other network providers",
|
||||
provider_name or provider.__class__.__name__,
|
||||
exc.retry_after or 0,
|
||||
)
|
||||
any_rate_limited = True
|
||||
skip_network_providers = True
|
||||
continue
|
||||
except Exception as exc: # pragma: no cover - defensive logging
|
||||
logger.error("Provider %s failed for hash %s: %s", provider_name, sha256, exc)
|
||||
|
||||
Reference in New Issue
Block a user