CivitaiClient._make_request now retries 5xx/524/network errors up to 3 times with exponential backoff (1s, 2s) before giving up to the fallback provider chain.
get_model_version_info gains an in-memory OrderedDict cache (LRU, max 500 entries) so duplicate lookups of the same version ID within a single import/scan flow return instantly without a redundant API call.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Extend _is_transient_server_error() check introduced in 15dfaed4 to
get_image_info(), so Cloudflare 524 and generic 5xx errors during
remote recipe import are logged as info instead of error and do not
produce scary tracebacks.
Same pattern as get_model_versions() - transient upstream failures
return None gracefully rather than being logged as errors.
Adds lora_syntax_format setting (full/legacy) that controls whether <lora:...> syntax uses relative paths (full) or filename only (legacy). Default is legacy for backward compatibility with A1111 convention. The full path format (<lora:relative/path/filename:strength>) enables lossless model resolution across subfolders.
Ultraworked with Sisyphus (https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Teach CivitaiClient.get_model_versions() to recognise Cloudflare 524, generic
5xx, and connection-level errors as transient failures and return None
instead of raising RuntimeError, so a single upstream glitch does not
block the entire batch update or produce a scary traceback.
Also downgrade the generic except Exception log level in
ModelUpdateService._refresh_single_model() from error (with exc_info)
to warning (message only), since the full traceback is already logged
upstream in CivitaiClient.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
- Layer 2 fallback: user tags overlapping with auto-tag categories
(HIGH/LOW/I2V/T2V/TI2V/Lightning/Turbo) are merged into auto_tags,
providing manual override when filename-based detection fails.
Matching is case-insensitive so "high"/"High"/"HIGH" all work.
- Refresh on tag edit: save_metadata and add_tags handlers now return
recalculated auto_tags in the response; the frontend passes them to
VirtualScroller.updateSingleItem so badges update immediately without
requiring a page reload.
- 8 new test cases for Layer 2 fallback and case-insensitive matching.
Autocomplete, copy/send-to-workflow, and recipe syntax now emit
<lora:folder/name:strength> instead of <lora:name:strength>, using
relative paths to disambiguate identically-named loras in different
subfolders without requiring file renames.
Backend: 3-tier hybrid resolution (path → bare → basename fallback)
across get_lora_info, get_lora_info_absolute, get_model_preview_url,
get_model_civitai_url, get_model_info_by_name, get_lora_metadata_by_filename,
and get_hash_by_filename. Also fix get_random_loras and get_cycler_list
to return path-prefixed names for randomizer/cycler consistency.
Frontend: autocomplete, copyLoraSyntax, handleSendToWorkflow emit
folder-prefixed syntax. extract_lora_name preserves relative paths.
Saved image metadata (<lora:...> in EXIF) intentionally keeps basename-only
for compatibility with A1111/Forge ecosystem.
Duplicate filename detection is only relevant for LoRAs, which use
basename-only syntax (<lora:name:strength>). Checkpoints and diffusion
models reference files via relative paths with extensions, so filename
conflicts there are false positives — there is no resolution ambiguity.
Both _log_duplicate_filename_summary() and DoctorHandler's
_check_filename_conflicts() now skip scanners with model_type != 'lora'.
- Backend auto-tag extraction service: detect HIGH/LOW (Wan-only), I2V/T2V/TI2V,
Lightning/Turbo from filename, base_model, and CivitAI version name
- HIGH/LOW badge in card footer (inline before version name), color-coded:
blue for HIGH, teal for LOW; abbreviated to H/L in medium/compact density
- Auto-tag filter panel (I2V, T2V, TI2V, Lightning, Turbo) with tri-state
include/exclude filtering
- Full filter pipeline: FilterCriteria → ModelFilterSet → baseModelApi params
- AUTO_TAG_GROUPS exported for frontend use
- 19 unit tests for auto-tag extraction edge cases
The CivitAI /api/v1/models endpoint defaults to filtering out NSFW
content when the nsfw query parameter is omitted. Both get_user_models()
and get_model_versions_bulk() hit this endpoint without passing nsfw=true,
causing models whose nsfwLevel doesn't include the PG bit to be silently
dropped from results.
Add nsfw=true to both call sites so all browsing levels are returned.
- When downloaded Civitai image has no embedded EXIF, parse the
already-fetched Civitai API meta (resources, hashes) directly
instead of skipping parser altogether.
- Extract loras and model from parser output to fill metadata gaps
when the primary import path doesn't provide them.
- Read modelVersionIds[0] as fallback when modelVersionId is None
(Civitai API returns both but the singular form can be absent).
- Run RecipeEnricher in analyze_remote_image before returning, so
the LM UI receives complete metadata including checkpoint with
zero additional API calls (reuses the image_info already fetched).
- Wrap ExifUtils.extract_image_metadata() with asyncio.to_thread() in
both import handlers and analysis_service to prevent Pillow/piexif
from blocking ComfyUI's event loop during batch imports.
- Add asyncio.Semaphore(2) to import_remote_recipe and import_from_url
endpoints to cap concurrent heavy work and prevent event loop starvation.
- Pre-fetch Civitai image_info during download and pass it to the recipe
enricher, eliminating a redundant get_image_info() API round-trip.
The method mark_not_downloaded() was misleading — it doesn't negate
'downloaded' history (the model was indeed downloaded before), but
rather sets is_deleted_override = 1 to indicate the version was
downloaded and subsequently deleted. This flag allows re-download when
the 'skip previously downloaded' setting is enabled.
Rename to mark_as_deleted() to accurately reflect its semantics.
After deleting a model, the in-memory scanner cache was updated but the
SQLite persistent cache was not. On server restart, the stale persistent
cache caused check_model_version_exists() to return True, blocking
re-download with 'Model version already exists'.
Add _persist_current_cache() calls in both deletion paths:
- ModelLifecycleService.delete_model() (used by versions tab delete)
- delete_model_version handler in MiscHandlers
- Add source_path column to PersistentRecipeCache SQLite schema with
migration for existing databases (ALTER TABLE ADD COLUMN)
- Backfill source_path from recipe JSON files on first startup after
migration to avoid requiring manual cache rebuild
- Remove all source_url recipe field references (import_remote_recipe,
import_from_url, check_image_exists, enrichment, batch_import)
and consolidate on source_path as the single source of truth
- Add civitai.green to supported Civitai page hosts
- Register check-image-exists and import-from-url recipe endpoints
- Ernie & Anima: auto-fetched via CivitaiBaseModelService from Civitai API
- Ernie Turbo & Nucleus: pre-added as hardcoded constants (not yet in Civitai API)
- Added abbreviations (ERNI, ETRB, NUCL) and category entries across all layers
The model-level API (GET /api/v1/models/{id}) does not include usageControl
on version entries, causing generation-only models to show as downloadable.
Backend changes:
- Add get_model_versions_by_hashes() to CivitaiClient (POST by-hash batch)
- Propagate through all provider classes including RateLimitRetryingProvider
- Add _enrich_version_entries() pipeline: extract SHA256 from files[].hashes,
batch-call by-hash endpoint, inject usageControl+earlyAccessEndsAt in-place
- Wire enrichment into both bulk (_fetch_model_versions_bulk) and individual
(_refresh_single_model) refresh paths
- Fix _build_record_from_remote dropping usage_control field
- Fix POST by-hash request format (plain JSON array, not {hashes:[...]} object)
Frontend changes:
- Fix disabled download button tooltip: wrap in <span> since HTML title
attribute does not fire on disabled elements
Root cause: 231 concurrent /check-model-exists requests on 175K-lora library
caused ~9.4s wall clock time. The bottleneck was two-fold:
1. DownloadedVersionHistoryService opened a new sqlite3.connect() for every
query under asyncio.Lock. With a large WAL from 175K entries, each
connect() took ~8ms. Serialized by the lock across 231 requests, the
230th request waited ~1848ms just for lock acquisition.
2. check_model_exists always queried download history even when the model
was found locally. The history result (hasBeenDownloaded /
downloadedVersionIds) is only used by the UI when the model is NOT
found locally; when found, the 'in library' indicator takes priority.
Changes:
- downloaded_version_history_service.py: added persistent _get_conn() that
creates the SQLite connection once and reuses it across all queries
- misc_handlers.py: early-return from check_model_exists when the model
exists locally, bypassing the history service entirely (lock skipped)
Expected: per-request wait time drops from ~1912ms to <3ms, wall clock
from ~9.4s to <0.3s for the 175K-lora user's 231-card page.
Handle models that are only available for on-site generation (usageControl:
"Generation" or "InternalGeneration") rather than downloadable.
Backend changes:
- Add usage_control field to ModelVersionRecord dataclass
- Extract usageControl from Civitai API responses
- Filter non-downloadable versions from update availability checks
- Add database schema migration for usage_control column
- Include usageControl in version response JSON
Frontend changes:
- Add isDownloadAllowed() helper function
- Show disabled download button for non-downloadable versions
- Add "On-Site Only" badge for restricted versions
- Update resolveUpdateAvailability() to filter non-downloadable versions
- Add CSS styling for disabled action button
Internationalization:
- Add translations for onSiteOnly badge and downloadNotAllowedTooltip
- Complete translations for all 10 supported languages
Detects when multiple model files share the same basename (causing
ambiguity in LoRA resolution), logs warnings during scanning, and
provides a "Resolve Conflicts" button in the Doctor panel. Resolution
renames duplicates with hash-prefixed unique filenames, migrates all
sidecar and preview files, and updates the cache and frontend scroller
in-place so the model modal immediately reflects the new filename.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CivitAI returns file type "Diffusion Model" for checkpoint files (e.g., Anima
models), but the file selection logic only accepted "Model" and "Negative",
causing "No suitable file found in metadata" errors.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
_split _get_records_bulk into 500-id batches so the WHERE IN clause
never exceeds SQLite's 999-parameter ceiling.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
