Installing a second LoRA Manager instance (standalone or a second
ComfyUI install) that shares the settings directory puts two processes
on the same cache databases. Three things made that unsafe.
- The updater preserved cache/ and model_cache/ but not a legacy
recipe_cache/ directory, so a portable install predating the cache/
move lost its recipe database on a git-based update. Add it to
_PRESERVE_DIRS and to .gitignore.
- Cache connections used the sqlite3 default 5s timeout, which a
scanning instance can exceed, turning a concurrent write into
"database is locked". Route every shared cache connection through
connect_cache_db(), which raises the timeout to 30s and sets
busy_timeout + synchronous=NORMAL to match the existing WAL mode.
App-private databases (download queue, update history) are unchanged.
- A full-table cache replace is a read-modify-write that SQLite cannot
make atomic across processes, so two instances could interleave and
one snapshot could overwrite the other. Guard the recipe and model
save_cache paths with a cross-process advisory lock (flock on POSIX,
msvcrt on Windows). Locking is best-effort: if it is unavailable the
call proceeds and the SQLite busy timeout is the fallback.
The lock file is a hidden sibling of the database and is deliberately
never unlinked, so a second process cannot lock a fresh inode.
A scan that finds no recipe files at all is not a reliable deletion
signal: an unmounted drive, a recipes_path that silently falls back to
another LoRA root, or a cache shared with a second instance all look
exactly like a real wipe. The reconcile step treated them all as
deletions and overwrote the persistent cache with an empty one, so
DELETE FROM recipes destroyed the user's only record of their recipes
and the FTS index was rebuilt from the empty view (#1116).
Guard the prune:
- _reconcile_recipe_cache reports an all-missing result when every
persisted recipe file is gone AND the stored rows match the recorded
file stats. An internally inconsistent cache (leftover orphans) is
stale, not evidence of a fresh disappearance, and still prunes.
- The caller keeps the stored cache and logs a warning naming the
directory it scanned and the number of recipes it preserved, instead
of writing the empty result. It also skips the FTS rebuild so the
index stays aligned with the stored rows.
- save_cache gains skip_if_empty as a storage-level backstop: refuse to
empty a populated cache. Intentional clears (manual rebuild) keep the
default behaviour.
- Log the resolved scan directory per run so a support reader can tell a
real wipe apart from a scan that looked elsewhere.
Partial orphans (ordinary manual deletions) keep pruning as before.
- Snapshot pre-rematch entry state (reconnectSnapshot) so rematched
entries can be undone via the existing restore flow
- Bulk missing-LoRA downloads mark unresolvable failures hash-invalid,
flipping those entries from download to reconnect candidacy
- Recipe modal always offers a reconnect action next to download for
missing LoRA entries
- Rematch runs collect an opt-in relaxed-matching choice (also reconnect
missing models by file name) via a pre-run options dialog on the
global, bulk and single-recipe entries
- L4 (filename-level) matches are listed in a results dialog with
per-entry undo
The recipe "Repair Metadata" action has been marked Deprecated in the UI
for a while and cannot reliably recover recipes imported from CivitAI URLs
whose REST meta has no resources/hashes and whose image has no embedded
metadata (e.g. CivitAI-only generation data). Drop the feature end to end.
Backend:
- remove repair routes (repair, cancel-repair, recipe/{id}/repair,
repair-bulk, repair-progress) and their handler mappings/methods
- remove RecipeScanner repair_all_recipes / repair_recipe_by_id /
_repair_single_recipe and REPAIR_VERSION
- remove WebSocketManager recipe-repair progress channel
- drop repair_version column from the persistent recipe cache
- rematch mutual-exclusion now only checks rematch
Frontend:
- remove repair entries from per-recipe, bulk and global context menus
- remove repairRecipe / repairSelectedRecipes / repairRecipes + cancelRepair
and the repairBulk API client method/endpoint
- drop recipe-repair i18n keys (synced across locales; doctor keys kept)
Tests/docs: delete test_recipe_repair.py, update scaffolding/routes/ws/
persistent-cache/integration tests and i18n guideline examples.
Broadcast typed scan_progress messages over /ws/fetch-progress from the
manual refresh/rebuild paths of ModelScanner and RecipeScanner, and
render percent, processed/total, current file name and an EMA-smoothed
ETA in the loading overlay. Hardcoded refresh strings move to i18n
(common.scanProgress); WS connection failure falls back to the previous
static loading behavior.
Normalize undetermined recipe base_model to None in RecipeFormatParser
(previously ''). get_base_models now reports an "Unknown" bucket backed
by a dedicated __unknown__ marker, and the listing filter matches it
against recipes whose base model is falsy. Frontend renders the bucket
label as "Unknown" while filtering via the marker.
Tests: handler, scanner, parser, and frontend filtering.
Checkpoint entries that cannot be restored by download (deleted,
unresolvable hash, or name-only remnants with no CivitAI identifiers)
now get the same remediation chain LoRAs already had:
- scanner: parameterized reconnect-suggestion ranking, update/restore/
set-hash-invalid for the checkpoint entry, and clear hashInvalid on
rematch write-back (was only done for LoRAs)
- persistence/handlers/routes: reconnect/restore/reconnect-suggestions/
mark-hash-invalid endpoints under /api/lm/recipe/checkpoint/*
- modal: checkpoint reconnect UI (deleted/hash-invalid badges, inline
form with suggestions, undo for reconnected entries); download
failures mark the hash invalid only on explicit unresolvable signals
(not found/deleted/404/410), matching the LoRA rule
- css: checkpoint undo button shares the LoRA undo styles
- i18n: the 14 new keys translated in all 9 locales
Enhance the deleted-LoRA reconnect flow in the recipe modal:
- Suggest local reconnect candidates when the panel opens, ranked by
identity (same hash / same CivitAI version) then filename/name
similarity, with a hard filter on confident base-model mismatches;
the input gets a Combobox backed by the same endpoint as you type.
- Snapshot the pre-reconnect entry and offer a permanent restore:
reconnected entries show an undo icon at the right end of the info
row, with the original filename in the tooltip.
- Relax the manual reconnect base-model guard to a three-tier check:
exact/unknown labels pass silently, same-architecture families
(e.g. Pony <-> Illustrious) pass with a warning toast, and only
cross-architecture mismatches stay hard-rejected.
The recipe card pill counted LoRAs deleted from the source (isDeleted) as
available, showing a green 'ready 2/2' for recipes that cannot be fully
reproduced. LoRAs with an unresolvable hash (hashInvalid) were counted as
missing/downloadable even though downloads always fail, and recipe syntax
generation emitted broken tokens for them.
- Four-state status on RecipeCard pill and RecipeTab badge: ready (all in
library), missing (downloadable, red, keeps the action cue), partial
(unobtainable entries skipped when used, amber, fa-circle-minus),
unavailable (nothing usable, gray, fa-ban)
- Pill numerator is now the real in-library count; tooltips spell out
missing vs unavailable (deleted from source or unresolvable hash)
- get_recipe_syntax_tokens skips hashInvalid entries like deleted ones
instead of emitting tokens pointing at nonexistent files
- Bulk missing-download manager and recipe context menu exclude
hashInvalid LoRAs, matching the modal's per-item download block
- New locale keys loraStatus.missingAndUnavailable/partial/noneUsable,
translated for all 9 non-en locales
- import: prefer A1111 Lora hashes (12-char AutoV3) over conflicting Hashes
JSON values; recover the quote-wrapped AutoV3 from CivitAI image API meta;
merge EXIF-parsed LoRAs when the API-only parse yields none (meta=null)
- rematch: treat entries whose hash failed CivitAI resolution (hashInvalid)
as unresolved candidates; clear the flag on rematch/reconnect write-back
- download: persist hashInvalid and show a distinct toast when hash lookup
returns "Model not found", so unresolvable entries become recoverable
- ui: add Unresolvable Hash badge styling and reconnect affordance
- i18n: translate the new keys across all 10 locales
The recipes page always rendered with is_initializing=False, so a cold
start displayed an empty grid that never updated until a manual refresh.
Mirror the model pages: gate render_page on the scanner state, broadcast
init progress from RecipeScanner (including a completion message, and a
failure fallback so the page never stalls), and teach initialization.js
to detect the /loras/recipes page before the generic /loras match.
get_cached_data() claimed to wait for a running initialization but
actually returned the placeholder empty cache, so API requests during
startup saw zero recipes. The initializing flag was also set only after
the LoRA scanner wait, leaving an unguarded window. Mark initialization
before the first await and have callers await the in-flight task.
Consolidate the duplicate name-matching logic into ModelScanner:
find_matching_models is now the single core, using each scanner's own
file_extensions for suffix stripping. get_model_info_by_name gains
require_unique/base_model kwargs while legacy route behavior is kept
byte-identical. reconnect_lora passes the recipe base model as a guard
and distinguishes ambiguous, base-model-mismatched, and missing LoRAs
in its error messages.
get_recipe_syntax_tokens() previously skipped all LoRAs with
isDeleted=True unconditionally. Now it tries to resolve the file
locally first (via hash index or modelVersionId); only skips if
the LoRA is truly unavailable.
This is a companion fix to #946 (AutoV2 hash matching).
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Track recipe modal opens in a separate stats file (never touching recipe
JSON/EXIF), expose a fire-and-forget POST endpoint, and add an 'opened'
sort that hides never-opened recipes as a true recently-opened view.
Includes i18n for all locales and backend/frontend tests.
Drop the SequenceMatcher-based fuzzy_match fallback that froze the server
when FTS returned empty results. FTS now returns empty set for zero results
(no fallback), and when the index is not yet ready, search returns empty
rather than scanning all items in Python.
Build a civitai_image_id → recipe_id mapping once during cache
initialization instead of scanning all recipes on every
check_image_exists and import_from_url call.
- RecipeCache gains an image_id_map field populated by
_build_image_id_map() during cache init
- check_image_exists and import_from_url duplicate detection
now use the precomputed map (O(k) / O(1) vs O(n))
- Map is persisted in SQLite cache_metadata for fast startup
- Incrementally updated on add/remove/bulk_remove paths
- Fix: conn.close() before cache_metadata query (dead connection)
Add corruption detection to _repair_single_recipe: if checkpoint.modelVersionId matches any LoRA's modelVersionId, the checkpoint is corrupted (a LoRA was saved as checkpoint). Clear the checkpoint and remove the matching LoRA entry, then let enrichment re-resolve the correct checkpoint from CivitAI metadata.
This fixes the retroactive repair path for the modelVersionIds[0] fallback bug.
Autocomplete, copy/send-to-workflow, and recipe syntax now emit
<lora:folder/name:strength> instead of <lora:name:strength>, using
relative paths to disambiguate identically-named loras in different
subfolders without requiring file renames.
Backend: 3-tier hybrid resolution (path → bare → basename fallback)
across get_lora_info, get_lora_info_absolute, get_model_preview_url,
get_model_civitai_url, get_model_info_by_name, get_lora_metadata_by_filename,
and get_hash_by_filename. Also fix get_random_loras and get_cycler_list
to return path-prefixed names for randomizer/cycler consistency.
Frontend: autocomplete, copyLoraSyntax, handleSendToWorkflow emit
folder-prefixed syntax. extract_lora_name preserves relative paths.
Saved image metadata (<lora:...> in EXIF) intentionally keeps basename-only
for compatibility with A1111/Forge ecosystem.
- Add source_path column to PersistentRecipeCache SQLite schema with
migration for existing databases (ALTER TABLE ADD COLUMN)
- Backfill source_path from recipe JSON files on first startup after
migration to avoid requiring manual cache rebuild
- Remove all source_url recipe field references (import_remote_recipe,
import_from_url, check_image_exists, enrichment, batch_import)
and consolidate on source_path as the single source of truth
- Add civitai.green to supported Civitai page hosts
- Register check-image-exists and import-from-url recipe endpoints
Refactor force_refresh path to use thread pool execution instead of blocking
the event loop shared with ComfyUI. Key changes:
- Fix 1: Route force_refresh through _initialize_recipe_cache_sync() in thread pool
- Fix 2: Add GIL release points (time.sleep(0)) every 100 files in sync loops
- Fix 3: Move RecipeCache.resort() to thread pool via run_in_executor
- Fix 4: Persist cache automatically after force_refresh
- Fix 5: Increase yield frequency in _enrich_cache_metadata (every recipe)
This eliminates the ~5 minute freeze when rebuilding 30K recipe cache.
Fixes performance issue where ComfyUI became unresponsive during recipe
scanning due to shared Python event loop blocking.
Fix KeyError when 'hashes', 'name', or 'model' fields are missing from
Civitai API responses. Use .get() with defaults instead of direct dict
access in:
- LoraMetadata.from_civitai_info()
- CheckpointMetadata.from_civitai_info()
- EmbeddingMetadata.from_civitai_info()
- RecipeScanner._get_hash_from_civitai()
- DownloadManager._process_download()
Fixes#820
- Use modelVersionId as fallback for all loras in fingerprint calculation (not just deleted)
- Add URL-based duplicate detection using source_path field
- Combine both fingerprint and URL-based duplicate detection in API response
- Fix _download_remote_media return type and unbound variable issue
Refactor recipe lookup logic to improve efficiency from O(n²) to O(n + m):
- Build recipe_by_id dictionary for O(1) recipe ID lookups
- Simplify persisted_by_path construction using recipe_id extraction
- Add fallback lookup by recipe_id when path lookup fails
- Maintain same functionality while reducing computational complexity
Introduce a new PersistentRecipeCache service that stores recipe metadata in an SQLite database to significantly reduce application startup time. The cache eliminates the need to walk directories and parse JSON files on each launch by persisting recipe data between sessions.
Key features:
- Thread-safe singleton implementation with library-specific instances
- Automatic schema initialization and migration support
- JSON serialization for complex recipe fields (LoRAs, checkpoints, generation parameters, tags)
- File system monitoring with mtime/size validation for cache invalidation
- Environment variable toggle (LORA_MANAGER_DISABLE_PERSISTENT_CACHE) for debugging
- Comprehensive test suite covering save/load cycles, cache invalidation, and edge cases
The cache improves user experience by enabling near-instantaneous recipe loading after the initial cache population, while maintaining data consistency through file change detection.
Fixes a critical bug in FTS query building where multi-word searches
with field restrictions incorrectly used OR between all word+field
combinations instead of requiring ALL words to match within at least
one field.
Example: searching "cute cat" in {title, tags} previously produced:
title:cute* OR title:cat* OR tags:cute* OR tags:cat*
Which matched recipes with ANY word in ANY field.
Now produces:
(title:cute* title:cat*) OR (tags:cute* tags:cat*)
Which requires ALL words to match within at least one field.
Also adds fallback to fuzzy search when FTS returns empty results,
improving search reliability.
Co-Authored-By: Claude <noreply@anthropic.com>
Introduce a new RecipeFTSIndex class that provides fast prefix-based search across recipe fields (title, tags, LoRA names/models, prompts) using SQLite's FTS5 extension. The implementation supports sub-100ms search times for large datasets (20k+ recipes) and includes asynchronous indexing, incremental updates, and comprehensive unit tests.
Updated image path resolution logic to prioritize local sibling images in the same directory as recipes. When a stored image path differs from a local sibling, the system now automatically updates the recipe file to use the local path and persists this repair. This improves reliability when recipe assets are moved or reorganized, ensuring images remain accessible even if original paths become invalid.
Will Miao