Merge skill.yaml (metadata) and prompt.md (prompt template) into a
single SKILL.md file with YAML frontmatter, matching the agent-skill
convention used by opencode and Claude Code.
- Add frontmatter parser (_parse_skill_file) to SkillRegistry
- Remove skill.yaml, prompt.md, empty skills/__init__.py
- Remove obsolete load_handler method
- Update tests for new format and cleaned-up fields
Introduce an agent skill framework for LLM-driven metadata enrichment:
- AgentCLI (py/agent_cli/): in-process wrappers around internal services
using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
(replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales
Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
A model not being found on CivArchive by hash is a routine case (the
model simply isn't published there), not an error. The callers already
log the outcome at WARNING (bulk_metadata_refresh) or DEBUG
(metadata_sync_service) with full context, making this ERROR-level log
both misleading and redundant.
Cache corruption (NULL model_name/file_name from legacy DB rows or partial
writes) caused format_response to raise KeyError/AttributeError, failing the
entire /loras/list request and showing no models in the UI.
Fix across three layers:
- format_response (lora/checkpoint/embedding): replace direct dict[] access
with .get() fallbacks; return None for entries missing file_path
- handlers: filter None entries from list/excluded/fetch/duplicate/conflict
endpoints instead of letting them crash or appear as null in responses
- model_scanner: always use validate_batch repaired copies (previously
discarded when no invalid entries, leaving None values in raw_data)
- persistent_model_cache: add or-empty-string guards on read and write for
nullable TEXT columns (model_name, file_name, folder, base_model, etc.)
- Add &withMeta=true to image info URL so API returns full generation
metadata (resources with hash/type) instead of null meta
- Fix checkpoint assignment guard: check modelId instead of id so non-
checkpoint types (upscaler) are not wrongly set as recipe checkpoint
- Skip modelVersionIds loop when resources/civitaiResources already
provided LoRAs, preventing hash-resolved duplicates
- Fix int/str type comparison in CivArchive get_model_version so
version ID matching works correctly
When CivitAI image API returns meta=null and modelVersionIds at root
level, the import flow now:
- Injects modelVersionIds + browsingLevel into a minimal metadata dict
so the parser can discover LoRAs and checkpoints (both import-from-url
and analyze-image paths)
- Adds checkpoint dedup + fallback in the parser's modelVersionIds
handler to avoid duplicate API calls
- Runs EXIF extraction unconditionally in analyze-image path, then
merges with API metadata (fixes gen params loss)
- Propagates preview_nsfw_level through all three import paths:
import-from-url, analyze-image (UI Import), and batch-import,
plus the frontend save flow
- Prefer file type (UNet/Diffusion Model) over baseModel name when
deciding whether a checkpoint routes to the unet folder
- Add UNet to backend primary file type whitelist
- Add Krea 2 to DIFFUSION_MODEL_BASE_MODELS
- Include UNet/Diffusion Model files in frontend file selection UI
- Use actual file type from CivitAI in download params instead of
hardcoded 'Model'
When refreshing updates with a folder filter, versions already present in
other folders were excluded from the is_in_library check, making them
appear as available updates. When the user tried to download, the global
check found the file already exists and returned 'model already exists'.
Fix by also collecting the cross-folder version set when folder_path is
provided, and using the union (folder-filtered + cross-folder) for
is_in_library in both _build_record_from_remote and
_merge_with_local_versions.
_drain_stderr and _wait_until_ready both read from the same stderr pipe.
Starting the drain task before _wait_until_ready creates a race where the
drain task consumes aria2's early-exit error message before the startup
waiter can read it, resulting in an empty error message in the logs.
Also confirmed that --fsync does not exist as an aria2 option (exit code
28 = Invalid argument).
Exit code 28 (Invalid argument) indicates this user's aria2c does not
support the --fsync option. Remove it unconditionally; the stderr drain,
relaxed RPC timeouts, and increased retry coverage remain in place.
aria2 default --fsync=true calls fsync() after each write, which blocks
the entire single-threaded process on large files under Docker overlay.
Add --fsync=false to eliminate this blocking source.
Relax aiohttp session timeout: total=30 → sock_connect=10, sock_read=60
so that transient I/O delays don't cut off legitimate tellStatus RPCs.
Increase retry params (4 attempts, 3s delay) to give aria2 more recovery
time when blocked on synchronous I/O.
Root cause: aria2c subprocess stderr pipe (64 KB buffer) was never
drained. When enough error/warning output accumulated, aria2's write()
blocked, freezing the entire process including its RPC handler. The
tellStatus call then timed out after 30s with asyncio.TimeoutError(),
producing the empty error message in 'Failed to query aria2 download
status: '.
Fixes:
- Drain stderr in a background task so pipe never fills up
- Retry get_status() RPC calls up to 3 times on transient failure
- In the failure path, preserve .safetensors when .aria2 is absent
(the download was likely complete on disk)
Sort by Most/Fewest versions first now works when Group by model is off.
- Backend: group items by modelId (respecting version_grouping setting),
count versions per group, sort groups by count, expand groups with
versions sorted by version id descending
- CSS: remove rule that hid the sort option in non-grouped mode
- Tests: add 3 tests covering desc, asc, and same_base variants
When viewing all versions of a model (VLM mode via 'x versions' button):
- Backend always sorts by version ID descending, ignoring current sort_by
- A temporary 'Newest version first' option is injected into the sort
dropdown (removed on exit, not a permanent option)
- The sort dropdown is disabled (greyed out) while VLM is active
- On clearing VLM, the previous sort preference is restored and the
dropdown re-enabled
- Handles stale VLM state (e.g. after page reload with leftover session)
- Covers all three model page types: loras, checkpoints, embeddings
Also fixes review nits:
- Correct i18n call pattern (defaultValue in options object)
- Shared _restoreSortAfterVlm() helper to avoid triple duplication
- group_by_model dedup now counts versions per group and attaches
version_count; respects update_flag_strategy (same_base) by
sub-grouping on base_model
- Card footer shows clickable 'x versions' link instead of version
name when grouped (hides HIGH/LOW badges); clicking triggers
View Local Versions without page reload
- Added 'Local Versions' sort option (versions_count), auto-hidden
when group_by_model is off
- Sort preference is saved/restored separately for normal and
grouped modes
- VLM flow (triggerVlmView, clearCustomFilter) uses resetAndReload()
via API instead of window.location.reload()
- Fixed cache mutation bug: version_count is now set on a shallow
copy, not the cached dict, preventing stale version_count leaking
into VLM responses
- i18n: all 9 locale files translated
Clicking the button closes the modal, writes filter params to sessionStorage,
and reloads the page to show all local versions of the model as individual
cards (bypassing group-by-model dedup). The filter respects the update flag
strategy and the versions-filter-toggle state (same-base vs all versions).
Supporting changes:
- sessionStorage keys vlm_model_id / vlm_model_name / vlm_base_model
- BaseModelApiClient._addModelSpecificParams adds civitai_model_id param
- LoraApiClient calls super._addModelSpecificParams for VLM detection
- LorasControls / CheckpointsControls clearCustomFilter checks VLM first
- PageControls.checkVlmFilter shows customFilterIndicator with label
- Backend parses civitai_model_id, filters before group_by_model dedup
Adds a 'Group by Model' toggle in Layout Settings. When enabled, only the
latest version (highest civitai.id) of each Civitai model is shown as a
single card — older versions sharing the same modelId are hidden.
Backend dedup runs in BaseModelService.get_paginated_data() before
filtering/pagination, ensuring correct paginated results. The setting
is persisted via the existing settings pipeline and passed as a query
parameter to the listing endpoint.
Includes:
- Backend: dedup logic, route param parsing, settings default
- Frontend: API param, SettingsManager wiring, toggle UI
- i18n: translations for all 10 locales
- Tests: unit test covering dedup on/off and standalone items
- save_metadata_updates now trims/lowercases/dedupes tags on write
- ModelFilterSet tag matching is now case-insensitive (both include/exclude)
- Removed redundant .lower() calls in tag_update_service.py
Backend changes:
- Add civitai_api_key to _NO_SYNC_KEYS, return only boolean civitai_api_key_set
- Clean up known template placeholder on load to prevent false positive
Frontend changes:
- Replace type=password with type=text + CSS masking (-webkit-text-security)
- Replace pre-filled input with status display (Configured/Not configured)
- Add inline edit view with Save/Cancel buttons
- Re-add eye toggle via CSS class toggle (not type switching)
- Use CSS transitions for smooth status/edit view switching
This prevents Chromium/Vivaldi password manager from triggering
'save password' prompts when opening the settings modal.
- Add 5 new Tabler SVG icons (currency-dollar, brush, user, git-merge, license)
- Implement Set 2 rendering in ModelModal.js (standalone UI) with green/red
permission indicators and preview_tooltip.js (ComfyUI widget)
- Add use_new_license_icons setting (default: true) with toggle in settings UI
- ComfyUI tooltip reads setting directly from preview-url API response to
eliminate race conditions and respect standalone settings changes
- Remove the now-unused separate ComfyUI setting loramanager.license_icon_style
- Add CSS for both standalone (lora-modal.css) and widget (lm_styles.css)
- i18n: translate licenseIcons keys into all 10 supported languages
- Fix test to use classic style explicitly for continued coverage
- New GET /api/lm/downloads/queue/status handler for non-terminal status
transitions (queued -> downloading, downloading -> paused, etc.)
- Queue lifecycle auto-integration in DownloadManager._download_with_semaphore:
downloading -> SQLite update_status('downloading') on semaphore acquire
completed -> complete_download('completed') on success
canceled -> complete_download('canceled') on CancelledError
failed -> complete_download('failed') on Exception
- All queue operations wrapped in try/except to never break the download flow
- Remove pin/unpin and auto-hide hover mechanism (isPinned, isHovering,
hoverTimeout, showSidebar/hideSidebar, updateAutoHideState, etc.)
- Remove global show_folder_sidebar setting (SettingsManager,
PageControls, recipes, backend default)
- Simplify sidebar visibility to a single per-page toggle:
· Dedicated chevron-left button in header to hide sidebar
· Edge indicator (chevron-right) to restore when hidden
· No dropdown, no hover area, no pin button
- Add _migrateOldSettings() to convert old sidebarPinned and
show_folder_sidebar states to per-page sidebarDisabled
- Fix sidebar flicker on page load: CSS defaults to off-screen,
JS explicitly sets .visible or .hidden-by-setting
- Remove obsolete CSS classes: auto-hide, hover-active, collapsed
- Remove i18n keys: pinSidebar, unpinSidebar, moreOptions
- Update test mocks for the new initialize() interface
When a model is already classified as civitai_deleted=True via
.metadata.json but re-enters the failure block through the
civarchive/sqlite provider path (not the default provider),
needs_save was never set to True because civitai_api_not_found
and sqlite_attempted were both False. The flags were never
persisted to SQLite, causing the model to be re-fetched on
every restart.
Also demoted duplicate INFO/ERROR logging in fetch_and_update_model
to DEBUG (the use case already logs at WARNING), and added
exc_info=True to the fetch_all_civitai error handler.
When CivArchive returns HTTP 429 with a large retry_after, the bulk
metadata refresh would block for hours because:
1. FallbackMetadataProvider raised RateLimitError instead of continuing
to the next provider (e.g., SQLite archive was never reached).
2. _RateLimitRetryHelper retried long-rate-limit 429s 3 times — all
futile since the hourly cap hasn't reset.
3. The batch loop had no awareness of persistent rate-limiting,
causing 192+ models to each hammer the same rate-limited endpoint.
Changes:
- FallbackMetadataProvider: all 6 methods now continue to next provider
on RateLimitError instead of raising (model_metadata_provider.py)
- fetch_and_update_model: deleted-model path also continues on
RateLimitError so sqlite provider gets a chance (metadata_sync_service.py)
- _RateLimitRetryHelper: when retry_after >= 120s, only 1 attempt is
made — retries are futile for hour-scale rate limits
- BulkMetadataRefreshUseCase: tracks consecutive rate-limit failures
and aborts early after 3 (bulk_metadata_refresh_use_case.py)
Tests: updated test_fallback_respects_retry_limit for new continue
behavior; added tests for large/small retry_after thresholds.
- retry_from_history() and retry_all_failed() now DELETE the original
history entry after re-queuing it. Previously the old entry stayed
in history causing exponential growth on repeated retry→cancel→retry
cycles.
- Add deduplicate() called once on singleton creation to clean up
existing duplicate queue/history entries left by the bug:
1. In-status dedup (keep highest id per model+version+status)
2. Cross-status dedup (prefer completed > failed > canceled)
3. Queue dedup (keep highest rowid per model+version)
4. Orphan queue cleanup (source='retry' entries obsoleted by
terminal history entries)
The proxy settings allow selecting a SOCKS proxy type, but the SOCKS
URL was passed to aiohttp's per-request `proxy=` argument, which only
supports http(s) proxies. With a SOCKS proxy this opens a plain TCP
connection to the proxy port and sends an HTTP request; the SOCKS
server replies with its handshake bytes (e.g. b"\x05\xff") and aiohttp
fails with "Bad status line ... Expected HTTP/, RTSP/ or ICE/".
Route SOCKS proxy types through an aiohttp-socks ProxyConnector on the
session instead, leaving the `proxy=` kwarg for http(s) proxies only.
trust_env now keys off whether an app-level proxy is active. Adds
aiohttp-socks to requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Build a civitai_image_id → recipe_id mapping once during cache
initialization instead of scanning all recipes on every
check_image_exists and import_from_url call.
- RecipeCache gains an image_id_map field populated by
_build_image_id_map() during cache init
- check_image_exists and import_from_url duplicate detection
now use the precomputed map (O(k) / O(1) vs O(n))
- Map is persisted in SQLite cache_metadata for fast startup
- Incrementally updated on add/remove/bulk_remove paths
- Fix: conn.close() before cache_metadata query (dead connection)
- Add local file reimport support via _do_reimport_from_local
- Validate source_path BEFORE deleting old recipe (prevent data loss)
- Move delete_recipe after save_recipe (safe ordering)
- Preserve folder location, NSFW level, and carry over user edits
- Remove old timestamp preservation (use current time)
- Add scrollTop reset in resetAndReloadWithVirtualScroll
- Only reload on successful bulk reimport (avoid empty grid)
- Disable preserveScroll for both single and bulk reimport
- Change _get_stats_file_path() to use get_settings_dir()/stats/ instead of
first loras root directory
- Add _migrate_from_old_location() to copy existing stats from loras root
to new location on first access, then clean up old file
- Add 'stats' to update protection skip lists (clean, extract, tracking)
to prevent data loss during ZIP/git upgrades in portable mode
- Add usage_stats entry to backup targets and restore resolver so stats
are included in automatic snapshots
When CivitAI API returns meta=null and the optimized CDN image has no
embedded generation parameters (e.g. PNG tEXt chunks stripped by
Cloudflare Images), download the original image as fallback to recover
full recipe metadata (prompt, seed, LoRAs, etc.).
Also fixes Chrome password manager popping up on recipe save by adding
autocomplete="new-password" to the settings API key and proxy password
fields.
On Windows, shutil.rmtree() fails when deleting a directory that contains
an open SQLite database file. The ZIP update path in _download_and_replace_zip()
calls _clean_plugin_folder() which tries to delete the cache/ directory,
but downloaded_versions.sqlite is held open by DownloadedVersionHistoryService.
Fix:
- Add close() method to DownloadedVersionHistoryService to release
the persistent SQLite connection
- Call close() before _clean_plugin_folder() in the ZIP update flow
- Add 'cache' to the skip_files list so the runtime cache directory is
never deleted during plugin updates
When certifi is available, pass its CA bundle path as --ca-certificate
to the aria2c subprocess so that aria2 downloads use the same
certificate store as Python aiohttp downloads. Graceful fallback when
certifi is not installed.
Add corruption detection to _repair_single_recipe: if checkpoint.modelVersionId matches any LoRA's modelVersionId, the checkpoint is corrupted (a LoRA was saved as checkpoint). Clear the checkpoint and remove the matching LoRA entry, then let enrichment re-resolve the correct checkpoint from CivitAI metadata.
This fixes the retroactive repair path for the modelVersionIds[0] fallback bug.
