Compare commits

...

339 Commits

Author SHA1 Message Date
Will Miao fc9088bfd6 feat(recipes): restore Copy Recipe Syntax button in recipe modal header
- Add an icon-only copy button next to Send to ComfyUI in the header
  actions row, styled as a textless variant of the neighboring pill
  buttons
- Restore fetchAndCopyRecipeSyntax() wiring against the existing
  /api/lm/recipe/{id}/syntax endpoint (context menu action unaffected)
- Add recipes.actions.copyRecipeSyntax i18n key, reusing the existing
  per-locale translations of the identical context menu string
- Sync modal test fixtures and add copySyntax tests
2026-08-29 17:05:31 +08:00
Will Miao 675421ea84 fix(recipes): render reconnect form for hash-invalid LoRAs in recipe modal
The Reconnect action button was rendered for both deleted and hash-invalid
(Unresolvable Hash) LoRA entries, but the .lora-reconnect-container input
form was only rendered for deleted ones. Clicking Reconnect on a
hash-invalid item silently did nothing because showReconnectInput() could
not find the container. Align the container render condition with the
button condition, and extend the resource-items test to assert the form
opens on click.
2026-08-29 16:49:50 +08:00
Will Miao 2ff98ae089 docs(i18n): defer non-en translations until UI wording is final
[TODO: Translate] placeholders are now the sanctioned intermediate state
during feature development; translate all pending keys in one pass only
when the feature owner asks. R8 notes the exemption so placeholders are
not 'fixed' prematurely.
2026-08-29 16:41:53 +08:00
Will Miao c972c755fc fix(recipes): distinguish unobtainable LoRAs in recipe status and skip them in syntax
The recipe card pill counted LoRAs deleted from the source (isDeleted) as
available, showing a green 'ready 2/2' for recipes that cannot be fully
reproduced. LoRAs with an unresolvable hash (hashInvalid) were counted as
missing/downloadable even though downloads always fail, and recipe syntax
generation emitted broken tokens for them.

- Four-state status on RecipeCard pill and RecipeTab badge: ready (all in
  library), missing (downloadable, red, keeps the action cue), partial
  (unobtainable entries skipped when used, amber, fa-circle-minus),
  unavailable (nothing usable, gray, fa-ban)
- Pill numerator is now the real in-library count; tooltips spell out
  missing vs unavailable (deleted from source or unresolvable hash)
- get_recipe_syntax_tokens skips hashInvalid entries like deleted ones
  instead of emitting tokens pointing at nonexistent files
- Bulk missing-download manager and recipe context menu exclude
  hashInvalid LoRAs, matching the modal's per-item download block
- New locale keys loraStatus.missingAndUnavailable/partial/noneUsable,
  translated for all 9 non-en locales
2026-08-29 16:41:48 +08:00
Will Miao ebe3df7d22 docs(i18n): mark guidelines as the post-sweep target state; translate de playlist title
§3/§5/§6 now describe the resolved state (regression watch-list instead of a
to-do list), §4 documents the single intentional placeholder deviation
(mappingsUpdated drops {plural} where '<noun>s' cannot be appended). de
help.updateVlogs.playlistTitle translated.
2026-08-29 12:49:38 +08:00
Will Miao be44a75b74 fix(i18n): punctuation polish — ASCII colons/parens, '...' ellipsis, fr apostrophe
- fullwidth ':{message}/{error}' in fr/de/es/ru/he toasts -> ASCII
  (fr keeps the spaced ' : ' convention)
- fullwidth parens in bulk skip/resume count labels -> ASCII
- ru modals.download.selectHfFiles trailing fullwidth colon -> ASCII
- '…' -> '...' in all 9 locales (project style)
- fr header.filter.allowSellingGeneratedContentTooltip: d"images -> d'images
2026-08-29 12:48:05 +08:00
Will Miao fd1227d3b8 fix(i18n): translate banners, license labels, doctor UI and remaining leftovers
- banners.communitySupport.* (title/content/CTA/learnMore): 8 locales (zh-CN done)
- modals.model.license.noImageSell/noRentCivit/noRent/noSell: all 9 locales
- globalContextMenu.fetchMissingLicenses.*: 7 locales
- doctor.* issue titles, action labels, conflicts/version labels + es title
- toasts/settings: libraryLoadFailed/libraryActivateFailed, moveFailed,
  restartRequired, recipeSaved across locales; fr recipes storage path strings;
  zh-CN import lora count; ru/ko Recipe Manager init title
- checkpoints.modelTypes.diffusion_model translated in 7 locales (ja/ko keep
  the English loanword, consistent with their model-type names)
2026-08-29 12:47:17 +08:00
Will Miao 3a9e02137d fix(i18n): translate the batch-import UI for fr/de/es/ru/he/ja/ko
The whole recipes.batchImport section (~54 keys) and the
toast.recipes.batchImport* toasts (~8 keys) were byte-identical to en.json.
Translated using the normalized terminology (Recipe/Rezept/receta/рецепт/
מתכון/レシピ/레시피, bulk names: groupé/Massenimport/por lotes/пакетный/
בכמות גדולה/一括/일괄). URL/path placeholders stay as-is; identical words
(French 'Total', 'images') are legitimately unchanged.
2026-08-29 12:45:04 +08:00
Will Miao d8a2be8edc fix(i18n): normalize terminology and register across all locales
One term = one rendering per language; the mandatory fixes (see
docs/i18n-translation-guidelines.md §2/§5):
- fr: recette(s) -> Recipe(s) per glossary decision; checkpoint literal
  'Point de contrôle'/'hachage'/'étiquettes'/'dupliquées'/'mode lot' unified
- de: leftover English 'Recipe' -> Rezept; Basis-Modell -> Basismodell;
  Modelldaten -> Metadaten; bulk action label; du -> Sie (formal)
- es: 'Punto(s) de control' -> Checkpoint(s); flujo de trabajo -> workflow;
  palabras clave -> palabras de activación; preset -> preajuste; bulk -> por lotes
- ru: Контрольные точки/Чекпойнт -> Checkpoint; Эмбеддинг -> Embedding;
  запрос -> промпт (prompt sense only); рабочий процесс -> workflow; хэш -> хеш;
  безпотерьного typo
- he: נקודות ביקורת -> Checkpoint(s) (was literal road checkpoint); הטמעות ->
  Embedding; האש/גיבוב -> hash (האש reads as 'the fire'); הנחיה -> פרומפט;
  מטא-דאטה -> מטא-נתונים; דגם -> מודל; bulk feature name unified
- ja: チェックポイント/checkpoint -> Checkpoint; バルクモード -> 一括モード;
  recipe counter 個 -> 件; leftover English Recipe Manager translated
- ko: 체크포인트 -> Checkpoint; 임베딩 -> Embedding; 기본 모델 -> 베이스 모델;
  워크플로우 -> 워크플로; 벌크 모드 -> 일괄 모드; Checkpoint을 -> Checkpoint를
- zh-CN: 食谱 -> 配方; 检查点 -> Checkpoint; 基模型 -> 基础模型; 您 -> 你
- zh-TW: 食譜 -> 配方; 檢查點 -> Checkpoint; 你 -> 您 (18 keys)
2026-08-29 12:42:32 +08:00
Will Miao 1c46b2e8c3 fix(i18n): normalize CivitAI brand casing and civitai.red URL placeholders
- en.json: 49 values used 'Civitai' (lowercase 'ai'); normalize to the
  official 'CivitAI' casing and mirror in all 9 locales (key names like
  relinkCivitai/civitaiApiKey intentionally untouched)
- modals.relinkCivitai.helpText.format4: fix 'CivitArchive' typo -> 'CivArchive'
  in all locales (mirrored from en.json)
- recipes.controls.import.urlPlaceholder / modals.relinkCivitai.urlPlaceholder:
  restore the dropped 'https://civitai.red/...' alternative in 8 locales
  (zh-CN already had it)
2026-08-29 12:37:30 +08:00
Will Miao 3c3ac49f2f fix(i18n): correct stale help texts, inverted ko tag logic and placeholder contracts
- viewLocalTooltip: all 9 locales said 'coming soon'; describe the actual
  action (show local versions on main page)
- settings.downloadSkipBaseModels.help / hideEarlyAccessUpdates.help /
  aiProvider.apiBaseHelp: retranslate all locales to the current en wording
  (previous translations described an older source string)
- ko header.filter.tagLogicAny: 'all tags match' was inverted and identical
  to tagLogicAll; fix zh-TW typo 票籤 -> 標籤
- modals.checkUpdates.title/message: restore {typePlural} in zh-CN/zh-TW/ja/ko
- zh-CN recipes.controls.import.downloadLocationPreview: drop invented {path}
  (caller passes no params; it rendered literally)
- zh-TW toast.controls.refreshFailed: restore {action} placeholder
- toast.settings.mappingsUpdated: drop English-inflection {plural} where '<noun>s'
  would corrupt the noun (zh-CN/zh-TW/ja/ko/de/ru/he); caller passes hardcoded 's'
2026-08-29 12:36:18 +08:00
Will Miao 1a1be95a64 docs(i18n): add translation guidelines with per-locale term conventions
Audit of all 10 locale files found recipe/checkpoint mistranslations,
inverted ko tag logic, stale help texts, placeholder contract deviations,
and untranslated feature blocks. Document the conventions (R1-R9), per-
language term maps, confusion hot-spots, and the translation workflow so
future agents and translators follow the established decisions (e.g. keep
'Recipe' untranslated in French, use 配方 in Chinese).
2026-08-29 12:16:52 +08:00
Will Miao 7a36659a20 fix(downloads): preserve aria2 partial pair and refresh expired CivitAI signed URLs
A failed aria2 transfer deleted the partial payload while keeping its
.aria2 control file, and "No URI available" (expired CivitAI signed URL)
was treated as a permanent failure, wasting nearly-complete downloads.

- Re-schedule the transfer with a freshly resolved signed URL and
  continue=true when aria2 reports "No URI available", bounded by
  MAX_TRANSFER_RECOVERY_ATTEMPTS
- Keep payload and .aria2 control file together as a resumable pair
  after a failed transfer instead of deleting the payload
- Report and remove orphaned .aria2 control files that have no payload,
  both after failures and when restoring persisted downloads

Fixes #1088
2026-08-29 11:31:16 +08:00
Will Miao cb18281b14 fix(recipes): pin recipe modal badge sizing against import-modal.css collision
import-modal.css is loaded after recipe-modal.css and its unscoped
.missing-badge/.deleted-badge (equal specificity) were clobbering the
recipe modal's badge family, leaving invalid-hash-badge (no import
counterpart) at a different size. Scope the recipe status-badge sizing
under #recipeModal so import-modal.css can't override it. Also remove the
duplicate .deleted-badge block in import-modal.css.
2026-08-28 22:54:25 +08:00
Will Miao 856c9a87ac fix(recipes): resolve stale LoRA hash on import and add hashInvalid state
- import: prefer A1111 Lora hashes (12-char AutoV3) over conflicting Hashes
  JSON values; recover the quote-wrapped AutoV3 from CivitAI image API meta;
  merge EXIF-parsed LoRAs when the API-only parse yields none (meta=null)
- rematch: treat entries whose hash failed CivitAI resolution (hashInvalid)
  as unresolved candidates; clear the flag on rematch/reconnect write-back
- download: persist hashInvalid and show a distinct toast when hash lookup
  returns "Model not found", so unresolvable entries become recoverable
- ui: add Unresolvable Hash badge styling and reconnect affordance
- i18n: translate the new keys across all 10 locales
2026-08-28 22:24:07 +08:00
Will Miao a7d65fe84a feat(recipes): redesign resource item badges and actions in recipe modal
- Badges are pure status indicators with tooltips; remediation moves to a
  per-item action row (Download / Reconnect), matching the versions-tab
  badge/button pattern
- Civitai link inlines with the model title; the action row renders only
  when real actions exist, removing empty-row whitespace
- Single-LoRA download resolves identifiers from hash on demand (same
  fallback as the bulk missing-download flow) and shows immediate
  'Preparing download' feedback while resolving
- Successful downloads (LoRA and checkpoint) refresh the resources
  section and the recipe card in place, mirroring the bulk flow
- Row navigation is limited to in-library items with keyboard support;
  checkpoint type renders as muted text instead of a chip; badges use
  tonal styling; the local-path hover tooltip is removed
- Add resourceItems frontend tests and translate the new keys for all
  10 locales
2026-08-28 09:30:37 +08:00
Will Miao 15bf079af2 docs: remove git commit message guidelines from AGENTS.md 2026-08-27 22:38:52 +08:00
Will Miao 65ba750634 feat(recipes): improve recipe LoRA status indicators and missing-badge affordance (#1076)
- Recipe card: compact status pill with state icon + available/total
  fraction (e.g. "2/3"), pinned to the footer bottom-right like model
  card actions; status is encoded by icon + color, never color alone
- Recipe modal: "N missing" is now a real <button> with a persistent
  border, leading download icon, focus-visible ring and aria-label;
  clicking opens the download-missing flow
- Fix context menu missing-LoRA detection selector after badge refactor
- i18n: add recipes.status/loraStatus keys with translations for all
  10 locales, and fill pending rate-limit translations
2026-08-27 22:37:10 +08:00
Will Miao 17dcbd3d4f fix(delete): merge delete batches manifest-only, never move files
Bulk delete merged staged batches by physically moving each loser's
files into the winner's batch dir with os.rename. Cross-volume bulks
(winner and loser on different filesystems) always hit EXDEV, forcing a
rollback and degrading to the batch_ids array with per-batch undo.

Merge is now manifest-only: loser entries are appended to the winner's
manifest with their staged paths unchanged, so staged files keep living
in each model's own .lm-pending-delete/<batch_id> dir (no data IO, no
EXDEV). Loser dirs are recorded in the winner manifest's merged_sources
and each loser manifest is stamped merged_into so its own purge timer, a
post-restart sweep or a direct undo call no-op. A cross-volume bulk is
one undoable batch again, and undo/purge clean up the loser dirs once
the merged batch settles.
2026-08-27 19:32:17 +08:00
Will Miao e914a0e19d fix(ui): reconcile model listing in place after download (#1078)
Stop resetting the whole listing after a successful download. The legacy
flow reloaded page 1, scrolled to the top and hijacked the sidebar's
active folder whenever a custom target folder was used, which made the
Updates view lose its place (and sometimes render as an empty page).

Downloads only flip the update flag for one model, so the listing is now
reconciled in place through the virtual scroller:

- Updates view: the model's cards are removed once its newest eligible
  version is installed (the flag is model-level).
- Normal listing: the card stays; only update_available is cleared.
- Model not in the current view (different folder/filter/window):
  no-op; the sidebar folder tree alone is refreshed.
- Falling back to the legacy reload only when no virtual scroller is
  available (e.g. recipes page, duplicates mode, HF downloads).
2026-08-27 18:41:57 +08:00
Will Miao 2ba04bb1bd docs: merge CLAUDE.md content into AGENTS.md and remove CLAUDE.md 2026-08-27 18:41:57 +08:00
Will Miao 1b7314591a docs(skill): streamline lora-manager-e2e and gate usage to true integration checks
- Add a 'when to use / when not to use' gate: UI behavior questions
  default to Vitest/jsdom, E2E only for behavior spanning server +
  browser; description updated so the skill triggers less eagerly
- Pin the browser driver to Chrome DevTools MCP and explain why
  kimi-webbridge (user's real browser) is not a substitute
- Drop generic MCP pattern boilerplate duplicated by
  references/mcp-cheatsheet.md (SKILL.md 385 -> 145 lines)
- Move recipe rematch fixture / fresh-state / cancel-gap notes to
  references/recipe-rematch-fixtures.md
2026-08-27 18:15:04 +08:00
Will Miao 2bfb987312 feat(models): add shared searchable base model picker and overhaul bulk base model modal
- Extract a shared BaseModelPicker (search, keyboard navigation,
  filename-based suggestions, dynamic API models such as MiniMax H3
  under 'Other (API)') used by both the single-model metadata modal
  and the bulk base model modal
- Rework the bulk base model modal into a dedicated inline-list
  layout: fixed modal size, sticky-free footer with app-standard
  modal-actions/primary-btn/cancel-btn buttons, and an inline option
  list that scrolls itself instead of an overlay dropdown covering
  the footer
- Selecting an option in change mode now filters the list to the
  selection instead of resetting and scroll-jumping to it
- Restore opaque sticky section headers in the bulk modal so scrolled
  items no longer bleed through
2026-08-27 18:07:28 +08:00
Will Miao df34efafbc feat(recipes): skip rate-limited batch-import items and register download 429s (#1085)
Phase 2 of docs/plans/issue-1085-rate-limit-design.md:

- Batch import: items that fail due to vendor rate limiting are now
  SKIPPED with a "re-run the import later" hint instead of FAILED, so a
  transient 429 no longer pollutes failure accounting; the progress
  broadcast carries a rate_limited flag.
- Batch import UI: show a one-time "rate limited — slowing down" toast
  and swap the running status text while rate_limited; i18n keys synced
  to all locales.
- Downloader: download_file / download_to_memory / get_response_headers
  register 429 cooldowns with the RateLimitCoordinator, so subsequent
  API calls queue behind a download-triggered rate-limit window.
2026-08-27 10:08:32 +08:00
Will Miao c2a2048c8b feat(services): add per-destination rate-limit gate for API traffic (#1085)
Implement Phase 1 of docs/plans/issue-1085-rate-limit-design.md:

- New RateLimitCoordinator: per-host shared Retry-After gate with
  exponential backoff (30s base, 1800s cap), minimum inter-request pacing
  (default 0.75s), herd-free waiter serialization via per-destination
  locks, and a bounded wait (default 300s) that raises instead of parking.
- Downloader.make_request: connectivity-guard fail-fast first, then gate
  pacing; on 429 register the cooldown and wait-and-resend (bounded);
  errors that passed through the gate are marked gate_handled.
- FallbackMetadataProvider / MetadataSyncService: a network provider 429
  no longer fails over to other network providers (stops the CivArchive
  flood); sqlite stays as local last resort. Rate-limited lookups now
  report "Rate limited" instead of "Model not found", so transient 429s
  no longer mark models civitai_deleted.
- _RateLimitRetryHelper skips its own sleep for gate_handled errors,
  removing the double wait.
- New settings: rate_limit_gate_enabled, rate_limit_max_wait_seconds,
  rate_limit_min_interval_seconds.
2026-08-27 09:53:07 +08:00
Will Miao 1e1921cabb docs(plans): rate-limit abidance design for recipe ingest (#1085) 2026-08-27 09:02:42 +08:00
Will Miao ee233548e5 fix(recipes): enforce batch-import concurrency bound and harden ingest errors (#1085)
Address the rate-limit flood and secondary errors seen during large
recipe ingestion (example-images directory import):

- batch import: share one adaptive-concurrency semaphore across the whole
  batch (previously each item got a fresh semaphore, so the min/max
  concurrency bounds never applied and every item ran concurrently);
  synchronize the shared semaphore capacity after each completed item.
- comfy parser: guard ckpt_name against list/None values so re.search no
  longer raises TypeError and fails the whole image import.
- civarchive client: normalize empty-string failure payloads to
  "Request failed" and treat a missing payload as an error, fixing the
  "'NoneType' object has no attribute 'get'" crash.
- civarchive client: log connectivity-guard offline-cooldown
  short-circuits at DEBUG instead of one ERROR per request.
2026-08-27 07:58:48 +08:00
Will Miao 574dfbbe55 feat(settings): add explicit settings dir override for sandboxed runs
Add LORA_MANAGER_SETTINGS_DIR env var and standalone --settings-path to pin
the settings location (settings.json, cache/, wildcards/, backups/, logs/,
stats/) to an arbitrary directory. The override takes precedence over
portable mode and the platform user config dir, and skips legacy migration,
so sandboxed dev/E2E runs no longer need to write settings.json in the repo
root or collide with the real instance.

standalone.py pre-scans argv for --settings-path at import time because the
settings location is resolved before main() parses arguments. SettingsManager
portable-switch migration is a no-op while the directory is pinned.

Update the lora-manager-e2e skill (prefer --settings-path sandboxing;
start_server.py passes it through) and the lora-manager-runtime-context
skill (document precedence; inspect script honors the override).
2026-08-27 00:03:50 +08:00
Will Miao 1d3bcdfe47 fix(skill): quote lora-manager-e2e description so YAML frontmatter parses 2026-08-26 22:56:54 +08:00
Will Miao 74369940bf fix(recipes): log batch import progress only when it changes (#1084) 2026-08-26 22:39:51 +08:00
Will Miao d188cec306 fix(recipes): restore batch import modal on reopen and log recipe ingest progress (#1084) 2026-08-26 22:32:20 +08:00
Will Miao 641a61f804 feat(relink): accept CivitArchive URLs when linking models 2026-08-26 21:31:30 +08:00
Will Miao 3025c64fea fix(recipes): serve duplicate scan from cache and guard against re-entry 2026-08-26 20:34:50 +08:00
Will Miao c52cfc7e7a fix(download): serialize concurrent downloads resolving to the same target path 2026-08-26 12:17:14 +08:00
Will Miao 4ed9f775f6 feat(bulk): add shift+click range selection in bulk mode 2026-08-26 10:09:35 +08:00
Will Miao 0b08ad283a fix(bulk): restore card selection when virtual scroller recreates cards 2026-08-26 09:28:17 +08:00
Will Miao 08895f77ff fix(update): align update-check summary count with Updates filter scope (#1083) 2026-08-25 20:36:46 +08:00
Will Miao 74f889f160 fix(recipes): validate FTS index from stored metadata instead of scanning 2026-08-25 17:38:31 +08:00
Will Miao c51090ab16 fix(recipes): make source_path backfill a one-shot migration 2026-08-25 17:38:17 +08:00
Will Miao cdb044cb45 fix(scanner): offload persisted-cache hydration from the event loop 2026-08-25 17:38:09 +08:00
Will Miao c83b26b556 fix(delete): run startup reconciliation walk off the event loop 2026-08-25 17:38:01 +08:00
Will Miao a202c666bc fix(download): return 200 for missing queue items and quiet download-progress 404s
The browser extension's apiFetch treats any 404 as a missing endpoint and
retries the legacy non-/api/lm URL, producing two spurious
'error_middleware - WARNING - API GET ... 404' log lines per occurrence.

- complete_download_in_queue / update_download_queue_status /
  retry_download_from_history: 'not found' is a normal business outcome,
  return 200 + success:false instead of 404 (extension behavior unchanged;
  apiGet ignores the HTTP status)
- error_middleware: downgrade /api/lm/download-progress/ 404s to debug like
  previews - the 404 status itself stays (extension uses it for failure
  detection), only the log level is lowered
2026-08-25 09:59:33 +08:00
Will Miao e05046af10 fix(loaders): sanitize invalid control_after_generate values when loading old workflows
Old workflows (saved before the control_after_generate feature) carry a
shorter widgets_values array. The frontend's index-based widget restore
then shifts the old weight_dtype value into the hidden control widget
(leaving an invalid value like 'default') and silently resets
weight_dtype to its default. On graph load, hand the shifted value back
to weight_dtype when it still sits at its default, then reset the
control mode to 'fixed' so old workflows keep loading deterministically.
2026-08-25 08:30:17 +08:00
Will Miao 41ed03e5c6 fix(download): stop stale aria2 GIDs from spamming errors after queue clears
- Log expected "GID not found" tellStatus probes at DEBUG, and treat a
  forgotten GID as permanent so the poll loop recovers immediately
  instead of burning 4 retries x 3s of ERROR lines per cycle
- cancel_download tolerates a forgotten GID and always pops the
  in-memory transfer so concurrent polls cannot re-register a
  cancelled download
- Restore sweep deletes aria2 state records with no resolvable target
  path instead of skipping them forever
- Clearing the download queue now also cancels in-memory tasks, removes
  live aria2 transfers and drops persisted state for the cleared ids
  (partial files on disk are preserved)
2026-08-25 08:19:54 +08:00
Will Miao da071e8452 feat(versions): add file-variant badge and hide download button for in-library versions (#1058) 2026-08-24 23:17:10 +08:00
Will Miao a0bb6df2b8 test(recipe): reset RecipeScanner singleton in lora availability fixture 2026-08-24 17:23:20 +08:00
Will Miao 6f5c444ec5 feat(recipes): add lora availability filter to recipe filter panel 2026-08-24 17:00:02 +08:00
Will Miao 20f66a4fe1 fix(ui): reload listing when an invalid folder selection falls back to root
After a drag move empties the selected folder, refresh() resets the
stale activeFolder to root but the grid kept showing the old filtered
(empty) view until a manual reload. Trigger resetAndReload when the
fallback happens post-initialization; the initial page load is untouched
because it picks up the cleared filter on its own.
2026-08-24 14:17:06 +08:00
Will Miao 879745da53 fix(init): add missing /api/lm/init-status endpoint used by polling fallback
initialization.js falls back to polling /api/lm/init-status when the
/ws/init-progress WebSocket cannot be established, but no route ever
registered that path — each poll 404'd and the page never reloaded after
the scan completed. Report the aggregate status of all four scanners and
omit pageType so every initialization page accepts the update.
2026-08-24 14:17:06 +08:00
Will Miao 3afec0a0be fix(ui): fall back to folder root when persisted active folder no longer exists
restoreSelectedFolder trusted localStorage blindly: a stale activeFolder
(moved/deleted, or saved while the tree was still empty) left the grid
filtered to a nonexistent folder with a phantom breadcrumb and no way to
recover short of clicking the root breadcrumb. Validate the persisted
path against the freshly loaded tree and reset to root when it is gone;
skip validation when the tree load failed so transient errors don't wipe
the saved location.
2026-08-24 14:17:06 +08:00
Will Miao 06c270a6e1 fix(recipes): show initialization screen and auto-reload during first scan
The recipes page always rendered with is_initializing=False, so a cold
start displayed an empty grid that never updated until a manual refresh.
Mirror the model pages: gate render_page on the scanner state, broadcast
init progress from RecipeScanner (including a completion message, and a
failure fallback so the page never stalls), and teach initialization.js
to detect the /loras/recipes page before the generic /loras match.
2026-08-24 14:17:06 +08:00
Will Miao 87e93636dc fix(recipes): wait for in-flight cache initialization instead of returning empty cache
get_cached_data() claimed to wait for a running initialization but
actually returned the placeholder empty cache, so API requests during
startup saw zero recipes. The initializing flag was also set only after
the LoRA scanner wait, leaving an unguarded window. Mark initialization
before the first await and have callers await the in-flight task.
2026-08-24 14:17:06 +08:00
Will Miao 074d1f2e51 feat(ui): improve tag autocomplete toggle discoverability in prompt nodes
- Add Tag Autocomplete ON/OFF entry to the Prompt (LoraManager) node
  right-click menu, cross-referencing the slash commands
- Show the current autocomplete state (/autocomplete or /noautocomplete
  hint) below the slash command list
- Show a one-time dismissible tip in the suggestion dropdown on first use
- Clarify toggle command labels (Turn autocomplete ON/OFF) and cross-link
  all three entry points in the settings tooltip
- Share the setting write path via setLoraManagerSettingValue()
2026-08-24 12:21:31 +08:00
Will Miao 40f922b0e8 fix(ui): right-anchor license icons and delete button as one group in model modal 2026-08-24 11:39:17 +08:00
Will Miao a7214b6cff fix(i18n): translate remaining workflow-related UI strings 2026-08-24 09:31:17 +08:00
Will Miao 8ca66e72eb feat(ui): add delete button and Del shortcut to model and recipe modals 2026-08-24 09:27:00 +08:00
Will Miao 90be5799e4 fix(ui): preserve group editor scroll position when toggling tags 2026-08-24 08:17:47 +08:00
Will Miao 1a93b0eca2 feat(recipes): add prev/next navigation buttons and keyboard shortcuts to recipe modal 2026-08-24 08:15:19 +08:00
Will Miao c2360a35ad fix(ui): show empty folders as move and download destinations (#999) 2026-08-23 21:09:16 +08:00
Will Miao 030a32f8fa feat(ui): add hash search option and de-emphasized hash display in model modal 2026-08-23 10:09:55 +08:00
Will Miao 25e72b43ce fix(download): disable netrc auto-auth to avoid Authorization header conflict (#1070)
With trust_env=True, aiohttp auto-loads credentials from ~/.netrc (e.g. a
'machine civitai.red' or 'default' entry) and refuses to combine them with
the explicit Authorization: Bearer header, aborting every authenticated
CivitAI request with 'Cannot combine AUTHORIZATION header with AUTH
argument or credentials encoded in URL'.
2026-08-22 19:33:01 +08:00
Will Miao 41e9883daa test(recipe): cover send-workflow frontend paths 2026-08-21 21:09:58 +08:00
Will Miao ae461ebc81 fix(registry): replace one %s placeholder per log argument 2026-08-21 21:09:58 +08:00
Will Miao 3ebf256c5d feat(recipe): send embedded recipe workflow to ComfyUI canvas 2026-08-21 21:09:58 +08:00
Will Miao 0905e2be6e fix(recipes): restore primary style on checkpoint Send to ComfyUI button 2026-08-21 10:00:08 +08:00
Will Miao bd380bc1a1 fix(ui): replace stale command abbreviations in autocomplete messages 2026-08-21 09:20:11 +08:00
Will Miao cb4fd3a0e6 refactor(ui): split settings modal into section templates with shared macros 2026-08-21 09:14:52 +08:00
Will Miao bbe0acac5c fix(ui): keep loras widget context menu within viewport bounds 2026-08-21 08:07:50 +08:00
Will Miao 45e7c25308 feat(recipes): redesign import modal with URL-first input and unified drop zone 2026-08-21 00:01:50 +08:00
Will Miao 86aa1d8059 fix(ui): position toasts below header to avoid overlapping page controls 2026-08-20 22:08:27 +08:00
Will Miao 74254756ef fix(ui): unify modal backdrop blur across all modals 2026-08-20 21:25:05 +08:00
Will Miao 259e08e47c feat(download): expose per-file downloadedFiles in check-model-exists (#1058)
The version branch of check-model-exists now returns
downloadedFiles: [{fileId, fileName, filePath}] so clients (e.g. the
browser extension) can tell a partially downloaded version apart from a
fully downloaded one. Reuses ModelCivitaiHandler._match_downloaded_files
(D2 rule) against the local cache; unmatchable local files are reported
with fileId: None. No CivitAI API call added.
2026-08-20 20:59:19 +08:00
Will Miao 6647c45731 fix(download): include file identity in queue/history dedup (#1058)
Distinct files of the same model version queued before a backend restart
were silently collapsed by deduplicate(), which grouped rows by
(model_id, model_version_id) only. Extract the file id from file_params
via json_extract and add it to the dedup key; rows without file identity
keep the old per-version behavior (NULL matches NULL).
2026-08-20 20:58:44 +08:00
Will Miao b614a5c447 docs: remove broken star history chart (#1066) 2026-08-20 20:56:22 +08:00
Will Miao b80830913c refactor(nodes): declare loras widget as LORAS input type on lora nodes 2026-08-20 13:22:11 +08:00
Will Miao e57e11897e refactor(services): share weight-file extension set between rematch and find_matching_models 2026-08-19 21:54:22 +08:00
Will Miao 8a16034135 refactor(services): unify local model name matching with uniqueness and base-model guards (#1065)
Consolidate the duplicate name-matching logic into ModelScanner:
find_matching_models is now the single core, using each scanner's own
file_extensions for suffix stripping. get_model_info_by_name gains
require_unique/base_model kwargs while legacy route behavior is kept
byte-identical. reconnect_lora passes the recipe base model as a guard
and distinguishes ambiguous, base-model-mismatched, and missing LoRAs
in its error messages.
2026-08-19 21:02:04 +08:00
Will Miao 7fc3b7e5be docs: fix star history chart with official token-based embed (#1066) 2026-08-19 20:51:23 +08:00
Aaalice b0c7a1baae Fix recipe parsing for metadata-free local LoRAs (#1065)
* fix(recipes): resolve metadata-free local LoRAs

* fix(recipes): prioritize LoRA hashes over names
2026-08-19 19:07:17 +08:00
Will Miao 6411d83d46 fix(i18n): translate remaining untranslated UI strings 2026-08-19 18:59:55 +08:00
Will Miao 74a063b0e5 fix(i18n): complete translations for per-file download UI (#1058) 2026-08-19 18:53:31 +08:00
Will Miao 96376e5cce fix(download): hide URL step when file dialog opens from versions tab (#1058) 2026-08-19 18:35:16 +08:00
Will Miao e7c26bf722 feat(download): per-file download status and multi-file selection (#1058) 2026-08-19 17:51:31 +08:00
Will Miao cef4129fc9 fix(download): allow downloading additional files of an in-library model version (#1058) 2026-08-19 16:29:59 +08:00
Will Miao 0a28500848 fix(loaders): default control_after_generate to fixed on checkpoint/unet loaders
The previous boolean 'control_after_generate': true defaulted the control
widget to 'randomize', silently changing existing workflows into random
model selection on every queue. A string value sets the default mode, so
'fixed' preserves the prior behavior; users opt into randomization
explicitly.
2026-08-19 10:33:23 +08:00
Will Miao fc3f3f3bdb feat(loaders): add control_after_generate random model selection to checkpoint/unet loaders
The Checkpoint/Unet Loader (LoraManager) nodes now support ComfyUI's
built-in control_after_generate mechanism on the ckpt_name/unet_name combos,
letting users pick a random model on every queue with the selected model
written back into the widget (visible, and lockable via the 'fixed' mode).

A base_model input narrows the random pool: a front-end extension fetches
the name/base_model mapping from the new /api/lm/checkpoints/loader-pool
endpoint and filters the combo options, wired through the node callback,
the refreshComboInNodes extension hook, and a graph.onConfigure hook
installed from onAdded (onNodeCreated fires before the node is attached to
a graph, so the graph reference is unavailable there).
2026-08-19 05:13:51 +08:00
Will Miao fa58297973 fix(ui): stop media viewer Escape from closing underlying modal 2026-08-18 20:51:56 +08:00
Will Miao 5d1a22fb8f fix(ui): ignore internal card drags in model card preview drop (#1034)
Tag move-to-folder drags with a custom dataTransfer MIME type so card
preview-drop handlers skip them entirely (no highlight, no upload), and
mark the preview image non-draggable so the browser no longer synthesizes
a File payload when a drag starts on the image. Fixes card-on-card drops
and click-jitter self-drops replacing the preview with itself.
2026-08-18 20:38:29 +08:00
Will Miao d2f50f26f1 feat(ui): redesign model modal showcase as on-demand gallery 2026-08-18 20:38:29 +08:00
hein 4a6042d0b4 fix: include locally available LoRAs in recipe syntax even if deleted from Civitai (#948)
get_recipe_syntax_tokens() previously skipped all LoRAs with
isDeleted=True unconditionally. Now it tries to resolve the file
locally first (via hash index or modelVersionId); only skips if
the LoRA is truly unavailable.

This is a companion fix to #946 (AutoV2 hash matching).

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-08-18 15:10:53 +08:00
Will Miao 846206d958 fix(ui): add model modal backdrop blur to match recipe modal 2026-08-18 09:09:05 +08:00
Will Miao 0daf4924f0 feat(recipes): redesign recipe detail modal with three-column workspace layout
- Three-column layout (preview | generation parameters | resources) with
  independent per-pane scrolling and a content-sized modal shell that
  shrinks to fit short recipes and caps at viewport height for long ones
- Blurred, darker backdrop to focus attention on the modal
- Preview frame hugs the image instead of a fixed-size box
- Move recipe-level 'Send to ComfyUI' into the header actions row to match
  the model detail modal convention; remove the modal 'Copy Recipe Syntax'
  button (context menu action is unaffected)
- Add recipes.actions.sendRecipe i18n keys with translations
- Sync modal test fixtures to the new structure
2026-08-18 09:09:05 +08:00
willmiao d38a3d091d docs: auto-update supporters list in README 2026-08-16 11:47:29 +00:00
Will Miao 94dd08646d chore(release): bump version to v1.2.1 2026-08-16 19:47:15 +08:00
Will Miao 658f88ca48 feat(recipes): add toolbar toggle and settings preview for masonry layout 2026-08-16 15:23:30 +08:00
Will Miao f53352efb2 feat(metadata): collect generation params from Krea two/three stage samplers 2026-08-16 09:53:08 +08:00
Will Miao 38809a9d1b feat(recipes): add filename fallback tier to recipe rematch 2026-08-16 09:17:59 +08:00
Will Miao 395682509c feat(autocomplete): replace /af and /ac toggle abbreviations with full command names 2026-08-15 22:03:54 +08:00
Will Miao ef3e7d7bf4 feat(update): detect CivitAI paidAccess versions and add hide paid updates (#1060)
CivitAI's PaidAccess cutover deprecated the availability=EarlyAccess and
earlyAccessEndsAt signals; gated versions now report availability=Public
with a paidAccess DTO that LoRA Manager previously ignored, so "Hide
Early Access Updates" missed paid/early-access models and downloads
failed with 401.

Parse and persist paidAccess from model-level, bulk, and by-hash
responses; treat timed paid gates as early access and permanent paid
versions as a distinct is_paid state; add a hide_paid_updates setting
with a "Paid" badge in the versions tab; warn before downloading gated
versions. Includes SQLite migration, i18n for all locales, and
backend/frontend tests.
2026-08-15 18:08:14 +08:00
Will Miao c85b6b64a1 feat(recipes): add recently opened sort with modal open tracking
Track recipe modal opens in a separate stats file (never touching recipe
JSON/EXIF), expose a fire-and-forget POST endpoint, and add an 'opened'
sort that hides never-opened recipes as a true recently-opened view.
Includes i18n for all locales and backend/frontend tests.
2026-08-15 11:37:46 +08:00
Will Miao 34c87d4934 refactor(sort): extract seeded random sort helpers into SortDropdown 2026-08-15 09:53:28 +08:00
Will Miao 93472e5d67 feat(recipes): add sort by random option with seeded stable pagination 2026-08-15 09:50:46 +08:00
Will Miao ae185ee714 fix(loaders): correct random checkpoint loader return type annotation
load_checkpoint returns a 4-tuple (MODEL, CLIP, VAE, model_name) since the
random loader exposes the selected model name; the annotation still claimed
a 3-tuple.
2026-08-15 08:51:39 +08:00
Martial Michel 795036275a feat(loaders): add random model selection by base model to checkpoint/unet loaders
Add dedicated Random Checkpoint/Unet Loader (LoraManager) nodes that pick a random model from the indexed pool on every run, optionally filtered by base_model, and expose the selected model name via a STRING output.
2026-08-15 08:48:57 +08:00
Will Miao d43ab6e32f fix(vue-widgets): make text widget clear button undoable via Ctrl+Z (#1056) 2026-08-14 23:16:12 +08:00
Will Miao 280181f92e feat(metadata-overwrite): support wired SAMPLER input on sampler field
The sampler field now accepts either a manual string or a SAMPLER
connection. When wired, the sampler name is extracted from the
KSAMPLER object's sampler_function __name__ (sample_euler -> euler),
with special-casing for dpm_fast/dpm_adaptive local closures and
uni_pc/uni_pc_bh2 function names.

- sampler input declared as "STRING,SAMPLER" with widgetType STRING,
  mirroring the existing model field union pattern
- shared collect_overwrite_params() handles the non-str branch so the
  node and the metadata extractor conversion logic stay in sync;
  unrecognized sampler functions are logged and skipped
- note: ddim is constructed by ComfyUI as euler with random inpaint,
  so the ddim name is unrecoverable and extracts as euler
2026-08-14 15:21:28 +08:00
Will Miao f8d98934ad feat(ui): set preview via drag and drop on model cards (#1034) 2026-08-14 13:01:10 +08:00
Will Miao 303cca0d85 fix(download): accept newer CivitAI file types for primary file selection
Downloads failed with "No suitable file found in metadata" for models whose
only file uses newer CivitAI file types (e.g. 'Enhancement LoRA' for
Anima/AIR image-editing LoRAs) because the primary-file allowlist only
covered legacy types.

- unify the weights-type allowlist as MODEL_WEIGHT_FILE_TYPES
  (py/utils/constants.py) and apply it across download, recipe and
  metadata-refresh lookups
- mirror CivitAI's getPrimaryFile() semantics: prefer weights-type primary,
  fall back to weights files, then trust CivitAI's primary flag (excluding
  non-downloadable artifacts like Config/Archive/Workflow)
- mirror the allowlist in the frontend via shared isModelWeightFile() helper
- add regression tests for the Enhancement LoRA primary-file download,
  primary-flag fallback and weights-over-non-weights-primary preference
2026-08-12 21:14:23 +08:00
Will Miao c2f16784b3 fix(metadata): keep identity selectors from leaking unselected prompts 2026-08-12 19:44:43 +08:00
Will Miao 5bc6d8286c fix(metadata): exclude scalar fields from conditioning provenance inputs 2026-08-12 19:18:46 +08:00
Luna_K 3f8381ffee Fix prompt tracking through conditioning transforms 2026-08-12 19:16:30 +08:00
Will Miao 1ca99294c9 feat(delete): shorten undo window to 20s and make undo toast dismissible 2026-08-12 19:15:03 +08:00
Will Miao 680f0a57f5 fix(update): resolve template path when updating to a different base model (#1059)
Version-tab updates reused the current version's folder, so updating a LoRA
to a version with a different base model (e.g. Illustrious -> Anima) ignored
the download path template and landed in the old version's directory.

When the target version's base model differs from the current local version
and a path template is configured, re-resolve the template under the same
model root. The backend keeps an explicitly provided root when
use_save_dir_as_root is set, so regular downloads still use the default root.
2026-08-12 18:45:54 +08:00
Will Miao 94e3f54571 feat(workflow): exclude text-capable nodes with connected text from send targets
CLIP Text Encode and friends whose text widget is backed by a connected
input cannot have their text changed via the widget (execution reads the
linked input), so sending to them was a silent no-op.

- Registry: compute text_widget_connected capability from the widget's
  backing input link state; has_text_widget drops to false when wired;
  include the flag in the registration fingerprint so link changes
  re-register the affected nodes
- Registry: hook link connect/disconnect (graph events on new litegraph,
  onAfterChange fallback for classic) on root and subgraphs, plus
  subgraph-created for future subgraphs
- applyWidgetUpdate: skip inject_text when the target widget is connected
  and self-heal the registry instead of writing a value that is ignored
- Web UI: drop text_widget_connected nodes from prompt/embedding send
  candidates; show a Mark as -> Send Prompt Target hint toast when no
  candidates remain (new uiHelpers.workflow.noPromptTargets key, synced
  to all locales; zh-CN/zh-TW translated)
- Extract shared resolveTextWidget() used by both the candidate-set
  logic and the write path so the two cannot drift apart
- Tests: workflow registry connection-state registration, subgraph
  handling, fingerprint re-registration, inject_text write/skip paths,
  setup link-change hooks; uiHelpers candidate filtering and hint toast
2026-08-12 16:50:07 +08:00
Will Miao 5c2b2aedcc fix(i18n): complete recipe delete undo warning translations
Translate modals.deleteRecipe.recoverableWarning in all 9 non-English
locales (de, es, fr, he, ja, ko, ru, zh-CN, zh-TW)
2026-08-11 21:20:54 +08:00
Will Miao ebc31fb963 fix(i18n): translate recipe delete undo warning
Move the recipe delete modal's undo warning into modals.deleteRecipe.
recoverableWarning instead of hardcoded English; sync placeholders into
all 9 non-English locales
2026-08-11 21:18:36 +08:00
Will Miao 9659df6ad9 refactor(delete): make undo unconditional, remove undo toggle and button delay
- Remove delete_undo_enabled setting (backend default, frontend state,
  settings modal UI, 10 locales); staged deletes with 30s undo are now
  the only delete path and stale settings keys are silently ignored
- Remove the 1500ms delete-button arm delay (armDeleteButton) from all
  delete modals; misclicks are recoverable via the undo toast
- Delete modal always shows the recoverable warning
- Log the first staged file path in staging log lines for easier support
2026-08-11 21:15:59 +08:00
Will Miao 04d131e9dc docs(delete): correct same-volume guarantee after symlink fix 2026-08-11 19:01:44 +08:00
Will Miao 78fe6282c7 test(delete): symlink and restart regression for staged deletes 2026-08-11 19:00:28 +08:00
Will Miao 0c00ee22fc fix(delete): stage model deletes into the model folder (avoid EXDEV) 2026-08-11 18:48:03 +08:00
Will Miao 5fd4946b1f fix(delete): track staged batches in-process; reconcile at startup 2026-08-11 18:36:30 +08:00
Will Miao f1d3ac0cdc fix(metadata): fill local file facts when self-heal recreates sidecar
Refresh after manual .metadata.json deletion rebuilds the payload without
file_name/size/modified, which are required by BaseModelMetadata.from_dict.
The recreated sidecar then fails to parse and the scanner skips the model.

- load_metadata_payload fills missing file facts from os.stat
- hydrate_model_data restores every missing key from the cache snapshot
  only when the sidecar is missing entirely (disk stays authoritative
  otherwise), preferring the cached import timestamp for modified
- save_metadata fills file facts on write so no write path can produce
  an unparseable sidecar
2026-08-11 14:57:23 +08:00
Will Miao e2c45905f0 test(recipe): await background resort deterministically in pagination tests 2026-08-11 14:09:24 +08:00
Will Miao b2c68e6a65 feat(delete): add undo toasts and harden delete modals 2026-08-11 14:09:10 +08:00
Will Miao eb0f6dd3b6 feat(settings): add delete_undo_enabled toggle 2026-08-11 14:08:55 +08:00
Will Miao 0bf87f9092 chore(i18n): add undo-delete and delete-confirmation strings 2026-08-11 14:08:41 +08:00
Will Miao 1da2433bb2 feat(delete): add undo-delete endpoint and purge scheduling 2026-08-11 14:08:28 +08:00
Will Miao 2d6cf545b9 feat(delete): stage model and recipe deletes for 30s undo 2026-08-11 14:08:15 +08:00
Will Miao 6a259a14fa feat(nodes): flag missing local models at queue and load time (#1057) 2026-08-10 12:31:36 +08:00
Will Miao 41e1fd1e1f feat(download): expose aria2 disk write failure root cause at INFO level
Promote aria2 stderr lines that indicate disk write failures (e.g. the
'cause: No space left on device' line following 'Write disk cache flush
failure') from DEBUG to INFO so the root cause is visible in default logs,
including Windows-specific phrases (file locked by another process, sharing
violation). The same line is rate-limited to one INFO report per 60s window
and the report map is pruned on insert so repeated failures cannot spam the
log or grow memory. All other stderr output stays at DEBUG.
2026-08-10 09:45:13 +08:00
Will Miao 95fb3c7fc9 feat(recipes): add prompt-aware duplicate detection toggle 2026-08-10 00:07:14 +08:00
Will Miao 8237e5f9ea feat(ui): mark repair recipe data entries as deprecated
Add (Deprecated) label suffix to the three context menu entries for
repairing recipe data (global, bulk, single) ahead of their removal.
2026-08-09 15:50:02 +08:00
Will Miao aa75986178 fix(ui): hide context menu separator with no visible items 2026-08-09 15:44:10 +08:00
Will Miao b887922055 fix(i18n): translate rematch metadata strings 2026-08-09 15:33:24 +08:00
Will Miao 68fa0f29c7 feat(recipes): report rematch results with aggregate logs and toast feedback 2026-08-09 14:37:23 +08:00
Will Miao d9d362c9c9 fix(download): self-heal aria2 transfers lost on daemon restart 2026-08-09 12:48:36 +08:00
Will Miao d0bc4be0dc chore(skill): harden lora-manager-e2e for sandboxed E2E 2026-08-09 11:31:04 +08:00
Will Miao 420530f532 feat(ui): add global, bulk and per-recipe rematch actions 2026-08-09 11:31:00 +08:00
Will Miao 3001f0f0ef feat(recipes): add recipe rematch API endpoints 2026-08-09 11:30:50 +08:00
Will Miao b2a1307d23 feat(recipes): add recipe rematch WebSocket progress channel 2026-08-09 11:30:46 +08:00
Will Miao 64da845a58 feat(recipes): add local-only recipe rematch to scanner 2026-08-09 11:30:43 +08:00
Will Miao 27027c4497 refactor(recipes): reuse shared local hash cache in create-from-example 2026-08-08 22:45:29 +08:00
Will Miao 86c85c08ec feat(recipes): pass local hash cache to remote and url recipe imports 2026-08-08 22:13:19 +08:00
Will Miao 196c8ffc3e feat(recipes): match civitai image hash sections against local hash cache 2026-08-08 22:12:47 +08:00
Will Miao cfc95ee02a feat(recipes): pass local hash cache through analysis recipe parsing 2026-08-08 22:11:50 +08:00
Will Miao 479fa36997 feat(recipes): add version-cached local hash cache builder 2026-08-08 22:04:09 +08:00
Will Miao 3e1216e9bc feat(recipes): add cache version counter to model scanners 2026-08-08 21:57:36 +08:00
Will Miao 007883b7d1 fix(recipes): backfill lora cache item by autov2/autov3 hash too 2026-08-08 21:51:34 +08:00
Will Miao dc9200a12c fix(recipes): match recipe-format lora cache item by autov2/autov3 hash 2026-08-08 21:50:33 +08:00
Will Miao d2f955266d fix(types): resolve pre-existing basedpyright errors in tests
Fix ~790 basedpyright errors across the test suite:
- Type stub subclasses of real production classes with super().__init__()
- Add missing generic type arguments and Dict[str, Any] annotations
- Add None guards before subscript/member access
- Adapt tests to production API changes (removed dead handlers,
  PersistentModelCache.get_default, _i18n_filter_added location)
2026-08-08 20:12:59 +08:00
Will Miao 8e724538bd fix(types): resolve pre-existing basedpyright errors in py/ and standalone.py
Fix ~950 basedpyright errors across the backend:
- Convert ineffective # type: ignore comments to # pyright: ignore[rule]
- Add missing generic type arguments (Dict[str, Any], list[Any], ...)
- Annotate dynamic dict literals and runtime-initialized attributes
- Widen CivitAI provider tuple signatures in recipe parsers
- Remove dead LoraRoutes handlers calling nonexistent LoraService methods
- Suppress unavoidable ServiceRegistry import cycles (basedpyright counts
  function-local imports as cycle edges)
2026-08-08 20:12:52 +08:00
Will Miao 6fcdeb799d feat(metadata): resolve AutoV3 at download time without waiting for backfill
- Read AutoV3 directly from the downloaded file's own file_info hashes
  (no SHA256 cross-matching against version_info.files, so the value is
  captured even when the API omits SHA256)
- Extract normalize_autov3() validation helper shared with the
  sha256-matching autov3_from_civitai_files path
- Fall back to the embedded safetensors header hash at download
  completion; mark '' (checked-unavailable) so the startup backfill
  query (autov3 IS NULL) never revisits the row
- Clear archive-level AutoV3 for zip-extracted models so per-file
  header resolution applies to every extracted model
2026-08-08 15:20:43 +08:00
Will Miao 97b9b1f62b feat(metadata): add CivitAI AutoV3 hash support across all storage layers
- Three-state autov3 field (not-checked / checked-unavailable / 12-hex value)
  in .metadata.json sidecars, in-memory ModelHashIndex, and SQLite
  (models.autov3 column + autov3_index table) with column-presence migration
- Background self-terminating backfill for legacy rows: per-model-type
  concurrency guard, executor-offloaded I/O, Civitai-first resolution
  (SHA256-matched version file) falling back to the embedded safetensors
  header hash
- Civitai-first propagation on metadata refresh, scan, and download paths;
  reject the empty-string SHA256 placeholder and strip OneTrainer 0x prefix
- List API hash filters and hash index lookups accept 12-char AutoV3
- Cap safetensors header reads at 64 MiB to prevent crafted-file allocation
- Prevent stale AutoV3 mappings on file replacement while preserving them on
  same-file re-registration (lazy-hash completion)
2026-08-08 14:30:34 +08:00
Will Miao 4bf9a4b640 refactor(download): rename locationStep id to downloadLocationStep
The download modal's step shared the 'locationStep' id with the import
modal, so getElementById('locationStep') could resolve to the wrong
element depending on template include order. The import flow relied on
an injected display:block !important rule to work around it.

Rename the download modal's step id and update all references so each
modal owns a unique step id.
2026-08-08 08:50:36 +08:00
Will Miao c5088772e8 fix(ui): pin import modal action buttons with sticky footer
Make the import modal a flex column with a scrollable step area so the
Back/Import buttons stay visible on short viewports (1080p / 150% zoom)
instead of being cut off at the bottom of the scroll flow.

Also reset step scroll positions via class since 'locationStep' has a
duplicate id in the download modal template.
2026-08-08 08:49:20 +08:00
Will Miao 56acefbd6c feat(autocomplete): search loras within active filters of LoRA Manager page
Add /af and /noaf toggle commands (plus /activefilters aliases) to the
loras autocomplete widget. When enabled (default off), suggestions are
matched within the active filters (folder, base model, tags, auto-tags,
license, tag logic) persisted by the LoRA Manager page in localStorage,
keeping the match pool consistent with the list endpoint, including the
global show_only_sfw setting.

Backend: /lm/{prefix}/relative-paths accepts the filter query params and
pre-filters the scanner cache with ModelFilterSet. The presence of the
recursive param signals the filter pipeline to run even without concrete
filters so global settings stay in parity with the list endpoint.
2026-08-07 20:07:34 +08:00
Will Miao 5ab06c4aae docs: rename "Standalone Web UI" to "LoRA Manager Web UI" in AGENTS.md 2026-08-07 17:57:01 +08:00
Will Miao c11f4b5c68 feat(ui): widen filter panel and preset name limit 2026-08-07 16:53:14 +08:00
Will Miao 86376284f4 fix(ui): clamp filter panel height to viewport 2026-08-07 16:53:04 +08:00
Will Miao 2b8a2fc7d8 feat(filters): remove preset count limit 2026-08-07 16:52:53 +08:00
Will Miao f26e1b41c8 fix(i18n): translate zh-TW api key placeholder 2026-08-07 16:25:40 +08:00
Will Miao c1671af99f feat(downloads): translate batch download summary strings 2026-08-07 16:23:55 +08:00
Will Miao ac7707d0f6 fix(cards): clear model-card min-width on the item element itself 2026-08-07 16:09:51 +08:00
Will Miao 381cd710a2 feat(recipes): translate recipes layout setting strings 2026-08-07 15:36:30 +08:00
Will Miao ad0d18cb79 chore: ignore .playwright-mcp working directory 2026-08-07 15:33:43 +08:00
Will Miao 7980ee77d0 perf(recipes): batch preview dimension reads via asyncio.gather 2026-08-07 15:31:13 +08:00
Will Miao 916b8bb327 fix(recipes): skip stale scroller re-enable on deferred layout switch 2026-08-07 14:03:13 +08:00
Will Miao 87e3d4dea9 feat(recipes): wire recipes layout switch event and rebuild 2026-08-07 12:49:30 +08:00
Will Miao 76a913f5e0 feat(recipes): complete MasonryScroller public API parity with VirtualScroller 2026-08-07 12:40:28 +08:00
Will Miao d8c192e647 feat(recipes): branch masonry scroller instantiation for recipes page 2026-08-07 12:38:17 +08:00
Will Miao c453437620 feat(recipes): add MasonryScroller with column-based virtual scrolling 2026-08-07 12:31:13 +08:00
Will Miao 720fa6d909 feat(recipes): expose preview width/height in recipe listing API 2026-08-07 11:59:26 +08:00
Will Miao b4f71089f4 feat(recipes): add recipes_layout setting (grid|masonry) with i18n 2026-08-07 11:49:12 +08:00
Will Miao 83e6657ead feat(recipes): add get_image_dimensions helper with LRU cache 2026-08-07 11:47:28 +08:00
Will Miao 7ea6df4111 feat(downloads): default to latest version when URL lacks modelVersionId
Auto-select the first (newest) version for URLs without an explicit
modelVersionId, matching the existing batch flow, so users can proceed
to location/download without manually picking a version.
2026-08-07 11:24:45 +08:00
Will Miao d9ab92602a feat(downloads): show failure summary modal for single downloads too 2026-08-07 10:49:14 +08:00
pixelpaws 5ffadaed31 Merge pull request #1054 from willmiao/feat/gemini-provider
feat(llm): add Gemini as a preset AI provider
2026-08-07 10:30:45 +08:00
Will Miao 24f5f7df5d feat(llm): add Gemini as a preset AI provider 2026-08-07 10:27:51 +08:00
Will Miao daf01fb1d6 feat(downloads): show batch download summary with failure details and retry 2026-08-07 10:23:17 +08:00
Will Miao 0f11b6def9 fix(recipes): allow recipes storage path on a different drive (Windows)
os.path.commonpath raises ValueError for paths on different Windows
drives. Treat that as no common root so cross-drive recipes migrations
succeed instead of failing with 'Invalid recipes path change'.
2026-08-06 22:18:24 +08:00
Will Miao 7df83f44b8 feat(SaveImageLM): add add_loras_to_prompt toggle to restore legacy lora syntax line in metadata 2026-08-06 15:58:18 +08:00
Will Miao 169fa7bed6 fix(vue-widgets): resolve pre-existing typecheck errors 2026-08-06 15:33:02 +08:00
Will Miao 027b504fe8 refactor(autocomplete): remove unused custom_words and embeddings modelTypes 2026-08-06 15:28:58 +08:00
Will Miao 186ef4da78 refactor(ui): group example image download actions into a submenu
Move the 'Download Missing' / 'Re-process All' example image actions
under a single 'Download Example Images' submenu item in the single-model
and bulk context menus, matching the existing send-to-workflow submenu
pattern. Shorten the submenu labels and update all locale translations.
2026-08-03 21:18:05 +08:00
pixelpaws dc674098e7 Merge pull request #1050 from willmiao/fix/recipes-bulk-content-rating
fix(recipes): enable bulk content rating for selected recipes
2026-08-03 20:58:24 +08:00
Will Miao 9087b4b07c feat(example-images): add missing-only download path and skip existing files
Split the single-model and bulk context menu actions into 'Download
Missing Example Images' (regular endpoint, skips already-processed
models) and 'Re-process Example Images' (force endpoint, retries
failed models).

- start_download accepts model_hashes so a selected subset can be
  processed with the progress-aware skip logic; explicitly targeted
  models bypass the failed/processed model-level guards so per-image
  gaps are filled
- pre-download existence check in the processor skips network requests
  for image files already on disk across all download paths
- force download retries previously failed models and clears their
  failed status on success
- add i18n keys for the new menu items across all locales
2026-08-03 20:52:46 +08:00
Will Miao 8e45c22d7a fix(recipes): enable bulk content rating for selected recipes 2026-08-03 19:31:58 +08:00
Will Miao 191c4e03cd feat(metadata-overwrite): support wired MODEL input on model field
The model field now accepts either a manual string or a MODEL connection.
When wired, the model name is extracted from the patcher's
cached_patcher_init (registered by core loaders load_checkpoint_guess_config
and load_diffusion_model, preserved through LoRA clones) and converted to a
ComfyUI-style relative name via config model roots.

- model input declared as "STRING,MODEL" with widgetType STRING, so the
  text widget and the dual-type connection slot coexist; non-STRING/MODEL
  links are rejected by frontend and backend type validation
- UNETLoaderLM GGUF branch now registers a custom cached_patcher_init reload
  factory so GGUF models participate in name extraction and ModelPatcher
  deepclone/dynamic machinery
- shared collect_overwrite_params() helper keeps the node and the metadata
  extractor conversion logic in sync; extraction failures are logged instead
  of silently dropping the overwrite
2026-08-03 16:44:03 +08:00
Will Miao ab4154c57d feat(ui): add seeded random sort option to model pages (#1049) 2026-08-03 15:02:49 +08:00
Will Miao 28e93d12ff fix(example-images): use in-place cache sync and bulk pending-check index for large libraries 2026-08-03 12:04:56 +08:00
Will Miao 75e63c758b feat(api): add cursor-based pagination to civitai user-models endpoint 2026-08-03 11:07:06 +08:00
Will Miao 823f71f269 feat(nodes): make Lora Stack Combiner inputs dynamic 2026-08-02 22:04:40 +08:00
Will Miao 042dd4088d fix(nodes): make Lora Stack Combiner inputs optional 2026-08-01 17:14:00 +08:00
willmiao eaa791a9eb docs: auto-update supporters list in README 2026-07-31 13:25:56 +00:00
Will Miao 2228627ff4 chore(release): bump version to v1.2.0 2026-07-31 21:25:38 +08:00
Will Miao 4c647ad9c8 fix(update): throttle nightly update badge to once per day 2026-07-31 21:18:58 +08:00
Will Miao 8ca3e6c33f fix(ui): guard marquee bulk-mode entry against click jitter and stale drag state 2026-07-31 18:40:14 +08:00
Will Miao dd6bdbf297 fix(update): persist update_channel via settings.json instead of hasGit
After b464fdc3 (preserve .git on release switch), the hasGit-based
channel detection is unreliable — .git now exists for both release
and nightly installs, so page refresh always reset the channel.

- Add _resolveChannelFromSettings() with migration heuristic:
  !hasGit → release (ZIP), detached HEAD → release (on tag),
  on branch → nightly. Uses gitInfo.branch from check-updates.
- Persist resolved channel to settings.json on first load
  (one-time migration) and on explicit switchChannel.
- Add update_channel validation (release|nightly) in backend
  update_settings handler.
- Remove hasGit-based guessing from initialize(); defer to
  checkForUpdates where full gitInfo is available.
- Channel resolution runs before checkForUpdates early-returns
  to avoid null channelMode on reload-within-interval.

Tests: 361 passed.
2026-07-31 13:23:54 +08:00
Will Miao b47dde87e4 fix(settings): suppress error toasts when optional model roots are empty 2026-07-31 10:07:52 +08:00
Will Miao 99e65cccd8 fix(update): downgrade settings backup/restore logs from INFO to DEBUG 2026-07-30 20:32:00 +08:00
Will Miao 3bdacb8f46 fix(test): update release channel git test to mock _perform_git_update instead of _download_and_replace_zip 2026-07-30 18:35:43 +08:00
Will Miao b4f9c224d3 fix(example-images): move multi→single-library consolidation to startup, eliminate per-request os.listdir()
Move reverse-migration logic from get_model_folder() (hot path, called on
every metadata/example-images request) to ExampleImagesMigration, where it
runs once at startup.  On network storage this was causing 22-38s delays
per LoRA card click.

Additionally optimize prune_stale_example_images() to read the directory
listing once instead of per image entry (O(N*M) → O(M)).  Also reorder
consolidation checks so regex filters run before filesystem stat calls.
2026-07-30 18:11:40 +08:00
Will Miao 5ec0399c81 fix(i18n): remove redundant 'preserved' sentence from release channel message, sync all 10 locales 2026-07-30 16:38:04 +08:00
Will Miao b464fdc333 fix(update): preserve .git on release channel switch, use git checkout tag
Previously, switching to the release channel would delete .git/ and
fall back to a ZIP download. This broke update.bat, manual git
commands, and CM git-based update detection.

Now the release path uses git checkout <latest-tag> when .git exists,
and only falls back to ZIP when .git is absent (CM CNR installs).
.git is never deleted - the ZIP→nightly path remains a one-way
upgrade via _init_git_repo.

Also updates locale strings (en, zh-CN, zh-TW, ja) to remove the
now-inaccurate "remove the Git repository" wording.
2026-07-29 21:23:39 +08:00
Will Miao 53825500db fix(update): add staging protection to switch_channel
switch_channel has three destructive code paths (git reset + clean,
git init + checkout --force, and rmtree + ZIP replace) that were
missing the _stage_preserved_items / _restore_preserved_items safety
net already applied to perform_update.

Wrap the channel-specific logic in a try/finally so preserved user
data (settings.json, civitai/, cache/, etc.) is physically moved
outside plugin_root before any git operation and always restored.
2026-07-29 20:41:36 +08:00
Will Miao f2ac790752 fix(update): stage preserved items outside repo before git/ZIP update
Move settings.json, civitai/, wildcards/, backups/, stats/, logs/,
cache/, and model_cache/ to a temp directory before git reset/clean
or ZIP replacement, then restore them in a try/finally block.

This prevents data loss on Windows where git clean -e exclusion
patterns can fail due to path-separator mismatches or where file
locks (open SQLite/log handles) cause the restore step to be skipped
on failure.

Also unifies three hardcoded skip lists (_clean_plugin_folder,
skip_items, skip_tracked) to derive from the single _PRESERVE_DIRS
constant, fixing drift where logs/ was missing from the ZIP path.
2026-07-29 19:49:50 +08:00
Will Miao 0d8805cdee fix(recipes): update cards in-place after LoRA download, preventing scroll reset 2026-07-29 11:35:28 +08:00
pixelpaws 656e24ac9b Merge pull request #1044 from d1udiu/fix-filter
fix(filters): prevent search query from being persisted in localStorage
2026-07-29 11:30:40 +08:00
d1udiu 6718b37403 fix(filters): prevent search query from being persisted in localStorage 2026-07-29 10:12:42 +08:00
Will Miao c9e5e784fc fix(metadata-overwrite): use sentinel default for clip_skip to accept wired 0 2026-07-28 23:13:00 +08:00
Will Miao f92f958682 fix(SaveImageLM): correct scheduler mapping and deduplicate sampler map
- Fix incorrect mapping: "normal" -> "Normal" (was "Simple")
- Replace inline sampler_mapping with CIVITAI_SAMPLER_MAP reference
  to eliminate duplicate definition
2026-07-28 21:39:09 +08:00
Will Miao f63fab0676 fix(cache): deduplicate model entries on add and reconcile to prevent duplicate cards (#1041) 2026-07-28 20:44:57 +08:00
Will Miao cfc4903c0c fix(update): read ahead_by from GitHub compare API when status is ahead/diverged
The compare API URL format compare/{local_hash}...main returns
status='ahead' when main is ahead of the local commit. The count is
in the ahead_by field, not behind_by. The old code only read behind_by
which is always 0 in this case, causing the UI to show 'Up to date'
when actually several commits behind.

Also handle status='diverged' (both sides have unique commits) by
reading ahead_by for the remote-ahead count.

Frontend adds a hash comparison fallback: if behind_by is 0 but local
and remote commit hashes differ, show 'Behind main' instead of the
incorrect 'Up to date'.

Tests: _AheadCompareDownloader and _DivergedCompareDownloader mocks
for the two status paths.
2026-07-28 17:47:38 +08:00
Will Miao a527a847fe fix(download): route UNet/diffusion model downloads to unet roots in location step
When downloading a diffusion model (UNet) from the checkpoints page, the
download modal's location step always showed checkpoint roots and paths.
Now the modal detects the file subtype and switches to unet_roots endpoint,
default_unet_root key, and 'unet' path template.
2026-07-28 17:21:12 +08:00
Will Miao 91b0bf8933 fix(download_queue): deduplicate download_history rows before creating unique index (#1041) 2026-07-27 21:36:58 +08:00
Will Miao 66d1c96783 feat(update): add Release/Nightly channel switching
- Add POST /api/lm/switch-channel endpoint with git init / ZIP fallback
- Add _backup_git/_restore_git helpers with safe rollback
- Version-info endpoint now returns has_git flag for auto-detection
- Check-updates always returns releases (changelog) regardless of channel
- Nightly mode shows 'N commits behind main' with commit hash and date
- View on GitHub link points to /commits/main in nightly mode
- Channel toggle UI with pill-style buttons in update modal
- Confirmation dialog with Esc / backdrop-dismiss support
- Channel derived from has_git on every page load, no localStorage
- i18n: 11 new keys translated across 9 non-English locales
- CSS: unified card-style sections in _base.css
- Tests: 8 new tests covering switch-channel, nightly response, init_git_repo
2026-07-27 20:27:05 +08:00
Will Miao 986128076e fix(widget): guard setValue against non-array input to prevent workflow load crash (#1039) 2026-07-26 21:54:37 +08:00
Will Miao 1de0a53241 feat(grouping): version-group library cards by HuggingFace repo for non-Civitai sources (#1040) 2026-07-26 21:46:55 +08:00
Will Miao 0ec7eaf606 fix(wildcards): resolve weighted N::value syntax inside wildcard YAML lists (#1039) 2026-07-26 18:33:31 +08:00
Will Miao d9fcb0e92b fix(filter): preserve search term through filter apply/clear operations 2026-07-26 16:49:57 +08:00
Will Miao f49b4ba4db fix(metadata-overwrite): rename 'checkpoint' input to 'model' 2026-07-26 10:59:38 +08:00
Will Miao 84e708328b fix: correct return_types propagation to GenericNodeExtractor
Two bugs prevented type-signature-based fallback from working:

- metadata_hook.py used getattr(obj.__class__, 'RETURN_TYPES')
  which fails when _async_map_node_over_list is called with
  a class (not instance) — obj.__class__ is the metaclass
  'type', which has no RETURN_TYPES. Fixed: getattr(obj, ...).

- metadata_registry.py used type(extractor) is GenericNodeExtractor
  to dispatch return_types. NODE_EXTRACTORS stores class
  references, not instances; type(Class) is always 'type',
  never the class. Fixed: extractor is GenericNodeExtractor.
2026-07-26 10:34:20 +08:00
Will Miao 125bed3f09 feat: add Metadata Overwrite node for manual generation params override 2026-07-26 08:50:21 +08:00
Will Miao 077e70169d feat: add type-signature-based fallback for unregistered nodes
GenericNodeExtractor (previously a no-op) now inspects
RETURN_TYPES to detect MODEL loaders and CONDITIONING
encoders in nodes not registered in NODE_EXTRACTORS.

- Propagate return_types from the hook layer through the
  registry to GenericNodeExtractor.extract() and update().
- MODEL detection: scan ckpt_name/unet_name/model_path/
  model_name/gguf_name fields, validate by extension.
- CONDITIONING detection: scan text/clip_l/t5xxl/prompt
  fields, store prompt text and conditioning tensor.
- _fill_missing_metadata also checks node_cache, so
  GenericNodeExtractor-handled nodes survive cache.
2026-07-25 22:14:51 +08:00
Will Miao e6dc169a05 feat: add meta hints user marks for metadata heuristic override
Users can now right-click nodes and assign meta hints
(primary_model, primary_sampler, positive_prompt,
negative_prompt) to override the metadata processor's
heuristic inference.

- Store extra_data from the API request so workflow node
  properties (including lm_marker_role) are accessible
  during metadata processing.
- _get_user_marks scans extra_data.extra_pnginfo.workflow
  for meta_* marks, falling back to prompt.original_prompt.
- extract_generation_params checks user marks before
  heuristic inference for sampler, model, and prompts.
- Warn on duplicate marks or invalid marked nodes.
2026-07-25 22:13:52 +08:00
Will Miao f34c02756d fix(recipes): eliminate O(n) fuzzy search fallback over 42k+ recipes
Drop the SequenceMatcher-based fuzzy_match fallback that froze the server
when FTS returned empty results. FTS now returns empty set for zero results
(no fallback), and when the index is not yet ready, search returns empty
rather than scanning all items in Python.
2026-07-25 17:34:37 +08:00
Will Miao 1e4c315481 fix(ModelModal): respect civitai_host setting for creator profile link 2026-07-25 07:15:12 +08:00
Will Miao a8283a0d00 fix(SaveImageLM): clarify embed_workflow tooltip — explains drag-and-drop workflow restoration
The previous tooltip was misleading: users thought workflow embedding was
automatic. New wording explains this opt-in flag stores the complete
workflow inside images, allowing one-click restoration via drag-and-drop.
PNG and WebP only.
2026-07-24 19:53:59 +08:00
Will Miao 55896669fc feat(SaveImageLM): expose webp_method and jpeg_subsampling as conditional node inputs
Add two new optional parameters to the Save Image node:

- webp_method (INT, 0-6, default 6): Controls WebP compression level.
  0=fastest/largest, 6=slowest/smallest. Previously hardcoded to 0.
- jpeg_subsampling (INT, 0-2, default 0): Controls JPEG chroma
  subsampling. 0=4:4:4 (best quality), 1=4:2:2, 2=4:2:0.

Frontend JS extension hides/disables each parameter when the
selected file_format doesn't apply (e.g., webp_method is hidden
when saving as PNG or JPEG). 7 new tests cover parameter plumbing
and default consistency across INPUT_TYPES, save_images(), and
process_image().
2026-07-24 19:32:51 +08:00
Will Miao e341e0b9d2 fix(test): update parameters assertion to include Version: ComfyUI after metadata format upgrade 2026-07-24 18:29:07 +08:00
Will Miao e6538c83bb fix(metadata): restore sha256 after hydrate_model_data to prevent KeyError in CivitAI fetch
hydrate_model_data replaces model_data with .metadata.json content which
may lack sha256 (corrupted file, concurrent write, etc.). Restore the
cached sha256 after hydration and persist the fix back to disk so
subsequent lookups don't hit the same error.

Also improve error log to include file_path for debugging.
2026-07-24 12:07:18 +08:00
Will Miao 92e1285ea5 feat(SaveImageLM): upgrade metadata output to A1111/Civitai-compatible format
- Replace plain-text Lora hashes with Hashes JSON dict matching A1111 convention
- Add Civitai resources JSON array with AIR URNs for direct model version linking
- Add Clip skip, Version: ComfyUI fields to generation params line
- Build AIR strings from local scanner cache (no API calls needed)
- Add complete sampler name mapping (CIVITAI_SAMPLER_MAP) and base model → AIR slug mapping (BASE_MODEL_AIR_SLUG) sourced from civitai ecosystem constants
- Remove lora text prepending from prompt line; LoRA info now in structured JSON sections
2026-07-24 06:20:28 +08:00
Will Miao 2aabd1d90e fix(ai): use json_schema instead of json_object for broader provider compatibility (#1033)
LM Studio and some other OpenAI-compatible servers reject
response_format=json_object but accept json_schema. Switch to the
equivalent json_schema format and add a fallback that retries
without response_format when the provider rejects the format type.
2026-07-23 09:17:29 +08:00
Will Miao 7b8b778f83 fix(widget): restore strength drag on lora entries and header
widget.value is a getter/setter that returns a new array on every read,
so handleStrengthDrag with updateWidget=false mutated a discarded copy.
Introduce __dragActive flag to suppress renderLoras in setValue during
drag, allowing mutations to persist through widget.value without
destroying the DOM. Use try-finally to guarantee flag cleanup.
2026-07-23 08:31:34 +08:00
Will Miao 7c8dc57d55 fix(security): use abspath instead of realpath in containment checks to support symlinks (#1028) 2026-07-23 07:06:41 +08:00
Will Miao fe95fae5f2 fix(workflow): include Create Hook LoRA in lora_code_update handler 2026-07-22 11:40:56 +08:00
Will Miao ce8a95abf7 chore(release): bump version to v1.1.9 2026-07-21 22:22:39 +08:00
Will Miao c8e7e543d6 fix(api): remove overstrict model type validation in getApiEndpoints
The validation in getApiEndpoints threw for page types not in
MODEL_TYPES (e.g. 'recipes'), crashing the recipes page initialization
when FilterManager calls it via createBaseModelTags(). The throw was
synchronous and outside the fetch().catch() chain, causing an uncaught
promise rejection that aborted the entire app initialization.

getApiEndpoints is a URL builder -- validation belongs to callers that
need strict type checking (they already use isValidModelType()). For
non-model-type pages like recipes, the generated URLs are correct
(the backend does have /api/lm/recipes/* routes).

Fixes regression from f53f859a (feat(filter): add debounced tag search).
2026-07-21 22:09:30 +08:00
Will Miao a9dbb15ffa fix(create_hook_lora): lazy import comfy.hooks/comfy.utils to fix CI pipeline (#744) 2026-07-21 18:44:04 +08:00
Will Miao cf64043f7d fix(security): add library root containment check for delete/move/rename operations (#1028) 2026-07-21 15:23:38 +08:00
Will Miao ccaff92c18 fix(nodes): register Create Hook LoRA node in workflow target registries 2026-07-21 14:56:39 +08:00
Will Miao 585b5c922a feat(nodes): add Create Hook LoRA (LoraManager) node for multi-LoRA hook pipelines 2026-07-21 09:44:28 +08:00
Will Miao ea80c2224c fix(download): prevent path traversal in download template resolution (#1028) 2026-07-20 21:08:43 +08:00
willmiao 8b0f56c1a6 docs: auto-update supporters list in README 2026-07-20 12:42:20 +00:00
Will Miao 8022d12f03 chore(release): bump version to v1.1.8 2026-07-20 20:42:04 +08:00
Will Miao 3939f7f91b chore: Update lora manager basic example workflow 2026-07-20 20:20:35 +08:00
Will Miao aebf2e37dd fix(filter): apply preset on full tile click and suppress i18n double-translate warnings
- Move preset apply handler from span.preset-name to div.filter-preset so
  clicking anywhere on the tile triggers the preset, not just the label text.
- Add whitespace heuristic in showToast() to skip translate() for plain
  messages that are already translated at the call site. This prevents
  i18next from logging 'Translation key not found' for pre-translated
  strings like 'Preset "name" applied'.
2026-07-20 17:57:14 +08:00
Will Miao f53f859a71 feat(filter): add debounced tag search with backend search-tags endpoint 2026-07-20 17:37:47 +08:00
Will Miao d916375abe fix(checkpoint): populate hash index from pre-computed metadata to prevent repeated hash re-calculation (#1002) 2026-07-20 12:24:54 +08:00
Will Miao 57983df4bd fix(recipe): resolve recipe metadata update bugs in cache sort, allowed fields, and bulk API routing
- Use safe .get() in RecipeCache._resort_locked instead of itemgetter to prevent KeyError when recipe missing created_date; align sort key with _sort_cache_sync (prefer modified, fallback created_date, fallback 0)
- Add base_model to allowed_fields in persistence_service.update_recipe() so the field passes validation
- Route bulk base model updates through updateRecipeMetadata() on recipes page instead of generic saveModelMetadata(), matching existing isRecipesPage pattern used in setBulkFavorites and saveBulkTags
2026-07-20 11:20:06 +08:00
Will Miao c68d7559a0 fix(widget): correct reorder drop indicator position when container is scrolled
The drop indicator top position was calculated using only
getBoundingClientRect() offsets (post-CSS-transform viewport space)
without accounting for container.scrollTop (pre-transform layout space).
This caused the indicator to drift upward as the user scrolled down,
eventually disappearing entirely.

Fixed by adding container.scrollTop to the position calculation and
only dividing the GBCR visual-diff portion by scale, since scrollTop
is already in pre-transform coordinate space.
2026-07-19 22:40:07 +08:00
Will Miao 9a8f5bf2d6 fix(ui): reposition download settings before AI provider section 2026-07-19 17:57:02 +08:00
Will Miao a8d742b031 feat(metadata): add CivArchive API toggle and provider fallback order settings
- Add enable_civarchive_api toggle (default on) to allow disabling
  CivArchive to avoid its rate-limit windows entirely
- Add metadata_provider_order dropdown with two presets:
  CivitAI → CivArchive → Archive DB (default) and
  CivitAI → Archive DB → CivArchive
- Wire both settings through backend (metadata_service, settings_manager,
  misc_handlers) and frontend (SettingsManager, state, settings modal)
- Reorder Metadata section in settings modal: toggles → status/management
  → fallback order, for natural top-down workflow
- Make update_metadata_providers() log the effective provider chain
  using actually-registered providers rather than settings assumptions
- Add 5 test cases covering all provider-combination paths
- Complete i18n translations for 6 new keys across all 9 non-English locales
2026-07-19 17:51:50 +08:00
Will Miao c27e4d1bfc feat(cache): opportunistic cache sync on metadata read with in-place update
- Add PersistentModelCache.update_single_model() for lightweight targeted
  SQL update (single row + incremental tag/hash deltas, no full table scan)
- Add ModelScanner.sync_cache_from_metadata() with compare-first logic:
  skips entirely when cache is already in sync; when stale, updates the
  entry in-place (O(1) instead of O(n) remove+append), incrementally
  adjusts tag counts/hash index/version index, and resorts only when
  sort-relevant fields changed
- Wire sync_cache_from_metadata() into BaseModelService.get_model_metadata()
  via fire-and-forget asyncio.create_task — disk I/O is already paid for
- Include identity re-validation guard against concurrent cache replacement
- Add 16 tests covering _cache_entries_differ, sync_cache_from_metadata
  (no-change, in-place, fallback, conditional resort), and
  update_single_model (insert, tag delta, hash delta)
2026-07-19 08:32:21 +08:00
Will Miao d15a8aa9a2 fix(workflow): accept non-string widget values and support GlobalSeed node in gen-params (#1026) 2026-07-18 22:57:20 +08:00
Will Miao 74a7d12ca4 fix(test): add missing options mock in LoraInfoWidget test 2026-07-18 22:14:03 +08:00
Will Miao 2f94a9773e feat(workflow): redirect gen-params updates to connected Primitive nodes (#1026)
When a KSampler marked as 'Send Gen Params Target' has widget inputs
wired to Primitive nodes (PrimitiveNode, PrimitiveInt, PrimitiveFloat,
etc.), sending gen params from the Lora Manager UI now updates the
Primitive node's value instead of the KSampler widget. This is
necessary because ComfyUI's execution engine reads from the connected
input, ignoring the widget value when a wire is present.

Also fix two minor issues found during review:
- Remove unnecessary String() wrapping on numeric gen params (seed,
  steps, cfg) to preserve native types through the JSON/WS path
- Correct misleading isNodeEnabled comment: LGraphEventMode values
  are 0=Always, 2=Never, 4=Bypass (not 'Normal/Enabled')
2026-07-18 22:10:48 +08:00
Will Miao 37bdfa21ea fix(standalone): ensure sys.path includes script dir for python_embeded compatibility (#1025) 2026-07-18 21:22:25 +08:00
Will Miao f0bf2728c9 fix(downloads): accept download_id in history delete/retry endpoints, add unique index 2026-07-18 21:10:42 +08:00
Will Miao dc715aa273 fix(download): fallback to downloadUrl when all mirrors are deleted
When Civitai returns 404 for /models/{id} (e.g. due to Civitai API bug
where un-deleted models still get 404), the fallback to CivArchive
provides metadata.  However CivArchive may return mirrors with every
entry marked deletedAt, while the file's downloadUrl is still valid.

Before this fix, _build_download_urls_from_file_info used an if/else
that skipped the downloadUrl fallback whenever the mirrors array was
non-empty, even when all mirrors were filtered out.  Now downloadUrl
is always tried when no usable mirror remains.

Also deduplicated the inline mirror-processing code at the second call
site by replacing it with a call to the shared helper.
2026-07-18 18:28:32 +08:00
Will Miao 7ee2361e87 fix(config): remove stale 'default' library entry and consolidate example images on startup 2026-07-18 17:25:36 +08:00
Will Miao e04c22f83f fix(widgets): allow text selection in LoraInfoWidget description tab 2026-07-17 18:33:59 +08:00
Will Miao 681cc13e90 fix(widgets): persist LoRA entry selection and active tab across save/load 2026-07-17 18:27:34 +08:00
Will Miao 090e0297d4 fix(downloader): hold session lock in retry paths to prevent session close race
Refactor _create_session() to make-before-break: snapshot old session,
assign new one first, then close old.  Previously, concurrent download
retries called _create_session() without the session lock (violating its
docstring contract) and closed the old session while other coroutines
held active references — causing aiohttp to raise "NoneType has no
attribute connect" when dereferencing the torn-down connector.

Also wrap the two _create_session() calls in the integrity-retry and
network-retry paths with self._session_lock to match the locking
discipline used by the session property and refresh_session().
2026-07-17 17:21:05 +08:00
Will Miao 6f71335be4 feat(widgets): add Description tab to LoraInfoWidget with dual-mode rendering support
- Add Notes/Description tab switching with tab state persistence in widget value
- Lazy-load model description and version description from /lm/loras/metadata
- Render CivitAI HTML descriptions inline via v-html
- Auto-fetch description when LoRA selection changes while on Description tab
- Fix Vue mode height containment via contain:layout size (lm-vue-node class)
- Fix scroll wheel isolation: widget scroll vs canvas zoom in both render modes
- Add docs/comfyui-dual-mode-widgets.md with widget rendering patterns
2026-07-17 15:04:47 +08:00
Will Miao 7f51812c1e feat(nodes): add LoRA Syntax → Path node (#1015) 2026-07-16 19:57:29 +08:00
Will Miao a9dc4d7b9d fix(widgets): reuse orphaned DOM containers after undo/redo in Vue render mode
In ComfyUI Vue render mode, WidgetDOM.vue reuses its component instance
during undo/redo without re-calling mountWidgetElement(), leaving newly
created widget containers detached from the DOM.

- AutocompleteTextWidget: scan for empty containers by ID prefix and reuse
- Loras widget: scan for empty .lm-loras-container elements and reuse
- Prevent duplicate event listeners by guarding listener setup on new
  containers only
- Keep container in DOM on cleanup (clearChildren instead of remove)
  so it can be found and reused by the next factory invocation
2026-07-16 18:54:00 +08:00
Will Miao 5d50ddb5d4 fix(ui): exit bulk mode after send-to-workflow completes 2026-07-16 18:54:00 +08:00
Will Miao f86198d234 fix(loras): include folder prefix in context menu and bulk send-to-workflow
When using full path lora syntax, the context menu (single/bulk)
and bulk copy actions were passing only the file basename to
buildLoraSyntax(), ignoring the folder prefix. This caused the
output to look like legacy A1111 format even when full path mode
was enabled.

Aligns all entry points with ModelCard.handleSendToWorkflow(),
which correctly includes the folder prefix.

Also fixes selectAllVisibleModels() to cache the folder field,
preventing missing prefix on select-all-then-send flows.
2026-07-16 18:54:00 +08:00
Will Miao ffe65d983c feat(api): add GET endpoints for update-lora-code and update-node-widget
Add GET variants of the two POST endpoints used by the send-to-workflow
feature. Parameters are read from query string instead of JSON body,
supporting both simple repeated node_id params and JSON-encoded node_ids
for complex graph references.
2026-07-15 21:49:22 +08:00
Will Miao b0b5be913c fix(downloads): reject re-insertion of download_ids already in history
In add_to_queue, check download_history before INSERT OR IGNORE.  Without
this check, a fire-and-forget /queue/complete failure on the extension side
would allow the same download_id to be re-inserted after complete_download()
deleted it from the queue — creating phantom queued entries for already-
finished downloads.
2026-07-15 19:12:22 +08:00
Will Miao 01efcbc584 fix(loras): allow toggle deselect on LoRA entry click 2026-07-14 18:23:06 +08:00
Will Miao 02c249917a fix(recipe): ensure custom recipes_path is added to preview allowed roots on startup 2026-07-14 18:15:20 +08:00
Will Miao 419bbc90b2 feat(lora-info): add Lora Info display node
Add a pure frontend node that shows filename and editable notes for
a selected LoRA. Connect any output from a LoRA Loader/Stacker/Randomizer/
WanVideoSelect to the lora_source input — selecting a LoRA in the source
widget updates the info display automatically.

- Python node (LoraInfoLM): display-only, no workflow execution
- Vue widget: filename label, auto-sizing notes textarea, save button
  with ComfyUI toast feedback on save
- Frontend extension: wire-based selection propagation with stale-response
  race guard; clears display on wire disconnect
- Backend: get-notes endpoint now returns file_path alongside notes;
  matching supports full-path lora syntax; fix NoneType crash in
  trigger words endpoint; document cache file_name invariant
- Wired into all four lora widget nodes (Loader, Stacker, Randomizer,
  WanVideoSelect)
2026-07-14 18:00:31 +08:00
willmiao b0c4510fdb docs: auto-update supporters list in README 2026-07-13 14:18:36 +00:00
Will Miao bf6a614e0d chore(release): bump version to v1.1.7 2026-07-13 22:18:16 +08:00
Will Miao feab01cd9c fix(preview): hide license icons for models without CivitAI metadata 2026-07-13 19:49:10 +08:00
Will Miao 966024e534 fix(registry): force re-registration on WS refresh to prevent timeout, demote empty-registry log to debug
- workflow_registry.js: add force param to refreshRegistry(), bypass fingerprint
  dedup when responding to lora_registry_refresh WS message. Without this, the
  backend's wait_for_all() times out after 0.5s because the frontend skips the
  register-nodes POST when the workflow fingerprint hasn't changed (common after
  ComfyUI restart with an empty or unchanged workflow).
- misc_handlers.py: demote 'No nodes registered after refresh' from WARNING to
  DEBUG — empty workflows are a normal operational state, not a warning-worthy
  condition.
2026-07-13 19:10:48 +08:00
Will Miao 2018722cc8 fix(registry): handle compound subgraph node IDs, add proactive node push from graph hooks
- Handle compound node IDs (e.g. "252:0") from expanded group subgraphs
  to fix 400 Bad Request on workflows with group nodes
- Frontend proactively pushes node data via afterConfigureGraph and
  LiteGraph hooks (onNodeAdded/onNodeRemoved/graphChanged), eliminating
  WebSocket round-trip latency for most "Send to Workflow" operations
- Add content-fingerprint dedup to skip duplicate register-nodes POSTs
- Fast-path cache returns immediately when tabs are registered (including
  0-node registrations), avoiding unnecessary WS refresh cycles
- Distinguish "Empty Registry" from other errors in standalone UI toast
- Reduce WS refresh timeout 2s→0.5s, add cooldown and lock to prevent
  concurrent refresh storms
- All [LM:Registry] logs at DEBUG level
2026-07-13 18:02:26 +08:00
Will Miao 9d85c2a44a fix(ui): prevent tags widget from auto-resizing in Vue mode when tags change 2026-07-13 14:55:40 +08:00
Will Miao 03dd047e62 fix(download): return 200 instead of 500 when user cancels download 2026-07-13 11:47:48 +08:00
Will Miao 86b547c1e0 fix(locales): add missing downloadStopped key to toast.downloads section 2026-07-13 11:35:48 +08:00
Will Miao bab9752c8b fix(download): close modal before progress overlay and fix downloadId ReferenceError on cancel 2026-07-13 11:29:47 +08:00
Will Miao 774cc1be86 fix(download): use file ID for exact match, add debug logging for multi-file selection (#1023)
- Frontend: send file.id in file_params, use null instead of hardcoded defaults
- Backend: priority matching (ID exact → primary → lenient metadata)
- Lenient metadata: only compare fields present on both sides (fixes GGUF size mismatch)
- Add debug logs at key points: entry, file_params received, match result, anomaly signals
2026-07-13 11:15:03 +08:00
Will Miao 234b73c8a2 feat(ui): add cancel button to download progress modal 2026-07-13 09:40:53 +08:00
Will Miao abd06c48f4 fix(settings): reject checkpoints↔unet path overlap in extra folder paths with inline error UI
Changes:
- Backend: _validate_folder_paths() now checks checkpoints↔unet overlap
  within the same library using os.path.realpath() for symlink resolution
- Backend: set() calls _validate_folder_paths() for both folder_paths and
  extra_folder_paths before writing
- Backend: extracted _normalize_path_set() helper to eliminate duplicated
  normalization logic
- Frontend: inline error display with red border + error message below the
  conflicting input, no save triggered
- Frontend: path normalization (strip trailing slash, lowercase) in pre-check
  to reduce false negatives vs backend realpath
- Frontend: asymmetric error UX — message only on the user-edited side,
  red border on the pre-existing conflict side
- CSS: has-error styles with hardcoded rgba fallback for older browsers
- i18n: checkpointUnetOverlap + checkpointUnetOverlapInline keys added to
  all 10 locale files
2026-07-13 08:22:40 +08:00
Will Miao 6ca411e4e4 fix(ui): make loras widget fixed-size with user-controlled node resize
Remove dynamic height calculation that auto-resized the node when
LoRAs are added or removed. The widget now stays at the size the user
sets via the node resize handle, scrolling when content overflows.

- Drop updateWidgetHeight() and hardcoded entry-count height math
- Set --comfy-widget-min-height once (200px) instead of recalculating
- In Vue mode: add contain:layout+size to break the ResizeObserver
  feedback loop that forced node growth with content (CSS via
  .lm-loras-container.lm-vue-node scoped to vueNodesMode only)
- Remove unused "Node 2.0: Maximum visible LoRA entries" setting
2026-07-12 22:35:58 +08:00
Will Miao 6470021e77 feat(settings): persist LORA_MANAGER_PORTABLE to settings.json on first use (#1018) 2026-07-12 09:32:30 +08:00
Will Miao 71658ab37b feat(settings): add LORA_MANAGER_PORTABLE env var for per-instance settings isolation (#1018) 2026-07-12 07:44:31 +08:00
Will Miao 4f016a8024 feat(fetch): skip CivArchive API for HuggingFace-sourced models
- Bulk refresh filter now excludes models with hf_url
- Individual refresh for HF models only checks CivitAI API
- CivArchive client validates model IDs before querying
2026-07-11 20:29:54 +08:00
Will Miao f362ed585b fix(preview): gracefully handle deleted preview files - image fallback, cache cleanup, quieter logs
- Add onerror handler on <img> previews to fallback to no-preview.png
- Fire async cache cleanup when preview file returns 404
- Add ModelCache.clear_preview_by_path() for safe stale-url removal
- Downgrade /api/lm/previews 404 log from warning to debug
2026-07-10 21:25:07 +08:00
Will Miao 196172624f fix(ui): allow autocomplete textarea resize in app mode (#1020) 2026-07-09 11:59:09 +08:00
Will Miao 316702b7ab fix(hf): allow subdirectory paths in HF resolve URLs, strip repo-internal dirs on save (#1019) 2026-07-09 09:18:38 +08:00
Will Miao a7625b009f fix(ui): also exit bulk mode after enrich-hf-llm-bulk completes 2026-07-07 20:31:16 +08:00
Will Miao 5d4a33c90d fix(hf): stop using realpath for download path construction, match CivitAI approach 2026-07-07 20:24:47 +08:00
Will Miao 041a6b8525 Revert "fix(hf): pass computed folder to _save_hf_metadata instead of re-deriving from paths"
This reverts commit 54b44131b6.
2026-07-07 20:13:20 +08:00
Will Miao 2638109ad6 feat(hf): add Link to HuggingFace feature with unified Link Model submenu
- Merge Relink to Civitai and new Link to HuggingFace into a single
  'Link Model' submenu with sub-options for each source
- Add POST /api/lm/set-hf-url endpoint to associate a model with a
  HuggingFace repo URL, saving hf_url to .metadata.json
- Add link_hf_modal.html for URL input, following relink-civitai pattern
- Use update_single_model_cache instead of add_model_to_cache to
  prevent duplicate cache entries after linking
- Remove os.path.realpath usage for consistency with relink-civitai
- Raise errors instead of silently falling back to LoRA scanner when
  model root cannot be determined
- Scope .input-group CSS rules to modal IDs to fix style conflicts
  with download-modal.css
- Add i18n keys across all 10 locales with translations for
  zh-CN, zh-TW, ja, ko, de, es, fr, he, ru
2026-07-07 20:04:47 +08:00
Will Miao b019326747 feat(ui): auto-exit bulk mode after all bulk operations complete 2026-07-06 18:51:33 +08:00
Will Miao 54b44131b6 fix(hf): pass computed folder to _save_hf_metadata instead of re-deriving from paths 2026-07-06 17:34:43 +08:00
Will Miao a1d948025c fix(hf): strip empty trainedWords from metadata JSON to keep sidecar clean 2026-07-06 16:49:51 +08:00
Will Miao a90b2514ba feat(ui): group HF batch files by repo with collapse/expand, fix nested scroll & collapse animation
- Group HF batch download files by repo with collapsible group headers
- Fix nested scrollbar conflict (inner scrollbar undraggable) by making batch-preview-list flex-fill
- Fix collapse animation glitch (items disappearing before container shrinks) by keeping expanded during max-height transition
- Visual polish: hover lift, backdrop-filter glass, design token alignment
- Remove redundant database icon from group header
- Guard transitionend handlers against rapid-click races
2026-07-06 16:36:26 +08:00
pixelpaws cb4ad27813 Merge pull request #1013 from willmiao/agent
Hugging Face model metadata AI enrichment
2026-07-06 12:21:19 +08:00
Will Miao 637831248b fix(agent): route WS error events through onError instead of dead onComplete branch 2026-07-06 12:18:17 +08:00
Will Miao 00228deaaa fix(download): retry on Civitai 429 rate limit instead of removing images from metadata
When Civitai returns 429 (Too Many Requests) during example image
downloads, the previous behavior treated all failures identically and
permanently removed the corresponding images from model metadata —
making them impossible to retry.

This commit adds:
- 429 detection + Retry-After header parsing in download_to_memory
- Exponential backoff retry (up to 3 attempts) in
  download_model_images_with_tracking
- Separate tracking of rate-limited vs permanently failed URLs
- rate_limited_models progress tracking persisted to disk
- Rate-limited models are NOT added to failed_models/processed_models
  so they are automatically retried on subsequent download runs
- Force mode clears failed_models when rate-limited images exist
2026-07-06 11:58:19 +08:00
Will Miao 2373edf73c feat(ui): load provider model catalog asynchronously to avoid blocking page render 2026-07-06 10:02:09 +08:00
Will Miao e0e1b804a7 fix(llm): require api_base for custom provider without preset default 2026-07-06 10:02:04 +08:00
Will Miao fecbe8241f fix(agent): use status= instead of status_code in json_response calls 2026-07-06 10:02:00 +08:00
Will Miao 5983eaa1ce refactor(llm): use catalog-based max_tokens, remove JSON retry, reduce Ollama num_ctx
- Parse limit.output from model catalog alongside model IDs
  for per-model max output token limits
- Use catalog lookup in chat_completion_json() to set max_tokens;
  fall back to 4096 for unknown models (e.g. local Ollama)
- Remove the JSON retry (response_format → plain text fallback);
  keep _try_salvage_json as last-resort for truncated responses
- Reduce Ollama num_ctx from 32768 to 8192 (sufficient for
  metadata enrichment, saves VRAM)
- Fix stale test comment referencing removed retry
2026-07-06 09:13:42 +08:00
Will Miao 07fa454f72 chore(tests): stop tracking HF enrichment baseline snapshots
Remove tests/enrich_hf_validation/baselines/ from git tracking
(.gitignore entry + git rm --cached). These contain README snapshots
from community HF repos that may include NSFW/sensitive content.

Local files are preserved on disk for offline reference.
2026-07-06 01:08:25 +08:00
Will Miao 4b5aa45379 chore(tests): update bash code block tests to match preserved-bash behavior
Commit 9a0d866b changed _strip_fenced_code_blocks to preserve bash/shell
code blocks (they carry CLI setup and trigger-word metadata signal).
Update the two affected tests to expect bash content in the output
instead of asserting it is stripped.

- Rename test_bash_code_block_stripped → test_bash_code_block_preserved
- Update assertions: expect 'pip install' in result
2026-07-06 01:02:04 +08:00
Will Miao 9a0d866be4 fix(agent): preserve bash/shell code blocks in readme_processor during README cleaning 2026-07-06 00:40:35 +08:00
Will Miao 308d8f71b8 feat(ui): gray out enrich-hf-llm when no hf_url, add backend fast-fail, rename labels across locales, reposition menu item 2026-07-06 00:34:18 +08:00
Will Miao d0e8938039 fix(agent): call _format_base_models via self. to prevent NameError
The bare call  inside _build_prompt_context
would raise NameError because class methods don't close over class-level
scope. Use  instead to trigger attribute lookup.

Update enrich_hf_metadata prompt.md clue locations for better LLM accuracy.
Update baseline report to v2 (mean 69.0, 46 models, +2.2pp vs baseline 71.1%).
Consolidate README snapshots into baselines/readmes/.
2026-07-06 00:10:30 +08:00
Will Miao 13ed898b6b chore(tests): add base_model ground truth mapping for all 46 test entries 2026-07-05 20:47:30 +08:00
Will Miao e1dfd1c2a6 chore(tests): add two test entries and their HF README snapshots 2026-07-05 20:45:01 +08:00
Will Miao e3e944911b refactor(agent): extract shared scanner iteration into _find_model_entry
_Previous_ _find_scanner_for_model and identify_model_type contained ~25 lines
of identical scanner-iteration + path-matching logic.  Factor it into
_find_model_entry() so a new scanner type or edge-case fix can't drift apart.
2026-07-05 18:03:57 +08:00
Will Miao 51c0135250 refactor(agent): rename agent_cli to metadata_ops, strip temp debug logs
- Rename py/agent_cli/ -> py/metadata_ops/ (module was never agent-related)
- Rename tests/agent_cli/ -> tests/metadata_ops/
- Remove 9 low-value/debug INFO log points across agent_handlers.py,
  agent_service.py, llm_service.py, and metadata_ops/__init__.py
- Keep LLM raw response at DEBUG level for diagnostics
- Consolidate per-model progress + LLM result into single concise
  log line with basename instead of full path
- Update package/class/method docstrings to clarify this is a
  pipeline infrastructure, not a true agent loop
2026-07-05 18:00:58 +08:00
Will Miao 7b19bbb14e fix(agent): preserve preview URLs for collection repo models with flat heading structure
Three-part fix for enrich_hf_metadata failing to extract correct preview_url
from HuggingFace collection repos where models share flat heading levels:

1. _strip_standalone_images() now converts <img> tags to markdown image
   syntax ![alt](src) instead of stripping the URL entirely, so the LLM
   can still extract preview URLs.

2. _extract_section() uses a line-count-based forward window (stopping at
   <a id> anchors) for non-heading matches, instead of stopping at the
   very next heading. This prevents same-level sub-headings (# Download,
   # Trigger, # Sample prompt within a single model section) from
   truncating the window before sample images are included.

3. Post-processor preview fallback now filters gallery images to the
   model-specific README section before falling back to the repo-wide
   first image.
2026-07-05 17:05:47 +08:00
Will Miao 5494a70f40 chore(tests): commit validation dataset and baseline reports into repo
Move the HF model list from ~/Documents/ into tests/enrich_hf_validation/test_data/
and commit the pipeline validation baseline artifacts (report.json,
preprocessing_audit.json, README snapshots) into baselines/.

Update config.py and run_validation.py defaults to use repo-relative paths
via os.path.dirname(__file__) instead of ~/Documents/ hardcode.

Originates from changes in 8fb00998 (validation pipeline audit).
2026-07-05 17:03:45 +08:00
Will Miao 26c9ade1c9 feat(agent): optimize base model prompt — grouped display, comprehensive mapping rules, filename inference
- agent_service._format_base_models: output bullet list instead of
  JSON array for cleaner LLM parsing
- prompt.md mapping section: replace 14-row HF→CivitAI table with
  compact rule set covering 14 mapping paths including new entries
  for HiDream-ai, OnomaAIResearch/Illustrious, ideogram-ai/ideogram,
  Tongyi-MAI/Z-Image-Turbo, and Wan-AI/Wan2.*
- base_model extraction instruction: add guidance to infer from
  model filename, YAML tags, and README body text when YAML
  frontmatter has no explicit base_model:
2026-07-05 15:45:17 +08:00
Will Miao 87db23825f feat(constants): add 12 new CivitAI base models from API, sync JS/Python abbreviations and categories 2026-07-05 11:44:53 +08:00
Will Miao 8fb00998a7 feat(agent): fix extract_relevant_section false positives, add validation pipeline audit
- extract_relevant_section: raise token threshold >3, verify anchor
  sections contain basename, require 2+ heading token overlaps, skip
  TOC-style headings (markdown links), verify heading section size
- metadata_constructor: parse repo_id,model_name.safetensors format
  so model_path basename matches real filename
- config: replace hardcoded SUPPORTED_BASE_MODELS with dynamic
  init_supported_base_models() using production list_base_models()
- preprocessing_auditor: new Phase 1.5 audit module — fetches each
  README, runs extract_relevant_section + clean_readme_for_llm,
  records stats and flags, saves raw READMEs for cross-reference
- run_validation: integrate audit phase, add --audit-only mode,
  add LLM config consistency check, add ComfyUI root to sys.path
- report_generator: add Preprocessing Audit and Config Warnings
  sections to both markdown and JSON reports
2026-07-05 11:18:48 +08:00
Will Miao dd3aa97d0a refactor(agent): rename md_to_html to readme_processor, fix section extraction, widget parsing, and list_base_models
- Rename md_to_html.py → readme_processor.py (file no longer just HTML conversion)
- _extract_section: include YAML frontmatter, use heading-level-aware forward
  walk (sub-headings under # are included), increase walk limit past 30 lines
- _is_heading: exclude </hN> closing tags from boundary detection
- _heading_level: new helper for heading-level-aware section matching
- css: yield 0 for heading like closing tags, was unexpectedly caught by _is_heading
- extract_gallery_images: fix YAML block scalar (text: >-) prompt extraction;
  use endswith instead of == to detect the block marker
- _strip_widget_section: add to clean_readme_for_llm (widget text is handled
  by post-processor, not needed in LLM prompt)
- _strip_standalone_images: keep markdown image URLs intact for LLM preview
  extraction (was stripping to alt text only)
- list_base_models: switch from scanner-cache aggregation to
  CivitaiBaseModelService.get_base_models() - always returns full list
- Ollama: add num_ctx=32768 to payload options so thinking models have room
  to both reason and produce output
- Add tests/agent_cli/test_readme_processor.py: 59 tests covering extraction,
  cleaning, section matching, heading detection
- Update existing tests for behavioral changes
2026-07-05 06:39:54 +08:00
Will Miao 8bee8f4069 fix(recipe): fallback to locate custom example image on disk by model hash and image id (#1012) 2026-07-04 18:40:34 +08:00
Will Miao 817fe21b3e fix(ui): read cfg_scale and clip_skip with snake_case fallback, pass custom image id for recipe creation (#1012) 2026-07-04 18:40:24 +08:00
Will Miao 905c37290f chore: update runtime logs to use 'LLM enrichment' instead of 'Agent skill'
- agent_handlers.py: 'Agent skill' -> 'LLM enrichment' in all log messages
- skill_registry.py: 'agent skills' -> 'prompt-based skills' in discovery log
- llm_service.py: docstring 'agent skills' -> 'LLM-based enrichment features'
2026-07-04 16:53:41 +08:00
Will Miao f7632a47f9 feat(agent): enrich_hf_metadata with per-model progress and in-place card update
- PostProcessor returns updates dict from enrich_hf_metadata
- AgentService includes updated_data per model in WebSocket progress events
- Convert preview_url to HTTP URL via config.get_preview_static_url()
- LoraContextMenu: showEnhancedProgress + updateSingleItem per model
- BulkContextMenu: same pattern, remove window.location.reload()
- Guard empty updated_data and clean up callbacks on HTTP error
2026-07-04 16:50:56 +08:00
Will Miao 646f1ddfb1 refactor(agent): align 'Agent' naming to 'AI/LLM' to match current implementation
- locales/en.json: 'Enrich Metadata (Agent)' -> 'Enrich Metadata (AI)'
- Rename SKILL.md -> prompt.md with backward compat in skill_registry.py
- JS context menu action IDs: enrich-hf-agent -> enrich-hf-llm
- HTML template data-action attributes synced to match
- docstring cleanup: 'agent skill' -> 'skill pipeline' / 'feature'
2026-07-04 14:06:50 +08:00
Will Miao 170c8068c5 feat(agent): enrich_hf_metadata — filename-aware section matching, preview extraction for markdown/HTML/widget, JSON salvage, instance_prompt fallback, and validation suite
- extract_relevant_section(): trim README to model-filename-matching section
  for collection repos (download link, anchor ID, heading strategies)
- _strip_standalone_images(): preserve markdown image URLs so LLM can
  extract preview_url; strip only HTML <img> tags
- extract_simple_markdown_images(): extract civitai.images from ![]() body
- extract_html_img_tags(): extract from <img src="..."> (deadman44-style)
- extract_gallery_images(): fix widget parser for YAML - output: dash prefix
- _is_heading: exclude </hN> closing tags from boundary detection
- _extract_section: start at matching heading when match IS a heading line
- _try_salvage_json(): recover truncated JSON (close braces/brackets in
  LIFO order, close unterminated strings, strip trailing commas)
- PostProcessor: store _llm_confidence, add instance_prompt YAML fallback
- agent_service: pass model_basename to prompt, trim README via
  extract_relevant_section before clean_readme_for_llm
- Add tests/enrich_hf_validation/ suite: 100-model pipeline with progress
  checkpoint/resume, per-field scoring, markdown+JSON reporting
- Fix evaluation_engine: read _llm_confidence (not _llm_response)
2026-07-04 12:00:15 +08:00
Will Miao 3494037d20 fix(download): pass proxy to aria2 for actual file transfers (#1010) 2026-07-04 11:07:18 +08:00
Will Miao a1fd4e150b feat(agent): optimize enrich_hf_metadata with README cleaning, Ollama native API, and expanded fields
- Add clean_readme_for_llm() to strip noise from README before LLM injection
- Keep widget section text (valuable tag signal) and unmarked code blocks (trigger words)
- Preserve standalone image alt text instead of removing entirely
- Switch Ollama to native /api/chat with think:false to fix empty content on thinking models
- Extract Sample Gallery table images and deduplicate with widget images
- Only strip code blocks with explicit language tags (bash)
- Add notes and usage_tips fields to SKILL.md output format and post-processor
- Clean up dead code, fix regex edge cases, remove double type annotation
2026-07-04 08:01:50 +08:00
Will Miao b22f09bd1d fix(standalone): load extra folder paths from library settings in standalone mode 2026-07-03 19:21:56 +08:00
Will Miao 4ed9169646 feat(ui): redesign AI Provider settings with provider presets and model catalog
- Replace hardcoded provider list with PROVIDER_PRESETS (OpenAI, Ollama,
  DeepSeek, Groq, OpenRouter, OpenCode Go, Custom)
- Load model lists from models.dev/api.json catalog at startup
- Add Combobox vanilla JS component for model/base-URL selection
- Fetch local Ollama models via live API instead of catalog
- Hide API key values from frontend (boolean-only llm_api_key_set)
- Add i18n translations for all 9+ locales
- Update snapshot tests for new response fields
2026-07-03 16:08:51 +08:00
Will Miao f06c60bd47 fix(agent): handle plain YAML scalar text in extract_gallery_images
Widget entries with unquoted multi-line YAML scalars (e.g. "text: two samurais...\n  continuation") were not parsed, leaving gallery image prompts empty. Add a third branch for plain scalar format alongside the existing quoted and >- folded block handlers.
2026-07-03 07:34:24 +08:00
Will Miao ee8250c26c feat(agent): extract HF widget gallery images into civitai.images with recommended dimensions
- Add extract_gallery_images() to parse YAML widget entries from README
  frontmatter, convert relative image URLs to absolute HF URLs, and
  build civitai.images-compatible entries with prompt metadata
- LLM now extracts recommended_width/recommended_height from README
  (e.g. "Best Dimensions"), used as gallery image dimensions
- extract_gallery_images() accepts default_width/height parameters,
  falling back to 512x512 when LLM provides no recommendation
- Frontend ShowcaseView.js: defensive NaN guard for 0 width/height
- post_processor: consistently merge civitai updates across triggers,
  description, and gallery blocks with distinct variable names
- SKILL.md: add recommended_width/recommended_height to output schema
- 62 tests pass, including gallery extraction and dimension tests
2026-07-03 07:07:19 +08:00
Will Miao 88349bf944 feat(agent): render HF README as HTML in modelDescription, move converter to skill-local module
- Add inline convert_readme_to_html() in new skill-local md_to_html.py
  (zero external deps, handles h1-h4/bold/italic/code/lists/tables/links/hr)
- Strip YAML frontmatter, <Gallery />, badge images, HTML comments pre-conversion
- Fix indented whitespace after lists being misidentified as code blocks
- Fix HTML double-escaping in _inline_md (each pattern escapes independently)
- LLM short_description → civitai.description ("About this version" sidebar)
- raw README HTML → modelDescription (description tab, always available offline)
- Pass full readme_content from agent_service to post_processor
- 51 tests for converter + 4 updated/added post-processor tests
2026-07-02 23:34:52 +08:00
Will Miao a8adcaf023 feat(agent): improve enrich_hf_metadata skill with priority_tags, preview_url fix, civitai.trainedWords
- Add identify_model_type() helper to determine lora/checkpoint/embedding
- Pass priority_tags from user settings to LLM prompt for tag relevance
- SKILL.md: instruct LLM to exclude technical/generic HF tags, cross-reference
  against priority_tags; forbid ['None'] placeholder for trigger words
- post_processor: fix preview_url not updated after download (now writes local
  .webp path to metadata); write trigger words to civitai.trainedWords instead
  of top-level; sanitize ['None']/'null'/'n/a' placeholder values to []
- download_preview() now returns str | None (local path) instead of bool
- Update tests for new return type and nested civitai.trainedWords structure
2026-07-02 22:14:44 +08:00
Will Miao 63785f82b5 refactor(agent): consolidate skill definition into single SKILL.md with YAML frontmatter
Merge skill.yaml (metadata) and prompt.md (prompt template) into a
single SKILL.md file with YAML frontmatter, matching the agent-skill
convention used by opencode and Claude Code.

- Add frontmatter parser (_parse_skill_file) to SkillRegistry
- Remove skill.yaml, prompt.md, empty skills/__init__.py
- Remove obsolete load_handler method
- Update tests for new format and cleaned-up fields
2026-07-02 21:29:02 +08:00
Will Miao cf898da193 feat(agent): add LLM-powered metadata enrichment system with AgentCLI and PostProcessor
Introduce an agent skill framework for LLM-driven metadata enrichment:

- AgentCLI (py/agent_cli/): in-process wrappers around internal services
  using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
  OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
  (replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales

Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
2026-07-02 21:27:01 +08:00
523 changed files with 87200 additions and 12382 deletions
+116 -171
View File
@@ -1,201 +1,146 @@
---
name: lora-manager-e2e
description: End-to-end testing and validation for LoRa Manager features. Use when performing automated E2E validation of LoRa Manager standalone mode, including starting/restarting the server, using Chrome DevTools MCP to interact with the web UI at http://127.0.0.1:8188/loras, and verifying frontend-to-backend functionality. Covers workflow validation, UI interaction testing, and integration testing between the standalone Python backend and the browser frontend.
description: "End-to-end testing and validation for LoRa Manager features. Use ONLY for sandboxed E2E validation of LoRa Manager standalone mode: start the standalone server on a free port with --settings-path, drive the web UI (http://127.0.0.1:{PORT}/loras) via Chrome DevTools MCP, and verify frontend-to-backend integration. NOT for UI behavior checks that unit tests (Vitest/jsdom) can cover. Trigger keywords: E2E, standalone, Chrome DevTools MCP, lora-manager-e2e, sandbox."
---
# LoRa Manager E2E Testing
This skill provides workflows and utilities for end-to-end testing of LoRa Manager using Chrome DevTools MCP.
End-to-end testing of LoRa Manager standalone mode using Chrome DevTools MCP.
## Prerequisites
## When to Use — and When NOT To
- LoRa Manager project cloned and dependencies installed (`pip install -r requirements.txt`)
- Chrome browser available for debugging
- Chrome DevTools MCP connected
E2E runs are slow and token-heavy. Reach for them only when the question genuinely
spans server + browser (routing, scan persistence, websocket updates, EXIF writes).
## Quick Start Workflow
- **Default to unit/component tests first**: `npm run test:js` (Vitest/jsdom) covers
DOM rendering, modal behavior, event handling and API-client calls deterministically
in seconds. Backend logic goes through `pytest`. A UI-behavior question answered by
jsdom MUST NOT be escalated to E2E.
- **Use E2E only when** the behavior cannot be observed without a live server and a
real browser, e.g. template rendering through the aiohttp server, scanner → SQLite
persistence → API → DOM round-trips, or real EXIF/image writes.
- If you start an E2E and realize a unit test would answer the question, stop and
switch.
### 1. Start LoRa Manager Standalone
**Browser driver is fixed: Chrome DevTools MCP.** Do not substitute kimi-webbridge —
it operates on the user's real browser (real tabs, real sessions, synthetic
`isTrusted=false` events), which breaks the isolation this skill requires and lacks
the console/network inspection E2E debugging relies on. kimi-webbridge is for
interactive browsing with the user's real login sessions, not for sandboxed E2E.
```python
# Use the provided script to start the server
python .agents/skills/lora-manager-e2e/scripts/start_server.py --port 8188
```
## Conventions
Or manually:
```bash
cd /home/miao/workspace/ComfyUI/custom_nodes/ComfyUI-Lora-Manager
python standalone.py --port 8188
```
- **`{PORT}`**: default candidate `8188`, but it is **commonly occupied by a live
ComfyUI** — always check first (`ss -tlnp | grep ':{PORT}'`) and use a free port
(e.g. `8199`). Substitute the chosen port everywhere below. Never kill a process
you did not start for this E2E.
- **`<repo-root>`**: the repository/worktree root; run all commands from there.
- **`<sandbox>`**: a throwaway dir, e.g. `/tmp/opencode/<plan>-e2e`.
Wait for server ready message before proceeding.
## SANDBOX (MANDATORY)
### 2. Open Chrome Debug Mode
> Every E2E run MUST target a throwaway sandbox, never real user data.
1. **Explicit settings directory**: always launch with `--settings-path <sandbox>/settings`.
This pins ALL runtime data (`settings.json`, `cache/`, `backups/`, `logs/`, `stats/`,
`wildcards/`) under the sandbox. **Never** create `<repo-root>/settings.json` — the repo
folder is usually the real ComfyUI plugin folder and a portable settings file there is
read by the real instance.
2. **Sandboxed library paths**: point `folder_paths` / `recipes_path` /
`example_images_path` at disposable dirs under `<sandbox>` — never the real library,
real recipe dir, or real settings:
```json
{
"folder_paths": {
"loras": ["<sandbox>/models/loras"],
"checkpoints": ["<sandbox>/models/checkpoints"],
"unet": ["<sandbox>/models/checkpoints"],
"diffusers": []
},
"recipes_path": "<sandbox>/recipes",
"example_images_path": "<sandbox>/example_images"
}
```
3. **Real-data protection proof**: before starting and after finishing, snapshot the real
config and recipe library and confirm they are byte-identical; also confirm
`<repo-root>` gained no `settings.json` or `cache/`:
```bash
sha256sum ~/.config/ComfyUI-LoRA-Manager/settings.json > <sandbox>/settings.before.sha256
ls ~/models/recipes/*.recipe.json 2>/dev/null | wc -l > <sandbox>/recipes-count.before.txt
# AFTER the run: record again and diff. Any change = the run leaked into real data.
```
## Quick Start
```bash
# Chrome with remote debugging on port 9222
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:8188/loras
cd <repo-root>
# 1. Sandbox
mkdir -p <sandbox>/settings <sandbox>/models/{loras,checkpoints} <sandbox>/{recipes,example_images}
# write <sandbox>/settings/settings.json per the SANDBOX example
# 2. Port
ss -tlnp | grep ':{PORT}' || echo "port {PORT} is free"
# 3. Server — MUST be fully detached (a plain background & dies with the shell);
# the helper enforces this and manages its own pidfile
python .agents/skills/lora-manager-e2e/scripts/start_server.py \
--port {PORT} --settings-path <sandbox>/settings --wait --timeout 30 --detach
ss -tlnp | grep ':{PORT}' # verify listening BEFORE proceeding
# 4. Chrome with remote debugging, then connect Chrome DevTools MCP (verify via list_pages)
google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:{PORT}/loras
```
### 3. Connect Chrome DevTools MCP
Then drive the UI with the MCP tools (`take_snapshot`, `click`, `fill`, `fill_form`,
`evaluate_script`, `wait_for`, `list_network_requests`, `list_console_messages`) —
see [references/mcp-cheatsheet.md](references/mcp-cheatsheet.md) for patterns.
Ensure the MCP server is connected to Chrome at `http://localhost:9222`.
### 4. Navigate and Interact
Use Chrome DevTools MCP tools to:
- Take snapshots: `take_snapshot`
- Click elements: `click`
- Fill forms: `fill` or `fill_form`
- Evaluate scripts: `evaluate_script`
- Wait for elements: `wait_for`
## Common E2E Test Patterns
### Pattern: Full Page Load Verification
```python
# Navigate to LoRA list page
navigate_page(type="url", url="http://127.0.0.1:8188/loras")
# Wait for page to load
wait_for(text="LoRAs", timeout=10000)
# Take snapshot to verify UI state
snapshot = take_snapshot()
```
### Pattern: Restart Server for Configuration Changes
```python
# Stop current server (if running)
# Start with new configuration
python .agents/skills/lora-manager-e2e/scripts/start_server.py --port 8188 --restart
# Wait and refresh browser
navigate_page(type="reload", ignoreCache=True)
wait_for(text="LoRAs", timeout=15000)
```
### Pattern: Verify Backend API via Frontend
```python
# Execute script in browser to call backend API
result = evaluate_script(function="""
async () => {
const response = await fetch('/loras/api/list');
const data = await response.json();
return { count: data.length, firstItem: data[0]?.name };
}
""")
```
### Pattern: Form Submission Flow
```python
# Fill a form (e.g., search or filter)
fill_form(elements=[
{"uid": "search-input", "value": "character"},
])
# Click submit button
click(uid="search-button")
# Wait for results
wait_for(text="Results", timeout=5000)
# Verify results via snapshot
snapshot = take_snapshot()
```
### Pattern: Modal Dialog Interaction
```python
# Open modal (e.g., add LoRA)
click(uid="add-lora-button")
# Wait for modal to appear
wait_for(text="Add LoRA", timeout=3000)
# Fill modal form
fill_form(elements=[
{"uid": "lora-name", "value": "Test LoRA"},
{"uid": "lora-path", "value": "/path/to/lora.safetensors"},
])
# Submit
click(uid="modal-submit-button")
# Wait for success message or close
wait_for(text="Success", timeout=5000)
```
## Available Scripts
### scripts/start_server.py
Starts or restarts the LoRa Manager standalone server.
Server restart after config/fixture changes:
```bash
python scripts/start_server.py [--port PORT] [--restart] [--wait]
python .agents/skills/lora-manager-e2e/scripts/start_server.py \
--port {PORT} --settings-path <sandbox>/settings --restart --wait --detach
# then reload the browser page (ignoreCache=True)
```
Options:
- `--port`: Server port (default: 8188)
- `--restart`: Kill existing server before starting
- `--wait`: Wait for server to be ready before exiting
`--restart` only kills the E2E server the script itself started (via its pidfile) and
aborts instead of killing unrelated processes on the port.
### scripts/wait_for_server.py
## Abort Rule
Polls server until ready or timeout.
A sandboxed E2E should finish in well under 30 minutes. If any phase exceeds ~2x its
expected duration (server readiness > 60 s, MCP connect > 2 min, a single scenario >
10 min), or any single tool call fails 3+ times in a row, **STOP** — do not retry
blindly. Report `BLOCKED` with the phase, last observed state (server PID,
`ss -tlnp` output, page snapshot, last API response) and suspected cause. A clean
BLOCKED report beats an hour of retries.
```bash
python scripts/wait_for_server.py [--port PORT] [--timeout SECONDS]
```
## Troubleshooting
## Test Scenarios Reference
See [references/test-scenarios.md](references/test-scenarios.md) for detailed test scenarios including:
- LoRA list display and filtering
- Model metadata editing
- Recipe creation and management
- Settings configuration
- Import/export functionality
## Network Request Verification
Use `list_network_requests` and `get_network_request` to verify API calls:
```python
# List recent XHR/fetch requests
requests = list_network_requests(resourceTypes=["xhr", "fetch"])
# Get details of specific request
details = get_network_request(reqid=123)
```
## Console Message Monitoring
```python
# Check for errors or warnings
messages = list_console_messages(types=["error", "warn"])
```
## Performance Testing
```python
# Start performance trace
performance_start_trace(reload=True, autoStop=False)
# Perform actions...
# Stop and analyze
results = performance_stop_trace()
```
- **"browser is already running" / `list_pages` fails**: a stale Chrome holds the
profile dir. Find it (`ps -ef | grep -i '[c]hrome.*user-data-dir'`), confirm it is a
leftover QA Chrome (not the live ComfyUI, not your current MCP browser), kill only
that PID, then retry `list_pages`.
- **MCP refuses to write screenshots into the worktree**: save to `/tmp` via
`take_screenshot(filePath="/tmp/...")` and copy into the evidence dir from the shell.
## Cleanup
Always ensure proper cleanup after tests:
1. Stop the standalone server
2. Close browser pages (keep at least one open)
3. Clear temporary data if needed
1. Stop the standalone server: `kill <recorded-pid>` (only the PID you started), then
confirm `ss -tlnp | grep ':{PORT}'` is empty.
2. Close browser pages (keep at least one open).
3. `rm -rf <sandbox>`; verify `<repo-root>` gained no `settings.json` or `cache/`.
4. Re-run the real-data protection check from the SANDBOX section and record the result.
## References & Scripts
- [references/mcp-cheatsheet.md](references/mcp-cheatsheet.md) — Chrome DevTools MCP
command patterns (navigation, waiting, snapshots, forms, network, console, performance).
- [references/test-scenarios.md](references/test-scenarios.md) — detailed test scenarios
(list display, metadata editing, recipes, settings, import/export).
- [references/recipe-rematch-fixtures.md](references/recipe-rematch-fixtures.md) —
fixture format, fresh-state reset and known gaps for recipe rematch/repair E2E runs.
- `scripts/start_server.py` — start/restart the standalone server
(`--port --settings-path --restart --wait --timeout --detach`); refuses to touch
unrelated processes on the port.
- `scripts/wait_for_server.py` — poll readiness (`--port --timeout`).
@@ -2,11 +2,13 @@
Quick reference for common MCP commands used in LoRa Manager E2E testing.
> **Port convention**: `{PORT}` is the port chosen for the E2E run (default candidate `8188`, but only if actually free — see the SKILL.md Port Selection section; use e.g. `8199` when `8188` is occupied by a live ComfyUI). Always run against the **sandboxed** standalone server, never a live instance.
## Navigation
```python
# Navigate to LoRA list page
navigate_page(type="url", url="http://127.0.0.1:8188/loras")
navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras")
# Reload page with cache clear
navigate_page(type="reload", ignoreCache=True)
@@ -179,7 +181,7 @@ pages = list_pages()
select_page(pageId=0, bringToFront=True)
# Create new page
new_page(url="http://127.0.0.1:8188/loras")
new_page(url="http://127.0.0.1:{PORT}/loras")
# Close page (keep at least one open!)
close_page(pageId=1)
@@ -261,7 +263,7 @@ drag(from_uid="draggable-item", to_uid="drop-zone")
### Verify LoRA Cards Loaded
```python
navigate_page(type="url", url="http://127.0.0.1:8188/loras")
navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras")
wait_for(text="LoRAs", timeout=10000)
# Check if cards loaded
@@ -322,3 +324,37 @@ navigate_page(type="reload")
errors = list_console_messages(types=["error"])
assert len(errors) == 0, f"Console errors: {errors}"
```
## Troubleshooting
### Stale profile lock ("browser is already running" / `list_pages` fails)
A Chrome profile held by a stale Chrome from a prior MCP session makes `list_pages`
fail with "browser is already running". Fix:
1. Find the stale Chrome that owns the profile dir (e.g. `~/.config/chrome-dev-profile`):
```bash
ps -ef | grep -i '[c]hrome.*user-data-dir'
```
2. Confirm it is a QA Chrome from a completed task (NOT the live ComfyUI server, NOT
your current MCP instance).
3. Kill ONLY that stale Chrome (`kill <stale-pid>`), then retry `list_pages`.
### Screenshot-write restrictions
The MCP may refuse to write into paths outside its configured workspace roots
(e.g. `.omo/evidence/screenshots/` under a worktree that canonicalizes to an unmapped
path). Save the screenshot to `/tmp` via the MCP, then copy it into the evidence dir:
```bash
# MCP: take_screenshot(filePath="/tmp/<plan>-e2e/recipe-b-after.png", format="png")
# Shell:
mkdir -p <repo-root>/.omo/evidence/screenshots
cp /tmp/<plan>-e2e/recipe-b-after.png <repo-root>/.omo/evidence/screenshots/
```
### Time budgets & abort rule
See SKILL.md "Time Budgets & Abort Guidance": if a phase exceeds ~2x its budget or a
tool call retries 3+ times in a row, STOP and report BLOCKED with the last observed
state (server PID + `ss -tlnp`, page snapshot, last API response). Do not loop.
@@ -0,0 +1,72 @@
# Recipe Rematch/Repair E2E — Fixtures, Fresh State, Known Gaps
Specialized guidance for recipe rematch/repair E2E runs, extracted from the SKILL.md
main flow. Read the SKILL.md SANDBOX section first — everything here assumes a
sandboxed run.
## Fixture Rules (validated by the task-8 E2E)
Seed the **sandboxed** `recipes_path` with hand-written fixture recipes:
1. **Filename constraint**: each file MUST be named `f"{id}.recipe.json"` **and** the
in-JSON `id` field MUST equal the filename. Discovery accepts any `*.recipe.json`,
but persistence resolves the path via `get_recipe_json_path` and
`_save_recipe_persistently` returns `False` on a mismatch → the fixture would be
counted as an error.
- `recipe-a.recipe.json` → in-JSON `"id": "recipe-a"`
2. **File format**: mirror an existing recipe JSON — top-level `id`, `file_path`,
`title`, `loras`, `fingerprint`, `gen_params`; lora entries per the persistence
conventions (`hash`, `file_name`, `modelVersionId`, `isDeleted`, ...).
3. **Companion image**: each recipe needs an image (e.g. a `.webp` generated with PIL)
referenced by `file_path`, used for EXIF verification
(`ExifUtils.append_recipe_metadata` writes a `"Recipe metadata: ..."` marker; a
freshly generated `.webp` with no marker is the clean "untouched" control).
4. **autov3 three-state contract**: for L3 (autov3-only, renamed-file) fixtures the
local model's `.metadata.json` sidecar MUST have the `autov3` key **ABSENT** (the
"unchecked" state), NOT `""``""` is the TERMINAL "checked but unavailable" state
that L3 deliberately skips. The scanner computes + persists `autov3` from the file
header during the normal library scan (`model_scanner.py` `_process_model_file`), so
the live L3 match resolves through the local autov3/hash cache; the
computed-autov3 branch for unchecked items is covered by the unit suite.
5. **Fixture design for a rematch run** (mirrors the task-8 E2E):
- `recipe-a`: lora entry `isDeleted=True`, `hash` = 12-char autov3 computed from the
local model (`calculate_autov3`, `py/utils/file_utils.py`), whose local model file
was RENAMED after the recipe was written so `file_name` differs (proves L3 match
without filename).
- `recipe-b`: parser-convention checkpoint entry (uses `id`, no `modelVersionId`)
matching a local checkpoint via L2 — the local checkpoint's `.metadata.json` MUST
carry civitai version data with that `id` so `version_index` contains it (L2
cannot match otherwise).
- `recipe-c`: healthy recipe (no deleted entries) → must remain untouched.
The scanner computes and persists model hashes during the library scan, so the sandbox
model dirs just need the model files + `.metadata.json` sidecars. With
`--settings-path`, all derived data lands under the sandbox settings dir (`cache/`,
`backups/`, `logs/`, `stats/`, `wildcards/`), and NO `cache/` appears in the repo root.
## Fresh State Between Entry-Point Runs
Each entry point (global / per-recipe / selection-bulk) must start from the same
deleted state. Between runs (keep a pristine copy in `<sandbox>/recipes-before/`):
```bash
# 1. Reset fixtures to the before-state snapshot
cp <sandbox>/recipes-before/*.recipe.json <sandbox>/recipes/
# 2. Clear the recipe/FTS caches (with --settings-path these live under the sandbox
# settings dir, NOT <repo-root>/cache)
rm -f <sandbox>/settings/cache/recipe/*.sqlite
rm -rf <sandbox>/settings/cache/fts/*
# 3. Restart the server (fresh process, fresh scan)
python .agents/skills/lora-manager-e2e/scripts/start_server.py \
--port {PORT} --settings-path <sandbox>/settings --restart --wait --timeout 30 --detach
# 4. Re-verify the server is listening + reload the browser page
```
## Cancellation Testing (KNOWN GAP)
Testing the rematch-cancel path E2E requires a run long enough to cancel mid-flight. A
tiny 3-recipe fixture set completes in **seconds** — too fast to reliably cancel. The
cancel path is currently **unit-covered only** (`rematch_all_recipes` cancellation
tests); do not block an E2E run on cancel-path verification. If you must attempt it,
you would need an artificially large/deferred fixture set to create a cancellable
window — treat this as a research task, not part of the standard E2E.
@@ -2,6 +2,14 @@
This document provides detailed test scenarios for end-to-end validation of LoRa Manager features.
> **Run preconditions (from SKILL.md)**: every run uses the **sandboxed** standalone
> server on a free port `{PORT}` (default candidate `8188`, only if actually free — pick
> e.g. `8199` when `8188` is occupied by a live ComfyUI). Fixtures live in the sandboxed
> `recipes_path` as `f"{id}.recipe.json"` files with matching in-JSON `id`; the real user
> config and real library are never touched (record protection proof before/after).
> Abort if a phase exceeds ~2x its budget or a tool call retries 3+ times (SKILL.md
> "Time Budgets & Abort Guidance").
## Table of Contents
1. [LoRA List Page](#lora-list-page)
@@ -19,7 +27,7 @@ This document provides detailed test scenarios for end-to-end validation of LoRa
**Objective**: Verify the LoRA list page loads correctly and displays models.
**Steps**:
1. Navigate to `http://127.0.0.1:8188/loras`
1. Navigate to `http://127.0.0.1:{PORT}/loras`
2. Wait for page title "LoRAs" to appear
3. Take snapshot to verify:
- Header with "LoRAs" title is visible
@@ -134,7 +142,7 @@ evaluate_script(function="""
**Objective**: Verify recipes page loads and displays recipes.
**Steps**:
1. Navigate to `http://127.0.0.1:8188/recipes`
1. Navigate to `http://127.0.0.1:{PORT}/recipes`
2. Wait for "Recipes" title
3. Take snapshot
@@ -176,7 +184,7 @@ evaluate_script(function="""
**Objective**: Verify settings page displays correctly.
**Steps**:
1. Navigate to `http://127.0.0.1:8188/settings`
1. Navigate to `http://127.0.0.1:{PORT}/settings`
2. Wait for "Settings" title
3. Take snapshot
@@ -190,7 +198,7 @@ evaluate_script(function="""
1. Navigate to settings page
2. Change a setting (e.g., default view mode)
3. Save settings
4. Restart server: `python scripts/start_server.py --restart --wait`
4. Restart server: `python scripts/start_server.py --port {PORT} --restart --wait --timeout 30 --detach`
5. Refresh browser page
6. Navigate to settings
@@ -8,11 +8,18 @@ This script shows how to:
3. Verify functionality end-to-end
Note: This is a template. Actual execution requires Chrome DevTools MCP.
Port: pick a FREE port for the run — 8188 is commonly occupied by a live
ComfyUI (see the skill's Port Selection section). Set PORT below to e.g. 8199
when 8188 is taken. Always run against a SANDBOXED standalone server.
"""
import subprocess
import sys
import time
# Choose the E2E port. 8188 is only the default candidate; use 8199 (or any
# free port checked with `ss -tlnp`) when 8188 is occupied by a live ComfyUI.
PORT = "8188"
def run_test():
@@ -22,12 +29,12 @@ def run_test():
print("LoRa Manager E2E Test Example")
print("=" * 60)
# Step 1: Start server
# Step 1: Start server (detached so it survives the shell)
print("\n[1/5] Starting LoRa Manager standalone server...")
result = subprocess.run(
[sys.executable, "start_server.py", "--port", "8188", "--wait", "--timeout", "30"],
[sys.executable, "start_server.py", "--port", PORT, "--wait", "--timeout", "30", "--detach"],
capture_output=True,
text=True
text=True,
)
if result.returncode != 0:
print(f"Failed to start server: {result.stderr}")
@@ -36,46 +43,55 @@ def run_test():
# Step 2: Open Chrome (manual step - show command)
print("\n[2/5] Open Chrome with debug mode:")
print("google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:8188/loras")
print(
f"google-chrome --remote-debugging-port=9222 "
f"--user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:{PORT}/loras"
)
print("(In actual test, this would be automated via MCP)")
# Step 3: Navigate and verify page load
print("\n[3/5] Page Load Verification:")
print("""
print(
f"""
MCP Commands to execute:
1. navigate_page(type="url", url="http://127.0.0.1:8188/loras")
1. navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras")
2. wait_for(text="LoRAs", timeout=10000)
3. snapshot = take_snapshot()
""")
"""
)
# Step 4: Test search functionality
print("\n[4/5] Search Functionality Test:")
print("""
print(
"""
MCP Commands to execute:
1. fill(uid="search-input", value="test")
2. press_key(key="Enter")
3. wait_for(text="Results", timeout=5000)
4. result = evaluate_script(function="""
4. result = evaluate_script(function=`
() => {
const cards = document.querySelectorAll('.lora-card');
return { count: cards.length };
}
""")
""")
`)
"""
)
# Step 5: Verify API
print("\n[5/5] API Verification:")
print("""
print(
"""
MCP Commands to execute:
1. api_result = evaluate_script(function="""
1. api_result = evaluate_script(function=`
async () => {
const response = await fetch('/loras/api/list');
const data = await response.json();
return { count: data.length, status: response.status };
}
""")
`)
2. Verify api_result['status'] == 200
""")
"""
)
print("\n" + "=" * 60)
print("Test flow completed!")
@@ -91,29 +107,31 @@ def example_restart_flow():
print("Example: Server Restart Flow")
print("=" * 60)
print("""
print(
f"""
Scenario: Change setting and verify after restart
Steps:
1. Navigate to settings page
- navigate_page(type="url", url="http://127.0.0.1:8188/settings")
- navigate_page(type="url", url="http://127.0.0.1:{PORT}/settings")
2. Change a setting (e.g., theme)
- fill(uid="theme-select", value="dark")
- click(uid="save-settings-button")
3. Restart server
- subprocess.run([python, "start_server.py", "--restart", "--wait"])
- subprocess.run([python, "start_server.py", "--port", "{PORT}", "--restart", "--wait", "--detach"])
4. Refresh browser
- navigate_page(type="reload", ignoreCache=True)
- wait_for(text="LoRAs", timeout=15000)
5. Verify setting persisted
- navigate_page(type="url", url="http://127.0.0.1:8188/settings")
- navigate_page(type="url", url="http://127.0.0.1:{PORT}/settings")
- theme = evaluate_script(function="() => document.querySelector('#theme-select').value")
- assert theme == "dark"
""")
"""
)
def example_modal_interaction():
@@ -123,7 +141,8 @@ def example_modal_interaction():
print("Example: Modal Dialog Interaction")
print("=" * 60)
print("""
print(
"""
Scenario: Add new LoRA via modal
Steps:
@@ -143,7 +162,8 @@ def example_modal_interaction():
4. Verify success
- wait_for(text="Successfully added", timeout=5000)
- snapshot = take_snapshot()
""")
"""
)
def example_network_monitoring():
@@ -153,12 +173,13 @@ def example_network_monitoring():
print("Example: Network Request Monitoring")
print("=" * 60)
print("""
print(
f"""
Scenario: Verify API calls during user interaction
Steps:
1. Clear network log (implicit on navigation)
- navigate_page(type="url", url="http://127.0.0.1:8188/loras")
- navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras")
2. Perform action that triggers API call
- fill(uid="search-input", value="character")
@@ -175,7 +196,8 @@ def example_network_monitoring():
- if search_requests:
details = get_network_request(reqid=search_requests[0]["reqid"])
- Verify request method, response status, etc.
""")
"""
)
if __name__ == "__main__":
@@ -1,15 +1,78 @@
#!/usr/bin/env python3
"""
Start or restart LoRa Manager standalone server for E2E testing.
Backward-compatible CLI: --port, --restart, --wait, --timeout all work as before.
New options: --detach (setsid-style fully detached launch, survives shell death).
Safety rules implemented here:
- Never kill processes the script did not start. The script tracks the PIDs it
manages in a pidfile (/tmp/lora-manager-e2e-server-{PORT}.pid).
- If the port is held by an unrelated process (e.g. a live ComfyUI) the script
reports the conflict and exits early instead of killing it.
- --restart only kills managed PIDs; if unrelated processes still hold the port
afterwards, the script reports them and aborts.
"""
from __future__ import annotations
import argparse
import os
import signal
import socket
import subprocess
import sys
import time
import socket
import signal
import os
PIDFILE_PREFIX = "/tmp/lora-manager-e2e-server"
def pidfile_path(port: int) -> str:
"""Path of the pidfile that records PIDs this script started for a port."""
return f"{PIDFILE_PREFIX}-{port}.pid"
def read_managed_pids(port: int) -> list[int]:
"""Read PIDs this script previously managed for the port (may be stale)."""
path = pidfile_path(port)
if not os.path.exists(path):
return []
try:
with open(path, "r", encoding="utf-8") as fh:
return [int(line.strip()) for line in fh if line.strip().isdigit()]
except (OSError, ValueError):
return []
def write_managed_pids(port: int, pids: list[int]) -> None:
"""Record PIDs this script manages for the port."""
try:
with open(pidfile_path(port), "w", encoding="utf-8") as fh:
for pid in pids:
fh.write(f"{pid}\n")
except OSError as exc:
print(f"Warning: could not write pidfile for port {port}: {exc}")
def clear_managed_pids(port: int) -> None:
"""Remove the pidfile for the port (no longer managed)."""
path = pidfile_path(port)
try:
if os.path.exists(path):
os.remove(path)
except OSError as exc:
print(f"Warning: could not remove pidfile {path}: {exc}")
def process_alive(pid: int) -> bool:
"""Return True if a process with the given pid exists."""
try:
os.kill(pid, 0)
return True
except ProcessLookupError:
return False
except PermissionError:
return True # exists but owned by someone else
def find_server_process(port: int) -> list[int]:
@@ -19,7 +82,7 @@ def find_server_process(port: int) -> list[int]:
["lsof", "-ti", f":{port}"],
capture_output=True,
text=True,
check=False
check=False,
)
if result.returncode == 0 and result.stdout.strip():
return [int(pid) for pid in result.stdout.strip().split("\n") if pid]
@@ -30,7 +93,7 @@ def find_server_process(port: int) -> list[int]:
["netstat", "-tlnp"],
capture_output=True,
text=True,
check=False
check=False,
)
pids = []
for line in result.stdout.split("\n"):
@@ -49,30 +112,48 @@ def find_server_process(port: int) -> list[int]:
return []
def kill_server(port: int) -> None:
"""Kill processes using the specified port."""
pids = find_server_process(port)
def describe_processes(pids: list[int]) -> str:
"""Human-readable description of a pid list (pid + command line)."""
descriptions = []
for pid in pids:
cmdline = ""
try:
with open(f"/proc/{pid}/cmdline", "rb") as fh:
raw = fh.read().replace(b"\x00", b" ").decode("utf-8", "replace")
cmdline = raw.strip()
except OSError:
pass
descriptions.append(f"pid {pid}{' (' + cmdline + ')' if cmdline else ''}")
return ", ".join(descriptions) if descriptions else "none"
def kill_pids(pids: list[int], what: str) -> None:
"""Send SIGTERM (then SIGKILL) to the given PIDs, only after reporting."""
for pid in pids:
print(f"Sent SIGTERM to {what} pid {pid}")
try:
os.kill(pid, signal.SIGTERM)
print(f"Sent SIGTERM to process {pid}")
except ProcessLookupError:
pass
# Wait for processes to terminate
time.sleep(1)
deadline = time.time() + 5
while time.time() < deadline:
if not any(process_alive(pid) for pid in pids):
break
time.sleep(0.2)
# Force kill if still running
pids = find_server_process(port)
for pid in pids:
if process_alive(pid):
try:
os.kill(pid, signal.SIGKILL)
print(f"Sent SIGKILL to process {pid}")
print(f"Sent SIGKILL to {what} pid {pid}")
except ProcessLookupError:
pass
def is_server_ready(port: int, timeout: float = 0.5) -> bool:
def is_server_ready(port: int, timeout: float = 2.0) -> bool:
"""Check if server is accepting connections."""
try:
with socket.create_connection(("127.0.0.1", port), timeout=timeout):
@@ -84,9 +165,15 @@ def is_server_ready(port: int, timeout: float = 0.5) -> bool:
def wait_for_server(port: int, timeout: int = 30) -> bool:
"""Wait for server to become ready."""
start = time.time()
last_report = 0.0
while time.time() - start < timeout:
if is_server_ready(port):
return True
# Report progress every ~5s so a slow boot is visible, not silent.
elapsed = time.time() - start
if elapsed - last_report >= 5:
print(f" ...still waiting ({int(elapsed)}s/{timeout}s)")
last_report = elapsed
time.sleep(0.5)
return False
@@ -99,23 +186,41 @@ def main() -> int:
"--port",
type=int,
default=8188,
help="Server port (default: 8188)"
help="Server port (default: 8188)",
)
parser.add_argument(
"--restart",
action="store_true",
help="Kill existing server before starting"
help="Kill the E2E server previously managed by this script for the port "
"(tracked via pidfile) before starting; refuse to kill unrelated processes",
)
parser.add_argument(
"--wait",
action="store_true",
help="Wait for server to be ready before exiting"
help="Wait for server to be ready before exiting",
)
parser.add_argument(
"--timeout",
type=int,
default=30,
help="Timeout for waiting (default: 30)"
help="Timeout for waiting (default: 30)",
)
parser.add_argument(
"--detach",
action="store_true",
help="Launch the server fully detached (setsid-style) so it survives shell "
"death. REQUIRED for E2E: a plain background process dies with the shell",
)
parser.add_argument(
"--settings-path",
type=str,
default=None,
metavar="DIR",
help="Explicit settings directory passed to standalone.py (--settings-path, "
"equivalent to LORA_MANAGER_SETTINGS_DIR). settings.json, cache/, "
"wildcards/, backups/, logs/, stats/ all live under this directory instead "
"of the project root or the user config dir. Recommended for sandboxed E2E "
"so the real instance and the repo stay untouched",
)
args = parser.parse_args()
@@ -125,31 +230,115 @@ def main() -> int:
skill_dir = os.path.dirname(script_dir)
project_root = os.path.dirname(os.path.dirname(os.path.dirname(skill_dir)))
# Restart if requested
if args.restart:
print(f"Killing existing server on port {args.port}...")
kill_server(args.port)
time.sleep(1)
managed_pids = read_managed_pids(args.port)
# Check if already running
if is_server_ready(args.port):
print(f"Server already running on port {args.port}")
# Restart if requested: kill ONLY managed PIDs.
if args.restart:
alive_managed = [pid for pid in managed_pids if process_alive(pid)]
if alive_managed:
print(
f"Killing E2E server previously started by this script on port "
f"{args.port} ({describe_processes(alive_managed)})..."
)
kill_pids(alive_managed, "managed E2E server")
else:
print(
f"No live managed E2E server for port {args.port} "
f"(pidfile: {pidfile_path(args.port)})"
)
time.sleep(1)
# Refuse to kill anything the script did not manage.
remaining = find_server_process(args.port)
if remaining:
print(
f"ERROR: port {args.port} is still held by process(es) this script "
f"did not start: {describe_processes(remaining)}."
)
print(
"These may be unrelated (e.g. a live ComfyUI). The script will NOT "
"kill them. Pick a different --port, or stop them manually if you "
"are certain they are stale E2E servers."
)
return 2
clear_managed_pids(args.port)
# Port conflict check before starting: never blind-kill.
port_pids = find_server_process(args.port)
if port_pids:
alive_managed = [pid for pid in port_pids if pid in managed_pids]
unmanaged = [pid for pid in port_pids if pid not in managed_pids]
if alive_managed and not unmanaged:
print(
f"Server already running on port {args.port} "
f"({describe_processes(alive_managed)}, started by this script). "
f"Use --restart to recycle it."
)
return 0
print(
f"ERROR: port {args.port} is already in use by process(es): "
f"{describe_processes(port_pids)}."
)
print(
"This is likely an unrelated process (e.g. a live ComfyUI holding 8188). "
"The script will NOT kill it. Pick a free port with --port, e.g. 8199."
)
return 2
# Start server
print(f"Starting LoRa Manager standalone server on port {args.port}...")
cmd = [sys.executable, "standalone.py", "--port", str(args.port)]
cmd = [
sys.executable,
"standalone.py",
"--host",
"127.0.0.1",
"--port",
str(args.port),
]
if args.settings_path:
settings_dir = os.path.abspath(os.path.expanduser(args.settings_path))
if os.path.exists(settings_dir) and not os.path.isdir(settings_dir):
print(
f"ERROR: --settings-path '{settings_dir}' exists but is not a directory."
)
return 2
os.makedirs(settings_dir, exist_ok=True)
cmd.extend(["--settings-path", settings_dir])
print(f"Settings directory: {settings_dir}")
# Start in background
if args.detach:
# Fully detached launch: new session (setsid), no controlling terminal,
# stdin from /dev/null, stdout/stderr to a log file. Survives the shell.
log_dir = os.path.join(script_dir, "logs")
os.makedirs(log_dir, exist_ok=True)
log_path = os.path.join(log_dir, f"server-{args.port}.log")
with open(log_path, "ab") as log_fh:
process = subprocess.Popen(
cmd,
cwd=project_root,
stdin=subprocess.DEVNULL,
stdout=log_fh,
stderr=subprocess.STDOUT,
start_new_session=True,
close_fds=True,
)
print(f"Detached server process started with PID {process.pid} (setsid)")
print(f"Log: {log_path}")
else:
# Plain background process (legacy behavior): dies with the shell.
process = subprocess.Popen(
cmd,
cwd=project_root,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
start_new_session=True
start_new_session=True,
)
print(f"Server process started with PID {process.pid}")
print(
"NOTE: not detached — this process dies when the launching shell exits. "
"For E2E use --detach."
)
print(f"Server process started with PID {process.pid}")
write_managed_pids(args.port, [process.pid])
# Wait for ready if requested
if args.wait:
@@ -157,8 +346,7 @@ def main() -> int:
if wait_for_server(args.port, args.timeout):
print(f"Server ready at http://127.0.0.1:{args.port}/loras")
return 0
else:
print(f"Timeout waiting for server")
print(f"Timeout waiting for server on port {args.port}")
return 1
print(f"Server starting at http://127.0.0.1:{args.port}/loras")
@@ -1,15 +1,20 @@
#!/usr/bin/env python3
"""
Wait for LoRa Manager server to become ready.
Timeout is configurable via --timeout (default 30s); the script polls the port
until the server accepts connections or the timeout expires.
"""
from __future__ import annotations
import argparse
import socket
import sys
import time
def is_server_ready(port: int, timeout: float = 0.5) -> bool:
def is_server_ready(port: int, timeout: float = 2.0) -> bool:
"""Check if server is accepting connections."""
try:
with socket.create_connection(("127.0.0.1", port), timeout=timeout):
@@ -21,9 +26,15 @@ def is_server_ready(port: int, timeout: float = 0.5) -> bool:
def wait_for_server(port: int, timeout: int = 30) -> bool:
"""Wait for server to become ready."""
start = time.time()
last_report = 0.0
while time.time() - start < timeout:
if is_server_ready(port):
return True
# Report progress every ~5s so a slow boot is visible, not silent.
elapsed = time.time() - start
if elapsed - last_report >= 5:
print(f" ...still waiting ({int(elapsed)}s/{timeout}s)")
last_report = elapsed
time.sleep(0.5)
return False
@@ -36,13 +47,13 @@ def main() -> int:
"--port",
type=int,
default=8188,
help="Server port (default: 8188)"
help="Server port (default: 8188)",
)
parser.add_argument(
"--timeout",
type=int,
default=30,
help="Timeout in seconds (default: 30)"
help="Timeout in seconds (default: 30)",
)
args = parser.parse_args()
@@ -52,7 +63,6 @@ def main() -> int:
if wait_for_server(args.port, args.timeout):
print(f"Server ready at http://127.0.0.1:{args.port}/loras")
return 0
else:
print(f"Timeout: Server not ready after {args.timeout}s")
return 1
@@ -9,7 +9,10 @@ description: Inspect ComfyUI LoRA Manager runtime configuration and local diagno
- Treat runtime state as local user data. Prefer read-only inspection unless the user explicitly asks for mutation.
- Never print secret-like settings values. Redact keys containing `key`, `token`, `secret`, `password`, `auth`, or `credential`, including `civitai_api_key`.
- Resolve paths from the runtime configuration before guessing. In this environment the settings file is normally `/home/miao/.config/ComfyUI-LoRA-Manager/settings.json`, but portable settings can override this through the repository `settings.json`.
- Resolve paths from the runtime configuration before guessing. Settings-directory precedence (highest first):
1. **Explicit override** — env `LORA_MANAGER_SETTINGS_DIR` or standalone `--settings-path` (also accepted by the inspect script as `--settings-path DIR`). Pins EVERYTHING (`settings.json`, `cache/`, `wildcards/`, `backups/`, `logs/`, `stats/`) under the given directory; bypasses portable mode and the user config dir. Common when inspecting a sandboxed/E2E instance.
2. **Portable** — repository `<repo-root>/settings.json` with `"use_portable_settings": true` (or `LORA_MANAGER_PORTABLE=1`): settings dir = `<repo-root>`.
3. **Default**`~/.config/ComfyUI-LoRA-Manager` on this machine (`platformdirs.user_config_dir("ComfyUI-LoRA-Manager", appauthor=False)`).
- Use the active library when selecting per-library caches and paths. Read `active_library` from settings; fall back to `default` if missing.
- Normalize and expand `~` before comparing paths. Symlinks are common in this repo.
@@ -32,9 +35,17 @@ python .agents/skills/lora-manager-runtime-context/scripts/inspect_runtime_conte
python .agents/skills/lora-manager-runtime-context/scripts/inspect_runtime_context.py sqlite --db /path/to/cache.sqlite --limit 3
```
To inspect a sandboxed/E2E instance that pins its settings directory:
```bash
# --settings-path DIR (or LORA_MANAGER_SETTINGS_DIR) works with every subcommand:
python .agents/skills/lora-manager-runtime-context/scripts/inspect_runtime_context.py \
--settings-path /tmp/opencode/<plan>-e2e/settings summary
```
## Runtime Path Rules
- Settings directory: use `py/utils/settings_paths.py`. Default platform path is `platformdirs.user_config_dir("ComfyUI-LoRA-Manager", appauthor=False)`.
- Settings directory: resolve via `py/utils/settings_paths.py``get_settings_dir()` honors the `LORA_MANAGER_SETTINGS_DIR` / programmatic override first, then portable mode, then `platformdirs.user_config_dir("ComfyUI-LoRA-Manager", appauthor=False)`. The inspect script mirrors this precedence in `resolve_settings_path()`.
- Settings file: `<settings_dir>/settings.json`.
- Cache root: `<settings_dir>/cache`.
- Canonical cache files:
@@ -14,6 +14,7 @@ from typing import Any
SECRET_PATTERN = re.compile(r"(key|token|secret|password|auth|credential)", re.IGNORECASE)
APP_NAME = "ComfyUI-LoRA-Manager"
SETTINGS_DIR_ENV = "LORA_MANAGER_SETTINGS_DIR"
CACHE_SQLITE = {
"model": ("model", "{library}.sqlite"),
"recipe": ("recipe", "{library}.sqlite"),
@@ -30,6 +31,15 @@ CACHE_JSON = {
def main() -> int:
parser = argparse.ArgumentParser(description="Inspect LoRA Manager runtime state read-only.")
parser.add_argument(
"--settings-path",
type=str,
default=None,
metavar="DIR",
help="Explicit settings directory (same as LORA_MANAGER_SETTINGS_DIR / "
"standalone --settings-path). Overrides portable mode and the default "
"user config dir.",
)
subparsers = parser.add_subparsers(dest="command", required=True)
subparsers.add_parser("summary", help="Print redacted settings and resolved paths.")
@@ -44,6 +54,8 @@ def main() -> int:
sqlite_parser.add_argument("--limit", type=int, default=3, help="Rows to sample from each user table.")
args = parser.parse_args()
if args.settings_path:
os.environ[SETTINGS_DIR_ENV] = args.settings_path
context = build_context()
if args.command == "summary":
@@ -78,6 +90,11 @@ def build_context() -> dict[str, Any]:
def resolve_settings_path() -> Path:
# Explicit override: LORA_MANAGER_SETTINGS_DIR env or --settings-path.
explicit = os.environ.get(SETTINGS_DIR_ENV)
if explicit:
return Path(explicit).expanduser() / "settings.json"
repo_root = find_repo_root()
portable = repo_root / "settings.json"
if portable.exists():
+5
View File
@@ -25,6 +25,7 @@ model_cache/
reasonix.toml
.reasonix/
.codegraph/
.playwright-mcp/
# Vue widgets development cache (but keep build output)
vue-widgets/node_modules/
@@ -36,3 +37,7 @@ vue-widgets/dist/
# Working/research notes (not committed)
.docs/
# HF enrichment validation baseline snapshots (contain potentially
# NSFW README content fetched from community model repos)
tests/enrich_hf_validation/baselines/
+202
View File
@@ -0,0 +1,202 @@
---
slug: undo-delete-staging
status: drafting
intent: clear
review_required: false
pending-action: write .omo/plans/undo-delete-staging.md
approach: "Option B: delayed physical deletion with Undo. Backend: same-volume rename to per-root staging dir (.lm-pending-delete/) [updated 2026-08: model staging moved to a SIBLING dir inside each deleted model's own folder — see 'Symlink fix (2026-08)' under Decisions] + manifest JSON (batch_id, expires_at, staged->original map) + purge (30s TTL timer + startup sweep + opportunistic) + undo-delete endpoint + settings toggle 'skip undo'. Small files (recipes: JSON+preview) copy to global staging under settings dir instead of rename. Frontend: extend toast system with action button + 30s countdown; delete flows (single model / recipe / bulk / duplicates) consume batch_id from delete response and show Undo toast; expired undo -> 'undo expired' toast. Plus confirm-modal friction (C-friction, NO type-to-confirm): delete button delay-activation 1.5s + modal shows file size 'will free X GB' + Cancel gets initial focus. i18n keys + sync_translation_keys.py."
---
# Draft: undo-delete-staging
## Components (topology ledger)
<!-- Lock the SHAPE before depth. One row per top-level component that can succeed or fail independently. -->
<!-- id | outcome (one line) | status: active|deferred | evidence path -->
- backend staging module (stage/purge/undo + manifest + per-volume dir resolution) | new module, active | pending exploration: model_lifecycle_service.py delete_model / delete_model_artifacts
- delete endpoints return batch_id (model/recipe/bulk/duplicates) | active | pending exploration: handlers + response shapes
- undo-delete HTTP endpoint + route registration | active | pending exploration: route registrar pattern
- purge scheduling (30s timer + startup sweep + opportunistic) | active | pending exploration: app on_startup hooks
- settings toggle "skip undo window" | active | pending exploration: settings service read pattern
- frontend toast extension (action button + countdown) | active | pending exploration: showToast impl
- frontend delete flows consume batch_id + Undo toast | active | pending exploration: call sites
- confirm-modal friction (delay-activate + size display + cancel focus) | active | pending exploration: modal focus behavior
- i18n keys + sync_translation_keys.py | active | known
## Open assumptions (announced defaults)
<!-- Record any default you adopt instead of asking, so the user can veto it at the gate. -->
<!-- assumption | adopted default | rationale | reversible? -->
- Undo window TTL = 30s | 30s balances space-freeing intent vs accident recovery | yes (constant)
- Staging dir name: `.lm-pending-delete/` under each model root; recipes: `{settings_dir}/.lm-pending-delete/` | hidden, same-volume [updated 2026-08: same-volume is now guaranteed by sibling staging inside the model's own folder, not by the root location], consistent | yes
- Staging failure falls back to existing hard delete | user intent is delete; staging is best-effort; hard delete likely fails identically under same conditions | yes
- Purge on startup uses expires_at (not purge-all) so a <30s restart with live tab can still undo | robust, matches client-side timer | yes
- Settings toggle label: "Delete permanently immediately (skip undo window)" | power users freeing space | yes
- C-friction: delete button enabled after 1.5s + modal shows freed size; NO type-to-confirm (user vetoed) | user explicitly rejected type-to-confirm | n/a
- Bulk/duplicates delete: one batch id for whole action, one undo restores all | simplest consistent semantics | yes
## Findings (cited - path:lines)
### Backend
- `delete_model_artifacts` (py/services/model_lifecycle_service.py:19-48) = physical delete via os.remove; patterns: main file + `{name}.metadata.json` + PREVIEW_EXTENSIONS (py/utils/constants.py:22-37). ALSO called by ModelScanner.bulk_delete_models (py/services/model_scanner.py:2221) - single swap point covers bulk models.
- `ModelLifecycleService.delete_model` (model_lifecycle_service.py:101-154): fetches `cached_entry` (111-116) - SNAPSHOT available for cache restore; after delete: cache.raw_data removal + resort + bump_cache_version (136-143), `_hash_index.remove_by_path` (145-146), `_sync_update_for_model` (148; update-service only, no recipe JSON rewrites - recipe refs are hash-based, re-resolve on restore), `_persist_current_cache` (150-152), returns `{"success": True, "deleted_files": [...]}` (154).
- Handler `delete_model` (py/routes/handlers/model_handlers.py:478-492): POST /api/lm/{prefix}/delete; response passthrough; `_broadcast_models_changed()` (57-74) after success; 400 `{"success":false,"error"}`; 500 plain text.
- Recipe delete: handler (recipe_handlers.py:1422-1438) DELETE /api/lm/recipe/{recipe_id} -> persistence_service.delete_recipe (py/services/recipes/persistence_service.py:193-209): os.remove(recipe_json_path) + os.remove(image_path) (204-206), recipe_scanner.remove_recipe (208), returns `{"success": true, "message": ...}`. PersistenceResult dataclass (20-25).
- Bulk models: POST /api/lm/{prefix}/bulk-delete (model_route_registrar.py:39) -> handler (model_handlers.py:974-994) -> lifecycle_service.bulk_delete_models (model_lifecycle_service.py:308-318) -> scanner.bulk_delete_models (model_scanner.py:2181-2269) which calls delete_model_artifacts per file (2221) + `_batch_update_cache_for_deleted_models` (2271-2335); response `{"success","status","total_deleted","total_attempted","cache_updated","results"}` (2254-2269).
- Bulk recipes: POST /api/lm/recipes/bulk-delete (recipe_route_registrar.py:50) -> handler (recipe_handlers.py:1554-1573) -> persistence_service.bulk_delete (persistence_service.py:439-482): per-id os.remove x2 (464-466), recipe_scanner.bulk_remove (472); response `{"success","deleted","failed","total_deleted","total_failed"}` (474-482).
- Duplicates: NO dedicated delete endpoints (find-only: GET /api/lm/{prefix}/find-duplicates model_route_registrar.py:59, GET /api/lm/recipes/find-duplicates recipe_route_registrar.py:49). Duplicate deletion reuses bulk-delete endpoints.
- Startup hooks: lora_manager.py:183-187 `app.on_startup.append(lambda app: cls._initialize_services())` (ComfyUI mode, app = PromptServer.instance.app at :78); standalone.py:370-374 same (StandaloneLoraManager.add_routes). Background tasks: `asyncio.create_task(name=...)` (lora_manager.py:224-239; recipe_handlers.py:793). Singleton+asyncio.Lock pattern: model_scanner.py:40-63.
- Settings: DEFAULT_SETTINGS (py/services/settings_manager.py:57-119), `get(key, default)` (1390-1392), get_settings_manager() (2215-2228), reset_settings_manager() (2231). Typed-bool getter example: get_skip_previously_downloaded_model_versions (1253-1262). Handlers: base_model_routes.py:70, base_recipe_routes.py:54.
- Model roots: ModelScanner.get_model_roots base NotImplementedError (model_scanner.py:1073-1075); impls lora_scanner.py:31-45, checkpoint_scanner.py:428-441, embedding_scanner.py:24-36. `_find_root_for_file(file_path)` (model_scanner.py:1108-1124) returns containing root - for per-root staging dir computation [updated 2026-08: staging no longer uses the containing root; batches are siblings inside the model's own folder]. Business-path rule (AGENTS.md): use os.path.abspath, never realpath, for staging/undo routing.
- Cache restore methods: ModelCache has raw_data + resort (conftest mocks: tests/conftest.py:144-154); ModelHashIndex.add_entry(sha256, file_path, autov3) (py/services/model_hash_index.py:16); RecipeScanner.add_recipe(recipe_data) (recipe_scanner.py:2136) -> recipe_cache.add_recipe (recipe_cache.py:64). No single-file incremental model rescan - use snapshot restore instead of rescan.
- Route registrar: model_route_registrar.py:177 add_route(method, path, handler), :180 add_prefixed_route - undo endpoint can be a non-prefixed route via add_route.
- Tests: tests/services/test_model_lifecycle_service.py (inline tmp_path files, per-test stub scanners ScannerForDelete/VersionAwareScanner etc); conftest MockScanner/MockCache/MockHashIndex (tests/conftest.py:134-212); integration fixtures tests/integration/conftest.py; lifecycle hook tests tests/routes/test_lora_manager_lifecycle.py:177-178, tests/standalone/test_standalone_server.py:83-84.
### Frontend
- 5 delete call sites:
a) Single model: static/js/utils/modalUtils.js confirmDelete (27-42) -> getModelApiClient().deleteModel(path); ignores return.
b) Recipe single: static/js/components/RecipeCard.js confirmDeleteRecipe (405-449) - RAW fetch DELETE /api/lm/recipe/{id}, checks only response.ok, showToast toast.recipes.deletedSuccessfully, state.virtualScroller.removeItemByFilePath.
c) Bulk: static/js/managers/BulkManager.js confirmBulkDelete (633-672) -> getActiveApiClient() (134-142) -> bulkDeleteModels(filePaths); reads result.cancelled/success/deleted_count/error.
d) Recipe duplicates: static/js/components/DuplicatesManager.js confirmDeleteDuplicates (457-494) - RAW fetch POST /api/lm/recipes/bulk-delete, reads data.success/data.total_deleted, exitDuplicateMode().
e) Model duplicates: static/js/components/ModelDuplicatesManager.js confirmDeleteDuplicates (710-776) - RAW fetch POST /api/lm/{type}/bulk-delete, reads data.total_deleted, then resetAndReload(true) + find-duplicates re-check.
Bonus: static/js/components/shared/ModelVersionsTab.js:1136-1144 client.deleteModel (ignores return).
- API clients: BaseModelApiClient.deleteModel (static/js/api/baseModelApi.js:184-216) returns true/false, shows its own toasts, does removeItemByFilePath inside; bulkDeleteModels (1591-1642) returns {success, deleted_count, failed_count, errors} or {success:false, cancelled:true}; RecipeSidebarApiClient.bulkDeleteModels (recipeApi.js:623-664) returns {success, deleted_count: total_deleted, ...}. Endpoint map apiConfig.js:56,64.
- Toast: showToast(key, params={}, type='info', fallback=null) (static/js/utils/uiHelpers.js:136-193) - textContent only, NO action/button support; durations 2000/5000ms; CSS static/css/components/toast.css (.toast flex gap:12px - button can be added). Closest action pattern: bannerService.registerBanner actions array + onRegister (static/js/managers/BannerService.js; used uiHelpers.js:18-57).
- i18n: locales/en.json delete keys (1303-1314 bulkDelete, 1945-1948 recipes, 1987-1991 models, 2124-2130 duplicates, 2166-2170 toast.api); t()/interpolate (static/js/i18n/index.js:193-248); translate wrapper (utils/i18nHelpers.js:13-23); sync script scripts/sync_translation_keys.py (en reference, [TODO: Translate] placeholders).
- Refresh after undo: recipes -> window.recipeManager.loadRecipes(true) (recipes.js:359; used by FilterManager.js:752 etc) or refreshRecipes (recipeApi.js:308); models -> resetAndReload(true) from modelApiFactory (used by ModelDuplicatesManager.js:740).
- Size for modal: card.dataset.file_size (ModelCard.js:467), formatFileSize (ModelModal.js:615).
- Tests: tests/frontend/utils/uiHelpers.dom.test.js (toast), api/recipeApi.bulk.test.js, components/duplicatesManager.test.js, components/modelDuplicatesManager.test.js, pages/*Page.test.js, i18n tests tests/i18n/test_i18n.py.
## Decisions (with rationale)
1. Same-volume rename staging for model files (atomic, no copy cost for multi-GB files); cross-volume rename forbidden. [CORRECTED 2026-08: "same-volume because under the containing root" was only true for plain directories — nested symlinked subdirs could cross volumes. Superseded by sibling staging: `.lm-pending-delete/<batch_id>/` inside the deleted model's own folder makes stage/undo same-device by construction; see "Symlink fix (2026-08)" below.]
2. Copy-to-global-staging for recipes (small files; avoids recipe JSON vs preview image cross-volume problem).
3. Manifest JSON files are the only state - no DB changes. Manifest includes model cached_entry snapshot for exact cache restore (no rescan needed).
4. Undo endpoint returns restored paths; expired batch -> 404-style error -> frontend 'undo expired' toast.
5. Skip-undo setting honored server-side (no batch_id in response -> no undo toast client-side).
6. Staging failure falls back to existing hard delete (best-effort undo, never blocks delete).
7. Undo window TTL = 30s constant (PENDING_DELETE_TTL_SECONDS); startup sweep uses expires_at (survives restart; browser-tab timer survives).
8. Purge triple-trigger: per-batch asyncio timer task + on_startup sweep + opportunistic purge at each stage/undo.
9. Frontend: new showActionToast (keep showToast signature untouched; extract shared createToastElement/appendToast internals); undo click -> shared handleUndoDelete(batchId, refreshFn); full list refresh after undo (recipes: window.recipeManager.loadRecipes(true); models: resetAndReload(true)).
10. C-friction wave (NO type-to-confirm - user vetoed): delete buttons delay-activate 1.5s after modal open, initial focus on Cancel, model delete modal gains "permanently deleted from disk" warning + file size display (card.dataset.file_size + formatFileSize).
11. Model cache restore on undo: append snapshot to cache.raw_data (dedupe by file_path) + resort + bump_cache_version + _persist_current_cache + _hash_index.add_entry + _broadcast_models_changed. Recipe restore: copy back files + recipe_scanner.add_recipe(recipe_data loaded from restored JSON).
### Symlink fix (2026-08)
Post-execution addendum (plan `.omo/plans/undo-delete-symlink-fix.md`, commits 5fd4946b / 0c00ee22):
12. Model staging moved from `<model_root>/.lm-pending-delete/<batch_id>/` to `<model_dir>/.lm-pending-delete/<batch_id>/` (sibling of the model artifacts, inside the deleted model's own folder). Stage/undo renames are same-device BY CONSTRUCTION — EXDEV is impossible even when the business path traverses nested symlinks to other volumes (the decision-1 "containing root" guarantee covered only plain directories). EXDEV remains possible only for cross-volume merges, which keep the batch_ids-array fallback. Accepted edge: deleting the model's whole FOLDER during the 30s window destroys that batch (undo returns 404). Batch discovery uses an in-memory registry (`_known_batch_dirs`) with a startup reconciliation scan (`purge_expired(scan_roots=True)`) covering restarts and crash leftovers. Recipe batches unchanged (copy-based settings-dir staging with the `_restore_file` EXDEV fallback).
## Scope IN
- Model single delete (model_handlers delete_model / model_lifecycle_service)
- Recipe delete (recipe_handlers delete_recipe / persistence_service)
- Bulk delete (models scanner + recipes persistence) + duplicates (reuse bulk endpoints)
- Undo endpoint POST /api/lm/undo-delete (models + recipes, one batch space)
- Purge: timer + startup sweep + opportunistic
- Settings toggle delete_undo_enabled + settings page checkbox
- Frontend: showActionToast + all 5 delete flows + shared undo handler
- C-friction modal changes (delay-activate + cancel focus + warning copy + size display)
- i18n keys + sync_translation_keys.py
- Backend + frontend tests
## Scope OUT (Must NOT have)
- NO type-to-confirm / hold-to-confirm friction (user vetoed)
- NO OS trash integration (send2trash) in this iteration
- NO persistent recycle-bin UI (no trash browsing page)
- NO changes to exclude/unexclude flow
- NO DB migrations
- NO new dependencies (no send2trash)
- NO changes to download flows
- NO recipe-JSON rewriting on model undo (hash-based refs re-resolve themselves)
## Open questions
None - all implementation details resolved by exploration. Design decisions settled in conversation (B+C, no type-to-confirm).
## Approval gate
status: approved
<!-- Approach approved -> rerun scaffold without --draft-only, run Metis gap analysis, APPEND todo batches, fill TL;DR last, run structural self-check, then Phase 4 handoff. -->
## Review round state (ulw-plan-review-round-state-contract)
```json
{
"transition": "replace",
"phase": "review_round_initialized",
"applies_when": ["retry_after_plan_change"],
"atomic": true,
"review_required": true,
"plan_path": ".omo/plans/undo-delete-staging.md",
"plan_sha256": "8cf7c9be38a76d8ef1fb832aba043d6d7e82b60465b1bf28c6eafa7045117adc",
"review_round_id": "rr-undo-del-20260811-006",
"round_status": "active",
"pending-action": "review .omo/plans/undo-delete-staging.md",
"review": {
"momus": { "status": "pending", "workspace_root": "/mnt/data/reinstall-backup-2026-04-12/data/workspace/ComfyUI/custom_nodes/ComfyUI-Lora-Manager", "runtime_home": null, "target": ".omo/plans/undo-delete-staging.md", "round_id": "rr-undo-del-20260811-006", "plan_sha256": "8cf7c9be38a76d8ef1fb832aba043d6d7e82b60465b1bf28c6eafa7045117adc", "launch_id": null, "session": null, "result": null },
"independent": { "status": "pending", "workspace_root": "/mnt/data/reinstall-backup-2026-04-12/data/workspace/ComfyUI/custom_nodes/ComfyUI-Lora-Manager", "runtime_home": null, "target": ".omo/plans/undo-delete-staging.md", "round_id": "rr-undo-del-20260811-006", "plan_sha256": "8cf7c9be38a76d8ef1fb832aba043d6d7e82b60465b1bf28c6eafa7045117adc", "launch_id": null, "session": null, "result": null }
}
}
```
## Review results + fix/retry ledger
### Round 1 (rr-undo-del-20260811-001, plan sha256 6c52bf99...)
- momus: APPROVE (non-blocking notes: todo1+7 duplicate DEFAULT_SETTINGS key -> fixed todo 7 to verify-only; "batch_ids" plural in todos 8/9 acceptance -> fixed; purge OSError note -> folded into todo 1 purge semantics)
- independent (oracle): CHANGES_REQUESTED
- BLOCKING S1: scanner walks would index .lm-pending-delete staged files as ghost entries -> fixed: todo 1 now mandates scanner walk exclusion at model_scanner.py:706/:867/:1404/_process_model_file + acceptance (o) scanner-visibility test
- BLOCKING S2: manifest lacks model_type, undo could restore into wrong cache/hash index -> fixed: manifest now carries model_type + todo 5 resolves per-type scanner via registrar pattern + acceptance (b) checkpoint-batch test
- S3 merged-batch expires_at re-anchor -> fixed: merge_batches re-anchors now+TTL in todo 1 + todo 3/4 assertions
- S4 manifest-less dir policy -> fixed: quarantine to <batch_id>.orphaned, never delete (todo 1 + acceptance g)
- S5 partial-undo retry semantics -> fixed: per-entry restored flag write-through + retry test (acceptance e)
- S6 purge locked-file failure semantics -> fixed: skip file, keep batch, never rmtree past errors (todo 1 + acceptance i)
- T8 undo-after-restart test -> fixed: todo 5 acceptance (f)
- T9 recipe undo -> re-delete test -> fixed: todo 5 acceptance (h)
- T7 rescan-stale-entry test -> fixed: todo 5 acceptance (g)
- Route registration pinned to shared routes class per mode (NOT per-model-type registrar which registers 3x) -> fixed: todo 5 now creates py/routes/pending_delete_routes.py registered once in lora_manager.py:170-172 + standalone.py:356-358 + duplicate-route test (e)
- Version-index staleness on single-delete undo -> fixed: todo 5 follows bulk cache-update pattern incl. rebuild_version_index (model_scanner.py:2324)
- Cancelled-bulk batch_id frontend handling -> fixed: todo 9 shows action toast on cancelled+staged-subset
- Single-instance assumption -> added to Scope OUT
- Occupied-refusal loss UX -> accepted-intent documented in success criteria + modal copy
### Round 2 (rr-undo-del-20260811-002, plan sha256 f3d52235...)
- momus: APPROVE (all 12 round-1 fixes verified present; zero dead references; non-blocking nits only)
- independent (oracle): CHANGES_REQUESTED
- BLOCK-1: merge_batches file-movement semantics unspecified (silent data-loss vector) -> fixed: todo 1 now specifies move-into-winner-dir + entry re-point + loser-dirs-removed-only-when-empty + abort-on-move-failure (all batches intact) + merge inside service lock + acceptance (k) file-survival assertions + acceptance (l) merge-failure abort test
- BLOCK-2: same-file parallel edits within waves (todo 5 vs 6 on lora_manager.py; todo 8 vs 9 on baseModelApi.js) -> fixed: waves/matrix now serialize 5->6 and 8->9 with explicit reasons; matrix updated
- Recommended: checkpoint_scanner.py:331 exclusion -> fixed (todo 1 + acceptance p); S5 pre-check skips restored:true entries -> fixed (todo 1); _tags_count restore on undo -> fixed (todo 5 + acceptance j); undo-blind flows documented (ModelVersionsTab + misc_handlers:2456) -> fixed (todo 8 note + Scope OUT); merge-failure no-merge fallback contract (batch_ids array) -> fixed (todos 3/4/9)
### Round 3 (rr-undo-del-20260811-003, plan sha256 8f2dfd46...)
- momus: APPROVE (all round-2 fixes verified present + spot-checked refs; no new contradictions)
- independent (oracle): CHANGES_REQUESTED
- BLOCKING A: merged batches never timer-purged after re-anchor (winner's original timer no-ops at old expiry; no fresh timer for re-anchored expiry; idle server -> merged batch lingers, violating "30s purge" success criterion; affects EVERY bulk delete) -> fixed: todo 1 merge_batches now ARMS A FRESH PURGE TIMER for the winner with re-anchored expiry + acceptance (q) fresh-timer test + purge_expired must enumerate ALL scanner types' roots (explicit in todo 1)
- BLOCKING B: dependency matrix contradicted same-file policy for todos 8/9<->11 (5 shared files) and 12<->11 -> fixed: todo 11 now "Blocked by: 8, 9 (same files...)"; todo 12 blocked by 11 (sync after 11); wave text updated (11, then 12 AFTER 11); "Can parallelize with" columns corrected
- BLOCKING C: frontend batch_ids sequential-undo fallback has NO test + merge->undo loser-restore + merge->purge assertions missing -> fixed: todo 9 acceptance now tests the batch_ids fallback path; todo 1 acceptance now has (k2)/(k3)
- Notes folded: sub-second toast-tail expiry race accepted; EXDEV fallback = NORMAL path for cross-volume bulks [annotated 2026-08: after the sibling-staging fix, EXDEV can only arise during cross-volume MERGES, never during single stage/undo renames]
### Round 4 (rr-undo-del-20260811-004, plan sha256 179e7ff7...)
- momus: APPROVE (round-3 fixes verified; one non-blocking nit: todo 11 inline "Blocked by: —" stale -> fixed to "8, 9")
- independent (oracle): CHANGES_REQUESTED
- BLOCKING GAP-1 (NEW, introduced by round-3 fix): todo 8 handleUndoDelete always-refresh/always-toast contract contradicted todo 9's sequential loop "exactly ONE final refresh" -> fixed: handleUndoDelete(batchId, refreshFn, {showToast, refresh}) suppression options; todo 9 loop uses suppressed calls + one final refresh/toast; acceptance extended (loop failure mid-way -> stop + error toast + no final refresh; 404 body discrimination expired vs occupied)
- BLOCKING GAP-2: no cross-type purge enumeration test -> fixed: todo 1 acceptance (r) purges expired batches across lora root + checkpoint root + recipe staging dir in one call
- Non-blocking folded: GAP-3 404-copy discrimination -> fixed in todo 8 (d); GAP-4 merge partial-failure rollback direction (move back + restore manifests, extended (l) asserts sequential constituent undo still restores everything) -> fixed in todo 1; GAP-5 post-restart timer-loss residual gap documented -> fixed in todo 6; GAP-6 usage_stats.py:424 walk added to exclusion mandate + todo 5 acceptance (k) embeddings undo test
### Round 5 (rr-undo-del-20260811-005, plan sha256 dfaa39ea...)
- momus: APPROVE (all round-4 fixes verified; no new contradictions)
- independent (oracle): CHANGES_REQUESTED
- BLOCK-1: lock-ordering deadlock ambiguity (asyncio.Lock not re-entrant: opportunistic purge_expired called while stage/undo hold the lock would deadlock on first use) -> fixed: todo 1 now has explicit LOCK HIERARCHY (lock acquired ONLY by stage/merge/undo/purge_batch; purge_expired is lock-free and must be called BEFORE lock acquisition); todo 6 (c) updated with the same rule + acceptance (u) lock-no-deadlock test
- BLOCK-2: purge edge semantics unspecified -> fixed: purge_batch treats missing staged files (partially-restored batches) as already-purged (FileNotFoundError silent no-op); sweep skips `.orphaned`-suffixed dirs (quarantine is terminal); acceptance (s) partially-restored purge + (t) quarantine-terminal tests
- Non-blocking folded: todo 2/3 test-file collision -> todo 3's bulk tests moved to tests/services/test_model_scanner.py; todo 9 (d) DuplicatesManager refreshFn stated explicitly (recipes loadRecipes / models resetAndReload); modal-copy + bulk-count trade-offs acknowledged in success criteria; acceptance (r) extended with embeddings root
### Round 6 (rr-undo-del-20260811-006, plan sha256 8cf7c9be...)
- momus: APPROVE (all round-5 fixes verified; no new contradictions; references verified)
- independent (oracle): APPROVE — no blocking issues; all round-5 items fixed with working, tested solutions; no new race/data-loss/consistency defects
- Deferred optional improvements (non-blocking, recorded for executor awareness; plan file left untouched to preserve the approved digest):
1. Tag-count asymmetry: single delete_model never decrements _tags_count (lifecycle 101-154), bulk does (scanner 2297-2303); undo re-increment is exact for bulk, over-counts for single until rescan (cosmetic, self-healing). Optional fix riding in todo 2: decrement tags in the single-delete path to mirror bulk.
2. Todo 5 factual nit: ModelCache.resort() already rebuilds the version index — explicit rebuild in undo is belt-and-braces, no action needed.
3. Todo 8 premise nit: ModelVersionsTab call ignores deleteModel's return entirely — nothing breaks, no adaptation needed.
4. Todo 3's pytest command includes test_model_lifecycle_service.py which todo 2 edits in the same wave — run that file's tests after todo 2 lands.
5. merge_batches with a missing/quarantined constituent id: any sane fallback (abort -> batch_ids, or skip missing) acceptable — files stay staged either way.
## Review lifecycle
- rounds: 6 (rr-undo-del-20260811-001..006); final round both lanes APPROVE
- final live-plan validation: sha256 = 8cf7c9be38a76d8ef1fb832aba043d6d7e82b60465b1bf28c6eafa7045117adc — MATCHES approved round-6 digest
- status: APPROVED — ready for execution handoff ($start-work undo-delete-staging)
File diff suppressed because one or more lines are too long
+122 -49
View File
@@ -2,6 +2,10 @@
This file provides guidance for agentic coding assistants working in this repository.
## Overview
ComfyUI LoRA Manager is a comprehensive LoRA management system for ComfyUI that combines a Python backend with browser-based widgets. It provides model organization, downloading from CivitAI/CivArchive, recipe management, and one-click workflow integration.
## Development Commands
### Backend Development
@@ -28,16 +32,21 @@ COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py --cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
### Frontend Development (Standalone Web UI)
### Frontend Development (LoRA Manager Web UI)
```bash
# Install dependencies (root and Vue widgets)
npm install
cd vue-widgets && npm install && cd ..
npm test # Run all tests (JS + Vue)
npm run test:js # Run JS tests only
npm run test:watch # Watch mode
npm run test:vue # Run Vue widget tests only
npm run test:watch # Watch mode (JS tests only)
npm run test:coverage # Generate coverage report
```
@@ -54,105 +63,169 @@ npm run test:watch # Watch mode
npm run test:coverage # Generate coverage report
```
## Python Code Style
### Localization
### Imports & Formatting
```bash
# Sync translation keys after UI string updates
python scripts/sync_translation_keys.py
```
Locale files are in `locales/` (en, zh-CN, zh-TW, ja, ko, fr, de, es, ru, he).
**Before translating anything, read `docs/i18n-translation-guidelines.md`** — it defines the
term conventions (e.g. "Recipe" stays untranslated in French, 配方 in Chinese; model-type and
brand names are never translated), per-locale preferred renderings, placeholder rules, and
the known confusion hot-spots.
## Code Style
### Python
#### Imports & Formatting
- Use `from __future__ import annotations` for forward references
- Group imports: standard library, third-party, local (blank line separated)
- Use `TYPE_CHECKING` guard for type-checking-only imports
- Absolute imports within `py/`: `from ..services import X`
- PEP 8 with 4-space indentation, type hints required
### Naming Conventions
#### Naming Conventions
- Files: `snake_case.py`, Classes: `PascalCase`, Functions/vars: `snake_case`
- Constants: `UPPER_SNAKE_CASE`, Private: `_protected`, `__mangled`
### Error Handling & Async
#### Error Handling & Async
- Use `logging.getLogger(__name__)`, define custom exceptions in `py/services/errors.py`
- `async def` for I/O, `@pytest.mark.asyncio` for async tests
- Singleton with `asyncio.Lock`: see `ModelScanner.get_instance()`
- Return `aiohttp.web.json_response` or `web.Response`
### Testing
### JavaScript/TypeScript
- `pytest` with `--import-mode=importlib`
- Fixtures in `tests/conftest.py`, use `tmp_path_factory` for isolation
- Mark tests needing real paths: `@pytest.mark.no_settings_dir_isolation`
- Mock ComfyUI dependencies via conftest patterns
## JavaScript/TypeScript Code Style
### Imports & Modules
#### Imports & Modules
- ES modules: `import { app } from "../../scripts/app.js"` for ComfyUI
- Vue: `import { ref, computed } from 'vue'`, type imports: `import type { Foo }`
- Export named functions: `export function foo() {}`
### Naming & Formatting
#### Naming & Formatting
- camelCase for functions/vars/props, PascalCase for classes
- Constants: `UPPER_SNAKE_CASE`, Files: `snake_case.js` or `kebab-case.js`
- 2-space indentation preferred (follow existing file conventions)
- Vue Single File Components: `<script setup lang="ts">` preferred
### Widget Development
#### Widget Development
- Prefer vanilla JS for `web/comfyui/` widgets; avoid framework dependencies (except the Vue widgets in `vue-widgets/`)
- ComfyUI: `app.registerExtension()`, `node.addDOMWidget(name, type, element, options)`
- Event handlers via `addEventListener` or widget callbacks
- Shared utilities: `web/comfyui/utils.js`
- Dual-mode rendering patterns (canvas vs Vue): see `docs/comfyui-dual-mode-widgets.md`
### Vue Composables Pattern
#### Vue Composables Pattern
- Use composition API: `useXxxState(widget)`, return reactive refs and methods
- Guard restoration loops with flag: `let isRestoring = false`
- Build config from state: `const buildConfig = (): Config => { ... }`
## Architecture Patterns
## Architecture
### Dual Mode Operation
The system runs in two modes:
- **ComfyUI plugin mode**: Integrates with ComfyUI's PromptServer, uses `folder_paths` for model discovery
- **Standalone mode**: `standalone.py` mocks ComfyUI dependencies, reads paths from `settings.json`
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
### Backend Entry Points
- `__init__.py` — ComfyUI plugin entry: registers nodes via `NODE_CLASS_MAPPINGS`, sets `WEB_DIRECTORY`, calls `LoraManager.add_routes()`
- `standalone.py` — Standalone server: mocks `folder_paths` and node modules, starts aiohttp server
- `py/lora_manager.py` — Main `LoraManager` class that registers all HTTP routes
### Service Layer
- `ServiceRegistry` singleton for DI, services use `get_instance()` classmethod
- `BaseModelService` abstract base → `LoraService`, `CheckpointService`, `EmbeddingService`
- `ModelScanner` base → `LoraScanner`, `CheckpointScanner`, `EmbeddingScanner` for file discovery with hash-based deduplication
- `PersistentModelCache` (SQLite) for metadata persistence
- `MetadataSyncService` — background sync from CivitAI/CivArchive APIs
- `SettingsManager` — settings with schema migration support
- `WebSocketManager` — real-time progress broadcasting
- `ModelServiceFactory` — creates the right service for each model type
- Use cases in `py/services/use_cases/` orchestrate complex business logic (auto-organize, bulk refresh, downloads)
- Separate scanners (discovery) from services (business logic)
- Handlers in `py/routes/handlers/` are pure functions with deps as params
### Model Types & Routes
- `BaseModelService` base for LoRA, Checkpoint, Embedding
- `ModelScanner` for file discovery, hash deduplication
- `PersistentModelCache` (SQLite) for persistence
- Route registrars: `ModelRouteRegistrar`, endpoints: `/loras/*`, `/checkpoints/*`, `/embeddings/*`
- WebSocket via `WebSocketManager` for real-time updates
- API endpoints follow `/loras/*`, `/checkpoints/*`, `/embeddings/*` patterns
- Route registrars organize endpoints by domain: `ModelRouteRegistrar`, `RecipeRouteRegistrar`, etc.
- Request handlers in `py/routes/handlers/` implement route logic
- All routes use aiohttp, return `web.json_response` or `web.Response`
### Recipe System
- Base: `py/recipes/base.py`, Enrichment: `RecipeEnrichmentService`
- Parsers: `py/recipes/parsers/`
- Base: `py/recipes/base.py`, Enrichment: `RecipeEnrichmentService` in `py/recipes/enrichment.py`
- Parsers: `py/recipes/parsers/` for PNG metadata, JSON, and workflow formats
### Custom Nodes
- Location: `py/nodes/`, all nodes registered in `__init__.py`
- Each node class has a `NAME` class attribute used as key in `NODE_CLASS_MAPPINGS`
- Standard ComfyUI node pattern: `INPUT_TYPES()` classmethod, `RETURN_TYPES`, `FUNCTION`
### Configuration
- `py/config.py` manages folder paths for models and handles symlink mappings
- Auto-saves paths to `settings.json` in ComfyUI mode
### Frontend UI Architecture
#### 1. LoRA Manager Web UI
- Location: `./static/` (JS/CSS) and `./templates/` (HTML)
- Tech: Vanilla JS + CSS, served by the hosting server (ComfyUI app in plugin mode, `standalone.py` in standalone mode)
- Tests: `tests/frontend/**/*.test.js` (vitest + jsdom)
#### 2. ComfyUI Custom Node Widgets
- Location: `./web/comfyui/` (Vanilla JS) + `./vue-widgets/` (Vue)
- Primary styles: `./web/comfyui/lm_styles.css` (NOT `./static/css/`)
- Vue widgets: Vue 3 + TypeScript + PrimeVue + vue-i18n, e.g. `LoraPoolWidget`, `LoraRandomizerWidget`, `LoraCyclerWidget`, `AutocompleteTextWidget`
- Vue builds to `./web/comfyui/vue-widgets/`; auto-built on ComfyUI startup via `py/vue_widget_builder.py`, typecheck via `vue-tsc`
- Widget registration: `app.registerExtension()` and `getCustomWidgets` hooks; `node.addDOMWidget(...)` embeds HTML in LiteGraph nodes
- See `docs/dom_widget_dev_guide.md` for the DOMWidget development guide
## Testing
### Backend (pytest)
- Config in `pytest.ini`: `--import-mode=importlib`, testpaths=`tests`
- Fixtures in `tests/conftest.py` mock ComfyUI dependencies; use `tmp_path_factory` for isolation
- Markers: `@pytest.mark.asyncio`, `@pytest.mark.no_settings_dir_isolation` (tests needing real settings paths)
### Frontend (vitest)
- Vanilla JS tests: `tests/frontend/**/*.test.js` with jsdom; setup in `tests/frontend/setup.js`
- Vue widget tests: `vue-widgets/tests/**/*.test.ts` with jsdom + `@vue/test-utils`
## Key Integration Points
- **Settings:** Stored in the user config directory (via `platformdirs`) or portable mode (`"use_portable_settings": true`)
- **CivitAI/CivArchive:** API clients for metadata sync and model downloads; CivitAI API key stored in settings
- **Symlinks:** Config scans symlinks to map virtual→physical paths; fingerprinting prevents redundant rescans
- **WebSocket:** Broadcasts real-time progress for downloads, scans, and metadata sync
- **Model scanning flow:** Walk folders → compute hashes → deduplicate → extract safetensors metadata → cache in SQLite → background CivitAI sync → WebSocket broadcast
## Important Notes
- ALWAYS use English for comments (per copilot-instructions.md)
- Dual mode: ComfyUI plugin (folder_paths) vs standalone (settings.json)
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
- Run `python scripts/sync_translation_keys.py` after adding UI strings to `locales/en.json`
- Symlinks require normalized paths
## Git / Commit Messages
- Follow the style of recent repository commits when writing commit messages
- Prefer the repo's existing `feat(...)`, `fix(...)`, `chore:` style where applicable
- If the user has provided a GitHub issue link or issue ID for the task, mention that issue in the commit message, for example `(#871)`
- When unrelated local changes exist, stage and commit only the files relevant to the requested task
## Frontend UI Architecture
### 1. Standalone Web UI
- Location: `./static/` and `./templates/`
- Tech: Vanilla JS + CSS, served by standalone server
- Tests via npm in root directory
### 2. ComfyUI Custom Node Widgets
- Location: `./web/comfyui/` (Vanilla JS) + `./vue-widgets/` (Vue)
- Primary styles: `./web/comfyui/lm_styles.css` (NOT `./static/css/`)
- Vue builds to `./web/comfyui/vue-widgets/`, typecheck via `vue-tsc`
- Symlinks require normalized paths.
**Business paths vs real paths**: All stored paths and operation routing use the
original paths as they appear under configured model roots — symlinks are NOT
resolved. `os.path.realpath` is only for scanner dedup and the symlink cache.
Any path passed to `os.remove`/`os.rename`/`shutil.move` or validated by a
containment check MUST use the business path (i.e. `os.path.abspath`, not
`realpath`).
-189
View File
@@ -1,189 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Overview
ComfyUI LoRA Manager is a comprehensive LoRA management system for ComfyUI that combines a Python backend with browser-based widgets. It provides model organization, downloading from CivitAI/CivArchive, recipe management, and one-click workflow integration.
## Development Commands
### Backend
```bash
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run standalone server (port 8188 by default)
python standalone.py --port 8188
# Run all backend tests
pytest
# Run specific test file or function
pytest tests/test_recipes.py
pytest tests/test_recipes.py::test_function_name
# Run backend tests with coverage
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
### Frontend
There are three test suites run by `npm test`: vanilla JS tests (vitest at root) and Vue widget tests (`vue-widgets/` vitest).
```bash
npm install
cd vue-widgets && npm install && cd ..
# Run all frontend tests (JS + Vue)
npm test
# Run only vanilla JS tests
npm run test:js
# Run only Vue widget tests
npm run test:vue
# Watch mode (JS tests only)
npm run test:watch
# Frontend coverage
npm run test:coverage
# Build Vue widgets (output to web/comfyui/vue-widgets/)
cd vue-widgets && npm run build
# Vue widget dev mode (watch + rebuild)
cd vue-widgets && npm run dev
# Typecheck Vue widgets
cd vue-widgets && npm run typecheck
```
### Localization
```bash
# Sync translation keys after UI string updates
python scripts/sync_translation_keys.py
```
Locale files are in `locales/` (en, zh-CN, zh-TW, ja, ko, fr, de, es, ru, he).
## Architecture
### Dual Mode Operation
The system runs in two modes:
- **ComfyUI plugin mode**: Integrates with ComfyUI's PromptServer, uses `folder_paths` for model discovery
- **Standalone mode**: `standalone.py` mocks ComfyUI dependencies, reads paths from `settings.json`
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
### Backend (Python)
**Entry points:**
- `__init__.py` — ComfyUI plugin entry: registers nodes via `NODE_CLASS_MAPPINGS`, sets `WEB_DIRECTORY`, calls `LoraManager.add_routes()`
- `standalone.py` — Standalone server: mocks `folder_paths` and node modules, starts aiohttp server
- `py/lora_manager.py` — Main `LoraManager` class that registers all HTTP routes
**Service layer** (`py/services/`):
- `ServiceRegistry` singleton for dependency injection; services follow `get_instance()` singleton pattern
- `BaseModelService` abstract base → `LoraService`, `CheckpointService`, `EmbeddingService`
- `ModelScanner` base → `LoraScanner`, `CheckpointScanner`, `EmbeddingScanner` for file discovery with hash-based deduplication
- `PersistentModelCache` — SQLite-based metadata cache
- `MetadataSyncService` — Background sync from CivitAI/CivArchive APIs
- `SettingsManager` — Settings with schema migration support
- `WebSocketManager` — Real-time progress broadcasting
- `ModelServiceFactory` — Creates the right service for each model type
- Use cases in `py/services/use_cases/` orchestrate complex business logic (auto-organize, bulk refresh, downloads)
**Routes** (`py/routes/`):
- Route registrars organize endpoints by domain: `ModelRouteRegistrar`, `RecipeRouteRegistrar`, etc.
- Request handlers in `py/routes/handlers/` implement route logic
- API endpoints follow `/loras/*`, `/checkpoints/*`, `/embeddings/*` patterns
- All routes use aiohttp, return `web.json_response` or `web.Response`
**Recipe system** (`py/recipes/`):
- `base.py` — Recipe metadata structure
- `enrichment.py` — Enriches recipes with model metadata
- `parsers/` — Parsers for PNG metadata, JSON, and workflow formats
**Custom nodes** (`py/nodes/`):
- Each node class has a `NAME` class attribute used as key in `NODE_CLASS_MAPPINGS`
- Standard ComfyUI node pattern: `INPUT_TYPES()` classmethod, `RETURN_TYPES`, `FUNCTION`
- All nodes registered in `__init__.py`
**Configuration** (`py/config.py`):
- Manages folder paths for models, handles symlink mappings
- Auto-saves paths to settings.json in ComfyUI mode
### Frontend — Two Distinct UI Systems
#### 1. Standalone Manager Web UI
- **Location:** `static/` (JS/CSS) and `templates/` (HTML)
- **Tech:** Vanilla JS + CSS, served by standalone server
- **Structure:** `static/js/core.js` (shared), `loras.js`, `checkpoints.js`, `embeddings.js`, `recipes.js`, `statistics.js`
- **Tests:** `tests/frontend/**/*.test.js` (vitest + jsdom)
#### 2. ComfyUI Custom Node Widgets
- **Vanilla JS widgets:** `web/comfyui/*.js` — ES modules extending ComfyUI's LiteGraph UI
- `loras_widget.js` / `loras_widget_events.js` — Main LoRA selection widget
- `autocomplete.js` — Trigger word and embedding autocomplete
- `preview_tooltip.js` — Model card preview tooltips
- `top_menu_extension.js` — "Launch LoRA Manager" menu item
- `utils.js` — Shared utilities and API helpers
- Widget styling in `web/comfyui/lm_styles.css` (NOT `static/css/`)
- **Vue widgets:** `vue-widgets/src/` → built to `web/comfyui/vue-widgets/`
- Vue 3 + TypeScript + PrimeVue + vue-i18n
- Vite build with CSS-injected-by-JS plugin
- Components: `LoraPoolWidget`, `LoraRandomizerWidget`, `LoraCyclerWidget`, `AutocompleteTextWidget`
- Auto-built on ComfyUI startup via `py/vue_widget_builder.py`
- Tests: `vue-widgets/tests/**/*.test.ts` (vitest)
**Widget registration pattern:**
- Widgets use `app.registerExtension()` and `getCustomWidgets` hooks
- `node.addDOMWidget(name, type, element, options)` embeds HTML in LiteGraph nodes
- See `docs/dom_widget_dev_guide.md` for DOMWidget development guide
## Code Style
**Python:**
- PEP 8, 4-space indentation, English comments only
- Use `from __future__ import annotations` for forward references
- Use `TYPE_CHECKING` guard for type-checking-only imports
- Loggers via `logging.getLogger(__name__)`
- Custom exceptions in `py/services/errors.py`
- Async patterns: `async def` for I/O, `@pytest.mark.asyncio` for async tests
- Singleton pattern with class-level `asyncio.Lock` (see `ModelScanner.get_instance()`)
**JavaScript:**
- ES modules, camelCase functions/variables, PascalCase classes
- Widget files use `*_widget.js` suffix
- Prefer vanilla JS for `web/comfyui/` widgets, avoid framework dependencies (except Vue widgets)
## Testing
**Backend (pytest):**
- Config in `pytest.ini`: `--import-mode=importlib`, testpaths=`tests`
- Fixtures in `tests/conftest.py` handle ComfyUI dependency mocking
- Markers: `@pytest.mark.asyncio`, `@pytest.mark.no_settings_dir_isolation`
- Uses `tmp_path_factory` for directory isolation
**Frontend (vitest):**
- Vanilla JS tests: `tests/frontend/**/*.test.js` with jsdom
- Vue widget tests: `vue-widgets/tests/**/*.test.ts` with jsdom + @vue/test-utils
- Setup in `tests/frontend/setup.js`
## Key Integration Points
- **Settings:** Stored in user directory (via `platformdirs`) or portable mode (`"use_portable_settings": true`)
- **CivitAI/CivArchive:** API clients for metadata sync and model downloads; CivitAI API key in settings
- **Symlink handling:** Config scans symlinks to map virtual→physical paths; fingerprinting prevents redundant rescans
- **WebSocket:** Broadcasts real-time progress for downloads, scans, and metadata sync
- **Model scanning flow:** Walk folders → compute hashes → deduplicate → extract safetensors metadata → cache in SQLite → background CivitAI sync → WebSocket broadcast
+2 -7
View File
File diff suppressed because one or more lines are too long
+28
View File
@@ -3,6 +3,8 @@ try: # pragma: no cover - import fallback for pytest collection
from .py.nodes.lora_loader import LoraLoaderLM, LoraTextLoaderLM
from .py.nodes.checkpoint_loader import CheckpointLoaderLM
from .py.nodes.unet_loader import UNETLoaderLM
from .py.nodes.random_checkpoint_loader import RandomCheckpointLoaderLM
from .py.nodes.random_unet_loader import RandomUNETLoaderLM
from .py.nodes.trigger_word_toggle import TriggerWordToggleLM
from .py.nodes.prompt import PromptLM
from .py.nodes.text import TextLM
@@ -15,6 +17,10 @@ try: # pragma: no cover - import fallback for pytest collection
from .py.nodes.lora_pool import LoraPoolLM
from .py.nodes.lora_randomizer import LoraRandomizerLM
from .py.nodes.lora_cycler import LoraCyclerLM
from .py.nodes.lora_info import LoraInfoLM
from .py.nodes.lora_syntax_to_path import LoraSyntaxToPath
from .py.nodes.create_hook_lora import CreateHookLoraLM
from .py.nodes.metadata_overwrite import MetadataOverwriteLM
from .py.metadata_collector import init as init_metadata_collector
except (
ImportError
@@ -36,6 +42,12 @@ except (
"py.nodes.checkpoint_loader"
).CheckpointLoaderLM
UNETLoaderLM = importlib.import_module("py.nodes.unet_loader").UNETLoaderLM
RandomCheckpointLoaderLM = importlib.import_module(
"py.nodes.random_checkpoint_loader"
).RandomCheckpointLoaderLM
RandomUNETLoaderLM = importlib.import_module(
"py.nodes.random_unet_loader"
).RandomUNETLoaderLM
TriggerWordToggleLM = importlib.import_module(
"py.nodes.trigger_word_toggle"
).TriggerWordToggleLM
@@ -56,6 +68,16 @@ except (
"py.nodes.lora_randomizer"
).LoraRandomizerLM
LoraCyclerLM = importlib.import_module("py.nodes.lora_cycler").LoraCyclerLM
LoraInfoLM = importlib.import_module("py.nodes.lora_info").LoraInfoLM
LoraSyntaxToPath = importlib.import_module(
"py.nodes.lora_syntax_to_path"
).LoraSyntaxToPath
CreateHookLoraLM = importlib.import_module(
"py.nodes.create_hook_lora"
).CreateHookLoraLM
MetadataOverwriteLM = importlib.import_module(
"py.nodes.metadata_overwrite"
).MetadataOverwriteLM
init_metadata_collector = importlib.import_module("py.metadata_collector").init
NODE_CLASS_MAPPINGS = {
@@ -65,6 +87,8 @@ NODE_CLASS_MAPPINGS = {
LoraTextLoaderLM.NAME: LoraTextLoaderLM,
CheckpointLoaderLM.NAME: CheckpointLoaderLM,
UNETLoaderLM.NAME: UNETLoaderLM,
RandomCheckpointLoaderLM.NAME: RandomCheckpointLoaderLM,
RandomUNETLoaderLM.NAME: RandomUNETLoaderLM,
TriggerWordToggleLM.NAME: TriggerWordToggleLM,
LoraStackerLM.NAME: LoraStackerLM,
LoraStackCombinerLM.NAME: LoraStackCombinerLM,
@@ -75,6 +99,10 @@ NODE_CLASS_MAPPINGS = {
LoraPoolLM.NAME: LoraPoolLM,
LoraRandomizerLM.NAME: LoraRandomizerLM,
LoraCyclerLM.NAME: LoraCyclerLM,
LoraInfoLM.NAME: LoraInfoLM,
LoraSyntaxToPath.NAME: LoraSyntaxToPath,
CreateHookLoraLM.NAME: CreateHookLoraLM,
MetadataOverwriteLM.NAME: MetadataOverwriteLM,
}
WEB_DIRECTORY = "./web/comfyui"
+579 -496
View File
File diff suppressed because it is too large Load Diff
+208
View File
@@ -0,0 +1,208 @@
# Agent Skills System
The LoRA Manager agent skills system enables LLM-powered metadata enrichment and other AI-driven tasks. Users configure their own LLM provider (BYOK), and skills are executed through right-click context menu actions.
## Architecture
```
┌──────────────────────────────────────────────┐
│ LoRA Manager Backend │
│ │
│ ┌──────────────┐ ┌────────────────┐ │
│ │ LLMService │───▶│ LLM Provider │ │
│ │ (BYOK config, │◀───│ (OpenAI/Ollama │ │
│ │ API calls) │ │ /custom) │ │
│ └───────┬───────┘ └────────────────┘ │
│ │ │
│ ┌───────▼───────────────────────┐ │
│ │ AgentService │ │
│ │ (orchestration: validate │ │
│ │ → LLM call → post-process │ │
│ │ → WebSocket broadcast) │ │
│ └───────┬───────────────────────┘ │
│ │ │
│ ┌───────▼───────────────────────┐ │
│ │ SkillRegistry │ │
│ │ ┌─────────────────────────┐ │ │
│ │ │ enrich_hf_metadata: │ │ │
│ │ │ - skill.yaml │ │ │
│ │ │ - prompt.md │ │ │
│ │ │ - handler.py │ │ │
│ │ └─────────────────────────┘ │ │
│ └───────────────────────────────┘ │
└──────────────────────────────────────────────┘
```
### Key Design Principle
**Skills define *what* to do (prompt + post-processing). The AgentService handles *how* (LLM calls, validation, progress).**
Skills never call the LLM directly. This keeps BYOK configuration centralized and provider-agnostic.
## BYOK Configuration
Users configure their LLM provider in **Settings → AI Provider**:
| Setting | Description | Example |
|---|---|---|
| `llm_provider` | Provider type | `openai`, `ollama`, or `custom` |
| `llm_api_key` | API key (not needed for local Ollama) | `sk-...` |
| `llm_api_base` | Custom API base URL (empty = provider default) | `https://api.openai.com/v1` |
| `llm_model` | Model name | `gpt-4o-mini` |
Environment variable overrides: `LLM_API_KEY`, `LLM_MODEL`, `LLM_API_BASE`, `LLM_PROVIDER`.
### Supported Providers
- **OpenAI**: Uses `https://api.openai.com/v1` by default
- **Ollama** (local): Uses `http://localhost:11434/v1`, no API key required
- **Custom**: Any OpenAI-compatible endpoint (vLLM, LM Studio, etc.) — set `llm_api_base` explicitly
## Available Skills
### enrich_hf_metadata
Enriches HuggingFace-downloaded models with metadata extracted by an LLM from the HF model card.
**Entry point**: Right-click context menu → "Enrich Metadata (Agent)"
**What it does**:
1. Reads the model's `.metadata.json` to get the `hf_url`
2. Fetches the README.md from the HuggingFace repository
3. Sends the README + local metadata to the LLM for structured extraction
4. Writes extracted fields to `.metadata.json`:
- `base_model` — only if current value is empty
- `trainedWords` — trigger words (LoRA only, if none exist)
- `modelDescription` — concise summary (if none exists)
- `tags` — merged with existing tags, deduplicated
- `metadata_source` — audit trail: `agent:enrich_hf_metadata`
- `llm_enriched_at` — ISO timestamp
5. Downloads and optimizes preview image (if LLM found one in the README)
6. Updates the scanner cache
7. Broadcasts WebSocket progress events
**Model types**: LoRA, Checkpoint, Embedding
## Adding a New Skill
### 1. Create the skill directory
```
py/services/agent/skills/<skill_name>/
├── skill.yaml # Skill metadata and schemas
├── prompt.md # LLM prompt template
└── handler.py # Pre-processing and post-processing
```
### 2. Write skill.yaml
```yaml
name: my_skill
title: "My Skill"
description: "What this skill does"
llm_required: true
model_type_filter: ["lora"] # or null for all types
input_schema:
type: object
properties:
model_paths:
type: array
items:
type: string
required:
- model_paths
output_schema:
type: object
properties:
# ... JSON schema for LLM output
permissions:
write_metadata: true
write_previews: false
network_domains:
- "example.com"
```
### 3. Write prompt.md
Use `{{variable}}` placeholders that will be replaced with data from the `prepare` function:
```markdown
You are an expert assistant...
Model URL: {{hf_url}}
README content:
{{readme_content}}
Current metadata:
{{current_metadata}}
```
### 4. Write handler.py
```python
async def prepare(model_path: str, input_data: dict) -> dict:
"""Gather context for the LLM prompt. Returns variables for template rendering."""
return {
"model_path": model_path,
# ... other variables used in prompt.md
}
async def post_process(context) -> dict:
"""Apply the LLM-extracted data to the model."""
llm_response = context.llm_response
# ... write metadata, download previews, update cache
return {
"success": True,
"updated_fields": ["base_model", "tags"],
"errors": [],
}
```
**Important**: Use absolute imports (`from py.utils.metadata_manager import MetadataManager`) because skills are loaded via `importlib.util.spec_from_file_location`, which doesn't support relative imports.
### 5. Test
The skill is automatically discovered by `SkillRegistry` on startup. Test with:
```python
pytest tests/services/test_agent_service.py
```
## API Endpoints
| Method | Path | Description |
|---|---|---|
| GET | `/api/lm/agent/skills` | List available skills |
| POST | `/api/lm/agent/execute/{skill_name}` | Execute a skill (body: `{"model_paths": [...]}`) |
| POST | `/api/lm/agent/cancel` | Cancel running skill (stub) |
## WebSocket Events
| Type | When | Key fields |
|---|---|---|
| `agent_progress` | Skill started/processing | `skill`, `status`, `total`, `processed`, `success`, `current_path` |
| `agent_progress` | Skill completed | `skill`, `status`, `updated_models`, `errors`, `summary` |
| `agent_progress` | Skill error | `skill`, `status`, `error` |
## Security Model
Skills declare permissions in `skill.yaml`:
- `write_metadata` — can write `.metadata.json` files
- `write_previews` — can download/replace preview images
- `network_domains` — allowed domains for HTTP requests
These are declarative constraints checked by `AgentService`. They are defense-in-depth, not a sandbox — the Python process can technically do anything, but the contract is clear and auditable.
## File Locations
| Component | Path |
|---|---|
| LLMService | `py/services/llm_service.py` |
| AgentService | `py/services/agent/agent_service.py` |
| SkillRegistry | `py/services/agent/skill_registry.py` |
| SkillDefinition | `py/services/agent/skill_definition.py` |
| Skills directory | `py/services/agent/skills/` |
| Route handlers | `py/routes/handlers/agent_handlers.py` |
| Frontend manager | `static/js/managers/AgentManager.js` |
| Settings UI | `templates/components/modals/settings_modal.html` |
| Context menu | `templates/components/context_menu.html` |
+65
View File
@@ -0,0 +1,65 @@
# ComfyUI Dual-Mode Widget Rendering
ComfyUI custom node widgets render in one of two modes. Patterns that work in one often fail silently in the other. Test both.
## Mode Detection
```js
typeof LiteGraph !== 'undefined' && LiteGraph.vueNodesMode
```
In Vue SFCs, `window.LiteGraph` is unavailable — pass as a prop from `main.ts`.
## Canvas Mode Layout
Uses `computeLayoutSize()` + `distributeSpace()` to allocate widget height within the node. Widgets with `computeLayoutSize` participate in space distribution; those with `computeSize` have fixed height.
- `getMinHeight()` in `addDOMWidget` options → minimum widget height
- `widget.computeLayoutSize()``{ minHeight, minWidth, maxHeight? }`
- Avoid `getMaxHeight()` unless the widget genuinely needs a fixed cap (prevents user resize)
## Vue Mode Layout
Uses CSS Grid (`grid-template-rows`) + `ResizeObserver`. The ResizeObserver watches the widget's DOM and feeds back into grid row sizing. This creates a feedback loop: content grows → row resizes → more space for content → content reflows/grows → row resizes again.
### Height Containment
The fix: `contain: layout size` on the widget root. This tells the browser the element's intrinsic size is CSS-determined, not driven by descendant content. The ResizeObserver sees a stable size and the loop is broken.
```css
.widget-root.lm-vue-node {
height: 100%;
min-height: var(--comfy-widget-min-height, 200px);
contain: layout size;
}
```
Existing examples: `.lm-loras-container.lm-vue-node` and `.comfy-tags-container.lm-vue-node` in `web/comfyui/lm_styles.css`.
**Do NOT** fix height issues with `maxHeight`, `getMaxHeight()`, or inline `max-height` — these prevent the user from resizing the node.
## Scroll Wheel Isolation
Both modes need to distinguish "user wants to scroll widget content" from "user wants to zoom canvas".
**Canvas mode:** Add `@wheel` on widget root. Check `event.target.closest(selector)` for scrollable sub-areas. If scrollable → `event.stopPropagation()`. Otherwise → `app.canvas.processMouseWheel(event)`.
**Vue mode:** Add CSS class `lm-wheel-scrollable` to scrollable elements. The global capture-phase hook in `web/comfyui/utils.js` (`enableListWheelScroll`) detects wheel events on marked elements and manually scrolls them via `element.scrollTop`, consuming the event before canvas zoom sees it.
## DOM Structure
`main.ts` creates an outer `<div>` container, then `vueApp.mount(container)`. The Vue app renders its own root element inside.
- `container.id` / `container.style.*` → outer element
- Vue scoped `<style>``[data-v-hash]` applies only to Vue root
Classes needed by scoped Vue CSS must go on the Vue root element. Pass data as props and bind with `:class` rather than manipulating the DOM from `main.ts`.
## Serialization
For stateful widgets that need workflow persistence:
- `serialize: true` in `addDOMWidget` options
- `serializeValue()` → state snapshot (called on workflow save)
- `onSetValue(v)` → restore state (called on workflow load)
- Always handle missing keys in restored value for backward compatibility with old workflows
+368
View File
@@ -0,0 +1,368 @@
# i18n Translation Guidelines
This document is the canonical set of conventions for translating LoRA Manager UI strings.
It applies to **human translators and AI agents** alike. Read it before editing anything in
`locales/`.
Source of truth: `locales/en.json` (10 locales, 1810 leaf keys; all locales share the exact
same key structure).
Locales: `en`, `zh-CN`, `zh-TW`, `ja`, `ko`, `fr`, `de`, `es`, `ru`, `he` (RTL).
> **Status (2026-08 sweep):** a full audit was executed and the terminology, placeholder,
> stale-text, and untranslated-block fixes described in §2–§6 were applied across all locales
> (commits `3c3ac49f` … `fd1227d3`). The tables below are now the **normative target state**,
> not a to-do list — future edits should preserve these renderings and only add what is new.
---
## 1. Hard rules (do not violate)
### R1 — Key structure is sacred
- Only `locales/en.json` may add/remove/rename keys. All other locales must keep the exact
same nested key set. `tests/i18n/test_i18n.py` enforces this.
- When a new UI string is added to `en.json`, run
`python scripts/sync_translation_keys.py` (adds the missing keys to all locales with
placeholder copies), then translate the newly added keys in every locale.
- Never reorder, re-indent, or reformat a locale file "for tidiness". The sync script
preserves formatting; manual reformatting creates noisy diffs.
### R2 — Placeholders and HTML must be preserved verbatim
- `{name}`-style placeholders must appear in the translation exactly as in `en.json`.
Do not invent placeholders the source string does not have — the caller may not pass them
(example bug: `zh-CN recipes.controls.import.downloadLocationPreview` added `{path}`; the
template renders this key with no parameters, so the literal text `{path}` shows in the UI).
- `{{...}}` in a locale value is an escaped literal brace — keep it identical.
- Keep embedded HTML tags (e.g. `<strong>...</strong>`, `<code>...</code>`) intact.
You may move the tag around the sentence if the target language needs different word order.
### R3 — Never translate or transliterate these
- Model types: **LoRA, Checkpoint, Embedding, Diffusion Model**
- Products/brands: **LoRA Manager, ComfyUI, CivitAI, CivArchive, HuggingFace, Ko-fi**
- Ecosystem names: **LyCORIS, DoRA**, trigger-adjacent jargon **Prompt, Workflow**
(these are used as-is in the target-language SD community; see §2 per-language policy)
- Theme names: **Nord, Midnight, Monokai, Dracula, Solarized**
### R4 — The "Recipe" convention (the most important domain term)
Product intent: a *Recipe* records a **LoRA combination + generation parameters**
(prompt, seed, sampler, …) that reproduces an image style. The metaphor is a **cooking
recipe** — "follow it and you get a similar dish". It is **not** a menu, not a dish list,
not a prescription.
Decision per language — translate only into a word whose everyday primary meaning is a
cooking recipe; where that word would mislead users, **keep the English "Recipe(s)"**:
| Locale | Use | Never use |
|---|---|---|
| fr | **Recipe / Recipes** (keep English) | recette(s) — cooking reading is secondary and it was explicitly judged misleading |
| zh-CN / zh-TW | 配方 | 食谱 (reads as "food cookbook") |
| ja | レシピ | — (leftover English "Recipe" in `initialization.recipes.title` / `toast.recipes.recipeSaved` → translate) |
| ko | 레시피 | — |
| de | Rezept / Rezepte | — (cooking meaning dominant; prescription reading acceptable) |
| es | receta / recetas | — (cooking meaning dominant) |
| ru | рецепт / рецепты | — (leftover English "Recipe" in `initialization.recipes.title` / `toast.recipes.recipeSaved` → translate) |
| he | מתכון / מתכונים | — (cooking meaning dominant) |
Whatever the choice, **one concept = one noun within a locale**. Currently violated in:
- `fr` — "Recipe" (~97 keys, incl. nav) mixed with "recette" (~58 keys)
- `zh-CN` / `zh-TW` — 配方 (126/122 keys) mixed with 食谱 / 食譜 (14/17 keys, all in the
*rematch* flow: `globalContextMenu.rematchRecipes.*`, `toast.recipes.rematch*`)
- `de` — "Rezept" (136 keys) mixed with leftover English "Recipe" (5 keys)
- `ja` / `ru` — leftover English "Recipe" in `initialization.recipes.title` ("Recipe Manager
zu initialisieren" / «Инициализация Recipe Manager») and `toast.recipes.recipeSaved`
### R5 — One term, one rendering (within each locale)
Same source word must not be translated several ways in one file. Known offender areas
(see §5 for the full fix list): recipe, Checkpoint, Embedding, prompt, base model, preset,
workflow, hash, metadata, tags, bulk. Every locale currently mixes variants of at least one
of these — pick the preferred form in the §2 tables and normalize.
### R6 — Register consistency
- `zh-CN` / `zh-TW`: pick 你 or 您 once. Do not mix (zh-CN has 44×你 + 5×您; zh-TW has
27×您 + 18×你).
- `de`: pick "du" or "Sie" once (currently 143×Sie + ~7×du).
- `es`: pick "tú" or "usted" once.
### R7 — Punctuation per script
- Full-width punctuation `:()` is correct **only in CJK locales** (zh-CN, zh-TW, ja, ko).
- Latin/Cyrillic/Hebrew locales must use ASCII `: ()` — full-width colons leaked in there
are machine-translation artifacts. Known: `fr toast.recipes.createError/createFailed`,
`es toast.recipes.createError/createFailed` (e.g. "…de la receta" should be "…de la receta:").
- `fr` apostrophes must be U+2019 `'` / ASCII `'`, never a straight double quote:
`fr header.filter.allowSellingGeneratedContentTooltip` currently reads
`vendre d"images` → fix to `d'images`. Do not mix `'` and `'` in one file (fr has 299 vs 15).
- Ellipsis: use ASCII `...` (project style). Don't introduce `…`.
- Keep the sentence-ending period/omission consistent with the source string where the
language allows it.
- `he` is RTL: mix of Hebrew and Latin scripts is normal; keep Latin term ordering natural.
### R8 — No untranslated English leftovers
Full sentences left byte-identical to `en.json` are bugs (brand names and URL placeholders
are the exception). Every locale has them; see §6 for the per-locale checklist.
`[TODO: Translate]` placeholders are the sanctioned intermediate state during feature
development (see §7) — do not "fix" them unless the feature owner asked for translations.
### R9 — Mirror the source even when the source is wrong
If `en.json` itself contains an inconsistency (e.g. the `Civitai` vs `CivitAI` casing split,
or the `CivitArchive` typo in `modals.relinkCivitai.helpText.format4`), translate/transcribe
it as-is in your locale and instead **fix the source** in `en.json` (then propagate by
re-syncing and re-translating affected keys). Do not silently diverge in one locale only.
---
## 2. Per-language term maps
Preferred rendering per term. "Fix" means the locale currently contains the wrong variant
and must be normalized. `en` = keep the English word as-is.
### fr
| Term | Use | Fix |
|---|---|---|
| recipe | Recipe(s) | Replace all "recette(s)" (58 keys, e.g. `recipes.actions.deleteRecipeWithShortcut`, `toast.recipes.rematchComplete`) with "Recipe(s)" |
| Checkpoint | Checkpoint | `statistics.modelTypes.checkpoint` = "Point de contrôle" → "Checkpoint" |
| trigger words | mot(s)-clé(s) | unify: `modals.model.triggerWords.editWord` uses "mot déclencheur" — pick one |
| prompt / negative prompt | Prompt / prompt négatif | — |
| base model | modèle(s) de base | — |
| preset | préréglage | unify: `modals.model.usageTips.addPresetParameter` "prédéfini", `toast.presets.restored` "par défaut" |
| hash | hash | `conflictConfirm.message` "hachage" → "hash" |
| tags | tags | `settings.sections.priorityTags` "Étiquettes" → "Tags" |
| metadata | métadonnées | `loras.controls.refresh.fullTooltip` keeps English "metadata" |
| duplicates | doublon(s) | unify with "dupliqué(e)s" |
| bulk | groupé(e) | unify with "par lot / mode lot" variants |
### de
| Term | Use | Fix |
|---|---|---|
| recipe | Rezept/Rezepte | 5 leftover English "Recipe" keys → Rezept (e.g. `globalContextMenu.repairRecipes.label`, `toast.recipes.recipeSaved`) |
| base model | pick Basis-Modell or Basismodell | currently 27× hyphenated vs 15× closed |
| metadata | Metadaten | 4 keys use "Modelldaten" (`onboarding.steps.fetch.title/content`) → Metadaten |
| bulk | pick Massen- or Sammelmodus | `loras.controls.bulk.action` = "Massen" reads as "crowds" — use "Massenbearbeitung"/"Mehrfachauswahl" |
| register | Sie (formal) | 7 keys use "du/dein" (`settings.backup.managementHelp`, `modals.checkUpdates.message/tip`, `doctor.footer`, …) |
### es
| Term | Use | Fix |
|---|---|---|
| recipe | receta(s) | — |
| Checkpoint | Checkpoint | 5 statistics keys "Punto(s) de control" → "Checkpoints" (`statistics.metrics.checkpoints`, `statistics.insights.unusedCheckpoints.*`, `statistics.modelTypes.checkpoint`) |
| trigger words | palabra(s) de activación | 2 keys already use it; ~15 keys "palabra(s) clave" (reads as search keyword) → unify |
| base model | modelo base | — |
| preset | preajuste | 3 keys keep English "preset", 1 "preestablecido" → preajuste |
| workflow | pick flujo de trabajo or workflow | currently 21× "flujo de trabajo" vs 10× "workflow" |
| bulk | masivo / por lotes | unify; "Batch Import" → traducción |
| tags | etiquetas | — |
### ru
| Term | Use | Fix |
|---|---|---|
| recipe | рецепт(ы) | English leftovers: `initialization.recipes.title`, `recipes.batchImport.*`, `toast.recipes.recipeSaved` → translate |
| Checkpoint | Checkpoint (recommended) | 3 variants today: "Checkpoint" (17 keys), «Чекпойнт», «Контрольная точка» (statistics, 6 keys) — statistics MUST drop «Контрольная точка» |
| Embedding | Embedding | «Эмбеддинг» variant exists in `settings.priorityTags.modelTypes.embedding` — unify |
| prompt | промпт | 8 keys use «запрос» (reads as "database/HTTP request") → «промпт» |
| base model | базовая модель | — |
| preset | пресет | `header.theme.presets` "Предустановки" → пресеты |
| workflow | Workflow (recommended) | «рабочий процесс» used in 4 keys — unify |
| hash | pick хеш or хэш | both spellings co-occur |
| tag(s) | тег(и) | — |
| typos | — | `settings.misc.loraSyntaxFormatHelp`: «безпотерьного» → «беспотерьного» |
### he
| Term | Use | Fix |
|---|---|---|
| recipe | מתכון / מתכונים | — |
| Checkpoint | Checkpoint | 5 statistics keys «נקודת/נקודות ביקורת» (road/security checkpoint) → "Checkpoint(s)" (`statistics.metrics.checkpoints`, `statistics.modelTypes.checkpoint`, `statistics.insights.unusedCheckpoints.*`) |
| Embedding | Embedding | `statistics` keys use הטמעות → Embedding |
| prompt | pick הנחיה or פרומפט | 9 keys הנחיה vs 3 פרומפט — unify (recommend פרומפט, SD-community loanword) |
| preset | קביעה מראש | `header.filter.presetOverwriteConfirm` uses פריסט → unify |
| hash | pick one of האש / גיבוב / hash | 3 variants co-occur — unify (recommend hash or גיבוב) |
| metadata | pick מטא-דאטה or מטא-נתונים | 38 vs 17 keys — unify |
| model | מודל | 13 keys use דגם/דגמים — unify |
| bulk | pick one of 5 variants | 5 different renderings ("כמות גדולה", "המוני", "קבוצתי", "אצווה", …) — unify; `loras.controls.bulk.action` "כמות גדולה" reads as "large quantity" |
### ja
| Term | Use | Fix |
|---|---|---|
| recipe | レシピ | `initialization.recipes.title` keeps English "Recipe Manager" — translate to レシピマネージャー |
| Checkpoint | Checkpoint or チェックポイント (pick one) | 3 variants: Checkpoint (~14), checkpoint lowercase (4), チェックポイント (4, e.g. `settings.priorityTags.modelTypes.checkpoint`) |
| Embedding | Embedding | 4 keys lowercase "embedding" mid-sentence |
| bulk | 一括 | `modals.checkUpdates.tip` "バルクモード" → 一括モード |
| recipe counter | 件 or 個 | `repairRecipes.success` uses 件, `.cancelled` uses 個 — unify |
### ko
| Term | Use | Fix |
|---|---|---|
| recipe | 레시피 | — |
| Checkpoint | Checkpoint (recommended) | 4 keys transliterate 체크포인트 (`settings.priorityTags.modelTypes.checkpoint`, `toast.recipes.missingCheckpointPath/missingCheckpointInfo/downloadCheckpointFailed`) |
| Embedding | Embedding | 3 keys 임베딩 (`settings.priorityTags.modelTypes.embedding`, `uiHelpers.nodeSelector.embedding`) |
| base model | 베이스 모델 | 6 keys «기본 모델» read as "default model" → 베이스 모델 (`settings.downloadSkipBaseModels.*`, `toast.loras.downloadSkippedByBaseModel`) |
| workflow | pick 워크플로 or 워크플로우 | 26 vs 6 keys — unify |
| bulk | 일괄 | `modals.checkUpdates.tip` "벌크 모드" → 일괄 모드 |
| tag logic | — | `header.filter.tagLogicAny` = "모든 태그 일치 (OR)" is **inverted** (should be "하나 이상의 태그 일치") and identical to `tagLogicAll` |
| particle | — | `modelCard.sendToWorkflow.checkpointNotImplemented`: "Checkpoint을" → "Checkpoint를" |
### zh-CN / zh-TW
| Term | zh-CN | zh-TW |
|---|---|---|
| recipe | 配方 (fix 食谱 → 配方, 14 keys in rematch flow) | 配方 (fix 食譜 → 配方, 17 keys in rematch flow) |
| Checkpoint | Checkpoint (fix 检查点 → Checkpoint, 5 keys: `toast.recipes.missingCheckpointPath/missingCheckpointInfo/downloadCheckpointFailed`, `modelCard.actions.checkpointNameCopied`, `modelCard.sendToWorkflow.checkpointNotImplemented`) | Checkpoint (fix 檢查點 → Checkpoint, 4 keys: `modelCard.actions.copyCheckpointName`, `toast.recipes.missing*`×2, `toast.recipes.downloadCheckpointFailed`) |
| base model | 基础模型 (fix 基模型 → 基础模型, 3 keys in `modals.model.versions.filters.*`) | 基礎模型 ✓ consistent |
| prompt | 提示词 ✓ | 提示詞 ✓ |
| preset | 预设 ✓ | 預設 ✓ |
| workflow | 工作流 ✓ | 工作流 ✓ |
| trigger words | 触发词 ✓ | 觸發詞 ✓ |
| hash | 哈希 (哈希值 variant OK) | 雜湊 ✓ |
| register | 你 (fix 5×您 → 你) | 您 (fix 18×你 → 您) |
---
## 3. Cross-cutting confusion hot-spots (must-fix list)
All items below were **resolved** in the 2026-08 sweep — treat them as a regression
watch-list: do not reintroduce these renderings.
1. **Checkpoint rendered as a literal security/road checkpoint** — fr, es, ru, he, zh-CN,
zh-TW all had 46 keys in the `statistics.*` domain reading as "control point"; reverted
to "Checkpoint".
2. **"recipe" variants that break the one-noun rule** — fr "recette" → "Recipe", zh
食谱/食譜 → 配方, de/ja/ru leftover English "Recipe" translated.
3. **ko `header.filter.tagLogicAny`** — was inverted ("모든 태그 일치 (OR)") and identical
to `tagLogicAll`; now "어느 하나의 태그와 일치 (OR)".
4. **ja `modals.model.versions.actions.viewLocalTooltip`** — was the stale "近日対応予定"
("coming soon"); all 9 locales now describe the actual action.
5. **Stale help texts**`settings.downloadSkipBaseModels.help`,
`settings.aiProvider.apiBaseHelp`, `settings.hideEarlyAccessUpdates.help` retranslated
in all locales to the current `en.json` wording.
6. **en.json source bugs** (fixed in source, then mirrored):
- "Civitai" → "CivitAI" brand casing (values only; key names `relinkCivitai` etc. keep
their lowercase form and must not be renamed)
- `modals.relinkCivitai.helpText.format4` "CivitArchive" typo → "CivArchive"
- `zh-CN recipes.controls.import.downloadLocationPreview` invented `{path}` removed
---
## 4. Placeholder contract deviations (current)
`{...}` token sets must match `en.json` per key. All deviations found in the 2026-08 sweep
were fixed, with one *intentional* exception:
**`toast.settings.mappingsUpdated`** — the caller passes a hardcoded English inflection
(`plural: count !== 1 ? 's' : ''`). Languages that cannot build a plural by appending that
`s` (zh-CN/zh-TW, ja, ko, de, ru, he) **drop `{plural}`** and render a count-friendly form
(`({count})` or a measure word); fr and es keep it (`mappage{plural}`, `mapeo{plural}`).
```python
# keep a copy of this rule next to the key if it ever moves:
# fr/es: "... ({count} mappage{plural})"
# de/ru/he: "... ({count})"
# zh-CN: "{count} 条映射)" / zh-TW: "{count} 個對應)" / ja: "{count} マッピング)"
```
Do NOT add `{...}` tokens the source lacks (the caller will not supply them, and the literal
text renders in the UI), and do NOT rename source tokens (`{typePlural}` stays `{typePlural}`).
---
## 5. One term, one rendering — offender matrix
Cross-locale summary of §2 inconsistencies. "✓" = already consistent. All ✗ cells were
resolved in the 2026-08 sweep; the row shows the single rendering now in force per locale.
| Term | fr | de | es | ru | he | ja | ko | zh-CN | zh-TW |
|---|---|---|---|---|---|---|---|---|---|
| recipe | Recipe | Rezept | receta | рецепт | מתכון | レシピ | 레시피 | 配方 | 配方 |
| Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint | Checkpoint |
| Embedding | Embedding | Embedding | Embedding | Embedding | Embedding | Embedding | Embedding | Embedding | Embedding |
| prompt | Prompt | Prompt | prompt | промпт | פרומפט | プロンプト | 프롬프트 | 提示词 | 提示詞 |
| base model | modèle de base | Basismodell | modelo base | базовая модель | מודל בסיס | ベースモデル | 베이스 모델 | 基础模型 | 基礎模型 |
| preset | préréglage | Voreinstellung | preajuste | пресет | קביעה מראש | プリセット | 프리셋 | 预设 | 預設 |
| workflow | Workflow | Workflow | workflow | Workflow | workflow | ワークフロー | 워크플로 | 工作流 | 工作流 |
| hash | hash | Hash | hash | хеш | hash | ハッシュ | 해시 | 哈希 | 雜湊 |
| metadata | métadonnées | Metadaten | metadatos | метаданные | מטא-נתונים | メタデータ | 메타데이터 | 元数据 | 中繼資料 |
| tags | Tags | Tags | etiquetas | теги | תגיות | タグ | 태그 | 标签 | 標籤 |
| duplicates | en double | Duplikate | duplicados | дубликаты | כפילויות | 重複 | 중복 | 重复项 | 重複項 |
| bulk | groupé | Massen- | por lotes | пакетный | בכמות גדולה | 一括 | 일괄 | 批量 | 批量 |
Watch: ja/ko keep the model-type names **Checkpoint/Embedding** and `Diffusion Model` in
Latin (consistent with their model-type sections) — do not transliterate them as
チェックポイント/체크포인트.
---
## 6. Untranslated English leftovers (status)
Values byte-identical to `en.json` that are actual UI sentences are bugs (brand names and
URL placeholders are the exception). As of the 2026-08 sweep, **all previously untranslated
blocks are translated** in every locale: `recipes.batchImport.*` + `toast.recipes.batchImport*`
(fr/de/es/ru/he/ja/ko), `banners.communitySupport.*`, `modals.model.license.*`,
`globalContextMenu.fetchMissingLicenses.*`, the `doctor.*` issue/action/label subset,
`toast.settings.libraryLoadFailed` / `libraryActivateFailed`, `toast.api.moveFailed`,
`settings.extraFolderPaths.restartRequired`, `toast.recipes.recipeSaved`,
`sidebar.dragDrop.moveUnsupported`, `checkpoints.modelTypes.diffusion_model`
(ja/ko keep the English loanword), `initialization.recipes.title`.
The only values that remain intentionally identical to `en.json` are non-translatable:
URL/path placeholders (`https://…`, `C:/…`), numeric presets (`5 (1080p), 6 (2K), 8 (4K)`),
example token lists (`character, concept, style(toon|toon_style)`), service/provider names
(`CivitAI → CivArchive → Archive DB`), and the external playlist title
(`help.updateVlogs.playlistTitle`, de: translated to "LoRA Manager-Update-Playlist").
Rule for `uiHelpers.workflow.noPromptTargets`: the second line (`Mark as → Send Prompt
Target`) quotes literal ComfyUI context-menu items — keep those menu labels in English in
every locale because that is what the user actually sees in ComfyUI.
License labels (`modals.model.license.*`): the restriction labels are now translated in all
locales (the sibling `creditRequired` has always been translated).
---
## 7. Workflow for agents and translators
### Adding a new UI string
1. Add the key to `locales/en.json` only.
2. Run `python scripts/sync_translation_keys.py` — it inserts the key into the other 9
locales (as a `[TODO: Translate]` placeholder) preserving formatting.
3. **During feature development, stop here.** While the UI copy is still in flux, leave the
`[TODO: Translate]` placeholders as-is — translating churning strings into 9 locales is
wasted work. Placeholders are a normal intermediate state, not a bug.
4. Once the wording is final and the feature owner explicitly asks for translations,
translate **all** pending `[TODO: Translate]` keys in every locale (not just the latest
feature's), applying §1–§3 (placeholders verbatim, Recipe rule, term maps, register).
Find pending keys with: `grep -c "TODO: Translate" locales/*.json`
5. If the new string contains new terminology, extend §2 tables.
### Fixing a translation bug
1. Locate the key (dotted path) in the relevant locale file.
2. Check the corresponding `en.json` value and the actual caller (grep `static/js` or
`web/comfyui` for the key) to learn which placeholders are passed.
3. Fix trivially; for normalization sweeps (e.g. "recette" → "Recipe"), do it file-wide for
the offending keys only — do not touch unrelated lines.
4. If the bug is in `en.json` itself (R9), fix the source first, then re-sync and update all
locales.
### Verification
```bash
pytest tests/i18n/test_i18n.py # key parity + JSON validity + JS key references
python scripts/sync_translation_keys.py --dry-run # shows which keys would change; add --verbose for per-key detail
npm test # frontend tests incl. i18n helpers
```
`pytest tests/i18n` only checks structure. Quality conventions in this document are not
machine-enforced — a human/agent review pass is required.
### Anti-patterns checklist
- [ ] Placeholders `{x}` / `{{x}}` differ from `en.json`
- [ ] Same source term translated 2+ ways in the same file (see §5)
- [ ] "Checkpoint" became a literal checkpoint; "recipe" became menu/prescription/food-cookbook
- [ ] Brand names translated or transliterated (LoRA, CivitAI, ComfyUI, …)
- [ ] Latin locale using full-width `:()`; fr using `"` as apostrophe
- [ ] Mixed 你/您, du/Sie, tú/usted
- [ ] Full English sentences left behind (see §6)
- [ ] Register/typos/mojibake; source string is stale vs `en.json` (compare semantics, not
just words)
+4
View File
@@ -39,6 +39,7 @@ These fields are present in all model metadata files.
| `metadata_source` | string\|null | ❌ No | ✅ Yes | Last provider that supplied metadata (see below) |
| `last_checked_at` | float | ❌ No (default: `0`) | ✅ Yes | Unix timestamp of last metadata check |
| `hash_status` | string | ❌ No (default: `"completed"`) | ✅ Yes | Hash calculation status: `"pending"`, `"calculating"`, `"completed"`, `"failed"` |
| `autov3` | string\|null | ❌ No | ✅ Yes | CivitAI AutoV3 hash (first 12 chars, lowercase hex) sourced from the safetensors embedded metadata (`sshs_model_hash` / `modelspec.hash_sha256`). **Absent** = not yet checked (may be backfilled later); **`null`** = checked but unavailable (header has no recognized hash); **12-char hex string** = value |
---
@@ -287,6 +288,7 @@ These fields are automatically synchronized with the filesystem:
- `preview_url` — Updated if preview file is moved/removed
- `sha256` — Updated during hash calculation (when `hash_status="pending"`)
- `hash_status` — Updated during hash calculation
- `autov3` — Set when metadata is first created (from safetensors header); may be backfilled later for entries where it is absent
- `last_checked_at` — Timestamp of scan
- `metadata_source` — Set based on metadata provider
@@ -345,6 +347,7 @@ These fields can be edited by users at any time through the Lora Manager UI or b
| `metadata_source` | `null` |
| `last_checked_at` | `0` |
| `hash_status` | `"completed"` |
| `autov3` | absent (not checked) or `null` (checked, no value) |
| `usage_tips` | `"{}"` (LoRA only) |
| `model_type` | `"checkpoint"` or `"embedding"` (not present in LoRA models) |
@@ -354,6 +357,7 @@ These fields can be edited by users at any time through the Lora Manager UI or b
| Version | Date | Changes |
|---------|------|---------|
| 1.1 | 2026-08 | Added `autov3` field (CivitAI AutoV3 hash with three-state semantics) |
| 1.0 | 2026-03 | Initial schema documentation |
---
@@ -0,0 +1,206 @@
# Plan: Multi-File Downloads Within a Single CivitAI Model Version
**Issue:** [#1058 — Cannot download multiple file variants from the same model version](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1058)
**Status:** v2 — revised after adversarial review (backend correctness + frontend/tests)
**Scope:** CivitAI/CivArchive downloads of `lora`, `checkpoint`, `embedding` model types. HuggingFace downloads are out of scope (already per-file).
> v2 changelog: incorporated 18 review findings. Key changes vs v1:
> shared file resolver + `resolved_version_id` for the gate (R1); `file_params` normalization at API boundary (R2); D2 hash-matching rule fixed for empty-hash cases (R6/R7); D3 extended to re-point `version_index` on removal (R4); D4 replaced with a child table (R3); `delete_model_version` interaction documented (R5); `ModelVersionsTab` surface added to phase 2 (F6); phase-2 multi-file loop requires a reload-deferred download variant (F7); queue-retry `file_params=NULL` known issue recorded (R9); test-fixture gaps and revised estimates (F10).
---
## 1. Problem Statement
A CivitAI model version can contain multiple downloadable weight files (e.g. fp16/fp32, safetensors/ckpt, different sizes). LoRA Manager already has a working file-selection pipeline (frontend file dialog → `fileParams` → backend file matching), but downloaded state is tracked at the **model-version** level. After any single file of a version is downloaded:
1. The version is marked **In Library** and the file-selection entry point disappears.
2. The backend rejects further download attempts for that version.
There is no way to download the remaining files of the same version through LoRA Manager.
## 2. Current State (verified against code; all references confirmed by review)
### 2.1 Download gating — backend (`py/services/download_manager.py`)
`_execute_original_download` enforces two version-level gates:
- **Library gate, early** (lines 11571184, before metadata fetch, fires when `model_version_id` given) and **late** (lines 13501376, fires only when `model_version_id is None`): `scanner.check_model_version_exists(version_id)` across lora/checkpoint/embedding scanners → hard error `"Model version already exists in ... library"`.
- **History gate** (lines 12381279): when `skip_previously_downloaded_model_versions` setting is on, `_has_been_downloaded(model_type, version_id)` → silent skip. History DB primary key is `(model_type, version_id)` (`py/services/downloaded_version_history_service.py:61`).
File selection works: `file_params {id, type, format, size, fp}` is matched against `version_info.files` (lines 14981569), **but only under `if file_params and model_version_id:` (line 1499)** — with `model_id`-only requests the selection silently falls back to the primary file (15711619). `file_params` currently carries no file `name` or hash.
### 2.2 Downloaded-state surfacing — backend (`py/routes/handlers/model_handlers.py`)
`get_civitai_versions` (lines 21482188) sets per-version `existsLocally` via `cache.version_index.get(version_id)` (plus a single `localPath` from that entry) and `hasBeenDownloaded` via the history service. No per-file granularity.
### 2.3 Frontend blockers (`static/js/managers/DownloadManager.js`)
Three independent gates prevent re-entering the file dialog:
1. **Line 598:** file-select badge rendered only when `modelFiles.length > 1 && !existsLocally`.
2. **Lines 666681 (`updateNextButtonState`):** Next button disabled with "Already in Library" when `currentVersion.existsLocally`.
3. **Lines 784787 (`proceedToLocation`):** toast + abort when `currentVersion.existsLocally`.
The badge path (`confirmFileSelection` lines 737759 → `proceedToLocationContent``startDownload` single mode → `executeDownloadWithProgress` → POST `file_params`, `static/js/api/baseModelApi.js:12361250`) has **zero** `existsLocally` guards (all 12 occurrences enumerated; none on this path; `import/DownloadManager.js` has none either). The `.exists-locally` CSS class is purely visual (`download-modal.css:496499`). **Making the badge visible again is sufficient to unlock the flow** for phase 1.
Post-download refresh is clean: the modal closes and `resetAndReload(true)` performs a full library refetch (`DownloadManager.js:1063`); dialog reopen resets state and refetches versions with no client-side cache. No same-session staleness.
### 2.4 Local identity of the downloaded file
`LoraMetadata/CheckpointMetadata/EmbeddingMetadata.from_civitai_info(version_info, file_info, ...)` (`py/utils/models.py:245369`) persists:
- `sha256` = `file_info.hashes.SHA256` (lowercased, defaults to `""`) — a stable per-file identity;
- `civitai` = the full `version_info` payload (including the `files` list).
Metadata refresh (`metadata_sync_service.py:104105`) replaces the `civitai` blob wholesale but never overwrites top-level `sha256`; `verify_duplicate_hashes` (481526) corrects it to the on-disk hash. Top-level-sha256 matching is refresh-robust.
**Caveats (review R6/R7):**
- SHA256 is not guaranteed: CivArchive's transform only sets `hashes` when source data carries it (`civarchive_client.py:185189`); `from_civitai_info` defaults to `""`.
- Name fallback is unreliable exactly when it matters: local `file_name` is extension-less (`models.py:264`) and `generate_unique_filename` rewrites it with a hash suffix on conflict (`download_manager.py:11251136`); checkpoints with `hash_status='pending'` keep empty sha256 until on-demand hashing (`model_scanner.py:12321240`).
### 2.5 Version index collision (pre-existing hazard)
`ModelCache.version_index` is single-valued (`model_cache.py:133`: `version_index[version_id] = item`). Two files of the same version in the library → second entry overwrites the first; `remove_from_version_index` (lines 151181) drops the whole version key when the indexed entry is removed, even if a sibling file remains. ~10 read sites depend on this index (48 grep touch points total; readers include `recipe_scanner.py:26822726`, `recipe_format.py:3740`, `misc_handlers.py:24402444`, `model_handlers.py`, `model_scanner.check_model_version_exists:2444`).
Review correction (F3): bulk paths `remove_models` (`model_scanner.py:2376`) and `update_single_model_cache` (`:1689`) call `rebuild_version_index()` right after, so a sibling re-enters the index in those flows — the hazard is narrower than v1 stated, but direct `remove_from_version_index` callers (e.g. `model_scanner.py:1018`) still drop the key, and the user-visible artifact in phase 1 is real: `localPath` in the dialog flips to whichever file was indexed last.
### 2.6 Entry points that send / don't send `file_params` (fully enumerated by review)
**Send `file_params` (user-initiated dialog flows only):** `DownloadManager.js:16111639` (single mode). API surface accepting arbitrary JSON `file_params`: GET `/api/lm/download-model-get` (`model_handlers.py:16341686`), POST `/api/lm/downloads/queue/add` (`model_handlers.py:17991832`).
**Never send `file_params` (keep version-level semantics):** batch download (`DownloadManager.js:17561766`; batch also filters out in-library versions at `:1648`), `downloadVersionWithDefaults` (`:18101830`), recipe import (`import/DownloadManager.js:269276`), bulk missing-LoRA (`BulkMissingLoraDownloadManager.js:292299`), `RecipeModal.js:17281736`, `ModelVersionsTab.js:1427`. `web/comfyui/` and `vue-widgets/src` contain **no** download triggers at all (grep-verified). `py/services/use_cases/` has only `download_model_use_case.py` (pass-through).
### 2.7 Paths that do NOT need changes (verified)
- **aria2 pause/resume** (`_resume_restored_aria2_download`, line 754+): resumes from persisted `resume_context`; never re-runs existence gates.
- **`download_coordinator.py:90`**: pure pass-through of `file_params`.
- **Update checker / plugin self-update** (`update_routes.py:496501`): only closes the history DB handle.
- **History delete semantics**: `mark_as_deleted` sets `is_deleted_override=1` and `has_been_downloaded` then returns False (`downloaded_version_history_service.py:276`) — LM-initiated deletes already reset the history skip.
### 2.8 Related pre-existing issues (record, not necessarily fix)
- **Queue retry drops file selection** (R9): `download_queue_service.retry_from_history` / `retry_all_failed` re-queue with `file_params=NULL` (`download_queue_service.py:705, 758`) although the queue table has a `file_params` column (`:43`) — a retried non-primary download silently reverts to the primary file. Fix alongside phase 1 (small: persist and reuse the column).
- **`delete_model_version`** (`misc_handlers.py:24102487`): resolves the file via the single-valued `version_index` (24402444), deletes only that one file, and `mark_as_deleted` flags the **entire version** as deleted in history (2479) even when a sibling file remains in the library. See phase 2 item 6.1.5.
## 3. Goals / Non-Goals
**Goals**
- G1: A user can download any not-yet-downloaded file of a version already partially in the library (issue repro steps 68).
- G2: True duplicates stay blocked: downloading the *same* file of the same version twice is rejected.
- G3: Per-file downloaded state visible in the file dialog; multiple files selectable and downloadable in one pass.
- G4: No regression for version-level semantics relied on by batch download, recipe missing-LoRA detection, and `skip_previously_downloaded_model_versions`.
**Non-Goals**
- No change to recipe `inLibrary` semantics ("any file of the version present" remains sufficient).
- No change to the update-checker (version-level comparison).
- No primary-key rebuild of the history database.
- HuggingFace download flow untouched.
## 4. Design Decisions
- **D1 — Explicit file selection bypasses the history gate, version-level gates stay for everyone else.** The history skip exists to dedupe automated flows. A user explicitly picking a file is unambiguous intent; the file-level library gate (G2) still prevents real duplicates. **Guard conditions use normalized truthiness** (see D1a). All confirmed `file_params` senders are user-initiated dialog flows (2.6), and LM-initiated deletes already reset history (2.7), so the bypass only affects "downloaded but not LM-deleted" versions with the setting on — intended.
- **D1a — `file_params` normalization at the boundary (R2).** `download-model-get` and `downloads/queue/add` accept arbitrary JSON; `{}` is `not None` but falsy and would bypass gates while downloading the primary file. Normalize `file_params = file_params or None` in the coordinator/handlers, and treat the bypass as active only when a target file id is resolvable.
- **D2 — File identity matching rule (R6/R7):** hash-compare **only when both sides are non-empty** (lowercase SHA256 equality); name-compare when either side is empty. Never let `"" == ""` match. Name fallback caveats from 2.4 apply (renamed files, pending checkpoint hashes) — acceptable residual risk, worst case is a blocked re-download the user can retry after hashing completes.
- **D3 — Cache indexes: additive multi-index + removal re-pointing (R4).** Add `version_files_index: Dict[int, List[dict]]` maintained alongside `version_index` by the same add/remove/rebuild methods; existing readers of `version_index` untouched. Additionally fix `remove_from_version_index`: when the popped entry has a surviving sibling (per the multi-index), re-point `version_index[version_id]` to the sibling instead of dropping the key; same for the `model_id_index` descriptor. This closes the 2.5 hazard for existing readers (`check_model_version_exists`, `existsLocally`, recipe matching) without restructuring anything.
- **D4 — Per-file history via a child table (R3).** v1's additive-column approach is structurally impossible on a `(model_type, version_id)` PK (`ON CONFLICT DO UPDATE` would keep only the last file). Instead add `downloaded_version_files(model_type, version_id, file_id, file_name, downloaded_at, PRIMARY KEY(model_type, version_id, file_id))` — additive, no PK rebuild, honors the Non-Goal. Existing version-level table and queries unchanged. New per-file queries are opt-in. `_initialize_schema` uses `CREATE TABLE IF NOT EXISTS`, so the new table is created for existing DBs without any ALTER.
- **D5 — UI flow reuse, with an extracted inner download function for multi-file (F7).** Phase 1 unlocks the existing badge → file dialog → location → download pipeline. Phase 2 upgrades the dialog to multi-select; iterating `executeDownloadWithProgress` as-is would produce N full library reloads, N toasts, and competing failure-summary modals — so phase 2 extracts a reload-deferred, failure-aggregating inner variant and runs one reload + one summary at the end.
## 5. Implementation — Phase 1 (fix the issue; independently shippable)
### 5.1 Backend — `py/services/download_manager.py`
1. **Normalize `file_params`** at the boundary (D1a): `download_coordinator.schedule_download` and the two API handlers (`model_handlers.py:16491666`, `18101832`) apply `file_params = file_params or None`.
2. **Extract a shared file resolver** (R1): pull the matching logic at 14981569 into `_resolve_target_file(version_info, file_params) -> Optional[dict]`, used by **both** the new gate and the download-selection path. The selection path's condition (line 1499) switches from `model_version_id` to `resolved_version_id` (already computed at 12301236 from `version_info.id`), so gate and download always agree on the target file — including the `model_id`-only case.
3. **New helper** `_find_local_file_entry(version_id, target_file) -> Optional[dict]`: iterate the three scanners' cached `raw_data` (NOT `version_index` — single-valued); candidates = entries whose `civitai.id` normalizes to `version_id`; match per D2.
4. **Gate restructure in `_execute_original_download`**:
- Early scanner gate (11571184): add `file_params is None` guard; with normalized `file_params`, defer (file identity not resolvable before metadata fetch).
- After `version_info` fetch + `resolved_version_id` (~1229): when `file_params` present, resolve target file via the shared resolver; unresolvable → hard error "No matching file" (fail closed, prevents empty-dict bypass). Resolvable → `_find_local_file_entry`; hit → same hard error shape as today with the file name in the message.
- History gate (12381279): add `file_params is None` (D1). Base-model skip (12811324) unchanged — still applies.
- Late gate (13501376): add `file_params is None` guard (F2) — the post-fetch file-level check above already covers this case.
- Nothing between the early gate and the post-fetch point assumes the version is absent (review task 6: only provider selection + metadata fetch; no DB writes; `_persist_aria2_state` runs only when actually downloading at 1659).
5. **Queue retry fix** (2.8, small): persist `file_params` into the queue table on enqueue and reuse it in `retry_from_history` / `retry_all_failed`.
6. Logging: `[download]` lines for file-level allow/block, consistent with existing style.
**Estimated:** ~150220 LOC + resolver extraction.
### 5.2 Frontend — `static/js/managers/DownloadManager.js`
1. Line 598: drop `&& !existsLocally` from the badge condition (badge shows whenever `modelFiles.length > 1`).
2. `fileParams` construction (16111616): add `name: this.selectedFile.name`.
3. Surface the backend "file already in library" hard error as a toast instead of only the batch-summary modal (R10/F12 nit; reuse existing error message field).
4. No changes to `updateNextButtonState` / `proceedToLocation` in phase 1; no template or CSS changes.
**Known phase-1 UX limitations (acknowledged, fixed in phase 2):** with all files downloaded the badge still renders and re-picking a downloaded file fails late (backend error after the location step); `localPath` may point at a sibling file; batch-preview "In Library" badge stays version-level and gives no hint of remaining files.
**Estimated:** ~1030 LOC (confirmed realistic by review).
### 5.3 Phase 1 tests
Backend — extend `tests/services/test_download_manager_basic.py` (1694 lines; all fixture patterns exist):
- **Fixture gaps to add (F10):** `DummyScanner.get_cached_data()`/`raw_data` stub (~10 lines); `hashes.SHA256` in the metadata-provider payload's `files`.
- Cases: same version + different SHA256 in library + `file_params` → proceeds; same SHA256 → hard error; `file_params=None` + version in library → hard error (unchanged); history-skip on + `file_params` → not skipped; without → skipped (unchanged); empty-dict `file_params` normalized → version-level behavior; `model_id`-only + `file_params` → gate and selection resolve the same file; legacy metadata (empty local sha256) matched by name; target file with empty SHA256 → name fallback, no `""==""` false positive.
- Queue retry: `file_params` survives retry.
- Assert proceed/abort via the existing `_execute_download` mock pattern.
Frontend (`tests/frontend/`): badge renders for multi-file version with `existsLocally=true` (pattern from `downloadManager.history.test.js`).
**Estimated:** ~150250 LOC (confirmed realistic).
## 6. Implementation — Phase 2 (per-file status + multi-select + index hardening)
### 6.1 Backend
1. **`py/services/model_cache.py`** (D3): add `version_files_index`; maintain in `add_to_version_index` / `remove_from_version_index` / `rebuild_version_index`; removal re-points `version_index[version_id]` (and the `model_id_index` descriptor) to a surviving sibling instead of dropping the key.
2. **`py/services/model_scanner.py`**: expose `get_files_for_version(version_id) -> List[dict]`.
3. **`py/routes/handlers/model_handlers.py` `get_civitai_versions`**: annotate each version with `downloadedFiles: [{fileId, fileName, filePath}]` via `version_files_index` + D2 matching against `version.files`.
4. **`py/services/downloaded_version_history_service.py`** (D4): new child table `downloaded_version_files`; `mark_downloaded` also upserts the child row when `file_id` known; `mark_as_deleted` clears the version's child rows only when no sibling remains in the library; new `get_downloaded_file_ids(model_type, version_id) -> set[int]`. `_record_downloaded_version_history` passes `file_info` through.
5. **`delete_model_version`** (`misc_handlers.py:24102487`, R5): resolve **all** local files of the version via `version_files_index`; delete all (current endpoint semantics are version-level) or — if kept per-file — only `mark_as_deleted` when no sibling remains. Decide at implementation time; minimum is documenting current behavior.
6. **`ModelVersionsTab` backend support**: none needed beyond item 3 (`downloadedFiles`); the tab consumes the same versions payload.
### 6.2 Frontend
1. **File dialog multi-select** — change surface (F8): option markup (`DownloadManager.js:712724`), the single-select click handler (`727734`), the `input[type="radio"]:checked` selector in `confirmFileSelection` (`738`); template `templates/components/modals/download_modal.html:4860` (confirm-button label only); CSS `download-modal.css` — checkbox variant of `.file-option-radio input` (595604) and a **new** `.file-option.disabled` style (does not exist). Files whose id ∈ `downloadedFiles` render disabled with an "In Library" tag.
2. **Mixed-type guard (F8):** multi-select is restricted to files sharing the same routing target (`_isDiffusionModel` is computed once from a single `selectedFile` at 798803; e.g. "Model" + "UNet" files route to different roots). Disallow mixed-type multi-select (simplest, predictable); single-file selection unchanged.
3. **Multi-file download loop (D5/F7):** extract from `executeDownloadWithProgress` a reload-deferred, no-toast inner function; iterate per selected file with per-file progress; one `resetAndReload(true)` + one aggregated success/failure summary at the end (reuse `showDownloadBatchSummary`).
4. **`updateNextButtonState` / `proceedToLocation`:** for multi-file versions, Next routes into the file dialog; hard block only when *every* weight file is downloaded.
5. **`ModelVersionsTab.js` (F6):** the Download action (`:576` hidden when `isInLibrary`) — for multi-file versions with remaining files, show it and route into the download modal's file dialog; keep hidden when all files present.
6. **Batch preview (F5):** `batch-preview-local-badge` (`:1320`) gains a "partially downloaded" hint for multi-file versions with remaining files.
7. New i18n keys (`modals.download.fileSelection.inLibrary`, `downloadSelected`, partial-download tooltip, etc.) → run `python scripts/sync_translation_keys.py`.
### 6.3 Phase 2 tests
- `model_cache` (`tests/services/test_model_cache.py` already covers add/remove at 4455): multi-valued index; sibling re-point on removal; rebuild.
- `get_civitai_versions`: `downloadedFiles` correctness (hash match, name fallback, no match, CivArchive no-hash payload).
- History service (`tests/services/test_downloaded_version_history_service.py` uses real SQLite on tmp_path): child-table creation on a legacy DB; per-file record/query; `mark_as_deleted` sibling semantics.
- Frontend: dialog checkbox rendering/disabled state and multi-file confirm — **greenfield behavior coverage** (F10: no existing test exercises `showFileSelectionStep`/`confirmFileSelection`; infra exists, patterns must be built).
## 7. Risks and Mitigations
| Risk | Impact | Mitigation |
|---|---|---|
| History-gate bypass (D1) causes unwanted re-downloads in automated flows | Large checkpoint files re-downloaded | Bypass only with normalized, resolvable `file_params` (D1a); all such senders are user-initiated dialog flows (2.6, verified); tests pin batch/recipe/bulk behavior. |
| Empty-hash matching edge cases (R6) | Duplicate download of the same file, or false block | D2 rule: hash only when both non-empty; name otherwise; never `""==""`. Residual risk documented (2.4). |
| Phase-1 late-failure UX (F12) | User picks a downloaded file, fails only after location step | Toast surfacing (5.2.3); phase 2 disables downloaded files up front. |
| Phase-2 index change corrupts existing behavior | Recipe matching, delete flows | Additive index + re-point only; `version_index` read semantics unchanged; `remove_models`/`update_single_model_cache` already rebuild (F3); tests. |
| `delete_model_version` marks whole version deleted while sibling remains (R5) | History wrongly suppresses re-download of the surviving sibling's version | Phase 2 item 6.1.5; documented until then. |
| History child-table migration failure on user installs | Service init crash | `CREATE TABLE IF NOT EXISTS` in `_initialize_schema`; failure degrades to version-level behavior (per-file queries return empty). |
| Batch-preview badge misleading for partial versions (F5) | Minor UX confusion | Acknowledged in phase 1; fixed in phase 2 item 6.2.6. |
| UI confusion: version shows "In Library" while files remain downloadable | Support burden | Phase 2: per-file disabled state + partial-download tooltip. |
| Hash-identical sibling files (repacked content) | Second file blocked | Acceptable: scanner hash dedup already collapses them. |
## 8. Rollout
1. **Commit 1**`fix(download): allow downloading additional files of an in-library model version (#1058)` → Phase 1 (5.15.3).
2. **Commit 2**`feat(download): per-file download status and multi-file selection (#1058)` → Phase 2 (6.16.3).
Phase 1 alone resolves the issue as reported; phase 2 can ship in a later release if review prefers smaller increments.
## 9. Effort Estimate (revised after review)
| Phase | Backend | Frontend | Tests | Risk |
|---|---|---|---|---|
| 1 | ~150220 LOC (+ queue-retry fix ~30) | ~1030 LOC | ~150250 LOC | Low |
| 2 | ~250350 LOC | ~250350 LOC (multi-file loop refactor + ModelVersionsTab + batch badge) | ~250350 LOC (dialog tests greenfield) | Medium |
+337
View File
@@ -0,0 +1,337 @@
# Plan: Global Rate-Limit Abidance for Recipe Ingest & Metadata Fetching
**Issue:** [#1085 — Large Recipe Ingest Appears to not abide by vendor rate limits, possibly a few other errors?](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1085)
**Status:** v2 — reviewed; decisions recorded in §10. **Phase 1 implemented**
(2026-08-27, commit `c2a2048c`): coordinator + downloader gate + Fix C
failover semantics + helper double-wait fix + settings. **Phase 2
implemented** (2026-08-27): batch-import rate-limit failures map to
`SKIPPED` + `rate_limited` WebSocket flag + UI slowdown hint (toast + status
text, i18n keys synced); `download_to_memory` / `get_response_headers` /
`download_file` register 429 cooldowns. Changes vs v1: Fix C moved to
Phase 1, helper double-wait resolved in Phase 1, gate/guard ordering
specified.
**Scope:** HTTP API traffic to CivitAI (`civitai.red`) and CivArchive (`civarchive.com`) from metadata fetching (bulk refresh, metadata sync, recipe analysis/enrichment, usage-control lookups). Large binary downloads (model files / preview images via `download_file`) are out of scope for *pacing* (they are already single-connection transfers) but their 429 responses should still be *registered*.
> Context: a first batch of fixes for this issue was already committed as
> `ee233548` ("fix(recipes): enforce batch-import concurrency bound and harden
> ingest errors (#1085)"): the batch-import concurrency controller now shares a
> real semaphore (bounds 15 actually apply), the Comfy parser tolerates
> list/`None` `ckpt_name`, CivArchive treats empty error payloads as failures,
> and offline-cooldown short-circuits log at DEBUG. This plan covers the two
> remaining orchestration-level fixes:
> **Fix 2** — slow down globally when a vendor rate limit is hit (respect
> `Retry-After`, queue instead of hammering); **Fix 3** — stop immediately
> failing over to CivArchive when CivitAI is rate-limited.
---
## 1. Problem Statement
During a large recipe ingest (e.g. importing the example-images directory,
which can be thousands of images), the manager fires one metadata request per
checkpoint + per LoRA per image through the fallback provider chain
(`civitai_api → civarchive_api → sqlite`). Consequences observed in #1085:
1. **CivitAI gets hammered** → 429s. The consumer then *immediately* tries
CivArchive for the same lookup, so **CivArchive gets hammered too** before
it was ever naturally needed (its only real job is recovering metadata for
models deleted from CivitAI).
2. Requests are retried per-call after `Retry-After`, but **each concurrent
call sleeps independently** → thundering herd: thousands of coroutines wake
at the same moment and re-flood the vendor.
3. While CivArchive is in the `ConnectivityGuard` cooldown, every batch item
short-circuits and is marked `FAILED` — the batch import's success/failure
accounting is polluted by a transient vendor state (log spam was fixed in
`ee233548`; the item-failure accounting is not).
4. `ConnectivityGuard` (`py/services/connectivity_guard.py`) only treats
transport-level unreachability as offline; **HTTP 429 is invisible to it**,
so nothing ever intentionally paces request rate.
User expectation from the issue: *"once a vendor rate limit time out is hit,
you should trigger a slow down with intentional reduction in request rate"*.
## 2. Current State (verified against code)
### 2.1 Where 429s are surfaced
- `Downloader.make_request` (`py/services/downloader.py:1120-1132`): HTTP 429 →
returns `RateLimitError(message, retry_after=…)` parsed from `Retry-After`
(missing header defaults to `None`).
- `CivitaiClient._make_request` (`py/services/civitai_client.py:97-100`):
converts `RateLimitError` to a raise immediately; no waiting. Transient
5xx/connection errors are retried 3× with 1s/2s/4s backoff.
- `CivArchiveClient._make_request` (`py/services/civarchive_client.py`):
raises `RateLimitError` with `provider="civarchive_api"` when not set.
- `_RateLimitRetryHelper` (`py/services/model_metadata_provider.py:45-102`):
per-call retry loop — sleeps `retry_after` (capped at 1800 s; `≥120 s` ⇒ no
retry), then re-raises. Because every concurrent call runs its own helper,
they sleep in parallel and re-fire in parallel.
- `FallbackMetadataProvider` (`py/services/model_metadata_provider.py:488-508,
564-584` etc.): on a final `RateLimitError` from one provider it logs
"skipping to next provider" and **continues to the next network provider** —
this is the direct cause of the CivArchive flood.
- `MetadataSyncService.fetch_and_update_model`
(`py/services/metadata_sync_service.py:248-333`): manually iterates
`provider_attempts`; on `RateLimitError` it `continue`s to the next provider
(same failover problem), then reports `"Rate limited"` when nothing
succeeded.
- `Downloader.make_request` has a per-destination scope already available:
`_guard_destination(url)` returns the hostname (`downloader.py:1194-1199`),
used by `ConnectivityGuard`.
### 2.2 What pacing exists today
- `ConnectivityGuard`: per-destination cooldown (30 s base, ×2 per extra
failure batch, 300 s cap) triggered only by transport errors
(`connectivity_guard.py:168-197`).
- `AdaptiveConcurrencyController` (batch import, fixed in `ee233548`): shared
semaphore enforces 15 concurrent items; *duration*-based adjustment only —
it never sees HTTP statuses, so it cannot distinguish "slow because rate
limited" from "slow because big image".
- No token bucket, no minimum inter-request interval, no shared
`Retry-After` gate anywhere (`grep` for throttle/token-bucket/rate-limiter:
0 hits).
## 3. Requirements & Constraints
R1. **Respect `Retry-After`.** After a 429, no further request to that
destination may be sent before the vendor's retry window elapses.
R2. **No thundering herd.** Concurrent waiters must share one wake-up (gate),
not sleep independently.
R3. **No double load.** A CivitAI 429 must not trigger a CivArchive request
for the same lookup. CivArchive should only be consulted when CivitAI
legitimately has no answer (404 / "not found"), or when CivitAI is
unreachable long-term.
R4. **No spurious item failures.** A rate-limited request must not turn a
batch-import item into `FAILED`; it should wait (bounded) and retry, or at
worst be `SKIPPED` with a clear "rate limited" reason (re-runnable import).
R5. **Never hang forever.** All waiting is bounded by a configurable cap; on
expiry the caller receives the `RateLimitError` and can decide.
R6. **Keep legitimate failover.** Deleted-model recovery via CivArchive/sqlite
must keep working (404 paths unchanged).
R7. **Single choke point.** The pacing gate should live where every API call
passes (the `Downloader`), so bulk refresh, metadata sync, recipe
analysis, and usage-control lookups all benefit without per-feature work.
## 4. Approach Comparison
### A. Reactive gate — shared `Retry-After` deadman clock (recommended core)
A process-wide, per-destination coordinator records the *next-allowed-send*
timestamp from each 429 (`now + max(retry_after, backoff)`). Every request
through `Downloader.make_request` consults the gate *before sending* and *when
a 429 arrives*; waiters block on a shared `asyncio.Event` that fires when the
cooldown expires.
- Pros: single choke point (R7); herd-free (R2); honors server guidance (R1);
no guessing at vendor limits; covers all providers automatically; reuses
existing per-destination scoping.
- Cons: still experiences 429s before slowing down (reactive); long
`Retry-After` windows (CivArchive has been observed at ~1500 s) need a sane
wait cap + skip/retry UX.
### B. Preemptive pacing — minimum inter-request interval (recommended companion)
Per-destination token bucket (simplest form: capacity 1 — at least `N` seconds
between consecutive API requests; `N` configurable, default ~0.75 s ≈ 80
r/min ceiling).
- Pros: prevents most 429s before they happen — exactly the "intentional
reduction in request rate" the issue asks for; trivial to implement on top
of A's coordinator.
- Cons: adds latency to bulk operations (thousands of models × `N`); the *exact*
vendor limits are unknown (CivitAI anonymous vs keyed vs `civitai.red`
mirror differ), so the default must be conservative-but-not-crippling and
settings-tunable.
### C. Fallback semantics change — stop network→network failover on 429 (must-do, low risk)
`FallbackMetadataProvider` (and `MetadataSyncService.fetch_and_update_model`'s
manual loop) must treat a final `RateLimitError` as a **terminal, non-failover
result** for network providers. Local-only providers (sqlite archive DB) may
stay as a last resort (no vendor cost).
- Pros: directly removes the CivArchive flood; small, surgical change.
- Cons: none significant; requires care to keep 404-failover intact (R6).
### Rejected / deferred
- **Per-feature retry queues** (batch import pauses & resumes whole batches):
richer UX but much larger change (batch state machine, WebSocket states);
unnecessary once A+B make requests wait at the choke point. Defer unless
review finds the bounded-wait UX insufficient.
- **Full token bucket with burst credit**: overkill; capacity-1 interval is
enough given the shared semaphore already caps concurrency at 5.
- **Retrying in `connectivity_guard`**: wrong layer — the guard is about
transport reachability, not vendor quota.
## 5. Recommended Architecture
New singleton **`RateLimitCoordinator`** (`py/services/rate_limit_coordinator.py`,
mirroring `ConnectivityGuard`'s singleton + per-destination patterns):
```
state per destination (hostname):
next_allowed_send: float (monotonic) # from 429 Retry-After + backoff
consecutive_429: int # for backoff growth
last_send_at: float # for min-interval pacing
waiters: list[Future] | asyncio.Event # shared wake-up per cooldown cycle
```
API:
- `async wait_for_slot(destination, request_started_within_window: bool)`
— called by `Downloader.make_request` *before* sending (blocks until
`min(now >= next_allowed_send)` and inter-request interval elapses) and
re-armable after a 429.
- `register_rate_limit(destination, retry_after: float | None)`
— called on 429: `next_allowed_send = max(now + retry_after_or_backoff, current)`;
`consecutive_429 += 1`; backoff = `retry_after` honored, else exponential
`30 · 2^(n-1)` capped at 1800 s; creates/re-arms the shared wake-up event.
- `register_success(destination)` — resets `consecutive_429` (called from the
existing 200 path in `make_request`).
- `remaining_seconds(destination)`, `in_cooldown(destination)` — for tests and
diagnostics.
Enforcement points:
1. **`Downloader.make_request`** (`downloader.py:1102-1132`): ordering inside
the method is **connectivity-guard fail-fast first** (offline short-circuit
costs nothing to check), **then** `await coordinator.wait_for_slot(destination)`
before `session.request`. On 429: `coordinator.register_rate_limit(...)`,
then *wait for the gate and re-send* (loop, bounded by
`rate_limit_max_wait_seconds`, default 300; `retry_after ≥ cap` ⇒ fail
immediately). After the loop, return the `RateLimitError` to the caller
(unchanged contract) **with `exc.gate_handled = True` set** so downstream
retry helpers know the wait already happened. 200 path calls
`register_success`.
2. **`Downloader.download_to_memory` / `get_response_headers`** (phase 2):
register 429s (so API calls queue); waiting only in `make_request`
initially.
3. **`FallbackMetadataProvider`** (`model_metadata_provider.py`): remove
network→network failover on `RateLimitError` — re-raise; only sqlite stays
as a local last resort (implementation: per-method `except RateLimitError`
handler that marks the chain rate-limited and stops iterating).
4. **`MetadataSyncService.fetch_and_update_model`**
(`metadata_sync_service.py:248-333`): on `RateLimitError` from the default
provider, stop appending further network providers (sqlite may remain);
the existing `any_rate_limited` merge already produces `"Rate limited"`.
5. **Batch import** (`batch_import_service.py`): no structural change needed —
items now wait inside `make_request`; optionally (phase 2) map residual
rate-limit failures (after the wait cap) to `SKIPPED` with
`"rate limited (retry_after=…s); re-run the import later"` instead of
`FAILED`, and surface a `rate_limited` flag in the WebSocket progress
broadcast.
6. **`_RateLimitRetryHelper` retries** (`model_metadata_provider.py`):
**Phase 1** — when the raised `RateLimitError` carries `gate_handled = True`
(set by the downloader after honoring the gate), the helper skips its own
`retry_after` sleep and re-raises immediately, eliminating the double wait.
The wiring stays so a `RateLimitError` still propagates cleanly; full
demotion/removal can follow once the gate proves out.
Settings (`settings.json`, schema extension in `SettingsManager`):
| key | default | meaning |
|---|---|---|
| `rate_limit_gate_enabled` | `true` | master switch for the coordinator |
| `rate_limit_max_wait_seconds` | `300` | how long `make_request` waits on a 429 gate before returning the error |
| `rate_limit_min_interval_seconds` | `0.75` | minimum seconds between API requests per destination (pacing, R6-friendly conservative default) |
## 6. Changes by File
| File | Change |
|---|---|
| `py/services/rate_limit_coordinator.py` (new) | coordinator singleton + per-destination state + tests seam |
| `py/services/downloader.py` | gate pre-check + 429 register/wait/retry loop + `register_success`; log the 429 notice at INFO once per cooldown, then DEBUG |
| `py/services/model_metadata_provider.py` | `FallbackMetadataProvider`: stop network failover on `RateLimitError`; helper skips its sleep when the error is marked `gate_handled` |
| `py/services/metadata_sync_service.py` | `fetch_and_update_model`: same failover semantics; keep sqlite last resort |
| `py/services/batch_import_service.py` | (phase 2) rate-limit failures → `SKIPPED` + `rate_limited` progress flag |
| `py/services/settings_manager.py` | new settings keys + defaults |
| `tests/services/test_rate_limit_coordinator.py` (new) | gate unit tests |
| `tests/services/test_civitai_client.py` / `test_civarchive_client.py` | provider-level 429 behavior |
| `tests/services/test_metadata_service.py` | failover-chain tests |
| `tests/services/test_batch_import_service.py` | SKIPPED-on-rate-limit |
## 7. Impact, Risks, Open Questions
- **Behavior change**: with the gate in `make_request`, any request can block
up to the wait cap — UI actions that call the API (e.g. a model-details
fetch) may take longer during cooldowns. Mitigation: bounded cap + INFO log
+ the existing async request handling already tolerates slow responses.
**Decided (§10): interactive requests take the same bounded wait** — one
behavior, no call-source plumbing; cooldowns are usually short.
- **Gate waits occupy batch slots**: with the 15 batch semaphore, all slots
can park on a gate simultaneously, freezing visible progress for up to one
wait cap per wave. Bounded and acceptable; the phase-2 `SKIPPED` mapping +
WebSocket `rate_limited` flag (both confirmed in scope, §10) make the stall
visible and recoverable.
- **Rate limit reality check**: CivitAI anonymous vs keyed limits, and whether
`civitai.red` differs, is unverified. Default pacing `0.75 s/req` is a
conservative guess (R6). Open question for maintainer: preferred default
and whether an API-keyed ceiling should be higher.
- **Long CivArchive windows**: `Retry-After ~1500 s` observed in code
comments. **Decided (§10): keep the 300 s default cap** — such lookups
fail/skip rather than park a request path for 25 minutes; batch import maps
them to `SKIPPED` (phase 2) so the user can re-run later.
- **Double waiting**: `_RateLimitRetryHelper` + gate could stack waits.
**Resolved in Phase 1**: the downloader marks gate-honored errors with
`gate_handled = True` and the helper skips its own sleep for those.
- **Downloads**: `download_file` 429s return an error to download managers
unchanged (already handled); only *registration* is proposed, so future
API calls queue behind a large `Retry-After` from a download burst.
## 8. Test Plan
1. **Coordinator unit tests** (new file):
- 429 with `retry_after` → `wait_for_slot` blocks ~that long, then passes.
- N concurrent waiters all wake together (herd test, wall-clock ≈ one
window, not N windows).
- Consecutive 429s grow backoff; `register_success` resets.
- Missing `Retry-After` → default backoff path.
- Wait cap: request fails after `rate_limit_max_wait_seconds` with
`RateLimitError`.
2. **Downloader tests** (mock aiohttp session): 429 then 200 → `make_request`
returns success after gate delay; two back-to-back calls to the same
destination are spaced ≥ `min_interval`; different destinations are not
spaced.
3. **Provider tests**: `FallbackMetadataProvider.get_model_version_info` —
Civitai raises `RateLimitError` → CivArchive mock **not called**; 404 still
falls through to CivArchive; sqlite still tried after network 429.
4. **Sync-service test**: `fetch_and_update_model` with a rate-limited default
provider → result error contains `"Rate limited"` and sqlite attempt state
unchanged.
5. **Batch-import test**: analysis provider 429s first, then succeeds →
item ends `SUCCESS` (wait path), and post-cap 429 → `SKIPPED` with
rate-limit reason (phase 2).
6. Full regression: `pytest tests/services tests/routes tests/standalone`
(currently 1582 passing).
## 9. Implementation Phases
- **Phase 1 (this plan, reviewed):** `RateLimitCoordinator` +
`Downloader.make_request` integration (guard fail-fast → gate pre-check
pacing → 429 register/wait/retry loop with cap → `gate_handled` marking) +
settings + **Fix C failover semantics** (`FallbackMetadataProvider`,
`fetch_and_update_model` — moved up from phase 2: smallest diff, kills the
CivArchive flood immediately, independent of coordinator correctness) +
`_RateLimitRetryHelper` double-wait fix + coordinator/downloader/provider/
sync tests.
- **Phase 2:** batch-import `SKIPPED`-on-rate-limit + `rate_limited` WebSocket
progress flag + slowdown hint (confirmed, §10),
`download_to_memory`/HEAD 429 registration, batch tests.
- **Phase 3:** full regression + docs + commit referencing `(#1085)`.
## 10. Review Checklist — Decisions (2026-08-27)
- [x] Default pacing interval `0.75 s` — **accepted** as conservative default;
tunable via `rate_limit_min_interval_seconds`. Revisit if CivitAI
publishes keyed/anonymous ceilings.
- [x] Wait cap `300 s` — **accepted**; long-window CivArchive lookups fail →
batch import marks them `SKIPPED` with a rate-limit reason (phase 2).
- [x] Interactive API calls also wait (bounded) — **yes**, same behavior for
all callers.
- [x] Keep sqlite as last resort behind a network rate limit — **yes**
(local-only, no vendor cost).
- [x] UI hint — **yes**: WebSocket `rate_limited` flag + "rate limited —
slowing down" hint in batch-import progress (phase 2); INFO logging
regardless.
File diff suppressed because one or more lines are too long
+462 -208
View File
File diff suppressed because it is too large Load Diff
+328 -74
View File
@@ -67,11 +67,11 @@
"steps": {
"fetch": {
"title": "Fetch Models Metadata",
"content": "Click the <strong>Fetch</strong> button to download model metadata and preview images from Civitai."
"content": "Click the <strong>Fetch</strong> button to download model metadata and preview images from CivitAI."
},
"download": {
"title": "Download New Models",
"content": "Use the <strong>Download</strong> button to download models directly from Civitai URLs."
"content": "Use the <strong>Download</strong> button to download models directly from CivitAI URLs."
},
"bulk": {
"title": "Bulk Operations",
@@ -103,8 +103,8 @@
"actions": {
"addToFavorites": "Add to favorites",
"removeFromFavorites": "Remove from favorites",
"viewOnCivitai": "View on Civitai",
"notAvailableFromCivitai": "Not available from Civitai",
"viewOnCivitai": "View on CivitAI",
"notAvailableFromCivitai": "Not available from CivitAI",
"viewOnHuggingFace": "View on Hugging Face",
"sendToWorkflow": "Send to ComfyUI (Click: Append, Shift+Click: Replace)",
"copyLoRASyntax": "Copy LoRA Syntax",
@@ -137,7 +137,7 @@
"exampleImages": {
"checkError": "Error checking for example images",
"missingHash": "Missing model hash information.",
"noRemoteImagesAvailable": "No remote example images available for this model on Civitai"
"noRemoteImagesAvailable": "No remote example images available for this model on CivitAI"
},
"badges": {
"update": "Update",
@@ -186,6 +186,16 @@
"cancelled": "Repair cancelled. {count} recipes were repaired.",
"error": "Recipe repair failed: {message}"
},
"rematchRecipes": {
"label": "Rematch recipes to local models",
"loading": "Rematching recipes to local models...",
"success": "Matched {entries} entries across {recipes} recipes",
"successErrors": "Matched {entries} entries across {recipes} recipes, {failures} failed",
"allFailed": "Rematch failed for {failures} of {total} recipes",
"noMatch": "No local match found for {entries} entries in {recipes} recipes",
"cancelled": "Rematch cancelled. {recipes} recipes updated ({entries} entries).",
"error": "Recipe rematch failed: {message}"
},
"manageExcludedModels": {
"label": "Manage Excluded Models"
},
@@ -212,6 +222,7 @@
"modelname": "Model Name",
"tags": "Tags",
"creator": "Creator",
"hash": "Hash",
"title": "Recipe Title",
"loraName": "LoRA Filename",
"loraModel": "LoRA Model Name",
@@ -233,7 +244,7 @@
"presetNamePlaceholder": "Preset name...",
"baseModel": "Base Model",
"baseModelSearchPlaceholder": "Search base models...",
"modelTags": "Tags (Top 20)",
"modelTags": "Tags",
"modelTypes": "Model Types",
"license": "License",
"noCreditRequired": "No Credit Required",
@@ -241,13 +252,19 @@
"allowSellingGeneratedContentTooltip": "Allow selling generated images",
"noCreditRequiredTooltip": "Use the model without crediting the creator",
"noTags": "No tags",
"tagSearchPlaceholder": "Search tags...",
"noTagMatches": "No tags match the current search.",
"autoTags": "Auto Tags",
"noBaseModelMatches": "No base models match the current search.",
"clearAll": "Clear All Filters",
"any": "Any",
"all": "All",
"tagLogicAny": "Match any tag (OR)",
"tagLogicAll": "Match all tags (AND)"
"tagLogicAll": "Match all tags (AND)",
"loraAvailability": "Lora Availability",
"availabilityReady": "Ready to use",
"availabilityMissing": "Has missing",
"availabilityDeleted": "Has deleted"
},
"theme": {
"toggle": "Toggle theme",
@@ -273,15 +290,15 @@
}
},
"settings": {
"civitaiApiKey": "Civitai API Key",
"civitaiApiKeyPlaceholder": "Enter your Civitai API key",
"civitaiApiKeyHelp": "Used for authentication when downloading models from Civitai",
"civitaiApiKey": "CivitAI API Key",
"civitaiApiKeyPlaceholder": "Enter your CivitAI API key",
"civitaiApiKeyHelp": "Used for authentication when downloading models from CivitAI",
"civitaiApiKeyConfigured": "Configured",
"civitaiApiKeyNotConfigured": "Not configured",
"civitaiApiKeySet": "Set up",
"civitaiHost": {
"label": "Civitai host",
"help": "Choose which Civitai site opens when using View on Civitai links.",
"label": "CivitAI host",
"help": "Choose which CivitAI site opens when using View on CivitAI links.",
"options": {
"com": "civitai.com (SFW)",
"red": "civitai.red (unrestricted)"
@@ -302,8 +319,8 @@
},
"aria2HelpLink": "Learn how to set up the aria2 download backend",
"civitaiHostBanner": {
"title": "Civitai host preference available",
"content": "Civitai now uses civitai.com for SFW content and civitai.red for unrestricted content. You can change which site opens by default in Settings.",
"title": "CivitAI host preference available",
"content": "CivitAI now uses civitai.com for SFW content and civitai.red for unrestricted content. You can change which site opens by default in Settings.",
"openSettings": "Open Settings"
},
"openSettingsFileLocation": {
@@ -433,7 +450,7 @@
},
"layoutSettings": {
"groupByModel": "Group by Model",
"groupByModelHelp": "When enabled, only the latest version of each Civitai model is shown as a single card. Older versions are hidden.",
"groupByModelHelp": "When enabled, only the latest version of each CivitAI model is shown as a single card. Older versions are hidden.",
"displayDensity": "Display Density",
"displayDensityOptions": {
"default": "Default",
@@ -447,6 +464,12 @@
"compact": "7 (1080p), 8 (2K), 10 (4K)"
},
"displayDensityWarning": "Warning: Higher densities may cause performance issues on systems with limited resources.",
"recipesLayout": "Recipes Layout",
"recipesLayoutHelp": "Choose how recipe cards are arranged: a uniform grid or a masonry (Pinterest-style) layout that preserves each image's aspect ratio.",
"recipesLayoutOptions": {
"grid": "Grid",
"masonry": "Masonry"
},
"showFolderSidebar": "Show Folder Sidebar",
"showFolderSidebarHelp": "Toggle the folder navigation sidebar on model pages. When disabled, the sidebar and hover area stay hidden.",
"cardInfoDisplay": "Card Info Display",
@@ -505,7 +528,9 @@
"saveSuccess": "Extra folder paths updated. Restart required to apply changes.",
"saveError": "Failed to update extra folder paths: {message}",
"validation": {
"duplicatePath": "This path is already configured"
"duplicatePath": "This path is already configured",
"checkpointUnetOverlap": "Cannot use the same path for both checkpoints and diffusion models: {paths}",
"checkpointUnetOverlapInline": "This path is also used for a different model type. Use separate folders for checkpoints and diffusion models."
}
},
"priorityTags": {
@@ -530,7 +555,7 @@
},
"downloadPathTemplates": {
"title": "Download Path Templates",
"help": "Configure folder structures for different model types when downloading from Civitai.",
"help": "Configure folder structures for different model types when downloading from CivitAI.",
"availablePlaceholders": "Available placeholders:",
"templateOptions": {
"flatStructure": "Flat Structure",
@@ -567,7 +592,7 @@
"exampleImages": {
"downloadLocation": "Download Location",
"downloadLocationPlaceholder": "Enter folder path for example images",
"downloadLocationHelp": "Enter the folder path where example images from Civitai will be saved",
"downloadLocationHelp": "Enter the folder path where example images from CivitAI will be saved",
"autoDownload": "Auto Download Example Images",
"autoDownloadHelp": "Automatically download example images for models that don't have them (requires download location to be set)",
"openMode": "Open Example Images Action",
@@ -602,6 +627,10 @@
"label": "Hide Early Access Updates",
"help": "When enabled, models with only early access updates will not show 'Update available' badge"
},
"hidePaidUpdates": {
"label": "Hide Paid Updates",
"help": "When enabled, models with only paid updates will not show 'Update available' badge"
},
"licenseIcons": {
"useNewStyle": "Use updated license icons",
"useNewStyleHelp": "Display license permissions with colored indicators (new style) or restriction-only icons (classic style). Mirroring the current CivitAI design."
@@ -618,7 +647,7 @@
},
"metadataArchive": {
"enableArchiveDb": "Enable Metadata Archive Database",
"enableArchiveDbHelp": "Use a local database to access metadata for models that have been deleted from Civitai.",
"enableArchiveDbHelp": "Use a local database to access metadata for models that have been deleted from CivitAI.",
"status": "Status",
"statusAvailable": "Available",
"statusUnavailable": "Not Available",
@@ -638,7 +667,13 @@
"preparing": "Preparing download...",
"connecting": "Connecting to download server...",
"completed": "Completed",
"downloadComplete": "Download completed successfully"
"downloadComplete": "Download completed successfully",
"enableCivarchiveApi": "Enable CivArchive API as metadata provider",
"enableCivarchiveApiHelp": "When on, CivArchive API is used as a fallback source for model metadata (e.g. for models deleted from CivitAI). Turn off to avoid CivArchive rate limits entirely.",
"providerOrder": "Metadata provider fallback order",
"providerOrderHelp": "CivitAI API is always tried first. Choose the order of the remaining providers when looking up metadata.",
"providerOrderCivitaiArchiveSqlite": "CivitAI → CivArchive → Archive DB",
"providerOrderCivitaiSqliteArchive": "CivitAI → Archive DB → CivArchive"
},
"proxySettings": {
"enableProxy": "Enable App-level Proxy",
@@ -657,6 +692,33 @@
"proxyPassword": "Password (Optional)",
"proxyPasswordPlaceholder": "password",
"proxyPasswordHelp": "Password for proxy authentication (if required)"
},
"aiProvider": {
"title": "AI Provider",
"provider": "Provider",
"providerHelp": "Choose your LLM provider. Preset providers set the API base URL automatically. Custom lets you specify any OpenAI-compatible endpoint.",
"providerOptions": {
"openai": "OpenAI",
"ollama": "Ollama (local)",
"deepseek": "DeepSeek",
"groq": "Groq",
"openrouter": "OpenRouter",
"google": "Gemini",
"opencode-go": "OpenCode Go",
"custom": "Custom (OpenAI-compatible)"
},
"apiBase": "API Base URL",
"apiBaseHelp": "The base URL for the LLM API. Select a preset or enter a custom URL. The dropdown shows presets for all supported providers.",
"apiBasePlaceholder": "https://api.openai.com/v1",
"apiKey": "API Key",
"apiKeyHelp": "Your LLM provider API key. Stored locally, never sent to any server except your chosen LLM provider.",
"apiKeyPlaceholder": "sk-...",
"apiKeyNotSet": "Not set",
"apiKeyConfigured": "Configured",
"apiKeySet": "Set up",
"model": "Model",
"modelHelp": "The model to use. Select from the dropdown (fetched from your provider) or type a custom model name.",
"modelPlaceholder": "Select a model..."
}
},
"loras": {
@@ -678,7 +740,9 @@
"versionsCount": "Local Versions",
"versionsCountDesc": "Most versions first",
"versionsCountAsc": "Fewest versions first",
"versionIdDesc": "Newest version first"
"versionIdDesc": "Newest version first",
"random": "Random",
"randomAction": "Randomize (shuffle)"
},
"refresh": {
"title": "Refresh model list",
@@ -686,7 +750,7 @@
"fullTooltip": "Reload all model details from metadata files—use if the library looks out of date or after manual edits."
},
"fetch": {
"title": "Fetch metadata from Civitai",
"title": "Fetch metadata from CivitAI",
"action": "Fetch"
},
"download": {
@@ -723,6 +787,7 @@
"copyAll": "Copy Selected Syntax",
"refreshAll": "Refresh Selected Metadata",
"repairMetadata": "Repair Metadata for Selected",
"rematchMetadata": "Rematch Selected to Local Models",
"reimportMetadata": "Re-import from Source",
"checkUpdates": "Check Updates for Selected",
"moveAll": "Move Selected to Folder",
@@ -735,6 +800,8 @@
"deleteAll": "Delete Selected",
"downloadMissingLoras": "Download Missing LoRAs",
"downloadExamples": "Download Example Images",
"downloadMissingExamples": "Download Missing",
"reprocessExamples": "Re-process All",
"clear": "Clear Selection",
"skipMetadataRefreshCount": "Skip ({count} models)",
"resumeMetadataRefreshCount": "Resume ({count} models)",
@@ -754,12 +821,15 @@
"completed": "Completed: {success} moved, {skipped} skipped, {failures} failed",
"complete": "Auto-organize complete",
"error": "Error: {error}"
}
},
"enrichHfAgent": "Enrich HF Metadata (AI)"
},
"contextMenu": {
"refreshMetadata": "Refresh Civitai Data",
"refreshMetadata": "Refresh CivitAI Data",
"checkUpdates": "Check Updates",
"relinkCivitai": "Re-link to Civitai",
"linkModel": "Link Model",
"linkCivitai": "Link to CivitAI",
"linkHuggingFace": "Link to HuggingFace",
"copySyntax": "Copy LoRA Syntax",
"copyFilename": "Copy Model Filename",
"copyRecipeSyntax": "Copy Recipe Syntax",
@@ -767,10 +837,13 @@
"sendToWorkflowReplace": "Send to Workflow (Replace)",
"openExamples": "Open Examples Folder",
"downloadExamples": "Download Example Images",
"downloadMissingExamples": "Download Missing",
"reprocessExamples": "Re-process All",
"replacePreview": "Replace Preview",
"setContentRating": "Set Content Rating",
"moveToFolder": "Move to Folder",
"repairMetadata": "Repair metadata",
"rematchMetadata": "Rematch to local models",
"reimportMetadata": "Re-import from Source",
"excludeModel": "Exclude Model",
"restoreModel": "Restore Model",
@@ -778,26 +851,72 @@
"shareRecipe": "Share Recipe",
"viewAllLoras": "View All LoRAs",
"downloadMissingLoras": "Download Missing LoRAs",
"deleteRecipe": "Delete Recipe"
"deleteRecipe": "Delete Recipe",
"enrichHfAgent": "Enrich HF Metadata (AI)"
}
},
"recipes": {
"title": "LoRA Recipes",
"actions": {
"sendCheckpoint": "Send to ComfyUI"
"sendCheckpoint": "Send to ComfyUI",
"sendRecipe": "Send to ComfyUI",
"copyRecipeSyntax": "Copy Recipe Syntax",
"deleteRecipeWithShortcut": "Delete recipe (Del)"
},
"navigation": {
"label": "Recipe navigation",
"previousWithShortcut": "Previous recipe (←)",
"nextWithShortcut": "Next recipe (→)"
},
"workflow": {
"sendWorkflow": "Send Workflow to ComfyUI",
"sent": "Workflow sent to ComfyUI",
"sendFailed": "Failed to send workflow to ComfyUI",
"noWorkflow": "No embedded workflow found in this recipe"
},
"status": {
"ready": "Ready to use",
"missingCount": "{count} missing",
"deletedCount": "{count} deleted",
"downloadMissing": "Download {count} missing LoRAs",
"downloadMissingTooltip": "Click to download missing LoRAs"
},
"loraStatus": {
"none": "No LoRAs in this recipe",
"allAvailable": "All LoRAs available - Ready to use",
"missing": "{missing} of {total} LoRAs missing",
"missingAndUnavailable": "{missing} of {total} LoRAs missing, {unavailable} unavailable (deleted from source or unresolvable hash)",
"partial": "{unavailable} of {total} LoRAs unavailable (deleted from source or unresolvable hash) - skipped when recipe is used",
"noneUsable": "No usable LoRAs - {unavailable} of {total} deleted from source or unresolvable hash"
},
"resources": {
"inLibrary": "In Library",
"notInLibrary": "Not in Library",
"deleted": "Deleted",
"hashInvalid": "Unresolvable Hash",
"inLibraryTooltip": "This model exists in your local library",
"notInLibraryTooltip": "This model is not in your library",
"deletedTooltip": "This LoRA was deleted from the source and is no longer available for download",
"hashInvalidTooltip": "This LoRA hash cannot be resolved on CivitAI - the model may have been updated",
"download": "Download",
"downloadLoraTooltip": "Download this LoRA",
"preparingDownload": "Preparing download...",
"reconnect": "Reconnect",
"reconnectTooltip": "Reconnect with a local LoRA",
"viewOnCivitai": "View on CivitAI",
"openLoraDetails": "View {name} in the LoRA library",
"openCheckpointDetails": "View {name} in the model library"
},
"controls": {
"import": {
"action": "Import",
"title": "Import a recipe from image or URL",
"urlLocalPath": "URL / Local Path",
"uploadImage": "Upload Image",
"urlSectionDescription": "Input a Civitai image URL from civitai.com or civitai.red, or a local file path, to import as a recipe.",
"dropZoneLabel": "Upload image",
"dropZoneHint": "Drag & drop an image here, paste from clipboard, or click to browse",
"orDivider": "or drag & drop / paste an image",
"imageUrlOrPath": "Image URL or File Path:",
"urlPlaceholder": "https://civitai.com/images/... or https://civitai.red/images/... or C:/path/to/image.png",
"fetchImage": "Fetch Image",
"uploadSectionDescription": "Upload an image with LoRA metadata to import as a recipe.",
"selectImage": "Select Image",
"recipeName": "Recipe Name",
"recipeNamePlaceholder": "Enter recipe name",
"tagsOptional": "Tags (optional)",
@@ -825,7 +944,7 @@
"downloadingLoras": "Downloading LoRAs...",
"savingRecipe": "Saving recipe...",
"startingDownload": "Starting download for LoRA {current}/{total}",
"deletedFromCivitai": "Deleted from Civitai",
"deletedFromCivitai": "Deleted from CivitAI",
"inLibrary": "In Library",
"notInLibrary": "Not in Library",
"earlyAccessRequired": "This LoRA requires early access payment to download.",
@@ -842,6 +961,8 @@
"errors": {
"selectImageFile": "Please select an image file",
"enterUrlOrPath": "Please enter a URL or file path",
"invalidUrl": "Please enter a valid URL",
"invalidInputFormat": "Please enter an image URL or a local image file path",
"selectLoraRoot": "Please select a LoRA root directory"
}
},
@@ -855,7 +976,9 @@
"dateAsc": "Oldest",
"lorasCount": "LoRA Count",
"lorasCountDesc": "Most",
"lorasCountAsc": "Least"
"lorasCountAsc": "Least",
"opened": "Recently Opened",
"openedDesc": "Recently opened"
},
"refresh": {
"title": "Refresh recipe list",
@@ -866,12 +989,26 @@
"favorites": {
"title": "Show Favorites Only",
"action": "Favorites"
},
"layout": {
"title": "Recipes Layout",
"grid": "Grid layout",
"masonry": "Masonry layout (Pinterest-style, preserves image aspect ratio)"
}
},
"duplicates": {
"finding": "Scanning for duplicate recipes...",
"found": "Found {count} duplicate groups",
"noGroups": "No duplicate groups found with the current matching basis",
"keepLatest": "Keep Latest Versions",
"deleteSelected": "Delete Selected"
"deleteSelected": "Delete Selected",
"includePromptLabel": "Include prompt in matching",
"basis": {
"loraCombo": "Matched by: LoRA combination",
"loraComboAndPrompt": "Matched by: LoRA combination + prompt",
"hintLoraCombo": "Recipes with the same LoRAs at identical strengths are grouped.",
"hintPromptIncluded": "Recipes are grouped only when they use the same LoRAs at identical strengths AND have the same prompt."
}
},
"contextMenu": {
"copyRecipe": {
@@ -930,6 +1067,8 @@
"start": "Start Import",
"startImport": "Start Import",
"importing": "Importing...",
"rateLimitedSlowdown": "Rate limited — slowing down...",
"rateLimitedHint": "Some items were skipped due to metadata provider rate limits. Re-run the import later to retry them.",
"progress": "Progress",
"total": "Total",
"success": "Success",
@@ -1133,7 +1272,7 @@
"download": {
"title": "Download Model from URL",
"titleWithType": "Download {type} from URL",
"civitaiUrl": "Civitai URL(s):",
"civitaiUrl": "CivitAI URL(s):",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Enter one CivitAI, CivArchive, or Hugging Face URL per line. Supports multiple URLs for batch download.",
"selectHfFiles": "Select file(s) to download from this repository:",
@@ -1159,14 +1298,16 @@
"downloaded": "Downloaded",
"downloadedTooltip": "Previously downloaded, but it is not currently in your library.",
"alreadyInLibrary": "Already in Library",
"partiallyDownloaded": "Partially downloaded",
"autoOrganizedPath": "[Auto-organized by path template]",
"fileSelection": {
"title": "Select File Format",
"files": "files",
"select": "Select File"
"select": "Select File",
"inLibrary": "In Library"
},
"errors": {
"invalidUrl": "Invalid Civitai URL format",
"invalidUrl": "Invalid CivitAI URL format",
"noVersions": "No versions available for this model",
"mixedSources": "Cannot mix CivitAI and Hugging Face URLs in the same batch.",
"noModelFiles": "No model files found in this repository."
@@ -1175,7 +1316,9 @@
"preparing": "Preparing download...",
"downloadedPreview": "Downloaded preview image",
"downloadingFile": "Downloading {type} file",
"finalizing": "Finalizing download..."
"finalizing": "Finalizing download...",
"cancelling": "Cancelling download...",
"cancelled": "Download cancelled"
},
"progress": {
"currentFile": "Current file:",
@@ -1202,8 +1345,13 @@
}
},
"deleteModel": {
"freesSpace": "Frees {size}",
"title": "Delete Model",
"message": "Are you sure you want to delete this model and all associated files?"
"message": "Are you sure you want to delete this model and all associated files?",
"recoverableWarning": "This will permanently delete the file after 20 seconds unless you undo."
},
"deleteRecipe": {
"recoverableWarning": "This action can be undone for 20 seconds."
},
"excludeModel": {
"title": "Exclude Model",
@@ -1273,7 +1421,7 @@
"title": "Local Example Images",
"message": "No local example images found for this model. View options:",
"downloadOption": {
"title": "Download from Civitai",
"title": "Download from CivitAI",
"description": "Save remote examples locally for offline use and faster loading"
},
"importOption": {
@@ -1291,8 +1439,16 @@
"pathPlaceholder": "Type folder path or select from tree below...",
"root": "Root"
},
"linkHuggingFace": {
"title": "Link to HuggingFace",
"infoText": "Paste the HuggingFace repository URL to associate this model with its source. This enables AI-powered metadata enrichment.",
"urlLabel": "HuggingFace Repository URL:",
"urlPlaceholder": "https://huggingface.co/user/repo",
"helpText": "Enter the full URL of the HuggingFace repository.",
"confirmAction": "Save & Link"
},
"relinkCivitai": {
"title": "Re-link to Civitai",
"title": "Re-link to CivitAI",
"warning": "Warning:",
"warningText": "This is a potentially destructive operation. Re-linking will:",
"warningList": {
@@ -1301,14 +1457,15 @@
"unintendedConsequences": "May have other unintended consequences"
},
"proceedText": "Only proceed if you're sure this is what you want.",
"urlLabel": "Civitai Model URL:",
"urlPlaceholder": "https://civitai.com/models/649516/model-name?modelVersionId=726676 or https://civitai.red/models/649516/model-name?modelVersionId=726676",
"urlLabel": "CivitAI Model URL:",
"urlPlaceholder": "https://civitai.com/models/12345/model-name?modelVersionId=67890 or https://civitai.red/models/12345/model-name?modelVersionId=67890",
"helpText": {
"title": "Paste any Civitai model URL from civitai.com or civitai.red. Supported formats:",
"format1": "https://civitai.com/models/649516",
"format2": "https://civitai.com/models/649516?modelVersionId=726676",
"format3": "https://civitai.com/models/649516/model-name?modelVersionId=726676",
"note": "Note: If no modelVersionId is provided, the latest version will be used."
"title": "Paste any CivitAI or CivitArchive model URL. Supported formats:",
"format1": "https://civitai.com/models/12345",
"format2": "https://civitai.com/models/12345?modelVersionId=67890",
"format3": "https://civitai.com/models/12345/model-name?modelVersionId=67890",
"note": "Note: If no modelVersionId is provided, the latest version will be used.",
"format4": "https://civarchive.com/models/12345 (CivArchive)"
},
"confirmAction": "Confirm Re-link"
},
@@ -1318,14 +1475,16 @@
"editFileName": "Edit file name",
"editBaseModel": "Edit base model",
"editVersionName": "Edit version name",
"viewOnCivitai": "View on Civitai",
"viewOnCivitaiText": "View on Civitai",
"viewOnCivitai": "View on CivitAI",
"viewOnCivitaiText": "View on CivitAI",
"viewOnHuggingFace": "View on Hugging Face",
"viewOnHuggingFaceText": "View on Hugging Face",
"viewCreatorProfile": "View Creator Profile",
"openFileLocation": "Open File Location",
"sendToWorkflow": "Send to ComfyUI",
"sendToWorkflowText": "Send to ComfyUI"
"sendToWorkflowText": "Send to ComfyUI",
"copyHash": "Copy hash",
"deleteModelWithShortcut": "Delete model (Del)"
},
"openFileLocation": {
"success": "File location opened successfully",
@@ -1342,6 +1501,7 @@
"location": "Location",
"baseModel": "Base Model",
"size": "Size",
"hashes": "Hashes",
"unknown": "Unknown",
"usageTips": "Usage Tips",
"additionalNotes": "Additional Notes",
@@ -1418,7 +1578,7 @@
},
"license": {
"noImageSell": "No selling generated content",
"noRentCivit": "No Civitai generation",
"noRentCivit": "No CivitAI generation",
"noRent": "No generation services",
"noSell": "No selling models",
"creditRequired": "Creator credit required",
@@ -1433,6 +1593,30 @@
"examples": "Loading examples...",
"versions": "Loading versions..."
},
"showcase": {
"hiddenBySfw": "{count} hidden by SFW-only setting",
"showExamples": "Show examples",
"showCount": "Show examples ({count})",
"hideExamples": "Hide examples",
"addExamples": "Add examples",
"previousExample": "Previous example",
"nextExample": "Next example",
"noExamples": "No example images available",
"addMoreExamples": "Add more examples",
"dragDrop": "Drag & drop images or videos here",
"or": "or",
"selectFiles": "Select Files",
"supportedFormats": "Supported formats: jpg, png, gif, webp, avif, jxl, mp4, webm",
"importing": "Importing files...",
"noSupportedFiles": "No supported files selected. Please select image or video files.",
"allFiltered": "All example images are filtered due to NSFW content settings",
"sfwOnlyEnabled": "Your settings are currently set to show only safe-for-work content",
"changeInSettings": "You can change this in Settings",
"nsfwMature": "Mature Content",
"nsfwR": "R-rated Content",
"nsfwX": "X-rated Content",
"nsfwXxx": "XXX-rated Content"
},
"versions": {
"heading": "Model versions",
"copy": "Track and manage every version of this model in one place.",
@@ -1459,24 +1643,28 @@
"newer": "Newer Version",
"newerTooltip": "This version is newer than your latest local version",
"earlyAccess": "Early Access",
"earlyAccessTooltip": "This version currently requires Civitai early access",
"earlyAccessTooltip": "This version currently requires CivitAI early access",
"paid": "Paid",
"paidTooltip": "This version requires payment to download",
"ignored": "Ignored",
"ignoredTooltip": "Update notifications are disabled for this version",
"onSiteOnly": "On-Site Only",
"onSiteOnlyTooltip": "This version is only available for on-site generation on Civitai"
"onSiteOnlyTooltip": "This version is only available for on-site generation on CivitAI"
},
"actions": {
"download": "Download",
"downloadTooltip": "Download this version",
"downloadEarlyAccessTooltip": "Download this early access version from Civitai",
"downloadNotAllowedTooltip": "This version is only available for on-site generation on Civitai",
"downloadChooseFilesTooltip": "Choose which files to download",
"downloadEarlyAccessTooltip": "Download this early access version from CivitAI",
"downloadPaidTooltip": "Download this paid version from CivitAI",
"downloadNotAllowedTooltip": "This version is only available for on-site generation on CivitAI",
"delete": "Delete",
"deleteTooltip": "Delete this local version",
"ignore": "Ignore",
"unignore": "Unignore",
"ignoreTooltip": "Ignore update notifications for this version",
"unignoreTooltip": "Resume update notifications for this version",
"viewVersionOnCivitai": "View version on Civitai",
"viewVersionOnCivitai": "View version on CivitAI",
"earlyAccessTooltip": "Requires early access purchase",
"resumeModelUpdates": "Resume updates for this model",
"ignoreModelUpdates": "Ignore updates for this model",
@@ -1497,7 +1685,8 @@
},
"empty": "No version history available for this model yet.",
"error": "Failed to load versions.",
"missingModelId": "This model is missing a Civitai model id.",
"missingModelId": "This model is missing a CivitAI model id.",
"hfGroupInfo": "This is a HuggingFace model group. Open the library to see all versions in the grid.",
"confirm": {
"delete": "Delete this version from your library?"
},
@@ -1524,6 +1713,21 @@
"downloadCsv": "Download CSV",
"columnModelName": "Model Name",
"columnError": "Error"
},
"downloadBatchSummary": {
"title": "Batch Download Summary",
"statSuccess": "Success",
"statFailed": "Failed",
"statTotal": "Total",
"successMessage": "All {count} models downloaded successfully",
"completedWithErrors": "Completed with errors",
"failed": "Download failed",
"failedItems": "Failed Items ({count})",
"columnName": "Model Name",
"columnError": "Error",
"close": "Close",
"copyReport": "Copy Report",
"retryFailed": "Retry Failed ({count})"
}
},
"modelTags": {
@@ -1565,14 +1769,14 @@
"tips": {
"title": "Tips & Tricks",
"civitai": {
"title": "Civitai Integration",
"description": "Connect your Civitai account: Visit Profile Avatar → Settings → API Keys → Add API Key, then paste it in Lora Manager settings.",
"alt": "Civitai API Setup"
"title": "CivitAI Integration",
"description": "Connect your CivitAI account: Visit Profile Avatar → Settings → API Keys → Add API Key, then paste it in Lora Manager settings.",
"alt": "CivitAI API Setup"
},
"download": {
"title": "Easy Download",
"description": "Use Civitai URLs to quickly download and install new models.",
"alt": "Civitai Download"
"description": "Use CivitAI URLs to quickly download and install new models.",
"alt": "CivitAI Download"
},
"recipes": {
"title": "Save Recipes",
@@ -1622,6 +1826,7 @@
"recipeReplaced": "Recipe replaced in workflow",
"recipeFailedToSend": "Failed to send recipe to workflow",
"noMatchingNodes": "No compatible nodes available in the current workflow",
"noPromptTargets": "No compatible prompt targets in the workflow.\nRight-click a node in ComfyUI → Mark as → Send Prompt Target",
"noTargetNodeSelected": "No target node selected",
"modelUpdated": "Model updated in workflow",
"modelFailed": "Failed to update model node",
@@ -1701,6 +1906,12 @@
"checkingMessage": "Please wait while we check for the latest version.",
"showNotifications": "Show update notifications",
"latestBadge": "Latest",
"latestMain": "Latest main",
"channel": "Update Channel",
"channels": {
"release": "Release",
"nightly": "Nightly"
},
"updateProgress": {
"preparing": "Preparing update...",
"installing": "Installing update...",
@@ -1721,6 +1932,15 @@
"warning": "Warning: Nightly builds may contain experimental features and could be unstable.",
"enable": "Enable Nightly Updates"
},
"channelSwitch": {
"nightlyTitle": "Switch to Nightly Channel",
"nightlyMessage": "Switching to Nightly will initialize a Git repository and track the latest main branch commits. Updates will be more frequent but may be unstable. You can switch back to Release at any time.",
"releaseTitle": "Switch to Release Channel",
"releaseMessage": "Switching to Release will checkout the latest stable release tag. You can switch back to Nightly at any time.",
"switching": "Switching to {channel} channel...",
"completed": "Successfully switched to {channel} channel",
"failed": "Failed to switch channel"
},
"banners": {
"recent": "Recent messages",
"empty": "No recent banners yet.",
@@ -1740,7 +1960,7 @@
"submitGithubIssue": "Submit GitHub Issue",
"joinDiscord": "Join Discord",
"youtubeChannel": "YouTube Channel",
"civitaiProfile": "Civitai Profile",
"civitaiProfile": "CivitAI Profile",
"supportKofi": "Support on Ko-fi",
"supportPatreon": "Support on Patreon"
},
@@ -1783,6 +2003,7 @@
"downloadPartialSuccess": "Downloaded {completed} of {total} LoRAs",
"downloadPartialWithAccess": "Downloaded {completed} of {total} LoRAs. {accessFailures} failed due to access restrictions. Check your API key in settings or early access status.",
"pleaseSelectVersion": "Please select a version",
"pleaseSelectFile": "Please select at least one file",
"versionExists": "This version already exists in your library",
"downloadCompleted": "Download completed successfully",
"downloadSkippedByBaseModel": "Skipped download because base model {baseModel} is excluded",
@@ -1816,6 +2037,8 @@
"createMissingData": "Missing required data to create recipe",
"created": "Recipe created successfully",
"noMissingLoras": "No missing LoRAs to download",
"noPreviousRecipe": "No previous recipe available",
"noNextRecipe": "No next recipe available",
"missingLorasInfoFailed": "Failed to get information for missing LoRAs",
"preparingForDownloadFailed": "Error preparing LoRAs for download",
"enterLoraName": "Please enter a LoRA name or syntax",
@@ -1828,6 +2051,9 @@
"missingCheckpointPath": "Checkpoint path not available",
"missingCheckpointInfo": "Missing checkpoint information",
"downloadCheckpointFailed": "Failed to download checkpoint: {message}",
"missingLoraDownloadInfo": "Missing download information for this LoRA",
"hashNotFoundOnCivitai": "This LoRA hash cannot be resolved on CivitAI - the model may have been updated or the hash is invalid",
"downloadLoraFailed": "Failed to download LoRA: {message}",
"cannotDelete": "Cannot delete recipe: Missing recipe ID",
"deleteConfirmationError": "Error showing delete confirmation",
"deletedSuccessfully": "Recipe deleted successfully",
@@ -1851,18 +2077,28 @@
"batchImportCancelFailed": "Failed to cancel batch import: {message}",
"batchImportNoUrls": "Please enter at least one URL or file path",
"batchImportNoDirectory": "Please enter a directory path",
"batchImportRateLimited": "Metadata provider rate limit reached — requests are being slowed and some items may be skipped. You can re-run the import later.",
"batchImportBrowseFailed": "Failed to browse directory: {message}",
"batchImportDirectorySelected": "Directory selected: {path}",
"noRecipesSelected": "No recipes selected",
"repairBulkComplete": "Repair complete: {repaired} repaired, {skipped} skipped (of {total})",
"repairBulkSkipped": "No repair needed for any of the {total} selected recipes",
"repairBulkFailed": "Failed to repair selected recipes: {message}",
"rematchComplete": "Matched {entries} entries across {recipes} recipes",
"rematchCompleteErrors": "Matched {entries} entries across {recipes} recipes, {failures} failed",
"rematchAllFailed": "Rematch failed for {failures} of {total} selected recipes",
"rematchUnmatched": "No local match found for {entries} entries in {recipes} recipes",
"rematchSkipped": "No rematch needed for any of the {total} selected recipes",
"rematchFailed": "Failed to rematch selected recipes: {message}",
"reimporting": "Re-importing recipe from source...",
"reimportSuccess": "Recipe re-imported successfully",
"reimportBulkComplete": "Re-import complete: {completed} re-imported, {failed} failed (of {total})",
"reimportBulkFailed": "Failed to re-import some recipes",
"noMissingLorasInSelection": "No missing LoRAs found in selected recipes",
"noLoraRootConfigured": "No LoRA root directory configured. Please set a default LoRA root in settings."
"noLoraRootConfigured": "No LoRA root directory configured. Please set a default LoRA root in settings.",
"workflowSent": "Workflow sent to ComfyUI",
"workflowSendFailed": "Failed to send workflow to ComfyUI: {error}",
"workflowNoWorkflow": "No embedded workflow found in this recipe"
},
"models": {
"noModelsSelected": "No models selected",
@@ -1899,8 +2135,8 @@
"bulkUpdatesChecking": "Checking selected {type}(s) for updates...",
"bulkUpdatesSuccess": "Updates available for {count} selected {type}(s)",
"bulkUpdatesNone": "No updates found for selected {type}(s)",
"bulkUpdatesMissing": "Selected {type}(s) are not linked to Civitai updates",
"bulkUpdatesPartialMissing": "Skipped {missing} selected {type}(s) without Civitai links",
"bulkUpdatesMissing": "Selected {type}(s) are not linked to CivitAI updates",
"bulkUpdatesPartialMissing": "Skipped {missing} selected {type}(s) without CivitAI links",
"bulkUpdatesFailed": "Failed to check updates for selected {type}(s): {message}",
"invalidCharactersRemoved": "Invalid characters removed from filename",
"filenameCannotBeEmpty": "File name cannot be empty",
@@ -1965,7 +2201,6 @@
"presetNameTooLong": "Preset name must be {max} characters or less",
"presetNameInvalidChars": "Preset name contains invalid characters",
"presetNameExists": "A preset with this name already exists",
"maxPresetsReached": "Maximum {max} presets allowed. Delete one to add more.",
"presetNotFound": "Preset not found",
"invalidPreset": "Invalid preset data",
"deletePresetFailed": "Failed to delete preset",
@@ -1975,7 +2210,8 @@
"imagesCompleted": "Example images {action} completed",
"imagesFailed": "Example images {action} failed",
"loadError": "Error loading downloads: {message}",
"downloadError": "Download error: {message}"
"downloadError": "Download error: {message}",
"downloadStopped": "Download cancelled"
},
"import": {
"folderTreeFailed": "Failed to load folder tree",
@@ -1993,6 +2229,14 @@
"updateFailed": "Failed to update trigger words",
"copyFailed": "Copy failed"
},
"undo": {
"action": "Undo",
"deleted": "Deleted {name}",
"deletedBulk": "Deleted {count} item(s)",
"expired": "Undo window expired. The item was permanently deleted.",
"failed": "Undo failed: {error}",
"restored": "Item restored"
},
"virtual": {
"loadFailed": "Failed to load items",
"loadMoreFailed": "Failed to load more items",
@@ -2018,8 +2262,11 @@
"contextMenu": {
"contentRatingSet": "Content rating set to {level}",
"contentRatingFailed": "Failed to set content rating: {message}",
"relinkSuccess": "Model successfully re-linked to Civitai",
"relinkSuccess": "Model successfully re-linked to CivitAI",
"relinkFailed": "Error: {message}",
"linkHfSuccess": "Model successfully linked to HuggingFace",
"linkHfFailed": "Error: {message}",
"linkCivArchSuccess": "Model successfully re-linked via CivitArchive",
"fetchMetadataFirst": "Please fetch metadata from CivitAI first",
"noCivitaiInfo": "No CivitAI information available",
"missingHash": "Model hash not available"
@@ -2054,6 +2301,7 @@
"fileRenameFailed": "Failed to rename file: {error}",
"previewUpdated": "Preview updated successfully",
"previewUploadFailed": "Failed to upload preview image",
"previewDropInvalid": "Unsupported file type: {name}. Drop an image or MP4 video instead.",
"refreshComplete": "{action} complete",
"refreshFailed": "Failed to {action} {type}s",
"metadataRefreshed": "Metadata refreshed successfully",
@@ -2081,6 +2329,12 @@
"moveFailed": "Failed to move item: {message}",
"copiedToClipboard": "Copied to clipboard",
"downloadStarted": "Download started"
},
"agent": {
"llmNotConfigured": "AI provider not configured. Enable it in Settings → AI Provider.",
"enrichStarted": "Enriching metadata with AI...",
"enrichComplete": "Metadata enrichment complete: {{summary}}",
"enrichFailed": "Metadata enrichment failed: {{error}}"
}
},
"doctor": {
@@ -2102,7 +2356,7 @@
},
"issues": {
"civitai_api_key": {
"title": "Civitai API Key"
"title": "CivitAI API Key"
},
"cache_health": {
"title": "Model Cache Health"
@@ -2156,9 +2410,9 @@
},
"communitySupport": {
"title": "Keep LoRA Manager Thriving with Your Support ❤️",
"content": "LoRA Manager is a passion project maintained full-time by a solo developer. Your support on Ko-fi helps cover development costs, keeps new updates coming, and unlocks a license key for the LM Civitai Extension as a thank-you gift. Every contribution truly makes a difference.",
"content": "LoRA Manager is a passion project maintained full-time by a solo developer. Your support on Ko-fi helps cover development costs, keeps new updates coming, and unlocks a license key for the LM CivitAI Extension as a thank-you gift. Every contribution truly makes a difference.",
"supportCta": "Support on Ko-fi",
"learnMore": "LM Civitai Extension Tutorial"
"learnMore": "LM CivitAI Extension Tutorial"
},
"cacheHealth": {
"corrupted": {
+470 -216
View File
File diff suppressed because it is too large Load Diff
+472 -218
View File
File diff suppressed because it is too large Load Diff
+486 -232
View File
File diff suppressed because it is too large Load Diff
+424 -170
View File
File diff suppressed because it is too large Load Diff
+429 -175
View File
File diff suppressed because it is too large Load Diff
+444 -190
View File
File diff suppressed because it is too large Load Diff
+354 -100
View File
@@ -67,11 +67,11 @@
"steps": {
"fetch": {
"title": "获取模型元数据",
"content": "点击 <strong>获取</strong> 按钮,从 Civitai 下载模型元数据和预览图片。"
"content": "点击 <strong>获取</strong> 按钮,从 CivitAI 下载模型元数据和预览图片。"
},
"download": {
"title": "下载新模型",
"content": "使用 <strong>下载</strong> 按钮,可直接通过 Civitai URL 下载模型。"
"content": "使用 <strong>下载</strong> 按钮,可直接通过 CivitAI URL 下载模型。"
},
"bulk": {
"title": "批量操作",
@@ -103,12 +103,12 @@
"actions": {
"addToFavorites": "添加到收藏",
"removeFromFavorites": "从收藏移除",
"viewOnCivitai": "在 Civitai 查看",
"notAvailableFromCivitai": "Civitai 上不可用",
"viewOnCivitai": "在 CivitAI 查看",
"notAvailableFromCivitai": "CivitAI 上不可用",
"viewOnHuggingFace": "在 Hugging Face 查看",
"sendToWorkflow": "发送到 ComfyUI(点击:追加,Shift+点击:替换)",
"copyLoRASyntax": "复制 LoRA 语法",
"checkpointNameCopied": "检查点名称已复制",
"checkpointNameCopied": "Checkpoint 名称已复制",
"toggleBlur": "切换模糊",
"show": "显示",
"openExampleImages": "打开示例图片文件夹",
@@ -131,13 +131,13 @@
"updateFailed": "收藏状态更新失败"
},
"sendToWorkflow": {
"checkpointNotImplemented": "发送检查点到工作流 - 功能待实现",
"checkpointNotImplemented": "发送Checkpoint到工作流 - 功能待实现",
"missingPath": "无法确定此卡片的模型路径"
},
"exampleImages": {
"checkError": "检查示例图片时出错",
"missingHash": "缺少模型哈希信息。",
"noRemoteImagesAvailable": "此模型在 Civitai 上没有远程示例图片"
"noRemoteImagesAvailable": "此模型在 CivitAI 上没有远程示例图片"
},
"badges": {
"update": "更新",
@@ -186,6 +186,16 @@
"cancelled": "修复已取消。已修复 {count} 个配方。",
"error": "配方修复失败:{message}"
},
"rematchRecipes": {
"label": "将配方重新匹配到本地模型",
"loading": "正在将配方重新匹配到本地模型...",
"success": "已匹配 {entries} 个条目,涉及 {recipes} 个配方",
"successErrors": "已匹配 {entries} 个条目,涉及 {recipes} 个配方,{failures} 个失败",
"allFailed": "{failures}/{total} 个配方重新匹配失败",
"noMatch": "在 {recipes} 个配方中未找到 {entries} 个条目的本地匹配",
"cancelled": "已取消重新匹配。{recipes} 个配方已更新({entries} 个条目)。",
"error": "配方重新匹配失败:{message}"
},
"manageExcludedModels": {
"label": "管理已排除的模型"
},
@@ -212,6 +222,7 @@
"modelname": "模型名称",
"tags": "标签",
"creator": "创作者",
"hash": "哈希",
"title": "配方标题",
"loraName": "LoRA 文件名",
"loraModel": "LoRA 模型名称",
@@ -233,7 +244,7 @@
"presetNamePlaceholder": "预设名称...",
"baseModel": "基础模型",
"baseModelSearchPlaceholder": "搜索基础模型...",
"modelTags": "标签(前20",
"modelTags": "标签",
"modelTypes": "模型类型",
"license": "许可证",
"noCreditRequired": "无需署名",
@@ -241,13 +252,19 @@
"allowSellingGeneratedContentTooltip": "允许出售生成的图片",
"noCreditRequiredTooltip": "使用模型时无需注明原作者",
"noTags": "无标签",
"tagSearchPlaceholder": "搜索标签...",
"noTagMatches": "没有匹配当前搜索的标签。",
"autoTags": "自动标签",
"noBaseModelMatches": "没有基础模型符合当前搜索。",
"clearAll": "清除所有筛选",
"any": "任一",
"all": "全部",
"tagLogicAny": "匹配任一标签 (或)",
"tagLogicAll": "匹配所有标签 (与)"
"tagLogicAll": "匹配所有标签 (与)",
"loraAvailability": "LoRA 可用性",
"availabilityReady": "可直接使用",
"availabilityMissing": "包含缺失 LoRA",
"availabilityDeleted": "包含已删除 LoRA"
},
"theme": {
"toggle": "切换主题",
@@ -273,15 +290,15 @@
}
},
"settings": {
"civitaiApiKey": "Civitai API 密钥",
"civitaiApiKeyPlaceholder": "请输入你的 Civitai API 密钥",
"civitaiApiKeyHelp": "用于从 Civitai 下载模型时的身份验证",
"civitaiApiKey": "CivitAI API 密钥",
"civitaiApiKeyPlaceholder": "请输入你的 CivitAI API 密钥",
"civitaiApiKeyHelp": "用于从 CivitAI 下载模型时的身份验证",
"civitaiApiKeyConfigured": "已配置",
"civitaiApiKeyNotConfigured": "未配置",
"civitaiApiKeySet": "设置",
"civitaiHost": {
"label": "Civitai 站点",
"help": "选择使用“在 Civitai 中查看”时默认打开的 Civitai 站点。",
"label": "CivitAI 站点",
"help": "选择使用“在 CivitAI 中查看”时默认打开的 CivitAI 站点。",
"options": {
"com": "civitai.com(仅 SFW",
"red": "civitai.red(无限制)"
@@ -302,8 +319,8 @@
},
"aria2HelpLink": "了解如何配置 aria2 下载后端",
"civitaiHostBanner": {
"title": "已提供 Civitai 站点偏好设置",
"content": "Civitai 现在使用 civitai.com 提供 SFW 内容,使用 civitai.red 提供无限制内容。你可以在设置中更改默认打开的站点。",
"title": "已提供 CivitAI 站点偏好设置",
"content": "CivitAI 现在使用 civitai.com 提供 SFW 内容,使用 civitai.red 提供无限制内容。你可以在设置中更改默认打开的站点。",
"openSettings": "打开设置"
},
"openSettingsFileLocation": {
@@ -411,7 +428,7 @@
},
"downloadSkipBaseModels": {
"label": "跳过这些基础模型的下载",
"help": "适用于所有下载流程。这里只能选择受支持的基础模型。",
"help": "启用后,使用所选基础模型的版本将被跳过。",
"searchPlaceholder": "筛选基础模型...",
"empty": "没有与当前搜索匹配的基础模型。",
"summary": {
@@ -433,7 +450,7 @@
},
"layoutSettings": {
"groupByModel": "按模型分组",
"groupByModelHelp": "开启后,每个 Civitai 模型仅显示最新版本的单张卡片,旧版本将被隐藏。",
"groupByModelHelp": "开启后,每个 CivitAI 模型仅显示最新版本的单张卡片,旧版本将被隐藏。",
"displayDensity": "显示密度",
"displayDensityOptions": {
"default": "默认",
@@ -447,6 +464,12 @@
"compact": "71080p),82K),104K"
},
"displayDensityWarning": "警告:高密度可能导致资源有限的系统性能下降。",
"recipesLayout": "配方布局",
"recipesLayoutHelp": "选择配方卡片的排列方式:统一网格,或保留每张图片原始宽高比的瀑布流(Pinterest 风格)布局。",
"recipesLayoutOptions": {
"grid": "网格",
"masonry": "瀑布流"
},
"showFolderSidebar": "显示文件夹侧边栏",
"showFolderSidebarHelp": "在模型页面启用或禁用文件夹导航侧边栏。关闭后,侧边栏和悬停区域将保持隐藏。",
"cardInfoDisplay": "卡片信息显示",
@@ -505,7 +528,9 @@
"saveSuccess": "额外文件夹路径已更新,需要重启才能生效。",
"saveError": "更新额外文件夹路径失败:{message}",
"validation": {
"duplicatePath": "此路径已配置"
"duplicatePath": "此路径已配置",
"checkpointUnetOverlap": "checkpoints 和 diffusion models 不能使用相同的路径:{paths}",
"checkpointUnetOverlapInline": "此路径已被用于另一种模型类型。请为 checkpoints 和 diffusion models 使用不同的文件夹。"
}
},
"priorityTags": {
@@ -530,7 +555,7 @@
},
"downloadPathTemplates": {
"title": "下载路径模板",
"help": "配置从 Civitai 下载不同模型类型的文件夹结构。",
"help": "配置从 CivitAI 下载不同模型类型的文件夹结构。",
"availablePlaceholders": "可用占位符:",
"templateOptions": {
"flatStructure": "扁平结构",
@@ -567,7 +592,7 @@
"exampleImages": {
"downloadLocation": "下载位置",
"downloadLocationPlaceholder": "输入示例图片文件夹路径",
"downloadLocationHelp": "输入保存从 Civitai 下载的示例图片的文件夹路径",
"downloadLocationHelp": "输入保存从 CivitAI 下载的示例图片的文件夹路径",
"autoDownload": "自动下载示例图片",
"autoDownloadHelp": "自动为没有示例图片的模型下载示例图片(需设置下载位置)",
"openMode": "打开示例图片操作",
@@ -600,7 +625,11 @@
},
"hideEarlyAccessUpdates": {
"label": "隐藏抢先体验更新",
"help": "抢先体验更新"
"help": "启用后,仅有抢先体验更新的模型将不显示“可更新”徽章。"
},
"hidePaidUpdates": {
"label": "隐藏付费更新",
"help": "启用后,仅有付费更新的模型将不显示“有可用更新”徽标"
},
"licenseIcons": {
"useNewStyle": "使用新版许可协议图标",
@@ -618,7 +647,7 @@
},
"metadataArchive": {
"enableArchiveDb": "启用元数据归档数据库",
"enableArchiveDbHelp": "使用本地数据库访问已从 Civitai 删除的模型元数据。",
"enableArchiveDbHelp": "使用本地数据库访问已从 CivitAI 删除的模型元数据。",
"status": "状态",
"statusAvailable": "可用",
"statusUnavailable": "不可用",
@@ -638,7 +667,13 @@
"preparing": "正在准备下载...",
"connecting": "正在连接下载服务器...",
"completed": "已完成",
"downloadComplete": "下载成功完成"
"downloadComplete": "下载成功完成",
"enableCivarchiveApi": "启用 CivArchive API 作为元数据提供者",
"enableCivarchiveApiHelp": "开启后,CivArchive API 将作为模型元数据的备用来源(例如用于已从 CivitAI 删除的模型)。关闭可完全避免 CivArchive 的速率限制。",
"providerOrder": "元数据提供者回退顺序",
"providerOrderHelp": "CivitAI API 始终优先尝试。选择查找元数据时其余提供者的顺序。",
"providerOrderCivitaiArchiveSqlite": "CivitAI → CivArchive → Archive DB",
"providerOrderCivitaiSqliteArchive": "CivitAI → Archive DB → CivArchive"
},
"proxySettings": {
"enableProxy": "启用应用级代理",
@@ -657,6 +692,33 @@
"proxyPassword": "密码 (可选)",
"proxyPasswordPlaceholder": "密码",
"proxyPasswordHelp": "代理认证的密码 (如果需要)"
},
"aiProvider": {
"title": "AI 提供商",
"provider": "提供商",
"providerHelp": "选择你的 LLM 提供商。OpenAI 和 Ollama 使用预设的 API 端点。自定义允许你指定任何兼容 OpenAI 的端点。",
"providerOptions": {
"openai": "OpenAI",
"ollama": "Ollama(本地)",
"deepseek": "DeepSeek",
"groq": "Groq",
"openrouter": "OpenRouter",
"google": "Gemini",
"opencode-go": "OpenCode Go",
"custom": "自定义(OpenAI 兼容)"
},
"apiBase": "API 基础地址",
"apiBaseHelp": "LLM API 的基础地址。选择预设或输入自定义地址,下拉框显示所有支持的提供商预设。",
"apiBasePlaceholder": "https://api.openai.com/v1",
"apiKey": "API 密钥",
"apiKeyHelp": "LLM 提供商的 API 密钥。本地存储,除你选择的 LLM 提供商外不会发送到任何服务器。",
"apiKeyPlaceholder": "sk-...",
"apiKeyNotSet": "未设置",
"apiKeyConfigured": "已配置",
"apiKeySet": "设置",
"model": "模型",
"modelHelp": "要使用的模型。从下拉框选择(从提供商获取)或输入自定义模型名称。",
"modelPlaceholder": "选择一个模型..."
}
},
"loras": {
@@ -678,7 +740,9 @@
"versionsCount": "本地版本数",
"versionsCountDesc": "版本数从多到少",
"versionsCountAsc": "版本数从少到多",
"versionIdDesc": "最新版本优先"
"versionIdDesc": "最新版本优先",
"random": "随机",
"randomAction": "随机排序(洗牌)"
},
"refresh": {
"title": "刷新模型列表",
@@ -686,7 +750,7 @@
"fullTooltip": "从元数据文件重新加载所有模型信息;用于列表过时或手动编辑后。"
},
"fetch": {
"title": "从 Civitai 获取元数据",
"title": "从 CivitAI 获取元数据",
"action": "获取"
},
"download": {
@@ -723,6 +787,7 @@
"copyAll": "复制所选中语法",
"refreshAll": "刷新所选中元数据",
"repairMetadata": "修复所选中元数据",
"rematchMetadata": "将所选中重新匹配到本地模型",
"reimportMetadata": "从源重新导入",
"checkUpdates": "检查所选更新",
"moveAll": "移动所选中到文件夹",
@@ -735,6 +800,8 @@
"deleteAll": "删除已选",
"downloadMissingLoras": "下载缺失的 LoRAs",
"downloadExamples": "下载示例图片",
"downloadMissingExamples": "下载缺失的",
"reprocessExamples": "重新处理全部",
"clear": "清除选择",
"skipMetadataRefreshCount": "跳过({count} 个模型)",
"resumeMetadataRefreshCount": "恢复({count} 个模型)",
@@ -754,12 +821,15 @@
"completed": "完成:已移动 {success} 个,跳过 {skipped} 个,失败 {failures} 个",
"complete": "自动整理已完成",
"error": "错误:{error}"
}
},
"enrichHfAgent": "AI HF 元数据增强"
},
"contextMenu": {
"refreshMetadata": "刷新 Civitai 数据",
"refreshMetadata": "刷新 CivitAI 数据",
"checkUpdates": "检查更新",
"relinkCivitai": "重新关联到 Civitai",
"linkModel": "链接模型",
"linkCivitai": "链接到 CivitAI",
"linkHuggingFace": "链接到 HuggingFace",
"copySyntax": "复制 LoRA 语法",
"copyFilename": "复制模型文件名",
"copyRecipeSyntax": "复制配方语法",
@@ -767,10 +837,13 @@
"sendToWorkflowReplace": "发送到工作流(替换)",
"openExamples": "打开示例文件夹",
"downloadExamples": "下载示例图片",
"downloadMissingExamples": "下载缺失的",
"reprocessExamples": "重新处理全部",
"replacePreview": "替换预览",
"setContentRating": "设置内容评级",
"moveToFolder": "移动到文件夹",
"repairMetadata": "修复元数据",
"rematchMetadata": "重新匹配到本地模型",
"reimportMetadata": "从源重新导入",
"excludeModel": "排除模型",
"restoreModel": "恢复模型",
@@ -778,26 +851,72 @@
"shareRecipe": "分享配方",
"viewAllLoras": "查看所有 LoRA",
"downloadMissingLoras": "下载缺失的 LoRA",
"deleteRecipe": "删除配方"
"deleteRecipe": "删除配方",
"enrichHfAgent": "AI HF 元数据增强"
}
},
"recipes": {
"title": "LoRA 配方",
"actions": {
"sendCheckpoint": "发送到 ComfyUI"
"sendCheckpoint": "发送到 ComfyUI",
"sendRecipe": "发送到 ComfyUI",
"copyRecipeSyntax": "复制配方语法",
"deleteRecipeWithShortcut": "删除配方(Del"
},
"navigation": {
"label": "配方导航",
"previousWithShortcut": "上一个配方(←)",
"nextWithShortcut": "下一个配方(→)"
},
"workflow": {
"sendWorkflow": "发送工作流到 ComfyUI",
"sent": "工作流已发送到 ComfyUI",
"sendFailed": "发送工作流到 ComfyUI 失败",
"noWorkflow": "此配方中未找到内嵌工作流"
},
"status": {
"ready": "可直接使用",
"missingCount": "缺失 {count} 个",
"deletedCount": "已删除 {count} 个",
"downloadMissing": "下载 {count} 个缺失的 LoRA",
"downloadMissingTooltip": "点击下载缺失的 LoRA"
},
"loraStatus": {
"none": "此配方不包含 LoRA",
"allAvailable": "所有 LoRA 均已就绪 - 可直接使用",
"missing": "{total} 个 LoRA 中缺失 {missing} 个",
"missingAndUnavailable": "{total} 个 LoRA 中缺失 {missing} 个,{unavailable} 个不可用(已从源站删除或哈希无法解析)",
"partial": "{total} 个 LoRA 中 {unavailable} 个不可用(已从源站删除或哈希无法解析)- 使用配方时将被跳过",
"noneUsable": "没有可用的 LoRA - {total} 个中 {unavailable} 个已从源站删除或哈希无法解析"
},
"resources": {
"inLibrary": "在库中",
"notInLibrary": "不在库中",
"deleted": "已删除",
"hashInvalid": "无法解析的哈希",
"inLibraryTooltip": "该模型已存在于本地库中",
"notInLibraryTooltip": "该模型不在你的本地库中",
"deletedTooltip": "该 LoRA 已从来源站删除,无法下载",
"hashInvalidTooltip": "此 LoRA 哈希无法在 CivitAI 上解析——模型可能已更新",
"download": "下载",
"downloadLoraTooltip": "下载此 LoRA",
"preparingDownload": "正在准备下载...",
"reconnect": "重新关联",
"reconnectTooltip": "与本地 LoRA 重新关联",
"viewOnCivitai": "在 CivitAI 上查看",
"openLoraDetails": "在 LoRA 库中查看 {name}",
"openCheckpointDetails": "在模型库中查看 {name}"
},
"controls": {
"import": {
"action": "导入",
"title": "从图片或 URL 导入配方",
"urlLocalPath": "URL / 本地路径",
"uploadImage": "上传图片",
"urlSectionDescription": "输入来自 civitai.com 或 civitai.red 的 Civitai 图片 URL,或本地文件路径以导入为配方。",
"dropZoneLabel": "上传图片",
"dropZoneHint": "将图片拖拽到此处、从剪贴板粘贴,或点击浏览",
"orDivider": "或拖拽 / 粘贴图片",
"imageUrlOrPath": "图片 URL 或文件路径:",
"urlPlaceholder": "https://civitai.com/images/... 或 https://civitai.red/images/... 或 C:/path/to/image.png",
"fetchImage": "获取图片",
"uploadSectionDescription": "上传带有 LoRA 元数据的图片以导入为配方。",
"selectImage": "选择图片",
"recipeName": "配方名称",
"recipeNamePlaceholder": "输入配方名称",
"tagsOptional": "标签(可选)",
@@ -805,7 +924,7 @@
"addTag": "添加",
"noTagsAdded": "未添加标签",
"lorasInRecipe": "此配方中的 LoRA",
"downloadLocationPreview": "下载位置预览:{path}",
"downloadLocationPreview": "下载位置预览:",
"useDefaultPath": "使用默认路径",
"useDefaultPathTooltip": "启用后,文件将自动使用配置的路径模板进行组织",
"selectLoraRoot": "选择 LoRA 根目录",
@@ -819,20 +938,20 @@
"importAndDownload": "导入并下载",
"downloadMissingLoras": "下载缺失的 LoRA",
"saveRecipe": "保存配方",
"loraCountInfo": "({existing}/{total} in library)",
"loraCountInfo": "(库中 {existing}/{total}",
"processingInput": "处理输入...",
"analyzingMetadata": "分析图像元数据...",
"downloadingLoras": "下载 LoRA...",
"savingRecipe": "保存配方...",
"startingDownload": "开始下载 LoRA {current}/{total}",
"deletedFromCivitai": "从 Civitai 中删除",
"deletedFromCivitai": "从 CivitAI 中删除",
"inLibrary": "在库中",
"notInLibrary": "不在库中",
"earlyAccessRequired": "此 LoRA 需要提前访问权限才能下载。",
"earlyAccessEnds": "提前访问权限将于 {date} 结束。",
"earlyAccess": "提前访问",
"verifyEarlyAccess": "在下载之前,请验证您是否已购买提前访问权限。",
"duplicateRecipesFound": "在的库中找到 {count} 个相同的配方。",
"verifyEarlyAccess": "在下载之前,请确认你已购买提前访问权限。",
"duplicateRecipesFound": "在的库中找到 {count} 个相同的配方。",
"duplicateRecipesDescription": "这些配方包含相同的 LoRA,权重完全相同。",
"showDuplicates": "显示重复项",
"hideDuplicates": "隐藏重复项",
@@ -842,6 +961,8 @@
"errors": {
"selectImageFile": "请选择一个图像文件",
"enterUrlOrPath": "请输入 URL 或文件路径",
"invalidUrl": "请输入有效的 URL",
"invalidInputFormat": "请输入图片 URL 或本地图片文件路径",
"selectLoraRoot": "请选择 LoRA 根目录"
}
},
@@ -855,7 +976,9 @@
"dateAsc": "最早",
"lorasCount": "LoRA 数量",
"lorasCountDesc": "最多",
"lorasCountAsc": "最少"
"lorasCountAsc": "最少",
"opened": "最近打开",
"openedDesc": "最近打开"
},
"refresh": {
"title": "刷新配方列表",
@@ -866,12 +989,26 @@
"favorites": {
"title": "仅显示收藏",
"action": "收藏"
},
"layout": {
"title": "配方布局",
"grid": "网格布局",
"masonry": "瀑布流布局(Pinterest 风格,保留图片原始宽高比)"
}
},
"duplicates": {
"finding": "正在扫描重复配方...",
"found": "发现 {count} 个重复组",
"noGroups": "按当前判重依据未找到重复组",
"keepLatest": "保留最新版本",
"deleteSelected": "删除已选"
"deleteSelected": "删除已选",
"includePromptLabel": "将提示词纳入判重",
"basis": {
"loraCombo": "判重依据:LoRA 组合",
"loraComboAndPrompt": "判重依据:LoRA 组合 + 提示词",
"hintLoraCombo": "使用相同 LoRA(强度一致)的配方会被分组。",
"hintPromptIncluded": "仅当配方使用相同的 LoRA(强度一致)且提示词相同时才会被分组。"
}
},
"contextMenu": {
"copyRecipe": {
@@ -930,6 +1067,8 @@
"start": "开始导入",
"startImport": "开始导入",
"importing": "正在导入配方...",
"rateLimitedSlowdown": "触发速率限制 — 正在减速...",
"rateLimitedHint": "部分条目因元数据提供方的速率限制而被跳过。稍后重新运行导入即可重试这些条目。",
"progress": "进度",
"total": "总计",
"success": "成功",
@@ -971,7 +1110,7 @@
"title": "Checkpoint 模型",
"modelTypes": {
"checkpoint": "Checkpoint",
"diffusion_model": "Diffusion Model"
"diffusion_model": "扩散模型"
},
"contextMenu": {
"moveToOtherTypeFolder": "移动到 {otherType} 文件夹",
@@ -995,7 +1134,7 @@
"collapseAllDisabled": "列表视图下不可用",
"dragDrop": {
"unableToResolveRoot": "无法确定移动的目标路径。",
"moveUnsupported": "Move is not supported for this item.",
"moveUnsupported": "此条目不支持移动。",
"createFolderHint": "释放以创建新文件夹",
"newFolderName": "新文件夹名称",
"folderNameHint": "按 Enter 确认,Escape 取消",
@@ -1133,7 +1272,7 @@
"download": {
"title": "从 URL 下载模型",
"titleWithType": "从 URL 下载 {type}",
"civitaiUrl": "Civitai URL:",
"civitaiUrl": "CivitAI URL:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "每行输入一个 CivitAI、CivArchive 或 Hugging Face URL。支持批量下载多个 URL。",
"selectHfFiles": "选择从此仓库下载的文件:",
@@ -1159,14 +1298,16 @@
"downloaded": "已下载",
"downloadedTooltip": "之前已下载,但当前不在你的库中。",
"alreadyInLibrary": "已存在于库中",
"partiallyDownloaded": "部分已下载",
"autoOrganizedPath": "【已按路径模板自动整理】",
"fileSelection": {
"title": "选择文件格式",
"files": "个文件",
"select": "选择文件"
"select": "选择文件",
"inLibrary": "已在库中"
},
"errors": {
"invalidUrl": "无效的 Civitai URL 格式",
"invalidUrl": "无效的 CivitAI URL 格式",
"noVersions": "此模型没有可用版本",
"mixedSources": "无法在同一批次中混合使用 CivitAI 和 Hugging Face URL。",
"noModelFiles": "在此仓库中未找到模型文件。"
@@ -1175,7 +1316,9 @@
"preparing": "正在准备下载...",
"downloadedPreview": "预览图片已下载",
"downloadingFile": "正在下载 {type} 文件",
"finalizing": "正在完成下载..."
"finalizing": "正在完成下载...",
"cancelling": "取消下载中...",
"cancelled": "下载已取消"
},
"progress": {
"currentFile": "当前文件:",
@@ -1202,8 +1345,13 @@
}
},
"deleteModel": {
"freesSpace": "释放 {size}",
"title": "删除模型",
"message": "你确定要删除此模型及所有相关文件吗?"
"message": "你确定要删除此模型及所有相关文件吗?",
"recoverableWarning": "如果不撤销,文件将在 20 秒后被永久删除。"
},
"deleteRecipe": {
"recoverableWarning": "此操作可在 20 秒内撤销。"
},
"excludeModel": {
"title": "排除模型",
@@ -1238,8 +1386,8 @@
"action": "全部删除"
},
"checkUpdates": {
"title": "检查所有 {type} 的更新?",
"message": "这会库中的每个 {type} 检查更新,大型集合可能需要一些时间。",
"title": "检查所有 {typePlural} 的更新?",
"message": "这会检查库中的每个 {typePlural} 的更新,大型集合可能需要一些时间。",
"tip": "想分批进行?切换到批量模式,选中需要的模型,然后使用“检查所选更新”。",
"action": "检查全部"
},
@@ -1273,7 +1421,7 @@
"title": "本地示例图片",
"message": "未找到此模型的本地示例图片。可选操作:",
"downloadOption": {
"title": "从 Civitai 下载",
"title": "从 CivitAI 下载",
"description": "将远程示例保存到本地,便于离线使用和更快加载"
},
"importOption": {
@@ -1291,8 +1439,16 @@
"pathPlaceholder": "输入文件夹路径或从下方树中选择...",
"root": "根目录"
},
"linkHuggingFace": {
"title": "链接到 HuggingFace",
"infoText": "粘贴 HuggingFace 仓库 URL 以关联此模型。关联后可启用 AI 元数据增强功能。",
"urlLabel": "HuggingFace 仓库 URL",
"urlPlaceholder": "https://huggingface.co/user/repo",
"helpText": "请输入完整的 HuggingFace 仓库 URL。",
"confirmAction": "保存并链接"
},
"relinkCivitai": {
"title": "重新关联到 Civitai",
"title": "重新关联到 CivitAI",
"warning": "警告:",
"warningText": "这是一个有潜在风险的操作。重新关联将:",
"warningList": {
@@ -1301,14 +1457,15 @@
"unintendedConsequences": "可能有其他不可预期的后果"
},
"proceedText": "仅在你确定需要此操作时继续。",
"urlLabel": "Civitai 模型 URL",
"urlPlaceholder": "https://civitai.com/models/649516/model-name?modelVersionId=726676 或 https://civitai.red/models/649516/model-name?modelVersionId=726676",
"urlLabel": "CivitAI 模型 URL",
"urlPlaceholder": "https://civitai.com/models/12345/model-name?modelVersionId=67890 或 https://civitai.red/models/12345/model-name?modelVersionId=67890",
"helpText": {
"title": "粘贴任意来自 civitai.comcivitai.red 的 Civitai 模型 URL。支持格式:",
"format1": "https://civitai.com/models/649516",
"format2": "https://civitai.com/models/649516?modelVersionId=726676",
"format3": "https://civitai.com/models/649516/model-name?modelVersionId=726676",
"note": "注意:如果未提供 modelVersionId,将使用最新版本。"
"title": "粘贴任意 CivitAICivitArchive 模型 URL。支持格式:",
"format1": "https://civitai.com/models/12345",
"format2": "https://civitai.com/models/12345?modelVersionId=67890",
"format3": "https://civitai.com/models/12345/model-name?modelVersionId=67890",
"note": "注意:如果未提供 modelVersionId,将使用最新版本。",
"format4": "https://civarchive.com/models/12345 (CivArchive)"
},
"confirmAction": "确认重新关联"
},
@@ -1318,14 +1475,16 @@
"editFileName": "编辑文件名",
"editBaseModel": "编辑基础模型",
"editVersionName": "编辑版本名称",
"viewOnCivitai": "在 Civitai 查看",
"viewOnCivitaiText": "在 Civitai 查看",
"viewOnCivitai": "在 CivitAI 查看",
"viewOnCivitaiText": "在 CivitAI 查看",
"viewOnHuggingFace": "在 Hugging Face 查看",
"viewOnHuggingFaceText": "在 Hugging Face 查看",
"viewCreatorProfile": "查看创作者主页",
"openFileLocation": "打开文件位置",
"sendToWorkflow": "发送到 ComfyUI",
"sendToWorkflowText": "发送到 ComfyUI"
"sendToWorkflowText": "发送到 ComfyUI",
"copyHash": "复制哈希值",
"deleteModelWithShortcut": "删除模型(Del"
},
"openFileLocation": {
"success": "文件位置已成功打开",
@@ -1342,13 +1501,14 @@
"location": "位置",
"baseModel": "基础模型",
"size": "大小",
"hashes": "哈希值",
"unknown": "未知",
"usageTips": "使用提示",
"additionalNotes": "附加备注",
"notesHint": "回车保存,Shift+回车换行",
"addNotesPlaceholder": "在此添加你的备注...",
"aboutThisVersion": "关于此版本",
"baseModelSearchPlaceholder": "搜索基础模型",
"baseModelSearchPlaceholder": "搜索基础模型...",
"baseModelSuggested": "推荐",
"baseModelNoMatch": "没有匹配的基础模型"
},
@@ -1417,10 +1577,10 @@
"noNext": "没有下一个模型"
},
"license": {
"noImageSell": "No selling generated content",
"noRentCivit": "No Civitai generation",
"noRent": "No generation services",
"noSell": "No selling models",
"noImageSell": "禁止出售生成的图片",
"noRentCivit": "禁止在 CivitAI 上生成",
"noRent": "禁止生成服务",
"noSell": "禁止出售模型",
"creditRequired": "需要创作者署名",
"noDerivatives": "禁止分享合并作品",
"noReLicense": "需要相同权限",
@@ -1433,6 +1593,30 @@
"examples": "正在加载示例...",
"versions": "正在加载版本..."
},
"showcase": {
"hiddenBySfw": "{count} 张因仅显示 SFW 设置而被隐藏",
"showExamples": "显示示例",
"showCount": "显示示例({count}",
"hideExamples": "隐藏示例",
"addExamples": "添加示例",
"previousExample": "上一个示例",
"nextExample": "下一个示例",
"noExamples": "暂无示例图片",
"addMoreExamples": "添加更多示例",
"dragDrop": "将图片或视频拖放到此处",
"or": "或",
"selectFiles": "选择文件",
"supportedFormats": "支持的格式:jpg, png, gif, webp, avif, jxl, mp4, webm",
"importing": "正在导入文件...",
"noSupportedFiles": "未选择受支持的文件。请选择图片或视频文件。",
"allFiltered": "所有示例图片均因 NSFW 内容设置而被过滤",
"sfwOnlyEnabled": "你当前的设置为仅显示 SFW 内容",
"changeInSettings": "你可以在设置中更改此选项",
"nsfwMature": "成熟内容",
"nsfwR": "R 级内容",
"nsfwX": "X 级内容",
"nsfwXxx": "XXX 级内容"
},
"versions": {
"heading": "模型版本",
"copy": "在一个位置管理该模型的所有版本。",
@@ -1459,45 +1643,50 @@
"newer": "较新的版本",
"newerTooltip": "此版本比你本地的最新版本更新",
"earlyAccess": "抢先体验",
"earlyAccessTooltip": "此版本当前需要 Civitai 抢先体验权限",
"earlyAccessTooltip": "此版本当前需要 CivitAI 抢先体验权限",
"paid": "付费",
"paidTooltip": "此版本需要付费后才能下载",
"ignored": "已忽略",
"ignoredTooltip": "此版本已关闭更新通知",
"onSiteOnly": "仅站内生成",
"onSiteOnlyTooltip": "此版本仅在 Civitai 站内可用,无法下载"
"onSiteOnlyTooltip": "此版本仅在 CivitAI 站内可用,无法下载"
},
"actions": {
"download": "下载",
"downloadTooltip": "下载此版本",
"downloadEarlyAccessTooltip": "从 Civitai 下载此抢先体验版本",
"downloadNotAllowedTooltip": "此版本仅在 Civitai 站内可用,无法下载",
"downloadChooseFilesTooltip": "选择要下载的文件",
"downloadEarlyAccessTooltip": " CivitAI 下载此抢先体验版本",
"downloadPaidTooltip": "从 CivitAI 下载此付费版本",
"downloadNotAllowedTooltip": "此版本仅在 CivitAI 站内可用,无法下载",
"delete": "删除",
"deleteTooltip": "删除此本地版本",
"ignore": "忽略",
"unignore": "取消忽略",
"ignoreTooltip": "忽略此版本的更新通知",
"unignoreTooltip": "恢复此版本的更新通知",
"viewVersionOnCivitai": "在 Civitai 上查看版本",
"viewVersionOnCivitai": "在 CivitAI 上查看版本",
"earlyAccessTooltip": "需要购买抢先体验",
"resumeModelUpdates": "继续跟踪该模型的更新",
"ignoreModelUpdates": "忽略该模型的更新",
"viewLocalVersions": "查看所有本地版本",
"viewLocalTooltip": "敬请期待"
"viewLocalTooltip": "在主页面上显示该模型的所有本地版本"
},
"filters": {
"label": "基础筛选",
"state": {
"showAll": "全部版本",
"showSameBase": "相同基模型"
"showSameBase": "相同基模型"
},
"tooltip": {
"showAllVersions": "切换为显示所有版本",
"showSameBaseVersions": "仅显示与当前基模型匹配的版本"
"showSameBaseVersions": "仅显示与当前基模型匹配的版本"
},
"empty": "没有与当前基模型筛选匹配的版本。"
"empty": "没有与当前基模型筛选匹配的版本。"
},
"empty": "该模型还没有版本历史。",
"error": "加载版本失败。",
"missingModelId": "该模型缺少 Civitai 模型 ID。",
"missingModelId": "该模型缺少 CivitAI 模型 ID。",
"hfGroupInfo": "这是一个 HuggingFace 模型组。打开库页面即可在网格中查看所有版本。",
"confirm": {
"delete": "从库中删除此版本?"
},
@@ -1524,6 +1713,21 @@
"downloadCsv": "下载 CSV",
"columnModelName": "模型名称",
"columnError": "错误"
},
"downloadBatchSummary": {
"title": "批量下载摘要",
"statSuccess": "成功",
"statFailed": "失败",
"statTotal": "总数",
"successMessage": "全部 {count} 个模型下载成功",
"completedWithErrors": "已完成,但有错误",
"failed": "下载失败",
"failedItems": "失败项({count}",
"columnName": "模型名称",
"columnError": "错误",
"close": "关闭",
"copyReport": "复制报告",
"retryFailed": "重试失败项({count}"
}
},
"modelTags": {
@@ -1565,14 +1769,14 @@
"tips": {
"title": "技巧与提示",
"civitai": {
"title": "Civitai 集成",
"description": "连接你的 Civitai 账号:访问头像 → 设置 → API 密钥 → 添加密钥,然后粘贴到 LoRA 管理器设置中。",
"alt": "Civitai API 设置"
"title": "CivitAI 集成",
"description": "连接你的 CivitAI 账号:访问头像 → 设置 → API 密钥 → 添加密钥,然后粘贴到 LoRA 管理器设置中。",
"alt": "CivitAI API 设置"
},
"download": {
"title": "便捷下载",
"description": "使用 Civitai URL 快速下载和安装新模型。",
"alt": "Civitai 下载"
"description": "使用 CivitAI URL 快速下载和安装新模型。",
"alt": "CivitAI 下载"
},
"recipes": {
"title": "保存配方",
@@ -1622,6 +1826,7 @@
"recipeReplaced": "配方已替换到工作流",
"recipeFailedToSend": "发送配方到工作流失败",
"noMatchingNodes": "当前工作流中没有兼容的节点",
"noPromptTargets": "工作流中没有兼容的 prompt 目标节点。\n在 ComfyUI 中右键节点 → Mark as → Send Prompt Target",
"noTargetNodeSelected": "未选择目标节点",
"modelUpdated": "模型已更新到工作流",
"modelFailed": "更新模型节点失败",
@@ -1649,7 +1854,7 @@
"copiedUri": "链接已复制到剪贴板:{{uri}}",
"uriClipboardFallback": "链接:{{uri}}",
"setupRequired": "示例图片存储",
"setupDescription": "要添加自定义示例图片,需要先设置下载位置。",
"setupDescription": "要添加自定义示例图片,需要先设置下载位置。",
"setupUsage": "此路径用于存储下载的示例图片和自定义图片。",
"openSettings": "打开设置"
}
@@ -1701,6 +1906,12 @@
"checkingMessage": "请稍候,正在检查最新版本。",
"showNotifications": "显示更新通知",
"latestBadge": "最新",
"latestMain": "Main 分支",
"channel": "更新频道",
"channels": {
"release": "稳定版",
"nightly": "Nightly"
},
"updateProgress": {
"preparing": "正在准备更新...",
"installing": "正在安装更新...",
@@ -1721,6 +1932,15 @@
"warning": "警告:Nightly 版本可能包含实验性功能,可能不稳定。",
"enable": "启用 Nightly 更新"
},
"channelSwitch": {
"nightlyTitle": "切换到 Nightly",
"nightlyMessage": "切换到 Nightly 将初始化 Git 仓库并跟踪 main 分支的最新提交。更新更频繁但可能不稳定,可随时切回稳定版。",
"releaseTitle": "切换到稳定版",
"releaseMessage": "切换到稳定版将检出最新的发布标签。可随时切换回每日构建版。",
"switching": "正在切换到 {channel} 频道...",
"completed": "已切换到 {channel} 频道",
"failed": "切换频道失败"
},
"banners": {
"recent": "最近的通知",
"empty": "暂无最近的横幅通知。",
@@ -1740,7 +1960,7 @@
"submitGithubIssue": "提交 GitHub 问题",
"joinDiscord": "加入 Discord",
"youtubeChannel": "YouTube 频道",
"civitaiProfile": "Civitai 个人资料",
"civitaiProfile": "CivitAI 个人资料",
"supportKofi": "支持 Ko-fi",
"supportPatreon": "支持 Patreon"
},
@@ -1783,6 +2003,7 @@
"downloadPartialSuccess": "已下载 {completed}/{total} 个 LoRA",
"downloadPartialWithAccess": "已下载 {completed}/{total} 个 LoRA。{accessFailures} 个因访问限制失败。请检查设置中的 API 密钥或早期访问状态。",
"pleaseSelectVersion": "请选择版本",
"pleaseSelectFile": "请至少选择一个文件",
"versionExists": "该版本已存在于你的库中",
"downloadCompleted": "下载成功完成",
"downloadSkippedByBaseModel": "由于基础模型 {baseModel} 已被排除,已跳过下载",
@@ -1816,6 +2037,8 @@
"createMissingData": "缺少创建配方所需的数据",
"created": "配方创建成功",
"noMissingLoras": "没有缺失的 LoRA 可下载",
"noPreviousRecipe": "没有上一个配方",
"noNextRecipe": "没有下一个配方",
"missingLorasInfoFailed": "获取缺失 LoRA 信息失败",
"preparingForDownloadFailed": "准备下载 LoRA 时出错",
"enterLoraName": "请输入 LoRA 名称或语法",
@@ -1825,9 +2048,12 @@
"cannotSend": "无法发送配方:缺少配方 ID",
"sendFailed": "发送配方到工作流失败",
"sendError": "发送配方到工作流出错",
"missingCheckpointPath": "缺少检查点路径",
"missingCheckpointInfo": "缺少检查点信息",
"downloadCheckpointFailed": "下载检查点失败:{message}",
"missingCheckpointPath": "缺少Checkpoint路径",
"missingCheckpointInfo": "缺少Checkpoint信息",
"downloadCheckpointFailed": "下载Checkpoint失败:{message}",
"missingLoraDownloadInfo": "缺少此 LoRA 的下载信息",
"hashNotFoundOnCivitai": "此 LoRA 哈希无法在 CivitAI 上解析——模型可能已更新或哈希无效",
"downloadLoraFailed": "下载 LoRA 失败:{message}",
"cannotDelete": "无法删除配方:缺少配方 ID",
"deleteConfirmationError": "显示删除确认出错",
"deletedSuccessfully": "配方删除成功",
@@ -1851,18 +2077,28 @@
"batchImportCancelFailed": "取消批量导入失败:{message}",
"batchImportNoUrls": "请输入至少一个 URL 或文件路径",
"batchImportNoDirectory": "请输入目录路径",
"batchImportRateLimited": "已达到元数据提供方的速率限制 — 请求正在放缓,部分条目可能被跳过。你可以稍后重新运行导入。",
"batchImportBrowseFailed": "浏览目录失败:{message}",
"batchImportDirectorySelected": "已选择目录:{path}",
"noRecipesSelected": "未选择任何配方",
"repairBulkComplete": "修复完成:{repaired} 个已修复,{skipped} 个已跳过(共 {total} 个)",
"repairBulkSkipped": "所选 {total} 个配方无需修复",
"repairBulkFailed": "修复所选配方失败:{message}",
"rematchComplete": "已匹配 {entries} 个条目,涉及 {recipes} 个配方",
"rematchCompleteErrors": "已匹配 {entries} 个条目,涉及 {recipes} 个配方,{failures} 个失败",
"rematchAllFailed": "{failures}/{total} 个所选配方重新匹配失败",
"rematchUnmatched": "在 {recipes} 个配方中未找到 {entries} 个条目的本地匹配",
"rematchSkipped": "{total} 个所选配方均无需重新匹配",
"rematchFailed": "重新匹配所选配方失败:{message}",
"reimporting": "正在从源重新导入配方...",
"reimportSuccess": "配方已从源重新导入成功",
"reimportBulkComplete": "重新导入完成:{completed} 个已导入,{failed} 个失败(共 {total} 个)",
"reimportBulkFailed": "重新导入某些配方失败",
"noMissingLorasInSelection": "在选定的配方中未找到缺失的 LoRAs",
"noLoraRootConfigured": "未配置 LoRA 根目录。请在设置中设置默认的 LoRA 根目录。"
"noLoraRootConfigured": "未配置 LoRA 根目录。请在设置中设置默认的 LoRA 根目录。",
"workflowSent": "工作流已发送到 ComfyUI",
"workflowSendFailed": "发送工作流到 ComfyUI 失败: {error}",
"workflowNoWorkflow": "此配方中未找到内嵌工作流"
},
"models": {
"noModelsSelected": "未选中模型",
@@ -1899,8 +2135,8 @@
"bulkUpdatesChecking": "正在检查所选 {type} 的更新...",
"bulkUpdatesSuccess": "{count} 个所选 {type} 有可用更新",
"bulkUpdatesNone": "所选 {type} 未发现更新",
"bulkUpdatesMissing": "所选 {type} 未关联 Civitai 更新",
"bulkUpdatesPartialMissing": "已跳过 {missing} 个未关联 Civitai 的所选 {type}",
"bulkUpdatesMissing": "所选 {type} 未关联 CivitAI 更新",
"bulkUpdatesPartialMissing": "已跳过 {missing} 个未关联 CivitAI 的所选 {type}",
"bulkUpdatesFailed": "检查所选 {type} 的更新失败:{message}",
"invalidCharactersRemoved": "文件名中的无效字符已移除",
"filenameCannotBeEmpty": "文件名不能为空",
@@ -1929,7 +2165,7 @@
"checkpointRootsFailed": "加载 Checkpoint 根目录失败:{message}",
"unetRootsFailed": "加载 Diffusion Model 根目录失败:{message}",
"embeddingRootsFailed": "加载 Embedding 根目录失败:{message}",
"mappingsUpdated": "基础模型路径映射已更新({count} 条映射{plural}",
"mappingsUpdated": "基础模型路径映射已更新({count} 条映射)",
"mappingsCleared": "基础模型路径映射已清除",
"mappingSaveFailed": "保存基础模型映射失败:{message}",
"downloadTemplatesUpdated": "下载路径模板已更新",
@@ -1940,8 +2176,8 @@
"compactModeToggled": "紧凑模式 {state}",
"settingSaveFailed": "保存设置失败:{message}",
"displayDensitySet": "显示密度已设置为 {density}",
"libraryLoadFailed": "Failed to load libraries: {message}",
"libraryActivateFailed": "Failed to activate library: {message}",
"libraryLoadFailed": "加载模型库失败:{message}",
"libraryActivateFailed": "激活模型库失败:{message}",
"languageChangeFailed": "切换语言失败:{message}",
"cacheCleared": "缓存文件已成功清除。下次操作将重建缓存。",
"cacheClearFailed": "清除缓存失败:{error}",
@@ -1965,7 +2201,6 @@
"presetNameTooLong": "预设名称不能超过 {max} 个字符",
"presetNameInvalidChars": "预设名称包含无效字符",
"presetNameExists": "已存在同名预设",
"maxPresetsReached": "最多允许 {max} 个预设。删除一个以添加更多。",
"presetNotFound": "预设未找到",
"invalidPreset": "无效的预设数据",
"deletePresetFailed": "删除预设失败",
@@ -1975,7 +2210,8 @@
"imagesCompleted": "示例图片{action}完成",
"imagesFailed": "示例图片{action}失败",
"loadError": "加载下载项出错:{message}",
"downloadError": "下载错误:{message}"
"downloadError": "下载错误:{message}",
"downloadStopped": "下载已取消"
},
"import": {
"folderTreeFailed": "加载文件夹树失败",
@@ -1993,6 +2229,14 @@
"updateFailed": "触发词更新失败",
"copyFailed": "复制失败"
},
"undo": {
"action": "撤销",
"deleted": "已删除 {name}",
"deletedBulk": "已删除 {count} 个项目",
"expired": "撤销窗口已过期,项目已被永久删除。",
"failed": "撤销失败:{error}",
"restored": "项目已恢复"
},
"virtual": {
"loadFailed": "加载项目失败",
"loadMoreFailed": "加载更多项目失败",
@@ -2018,8 +2262,11 @@
"contextMenu": {
"contentRatingSet": "内容评级已设置为 {level}",
"contentRatingFailed": "设置内容评级失败:{message}",
"relinkSuccess": "模型已成功重新关联到 Civitai",
"relinkSuccess": "模型已成功重新关联到 CivitAI",
"relinkFailed": "错误:{message}",
"linkHfSuccess": "模型已成功链接到 HuggingFace",
"linkHfFailed": "错误:{message}",
"linkCivArchSuccess": "模型已成功通过 CivitArchive 重新关联",
"fetchMetadataFirst": "请先从 CivitAI 获取元数据",
"noCivitaiInfo": "无 CivitAI 信息",
"missingHash": "模型哈希不可用"
@@ -2054,6 +2301,7 @@
"fileRenameFailed": "重命名文件失败:{error}",
"previewUpdated": "预览图片更新成功",
"previewUploadFailed": "上传预览图片失败",
"previewDropInvalid": "不支持的文件类型:{name}。请拖入图片或 MP4 视频。",
"refreshComplete": "{action} 完成",
"refreshFailed": "{action} {type} 失败",
"metadataRefreshed": "元数据刷新成功",
@@ -2078,9 +2326,15 @@
"bulkMoveSuccess": "成功移动 {successCount} 个 {type}",
"exampleImagesDownloadSuccess": "示例图片下载成功!",
"exampleImagesDownloadFailed": "示例图片下载失败:{message}",
"moveFailed": "Failed to move item: {message}",
"moveFailed": "移动条目失败:{message}",
"copiedToClipboard": "已复制到剪贴板",
"downloadStarted": "下载已开始"
},
"agent": {
"llmNotConfigured": "AI 提供商未配置。请在 设置 → AI 提供商 中进行配置。",
"enrichStarted": "正在使用 AI 增强元数据...",
"enrichComplete": "元数据增强完成:{{summary}}",
"enrichFailed": "元数据增强失败:{{error}}"
}
},
"doctor": {
@@ -2102,7 +2356,7 @@
},
"issues": {
"civitai_api_key": {
"title": "Civitai API 密钥"
"title": "CivitAI API 密钥"
},
"cache_health": {
"title": "模型缓存健康状态"
+368 -114
View File
File diff suppressed because it is too large Load Diff
+79 -14
View File
@@ -1,13 +1,19 @@
# pyright: reportImportCycles=false
# Lazy (function-local) imports still count as static edges in basedpyright's
# reportImportCycles, so the ServiceRegistry singleton pattern necessarily forms
# import cycles. Breaking them would require an architectural refactor.
import os
import platform
import posixpath
import threading
from pathlib import Path
import folder_paths # type: ignore
import folder_paths # pyright: ignore[reportMissingImports]
from typing import Any, Dict, Iterable, List, Mapping, Optional, Set, Tuple
import logging
import json
import urllib.parse
import sys as _sys
import types as _types
import time
from .utils.cache_paths import CacheType, get_cache_file_path, get_legacy_cache_paths
@@ -88,7 +94,7 @@ def _resolve_valid_default_root(
def _normalize_folder_paths_for_comparison(
folder_paths: Mapping[str, Iterable[str]],
folder_paths: Mapping[str, Any],
) -> Dict[str, Set[str]]:
"""Normalize folder paths for comparison across libraries."""
@@ -175,7 +181,6 @@ class Config:
# Load extra folder paths from active library settings before symlink scan
# so both primary and extra paths are discovered in a single pass.
if not standalone_mode:
self._load_extra_paths_from_settings()
# Scan symbolic links during initialization
@@ -191,7 +196,7 @@ class Config:
Called during ``Config.__init__`` before the symlink scan so both primary and
extra paths are discovered in a single pass. Mirrors the extra-path
portion of ``_apply_library_paths`` without replacing the primary roots
that were already resolved from ComfyUI's ``folder_paths``.
that were already resolved via ``folder_paths.get_folder_paths``.
"""
try:
from .services.settings_manager import get_settings_manager
@@ -207,6 +212,12 @@ class Config:
if not isinstance(library_config, dict):
return
# Always read recipes_path — it is independent of extra folder paths
# and must be set before any early returns below.
recipes_path = library_config.get("recipes_path", "")
if isinstance(recipes_path, str) and recipes_path:
self.recipes_path = recipes_path
extra_folder_paths = library_config.get("extra_folder_paths")
if not isinstance(extra_folder_paths, dict):
return
@@ -232,10 +243,6 @@ class Config:
extra_embedding
)
recipes_path = library_config.get("recipes_path", "")
if isinstance(recipes_path, str) and recipes_path:
self.recipes_path = recipes_path
if self.extra_loras_roots:
logger.info(
"Found extra LoRA roots:"
@@ -356,6 +363,47 @@ class Config:
"Failed to rename legacy 'default' library: %s", rename_error
)
# Clean up a stale "default" library entry that has no meaningful
# paths configured (e.g. leftover bootstrap artifact). This only
# fires when "comfyui" already exists so we never delete the last
# remaining library.
if (
"default" in libraries
and "comfyui" in libraries
and isinstance(default_library, Mapping)
):
default_folder_paths = _normalize_library_folder_paths(
default_library
)
default_extra_paths = default_library.get("extra_folder_paths", {})
has_meaningful_paths = bool(default_folder_paths) or bool(
default_extra_paths
) or any(
default_library.get(key)
for key in (
"default_lora_root",
"default_checkpoint_root",
"default_unet_root",
"default_embedding_root",
"recipes_path",
)
)
if not has_meaningful_paths:
try:
settings_service.delete_library("default")
libraries_changed = True
logger.info(
"Removed stale 'default' library entry "
"with no meaningful paths configured"
)
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
except Exception as delete_error:
logger.debug(
"Failed to remove stale 'default' library: %s",
delete_error,
)
default_lora_root = _resolve_valid_default_root(
comfy_library.get("default_lora_root", ""),
list(self.loras_roots or []),
@@ -438,7 +486,7 @@ class Config:
import ctypes
FILE_ATTRIBUTE_REPARSE_POINT = 0x400
attrs = ctypes.windll.kernel32.GetFileAttributesW(str(path)) # type: ignore[attr-defined]
attrs = ctypes.windll.kernel32.GetFileAttributesW(str(path)) # pyright: ignore[reportAttributeAccessIssue]
return attrs != -1 and (attrs & FILE_ATTRIBUTE_REPARSE_POINT)
except Exception as e:
logger.error(f"Error checking Windows reparse point: {e}")
@@ -447,7 +495,7 @@ class Config:
logger.error(f"Error checking link status for {path}: {e}")
return False
def _entry_is_symlink(self, entry: os.DirEntry) -> bool:
def _entry_is_symlink(self, entry: os.DirEntry[str]) -> bool:
"""Check if a directory entry is a symlink, including Windows junctions."""
if entry.is_symlink():
return True
@@ -456,7 +504,7 @@ class Config:
import ctypes
FILE_ATTRIBUTE_REPARSE_POINT = 0x400
attrs = ctypes.windll.kernel32.GetFileAttributesW(entry.path) # type: ignore[attr-defined]
attrs = ctypes.windll.kernel32.GetFileAttributesW(entry.path) # pyright: ignore[reportAttributeAccessIssue]
return attrs != -1 and (attrs & FILE_ATTRIBUTE_REPARSE_POINT)
except Exception:
pass
@@ -1082,8 +1130,8 @@ class Config:
def _apply_library_paths(
self,
folder_paths: Mapping[str, Iterable[str]],
extra_folder_paths: Optional[Mapping[str, Iterable[str]]] = None,
folder_paths: Mapping[str, Any],
extra_folder_paths: Optional[Mapping[str, Any]] = None,
recipes_path: str = "",
) -> None:
self._path_mappings.clear()
@@ -1380,4 +1428,21 @@ class Config:
# Global config instance
config = Config()
# NOTE: Guard against re-import. When ServiceRegistry.get_lora_scanner() triggers
# a fresh import of lora_scanner → config, we must NOT re-execute Config.__init__()
# (which re-scans all roots, re-registers libraries, etc.).
#
# Strategy: store the config instance in a dedicated sentinel module
# ('_lm_config_cache') that is NEVER removed from sys.modules (its key does
# NOT start with 'py.'), so it survives re-imports of py.* modules.
_CONFIG_SENTINEL = "_lm_config_cache"
config: Config
if _CONFIG_SENTINEL in _sys.modules:
# Re-import: reuse the existing singleton from the sentinel.
config = _sys.modules[_CONFIG_SENTINEL].config
else:
config = Config()
# Register the sentinel so re-imports of py.config find us.
_sentinel_mod = _types.ModuleType(_CONFIG_SENTINEL)
setattr(_sentinel_mod, "config", config)
_sys.modules[_CONFIG_SENTINEL] = _sentinel_mod
+29 -1
View File
@@ -14,7 +14,7 @@ standalone_mode = (
if not standalone_mode:
setup_logging()
from server import PromptServer # type: ignore
from server import PromptServer # pyright: ignore[reportMissingImports]
from .config import config
from .services.model_service_factory import (
@@ -25,10 +25,12 @@ from .routes.recipe_routes import RecipeRoutes
from .routes.stats_routes import StatsRoutes
from .routes.update_routes import UpdateRoutes
from .routes.misc_routes import MiscRoutes
from .routes.pending_delete_routes import PendingDeleteRoutes
from .routes.preview_routes import PreviewRoutes
from .routes.example_images_routes import ExampleImagesRoutes
from .services.service_registry import ServiceRegistry
from .services.settings_manager import get_settings_manager
from .services.pending_delete_service import get_pending_delete_service
from .utils.example_images_migration import ExampleImagesMigration
from .services.websocket_manager import ws_manager
from .services.example_images_cleanup_service import ExampleImagesCleanupService
@@ -170,6 +172,7 @@ class LoraManager:
RecipeRoutes.setup_routes(app)
UpdateRoutes.setup_routes(app)
MiscRoutes.setup_routes(app)
PendingDeleteRoutes.setup_routes(app)
ExampleImagesRoutes.setup_routes(app, ws_manager=ws_manager)
PreviewRoutes.setup_routes(app)
@@ -208,6 +211,10 @@ class LoraManager:
# Initialize WebSocket manager
await ServiceRegistry.get_websocket_manager()
# Preload LLM model catalog (background task, non-blocking)
from .services.llm_service import LLMService
await LLMService.get_instance()
# Initialize scanners in background
lora_scanner = await ServiceRegistry.get_lora_scanner()
checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
@@ -241,6 +248,20 @@ class LoraManager:
cls._run_post_initialization_tasks(init_tasks), name="post_init_tasks"
)
# Startup sweep: purge pending-delete batches that expired during a
# previous run. Non-blocking (fire-and-forget); purge_expired only
# removes already-expired batches, so a staged undo that survived a
# restart stays restorable. scan_roots=True runs the reconciliation
# pass first so leftover batches (the in-process registry is empty
# after a restart) are re-discovered on disk. Covers both plugin
# and standalone modes (StandaloneLoraManager reuses this
# classmethod).
pending_delete_service = await get_pending_delete_service()
asyncio.create_task(
pending_delete_service.purge_expired(scan_roots=True),
name="pending_delete_startup_sweep",
)
logger.debug(
"LoRA Manager: All services initialized and background tasks scheduled"
)
@@ -445,5 +466,12 @@ class LoraManager:
scanner.cancel_task()
logger.debug("LoRA Manager: Cancelled %s", name)
# Close shared aiohttp sessions to avoid "Unclosed client session" warnings
try:
from py.routes.handlers.hf_handlers import close_hf_api_session
await close_hf_api_session()
except Exception as exc:
logger.debug("Error closing HF API session: %s", exc)
except Exception as e:
logger.error(f"Error during cleanup: {e}", exc_info=True)
+2 -2
View File
@@ -22,7 +22,7 @@ if not standalone_mode:
logger.info("ComfyUI Metadata Collector initialized")
def get_metadata(prompt_id=None): # type: ignore[no-redef]
def get_metadata(prompt_id=None): # pyright: ignore[reportRedeclaration]
"""Helper function to get metadata from the registry"""
registry = MetadataRegistry()
return registry.get_metadata(prompt_id)
@@ -31,6 +31,6 @@ else:
def init():
logger.info("ComfyUI Metadata Collector disabled in standalone mode")
def get_metadata(prompt_id=None): # type: ignore[no-redef]
def get_metadata(prompt_id=None): # pyright: ignore[reportRedeclaration]
"""Dummy implementation for standalone mode"""
return {}
+15 -1
View File
@@ -1,5 +1,11 @@
"""Constants used by the metadata collector"""
# Sentinel value for clip_skip to distinguish "unconnected / widget default"
# from "user wired value 0". Both ComfyUI CLIPSetLastLayer (-24..-1) and
# A1111 conventions treat 0 as meaningless for clip skipping, but users may
# explicitly wire 0 to the overwrite node to express "no clip skip / default".
CLIP_SKIP_SENTINEL = -25
# Metadata categories
MODELS = "models"
PROMPTS = "prompts"
@@ -9,6 +15,14 @@ EMBEDDINGS = "embeddings"
SIZE = "size"
IMAGES = "images"
IS_SAMPLER = "is_sampler" # New constant to mark sampler nodes
OVERWRITE = "overwrite" # Manual metadata overwrite from MetadataOverwriteLM node
# Field names that the MetadataOverwriteLM node and its extractor share
METADATA_OVERWRITE_FIELDS = (
"prompt", "negative_prompt", "seed", "steps", "cfg_scale",
"sampler", "scheduler", "model", "loras", "size",
"clip_skip", "additional_data",
)
# Complete list of categories to track
METADATA_CATEGORIES = [MODELS, PROMPTS, SAMPLING, LORAS, EMBEDDINGS, SIZE, IMAGES]
METADATA_CATEGORIES = [MODELS, PROMPTS, SAMPLING, LORAS, EMBEDDINGS, SIZE, IMAGES, OVERWRITE]
+15 -5
View File
@@ -16,7 +16,7 @@ class MetadataHook:
execution = None
try:
# Try direct import first
import execution # type: ignore
import execution # pyright: ignore[reportMissingImports]
except ImportError:
# Try to locate from system modules
for module_name in sys.modules:
@@ -83,7 +83,8 @@ class MetadataHook:
# Record inputs before execution
if node_id is not None:
registry.record_node_execution(node_id, class_type, input_data_all, None)
return_types = getattr(obj, 'RETURN_TYPES', None)
registry.record_node_execution(node_id, class_type, input_data_all, None, return_types=return_types)
except Exception as e:
logger.error(f"Error collecting metadata (pre-execution): {str(e)}")
@@ -114,7 +115,8 @@ class MetadataHook:
# Record outputs after execution
if node_id is not None:
registry.update_node_execution(node_id, class_type, results)
return_types = getattr(obj, 'RETURN_TYPES', None)
registry.update_node_execution(node_id, class_type, results, return_types=return_types)
except Exception as e:
logger.error(f"Error collecting metadata (post-execution): {str(e)}")
@@ -136,6 +138,9 @@ class MetadataHook:
if hasattr(prompt, 'original_prompt'):
registry.set_current_prompt(prompt)
# Store extra_data for accessing full workflow node properties
registry.set_extra_data(extra_data)
# Execute the original function
return original_execute(*args, **kwargs)
@@ -163,7 +168,8 @@ class MetadataHook:
class_type = obj.__class__.__name__
node_id = unique_id
if node_id is not None:
registry.record_node_execution(node_id, class_type, input_data_all, None)
return_types = getattr(obj, 'RETURN_TYPES', None)
registry.record_node_execution(node_id, class_type, input_data_all, None, return_types=return_types)
except Exception as e:
logger.error(f"Error collecting metadata (pre-execution): {str(e)}")
@@ -180,7 +186,8 @@ class MetadataHook:
class_type = obj.__class__.__name__
node_id = unique_id
if node_id is not None:
registry.update_node_execution(node_id, class_type, results)
return_types = getattr(obj, 'RETURN_TYPES', None)
registry.update_node_execution(node_id, class_type, results, return_types=return_types)
except Exception as e:
logger.error(f"Error collecting metadata (post-execution): {str(e)}")
@@ -202,6 +209,9 @@ class MetadataHook:
if hasattr(prompt, 'original_prompt'):
registry.set_current_prompt(prompt)
# Store extra_data for accessing full workflow node properties
registry.set_extra_data(extra_data)
# Execute the original function
return await original_execute(*args, **kwargs)
+147 -5
View File
@@ -1,15 +1,68 @@
import json
import logging
import os
from .constants import IMAGES
# Check if running in standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER, OVERWRITE
from .node_extractors import NODE_EXTRACTORS
logger = logging.getLogger(__name__)
# Keys that identify metadata hint marks stored in node.properties.lm_marker_role
_META_MARK_PREFIX = "meta_"
_MARK_PRIMARY_MODEL = "primary_model"
_MARK_PRIMARY_SAMPLER = "primary_sampler"
_MARK_POSITIVE_PROMPT = "positive_prompt"
_MARK_NEGATIVE_PROMPT = "negative_prompt"
class MetadataProcessor:
"""Process and format collected metadata"""
@staticmethod
def _get_user_marks(metadata):
"""Scan workflow nodes (from extra_data.extra_pnginfo.workflow) for user-assigned
metadata hint marks stored in node.properties.lm_marker_role.
Returns a dict mapping mark type keys to node IDs.
Example: {'primary_model': '42', 'primary_sampler': '17'}
"""
marks: dict[str, str] = {}
# Primary source: extra_data.extra_pnginfo.workflow.nodes (has full properties)
extra_data = metadata.get("extra_data")
if extra_data and isinstance(extra_data, dict):
extra_pnginfo = extra_data.get("extra_pnginfo", {})
if isinstance(extra_pnginfo, dict):
workflow = extra_pnginfo.get("workflow", {})
nodes = workflow.get("nodes", [])
for node in nodes:
node_id = str(node.get("id", ""))
role = node.get("properties", {}).get("lm_marker_role", "")
if role.startswith(_META_MARK_PREFIX):
mark_type = role[len(_META_MARK_PREFIX):]
if mark_type in marks:
logger.warning(
"Duplicate meta hint '%s': node %s (previous: %s), "
"last match wins",
mark_type, node_id, marks[mark_type],
)
marks[mark_type] = node_id
# Fallback: try prompt.original_prompt (API-only submissions may not have workflow)
if not marks:
prompt = metadata.get("current_prompt")
if prompt and getattr(prompt, "original_prompt", None):
for node_id, node_data in prompt.original_prompt.items():
role = node_data.get("properties", {}).get("lm_marker_role", "")
if role.startswith(_META_MARK_PREFIX):
mark_type = role[len(_META_MARK_PREFIX):]
marks[mark_type] = node_id
return marks
@staticmethod
def find_primary_sampler(metadata, downstream_id=None):
"""
@@ -162,6 +215,24 @@ class MetadataProcessor:
primary_sampler = sampler_info
primary_sampler_id = node_id
# Last resort: any registered sampler. Samplers without a denoise or
# add_noise parameter (e.g. multi-stage samplers like KreaTwoStageSampler)
# are not caught by the criteria above. Prefer execution order so the
# first executed sampler wins, matching the downstream_id branch.
if primary_sampler is None:
sampler_ids = [
node_id
for node_id, sampler_info in metadata.get(SAMPLING, {}).items()
if sampler_info.get(IS_SAMPLER, False)
]
if sampler_ids:
if downstream_id and "execution_order" in metadata:
for node_id in metadata["execution_order"]:
if node_id in sampler_ids:
return node_id, metadata[SAMPLING][node_id]
primary_sampler_id = sampler_ids[0]
primary_sampler = metadata[SAMPLING][sampler_ids[0]]
return primary_sampler_id, primary_sampler
@staticmethod
@@ -471,17 +542,54 @@ class MetadataProcessor:
"checkpoint": None,
"loras": "",
"size": None,
"clip_skip": None
"clip_skip": None,
"additional_data": "",
}
# Get the prompt object for node relationship tracing
prompt = metadata.get("current_prompt")
# Find the primary KSampler node
# ---- User marks: override heuristic inference with user-assigned hints ----
user_marks = MetadataProcessor._get_user_marks(metadata)
# Find the primary KSampler node (user mark takes priority)
primary_sampler_id = None
primary_sampler = None
if _MARK_PRIMARY_SAMPLER in user_marks:
marked_id = user_marks[_MARK_PRIMARY_SAMPLER]
sampler_data = metadata.get(SAMPLING, {}).get(marked_id)
if sampler_data and sampler_data.get(IS_SAMPLER):
primary_sampler_id = marked_id
primary_sampler = sampler_data
else:
logger.warning(
"User-marked primary sampler %s has no runtime metadata, "
"falling back to heuristic",
marked_id,
)
if primary_sampler is None:
primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id)
# Directly get checkpoint from metadata instead of tracing
# Pass primary_sampler_id to avoid redundant calculation
# Resolve checkpoint / model (user mark takes priority)
if _MARK_PRIMARY_MODEL in user_marks:
marked_id = user_marks[_MARK_PRIMARY_MODEL]
if marked_id in metadata.get(MODELS, {}):
params["checkpoint"] = metadata[MODELS][marked_id].get("name")
else:
extra_data = metadata.get("extra_data")
extra_pnginfo = extra_data.get("extra_pnginfo", {}) if extra_data and isinstance(extra_data, dict) else {}
workflow = extra_pnginfo.get("workflow", {}) if isinstance(extra_pnginfo, dict) else {}
node_type = "unknown"
for n in workflow.get("nodes", []):
if str(n.get("id", "")) == marked_id:
node_type = n.get("type", "unknown")
break
logger.warning(
"User-marked primary model %s (type=%s, registered=%s) has no runtime metadata, "
"falling back to heuristic",
marked_id, node_type, node_type in NODE_EXTRACTORS,
)
if params["checkpoint"] is None:
checkpoint = MetadataProcessor.find_primary_checkpoint(metadata, id, primary_sampler_id)
if checkpoint:
params["checkpoint"] = checkpoint
@@ -540,6 +648,21 @@ class MetadataProcessor:
# For SamplerCustom, handle any additional parameters
MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, primary_sampler_id, params)
# ---- User marks: override prompts with explicitly tagged nodes ----
prompts_data = metadata.get(PROMPTS, {})
if _MARK_POSITIVE_PROMPT in user_marks:
pos_id = user_marks[_MARK_POSITIVE_PROMPT]
if pos_id in prompts_data:
prompt_text = prompts_data[pos_id].get("text") or prompts_data[pos_id].get("positive_text")
if prompt_text:
params["prompt"] = prompt_text
if _MARK_NEGATIVE_PROMPT in user_marks:
neg_id = user_marks[_MARK_NEGATIVE_PROMPT]
if neg_id in prompts_data:
prompt_text = prompts_data[neg_id].get("text") or prompts_data[neg_id].get("negative_text")
if prompt_text:
params["negative_prompt"] = prompt_text
# Size extraction is same for all sampler types
# Check if the sampler itself has size information (from latent_image)
if primary_sampler_id in metadata.get(SIZE, {}):
@@ -569,6 +692,25 @@ class MetadataProcessor:
if params["clip_skip"] is None:
params["clip_skip"] = "1"
# ---- Apply manual metadata overwrites ----
for overwrite_info in metadata.get(OVERWRITE, {}).values():
overwrite_params = overwrite_info.get("parameters", {})
for key, value in overwrite_params.items():
if key == "clip_skip":
# Accept any value from overwrite node (sentinel -25 already
# filtered upstream). Needed because falsy check treats 0
# as "not set" even though 0 is a valid wired input here.
params[key] = value
elif value: # truthy check — only overwrite when user provided a real value
params[key] = value
# Bridge: the overwrite node exposes the field as "model" (more accurate),
# but the internal pipeline key remains "checkpoint" for backward compatibility
# with A1111 metadata format and downstream consumers.
if params.get("model"):
params["checkpoint"] = params["model"]
del params["model"]
return params
@staticmethod
+44 -11
View File
@@ -1,7 +1,8 @@
import time
from nodes import NODE_CLASS_MAPPINGS # type: ignore
from typing import Any
from nodes import NODE_CLASS_MAPPINGS # pyright: ignore[reportMissingImports, reportAttributeAccessIssue]
from .node_extractors import NODE_EXTRACTORS, GenericNodeExtractor
from .constants import METADATA_CATEGORIES, IMAGES
from .constants import METADATA_CATEGORIES, IMAGES, OVERWRITE
class MetadataRegistry:
@@ -9,6 +10,15 @@ class MetadataRegistry:
_instance = None
current_prompt_id: Any = None
current_prompt: Any = None
metadata: dict[str, Any] = {}
prompt_metadata: dict[str, Any] = {}
executed_nodes: set[str] = set()
node_cache: dict[str, Any] = {}
max_prompt_history: int = 3
metadata_categories: list[str] = METADATA_CATEGORIES
def __new__(cls):
if cls._instance is None:
cls._instance = super().__new__(cls)
@@ -61,6 +71,7 @@ class MetadataRegistry:
{
"execution_order": [],
"current_prompt": None, # Will store the prompt object
"extra_data": None, # Will store the API extra_data for workflow metadata
"timestamp": time.time(),
}
)
@@ -75,6 +86,11 @@ class MetadataRegistry:
# Store the prompt in the metadata for later relationship tracing
self.prompt_metadata[self.current_prompt_id]["current_prompt"] = prompt
def set_extra_data(self, extra_data):
"""Store the API extra_data (contains extra_pnginfo.workflow with node properties)"""
if self.current_prompt_id and self.current_prompt_id in self.prompt_metadata:
self.prompt_metadata[self.current_prompt_id]["extra_data"] = extra_data
def get_metadata(self, prompt_id=None):
"""Get collected metadata for a prompt"""
key = prompt_id if prompt_id is not None else self.current_prompt_id
@@ -122,20 +138,28 @@ class MetadataRegistry:
cache_key = f"{node_id}:{class_type}"
# Check if this node type is relevant for metadata collection
if class_type in NODE_EXTRACTORS:
if class_type in NODE_EXTRACTORS or cache_key in self.node_cache:
# Check if we have cached metadata for this node
if cache_key in self.node_cache:
cached_data = self.node_cache[cache_key]
# Detect bypass (mode=4) / mute (mode=2) — these nodes
# were intentionally disabled and should not contribute
# overwrite values from a previous execution's cache.
node_mode = node_data.get("mode", 0)
node_is_disabled = node_mode in (2, 4)
# Apply cached metadata to the current metadata
for category in self.metadata_categories:
if category == OVERWRITE and node_is_disabled:
continue
if category in cached_data and node_id in cached_data[category]:
if node_id not in metadata[category]:
metadata[category][node_id] = cached_data[category][
node_id
]
def record_node_execution(self, node_id, class_type, inputs, outputs):
def record_node_execution(self, node_id, class_type, inputs, outputs, return_types=None):
"""Record information about a node's execution"""
if not self.current_prompt_id:
return
@@ -158,17 +182,18 @@ class MetadataRegistry:
# Extract node-specific metadata
extractor = NODE_EXTRACTORS.get(class_type, GenericNodeExtractor)
extractor.extract(
node_id,
processed_inputs,
outputs,
if extractor is GenericNodeExtractor:
extractor.extract(node_id, processed_inputs, outputs,
self.prompt_metadata[self.current_prompt_id],
)
return_types=return_types)
else:
extractor.extract(node_id, processed_inputs, outputs,
self.prompt_metadata[self.current_prompt_id])
# Cache this node's metadata
self._cache_node_metadata(node_id, class_type)
def update_node_execution(self, node_id, class_type, outputs):
def update_node_execution(self, node_id, class_type, outputs, return_types=None):
"""Update node metadata with output information"""
if not self.current_prompt_id:
return
@@ -179,8 +204,16 @@ class MetadataRegistry:
# Use the same extractor to update with outputs
extractor = NODE_EXTRACTORS.get(class_type, GenericNodeExtractor)
if hasattr(extractor, "update"):
if extractor is GenericNodeExtractor:
extractor.update(
node_id, processed_outputs, self.prompt_metadata[self.current_prompt_id]
node_id, processed_outputs,
self.prompt_metadata[self.current_prompt_id],
return_types=return_types,
)
else:
extractor.update(
node_id, processed_outputs,
self.prompt_metadata[self.current_prompt_id],
)
# Update the cached metadata for this node
+243 -11
View File
@@ -2,7 +2,8 @@ import json
import os
import re
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER, OVERWRITE
from .overwrite_utils import collect_overwrite_params
def _store_checkpoint_metadata(metadata, node_id, model_name):
@@ -31,10 +32,94 @@ class NodeMetadataExtractor:
pass
class GenericNodeExtractor(NodeMetadataExtractor):
"""Default extractor for nodes without specific handling"""
"""Fallback extractor with type-signature-based detection.
When a node is not in the NODE_EXTRACTORS registry, the hook layer
passes ``return_types`` from ``obj.RETURN_TYPES``:
* ``MODEL`` output: common input fields (ckpt_name, unet_name, etc.)
are checked for a model file name and stored as checkpoint metadata.
* ``CONDITIONING`` output: common text input fields are checked for
prompt text, and conditioning inputs are tracked through transforms.
"""
# Input field names that carry a model path in loader-style nodes.
_MODEL_NAME_FIELDS = (
"ckpt_name", "unet_name", "model_path", "model_name", "gguf_name",
)
# Extensions used by checkpoint_scanner.py — only record values that look
# like real model filenames to avoid capturing unrelated string fields.
_MODEL_EXTENSIONS = {
".ckpt", ".pt", ".pt2", ".bin", ".pth", ".safetensors", ".pkl", ".sft", ".gguf",
}
# Input field names that may carry prompt text in encoder-style nodes.
_TEXT_FIELDS = ("text", "clip_l", "t5xxl", "prompt", "positive", "negative")
@staticmethod
def extract(node_id, inputs, outputs, metadata):
pass
def extract(node_id, inputs, outputs, metadata, return_types=None):
if return_types is None:
return
# — MODEL loader detection (checkpoint / UNET / GGUF) —
if "MODEL" in return_types or any("MODEL" in str(t) for t in return_types):
for field in GenericNodeExtractor._MODEL_NAME_FIELDS:
val = inputs.get(field)
if val and isinstance(val, str) and val.strip():
name = val.strip()
if not any(name.lower().endswith(ext) for ext in GenericNodeExtractor._MODEL_EXTENSIONS):
continue
_store_checkpoint_metadata(metadata, node_id, name)
return
# — CONDITIONING encoder / transform detection —
if "CONDITIONING" in return_types or any("CONDITIONING" in str(t) for t in return_types):
text = None
for field in GenericNodeExtractor._TEXT_FIELDS:
val = inputs.get(field)
if val and isinstance(val, str) and val.strip():
text = val.strip()
break
input_conditionings = _collect_conditioning_inputs(inputs)
if text or input_conditionings:
prompt_metadata = _ensure_prompt_metadata(metadata, node_id)
if text:
prompt_metadata["text"] = text
if input_conditionings:
prompt_metadata["orig_conditionings"] = input_conditionings
@staticmethod
def update(node_id, outputs, metadata, return_types=None):
if return_types is None:
return
if "CONDITIONING" not in return_types and not any(
"CONDITIONING" in str(t) for t in return_types
):
return
if node_id not in metadata.get(PROMPTS, {}):
return
output_tuple = _first_output_tuple(outputs)
if not output_tuple or len(output_tuple) < 1:
return
conditioning_index = _first_conditioning_index(return_types)
if conditioning_index is None or len(output_tuple) <= conditioning_index:
return
output_conditioning = output_tuple[conditioning_index]
if output_conditioning is None:
return
prompt_metadata = metadata[PROMPTS][node_id]
prompt_metadata["conditioning"] = output_conditioning
_record_conditioning_source(
metadata,
node_id,
output_conditioning,
prompt_metadata.get("orig_conditionings", []),
)
class CheckpointLoaderExtractor(NodeMetadataExtractor):
@staticmethod
@@ -349,6 +434,34 @@ def _first_output_tuple(outputs):
return None
def _first_conditioning_index(return_types):
"""Return the index of the first CONDITIONING output slot, or None."""
if not return_types:
return None
for index, return_type in enumerate(return_types):
if "CONDITIONING" in str(return_type):
return index
return None
def _collect_conditioning_inputs(inputs):
"""Collect conditioning object inputs (``conditioning*`` keys).
Primitive values (None, str, int, float, bool) are excluded so scalar
fields like ``conditioning_strength`` are not mistaken for conditioning
objects during provenance tracking.
"""
if not inputs:
return []
return [
value
for input_name, value in inputs.items()
if input_name.startswith("conditioning")
and value is not None
and not isinstance(value, (str, int, float, bool))
]
def _record_conditioning_source(
metadata, node_id, output_conditioning, input_conditionings
):
@@ -361,6 +474,14 @@ def _record_conditioning_source(
if not sources:
return
# Identity-preserving selectors return one of their inputs unchanged:
# only that input contributed to the output, so record it alone instead
# of treating every input as a combination source.
for conditioning in sources:
if id(conditioning) == id(output_conditioning):
sources = [conditioning]
break
prompt_metadata = _ensure_prompt_metadata(metadata, node_id)
prompt_metadata.setdefault("conditioning_sources", []).append(
{
@@ -440,13 +561,7 @@ class ConditioningCombineExtractor(NodeMetadataExtractor):
if not inputs:
return
input_conditionings = []
for input_name in inputs:
if (
input_name.startswith("conditioning")
and inputs[input_name] is not None
):
input_conditionings.append(inputs[input_name])
input_conditionings = _collect_conditioning_inputs(inputs)
if input_conditionings:
prompt_metadata = _ensure_prompt_metadata(metadata, node_id)
@@ -746,6 +861,65 @@ class TSCKSamplerAdvancedExtractor(KSamplerAdvancedExtractor, TSCSamplerBaseExtr
# Update method is inherited from TSCSamplerBaseExtractor
class KreaTwoStageSamplerExtractor(BaseSamplerExtractor):
"""Extractor for Krea Two/Three Stage Samplers (Auryg/Krea-2-Two-Stage-Sampler).
The node samples in two (or three) stages with per-stage settings
(stage1_steps/stage2_steps, stage1_cfg/stage2_cfg, ...). The canonical
metadata fields consumed by ``extract_generation_params`` (steps, cfg,
sampler_name, scheduler) are derived from the base stage (stage 1; the
three-stage variant reuses stage 1 settings for stage 3), while the full
per-stage breakdown is preserved in the raw parameters.
"""
# All per-stage parameter keys present on both node variants.
_STAGE_PARAM_KEYS = (
"stage1_steps", "stage1_cfg", "stage1_sampler_name", "stage1_scheduler",
"stage2_steps", "stage2_cfg", "stage2_sampler_name", "stage2_scheduler",
)
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs:
return
BaseSamplerExtractor.extract_sampling_params(
node_id,
inputs,
metadata,
("seed", "handoff_percent", "stage3_handoff_percent")
+ KreaTwoStageSamplerExtractor._STAGE_PARAM_KEYS,
)
# Derive the canonical fields expected by extract_generation_params.
sampling_params = metadata[SAMPLING][node_id]["parameters"]
if "stage1_steps" in sampling_params or "stage2_steps" in sampling_params:
sampling_params["steps"] = (
(sampling_params.get("stage1_steps") or 0)
+ (sampling_params.get("stage2_steps") or 0)
)
if "stage1_cfg" in sampling_params:
sampling_params["cfg"] = sampling_params["stage1_cfg"]
if "stage1_sampler_name" in sampling_params:
sampling_params["sampler_name"] = sampling_params["stage1_sampler_name"]
if "stage1_scheduler" in sampling_params:
sampling_params["scheduler"] = sampling_params["stage1_scheduler"]
BaseSamplerExtractor.extract_conditioning(node_id, inputs, metadata)
# Prefer the final generation resolution; latent dims are the fallback.
BaseSamplerExtractor.extract_latent_dimensions(node_id, inputs, metadata)
final_width = inputs.get("final_width")
final_height = inputs.get("final_height")
if final_width and final_height:
if SIZE not in metadata:
metadata[SIZE] = {}
metadata[SIZE][node_id] = {
"width": final_width,
"height": final_height,
"node_id": node_id,
}
class LoraLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
@@ -786,6 +960,37 @@ class ImageSizeExtractor(NodeMetadataExtractor):
"node_id": node_id
}
class KreaDualResolutionSelectorExtractor(NodeMetadataExtractor):
"""Extract base resolution from Krea Dual Resolution Selector outputs
(Auryg/Krea-2-Two-Stage-Sampler).
The node computes base/final dimensions at runtime from aspect ratio and
megapixel settings, so the values are only available in the update phase
(outputs: base_width, base_height, final_width, final_height, seed).
"""
@staticmethod
def extract(node_id, inputs, outputs, metadata):
# Dimensions are computed at runtime; nothing to do here.
pass
@staticmethod
def update(node_id, outputs, metadata):
output_tuple = _first_output_tuple(outputs)
if not output_tuple or len(output_tuple) < 2:
return
width, height = output_tuple[0], output_tuple[1]
if not isinstance(width, int) or not isinstance(height, int):
return
if SIZE not in metadata:
metadata[SIZE] = {}
metadata[SIZE][node_id] = {
"width": width,
"height": height,
"node_id": node_id,
}
class RgthreePowerLoraLoaderExtractor(NodeMetadataExtractor):
"""Extract LoRA metadata from rgthree Power Lora Loader.
@@ -1154,6 +1359,28 @@ class CR_ApplyControlNetStackExtractor(NodeMetadataExtractor):
metadata[PROMPTS][node_id]["positive_encoded"] = transformed_positive
metadata[PROMPTS][node_id]["negative_encoded"] = transformed_negative
class MetadataOverwriteExtractor(NodeMetadataExtractor):
"""Extract manually specified metadata from MetadataOverwriteLM node.
Stores truthy input values under the OVERWRITE category so that
extract_generation_params can merge them over the inferred params.
"""
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs:
return
overwrite_params = collect_overwrite_params(inputs)
if overwrite_params:
metadata.setdefault(OVERWRITE, {})
metadata[OVERWRITE][node_id] = {
"parameters": overwrite_params,
"node_id": node_id,
}
# Registry of node-specific extractors
# Keys are node class names
NODE_EXTRACTORS = {
@@ -1165,6 +1392,8 @@ NODE_EXTRACTORS = {
"ClownsharKSampler_Beta": SamplerExtractor,
"TSC_KSampler": TSCKSamplerExtractor, # Efficient Nodes
"TSC_KSamplerAdvanced": TSCKSamplerAdvancedExtractor, # Efficient Nodes
"KreaTwoStageSampler": KreaTwoStageSamplerExtractor, # Auryg/Krea-2-Two-Stage-Sampler
"KreaThreeStageSampler": KreaTwoStageSamplerExtractor, # Auryg/Krea-2-Two-Stage-Sampler
"KSamplerBasicPipe": KSamplerBasicPipeExtractor, # comfyui-impact-pack
"KSamplerAdvancedBasicPipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-impact-pack
"KSampler_inspire_pipe": KSamplerBasicPipeExtractor, # comfyui-inspire-pack
@@ -1216,10 +1445,13 @@ NODE_EXTRACTORS = {
"GetNode": GetNodeExtractor,
# Latent
"EmptyLatentImage": ImageSizeExtractor,
"KreaDualResolutionSelector": KreaDualResolutionSelectorExtractor, # Auryg/Krea-2-Two-Stage-Sampler
# Flux
"FluxGuidance": FluxGuidanceExtractor, # Add FluxGuidance
"CFGGuider": CFGGuiderExtractor, # Add CFGGuider
# Image
"VAEDecode": VAEDecodeExtractor, # Added VAEDecode extractor
# Metadata overwrite
"MetadataOverwriteLM": MetadataOverwriteExtractor,
# Add other nodes as needed
}
+51
View File
@@ -0,0 +1,51 @@
"""Shared helpers for Metadata Overwrite node metadata collection.
Used by both the MetadataOverwriteLM node (execution time) and the
MetadataOverwriteExtractor (hook time) so the conversion/filtering logic
cannot drift between the two paths.
"""
import logging
from typing import Any, Dict
from ..utils.utils import model_patcher_to_name, sampler_object_to_name
from .constants import CLIP_SKIP_SENTINEL, METADATA_OVERWRITE_FIELDS
logger = logging.getLogger(__name__)
def collect_overwrite_params(values: Dict[str, Any]) -> Dict[str, Any]:
"""Convert node input values into non-default overwrite parameters.
For most fields, a falsy value (empty string, 0) means "not set" and is
skipped. clip_skip uses a dedicated sentinel (-25) so that a wired value
of 0 is preserved. The ``model`` field accepts either a manual string or
a wired MODEL (ModelPatcher) connection; in the latter case the source
model name is extracted from the patcher's ``cached_patcher_init`` and
stored as a ComfyUI-style relative path. The ``sampler`` field likewise
accepts a manual string or a wired SAMPLER (KSAMPLER) connection, from
which the sampler name is extracted via the sampler function's name.
"""
result: Dict[str, Any] = {}
for key in METADATA_OVERWRITE_FIELDS:
value = values.get(key)
if key == "model" and not isinstance(value, str):
value = model_patcher_to_name(value)
if value is None:
logger.warning(
"Could not extract model name from wired MODEL input "
"(no cached_patcher_init); model metadata overwrite skipped"
)
elif key == "sampler" and not isinstance(value, str):
value = sampler_object_to_name(value)
if value is None:
logger.warning(
"Could not extract sampler name from wired SAMPLER input "
"(unrecognized sampler function); sampler metadata overwrite skipped"
)
if key == "clip_skip":
if value != CLIP_SKIP_SENTINEL:
result[key] = value
elif value:
result[key] = value
return result
+233
View File
@@ -0,0 +1,233 @@
"""Metadata operations — thin in-process wrappers around LoRA Manager internal services.
All functions are simple Python async functions that delegate to the
appropriate internal service. They use **relative imports** within the
``py`` package, so ``sys.modules`` caching works normally and there is no
risk of double import or circular dependencies.
Usage (in-process, primary)::
from py.metadata_ops import list_base_models, read_metadata
models = await list_base_models()
meta = await read_metadata("/path/to/model.safetensors")
Usage (subprocess, debugging / external)::
python -m py.metadata_ops base-models list
python -m py.metadata_ops metadata read /path/to/model.safetensors
"""
from __future__ import annotations
import asyncio
import logging
import os
from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
SCANNER_TYPE_MAP: dict[str, str] = {
"get_lora_scanner": "lora",
"get_checkpoint_scanner": "checkpoint",
"get_embedding_scanner": "embedding",
}
SCANNER_GETTER_NAMES = tuple(SCANNER_TYPE_MAP.keys())
async def _find_model_entry(
model_path: str,
) -> tuple[Any, object, str | None] | tuple[None, None, None]:
"""Iterate all scanners and return the first (scanner, entry, getter_name)
that owns *model_path*. Returns ``(None, None, None)`` when no scanner
claims it.
"""
from ..services.service_registry import ServiceRegistry
normalized = os.path.normpath(model_path)
for getter_name in SCANNER_GETTER_NAMES:
getter = getattr(ServiceRegistry, getter_name, None)
if getter is None:
continue
try:
scanner = await getter()
if scanner is None:
continue
cache = await scanner.get_cached_data()
for entry in cache.raw_data:
if os.path.normpath(entry.get("file_path", "")) == normalized:
return scanner, entry, getter_name
except Exception as exc:
logger.debug(
"Scanner %s check failed for %s: %s",
getter_name, model_path, exc,
)
return None, None, None
async def _find_scanner_for_model(
model_path: str,
) -> tuple[Any, object] | tuple[None, None]:
"""Find the (scanner, cache_entry) responsible for *model_path*."""
scanner, entry, _ = await _find_model_entry(model_path)
return scanner, entry
async def identify_model_type(model_path: str) -> str:
"""Determine the model type (``\"lora\"``, ``\"checkpoint\"``, or
``\"embedding\"``) for *model_path*.
Falls back to ``\"lora\"`` when unknown.
"""
_, _, getter_name = await _find_model_entry(model_path)
return SCANNER_TYPE_MAP[getter_name] if getter_name else "lora"
# ---------------------------------------------------------------------------
# Public API
# ---------------------------------------------------------------------------
async def list_base_models(limit: int = 0) -> List[str]:
"""Return all valid CivitAI base model names.
Uses ``CivitaiBaseModelService.get_base_models()`` which merges a
hardcoded list (``SUPPORTED_DOWNLOAD_SKIP_BASE_MODELS``) with remote
models fetched from the CivitAI API. Never empty the hardcoded
fallback always provides a complete set.
The result is sorted alphabetically. Pass *limit* = 0 for all models.
"""
from ..services.civitai_base_model_service import (
CivitaiBaseModelService,
)
try:
service = await CivitaiBaseModelService.get_instance()
response = await service.get_base_models()
names: List[str] = response.get("models", [])
except Exception as exc:
logger.warning("list_base_models failed: %s", exc)
names = []
if limit > 0:
return names[:limit]
return names
async def read_metadata(model_path: str) -> Dict[str, Any]:
"""Load the full metadata payload for *model_path* from disk.
Returns an empty dict when the metadata file does not exist or cannot
be parsed never raises.
"""
from ..utils.metadata_manager import MetadataManager
try:
return await MetadataManager.load_metadata_payload(model_path) or {}
except Exception as exc:
logger.warning("read_metadata failed for %s: %s", model_path, exc)
return {}
async def apply_metadata_updates(
model_path: str,
updates: Dict[str, Any],
) -> List[str]:
"""Merge *updates* into the model's on-disk metadata and persist.
Returns the list of field names that actually changed.
"""
from ..utils.metadata_manager import MetadataManager
metadata = await read_metadata(model_path)
updated_fields: List[str] = []
for key, value in updates.items():
old = metadata.get(key)
if old != value:
metadata[key] = value
updated_fields.append(key)
if updated_fields:
await MetadataManager.save_metadata(model_path, metadata)
return updated_fields
async def download_preview(
model_path: str,
url: str,
*,
target_width: int = 480,
quality: int = 85,
) -> str | None:
"""Download a preview image from *url*, optimise to .webp, and save it.
The output file is placed alongside the model file with a ``.webp``
extension. Returns the local file path on success, ``None`` on failure.
"""
from ..services.downloader import get_downloader
from ..utils.exif_utils import ExifUtils
if not url or not url.strip():
return None
base_name = os.path.splitext(os.path.basename(model_path))[0]
preview_dir = os.path.dirname(model_path)
output_path = os.path.join(preview_dir, base_name + ".webp")
downloader = await get_downloader()
# Try in-memory download + optimise first
success, content, _headers = await downloader.download_to_memory(
url, use_auth=False,
)
if success and content:
try:
optimized_data, _ = ExifUtils.optimize_image(
image_data=content,
target_width=target_width,
format="webp",
quality=quality,
preserve_metadata=False,
)
with open(output_path, "wb") as f:
f.write(optimized_data)
return output_path
except Exception as exc:
logger.warning("Preview optimisation failed, saving raw: %s", exc)
# Fall through to raw save
# Fallback: download directly to file
try:
ok, _ = await downloader.download_file(url, output_path, use_auth=False)
if ok:
return output_path
except Exception as exc:
logger.warning("Preview fallback download failed for %s: %s", model_path, exc)
return None
async def refresh_cache(model_path: str) -> bool:
"""Invalidate and reload the scanner cache entry for *model_path*.
Returns ``True`` when the model was found and the cache was refreshed.
"""
scanner, entry = await _find_scanner_for_model(model_path)
if scanner is None:
logger.warning("refresh_cache: no scanner found for %s", model_path)
return False
try:
metadata = await read_metadata(model_path)
if not metadata:
logger.warning("refresh_cache: no metadata for %s", model_path)
return False
await scanner.update_single_model_cache(model_path, model_path, metadata)
return True
except Exception as exc:
logger.warning("refresh_cache failed for %s: %s", model_path, exc)
return False
+113
View File
@@ -0,0 +1,113 @@
"""Subprocess entry point for ``metadata_ops`` (debugging / external use).
Usage::
python -m py.metadata_ops base-models list [--limit N]
python -m py.metadata_ops metadata read <path>
python -m py.metadata_ops metadata update <path> --json '{...}'
python -m py.metadata_ops preview download <path> --url <url>
python -m py.metadata_ops cache refresh <path>
"""
from __future__ import annotations
import argparse
import asyncio
import json
import sys
from typing import Any, Dict, List
def _build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(prog="lmcli", description="LoRA Manager Agent CLI")
sub = parser.add_subparsers(dest="command", required=True)
# base-models list
base_models = sub.add_parser("base-models", aliases=["bm"])
base_models_cmds = base_models.add_subparsers(dest="subcommand", required=True)
base_models_list = base_models_cmds.add_parser("list")
base_models_list.add_argument(
"--limit", type=int, default=0, help="Max number of models (0 = all)"
)
# metadata read
meta = sub.add_parser("metadata", aliases=["md"])
meta_cmds = meta.add_subparsers(dest="subcommand", required=True)
meta_read = meta_cmds.add_parser("read")
meta_read.add_argument("path", type=str, help="Model file path")
# metadata update
meta_update = meta_cmds.add_parser("update")
meta_update.add_argument("path", type=str, help="Model file path")
meta_update.add_argument(
"--json",
type=str,
required=True,
help='JSON object of fields to update, e.g. \'{"base_model": "SDXL 1.0"}\'',
)
# preview download
prev = sub.add_parser("preview", aliases=["pv"])
prev_cmds = prev.add_subparsers(dest="subcommand", required=True)
prev_dl = prev_cmds.add_parser("download")
prev_dl.add_argument("path", type=str, help="Model file path")
prev_dl.add_argument("--url", type=str, required=True, help="Preview image URL")
# cache refresh
cache = sub.add_parser("cache")
cache_cmds = cache.add_subparsers(dest="subcommand", required=True)
cache_refresh = cache_cmds.add_parser("refresh")
cache_refresh.add_argument("path", type=str, help="Model file path")
return parser
async def _run(args: argparse.Namespace) -> Any:
from . import ( # lazy import so startup is fast
list_base_models,
read_metadata,
apply_metadata_updates,
download_preview,
refresh_cache,
)
cmd = args.command
sub = args.subcommand
if cmd in ("base-models", "bm") and sub == "list":
return await list_base_models(limit=args.limit)
if cmd in ("metadata", "md") and sub == "read":
return await read_metadata(args.path)
if cmd in ("metadata", "md") and sub == "update":
updates: Dict[str, Any] = json.loads(args.json)
return await apply_metadata_updates(args.path, updates)
if cmd in ("preview", "pv") and sub == "download":
return await download_preview(args.path, args.url)
if cmd == "cache" and sub == "refresh":
return await refresh_cache(args.path)
raise ValueError(f"Unknown command: {cmd} {sub}")
def main() -> None:
parser = _build_parser()
args = parser.parse_args()
result = asyncio.run(_run(args))
# Always print as JSON so callers can parse reliably
if isinstance(result, list):
for item in result:
print(item)
elif isinstance(result, dict):
json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
print()
else:
print(json.dumps(result))
if __name__ == "__main__":
main()
+16 -1
View File
@@ -41,7 +41,22 @@ async def api_json_error(
if exc.status < 400:
raise
logger.warning(
# Preview 404 is routine (file deleted from disk) — not worth a warning.
logger_method = logger.warning
if request.path.startswith("/api/lm/previews") and exc.status == 404:
logger_method = logger.debug
# Download-progress 404 is routine too: in-memory tracking is removed
# once a download finishes/fails, so the extension's final polls 404.
# The extension relies on the 404 status itself (failure detection),
# so only the log level is lowered.
if (
request.path.startswith("/api/lm/download-progress/")
and exc.status == 404
):
logger_method = logger.debug
logger_method(
"API %s %s returned HTTP %d: %s",
request.method,
request.path,
+87 -8
View File
@@ -1,7 +1,8 @@
import logging
from typing import List, Tuple
import comfy.sd # type: ignore
import folder_paths # type: ignore
import os
from typing import Any, List, Tuple
import comfy.sd # pyright: ignore[reportMissingImports]
import folder_paths # pyright: ignore[reportMissingImports]
from ..utils.utils import get_checkpoint_info_absolute, _format_model_name_for_comfyui
logger = logging.getLogger(__name__)
@@ -12,20 +13,42 @@ class CheckpointLoaderLM:
Loads checkpoints from both standard ComfyUI folders and LoRA Manager's
extra folder paths, providing a unified interface for checkpoint loading.
The ckpt_name combo supports ComfyUI's control_after_generate, letting
users pick a random checkpoint on every run; the base_model input narrows
the random pool through a front-end extension that filters the combo
options.
"""
NAME = "Checkpoint Loader (LoraManager)"
CATEGORY = "Lora Manager/loaders"
@classmethod
def INPUT_TYPES(s):
def INPUT_TYPES(cls):
# Get list of checkpoint names from scanner (includes extra folder paths)
checkpoint_names = s._get_checkpoint_names()
checkpoint_names = cls._get_checkpoint_names()
base_models = cls._get_available_base_models()
return {
"required": {
"ckpt_name": (
checkpoint_names,
{"tooltip": "The name of the checkpoint (model) to load."},
{
"tooltip": (
"The name of the checkpoint (model) to load. Use "
"control_after_generate to pick a random model on "
"every run."
),
"control_after_generate": "fixed",
},
),
"base_model": (
base_models,
{
"default": "Any",
"tooltip": (
"Restrict the random selection pool to this base "
"model. 'Any' uses the full pool."
),
},
),
}
}
@@ -58,7 +81,10 @@ class CheckpointLoaderLM:
for item in cache.raw_data:
if item.get("sub_type") == "checkpoint":
file_path = item.get("file_path", "")
if file_path:
# Only offer models that still exist on disk so ComfyUI
# flags missing checkpoints at queue time via
# "value not in list" (the scanner cache can be stale).
if file_path and os.path.exists(file_path):
# Format using relative path with OS-native separator
formatted_name = _format_model_name_for_comfyui(
file_path, model_roots
@@ -89,15 +115,68 @@ class CheckpointLoaderLM:
logger.error(f"Error getting checkpoint names: {e}")
return []
def load_checkpoint(self, ckpt_name: str) -> Tuple:
@classmethod
def _get_available_base_models(cls) -> List[str]:
"""Get distinct base_model values present among indexed checkpoints, for the random-selection filter."""
try:
from ..services.service_registry import ServiceRegistry
async def _get_base_models():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
base_models = set()
for item in cache.raw_data:
if item.get("sub_type") != "checkpoint":
continue
base_model = item.get("base_model")
file_path = item.get("file_path", "")
if base_model and file_path and os.path.exists(file_path):
base_models.add(base_model)
return sorted(base_models)
return ["Any"] + cls._run_async(_get_base_models)
except Exception as e:
logger.error(f"Error getting available base models: {e}")
return ["Any"]
@staticmethod
def _run_async(coro_fn):
"""Run an async fetcher, handling the case where an event loop is already running."""
import asyncio
try:
asyncio.get_running_loop()
import concurrent.futures
def run_in_thread():
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
return new_loop.run_until_complete(coro_fn())
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
return future.result()
except RuntimeError:
return asyncio.run(coro_fn())
def load_checkpoint(
self, ckpt_name: str, base_model: str = "Any"
) -> Tuple[Any, Any, Any]:
"""Load a checkpoint by name, supporting extra folder paths
Args:
ckpt_name: The name of the checkpoint to load (relative path with extension)
base_model: Only used by the front-end to filter the random pool
Returns:
Tuple of (MODEL, CLIP, VAE)
"""
del base_model
# Get absolute path from cache using ComfyUI-style name
ckpt_path, metadata = get_checkpoint_info_absolute(ckpt_name)
+124
View File
@@ -0,0 +1,124 @@
"""Create Hook LoRA (LoraManager) — multi-LoRA hook node compatible with ComfyUI's built-in hook pipeline.
Produces ``("HOOKS",)`` output that chains seamlessly with downstream hook consumers
(ConditioningSetProperties, SetHookKeyframes, CombineHooks, SetClipHooks, etc.).
"""
from __future__ import annotations
import logging
import os
from ..utils.utils import get_lora_info_absolute
from .utils import (
FlexibleOptionalInputType,
any_type,
apply_lora_syntax_format,
get_loras_list,
validate_lora_entries,
)
logger = logging.getLogger(__name__)
class CreateHookLoraLM:
NAME = "Create Hook LoRA (LoraManager)"
CATEGORY = "Lora Manager/hooks"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"text": (
"AUTOCOMPLETE_TEXT_LORAS",
{
"placeholder": "Search LoRAs to add...",
"tooltip": (
"Search and select LoRAs. Each LoRA gets its own "
"model/clip strength. Hooks chain with prev_hooks."
),
},
),
"loras": ("LORAS", {}),
},
"optional": FlexibleOptionalInputType(any_type),
}
@classmethod
def VALIDATE_INPUTS(cls, loras=None):
"""Queue-time validation: reject missing local LoRAs before execution."""
return validate_lora_entries({"loras": loras}) or True
RETURN_TYPES = ("HOOKS", "STRING", "STRING")
RETURN_NAMES = ("HOOKS", "trigger_words", "active_loras")
FUNCTION = "create_hook"
def create_hook(self, text: str, loras, **kwargs):
"""Create a HookGroup from the selected LoRAs, chained with prev_hooks.
Each active LoRA from the widget is loaded and wrapped in a WeightHook
via :func:`comfy.hooks.create_hook_lora`. All hooks are combined into a
single group and returned alongside trigger words and a human-readable
summary of the active LoRAs.
"""
del text # used by the frontend widget only
# Lazy imports: comfy is not available in CI/test environment at module level
import comfy.hooks # pyright: ignore[reportMissingImports] # noqa: C0415
import comfy.utils # pyright: ignore[reportMissingImports] # noqa: C0415
prev_hooks: comfy.hooks.HookGroup | None = kwargs.get("prev_hooks")
hook_group = prev_hooks.clone() if prev_hooks is not None else comfy.hooks.HookGroup()
all_trigger_words: list[str] = []
active_loras: list[tuple[str, float, float]] = []
for lora in get_loras_list({"loras": loras}):
if not lora.get("active", False):
continue
lora_name = apply_lora_syntax_format(lora["name"])
model_strength = float(lora["strength"])
clip_strength = float(lora.get("clipStrength", model_strength))
# Skip useless no-op entries (both strengths are zero)
if model_strength == 0.0 and clip_strength == 0.0:
continue
lora_path, trigger_words = get_lora_info_absolute(lora_name)
if not lora_path or not os.path.isfile(lora_path):
logger.warning("LoRA '%s' not found — skipping", lora_name)
continue
try:
lora_weights = comfy.utils.load_torch_file(lora_path, safe_load=True)
lora_hooks = comfy.hooks.create_hook_lora(
lora=lora_weights,
strength_model=model_strength,
strength_clip=clip_strength,
)
except Exception:
logger.exception("Failed to load LoRA '%s' — skipping", lora_name)
continue
hook_group = hook_group.clone_and_combine(lora_hooks)
active_loras.append((lora_name, model_strength, clip_strength))
all_trigger_words.extend(trigger_words)
# Format trigger words (group mode separator)
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
# Format active LoRAs summary
formatted_loras = []
for name, model_s, clip_s in active_loras:
if abs(model_s - clip_s) > 0.001:
formatted_loras.append(
f"<lora:{name}:{model_s}:{clip_s}>"
)
else:
formatted_loras.append(f"<lora:{name}:{model_s}>")
active_loras_text = " ".join(formatted_loras)
return (hook_group, trigger_words_text, active_loras_text)
+45
View File
@@ -0,0 +1,45 @@
"""Lora Info display node — pure frontend node for showing selected LoRA info.
This node does NOT participate in workflow execution. Its single optional
"lora_source" input exists solely as a wire-connection anchor so that the
frontend can traverse the graph and push selection data to connected info nodes.
"""
from __future__ import annotations
class LoraInfoLM:
"""Display node that shows filename and notes for the selected LoRA."""
NAME = "Lora Info (LoraManager)"
CATEGORY = "Lora Manager/utils"
DESCRIPTION = (
"Displays information (filename, notes) about the currently selected "
"LoRA. Connect any output from a LoRA Loader or Stacker to the "
"lora_source input, then select a LoRA in the source widget — the "
"info updates automatically. Does not affect workflow execution."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {},
}
RETURN_TYPES = ()
RETURN_NAMES = ()
OUTPUT_NODE = False
FUNCTION = "noop"
def noop(self, **kwargs):
# This node is display-only — no workflow execution needed.
return ()
NODE_CLASS_MAPPINGS = {
LoraInfoLM.NAME: LoraInfoLM,
}
NODE_DISPLAY_NAME_MAPPINGS = {
LoraInfoLM.NAME: "Lora Info (LoraManager)",
}
+16 -24
View File
@@ -1,9 +1,8 @@
import importlib
import logging
import re
import comfy.sd # type: ignore
import comfy.utils # type: ignore
import comfy.sd # pyright: ignore[reportMissingImports]
import comfy.utils # pyright: ignore[reportMissingImports]
from ..utils.utils import get_lora_info_absolute
from .utils import (
@@ -14,6 +13,8 @@ from .utils import (
extract_lora_name,
get_loras_list,
nunchaku_load_lora,
parse_lora_syntax,
validate_lora_entries,
)
logger = logging.getLogger(__name__)
@@ -48,9 +49,9 @@ def _collect_stack_entries(lora_stack):
return entries
def _collect_widget_entries(kwargs):
def _collect_widget_entries(loras):
entries = []
for lora in get_loras_list(kwargs):
for lora in get_loras_list({"loras": loras}):
if not lora.get("active", False):
continue
lora_name = apply_lora_syntax_format(lora["name"])
@@ -138,20 +139,26 @@ class LoraLoaderLM:
"placeholder": "Search LoRAs to add...",
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
}),
"loras": ("LORAS", {}),
},
"optional": FlexibleOptionalInputType(any_type),
}
@classmethod
def VALIDATE_INPUTS(cls, loras=None):
"""Queue-time validation: reject missing local LoRAs before execution."""
return validate_lora_entries({"loras": loras}) or True
RETURN_TYPES = ("MODEL", "CLIP", "STRING", "STRING")
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras"
def load_loras(self, model, text, **kwargs):
"""Loads multiple LoRAs based on the kwargs input and lora_stack."""
def load_loras(self, model, text, loras, **kwargs):
"""Loads multiple LoRAs based on the widget input and lora_stack."""
del text
clip = kwargs.get("clip", None)
lora_entries = _collect_stack_entries(kwargs.get("lora_stack", None))
lora_entries.extend(_collect_widget_entries(kwargs))
lora_entries.extend(_collect_widget_entries(loras))
nunchaku_model_kind = detect_nunchaku_model_kind(model)
if nunchaku_model_kind == "flux":
@@ -189,25 +196,10 @@ class LoraTextLoaderLM:
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras_from_text"
def parse_lora_syntax(self, text):
"""Parse LoRA syntax from text input."""
pattern = r"<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>"
matches = re.findall(pattern, text, re.IGNORECASE)
loras = []
for match in matches:
model_strength = float(match[1])
loras.append({
"name": match[0],
"model_strength": model_strength,
"clip_strength": float(match[2]) if match[2] else model_strength,
})
return loras
def load_loras_from_text(self, model, lora_syntax, clip=None, lora_stack=None):
"""Load LoRAs based on text syntax input."""
lora_entries = _collect_stack_entries(lora_stack)
for lora in self.parse_lora_syntax(lora_syntax):
for lora in parse_lora_syntax(lora_syntax):
lora_path, trigger_words = get_lora_info_absolute(lora["name"])
lora_entries.append({
"name": lora["name"],
+6
View File
@@ -9,6 +9,7 @@ and tracks the last used combination for reuse.
import logging
import os
from ..utils.utils import get_lora_info
from .utils import validate_lora_entries
logger = logging.getLogger(__name__)
@@ -31,6 +32,11 @@ class LoraRandomizerLM:
},
}
@classmethod
def VALIDATE_INPUTS(cls, loras=None):
"""Queue-time validation: reject missing local LoRAs before execution."""
return validate_lora_entries({"loras": loras}) or True
RETURN_TYPES = ("LORA_STACK",)
RETURN_NAMES = ("LORA_STACK",)
+86 -10
View File
@@ -1,26 +1,102 @@
from __future__ import annotations
import inspect
import re
from typing import Any
_STACK_INPUT_PATTERN = re.compile(r"^lora_stack(?:_([ab])|(\d+))$")
def _is_stack_input(name: str) -> bool:
return bool(_STACK_INPUT_PATTERN.match(name))
def _stack_slot_number(name: str) -> int:
"""Numeric slot used to order stack inputs; legacy a/b map to 1/2."""
match = _STACK_INPUT_PATTERN.match(name)
if not match:
return -1
letter, digits = match.group(1), match.group(2)
if digits is not None:
return int(digits)
return 1 if letter == "a" else 2
class _LoraStackOptionalInputs:
"""Lookup that preserves explicit optional inputs and dynamic lora_stack slots."""
def __init__(self, explicit_inputs: dict[str, tuple[str, dict[str, Any]]]) -> None:
self._explicit_inputs = explicit_inputs
def __contains__(self, item: object) -> bool:
if not isinstance(item, str):
return False
return item in self._explicit_inputs or _is_stack_input(item)
def __getitem__(self, key: str) -> tuple[str, dict[str, Any]]:
if key in self._explicit_inputs:
return self._explicit_inputs[key]
if _is_stack_input(key):
return (
"LORA_STACK",
{
"tooltip": "A LoRA stack to combine. Connect to add more inputs.",
},
)
raise KeyError(key)
class LoraStackCombinerLM:
NAME = "Lora Stack Combiner (LoraManager)"
CATEGORY = "Lora Manager/stackers"
DESCRIPTION = (
"Combines multiple LoRA stacks into a single stack. "
"Supports dynamic inputs: connect a stack to add more inputs."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"lora_stack_a": ("LORA_STACK",),
"lora_stack_b": ("LORA_STACK",),
optional_inputs: dict[str, tuple[str, dict[str, Any]]] = {
"lora_stack1": (
"LORA_STACK",
{
"tooltip": "A LoRA stack to combine. Connect to add more inputs.",
},
),
"lora_stack2": (
"LORA_STACK",
{
"tooltip": "A LoRA stack to combine. Connect to add more inputs.",
},
),
}
stack = inspect.stack()
if len(stack) > 2 and stack[2].function == "get_input_info":
optional_inputs = _LoraStackOptionalInputs(optional_inputs) # pyright: ignore[reportAssignmentType]
return {
"required": {},
"optional": optional_inputs,
}
RETURN_TYPES = ("LORA_STACK",)
RETURN_NAMES = ("LORA_STACK",)
FUNCTION = "combine_stacks"
def combine_stacks(self, lora_stack_a, lora_stack_b):
combined_stack = []
def combine_stacks(self, lora_stack1=None, lora_stack2=None, **kwargs):
stacks = {
"lora_stack1": lora_stack1,
"lora_stack2": lora_stack2,
}
for key, value in kwargs.items():
if _is_stack_input(key) and value is not None:
stacks[key] = value
if lora_stack_a:
combined_stack.extend(lora_stack_a)
if lora_stack_b:
combined_stack.extend(lora_stack_b)
combined_stack = []
for key in sorted(stacks, key=_stack_slot_number):
stack = stacks[key]
if stack:
combined_stack.extend(stack)
return (combined_stack,)
+11 -5
View File
@@ -1,6 +1,6 @@
import os
from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, apply_lora_syntax_format, extract_lora_name, get_loras_list
from .utils import FlexibleOptionalInputType, any_type, apply_lora_syntax_format, extract_lora_name, get_loras_list, validate_lora_entries
import logging
@@ -18,16 +18,22 @@ class LoraStackerLM:
"placeholder": "Search LoRAs to add...",
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
}),
"loras": ("LORAS", {}),
},
"optional": FlexibleOptionalInputType(any_type),
}
@classmethod
def VALIDATE_INPUTS(cls, loras=None):
"""Queue-time validation: reject missing local LoRAs before execution."""
return validate_lora_entries({"loras": loras}) or True
RETURN_TYPES = ("LORA_STACK", "STRING", "STRING")
RETURN_NAMES = ("LORA_STACK", "trigger_words", "active_loras")
FUNCTION = "stack_loras"
def stack_loras(self, text, **kwargs):
"""Stacks multiple LoRAs based on the kwargs input without loading them."""
def stack_loras(self, text, loras, **kwargs):
"""Stacks multiple LoRAs based on the widget input without loading them."""
stack = []
active_loras = []
all_trigger_words = []
@@ -42,8 +48,8 @@ class LoraStackerLM:
_, trigger_words = get_lora_info(lora_name)
all_trigger_words.extend(trigger_words)
# Process loras from kwargs with support for both old and new formats
loras_list = get_loras_list(kwargs)
# Process loras from the widget with support for both old and new formats
loras_list = get_loras_list({"loras": loras})
for lora in loras_list:
if not lora.get('active', False):
continue
+62
View File
@@ -0,0 +1,62 @@
"""Node to resolve `<lora:name:strength>` syntax to absolute file system paths.
Takes the loaded_loras / active_loras STRING output from LoraLoaderLM or
LoraStackerLM and resolves each lora name to its absolute path on disk via
the scanner cache. Unknown names are returned as-is.
"""
import logging
from ..utils.utils import get_lora_info_absolute
from .utils import parse_lora_syntax
logger = logging.getLogger(__name__)
class LoraSyntaxToPath:
NAME = "LoRA Syntax → Path (LoraManager)"
CATEGORY = "Lora Manager/utils"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"lora_syntax": (
"STRING",
{
"forceInput": True,
"multiline": True,
"tooltip": (
"<lora:name:strength> formatted text from "
"loaded_loras / active_loras output"
),
},
),
},
}
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("paths",)
FUNCTION = "resolve"
def resolve(self, lora_syntax: str) -> tuple[str]:
"""Parse <lora:...> syntax and resolve each name to its absolute path."""
if not lora_syntax or not lora_syntax.strip():
logger.info("Received empty lora_syntax input")
return ("",)
parsed = parse_lora_syntax(lora_syntax)
if not parsed:
logger.info("No valid <lora:...> entries found in input")
return ("",)
paths: list[str] = []
for entry in parsed:
try:
absolute_path, _ = get_lora_info_absolute(entry["name"])
paths.append(absolute_path)
except Exception:
logger.warning("Failed to resolve lora '%s', skipping", entry["name"])
continue
return ("\n".join(paths),)
+179
View File
@@ -0,0 +1,179 @@
"""Metadata Overwrite node — allows users to manually specify generation parameters
that override the automatically collected/inferred metadata.
Most inputs have falsy defaults (empty string / 0) which are skipped.
clip_skip uses a sentinel default (-25) so that a wired value of 0 is
preserved both ComfyUI and A1111 conventions have no meaningful 0 value,
but users may wire 0 to express "no clip skip / default".
"""
from typing import Any
from ..metadata_collector.constants import CLIP_SKIP_SENTINEL as _CLIP_SKIP_SENTINEL
from ..metadata_collector.overwrite_utils import collect_overwrite_params
class MetadataOverwriteLM:
NAME = "Metadata Overwrite (LoraManager)"
CATEGORY = "Lora Manager/utils"
DESCRIPTION = (
"Manually specify generation parameters to override automatically collected "
"metadata. Only filled/connected inputs will take effect — empty defaults "
"are ignored."
)
@classmethod
def INPUT_TYPES(cls) -> dict[str, Any]:
return {
"optional": {
"prompt": (
"STRING",
{
"default": "",
"multiline": True,
"tooltip": "Positive prompt. Only overwrites when non-empty.",
},
),
"negative_prompt": (
"STRING",
{
"default": "",
"multiline": True,
"tooltip": "Negative prompt. Only overwrites when non-empty.",
},
),
"seed": (
"INT",
{
"default": 0,
"min": 0,
"max": 0xFFFFFFFFFFFFFFFF,
"control_after_generate": False,
"tooltip": "Seed value. Only overwrites when > 0.",
},
),
"steps": (
"INT",
{
"default": 0,
"min": 0,
"max": 10000,
"tooltip": "Number of steps. Only overwrites when > 0.",
},
),
"cfg_scale": (
"FLOAT",
{
"default": 0.0,
"min": 0.0,
"max": 100.0,
"tooltip": "CFG scale. Only overwrites when > 0.",
},
),
"sampler": (
"STRING,SAMPLER",
{
"default": "",
"widgetType": "STRING",
"tooltip": (
"Sampler name. Fill in the name manually or "
"connect a SAMPLER output (e.g. KSamplerSelect) "
"— the sampler name is then extracted "
"automatically. Note: ddim is recorded as "
"euler (ComfyUI internal representation). "
"Only overwrites when non-empty."
),
},
),
"scheduler": (
"STRING",
{
"default": "",
"tooltip": "Scheduler name. Only overwrites when non-empty.",
},
),
"model": (
"STRING,MODEL",
{
"default": "",
"widgetType": "STRING",
"tooltip": (
"The checkpoint or diffusion model (UNet) used "
"for generation. Fill in the name manually or "
"connect a MODEL output — the model name is then "
"extracted automatically. Only overwrites when "
"non-empty."
),
},
),
"loras": (
"STRING",
{
"default": "",
"multiline": True,
"tooltip": (
"LoRA syntax, e.g. <lora:name:strength> "
"or <lora:name:model_strength:clip_strength>, "
"separated by spaces. Only overwrites when non-empty."
),
},
),
"size": (
"STRING",
{
"default": "",
"tooltip": (
"Image size in WIDTHxHEIGHT format (e.g. 512x768). "
"Only overwrites when non-empty."
),
},
),
"clip_skip": (
"INT",
{
"default": _CLIP_SKIP_SENTINEL,
"min": -25,
"max": 24,
"tooltip": (
"Clip skip (ComfyUI: -24..-1, A1111: 1+). "
"Default -25 means not set — any other value "
"overwrites."
),
},
),
"additional_data": (
"STRING",
{
"default": "",
"multiline": True,
"tooltip": (
"Additional data to embed in the image metadata. "
"Inserted between Clip skip and Model hash in the "
"A1111-compatible parameters string. "
'Example: "Copyright": "Some license info"'
),
},
),
},
}
RETURN_TYPES = ("METADATA",)
RETURN_NAMES = ("metadata",)
FUNCTION = "collect_metadata"
OUTPUT_NODE = True
def collect_metadata(self, **kwargs: Any) -> tuple[dict[str, Any]]:
"""Collect non-default input values into a metadata dict.
For most fields, a falsy value (empty string, 0) means "not set"
and is skipped. clip_skip uses a dedicated sentinel (-25) so that
a wired value of 0 is preserved and reaches the metadata pipeline.
The ``model`` field accepts either a manual string or a wired MODEL
(ModelPatcher) connection; in the latter case the underlying model
name is extracted from the patcher's ``cached_patcher_init`` and
stored as a ComfyUI-style relative path. The ``sampler`` field
likewise accepts a manual string or a wired SAMPLER (KSAMPLER)
connection, from which the sampler name is extracted automatically.
"""
return (collect_overwrite_params(kwargs),)
+12 -13
View File
@@ -15,15 +15,15 @@ import os
import re
from collections import defaultdict
from pathlib import Path
from typing import Dict, List, Optional, Tuple, Union
from typing import Any, Dict, List, Optional, Tuple, Union, cast
import comfy.utils # type: ignore
import folder_paths # type: ignore
import comfy.utils # pyright: ignore[reportMissingImports]
import folder_paths # pyright: ignore[reportMissingImports]
import torch
import torch.nn as nn
from safetensors import safe_open
from nunchaku.lora.flux.nunchaku_converter import (
from nunchaku.lora.flux.nunchaku_converter import ( # pyright: ignore[reportMissingTypeStubs]
pack_lowrank_weight,
unpack_lowrank_weight,
)
@@ -87,10 +87,6 @@ def _rename_layer_underscore_layer_name(old_name: str) -> str:
return new_name
def _is_indexable_module(module):
return isinstance(module, (nn.ModuleList, nn.Sequential, list, tuple))
def _get_module_by_name(model: nn.Module, name: str) -> Optional[nn.Module]:
if not name:
return model
@@ -100,7 +96,7 @@ def _get_module_by_name(model: nn.Module, name: str) -> Optional[nn.Module]:
continue
if hasattr(module, part):
module = getattr(module, part)
elif part.isdigit() and _is_indexable_module(module):
elif part.isdigit() and isinstance(module, (nn.ModuleList, nn.Sequential, list, tuple)):
try:
module = module[int(part)]
except (IndexError, TypeError):
@@ -267,7 +263,9 @@ def _handle_proj_out_split(lora_dict: Dict[str, Dict[str, torch.Tensor]], base_k
return result, consumed
def _apply_lora_to_module(module: nn.Module, a_tensor: torch.Tensor, b_tensor: torch.Tensor, module_name: str, model: nn.Module) -> None:
def _apply_lora_to_module(module: Any, a_tensor: torch.Tensor, b_tensor: torch.Tensor, module_name: str, model: Any) -> None:
# These modules are dynamic torch containers; monkey-patched attributes
# below are set at runtime, so the module/model types are deliberately Any.
if not hasattr(module, "in_features") or not hasattr(module, "out_features"):
raise ValueError(f"{module_name}: unsupported module without in/out features")
if a_tensor.shape[1] != module.in_features or b_tensor.shape[0] != module.out_features:
@@ -336,7 +334,7 @@ def _apply_lora_to_module(module: nn.Module, a_tensor: torch.Tensor, b_tensor: t
raise ValueError(f"{module_name}: unsupported module type {type(module)}")
def reset_lora_v2(model: nn.Module) -> None:
def reset_lora_v2(model: Any) -> None:
slots = getattr(model, "_lora_slots", None)
if not slots:
return
@@ -344,6 +342,7 @@ def reset_lora_v2(model: nn.Module) -> None:
module = _get_module_by_name(model, name)
if module is None:
continue
module = cast(Any, module)
module_type = info.get("type", "nunchaku")
if module_type == "nunchaku":
base_rank = info["base_rank"]
@@ -371,7 +370,7 @@ def reset_lora_v2(model: nn.Module) -> None:
def compose_loras_v2(model: nn.Module, lora_configs: List[Tuple[Union[str, Path, Dict[str, torch.Tensor]], float]], apply_awq_mod: bool = True) -> bool:
del apply_awq_mod # retained for interface compatibility
reset_lora_v2(model)
aggregated_weights: Dict[str, List[Dict[str, object]]] = defaultdict(list)
aggregated_weights: Dict[str, List[Dict[str, Any]]] = defaultdict(list)
saw_supported_format = False
unresolved_targets = 0
@@ -471,7 +470,7 @@ def compose_loras_v2(model: nn.Module, lora_configs: List[Tuple[Union[str, Path,
class ComfyQwenImageWrapperLM(nn.Module):
def __init__(self, model: nn.Module, config=None, apply_awq_mod: bool = True):
super().__init__()
self.model = model
self.model: Any = model
self.config = {} if config is None else config
self.dtype = next(model.parameters()).dtype
self.loras: List[Tuple[Union[str, Path, Dict[str, torch.Tensor]], float]] = []
+2 -2
View File
@@ -67,7 +67,7 @@ class PromptLM:
stack = inspect.stack()
if len(stack) > 2 and stack[2].function == "get_input_info":
optional_inputs = _PromptOptionalInputs(optional_inputs) # type: ignore[assignment]
optional_inputs = _PromptOptionalInputs(optional_inputs) # pyright: ignore[reportAssignmentType]
return {
"required": {
@@ -126,7 +126,7 @@ class PromptLM:
else:
prompt = expanded_text
from nodes import CLIPTextEncode # type: ignore
from nodes import CLIPTextEncode # pyright: ignore[reportMissingImports, reportAttributeAccessIssue]
conditioning = CLIPTextEncode().encode(clip, prompt)[0]
return (conditioning, prompt)
+214
View File
@@ -0,0 +1,214 @@
import logging
import os
import random
from typing import Any, List, Optional, Tuple
import comfy.sd # pyright: ignore[reportMissingImports]
import folder_paths # pyright: ignore[reportMissingImports]
from ..utils.utils import get_checkpoint_info_absolute, _format_model_name_for_comfyui
logger = logging.getLogger(__name__)
class RandomCheckpointLoaderLM:
"""Checkpoint Loader that can randomly pick a checkpoint from the pool
Loads checkpoints from both standard ComfyUI folders and LoRA Manager's
extra folder paths. When select_at_random is enabled, ignores ckpt_name
and picks a random checkpoint (optionally filtered by base_model) on
every run.
"""
NAME = "Random Checkpoint Loader (LoraManager)"
CATEGORY = "Lora Manager/loaders"
@classmethod
def INPUT_TYPES(cls):
# Get list of checkpoint names from scanner (includes extra folder paths)
checkpoint_names = cls._get_checkpoint_names()
base_models = cls._get_available_base_models()
return {
"required": {
"ckpt_name": (
checkpoint_names,
{"tooltip": "The name of the checkpoint (model) to load."},
),
"select_at_random": (
"BOOLEAN",
{
"default": False,
"tooltip": (
"Ignore ckpt_name and pick a random checkpoint from the "
"pool (optionally filtered by base_model) on every run."
),
},
),
"base_model": (
base_models,
{
"default": "Any",
"tooltip": "Restrict random selection to this base model. 'Any' uses the full pool.",
},
),
}
}
RETURN_TYPES = ("MODEL", "CLIP", "VAE", "STRING")
RETURN_NAMES = ("MODEL", "CLIP", "VAE", "model_name")
OUTPUT_TOOLTIPS = (
"The model used for denoising latents.",
"The CLIP model used for encoding text prompts.",
"The VAE model used for encoding and decoding images to and from latent space.",
"The name of the checkpoint that was loaded (useful when select_at_random is enabled).",
)
FUNCTION = "load_checkpoint"
@classmethod
def IS_CHANGED(cls, ckpt_name, select_at_random=False, base_model="Any"):
# Force re-execution on every run while randomizing, since the widget
# values themselves don't change between queue runs.
if select_at_random:
return float("nan")
return ckpt_name
@staticmethod
def _run_async(coro_fn):
"""Run an async fetcher, handling the case where an event loop is already running."""
import asyncio
try:
asyncio.get_running_loop()
import concurrent.futures
def run_in_thread():
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
return new_loop.run_until_complete(coro_fn())
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
return future.result()
except RuntimeError:
return asyncio.run(coro_fn())
@classmethod
def _get_checkpoint_names(cls, base_model: Optional[str] = None) -> List[str]:
"""Get list of checkpoint names from scanner cache in ComfyUI format (relative path with extension)
Args:
base_model: If given (and not "Any"), only include checkpoints matching this base model.
"""
try:
from ..services.service_registry import ServiceRegistry
async def _get_names():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
# Get all model roots for calculating relative paths
model_roots = scanner.get_model_roots()
# Filter only checkpoint type (not diffusion_model) and format names
names = []
for item in cache.raw_data:
if item.get("sub_type") != "checkpoint":
continue
if (
base_model
and base_model != "Any"
and item.get("base_model") != base_model
):
continue
file_path = item.get("file_path", "")
# Only offer models that still exist on disk so ComfyUI
# flags missing checkpoints at queue time via
# "value not in list" (the scanner cache can be stale).
if file_path and os.path.exists(file_path):
# Format using relative path with OS-native separator
formatted_name = _format_model_name_for_comfyui(
file_path, model_roots
)
if formatted_name:
names.append(formatted_name)
return sorted(names)
return cls._run_async(_get_names)
except Exception as e:
logger.error(f"Error getting checkpoint names: {e}")
return []
@classmethod
def _get_available_base_models(cls) -> List[str]:
"""Get distinct base_model values present among indexed checkpoints, for the random-selection filter."""
try:
from ..services.service_registry import ServiceRegistry
async def _get_base_models():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
base_models = set()
for item in cache.raw_data:
if item.get("sub_type") != "checkpoint":
continue
base_model = item.get("base_model")
file_path = item.get("file_path", "")
if base_model and file_path and os.path.exists(file_path):
base_models.add(base_model)
return sorted(base_models)
return ["Any"] + cls._run_async(_get_base_models)
except Exception as e:
logger.error(f"Error getting available base models: {e}")
return ["Any"]
def load_checkpoint(
self,
ckpt_name: str,
select_at_random: bool = False,
base_model: str = "Any",
) -> Tuple[Any, Any, Any, str]:
"""Load a checkpoint by name, supporting extra folder paths
Args:
ckpt_name: The name of the checkpoint to load (relative path with extension)
select_at_random: If True, ignore ckpt_name and pick randomly from the pool
base_model: Restricts random selection to this base model ("Any" = no filter)
Returns:
Tuple of (MODEL, CLIP, VAE, model_name)
"""
if select_at_random:
pool = self._get_checkpoint_names(base_model)
if not pool:
raise FileNotFoundError(
f"No checkpoints found for base model '{base_model}'. "
"Pick a different base model or disable 'select_at_random'."
)
ckpt_name = random.choice(pool)
logger.info(
f"[RandomCheckpointLoaderLM] Randomly selected checkpoint: {ckpt_name}"
)
# Get absolute path from cache using ComfyUI-style name
ckpt_path, metadata = get_checkpoint_info_absolute(ckpt_name)
if metadata is None:
raise FileNotFoundError(
f"Checkpoint '{ckpt_name}' not found in LoRA Manager cache. "
"Make sure the checkpoint is indexed and try again."
)
# Load regular checkpoint using ComfyUI's API
logger.info(f"Loading checkpoint from: {ckpt_path}")
out = comfy.sd.load_checkpoint_guess_config(
ckpt_path,
output_vae=True,
output_clip=True,
embedding_directory=folder_paths.get_folder_paths("embeddings"),
)
return out[:3] + (ckpt_name,)
+326
View File
@@ -0,0 +1,326 @@
import logging
import os
import random
from typing import Any, List, Optional, Tuple
import comfy.sd # pyright: ignore[reportMissingImports]
from ..utils.utils import get_checkpoint_info_absolute, _format_model_name_for_comfyui
logger = logging.getLogger(__name__)
def _reload_gguf_unet(
unet_path: str, weight_dtype: str, disable_dynamic: bool = False
) -> object:
"""Reload a GGUF diffusion model from disk (cached_patcher_init factory).
Mirrors the GGUF branch of RandomUNETLoaderLM.load_unet so ModelPatcher
deepclone/dynamic machinery can rebuild GGUF models with the correct
GGMLOps. ``disable_dynamic`` is accepted for signature compatibility
with core ComfyUI loaders.
"""
loader = RandomUNETLoaderLM()
model, _unet_name = loader._load_gguf_unet(unet_path, unet_path, weight_dtype)
return model
class RandomUNETLoaderLM:
"""UNET Loader that can randomly pick a diffusion model from the pool
Loads diffusion models/UNets from both standard ComfyUI folders and LoRA
Manager's extra folder paths. Supports both regular diffusion models and
GGUF format models. When select_at_random is enabled, ignores unet_name
and picks a random diffusion model (optionally filtered by base_model)
on every run.
"""
NAME = "Random Unet Loader (LoraManager)"
CATEGORY = "Lora Manager/loaders"
@classmethod
def INPUT_TYPES(cls):
# Get list of unet names from scanner (includes extra folder paths)
unet_names = cls._get_unet_names()
base_models = cls._get_available_base_models()
return {
"required": {
"unet_name": (
unet_names,
{"tooltip": "The name of the diffusion model to load."},
),
"weight_dtype": (
["default", "fp8_e4m3fn", "fp8_e4m3fn_fast", "fp8_e5m2"],
{"tooltip": "The dtype to use for the model weights."},
),
"select_at_random": (
"BOOLEAN",
{
"default": False,
"tooltip": (
"Ignore unet_name and pick a random diffusion model from "
"the pool (optionally filtered by base_model) on every run."
),
},
),
"base_model": (
base_models,
{
"default": "Any",
"tooltip": "Restrict random selection to this base model. 'Any' uses the full pool.",
},
),
}
}
RETURN_TYPES = ("MODEL", "STRING")
RETURN_NAMES = ("MODEL", "model_name")
OUTPUT_TOOLTIPS = (
"The model used for denoising latents.",
"The name of the diffusion model that was loaded (useful when select_at_random is enabled).",
)
FUNCTION = "load_unet"
@classmethod
def IS_CHANGED(
cls, unet_name, weight_dtype, select_at_random=False, base_model="Any"
):
# Force re-execution on every run while randomizing, since the widget
# values themselves don't change between queue runs.
if select_at_random:
return float("nan")
return unet_name
@staticmethod
def _run_async(coro_fn):
"""Run an async fetcher, handling the case where an event loop is already running."""
import asyncio
try:
asyncio.get_running_loop()
import concurrent.futures
def run_in_thread():
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
return new_loop.run_until_complete(coro_fn())
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
return future.result()
except RuntimeError:
return asyncio.run(coro_fn())
@classmethod
def _get_unet_names(cls, base_model: Optional[str] = None) -> List[str]:
"""Get list of diffusion model names from scanner cache in ComfyUI format (relative path with extension)
Args:
base_model: If given (and not "Any"), only include models matching this base model.
"""
try:
from ..services.service_registry import ServiceRegistry
async def _get_names():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
# Get all model roots for calculating relative paths
model_roots = scanner.get_model_roots()
# Filter only diffusion_model type and format names
names = []
for item in cache.raw_data:
if item.get("sub_type") != "diffusion_model":
continue
if (
base_model
and base_model != "Any"
and item.get("base_model") != base_model
):
continue
file_path = item.get("file_path", "")
# Only offer models that still exist on disk so ComfyUI
# flags missing diffusion models at queue time via
# "value not in list" (the scanner cache can be stale).
if file_path and os.path.exists(file_path):
# Format using relative path with OS-native separator
formatted_name = _format_model_name_for_comfyui(
file_path, model_roots
)
if formatted_name:
names.append(formatted_name)
return sorted(names)
return cls._run_async(_get_names)
except Exception as e:
logger.error(f"Error getting unet names: {e}")
return []
@classmethod
def _get_available_base_models(cls) -> List[str]:
"""Get distinct base_model values present among indexed diffusion models, for the random-selection filter."""
try:
from ..services.service_registry import ServiceRegistry
async def _get_base_models():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
base_models = set()
for item in cache.raw_data:
if item.get("sub_type") != "diffusion_model":
continue
base_model = item.get("base_model")
file_path = item.get("file_path", "")
if base_model and file_path and os.path.exists(file_path):
base_models.add(base_model)
return sorted(base_models)
return ["Any"] + cls._run_async(_get_base_models)
except Exception as e:
logger.error(f"Error getting available base models: {e}")
return ["Any"]
def load_unet(
self,
unet_name: str,
weight_dtype: str,
select_at_random: bool = False,
base_model: str = "Any",
) -> Tuple[Any, ...]:
"""Load a diffusion model by name, supporting extra folder paths
Args:
unet_name: The name of the diffusion model to load (relative path with extension)
weight_dtype: The dtype to use for model weights
select_at_random: If True, ignore unet_name and pick randomly from the pool
base_model: Restricts random selection to this base model ("Any" = no filter)
Returns:
Tuple of (MODEL, model_name)
"""
import torch
if select_at_random:
pool = self._get_unet_names(base_model)
if not pool:
raise FileNotFoundError(
f"No diffusion models found for base model '{base_model}'. "
"Pick a different base model or disable 'select_at_random'."
)
unet_name = random.choice(pool)
logger.info(
f"[RandomUNETLoaderLM] Randomly selected diffusion model: {unet_name}"
)
# Get absolute path from cache using ComfyUI-style name
unet_path, metadata = get_checkpoint_info_absolute(unet_name)
if metadata is None:
raise FileNotFoundError(
f"Diffusion model '{unet_name}' not found in LoRA Manager cache. "
"Make sure the model is indexed and try again."
)
# Check if it's a GGUF model
if unet_path.endswith(".gguf"):
return self._load_gguf_unet(unet_path, unet_name, weight_dtype)
# Load regular diffusion model using ComfyUI's API
logger.info(f"Loading diffusion model from: {unet_path}")
# Build model options based on weight_dtype
model_options = {}
if weight_dtype == "fp8_e4m3fn":
model_options["dtype"] = torch.float8_e4m3fn
elif weight_dtype == "fp8_e4m3fn_fast":
model_options["dtype"] = torch.float8_e4m3fn
model_options["fp8_optimizations"] = True
elif weight_dtype == "fp8_e5m2":
model_options["dtype"] = torch.float8_e5m2
model = comfy.sd.load_diffusion_model(unet_path, model_options=model_options)
return (model, unet_name)
def _load_gguf_unet(
self, unet_path: str, unet_name: str, weight_dtype: str
) -> Tuple[Any, ...]:
"""Load a GGUF format diffusion model
Args:
unet_path: Absolute path to the GGUF file
unet_name: Name of the model for error messages
weight_dtype: The dtype to use for model weights
Returns:
Tuple of (MODEL, model_name)
"""
import torch
from .gguf_import_helper import get_gguf_modules
# Get ComfyUI-GGUF modules using helper (handles various import scenarios)
try:
loader_module, ops_module, nodes_module = get_gguf_modules()
gguf_sd_loader = getattr(loader_module, "gguf_sd_loader")
GGMLOps = getattr(ops_module, "GGMLOps")
GGUFModelPatcher = getattr(nodes_module, "GGUFModelPatcher")
except RuntimeError as e:
raise RuntimeError(f"Cannot load GGUF model '{unet_name}'. {str(e)}")
logger.info(f"Loading GGUF diffusion model from: {unet_path}")
try:
# Load GGUF state dict
sd, extra = gguf_sd_loader(unet_path)
# Prepare kwargs for metadata if supported
kwargs = {}
import inspect
valid_params = inspect.signature(
comfy.sd.load_diffusion_model_state_dict
).parameters
if "metadata" in valid_params:
kwargs["metadata"] = extra.get("metadata", {})
# Setup custom operations with GGUF support
ops = GGMLOps()
# Handle weight_dtype for GGUF models
if weight_dtype in ("default", None):
ops.Linear.dequant_dtype = None
elif weight_dtype in ["target"]:
ops.Linear.dequant_dtype = weight_dtype
else:
ops.Linear.dequant_dtype = getattr(torch, weight_dtype, None)
# Load the model
model = comfy.sd.load_diffusion_model_state_dict(
sd, model_options={"custom_operations": ops}, **kwargs
)
if model is None:
raise RuntimeError(
f"Could not detect model type for GGUF diffusion model: {unet_path}"
)
# Wrap with GGUFModelPatcher
model = GGUFModelPatcher.clone(model)
# Register a reload factory so the MODEL carries its source path
# (cached_patcher_init) like core ComfyUI loaders do — required
# for model-name extraction downstream and for ModelPatcher
# deepclone/dynamic machinery.
model.cached_patcher_init = (_reload_gguf_unet, (unet_path, weight_dtype))
return (model, unet_name)
except Exception as e:
logger.error(f"Error loading GGUF diffusion model '{unet_name}': {e}")
raise RuntimeError(
f"Failed to load GGUF diffusion model '{unet_name}': {str(e)}"
)
+360 -120
View File
@@ -5,7 +5,7 @@ import time
import uuid
from typing import Any, Dict, Optional
import numpy as np
import folder_paths # type: ignore
import folder_paths # pyright: ignore[reportMissingImports]
from ..services.service_registry import ServiceRegistry
from ..metadata_collector.metadata_processor import MetadataProcessor
from ..metadata_collector import get_metadata
@@ -13,9 +13,159 @@ from ..utils.constants import CARD_PREVIEW_WIDTH
from ..utils.exif_utils import ExifUtils
from ..utils.utils import calculate_recipe_fingerprint, sanitize_folder_name
from PIL import Image, PngImagePlugin
import piexif
import piexif # pyright: ignore[reportMissingTypeStubs]
import logging
# Civitai-compatible sampler name mapping: ComfyUI internal → A1111 display name
CIVITAI_SAMPLER_MAP = {
"euler": "Euler",
"euler_ancestral": "Euler a",
"lms": "LMS",
"heun": "Heun",
"dpm_2": "DPM2",
"dpm_2_ancestral": "DPM2 a",
"dpmpp_2s_ancestral": "DPM++ 2S a",
"dpmpp_2m": "DPM++ 2M",
"dpmpp_sde": "DPM++ SDE",
"dpmpp_sde_gpu": "DPM++ SDE",
"dpmpp_2m_sde": "DPM++ 2M SDE",
"dpmpp_2m_sde_gpu": "DPM++ 2M SDE",
"dpmpp_3m_sde": "DPM++ 3M SDE",
"dpm_fast": "DPM fast",
"dpm_adaptive": "DPM adaptive",
"ddim": "DDIM",
"plms": "PLMS",
"uni_pc_bh2": "UniPC",
"uni_pc": "UniPC",
"lcm": "LCM",
}
# Base model display name → AIR URN slug
# Sourced from civitai source: src/shared/constants/basemodel.constants.ts
BASE_MODEL_AIR_SLUG = {
# Stable Diffusion family
"SD 1.4": "sd1",
"SD 1.5": "sd1",
"SD 1.5 LCM": "sd1",
"SD 1.5 Hyper": "sd1",
"SD 2.0": "sd2",
"SD 2.0 768": "sd2",
"SD 2.1": "sd2",
"SD 2.1 768": "sd2",
"SD 2.1 Unclip": "sd2",
"SD 3.0": "sd3",
"SD 3.5": "sd35",
"SD 3.5 Large": "sd35",
"SD 3.5 Large Turbo": "sd35",
"SD 3.5 Medium": "sd35",
"SDXL 0.9": "sdxl",
"SDXL 1.0": "sdxl",
"SDXL 1.0 LCM": "sdxl",
"SDXL Lightning": "sdxl",
"SDXL Hyper": "sdxl",
"SDXL Turbo": "sdxl",
"SDXL Distilled": "sdxldistilled",
"Stable Cascade": "scascade",
"Stable Video Diffusion": "svd",
"SVD": "svd",
"SVD XT": "svdxt",
# SDXL community fine-tunes
"Pony": "pony",
"Pony Diffusion": "pony",
"Illustrious": "illustrious",
"NoobAI": "noobai",
"Animagine": "illustrious",
# Flux family
"Flux.1": "flux1",
"Flux.1 D": "flux1",
"Flux.1 S": "flux1",
"Flux.1 Krea": "fluxkrea",
"Flux.1 Kontext": "flux1kontext",
"Flux.2": "flux2",
"Flux.2 D": "flux2",
"Flux.2 Klein 9B": "flux2klein_9b",
"Flux.2 Klein 9B Base": "flux2klein_9b_base",
"Flux.2 Klein 4B": "flux2klein_4b",
"Flux.2 Klein 4B Base": "flux2klein_4b_base",
# Other image models (sorted alphabetically)
"AuraFlow": "auraflow",
"Chroma": "chroma",
"HiDream": "hidream",
"HiDream-O1": "hidream-o1",
"Hunyuan DiT": "hydit1",
"Hunyuan Video": "hyv1",
"Kolors": "kolors",
"Lumina": "lumina",
"Mochi": "mochi",
"ODOR": "odor",
"PixArt Alpha": "pixarta",
"PixArt Sigma": "pixarte",
"Playground v2": "playgroundv2",
"Playground v2.5": "playgroundv2",
"Pony Diffusion V7": "ponyv7",
# Video models
"CogVideoX": "cogvideox",
"LTX Video": "ltxv",
"LTX Video 2": "ltxv2",
"LTX Video 2.3": "ltxv23",
"Wan Video": "wanvideo",
"Wan Video 1.3B T2V": "wanvideo_13b_t2v",
"Wan Video 14B T2V": "wanvideo_14b_t2v",
"Wan Video 14B I2V 480p": "wanvideo_14b_i2v_480p",
"Wan Video 14B I2V 720p": "wanvideo_14b_i2v_720p",
# Third-party / proprietary image models
"Boogu": "boogu",
"Ernie": "ernie",
"Grok": "grok",
"HappyHorse": "happyhorse",
"Ideogram": "ideogram",
"Ideogram 4.0": "ideogram",
"Imagen": "imagen4",
"Imagen 4": "imagen4",
"Krea": "krea2",
"Krea 2": "krea2",
"Lens": "lens",
"MAI": "mai",
"Nano Banana": "nanobanana",
"OpenAI": "openai",
"Reve": "reve",
"Reve 2": "reve",
"Reve 2.1": "reve",
"Seedream": "seedream",
"Sora": "sora2",
"Sora 2": "sora2",
"Veo": "veo3",
"Veo 2": "veo3",
"Veo 3": "veo3",
"ZImageTurbo": "zimageturbo",
"ZImageBase": "zimagebase",
"ZImage": "zimagebase",
# Third-party video models
"Hailuo by MiniMax": "minimax",
"Haiper": "haiper",
"Kling": "kling",
"Lightricks": "lightricks",
"Seedance": "seedance",
"Vidu": "vidu",
# Qwen family
"Qwen": "qwen",
"Qwen 2": "qwen2",
# Anima
"Anima": "anima",
# Special
"Upscaler": "upscaler",
"Other": "other",
}
logger = logging.getLogger(__name__)
@@ -70,11 +220,29 @@ class SaveImageLM:
"tooltip": "Compression quality for JPEG and lossy WebP formats (1-100). Higher values mean better quality but larger files.",
},
),
"webp_method": (
"INT",
{
"default": 6,
"min": 0,
"max": 6,
"tooltip": "WebP compression method (0-6). 0=fastest/largest, 6=slowest/smallest. Only applies when file_format is 'webp'.",
},
),
"jpeg_subsampling": (
"INT",
{
"default": 0,
"min": 0,
"max": 2,
"tooltip": "JPEG chroma subsampling level. 0=4:4:4 (best quality), 1=4:2:2, 2=4:2:0 (smallest files). Only applies when file_format is 'jpeg'.",
},
),
"embed_workflow": (
"BOOLEAN",
{
"default": False,
"tooltip": "Embeds the complete workflow data into the image metadata. Only works with PNG and WebP formats.",
"tooltip": "When enabled, saved images store the complete workflow. Drag the image back into ComfyUI to restore the original node graph. PNG and WebP only.",
},
),
"save_with_metadata": (
@@ -84,6 +252,13 @@ class SaveImageLM:
"tooltip": "When enabled, embeds generation parameters into the saved image metadata. Disable to skip writing generation metadata.",
},
),
"add_loras_to_prompt": (
"BOOLEAN",
{
"default": False,
"tooltip": "When enabled, appends the LoRA syntax line (e.g. <lora:name:strength>) after the positive prompt in the saved metadata.",
},
),
"add_counter_to_filename": (
"BOOLEAN",
{
@@ -142,148 +317,197 @@ class SaveImageLM:
return None
def format_metadata(self, metadata_dict):
"""Format metadata in the requested format similar to userComment example"""
if not metadata_dict:
return ""
def _resolve_model_cache_entry(self, scanner_type: str, name: str):
"""Resolve model hash, civitai metadata, and base_model from scanner cache.
Returns (hash_str, civitai_dict, base_model_str). All values are empty defaults when not found."""
scanner = ServiceRegistry.get_service_sync(scanner_type)
if scanner is None or not name:
return "", {}, ""
# Helper function to only add parameter if value is not None
def add_param_if_not_none(param_list, label, value):
if value is not None:
param_list.append(f"{label}: {value}")
entry = self._get_cached_model_by_name(scanner, name)
if entry is None:
basename = os.path.splitext(os.path.basename(name))[0]
hash_val = scanner.get_hash_by_filename(basename)
return (hash_val or "").lower(), {}, ""
hash_val = (entry.get("sha256") or "").lower()
civitai = entry.get("civitai") or {}
base_model = entry.get("base_model") or ""
return hash_val, civitai, base_model
@staticmethod
def _get_civitai_sampler_name(sampler_name: str, scheduler: str) -> str:
if sampler_name in CIVITAI_SAMPLER_MAP:
civitai_name = CIVITAI_SAMPLER_MAP[sampler_name]
if scheduler == "karras":
civitai_name += " Karras"
elif scheduler == "exponential":
civitai_name += " Exponential"
return civitai_name
else:
if scheduler and scheduler != "normal":
return f"{sampler_name}_{scheduler}"
return sampler_name
@staticmethod
def _build_air_string(base_model: str, model_type: str, model_id: int, version_id: int) -> str:
slug = BASE_MODEL_AIR_SLUG.get(base_model, "other")
type_lower = model_type.lower() if model_type else "other"
return f"urn:air:{slug}:{type_lower}:civitai:{model_id}@{version_id}"
def format_metadata(self, metadata_dict: dict[str, Any], add_loras_to_prompt: bool = False) -> str:
"""Format metadata as A1111-compatible parameters string with Hashes JSON and Civitai resources."""
if not metadata_dict: return ""
# Extract the prompt and negative prompt
prompt = metadata_dict.get("prompt", "")
negative_prompt = metadata_dict.get("negative_prompt", "")
# Extract loras from the prompt if present
steps = metadata_dict.get("steps")
cfg = metadata_dict.get("guidance")
if cfg is None:
cfg = metadata_dict.get("cfg_scale")
if cfg is None:
cfg = metadata_dict.get("cfg")
seed = metadata_dict.get("seed")
size = metadata_dict.get("size")
sampler = metadata_dict.get("sampler") or ""
scheduler = metadata_dict.get("scheduler") or "normal"
checkpoint = metadata_dict.get("checkpoint") or ""
loras_text = metadata_dict.get("loras", "")
lora_hashes = {}
clip_skip = metadata_dict.get("clip_skip")
# If loras are found, add them on a new line after the prompt
# Parse LoRA entries from <lora:name:strength> format
lora_entries: list[tuple[str, float]] = []
if loras_text:
prompt_with_loras = f"{prompt}\n{loras_text}"
for match in re.findall(r"<lora:([^:]+):([^>]+)>", loras_text):
lora_name, strength_str = match
try:
strength = float(strength_str)
except (ValueError, TypeError):
strength = 1.0
lora_entries.append((lora_name, strength))
# Extract lora names from the format <lora:name:strength>
lora_matches = re.findall(r"<lora:([^:]+):([^>]+)>", loras_text)
# Resolve checkpoint hash and Civitai data from local cache
ckpt_hash, ckpt_civitai, ckpt_base_model = "", {}, ""
ckpt_display_name = ""
if checkpoint:
ckpt_hash, ckpt_civitai, ckpt_base_model = self._resolve_model_cache_entry(
"checkpoint_scanner", checkpoint
)
ckpt_display_name = os.path.splitext(os.path.basename(checkpoint))[0]
# Get hash for each lora
for lora_name, strength in lora_matches:
hash_value = self.get_lora_hash(lora_name)
if hash_value:
lora_hashes[lora_name] = hash_value
else:
prompt_with_loras = prompt
# Resolve LoRA hash and Civitai data from local cache
loras_data: list[dict[str, Any]] = []
for lora_name, strength in lora_entries:
lora_hash, lora_civitai, lora_base_model = self._resolve_model_cache_entry(
"lora_scanner", lora_name
)
loras_data.append({
"name": lora_name,
"strength": strength,
"hash": lora_hash,
"civitai": lora_civitai,
"base_model": lora_base_model,
})
# Format the first part (prompt and loras)
metadata_parts = [prompt_with_loras]
# Build Hashes JSON (A1111 / Civitai standard format)
hashes: dict[str, str] = {}
if ckpt_hash:
hashes["model"] = ckpt_hash[:10].upper()
for lora in loras_data:
if lora["hash"]:
hashes[f"LORA:{lora['name']}"] = lora["hash"][:10].upper()
# Add negative prompt
if negative_prompt:
metadata_parts.append(f"Negative prompt: {negative_prompt}")
# Build Civitai resources JSON array
civitai_resources: list[dict[str, Any]] = []
if ckpt_civitai.get("id", 0) > 0:
ckpt_resource: dict[str, Any] = {}
ckpt_type = (ckpt_civitai.get("model") or {}).get("type", "Checkpoint")
model_id = ckpt_civitai.get("modelId", 0)
version_id = ckpt_civitai.get("id", 0)
if model_id and version_id:
ckpt_resource["air"] = self._build_air_string(
ckpt_base_model, ckpt_type, int(model_id), int(version_id)
)
elif version_id:
ckpt_resource["modelVersionId"] = int(version_id)
if ckpt_civitai.get("name"):
ckpt_resource["versionName"] = ckpt_civitai["name"]
if ckpt_resource:
civitai_resources.append(ckpt_resource)
# Format the second part (generation parameters)
params = []
for lora in loras_data:
lora_civitai = lora["civitai"]
if not lora_civitai or lora_civitai.get("id", 0) <= 0:
continue
lora_resource: dict[str, Any] = {"weight": lora["strength"]}
lora_type = (lora_civitai.get("model") or {}).get("type", "LORA")
model_id = lora_civitai.get("modelId", 0)
version_id = lora_civitai.get("id", 0)
if model_id and version_id:
lora_resource["air"] = self._build_air_string(
lora["base_model"], lora_type, int(model_id), int(version_id)
)
elif version_id:
lora_resource["modelVersionId"] = int(version_id)
if lora_civitai.get("name"):
lora_resource["versionName"] = lora_civitai["name"]
civitai_resources.append(lora_resource)
# Add standard parameters in the correct order
if "steps" in metadata_dict:
add_param_if_not_none(params, "Steps", metadata_dict.get("steps"))
sampler_name = CIVITAI_SAMPLER_MAP.get(sampler, sampler) if sampler else None
# Combine sampler and scheduler information
sampler_name = None
scheduler_name = None
if "sampler" in metadata_dict:
sampler = metadata_dict.get("sampler")
# Convert ComfyUI sampler names to user-friendly names
sampler_mapping = {
"euler": "Euler",
"euler_ancestral": "Euler a",
"dpm_2": "DPM2",
"dpm_2_ancestral": "DPM2 a",
"heun": "Heun",
"dpm_fast": "DPM fast",
"dpm_adaptive": "DPM adaptive",
"lms": "LMS",
"dpmpp_2s_ancestral": "DPM++ 2S a",
"dpmpp_sde": "DPM++ SDE",
"dpmpp_sde_gpu": "DPM++ SDE",
"dpmpp_2m": "DPM++ 2M",
"dpmpp_2m_sde": "DPM++ 2M SDE",
"dpmpp_2m_sde_gpu": "DPM++ 2M SDE",
"ddim": "DDIM",
}
sampler_name = sampler_mapping.get(sampler, sampler)
if "scheduler" in metadata_dict:
scheduler = metadata_dict.get("scheduler")
scheduler_mapping = {
"normal": "Simple",
"normal": "Normal",
"karras": "Karras",
"exponential": "Exponential",
"sgm_uniform": "SGM Uniform",
"sgm_quadratic": "SGM Quadratic",
}
scheduler_name = scheduler_mapping.get(scheduler, scheduler)
scheduler_name = scheduler_mapping.get(scheduler, scheduler) if scheduler else None
# Add combined sampler and scheduler information
# Build output lines
prompt_line = prompt if prompt else ""
if add_loras_to_prompt and loras_text:
prompt_line = f"{prompt_line}\n{loras_text}" if prompt_line else loras_text
lines = [prompt_line] if prompt_line else [""]
if negative_prompt:
lines.append(f"Negative prompt: {negative_prompt}")
params: list[str] = []
if steps is not None:
params.append(f"Steps: {steps}")
if sampler_name:
if scheduler_name:
params.append(f"Sampler: {sampler_name} {scheduler_name}")
else:
params.append(f"Sampler: {sampler_name}")
# CFG scale (Use guidance if available, otherwise fall back to cfg_scale or cfg)
if "guidance" in metadata_dict:
add_param_if_not_none(params, "CFG scale", metadata_dict.get("guidance"))
elif "cfg_scale" in metadata_dict:
add_param_if_not_none(params, "CFG scale", metadata_dict.get("cfg_scale"))
elif "cfg" in metadata_dict:
add_param_if_not_none(params, "CFG scale", metadata_dict.get("cfg"))
# Seed
if "seed" in metadata_dict:
add_param_if_not_none(params, "Seed", metadata_dict.get("seed"))
# Size
if "size" in metadata_dict:
add_param_if_not_none(params, "Size", metadata_dict.get("size"))
# Model info
if "checkpoint" in metadata_dict:
# Ensure checkpoint is a string before processing
checkpoint = metadata_dict.get("checkpoint")
if checkpoint is not None:
# Get model hash
model_hash = self.get_checkpoint_hash(checkpoint)
# Extract basename without path
checkpoint_name = os.path.basename(checkpoint)
# Remove extension if present
checkpoint_name = os.path.splitext(checkpoint_name)[0]
# Add model hash if available
if model_hash:
if cfg is not None:
params.append(f"CFG scale: {cfg}")
if seed is not None:
params.append(f"Seed: {seed}")
if size:
params.append(f"Size: {size}")
if clip_skip is not None:
try:
params.append(f"Clip skip: {abs(int(clip_skip))}")
except (ValueError, TypeError):
pass
additional_data = metadata_dict.get("additional_data", "")
if additional_data:
params.append(additional_data)
if ckpt_hash:
params.append(f"Model hash: {ckpt_hash[:10].upper()}")
if ckpt_display_name:
params.append(f"Model: {ckpt_display_name}")
if hashes:
params.append(f"Hashes: {json.dumps(hashes, separators=(',', ':'))}")
params.append("Version: ComfyUI")
if civitai_resources:
params.append(
f"Model hash: {model_hash[:10]}, Model: {checkpoint_name}"
f"Civitai resources: {json.dumps(civitai_resources, separators=(',', ':'))}"
)
else:
params.append(f"Model: {checkpoint_name}")
# Add LoRA hashes if available
if lora_hashes:
lora_hash_parts = []
for lora_name, hash_value in lora_hashes.items():
lora_hash_parts.append(f"{lora_name}: {hash_value[:10]}")
if lora_hash_parts:
params.append(f'Lora hashes: "{", ".join(lora_hash_parts)}"')
# Combine all parameters with commas
metadata_parts.append(", ".join(params))
# Join all parts with a new line
return "\n".join(metadata_parts)
lines.append(", ".join(params))
return "\n".join(lines)
# credit to nkchocoai
# Add format_filename method to handle pattern substitution
@@ -554,6 +778,14 @@ class SaveImageLM:
if checkpoint_entry:
recipe_data["checkpoint"] = checkpoint_entry
# The recipe image is the WebP produced above from the output file;
# reuse the same metadata extraction to record workflow presence.
try:
metadata = ExifUtils._load_structured_metadata(image_path)
recipe_data["has_workflow"] = bool(metadata.get("workflow"))
except Exception:
recipe_data["has_workflow"] = False
json_path = os.path.normpath(
os.path.join(recipes_dir, f"{recipe_id}.recipe.json")
)
@@ -573,10 +805,13 @@ class SaveImageLM:
extra_pnginfo=None,
lossless_webp=True,
quality=100,
webp_method=6,
jpeg_subsampling=0,
embed_workflow=False,
save_with_metadata=True,
add_counter_to_filename=True,
save_as_recipe=False,
add_loras_to_prompt=False,
):
"""Save images with metadata"""
results = []
@@ -585,7 +820,7 @@ class SaveImageLM:
raw_metadata = get_metadata()
metadata_dict = MetadataProcessor.to_dict(raw_metadata, id)
metadata = self.format_metadata(metadata_dict)
metadata = self.format_metadata(metadata_dict, add_loras_to_prompt)
# Process filename_prefix with pattern substitution
filename_prefix = self.format_filename(filename_prefix, metadata_dict)
@@ -627,15 +862,14 @@ class SaveImageLM:
elif file_format == "jpeg":
file = base_filename + ".jpg"
file_extension = ".jpg"
save_kwargs = {"quality": quality, "optimize": True}
save_kwargs = {"quality": quality, "optimize": True, "subsampling": jpeg_subsampling}
elif file_format == "webp":
file = base_filename + ".webp"
file_extension = ".webp"
# Add optimization param to control performance
save_kwargs = {
"quality": quality,
"lossless": lossless_webp,
"method": 0,
"method": webp_method,
}
else:
raise ValueError(f"Unsupported file format: {file_format}")
@@ -722,10 +956,13 @@ class SaveImageLM:
extra_pnginfo=None,
lossless_webp=True,
quality=100,
webp_method=6,
jpeg_subsampling=0,
embed_workflow=False,
save_with_metadata=True,
add_counter_to_filename=True,
save_as_recipe=False,
add_loras_to_prompt=False,
):
"""Process and save image with metadata"""
# Make sure the output directory exists
@@ -751,10 +988,13 @@ class SaveImageLM:
extra_pnginfo,
lossless_webp,
quality,
webp_method,
jpeg_subsampling,
embed_workflow,
save_with_metadata,
add_counter_to_filename,
save_as_recipe,
add_loras_to_prompt,
)
return {
+107 -8
View File
@@ -1,37 +1,74 @@
import logging
import os
from typing import List, Tuple
import comfy.sd # type: ignore
from typing import Any, List, Tuple
import comfy.sd # pyright: ignore[reportMissingImports]
from ..utils.utils import get_checkpoint_info_absolute, _format_model_name_for_comfyui
logger = logging.getLogger(__name__)
def _reload_gguf_unet(
unet_path: str, weight_dtype: str, disable_dynamic: bool = False
) -> object:
"""Reload a GGUF diffusion model from disk (cached_patcher_init factory).
Mirrors the GGUF branch of UNETLoaderLM.load_unet so ModelPatcher
deepclone/dynamic machinery can rebuild GGUF models with the correct
GGMLOps. ``disable_dynamic`` is accepted for signature compatibility
with core ComfyUI loaders.
"""
loader = UNETLoaderLM()
model, = loader._load_gguf_unet(unet_path, unet_path, weight_dtype)
return model
class UNETLoaderLM:
"""UNET Loader with support for extra folder paths
Loads diffusion models/UNets from both standard ComfyUI folders and LoRA Manager's
extra folder paths, providing a unified interface for UNET loading.
Supports both regular diffusion models and GGUF format models.
The unet_name combo supports ComfyUI's control_after_generate, letting
users pick a random diffusion model on every run; the base_model input
narrows the random pool through a front-end extension that filters the
combo options.
"""
NAME = "Unet Loader (LoraManager)"
CATEGORY = "Lora Manager/loaders"
@classmethod
def INPUT_TYPES(s):
def INPUT_TYPES(cls):
# Get list of unet names from scanner (includes extra folder paths)
unet_names = s._get_unet_names()
unet_names = cls._get_unet_names()
base_models = cls._get_available_base_models()
return {
"required": {
"unet_name": (
unet_names,
{"tooltip": "The name of the diffusion model to load."},
{
"tooltip": (
"The name of the diffusion model to load. Use "
"control_after_generate to pick a random model on "
"every run."
),
"control_after_generate": "fixed",
},
),
"weight_dtype": (
["default", "fp8_e4m3fn", "fp8_e4m3fn_fast", "fp8_e5m2"],
{"tooltip": "The dtype to use for the model weights."},
),
"base_model": (
base_models,
{
"default": "Any",
"tooltip": (
"Restrict the random selection pool to this base "
"model. 'Any' uses the full pool."
),
},
),
}
}
@@ -59,7 +96,10 @@ class UNETLoaderLM:
for item in cache.raw_data:
if item.get("sub_type") == "diffusion_model":
file_path = item.get("file_path", "")
if file_path:
# Only offer models that still exist on disk so ComfyUI
# flags missing diffusion models at queue time via
# "value not in list" (the scanner cache can be stale).
if file_path and os.path.exists(file_path):
# Format using relative path with OS-native separator
formatted_name = _format_model_name_for_comfyui(
file_path, model_roots
@@ -90,16 +130,69 @@ class UNETLoaderLM:
logger.error(f"Error getting unet names: {e}")
return []
def load_unet(self, unet_name: str, weight_dtype: str) -> Tuple:
@classmethod
def _get_available_base_models(cls) -> List[str]:
"""Get distinct base_model values present among indexed diffusion models, for the random-selection filter."""
try:
from ..services.service_registry import ServiceRegistry
async def _get_base_models():
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
base_models = set()
for item in cache.raw_data:
if item.get("sub_type") != "diffusion_model":
continue
base_model = item.get("base_model")
file_path = item.get("file_path", "")
if base_model and file_path and os.path.exists(file_path):
base_models.add(base_model)
return sorted(base_models)
return ["Any"] + cls._run_async(_get_base_models)
except Exception as e:
logger.error(f"Error getting available base models: {e}")
return ["Any"]
@staticmethod
def _run_async(coro_fn):
"""Run an async fetcher, handling the case where an event loop is already running."""
import asyncio
try:
asyncio.get_running_loop()
import concurrent.futures
def run_in_thread():
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
return new_loop.run_until_complete(coro_fn())
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
return future.result()
except RuntimeError:
return asyncio.run(coro_fn())
def load_unet(
self, unet_name: str, weight_dtype: str, base_model: str = "Any"
) -> Tuple[Any, ...]:
"""Load a diffusion model by name, supporting extra folder paths
Args:
unet_name: The name of the diffusion model to load (relative path with extension)
weight_dtype: The dtype to use for model weights
base_model: Only used by the front-end to filter the random pool
Returns:
Tuple of (MODEL,)
"""
del base_model
import torch
# Get absolute path from cache using ComfyUI-style name
@@ -133,7 +226,7 @@ class UNETLoaderLM:
def _load_gguf_unet(
self, unet_path: str, unet_name: str, weight_dtype: str
) -> Tuple:
) -> Tuple[Any, ...]:
"""Load a GGUF format diffusion model
Args:
@@ -196,6 +289,12 @@ class UNETLoaderLM:
# Wrap with GGUFModelPatcher
model = GGUFModelPatcher.clone(model)
# Register a reload factory so the MODEL carries its source path
# (cached_patcher_init) like core ComfyUI loaders do — required
# for model-name extraction downstream and for ModelPatcher
# deepclone/dynamic machinery.
model.cached_patcher_init = (_reload_gguf_unet, (unet_path, weight_dtype))
return (model,)
except Exception as e:
+178 -2
View File
@@ -1,3 +1,6 @@
from typing import Any
class AnyType(str):
"""A special class that is always equal in not equal comparisons. Credit to pythongosssss"""
@@ -6,7 +9,7 @@ class AnyType(str):
# Credit to Regis Gaughan, III (rgthree)
class FlexibleOptionalInputType(dict):
class FlexibleOptionalInputType(dict[str, Any]):
"""A special class to make flexible nodes that pass data to our python handlers.
Enables both flexible/dynamic input types (like for Any Switch) or a dynamic number of inputs
@@ -23,6 +26,7 @@ class FlexibleOptionalInputType(dict):
"""
def __init__(self, type):
super().__init__()
self.type = type
def __getitem__(self, key):
@@ -36,10 +40,12 @@ any_type = AnyType("*")
# Common methods extracted from lora_loader.py and lora_stacker.py
import os
import re
import logging
import copy
import sys
import folder_paths # type: ignore
import asyncio
import folder_paths # pyright: ignore[reportMissingImports]
logger = logging.getLogger(__name__)
@@ -69,6 +75,25 @@ def extract_lora_name(lora_path):
return apply_lora_syntax_format(name_no_ext)
def parse_lora_syntax(text: str) -> list[dict[str, Any]]:
"""Parse <lora:name:strength> syntax from text input into a list of dicts.
Each entry contains: name, model_strength, clip_strength.
Supports both ``<lora:name:strength>`` and ``<lora:name:model_strength:clip_strength>``.
"""
pattern = r"<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>"
matches = re.findall(pattern, text, re.IGNORECASE)
loras = []
for match in matches:
model_strength = float(match[1])
loras.append({
"name": match[0],
"model_strength": model_strength,
"clip_strength": float(match[2]) if match[2] else model_strength,
})
return loras
def get_loras_list(kwargs):
"""Helper to extract loras list from either old or new kwargs format"""
if "loras" not in kwargs:
@@ -87,6 +112,157 @@ def get_loras_list(kwargs):
return []
_LORA_EXTENSIONS = (".safetensors", ".ckpt", ".pt", ".bin")
def _strip_lora_extension(name: str) -> str:
"""Strip a known LoRA model extension from a name (case-insensitive)."""
lowered = name.lower()
for ext in _LORA_EXTENSIONS:
if lowered.endswith(ext):
return name[: -len(ext)]
return name
def _find_missing_loras(names: list[str]) -> list[str]:
"""Return the names that cannot be resolved to an existing local LoRA file.
Mirrors the matching semantics of ``get_lora_info_absolute``
(py/utils/utils.py): after stripping the extension, a name matches a cached
LoRA when it equals the cached file name or the ``folder/file`` path. As a
fallback, a name containing a folder that only matches by basename resolves
to the first basename match (same behavior as the runtime resolver). Raw
absolute paths that exist on disk are always considered available.
The scanner cache is fetched once for all names; the cache may be stale, so
resolved paths are additionally verified with ``os.path.isfile``.
"""
if not names:
return []
async def _check() -> list[str]:
from ..services.service_registry import ServiceRegistry
scanner = await ServiceRegistry.get_lora_scanner()
# The scanner cache may not be hydrated yet (startup, library path
# change). An empty cache is not authoritative — treat it as "cannot
# verify" and skip validation instead of flagging every active LoRA
# as missing.
if getattr(scanner, "_cache", None) is None or getattr(
scanner, "_is_initializing", False
):
return []
cache = await scanner.get_cached_data()
lookup = {}
basename_candidates = {}
for item in cache.raw_data:
file_path = item.get("file_path")
if not file_path:
continue
file_name = item.get("file_name", "")
folder = item.get("folder", "")
file_name_no_ext = _strip_lora_extension(file_name)
path_name_no_ext = (
f"{folder}/{file_name_no_ext}".replace("\\", "/")
if folder
else file_name_no_ext
)
lookup.setdefault(file_name_no_ext, file_path)
lookup.setdefault(path_name_no_ext, file_path)
basename_candidates.setdefault(file_name_no_ext, []).append(
(folder, file_path)
)
missing = []
for name in names:
if not name:
continue
normalized = name.replace("\\", "/")
# Raw absolute paths (outside the library) are usable as-is.
if os.path.isfile(normalized):
continue
no_ext = _strip_lora_extension(normalized)
file_path = lookup.get(no_ext)
if file_path is None and "/" in no_ext:
# A name with a folder that matches only by basename resolves
# at runtime like get_lora_info_absolute's fallback does:
# prefer a candidate whose folder prefixes the name, else the
# first basename match.
folder, basename = no_ext.rsplit("/", 1)
candidates = basename_candidates.get(basename, [])
file_path = next(
(
fp
for fld, fp in candidates
if fld and no_ext.startswith(fld + "/")
),
None,
)
if file_path is None and candidates:
file_path = candidates[0][1]
if file_path is None or not os.path.isfile(file_path):
missing.append(name)
return missing
try:
# Check if we're already in an event loop
loop = asyncio.get_running_loop()
# If we're in a running loop, run the async check in a separate thread
import concurrent.futures
def run_in_thread():
new_loop = asyncio.new_event_loop()
asyncio.set_event_loop(new_loop)
try:
return new_loop.run_until_complete(_check())
finally:
new_loop.close()
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(run_in_thread)
return future.result()
except RuntimeError:
# No event loop is running, we can use asyncio.run()
return asyncio.run(_check())
def validate_lora_entries(kwargs):
"""Validate active LoRA widget entries against the local library.
Used by node ``VALIDATE_INPUTS`` implementations so ComfyUI rejects the
prompt at queue time (``custom_validation_failed``) when an active entry
references a LoRA that is not available locally mirroring how built-in
loader nodes flag missing models before execution starts.
Returns:
None when every active entry resolves to an existing local file,
otherwise a descriptive error string listing the missing LoRAs.
Verification failures (e.g. scanner not ready) are treated as valid
so queueing is never blocked by validation machinery itself.
"""
# Missing/empty loras input is always valid; skip get_loras_list so it
# does not log a warning for the None case on every queue.
if not kwargs.get("loras"):
return None
loras = get_loras_list(kwargs)
active_names = []
for lora in loras:
if not isinstance(lora, dict):
continue
if not lora.get("active", False):
continue
active_names.append(apply_lora_syntax_format(str(lora.get("name") or "")))
try:
missing = _find_missing_loras(active_names)
except Exception:
logger.exception("Failed to validate LoRA entries against the local library")
return None
if not missing:
return None
return "Missing LoRA(s) in local library: " + ", ".join(missing)
def load_state_dict_in_safetensors(path, device="cpu", filter_prefix=""):
"""Simplified version of load_state_dict_in_safetensors that just loads from a local path"""
import safetensors.torch
+10 -4
View File
@@ -1,7 +1,7 @@
import os
from ..utils.utils import get_lora_info_absolute
from ..config import config
from .utils import FlexibleOptionalInputType, any_type, get_loras_list
from .utils import FlexibleOptionalInputType, any_type, get_loras_list, validate_lora_entries
import logging
logger = logging.getLogger(__name__)
@@ -31,15 +31,21 @@ class WanVideoLoraSelectLM:
"placeholder": "Search LoRAs to add...",
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
}),
"loras": ("LORAS", {}),
},
"optional": FlexibleOptionalInputType(any_type),
}
@classmethod
def VALIDATE_INPUTS(cls, loras=None):
"""Queue-time validation: reject missing local LoRAs before execution."""
return validate_lora_entries({"loras": loras}) or True
RETURN_TYPES = ("WANVIDLORA", "STRING", "STRING")
RETURN_NAMES = ("lora", "trigger_words", "active_loras")
FUNCTION = "process_loras"
def process_loras(self, text, low_mem_load=False, merge_loras=True, **kwargs):
def process_loras(self, text, loras, low_mem_load=False, merge_loras=True, **kwargs):
loras_list = []
all_trigger_words = []
active_loras = []
@@ -57,8 +63,8 @@ class WanVideoLoraSelectLM:
selected_blocks = blocks.get("selected_blocks", {})
layer_filter = blocks.get("layer_filter", "")
# Process loras from kwargs with support for both old and new formats
loras_from_widget = get_loras_list(kwargs)
# Process loras from the widget with support for both old and new formats
loras_from_widget = get_loras_list({"loras": loras})
for lora in loras_from_widget:
if not lora.get('active', False):
continue
+64 -8
View File
@@ -1,3 +1,7 @@
# pyright: reportImportCycles=false
# Lazy (function-local) imports still count as static edges in basedpyright's
# reportImportCycles, so the ServiceRegistry singleton pattern necessarily forms
# import cycles. Breaking them would require an architectural refactor.
"""Base classes for recipe parsers."""
import json
@@ -7,7 +11,7 @@ import re
from typing import Dict, List, Any, Optional, Tuple
from abc import ABC, abstractmethod
from ..config import config
from ..utils.constants import VALID_LORA_TYPES, VALID_CHECKPOINT_SUB_TYPES
from ..utils.constants import MODEL_WEIGHT_FILE_TYPES, VALID_LORA_TYPES, VALID_CHECKPOINT_SUB_TYPES
from ..utils.civitai_utils import rewrite_preview_url
logger = logging.getLogger(__name__)
@@ -38,7 +42,41 @@ class RecipeMetadataParser(ABC):
pass
@staticmethod
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any], Optional[str]],
def populate_lora_from_local(lora_entry: Dict[str, Any], local_lora: Dict[str, Any], base_model_counts=None) -> Dict[str, Any]:
"""Populate a recipe LoRA entry from the local scanner cache."""
local_path = local_lora.get('file_path') or ''
file_name = local_lora.get('file_name') or os.path.splitext(os.path.basename(local_path))[0]
base_model = local_lora.get('base_model') or ''
lora_entry['name'] = local_lora.get('model_name') or file_name or lora_entry.get('name', '')
lora_entry['file_name'] = file_name
lora_entry['hash'] = (local_lora.get('sha256') or lora_entry.get('hash') or '').lower()
lora_entry['localPath'] = local_path or None
lora_entry['size'] = local_lora.get('size', 0) or 0
lora_entry['baseModel'] = base_model
lora_entry['existsLocally'] = True
lora_entry['isDeleted'] = False
preview_url = local_lora.get('preview_url')
if preview_url:
lora_entry['thumbnailUrl'] = config.get_preview_static_url(preview_url)
civitai_info = local_lora.get('civitai') or {}
if isinstance(civitai_info, dict):
if civitai_info.get('id') is not None:
lora_entry['id'] = civitai_info['id']
if civitai_info.get('modelId') is not None:
lora_entry['modelId'] = civitai_info['modelId']
if civitai_info.get('name'):
lora_entry['version'] = civitai_info['name']
if base_model_counts is not None and base_model:
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
return lora_entry
@staticmethod
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any] | None, str | None] | Dict[str, Any],
recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]:
"""
Populate a lora entry with information from Civitai API response
@@ -151,9 +189,9 @@ class RecipeMetadataParser(ABC):
# Process file information if available
if 'files' in civitai_info:
# Find the primary model file (type="Model" and primary=true) in the files list
# Find the primary model file (weights-type and primary=true) in the files list
model_file = next((file for file in civitai_info.get('files', [])
if file.get('type') == 'Model' and file.get('primary') == True), None)
if file.get('type') in MODEL_WEIGHT_FILE_TYPES and file.get('primary') == True), None)
if model_file:
# Get size
@@ -175,10 +213,18 @@ class RecipeMetadataParser(ABC):
lora_entry['localPath'] = local_path
lora_entry['file_name'] = os.path.splitext(os.path.basename(local_path))[0]
# Get thumbnail from local preview if available
# Get thumbnail from local preview if available.
# Match the cache item by local path first (get_path_by_hash
# cascade: 10-char autov2 / 12-char autov3), then by hash.
lora_cache = await lora_scanner.get_cached_data()
h = (lora_entry.get("hash") or "").lower()
lora_item = next((item for item in lora_cache.raw_data
if item['sha256'].lower() == lora_entry['hash'].lower()), None)
if (item.get("file_path") or "") == local_path), None)
if lora_item is None:
lora_item = next((item for item in lora_cache.raw_data
if (item.get("sha256") or "").lower() == h
or (item.get("autov3") or "").lower() == h
or (item.get("sha256") or "")[:10].lower() == h), None)
if lora_item and 'preview_url' in lora_item:
lora_entry['thumbnailUrl'] = config.get_preview_static_url(lora_item['preview_url'])
except Exception as e:
@@ -194,7 +240,7 @@ class RecipeMetadataParser(ABC):
return lora_entry
@staticmethod
async def populate_checkpoint_from_civitai(checkpoint: Dict[str, Any], civitai_info: Dict[str, Any]) -> Dict[str, Any]:
async def populate_checkpoint_from_civitai(checkpoint: Dict[str, Any], civitai_info: Dict[str, Any] | Tuple[Dict[str, Any] | None, str | None] | None) -> Dict[str, Any]:
"""
Populate checkpoint information from Civitai API response
@@ -249,11 +295,21 @@ class RecipeMetadataParser(ABC):
checkpoint['id'] = civitai_data.get('id', 0)
if 'files' in civitai_data:
# Prefer the file CivitAI marked primary; fall back to any
# weights-type file (providers without primary flags).
model_file = next(
(
file
for file in civitai_data.get('files', [])
if file.get('type') == 'Model'
if file.get('type') in MODEL_WEIGHT_FILE_TYPES
and file.get('primary') is True
),
None,
) or next(
(
file
for file in civitai_data.get('files', [])
if file.get('type') in MODEL_WEIGHT_FILE_TYPES
),
None,
)
+4
View File
@@ -1,3 +1,7 @@
# pyright: reportImportCycles=false
# Lazy (function-local) imports still count as static edges in basedpyright's
# reportImportCycles, so the ServiceRegistry singleton pattern necessarily forms
# import cycles. Breaking them would require an architectural refactor.
import logging
import json
import os
+3 -1
View File
@@ -1,6 +1,7 @@
"""Factory for creating recipe metadata parsers."""
import logging
from typing import Any
from .parsers import (
RecipeFormatParser,
ComfyMetadataParser,
@@ -31,7 +32,8 @@ class RecipeParserFactory:
# First, try CivitaiApiMetadataParser for dict input
if isinstance(metadata, dict):
try:
if CivitaiApiMetadataParser().is_metadata_matching(metadata):
user_comment: Any = metadata
if CivitaiApiMetadataParser().is_metadata_matching(user_comment):
return CivitaiApiMetadataParser()
except Exception as e:
logger.debug(f"CivitaiApiMetadataParser check failed: {e}")
+176 -38
View File
@@ -52,7 +52,7 @@ class AutomaticMetadataParser(RecipeMetadataParser):
negative_and_params = ""
# Initialize metadata
metadata = {
metadata: Dict[str, Any] = {
"prompt": prompt,
"loras": []
}
@@ -146,14 +146,12 @@ class AutomaticMetadataParser(RecipeMetadataParser):
# Initialize hashes dict if it doesn't exist
if "hashes" not in metadata:
metadata["hashes"] = {}
# Add as lora type in the same format as
# regular hashes. Only override an
# existing entry if its value is empty
# (Lora hashes is the more reliable
# source when Hashes JSON has blanks).
# Lora hashes carries the 12-char AutoV3
# hash (resolvable on CivitAI and the local
# autov3 index); the Hashes JSON value is
# only the 10-char AutoV2 prefix, so on
# conflict the Lora hashes value wins.
key = f"lora:{lora_name}"
existing = metadata["hashes"].get(key, "")
if not existing:
metadata["hashes"][key] = lora_hash
# Remove lora hashes from params section
@@ -362,37 +360,47 @@ class AutomaticMetadataParser(RecipeMetadataParser):
checkpoint = checkpoint_entry
# If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"]
if not loras or len(loras) == 0:
# Extract lora weights from extranet tags in prompt (for later use)
lora_weights = {}
lora_matches = re.findall(self.EXTRANETS_REGEX, prompt)
for lora_type, lora_name, lora_weight in lora_matches:
key = f"{lora_type}:{lora_name}"
lora_weights[key] = round(float(lora_weight), 2)
def normalize_lora_name(name, basename=False):
normalized = str(name or '').replace('\\', '/')
if normalized.casefold().endswith('.safetensors'):
normalized = normalized[:-12]
if basename:
normalized = normalized.rsplit('/', 1)[-1]
return normalized.casefold()
# Use hashes from metadata as the primary source
if metadata.get("hashes"):
for hash_key, lora_hash in metadata.get("hashes", {}).items():
# Only process lora or hypernet types
if not hash_key.startswith(("lora:", "hypernet:")):
continue
def get_version_id(lora):
version_id = lora.get('id')
if version_id in (None, '', 0, '0'):
version_id = lora.get('modelVersionId')
if version_id in (None, '', 0, '0'):
return None
return str(version_id)
# Skip entries without a hash value — they can't be
# resolved via CivitAI and would only produce a
# useless "Deleted" entry in the recipe.
if not lora_hash:
continue
prompt_loras = {}
for match in re.findall(self.EXTRANETS_REGEX, prompt):
lora_type, lora_name, _ = match
prompt_loras[(lora_type, normalize_lora_name(lora_name))] = match
lora_type, lora_name = hash_key.split(':', 1)
prompt_by_basename = {}
for lora_type, lora_name, lora_weight in prompt_loras.values():
key = (lora_type, normalize_lora_name(lora_name, True))
prompt_by_basename.setdefault(key, []).append((lora_name, round(float(lora_weight), 2)))
# Get weight from extranet tags if available, else default to 1.0
weight = lora_weights.get(hash_key, 1.0)
hash_basenames = {
(hash_key.split(':', 1)[0], normalize_lora_name(hash_key.split(':', 1)[1], True))
for hash_key, hash_value in metadata.get("hashes", {}).items()
if hash_value and hash_key.startswith(("lora:", "hypernet:"))
}
recipe_base_model = checkpoint.get("baseModel") if checkpoint else None
if not recipe_base_model and len(base_model_counts) == 1:
recipe_base_model = next(iter(base_model_counts))
# Initialize lora entry
lora_entry = {
resource_lora_count = len(loras)
def make_lora_entry(lora_type, lora_name, weight, lora_hash=''):
return {
'name': lora_name,
'type': lora_type, # 'lora' or 'hypernet'
'type': lora_type,
'weight': weight,
'hash': lora_hash,
'existsLocally': False,
@@ -405,24 +413,154 @@ class AutomaticMetadataParser(RecipeMetadataParser):
'isDeleted': False
}
# Try to get info from Civitai
if metadata_provider:
def merge_or_append_civitai(civitai_entry, preserve_existing_weight=False):
civitai_id = get_version_id(civitai_entry)
civitai_hash = (civitai_entry.get('hash') or '').lower()
for index, existing in enumerate(loras):
existing_id = get_version_id(existing)
existing_hash = (existing.get('hash') or '').lower()
if not (
(civitai_id and existing_id == civitai_id)
or (civitai_hash and existing_hash == civitai_hash)
):
continue
if preserve_existing_weight:
civitai_entry['weight'] = existing.get('weight', civitai_entry['weight'])
existing_base = existing.get('baseModel')
if not civitai_entry.get('baseModel'):
civitai_entry['baseModel'] = existing_base or ''
elif existing_base:
remaining = base_model_counts.get(existing_base, 0) - 1
if remaining > 0:
base_model_counts[existing_base] = remaining
else:
base_model_counts.pop(existing_base, None)
loras[index] = civitai_entry
return
loras.append(civitai_entry)
def merge_or_append_local(local_entry):
local_id = get_version_id(local_entry)
local_hash = (local_entry.get('hash') or '').lower()
for existing in loras:
existing_id = get_version_id(existing)
existing_hash = (existing.get('hash') or '').lower()
if not (
(local_id and existing_id == local_id)
or (local_hash and existing_hash == local_hash)
):
continue
existing['weight'] = local_entry['weight']
existing['hash'] = local_entry['hash']
existing['file_name'] = local_entry['file_name']
existing['existsLocally'] = True
existing['localPath'] = local_entry['localPath']
existing['size'] = local_entry['size']
existing['isDeleted'] = False
if not existing.get('modelId') and local_entry.get('modelId'):
existing['modelId'] = local_entry['modelId']
if not existing.get('baseModel') and local_entry.get('baseModel'):
existing['baseModel'] = local_entry['baseModel']
base_model_counts[local_entry['baseModel']] = base_model_counts.get(local_entry['baseModel'], 0) + 1
thumbnail_url = local_entry.get('thumbnailUrl')
if thumbnail_url and not thumbnail_url.endswith('/images/no-preview.png'):
existing['thumbnailUrl'] = thumbnail_url
return
if local_entry.get('baseModel'):
base_model = local_entry['baseModel']
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
loras.append(local_entry)
resolved_prompt_basenames = set()
queried_local_basenames = set()
for lora_type, lora_name, lora_weight in prompt_loras.values():
weight = round(float(lora_weight), 2)
basename_key = (lora_type, normalize_lora_name(lora_name, True))
matching_resources = [
lora
for lora in loras[:resource_lora_count]
if lora.get('file_name')
and normalize_lora_name(lora['file_name'], True) == basename_key[1]
and (
(lora_type == 'hypernet' and str(lora.get('type', '')).casefold() in ('hypernet', 'hypernetwork'))
or (lora_type == 'lora' and str(lora.get('type', '')).casefold() not in ('hypernet', 'hypernetwork'))
)
]
if len(prompt_by_basename[basename_key]) == 1 and len(matching_resources) == 1:
matching_resources[0]['weight'] = weight
if basename_key not in hash_basenames:
resolved_prompt_basenames.add(basename_key)
continue
if basename_key in hash_basenames:
continue
if not recipe_scanner or lora_type != 'lora':
continue
queried_local_basenames.add(basename_key)
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if not local_lora:
continue
local_entry = self.populate_lora_from_local(
make_lora_entry(lora_type, lora_name, weight),
local_lora,
)
merge_or_append_local(local_entry)
resolved_prompt_basenames.add(basename_key)
for hash_key, lora_hash in metadata.get("hashes", {}).items():
if not hash_key.startswith(("lora:", "hypernet:")):
continue
lora_type, lora_name = hash_key.split(':', 1)
basename_key = (lora_type, normalize_lora_name(lora_name, True))
if basename_key in resolved_prompt_basenames:
continue
prompt_entries = prompt_by_basename.get(basename_key, [])
weight = prompt_entries[0][1] if len(prompt_entries) == 1 else 1.0
lora_entry = make_lora_entry(lora_type, lora_name, weight, lora_hash)
if lora_hash and recipe_scanner and lora_type == 'lora':
local_lora = await recipe_scanner.get_local_lora_by_hash(lora_hash)
if local_lora:
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
merge_or_append_local(local_entry)
continue
hash_resolved = False
if lora_hash and metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash
lora_hash,
)
if populated_entry is None:
continue # Skip invalid LoRA types
continue
lora_entry = populated_entry
hash_resolved = not lora_entry.get('isDeleted')
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA {lora_name}: {e}")
if hash_resolved:
merge_or_append_civitai(lora_entry, preserve_existing_weight=not prompt_entries)
continue
if recipe_scanner and lora_type == 'lora' and basename_key not in queried_local_basenames:
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if local_lora:
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
merge_or_append_local(local_entry)
continue
if lora_hash and not resource_lora_count:
loras.append(lora_entry)
# Try to get base model from resources or make educated guess
+95 -13
View File
@@ -4,7 +4,7 @@ import json
import logging
from typing import Dict, Any, Union
from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS
from ..constants import GEN_PARAM_KEYS, VALID_LORA_TYPES
from ...services.metadata_service import get_default_metadata_provider
from ...config import config
@@ -14,15 +14,16 @@ logger = logging.getLogger(__name__)
class CivitaiApiMetadataParser(RecipeMetadataParser):
"""Parser for Civitai image metadata format"""
def is_metadata_matching(self, metadata) -> bool:
def is_metadata_matching(self, user_comment) -> bool:
"""Check if the metadata matches the Civitai image metadata format
Args:
metadata: The metadata from the image (dict)
user_comment: The metadata from the image (dict)
Returns:
bool: True if this parser can handle the metadata
"""
metadata = user_comment
if not metadata or not isinstance(metadata, dict):
return False
@@ -73,7 +74,7 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
return False
async def parse_metadata( # type: ignore[override]
async def parse_metadata( # pyright: ignore[reportIncompatibleMethodOverride]
self, user_comment, recipe_scanner=None, civitai_client=None,
local_cache: dict[str, Any] | None = None,
) -> Dict[str, Any]:
@@ -89,8 +90,7 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
Returns:
Dict containing parsed recipe data
"""
metadata: Dict[str, Any] = user_comment # type: ignore[assignment]
metadata = user_comment
metadata: Dict[str, Any] = user_comment
try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
@@ -115,8 +115,29 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
):
metadata = inner_meta
# Civitai's image API meta parser mangles the A1111 "Lora hashes"
# text field into a quote-wrapped dict entry:
# '"Daphne Blake Cosplay_v1": "e67ebd5e315f"'
# The 12-char AutoV3 it carries is more reliable than the stale
# 10-char AutoV2 value in the "hashes" dict, so recover it and
# let it override the conflicting entry.
if isinstance(metadata, dict):
for key, hash_value in list(metadata.items()):
if (
isinstance(key, str)
and key.startswith('"')
and isinstance(hash_value, str)
and hash_value.endswith('"')
):
clean_name = key.strip('"').strip()
clean_hash = hash_value.strip('"').strip()
if clean_name and clean_hash:
hashes_dict = metadata.get("hashes")
if isinstance(hashes_dict, dict):
hashes_dict[f"lora:{clean_name}"] = clean_hash
# Initialize result structure
result = {
result: Dict[str, Any] = {
"base_model": None,
"loras": [],
"model": None,
@@ -125,10 +146,10 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
}
# Track already added LoRAs to prevent duplicates
added_loras = {} # key: model_version_id or hash, value: index in result["loras"]
added_loras: Dict[str, Any] = {} # key: model_version_id or hash, value: index in result["loras"]
# Extract hash information from hashes field for LoRA matching
lora_hashes = {}
lora_hashes: Dict[str, Any] = {}
if "hashes" in metadata and isinstance(metadata["hashes"], dict):
for key, hash_value in metadata["hashes"].items():
key_str = str(key)
@@ -184,7 +205,7 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
if model_info:
result["base_model"] = model_info.get("baseModel", "")
base_model_counts = {}
base_model_counts: Dict[str, int] = {}
# Process standard resources array
if "resources" in metadata and isinstance(metadata["resources"], list):
@@ -196,7 +217,7 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# identification because it has an explicit type field and hash,
# unlike modelVersionIds which is a flat list with no type info.
if resource_type == "model":
checkpoint_entry = {
checkpoint_entry: Dict[str, Any] = {
"id": 0,
"modelId": 0,
"name": resource.get("name", "Unknown Model"),
@@ -216,7 +237,8 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Try to look up base model from the checkpoint hash
cp_hash = checkpoint_entry.get("hash")
if cp_hash and metadata_provider:
local_cached = local_cache.get(cp_hash) if local_cache else None
# local_cache keys are stored lowercase
local_cached = local_cache.get(cp_hash.lower()) if local_cache else None
if local_cached:
self._populate_entry_from_cache(
checkpoint_entry, local_cached
@@ -294,8 +316,15 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Try to get info from Civitai if hash is available
if lora_hash and metadata_provider:
local_cached = local_cache.get(lora_hash) if local_cache else None
# local_cache keys are stored lowercase
local_cached = local_cache.get(lora_hash.lower()) if local_cache else None
if local_cached:
cached_type = self._cache_item_model_type(local_cached)
if cached_type and cached_type not in VALID_LORA_TYPES:
logger.debug(
f"Skipping non-LoRA cache item for hash {lora_hash}"
)
continue
self._populate_entry_from_cache(
lora_entry, local_cached
)
@@ -304,6 +333,12 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
added_loras[str(lora_entry["id"])] = len(
result["loras"]
)
# Mirror base.py:150-151 counts for API-path loras
bm = local_cached.get("base_model") or ""
if bm:
base_model_counts[bm] = base_model_counts.get(
bm, 0
) + 1
else:
try:
civitai_info = (
@@ -649,6 +684,23 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
}
if metadata_provider:
# local_cache keys are stored lowercase
local_cached = local_cache.get(lora_hash.lower()) if local_cache else None
if local_cached:
cached_type = self._cache_item_model_type(local_cached)
if cached_type and cached_type not in VALID_LORA_TYPES:
logger.debug(
f"Skipping non-LoRA cache item for hash {lora_hash}"
)
continue
self._populate_entry_from_cache(lora_entry, local_cached)
# Mirror base.py:150-151 counts for API-path loras
bm = local_cached.get("base_model") or ""
if bm:
base_model_counts[bm] = base_model_counts.get(bm, 0) + 1
if "id" in lora_entry and lora_entry["id"]:
added_loras[str(lora_entry["id"])] = len(result["loras"])
else:
try:
civitai_info = await metadata_provider.get_model_by_hash(
lora_hash
@@ -711,6 +763,25 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Try to get info from Civitai if hash is available
if lora_entry["hash"] and metadata_provider:
# local_cache keys are stored lowercase
local_cached = local_cache.get(lora_hash.lower()) if local_cache else None
if local_cached:
cached_type = self._cache_item_model_type(local_cached)
if cached_type and cached_type not in VALID_LORA_TYPES:
logger.debug(
f"Skipping non-LoRA cache item for hash {lora_hash}"
)
lora_index += 1
continue # Skip non-LoRA cache items
self._populate_entry_from_cache(lora_entry, local_cached)
# Mirror base.py:150-151 counts for API-path loras
bm = local_cached.get("base_model") or ""
if bm:
base_model_counts[bm] = base_model_counts.get(bm, 0) + 1
# If we have a version ID from Civitai, track it for deduplication
if "id" in lora_entry and lora_entry["id"]:
added_loras[str(lora_entry["id"])] = len(result["loras"])
else:
try:
civitai_info = await metadata_provider.get_model_by_hash(
lora_hash
@@ -795,3 +866,14 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
base_model = cache_item.get("base_model", "")
if base_model:
entry["baseModel"] = base_model
@staticmethod
def _cache_item_model_type(cache_item: dict[str, Any]) -> str:
"""Lowercased civitai.model.type of a cache item, or '' when unknown."""
civ = cache_item.get("civitai")
if not isinstance(civ, dict):
return ""
model_info = civ.get("model")
if not isinstance(model_info, dict):
return ""
return (model_info.get("type") or "").lower()
+104 -67
View File
@@ -31,79 +31,25 @@ class ComfyMetadataParser(RecipeMetadataParser):
metadata_provider = await get_default_metadata_provider()
data = json.loads(user_comment)
loras = []
# Find all LoraLoader nodes
lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'}
# Process each LoraLoader node
for node_id, node in lora_nodes.items():
if 'inputs' not in node or 'lora_name' not in node['inputs']:
continue
lora_name = node['inputs'].get('lora_name', '')
# Parse the URN to extract model ID and version ID
# Format: "urn:air:sdxl:lora:civitai:1107767@1253442"
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
if not lora_id_match:
continue
model_id = lora_id_match.group(1)
model_version_id = lora_id_match.group(2)
# Get strength from node inputs
weight = node['inputs'].get('strength_model', 1.0)
# Initialize lora entry with default values
lora_entry = {
'id': model_version_id,
'modelId': model_id,
'name': f"Lora {model_id}", # Default name
'version': '',
'type': 'lora',
'weight': weight,
'existsLocally': False,
'localPath': None,
'file_name': '',
'hash': '',
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
# Get additional info from Civitai if metadata provider is available
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
loras.append(lora_entry)
# Find checkpoint info
checkpoint_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'CheckpointLoaderSimple'}
checkpoint = None
checkpoint_id = None
checkpoint_version_id = None
if checkpoint_nodes:
# Get the first checkpoint node
checkpoint_node = next(iter(checkpoint_nodes.values()))
if 'inputs' in checkpoint_node and 'ckpt_name' in checkpoint_node['inputs']:
checkpoint_name = checkpoint_node['inputs']['ckpt_name']
# Parse checkpoint URN
# Some ComfyUI workflows serialize ckpt_name as a
# single-element list (e.g. ["model.safetensors"]) or leave
# the value unset (None). Neither is a string, so skip the
# CivitAI-URN lookup instead of crashing re.search with a
# TypeError that fails the whole image import.
if isinstance(checkpoint_name, list):
checkpoint_name = (
checkpoint_name[0] if checkpoint_name else None
)
if isinstance(checkpoint_name, str):
checkpoint_match = re.search(r'civitai:(\d+)@(\d+)', checkpoint_name)
if checkpoint_match:
checkpoint_id = checkpoint_match.group(1)
@@ -115,17 +61,108 @@ class ComfyMetadataParser(RecipeMetadataParser):
'version': '',
'type': 'checkpoint'
}
# Get additional checkpoint info from Civitai
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(checkpoint_version_id)
civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
# Populate checkpoint with Civitai info
checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint: {e}")
recipe_base_model = checkpoint.get('baseModel') if checkpoint else None
loras = []
lora_candidates = []
for node in data.values():
if not isinstance(node, dict):
continue
inputs = node.get('inputs')
if not isinstance(inputs, dict):
continue
if node.get('class_type') == 'LoraLoader':
lora_name = inputs.get('lora_name', '')
if isinstance(lora_name, str) and lora_name:
lora_candidates.append((lora_name, inputs.get('strength_model', 1.0)))
continue
if node.get('class_type') != 'LoraLoaderLM':
continue
loras_data = inputs.get('loras', [])
if isinstance(loras_data, dict):
loras_data = loras_data.get('__value__', [])
if isinstance(loras_data, list) and len(loras_data) == 1 and isinstance(loras_data[0], list):
loras_data = loras_data[0]
if not isinstance(loras_data, list):
continue
for lora in loras_data:
if not isinstance(lora, dict) or not lora.get('active', False) or lora.get('_isDummy', False):
continue
lora_name = lora.get('name', '')
if isinstance(lora_name, str) and lora_name:
lora_candidates.append((lora_name, lora.get('strength', 1.0)))
for lora_name, weight in lora_candidates:
if isinstance(weight, str):
try:
weight = float(weight)
except ValueError:
weight = 1.0
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
if lora_id_match:
model_id = lora_id_match.group(1)
model_version_id = lora_id_match.group(2)
entry_name = f"Lora {model_id}"
else:
model_id = 0
model_version_id = 0
entry_name = re.split(r'[\\/]', lora_name)[-1]
entry_name = re.sub(r'\.[^.]+$', '', entry_name)
lora_entry = {
'id': model_version_id,
'modelId': model_id,
'name': entry_name,
'version': '',
'type': 'lora',
'weight': weight,
'existsLocally': False,
'localPath': None,
'file_name': entry_name,
'hash': '',
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if lora_id_match:
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner
)
if populated_entry is None:
continue
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
else:
if not recipe_scanner:
continue
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if not local_lora:
continue
lora_entry = self.populate_lora_from_local(lora_entry, local_lora)
loras.append(lora_entry)
# Extract generation parameters
gen_params = {}
+1 -1
View File
@@ -30,7 +30,7 @@ class MetaFormatParser(RecipeMetadataParser):
prompt = parts[0].strip()
# Initialize metadata
metadata = {"prompt": prompt, "loras": []}
metadata: Dict[str, Any] = {"prompt": prompt, "loras": []}
# Extract negative prompt and parameters if available
if len(parts) > 1:
+10 -2
View File
@@ -91,7 +91,15 @@ class RecipeFormatParser(RecipeMetadataParser):
exists_locally = lora_scanner.has_hash(lora['hash'])
if exists_locally:
lora_cache = await lora_scanner.get_cached_data()
lora_item = next((item for item in lora_cache.raw_data if item['sha256'].lower() == lora['hash'].lower()), None)
# Cascade match: full sha256, stored autov3, or autov2 (sha256[:10]).
h = (lora.get('hash') or '').lower()
lora_item = next(
(item for item in lora_cache.raw_data
if (item.get("sha256") or "").lower() == h
or (item.get("autov3") or "").lower() == h
or (item.get("sha256") or "")[:10].lower() == h),
None
)
if lora_item:
lora_entry['existsLocally'] = True
lora_entry['inLibrary'] = True
@@ -148,7 +156,7 @@ class RecipeFormatParser(RecipeMetadataParser):
checkpoint_data = recipe_metadata.get('checkpoint') or {}
if isinstance(checkpoint_data, dict) and checkpoint_data:
version_id = checkpoint_data.get('modelVersionId') or checkpoint_data.get('id')
checkpoint_entry = {
checkpoint_entry: Dict[str, Any] = {
'id': version_id or 0,
'modelId': checkpoint_data.get('modelId', 0),
'name': checkpoint_data.get('name', 'Unknown Checkpoint'),
+9 -8
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import logging
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Callable, Dict, Mapping
from typing import TYPE_CHECKING, Awaitable, Callable, Dict, Mapping
import jinja2
from aiohttp import web
@@ -30,6 +30,7 @@ from ..services.websocket_progress_callback import (
WebSocketProgressCallback,
)
from ..utils.exif_utils import ExifUtils
from ..utils.constants import MODEL_WEIGHT_FILE_TYPES
from ..utils.metadata_manager import MetadataManager
from .model_route_registrar import COMMON_ROUTE_DEFINITIONS, ModelRouteRegistrar
from .handlers.model_handlers import (
@@ -84,7 +85,7 @@ class BaseModelRoutes(ABC):
self.metadata_progress_callback = WebSocketBroadcastCallback()
self._handler_set: ModelHandlerSet | None = None
self._handler_mapping: Dict[str, Callable[[web.Request], web.StreamResponse]] | None = None
self._handler_mapping: Dict[str, Callable[[web.Request], Awaitable[web.Response]]] | None = None
self._preview_service = PreviewAssetService(
metadata_manager=MetadataManager,
@@ -131,7 +132,7 @@ class BaseModelRoutes(ABC):
self._handler_set = None
self._handler_mapping = None
def _ensure_handler_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
def _ensure_handler_mapping(self) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
if self._handler_mapping is None:
handler_set = self._create_handler_set()
self._handler_set = handler_set
@@ -220,7 +221,7 @@ class BaseModelRoutes(ABC):
)
@property
def route_handlers(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
def route_handlers(self) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
return self._ensure_handler_mapping()
def setup_routes(self, app: web.Application, prefix: str) -> None:
@@ -237,7 +238,7 @@ class BaseModelRoutes(ABC):
"""Setup model-specific routes."""
raise NotImplementedError
def _parse_specific_params(self, request: web.Request) -> Dict:
def _parse_specific_params(self, request: web.Request) -> Dict[str, Any]:
"""Parse model-specific parameters - to be overridden by subclasses."""
return {}
@@ -251,9 +252,9 @@ class BaseModelRoutes(ABC):
def _find_model_file(self, files):
"""Find the appropriate model file from the files list - can be overridden by subclasses."""
return next((file for file in files if file.get("type") in ("Model", "Diffusion Model") and file.get("primary") is True), None)
return next((file for file in files if file.get("type") in MODEL_WEIGHT_FILE_TYPES and file.get("primary") is True), None)
def get_handler(self, name: str) -> Callable[[web.Request], web.StreamResponse]:
def get_handler(self, name: str) -> Callable[[web.Request], Awaitable[web.StreamResponse]]:
"""Expose handlers for subclasses or tests."""
return self._ensure_handler_mapping()[name]
@@ -285,7 +286,7 @@ class BaseModelRoutes(ABC):
)
return self.model_lifecycle_service
def _make_handler_proxy(self, name: str) -> Callable[[web.Request], web.StreamResponse]:
def _make_handler_proxy(self, name: str) -> Callable[[web.Request], Awaitable[web.StreamResponse]]:
async def proxy(request: web.Request) -> web.StreamResponse:
try:
handler = self.get_handler(name)
+27 -9
View File
@@ -4,7 +4,7 @@ from __future__ import annotations
import logging
import os
from typing import Callable, Mapping
from typing import Awaitable, Callable, Mapping
import jinja2
from aiohttp import web
@@ -32,6 +32,7 @@ from .handlers.recipe_handlers import (
RecipePageView,
RecipeQueryHandler,
RecipeSharingHandler,
RecipeWorkflowHandler,
)
from .recipe_route_registrar import ROUTE_DEFINITIONS
@@ -61,7 +62,9 @@ class BaseRecipeRoutes:
self._i18n_registered = False
self._startup_hooks_registered = False
self._handler_set: RecipeHandlerSet | None = None
self._handler_mapping: dict[str, Callable] | None = None
self._handler_mapping: Mapping[
str, Callable[[web.Request], Awaitable[web.StreamResponse]]
] | None = None
async def attach_dependencies(self, app: web.Application | None = None) -> None:
"""Resolve shared services from the registry."""
@@ -84,7 +87,9 @@ class BaseRecipeRoutes:
app.on_startup.append(self.attach_dependencies)
self._startup_hooks_registered = True
def to_route_mapping(self) -> Mapping[str, Callable]:
def to_route_mapping(
self,
) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
"""Return a mapping of handler name to coroutine for registrar binding."""
if self._handler_mapping is None:
@@ -124,17 +129,17 @@ class BaseRecipeRoutes:
or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
)
if not standalone_mode:
from ..metadata_collector import get_metadata # type: ignore[import-not-found]
from ..metadata_collector.metadata_processor import ( # type: ignore[import-not-found]
from ..metadata_collector import get_metadata # pyright: ignore[reportMissingImports]
from ..metadata_collector.metadata_processor import ( # pyright: ignore[reportMissingImports]
MetadataProcessor,
)
from ..metadata_collector.metadata_registry import ( # type: ignore[import-not-found]
from ..metadata_collector.metadata_registry import ( # pyright: ignore[reportMissingImports]
MetadataRegistry,
)
else: # pragma: no cover - optional dependency path
get_metadata = None # type: ignore[assignment]
MetadataProcessor = None # type: ignore[assignment]
MetadataRegistry = None # type: ignore[assignment]
get_metadata = None # pyright: ignore[reportAssignmentType]
MetadataProcessor = None # pyright: ignore[reportAssignmentType]
MetadataRegistry = None # pyright: ignore[reportAssignmentType]
analysis_service = RecipeAnalysisService(
exif_utils=ExifUtils,
@@ -196,6 +201,18 @@ class BaseRecipeRoutes:
sharing_service=sharing_service,
)
# Lazy import: standalone mode replaces the ``server`` module with a
# mock, so resolve PromptServer at handler-set build time instead of
# module import time. The handler's standalone check guards UX.
from server import PromptServer # pyright: ignore[reportMissingImports]
workflow = RecipeWorkflowHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
prompt_server=PromptServer,
logger=logger,
)
from ..services.websocket_manager import ws_manager
batch_import_service = BatchImportService(
@@ -220,4 +237,5 @@ class BaseRecipeRoutes:
analysis=analysis,
sharing=sharing,
batch_import=batch_import,
workflow=workflow,
)
+48 -8
View File
@@ -1,5 +1,6 @@
import logging
from typing import Dict, List, Set
import os
from typing import Any, Dict, List, Set
from aiohttp import web
from .base_model_routes import BaseModelRoutes
@@ -7,6 +8,7 @@ from .model_route_registrar import ModelRouteRegistrar
from ..services.checkpoint_service import CheckpointService
from ..services.service_registry import ServiceRegistry
from ..config import config
from ..utils.utils import _format_model_name_for_comfyui
logger = logging.getLogger(__name__)
@@ -28,13 +30,13 @@ class CheckpointRoutes(BaseModelRoutes):
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
def setup_routes(self, app: web.Application, prefix: str = "checkpoints"):
"""Setup Checkpoint routes"""
# Schedule service initialization on app startup
app.on_startup.append(lambda _: self.initialize_services())
# Setup common routes with 'checkpoints' prefix (includes page route)
super().setup_routes(app, 'checkpoints')
super().setup_routes(app, prefix)
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup Checkpoint-specific routes"""
@@ -45,6 +47,44 @@ class CheckpointRoutes(BaseModelRoutes):
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/checkpoints_roots', prefix, self.get_checkpoints_roots)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/unet_roots', prefix, self.get_unet_roots)
# Name/base_model pool for the Random Checkpoint/Unet Loader nodes
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/loader-pool', prefix, self.get_loader_pool)
async def get_loader_pool(self, request: web.Request) -> web.Response:
"""Return ComfyUI-formatted model names with their base_model.
Backing data for the Random Checkpoint/Unet Loader nodes: the front-end
filters the ckpt_name/unet_name combo options by base_model using this
pool, so control_after_generate randomizes within the narrowed set.
"""
try:
sub_type = request.query.get("sub_type", "checkpoint")
if sub_type not in ("checkpoint", "diffusion_model"):
return web.json_response({"error": "invalid sub_type"}, status=400)
scanner = await ServiceRegistry.get_checkpoint_scanner()
cache = await scanner.get_cached_data()
model_roots = scanner.get_model_roots()
items: List[Dict[str, str]] = []
for item in cache.raw_data:
if item.get("sub_type") != sub_type:
continue
file_path = item.get("file_path", "")
if not file_path or not os.path.exists(file_path):
continue
formatted_name = _format_model_name_for_comfyui(file_path, model_roots)
if formatted_name:
items.append(
{
"name": formatted_name,
"base_model": item.get("base_model", "") or "",
}
)
items.sort(key=lambda x: x["name"])
return web.json_response({"items": items})
except Exception as e:
logger.error(f"Error getting loader pool: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500)
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for Checkpoint"""
return model_type.lower() == 'checkpoint'
@@ -53,9 +93,9 @@ class CheckpointRoutes(BaseModelRoutes):
"""Get expected model types string for error messages"""
return "Checkpoint"
def _parse_specific_params(self, request: web.Request) -> Dict:
def _parse_specific_params(self, request: web.Request) -> Dict[str, Any]:
"""Parse Checkpoint-specific parameters"""
params: Dict = {}
params: Dict[str, Any] = {}
if 'checkpoint_hash' in request.query:
params['hash_filters'] = {'single_hash': request.query['checkpoint_hash'].lower()}
@@ -70,7 +110,7 @@ class CheckpointRoutes(BaseModelRoutes):
"""Get detailed information for a specific checkpoint by name"""
try:
name = request.match_info.get('name', '')
checkpoint_info = await self.service.get_model_info_by_name(name)
checkpoint_info = await self.service.get_model_info_by_name(name) # pyright: ignore[reportAttributeAccessIssue]
if checkpoint_info:
return web.json_response(checkpoint_info)
@@ -89,7 +129,7 @@ class CheckpointRoutes(BaseModelRoutes):
roots.extend(config.checkpoints_roots or [])
roots.extend(config.extra_checkpoints_roots or [])
# Remove duplicates while preserving order
seen: set = set()
seen: set[str] = set()
unique_roots: List[str] = []
for root in roots:
if root and root not in seen:
@@ -114,7 +154,7 @@ class CheckpointRoutes(BaseModelRoutes):
roots.extend(config.unet_roots or [])
roots.extend(config.extra_unet_roots or [])
# Remove duplicates while preserving order
seen: set = set()
seen: set[str] = set()
unique_roots: List[str] = []
for root in roots:
if root and root not in seen:
+3 -3
View File
@@ -26,13 +26,13 @@ class EmbeddingRoutes(BaseModelRoutes):
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
def setup_routes(self, app: web.Application, prefix: str = "embeddings"):
"""Setup Embedding routes"""
# Schedule service initialization on app startup
app.on_startup.append(lambda _: self.initialize_services())
# Setup common routes with 'embeddings' prefix (includes page route)
super().setup_routes(app, 'embeddings')
super().setup_routes(app, prefix)
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup Embedding-specific routes"""
@@ -51,7 +51,7 @@ class EmbeddingRoutes(BaseModelRoutes):
"""Get detailed information for a specific embedding by name"""
try:
name = request.match_info.get('name', '')
embedding_info = await self.service.get_model_info_by_name(name)
embedding_info = await self.service.get_model_info_by_name(name) # pyright: ignore[reportAttributeAccessIssue]
if embedding_info:
return web.json_response(embedding_info)
+8 -4
View File
@@ -1,7 +1,7 @@
from __future__ import annotations
import logging
from typing import Callable, Mapping
from typing import Any, Awaitable, Callable, Mapping
from aiohttp import web
@@ -35,7 +35,7 @@ class ExampleImagesRoutes:
*,
ws_manager,
download_manager: DownloadManager | None = None,
processor=ExampleImagesProcessor,
processor: Any = ExampleImagesProcessor,
file_manager=ExampleImagesFileManager,
cleanup_service: ExampleImagesCleanupService | None = None,
) -> None:
@@ -46,7 +46,9 @@ class ExampleImagesRoutes:
self._file_manager = file_manager
self._cleanup_service = cleanup_service or ExampleImagesCleanupService()
self._handler_set: ExampleImagesHandlerSet | None = None
self._handler_mapping: Mapping[str, Callable[[web.Request], web.StreamResponse]] | None = None
self._handler_mapping: Mapping[
str, Callable[[web.Request], Awaitable[web.StreamResponse]]
] | None = None
@classmethod
def setup_routes(cls, app: web.Application, *, ws_manager) -> None:
@@ -61,7 +63,9 @@ class ExampleImagesRoutes:
registrar = ExampleImagesRouteRegistrar(app)
registrar.register_routes(self.to_route_mapping())
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
def to_route_mapping(
self,
) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
"""Return the registrar-compatible mapping of handler names to callables."""
if self._handler_mapping is None:
+165
View File
@@ -0,0 +1,165 @@
"""HTTP route handlers for agent skill endpoints.
These handlers expose the :class:`AgentService` via HTTP, allowing the
frontend to list available skills and execute them on selected models.
Progress is reported via WebSocket broadcast.
"""
from __future__ import annotations
import asyncio
import logging
from typing import Any, Dict
from aiohttp import web
from ...services.agent import AgentService, AgentProgressReporter
from ...services.llm_service import LLMNotConfiguredError
logger = logging.getLogger(__name__)
class AgentHandler:
"""HTTP handler for agent skill operations."""
def __init__(self, agent_service: AgentService | None = None) -> None:
self._agent_service = agent_service
async def _ensure_service(self) -> AgentService:
if self._agent_service is None:
self._agent_service = await AgentService.get_instance()
return self._agent_service
# ------------------------------------------------------------------
# GET /api/lm/agent/skills
# ------------------------------------------------------------------
async def get_agent_skills(self, request: web.Request) -> web.Response:
"""Return a list of available agent skills."""
service = await self._ensure_service()
skills = await service.list_skills()
return web.json_response({"skills": skills})
# ------------------------------------------------------------------
# POST /api/lm/agent/execute/{skill_name}
# ------------------------------------------------------------------
async def execute_agent_skill(self, request: web.Request) -> web.Response:
"""Execute an agent skill on the provided model paths.
Request body::
{"model_paths": ["/path/to/model1.safetensors", ...], "options": {}}
Returns immediately with a task ID. Execution runs in the
background; progress and completion are pushed via WebSocket
events of type ``agent_progress``.
"""
skill_name = request.match_info.get("skill_name", "")
if not skill_name:
return web.json_response(
{"error": "Skill name is required"}, status=400
)
try:
body = await request.json()
except Exception:
return web.json_response(
{"error": "Invalid JSON body"}, status=400
)
model_paths = body.get("model_paths", [])
if not model_paths or not isinstance(model_paths, list):
return web.json_response(
{"error": "model_paths must be a non-empty array"},
status=400,
)
service = await self._ensure_service()
# Validate LLM configuration early for skills that need it
# (fail fast rather than after starting background work)
try:
from ...services.llm_service import LLMService
llm = await LLMService.get_instance()
if not llm.is_configured():
return web.json_response(
{
"error": "LLM provider is not configured. "
"Enable it in Settings → AI Provider.",
},
status=400,
)
except Exception as exc:
logger.error("Failed to check LLM configuration: %s", exc)
# Launch execution in the background
progress_reporter = AgentProgressReporter()
logger.info(
"LLM enrichment '%s' starting for %d model(s)",
skill_name, len(model_paths),
)
async def _run() -> None:
try:
result = await service.execute_skill(
skill_name=skill_name,
input_data={"model_paths": model_paths},
progress_callback=progress_reporter,
)
logger.info(
"LLM enrichment '%s' finished: success=%s, summary='%s', errors=%s",
skill_name, result.success, result.summary, result.errors,
)
except LLMNotConfiguredError as exc:
logger.warning("LLM enrichment '%s' not configured: %s", skill_name, exc)
await progress_reporter.on_progress(
{
"type": "agent_progress",
"skill": skill_name,
"status": "error",
"error": str(exc),
}
)
except Exception as exc:
logger.error("LLM enrichment '%s' failed: %s", skill_name, exc, exc_info=True)
await progress_reporter.on_progress(
{
"type": "agent_progress",
"skill": skill_name,
"status": "error",
"error": str(exc),
}
)
# Fire and forget — progress comes via WebSocket
asyncio.create_task(_run())
return web.json_response(
{
"status": "started",
"skill": skill_name,
"model_count": len(model_paths),
}
)
# ------------------------------------------------------------------
# POST /api/lm/agent/cancel
# ------------------------------------------------------------------
async def cancel_agent_skill(self, request: web.Request) -> web.Response:
"""Cancel a running agent skill.
NOTE: Cancellation is a stub for now the AgentService processes
models sequentially and does not yet support mid-execution
cancellation. This endpoint exists for API completeness.
"""
# TODO: implement cooperative cancellation in AgentService
return web.json_response(
{"status": "acknowledged", "note": "Cancellation not yet implemented"},
status=200,
)
@@ -3,7 +3,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import Callable, Mapping
from typing import Awaitable, Callable, Mapping
from aiohttp import web
@@ -170,7 +170,7 @@ class ExampleImagesHandlerSet:
management: ExampleImagesManagementHandler
files: ExampleImagesFileHandler
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]:
"""Flatten handler methods into the registrar mapping."""
return {
+138 -39
View File
@@ -49,6 +49,14 @@ async def _get_hf_api_session() -> aiohttp.ClientSession:
return _hf_api_session
async def close_hf_api_session() -> None:
"""Close the shared HF API session, if it was ever created."""
global _hf_api_session
if _hf_api_session is not None and not _hf_api_session.closed:
await _hf_api_session.close()
_hf_api_session = None
def _infer_model_type(model_root: str) -> tuple[Any, str]:
"""Determine model class and scanner by matching ``model_root`` against the
configured root paths for each model type (from ``Config``).
@@ -114,8 +122,12 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
metadata._unknown_fields["hf_url"] = hf_url
metadata.from_civitai = False # HF models are not from CivitAI
metadata_dict = metadata.to_dict()
if "trainedWords" in metadata_dict and not metadata_dict["trainedWords"]:
del metadata_dict["trainedWords"]
# 3. Save metadata atomically
await MetadataManager.save_metadata(dest_path, metadata)
await MetadataManager.save_metadata(dest_path, metadata_dict)
logger.info("Saved HF metadata (with hf_url) for %s", dest_path)
# 4. Determine relative folder path for cache
@@ -139,9 +151,117 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
logger.warning("Failed to save HF metadata for %s: %s", dest_path, exc)
def _find_matching_root(dest_dir: str) -> str | None:
"""Walk up *dest_dir* to find which configured scanner root it belongs to."""
norm = os.path.normpath(dest_dir).replace(os.sep, "/")
all_roots = []
for root_list in (
config.loras_roots or [],
config.extra_loras_roots or [],
config.checkpoints_roots or [],
config.extra_checkpoints_roots or [],
config.unet_roots or [],
config.extra_unet_roots or [],
config.embeddings_roots or [],
config.extra_embeddings_roots or [],
):
all_roots.extend([os.path.normpath(p).replace(os.sep, "/") for p in root_list])
# Find the longest matching prefix
match: str | None = None
for root in all_roots:
if norm.startswith(root):
if match is None or len(root) > len(match):
match = root
return match
async def _add_to_scanner_cache(dest_path: str, metadata: dict[str, Any]) -> None:
model_dir = os.path.dirname(dest_path)
model_root = _find_matching_root(model_dir)
if not model_root:
raise ValueError(f"File path {dest_path} is not within any configured scanner root")
scanner_getter_name = _infer_model_type(model_root)[1]
scanner_getter = getattr(ServiceRegistry, scanner_getter_name, None)
if scanner_getter is None:
raise RuntimeError(f"Scanner getter '{scanner_getter_name}' not found in ServiceRegistry")
scanner = await scanner_getter()
if scanner is None:
raise RuntimeError(f"Scanner '{scanner_getter_name}' returned None")
await scanner.update_single_model_cache(dest_path, dest_path, metadata)
class HfHandler:
"""Handle Hugging Face model browsing and download."""
async def set_hf_url(self, request: web.Request) -> web.Response:
try:
payload: dict[str, Any] = await request.json()
except json.JSONDecodeError:
return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
file_path = (payload.get("file_path") or "").strip()
hf_url = (payload.get("hf_url") or "").strip()
if not file_path or not hf_url:
return web.json_response(
{"success": False, "error": "Missing required fields: 'file_path' and 'hf_url'"},
status=400,
)
m = re.match(r"^https?://huggingface\.co/([^/]+/[^/]+)/?$", hf_url)
if not m:
return web.json_response(
{
"success": False,
"error": "Invalid HuggingFace URL. Expected format: https://huggingface.co/user/repo",
},
status=400,
)
if not os.path.isfile(file_path):
return web.json_response(
{"success": False, "error": f"File not found: {file_path}"},
status=404,
)
model_root = _find_matching_root(os.path.dirname(file_path))
if not model_root:
return web.json_response(
{
"success": False,
"error": "File is not within any configured model directory. Cannot link to HuggingFace.",
},
status=400,
)
try:
existing = await MetadataManager.load_metadata_payload(file_path)
if existing.get("hf_url") == hf_url:
return web.json_response({
"success": True,
"message": "hf_url already set",
"hf_url": hf_url,
})
existing["hf_url"] = hf_url
existing["from_civitai"] = False
await MetadataManager.save_metadata(file_path, existing)
await _add_to_scanner_cache(file_path, existing)
logger.info("Set hf_url=%s for %s", hf_url, file_path)
return web.json_response({
"success": True,
"message": f"hf_url set to {hf_url}",
"hf_url": hf_url,
})
except Exception as exc:
logger.error("Failed to set hf_url for %s: %s", file_path, exc)
return web.json_response(
{"success": False, "error": str(exc)},
status=500,
)
async def get_hf_repo_files(self, request: web.Request) -> web.Response:
"""List model-weight files from a HF repo with real file sizes.
@@ -243,8 +363,8 @@ class HfHandler:
if ".." in (author, repo_name) or "." in (author, repo_name):
return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
# Validate filename — must not contain path separators or ..
if "/" in filename or "\\" in filename or ".." in filename:
# Validate filename — must not contain path traversal
if ".." in filename:
return web.json_response({"error": "Invalid filename"}, status=400)
# Validate relative_path — must not be absolute or escape base directory
@@ -254,35 +374,17 @@ class HfHandler:
if ".." in relative_path.split("/") or "\\" in relative_path:
return web.json_response({"error": "Invalid relative_path"}, status=400)
# Validate model_root — must not contain path traversal
if not os.path.isabs(model_root):
# For relative model_root, check it doesn't escape
resolved_model_root = os.path.realpath(
os.path.join(os.getcwd(), "models", model_root)
)
# Use model_root directly as the base directory — same approach as
# CivitAI's download path (download_manager.py). No realpath, no
# allowed-roots validation, no path-traversal check; those are
# unnecessary when the frontend sends the path from its own dropdown
# (populated from scanner roots). Using the "business path" directly
# keeps dest_path consistent with scanner roots so that later folder
# derivation (in _save_hf_metadata) works correctly.
if os.path.isabs(model_root):
base_dir = os.path.normpath(model_root)
else:
resolved_model_root = os.path.realpath(model_root)
# Verify model_root is within a configured scanner root
allowed_roots = set()
for root_list in (
config.loras_roots or [],
config.extra_loras_roots or [],
config.checkpoints_roots or [],
config.extra_checkpoints_roots or [],
config.unet_roots or [],
config.extra_unet_roots or [],
config.embeddings_roots or [],
config.extra_embeddings_roots or [],
):
for r in root_list:
allowed_roots.add(os.path.realpath(r))
if not any(resolved_model_root == root or resolved_model_root.startswith(root + os.sep) for root in allowed_roots):
logger.warning("Invalid model_root rejected: %s", model_root)
return web.json_response({"error": f"Invalid model_root: {model_root}"}, status=400)
base_dir = resolved_model_root
base_dir = os.path.normpath(os.path.join(os.getcwd(), "models", model_root))
if use_default_paths:
target_dir = os.path.join(base_dir, "huggingface", author, repo_name)
@@ -291,15 +393,12 @@ class HfHandler:
else:
target_dir = base_dir
os.makedirs(target_dir, exist_ok=True)
dest_path = os.path.join(target_dir, filename)
# Strip HF repo subdirectory — "diffusion_models/xxx.safetensors"
# is an HF repo convention, not meaningful for local storage.
file_base = os.path.basename(filename)
# Resolve symlinks and check for path traversal escape
real_dest = os.path.realpath(dest_path)
real_base = os.path.realpath(target_dir)
if not real_dest.startswith(real_base + os.sep):
logger.warning("Path traversal blocked: %s -> %s", dest_path, real_dest)
return web.json_response({"error": "Path traversal detected"}, status=400)
os.makedirs(target_dir, exist_ok=True)
dest_path = os.path.join(target_dir, file_base)
# Check if already exists (simple skip)
if os.path.exists(dest_path) and os.path.getsize(dest_path) > 0:
File diff suppressed because it is too large Load Diff
+442 -56
View File
@@ -51,6 +51,29 @@ LICENSE_FIELDS = (
)
_broadcast_models_changed_tasks: set = set()
def _broadcast_models_changed() -> None:
"""Notify connected clients that the local model library changed.
The ComfyUI graph page listens for this event to invalidate its cached
model availability data (loras widget missing-model cues / error flags)
without waiting for the cache TTL to expire.
"""
try:
from ...services.websocket_manager import ws_manager
task = asyncio.create_task(ws_manager.broadcast({"type": "models_changed"}))
# Keep a reference so the task is not garbage-collected mid-await.
_broadcast_models_changed_tasks.add(task)
task.add_done_callback(_broadcast_models_changed_tasks.discard)
except Exception:
logging.getLogger(__name__).debug(
"Failed to broadcast models_changed", exc_info=True
)
class ModelPageView:
"""Render the HTML view for model listings."""
@@ -71,7 +94,7 @@ class ModelPageView:
self._server_i18n = server_i18n
self._logger = logger
def _load_supporters(self) -> dict:
def _load_supporters(self) -> dict[str, Any]:
"""Load supporters data from JSON file."""
try:
current_file = os.path.abspath(__file__)
@@ -152,7 +175,15 @@ class ModelPageView:
self._template_env.filters["t"] = (
self._server_i18n.create_template_filter()
)
self._template_env._i18n_filter_added = True # type: ignore[attr-defined]
self._template_env._i18n_filter_added = True # pyright: ignore[reportAttributeAccessIssue]
from ...services.llm_service import PROVIDER_PRESETS
# Provider presets are embedded directly (local, no await needed).
# Provider model catalogs are fetched asynchronously by the
# frontend via GET /api/lm/llm/provider-models so page rendering
# never blocks on the remote model catalog (which can take up to
# 30s on cold cache).
template_context = {
"is_initializing": is_initializing,
@@ -161,6 +192,8 @@ class ModelPageView:
"folders": [],
"t": self._server_i18n.get_translation,
"version": self._get_app_version(),
"provider_presets_json": json.dumps(PROVIDER_PRESETS),
"provider_models_json": "{}",
}
if not is_initializing:
@@ -189,7 +222,7 @@ class ModelListingHandler:
self,
*,
service,
parse_specific_params: Callable[[web.Request], Dict],
parse_specific_params: Callable[[web.Request], Dict[str, Any]],
logger: logging.Logger,
) -> None:
self._service = service
@@ -277,7 +310,7 @@ class ModelListingHandler:
)
return web.json_response({"error": str(exc)}, status=500)
def _parse_common_params(self, request: web.Request) -> Dict:
def _parse_common_params(self, request: web.Request) -> Dict[str, Any]:
page = int(request.query.get("page", "1"))
page_size = min(int(request.query.get("page_size", "20")), 100)
sort_by = request.query.get("sort_by", "name")
@@ -331,6 +364,7 @@ class ModelListingHandler:
== "true",
"tags": request.query.get("search_tags", "false").lower() == "true",
"creator": request.query.get("search_creator", "false").lower() == "true",
"hash": request.query.get("search_hash", "false").lower() == "true",
"recursive": request.query.get("recursive", "true").lower() == "true",
}
@@ -384,12 +418,14 @@ class ModelListingHandler:
)
# View-local-versions filter: show all local versions of a specific model
# Accepts either a CivitAI modelId (int) or a HF group key like "hf:user/repo"
civitai_model_id = request.query.get("civitai_model_id")
if civitai_model_id is not None:
try:
civitai_model_id = int(civitai_model_id)
except (TypeError, ValueError):
civitai_model_id = None
# Keep as string — could be an HF group key (e.g. "hf:user/repo")
pass
return {
"page": page,
@@ -448,6 +484,7 @@ class ModelManagementHandler:
return web.Response(text="Model path is required", status=400)
result = await self._lifecycle_service.delete_model(file_path)
_broadcast_models_changed()
return web.json_response(result)
except ValueError as exc:
return web.json_response({"success": False, "error": str(exc)}, status=400)
@@ -527,6 +564,7 @@ class ModelManagementHandler:
# Update model_data with new hash
model_data["sha256"] = sha256
model_data["hash_status"] = "completed"
hash_status = "completed"
else:
return web.json_response(
{"success": False, "error": "No SHA256 hash found"}, status=400
@@ -534,6 +572,32 @@ class ModelManagementHandler:
await MetadataManager.hydrate_model_data(model_data)
# hydrate_model_data replaces model_data with .metadata.json content,
# which may lack sha256. Restore from cache and persist the fix.
if not model_data.get("sha256"):
if sha256:
model_data["sha256"] = sha256
model_data["hash_status"] = model_data.get("hash_status", hash_status)
data_to_save = model_data.copy()
data_to_save.pop("folder", None)
await MetadataManager.save_metadata(file_path, data_to_save)
else:
sha256 = await calculate_sha256(file_path)
if sha256:
model_data["sha256"] = sha256.lower()
model_data["hash_status"] = "completed"
data_to_save = model_data.copy()
data_to_save.pop("folder", None)
await MetadataManager.save_metadata(file_path, data_to_save)
else:
return web.json_response(
{
"success": False,
"error": "Failed to compute SHA256 hash for model",
},
status=500,
)
success, error = await self._metadata_sync.fetch_and_update_model(
sha256=model_data["sha256"],
file_path=file_path,
@@ -556,7 +620,12 @@ class ModelManagementHandler:
{"success": False, "error": OFFLINE_FRIENDLY_MESSAGE},
status=503,
)
self._logger.error("Error fetching from CivitAI: %s", exc, exc_info=True)
self._logger.error(
"Error fetching from CivitAI for %s: %s",
locals().get("file_path", "unknown"),
exc,
exc_info=True,
)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def relink_civitai(self, request: web.Request) -> web.Response:
@@ -565,6 +634,16 @@ class ModelManagementHandler:
file_path = data.get("file_path")
model_id = data.get("model_id")
model_version_id = data.get("model_version_id")
source = data.get("source")
if source not in (None, "", "civarchive"):
return web.json_response(
{
"success": False,
"error": f"Unsupported relink source: {source}",
},
status=400,
)
if not file_path or model_id is None:
return web.json_response(
@@ -580,19 +659,32 @@ class ModelManagementHandler:
metadata_path
)
relink_kwargs = {
"file_path": file_path,
"metadata": local_metadata,
"model_id": int(model_id),
"model_version_id": int(model_version_id) if model_version_id else None,
}
if source == "civarchive":
relink_kwargs["provider_name"] = "civarchive_api"
updated_metadata = await self._metadata_sync.relink_metadata(
file_path=file_path,
metadata=local_metadata,
model_id=int(model_id),
model_version_id=int(model_version_id) if model_version_id else None,
**relink_kwargs
)
await self._service.scanner.update_single_model_cache(
file_path, file_path, updated_metadata
)
message = f"Model successfully re-linked to Civitai model {model_id}" + (
f" version {model_version_id}" if model_version_id else ""
if source == "civarchive":
message = (
f"Model successfully re-linked to CivArchive model {model_id}"
+ (f" version {model_version_id}" if model_version_id else "")
)
else:
message = (
f"Model successfully re-linked to Civitai model {model_id}"
+ (f" version {model_version_id}" if model_version_id else "")
)
return web.json_response(
{
@@ -601,6 +693,8 @@ class ModelManagementHandler:
"hash": updated_metadata.get("sha256", ""),
}
)
except ValueError as exc:
return web.json_response({"success": False, "error": str(exc)}, status=400)
except Exception as exc:
if is_expected_offline_error(str(exc)):
return web.json_response(
@@ -614,7 +708,7 @@ class ModelManagementHandler:
try:
reader = await request.multipart()
field = await reader.next()
field: Any = await reader.next()
if field is None or field.name != "preview_file":
raise ValueError("Expected 'preview_file' field")
content_type = field.headers.get("Content-Type", "image/png")
@@ -656,7 +750,7 @@ class ModelManagementHandler:
{
"success": True,
"preview_url": config.get_preview_static_url(
result["preview_path"]
str(result["preview_path"])
),
"preview_nsfw_level": result["preview_nsfw_level"],
}
@@ -737,7 +831,7 @@ class ModelManagementHandler:
result = await self._preview_service.replace_preview(
model_path=model_path,
preview_data=preview_data,
preview_data=preview_bytes,
content_type=content_type,
original_filename=original_filename,
nsfw_level=nsfw_level,
@@ -749,7 +843,7 @@ class ModelManagementHandler:
{
"success": True,
"preview_url": config.get_preview_static_url(
result["preview_path"]
str(result["preview_path"])
),
"preview_nsfw_level": result["preview_nsfw_level"],
}
@@ -887,6 +981,8 @@ class ModelManagementHandler:
file_path=file_path, new_file_name=new_file_name
)
_broadcast_models_changed()
return web.json_response(
{
**result,
@@ -915,6 +1011,7 @@ class ModelManagementHandler:
)
result = await self._lifecycle_service.bulk_delete_models(file_paths)
_broadcast_models_changed()
return web.json_response(result)
except ValueError as exc:
return web.json_response({"success": False, "error": str(exc)}, status=400)
@@ -958,11 +1055,18 @@ class ModelQueryHandler:
self._service = service
self._logger = logger
@staticmethod
def _parse_include_empty(request: web.Request) -> bool:
"""Parse the include_empty query flag (``1``/``true``)."""
return request.query.get("include_empty", "").lower() in ("1", "true")
async def get_top_tags(self, request: web.Request) -> web.Response:
try:
limit = int(request.query.get("limit", "20"))
if limit < 0:
limit = 20
elif limit > 200:
limit = 20
top_tags = await self._service.get_top_tags(limit)
return web.json_response({"success": True, "tags": top_tags})
except Exception as exc:
@@ -971,6 +1075,22 @@ class ModelQueryHandler:
{"success": False, "error": "Internal server error"}, status=500
)
async def search_tags(self, request: web.Request) -> web.Response:
try:
query = request.query.get("q", "")
limit = int(request.query.get("limit", "20"))
if limit < 0:
limit = 20
elif limit > 200:
limit = 20
tags = await self._service.search_tags(query, limit)
return web.json_response({"success": True, "tags": tags})
except Exception as exc:
self._logger.error("Error searching tags: %s", exc, exc_info=True)
return web.json_response(
{"success": False, "error": "Internal server error"}, status=500
)
async def get_base_models(self, request: web.Request) -> web.Response:
try:
limit = int(request.query.get("limit", "20"))
@@ -999,6 +1119,7 @@ class ModelQueryHandler:
await self._service.scan_models(
force_refresh=True, rebuild_cache=full_rebuild
)
_broadcast_models_changed()
if self._service.scanner.is_cancelled():
return web.json_response(
{
@@ -1033,8 +1154,14 @@ class ModelQueryHandler:
async def get_folders(self, request: web.Request) -> web.Response:
try:
include_empty = self._parse_include_empty(request)
if include_empty:
# Live enumeration includes empty OS-created directories.
folders = await self._service.scanner.get_all_folders()
else:
cache = await self._service.scanner.get_cached_data()
return web.json_response({"folders": cache.folders})
folders = cache.folders
return web.json_response({"folders": folders})
except Exception as exc:
self._logger.error("Error getting folders: %s", exc)
return web.json_response({"success": False, "error": str(exc)}, status=500)
@@ -1059,7 +1186,9 @@ class ModelQueryHandler:
{"success": False, "error": "model_root parameter is required"},
status=400,
)
folder_tree = await self._service.get_folder_tree(model_root)
folder_tree = await self._service.get_folder_tree(
model_root, include_empty=self._parse_include_empty(request)
)
return web.json_response({"success": True, "tree": folder_tree})
except Exception as exc:
self._logger.error("Error getting folder tree: %s", exc)
@@ -1067,7 +1196,9 @@ class ModelQueryHandler:
async def get_unified_folder_tree(self, request: web.Request) -> web.Response:
try:
unified_tree = await self._service.get_unified_folder_tree()
unified_tree = await self._service.get_unified_folder_tree(
include_empty=self._parse_include_empty(request)
)
return web.json_response({"success": True, "tree": unified_tree})
except Exception as exc:
self._logger.error("Error getting unified folder tree: %s", exc)
@@ -1265,9 +1396,13 @@ class ModelQueryHandler:
text=f"{self._service.model_type.capitalize()} file name is required",
status=400,
)
notes = await self._service.get_model_notes(model_name)
if notes is not None:
return web.json_response({"success": True, "notes": notes})
result = await self._service.get_model_notes(model_name)
if result is not None:
return web.json_response({
"success": True,
"notes": result["notes"],
"file_path": result["file_path"],
})
return web.json_response(
{
"success": False,
@@ -1303,6 +1438,17 @@ class ModelQueryHandler:
}
if include_license_flags:
model_data = await self._service.get_model_info_by_name(model_name)
# Only return license_flags when real CivitAI model license
# data exists. This mirrors ModelModal's guard
# (modelData?.civitai?.model) so the preview tooltip never
# shows misleading license icons for HF or other models
# without actual license metadata.
civitai_data = (model_data or {}).get("civitai") or {}
has_license_data = (
isinstance(civitai_data, dict)
and isinstance(civitai_data.get("model"), dict)
)
if has_license_data:
license_flags = (model_data or {}).get("license_flags")
if license_flags is not None:
response_payload["license_flags"] = int(license_flags)
@@ -1411,8 +1557,73 @@ class ModelQueryHandler:
search = request.query.get("search", "").strip()
limit = min(int(request.query.get("limit", "15")), 100)
offset = max(0, int(request.query.get("offset", "0")))
folder = request.query.get("folder")
recursive = request.query.get("recursive", "true").lower() == "true"
base_models = list(request.query.getall("base_model", []))
model_types = list(request.query.getall("model_type", []))
tag_filters: Dict[str, str] = {}
for tag in request.query.getall("tag_include", []):
if tag:
tag_filters[tag] = "include"
for tag in request.query.getall("tag_exclude", []):
if tag:
tag_filters[tag] = "exclude"
auto_tag_filters: Dict[str, str] = {}
for tag in request.query.getall("auto_tag_include", []):
if tag:
auto_tag_filters[tag] = "include"
for tag in request.query.getall("auto_tag_exclude", []):
if tag:
auto_tag_filters[tag] = "exclude"
tag_logic = request.query.get("tag_logic", "any").lower()
if tag_logic not in ("any", "all"):
tag_logic = "any"
credit_required = request.query.get("credit_required")
if credit_required is not None:
credit_required = credit_required.lower() not in ("false", "0", "")
allow_selling_generated_content = request.query.get(
"allow_selling_generated_content"
)
if allow_selling_generated_content is not None:
allow_selling_generated_content = (
allow_selling_generated_content.lower() not in ("false", "0", "")
)
# The presence of the recursive param (always sent by the loras
# widget when filter mode is on) signals that the filter pipeline
# must run even when no concrete filter is set, so global settings
# like show_only_sfw stay consistent with the list endpoint.
apply_filters = (
"recursive" in request.query
or folder is not None
or bool(base_models)
or bool(model_types)
or bool(tag_filters)
or bool(auto_tag_filters)
or credit_required is not None
or allow_selling_generated_content is not None
)
matching_paths = await self._service.search_relative_paths(
search, limit, offset
search,
limit,
offset,
folder=folder,
recursive=recursive,
base_models=base_models,
model_types=model_types,
tags=tag_filters,
auto_tags=auto_tag_filters,
tag_logic=tag_logic,
credit_required=credit_required,
allow_selling_generated_content=allow_selling_generated_content,
apply_filters=apply_filters,
)
return web.json_response(
{"success": True, "relative_paths": matching_paths}
@@ -1489,7 +1700,8 @@ class ModelDownloadHandler:
import json
try:
data["file_params"] = json.loads(file_params_json)
# Normalize falsy payloads (e.g. {}) to None (#1058)
data["file_params"] = json.loads(file_params_json) or None
except json.JSONDecodeError:
self._logger.warning(
"Invalid file_params JSON: %s", file_params_json
@@ -1641,7 +1853,8 @@ class ModelDownloadHandler:
model_id = int(model_id_str) if model_id_str else None
model_version_id = int(model_version_id_str) if model_version_id_str else None
file_params = json.loads(file_params_json) if file_params_json else None
# Normalize falsy payloads (e.g. {}) to None (#1058)
file_params = (json.loads(file_params_json) if file_params_json else None) or None
service = await DownloadQueueService.get_instance()
item = await service.add_to_queue(
@@ -1716,8 +1929,18 @@ class ModelDownloadHandler:
try:
status_filter = request.query.get("status") or None
service = await DownloadQueueService.get_instance()
cleared = await service.clear_queue(status_filter=status_filter)
return web.json_response({"success": True, "cleared": cleared})
cleared_ids = await service.clear_queue(status_filter=status_filter)
# Clearing the queue rows alone would orphan any in-memory tasks
# and persisted aria2 state for those downloads, leaving them
# polling the daemon invisibly. Tear that tracking down too.
try:
await self._download_coordinator.discard_cleared_downloads(cleared_ids)
except Exception:
self._logger.warning(
"Failed to discard in-memory state for cleared downloads",
exc_info=True,
)
return web.json_response({"success": True, "cleared": len(cleared_ids)})
except Exception as exc:
self._logger.error(
"Error clearing download queue: %s", exc, exc_info=True
@@ -1762,14 +1985,20 @@ class ModelDownloadHandler:
async def delete_download_history_item(self, request: web.Request) -> web.Response:
try:
item_id = int(request.query.get("id", "0"))
if not item_id:
download_id = request.query.get("download_id")
id_str = request.query.get("id")
item_id = int(id_str) if id_str else None
if not download_id and not item_id:
return web.json_response(
{"success": False, "error": "id is required"}, status=400
{"success": False, "error": "id or download_id is required"},
status=400,
)
service = await DownloadQueueService.get_instance()
deleted = await service.delete_history_item(item_id)
deleted = await service.delete_history_item(
id=item_id, download_id=download_id
)
return web.json_response({"success": deleted})
except Exception as exc:
self._logger.error(
@@ -1779,18 +2008,26 @@ class ModelDownloadHandler:
async def retry_download_from_history(self, request: web.Request) -> web.Response:
try:
item_id = int(request.query.get("id", "0"))
if not item_id:
download_id = request.query.get("download_id")
id_str = request.query.get("id")
item_id = int(id_str) if id_str else None
if not download_id and not item_id:
return web.json_response(
{"success": False, "error": "id is required"}, status=400
{"success": False, "error": "id or download_id is required"},
status=400,
)
service = await DownloadQueueService.get_instance()
item = await service.retry_from_history(item_id)
item = await service.retry_from_history(
item_id=item_id, download_id=download_id
)
if item is None:
# Missing or non-retryable history entry is a business
# outcome, not a routing error: 200 lets the extension's
# apiFetch 404-fallback and error middleware stay quiet.
return web.json_response(
{"success": False, "error": "History item not found or not retryable"},
status=404,
{"success": False, "error": "History item not found or not retryable"}
)
return web.json_response({"success": True, "item": item})
except Exception as exc:
@@ -1841,8 +2078,12 @@ class ModelDownloadHandler:
completed_at=completed_at,
)
if item is None:
# A missing queue item (already completed, or never queued) is
# a normal business outcome, not a routing error. Return 200
# so the browser extension's apiFetch 404-fallback and the
# error middleware stay quiet.
return web.json_response(
{"success": False, "error": "Download not found in queue"}, status=404
{"success": False, "error": "Download not found in queue"}
)
return web.json_response({"success": True, "item": item})
except Exception as exc:
@@ -1884,9 +2125,10 @@ class ModelDownloadHandler:
service = await DownloadQueueService.get_instance()
updated = await service.update_status(download_id, status)
if not updated:
# Same rationale as complete_download_in_queue: a missing
# queue item is a business outcome, not a routing error.
return web.json_response(
{"success": False, "error": "Download not found in queue"},
status=404,
{"success": False, "error": "Download not found in queue"}
)
return web.json_response({"success": True})
except Exception as exc:
@@ -1906,7 +2148,7 @@ class ModelCivitaiHandler:
settings_service: SettingsManager,
ws_manager: WebSocketManager,
logger: logging.Logger,
metadata_provider_factory: Callable[[], Awaitable],
metadata_provider_factory: Callable[[], Awaitable[Any]],
validate_model_type: Callable[[str], bool],
expected_model_types: Callable[[], str],
find_model_file: Callable[
@@ -1971,7 +2213,7 @@ class ModelCivitaiHandler:
downloaded_version_ids = set(
await history_service.get_downloaded_version_ids(
self._service.model_type,
model_id,
int(model_id),
)
)
except Exception as exc: # pragma: no cover - defensive logging
@@ -2005,6 +2247,19 @@ class ModelCivitaiHandler:
else:
version.pop("localPath", None)
# Per-file downloaded state so multi-file versions can show
# which individual files are already in the library (#1058)
local_entries: List[Any] = []
if version_id is not None and cache:
files_getter = getattr(cache, "get_files_by_version_id", None)
if files_getter is not None:
local_entries = files_getter(version_id)
elif cache_entry is not None:
local_entries = [cache_entry]
version["downloadedFiles"] = self._match_downloaded_files(
version, local_entries
)
model_file = (
self._find_model_file(version.get("files", []))
if isinstance(version.get("files"), Iterable)
@@ -2019,6 +2274,64 @@ class ModelCivitaiHandler:
)
return web.Response(status=500, text=str(exc))
@staticmethod
def _match_downloaded_files(
version: Mapping[str, Any], local_entries: List[Any]
) -> List[Dict[str, Any]]:
"""Map local library entries back to individual files of a version.
Matching follows rule D2 (#1058): SHA256 is authoritative when the
local entry carries one; otherwise fall back to extension-less file
name equality. Returns ``[{fileId, fileName, filePath}]``.
"""
files = version.get("files")
if not isinstance(files, list) or not local_entries:
return []
by_hash: Dict[str, Mapping[str, Any]] = {}
by_name: Dict[str, Mapping[str, Any]] = {}
for file_info in files:
if not isinstance(file_info, Mapping):
continue
sha = str(
(file_info.get("hashes") or {}).get("SHA256") or ""
).strip().lower()
if sha:
by_hash.setdefault(sha, file_info)
name = str(file_info.get("name") or "").strip()
if name:
by_name.setdefault(os.path.splitext(name)[0], file_info)
downloaded: List[Dict[str, Any]] = []
seen_keys: set = set()
for entry in local_entries:
if not isinstance(entry, Mapping):
continue
matched: Optional[Mapping[str, Any]] = None
local_hash = str(entry.get("sha256") or "").strip().lower()
if local_hash:
matched = by_hash.get(local_hash)
if matched is None:
local_name = str(entry.get("file_name") or "").strip()
if local_name:
matched = by_name.get(local_name)
if matched is None:
continue
file_id = matched.get("id")
dedupe_key = file_id if file_id is not None else matched.get("name")
if dedupe_key in seen_keys:
continue
seen_keys.add(dedupe_key)
downloaded.append(
{
"fileId": file_id,
"fileName": matched.get("name"),
"filePath": entry.get("file_path"),
}
)
return downloaded
async def get_civitai_model_by_version(self, request: web.Request) -> web.Response:
try:
model_version_id = request.match_info.get("modelVersionId")
@@ -2081,6 +2394,8 @@ class ModelMoveHandler:
result = await self._move_service.move_model(
file_path, target_path, use_default_paths=use_default_paths
)
if result.get("success"):
_broadcast_models_changed()
status = 200 if result.get("success") else 500
return web.json_response(result, status=status)
except Exception as exc:
@@ -2100,6 +2415,8 @@ class ModelMoveHandler:
result = await self._move_service.move_models_bulk(
file_paths, target_path, use_default_paths=use_default_paths
)
if result.get("success"):
_broadcast_models_changed()
return web.json_response(result)
except Exception as exc:
self._logger.error("Error moving models in bulk: %s", exc, exc_info=True)
@@ -2145,6 +2462,7 @@ class ModelAutoOrganizeHandler:
progress_callback=self._progress_callback,
exclusion_patterns=exclusion_patterns,
)
_broadcast_models_changed()
return web.json_response(result.to_dict())
except AutoOrganizeInProgressError:
return web.json_response(
@@ -2248,8 +2566,8 @@ class ModelUpdateHandler:
self._logger.error("Failed to fetch license info: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
updated: List[Dict[str, str]] = []
errors: List[Dict[str, str]] = []
updated: List[Dict[str, Any]] = []
errors: List[Dict[str, Any]] = []
for model_id in model_ids:
license_payload = license_map.get(model_id)
if not license_payload:
@@ -2262,6 +2580,7 @@ class ModelUpdateHandler:
model_section = civitai_section.get("model")
if not isinstance(model_section, Mapping):
model_section = {}
model_section = dict(model_section)
model_section.update(resolved_payload)
civitai_section["model"] = model_section
metadata_payload["civitai"] = civitai_section
@@ -2277,7 +2596,7 @@ class ModelUpdateHandler:
)
errors.append({"filePath": metadata_path, "error": str(exc)})
response_payload = {"success": True, "updated": updated}
response_payload: Dict[str, Any] = {"success": True, "updated": updated}
missing_model_ids = [mid for mid in model_ids if mid not in license_map]
if missing_model_ids:
response_payload["missingModelIds"] = missing_model_ids
@@ -2347,6 +2666,7 @@ class ModelUpdateHandler:
return web.json_response({"success": False, "error": str(exc)}, status=500)
hide_early_access = False
hide_paid = False
if self._settings is not None:
try:
hide_early_access = bool(
@@ -2354,12 +2674,27 @@ class ModelUpdateHandler:
)
except Exception:
pass
try:
hide_paid = bool(self._settings.get("hide_paid_updates", False))
except Exception:
pass
same_base_scope = self._uses_same_base_update_scope()
serialized_records = []
for record in records.values():
has_update_fn = getattr(record, "has_update", None)
if callable(has_update_fn) and has_update_fn(
hide_early_access=hide_early_access
if not callable(has_update_fn):
continue
scoped_fn = (
getattr(record, "has_update_for_local_bases", None)
if same_base_scope
else None
)
qualifies_fn = scoped_fn if callable(scoped_fn) else has_update_fn
if qualifies_fn(
hide_early_access=hide_early_access,
hide_paid=hide_paid,
):
serialized_records.append(self._serialize_record(record))
@@ -2370,6 +2705,26 @@ class ModelUpdateHandler:
}
)
def _uses_same_base_update_scope(self) -> bool:
"""Return True when update reporting must honor same-base scoping.
Mirrors ``BaseModelService._annotate_update_flags``: the Updates filter
evaluates updates per local base model when ``version_grouping`` is
``same_base`` (its default). The refresh summary counts with the same
scope so the "Found N update(s)" toast matches what the filter
displays. See issue #1083.
"""
if self._settings is None:
return True
try:
strategy_value = self._settings.get("version_grouping")
except Exception:
return True
if isinstance(strategy_value, str) and strategy_value.strip():
return strategy_value.strip().lower() == "same_base"
return True
async def set_model_update_ignore(self, request: web.Request) -> web.Response:
payload = await self._read_json(request)
model_id = self._normalize_model_id(payload.get("modelId"))
@@ -2513,10 +2868,16 @@ class ModelUpdateHandler:
if not record or not record.versions:
return record
# Find versions that need enrichment
# Find versions that need enrichment. Permanent paid versions are not
# early access (mirror _is_early_access_active) and never carry an end
# time, so skip them to avoid pointless per-version API calls.
versions_needing_update = []
for version in record.versions:
if version.is_early_access and not version.early_access_ends_at:
if (
version.is_early_access
and not version.early_access_ends_at
and not getattr(version, "is_paid", False)
):
versions_needing_update.append(version)
if not versions_needing_update:
@@ -2626,6 +2987,7 @@ class ModelUpdateHandler:
civitai_payload = metadata_payload.get("civitai")
if not isinstance(civitai_payload, Mapping):
civitai_payload = {}
civitai_payload = dict(civitai_payload)
model_payload = civitai_payload.get("model")
if not isinstance(model_payload, Mapping):
@@ -2670,7 +3032,7 @@ class ModelUpdateHandler:
return aggregated
def _extract_target_model_ids(self, payload: Dict) -> Optional[List[int]]:
def _extract_target_model_ids(self, payload: Dict[str, Any]) -> Optional[List[int]]:
if not isinstance(payload, Mapping):
return None
@@ -2698,7 +3060,7 @@ class ModelUpdateHandler:
return {}
to_dict = getattr(metadata, "to_dict", None)
if callable(to_dict):
if to_dict:
try:
return to_dict()
except Exception:
@@ -2709,7 +3071,7 @@ class ModelUpdateHandler:
return {}
async def _read_json(self, request: web.Request) -> Dict:
async def _read_json(self, request: web.Request) -> Dict[str, Any]:
if not request.can_read_body:
return {}
try:
@@ -2741,10 +3103,11 @@ class ModelUpdateHandler:
record,
*,
version_context: Optional[Dict[int, Dict[str, Any]]] = None,
) -> Dict:
) -> Dict[str, Any]:
context = version_context or {}
# Check user setting for hiding early access versions
hide_early_access = False
hide_paid = False
if self._settings is not None:
try:
hide_early_access = bool(
@@ -2752,6 +3115,10 @@ class ModelUpdateHandler:
)
except Exception:
pass
try:
hide_paid = bool(self._settings.get("hide_paid_updates", False))
except Exception:
pass
return {
"modelType": record.model_type,
"modelId": record.model_id,
@@ -2760,7 +3127,10 @@ class ModelUpdateHandler:
"inLibraryVersionIds": record.in_library_version_ids,
"lastCheckedAt": record.last_checked_at,
"shouldIgnore": record.should_ignore_model,
"hasUpdate": record.has_update(hide_early_access=hide_early_access),
"hasUpdate": record.has_update(
hide_early_access=hide_early_access,
hide_paid=hide_paid,
),
"versions": [
self._serialize_version(version, context.get(version.version_id))
for version in record.versions
@@ -2770,7 +3140,7 @@ class ModelUpdateHandler:
@staticmethod
def _serialize_version(
version, context: Optional[Dict[str, Any]]
) -> Dict:
) -> Dict[str, Any]:
context = context or {}
preview_override = context.get("preview_override")
preview_url = (
@@ -2779,8 +3149,11 @@ class ModelUpdateHandler:
# Determine if version is currently in early access
# Two-phase detection: use exact end time if available, otherwise fallback to basic flag
# Mirror _is_early_access_active: permanent paid versions (no end time) are NOT early access
is_early_access = False
if version.early_access_ends_at:
if getattr(version, "is_paid", False) and not version.early_access_ends_at:
is_early_access = False
elif version.early_access_ends_at:
try:
from datetime import datetime, timezone
@@ -2795,6 +3168,13 @@ class ModelUpdateHandler:
# Fallback to basic EA flag from bulk API
is_early_access = True
paid_access_payload = None
if getattr(version, "paid_access", None):
try:
paid_access_payload = json.loads(version.paid_access)
except (TypeError, ValueError):
paid_access_payload = None
return {
"versionId": version.version_id,
"name": version.name,
@@ -2808,8 +3188,13 @@ class ModelUpdateHandler:
"earlyAccessEndsAt": version.early_access_ends_at,
"isEarlyAccess": is_early_access,
"usageControl": version.usage_control,
"isPaid": bool(getattr(version, "is_paid", False)),
"paidAccess": paid_access_payload,
"filePath": context.get("file_path"),
"fileName": context.get("file_name"),
# Weight-file variant count (None when unknown); lets the UI hide
# the download affordance for single-file in-library versions.
"fileCount": getattr(version, "file_count", None),
}
async def _build_version_context(
@@ -2910,6 +3295,7 @@ class ModelHandlerSet:
"bulk_delete_models": self.management.bulk_delete_models,
"verify_duplicates": self.management.verify_duplicates,
"get_top_tags": self.query.get_top_tags,
"search_tags": self.query.search_tags,
"get_base_models": self.query.get_base_models,
"get_model_types": self.query.get_model_types,
"scan_models": self.query.scan_models,
@@ -0,0 +1,323 @@
"""Handler for the pending-delete undo endpoint.
Restores a staged delete batch (models or recipes) via
``PendingDeleteService.undo`` and then repairs the affected library caches:
the model cache entry is restored from the manifest's ``model_snapshot``
(including the version index and hash index), tag counts are re-incremented,
and the recipe cache is re-populated via ``RecipeScanner.add_recipe``.
The per-type scanner is resolved from the manifest's ``model_type`` page value
through the SAME ServiceRegistry getters the model route registrars use
(lora/checkpoint/embedding) - never a hardcoded lora scanner.
"""
from __future__ import annotations
import inspect
import json
import logging
import os
import re
from typing import Any, Awaitable, Callable, Dict, List, Optional, Set, cast
from aiohttp import web
from ...services.pending_delete_service import get_pending_delete_service
from .model_handlers import _broadcast_models_changed
logger = logging.getLogger(__name__)
# Manifest ``model_type`` page values -> ServiceRegistry scanner getter names.
# The model route registrars resolve per-type scanners via these getters
# (lora_routes / checkpoint_routes / embedding_routes); undo must do the same
# so the CORRECT cache is restored for the deleted model's type.
_MODEL_TYPE_GETTER_NAMES: Dict[str, str] = {
"loras": "get_lora_scanner",
"checkpoints": "get_checkpoint_scanner",
"embeddings": "get_embedding_scanner",
}
# Staged batch ids are ``uuid.uuid4().hex`` (32 lowercase hex chars). The id is
# joined into filesystem paths by ``_find_batch_dir``, so reject anything that
# does not match this exact shape (blocks path-traversal via batch_id).
_BATCH_ID_RE = re.compile(r"^[0-9a-f]{32}$")
class PendingDeleteHandler:
"""Handle undo requests for staged model/recipe deletions."""
def __init__(
self,
*,
service_factory: Callable[[], Awaitable[Any]] = get_pending_delete_service,
scanner_getter: Optional[Callable[[str], Awaitable[Any]]] = None,
recipe_scanner_getter: Optional[Callable[[], Awaitable[Any]]] = None,
) -> None:
self._service_factory: Callable[[], Awaitable[Any]] = service_factory
self._scanner_getter: Callable[[str], Awaitable[Any]] = (
scanner_getter or self._resolve_scanner
)
self._recipe_scanner_getter: Callable[[], Awaitable[Any]] = (
recipe_scanner_getter or self._resolve_recipe_scanner
)
@staticmethod
async def _resolve_scanner(model_type: str) -> Any:
"""Resolve the per-type scanner for a manifest ``model_type``.
The getter is looked up on the ServiceRegistry module namespace at call
time so tests (and the registry stubs) can patch it.
"""
from ...services import service_registry
getter_name = _MODEL_TYPE_GETTER_NAMES.get(model_type)
if getter_name is None:
raise ValueError(f"Unknown model type: {model_type}")
getter = getattr(service_registry.ServiceRegistry, getter_name, None)
if not callable(getter):
raise ValueError(f"No scanner getter for model type: {model_type}")
scanner = await cast(Callable[[], Awaitable[Any]], getter)()
if scanner is None:
raise ValueError(f"No scanner registered for model type: {model_type}")
return scanner
@staticmethod
async def _resolve_recipe_scanner() -> Any:
"""Resolve the recipe scanner via the ServiceRegistry module namespace."""
from ...services import service_registry
getter = getattr(service_registry.ServiceRegistry, "get_recipe_scanner", None)
if not callable(getter):
raise ValueError("Recipe scanner getter unavailable")
scanner = await cast(Callable[[], Awaitable[Any]], getter)()
if scanner is None:
raise ValueError("No recipe scanner registered")
return scanner
async def undo_delete(self, request: web.Request) -> web.Response:
"""Restore a staged batch and its library cache entry.
Body: ``{"batch_id": str}``. On success returns
``{"success": True, "restored": [<original paths>], "kind": kind}``.
Expired/unknown batches and occupied target paths -> 404.
"""
try:
data = await request.json()
except Exception:
return web.json_response(
{"success": False, "error": "Invalid JSON body"}, status=400
)
if not isinstance(data, dict):
return web.json_response(
{"success": False, "error": "Invalid JSON body"}, status=400
)
batch_id = data.get("batch_id")
if not batch_id or not isinstance(batch_id, str):
return web.json_response(
{"success": False, "error": "batch_id is required"}, status=400
)
if not _BATCH_ID_RE.fullmatch(batch_id):
# batch_id is joined into a path by _find_batch_dir - restrict to
# the exact staged-id shape so traversal payloads get 400.
return web.json_response(
{"success": False, "error": "Invalid batch_id"}, status=400
)
service = await self._service_factory()
try:
# Read the manifest BEFORE undo: undo() removes the batch dir.
manifest = await self._read_staged_manifest(service, batch_id)
result = await service.undo(batch_id)
except ValueError as exc:
return web.json_response({"success": False, "error": str(exc)}, status=404)
except Exception as exc:
logger.error("Unexpected error undoing batch %s: %s", batch_id, exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
kind = result.get("kind")
try:
if kind == "model":
if manifest is not None:
await self._restore_model_cache(manifest)
else:
# undo() raises when the manifest is missing, so this only
# happens defensively - files are restored regardless.
logger.warning(
"Manifest missing after undo of %s; skipping cache restore",
batch_id,
)
_broadcast_models_changed()
elif kind == "recipe":
# Recipe undo is client-refresh only: re-add to the scanner
# cache, no models_changed broadcast.
if manifest is not None:
await self._restore_recipe_cache(result, manifest)
else:
logger.warning(
"Manifest missing after undo of %s; skipping cache restore",
batch_id,
)
except Exception as exc:
# Files are already restored; only the cache restoration failed.
logger.error(
"Cache restoration failed after undo of %s: %s",
batch_id,
exc,
exc_info=True,
)
return web.json_response({"success": False, "error": str(exc)}, status=500)
return web.json_response(
{
"success": True,
"restored": result.get("restored", []),
"kind": kind,
}
)
@staticmethod
async def _read_staged_manifest(
service: Any, batch_id: str
) -> Optional[Dict[str, Any]]:
"""Locate and read the batch manifest while it still exists on disk."""
batch_dir = await service._find_batch_dir(batch_id)
if not batch_dir:
return None
manifest_path = os.path.join(batch_dir, "manifest.json")
try:
with open(manifest_path, "r", encoding="utf-8") as handle:
payload = json.load(handle)
except (OSError, json.JSONDecodeError) as exc:
logger.debug("Failed to read manifest for batch %s: %s", batch_id, exc)
return None
return payload if isinstance(payload, dict) else None
async def _restore_model_cache(self, manifest: Dict[str, Any]) -> None:
"""Re-add every deleted model's cache entry from the manifest.
Each main-file entry carries the deleted model's ``snapshot`` (added at
stage time), so a merged bulk manifest holds ALL snapshots - undo must
restore every one, not just the top-level winner's. Old-format
manifests without entry snapshots fall back to the top-level
``model_snapshot`` (backward compat / single-delete path).
"""
model_type = manifest.get("model_type")
if not model_type or not isinstance(model_type, str):
raise ValueError(f"Manifest carries no model_type: {manifest.get('batch_id')}")
scanner = await self._scanner_getter(model_type)
# Collect one snapshot per distinct file_path from the entry snapshots.
snapshots: List[Dict[str, Any]] = []
seen: Set[str] = set()
for entry in manifest.get("entries") or []:
snapshot = entry.get("snapshot")
if not isinstance(snapshot, dict):
continue
file_path = snapshot.get("file_path")
if not file_path or not isinstance(file_path, str):
continue
if file_path in seen:
continue
seen.add(file_path)
snapshots.append(snapshot)
if not snapshots:
# Backward compat: pre-F3 manifests carry only the top-level
# model_snapshot (single-delete path, unchanged behavior).
top = manifest.get("model_snapshot")
if isinstance(top, dict) and top.get("file_path"):
snapshots = [top]
else:
logger.warning(
"Manifest %s has no restorable model snapshot; skipping cache restore",
manifest.get("batch_id"),
)
return
cache = await scanner.get_cached_data()
if cache is None:
logger.warning(
"Scanner cache unavailable for %s; skipping cache restore", model_type
)
return
for snapshot in snapshots:
file_path = str(snapshot["file_path"])
# A rescan between delete and undo may have re-added a stale entry
# for this path - drop it so exactly one (the snapshot) remains.
cache.raw_data = [
item for item in cache.raw_data if item.get("file_path") != file_path
]
# Restore tag counts (mirror of the bulk-delete decrement in
# _batch_update_cache_for_deleted_models: undo re-increments).
tags = snapshot.get("tags")
if isinstance(tags, list):
for tag in tags:
if not isinstance(tag, str) or not tag:
continue
scanner._tags_count[tag] = scanner._tags_count.get(tag, 0) + 1
cache.raw_data.append(dict(snapshot))
# Re-register the path in the hash index (add_entry guards a
# missing sha256 internally; still guard defensively here).
sha256 = snapshot.get("sha256") or ""
autov3 = snapshot.get("autov3")
hash_index = getattr(scanner, "_hash_index", None)
if hash_index is not None and sha256 and file_path:
hash_index.add_entry(sha256, file_path, autov3)
# Follow the bulk-delete cache-update pattern ONCE after all entries,
# including the explicit version-index rebuild so the version index
# does not go stale.
cache.rebuild_version_index()
await cache.resort()
scanner.bump_cache_version()
persist = getattr(scanner, "_persist_current_cache", None)
if callable(persist):
result = persist()
if inspect.isawaitable(result):
await result
async def _restore_recipe_cache(
self, result: Dict[str, Any], manifest: Dict[str, Any]
) -> None:
"""Re-add a restored recipe via ``RecipeScanner.add_recipe``.
The recipe JSON embeds the full recipe_data (incl. id/file_path);
``add_recipe`` only READS the ``_json_path_map`` so the forced frontend
refresh self-heals any transient path-map gap.
"""
restored = result.get("restored") or []
json_path = next(
(p for p in restored if isinstance(p, str) and p.endswith(".json")),
None,
)
if not json_path or not os.path.exists(json_path):
# Defensive fallback to the manifest's recipe_snapshot file_path.
snapshot = manifest.get("recipe_snapshot") or {}
fallback = snapshot.get("file_path")
if fallback and os.path.exists(fallback):
json_path = fallback
else:
logger.warning(
"Restored recipe JSON not found in %s; skipping cache restore",
restored,
)
return
try:
with open(json_path, "r", encoding="utf-8") as handle:
recipe_data = json.load(handle)
except (OSError, json.JSONDecodeError) as exc:
logger.warning("Failed to load restored recipe JSON %s: %s", json_path, exc)
return
if not isinstance(recipe_data, dict):
return
recipe_scanner = await self._recipe_scanner_getter()
await recipe_scanner.add_recipe(recipe_data)
__all__ = ["PendingDeleteHandler"]
+31
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import asyncio
import logging
import mimetypes
import urllib.parse
@@ -53,6 +54,7 @@ class PreviewHandler:
if not resolved.is_file():
logger.debug("Preview file not found at %s", str(resolved))
asyncio.create_task(self._cleanup_stale_preview_url(normalized))
raise web.HTTPNotFound(text="Preview file not found")
# aiohttp's FileResponse handles range requests, content headers, and
@@ -69,6 +71,35 @@ class PreviewHandler:
resp.headers["Cache-Control"] = "public, max-age=86400"
return resp
async def _cleanup_stale_preview_url(self, normalized_preview_path: str) -> None:
"""Fire-and-forget: clear stale preview_url from all model caches.
When a preview file is no longer on disk, remove its reference from
every cached entry so subsequent list API responses return an empty
``preview_url``, letting the frontend show the no-preview placeholder.
"""
try:
from ...services.service_registry import ServiceRegistry
for service_name in ("lora_scanner", "checkpoint_scanner", "embedding_scanner"):
scanner = ServiceRegistry.get_service_sync(service_name)
if scanner is None or not hasattr(scanner, "_cache"):
continue
cache = getattr(scanner, "_cache", None)
if cache is None or not hasattr(cache, "clear_preview_by_path"):
continue
cleared = await cache.clear_preview_by_path(normalized_preview_path)
if cleared and hasattr(scanner, "_persist_current_cache"):
await scanner._persist_current_cache()
logger.info(
"Cleared stale preview_url for %d %s entries (%s)",
cleared,
service_name,
normalized_preview_path,
)
except Exception as exc:
logger.debug("Failed to clean up stale preview_url: %s", exc)
async def _stream_file(
self, request: web.Request, path: Path
) -> web.StreamResponse:
+571 -74
View File
@@ -10,7 +10,7 @@ import asyncio
import tempfile
from dataclasses import dataclass
from pathlib import Path
from typing import Any, Awaitable, Callable, Dict, List, Mapping, Optional
from typing import Any, Awaitable, Callable, Dict, List, Mapping, Optional, Protocol, Tuple
from aiohttp import web
@@ -34,6 +34,7 @@ from ...utils.civitai_utils import (
)
from ...utils.constants import NSFW_LEVELS
from ...utils.exif_utils import ExifUtils
from ...utils.recipe_open_stats import RecipeOpenStats
from ...recipes.merger import GenParamsMerger
from ...recipes.enrichment import RecipeEnricher
from ...services.websocket_manager import ws_manager as default_ws_manager
@@ -45,6 +46,33 @@ RecipeScannerGetter = Callable[[], Any]
CivitaiClientGetter = Callable[[], Any]
class PromptServerProtocol(Protocol):
"""Subset of PromptServer used by the recipe workflow handler."""
instance: "PromptServerProtocol"
def send_sync(
self, event: str, payload: dict[str, Any] | None = None, sid: str | None = None
) -> None: # pragma: no cover - protocol
...
# Cap concurrent preview-dimension reads across requests. With a cold LRU
# cache one page can touch up to page_size image files; 16 balances SSD and
# HDD throughput without starving the event loop.
_DIMS_READ_SEMAPHORE = asyncio.Semaphore(16)
async def _read_preview_dims(path: str) -> Optional[Tuple[int, int]]:
"""Read preview dimensions off the event loop under the concurrency cap.
PIL I/O runs in a worker thread so it never blocks the event loop, and the
semaphore bounds how many files are opened at once even when many list
requests land together.
"""
async with _DIMS_READ_SEMAPHORE:
return await asyncio.to_thread(ExifUtils.get_image_dimensions, path)
@dataclass(frozen=True)
class RecipeHandlerSet:
"""Group of handlers providing recipe route implementations."""
@@ -56,6 +84,7 @@ class RecipeHandlerSet:
analysis: "RecipeAnalysisHandler"
sharing: "RecipeSharingHandler"
batch_import: "BatchImportHandler"
workflow: "RecipeWorkflowHandler"
def to_route_mapping(
self,
@@ -72,6 +101,7 @@ class RecipeHandlerSet:
"save_recipe": self.management.save_recipe,
"delete_recipe": self.management.delete_recipe,
"get_top_tags": self.query.get_top_tags,
"search_tags": self.query.search_tags,
"get_base_models": self.query.get_base_models,
"get_roots": self.query.get_roots,
"get_folders": self.query.get_folders,
@@ -81,7 +111,9 @@ class RecipeHandlerSet:
"download_shared_recipe": self.sharing.download_shared_recipe,
"get_recipe_syntax": self.query.get_recipe_syntax,
"update_recipe": self.management.update_recipe,
"record_recipe_open": self.management.record_recipe_open,
"reconnect_lora": self.management.reconnect_lora,
"mark_lora_hash_invalid": self.management.mark_lora_hash_invalid,
"find_duplicates": self.query.find_duplicates,
"move_recipes_bulk": self.management.move_recipes_bulk,
"bulk_delete": self.management.bulk_delete,
@@ -95,6 +127,11 @@ class RecipeHandlerSet:
"repair_recipe": self.management.repair_recipe,
"repair_recipes_bulk": self.management.repair_recipes_bulk,
"get_repair_progress": self.management.get_repair_progress,
"rematch_recipes": self.management.rematch_recipes,
"cancel_rematch": self.management.cancel_rematch,
"rematch_recipe": self.management.rematch_recipe,
"rematch_recipes_bulk": self.management.rematch_recipes_bulk,
"get_rematch_progress": self.management.get_rematch_progress,
"start_batch_import": self.batch_import.start_batch_import,
"get_batch_import_progress": self.batch_import.get_batch_import_progress,
"cancel_batch_import": self.batch_import.cancel_batch_import,
@@ -104,6 +141,7 @@ class RecipeHandlerSet:
"import_from_url": self.management.import_from_url,
"create_from_example": self.management.create_from_example,
"reimport_recipe": self.management.reimport_recipe,
"send_recipe_workflow": self.workflow.send_recipe_workflow,
}
@@ -139,11 +177,19 @@ class RecipePageView:
user_language = self._settings.get("language", "en")
self._server_i18n.set_locale(user_language)
# While the initial scan is running, show the initialization
# screen (same as the model pages) instead of an empty grid; the
# page reloads itself when the scanner broadcasts completion.
is_initializing = (
recipe_scanner._cache is None or recipe_scanner.is_initializing()
)
try:
if not is_initializing:
await recipe_scanner.get_cached_data(force_refresh=False)
rendered = self._template_env.get_template(self._template_name).render(
recipes=[],
is_initializing=False,
is_initializing=is_initializing,
settings=self._settings,
request=request,
t=self._server_i18n.get_translation,
@@ -229,6 +275,14 @@ class RecipeListingHandler:
if tag_filters:
filters["tags"] = tag_filters
lora_availability = {
status.strip()
for status in request.query.get("lora_availability", "").split(",")
if status.strip() in ("ready", "missing", "deleted")
}
if lora_availability:
filters["lora_availability"] = lora_availability
lora_hash = request.query.get("lora_hash")
checkpoint_hash = request.query.get("checkpoint_hash")
@@ -245,7 +299,8 @@ class RecipeListingHandler:
recursive=recursive,
)
for item in result.get("items", []):
items = result.get("items", [])
for item in items:
file_path = item.get("file_path")
if file_path:
item["file_url"] = self.format_recipe_file_url(file_path)
@@ -254,6 +309,26 @@ class RecipeListingHandler:
item.setdefault("loras", [])
item.setdefault("base_model", "")
# Batch preview dimension reads with asyncio.gather. The previous
# loop awaited asyncio.to_thread once per item, so a page_size=100
# request submitted 100 sequential thread calls (50-300ms cold-page
# latency). gather runs them concurrently while the semaphore caps
# disk opens; dimensions stay omitted (not null) when a preview has
# no readable size (video, missing file).
to_read = [
(i, item.get("file_path"))
for i, item in enumerate(items)
if item.get("file_path")
]
if to_read:
dims_list = await asyncio.gather(
*(_read_preview_dims(path) for _, path in to_read)
)
for (idx, _), dims in zip(to_read, dims_list):
if dims:
item = items[idx]
item["width"], item["height"] = dims
return web.json_response(result)
except Exception as exc:
self._logger.error("Error retrieving recipes: %s", exc, exc_info=True)
@@ -317,12 +392,11 @@ class RecipeQueryHandler:
raise RuntimeError("Recipe scanner unavailable")
limit = int(request.query.get("limit", "20"))
cache = await recipe_scanner.get_cached_data()
tag_counts: Dict[str, int] = {}
for recipe in getattr(cache, "raw_data", []):
for tag in recipe.get("tags", []) or []:
tag_counts[tag] = tag_counts.get(tag, 0) + 1
if limit < 0:
limit = 20
elif limit > 200:
limit = 20
tag_counts = await self._get_recipe_tag_counts(recipe_scanner)
sorted_tags = [
{"tag": tag, "count": count} for tag, count in tag_counts.items()
@@ -333,6 +407,55 @@ class RecipeQueryHandler:
self._logger.error("Error retrieving top tags: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def search_tags(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
raise RuntimeError("Recipe scanner unavailable")
query = request.query.get("q", "")
limit = int(request.query.get("limit", "20"))
if limit < 0:
limit = 20
elif limit > 200:
limit = 20
tag_counts = await self._get_recipe_tag_counts(recipe_scanner)
normalized_query = (query or "").strip().lower()
if not normalized_query:
sorted_tags = [
{"tag": tag, "count": count} for tag, count in tag_counts.items()
]
sorted_tags.sort(key=lambda entry: entry["count"], reverse=True)
return web.json_response(
{"success": True, "tags": sorted_tags[: (limit if limit > 0 else 20)]}
)
matched = [
{"tag": tag, "count": count}
for tag, count in tag_counts.items()
if normalized_query in tag.lower()
]
matched.sort(key=lambda entry: entry["count"], reverse=True)
if limit == 0:
result = matched
else:
result = matched[:limit]
return web.json_response({"success": True, "tags": result})
except Exception as exc:
self._logger.error("Error searching recipe tags: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def _get_recipe_tag_counts(self, recipe_scanner) -> Dict[str, int]:
"""Compute tag->count mapping from cached recipe data."""
cache = await recipe_scanner.get_cached_data()
tag_counts: Dict[str, int] = {}
for recipe in getattr(cache, "raw_data", []):
for tag in recipe.get("tags", []) or []:
tag_counts[tag] = tag_counts.get(tag, 0) + 1
return tag_counts
async def get_base_models(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
@@ -489,18 +612,38 @@ class RecipeQueryHandler:
if recipe_scanner is None:
raise RuntimeError("Recipe scanner unavailable")
fingerprint_groups = await recipe_scanner.find_all_duplicate_recipes()
include_prompt = (
request.query.get("include_prompt", "false").lower() in ("1", "true")
)
fingerprint_groups = await recipe_scanner.find_all_duplicate_recipes(
include_prompt=include_prompt
)
url_groups = await recipe_scanner.find_duplicate_recipes_by_source()
# Assemble the response directly from the cached recipe summaries.
# Resolving each id via get_recipe_by_id would re-read every recipe
# JSON from disk — thousands of blocking reads on the event loop
# for large libraries — while all required fields already live in
# the cache.
cache = await recipe_scanner.get_cached_data()
recipes_by_id = {
str(recipe.get("id", "")): recipe for recipe in cache.raw_data
}
response_data = []
for fingerprint, recipe_ids in fingerprint_groups.items():
def append_groups(
groups: Dict[str, List[Any]], group_type: str
) -> None:
for group_key, recipe_ids in groups.items():
if len(recipe_ids) <= 1:
continue
recipes = []
for recipe_id in recipe_ids:
recipe = await recipe_scanner.get_recipe_by_id(recipe_id)
if recipe:
recipe = recipes_by_id.get(str(recipe_id))
if recipe is None:
continue
recipes.append(
{
"id": recipe.get("id"),
@@ -517,51 +660,21 @@ class RecipeQueryHandler:
if len(recipes) >= 2:
recipes.sort(
key=lambda entry: entry.get("modified", 0), reverse=True
key=lambda entry: entry.get("modified") or 0,
reverse=True,
)
response_data.append(
{
"type": "fingerprint",
"fingerprint": fingerprint,
"type": group_type,
"key": f"g-{len(response_data) + 1}",
"fingerprint": group_key,
"count": len(recipes),
"recipes": recipes,
}
)
for url, recipe_ids in url_groups.items():
if len(recipe_ids) <= 1:
continue
recipes = []
for recipe_id in recipe_ids:
recipe = await recipe_scanner.get_recipe_by_id(recipe_id)
if recipe:
recipes.append(
{
"id": recipe.get("id"),
"title": recipe.get("title"),
"file_url": recipe.get("file_url")
or self._format_recipe_file_url(
recipe.get("file_path", "")
),
"modified": recipe.get("modified"),
"created_date": recipe.get("created_date"),
"lora_count": len(recipe.get("loras", [])),
}
)
if len(recipes) >= 2:
recipes.sort(
key=lambda entry: entry.get("modified", 0), reverse=True
)
response_data.append(
{
"type": "source_path",
"fingerprint": url,
"count": len(recipes),
"recipes": recipes,
}
)
append_groups(fingerprint_groups, "fingerprint")
append_groups(url_groups, "source_path")
response_data.sort(key=lambda entry: entry["count"], reverse=True)
return web.json_response(
@@ -801,6 +914,159 @@ class RecipeManagementHandler:
self._logger.error("Error repairing single recipe: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def rematch_recipes(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
return web.json_response(
{"success": False, "error": "Recipe scanner unavailable"},
status=503,
)
# Mutual exclusion: a global rematch cannot start while a rematch
# OR a repair is already running — both mutate recipes under the
# same mutation lock.
if (
self._ws_manager.is_recipe_rematch_running()
or self._ws_manager.is_recipe_repair_running()
):
return web.json_response(
{"success": False, "error": "Recipe rematch already in progress"},
status=409,
)
recipe_scanner.reset_cancellation()
async def progress_callback(data):
await self._ws_manager.broadcast_recipe_rematch_progress(data)
# Run in background to avoid timeout
async def run_rematch():
try:
await recipe_scanner.rematch_all_recipes(
progress_callback=progress_callback
)
except Exception as e:
self._logger.error(
f"Error in recipe rematch task: {e}", exc_info=True
)
await self._ws_manager.broadcast_recipe_rematch_progress(
{"status": "error", "error": str(e)}
)
finally:
# Keep the final status for a while so the UI can see it
await asyncio.sleep(5)
self._ws_manager.cleanup_recipe_rematch_progress()
asyncio.create_task(run_rematch())
return web.json_response(
{"success": True, "message": "Recipe rematch started"}
)
except Exception as exc:
self._logger.error("Error starting recipe rematch: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def cancel_rematch(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
return web.json_response(
{"success": False, "error": "Recipe scanner unavailable"},
status=503,
)
recipe_scanner.cancel_task()
return web.json_response(
{"success": True, "message": "Cancellation requested"}
)
except Exception as exc:
self._logger.error("Error cancelling recipe rematch: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def rematch_recipes_bulk(self, request: web.Request) -> web.Response:
"""Rematch deleted resources for multiple recipes by their IDs.
Accepts a JSON body with a "recipe_ids" array. The per-recipe loop is
delegated to the scanner's rematch_recipes_bulk; this handler only
parses the request and returns the scanner's summary.
"""
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
return web.json_response(
{"success": False, "error": "Recipe scanner unavailable"},
status=503,
)
# A bulk rematch must not queue behind a running global rematch's
# mutation lock.
if self._ws_manager.is_recipe_rematch_running():
return web.json_response(
{"success": False, "error": "Recipe rematch already in progress"},
status=409,
)
data = await request.json()
recipe_ids = data.get("recipe_ids", [])
if not recipe_ids:
return web.json_response(
{"success": False, "error": "recipe_ids are required"},
status=400,
)
result = await recipe_scanner.rematch_recipes_bulk(recipe_ids)
return web.json_response(result)
except Exception as exc:
self._logger.error(
"Error performing bulk rematch: %s", exc, exc_info=True
)
return web.json_response(
{"success": False, "error": str(exc)}, status=500
)
async def rematch_recipe(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
return web.json_response(
{"success": False, "error": "Recipe scanner unavailable"},
status=503,
)
# Reject per-recipe rematches while a global run is in progress so
# they do not queue behind the mutation lock.
if self._ws_manager.is_recipe_rematch_running():
return web.json_response(
{"success": False, "error": "Recipe rematch already in progress"},
status=409,
)
recipe_id = request.match_info["recipe_id"]
result = await recipe_scanner.rematch_recipe_by_id(recipe_id)
return web.json_response(result)
except RecipeNotFoundError as exc:
return web.json_response({"success": False, "error": str(exc)}, status=404)
except Exception as exc:
self._logger.error("Error rematching single recipe: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def get_rematch_progress(self, request: web.Request) -> web.Response:
try:
progress = self._ws_manager.get_recipe_rematch_progress()
if progress:
return web.json_response({"success": True, "progress": progress})
return web.json_response(
{"success": False, "message": "No rematch in progress"}, status=404
)
except Exception as exc:
self._logger.error("Error getting rematch progress: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def reimport_recipe(self, request: web.Request) -> web.Response:
"""Delete a recipe and re-import it from its source URL.
@@ -996,10 +1262,10 @@ class RecipeManagementHandler:
*,
image_url: str,
name: str,
lora_entries: list,
checkpoint_entry: dict,
gen_params_request: dict,
tags: list,
lora_entries: list[Any],
checkpoint_entry: Dict[str, Any] | None,
gen_params_request: Dict[str, Any] | None,
tags: list[Any],
base_model: str,
source_path: str,
) -> web.Response:
@@ -1032,6 +1298,12 @@ class RecipeManagementHandler:
_original_image_url,
) = await self._download_remote_media(image_url)
# Build a version-cached map of local model hashes to cache items so
# CivitaiApiMetadataParser can skip CivitAI API calls for models that
# exist on disk. Built once and shared by every parse pass below.
local_cache = await recipe_scanner.build_local_hash_cache()
from ...recipes.parsers.civitai_image import CivitaiApiMetadataParser
# Extract embedded EXIF metadata (offloaded to thread pool in this call)
embedded_gen_params = {}
parsed_embedded = None
@@ -1053,6 +1325,13 @@ class RecipeManagementHandler:
)
)
if parser:
if isinstance(parser, CivitaiApiMetadataParser):
parsed_embedded = await parser.parse_metadata(
raw_embedded,
recipe_scanner=recipe_scanner,
local_cache=local_cache,
)
else:
parsed_embedded = await parser.parse_metadata(
raw_embedded, recipe_scanner=recipe_scanner
)
@@ -1086,6 +1365,13 @@ class RecipeManagementHandler:
civitai_inner_meta
)
if parser:
if isinstance(parser, CivitaiApiMetadataParser):
civitai_parsed = await parser.parse_metadata(
civitai_inner_meta,
recipe_scanner=recipe_scanner,
local_cache=local_cache,
)
else:
civitai_parsed = await parser.parse_metadata(
civitai_inner_meta, recipe_scanner=recipe_scanner
)
@@ -1187,6 +1473,33 @@ class RecipeManagementHandler:
self._logger.error("Error updating recipe: %s", exc, exc_info=True)
return web.json_response({"error": str(exc)}, status=500)
async def record_recipe_open(self, request: web.Request) -> web.Response:
"""Record that a recipe's detail modal was opened.
Lightweight fire-and-forget endpoint backing the "Recently Opened"
sort. It only writes the timestamp into the separate open-stats file
recipe JSON and EXIF are never touched.
"""
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
raise RuntimeError("Recipe scanner unavailable")
recipe_id = request.match_info["recipe_id"]
# Skip recording opens for recipes the scanner no longer knows.
recipe_json_path = await recipe_scanner.get_recipe_json_path(recipe_id)
if not recipe_json_path:
return web.json_response(
{"success": False, "error": "Recipe not found"}, status=404
)
RecipeOpenStats().record_open(recipe_id)
return web.json_response({"success": True})
except Exception as exc:
self._logger.error("Error recording recipe open: %s", exc, exc_info=True)
return web.json_response({"success": False, "error": str(exc)}, status=500)
async def move_recipe(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
@@ -1280,6 +1593,35 @@ class RecipeManagementHandler:
self._logger.error("Error reconnecting LoRA: %s", exc, exc_info=True)
return web.json_response({"error": str(exc)}, status=500)
async def mark_lora_hash_invalid(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
raise RuntimeError("Recipe scanner unavailable")
data = await request.json()
for field in ("recipe_id", "lora_index"):
if field not in data:
raise RecipeValidationError(f"Missing required field: {field}")
result = await self._persistence_service.mark_lora_hash_invalid(
recipe_scanner=recipe_scanner,
recipe_id=data["recipe_id"],
lora_index=int(data["lora_index"]),
hash_invalid=bool(data.get("hash_invalid", True)),
)
return web.json_response(result.payload, status=result.status)
except RecipeValidationError as exc:
return web.json_response({"error": str(exc)}, status=400)
except RecipeNotFoundError as exc:
return web.json_response({"error": str(exc)}, status=404)
except Exception as exc:
self._logger.error(
"Error marking LoRA hash invalid: %s", exc, exc_info=True
)
return web.json_response({"error": str(exc)}, status=500)
async def bulk_delete(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
@@ -1592,7 +1934,7 @@ class RecipeManagementHandler:
if not provider:
return ""
version_info = await provider.get_model_version_info(version_id)
version_info = await provider.get_model_version_info(str(version_id))
if isinstance(version_info, tuple):
version_info = version_info[0]
@@ -1712,6 +2054,12 @@ class RecipeManagementHandler:
await self._download_remote_media(image_url)
)
# Build a version-cached map of local model hashes to cache items so
# CivitaiApiMetadataParser can skip CivitAI API calls for models that
# exist on disk. Built once and shared by every parse pass below.
local_cache = await recipe_scanner.build_local_hash_cache()
from ...recipes.parsers.civitai_image import CivitaiApiMetadataParser
# Extract embedded EXIF metadata
embedded_gen_params = {}
parsed_embedded = None
@@ -1733,6 +2081,13 @@ class RecipeManagementHandler:
)
)
if parser:
if isinstance(parser, CivitaiApiMetadataParser):
parsed_embedded = await parser.parse_metadata(
raw_embedded,
recipe_scanner=recipe_scanner,
local_cache=local_cache,
)
else:
parsed_embedded = await parser.parse_metadata(
raw_embedded, recipe_scanner=recipe_scanner
)
@@ -1773,6 +2128,13 @@ class RecipeManagementHandler:
)
)
if parser:
if isinstance(parser, CivitaiApiMetadataParser):
parsed_embedded = await parser.parse_metadata(
raw_orig,
recipe_scanner=recipe_scanner,
local_cache=local_cache,
)
else:
parsed_embedded = await parser.parse_metadata(
raw_orig, recipe_scanner=recipe_scanner
)
@@ -1809,6 +2171,13 @@ class RecipeManagementHandler:
civitai_inner_meta
)
if parser:
if isinstance(parser, CivitaiApiMetadataParser):
civitai_parsed = await parser.parse_metadata(
civitai_inner_meta,
recipe_scanner=recipe_scanner,
local_cache=local_cache,
)
else:
civitai_parsed = await parser.parse_metadata(
civitai_inner_meta, recipe_scanner=recipe_scanner
)
@@ -1844,14 +2213,21 @@ class RecipeManagementHandler:
civitai_base_model = civitai_parsed.get("base_model")
if civitai_base_model and not metadata.get("base_model"):
metadata["base_model"] = civitai_base_model
elif parsed_embedded:
# EXIF fills whatever the API-only parse left open — when the image
# API meta is null (only modelVersionIds present) the API parse
# yields a checkpoint but no LoRAs, while the image EXIF carries the
# full resource list.
if parsed_embedded:
if not metadata.get("loras"):
parsed_loras = parsed_embedded.get("loras")
if parsed_loras and not metadata.get("loras"):
if parsed_loras:
metadata["loras"] = parsed_loras
if not metadata.get("checkpoint"):
parsed_model = parsed_embedded.get("model")
if parsed_model and not metadata.get("checkpoint"):
if parsed_model:
metadata["checkpoint"] = parsed_model
if parsed_embedded.get("base_model") and not metadata.get("base_model"):
if not metadata.get("base_model") and parsed_embedded.get("base_model"):
metadata["base_model"] = parsed_embedded["base_model"]
civitai_client = self._civitai_client_getter()
@@ -2023,30 +2399,41 @@ class RecipeManagementHandler:
parsed_input = {**image_data, **inner_meta}
parsed_input.pop("meta", None)
# Build a local cache of {hash cache_item} so the parser can
# skip CivitAI API calls for models that exist on disk.
local_cache: Dict[str, Dict[str, Any]] = {}
# Build the shared local hash cache so the parser can skip CivitAI
# API calls for models that exist on disk.
local_cache: Dict[str, Dict[str, Any]] = (
await recipe_scanner.build_local_hash_cache()
)
# Bounded supplement for un-backfilled parents. The shared builder
# never computes autov3; when the parent model exists on disk but
# its cached entry has no stored AutoV3, compute it for that single
# file and register the AutoV3 key so the parser can also match on
# that hash type (CivitAI metadata resources use AutoV3). This runs
# whenever the parent is found with an empty autov3, independent of
# whether the sha256 key is already present in the shared cache.
if model_hash:
lora_scanner = getattr(recipe_scanner, "_lora_scanner", None)
if lora_scanner and model_hash:
if lora_scanner:
try:
parent_cache_data = await lora_scanner.get_cached_data()
for item in getattr(parent_cache_data, "raw_data", []):
if item.get("sha256", "").lower() == model_hash.lower():
local_cache[model_hash.lower()] = item
# Compute AutoV3 so the parser can also match on
# that hash type (CivitAI metadata resources use
# AutoV3).
autov3 = (item.get("autov3") or "").lower()
if not autov3:
file_path = item.get("file_path")
if file_path and os.path.exists(file_path):
try:
from ...utils.file_utils import (
calculate_autov3,
)
autov3 = calculate_autov3(file_path)
if autov3:
local_cache[autov3.lower()] = item
autov3 = (
calculate_autov3(file_path) or ""
).lower()
except Exception:
pass
if autov3:
local_cache[autov3] = item
break
except Exception:
pass
@@ -2081,10 +2468,10 @@ class RecipeManagementHandler:
parent_model_id: int | None = None
parent_version_name: str | None = None
parent_model_name: str | None = None
# Prefer sha256 key; fall back to any cached entry.
# Resolve the parent strictly by its sha256 key. There is no
# arbitrary fallback: with a full-library cache, picking any entry
# would corrupt the isDeleted reconciliation below.
parent_item = local_cache.get(model_hash.lower()) if model_hash else None
if parent_item is None and local_cache:
parent_item = next(iter(local_cache.values()))
if parent_item:
civ = parent_item.get("civitai") or {}
if isinstance(civ, dict):
@@ -2218,6 +2605,31 @@ class RecipeManagementHandler:
"Failed to download image for recipe: %s", exc
)
# Fallback: try to locate a custom image on disk using model_hash + image id
if image_bytes is None:
image_id = image_data.get("id") or ""
if image_id and model_hash:
from ...utils.example_images_paths import get_model_folder
model_folder = get_model_folder(model_hash)
if model_folder and os.path.exists(model_folder):
for fname in os.listdir(model_folder):
if f"custom_{image_id}" in fname:
ext = os.path.splitext(fname)[1].lower()
if ext not in (".jpg", ".jpeg", ".png", ".webp", ".gif"):
continue
fpath = os.path.join(model_folder, fname)
if os.path.isfile(fpath):
try:
with open(fpath, "rb") as f:
image_bytes = f.read()
extension = ext
except Exception as exc:
self._logger.warning(
"Failed to read custom image file %s: %s",
fpath, exc,
)
break
prompt = (
(parsed.get("gen_params") or {}).get("prompt") or ""
)
@@ -2275,7 +2687,7 @@ class RecipeAnalysisHandler:
content_type = request.headers.get("Content-Type", "")
if "multipart/form-data" in content_type:
reader = await request.multipart()
field = await reader.next()
field: Any = await reader.next()
if field is None or field.name != "image":
raise RecipeValidationError("No image field found")
image_chunks = bytearray()
@@ -2392,6 +2804,91 @@ class RecipeSharingHandler:
return web.json_response({"error": str(exc)}, status=500)
class RecipeWorkflowHandler:
"""Extract an embedded workflow from a recipe image and broadcast it."""
def __init__(
self,
*,
ensure_dependencies_ready: EnsureDependenciesCallable,
recipe_scanner_getter: RecipeScannerGetter,
prompt_server: type[PromptServerProtocol],
logger: Logger,
) -> None:
self._ensure_dependencies_ready = ensure_dependencies_ready
self._recipe_scanner_getter = recipe_scanner_getter
self._prompt_server = prompt_server
self._logger = logger
async def send_recipe_workflow(self, request: web.Request) -> web.Response:
try:
await self._ensure_dependencies_ready()
recipe_scanner = self._recipe_scanner_getter()
if recipe_scanner is None:
raise RuntimeError("Recipe scanner unavailable")
recipe_id = request.match_info["recipe_id"]
recipe = await recipe_scanner.get_recipe_by_id(recipe_id)
if not recipe:
return web.json_response({"error": "Recipe not found"}, status=404)
if os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1":
return web.json_response(
{"error": "Standalone Mode Active"}, status=400
)
image_path = recipe.get("file_path")
if not image_path:
return web.json_response({"error": "no_workflow"}, status=404)
metadata = await asyncio.to_thread(
ExifUtils._load_structured_metadata, image_path
)
workflow_raw = metadata.get("workflow")
if not workflow_raw:
return web.json_response(
{
"error": "no_workflow",
"message": "No embedded workflow found in recipe image",
},
status=404,
)
# _load_structured_metadata always yields workflow as a JSON string;
# the frontend extension expects a parsed object for loadGraphData.
try:
workflow = (
json.loads(workflow_raw)
if isinstance(workflow_raw, str)
else workflow_raw
)
except (TypeError, ValueError):
self._logger.warning(
"Recipe %s embeds a non-JSON workflow payload; skipping send",
recipe_id,
)
return web.json_response(
{
"error": "no_workflow",
"message": "Embedded workflow data is not valid JSON",
},
status=404,
)
self._prompt_server.instance.send_sync(
"lm_load_workflow",
{
"workflow": workflow,
"name": recipe.get("title") or "",
"recipe_id": recipe_id,
},
)
return web.json_response({"success": True, "sent": True})
except Exception as exc:
self._logger.error("Error sending recipe workflow: %s", exc, exc_info=True)
return web.json_response({"error": str(exc)}, status=500)
class BatchImportHandler:
"""Handle batch import operations for recipes."""
+6 -71
View File
@@ -1,8 +1,8 @@
import asyncio
import logging
from aiohttp import web
from typing import Dict
from server import PromptServer # type: ignore
from typing import Any, Dict
from server import PromptServer # pyright: ignore[reportMissingImports]
from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
@@ -31,13 +31,13 @@ class LoraRoutes(BaseModelRoutes):
# Attach service dependencies
self.attach_service(self.service)
def setup_routes(self, app: web.Application):
def setup_routes(self, app: web.Application, prefix: str = "loras"):
"""Setup LoRA routes"""
# Schedule service initialization on app startup
app.on_startup.append(lambda _: self.initialize_services())
# Setup common routes with 'loras' prefix (includes page route)
super().setup_routes(app, "loras")
super().setup_routes(app, prefix)
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str):
"""Setup LoRA-specific routes"""
@@ -73,7 +73,7 @@ class LoraRoutes(BaseModelRoutes):
"POST", "/api/lm/{prefix}/get_trigger_words", prefix, self.get_trigger_words
)
def _parse_specific_params(self, request: web.Request) -> Dict:
def _parse_specific_params(self, request: web.Request) -> Dict[str, Any]:
"""Parse LoRA-specific parameters"""
params = {}
@@ -119,25 +119,6 @@ class LoraRoutes(BaseModelRoutes):
logger.error(f"Error getting letter counts: {e}")
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_notes(self, request: web.Request) -> web.Response:
"""Get notes for a specific LoRA file"""
try:
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text="Lora file name is required", status=400)
notes = await self.service.get_lora_notes(lora_name)
if notes is not None:
return web.json_response({"success": True, "notes": notes})
else:
return web.json_response(
{"success": False, "error": "LoRA not found in cache"}, status=404
)
except Exception as e:
logger.error(f"Error getting lora notes: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_trigger_words(self, request: web.Request) -> web.Response:
"""Get trigger words for a specific LoRA file"""
try:
@@ -168,52 +149,6 @@ class LoraRoutes(BaseModelRoutes):
logger.error(f"Error getting lora usage tips by path: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_preview_url(self, request: web.Request) -> web.Response:
"""Get the static preview URL for a LoRA file"""
try:
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text="Lora file name is required", status=400)
preview_url = await self.service.get_lora_preview_url(lora_name)
if preview_url:
return web.json_response({"success": True, "preview_url": preview_url})
else:
return web.json_response(
{
"success": False,
"error": "No preview URL found for the specified lora",
},
status=404,
)
except Exception as e:
logger.error(f"Error getting lora preview URL: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_civitai_url(self, request: web.Request) -> web.Response:
"""Get the Civitai URL for a LoRA file"""
try:
lora_name = request.query.get("name")
if not lora_name:
return web.Response(text="Lora file name is required", status=400)
result = await self.service.get_lora_civitai_url(lora_name)
if result["civitai_url"]:
return web.json_response({"success": True, **result})
else:
return web.json_response(
{
"success": False,
"error": "No Civitai data found for the specified lora",
},
status=404,
)
except Exception as e:
logger.error(f"Error getting lora Civitai URL: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_random_loras(self, request: web.Request) -> web.Response:
"""Get random LoRAs based on filters and strength ranges"""
try:
@@ -337,7 +272,7 @@ class LoraRoutes(BaseModelRoutes):
graph_identifier = entry.get("graph_id")
try:
parsed_node_id = int(node_identifier)
parsed_node_id = int(node_identifier) # pyright: ignore[reportArgumentType]
except (TypeError, ValueError):
parsed_node_id = node_identifier
+20 -2
View File
@@ -5,7 +5,7 @@ miscellaneous endpoints share a consistent registration flow.
"""
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from typing import Any, Callable, Iterable, Mapping
from aiohttp import web
@@ -22,6 +22,8 @@ class RouteDefinition:
MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/settings", "get_settings"),
RouteDefinition("POST", "/api/lm/settings", "update_settings"),
RouteDefinition("GET", "/api/lm/llm/models", "get_llm_models"),
RouteDefinition("GET", "/api/lm/llm/provider-models", "get_provider_models"),
RouteDefinition("GET", "/api/lm/doctor/diagnostics", "get_doctor_diagnostics"),
RouteDefinition("POST", "/api/lm/doctor/repair-cache", "repair_doctor_cache"),
RouteDefinition("POST", "/api/lm/doctor/resolve-filename-conflicts", "resolve_doctor_filename_conflicts"),
@@ -30,6 +32,7 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/settings/libraries", "get_settings_libraries"),
RouteDefinition("POST", "/api/lm/settings/libraries/activate", "activate_library"),
RouteDefinition("GET", "/api/lm/health-check", "health_check"),
RouteDefinition("GET", "/api/lm/init-status", "get_init_status"),
RouteDefinition("GET", "/api/lm/supporters", "get_supporters"),
RouteDefinition("GET", "/api/lm/wildcards/search", "search_wildcards"),
RouteDefinition("POST", "/api/lm/wildcards/open-location", "open_wildcards_location"),
@@ -37,10 +40,12 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("POST", "/api/lm/update-usage-stats", "update_usage_stats"),
RouteDefinition("GET", "/api/lm/get-usage-stats", "get_usage_stats"),
RouteDefinition("POST", "/api/lm/update-lora-code", "update_lora_code"),
RouteDefinition("GET", "/api/lm/update-lora-code", "get_update_lora_code"),
RouteDefinition("GET", "/api/lm/trained-words", "get_trained_words"),
RouteDefinition("GET", "/api/lm/model-example-files", "get_model_example_files"),
RouteDefinition("POST", "/api/lm/register-nodes", "register_nodes"),
RouteDefinition("POST", "/api/lm/update-node-widget", "update_node_widget"),
RouteDefinition("GET", "/api/lm/update-node-widget", "get_update_node_widget"),
RouteDefinition("GET", "/api/lm/get-registry", "get_registry"),
RouteDefinition("GET", "/api/lm/check-model-exists", "check_model_exists"),
RouteDefinition("GET", "/api/lm/check-models-exist", "check_models_exist"),
@@ -101,6 +106,19 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition(
"POST", "/api/lm/download-hf-model", "download_hf_model"
),
RouteDefinition(
"POST", "/api/lm/set-hf-url", "set_hf_url"
),
# Agent skill endpoints
RouteDefinition(
"GET", "/api/lm/agent/skills", "get_agent_skills"
),
RouteDefinition(
"POST", "/api/lm/agent/execute/{skill_name}", "execute_agent_skill"
),
RouteDefinition(
"POST", "/api/lm/agent/cancel", "cancel_agent_skill"
),
)
@@ -130,7 +148,7 @@ class MiscRouteRegistrar:
handler_lookup[definition.handler_name],
)
def _bind(self, method: str, path: str, handler: Callable) -> None:
def _bind(self, method: str, path: str, handler: Callable[..., Any]) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)
+4 -1
View File
@@ -7,7 +7,7 @@ import os
from typing import Awaitable, Callable, Mapping
from aiohttp import web
from server import PromptServer # type: ignore
from server import PromptServer # pyright: ignore[reportMissingImports]
from ..services.metadata_service import (
get_metadata_archive_manager,
@@ -40,6 +40,7 @@ from .handlers.misc_handlers import (
)
from .handlers.base_model_handlers import BaseModelHandlerSet
from .handlers.hf_handlers import HfHandler
from .handlers.agent_handlers import AgentHandler
from .misc_route_registrar import MiscRouteRegistrar
logger = logging.getLogger(__name__)
@@ -138,6 +139,7 @@ class MiscRoutes:
example_workflows = ExampleWorkflowsHandler()
base_model = BaseModelHandlerSet()
hf_handler = HfHandler()
agent_handler = AgentHandler()
return self._handler_set_factory(
health=health,
@@ -158,6 +160,7 @@ class MiscRoutes:
example_workflows=example_workflows,
base_model=base_model,
hf_handler=hf_handler,
agent_handler=agent_handler,
)
+5 -4
View File
@@ -3,7 +3,7 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from typing import Any, Callable, Iterable, Mapping
from aiohttp import web
@@ -46,6 +46,7 @@ COMMON_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
"GET", "/api/lm/{prefix}/auto-organize-progress", "get_auto_organize_progress"
),
RouteDefinition("GET", "/api/lm/{prefix}/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/{prefix}/search-tags", "search_tags"),
RouteDefinition("GET", "/api/lm/{prefix}/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/{prefix}/model-types", "get_model_types"),
RouteDefinition("GET", "/api/lm/{prefix}/scan", "scan_models"),
@@ -173,15 +174,15 @@ class ModelRouteRegistrar:
handler_lookup[definition.handler_name],
)
def add_route(self, method: str, path: str, handler: Callable) -> None:
def add_route(self, method: str, path: str, handler: Callable[..., Any]) -> None:
self._bind_route(method, path, handler)
def add_prefixed_route(
self, method: str, path_template: str, prefix: str, handler: Callable
self, method: str, path_template: str, prefix: str, handler: Callable[..., Any]
) -> None:
self._bind_route(method, path_template.replace("{prefix}", prefix), handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
def _bind_route(self, method: str, path: str, handler: Callable[..., Any]) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)
+25
View File
@@ -0,0 +1,25 @@
"""Route controller for the pending-delete undo endpoint."""
from __future__ import annotations
from aiohttp import web
from .handlers.pending_delete_handler import PendingDeleteHandler
class PendingDeleteRoutes:
"""Shared route controller mirroring MiscRoutes/UpdateRoutes.
Registered ONCE per mode (py/lora_manager.py, standalone.py); NEVER through
the per-model-type ModelRouteRegistrar, which is instantiated per model
type and would register this non-prefixed route three times.
"""
@staticmethod
def setup_routes(app: web.Application) -> None:
"""Register the shared undo-delete endpoint."""
handler = PendingDeleteHandler()
_ = app.router.add_post("/api/lm/undo-delete", handler.undo_delete)
__all__ = ["PendingDeleteRoutes"]
+17 -2
View File
@@ -3,7 +3,7 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Mapping
from typing import Any, Callable, Mapping
from aiohttp import web
@@ -29,6 +29,7 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("POST", "/api/lm/recipes/save", "save_recipe"),
RouteDefinition("DELETE", "/api/lm/recipe/{recipe_id}", "delete_recipe"),
RouteDefinition("GET", "/api/lm/recipes/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/recipes/search-tags", "search_tags"),
RouteDefinition("GET", "/api/lm/recipes/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/recipes/roots", "get_roots"),
RouteDefinition("GET", "/api/lm/recipes/folders", "get_folders"),
@@ -42,9 +43,15 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/syntax", "get_recipe_syntax"),
RouteDefinition("PUT", "/api/lm/recipe/{recipe_id}/update", "update_recipe"),
RouteDefinition(
"POST", "/api/lm/recipe/{recipe_id}/opened", "record_recipe_open"
),
RouteDefinition("POST", "/api/lm/recipe/move", "move_recipe"),
RouteDefinition("POST", "/api/lm/recipes/move-bulk", "move_recipes_bulk"),
RouteDefinition("POST", "/api/lm/recipe/lora/reconnect", "reconnect_lora"),
RouteDefinition(
"POST", "/api/lm/recipe/lora/mark-hash-invalid", "mark_lora_hash_invalid"
),
RouteDefinition("GET", "/api/lm/recipes/find-duplicates", "find_duplicates"),
RouteDefinition("POST", "/api/lm/recipes/bulk-delete", "bulk_delete"),
RouteDefinition(
@@ -60,6 +67,11 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("POST", "/api/lm/recipe/{recipe_id}/repair", "repair_recipe"),
RouteDefinition("POST", "/api/lm/recipes/repair-bulk", "repair_recipes_bulk"),
RouteDefinition("GET", "/api/lm/recipes/repair-progress", "get_repair_progress"),
RouteDefinition("POST", "/api/lm/recipes/rematch", "rematch_recipes"),
RouteDefinition("POST", "/api/lm/recipes/rematch-bulk", "rematch_recipes_bulk"),
RouteDefinition("POST", "/api/lm/recipe/{recipe_id}/rematch", "rematch_recipe"),
RouteDefinition("POST", "/api/lm/recipes/cancel-rematch", "cancel_rematch"),
RouteDefinition("GET", "/api/lm/recipes/rematch-progress", "get_rematch_progress"),
RouteDefinition("POST", "/api/lm/recipes/batch-import/start", "start_batch_import"),
RouteDefinition(
"GET", "/api/lm/recipes/batch-import/progress", "get_batch_import_progress"
@@ -81,6 +93,9 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition(
"POST", "/api/lm/recipe/{recipe_id}/reimport", "reimport_recipe"
),
RouteDefinition(
"POST", "/api/lm/recipe/{recipe_id}/send-workflow", "send_recipe_workflow"
),
)
@@ -104,7 +119,7 @@ class RecipeRouteRegistrar:
handler = handler_lookup[definition.handler_name]
self._bind_route(definition.method, definition.path, handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
def _bind_route(self, method: str, path: str, handler: Callable[..., Any]) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)
+11 -10
View File
@@ -40,10 +40,11 @@ class StatsRoutes:
"""Route handlers for Statistics page and API endpoints"""
def __init__(self):
self.lora_scanner = None
self.checkpoint_scanner = None
self.embedding_scanner = None
self.usage_stats = None
self.lora_scanner: Any = None
self.checkpoint_scanner: Any = None
self.embedding_scanner: Any = None
self.usage_stats: Any = None
self._i18n_filter_added = False
self.template_env = jinja2.Environment(
loader=jinja2.FileSystemLoader(config.templates_path),
autoescape=True
@@ -95,9 +96,9 @@ class StatsRoutes:
server_i18n.set_locale(user_language)
# 为模板环境添加i18n过滤器
if not hasattr(self.template_env, '_i18n_filter_added'):
if not self._i18n_filter_added:
self.template_env.filters['t'] = server_i18n.create_template_filter()
self.template_env._i18n_filter_added = True
self._i18n_filter_added = True
template = self.template_env.get_template('statistics.html')
rendered = template.render(
@@ -549,7 +550,7 @@ class StatsRoutes:
'error': str(e)
}, status=500)
def _count_unused_models(self, models: List[Dict], usage_data: Dict) -> int:
def _count_unused_models(self, models: List[Dict[str, Any]], usage_data: Dict[str, Any]) -> int:
"""Count models that have never been used"""
used_hashes = set(usage_data.keys())
unused_count = 0
@@ -560,7 +561,7 @@ class StatsRoutes:
return unused_count
def _get_top_used_models(self, usage_data: Dict, model_map: Dict, limit: int) -> List[Dict]:
def _get_top_used_models(self, usage_data: Dict[str, Any], model_map: Dict[str, Any], limit: int) -> List[Dict[str, Any]]:
"""Get top used models with their metadata"""
sorted_usage = sorted(usage_data.items(), key=lambda x: x[1].get('total', 0), reverse=True)
@@ -578,7 +579,7 @@ class StatsRoutes:
return top_models
def _get_usage_timeline(self, usage_data: Dict, days: int) -> List[Dict]:
def _get_usage_timeline(self, usage_data: Dict[str, Any], days: int) -> List[Dict[str, Any]]:
"""Get usage timeline for the past N days"""
timeline = []
today = datetime.now()
@@ -614,7 +615,7 @@ class StatsRoutes:
return list(reversed(timeline)) # Oldest to newest
def _format_size(self, size_bytes: int) -> str:
def _format_size(self, size_bytes: float) -> str:
"""Format file size in human readable format"""
for unit in ['B', 'KB', 'MB', 'GB', 'TB']:
if size_bytes < 1024.0:
+310 -39
View File
@@ -6,7 +6,7 @@ import shutil
import tempfile
import asyncio
from aiohttp import web, ClientError
from typing import Dict, List
from typing import Any, Dict, List, cast
from ..utils.settings_paths import ensure_settings_file
from ..services.downloader import get_downloader
@@ -38,6 +38,84 @@ def _clean_excludes() -> List[str]:
return excludes
def _stage_preserved_items(plugin_root: str) -> tuple[str, list[str]]:
"""Move preserved user-data items to a temp directory outside *plugin_root*.
This ensures that ``git reset --hard``, ``git clean -fd``, and ZIP-based
replacement cannot touch these files even when ``-e`` exclusion patterns
are mishandled (e.g. on Windows where forward-slash patterns may not
match backslash-prefixed paths in some Git builds, or where file locks
prevent deletion/recreation).
Returns:
``(backup_root, staged_names)``: the temp directory path and the
list of item names that were successfully moved.
"""
backup_root = tempfile.mkdtemp(prefix='lora_manager_update_')
staged: list[str] = []
for name in _PRESERVE_DIRS:
src = os.path.join(plugin_root, name)
if not os.path.lexists(src):
continue
dst = os.path.join(backup_root, name)
try:
shutil.move(src, dst)
staged.append(name)
logger.debug("Staged '%s' for update safety", name)
except OSError:
# ``shutil.move`` may fail on Windows if a file handle inside
# the directory is still open (e.g. a SQLite WAL file). Fall
# back to copy-then-remove.
logger.debug("Move failed for '%s', falling back to copy", name)
try:
if os.path.isdir(src) and not os.path.islink(src):
shutil.copytree(src, dst, symlinks=True)
shutil.rmtree(src, ignore_errors=True)
else:
shutil.copy2(src, dst)
os.remove(src)
staged.append(name)
logger.info("Copied (then removed) '%s' for update safety", name)
except Exception as exc:
logger.warning(
"Could not stage '%s': %s (will rely on git -e / skip lists)", name, exc
)
return backup_root, staged
def _restore_preserved_items(plugin_root: str, backup_root: str, staged: list[str]) -> None:
"""Move staged items back from *backup_root* into *plugin_root*.
Any leftover placeholder at the destination (created by git checkout or
ZIP extraction) is removed before the move.
"""
for name in staged:
src = os.path.join(backup_root, name)
dst = os.path.join(plugin_root, name)
try:
if os.path.lexists(dst):
if os.path.isdir(dst) and not os.path.islink(dst):
shutil.rmtree(dst, ignore_errors=True)
else:
os.remove(dst)
shutil.move(src, dst)
logger.debug("Restored '%s' after update", name)
except OSError:
logger.debug("Move failed restoring '%s', falling back to copy", name)
try:
if os.path.isdir(src) and not os.path.islink(src):
shutil.copytree(src, dst, symlinks=True, dirs_exist_ok=True)
shutil.rmtree(src, ignore_errors=True)
else:
shutil.copy2(src, dst)
os.remove(src)
logger.info("Copied '%s' back after update", name)
except Exception as exc:
logger.error("Failed to restore '%s': %s", name, exc)
shutil.rmtree(backup_root, ignore_errors=True)
class UpdateRoutes:
"""Routes for handling plugin update checks"""
@@ -47,6 +125,7 @@ class UpdateRoutes:
app.router.add_get('/api/lm/check-updates', UpdateRoutes.check_updates)
app.router.add_get('/api/lm/version-info', UpdateRoutes.get_version_info)
app.router.add_post('/api/lm/perform-update', UpdateRoutes.perform_update)
app.router.add_post('/api/lm/switch-channel', UpdateRoutes.switch_channel)
@staticmethod
async def check_updates(request):
@@ -65,10 +144,17 @@ class UpdateRoutes:
# Fetch remote version from GitHub
if nightly:
remote_version, changelog = await UpdateRoutes._get_nightly_version()
releases = None
local_hash = git_info.get('short_hash', '')
nightly_version, releases_result = await asyncio.gather(
UpdateRoutes._get_nightly_version(local_hash),
UpdateRoutes._get_remote_version()
)
remote_version, _, behind_by, commit_date = nightly_version
_, changelog, releases = releases_result
else:
remote_version, changelog, releases = await UpdateRoutes._get_remote_version()
behind_by = 0
commit_date = ''
# Compare versions
if nightly:
@@ -81,6 +167,10 @@ class UpdateRoutes:
remote_version.replace('v', '')
)
current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir))
has_git = os.path.exists(os.path.join(plugin_root, '.git'))
response_data = {
'success': True,
'current_version': local_version,
@@ -88,13 +178,13 @@ class UpdateRoutes:
'update_available': update_available,
'changelog': changelog,
'git_info': git_info,
'nightly': nightly
'nightly': nightly,
'has_git': has_git,
'releases': releases,
'behind_by': behind_by,
'commit_date': commit_date
}
# Include releases list for stable mode
if releases is not None:
response_data['releases'] = releases
return web.json_response(response_data)
except NETWORK_EXCEPTIONS as e:
@@ -126,9 +216,14 @@ class UpdateRoutes:
# Format: version-short_hash
version_string = f"{local_version}-{short_hash}"
current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir))
has_git = os.path.exists(os.path.join(plugin_root, '.git'))
return web.json_response({
'success': True,
'version': version_string
'version': version_string,
'has_git': has_git
})
except Exception as e:
@@ -156,20 +251,22 @@ class UpdateRoutes:
if os.path.exists(settings_path):
with open(settings_path, 'r', encoding='utf-8') as f:
settings_backup = f.read()
logger.info("Backed up settings.json")
logger.debug("Backed up settings.json (%d bytes)", len(settings_backup))
staged_backup_dir, staged_items = _stage_preserved_items(plugin_root)
try:
git_folder = os.path.join(plugin_root, '.git')
if os.path.exists(git_folder):
# Git update
success, new_version = await UpdateRoutes._perform_git_update(plugin_root, nightly)
else:
# Fallback: Download ZIP and replace files
success, new_version = await UpdateRoutes._download_and_replace_zip(plugin_root)
finally:
_restore_preserved_items(plugin_root, staged_backup_dir, staged_items)
if settings_backup and success:
with open(settings_path, 'w', encoding='utf-8') as f:
f.write(settings_backup)
logger.info("Restored settings.json")
logger.debug("Restored settings.json content (%d bytes)", len(settings_backup))
if success:
return web.json_response({
@@ -190,6 +287,164 @@ class UpdateRoutes:
'error': str(e)
})
@staticmethod
async def switch_channel(request):
"""
Switch between release and nightly update channels.
ZIP/CNR install Nightly: git init + checkout main (one-way upgrade)
Git install Release: git checkout latest tag (.git preserved)
ZIP/CNR install Release: ZIP download (no .git, stays in ZIP mode)
Git install Nightly: git checkout main + pull
"""
try:
body = await request.json() if request.has_body else {}
channel = body.get('channel', '')
if channel not in ('release', 'nightly'):
return web.json_response({
'success': False,
'error': f'Invalid channel: {channel}. Must be "release" or "nightly".'
})
current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir))
settings_path = ensure_settings_file(logger)
settings_backup = None
if os.path.exists(settings_path):
with open(settings_path, 'r', encoding='utf-8') as f:
settings_backup = f.read()
logger.debug("Backed up settings.json before channel switch (%d bytes)", len(settings_backup))
staged_backup_dir, staged_items = _stage_preserved_items(plugin_root)
try:
git_folder = os.path.join(plugin_root, '.git')
if channel == 'nightly':
git_backup = None
if os.path.exists(git_folder):
git_backup = UpdateRoutes._backup_git(git_folder, 'nightly')
success = False
new_version = ''
try:
if os.path.exists(git_folder):
success, new_version = await UpdateRoutes._perform_git_update(
plugin_root, nightly=True
)
else:
success, new_version = UpdateRoutes._init_git_repo(plugin_root)
finally:
UpdateRoutes._restore_git(git_backup, git_folder, success, 'nightly')
else:
success = False
new_version = ''
if os.path.exists(git_folder):
success, new_version = await UpdateRoutes._perform_git_update(
plugin_root, nightly=False
)
else:
tracking_file = os.path.join(plugin_root, '.tracking')
if os.path.exists(tracking_file):
os.remove(tracking_file)
success, new_version = await UpdateRoutes._download_and_replace_zip(plugin_root)
finally:
_restore_preserved_items(plugin_root, staged_backup_dir, staged_items)
if settings_backup and success:
with open(settings_path, 'w', encoding='utf-8') as f:
f.write(settings_backup)
logger.debug("Restored settings.json content after channel switch (%d bytes)", len(settings_backup))
if success:
return web.json_response({
'success': True,
'channel': channel,
'new_version': new_version,
'message': f'Switched to {channel} channel'
})
else:
return web.json_response({
'success': False,
'error': f'Failed to switch to {channel} channel'
})
except Exception as e:
logger.error("Failed to switch channel: %s", e, exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
})
@staticmethod
def _init_git_repo(plugin_root: str) -> tuple[bool, str]:
"""
Initialize a Git repository in a ZIP-installed plugin folder.
Clones the remote history and checks out main branch.
"""
try:
import git
except ImportError:
logger.error(
"GitPython is not available: cannot initialize git repo. "
"Install git or set $GIT_PYTHON_GIT_EXECUTABLE to the git binary path."
)
return False, ""
clean_excludes = _clean_excludes()
try:
repo = git.Repo.init(plugin_root)
origin = repo.create_remote(
'origin',
'https://github.com/willmiao/ComfyUI-Lora-Manager.git'
)
origin.fetch()
repo.create_head('main', origin.refs.main)
repo.git.checkout('main', '--force')
repo.git.reset('--hard')
repo.git.clean('-fd', *clean_excludes)
tracking_file = os.path.join(plugin_root, '.tracking')
if os.path.exists(tracking_file):
os.remove(tracking_file)
logger.info("Removed .tracking file (now in git mode)")
new_version = f"main-{repo.head.commit.hexsha[:7]}"
logger.info("Initialized git repo on main branch: %s", new_version)
return True, new_version
except Exception as e:
logger.error("Failed to initialize git repo: %s", e, exc_info=True)
return False, ""
@staticmethod
def _backup_git(git_folder, label):
try:
backup_dir = tempfile.mkdtemp()
backup = os.path.join(backup_dir, '.git')
shutil.copytree(git_folder, backup)
logger.info("Backed up .git before switching to %s", label)
return backup
except Exception as e:
logger.error("Failed to backup .git before %s switch: %s", label, e)
return None
@staticmethod
def _restore_git(git_backup, git_folder, success, label):
if git_backup and not success:
try:
if os.path.exists(git_folder):
shutil.rmtree(git_folder)
shutil.copytree(git_backup, git_folder)
logger.info("Restored .git after failed %s switch", label)
except Exception as e:
logger.error("Failed to restore .git after %s switch: %s", label, e)
if git_backup:
shutil.rmtree(os.path.dirname(git_backup), ignore_errors=True)
@staticmethod
async def _download_and_replace_zip(plugin_root: str) -> tuple[bool, str]:
"""
@@ -213,8 +468,9 @@ class UpdateRoutes:
logger.error(f"Failed to fetch release info: {data}")
return False, ""
zip_url = data.get("zipball_url")
version = data.get("tag_name", "unknown")
release_payload = cast(dict[str, Any], data)
zip_url = release_payload.get("zipball_url", "")
version = release_payload.get("tag_name", "unknown")
# Download ZIP to temporary file
with tempfile.NamedTemporaryFile(delete=False, suffix=".zip") as tmp_zip:
@@ -244,8 +500,7 @@ class UpdateRoutes:
except Exception:
logger.debug("Could not close downloaded-version history database", exc_info=True)
# Skip settings.json, civitai, model cache and runtime cache folders
UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json', 'civitai', 'model_cache', 'cache', 'wildcards', 'backups', 'stats'])
UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=list(_PRESERVE_DIRS))
# Extract ZIP to temp dir
with tempfile.TemporaryDirectory() as tmp_dir:
@@ -255,7 +510,7 @@ class UpdateRoutes:
extracted_root = next(os.scandir(tmp_dir)).path
# Copy files, skipping user data that should be preserved
skip_items = {'settings.json', 'civitai', 'wildcards', 'backups', 'stats'}
skip_items = set(_PRESERVE_DIRS)
for item in os.listdir(extracted_root):
if item in skip_items:
continue
@@ -272,7 +527,7 @@ class UpdateRoutes:
# for ComfyUI Manager to work properly
tracking_info_file = os.path.join(plugin_root, '.tracking')
tracking_files = []
skip_tracked = {'civitai', 'wildcards', 'backups', 'stats'}
skip_tracked = set(_PRESERVE_DIRS) - {'settings.json'}
for root, dirs, files in os.walk(extracted_root):
# Skip user data directories and their contents
rel_root = os.path.relpath(root, extracted_root)
@@ -296,6 +551,7 @@ class UpdateRoutes:
logger.error(f"ZIP update failed: {e}", exc_info=True)
return False, ""
@staticmethod
def _clean_plugin_folder(plugin_root, skip_files=None):
skip_files = skip_files or []
for item in os.listdir(plugin_root):
@@ -308,41 +564,56 @@ class UpdateRoutes:
os.remove(path)
@staticmethod
async def _get_nightly_version() -> tuple[str, List[str]]:
"""
Fetch latest commit from main branch
"""
async def _get_nightly_version(local_hash: str = "") -> tuple[str, List[str], int, str]:
repo_owner = "willmiao"
repo_name = "ComfyUI-Lora-Manager"
# Use GitHub API to fetch the latest commit from main branch
github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/commits/main"
try:
downloader = await get_downloader()
success, data = await downloader.make_request('GET', github_url, custom_headers={'Accept': 'application/vnd.github+json'})
success, data = await downloader.make_request(
'GET', github_url,
custom_headers={'Accept': 'application/vnd.github+json'}
)
if not success:
logger.warning(f"Failed to fetch GitHub commit: {data}")
return "main", []
logger.warning("Failed to fetch GitHub commit: %s", data)
return "main", [], 0, ""
commit_sha = data.get('sha', '')[:7] # Short hash
commit_message = data.get('commit', {}).get('message', '')
commit_payload = cast(dict[str, Any], data)
commit_sha = commit_payload.get('sha', '')[:7]
commit_message = commit_payload.get('commit', {}).get('message', '')
commit_date = commit_payload.get('commit', {}).get('committer', {}).get('date', '')[:10]
# Format as "main-{short_hash}"
version = f"main-{commit_sha}"
# Use commit message as changelog
changelog = [commit_message] if commit_message else []
return version, changelog
behind_by = 0
if local_hash and local_hash not in ('unknown', 'stable'):
compare_url = (
f"https://api.github.com/repos/{repo_owner}/{repo_name}"
f"/compare/{local_hash}...main"
)
c_ok, c_data = await downloader.make_request(
'GET', compare_url,
custom_headers={'Accept': 'application/vnd.github+json'}
)
if c_ok:
compare_payload = cast(dict[str, Any], c_data)
if compare_payload.get('status') in ('ahead', 'diverged'):
behind_by = compare_payload.get('ahead_by', 0)
else:
behind_by = compare_payload.get('behind_by', 0)
return version, changelog, behind_by, commit_date
except NETWORK_EXCEPTIONS as e:
logger.warning("Unable to reach GitHub for nightly version: %s", e)
return "main", []
return "main", [], 0, ""
except Exception as e:
logger.error(f"Error fetching nightly version: {e}", exc_info=True)
return "main", []
logger.error("Error fetching nightly version: %s", e, exc_info=True)
return "main", [], 0, ""
@staticmethod
def _compare_nightly_versions(local_git_info: Dict[str, str], remote_version: str) -> bool:
@@ -438,7 +709,7 @@ class UpdateRoutes:
logger.info(f"Successfully updated to {new_version}")
return True, new_version
except git.exc.GitError as e:
except git.exc.GitError as e: # pyright: ignore[reportAttributeAccessIssue]
logger.error(f"Git error during update: {e}")
return False, ""
except Exception as e:
@@ -499,7 +770,7 @@ class UpdateRoutes:
return git_info
@staticmethod
async def _get_remote_version() -> tuple[str, List[str], List[Dict]]:
async def _get_remote_version() -> tuple[str, List[str], List[Dict[str, Any]]]:
"""
Fetch remote version from GitHub
Returns:
@@ -521,7 +792,7 @@ class UpdateRoutes:
# Parse releases
releases = []
for i, release in enumerate(data):
for i, release in enumerate(cast(list[dict[str, Any]], data)):
version = release.get('tag_name', '')
if not version.startswith('v'):
version = f"v{version}"
+27
View File
@@ -0,0 +1,27 @@
"""LLM-powered metadata enrichment pipeline infrastructure.
This package provides the orchestration layer for LLM-powered features.
Skills define *what* to do (prompt template). The :class:`AgentService`
handles *how* (LLM calls, context gathering, validation, progress).
NOTE: The current implementation is a code-driven pipeline, not a true
agent loop. Future agent orchestration (LLM-driven tool selection) will
live alongside this package with its own namespace.
"""
from __future__ import annotations
from .skill_definition import SkillDefinition, SkillPermissions
from .skill_registry import SkillRegistry
from .agent_service import AgentService, AgentProgressReporter, SkillResult
from .post_processor import PostProcessor
__all__ = [
"AgentProgressReporter",
"AgentService",
"PostProcessor",
"SkillDefinition",
"SkillPermissions",
"SkillRegistry",
"SkillResult",
]
+489
View File
@@ -0,0 +1,489 @@
"""Pipeline orchestration service.
The :class:`AgentService` coordinates LLM-powered pipeline execution:
1. Look up the pipeline definition in :class:`SkillRegistry`
2. Validate input against its ``input_schema``
3. Prepare context via :mod:`~py.metadata_ops` (read metadata, list base models, fetch HF README)
4. If ``llm_required``: call :class:`LLMService` with the rendered prompt
5. Post-process via :class:`PostProcessor` (delegates I/O to :mod:`~py.metadata_ops`)
6. Broadcast progress and completion via :class:`WebSocketManager`
Pipeline definitions (*skills*) describe *what* to do (prompt template).
The AgentService handles *how* (LLM calls, context gathering, validation,
progress).
"""
from __future__ import annotations
import asyncio
import json
import logging
import re
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional
import aiohttp
import os
from ...config import config
from ..llm_service import LLMService
from ..websocket_manager import ws_manager
from .post_processor import PostProcessor
from .skill_registry import SkillRegistry
from .skills.enrich_hf_metadata.readme_processor import (
clean_readme_for_llm,
extract_relevant_section,
)
logger = logging.getLogger(__name__)
class AgentProgressReporter:
"""Protocol-compatible progress reporter backed by WebSocket broadcast."""
async def on_progress(self, payload: Dict[str, Any]) -> None:
await ws_manager.broadcast(payload)
@dataclass
class SkillResult:
"""Outcome of a skill execution."""
success: bool
updated_models: List[Dict[str, Any]] = field(default_factory=list)
errors: List[str] = field(default_factory=list)
summary: str = ""
def _validate_schema(data: Any, schema: Dict[str, Any], path: str = "") -> List[str]:
"""Minimal JSON schema validator.
Supports a subset of JSON Schema: ``type``, ``properties``, ``required``,
``items``, ``enum``. Returns a list of error messages (empty = valid).
"""
errors: List[str] = []
if not schema:
return errors
expected_type = schema.get("type")
if expected_type:
type_map = {
"string": str,
"number": (int, float),
"integer": int,
"boolean": bool,
"array": list,
"object": dict,
"null": type(None),
}
expected_py = type_map.get(expected_type)
if expected_py is not None and not isinstance(data, expected_py):
errors.append(f"{path or 'root'}: expected {expected_type}, got {type(data).__name__}")
return errors
if expected_type == "object" and isinstance(data, dict):
properties = schema.get("properties", {})
required = schema.get("required", [])
for req_key in required:
if req_key not in data:
errors.append(f"{path or 'root'}: missing required property '{req_key}'")
for key, value in data.items():
if key in properties:
errors.extend(_validate_schema(value, properties[key], f"{path}.{key}"))
if expected_type == "array" and isinstance(data, list):
items_schema = schema.get("items")
if items_schema:
for i, item in enumerate(data):
errors.extend(_validate_schema(item, items_schema, f"{path}[{i}]"))
if "enum" in schema and data not in schema["enum"]:
errors.append(f"{path or 'root'}: value '{data}' not in enum {schema['enum']}")
return errors
# ------------------------------------------------------------------
# Prompt template rendering
# ------------------------------------------------------------------
def _render_prompt(template: str, variables: Dict[str, Any]) -> str:
"""Render a prompt template with ``{{variable}}`` placeholders.
Uses simple regex substitution no Jinja2 dependency needed.
"""
def replace(match: re.Match[str]) -> str:
key = match.group(1).strip()
value = variables.get(key, "")
if isinstance(value, (dict, list)):
return json.dumps(value, ensure_ascii=False, indent=2)
return str(value)
return re.sub(r"\{\{(\w+)\}\}", replace, template)
class AgentService:
"""Orchestrate agent skill execution.
Usage::
service = await AgentService.get_instance()
result = await service.execute_skill(
skill_name="enrich_hf_metadata",
input_data={"model_paths": ["/path/to/model.safetensors"]},
progress_callback=AgentProgressReporter(),
)
"""
_instance: Optional["AgentService"] = None
_lock: asyncio.Lock = asyncio.Lock()
def __init__(
self,
*,
skill_registry: Optional[SkillRegistry] = None,
llm_service: Optional[LLMService] = None,
) -> None:
self._registry = skill_registry
self._llm_service = llm_service
@classmethod
async def get_instance(cls) -> "AgentService":
"""Return the lazily-initialised global ``AgentService``."""
if cls._instance is None:
async with cls._lock:
if cls._instance is None:
cls._instance = cls(
skill_registry=await SkillRegistry.get_instance(),
llm_service=await LLMService.get_instance(),
)
return cls._instance
@classmethod
def reset_instance(cls) -> None:
"""Reset the cached singleton — primarily for tests."""
cls._instance = None
async def _ensure_registry(self) -> SkillRegistry:
if self._registry is None:
self._registry = await SkillRegistry.get_instance()
return self._registry
async def _ensure_llm(self) -> LLMService:
if self._llm_service is None:
self._llm_service = await LLMService.get_instance()
return self._llm_service
async def list_skills(self) -> List[Dict[str, Any]]:
"""Return a JSON-serialisable list of available skills."""
registry = await self._ensure_registry()
return [
{
"name": s.name,
"title": s.title,
"description": s.description,
"llm_required": s.llm_required,
"model_type_filter": s.model_type_filter,
}
for s in registry.list_skills()
]
async def execute_skill(
self,
*,
skill_name: str,
input_data: Dict[str, Any],
progress_callback: Optional[AgentProgressReporter] = None,
) -> SkillResult:
"""Execute a pipeline (skill) on the given models.
Args:
skill_name: Name of the pipeline to execute
input_data: Input validated against the pipeline's ``input_schema``
progress_callback: Optional WebSocket progress reporter
Returns:
:class:`SkillResult` with success status and updated model info
"""
registry = await self._ensure_registry()
skill = registry.get_skill(skill_name)
if skill is None:
return SkillResult(
success=False,
errors=[f"Skill not found: {skill_name}"],
summary=f"Skill '{skill_name}' does not exist",
)
input_errors = _validate_schema(input_data, skill.input_schema)
if input_errors:
return SkillResult(
success=False,
errors=input_errors,
summary=f"Invalid input: {'; '.join(input_errors)}",
)
model_paths = input_data.get("model_paths", [])
if not model_paths:
return SkillResult(
success=False,
errors=["No model_paths provided"],
summary="No models to process",
)
total = len(model_paths)
processed = 0
success_count = 0
skipped_count = 0
updated_models: List[Dict[str, Any]] = []
errors: List[str] = []
post_processor = PostProcessor()
await self._emit_progress(
progress_callback, skill_name, status="started",
total=total, processed=0, success=0,
)
llm = await self._ensure_llm()
llm_configured = llm.is_configured() if skill.llm_required else True
for model_path in model_paths:
model_filename = os.path.basename(model_path)
logger.info(
"[%s] [%d/%d] %s",
skill_name, processed + 1, total, model_filename,
)
updated_data: Dict[str, Any] = {}
skip_model = False
try:
from ...metadata_ops import read_metadata
metadata = await read_metadata(model_path)
# Fast-fail: enrich_hf_metadata requires hf_url to have HF README context
if skill_name == "enrich_hf_metadata" and not metadata.get("hf_url", ""):
logger.info(
"[%s] SKIP %s — no hf_url in metadata",
skill_name, model_filename,
)
skipped_count += 1
skip_model = True
if not skip_model:
prompt_vars: Dict[str, Any] = {"model_path": model_path}
if skill.llm_required and llm_configured:
prompt_vars = await self._build_prompt_context(
skill_name, model_path, metadata, registry, llm,
)
llm_response: Optional[Dict[str, Any]] = None
if skill.llm_required and llm_configured:
prompt_template = registry.load_prompt(skill_name)
rendered = _render_prompt(prompt_template, prompt_vars)
llm_response = await llm.chat_completion_json(
system_prompt=prompt_vars.get(
"system_prompt",
"You are a helpful assistant that extracts structured metadata.",
),
user_prompt=rendered,
)
if llm_response:
logger.info(
"[%s] [%d/%d] %s → base_model=%s confidence=%s",
skill_name, processed + 1, total, model_filename,
(llm_response.get("base_model") or "?")[:50],
llm_response.get("confidence", "?"),
)
model_result = await post_processor.process(
skill_name=skill_name,
model_path=model_path,
llm_output=llm_response or {},
metadata=metadata,
readme_content=prompt_vars.get("readme_content_full", ""),
)
if model_result.get("success", True):
success_count += 1
uf = model_result.get("updated_fields", [])
if uf:
updated_models.append({"path": model_path, "updated_fields": uf})
updated_data = model_result.get("updates", {})
if "preview_url" in updated_data and updated_data["preview_url"]:
updated_data["preview_url"] = config.get_preview_static_url(
updated_data["preview_url"]
)
else:
errors.extend(
model_result.get("errors", [model_result.get("error", "Unknown error")])
)
except Exception as exc:
logger.error("Skill %s failed for %s: %s", skill_name, model_path, exc)
errors.append(f"{model_path}: {exc}")
processed += 1
await self._emit_progress(
progress_callback, skill_name, status="processing",
total=total, processed=processed, success=success_count,
skipped=skipped_count,
current_path=model_path,
updated_data=updated_data,
)
result = SkillResult(
success=success_count > 0,
updated_models=updated_models,
errors=errors,
summary=f"Processed {processed}/{total} models, {success_count} succeeded, {skipped_count} skipped",
)
await self._emit_progress(
progress_callback, skill_name, status="completed",
total=total, processed=processed, success=success_count,
skipped=skipped_count,
updated_models=updated_models, errors=errors, summary=result.summary,
)
return result
# ------------------------------------------------------------------
# Base model grouping (keeps the prompt compact)
# ------------------------------------------------------------------
@staticmethod
def _format_base_models(models: List[str]) -> str:
"""Format the base model list as a flat, one-per-line list.
Attempts to group by family consistently degraded LLM extraction
accuracy the LLM finds individual model names harder to spot
in comma-separated groups than in a simple ``- Name`` list.
"""
return "\n".join(f"- {m}" for m in models)
async def _build_prompt_context(
self,
skill_name: str,
model_path: str,
metadata: Dict[str, Any],
registry: SkillRegistry,
llm: Any,
) -> Dict[str, Any]:
"""Gather variables for the skill's prompt template.
Reads metadata, fetches the HF README (if applicable), lists available
base models, loads user priority tags, and returns a dict that maps to
``{{variable}}`` placeholders in ``prompt.md``.
"""
from ...metadata_ops import identify_model_type, list_base_models
from ..settings_manager import SettingsManager
context: Dict[str, Any] = {
"model_path": model_path,
"model_basename": "",
"hf_url": "",
"repo": "",
"readme_content": "",
"readme_content_full": "",
"current_metadata": {},
"base_models": [],
"priority_tags": "",
}
# Extract model basename (filename without extension) for the LLM
# to use when locating the matching section in collection repos.
raw_basename = os.path.splitext(os.path.basename(model_path))[0]
context["model_basename"] = raw_basename or ""
context["current_metadata"] = {
"file_name": metadata.get("file_name", ""),
"base_model": metadata.get("base_model", ""),
"tags": metadata.get("tags", []),
"modelDescription": metadata.get("modelDescription", ""),
"trainedWords": metadata.get("trainedWords", []),
"sha256": (metadata.get("sha256") or "")[:16] + "..." if metadata.get("sha256") else "",
"size": metadata.get("size", 0),
}
hf_url = metadata.get("hf_url", "")
context["hf_url"] = hf_url
repo = self._extract_repo_from_url(hf_url) if hf_url else ""
context["repo"] = repo or ""
if repo:
readme = await self._fetch_readme(repo)
# Trim README to the section relevant to this model file
# (collection repos often have multiple models in one README).
if readme and raw_basename:
trimmed = extract_relevant_section(readme, raw_basename)
cleaned = clean_readme_for_llm(trimmed) if trimmed else ""
else:
cleaned = clean_readme_for_llm(readme) if readme else ""
context["readme_content"] = cleaned if cleaned else "(README not available)"
context["readme_content_full"] = readme or ""
try:
raw_models = await list_base_models()
context["base_models"] = self._format_base_models(raw_models)
except Exception as exc:
logger.debug("Failed to list base models: %s", exc)
context["base_models"] = "</not available>"
# Determine model type and load the corresponding priority_tags
try:
model_type = await identify_model_type(model_path)
context["model_type"] = model_type
settings = SettingsManager()
priority_config = settings.get_priority_tag_config()
context["priority_tags"] = priority_config.get(model_type, "")
except Exception as exc:
logger.debug("Failed to load priority tags: %s", exc)
context["model_type"] = "lora"
context["priority_tags"] = ""
return context
@staticmethod
def _extract_repo_from_url(hf_url: str) -> Optional[str]:
"""Extract ``user/repo`` from a HuggingFace URL."""
if not hf_url:
return None
m = re.match(r"https?://huggingface\.co/([^/]+/[^/]+)", hf_url)
return m.group(1) if m else None
@staticmethod
async def _fetch_readme(repo: str) -> str:
"""Fetch README.md from HuggingFace (tries ``main``, then ``master``)."""
async with aiohttp.ClientSession(
headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
timeout=aiohttp.ClientTimeout(total=30),
) as session:
for branch in ("main", "master"):
url = f"https://huggingface.co/{repo}/raw/{branch}/README.md"
try:
async with session.get(url) as resp:
if resp.status == 200:
return await resp.text()
except Exception as exc:
logger.debug("Failed to fetch README from %s: %s", url, exc)
return ""
async def _emit_progress(
self,
callback: Optional[AgentProgressReporter],
skill_name: str,
*,
status: str,
**extra: Any,
) -> None:
"""Send a progress update via WebSocket (if callback is set)."""
payload: Dict[str, Any] = {"type": "agent_progress", "skill": skill_name, "status": status}
payload.update(extra)
if callback is not None:
await callback.on_progress(payload)
+336
View File
@@ -0,0 +1,336 @@
"""Post-processing engine for skill pipeline outputs.
The :class:`PostProcessor` takes the LLM's structured JSON output and applies
it to a model's on-disk metadata via the :mod:`~py.metadata_ops` functions.
It handles all the skill-specific business logic conditions, transformations,
and orchestration of multiple side-effects (write metadata, download preview,
refresh cache). All actual I/O is delegated to :mod:`~py.metadata_ops`.
"""
from __future__ import annotations
import json
import logging
import os
import re
from datetime import datetime, timezone
from typing import Any, Dict, List, Optional
logger = logging.getLogger(__name__)
class PostProcessor:
"""Deterministic post-processor for skill pipeline outputs.
Usage (called by :class:`~py.services.agent.agent_service.AgentService`)::
processor = PostProcessor()
result = await processor.process(
skill_name="enrich_hf_metadata",
model_path="/path/to/model.safetensors",
llm_output={...},
metadata={...}, # from metadata_ops.read_metadata()
)
"""
async def process(
self,
*,
skill_name: str,
model_path: str,
llm_output: Dict[str, Any],
metadata: Dict[str, Any],
readme_content: str = "",
) -> Dict[str, Any]:
"""Route *llm_output* to the correct skill post-processor.
*readme_content* is optional raw markdown content (e.g. HF README)
that is converted to HTML and stored as ``modelDescription`` for
the description tab.
Returns a dict with keys ``success`` (bool), ``updated_fields`` (list),
``preview_downloaded`` (bool), and ``errors`` (list).
"""
if skill_name == "enrich_hf_metadata":
return await self._process_enrich_hf_metadata(
model_path, llm_output, metadata, readme_content,
)
return {
"success": False,
"updated_fields": [],
"errors": [f"No post-processor registered for skill: {skill_name}"],
}
# ------------------------------------------------------------------
# enrich_hf_metadata
# ------------------------------------------------------------------
async def _process_enrich_hf_metadata(
self,
model_path: str,
llm_output: Dict[str, Any],
metadata: Dict[str, Any],
readme_content: str = "",
) -> Dict[str, Any]:
from ...metadata_ops import (
apply_metadata_updates,
download_preview,
refresh_cache,
)
from .skills.enrich_hf_metadata.readme_processor import (
convert_readme_to_html,
extract_gallery_images,
extract_gallery_table_images,
extract_relevant_section,
extract_simple_markdown_images,
extract_html_img_tags,
extract_repo_from_hf_url,
)
updated_fields: List[str] = []
preview_downloaded = False
# -- Determine whether this is an HF-sourced model -----------------
is_hf_model = not metadata.get("from_civitai", True)
# -- Collect updates -----------------------------------------------
updates: Dict[str, Any] = {}
# base_model
new_base = (llm_output.get("base_model") or "").strip()
current_base = metadata.get("base_model", "") or ""
if new_base and self._should_overwrite(current_base, is_hf_model):
updates["base_model"] = new_base
# trigger words → civitai.trainedWords
new_triggers = llm_output.get("trigger_words", [])
trigger_words_empty = True
if isinstance(new_triggers, list):
cleaned = [t.strip() for t in new_triggers if t.strip()]
cleaned = [t for t in cleaned if t.lower() not in ("none", "null", "n/a")]
trigger_words_empty = not cleaned
current_civitai = metadata.get("civitai") or {}
current_triggers = current_civitai.get("trainedWords") or []
if self._should_overwrite_list(current_triggers, is_hf_model):
trig_civitai = dict(current_civitai)
if "civitai" in updates and isinstance(updates["civitai"], dict):
trig_civitai.update(updates["civitai"])
trig_civitai["trainedWords"] = cleaned
updates["civitai"] = trig_civitai
# modelDescription — from raw README content (converted to HTML)
if readme_content and is_hf_model:
converted = convert_readme_to_html(readme_content)
if converted:
updates["modelDescription"] = converted
# short_description → civitai.description (for "About this version")
short_desc = (llm_output.get("short_description") or "").strip()
if short_desc and is_hf_model:
current_civitai = metadata.get("civitai") or {}
desc_civitai = dict(current_civitai)
if "civitai" in updates and isinstance(updates["civitai"], dict):
desc_civitai.update(updates["civitai"])
desc_civitai["description"] = short_desc
updates["civitai"] = desc_civitai
# gallery images → civitai.images (from YAML frontmatter widget entries
# and Sample Gallery markdown tables in the README body)
gallery_images: List[Dict[str, Any]] = []
if readme_content and is_hf_model:
hf_url = metadata.get("hf_url", "") or ""
repo = extract_repo_from_hf_url(hf_url)
if repo:
rec_w = llm_output.get("recommended_width") or 0
rec_h = llm_output.get("recommended_height") or 0
# 1. Widget images (YAML frontmatter)
gallery = extract_gallery_images(
readme_content, repo,
default_width=rec_w, default_height=rec_h,
)
# 2. Sample Gallery table images (markdown body), deduplicated
existing_urls = {img["url"] for img in gallery if img.get("url")}
table_images = extract_gallery_table_images(
readme_content, repo,
existing_urls=existing_urls,
default_width=rec_w, default_height=rec_h,
)
existing_urls.update(img["url"] for img in table_images if img.get("url"))
# 3. Simple markdown images `![alt](url)` in the body
simple_images = extract_simple_markdown_images(
readme_content, repo,
existing_urls=existing_urls,
default_width=rec_w, default_height=rec_h,
)
existing_urls.update(img["url"] for img in simple_images if img.get("url"))
# 4. HTML `<img>` tags (used by many collection repos)
html_images = extract_html_img_tags(
readme_content, repo,
existing_urls=existing_urls,
default_width=rec_w, default_height=rec_h,
)
all_images = gallery + table_images + simple_images + html_images
if all_images:
gallery_images = all_images
current_civitai = metadata.get("civitai") or {}
gallery_civitai = dict(current_civitai)
if "civitai" in updates and isinstance(updates["civitai"], dict):
gallery_civitai.update(updates["civitai"])
gallery_civitai["images"] = all_images
updates["civitai"] = gallery_civitai
# tags
new_tags = llm_output.get("tags", [])
if isinstance(new_tags, list) and new_tags:
existing_tags = metadata.get("tags") or []
merged = self._merge_tags(existing_tags, new_tags)
if len(merged) > len(existing_tags) or is_hf_model:
updates["tags"] = merged
# metadata_source & llm_enriched_at (always set)
updates["metadata_source"] = "agent:enrich_hf_metadata"
updates["llm_enriched_at"] = datetime.now(timezone.utc).isoformat()
# Store LLM confidence in metadata so it's accessible for evaluation
raw_confidence = (llm_output.get("confidence") or "").strip()
if raw_confidence:
updates["_llm_confidence"] = raw_confidence
# Fallback: extract instance_prompt from YAML frontmatter when the LLM
# returned empty trigger words but the README has instance_prompt.
if trigger_words_empty:
instance_prompt = _extract_yaml_instance_prompt(readme_content)
if instance_prompt:
current_civitai = metadata.get("civitai") or {}
trig_civitai = dict(current_civitai)
if "civitai" in updates and isinstance(updates["civitai"], dict):
trig_civitai.update(updates["civitai"])
trig_civitai["trainedWords"] = [instance_prompt]
updates["civitai"] = trig_civitai
preview_remote_url = (llm_output.get("preview_url") or "").strip()
# Fallback: if the LLM couldn't find a preview image in the cleaned
# README, find the first gallery image from the *model-specific
# section* of the README (not the repo-wide first image, which
# belongs to a different model in collection repos).
if not preview_remote_url and readme_content and is_hf_model:
model_basename = os.path.splitext(os.path.basename(model_path))[0]
relevant_section = extract_relevant_section(
readme_content, model_basename,
)
if relevant_section and relevant_section != readme_content:
for img in gallery_images:
img_url = img.get("url", "")
if img_url and img_url in relevant_section:
preview_remote_url = img_url
break
# Last resort: use the first gallery image from the full README.
if not preview_remote_url and gallery_images:
preview_remote_url = gallery_images[0].get("url", "")
current_preview = metadata.get("preview_url") or ""
if preview_remote_url and not (current_preview and os.path.exists(current_preview)):
local_path = await download_preview(model_path, preview_remote_url)
if local_path:
preview_downloaded = True
updates["preview_url"] = local_path
# notes — plain-text summary of usage info from the LLM
new_notes = (llm_output.get("notes") or "").strip()
if new_notes:
updates["notes"] = new_notes
# usage_tips — JSON string (e.g. {"strength_min":0.85,"strength_max":1.4})
raw_tips = (llm_output.get("usage_tips") or "").strip()
if raw_tips and raw_tips != "{}":
try:
json.loads(raw_tips)
updates["usage_tips"] = raw_tips
except (json.JSONDecodeError, TypeError):
logger.warning(
"LLM returned invalid usage_tips JSON: %s", raw_tips[:200]
)
if updates:
updated_fields = await apply_metadata_updates(model_path, updates)
# -- Refresh scanner cache ------------------------------------------
if updated_fields or preview_downloaded:
await refresh_cache(model_path)
return {
"success": True,
"updated_fields": updated_fields,
"preview_downloaded": preview_downloaded,
"updates": updates,
"errors": [],
}
# ------------------------------------------------------------------
# Helpers
# ------------------------------------------------------------------
@staticmethod
def _should_overwrite(current_value: str, is_hf_model: bool) -> bool:
"""Return ``True`` when a scalar field should be overwritten."""
return is_hf_model or not current_value or current_value.lower() in (
"", "unknown",
)
@staticmethod
def _should_overwrite_list(current_list: List[str], is_hf_model: bool) -> bool:
"""Return ``True`` when a list field should be overwritten."""
return is_hf_model or not current_list
@staticmethod
def _merge_tags(existing: List[str], new: List[str]) -> List[str]:
"""Merge *new* tags into *existing*, all lowercased.
This matches the behaviour of :class:`TagUpdateService` which
normalises every tag to lowercase for case-insensitive dedup.
"""
merged: List[str] = []
seen: set[str] = set()
for tag in list(existing) + list(new):
t = tag.strip().lower()
if t and t not in seen:
merged.append(t)
seen.add(t)
return merged
# ------------------------------------------------------------------
# Module-level helpers
# ------------------------------------------------------------------
def _extract_yaml_instance_prompt(readme_content: str) -> str:
"""Extract ``instance_prompt`` from the YAML frontmatter of a HF README.
Returns the prompt text, or empty string if not found. Handles
``null`` / ``~`` YAML null values by returning empty string.
"""
if not readme_content or not readme_content.startswith("---"):
return ""
# Find end of frontmatter
end = readme_content.find("---", 3)
if end == -1:
return ""
frontmatter = readme_content[3:end]
for line in frontmatter.split("\n"):
line = line.strip()
m = re.match(r"^instance_prompt:\s*(.*)", line)
if m:
val = m.group(1).strip().strip('"').strip("'")
if val.lower() in ("null", "~", "none", ""):
return ""
return val
return ""
+45
View File
@@ -0,0 +1,45 @@
"""Skill definition data structures.
Each skill is described by a :class:`SkillDefinition` that declares its
input/output schemas, whether it needs an LLM call, and what permissions
its post-processor has.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from typing import Any, Dict, List, Optional, Tuple
@dataclass(frozen=True)
class SkillPermissions:
"""Declarative permission scope for a skill's post-processor.
These are auditable constraints the :class:`AgentService` checks them
before invoking the handler. They are defense-in-depth, not a sandbox.
"""
write_metadata: bool = True
write_previews: bool = True
network_domains: Tuple[str, ...] = ()
@dataclass(frozen=True)
class SkillDefinition:
"""Immutable description of an agent skill."""
name: str
title: str
description: str
llm_required: bool
input_schema: Dict[str, Any] = field(default_factory=dict)
output_schema: Dict[str, Any] = field(default_factory=dict)
model_type_filter: Optional[List[str]] = None
permissions: SkillPermissions = field(default_factory=SkillPermissions)
def applies_to_model_type(self, model_type: str) -> bool:
"""Return ``True`` if this skill can run on the given model type."""
if self.model_type_filter is None:
return True
return model_type in self.model_type_filter
+210
View File
@@ -0,0 +1,210 @@
"""Discovery and loading of prompt-based skills.
Skills live in ``py/services/agent/skills/<name>/`` directories. Each
directory must contain a ``prompt.md`` file with YAML frontmatter::
---
name: my_skill
title: "My Skill"
description: "What this skill does"
llm_required: true
---
Prompt template with ``{{variable}}`` placeholders.
Legacy ``SKILL.md`` files are also supported for backward compatibility.
The registry scans the skills directory on first access and caches results.
"""
from __future__ import annotations
import asyncio
import logging
import re
from pathlib import Path
from typing import Any, Dict, List, Optional
import yaml
from .skill_definition import SkillDefinition, SkillPermissions
logger = logging.getLogger(__name__)
# Directory where built-in skills are stored
_SKILLS_DIR = Path(__file__).parent / "skills"
#: Preferred file names for prompt definition files (tried in order).
#: ``prompt.md`` is the current convention; ``SKILL.md`` is the legacy name
#: kept for backward compatibility.
_PROMPT_FILE_NAMES: tuple[str, ...] = ("prompt.md", "SKILL.md")
# ---------------------------------------------------------------------------
# Frontmatter parser
# ---------------------------------------------------------------------------
_FRONTMATTER_RE = re.compile(
r"^---\s*\n(.*?\n)---\s*\n?(.*)", re.DOTALL
)
def _parse_skill_file(path: Path) -> tuple[dict[str, Any], str]:
"""Read a prompt definition file (``prompt.md`` or legacy ``SKILL.md``) and
return (frontmatter_dict, body_text).
Raises ``ValueError`` if the file lacks valid YAML frontmatter.
"""
text = path.read_text(encoding="utf-8")
m = _FRONTMATTER_RE.match(text)
if not m:
raise ValueError(f"Missing or invalid YAML frontmatter in {path}")
frontmatter = yaml.safe_load(m.group(1))
if not isinstance(frontmatter, dict):
raise ValueError(f"Frontmatter in {path} is not a mapping")
body = m.group(2).strip()
return frontmatter, body
class SkillRegistry:
"""Discover and load agent skills from the filesystem."""
_instance: Optional["SkillRegistry"] = None
_lock: asyncio.Lock = asyncio.Lock()
def __init__(self, skills_dir: Path = _SKILLS_DIR) -> None:
self._skills_dir = skills_dir
self._skills: Dict[str, SkillDefinition] = {}
self._loaded: bool = False
# ------------------------------------------------------------------
# Singleton access
# ------------------------------------------------------------------
@classmethod
async def get_instance(cls) -> "SkillRegistry":
"""Return the lazily-initialised global ``SkillRegistry``."""
if cls._instance is None:
async with cls._lock:
if cls._instance is None:
registry = cls()
registry._discover()
cls._instance = registry
return cls._instance
@classmethod
def reset_instance(cls) -> None:
"""Reset the cached singleton — primarily for tests."""
cls._instance = None
# ------------------------------------------------------------------
# Discovery
# ------------------------------------------------------------------
@staticmethod
def _find_prompt_file(skill_dir: Path) -> Path | None:
"""Return the first prompt definition file that exists in *skill_dir*.
Tries ``_PROMPT_FILE_NAMES`` in order so that new conventions
(``prompt.md``) take precedence while legacy ``SKILL.md`` files
still load without changes.
"""
for name in _PROMPT_FILE_NAMES:
candidate = skill_dir / name
if candidate.exists():
return candidate
return None
def _discover(self) -> None:
"""Scan the skills directory and load all valid skill definitions."""
self._skills.clear()
if not self._skills_dir.is_dir():
logger.warning("Skills directory does not exist: %s", self._skills_dir)
self._loaded = True
return
for entry in sorted(self._skills_dir.iterdir()):
if not entry.is_dir():
continue
prompt_file = self._find_prompt_file(entry)
if prompt_file is None:
continue
try:
definition = self._load_skill_definition(prompt_file)
if definition is not None:
self._skills[definition.name] = definition
logger.debug("Loaded skill: %s", definition.name)
except Exception as exc:
logger.warning("Failed to load skill from %s: %s", prompt_file, exc)
self._loaded = True
logger.info("Discovered %d prompt-based skills", len(self._skills))
def _load_skill_definition(self, path: Path) -> Optional[SkillDefinition]:
"""Parse a prompt definition file's frontmatter into a
:class:`SkillDefinition`."""
try:
data, _body = _parse_skill_file(path)
except (ValueError, yaml.YAMLError) as exc:
logger.warning("Failed to parse prompt file %s: %s", path, exc)
return None
if "name" not in data:
logger.warning("Prompt file %s missing required 'name' field", path)
return None
perm_data = data.get("permissions", {})
permissions = SkillPermissions(
write_metadata=perm_data.get("write_metadata", True),
write_previews=perm_data.get("write_previews", True),
network_domains=tuple(perm_data.get("network_domains", [])),
)
return SkillDefinition(
name=data["name"],
title=data.get("title", data["name"]),
description=data.get("description", ""),
llm_required=data.get("llm_required", False),
input_schema=data.get("input_schema", {}),
output_schema=data.get("output_schema", {}),
model_type_filter=data.get("model_type_filter"),
permissions=permissions,
)
# ------------------------------------------------------------------
# Public API
# ------------------------------------------------------------------
def list_skills(self) -> List[SkillDefinition]:
"""Return all discovered skill definitions."""
if not self._loaded:
self._discover()
return list(self._skills.values())
def get_skill(self, name: str) -> Optional[SkillDefinition]:
"""Return the skill definition for ``name``, or ``None`` if not found."""
if not self._loaded:
self._discover()
return self._skills.get(name)
def load_prompt(self, name: str) -> str:
"""Load and return the prompt template body for the named skill."""
skill_dir = self._skills_dir / name
skill_path = self._find_prompt_file(skill_dir)
if skill_path is None:
raise FileNotFoundError(
f"Prompt file not found for skill '{name}' in {skill_dir} "
f"(tried {list(_PROMPT_FILE_NAMES)})"
)
try:
_frontmatter, body = _parse_skill_file(skill_path)
return body
except (ValueError, yaml.YAMLError) as exc:
raise ValueError(f"Failed to parse prompt from {skill_path}: {exc}") from exc
@@ -0,0 +1,165 @@
---
name: enrich_hf_metadata
title: "Enrich Metadata from HuggingFace"
description: >
Parse the HuggingFace model card via LLM to extract description, trigger
words, base model, tags, and preview image URL.
llm_required: true
---
You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a HuggingFace model card (README.md).
## Model Information
- **Repository**: {{hf_url}}
- **Model file path**: {{model_path}}
- **Model filename**: {{model_basename}}
- **Repository ID**: {{repo}}
## Current Metadata (may be incomplete)
```json
{{current_metadata}}
```
## User Priority Tags Reference
The user has configured the following list of **meaningful tag categories** for this model type (`{{model_type}}`):
```
{{priority_tags}}
```
These are the subjects, styles, and concepts the user considers useful for categorization. Use this list as a **reference** when evaluating tags (see the **tags** section below).
## Available Base Models
The following base models are currently valid in this system. Use the EXACT
name listed — do not invent aliases or modify variant suffixes.
{{base_models}}
## HuggingFace README Content
```
{{readme_content}}
```
## Extraction Instructions
Extract the following information from the README content above:
### base_model
The base model this model was trained on. Use EXACTLY one of the names from the **Available Base Models** list above. Do not invent new names or use aliases.
Check the YAML frontmatter for ``base_model:`` first. If the frontmatter has no ``base_model:``, look at the **model filename** (``{{model_basename}}``), YAML ``tags:``, README title and first paragraph for clues — the base model family is often embedded in the name
### trigger_words
The trigger words or activation prompts needed to use this LoRA. Look for:
- `instance_prompt:` in the YAML frontmatter
- Phrases like "trigger word:", "trigger:", "use this prompt:", "activation prompt:"
- In collection repos: the trigger section **specific to this model file** (look near matching download links or anchor IDs)
- Example prompts at the start (usually the first word or phrase before any description)
Return as an array of strings. If none found, return an empty array `[]`. **Never** return `["None"]` or any placeholder value — a truly empty list means no trigger words exist.
### short_description
A concise 1-2 sentence summary of what this model does. Extract from the "Model description" section or the first paragraph. For collection repos, focus on the **specific model version** matching `{{model_basename}}`, not the repo as a whole. Return empty string if the README is too minimal.
### tags
3-8 relevant tags for categorizing this model. **Quality over quantity.**
Sources to consider:
- The YAML frontmatter `tags:` list (filter out technical ones — see below)
- The subject, style, character, or concept the model represents
- The model filename itself may give clues (e.g. "pokemon", "anime", "pixelart")
**Critical filtering rules — apply them strictly:**
1. **Exclude technical/generic tags.** Reject any tag that describes the model's **training methodology, framework, architecture, or modality** rather than its content. Examples to exclude: `text-to-image`, `diffusers`, `lora`, `dreambooth`, `diffusers-training`, `flux`, `sdxl`, `checkpoint`, `pytorch`, `safetensors`, `fine-tuning`, `stable-diffusion`, and any variant of these.
2. **Cross-reference against the priority_tags reference.** Only include a tag if it meaningfully describes what the model actually creates (subject, style, character type) and is semantically close to one of the priority_tags. If none of the README's tags match meaningful categories, prefer returning a smaller set or an empty array over including low-value tags.
3. **All lowercase, no spaces, no hyphens** (use single words like `"photorealistic"`, `"anime"`, `"character"`).
Return empty array if no meaningful content tags remain after filtering.
### recommended_width, recommended_height
The recommended image generation resolution for this model, in pixels. Look for sections like "Best Dimensions", "Recommended size", "Suggested resolution", or similar phrasing in the README. Prefer the explicitly marked "Best" or default resolution. If the table/list has multiple entries (e.g. "768 x 1024 (Best)" and "1024 x 1024 (Default)"), use the one marked "Best". Return integers. If no resolution can be determined, return 0 for both.
### preview_url
The URL of the most suitable preview image from the README. Look for:
- Image tags near the section matching the model filename (`{{model_basename}}`)
- The YAML frontmatter `widget:` section (which often has `output.url` fields)
- In collection repos: the sample images listed **under the section** for this specific model version
- Generic `![alt](url)` in the body
Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL as `https://huggingface.co/{{repo}}/resolve/main/{filename}`. If no suitable image is found, return an empty string.
### notes
A plain-text summary of the model card's key practical usage information. Combine trigger words, style modifiers, recommended parameters (steps, CFG, resolution, sampler), and any setup tips into a readable paragraph. For collection repos, focus on the **specific model version** matching `{{model_basename}}`. Return empty string if the README has no useful usage info.
### usage_tips
A JSON string with structured usage recommendations. Extract from the README any explicit ranges or recommended values (e.g. "Set LoRA strength: **0.85 - 1.4**", "CLIP strength: 0.5"). Possible fields (include only those you can determine):
```json
{
"strength_min": 0.85,
"strength_max": 1.4,
"strength_range": "0.85-1.4",
"strength": 0.6,
"clip_strength": 0.5,
"clip_skip": 2
}
```
Return the JSON string (e.g. `'{"strength_min":0.85,"strength_max":1.4}'`). Return `"{}"` if nothing useful is found.
### confidence
Your confidence level in the extracted data:
- "high" — most fields were explicitly stated in the README
- "medium" — some fields were inferred from context
- "low" — most fields are guesses based on limited information
## Important: Handling Collection Repos (multiple model files)
Many HuggingFace repos contain **multiple model files** in a single repository
(e.g. a "LoRA collection" with different styles/characters in separate files).
The model file currently being enriched is: **`{{model_basename}}`**
To find the correct section in the README:
1. **Search for download links** containing the filename — the surrounding paragraph is your section.
2. **Search for anchor IDs** (`<a id="...">`) or section headings whose text matches words from the filename.
3. **Search for HTML headings** (`<h1>`, `<h2>`, `<span>`) containing parts of the filename.
4. If no match is found, use the full README as usual — the model may be the only one in the repo.
When a matching section IS found, prefer metadata from that section.
When no section matches (e.g. single-model repos or repos without per-file sections),
extract metadata from the full README normally. Do not return empty data just
because the filename doesn't appear in the README.
## Output Format
Return ONLY a JSON object with exactly these fields (no markdown fences, no extra text):
```json
{
"model_path": "{{model_path}}",
"base_model": "<canonical name or empty string>",
"trigger_words": ["<word1>", "<word2>"],
"short_description": "<1-2 sentence summary>",
"tags": ["<tag1>", "<tag2>"],
"recommended_width": 768,
"recommended_height": 1024,
"preview_url": "<image URL or empty string>",
"notes": "<plain-text usage summary or empty string>",
"usage_tips": "<JSON string like '{\"strength_min\":0.85,\"strength_max\":1.4}' or '{}'>",
"confidence": "<high|medium|low>"
}
```
Important:
- Only include the JSON object, no other text
- If a field cannot be determined, use an empty string or empty array
- Do not fabricate information not supported by the README
- Never use placeholder values like `"None"` or `"unknown"` for missing data — use empty string or empty array
File diff suppressed because it is too large Load Diff
+228 -16
View File
@@ -1,3 +1,7 @@
# pyright: reportImportCycles=false
# Lazy (function-local) imports still count as static edges in basedpyright's
# reportImportCycles, so the ServiceRegistry singleton pattern necessarily forms
# import cycles. Breaking them would require an architectural refactor.
from __future__ import annotations
import asyncio
@@ -7,6 +11,7 @@ import os
import secrets
import shutil
import socket
import time
from dataclasses import dataclass
from datetime import datetime
from pathlib import Path
@@ -20,10 +25,43 @@ from .settings_manager import get_settings_manager
logger = logging.getLogger(__name__)
# Maximum times the download poll loop will re-schedule a transfer after it
# is lost (daemon restart / RPC outage) before failing the download.
MAX_TRANSFER_RECOVERY_ATTEMPTS = 2
# stderr lines matching these markers indicate a disk write failure inside
# aria2 (piece cache flush or raw file write). They are promoted to INFO so
# the root cause (disk full, permission denied, file locked by another
# process, ...) is visible in the default logs; all other stderr output stays
# at DEBUG to avoid noise.
_DISK_WRITE_ERROR_MARKERS = (
# aria2 wrapper messages (write disk cache flush path)
"write disk cache flush failure",
"error when trying to flush write cache",
"failed to write into the file",
"failed to open the file",
"failed to seek the file",
# underlying root-cause phrases reported via "cause: ..." (POSIX + Windows)
"no space left on device",
"not enough space on the disk",
"input/output error",
"permission denied",
"access is denied",
"disk quota exceeded",
"used by another process",
"sharing violation",
)
# Minimum interval between INFO-level reports of the same stderr line so a
# repeated failure (e.g. aria2 retrying against a full disk) does not spam
# the log.
STDERR_ERROR_REPORT_INTERVAL = 60.0
def _try_certifi_ca_path() -> str | None:
"""Return the certifi CA bundle path if available, else None."""
try:
import certifi # type: ignore[import-untyped]
import certifi # pyright: ignore[reportMissingTypeStubs]
path = certifi.where()
if os.path.isfile(path):
@@ -44,6 +82,17 @@ CIVITAI_DOWNLOAD_URL_PREFIXES = (
)
def _is_no_uri_available_error(message: str) -> bool:
"""Return True for aria2's "No URI available" transfer failure.
aria2 reports this when every URI for the transfer has become unusable.
For CivitAI downloads this typically means the temporary signed URL
expired mid-download; the transfer can be recovered by resolving a fresh
signed URL and re-scheduling with ``continue=true``.
"""
return "no uri available" in message.lower()
class Aria2Error(RuntimeError):
"""Raised when aria2 integration fails."""
@@ -81,10 +130,12 @@ class Aria2Downloader:
self._rpc_session: Optional[aiohttp.ClientSession] = None
self._rpc_session_lock = asyncio.Lock()
self._process_lock = asyncio.Lock()
self._register_lock = asyncio.Lock()
self._transfers: Dict[str, Aria2Transfer] = {}
self._poll_interval = 0.5
self._state_store = Aria2TransferStateStore()
self._stderr_reader_task: Optional[asyncio.Task] = None
self._stderr_reader_task: Optional[asyncio.Task[Any]] = None
self._stderr_error_report: Dict[str, float] = {}
@property
def is_running(self) -> bool:
@@ -99,26 +150,61 @@ class Aria2Downloader:
progress_callback=None,
headers: Optional[Dict[str, str]] = None,
) -> Tuple[bool, str]:
"""Download a file using aria2 RPC and wait for completion."""
"""Download a file using aria2 RPC and wait for completion.
The poll loop is self-healing: when the in-memory transfer entry
disappears (e.g. another download restarted the daemon and
``close()`` cleared ``_transfers``) or the RPC becomes unreachable,
the transfer is re-scheduled with ``continue=true`` so the download
resumes from the on-disk ``.aria2`` control file. The same
re-scheduling happens when aria2 fails with "No URI available"
(typically an expired CivitAI signed URL): a fresh URL is resolved
and the partial download continues. Recovery is bounded by
``MAX_TRANSFER_RECOVERY_ATTEMPTS``.
"""
await self._ensure_process()
save_path = os.path.abspath(save_path)
async with self._register_lock:
transfer = self._transfers.get(download_id)
if transfer is None or os.path.abspath(transfer.save_path) != save_path:
gid = await self._schedule_download(
transfer = await self._register_transfer(
url,
save_path,
download_id=download_id,
headers=headers,
)
transfer = Aria2Transfer(gid=gid, save_path=save_path)
self._transfers[download_id] = transfer
recovery_attempts = 0
try:
while True:
try:
status = await self._get_status_with_retry(download_id)
except Aria2Error:
status = None
if status is None:
if recovery_attempts >= MAX_TRANSFER_RECOVERY_ATTEMPTS:
return False, "aria2 download not found"
recovery_attempts += 1
logger.warning(
"aria2 transfer %s lost; re-scheduling with resume "
"(attempt %d/%d)",
download_id,
recovery_attempts,
MAX_TRANSFER_RECOVERY_ATTEMPTS,
)
await asyncio.sleep(1.0)
await self._ensure_process()
async with self._register_lock:
transfer = await self._register_transfer(
url,
save_path,
download_id=download_id,
headers=headers,
)
continue
snapshot = self._build_progress_snapshot(status)
if progress_callback is not None:
@@ -129,12 +215,43 @@ class Aria2Downloader:
completed_path = self._resolve_completed_path(status, save_path)
return True, completed_path
if state == "error":
return False, status.get("errorMessage") or "aria2 download failed"
error_message = status.get("errorMessage") or "aria2 download failed"
if (
_is_no_uri_available_error(error_message)
and recovery_attempts < MAX_TRANSFER_RECOVERY_ATTEMPTS
):
# The signed URL (e.g. CivitAI's) expired before the
# transfer finished. Re-registering resolves a fresh
# URL and resumes from the on-disk partial payload and
# .aria2 control file via ``continue=true``.
recovery_attempts += 1
logger.warning(
"aria2 transfer %s failed with %r; refreshing the "
"URL and resuming the partial download "
"(attempt %d/%d)",
download_id,
error_message,
recovery_attempts,
MAX_TRANSFER_RECOVERY_ATTEMPTS,
)
await asyncio.sleep(1.0)
await self._ensure_process()
async with self._register_lock:
transfer = await self._register_transfer(
url,
save_path,
download_id=download_id,
headers=headers,
)
continue
return False, error_message
if state == "removed":
return False, "Download was cancelled"
await asyncio.sleep(self._poll_interval)
finally:
current = self._transfers.get(download_id)
if current is not None and current.gid == transfer.gid:
self._transfers.pop(download_id, None)
async def _get_status_with_retry(
@@ -143,8 +260,9 @@ class Aria2Downloader:
"""Call get_status with retry for transient RPC failures.
Only retries on :exc:`Aria2Error` (RPC-level failure). Returns
``None`` immediately when the download_id is not tracked (a missing
transfer is not a transient condition, so retrying is pointless).
``None`` immediately when the transfer is not tracked or its GID is
gone from the daemon (a missing transfer is not a transient
condition, so retrying is pointless).
A single failed RPC call should not immediately fail the download,
because aria2 may be temporarily busy (e.g. finalizing multiple
@@ -190,7 +308,7 @@ class Aria2Downloader:
download_id,
)
options: Dict[str, str] = {
options: Dict[str, Any] = {
"dir": save_dir,
"out": out_name,
"continue": "true",
@@ -201,6 +319,13 @@ class Aria2Downloader:
"auto-file-renaming": "false",
"file-allocation": "none",
}
# Pass proxy to aria2 so the actual file transfer goes through the
# same proxy used by the aiohttp-based URL resolution step above.
downloader = await get_downloader()
if downloader.proxy_url:
options["all-proxy"] = downloader.proxy_url
if request_headers:
options["header"] = [
f"{key}: {value}" for key, value in request_headers.items()
@@ -231,8 +356,33 @@ class Aria2Downloader:
)
return gid
async def _register_transfer(
self,
url: str,
save_path: str,
*,
download_id: str,
headers: Optional[Dict[str, str]] = None,
) -> Aria2Transfer:
"""Schedule a download and track it in the in-memory transfer registry."""
gid = await self._schedule_download(
url,
save_path,
download_id=download_id,
headers=headers,
)
transfer = Aria2Transfer(gid=gid, save_path=os.path.abspath(save_path))
self._transfers[download_id] = transfer
return transfer
async def get_status(self, download_id: str) -> Optional[Dict[str, Any]]:
"""Return the raw aria2 status payload for a known download."""
"""Return the raw aria2 status payload for a known download.
Returns ``None`` when the download_id is not tracked or the daemon no
longer knows the transfer's GID (daemon restart / forceRemove). A
forgotten GID is permanent, not transient, so the caller's recovery
path handles it instead of burning retry attempts on a dead GID.
"""
transfer = self._transfers.get(download_id)
if transfer is None:
@@ -248,8 +398,17 @@ class Aria2Downloader:
"files",
]
try:
status = await self._rpc_call("aria2.tellStatus", [transfer.gid, keys])
status = await self._rpc_call(
"aria2.tellStatus", [transfer.gid, keys], log_errors=False
)
except Exception as exc:
if "not found" in str(exc).lower():
logger.debug(
"aria2 GID %s for download %s is gone; treating as lost transfer",
transfer.gid,
download_id,
)
return None
raise Aria2Error(f"Failed to query aria2 download status: {exc}") from exc
if isinstance(status, dict):
@@ -267,7 +426,9 @@ class Aria2Downloader:
"files",
]
try:
status = await self._rpc_call("aria2.tellStatus", [gid, keys])
status = await self._rpc_call(
"aria2.tellStatus", [gid, keys], log_errors=False
)
except Exception as exc:
message = str(exc)
if "cannot be found" in message.lower() or "not found" in message.lower():
@@ -334,8 +495,19 @@ class Aria2Downloader:
try:
await self._rpc_call("aria2.forceRemove", [transfer.gid])
except Exception as exc:
if "not found" not in str(exc).lower():
return {"success": False, "error": str(exc)}
# The daemon already forgot this GID (restart / prior removal),
# so the transfer is effectively cancelled.
logger.debug(
"aria2 GID %s for download %s already gone during cancel",
transfer.gid,
download_id,
)
# Drop the in-memory entry as well so a concurrent poll loop does
# not mistake the removal for a lost transfer and re-register it.
self._transfers.pop(download_id, None)
await self._state_store.remove(download_id)
return {"success": True, "message": "Download cancelled successfully"}
@@ -378,16 +550,51 @@ class Aria2Downloader:
blocks, which freezes the entire ``aria2c`` process including its
RPC handler. This background task reads lines from stderr as they
arrive and forwards them to Python's logger.
Lines that indicate a disk write failure (e.g. the "cause: No space
left on device" line that follows "Write disk cache flush failure")
are promoted to INFO so the root cause is visible without enabling
debug logging; every other line stays at DEBUG to avoid noise.
"""
try:
assert self._process is not None and self._process.stderr is not None
async for line in self._process.stderr:
text = line.decode("utf-8", errors="replace").rstrip()
if text:
if self._is_disk_write_error(text):
self._report_stderr_error(text)
else:
logger.debug("aria2 stderr: %s", text)
except Exception:
pass
@staticmethod
def _is_disk_write_error(text: str) -> bool:
lowered = text.lower()
return any(marker in lowered for marker in _DISK_WRITE_ERROR_MARKERS)
def _report_stderr_error(self, text: str) -> None:
"""INFO-log a disk write failure line, rate-limited per line text.
aria2 re-emits the same error chain on every poll/retry while the
underlying condition persists; only the first occurrence within
``STDERR_ERROR_REPORT_INTERVAL`` seconds is promoted to INFO.
"""
now = time.monotonic()
last = self._stderr_error_report.get(text)
if last is not None and now - last < STDERR_ERROR_REPORT_INTERVAL:
logger.debug("aria2 stderr (repeated disk write error): %s", text)
return
# Drop entries older than the window so the map stays bounded even
# during a long disk-full episode (piece indexes change per line).
self._stderr_error_report = {
line: timestamp
for line, timestamp in self._stderr_error_report.items()
if now - timestamp < STDERR_ERROR_REPORT_INTERVAL
}
self._stderr_error_report[text] = now
logger.info("aria2 disk write failure: %s", text)
async def _dispatch_progress(self, callback, snapshot: DownloadProgress) -> None:
try:
result = callback(snapshot, snapshot)
@@ -590,7 +797,9 @@ class Aria2Downloader:
return isinstance(result, dict)
async def _rpc_call(self, method: str, params: list[Any]) -> Any:
async def _rpc_call(
self, method: str, params: list[Any], *, log_errors: bool = True
) -> Any:
if not self._rpc_url:
raise Aria2Error("aria2 RPC endpoint is not initialized")
@@ -621,7 +830,10 @@ class Aria2Downloader:
error = body["error"] or {}
code = error.get("code") if isinstance(error, dict) else None
message = error.get("message") if isinstance(error, dict) else str(error)
logger.error(
# Probing calls (e.g. tellStatus for a GID the daemon may have
# forgotten) pass log_errors=False: an expected "not found" must
# not spam the log at ERROR level.
(logger.error if log_errors else logger.debug)(
"aria2 RPC %s failed with HTTP %s, code=%s, message=%s",
method,
response.status,
@@ -636,7 +848,7 @@ class Aria2Downloader:
raise Aria2Error(status_message or "Unknown aria2 RPC error")
if response.status != 200:
logger.error(
(logger.error if log_errors else logger.debug)(
"aria2 RPC %s returned unexpected HTTP status %s without error payload: %s",
method,
response.status,

Some files were not shown because too many files have changed in this diff Show More