Add a reusable directory-picker modal backed by a new generic
POST /api/lm/browse-directory endpoint (browse logic extracted from the
recipe batch-import handler into py/utils/directory_browser.py) and wire
a browse button plus advisory validate-path feedback (POST
/api/lm/validate-path) into the settings path inputs: recipes path,
example images path/local root, and the extra-folder/model-path rows.
The browse button insets into the right edge of static inputs so narrow
settings rows keep their single-control layout.
Translations for the new settings.directoryPicker and
settings.pathValidation keys are filled in for all 9 locales.
Standalone users previously had to hand-edit settings.json to configure
primary folder_paths. Add a standalone-only Model Paths section to the
settings modal:
- Backend exposes standalone_mode, folder_paths (with template placeholder
values filtered out) and a data-driven folder_path_schema derived from
OTHER_MODEL_FOLDER_SUBTYPES via GET /api/lm/settings
- The new section renders multi-path editors per model type from the
schema, with inline enable_other_models / sub-type controls so other
model types are configured without leaving the tab
- Persistent restart-required cues after a save: nav dot, inline notice
and a global banner (unique id per change so dismissals don't mute
future reminders)
- The missing-model-paths startup banner and the Other Models no-paths
empty state now deep-link into the new section instead of pointing at
settings.json
The prune guard intentionally leaves the in-memory view empty while the
stored cache keeps the user's recipes, so there has to be a documented
way to accept the on-disk truth. That route is an explicit rebuild, which
clears the stored cache before a full directory scan. Cover it so the
FAQ recovery steps stay true.
Setting LORA_MANAGER_PORTABLE=1 once wrote use_portable_settings: true
into the plugin's own settings.json, and every later run of every
instance sharing that plugin folder then read and wrote the portable
settings directory. There was no way back except editing the file by
hand, which is exactly the trap a user hit while following the FAQ's
instructions for isolating a second instance (#1114).
LORA_MANAGER_PORTABLE=0 is now the explicit exit:
- _should_use_portable_settings honours "0" as a forced off, so the
resolved settings directory no longer depends on the persisted flag.
- SettingsManager clears the persisted flag in that case, so later runs
without the variable stay on the shared settings directory.
Unset or unrecognised values keep the previous behaviour: the persisted
flag decides, so existing portable installs are unaffected.
LORA_MANAGER_SETTINGS_DIR still takes precedence over both.
Installing a second LoRA Manager instance (standalone or a second
ComfyUI install) that shares the settings directory puts two processes
on the same cache databases. Three things made that unsafe.
- The updater preserved cache/ and model_cache/ but not a legacy
recipe_cache/ directory, so a portable install predating the cache/
move lost its recipe database on a git-based update. Add it to
_PRESERVE_DIRS and to .gitignore.
- Cache connections used the sqlite3 default 5s timeout, which a
scanning instance can exceed, turning a concurrent write into
"database is locked". Route every shared cache connection through
connect_cache_db(), which raises the timeout to 30s and sets
busy_timeout + synchronous=NORMAL to match the existing WAL mode.
App-private databases (download queue, update history) are unchanged.
- A full-table cache replace is a read-modify-write that SQLite cannot
make atomic across processes, so two instances could interleave and
one snapshot could overwrite the other. Guard the recipe and model
save_cache paths with a cross-process advisory lock (flock on POSIX,
msvcrt on Windows). Locking is best-effort: if it is unavailable the
call proceeds and the SQLite busy timeout is the fallback.
The lock file is a hidden sibling of the database and is deliberately
never unlinked, so a second process cannot lock a fresh inode.
A scan that finds no recipe files at all is not a reliable deletion
signal: an unmounted drive, a recipes_path that silently falls back to
another LoRA root, or a cache shared with a second instance all look
exactly like a real wipe. The reconcile step treated them all as
deletions and overwrote the persistent cache with an empty one, so
DELETE FROM recipes destroyed the user's only record of their recipes
and the FTS index was rebuilt from the empty view (#1116).
Guard the prune:
- _reconcile_recipe_cache reports an all-missing result when every
persisted recipe file is gone AND the stored rows match the recorded
file stats. An internally inconsistent cache (leftover orphans) is
stale, not evidence of a fresh disappearance, and still prunes.
- The caller keeps the stored cache and logs a warning naming the
directory it scanned and the number of recipes it preserved, instead
of writing the empty result. It also skips the FTS rebuild so the
index stays aligned with the stored rows.
- save_cache gains skip_if_empty as a storage-level backstop: refuse to
empty a populated cache. Intentional clears (manual rebuild) keep the
default behaviour.
- Log the resolved scan directory per run so a support reader can tell a
real wipe apart from a scan that looked elsewhere.
Partial orphans (ordinary manual deletions) keep pruning as before.
Example videos added through the "Add examples" flow were stored with a
hardcoded 720x1280 entry. The dimension probe next to it only ran for
images (PIL cannot open .mp4/.webm files), so every video entry stayed
portrait regardless of the source. The showcase viewer then sizes its
container straight from that value (--media-aspect in showcase.css), so
landscape clips were letterboxed inside a 9:16 box. CivitAI-sourced
examples were unaffected because their dimensions come from the API.
PIL cannot read video containers, so add a dependency-free reader that
parses the container headers instead: moov/trak/tkhd for ISO base media
(with the sample description as a fallback), Segment/Tracks/Pixel* for
WebM/Matroska, and RIFF/WebP for animated examples saved with a video
extension. The sniffed signature decides which reader runs, so a .mp4
that is really WebM still reports the right size; the extension is only
a fallback. Both readers seek past mdat rather than reading it, so a
large file costs the same as a small one.
Imported entries now record the file's real size and keep the previous
placeholder only when the file cannot be parsed.
Existing libraries keep their wrong entries, so backfill them once via
the existing naming migration: bump CURRENT_NAMING_VERSION to 3 and
repair each model's empty-url entries from the files on disk, then sync
the scanner cache. Only entries with no remote url are touched -- those
have no other source, which makes the rewrite lossless -- and entries
already carrying the right size are left byte-identical, so the pass is
idempotent and a no-op for libraries that never imported a video.
Civitai types with no scanner at all (Wildcards, Workflows, Hypernetwork,
Poses, AestheticGradient) used to get a 400 'Model type "x" is not
supported', which hid the Civitai version list from clients.
The handler now answers 200 with supported:false, a machine-readable
reason (model_type_unsupported, or other_models_disabled when the opt-in
master switch is off) and the versions marked read-only. The interactive
payload gains an explicit supported:true. Legacy clients only read
success/versions, so they are unaffected.
The standalone empty state showed the folder_paths keys but not where to
put them, and its Open Settings button led to a modal that cannot edit
primary folder paths. Now the page shows the real settings.json path and
an Open Settings Folder button backed by the existing open-location API.
Also stop open_settings_location from claiming success on headless Linux
sessions: with no DISPLAY/WAYLAND_DISPLAY, xdg-open cannot work, so the
handler now returns clipboard mode and the browser copies/shows the path
instead.
start_server.py computed the project root three levels up from scripts/,
assuming it lived under .agents/skills/<skill>/scripts/. After moving to
scripts/e2e/ that resolved to the ComfyUI root, so the launcher failed
with "can't open file 'standalone.py'".
updateOtherModelsControls() synced the sub-type checkboxes and default-root
selects but never set the master toggle's checked state, and the
setting_toggle macro renders no checked attribute, so after a page refresh
the toggle always appeared off regardless of the saved setting.
The reorder shortcut cannot be made reliable in this UI. `Alt + Arrow` is
the browser's tab-history / back-forward gesture on several platforms,
and the modal already binds bare `ArrowLeft`/`ArrowRight` to model
navigation, so the binding either did nothing — a keypress with nothing
focused never reaches a listener on the tag list — or fought the browser.
An affordance that occasionally navigates the page away is worse than
having no keyboard path at all, so drop it.
Reordering is pointer-only again: drag the chip (tags) or its `⠿` grip
(trigger words, whose chip body is click-to-edit). Everything that existed
only to serve the shortcut goes with it — the keydown listener, the hover
tracking used to resolve the target chip, the aria-live announcements, the
per-grip position labels and `moveItemWithinContainer`. The grip becomes a
decorative, non-focusable `<span>` (`aria-hidden`, behind a 5px drag
threshold) instead of a `<button>`, so it no longer promises a keyboard
action it cannot perform.
The tooltip and hint drop the shortcut mention in all 10 locales
(`common.reorder.dragHandle` = "Drag to reorder" and the localised
equivalents); `common.reorder.ariaLabel` and `common.reorder.announcement`
are pruned from every locale by the sync script. The i18n guidelines
record the decision so no shortcut is re-added without re-adding the keys.
Two stacked defects cut the side edges off the URL textareas in the
download and batch-import modals.
`#modelUrl` and `#batchUrlInput` are `width: 100%` with padding and a
border but no `box-sizing: border-box`, so the border box was wider than
the containing block and its right edge landed in the region the modal
clips: the right border column is missing in both screenshots while the
corner pixels of the top/bottom borders are drawn, and the batch
textarea's resize handle sits a padding-width to the right of the mode
toggle above it.
The download modal's `#downloadModal .download-step` additionally
scrolls with `overflow-x: hidden` and has no horizontal padding, so the
global `:focus-visible { outline-offset: 2px }` lost both vertical edges
there and only the top and bottom lines survived. Draw that ring inset
inside `#downloadModal`, mirroring the existing `#importModal` fix in
import-modal.css.
`.input-group input, .input-group select` gets the same border-box
treatment, which also repairs the standing clipped right border on the
other full-width fields the shared rule styles (the import modal's URL,
recipe-name and tag inputs, the batch directory and tags inputs, the
model root select and the target folder path).
Verified: `npx vitest run` 130 files / 1259 tests passed.
Fill in the 4 `modals.download.progress.*` keys added by the previous
commit across all 9 locales, so no `[TODO: Translate]` placeholder remains
and the "no remaining placeholders" claim in the guidelines holds again.
No new terminology: `metadata` reuses the §5 row (fr métadonnées, de
Metadaten, es metadatos, ru метаданные, he מטא-נתונים, ja メタデータ,
ko 메타데이터, zh-CN 元数据, zh-TW 中繼資料) and the fetching phrasing
mirrors each locale's existing `download.fetchingRepoFiles` /
`fetchingVersions` (de passive "werden abgerufen", es "Obteniendo", fr
"Récupération des", ru "Получение", he "מביא", ja "取得中", ko "가져오는
중"). "model file" follows `errors.noModelFiles` in each file.
`{name}` and `{source}` are verbatim §1-R2 placeholders. `{source}` is
replaced at runtime with the *untranslated* platform name, so its
surrounding spacing follows each locale's `modelCard.actions.viewOnSource`
precedent — ja `{source} から`, ko `{source}에서`, zh `从 {source}` /
`從 {source}`, he `מ-{source}` (as in the existing `מ-CivitAI`), ru
`из {source}` (as in `из Workflow`) — and no brand ever appears inside the
translated text.
Punctuation: ASCII `:` for the Latin / Cyrillic / Hebrew locales and for
ja / ko, whose four sibling keys in the same `progress` block already use
ASCII; French keeps this file's ` : `; zh-CN / zh-TW use full-width `：`
like their siblings.
The guidelines gain a status block recording the pass and those spacing
precedents, so a future source added to the same slot does not have to
re-derive them.
Verified: `pytest tests/i18n/test_i18n.py` 20 passed,
`sync_translation_keys.py --dry-run` reports no drift, `npm test` exits 0
(1259 JS + 91 Vue). Each locale file gains exactly 4 lines — the values
were substituted as literals rather than re-serialising the JSON, so no
formatting churn.
A ModelScope or Hugging Face download landed as a bare filename, hash and
source link; the model card stayed empty until the user ran "Enrich
Metadata with AI" by hand. But everything that makes a CivitAI download
useful — the display name, the description, the tags, the trigger words,
the example images, the preview — is already published by those sites'
public APIs, so asking for it at download time is deterministic work, not
model work.
Add `py/services/model_sources/hydration.py`, called by
`_save_source_metadata()` once the sidecar exists and the file is in the
scanner cache. It fetches the model card plus the site's card extras and
hands them to the same `PostProcessor` the AI skill uses, with an empty
`llm_output`, so the two paths cannot drift apart. What lands:
* `model_name` from the site's own display name (ModelScope's `Name`), so
the card stops showing the local filename — written only while the value
still equals the file stem, since once a user renames a model that
choice is theirs to keep
* `civitai.name` from the matched version's label (`showName`), which the
card renders as the version chip
* `civitai.description` / `modelDescription` from the author summary plus
the README as HTML
* `civitai.images` / `preview_url` from the per-file example images
* `civitai.trainedWords` from the per-file trigger words
* `base_model`, `tags` and `usage_tips` as before
Provenance stays honest: the pass records
`metadata_source = "source:<platform>"` rather than the skill's
`agent:enrich_hf_metadata`, and — because no provider ran — it no longer
stamps `llm_enriched_at`; that stamp is now conditional on the LLM
actually answering, which is what the field means. The five hand-rolled
`civitai` dict merges in the post-processor collapse into one
`_merge_civitai()` helper.
Two guards keep it safe. Only a model whose stored
`source_platform`/`source_url` match the repository being downloaded is
updated, so a local file that merely shares a name never receives another
model's card; and a file already on disk is topped up too, which
back-fills models downloaded before this existed. READMEs and detail
payloads describe the repository rather than the file, so a short-lived
process-wide `ModelSourceCache` (300 s, 32 entries) keeps a batch over one
repository to two HTTP requests. Every failure is logged and swallowed:
hydration can never fail a download.
Fix the hash policy while here. `_save_source_metadata()` went straight to
`MetadataManager.create_default_metadata()`, bypassing the per-type
factory on the owning scanner, so a checkpoint paid a full SHA256 inside
the download request — `CheckpointScanner`/`OtherScanner` deliberately
record `hash_status="pending"` with an empty `sha256` for their multi-GB
files. Metadata is now created through `scanner._create_default_metadata()`.
Hydration copes with the empty hash: `_matching_versions()` falls back to
the repository basename, which is exactly what the download just wrote.
Report both post-transfer stages, which advance no byte counter and so
read as a stall: the bar sat at 100% showing `0 B/s` for the seconds spent
hashing and fetching. `_report_phase()` broadcasts
`{"status": "metadata", "stage": "indexing" | "source", "platform": ...}`,
and `LoadingManager` names the stage in the status line (keeping the batch
position), retitles the item line, replaces the dead speed figure and runs
a sheen over the bar. `stage`/`platform` are machine-readable; the wording
is localised in the frontend.
Finally, `modelscope.ai` is its own catalogue rather than an alias of
`modelscope.cn` — `referall13/EM1` exists only on `.ai` and
`jj3550945163/Krea-2-LORA` only on `.cn` — so its URLs were rejected with
"Invalid model URL format". Register it as `ModelScopeIntlSource`
(`platform="modelscope-ai"`, `msai:` group prefix, its own default
download directory) and derive every URL either deployment builds from a
per-class `base_url`. `modelscope.com` stays an alias of `.cn`, which is
what it redirects to. The frontend source table, the link dialog hints and
the docs mirror the split.
Verified against the live APIs: both reported `.ai` repositories list
their files, read their READMEs and yield name / version / base model /
trigger words / example images. Backend 3092 passed; frontend 1259 JS +
91 Vue passed. The nine locales carry the new progress copy in the next
commit.
Adds a base model pill at the front of the recipe modal's tags row,
showing the full base model name (cards keep the abbreviation since
their overlay width is constrained). Falls back to a dimmed Unknown so
the header layout does not shift when hydration fills the value in.
Hydration now also merges base_model. Translated in all 9 locales.
`civitai.trainedWords` is an ordered array, and the order is what gets
pasted into a prompt: "Copy Trigger Words" and the insert-into-node
action join it as-is. The refresh merge unioned the stored words with the
freshly fetched ones via `list(set(...))`, so any metadata refresh
silently shuffled a user's ordering into an arbitrary one. Now that the
UI exposes reordering, that would look like the feature losing the change
at random.
Merge in order instead: stored words first (in their saved order), then
newly discovered ones, duplicates dropped. `_merge_ordered_unique` keeps
the behaviour easy to assert, and the existing merge test keeps passing
because it compares the result as a set.
The three `common.reorder.*` keys (grip tooltip and hint, the per-grip
aria label, and the aria-live announcement) are rendered in all 9
locales. They sit under `common` rather than in a feature namespace
because both the tag editor and the trigger-word editor render them, and
only `dragHandle` is visible copy — the other two are screen-reader text.
`Alt` and the `↑/↓` glyphs stay verbatim everywhere, the same precedent
as `Shift+Enter` in `modals.model.metadata.notesHint`, because they name
the keys rather than an action. "position X of Y" reuses each locale's
existing counting phrasing (ja `{total} 件中 … 番目`, ko
`총 {total}개 중 …번째`, fr `sur {total}`, ru `из {total}`), and
parentheses follow each file's own convention: full-width in zh-CN /
zh-TW / ja, ASCII in ko and the Latin/Cyrillic locales. No
`[TODO: Translate]` placeholder is left in any locale.
docs/i18n-translation-guidelines.md gains the matching §2 subsection and
status note so a later terminology sweep preserves these renderings; the
leaf-key count in its header is corrected to 2025 at the same time.
Model tags could already be reordered by dragging a chip, but the only
hint was a `cursor: grab` on `.metadata-item` — a hover-only, mouse-only
signal that also leaked into the bulk add-tags modal, where the chips are
not sortable at all. Trigger words could not be reordered, and their
order matters: "Copy Trigger Words" and the insert-into-node action join
the array as-is to build a prompt.
Both editors now share one vocabulary: a `⠿` grip that appears whenever
the list has something to order, plus `Alt + arrow` keyboard moves with
an aria-live announcement. Whether the chip body is draggable is a
property of the item rather than of the feature:
- tags have no click action of their own, so the whole chip stays
draggable (`handleSelector: null`), with a 5px threshold so a click on
the grip only focuses it
- trigger words keep click-to-edit on the body, so a drag starts from the
grip only
- the grip is the element that opts out of touch scrolling
(`touch-action: none`), so touch users drag by the grip in both editors
- reordering is offered only while editing: trigger words reveal the grip
from `.edit-mode`, tags from the edit container, which stays hidden
outside edit mode
The drag engine moves out of ModelTags.js into shared/pointerSort.js,
which now marks sortable containers with `pointer-sort-enabled` so only
lists that really sort show the grab cursor. Labels, the sortable flag,
the keyboard handler and the live region live in
shared/reorderSupport.js, and both editors render the same three
`common.reorder.*` keys (translations follow in the next commit).
Two inherited engine bugs are fixed on the way: the drop position was
only settled when an animation frame was still pending, so a fast drag
(or one that started by crossing the threshold) fell back into its
original slot; and the guard that stops a drop from triggering the chip's
own click handler was removed on a timer, swallowing unrelated clicks
until the next task.
The sidebar used the models-only folder list while the download and move
destination pickers list every directory, so a model downloaded into an
empty category folder did not appear in the sidebar at all. Empty folders
are also deliberate organization on disk, and a file-manager-shaped tree
that hides them is surprising. Default the preference to on.
Because the preference no longer gates fetching, both folder lists are
always loaded: the full list is the tree's single source of truth and the
empty-folder count, the models-only list is what "empty" is measured
against. That makes the view-options toggle a pure re-render, and lets the
new count decorate the "..." menu so the preference's effect is visible
without scanning the tree:
- empty-folder count shown next to the menu label, cleared when unknown
- the toggle is hidden entirely when there are no empty folders
- list view filters empty entries itself (the tree gets that for free
from the backend, the flat list is now always loaded in full)
- creating a folder still re-enables the preference, since the folder the
user just asked for would otherwise be invisible
Dim styling is decided by _isRenderedEmptyFolder() at render time; the
models-only set keeps its ancestor-expanded semantics for the delete
guard.
`Delete folder` and `Delete Model` rendered as plain menu text: the rule
was `color: var(--danger-color)`, and that token is defined nowhere in
the stylesheet tree. A var() reference to an undefined custom property is
invalid at computed-value time, so the declaration does not fall back to
a default — `color` inherits, and it silently matched the surrounding
menu text.
Points both the label and its icon (which inherits the colour) at
`--lora-error`, the themed error token the delete buttons already use.
The shared hover paints the accent background, which a red label does not
read against, so destructive entries also get their own `--lora-error-bg`
wash.
The same dead token was masked by a hardcoded fallback in the settings
priority-tags validation state; those now use the themed token too.
Fixes all seven destructive entries at once — the four model-card menus,
the exclude/duplicates `delete-all` entry and the folder sidebar menu.
Guard: tests/frontend/regression/contextMenuTokens.test.js fails if any
custom property used by menu.css stops resolving (verified by reverting
the token), and if var(--danger-color) ever comes back.
The folder context menu now reads: the content action (check for updates)
on top, then the folder operations as one group (new subfolder, rename),
then the destructive entry behind its own divider.
Gating the three folder entries per page could leave the menu with
dangling separators — the recipes sidebar hides all of them and keeps
only the update check, which already rendered one stray divider before
this change and would have rendered two after it. Add
_updateContextMenuSeparators: a divider survives only when a visible
entry sits on both sides, and a run of consecutive ones collapses to a
single line. The three per-item display toggles fold into one loop.
The template order is guarded by a regression test that parses
templates/components/context_menu.html, plus behaviour tests for the
divider collapsing.
Fills in the 35 `sidebar.*` placeholders the folder-sidebar feature series
left behind — view options (tree/list, empty folders), folder creation,
the delete confirmation modal and its result toasts, and folder
renaming, including the undo copy and the "deleting a folder never
cascades over model files" rule the backend enforces.
All nine locales are translated, so no `[TODO: Translate]` placeholder
remains anywhere: the state §7 of the translation guidelines describes
holds again. Terminology reuses the existing §2 maps (folder, model
root, sidebar) with tree/list view added, recorded in a new §2
subsection; the stale leaf-count in the header is refreshed too.
Follows the folder create/delete work: a typo'd directory could be
removed but not corrected, and for a folder holding models the only fix
was to move every model out by hand.
Adds POST /api/lm/{prefix}/rename-folder. Unlike the delete path this one
deliberately works on folders that hold models — a rename keeps every
file, so nothing is cascaded over: the directory is renamed on disk and
the scanner re-keys the records that pointed at the old prefix (recorded
folder list, cache file_path/folder/preview_url, hash and autov3 index
paths, excluded-model paths, and the metadata sidecars that travelled
with the directory). Ancestors are never touched, and only the leaf name
is accepted so a rename can never escape its parent.
Library roots, top-level symlinks and folders holding a staged delete are
refused; the last because a staging manifest records absolute
original/staged paths, so moving it would break undo and purge. A name
collision is a 409 target_exists conflict.
The sidebar reuses the inline-row idiom from folder creation: prefilled
with the current name, inserted in place of the node with that node
hidden while editing, Enter confirms and Escape/blur cancels. The
persisted selection and the expanded set are re-keyed across the rename
so the user keeps their place in the refreshed tree.
Folders created from the sidebar had no in-app way back out: the only
removal path was to leave ComfyUI, delete the directory by hand and
rescan. A typo'd folder also polluted the move/download destination
picker permanently, since it reads the same all_folders source.
Adds POST /api/lm/{prefix}/delete-folder, restricted to directories
whose subtree holds no model weight files — a folder-level cascade would
bypass the per-model lifecycle bookkeeping (metadata sidecars, previews,
cache entries, pending-delete staging, recipe references). The service
walks the directory itself instead of trusting the possibly stale cache,
reports what it would remove (models / files / subfolders / symlinks),
and refuses library roots, top-level symlinks (shutil.rmtree rejects
those) and folders holding a staged delete, whose manifest would be
invalidated by the move. Symbolic links inside the subtree are counted
but never followed.
ModelScanner.remove_known_folder mirrors add_known_folder: the removed
subtree leaves all_folders while ancestors are kept (every recorded
ancestor exists on disk in its own right), stale cache entries under the
prefix are purged and the folder list recomputed. The handler broadcasts
models_changed so destination pickers drop the folder too.
The sidebar entry is a destructive context-menu item. The modal opens in
a confirm state for model-free folders and an explanatory one when the
subtree still holds models, decided from the models-only set that already
dims empty nodes; a stale tree is caught by the 409 not_empty/busy
conflict. Truly empty folders get the existing 20s undo affordance,
implemented by re-creating the directory.
- Render the new-folder input as a temporary tree row at the creation
location (file-explorer style): full-width input confirmed with Enter
and canceled with Escape/blur; the parent folder auto-expands, and in
list mode the row is inserted after the parent item
- Remove the drag-to-blank-area folder creation (drop-zone strip,
sidebar-level drag handlers, performDragMoveWithState); dropping models
onto folder nodes still moves them
- Update empty-state hints and locale keys accordingly
Empty folders (tracked in the scan-recorded all_folders list, same source
the move/download destination picker uses) can now be surfaced in the
folder sidebar via a view-options toggle, dimmed when their subtree holds
no models. Folders can be created directly from the sidebar through a new
POST /api/lm/{prefix}/create-folder endpoint with library-root
containment checks; the scanner records the new directory incrementally
so the tree reflects it without a rescan.
The sidebar header moves its view toggles (tree/list, recursive, empty
folders) into a "..." menu to fit the new create-folder button.
A LoRA named `lora-sd1.5-backlight_slider_v10.safetensors` showed up in the
manager as `lora-sd1`, hid itself from searches for the rest of its name, and
collapsed into the same lora syntax tag as every sibling sharing the prefix.
The name was cut twice. `_process_model_file()` imports a third-party
`.civitai.info` sidecar by handing `from_civitai_info()` the local stem with
the extension already stripped, and the builder then stripped a second
"extension" from it -- `os.path.splitext` reads everything after the last dot
as one, so the version dot in `1.5` ended the name. The download path never
hit this because API filenames keep their extension and only need one strip.
Pass the real basename from the migration site, and make the builder strip
only a recognized model extension (`strip_model_extension`), so both input
shapes resolve to the same stem. The `model_name` fallback that reused the
same expression is fixed with it: on a sidecar without `model.name` the
display name was truncated too.
Libraries already corrupted do not heal on their own: the incremental Refresh
skips paths already in the cache (only a full rebuild reloads metadata) and
startup hydrates rows from SQLite as-is, so the wrong name survives restarts.
Reconcile now compares each cached row against the stem of its file path --
one string compare per file and no extra syscall, so a clean library pays
nothing -- and repairs mismatching rows through `load_metadata()` (which
normalizes the sidecar) and the existing in-place `_sync_cache_from_metadata_impl()`
path, which writes a targeted single-row SQL delta instead of a full save.
Repairs are one-shot, and a missing or corrupt sidecar keeps its row so a full
rebuild can recreate it without losing tags or civitai data.
Tests: the builder keeps dotted stems for all four model classes and still
strips real extensions; the migration writes the full local name to the
sidecar; and reconcile repairs memory, sidecar and SQLite row, runs exactly
once, and never reads metadata on a clean library.
A repository whose uploader wrote no README still gets a card. ModelScope
answers with a placeholder notice ("the contributor provided no further
description"), a block of SDK/git download instructions, and a closing
invitation to complete the card. None of it describes the model, yet it was
being sent to the LLM and, worse, stored as `modelDescription` — so a Krea 2
LoRA whose only real text was the author's summary showed 841 characters of
`pip install modelscope` scaffolding on its description tab.
Add `_strip_generated_card_boilerplate()` and run it on both paths:
`clean_readme_for_llm()` (the prompt) and `convert_readme_to_html()` (the
stored description). Markers are matched as substrings because the notices
are prose and because non-Latin scripts are not space-delimited — the notice
continues with a full-width period, so the `title == keyword` matching used
for the English boilerplate headings never fired.
A marker heading takes its whole section with it, which is what removes the
download block hanging off the notice; a stand-alone notice line is dropped
alone. Content the author added later, under a heading of equal or higher
level, is kept, so a card that was improved after the placeholder is not
thrown away.
Verified on the live repositories: the placeholder card's description went
from 841 characters to the 86-character author summary, while the repo with
a genuinely author-written card is byte-for-byte unchanged.
A file was matched to its published version by comparing basenames against
each version's `stats.fileList`. Renaming the weights — routine once a
model is filed away, and the reason the scanner records a sha256 at all —
made the match fail silently, so the file lost its example images and its
preview with no indication why.
The detail payload's `ModelInfos.safetensor.files[]` carries a real sha256
per published file, and the local hash is already on disk, so match on that
first: it is the one identifier a rename cannot invalidate. Exact basename
and `showName` matching remain as fallbacks, and an unknown hash falls
through to them rather than giving up, so a re-encoded file still resolves.
Verified against the live repository: a renamed `c1-st1000` file with its
hash yields the c1-st1000 image, the same rename without a hash yields
nothing, and supplying c1-st2000's hash resolves to the c1-st2000 image even
when the filename claims otherwise.
A collection repository publishes many model files under a single source id,
but enrichment re-read the README and the model-detail payload for every one
of them: eight checkpoints meant sixteen HTTP requests, each detail payload
being 10-22 KB of JSON.
Add `ModelSourceCache`, created by `execute_skill()` for the duration of a
run and passed to the provider through a new optional `cache` argument on
`fetch_model_card_context()`. The agent caches the README (repository-wide
and provider-agnostic), and ModelScope caches its detail payload under a
provider-namespaced key.
Only successful reads are memoised, so a transient failure is still retried
for the next file, and the per-file selection is redone from the cached
payload so a checkpoint never inherits a sibling's example images. Nothing
is retained across runs — a model card can change at any time — and download
URLs are not routed through the cache.
Measured over the eight checkpoints of one ModelScope repository: 16
requests before, 2 after.
To keep the two concerns separable, `_build_card_context()` now turns a
detail payload into a `ModelCardContext` as a pure function.
The tags instruction demanded "all lowercase, no spaces, no hyphens" with
single-word examples. That clause arrived in the same commit that added
the priority_tags cross-reference, so it reads as a crude way of pushing
the model towards that (entirely single-word) vocabulary rather than as a
requirement in its own right — and nothing in the codebase depends on it:
* `_merge_tags` only lowercases and de-duplicates;
* `resolve_priority_tag` matches aliases exactly, and the priority config
syntax already supports multi-word entries and aliases;
* the tag FTS index tokenises on non-alphanumerics, so a hyphenated tag is
indexed as two tokens and stays searchable;
* tags never reach a ComfyUI prompt — that is `trainedWords`.
It also fought the priority_tags rule it was meant to support. Handed the
site-curated `character-enhancement`, satisfying both rules produced
`character` as well; the run added generic priority-list tags and dropped
the site's own wording. The spelling used by the site, the frontmatter or
the author is now kept verbatim — hyphenated, multi-word or non-Latin —
and no separator-free synonym is invented for a tag already included.
Measured on a Krea 2 portrait LoRA, the proposal went from nine tags
(four of them generic priority-list words) to six grounded ones.
The post-processor stored the LLM's confidence as `_llm_confidence`, but
that value could never be read back: `BaseModelMetadata.from_dict()`
deliberately excludes underscore-prefixed keys from `_unknown_fields` and
`to_dict()` strips private fields, so it was erased by the next metadata
write and was invisible to `read_metadata()`. The enrichment evaluation
harness reads this field to score runs, so confidence was always scored
as blank.
Store it as `llm_confidence`, which round-trips as an ordinary unknown
field — the same mechanism `llm_enriched_at` already relies on. Nothing
else consumed the old name, and the harness still accepts it so sidecars
written by earlier versions keep evaluating.
Covered by a metadata load/save round-trip regression test plus
assertions that the post-processor writes the persisted key and no longer
writes the private one.
`_build_prompt_context()` was only reached when the LLM was configured,
so a user with no provider got nothing at all from a linked model source
— no preview, no example images, no author summary, no tags — even
though all of that is deterministic data from a public API.
Split the model-card fetch into `_load_source_card()`, which runs for
every source-backed enrichment, and have the post-processor apply its
result whether or not the LLM runs. The prompt is then built from the
already-fetched card rather than re-fetching it.
Invoking "Enrich Metadata with AI" still always calls the provider; a
model source supplying a description, images and tags is not treated as
a reason to skip it, since the LLM's summary and notes are richer and an
action that silently does not call out to the provider would be
unpredictable. The site data acts as a fallback for the gaps the LLM
leaves.
Add `base_model_resolver.resolve_base_model()` to map the site's own
names (`krea/Krea-2-Turbo`, `KREA_2_TURBO`) onto the canonical
vocabulary, used only when the LLM returns no base model. It is strictly
conservative — exact normalised matching plus a bounded set of variant
suffixes, and it only ever returns a name that is already in the
vocabulary — so an uncertain hint defers to the LLM instead of writing a
plausible-looking wrong value.
ModelScope's model card is not just README.md: the author's summary
(Description), the site-curated tags (OfficialTags), the internal
architecture enums (VisionFoundation/SubVisionFoundation) and — per
published version — the model filenames with that file's example images
(coverImages) and trigger words all live in the model-detail API.
AIGC repositories there frequently ship an auto-generated boilerplate
README and put the only useful text in Description, so reading just the
README yielded almost nothing.
Add `ModelSource.fetch_model_card_context()` returning a new
`ModelCardContext`, implemented by ModelScopeSource against the public
(no API key) detail endpoint. Example images are matched to the model's
basename through each version's `stats.fileList`, so every checkpoint in
a collection repository gets its own images rather than a sibling's.
Consume the context in the post-processor:
* example images seed `civitai.images` and, being per-file, take priority
in the preview fallback chain
* the author summary becomes a paragraph in `modelDescription` and fills
`civitai.description` when the LLM returns no short description
* site-curated tags are always merged in, which also fixes the official
`character-enhancement` being dropped by the prompt's no-hyphen rule
* per-file trigger words are used before the repo-wide YAML
`instance_prompt`
* an explicitly stated strength range is recovered by regex so
`usage_tips` is populated even without an LLM
The prompt gains a Site-Provided Metadata section so the LLM can prefer
the site's first-hand data over its own guesses.
Mirror the import modal fix (c5088772): make the download modal a flex
column with a scrollable step area so the Back/Download buttons stay
visible on short viewports (e.g. 1080p) instead of requiring a scroll
to the bottom of the location step.
Node code reads cache.raw_data while MetadataSyncService may mutate it
from a background thread; iterate over a list() snapshot to avoid a
possible 'list changed size during iteration' RuntimeError.
Enriching a model with `llm_provider=deepseek` failed outright with
HTTP 400 "This response_format type is unavailable now". Probing the
endpoint shows why:
response_format absent -> 200
{"type": "json_object"} -> 200
{"type": "json_schema",...} -> 400
`chat_completion_json` preferred `json_schema` for a real reason -- LM
Studio and other local OpenAI-compatible servers reject `json_object`
but accept `json_schema` -- and guarded the fallback with a substring
test for `'response_format.type'` (the wording of those servers'
rejection). DeepSeek's message is "This response_format type is
unavailable now", which does not contain that substring, so the guard
re-raised and the retry never ran.
Make the format a per-provider chain instead of a single guess:
- `_JSON_OBJECT_ONLY_PROVIDERS` lists providers known to reject
json_schema (currently just deepseek). They ask for `json_object`
first, so the common case costs one request and no wasted retry.
- Everyone else keeps `json_schema` first, then downgrades through
`json_object` and finally prompt-only mode.
- A downgrade now happens on any error mentioning `response_format`,
which covers wording variants without swallowing unrelated failures:
auth errors, unknown models, and rate limits still surface unchanged
because their messages never name the parameter.
`json_object` is sufficient here: the skill prompt already specifies the
exact JSON shape, and `_try_salvage_json` repairs imperfect output.
Verified against the real configured endpoint with the real
`enrich_hf_metadata` prompt, prompt renderer, and ModelScope model card
for jj3550945163/Krea-2-LORA: a 9,815-character prompt returns
parseable JSON (base_model "Flux.1 Krea", description, tags, notes).
Three regression tests cover the DeepSeek ordering, the
json_schema -> json_object downgrade, and the no-retry-on-unrelated-400
path. Full backend suite: 2856 passed.
The download dialog's URL field still said "CivitAI URL(s)" and rejected
anything that was not CivitAI, and the hint listed only CivitAI / CivArchive /
Hugging Face. Four en.json values were refreshed in the previous commit and
propagated here:
- modals.download.civitaiUrl -> "Model URL(s)" (模型 URL / モデル URL / modèle /
Modell / modelo / модель / מודל).
- modals.download.urlHint names all four supported sites.
- modals.download.errors.invalidUrl -> "Invalid model URL format"; it is the
generic "unrecognised URL" error, so naming CivitAI was wrong.
- modals.download.errors.mixedSources names Hugging Face / ModelScope.
Brand names stay Latin per R3, "model" follows the §2/§5 rendering already in
force in each locale, and the Latin/Cyrillic/Hebrew files keep ASCII
punctuation. en.json is unchanged in this commit; exactly four lines change in
each of the nine locale files, with no reindentation — the sync script does not
refresh an existing key's value, so this was done by exact-literal replacement.
pytest tests/i18n: 20 passed and sync_translation_keys.py --dry-run is a no-op.
ModelScope became a linkable source, but downloading from it was impossible:
the URL picker only recognised huggingface.co, the file listing hit a
huggingface-only endpoint, the resolve URL was hardcoded, and the default
path template always wrote into a `huggingface/` directory.
Move the download knowledge into the providers so the handlers stay generic:
- `ModelSource` gains `list_files()`, `file_download_url()`,
`default_revision` and `default_subdir`. `HuggingFaceSource` keeps the Hub
tree API (`/api/models/{id}/tree/{rev}`, LFS-aware sizes, `main`).
`ModelScopeSource` uses `/api/v1/models/{id}/repo/files?Revision=master`
— which reports real byte sizes for LFS files, so no HEAD probe is needed,
and which only accepts `master` (an HF-imported repo still 404s on `main`)
— and downloads through `/models/{id}/resolve/{rev}/{path}`. That URL
redirects to a CDN target carrying a time-limited `auth_key`, so it is
rebuilt on every request and never cached, which is also what keeps
resumable Range requests working.
- `hf_handlers.py`/`HfHandler` become `model_source_handlers.py`/
`ModelSourceHandler` with `list_model_source_files` and
`download_model_source`. New routes `/api/lm/model-source-files` and
`/api/lm/download-model-source`; the old `/api/lm/hf-repo-files` and
`/api/lm/download-hf-model` paths stay as aliases, and a payload without
`platform` still means Hugging Face, so existing callers are unaffected.
- A downloaded sidecar now records `source_platform` + `source_url` (with the
`hf_url` alias only for Hugging Face) instead of always writing `hf_url`,
and `use_default_paths` files ModelScope downloads under
`modelscope/<owner>/<repo>`. The now-unused shared HF aiohttp session and
its shutdown hook are gone; providers open short-lived sessions.
- Frontend: `detectUrlType` returns the platform-neutral
`model-source-repo` / `model-source-file` plus an explicit `platform`, the
DownloadManager's `hf*` state and methods are renamed to `source*`, every
`source === 'huggingface'` check becomes `isExternalModelSource()`, and
batch groups are keyed by `platform:repo` so the same `owner/name` on two
sites renders as two groups. A bare `owner/name` still means Hugging Face.
- `is_valid_source_id()` centralises repo-id validation (exactly
`owner/name`, no traversal, no leading dot). This also fixes the old HF
download check that rejected any dot in the name, i.e. legitimate repos
such as `black-forest-labs/FLUX.1-dev`.
Verified against the live APIs: the example repo lists 8 weight files with
correct sizes, and a ranged GET of the built resolve URL returns 206 after
following the redirect to the CDN. Backend 2853 passed; frontend 1143 JS +
91 Vue passed. The nine locales carry the refreshed download copy in the
next commit.
Complete the 15 [TODO: Translate] keys the model-source feature left behind
(modelCard.actions.viewOnSource, loras.contextMenu.linkModelSource,
modals.linkModelSource.*, modals.model.versions.sourceGroupInfo,
toast.contextMenu.enrichNeedsSource, toast.contextMenu.enrichUnsupportedSource),
and refresh the two enrichment labels that feature made stale.
- Brands stay Latin per R3: Hugging Face / ModelScope / TensorArt appear
verbatim, and {source} is substituted by the caller at runtime, so no locale
embeds a transliterated platform name. The placeholder-URL value
(modals.linkModelSource.urlPlaceholder) stays byte-identical to en.json per
the §6 URL exception.
- "model source" / "model page" / "model card" are new nouns and each locale
gets exactly one rendering; "AI enrichment" reuses the noun already in each
file from the previous enrichHfAgent copy. All of it is recorded in §2.
- modelCard.actions.viewOnSource follows each locale's existing
viewOnHuggingFace pattern rather than the neighbouring viewOnCivitai one, so
de/ru/he/ja/ko do not gain a third "View on ..." shape.
- loras.contextMenu.enrichHfAgent and loras.bulkOperations.enrichHfAgent read
"AI HF metadata" in all nine locales. The feature invalidated that by also
covering ModelScope, so both values drop the HF qualifier (the key names keep
the historical Hf, and the guidelines now say so).
- Script conventions: fr keeps ASCII apostrophes and a space before ':' (the
file is 351 ASCII vs 26 U+2019 and the modal being replaced was ASCII); ko
keeps ASCII ':' and '()' (188 vs 6); CJK locales keep full-width punctuation;
every ellipsis is ASCII '...'. Placeholders are verbatim per R2.
- modals.linkModelSource.enrichNote is phrased as a rule with the current
exception in parentheses, so the guidelines call that out for whoever adds
the next link-only source.
pytest tests/i18n: 20 passed, and scripts/sync_translation_keys.py --dry-run is
a no-op (no missing and no stale keys). Frontend: 1130 JS + 91 Vue passed.
Backend: 2815 passed. en.json is untouched by this commit.
A model file could only ever be linked to huggingface.co: `set_hf_url`
validated the URL with a huggingface-only regex, the agent fetched the card
from a hardcoded HF URL, and the readme processor built every relative image
path off `https://huggingface.co/{repo}/resolve/main`. ModelScope publishes the
same model-card convention (README.md + YAML frontmatter, often carrying
`base_model:` and `trigger_words:`) behind a public, key-less API, so the
enrichment pipeline could already serve it - it was the plumbing that was
HF-shaped, not the idea.
Make the external source a first-class, provider-driven concept:
- New `py/services/model_sources/` registry. A `ModelSource` owns URL
recognition (lenient for stored values, strict for user input), the
canonical page URL, model-card fetching, the asset base URL and the
capability flags. `HuggingFaceSource` is the previous logic relocated;
`ModelScopeSource` reads `/models/{o}/{n}/resolve/{master|main}/README.md`
and falls back to `/api/v1/models/{o}/{n}/repo`. `TensorArtSource` is
link-only on purpose: tensor.art answers plain HTTP clients with a
Cloudflare challenge and its internal API (ap-east-1.tensorart.cloud /
cn.tensorart.net) rejects every /v1/model/* route with "invalid
authorization header", so it declares supports_enrichment=False rather than
failing silently later.
- Metadata gains `source_platform` + `source_url`; `hf_url` stays as a
read/write alias, written only for Hugging Face, so existing sidecars,
cached rows and third-party consumers keep working. Normalisation runs at
the scanner, the persistent cache (both directions, plus two new columns
behind an ALTER migration) and the linking handler - which is what stops a
user who switches sources from leaving a stale `hf_url` on a ModelScope
model.
- The agent pipeline keys off the provider instead of `hf_url`: the fast-fail
gate now explains *why* a model is skipped (no source / unknown source /
source without a reachable card), the prompt context exposes
source_url/source_id/source_label/asset_base_url while still filling the
legacy hf_url/repo aliases, and the four README image extractors take a
base_url (defaulting to HF) so relative paths resolve against the right
site. Version grouping generalises to hf: / ms: / ta: keys.
- `POST /api/lm/set-hf-url` keeps its path and its legacy payload keys but
accepts `source_url`, validates against every provider and returns the
platform. `GET /api/lm/model-sources` lets the UI render the supported-site
list from the server.
- Frontend: a `modelSourceHelpers` mirror of the registry drives the link
dialog, the card/modal globe (branded "View on ModelScope/TensorArt"), the
version-group key and the enrichment gate; the versions tab no longer sends
ms:/ta: keys to the CivitAI API.
TensorArt stays in the list because provenance is worth keeping even when the
card is unreadable - the dialog says so plainly ("Sites that don't expose one
(currently TensorArt) can only be linked") and the context menu disables
enrichment with a matching tooltip, instead of the user getting
"Unsupported URL".
Verified against the real ModelScope API: jj3550945163/Krea-2-LORA returns a
1882-byte card whose frontmatter carries base_model/tags/trigger_words, and
relative images resolve to .../resolve/master/....
Tests: backend 2815 passed; frontend 1130 JS + 91 Vue passed; pytest
tests/i18n and a Jinja compile pass over templates/. The nine locales carry
[TODO: Translate] for the new strings, completed in the next commit.
Other Models management is opt-in and its folders come from
folder_paths.get_folder_paths(). In plugin mode ComfyUI registers vae,
upscale_models, text_encoders, clip_vision and controlnet out of the box, so
enabling the feature works immediately. Standalone only knows the keys present
in settings.json.folder_paths, and that file is edited by hand - there is no UI
for those keys - so a standalone user who followed the announcement banner
reached "Enable Other Models" and then an empty page.
Gate the announcement on the capability instead of on how the process was
started:
- Config.get_other_models_availability() probes every canonical other key
(legacy clip collapses into text_encoders where the host exposes
map_legacy) and reports which sub_types resolve to a folder that exists on
disk. It deliberately ignores enable_other_models: the question is "could
this work here at all?". An empty folder counts, because CivitAI downloads
can target it.
- /api/lm/settings exposes it as the derived, non-persisted
other_models_paths_available flag; a probe failure yields null and the
banner fails open.
- BannerService only registers the announcement when the flag is not false.
`=== false` (not falsy) keeps a cached/older payload working, and nothing is
written to dismissed_banners, so the banner can return once folders exist.
- The Other page grows an "enabled but nothing to scan" empty state driven by
config.other_roots, showing the settings.json snippet for standalone and a
pointer to ComfyUI model paths otherwise, plus an Open Settings action. It
also covers the corner where only a non-default sub_type has a folder.
Translate the six other.noPaths.* keys into all nine locales and record the
new "folder key" / "on disk" terminology in the i18n guidelines.
Backend tests and pytest tests/i18n could not run in this environment (no
pytest/platformdirs); the probe was exercised against a stubbed folder_paths.
Frontend: 120 files / 1101 JS tests passed.
A model could have CivitAI metadata and a HuggingFace link at the same time,
but only one of the two "View on ..." entries ever rendered, because both the
model modal and the card globe asked the `from_civitai` provenance flag which
source to show. `set_hf_url` wrote `false` and a CivitAI refresh wrote `true`,
so whichever ran last erased the other: linking HF hid "View on CivitAI" even
though the civitai payload was still in the sidecar, and (on the card) a later
refresh pointed the single globe icon back at CivitAI, hiding the HF entry.
Decide the links from the data itself instead:
- `set_hf_url` no longer touches `from_civitai`; it records where the metadata
came from, and HF provenance is already tracked by `hf_url`.
- Add `hasCivitaiSource(civitai)` in the shared card/modal utils and gate the
modal's CivitAI link, the card globe (title, enabled state, click target,
new `data-has_civitai`) and the context-menu `civitai` action on actual
CivitAI data (`modelId` / `model_id` / `id`). A dual-source model now shows
both links, and a CivitAI-only model with no `hf_url` stays as before.
- Agent HF enrichment (`PostProcessor.is_hf_model`) keyed off
`not from_civitai`, which stopped being a synonym for "has an HF source" once
both sources can coexist (and already broke after a CivitAI refresh flipped
the flag back to true). Key it off `hf_url` directly; the post-processor
tests move to that discriminator and gain a dual-source case.
Regression tests: the set-hf-url handler preserves civitai + `from_civitai`
and no longer forces the flag false, the modal renders both links (including
with `from_civitai: false`), and the card globe targets/opens the right source
and is disabled when neither is available.
Backend: 2749 passed. Frontend: 1098 JS + 91 Vue tests passed.
Three pre-enable strings listed exactly the old default set (VAE, upscaler,
text encoder, CLIP vision), so they read as "these are what enabling
manages" - now wrong twice over, since clip_vision became opt-in and
ControlNet was never named.
Point them at the capability instead: other.disabled.description and
banners.otherModels.content enumerate all five sub_types, and
settings.folderSettings.enableOtherModelsHelp names all five folder
categories the master switch gates. Model-type names stay in Latin per the
model-type rule; de compounds as CLIP-Vision- und ControlNet-Ordner and the
slash-list locales keep their existing VAE / Upscaler / Text Encoder / ...
casing. No placeholders or HTML are involved.
Editing en.json leaves the nine locales stale, and the sync script only adds
missing keys, so each locale is updated in the same pass by exact-literal
replacement of the one line - no JSON round-trip, no formatting churn (three
changed lines per file). Record the refreshed strings and the
"capability, not defaults" rule in the i18n guidelines.
DEFAULT_ENABLED_OTHER_SUB_TYPES managed vae, upscaler, text_encoder and
clip_vision while controlnet was the sole opt-in type. That split was not
defensible on demand breadth: ControlNet is the broader category by install
base, and clip_vision is the narrower one (IPAdapter/SVD image conditioning,
usually one to three files) whose CivitAI type is retired upstream.
Keep the default set to the dependency-style assets every pipeline needs and
where "which one am I actually using" is the real problem - VAE, upscalers
and text encoders - and treat clip_vision and controlnet symmetrically as
opt-in. The feature is still unreleased, so the change needs no migration.
- Sync all five surfaces holding a default: DEFAULT_ENABLED_OTHER_SUB_TYPES,
DEFAULT_SETTINGS, both DEFAULT_SETTINGS_BASE/createDefaultSettings lists,
updateOtherModelsControls()'s fallback and the Jinja fallback.
- The selection is persisted per user, so only the untouched default moves;
existing default_other_roots entries for a disabled sub_type are preserved.
- Fix the Jinja fallback using `or`, which treated an all-unchecked empty
allow-list as "unset" and re-checked every box on render; `is none` keeps
the empty list empty.
- Document the revised defaults and rationale in the plan.
Tests assert the new default trio, the normalize fallback, that both opt-in
types stay out of the default scan, and the auto-set iteration test now
enables clip_vision explicitly since it exercises the loop, not the default.
Enabling Other Models logged two warnings on a stock ComfyUI install:
Detected the same folder '.../clip' under multiple other-model categories
('.../clip' is already mapped). Keeping the first category; please fix
your path configuration.
Nothing was wrong with the configuration. ComfyUI's folder_paths rewrites
legacy names before every access (map_legacy: clip -> text_encoders,
unet -> diffusion_models) and registers both legacy directories under the
canonical key, so get_folder_paths("clip") returns exactly the same list as
get_folder_paths("text_encoders"). Both keys are in the enabled allow-list,
so the second pass hit the overlap guard for every text-encoder folder and
printed advice the user cannot act on. The path list itself was correct
(deduped), only the message was wrong.
- Config._collapse_legacy_folder_keys() drops a key when the host exposes
map_legacy and resolves it to another queried key. That is provably
lossless: an empty canonical list implies an empty alias list. The
standalone MockFolderPaths has no map_legacy and its keys are independent
settings.json entries, so every key is still queried there.
- _prepare_other_paths() now tracks the claiming sub_type alongside the
business path and downgrades a same-sub_type duplicate to debug, keeping
the warning for a genuine cross-category collision (and naming the other
category in the message).
Regression tests cover the aliased-key layout (no warning, no redundant
query, both folders still managed) and the same-sub_type duplicate, and the
opt-in test is parametrized over controlnet and clip_vision.
Other-model downloads now default to a flat layout, so a fresh library shows an
empty folder tree there while the sidebar still consumes 230px. Default the
per-page visibility to hidden for "other" through a small per-page default set.
The preference stays persisted per page, so an explicit show/hide toggle wins
afterwards, and the existing edge indicator keeps the hidden sidebar
discoverable and recoverable. Primary pages keep their visible default.
get_download_path_template() fell back to "{base_model}/{first_tag}" for any
unconfigured model type, so other-model downloads were silently nested under an
arbitrary CivitAI tag even though the settings UI exposes no template row for
"other" and priority_tags has no "other" entry (making {first_tag} resolve to
tags[0]).
Add DEFAULT_DOWNLOAD_PATH_TEMPLATES with other -> "" so unconfigured and
unknown types resolve to a flat layout under the already sub_type-scoped
default_other_roots; explicit settings.json values still win. Mirror the flat
default in the frontend DEFAULT_PATH_TEMPLATES and stop the download/move
default-path previews from rendering "/undefined" or a dangling slash.
At ~628px the header overflowed horizontally by 53px: the labelled nav held
383px that flex could not reclaim, so the search field was clamped to its
200px floor and had only ~96px of text room, letting the placeholder collide
with the Ctrl+F cue and the inline toggles.
Three rules drove that:
- .header-search had a hard min-width: 200px, so it parked at a fixed width
instead of shrinking with the space it was actually given.
- The input reserved 6.75rem for "options + filter + clear/cue", but that
declaration never applied: search-filter.css is imported after header.css
and its .search-container input (equal specificity) set the right padding.
The inline chrome actually needs 126px, so text ran underneath it.
- Labels stayed on the nav down to 600px, where a labelled nav (~383px) and a
readable search field (~300px) cannot coexist.
- Drop the min-width floors on .header-search and its container so the field
compresses naturally.
- Reserve exactly the inline chrome (cue 58 + clear 28 + toggles 56 + gaps and
edges 16 = 126px) and document why !important is required here.
- Add a 1366px breakpoint that hides the Ctrl+F cue and drops the reservation
to 68px; the shortcut itself keeps working, only the visual hint goes.
- Move the nav icon fallback from 600px to 700px and keep the <=600px
container tightening as its own query.
Verified in headless Chrome against the real stylesheet: no horizontal
overflow at any width (was 53px at 628px, 80px at 601px), and the placeholder
plus Ctrl+F cue never overlap (the same collision existed at ~1250px, where
the field now keeps 85px of text room instead of 2.8px).
Implements the backend slice (B1-B7) of
lm-civitai-extension/docs/other-models-support.md, which lets the companion
browser extension detect, badge and download the opt-in Other Models types
(VAE / upscaler / text encoder / CLIP vision / ControlNet).
ModelLibraryHandler:
- _normalize_model_type() learns the CivitAI other aliases (vae, upscaler,
textencoder, clip, clipvision, controlnet, other) and maps them to "other".
- _get_scanner_for_type() resolves "other" through the other scanner, but only
while enable_other_models is on, so model-versions-status and
model-version-download-status keep their legacy 400 when the feature is off.
- check_model_exists() / check_models_exist() consult the other scanner last
(lora -> checkpoint -> embedding -> other) and report modelType "other".
With the feature disabled both endpoints stay byte-identical to before and
the other scanner is never touched.
DownloadManager:
- The four other-type default-path failures now carry a machine-readable
"reason" (contract C4): other_models_disabled, other_sub_type_disabled,
other_no_default_root, other_sub_type_undecidable. The user-facing "error"
strings are unchanged; the key is additive and reaches the client because
both download endpoints pass the result dict through verbatim.
Tests cover the opt-in on/off branches for both existence endpoints, mixed
lora + other ids in the batch endpoint, the CivitAI alias acceptance and the
400 regression for unknown types, and the exact reason/error pairs for all
four download failure modes.
The action bar forced .controls-right (Doctor) onto its own full-width row
below 1500px. On high-DPI displays a maximized window reports a CSS viewport
of ~1280-1440px, so the Doctor button wrapped even with ~500px of free space
next to the action buttons.
- .actions / .action-buttons now wrap only on real overflow (flex-wrap plus
min-width: 0) instead of a viewport breakpoint.
- .controls-right relies on its auto margin to stay right-aligned on either
row, so the width: 100% + margin-top: 8px override is gone.
- Lower the button min-width floor from 100px to 90px; the old floor alone
made the row overflow the 1400px container at wide viewports.
- The <=1500px breakpoint now only tightens the buttons (min-width: 0,
padding, gap) and no longer forces a wrap; drop the no-op 0.8em font-size
override (base is already 0.85em).
- Keep the stacked mobile layout below 768px.
Verified in headless Chrome against the real stylesheet: one row with the
Doctor button inline down to 1200px (down to 1000px for shorter locales),
right-aligned wrap only when the content genuinely does not fit, and no
horizontal overflow at any width.
Undo the Doctor relocation from fe160134 while keeping that commit's
unrelated header changes (full-width header, 32px click targets,
role/tabindex plus Enter/Space activation).
- Restore the .doctor-control-group button in controls.html
- Drop the .doctor-toggle icon and the hamburger menu entry from the header
- Remove the Header.js 'doctor' dropdown action forwarding to the button
- Drop the header-scoped doctor-toggle styles
- Restore the .doctor-trigger styles (desktop and mobile) in doctor-modal.css
Complete the 36 keys left as [TODO: Translate] by the Other Models
feature (VAE / Upscaler / Text Encoder / CLIP Vision / ControlNet
management page and its opt-in toggles): settings.folderSettings.*,
other.*, initialization.other.*, toast.settings.otherRootsFailed and
banners.otherModels.*.
Model-type names (VAE, Upscaler, Text Encoder, CLIP Vision, ControlNet)
stay in Latin per the model-type rule, so the five subType* values are
intentionally identical to en.json; "Other Models" is a page/feature
name and is translated. Document the new terminology in the i18n
translation guidelines and note the completed i18n phase in the plan.
Documents the settings keys and defaults, the enabled/disabled behaviour
matrix, the backend and frontend touch points, cache consistency, the
discoverability surfaces (hidden nav + announcement banner + download CTA)
and the minimal settings.json.example policy.
- The Other nav entry is hidden while the feature is off
(nav-item--hidden, toggled client-side after enabling) and now uses the
fa-shapes icon.
- Shared utils/otherModels.js helpers (enable through the settings API,
open the settings Library section) are reused by the disabled page, the
announcement banner and the download modal.
- BannerService registers a one-time dismissible "other-models-announcement"
banner while the feature is off; SettingsManager drops the banner and
updates the nav when the master switch flips.
- A disabled download routing answer now surfaces a showActionToast with an
"Enable Other Models" action.
- Settings UI: master toggle + five sub_type checkboxes whose default-root
selects are disabled when unchecked; i18n keys added to en.json and synced
(other locales keep TODO placeholders).
Other Models management is now opt-in: enable_other_models (default false)
plus the enabled_other_sub_types allow-list replace the unreleased additive
enabled_other_folders key.
- config._get_enabled_other_folder_keys() is the single scan gate; a new
refresh_other_roots() rebuilds roots and preview roots on toggle.
- ModelScanner gains a _should_keep_cached_entry() hydration hook and
on_library_changed(reconcile=...) so switching a sub_type off drops its
entries (and hash/autov3 rows) at load time and switching it on rescans.
- OtherScanner filters location-derived entries accordingly.
- Other routes reject every other type while off (or a disabled sub_type) and
expose an "other_disabled" page flag; download routing returns a disabled
marker instead of guessing; the download manager refuses other-type
downloads and default-path routing for switched-off sub_types.
- Doctor / init-status / refresh-all skip the other scanner while off; the
scanner stays registered so staged pending-deletes still merge.
- Tests updated with explicit opt-in fixtures plus new gating coverage.
The example now only carries use_portable_settings, civitai_api_key and the
four core folder_paths keys (loras/checkpoints/unet/embeddings). Optional keys
such as the other-model folders, default_*_root and auto_organize_exclusions
are removed; their defaults live in DEFAULT_SETTINGS and reach the user's
settings.json on demand.
AGENTS.md now forbids adding optional/default keys to the example unless the
user explicitly asks for it.
- Drop the fixed max-width on .header-container so the header spans the
viewport while the card grid keeps its own content width
- Keep header icon click targets at 32px at all breakpoints and add
role/tabindex/aria-label plus Enter/Space activation
- Relocate the Doctor trigger from the page toolbar to the header icon
group (also available on the statistics page and in the hamburger
menu), removing the now-unused .doctor-trigger styles
The include_empty folder tree (download/move modals) walked every model
root synchronously on the event loop via get_all_folders(). On network
(NAS) roots this froze the whole server for the duration of the walk —
blocking WebSocket progress, aria2 RPC and the download queue — and the
5s TTL re-triggered the walk on nearly every modal interaction.
The scanners already visit every directory during cache scans, so record
the full directory list (including empty folders) there instead:
- _gather_model_data/_reconcile_cache collect directories during the
existing walks; reconcile refreshes and persists the list even when no
model files changed.
- ModelCache gains an all_folders field (None = never recorded).
- PersistentModelCache stores the list in a new folders table, with a
cache_meta flag distinguishing 'recorded empty' from legacy snapshots.
- get_all_folders() is now a pure in-memory read. A legacy snapshot
triggers a one-shot backfill walk in a worker thread (never on the
event loop) that records and persists the list.
- Moves add the destination folder (and parents) incrementally instead
of invalidating a TTL cache.
The browse endpoint and its frontend were written with POSIX-only
assumptions, so on Windows pressing Browse immediately failed with
"Access denied to this directory":
- The frontend opened the browser at "/", which resolves to the
current drive root on Windows.
- The allowlist check used Path("/"), which has no drive letter on
Windows, so relative_to() rejected every drive-qualified path —
anything outside the user profile was denied.
Fixes:
- Empty browse path now defaults to the user home directory instead of
erroring; the frontend sends "" rather than the POSIX-only "/".
- The access check is platform-aware (drive-qualified on Windows,
absolute on POSIX).
- Parent navigation uses the server-provided parent_path; the root
check is now path.parent == path (the old str/anchor comparison
self-looped at Windows drive roots).
- Browsing up from a Windows drive root shows a virtual list of
available drives so users can switch drives without typing a path.
The Windows-only case-insensitive match in ModelScanner._reconcile_cache
is the only pass left unverified by the recent realpath cleanup: realpath
may already cover case differences on Windows, and if the branch is ever
reachable it is O(files x cache entries). Records the reachability
question, the verification steps for a Windows run, and the two possible
fixes.
A no-change Refresh still computed os.path.realpath for every model file
in the library and for every cached entry. Both values are only ever
consulted when a discovered file is missing from the cache, so on a
50k-file library they cost ~1.3s and ~0.6s while being used zero times.
- Compute the per-file realpath only after the exact cache match fails
- Build the physical-path alias map lazily on the first miss; the
cross-run alias guard (overlapping roots / symlink layout changes)
still keeps the cached entry instead of a delete + re-add, which would
re-read metadata and re-hash the whole library
- Snapshot get_model_roots() once for the new-file pass instead of
re-reading it for every added file
- Run the duplicate-path integrity pass only when the snapshot already
contained duplicates or files were appended; a clean, unchanged cache
has nothing to clean. Duplicates can only be introduced by external
code rewriting raw_data or by this pass's own appends.
Zero-change reconcile drops from ~1400ms to ~120ms on 50k files, and an
alias flip still re-processes 0 files (#1108 investigation).
The download modal's location step decided between checkpoint and unet
roots using only the CivitAI file-type signal, while the backend also
falls back to DIFFUSION_MODEL_BASE_MODELS. Models like Anima (file type
"Model") were offered checkpoint roots in the UI even though
use_default_paths would route them to the unet root.
- Extract the two-tier decision into py/services/download_routing.py and
reuse it in DownloadManager._execute_download
- Add POST /api/lm/download/routing so the UI asks the backend for the
routing decision; fall back to the local file-type check on failure
- ModelVersionsTab: search both checkpoint and unet roots when resolving
an existing version's download path
A cancel landing between aria2.addUri acceptance and the _transfers
registration found no tracked transfer, so DownloadManager tolerated the
"not found" and only cancelled the asyncio task — the daemon kept
downloading the file untracked while history showed the download as
cancelled.
- Register the gid in _transfers immediately after addUri returns,
before any further await (state-store persist moved after it)
- Shield the addUri RPC so a mid-flight cancellation still learns the
accepted gid and forceRemoves it before re-raising CancelledError
- On cancellation during the state persist, remove the daemon transfer
unless it is paused (skip_download relies on paused gids surviving)
The option only existed in the LoRA page menu. Move updateEnrichMenuItem
and enrichWithAgent into ModelContextMenuMixin so both pages share the
implementation, and add the menu item to the checkpoints template.
Moving a checkpoint into a unet root (or vice versa) moved the file and
updated the in-memory cache, but three stale spots survived until a
manual cache rebuild:
- The moved .metadata.json kept the old sub_type, and the opportunistic
sync_cache_from_metadata path (fired by get_model_metadata and example
image metadata updates) trusted it, reverting the cache entry and the
SQLite snapshot to the pre-move sub_type. Loader nodes filter strictly
on sub_type, so the model stayed listed under the old type.
- The manager page discarded the move response's cache_entry, so the
card badge (CKPT/DM) and context menu label kept showing the old type.
Fixes:
- move_model now re-resolves sub_type from the target location (new
resolve_sub_type_for_path hook) and persists it into the moved
.metadata.json.
- _sync_cache_from_metadata_impl runs desired entries through
adjust_cached_entry so location-derived fields cannot be re-poisoned
by stale metadata snapshots.
- MoveManager carries cache_entry.sub_type into the in-place card
update so badge and context menu reflect the new type immediately.
detectUrlType only matched /resolve/ links, so pasting a HF web preview
(/blob/) URL fell through to direct-http and surfaced a misleading
'Invalid CivitAI URL format' error. Treat blob URLs as hf-resolve.
Replace the post-run toast cascade and the standalone L4 results modal
with a summary modal modeled on the batch download summary: 3-state
header, stat cards (matched / needs review / unresolved / errors),
an L4 review table with per-entry undo, and a copyable report. Wired
into the global, bulk and single-recipe rematch entries; complete
no-op runs keep the lightweight toast. Obsolete results-modal code,
styles and i18n keys are removed.
- Snapshot pre-rematch entry state (reconnectSnapshot) so rematched
entries can be undone via the existing restore flow
- Bulk missing-LoRA downloads mark unresolvable failures hash-invalid,
flipping those entries from download to reconnect candidacy
- Recipe modal always offers a reconnect action next to download for
missing LoRA entries
- Rematch runs collect an opt-in relaxed-matching choice (also reconnect
missing models by file name) via a pre-run options dialog on the
global, bulk and single-recipe entries
- L4 (filename-level) matches are listed in a results dialog with
per-entry undo
The .onboarding-target-highlight class sets position: relative, which
overrode .folder-sidebar's position: fixed (equal specificity, later
stylesheet). The sidebar left fixed positioning and moved in-flow, while
the spotlight/mask cutout stayed at the pre-highlight rect, leaving an
empty highlighted region during the folder sidebar step.
Whitespace cleanup in cleanupLoraSyntax() and the autocomplete blur
formatter collapsed all whitespace runs, including inside <lora:...>
tags. A file named 'test - 0021.safetensors' was rewritten to
'test - 0021' in the node text, so runtime file resolution failed.
Protect lora tags with placeholders (or segment splitting) so only
whitespace between entries is normalized; names inside tags are kept
byte-for-byte.
The field dates back to a development-stage bug in the enrich-metadata
(agent) pipeline, which briefly wrote trigger words at the top level of
model metadata instead of the established civitai.trainedWords location.
The write path was fixed before the feature merged to main (PR #1013)
and never shipped in any release, so no writer has existed since.
Remove the leftover pieces:
- BaseModelMetadata.trainedWords field (py/utils/models.py); sidecars
from that dev window now pass the key through _unknown_fields instead
- HF download handler's strip-empty-trainedWords special case, reverting
to saving the metadata object directly (py/routes/handlers/hf_handlers.py)
- trainedWords in the LLM enrichment context (agent_service.py)
- matching fallbacks/fixtures in the enrich_hf_validation harness and
post-processor test
Trigger words continue to live in civitai.trainedWords for all model
sources, which is what the UI, agent post-processor, and metadata sync
all read and write.
models.dev is served by Cloudflare with brotli compression when the
client advertises it, and brotli is a required dependency here, so
aiohttp always negotiates br. A corrupted br stream can crash the
native decoder with a Windows access violation (a Python-level
exception handler cannot catch it), or produce garbage bytes.
Send an explicit "Accept-Encoding: gzip, deflate" header on the model
catalog and Ollama model-list requests so the server never returns
brotli. zlib handles corrupt gzip data by raising ContentEncodingError
(an aiohttp.ClientError subclass), which the existing handlers already
catch and degrade to a warning with an empty-catalog fallback.
The public REST API rewrites files[].name to "{model}_{version}" for
non-LoRA model types, so every precision variant of a multi-file version
shared one name and landed on disk with a random short-hash suffix.
Fetch the raw stored filename from the model-versions/mini endpoint
(always pinned with modelFileId) and use it for the on-disk name and
metadata when available; fall back silently to the REST name otherwise.
CivArchive already serves raw names and is skipped.
Recipes imported from CivitAI image URLs can contain 0 LoRAs: the backend
only sees the REST image API + EXIF, while the complete generation data
lives in the image page's internal trpc payload (see
docs/recipe-civitai-image-no-metadata.md). When the companion
lm-civitai-extension is installed with a valid license, re-import (single
and bulk) of CivitAI-image-sourced recipes is now delegated to the
extension via DOM CustomEvents; the extension scrapes the image page with
the user's session and calls back into the reimport endpoint with the
full metadata payload. Without the extension (or with an invalid license)
the native path runs unchanged.
- POST /api/lm/recipe/{id}/reimport accepts optional payload params
(image_url/name/resources/gen_params/base_model/tags); the payload path
reuses the import-remote engine with reimport semantics (user-edit
carryover, delete-after-save), and malformed/failed payloads fall back
to the legacy URL import. Response gains loras_count.
- The endpoint also accepts GET: the extension is GET-only by convention
(documented in AGENTS.md).
- New static/js/utils/extensionReimportBridge.js (probeExtension /
delegateReimport / getCivitaiImageInfo) wired into RecipeContextMenu
and BulkManager with silent native fallback.
- i18n: toast.recipes.reimportingViaExtension added and translated in
all 9 locales.
Page-imported recipes can carry an exact CivitAI modelVersionId but no
modelId and no hash (CivitAI exposes no sha256 for e.g. Krea versions).
canDownloadLora() required (modelId && versionId) or a hash, so such
entries were misclassified as unrepairable and offered Reconnect instead
of Download.
- canDownloadLora: treat a bare version id as downloadable (it uniquely
pins the file; the model id is resolved on demand at download time).
A model id without an exact version id stays non-downloadable to avoid
silently grabbing the latest version.
- resolveLoraDownloadIdentifiers: when a hash is absent but a version id
exists, resolve the owning model id via /civitai/model/version/{id}
(same endpoint the bulk download missing flow uses). Hash-only and
direct (modelId+versionId) paths are unchanged.
resp.json() raises UnicodeDecodeError (not JSONDecodeError) when the
remote body contains invalid UTF-8 bytes, which the exception handler
did not catch and could crash the app. Apply the same fix to both
_load_model_catalog and fetch_ollama_models so they fall back to an
empty catalog. Add regression tests for both paths.
The recipe "Repair Metadata" action has been marked Deprecated in the UI
for a while and cannot reliably recover recipes imported from CivitAI URLs
whose REST meta has no resources/hashes and whose image has no embedded
metadata (e.g. CivitAI-only generation data). Drop the feature end to end.
Backend:
- remove repair routes (repair, cancel-repair, recipe/{id}/repair,
repair-bulk, repair-progress) and their handler mappings/methods
- remove RecipeScanner repair_all_recipes / repair_recipe_by_id /
_repair_single_recipe and REPAIR_VERSION
- remove WebSocketManager recipe-repair progress channel
- drop repair_version column from the persistent recipe cache
- rematch mutual-exclusion now only checks rematch
Frontend:
- remove repair entries from per-recipe, bulk and global context menus
- remove repairRecipe / repairSelectedRecipes / repairRecipes + cancelRepair
and the repairBulk API client method/endpoint
- drop recipe-repair i18n keys (synced across locales; doctor keys kept)
Tests/docs: delete test_recipe_repair.py, update scaffolding/routes/ws/
persistent-cache/integration tests and i18n guideline examples.
Commit 86aa1d80 added align-items: flex-end to .toast-container and
dropped the .toast min-width to 200px. With flex-end alignment each
toast now shrinks to its own content width, so toasts of different
message lengths render at inconsistent widths. Drop the align-items
override so the container falls back to stretch, giving every toast a
single shared width as before the change.
Adds a 'Keep Action Bar Visible' toggle (default off) under
Settings > Interface > Layout Settings. When enabled, the controls bar
(Refresh, Download, etc.) and the breadcrumb nav are wrapped in a shared
sticky container (.sticky-topbar) so both stay pinned as one unit; when
disabled, the wrapper is display: contents and the original behavior
(only the breadcrumb stays visible) is preserved.
Rebuilt from the preceding three commits' sources:
- active-filters chip and its dead settings-toggled broadcast removed
(the built bundle also no longer embeds the scripts/app.js test shim
that the chip's settings.js import used to pull in)
- scrollbar inset re-measured on programmatic value changes
- app/api bindings canonicalized to "../../../scripts/*" externals and
settings.js no longer inlined (bound at runtime via "../settings.js"
to the vanilla module instance), per the new build guard
Will Miao