`wait_for_slot` trusted a single `asyncio.sleep(delay)`. The event-loop timer can
fire before the delay elapsed -- Windows granularity is ~15ms, which the new
Windows job caught -- and the method then returned while the cooldown was still
armed, so the caller sent a request inside the window the gate exists to enforce.
It now re-checks the fixed `wake_at` and sleeps again, which also restores the
strict `elapsed >= 0.14` cooldown bounds the earlier relaxation had weakened (a
substring replace had turned two of them into `>= 0.09`), and a regression test
fails the gate if it ever returns on an early wake.
The last Windows failure was float noise on an upper bound: a 0.15s cooldown
window reports 0.15000000000000568 remaining, so the assertion missed by 6e-15.
The upper bound now carries an epsilon.
The two `elapsed >= 0.14` cooldown assertions had been weakened to `>= 0.09` by
a substring replace while relaxing the pacing bound -- that would have let a
missing wait pass. They are back at 0.12 (0.15s minus Windows timer rounding),
which still fails if the cooldown is not awaited.
Second pass over the informational Windows job (91 failures -> 11 -> 0):
* `_identity`/`_fold` now wrap the assertions my first sweep skipped: the
parenthesised and multi-line ones, and the ones whose token list was
incomplete (`base_models_roots`, `embeddings_roots`).
* The reconciler's `current_name` and `active_roots` carry the drive-prefixed
label on Windows ("C: loras"), so the last direct comparison goes through
`_root_label_names` too.
* `open_settings_location` reveals the settings *file* with
`explorer /select,` on Windows instead of opening the containing folder, so
that test asserts the platform's call instead of the POSIX one.
* Three more expectations are written in the business form the code returns
(metadata payload, extracted download paths, hydrated preview URL).
The new informational Windows job surfaced 91 failures: four real bugs (fixed in
the previous two commits) and a long tail of POSIX assumptions in the tests
themselves. None of them assert behaviour that differs on Windows, so they are
fixed rather than skipped wherever the behaviour is the same:
* Path spelling: expectations built with `str(tmp_path)` are compared through an
identity helper (`normalize_root_identity`) or the platform's separator, since
the code hands out forward-slashed business paths while `str()` is native.
* Root labels: `_root_display_labels` prefixes the drive on Windows ("C: loras"),
so label assertions compare the final path component or derive the expected
label from the same helper.
* `~` expansion reads USERPROFILE (ntpath), not HOME; timer granularity rounds
0.1s down; a case-only rename counts as "already named" via `os.path.normcase`.
* Genuinely POSIX-only strategies are skipped with a reason instead of failing:
the headless clipboard fallback, dbus file-manager selection, xdg-open, and
the case-fold index test that simulates a case-sensitive filesystem.
`with sqlite3.Connection` ends the transaction but keeps the file handle open,
so every cache class leaked one handle per instance until the garbage collector
happened to run. On Windows that makes the database file undeletable while the
process lives (WinError 32), which broke 30 cache tests in teardown and would
also block moving or clearing a cache at runtime.
All eight `with <connection>` sites now go through `closing_connection()`, and a
regression test asserts the schema init leaves no usable connection behind.
The layout is load-bearing for review: re-dumping a locale with a different
`indent` rewrites every line, so the diff hides the actual change. Assert that each
file still matches the canonical serialisation it is maintained in (`en.json`
indent=2, the translated locales indent=4, `ensure_ascii=False`, one trailing
newline), so the mistake fails in CI instead of in review.
Deleting a folder that holds models answered "A deletion is still pending inside
this folder. Wait for the undo window to expire." even though nothing was staged
and no window was running. `.lm-pending-delete` is only a container: the purge,
the undo and the staging rollback all removed the batch dir inside it and left the
container behind, and the folder manifest matched the directory *name* alone. Any
folder where a model had ever been deleted with the undo window was therefore
undeletable for good, and the same name match refused folder renames.
* `staging_dir_has_content()` is the shared answer to "is there still a batch
here?": an empty container is a leftover, a quarantined batch (`*.orphaned`) is
preserved data and still counts, and an unreadable directory counts too — never
delete over something that could not be listed.
* The container is removed with its last batch (purge, undo, staging rollback,
merged-loser cleanup). `rmdir` only ever succeeds on an empty directory, and the
path is checked to really be a `.lm-pending-delete`, so a model folder is never
at risk.
* A manifest-less batch dir with nothing in it is a crash leftover between
`makedirs` and the manifest write: it is dropped instead of quarantined, which
would have left an empty container that blocks deletion just the same.
Verified: purge, undo and the empty-manifest-less sweep all leave no container
behind, the container survives while another batch is staged, and an empty one no
longer blocks folder delete or rename while a real (or quarantined) batch still
does. 3818 backend tests pass.
Deleting a folder a file manager had already moved away answered "Folder no longer
exists" and stopped there: `remove_known_folder()` only ran after `shutil.rmtree()`
succeeded, so the records keeping the node in the tree could never be dropped, and
the modal's missing state hid its confirm button. The sidebar was a dead end for
exactly the case the folder list cannot self-heal.
* `ModelMoveService.forget_folder()` drops a folder's records (the recorded entry
plus the cached models under it) and refuses with `code: "exists"` while any root
still holds the directory, because then the caller wants a delete.
* `delete_folder()` forgets the records instead of failing when the directory is
already gone. The dry run stays a pure read and keeps answering `missing`, which
is what makes the dialog offer the removal. `POST`/`GET
/api/lm/{prefix}/forget-folder` is the endpoint for a node that maps to no
directory at all (the GET form is for the browser extension, which is GET-only).
* The sidebar falls back to it from the delete flow, offers "Remove from list" in
the missing state, and reports the folder as removed from the sidebar rather than
as a delete that never happened. An `exists` conflict says the directory is back
and should be deleted instead.
Verified: 3811 backend tests, 1543 JS tests, 96 Vue tests, 20 i18n tests.
`_reconcile_cache()` replaced the recorded folder list only when *nothing* was left
unverified: one switched-off drive, one directory `os.walk` could not enter, or one
symlink whose target was gone, and the whole list fell back to a union with what
the walk found. A folder moved with a file manager therefore stayed in the sidebar
for good — the walk kept adding the new location while the old entry was never
dropped, a restart re-hydrated the union from SQLite, and the ghost could not be
deleted in the UI either because its directory was gone.
* `_UnreachablePaths.covers_folder()` answers "could this specific entry have been
read?" for a library-relative folder: it is kept only while a directory the walk
failed to enter (or an offline symlink) may still hold it. A root that is
unreadable *as a whole* deliberately does not count: library-relative names carry
no root, so a missing drive would trivially cover every entry and no ghost could
ever be cleared. Its folders still show while they hold cached models
(`get_all_folders()` unions the model-derived folders) and a genuinely empty one
comes back the next time that root is walked.
* The merge keeps today's union for a scoped scan, and for a root-scoped scan that
does not cover every root: those cannot speak for the roots they were not asked
about. Asking for every root is a full refresh and cleans again.
Verified: a folder moved externally is dropped while another root is offline, the
offline root keeps its own folders and cached models, and a scoped scan still never
drops a folder of a root it did not scan. The Windows-only half of this (a
backslash `os.sep` making the configured twin shadow the live root, so deleted
models were never pruned) is now reproduced in CI by patching `os.sep` instead of
needing a Windows runner.
ComfyUI hands its model folders over in the host's own spelling --
`utils/extra_config.py` runs every configured path through `os.path.normpath()`,
which is all backslashes on Windows, and `main.py` registers `<output>/loras`
unconditionally for its save nodes. `Config` kept that spelling verbatim while
`_prepare_*_paths()` normalized the live root lists to forward slashes, and the
scanner compared the two as raw strings. On Windows every configured root
therefore looked like a root that was not in the live list: the Refresh menu
listed each directory twice (the twin showing "0 models"), the completion toast
claimed reachable roots were "currently inaccessible" and kept their whole cache
count as `kept_unreachable`, and the settings sync persisted host-registered
directories that were never created.
The duplication was not only cosmetic. `_UnreachablePaths.add()` normalizes the
prefix it is given, so the configured twin of a live root registered exactly the
live root's prefix as unreadable: in the prune phase every cached-but-missing
entry under it matched and was "kept", so models deleted from disk were never
removed from the cache again.
* `py/utils/root_paths.py` holds the single `normalize_root_identity()`
(separator style, redundant/trailing separators, and case where the filesystem
folds it: whole paths on Windows, drive letters everywhere). `config.py` and
`settings_manager.py` each carried a private copy and `model_scanner.py`
compared raw strings.
* `ModelScanner` compares root identities in `get_configured_model_roots()`,
`describe_model_roots()` (dedupe + `available`), the reconcile's
`configured_only` list and the `ReconcileScope.roots` membership test, so one
directory is one root whatever the host called it. `_normalized_root_prefix()`
is built on the same identity, and an empty prefix no longer matches every
path.
* A configured root that never made it into the live list is reported only while
it still holds cached entries -- in the menu and in `skipped_roots` /
`unavailable_paths` alike. A directory that neither exists nor was ever scanned
has nothing to keep, so `<output>/loras` stops being an unavailable root.
* The startup sync drops saved roots that neither exist nor hold cached entries
(`PersistentModelCache.roots_with_cached_entries()`, one scan per model type,
separator-agnostic and wildcard-escaped). A root named by the library's
`default_*_root` is kept even when empty: dropping it would let the next boot
"repair" the default onto another root. When the cache cannot be consulted,
every root is kept.
* `GET /scan?roots=` validates requested roots by identity too, so the browser
extension or a script passing the spelling settings.json holds selects the root
instead of answering 400 "Unknown model root(s)".
Verified: 8 of the new tests fail against the pre-fix source (checked with the
fix stashed); 3798 passed, 7 skipped, 11 deselected. The probe costs 71ms for a
100k-model library with seven missing roots (357ms when probing root by root).
Help tooltips were CSS pseudo-elements on the info icon, so they were
descendants of the scrollable settings pane (.settings-content uses
overflow-y: auto) and got clipped at its top edge. The Active Library
deprecation notice made this obvious: the text grew to ~10 lines while
only ~5 fit above the icon, so the tooltip started mid-sentence at
"page. Deprecated:".
Render the tooltip once into <body> with position: fixed instead, so no
overflow ancestor can clip it, and position it from JS: above the trigger
when it fits, below otherwise, clamped to the viewport. It follows the
trigger on scroll/resize and hides on mouseout, Escape or touch. The
delegated listeners also cover help icons rendered later by the settings
manager, and the box may now use 320px instead of 220px, which keeps long
help texts to fewer lines.
The "Migrate Sidecars Now" button reused the mode-change confirmation
copy, which told the user they could migrate later with the very button
they had just clicked. Add manual variants of the confirm messages
(confirmToCentralizedManual / confirmToAlongsideManual) that drop the
mode-change preamble and the circular backreference, and translate them
for all 9 locales.
- Surface Windows WinError 206 / ENAMETOOLONG during sidecar migration as
an actionable per-model error instead of a bare OSError
- Offer root relocation based on the resolved sidecar root, so moving off
(or back to) the default storage directory relocates existing sidecars
- Hint in the migration confirm dialog when the destination is the default
location, so a custom folder can be set before migrating
The example-images download loop marked a model as failed in
.download_progress.json whenever its metadata had no civitai images or
processing raised any exception. Since failed_models is a permanent skip
list for non-force runs (and excluded from the pending pre-check), models
whose metadata simply had not been fetched yet — or that hit a transient
error — were never retried. Users had to delete the progress file to
unstick them, and the blacklist re-grew afterwards.
- Distinguish 'metadata not fetched yet' (no civitai payload) from
confirmed absences (civitai entry present but image-less, or the model
is known to be gone from CivitAI via civitai_deleted/from_civitai).
Only confirmed absences are marked failed; the rest stay pending.
Retrying pending models only re-reads local metadata, so no extra
provider API calls are introduced.
- Transient processing errors no longer add the model to failed_models.
- Add unblock_failed_example_image_models() to remove hashes from the
persisted failed list (library-scoped and legacy locations) and from
the live download manager's in-memory state.
- Hook it into MetadataSyncService.update_model_metadata so a successful
metadata fetch/relink immediately re-enables example image downloads
for that model — no TTL polling needed.
Centralized sidecar storage moves .metadata.json into a mirror tree, so
the Location link in the model modal could no longer reveal a model's
sidecar. The Location row now shows an inline metadata icon button
(centralized mode only) that calls a new
POST /api/lm/models/open-sidecar-location endpoint: the backend resolves
the sidecar through py/utils/sidecar_paths, reveals the .metadata.json
selected in the file manager (falling back to the sidecar directory when
it has not been written yet), and returns the path for clipboard mode on
headless/Docker sessions. The platform reveal logic is factored into a
shared FileSystemHandler._reveal_file helper. All 9 locales translated.
open-file-location now asks the freedesktop org.freedesktop.FileManager1
ShowItems interface to reveal the file selected (GNOME Files, Dolphin,
Nemo, Caja, recent Thunar), falling back to plain xdg-open when the
interface is unavailable. Also align the Linux branch with the other
location handlers: headless sessions return clipboard mode instead of
silently failing.
trace_model_path dead-ended on KJNodes GetNode virtual links and fell back
to the first entry in MODELS, which could be a stale checkpoint name
resurrected from the process-lifetime node_cache — images ended up with a
wrong or missing Hashes.model.
- SetNodeExtractor now records MODEL passthrough as a model_variable entry
(variable name -> source node id), cached per node like other metadata
- trace_model_path resolves GetNode variable references through those
entries instead of giving up (last SetNode wins, per KJNodes semantics)
- _fill_missing_metadata validates cached checkpoint names against the
node's current widget values and rebuilds stale entries from the current
inputs instead of resurrecting the previous model
Warn at startup (log + dismissible startup message) when settings.json
contains more than one library, marking the feature as deprecated while
keeping it fully functional. Removal is planned for one of the next
releases, at which point all libraries' folder paths will be merged into
a single library. Multiple folder roots within one library remain fully
supported and unaffected.
- SettingsManager: add _warn_if_multi_library() invoked from
_migrate_to_library_registry
- Settings UI: label the Active Library selector as deprecated and note
the removal plan in its help text (en.json)
- Docs: add deprecation banners to multi_library_design.md and
library-switching.md
- Remove dead code: SettingsManager.update_active_library_paths had no
call sites
- Return the ResourceNotFoundError detail from get_civitai_versions and
get_model_versions_status instead of a bare "Model not found", and let
the download modal display the server-provided message
- Anchor extract_civitai_image_id / extractCivitaiImageId to
^/images/<id> so API paths and partial numeric IDs are rejected, same
as the model-page parsing fix in #1140
The pre-flight rate-limit gate checked civarchive.com for every download,
so a cooldown armed by bulk metadata fetches blocked plain CivitAI
downloads for the whole window. Only CivArchive-sourced downloads consult
civarchive.com now.
update_model_metadata ignored the save_metadata result, so a fetch on a
drive that went offline mid-run was counted as success: the cache got
the fresh CivitAI payload while the durable sidecar never landed, and
every later fetch skipped the model as already fetched.
The save result now raises MetadataPersistError. In the bulk fetch path
this fails the item (reported in the summary, cache untouched, model
stays eligible for the next run); relink surfaces an honest 500.
Also corrects the stale known-limitation note in the scoped-scan plan:
a startup-offline root has been listed and re-admitted since Wave 6.
Bulk metadata refresh picked candidates purely from cache state, so the
entries the scoped scan deliberately keeps for unreachable drives were
fetched anyway — burning CivitAI rate limit on sidecar/preview writes
that cannot land.
- Filter the fetch queue by root reachability (live check via
describe_model_roots), covering both startup-offline and mid-session
drive-off; probing failures degrade to no filtering
- Report offline_skipped / offline_roots in the WS terminal payloads,
the HTTP result and the completion message
- Show a "Drive Offline" stat card in the fetch summary modal when
any model was skipped this way
The default recipes dir (<first lora root>/recipes) is auto-created by the
recipe scanner and recorded by the model-root walks as an empty folder, so
it showed up in the folder sidebar — and since recipe files are not model
weight files, the folder delete endpoint's model_count check let a
right-click delete wipe the whole recipe library via shutil.rmtree.
- scan walks (full scan, reconcile, all-folders backfill) no longer descend
into the effective recipes dir, and get_all_folders() filters it out of
snapshots persisted before the exclusion existed
- delete_folder/rename_folder refuse the recipes dir and any ancestor of
it with code "protected"
- the delete modal renders a protected state (confirm hidden, per-row
checkbox disabled) instead of a deletable 'no models' prediction
- RecipeScanner.recipes_dir now shares the side-effect-free resolver in
py/utils/recipes_paths.py
`Config._dedupe_existing_paths()` drops roots whose directory does not exist
when the root list is built, so a drive that is switched off while LM starts was
not "an offline root" — it was not a root at all. The Refresh ▾ menu had no row
for it in either state, a full refresh reported nothing (its cached entries
survived only because they fall outside every configured prefix), and plugging
the drive back in changed nothing until the next restart.
* `Config` now records the configured (existence-unfiltered) paths per model
type — from ComfyUI's list in plugin mode, from the active library snapshot in
standalone mode, where the host mock filters non-existent paths itself — and
exposes them through `configured_roots_for()`.
* `ModelScanner.describe_model_roots()` appends configured-but-unavailable roots
with a live `reachable`, an `available` flag and their cached entry count, so
`/roots` (and therefore the menu) offers a row in both states: greyed while the
directory is missing, normal and clickable once it is back.
* `_reconcile_cache()` reports them as `skipped_roots` / `unavailable_paths` with
reason `root_unavailable` and counts their cached entries in `kept_unreachable`,
which is what makes the "N models kept" toast appear in the startup-offline
case. Report-only: they stay outside the scan scope, so nothing about pruning
changes. A root-scoped scan stays quiet about roots it was not asked about, and
a folder-scoped scan only mentions a missing root that holds cached entries of
that folder.
* `Config.admit_configured_roots()` re-runs the per-type prepare helpers against
the configured paths and APPENDS what is readable now (plus the
checkpoint/unet/other side maps), then refreshes the preview allowlist, so a
drive plugged in mid-session can be scanned without restarting. Append-only and
order-preserving by design: removing a root mid-session would let a later
settings save persist the loss, and re-sorting would move `*_roots[0]`, which
derives the recipes directory and the usage-stats file location. `_roots[0]`
never moves.
* `/roots` and `/scan` admit first, so `describe_model_roots()` stays a pure read
and a caller that never opens the menu (the browser extension) still gets a
drive that came back.
Verified on the sandbox with drive-Z switched off before startup: `/roots`
reports `drive-Z reachable=false available=false models=60`, a full refresh
returns `skipped_roots=[drive-Z root_unavailable]`,
`unavailable_paths=[{... kept: 60}]`, `kept_unreachable=60`, and all 432 models
stay cached. Renaming the drive back while the server runs admits it
(`available=true`) and `GET /scan?roots=<drive-Z>` walks it with no restart.
3714 passed, 7 skipped; frontend 1495 passed (149 files); vue widgets 96 passed.
Starting ComfyUI with an external drive switched off erased that drive's path
from `settings.json`. `Config._dedupe_existing_paths()` drops paths that do not
exist at the moment the root list is built, and `save_folder_paths_to_settings()`
(plugin mode, called from `Config.__init__`) persisted `list(self.loras_roots)`
through `upsert_library(folder_paths=...)`, which REPLACES the library's paths.
One boot with the drive off was therefore enough to lose the configuration, not
just to hide the root — the same applied to checkpoints/unet/embeddings and the
other-model keys.
* `Config` now records the *configured* (existence-unfiltered) primary paths per
model type while building the live lists, and the startup sync persists those.
A path the host (plugin mode) or the library (standalone mode) no longer
configures is still dropped, so removing a path keeps working.
* In standalone mode the host mock (`MockFolderPaths.get_folder_paths`) already
filters non-existent paths out of settings.json, so the active library snapshot
is the only record of what the user configured; `_remember_library_configured_paths()`
reads it there and leaves plugin mode to ComfyUI's own unfiltered list.
* `_resolve_valid_default_root()` receives the configured paths in
`allowed_paths`, so a `default_*_root` on a switched-off drive is no longer
"repaired" onto another root.
* `_append_new_paths()` is the shared append-only union (order preserved, case
insensitive) used by the sync.
Verified: a new config test builds the plugin-mode sync with one switched-off
drive, one readable root and one path the host dropped — the switched-off path
and the default root that points at it survive, the dropped path does not, and
the live `loras_roots` still holds only the readable root. tests/config 64 passed.
The Refresh menu can scope a scan to one model root, but the folder a user is
looking at lives in the sidebar's unified tree, which merges every root into one
relative-path namespace. "Scan this folder" therefore addresses the folder, not a
root: the backend walks that relative path under every root that holds it, which
is also what makes the action safe while another drive is switched off.
Backend:
* GET /scan accepts `folder=<rel>` (alone or with `roots=`), rejected together
with full_rebuild=true like the roots parameter. Validation reuses
normalize_relative_folder(), extracted to module scope from ModelMoveService so
the folder operations and the scan endpoint reject the same input (absolute
paths, drive letters, `..` climbing) instead of each carrying its own copy.
* The reconcile summary carries `scope_label` (the folder) for a folder scope, so
the result toast names the folder the user clicked instead of the roots it
happens to live under; the completed WS payload carries it too.
* `folder` is a scope prefix exactly like a root: only that subtree is re-read or
pruned, and an unreachable root keeps the entries that fall inside it.
Frontend:
* The sidebar folder context menu gains "Scan this folder" above "Check for
updates in this folder" (they share the refresh divider); the entry is gated by
the same supportsFolderManagement flag as the other folder operations.
* SidebarManager.scanFolder() resolves the node through the existing
_resolveFolderCandidates() before doing anything: a folder no root holds any
more explains itself ("no longer exists on disk") instead of scanning nothing,
and an unresolvable multi-root node is refused rather than guessed.
* PageControls.refreshModels() and BaseModelApi.refreshModels() forward the folder
scope, and _showRefreshSummary() prefers scope_label over the walked roots.
Verified live on the three-root sandbox with one drive switched off:
GET /scan?folder=pack000 walks drive-G and drive-Y, reports scope_label=pack000,
keeps drive-Z's 6 entries under that folder (kept_unreachable=6) and leaves all
420 models cached. 3704 passed, 7 skipped; frontend 1495 passed (150 files); vue
widgets 96 passed. The 2 new sidebar keys are [TODO: Translate] placeholders
pending the feature owner's go-ahead.
Refreshing had no way to say "scan only this drive": a user with three external
drives had to spin all of them up for every refresh, and switching a drive off
made the next refresh treat its whole library as deleted (rows pruned from the
memory cache and the SQLite cache, preview_url stripped on the next scroll).
Backend (py/services/model_scanner.py, py/config.py):
* ReconcileScope(roots, folder) + _reconcile_cache(scope=...): files inside the
scope reconcile normally, everything outside is neither re-read nor removed.
The folder half is plumbing for the sidebar entry in the next change.
* Path-level pruning guard: cached entries under a path this walk could not
read are kept and reported instead of removed. Sources: a configured root that
is not reachable (drive switched off while LM runs), a directory os.walk
failed to enter (permissions / I/O error / Windows junction to an offline
drive), and a known first-level symlink whose target is gone
(Config.iter_path_mappings()).
* The recorded folder list is unioned instead of replaced whenever the scan did
not verify every root, so a scoped scan cannot empty the sidebar.
* _reconcile_cache returns a summary (added / removed / repaired /
scanned_roots / skipped_roots / unavailable_paths / kept_unreachable),
exposed as ModelScanner.last_reconcile_summary, returned by
BaseModelService.scan_models() and broadcast in the completed WS payload.
* _root_display_labels(): set-aware labels ("G: loras", "usb/loras") grown
leftwards with real parent segments until unique, shared by the walk-progress
line and the roots API.
* GET /scan accepts repeated `roots` (400 for unknown roots, 400 combined with
full_rebuild=true); GET /roots gains root_details (label / reachable / cached
count) while `roots` stays a plain path list for existing callers.
* serve_preview: a 404 no longer clears the cached preview_url when the file's
own directory is unreachable - browsing the grid with a drive off used to
strip preview references from the persistent cache.
Frontend:
* Refresh ▾ gains a "Scan one folder" section listing the page's roots with
their cached counts; offline roots stay clickable and explain themselves; rows
are wired by delegation (new static/js/components/controls/ScanScopeMenu.js).
* A scoped scan reports "Scanned <root>: N new, M removed"; a scan that kept
entries reports "<N> models kept: <paths> not reachable".
* registerAPI() now injects the two cross-page passthroughs (fetchModelRoots and
an argument-forwarding refreshModels) so a page facade cannot drop them: the
first version rendered an empty menu and would have run a full refresh.
* createToastElement whitelists toast types, so a wrong `type` argument degrades
to the info style instead of rendering an unstyled box.
Verified in a sandbox instance with three roots: a scoped scan walks only the
requested root (progress roots=0/1, 240 files); a full refresh with one root
offline reports kept_unreachable=60 and leaves all 420 models cached; /roots
reports the offline root with its cached count. 3699 passed, 7 skipped;
frontend 1488 passed (148 files); vue widgets 96 passed.
The sidebar's folder tree merges every model root into one relative-path
namespace, but folder operations turned a node into a path by prefixing
default_*_root. A folder living under another root failed to delete with
"Folder no longer exists" (recipes under the primary lora root while
default_lora_root is the extra one), and where the same relative folder
exists in both roots the operation silently hit the other copy — 14 of the
16 top-level folders in the reporting library are shared, so guessing a root
was never safe.
Backend:
* ModelMoveService.resolve_folder() and GET /api/lm/{prefix}/resolve-folder
answer which directories a library-relative folder maps to
(folder_path/root/is_symlink), in scanner root order, skipping directories
no root holds and refusing absolute or climbing paths.
* delete_folder/rename_folder tag a vanished directory with code "missing"
so the sidebar can tell "this node is stale, refresh" from a failed
operation.
Frontend:
* _resolveFolderCandidates() is the single place that turns a node into
absolute paths: default root first, the old root-prefix fallback only
while a single root is configured, and an explicit unresolved error for a
multi-root library — nothing is guessed silently any more.
* One copy keeps the single-target modal, which now names the resolved
absolute path. Several copies render one checkbox row per copy, each
dry-run against the delete guard ("no models" / "contains N model
file(s)..." / a deletion is still pending / no longer exists / symbolic
link): a blocked copy is unticked, disabled and explained, the button
reads "Delete N folders", every ticked copy is deleted and guarded on its
own, and a partial failure is reported without discarding the successes.
* Rows are built once per open and only their status text is updated, so
ticking a box no longer rebuilds the list, steals focus or resizes the
modal mid-click; the action row keeps a fixed button width.
* Rename offers a root picker in its inline row, create inherits the
parent's root when the parent resolves to exactly one directory, and the
undo restores every copy a delete removed.
i18n: 26 new keys (sidebar.deleteFolderModal.*, .deleteFolderResult.*,
.renameFolderResult.*, .folderRoot.*, .folderResult.*) translated in all 9
locales, with the en wording normalized to the established "model root" noun
(it had said "library root") and the new terminology recorded in
docs/i18n-translation-guidelines.md.
Verified: pytest 3686 passed, vitest 1473 passed (86 in the folder-management
suite), pytest tests/i18n 20 passed, sync_translation_keys.py --dry-run
clean. A sandboxed standalone instance with two roots confirmed that deleting
one copy leaves the node in place, that the twin's model card survives the
purge, and that deleting both copies and undoing restores both directories.
The recorded folder list is a union over the model roots keyed by relative
path, but remove_known_folder() dropped an entry unconditionally. Deleting
<rootA>/test in the sidebar therefore hid a "test" that <rootB> still held:
the node disappeared from the next tree load and came back after the next
scan, which reads as "the delete did not work". The same call purged cache
entries by relative folder, so removing an empty <rootA>/test2 also evicted
the model cards of <rootB>/test2 until the next scan, and
rename_known_folder() re-keyed both the recorded folder and the "folder"
field of models that never moved.
* _folders_present_on_disk() answers "which of these relative folders does
some root still hold?" with stats off the event loop (model roots can live
on slow network shares) and reports nothing for stand-in scanners without
roots, which preserves their previous behaviour.
* remove_known_folder(folder, absolute_path=None) keeps the entries another
root still owns and purges by the removed directory's absolute path when
the caller knows it. Without a path the relative-folder filter stays as the
documented fallback: it prunes the entry (disk-verified either way) but
cannot tell same-named copies apart.
* rename_known_folder() re-adds the survivors of the old name and only
touches cache entries whose file_path sits inside the renamed directory.
Tests: the two delete tests now remove the directory from disk first, which
is what the contract always assumed, plus three new scanner tests (a twin
keeps the entry, the entry goes once no root holds it, the twin's cards
survive the purge), one for the legacy fallback and one for a rename that
leaves a twin behind.
A regular Refresh walked every configured root sequentially, so a full
25 TB drive delayed the roots behind it, and the dialog sat on "Checking
for changes..." at 0 % for the whole walk with no way to tell it was
working. On a cold external drive the walk itself dominates the cost, so
the fix is to overlap the drives and to show what the walk is doing.
Backend (py/services/model_scanner.py):
* Extract the per-root walk into the synchronous _walk_root_for_reconcile()
worker and merge its results on the event loop afterwards, in configured
root order: which business path wins a file reachable through several
roots must not depend on the order the workers happened to finish in.
* Group roots by device (_root_device_key: drive letter on Windows, st_dev
on POSIX) and run one worker per device. Roots sharing a device stay
sequential, so directory claims and the overlap dedup (#871, #1041) keep
their configured-order semantics; different devices run in parallel.
* Track walk progress per root (_ReconcileWalkTracker), weighted by each
root's cached entry count because the real file count is only known once
the walk ends. The bar splits into walk (0-50 %) and new-file (50-99 %)
phases so it never jumps backwards, and the walk broadcasts files seen,
active roots and the ETA counters.
* Replace the Windows case-insensitive fallback -- a scan of every cached
path per miss, i.e. O(files x cached) -- with a lazily built lower-cased
index (_CachedPathLookups, also now guarding the realpath alias index for
worker threads), and lift the os.name == "nt" gate into the module-level
_CASE_INSENSITIVE_PATHS so the branch is testable off Windows.
* Excluded-model membership is a set lookup instead of a list scan.
Frontend:
* render the walk phase as "Checking for changes... <roots> (N files)" with
the ETA, and reset the ETA tracker when the stage changes: the per-file
rate of counting files says nothing about processing them.
* add common.scanProgress.walkFiles; the other locales keep the sanctioned
[TODO: Translate] placeholder until the translation pass.
Verified: no-change reconcile over 10k files/500 dirs 101 ms and 10k/5000
dirs 230 ms (was 93/229 ms, within noise); a two-device sandbox walk runs
both roots concurrently, names them in the progress messages and finishes
with added=15, removed=0; 3665 passed, 7 skipped; frontend 1456 passed,
vue widgets 96 passed.
The Prompt and Lora Stack Combiner nodes expose unbounded dynamic input
slots (trigger_wordsN / lora_stackN). They resolved them by having
INPUT_TYPES() return a custom lookup object, but only when the caller was
ComfyUI's get_input_info() -- detected with inspect.stack(). That frame
inspection is what the registry security scan reports as
python_anti_debugging under the obfuscated-code admin tag.
Make the lookup a dict subclass instead, so INPUT_TYPES() can always
return it:
* /object_info (server.py) json.dumps INPUT_TYPES() directly, and a
dict subclass serializes its stored entries -- byte-identical to the
plain dict that was returned before.
* input_order (list(value.keys())), validate_inputs'
set(class_inputs["optional"]) and every other iteration still see only
the static slots.
* get_input_info() (graph.py) keeps resolving dynamic names through the
overridden __contains__/__getitem__, which no longer depends on who
the caller is.
The one behaviour change is in execution.py:get_input_data -- a dynamic
input passed as a constant rather than a link now reaches the node
instead of being silently dropped. These inputs are declared forceInput,
so the frontend only offers links; where it can happen the new behaviour
is the intended one.
Verified against ComfyUI's own consumer code: json.dumps output, keys(),
set(optional) and get_input_info() lookups all match the old behaviour,
and the two nodes no longer cross-resolve each other's slots.
3657 passed, 7 skipped.
Bring back the pre-v1.2.2 classic vertical list for example images as a
persisted showcase_layout setting (gallery stays the default), switchable
from both a Settings select and an in-modal segmented toggle.
- Revive the vertical list renderer (adapted from 4a6042d0) as
VerticalListView.js, reusing the shared MediaUtils/MetadataPanel
infrastructure; legacy CSS scoped under .showcase-vertical
- Dispatch showcase rendering on state.settings.showcase_layout;
collapsed bar, empty/filtered states and import flow stay shared
- Keep the back-to-top button available in vertical mode (the
showcase-expanded class that hides it now only applies to the gallery,
whose thumbnail strip occupies that corner)
- Persist the setting via DEFAULT_SETTINGS + DEFAULT_SETTINGS_BASE and a
new select under Settings > Layout Settings
- i18n: 6 new keys translated in all 9 locales; terminology recorded in
docs/i18n-translation-guidelines.md
Closes#1136
The owner could not tell from the UI what "Buzz Price Tracking" enabled, what the
"Price alert threshold" number meant, or what "Price Alerts" was alerting about.
That was not a copy problem: the implementation exposed our mechanism (a page
scrape) and our SQL predicates as the user's concepts. Two concrete defects came
from the same root:
- the alert population included versions the user already owns (neither the event
generator nor the query filtered on is_in_library; in the owner's library 28 of
52 gated versions were already downloaded, so most "alerts" were about files
already on disk, which cannot become cheaper *for them*);
- a threshold-filtered state list lived in a notification surface, so an empty
panel had three indistinguishable causes and read as a broken feature.
The information model is now the version plus ownership: cost is shown only where
a decision exists. Owned -> nothing. Not owned and free -> nothing. Not owned and
gated -> the price when it is known, `Paid` without a number when it is not, and
early access keeps its countdown because "free on <date>" decides between waiting
and paying. The numeric threshold has no place in that model: every decision is
categorical (wait / pay / skip), so the setting, the comparison and the whole
alert-state machine are gone.
- both alert-state columns are removed from the schema rather than left dead; a
database created by an unreleased build has them dropped on open (native
ALTER TABLE ... DROP COLUMN, guarded), which is a no-op for everyone else
- gate events are emitted only for versions the user does not have, and the
price-drop event goes with the threshold it belonged to
- both alert endpoints, PriceAlertsHandler and the service-registry adapter field
it needed are removed: events already reach the UI through the refresh response
- the bell tab, panel, CSS, both entry points, the unread watermark and their
locales are removed; the setting keeps only the enable flag and the refresh
interval and is framed as plumbing
- "Price unavailable" is replaced by `Paid`: the gate is certain from the public
API, only the number is best-effort, and that is our plumbing, not the user's
problem
Verified against a copy of the owner's real database: 52 gated versions ->
28 owned (now silent) + 24 the feature is actually about; the drop migration ran
and both removed endpoints 404.
The default threshold is 0 ("only tell me when a version becomes free") and the
settings copy says so, but the panel did not: a real instance with 52 priced paid
versions and an untouched threshold showed "Nothing is under your price threshold
right now" with only a small "Alert threshold: 0 Buzz" in the corner, which reads
as a broken feature.
- the payload now carries pricedCount, so the empty state can say how many paid
versions already have a known price
- when the threshold is 0 and prices are known, the empty state says so and
points at Settings - Library instead of implying there is nothing to show
- the read-time threshold comparison means setting one takes effect immediately;
measured on a copy of that instance: 0 Buzz -> 0 alerts, 100 -> 44,
500 -> 48, 5000 -> 51
Found in a real instance: after enabling price tracking, a normal "Check updates"
captured exactly one price out of 718 models, so the alerts panel looked broken
while the log said the refresh completed.
Price capture only ran when the version list was re-fetched, so it inherited the
metadata TTL: with 24 h metadata and 24 h price TTLs, only the handful of models
whose metadata happened to be stale that round were ever priced.
- the cached record already carries the gate, so the price pass now runs off
whichever version list is available (freshly fetched or stored) and applies the
result without touching last_checked_at, so a price-only pass cannot silently
extend the metadata TTL
- a failed attempt now satisfies the price TTL, so a mature model whose page no
host will serve is not retried on every single update check
- an explicitly forced check re-prices within the TTL
Verified by copying a real instance's update DB into a sandbox and running a
non-forced check: bulk metadata fetches 0 (version lists entirely from cache)
while priced versions went 1 -> 20 and the panel listed 19 alerts.
End-to-end verification against the live site found the price capture broken for
a whole class of users: the civitai page hosts are not interchangeable, and the
user's civitai_host preference was silently fatal. With civitai_host=civitai.red
the update DB held zero prices even with tracking enabled.
- civitai.red refuses non-browser HTTP clients outright (Cloudflare challenge,
403 for any User-Agent, aiohttp and httpx alike), while civitai.com and
civitai.green answer normally for anonymously visible models and 404 for
mature ones. An earlier manual check with curl passed on TLS fingerprint luck,
which is why this was missed.
- get_model_prices now tries the configured host first, then the others, and
takes the first parseable payload. The host that worked is remembered, and a
host that refuses outright is parked for 15 minutes so a library full of
mature models does not pay three requests each; a 404 is model-specific and
does not park the host. Links keep using the configured host, which is where
the user's own browser has clearance.
- Mature models still have no price source anywhere, so that is now stated
instead of silent: price_check_attempted_at separates "tried and unreadable"
from "never looked", gated versions show a muted "Price unavailable" badge,
and the alerts panel reports unavailableCount.
- Failures are logged at warning level, once per host per TTL, with the
per-host reason, instead of only at debug level.
- The recorded alternatives (internal tRPC with the user's API key, or an
extension-assisted fetch from the user's browser) and the strengthened
upstream ask for a public price field are documented in the plan.
P5a of docs/plans/paid-model-price-tracking.md: one surface that answers "what
got cheaper / became free", without a permanent button (the grid filter was
dropped by owner decision, so the panel carries the actions itself).
- price_alert_since records when an alert started, so the panel can say
"dropped 3 d ago" and count what is new since the user last looked; it is set
on the first sight of an already-cheap version, preserved while the alert
stands, and cleared when the price rises back above the threshold
- get_price_alerts() compares the threshold at read time (editing it takes
effect immediately, no refresh needed) and returns both kinds in one list;
model_type=None covers every type, which the shared update DB makes a single
query. "became free" needs no price data, so it is reported even while price
tracking is off
- GET /api/lm/price-alerts, registered once in MiscRoutes rather than per model
type, decorating rows best-effort with the local model name and file path from
the scanner indexes (a cold cache just omits them)
- a third tab in the notification bell: segments for under-threshold and
became-free, the three states (tracking off / nothing matching / stale), and
per-row actions (CivitAI always, Open when the model is local)
- two non-permanent entry points share one helper: the controls-bar updates
dropdown and the global context menu, whose label carries the unread count
- unread state stays client-side (localStorage watermark); the count is fetched
once on init and only when price tracking is enabled
- the per-type frontend client method is removed as dead code; the per-type
backend route stays for the companion extension
CivitAI's public API deliberately omits prices — paidAccess is trimmed to
{permanent, endsAt} because "pricing belongs to the purchase flow" — but the
public model page embeds the site's own model.getById result, including
paidAccess.terms, in its server-rendered payload. That is read anonymously
(no API key, no internal endpoint, no forged Origin), one request per gated
model, so only the ~2% of models that actually carry a gate pay for it.
- optional capture, off by default: price_tracking_enabled,
price_alert_threshold_buzz (0 = alert on "became free" only) and
price_check_ttl_hours; prices refresh on their own TTL and immediately when a
gate changes, and a failed fetch keeps the stored price instead of blanking it
- versions that stop carrying a gate are marked free (persisted gate_lapsed_at)
and gate transitions are reported as events on the refresh response, so a
version already in the library can announce that it became free
- price_alert_state plus a price_drop edge event; new
GET /api/lm/{type}/updates/price-alerts lists what is under the threshold
- versions tab shows the price (effective, with the list price struck through
and a Blue Buzz note) and a Free Now badge; an update check toasts the
transitions in one message
- the parser and the alerts query are unit-tested against a trimmed page
fixture, and every route definition is now asserted to resolve to a handler
Plan, verification notes and the deviations from it are in
docs/plans/paid-model-price-tracking.md.
CivitAI only returns a non-null paidAccess for an *active* gate: a lapsed gate
stays in the database as a tombstone and is filtered out server-side, so
{"permanent": false, "endsAt": null} — a timed gate whose window end has not
been recorded yet — is still enforced. Verified live: on model 1802980 that
version reports canDownload: false while its lapsed siblings report true.
Both the update service and the download gate dropped that shape, so such
versions read as free and "Hide Early Access Updates" missed them — the class
of bug reported in #1060.
The interpretation now lives in py/utils/paid_access.py and is shared, so the
badge, the update filter and the download warning cannot disagree.
Add OpenModelDB (openmodeldb.info) as a metadata and download source for
the existing upscaler model type.
Metadata:
- New OpenModelDBClient: fetches the site's bulk JSON dumps, caches them
on disk (24h TTL + ETag revalidation), and builds a local sha256 index
- New OpenModelDBModelMetadataProvider adapts catalogue entries to the
CivitAI-shaped version dict contract; registered in the fallback chain
behind the enable_openmodeldb_api setting (default on), gated to the
upscaler sub-type so other model types never trigger the dump download
- Persisted provenance uses metadata_source "openmodeldb" plus a nested
openmodeldb block (page URL, architecture, scale, license)
Images: paired-image LR/SR URLs are ephemeral imgdiff.net sessions, so
displayable images come from the site-hosted auto-generated thumbnails
(model-level cover leads images[], per-image thumbs for the rest); the
original comparison URL is kept in meta.comparisonUrl.
Downloads:
- New OpenModelDBSource (flat model ids, omdb: group prefix) with
resource filename derivation that recovers names hidden mid-path
(mediafire) or synthesizes {id}.{type} for folder links
- HTML-gateway mirrors (mediafire/mega/drive) are rejected with a clear
manual-download hint instead of silently saving an HTML page as .pth
- ModelSource base gains is_valid_source_id / default_subdir_parts /
resolve_download_url hooks so flat-id sources need no platform branches
UI: "View on OpenModelDB" link in the model modal (downloaded and
hash-enriched models), settings toggle next to the CivArchive one.
Applying a filename template to a large library re-did O(library) work for
every renamed file: a full natsort resort plus whole-table SQLite rewrite and
download-history resync after each rename, and a full scan plus resort of the
entire recipe collection per renamed LoRA. On a 20k-model library with 300k
recipes on a HDD this pushed "Apply to Library" into multi-day runs.
- ModelScanner.defer_cache_persist(): bulk loops update the in-memory entry
and indexes only; resort + persist + download-history sync run once at
context exit, forced even on cancellation/error since files are already
renamed on disk. Single-rename callers keep immediate per-call behavior.
- RecipeScanner.build_lora_hash_index(): one-shot hash -> recipes index so
per-file lookups are O(1); update_lora_filename_by_hash gains hash_index /
defer_maintenance params, with a single finalize_bulk_filename_updates()
resort at the end of a bulk session.
- ModelLifecycleService.bulk_rename_session() / BulkRenameContext wire the
deferred path through rename_model (hash index built lazily on first
recipe-touching rename).
- Blocking os.rename sequence offloaded via asyncio.to_thread so one file's
HDD I/O no longer stalls the event loop (no cross-file parallelism).
- Skip logic, per-batch WebSocket progress, cancellation, and result
counters unchanged.
- Model modal hash footnote now shows Civitai model id and version id
(right-aligned, quick-copy buttons); hidden for non-Civitai models
- Hash/id exact search (sha256/autov2/autov3/civitai ids) is now always
on: the search-options "hash" toggle is removed and the search_hash
query param is silently ignored for API compatibility
- Footnote render condition relaxed so autov3-only and id-only models
still show the line
- i18n: 4 new keys translated in all 9 locales; filters.hash key removed
Will Miao