When using full path lora syntax, the context menu (single/bulk)
and bulk copy actions were passing only the file basename to
buildLoraSyntax(), ignoring the folder prefix. This caused the
output to look like legacy A1111 format even when full path mode
was enabled.
Aligns all entry points with ModelCard.handleSendToWorkflow(),
which correctly includes the folder prefix.
Also fixes selectAllVisibleModels() to cache the folder field,
preventing missing prefix on select-all-then-send flows.
Add GET variants of the two POST endpoints used by the send-to-workflow
feature. Parameters are read from query string instead of JSON body,
supporting both simple repeated node_id params and JSON-encoded node_ids
for complex graph references.
In add_to_queue, check download_history before INSERT OR IGNORE. Without
this check, a fire-and-forget /queue/complete failure on the extension side
would allow the same download_id to be re-inserted after complete_download()
deleted it from the queue — creating phantom queued entries for already-
finished downloads.
Add a pure frontend node that shows filename and editable notes for
a selected LoRA. Connect any output from a LoRA Loader/Stacker/Randomizer/
WanVideoSelect to the lora_source input — selecting a LoRA in the source
widget updates the info display automatically.
- Python node (LoraInfoLM): display-only, no workflow execution
- Vue widget: filename label, auto-sizing notes textarea, save button
with ComfyUI toast feedback on save
- Frontend extension: wire-based selection propagation with stale-response
race guard; clears display on wire disconnect
- Backend: get-notes endpoint now returns file_path alongside notes;
matching supports full-path lora syntax; fix NoneType crash in
trigger words endpoint; document cache file_name invariant
- Wired into all four lora widget nodes (Loader, Stacker, Randomizer,
WanVideoSelect)
- workflow_registry.js: add force param to refreshRegistry(), bypass fingerprint
dedup when responding to lora_registry_refresh WS message. Without this, the
backend's wait_for_all() times out after 0.5s because the frontend skips the
register-nodes POST when the workflow fingerprint hasn't changed (common after
ComfyUI restart with an empty or unchanged workflow).
- misc_handlers.py: demote 'No nodes registered after refresh' from WARNING to
DEBUG — empty workflows are a normal operational state, not a warning-worthy
condition.
- Handle compound node IDs (e.g. "252:0") from expanded group subgraphs
to fix 400 Bad Request on workflows with group nodes
- Frontend proactively pushes node data via afterConfigureGraph and
LiteGraph hooks (onNodeAdded/onNodeRemoved/graphChanged), eliminating
WebSocket round-trip latency for most "Send to Workflow" operations
- Add content-fingerprint dedup to skip duplicate register-nodes POSTs
- Fast-path cache returns immediately when tabs are registered (including
0-node registrations), avoiding unnecessary WS refresh cycles
- Distinguish "Empty Registry" from other errors in standalone UI toast
- Reduce WS refresh timeout 2s→0.5s, add cooldown and lock to prevent
concurrent refresh storms
- All [LM:Registry] logs at DEBUG level
Changes:
- Backend: _validate_folder_paths() now checks checkpoints↔unet overlap
within the same library using os.path.realpath() for symlink resolution
- Backend: set() calls _validate_folder_paths() for both folder_paths and
extra_folder_paths before writing
- Backend: extracted _normalize_path_set() helper to eliminate duplicated
normalization logic
- Frontend: inline error display with red border + error message below the
conflicting input, no save triggered
- Frontend: path normalization (strip trailing slash, lowercase) in pre-check
to reduce false negatives vs backend realpath
- Frontend: asymmetric error UX — message only on the user-edited side,
red border on the pre-existing conflict side
- CSS: has-error styles with hardcoded rgba fallback for older browsers
- i18n: checkpointUnetOverlap + checkpointUnetOverlapInline keys added to
all 10 locale files
Remove dynamic height calculation that auto-resized the node when
LoRAs are added or removed. The widget now stays at the size the user
sets via the node resize handle, scrolling when content overflows.
- Drop updateWidgetHeight() and hardcoded entry-count height math
- Set --comfy-widget-min-height once (200px) instead of recalculating
- In Vue mode: add contain:layout+size to break the ResizeObserver
feedback loop that forced node growth with content (CSS via
.lm-loras-container.lm-vue-node scoped to vueNodesMode only)
- Remove unused "Node 2.0: Maximum visible LoRA entries" setting
- Bulk refresh filter now excludes models with hf_url
- Individual refresh for HF models only checks CivitAI API
- CivArchive client validates model IDs before querying
- Add onerror handler on <img> previews to fallback to no-preview.png
- Fire async cache cleanup when preview file returns 404
- Add ModelCache.clear_preview_by_path() for safe stale-url removal
- Downgrade /api/lm/previews 404 log from warning to debug
- Merge Relink to Civitai and new Link to HuggingFace into a single
'Link Model' submenu with sub-options for each source
- Add POST /api/lm/set-hf-url endpoint to associate a model with a
HuggingFace repo URL, saving hf_url to .metadata.json
- Add link_hf_modal.html for URL input, following relink-civitai pattern
- Use update_single_model_cache instead of add_model_to_cache to
prevent duplicate cache entries after linking
- Remove os.path.realpath usage for consistency with relink-civitai
- Raise errors instead of silently falling back to LoRA scanner when
model root cannot be determined
- Scope .input-group CSS rules to modal IDs to fix style conflicts
with download-modal.css
- Add i18n keys across all 10 locales with translations for
zh-CN, zh-TW, ja, ko, de, es, fr, he, ru
When Civitai returns 429 (Too Many Requests) during example image
downloads, the previous behavior treated all failures identically and
permanently removed the corresponding images from model metadata —
making them impossible to retry.
This commit adds:
- 429 detection + Retry-After header parsing in download_to_memory
- Exponential backoff retry (up to 3 attempts) in
download_model_images_with_tracking
- Separate tracking of rate-limited vs permanently failed URLs
- rate_limited_models progress tracking persisted to disk
- Rate-limited models are NOT added to failed_models/processed_models
so they are automatically retried on subsequent download runs
- Force mode clears failed_models when rate-limited images exist
- Parse limit.output from model catalog alongside model IDs
for per-model max output token limits
- Use catalog lookup in chat_completion_json() to set max_tokens;
fall back to 4096 for unknown models (e.g. local Ollama)
- Remove the JSON retry (response_format → plain text fallback);
keep _try_salvage_json as last-resort for truncated responses
- Reduce Ollama num_ctx from 32768 to 8192 (sufficient for
metadata enrichment, saves VRAM)
- Fix stale test comment referencing removed retry
Remove tests/enrich_hf_validation/baselines/ from git tracking
(.gitignore entry + git rm --cached). These contain README snapshots
from community HF repos that may include NSFW/sensitive content.
Local files are preserved on disk for offline reference.
Commit 9a0d866b changed _strip_fenced_code_blocks to preserve bash/shell
code blocks (they carry CLI setup and trigger-word metadata signal).
Update the two affected tests to expect bash content in the output
instead of asserting it is stripped.
- Rename test_bash_code_block_stripped → test_bash_code_block_preserved
- Update assertions: expect 'pip install' in result
The bare call inside _build_prompt_context
would raise NameError because class methods don't close over class-level
scope. Use instead to trigger attribute lookup.
Update enrich_hf_metadata prompt.md clue locations for better LLM accuracy.
Update baseline report to v2 (mean 69.0, 46 models, +2.2pp vs baseline 71.1%).
Consolidate README snapshots into baselines/readmes/.
_Previous_ _find_scanner_for_model and identify_model_type contained ~25 lines
of identical scanner-iteration + path-matching logic. Factor it into
_find_model_entry() so a new scanner type or edge-case fix can't drift apart.
- Rename py/agent_cli/ -> py/metadata_ops/ (module was never agent-related)
- Rename tests/agent_cli/ -> tests/metadata_ops/
- Remove 9 low-value/debug INFO log points across agent_handlers.py,
agent_service.py, llm_service.py, and metadata_ops/__init__.py
- Keep LLM raw response at DEBUG level for diagnostics
- Consolidate per-model progress + LLM result into single concise
log line with basename instead of full path
- Update package/class/method docstrings to clarify this is a
pipeline infrastructure, not a true agent loop
Three-part fix for enrich_hf_metadata failing to extract correct preview_url
from HuggingFace collection repos where models share flat heading levels:
1. _strip_standalone_images() now converts <img> tags to markdown image
syntax  instead of stripping the URL entirely, so the LLM
can still extract preview URLs.
2. _extract_section() uses a line-count-based forward window (stopping at
<a id> anchors) for non-heading matches, instead of stopping at the
very next heading. This prevents same-level sub-headings (# Download,
# Trigger, # Sample prompt within a single model section) from
truncating the window before sample images are included.
3. Post-processor preview fallback now filters gallery images to the
model-specific README section before falling back to the repo-wide
first image.
Move the HF model list from ~/Documents/ into tests/enrich_hf_validation/test_data/
and commit the pipeline validation baseline artifacts (report.json,
preprocessing_audit.json, README snapshots) into baselines/.
Update config.py and run_validation.py defaults to use repo-relative paths
via os.path.dirname(__file__) instead of ~/Documents/ hardcode.
Originates from changes in 8fb00998 (validation pipeline audit).
- agent_service._format_base_models: output bullet list instead of
JSON array for cleaner LLM parsing
- prompt.md mapping section: replace 14-row HF→CivitAI table with
compact rule set covering 14 mapping paths including new entries
for HiDream-ai, OnomaAIResearch/Illustrious, ideogram-ai/ideogram,
Tongyi-MAI/Z-Image-Turbo, and Wan-AI/Wan2.*
- base_model extraction instruction: add guidance to infer from
model filename, YAML tags, and README body text when YAML
frontmatter has no explicit base_model:
- Rename md_to_html.py → readme_processor.py (file no longer just HTML conversion)
- _extract_section: include YAML frontmatter, use heading-level-aware forward
walk (sub-headings under # are included), increase walk limit past 30 lines
- _is_heading: exclude </hN> closing tags from boundary detection
- _heading_level: new helper for heading-level-aware section matching
- css: yield 0 for heading like closing tags, was unexpectedly caught by _is_heading
- extract_gallery_images: fix YAML block scalar (text: >-) prompt extraction;
use endswith instead of == to detect the block marker
- _strip_widget_section: add to clean_readme_for_llm (widget text is handled
by post-processor, not needed in LLM prompt)
- _strip_standalone_images: keep markdown image URLs intact for LLM preview
extraction (was stripping to alt text only)
- list_base_models: switch from scanner-cache aggregation to
CivitaiBaseModelService.get_base_models() - always returns full list
- Ollama: add num_ctx=32768 to payload options so thinking models have room
to both reason and produce output
- Add tests/agent_cli/test_readme_processor.py: 59 tests covering extraction,
cleaning, section matching, heading detection
- Update existing tests for behavioral changes
- PostProcessor returns updates dict from enrich_hf_metadata
- AgentService includes updated_data per model in WebSocket progress events
- Convert preview_url to HTTP URL via config.get_preview_static_url()
- LoraContextMenu: showEnhancedProgress + updateSingleItem per model
- BulkContextMenu: same pattern, remove window.location.reload()
- Guard empty updated_data and clean up callbacks on HTTP error
- Add clean_readme_for_llm() to strip noise from README before LLM injection
- Keep widget section text (valuable tag signal) and unmarked code blocks (trigger words)
- Preserve standalone image alt text instead of removing entirely
- Switch Ollama to native /api/chat with think:false to fix empty content on thinking models
- Extract Sample Gallery table images and deduplicate with widget images
- Only strip code blocks with explicit language tags (bash)
- Add notes and usage_tips fields to SKILL.md output format and post-processor
- Clean up dead code, fix regex edge cases, remove double type annotation
- Replace hardcoded provider list with PROVIDER_PRESETS (OpenAI, Ollama,
DeepSeek, Groq, OpenRouter, OpenCode Go, Custom)
- Load model lists from models.dev/api.json catalog at startup
- Add Combobox vanilla JS component for model/base-URL selection
- Fetch local Ollama models via live API instead of catalog
- Hide API key values from frontend (boolean-only llm_api_key_set)
- Add i18n translations for all 9+ locales
- Update snapshot tests for new response fields
Widget entries with unquoted multi-line YAML scalars (e.g. "text: two samurais...\n continuation") were not parsed, leaving gallery image prompts empty. Add a third branch for plain scalar format alongside the existing quoted and >- folded block handlers.
- Add extract_gallery_images() to parse YAML widget entries from README
frontmatter, convert relative image URLs to absolute HF URLs, and
build civitai.images-compatible entries with prompt metadata
- LLM now extracts recommended_width/recommended_height from README
(e.g. "Best Dimensions"), used as gallery image dimensions
- extract_gallery_images() accepts default_width/height parameters,
falling back to 512x512 when LLM provides no recommendation
- Frontend ShowcaseView.js: defensive NaN guard for 0 width/height
- post_processor: consistently merge civitai updates across triggers,
description, and gallery blocks with distinct variable names
- SKILL.md: add recommended_width/recommended_height to output schema
- 62 tests pass, including gallery extraction and dimension tests
- Add identify_model_type() helper to determine lora/checkpoint/embedding
- Pass priority_tags from user settings to LLM prompt for tag relevance
- SKILL.md: instruct LLM to exclude technical/generic HF tags, cross-reference
against priority_tags; forbid ['None'] placeholder for trigger words
- post_processor: fix preview_url not updated after download (now writes local
.webp path to metadata); write trigger words to civitai.trainedWords instead
of top-level; sanitize ['None']/'null'/'n/a' placeholder values to []
- download_preview() now returns str | None (local path) instead of bool
- Update tests for new return type and nested civitai.trainedWords structure
Merge skill.yaml (metadata) and prompt.md (prompt template) into a
single SKILL.md file with YAML frontmatter, matching the agent-skill
convention used by opencode and Claude Code.
- Add frontmatter parser (_parse_skill_file) to SkillRegistry
- Remove skill.yaml, prompt.md, empty skills/__init__.py
- Remove obsolete load_handler method
- Update tests for new format and cleaned-up fields
Introduce an agent skill framework for LLM-driven metadata enrichment:
- AgentCLI (py/agent_cli/): in-process wrappers around internal services
using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
(replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales
Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
Extract auto-newline-on-paste logic into shared setupAutoNewlineOnPaste() utility in uiHelpers.js.
Apply it to both the Download modal (modelUrl) and Batch Import modal (batchUrlInput)
textarea, so users can paste multiple URLs in succession without manually pressing Enter.
Replace native <select> with a searchable dropdown that:
- Filters options as the user types
- Shows filename-inferred suggestions at the top in a "Suggested" section
- Supports keyboard navigation (ArrowUp/Down/Enter/Escape)
- Allows typing custom values not in the list
- Removes dead .base-model-selector CSS
Adds 3 new i18n keys (baseModelSearchPlaceholder, baseModelSuggested,
baseModelNoMatch) with translations for all 9 locales.
Security hardening:
- Validate repo format with strict regex (reject .. traversal)
- Validate filename rejects path separators and ..
- Validate relative_path rejects absolute paths and ..
- Verify model_root is within configured scanner roots using
realpath + os.sep guard to prevent prefix-match bypass
- Add realpath-based escape detection for final dest_path
Bug fixes:
- Fix WebSocket leak in _downloadHfSingle: wrap ws.close() in
try/finally so it closes even if downloadHfModel() throws
- Same fix for batch HF download per-file WebSocket loop
Frontend hardening:
- Tighten HF repo regex: require huggingface.co for full URLs,
reject bare .. patterns
- Add 12 unit tests for detectUrlType() covering HF resolve,
HF repo, CivitAI, CivArchive, direct HTTP, edge cases
- Unify single-URL and multi-URL HF repo flows to use the same batch
preview interface (remove separate repoFileStep)
- Remove unnecessary cloud icon from HF batch preview items
- Use formatFileSize() instead of hardcoded MB text
- Change default selection to unchecked (no preselected files)
- Add select all / deselect all checkbox with dynamic Next button
- Clean up dead CSS, HTML template, and JS methods from removed
repoFileStep
- Add selectAll i18n key with translations for all 10 locales
- Fix batch progress bar name fallback for HF items
A model not being found on CivArchive by hash is a routine case (the
model simply isn't published there), not an error. The callers already
log the outcome at WARNING (bulk_metadata_refresh) or DEBUG
(metadata_sync_service) with full context, making this ERROR-level log
both misleading and redundant.
Cache corruption (NULL model_name/file_name from legacy DB rows or partial
writes) caused format_response to raise KeyError/AttributeError, failing the
entire /loras/list request and showing no models in the UI.
Fix across three layers:
- format_response (lora/checkpoint/embedding): replace direct dict[] access
with .get() fallbacks; return None for entries missing file_path
- handlers: filter None entries from list/excluded/fetch/duplicate/conflict
endpoints instead of letting them crash or appear as null in responses
- model_scanner: always use validate_batch repaired copies (previously
discarded when no invalid entries, leaving None values in raw_data)
- persistent_model_cache: add or-empty-string guards on read and write for
nullable TEXT columns (model_name, file_name, folder, base_model, etc.)
git clean -fd in _perform_git_update deleted untracked, non-ignored
directories (wildcards, stats, backups, civitai, caches, logs) during
portable-mode updates, since released tags do not list them in .gitignore.
Add -e excludes for all user-managed paths to both nightly and stable
update branches. Add regression tests for both paths.
Move NodeRegistry from a single global _nodes dict to a per-client
(_tab_nodes) structure so that multiple ComfyUI browser tabs no
longer overwrite each other's workflow node data during a
lora_registry_refresh cycle. The merged result is a union of all
known tabs' target nodes, eliminating the non-deterministic failure
where send-to-workflow could randomly target a tab lacking valid
targets.
- NodeRegistry.register_nodes(sid, nodes) replaces per-tab data
without affecting other tabs.
- NodeRegistry.get_merged_registry() returns the union across all
connected clients, together with tab_count / per-tab metadata.
- prepare_for_refresh() snapshots the current active sockets; caller
re-reads before merging so that newly-connected tabs are not pruned.
- workflow_registry.js sends api.clientId in the POST body so the
backend can identify which tab is registering.
- Add &withMeta=true to image info URL so API returns full generation
metadata (resources with hash/type) instead of null meta
- Fix checkpoint assignment guard: check modelId instead of id so non-
checkpoint types (upscaler) are not wrongly set as recipe checkpoint
- Skip modelVersionIds loop when resources/civitaiResources already
provided LoRAs, preventing hash-resolved duplicates
- Fix int/str type comparison in CivArchive get_model_version so
version ID matching works correctly
When CivitAI image API returns meta=null and modelVersionIds at root
level, the import flow now:
- Injects modelVersionIds + browsingLevel into a minimal metadata dict
so the parser can discover LoRAs and checkpoints (both import-from-url
and analyze-image paths)
- Adds checkpoint dedup + fallback in the parser's modelVersionIds
handler to avoid duplicate API calls
- Runs EXIF extraction unconditionally in analyze-image path, then
merges with API metadata (fixes gen params loss)
- Propagates preview_nsfw_level through all three import paths:
import-from-url, analyze-image (UI Import), and batch-import,
plus the frontend save flow
- Prefer file type (UNet/Diffusion Model) over baseModel name when
deciding whether a checkpoint routes to the unet folder
- Add UNet to backend primary file type whitelist
- Add Krea 2 to DIFFUSION_MODEL_BASE_MODELS
- Include UNet/Diffusion Model files in frontend file selection UI
- Use actual file type from CivitAI in download params instead of
hardcoded 'Model'
- Convert marquee selection from viewport to document coordinates so
scrolling during a drag no longer deselects off-screen cards.
- Add RAF-based auto-scroll when dragging near viewport edges.
- Compute off-screen card positions from VirtualScroller layout
parameters instead of relying on DOM queries.
The document-level click handler in SortDropdown.js called trigger.focus()
unconditionally on every click outside the sort group. When a model card
was clicked to open the modal, focus() triggered scrollIntoView on the
.sort-trigger button, perturbing .page-content.scrollTop and causing the
card grid to jump up a few pixels.
The same interference also broke the back-to-top smooth-scroll animation:
frame-by-frame focus/scroll perturbations caused VirtualScroller to
schedule repeated re-renders, interrupting the compositor-thread scroll.
Fix: only return focus to the trigger when the dropdown was actually open,
so ordinary page clicks (e.g. clicking a model card) never force focus.
When refreshing updates with a folder filter, versions already present in
other folders were excluded from the is_in_library check, making them
appear as available updates. When the user tried to download, the global
check found the file already exists and returned 'model already exists'.
Fix by also collecting the cross-folder version set when folder_path is
provided, and using the union (folder-filtered + cross-folder) for
is_in_library in both _build_record_from_remote and
_merge_with_local_versions.
The back-to-top button used scrollTo({top:0, behavior:'smooth'}) which
conflicts with VirtualScroller's DOM manipulations during the smooth
scroll animation. Each animation frame triggered handleScroll() ->
scheduleRender() -> renderItems(), causing the browser to interrupt
the smooth scroll animation mid-way, resulting in only ~1 page of
upward scroll instead of reaching the top.
Root cause: commit 311e89e9 fixed VirtualScroller to listen on the
correct scroll container (.page-content), but this meant every scroll
event during smooth animation now triggers expensive DOM operations
that abort the browser's compositor-thread smooth scroll animation.
Fix: use instant scroll (scrollTop = 0) so the position is set
immediately without triggering frame-by-frame VirtualScroller
interference.
_drain_stderr and _wait_until_ready both read from the same stderr pipe.
Starting the drain task before _wait_until_ready creates a race where the
drain task consumes aria2's early-exit error message before the startup
waiter can read it, resulting in an empty error message in the logs.
Also confirmed that --fsync does not exist as an aria2 option (exit code
28 = Invalid argument).
Exit code 28 (Invalid argument) indicates this user's aria2c does not
support the --fsync option. Remove it unconditionally; the stderr drain,
relaxed RPC timeouts, and increased retry coverage remain in place.
Will Miao