512 Commits

Author SHA1 Message Date
Will Miao
75e63c758b feat(api): add cursor-based pagination to civitai user-models endpoint 2026-08-03 11:07:06 +08:00
Will Miao
dd6bdbf297 fix(update): persist update_channel via settings.json instead of hasGit
After b464fdc3 (preserve .git on release switch), the hasGit-based
channel detection is unreliable — .git now exists for both release
and nightly installs, so page refresh always reset the channel.

- Add _resolveChannelFromSettings() with migration heuristic:
  !hasGit → release (ZIP), detached HEAD → release (on tag),
  on branch → nightly. Uses gitInfo.branch from check-updates.
- Persist resolved channel to settings.json on first load
  (one-time migration) and on explicit switchChannel.
- Add update_channel validation (release|nightly) in backend
  update_settings handler.
- Remove hasGit-based guessing from initialize(); defer to
  checkForUpdates where full gitInfo is available.
- Channel resolution runs before checkForUpdates early-returns
  to avoid null channelMode on reload-within-interval.

Tests: 361 passed.
2026-07-31 13:23:54 +08:00
Will Miao
99e65cccd8 fix(update): downgrade settings backup/restore logs from INFO to DEBUG 2026-07-30 20:32:00 +08:00
Will Miao
b464fdc333 fix(update): preserve .git on release channel switch, use git checkout tag
Previously, switching to the release channel would delete .git/ and
fall back to a ZIP download. This broke update.bat, manual git
commands, and CM git-based update detection.

Now the release path uses git checkout <latest-tag> when .git exists,
and only falls back to ZIP when .git is absent (CM CNR installs).
.git is never deleted - the ZIP→nightly path remains a one-way
upgrade via _init_git_repo.

Also updates locale strings (en, zh-CN, zh-TW, ja) to remove the
now-inaccurate "remove the Git repository" wording.
2026-07-29 21:23:39 +08:00
Will Miao
53825500db fix(update): add staging protection to switch_channel
switch_channel has three destructive code paths (git reset + clean,
git init + checkout --force, and rmtree + ZIP replace) that were
missing the _stage_preserved_items / _restore_preserved_items safety
net already applied to perform_update.

Wrap the channel-specific logic in a try/finally so preserved user
data (settings.json, civitai/, cache/, etc.) is physically moved
outside plugin_root before any git operation and always restored.
2026-07-29 20:41:36 +08:00
Will Miao
f2ac790752 fix(update): stage preserved items outside repo before git/ZIP update
Move settings.json, civitai/, wildcards/, backups/, stats/, logs/,
cache/, and model_cache/ to a temp directory before git reset/clean
or ZIP replacement, then restore them in a try/finally block.

This prevents data loss on Windows where git clean -e exclusion
patterns can fail due to path-separator mismatches or where file
locks (open SQLite/log handles) cause the restore step to be skipped
on failure.

Also unifies three hardcoded skip lists (_clean_plugin_folder,
skip_items, skip_tracked) to derive from the single _PRESERVE_DIRS
constant, fixing drift where logs/ was missing from the ZIP path.
2026-07-29 19:49:50 +08:00
Will Miao
cfc4903c0c fix(update): read ahead_by from GitHub compare API when status is ahead/diverged
The compare API URL format compare/{local_hash}...main returns
status='ahead' when main is ahead of the local commit. The count is
in the ahead_by field, not behind_by. The old code only read behind_by
which is always 0 in this case, causing the UI to show 'Up to date'
when actually several commits behind.

Also handle status='diverged' (both sides have unique commits) by
reading ahead_by for the remote-ahead count.

Frontend adds a hash comparison fallback: if behind_by is 0 but local
and remote commit hashes differ, show 'Behind main' instead of the
incorrect 'Up to date'.

Tests: _AheadCompareDownloader and _DivergedCompareDownloader mocks
for the two status paths.
2026-07-28 17:47:38 +08:00
Will Miao
66d1c96783 feat(update): add Release/Nightly channel switching
- Add POST /api/lm/switch-channel endpoint with git init / ZIP fallback
- Add _backup_git/_restore_git helpers with safe rollback
- Version-info endpoint now returns has_git flag for auto-detection
- Check-updates always returns releases (changelog) regardless of channel
- Nightly mode shows 'N commits behind main' with commit hash and date
- View on GitHub link points to /commits/main in nightly mode
- Channel toggle UI with pill-style buttons in update modal
- Confirmation dialog with Esc / backdrop-dismiss support
- Channel derived from has_git on every page load, no localStorage
- i18n: 11 new keys translated across 9 non-English locales
- CSS: unified card-style sections in _base.css
- Tests: 8 new tests covering switch-channel, nightly response, init_git_repo
2026-07-27 20:27:05 +08:00
Will Miao
1de0a53241 feat(grouping): version-group library cards by HuggingFace repo for non-Civitai sources (#1040) 2026-07-26 21:46:55 +08:00
Will Miao
e6538c83bb fix(metadata): restore sha256 after hydrate_model_data to prevent KeyError in CivitAI fetch
hydrate_model_data replaces model_data with .metadata.json content which
may lack sha256 (corrupted file, concurrent write, etc.). Restore the
cached sha256 after hydration and persist the fix back to disk so
subsequent lookups don't hit the same error.

Also improve error log to include file_path for debugging.
2026-07-24 12:07:18 +08:00
Will Miao
f53f859a71 feat(filter): add debounced tag search with backend search-tags endpoint 2026-07-20 17:37:47 +08:00
Will Miao
a8d742b031 feat(metadata): add CivArchive API toggle and provider fallback order settings
- Add enable_civarchive_api toggle (default on) to allow disabling
  CivArchive to avoid its rate-limit windows entirely
- Add metadata_provider_order dropdown with two presets:
  CivitAI → CivArchive → Archive DB (default) and
  CivitAI → Archive DB → CivArchive
- Wire both settings through backend (metadata_service, settings_manager,
  misc_handlers) and frontend (SettingsManager, state, settings modal)
- Reorder Metadata section in settings modal: toggles → status/management
  → fallback order, for natural top-down workflow
- Make update_metadata_providers() log the effective provider chain
  using actually-registered providers rather than settings assumptions
- Add 5 test cases covering all provider-combination paths
- Complete i18n translations for 6 new keys across all 9 non-English locales
2026-07-19 17:51:50 +08:00
Will Miao
d15a8aa9a2 fix(workflow): accept non-string widget values and support GlobalSeed node in gen-params (#1026) 2026-07-18 22:57:20 +08:00
Will Miao
f0bf2728c9 fix(downloads): accept download_id in history delete/retry endpoints, add unique index 2026-07-18 21:10:42 +08:00
Will Miao
ffe65d983c feat(api): add GET endpoints for update-lora-code and update-node-widget
Add GET variants of the two POST endpoints used by the send-to-workflow
feature. Parameters are read from query string instead of JSON body,
supporting both simple repeated node_id params and JSON-encoded node_ids
for complex graph references.
2026-07-15 21:49:22 +08:00
Will Miao
419bbc90b2 feat(lora-info): add Lora Info display node
Add a pure frontend node that shows filename and editable notes for
a selected LoRA. Connect any output from a LoRA Loader/Stacker/Randomizer/
WanVideoSelect to the lora_source input — selecting a LoRA in the source
widget updates the info display automatically.

- Python node (LoraInfoLM): display-only, no workflow execution
- Vue widget: filename label, auto-sizing notes textarea, save button
  with ComfyUI toast feedback on save
- Frontend extension: wire-based selection propagation with stale-response
  race guard; clears display on wire disconnect
- Backend: get-notes endpoint now returns file_path alongside notes;
  matching supports full-path lora syntax; fix NoneType crash in
  trigger words endpoint; document cache file_name invariant
- Wired into all four lora widget nodes (Loader, Stacker, Randomizer,
  WanVideoSelect)
2026-07-14 18:00:31 +08:00
Will Miao
feab01cd9c fix(preview): hide license icons for models without CivitAI metadata 2026-07-13 19:49:10 +08:00
Will Miao
966024e534 fix(registry): force re-registration on WS refresh to prevent timeout, demote empty-registry log to debug
- workflow_registry.js: add force param to refreshRegistry(), bypass fingerprint
  dedup when responding to lora_registry_refresh WS message. Without this, the
  backend's wait_for_all() times out after 0.5s because the frontend skips the
  register-nodes POST when the workflow fingerprint hasn't changed (common after
  ComfyUI restart with an empty or unchanged workflow).
- misc_handlers.py: demote 'No nodes registered after refresh' from WARNING to
  DEBUG — empty workflows are a normal operational state, not a warning-worthy
  condition.
2026-07-13 19:10:48 +08:00
Will Miao
2018722cc8 fix(registry): handle compound subgraph node IDs, add proactive node push from graph hooks
- Handle compound node IDs (e.g. "252:0") from expanded group subgraphs
  to fix 400 Bad Request on workflows with group nodes
- Frontend proactively pushes node data via afterConfigureGraph and
  LiteGraph hooks (onNodeAdded/onNodeRemoved/graphChanged), eliminating
  WebSocket round-trip latency for most "Send to Workflow" operations
- Add content-fingerprint dedup to skip duplicate register-nodes POSTs
- Fast-path cache returns immediately when tabs are registered (including
  0-node registrations), avoiding unnecessary WS refresh cycles
- Distinguish "Empty Registry" from other errors in standalone UI toast
- Reduce WS refresh timeout 2s→0.5s, add cooldown and lock to prevent
  concurrent refresh storms
- All [LM:Registry] logs at DEBUG level
2026-07-13 18:02:26 +08:00
Will Miao
f362ed585b fix(preview): gracefully handle deleted preview files - image fallback, cache cleanup, quieter logs
- Add onerror handler on <img> previews to fallback to no-preview.png
- Fire async cache cleanup when preview file returns 404
- Add ModelCache.clear_preview_by_path() for safe stale-url removal
- Downgrade /api/lm/previews 404 log from warning to debug
2026-07-10 21:25:07 +08:00
Will Miao
316702b7ab fix(hf): allow subdirectory paths in HF resolve URLs, strip repo-internal dirs on save (#1019) 2026-07-09 09:18:38 +08:00
Will Miao
5d4a33c90d fix(hf): stop using realpath for download path construction, match CivitAI approach 2026-07-07 20:24:47 +08:00
Will Miao
041a6b8525 Revert "fix(hf): pass computed folder to _save_hf_metadata instead of re-deriving from paths"
This reverts commit 54b44131b6.
2026-07-07 20:13:20 +08:00
Will Miao
2638109ad6 feat(hf): add Link to HuggingFace feature with unified Link Model submenu
- Merge Relink to Civitai and new Link to HuggingFace into a single
  'Link Model' submenu with sub-options for each source
- Add POST /api/lm/set-hf-url endpoint to associate a model with a
  HuggingFace repo URL, saving hf_url to .metadata.json
- Add link_hf_modal.html for URL input, following relink-civitai pattern
- Use update_single_model_cache instead of add_model_to_cache to
  prevent duplicate cache entries after linking
- Remove os.path.realpath usage for consistency with relink-civitai
- Raise errors instead of silently falling back to LoRA scanner when
  model root cannot be determined
- Scope .input-group CSS rules to modal IDs to fix style conflicts
  with download-modal.css
- Add i18n keys across all 10 locales with translations for
  zh-CN, zh-TW, ja, ko, de, es, fr, he, ru
2026-07-07 20:04:47 +08:00
Will Miao
54b44131b6 fix(hf): pass computed folder to _save_hf_metadata instead of re-deriving from paths 2026-07-06 17:34:43 +08:00
Will Miao
a1d948025c fix(hf): strip empty trainedWords from metadata JSON to keep sidecar clean 2026-07-06 16:49:51 +08:00
pixelpaws
cb4ad27813 Merge pull request #1013 from willmiao/agent
Hugging Face model metadata AI enrichment
2026-07-06 12:21:19 +08:00
Will Miao
2373edf73c feat(ui): load provider model catalog asynchronously to avoid blocking page render 2026-07-06 10:02:09 +08:00
Will Miao
fecbe8241f fix(agent): use status= instead of status_code in json_response calls 2026-07-06 10:02:00 +08:00
Will Miao
51c0135250 refactor(agent): rename agent_cli to metadata_ops, strip temp debug logs
- Rename py/agent_cli/ -> py/metadata_ops/ (module was never agent-related)
- Rename tests/agent_cli/ -> tests/metadata_ops/
- Remove 9 low-value/debug INFO log points across agent_handlers.py,
  agent_service.py, llm_service.py, and metadata_ops/__init__.py
- Keep LLM raw response at DEBUG level for diagnostics
- Consolidate per-model progress + LLM result into single concise
  log line with basename instead of full path
- Update package/class/method docstrings to clarify this is a
  pipeline infrastructure, not a true agent loop
2026-07-05 18:00:58 +08:00
Will Miao
8bee8f4069 fix(recipe): fallback to locate custom example image on disk by model hash and image id (#1012) 2026-07-04 18:40:34 +08:00
Will Miao
905c37290f chore: update runtime logs to use 'LLM enrichment' instead of 'Agent skill'
- agent_handlers.py: 'Agent skill' -> 'LLM enrichment' in all log messages
- skill_registry.py: 'agent skills' -> 'prompt-based skills' in discovery log
- llm_service.py: docstring 'agent skills' -> 'LLM-based enrichment features'
2026-07-04 16:53:41 +08:00
Will Miao
4ed9169646 feat(ui): redesign AI Provider settings with provider presets and model catalog
- Replace hardcoded provider list with PROVIDER_PRESETS (OpenAI, Ollama,
  DeepSeek, Groq, OpenRouter, OpenCode Go, Custom)
- Load model lists from models.dev/api.json catalog at startup
- Add Combobox vanilla JS component for model/base-URL selection
- Fetch local Ollama models via live API instead of catalog
- Hide API key values from frontend (boolean-only llm_api_key_set)
- Add i18n translations for all 9+ locales
- Update snapshot tests for new response fields
2026-07-03 16:08:51 +08:00
Will Miao
cf898da193 feat(agent): add LLM-powered metadata enrichment system with AgentCLI and PostProcessor
Introduce an agent skill framework for LLM-driven metadata enrichment:

- AgentCLI (py/agent_cli/): in-process wrappers around internal services
  using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
  OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
  (replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales

Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
2026-07-02 21:27:01 +08:00
Will Miao
8b344ea39f feat(ui): add View on Hugging Face button, plumb hf_url through full cache pipeline 2026-07-01 08:38:16 +08:00
Will Miao
8348a0cef8 fix(download): harden HF download path validation, fix WebSocket leak, add URL detection tests (#965, #977)
Security hardening:
- Validate repo format with strict regex (reject .. traversal)
- Validate filename rejects path separators and ..
- Validate relative_path rejects absolute paths and ..
- Verify model_root is within configured scanner roots using
  realpath + os.sep guard to prevent prefix-match bypass
- Add realpath-based escape detection for final dest_path

Bug fixes:
- Fix WebSocket leak in _downloadHfSingle: wrap ws.close() in
  try/finally so it closes even if downloadHfModel() throws
- Same fix for batch HF download per-file WebSocket loop

Frontend hardening:
- Tighten HF repo regex: require huggingface.co for full URLs,
  reject bare .. patterns
- Add 12 unit tests for detectUrlType() covering HF resolve,
  HF repo, CivitAI, CivArchive, direct HTTP, edge cases
2026-07-01 05:51:58 +08:00
Will Miao
09ca91fc0e feat(download): add Hugging Face model download to standalone UI wizard (#965, #977)
Integrate HF model downloading into the existing CivitAI-style wizard flow:
- URL type detection (civitai / hf-resolve / hf-repo / direct-http)
- Repo file explorer with checkbox-based file selection
- Batch/queue download with per-file WebSocket progress
- Aria2 backend support (respects download_backend setting)
- Scanner cache integration via create_default_metadata + add_model_to_cache
- i18n updates for all 10 locales
2026-06-30 19:36:12 +08:00
Will Miao
16f5222efd fix(cache): prevent corrupted cache rows from breaking model listings (#730)
Cache corruption (NULL model_name/file_name from legacy DB rows or partial
writes) caused format_response to raise KeyError/AttributeError, failing the
entire /loras/list request and showing no models in the UI.

Fix across three layers:
- format_response (lora/checkpoint/embedding): replace direct dict[] access
  with .get() fallbacks; return None for entries missing file_path
- handlers: filter None entries from list/excluded/fetch/duplicate/conflict
  endpoints instead of letting them crash or appear as null in responses
- model_scanner: always use validate_batch repaired copies (previously
  discarded when no invalid entries, leaving None values in raw_data)
- persistent_model_cache: add or-empty-string guards on read and write for
  nullable TEXT columns (model_name, file_name, folder, base_model, etc.)
2026-06-30 09:02:42 +08:00
Will Miao
28f99c46d3 fix(update): preserve user data dirs during Git-based update via git clean -e excludes
git clean -fd in _perform_git_update deleted untracked, non-ignored
directories (wildcards, stats, backups, civitai, caches, logs) during
portable-mode updates, since released tags do not list them in .gitignore.
Add -e excludes for all user-managed paths to both nightly and stable
update branches. Add regression tests for both paths.
2026-06-29 21:10:38 +08:00
Will Miao
e9e8c31ad1 fix(registry): store nodes per-client to prevent multi-tab race condition
Move NodeRegistry from a single global _nodes dict to a per-client
(_tab_nodes) structure so that multiple ComfyUI browser tabs no
longer overwrite each other's workflow node data during a
lora_registry_refresh cycle.  The merged result is a union of all
known tabs' target nodes, eliminating the non-deterministic failure
where send-to-workflow could randomly target a tab lacking valid
targets.

- NodeRegistry.register_nodes(sid, nodes) replaces per-tab data
  without affecting other tabs.
- NodeRegistry.get_merged_registry() returns the union across all
  connected clients, together with tab_count / per-tab metadata.
- prepare_for_refresh() snapshots the current active sockets; caller
  re-reads before merging so that newly-connected tabs are not pruned.
- workflow_registry.js sends api.clientId in the POST body so the
  backend can identify which tab is registering.
2026-06-28 17:57:58 +08:00
Will Miao
283730cf38 fix(import): discover LoRA + checkpoint from modelVersionIds when API meta is null
When CivitAI image API returns meta=null and modelVersionIds at root
level, the import flow now:

- Injects modelVersionIds + browsingLevel into a minimal metadata dict
  so the parser can discover LoRAs and checkpoints (both import-from-url
  and analyze-image paths)
- Adds checkpoint dedup + fallback in the parser's modelVersionIds
  handler to avoid duplicate API calls
- Runs EXIF extraction unconditionally in analyze-image path, then
  merges with API metadata (fixes gen params loss)
- Propagates preview_nsfw_level through all three import paths:
  import-from-url, analyze-image (UI Import), and batch-import,
  plus the frontend save flow
2026-06-27 17:05:38 +08:00
Will Miao
85da7175bc feat: add Node Marker system with right-click marking 2026-06-23 20:54:32 +08:00
Will Miao
fc29cde82a feat(versions): add View all local versions button to model versions tab
Clicking the button closes the modal, writes filter params to sessionStorage,
and reloads the page to show all local versions of the model as individual
cards (bypassing group-by-model dedup). The filter respects the update flag
strategy and the versions-filter-toggle state (same-base vs all versions).

Supporting changes:
- sessionStorage keys vlm_model_id / vlm_model_name / vlm_base_model
- BaseModelApiClient._addModelSpecificParams adds civitai_model_id param
- LoraApiClient calls super._addModelSpecificParams for VLM detection
- LorasControls / CheckpointsControls clearCustomFilter checks VLM first
- PageControls.checkVlmFilter shows customFilterIndicator with label
- Backend parses civitai_model_id, filters before group_by_model dedup
2026-06-21 11:13:53 +08:00
Will Miao
559ca946dc feat(models): add group-by-model option to collapse multiple versions into one card
Adds a 'Group by Model' toggle in Layout Settings. When enabled, only the
latest version (highest civitai.id) of each Civitai model is shown as a
single card — older versions sharing the same modelId are hidden.

Backend dedup runs in BaseModelService.get_paginated_data() before
filtering/pagination, ensuring correct paginated results. The setting
is persisted via the existing settings pipeline and passed as a query
parameter to the listing endpoint.

Includes:
- Backend: dedup logic, route param parsing, settings default
- Frontend: API param, SettingsManager wiring, toggle UI
- i18n: translations for all 10 locales
- Tests: unit test covering dedup on/off and standalone items
2026-06-21 08:48:42 +08:00
Will Miao
cf0fd0e0ad feat(i18n): internationalize dynamic insights content with key/params architecture (#489) 2026-06-19 13:49:03 +08:00
Will Miao
ab6bb25d46 fix(example-images): skip hidden files in path validation, show offending items on failure (#807) 2026-06-19 11:54:55 +08:00
Will Miao
b24b1a7e57 feat(settings): hide API key from frontend, use status+edit instead of password field
Backend changes:
- Add civitai_api_key to _NO_SYNC_KEYS, return only boolean civitai_api_key_set
- Clean up known template placeholder on load to prevent false positive

Frontend changes:
- Replace type=password with type=text + CSS masking (-webkit-text-security)
- Replace pre-filled input with status display (Configured/Not configured)
- Add inline edit view with Save/Cancel buttons
- Re-add eye toggle via CSS class toggle (not type switching)
- Use CSS transitions for smooth status/edit view switching

This prevents Chromium/Vivaldi password manager from triggering
'save password' prompts when opening the settings modal.
2026-06-19 08:05:04 +08:00
Will Miao
9bbd26efe6 feat(license-icons): add second set of license icons matching current CivitAI design
- Add 5 new Tabler SVG icons (currency-dollar, brush, user, git-merge, license)
- Implement Set 2 rendering in ModelModal.js (standalone UI) with green/red
  permission indicators and preview_tooltip.js (ComfyUI widget)
- Add use_new_license_icons setting (default: true) with toggle in settings UI
- ComfyUI tooltip reads setting directly from preview-url API response to
  eliminate race conditions and respect standalone settings changes
- Remove the now-unused separate ComfyUI setting loramanager.license_icon_style
- Add CSS for both standalone (lora-modal.css) and widget (lm_styles.css)
- i18n: translate licenseIcons keys into all 10 supported languages
- Fix test to use classic style explicitly for continued coverage
2026-06-18 21:07:44 +08:00
Will Miao
b7721866e5 fix(stats): implement Model Types chart in Collection tab with correct type distribution 2026-06-18 06:48:46 +08:00
Will Miao
8314b9bedb feat(downloads): add /downloads/queue/status endpoint and integrate queue lifecycle
- New GET /api/lm/downloads/queue/status handler for non-terminal status
  transitions (queued -> downloading, downloading -> paused, etc.)
- Queue lifecycle auto-integration in DownloadManager._download_with_semaphore:
  downloading -> SQLite update_status('downloading') on semaphore acquire
  completed -> complete_download('completed') on success
  canceled -> complete_download('canceled') on CancelledError
  failed -> complete_download('failed') on Exception
- All queue operations wrapped in try/except to never break the download flow
2026-06-17 23:04:30 +08:00