Commit Graph

77 Commits

Author SHA1 Message Date
Will Miao 879745da53 fix(init): add missing /api/lm/init-status endpoint used by polling fallback
initialization.js falls back to polling /api/lm/init-status when the
/ws/init-progress WebSocket cannot be established, but no route ever
registered that path — each poll 404'd and the page never reloaded after
the scan completed. Report the aggregate status of all four scanners and
omit pageType so every initialization page accepts the update.
2026-08-24 14:17:06 +08:00
Will Miao 259e08e47c feat(download): expose per-file downloadedFiles in check-model-exists (#1058)
The version branch of check-model-exists now returns
downloadedFiles: [{fileId, fileName, filePath}] so clients (e.g. the
browser extension) can tell a partially downloaded version apart from a
fully downloaded one. Reuses ModelCivitaiHandler._match_downloaded_files
(D2 rule) against the local cache; unmatchable local files are reported
with fileId: None. No CivitAI API call added.
2026-08-20 20:59:19 +08:00
Will Miao e7c26bf722 feat(download): per-file download status and multi-file selection (#1058) 2026-08-19 17:51:31 +08:00
Will Miao 3e1216e9bc feat(recipes): add cache version counter to model scanners 2026-08-08 21:57:36 +08:00
Will Miao 8e724538bd fix(types): resolve pre-existing basedpyright errors in py/ and standalone.py
Fix ~950 basedpyright errors across the backend:
- Convert ineffective # type: ignore comments to # pyright: ignore[rule]
- Add missing generic type arguments (Dict[str, Any], list[Any], ...)
- Annotate dynamic dict literals and runtime-initialized attributes
- Widen CivitAI provider tuple signatures in recipe parsers
- Remove dead LoraRoutes handlers calling nonexistent LoraService methods
- Suppress unavoidable ServiceRegistry import cycles (basedpyright counts
  function-local imports as cycle edges)
2026-08-08 20:12:52 +08:00
Will Miao 75e63c758b feat(api): add cursor-based pagination to civitai user-models endpoint 2026-08-03 11:07:06 +08:00
Will Miao dd6bdbf297 fix(update): persist update_channel via settings.json instead of hasGit
After b464fdc3 (preserve .git on release switch), the hasGit-based
channel detection is unreliable — .git now exists for both release
and nightly installs, so page refresh always reset the channel.

- Add _resolveChannelFromSettings() with migration heuristic:
  !hasGit → release (ZIP), detached HEAD → release (on tag),
  on branch → nightly. Uses gitInfo.branch from check-updates.
- Persist resolved channel to settings.json on first load
  (one-time migration) and on explicit switchChannel.
- Add update_channel validation (release|nightly) in backend
  update_settings handler.
- Remove hasGit-based guessing from initialize(); defer to
  checkForUpdates where full gitInfo is available.
- Channel resolution runs before checkForUpdates early-returns
  to avoid null channelMode on reload-within-interval.

Tests: 361 passed.
2026-07-31 13:23:54 +08:00
Will Miao a8d742b031 feat(metadata): add CivArchive API toggle and provider fallback order settings
- Add enable_civarchive_api toggle (default on) to allow disabling
  CivArchive to avoid its rate-limit windows entirely
- Add metadata_provider_order dropdown with two presets:
  CivitAI → CivArchive → Archive DB (default) and
  CivitAI → Archive DB → CivArchive
- Wire both settings through backend (metadata_service, settings_manager,
  misc_handlers) and frontend (SettingsManager, state, settings modal)
- Reorder Metadata section in settings modal: toggles → status/management
  → fallback order, for natural top-down workflow
- Make update_metadata_providers() log the effective provider chain
  using actually-registered providers rather than settings assumptions
- Add 5 test cases covering all provider-combination paths
- Complete i18n translations for 6 new keys across all 9 non-English locales
2026-07-19 17:51:50 +08:00
Will Miao d15a8aa9a2 fix(workflow): accept non-string widget values and support GlobalSeed node in gen-params (#1026) 2026-07-18 22:57:20 +08:00
Will Miao ffe65d983c feat(api): add GET endpoints for update-lora-code and update-node-widget
Add GET variants of the two POST endpoints used by the send-to-workflow
feature. Parameters are read from query string instead of JSON body,
supporting both simple repeated node_id params and JSON-encoded node_ids
for complex graph references.
2026-07-15 21:49:22 +08:00
Will Miao 966024e534 fix(registry): force re-registration on WS refresh to prevent timeout, demote empty-registry log to debug
- workflow_registry.js: add force param to refreshRegistry(), bypass fingerprint
  dedup when responding to lora_registry_refresh WS message. Without this, the
  backend's wait_for_all() times out after 0.5s because the frontend skips the
  register-nodes POST when the workflow fingerprint hasn't changed (common after
  ComfyUI restart with an empty or unchanged workflow).
- misc_handlers.py: demote 'No nodes registered after refresh' from WARNING to
  DEBUG — empty workflows are a normal operational state, not a warning-worthy
  condition.
2026-07-13 19:10:48 +08:00
Will Miao 2018722cc8 fix(registry): handle compound subgraph node IDs, add proactive node push from graph hooks
- Handle compound node IDs (e.g. "252:0") from expanded group subgraphs
  to fix 400 Bad Request on workflows with group nodes
- Frontend proactively pushes node data via afterConfigureGraph and
  LiteGraph hooks (onNodeAdded/onNodeRemoved/graphChanged), eliminating
  WebSocket round-trip latency for most "Send to Workflow" operations
- Add content-fingerprint dedup to skip duplicate register-nodes POSTs
- Fast-path cache returns immediately when tabs are registered (including
  0-node registrations), avoiding unnecessary WS refresh cycles
- Distinguish "Empty Registry" from other errors in standalone UI toast
- Reduce WS refresh timeout 2s→0.5s, add cooldown and lock to prevent
  concurrent refresh storms
- All [LM:Registry] logs at DEBUG level
2026-07-13 18:02:26 +08:00
Will Miao 2638109ad6 feat(hf): add Link to HuggingFace feature with unified Link Model submenu
- Merge Relink to Civitai and new Link to HuggingFace into a single
  'Link Model' submenu with sub-options for each source
- Add POST /api/lm/set-hf-url endpoint to associate a model with a
  HuggingFace repo URL, saving hf_url to .metadata.json
- Add link_hf_modal.html for URL input, following relink-civitai pattern
- Use update_single_model_cache instead of add_model_to_cache to
  prevent duplicate cache entries after linking
- Remove os.path.realpath usage for consistency with relink-civitai
- Raise errors instead of silently falling back to LoRA scanner when
  model root cannot be determined
- Scope .input-group CSS rules to modal IDs to fix style conflicts
  with download-modal.css
- Add i18n keys across all 10 locales with translations for
  zh-CN, zh-TW, ja, ko, de, es, fr, he, ru
2026-07-07 20:04:47 +08:00
Will Miao 2373edf73c feat(ui): load provider model catalog asynchronously to avoid blocking page render 2026-07-06 10:02:09 +08:00
Will Miao 4ed9169646 feat(ui): redesign AI Provider settings with provider presets and model catalog
- Replace hardcoded provider list with PROVIDER_PRESETS (OpenAI, Ollama,
  DeepSeek, Groq, OpenRouter, OpenCode Go, Custom)
- Load model lists from models.dev/api.json catalog at startup
- Add Combobox vanilla JS component for model/base-URL selection
- Fetch local Ollama models via live API instead of catalog
- Hide API key values from frontend (boolean-only llm_api_key_set)
- Add i18n translations for all 9+ locales
- Update snapshot tests for new response fields
2026-07-03 16:08:51 +08:00
Will Miao cf898da193 feat(agent): add LLM-powered metadata enrichment system with AgentCLI and PostProcessor
Introduce an agent skill framework for LLM-driven metadata enrichment:

- AgentCLI (py/agent_cli/): in-process wrappers around internal services
  using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
  OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
  (replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales

Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
2026-07-02 21:27:01 +08:00
Will Miao 09ca91fc0e feat(download): add Hugging Face model download to standalone UI wizard (#965, #977)
Integrate HF model downloading into the existing CivitAI-style wizard flow:
- URL type detection (civitai / hf-resolve / hf-repo / direct-http)
- Repo file explorer with checkbox-based file selection
- Batch/queue download with per-file WebSocket progress
- Aria2 backend support (respects download_backend setting)
- Scanner cache integration via create_default_metadata + add_model_to_cache
- i18n updates for all 10 locales
2026-06-30 19:36:12 +08:00
Will Miao e9e8c31ad1 fix(registry): store nodes per-client to prevent multi-tab race condition
Move NodeRegistry from a single global _nodes dict to a per-client
(_tab_nodes) structure so that multiple ComfyUI browser tabs no
longer overwrite each other's workflow node data during a
lora_registry_refresh cycle.  The merged result is a union of all
known tabs' target nodes, eliminating the non-deterministic failure
where send-to-workflow could randomly target a tab lacking valid
targets.

- NodeRegistry.register_nodes(sid, nodes) replaces per-tab data
  without affecting other tabs.
- NodeRegistry.get_merged_registry() returns the union across all
  connected clients, together with tab_count / per-tab metadata.
- prepare_for_refresh() snapshots the current active sockets; caller
  re-reads before merging so that newly-connected tabs are not pruned.
- workflow_registry.js sends api.clientId in the POST body so the
  backend can identify which tab is registering.
2026-06-28 17:57:58 +08:00
Will Miao 85da7175bc feat: add Node Marker system with right-click marking 2026-06-23 20:54:32 +08:00
Will Miao ab6bb25d46 fix(example-images): skip hidden files in path validation, show offending items on failure (#807) 2026-06-19 11:54:55 +08:00
Will Miao b24b1a7e57 feat(settings): hide API key from frontend, use status+edit instead of password field
Backend changes:
- Add civitai_api_key to _NO_SYNC_KEYS, return only boolean civitai_api_key_set
- Clean up known template placeholder on load to prevent false positive

Frontend changes:
- Replace type=password with type=text + CSS masking (-webkit-text-security)
- Replace pre-filled input with status display (Configured/Not configured)
- Add inline edit view with Save/Cancel buttons
- Re-add eye toggle via CSS class toggle (not type switching)
- Use CSS transitions for smooth status/edit view switching

This prevents Chromium/Vivaldi password manager from triggering
'save password' prompts when opening the settings modal.
2026-06-19 08:05:04 +08:00
Will Miao d87863b423 feat(embedding): send embedding to workflow + fix copy button format
- Fix copy button on embedding cards to copy 'embedding:folder/name' format
- Add send-embedding-to-workflow for Prompt (LoraManager), Text (LoraManager),
  and CLIPTextEncode nodes, appending embedding code to text content
- Extend workflow registry to register text-capable nodes by comfyClass
  (not generic widget name 'text') to avoid false matches
- Add mode parameter to update_node_widget API/event for append support
- Fix single/bulk context menus: single shows plain 'Send to Workflow',
  bulk collapses submenu into direct action for embeddings (append-only)
2026-06-11 22:41:42 +08:00
Will Miao 1044fa3c83 feat(doctor): improve duplicate filename conflict UX with confirm modal, syntax-format nav, and i18n
- Remove [LoRAs] prefix noise from conflict detail display
- Limit inline conflict groups to 5, show remainder count
- Add 'Switch to Full Path Syntax' action in conflict card
- Add confirmation modal before resolving conflicts (shows rename strategy)
- Register resolveFilenameConflictsModal in ModalManager (fix no-op showModal)
- Switch to Interface section and add highlight animation on syntax-format nav
- Sync and translate conflictConfirm strings across all 10 locales
2026-05-25 21:25:35 +08:00
Will Miao f105500740 feat(doctor): suppress duplicate filename warnings when full path syntax is active (#917) 2026-05-22 22:35:06 +08:00
Will Miao 031d5e4f40 fix(doctor): exclude checkpoints/embeddings from duplicate filename detection (#934)
Duplicate filename detection is only relevant for LoRAs, which use
basename-only syntax (<lora:name:strength>). Checkpoints and diffusion
models reference files via relative paths with extensions, so filename
conflicts there are false positives — there is no resolution ambiguity.

Both _log_duplicate_filename_summary() and DoctorHandler's
_check_filename_conflicts() now skip scanners with model_type != 'lora'.
2026-05-18 13:57:28 +08:00
Will Miao 5c53968caa refactor(download-history): rename mark_not_downloaded to mark_as_deleted
The method mark_not_downloaded() was misleading — it doesn't negate
'downloaded' history (the model was indeed downloaded before), but
rather sets is_deleted_override = 1 to indicate the version was
downloaded and subsequently deleted. This flag allows re-download when
the 'skip previously downloaded' setting is enabled.

Rename to mark_as_deleted() to accurately reflect its semantics.
2026-05-12 22:50:30 +08:00
Will Miao b4f7dd75af fix(persistent-cache): persist scanner cache after model deletion
After deleting a model, the in-memory scanner cache was updated but the
SQLite persistent cache was not. On server restart, the stale persistent
cache caused check_model_version_exists() to return True, blocking
re-download with 'Model version already exists'.

Add _persist_current_cache() calls in both deletion paths:
- ModelLifecycleService.delete_model() (used by versions tab delete)
- delete_model_version handler in MiscHandlers
2026-05-12 22:50:10 +08:00
Will Miao b4a706651f feat(delete-model-version): add GET endpoint to delete a model version by version ID 2026-05-05 21:25:08 +08:00
Will Miao f028625ce9 feat(check-models-exist): add batch endpoint for checking multiple model IDs
New endpoint: GET /api/lm/check-models-exist?modelIds=1,2,3,...

Accepts comma-separated modelIds, returns a results array with one
entry per modelId. Uses a single scanner lookup batch - three
service-registry calls total, regardless of model count. Skips
history checks entirely (same rationale as the singleton endpoint:
when models exist locally, history is redundant).

Expected: reduces 231 HTTP round-trips to 1 for the browser
extension's model-card indicator flow. Combined with the prior
SQLite-connection and history-skip fixes, total wall-clock time
for a 175K-lora user's page load drops from ~9.4s to <10ms.
2026-05-02 13:43:53 +08:00
Will Miao d324b57274 perf(check-model-exists): eliminate SQLite connection-per-query overhead and skip redundant history checks
Root cause: 231 concurrent /check-model-exists requests on 175K-lora library
caused ~9.4s wall clock time. The bottleneck was two-fold:

1. DownloadedVersionHistoryService opened a new sqlite3.connect() for every
   query under asyncio.Lock. With a large WAL from 175K entries, each
   connect() took ~8ms. Serialized by the lock across 231 requests, the
   230th request waited ~1848ms just for lock acquisition.

2. check_model_exists always queried download history even when the model
   was found locally. The history result (hasBeenDownloaded /
   downloadedVersionIds) is only used by the UI when the model is NOT
   found locally; when found, the 'in library' indicator takes priority.

Changes:
- downloaded_version_history_service.py: added persistent _get_conn() that
  creates the SQLite connection once and reuses it across all queries
- misc_handlers.py: early-return from check_model_exists when the model
  exists locally, bypassing the history service entirely (lock skipped)

Expected: per-request wait time drops from ~1912ms to <3ms, wall clock
from ~9.4s to <0.3s for the 175K-lora user's 231-card page.
2026-05-02 13:31:20 +08:00
Will Miao 5dcfde36ea feat(doctor): add duplicate filename conflict detection and one-click resolution
Detects when multiple model files share the same basename (causing
ambiguity in LoRA resolution), logs warnings during scanning, and
provides a "Resolve Conflicts" button in the Doctor panel. Resolution
renames duplicates with hash-prefixed unique filenames, migrates all
sidecar and preview files, and updates the cache and frontend scroller
in-place so the model modal immediately reflects the new filename.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 15:21:26 +08:00
Will Miao cdd77029b6 fix(autocomplete): improve wildcard onboarding UX 2026-04-15 22:25:25 +08:00
Will Miao 083de395b1 chore(logging): remove autocomplete debug logs (#895) 2026-04-15 20:42:55 +08:00
Will Miao 4514ca94b7 fix(autocomplete): reduce tag search overhead (#895) 2026-04-15 20:42:33 +08:00
Will Miao 62247bdd87 feat(prompt): expand wildcards at runtime (#895) 2026-04-15 20:42:27 +08:00
Will Miao 1817142a7b feat(doctor): add system diagnostics feature 2026-04-11 16:03:38 +08:00
Will Miao 72f8e0d1be fix(backup): add user-state backup UI and storage 2026-04-10 20:49:30 +08:00
Will Miao 727d0ef043 feat(misc): add model download status aggregation 2026-04-03 22:17:09 +08:00
Will Miao 33a7f07558 feat(download-history): track downloaded model versions 2026-04-03 16:13:14 +08:00
Will Miao 00f5c1e887 feat: Dynamic base model fetching from Civitai API (#854)
Implement automatic fetching of base models from Civitai API to keep
data up-to-date without manual updates.

Backend:
- Add CivitaiBaseModelService with 7-day TTL caching
- Add /api/lm/base-models endpoints for fetching and refreshing
- Merge hardcoded and remote models for backward compatibility
- Smart abbreviation generation for unknown models

Frontend:
- Add civitaiBaseModelApi client for API communication
- Dynamic base model loading on app initialization
- Update SettingsManager to use merged model lists
- Add support for 8 new models: Anima, CogVideoX, LTXV 2.3, Mochi,
  Pony V7, Wan Video 2.5 T2V/I2V

API Endpoints:
- GET /api/lm/base-models - Get merged models
- POST /api/lm/base-models/refresh - Force refresh
- GET /api/lm/base-models/categories - Get categories
- GET /api/lm/base-models/cache-status - Check cache status

Closes #854
2026-03-29 00:18:15 +08:00
Will Miao cda271890a feat(workflow-template): add new tab template workflow with auto-zoom
- Add GET /api/lm/example-workflows endpoint to list available templates
- Add GET /api/lm/example-workflows/{filename} to retrieve specific workflow
- Add 'New Tab Template Workflow' setting in LoRA Manager settings
- Automatically apply 80% zoom level when loading template workflows
- Override workflow's saved view settings to prevent visual zoom flicker

The feature allows users to select a template workflow from example_workflows/
directory to load when creating new workflow tabs, with a hardcoded 0.8 zoom
level for better initial view experience.
2026-03-08 21:03:14 +08:00
Will Miao a802a89ff9 feat(autocomplete): implement virtual scrolling and pagination
- Add virtual scrolling with configurable visible items (default: 15)
- Implement pagination with offset/limit for backend APIs
- Support loading more items on scroll
- Fix width calculation for suggestions dropdown
- Update backend services to support offset parameter

Files modified:
- web/comfyui/autocomplete.js (virtual scroll, pagination)
- py/services/base_model_service.py (offset support)
- py/services/custom_words_service.py (offset support)
- py/services/tag_fts_index.py (offset support)
- py/routes/handlers/model_handlers.py (offset param)
- py/routes/handlers/misc_handlers.py (offset param)
2026-03-07 22:17:26 +08:00
Will Miao 78b55d10ba refactor: move supporters loading to separate API endpoint
- Add SupportersHandler in misc_handlers.py to serve /api/lm/supporters
- Register new endpoint in misc_route_registrar.py
- Remove supporters from page load template context in model_handlers.py
- Create supportersService.js for frontend data fetching
- Update Header.js to fetch supporters when support modal opens
- Modify support_modal.html to use client-side rendering

This change improves page load performance by loading supporters data
on-demand instead of during initial page render.
2026-02-28 20:14:20 +08:00
pixelpaws 7a04cec08d Merge pull request #825 from RanKaze/main
feat: filter node with mode:0
2026-02-23 16:39:45 +08:00
K1einB1ue 60324c1299 feat: filter node with mode:0 2026-02-22 07:19:08 +08:00
Will Miao 879588e252 refactor(settings): invert sync logic from whitelist to blacklist
Replace _SYNC_KEYS (37 keys) with _NO_SYNC_KEYS (5 keys) in SettingsHandler.
New settings automatically sync to frontend unless explicitly excluded.

Changes:
- SettingsHandler now syncs all settings except those in _NO_SYNC_KEYS
- Added keys() method to SettingsManager for iteration
- Updated tests to use new behavior

Benefits:
- No more missing keys when adding new settings
- Reduced maintenance burden
- Explicit exclusions for sensitive/internal settings only

Fixes: #86
2026-02-20 12:14:50 +08:00
Will Miao 67869f19ff feat(early-access): implement EA filtering and UI improvements
Add Early Access version support with filtering and improved UI:

Backend:
- Add is_early_access and early_access_ends_at fields to ModelVersionRecord
- Implement two-phase EA detection (bulk API + single API enrichment)
- Add hide_early_access_updates setting to filter EA updates
- Update has_update() and has_updates_bulk() to respect EA filter setting
- Add _enrich_early_access_details() for precise EA time fetching
- Fix setting propagation through base_model_service and model_update_service

Frontend:
- Add smart relative time display for EA (in Xh, in Xd, or date)
- Replace EA label with clock icon in metadata (fa-clock)
- Show Download button with bolt icon for EA versions (fa-bolt)
- Change EA badge color to #F59F00 (CivitAI Buzz theme)
- Fix toggle UI for hide_early_access_updates setting
- Add translation keys for EA time formatting

Tests:
- Update all tests to pass with new EA functionality
- Add test coverage for EA filtering logic

Closes #815
2026-02-20 10:32:51 +08:00
Will Miao 94edde7744 feat(settings): add metadata_refresh_skip_paths to sync keys for UI update 2026-02-09 10:09:53 +08:00
Will Miao 08267cdb48 refactor(filter): extract preset management logic into FilterPresetManager
Move filter preset creation, deletion, application, and storage logic
from FilterManager into a dedicated FilterPresetManager class to
improve separation of concerns and maintainability.

- Add FilterPresetManager with preset CRUD operations
- Update FilterManager to use preset manager via composition
- Handle EMPTY_WILDCARD_MARKER for wildcard base model filters
- Add preset-related translations to all locale files
- Update filter preset UI styling and interactions
2026-01-29 16:32:41 +08:00
Will Miao 58ae6b9de6 fix: persist onboarding and banner dismiss state to backend
Moves onboarding_completed and dismissed_banners from localStorage
to backend settings (settings.json) to survive incognito/private
browser modes.

Fixes #786
2026-01-29 08:48:04 +08:00