Commit Graph

2198 Commits

Author SHA1 Message Date
Will Miao
70c150bd80 fix(services): implement stable sorting for model and recipe caches
Add file_path as a tie-breaker for all sort modes in ModelCache, BaseModelService, LoraService, and RecipeCache to ensure deterministic ordering when primary keys are identical. Resolves issue #859.
2026-03-17 14:20:23 +08:00
Will Miao
9e81c33f8a fix(utils): make sanitize_folder_name idempotent by combining strip/rstrip calls 2026-03-17 11:24:59 +08:00
Will Miao
22c0dbd734 feat(recipes): persist 'Skip images without metadata' choice in batch import 2026-03-17 11:01:41 +08:00
Will Miao
d0c58472be fix(i18n): add missing common.actions.close translation key 2026-03-17 09:57:27 +08:00
Will Miao
b3c530bf36 fix(autocomplete): handle multi-word tag matching with normalized spaces
- Replace multiple consecutive spaces with single underscore for tag matching
  (e.g., 'looking  to   the side' → 'looking_to_the_side')
- Support prefix/suffix matching for flexible multi-word autocomplete
  (e.g., 'looking to the' → 'looking_to_the_side')
- Add comprehensive test coverage for multi-word scenarios

Test coverage:
- Multi-word exact match (Danbooru convention)
- Partial match with last token replacement
- Command mode with multi-word phrases
- Multiple consecutive spaces handling
- Backend LOG10 popularity weight validation

Fixes: 'looking to the side' input now correctly replaces with
'looking_to_the_side, ' (or 'looking to the side, ' with space replacement)
2026-03-17 09:34:01 +08:00
Will Miao
b5a0725d2c fix(autocomplete): improve tag search ranking with popularity-based sorting
- Add LOG10(post_count) weighting to BM25 score for better relevance ranking
- Prioritize tag_name prefix matches above alias matches using CASE statement
- Remove frontend re-scoring logic to trust backend排序 results
- Fix pagination consistency: page N+1 scores <= page N minimum score

Key improvements:
- '1girl' (6M posts) now ranks #1 instead of #149 for search '1'
- tag_name prefix matches always appear before alias matches
- Popular tags rank higher than obscure ones with same prefix
- Consistent ordering across pagination boundaries

Test coverage:
- Add test_search_tag_name_prefix_match_priority
- Add test_search_ranks_popular_tags_higher
- Add test_search_pagination_ordering_consistency
- Add test_search_rank_score_includes_popularity_weight
- Update test data with 15 tags starting with '1'

Fixes issues with autocomplete dropdown showing inconsistent results
when scrolling through paginated search results.
2026-03-16 19:09:07 +08:00
Will Miao
ef38bda04f docs: remove redundant example metadata files (#856)
- Delete examples/metadata/ directory and all example files
  - Real metadata.json files in model roots are better examples
  - Examples were artificial and could become outdated
  - Maintenance burden outweighs benefit

- Remove 'Complete Examples' section from docs/metadata-json-schema.md
- Remove reference to example files in 'See Also' section

Rationale:
Users have access to real-world metadata.json files in their actual
model directories, which contain complete Civitai API responses with
authentic data structures (images arrays with prompts, files with hashes,
creator information, etc.). These are more valuable than simplified
artificial examples.
2026-03-16 09:41:58 +08:00
Will Miao
58713ea6e0 fix(top-menu): use dynamic imports to eliminate deprecation warnings
- Replace static imports of deprecated ComfyButton and ComfyButtonGroup with dynamic imports
- Only loads legacy API files when frontend version < 1.33.9 (backward compatibility path)
- Frontend >= 1.33.9 users no longer see deprecation warnings since legacy code is never loaded
- Preserves full backward compatibility for older ComfyUI frontend versions
- All existing tests pass (159 JS + 65 Vue tests)
2026-03-16 09:41:58 +08:00
Will Miao
8b91920058 docs: add comprehensive metadata.json schema documentation (#856)
- Create docs/metadata-json-schema.md with complete field reference
  - All base fields for LoRA, Checkpoint, and Embedding models
  - Complete civitai object structure with Used vs Stored field classification
  - Model-level fields (allowCommercialUse, allowDerivatives, etc.)
  - Creator fields (username, image)
  - customImages structure with actual field names and types
  - Field behavior categories (Auto-Updated, Set Once, User-Editable)

- Add .specs/metadata.schema.json for programmatic validation
  - JSON Schema draft-07 format
  - oneOf schemas for each model type
  - Definitions for civitaiObject and usageTips

- Add example metadata files for each model type
  - lora-civitai.json: LoRA with full Civitai data
  - lora-custom.json: User-defined LoRA with trigger words
  - lora-no-triggerwords.json: LoRA without trigger words
  - checkpoint-civitai.json: Checkpoint from Civitai
  - embedding-custom.json: Custom embedding

Key clarifications:
  - modified: Import timestamp (Set Once, never changes after import)
  - size: File size at import time (Set Once)
  - base_model: Optional with actual values (SDXL 1.0, Flux.1 D, etc.)
  - model_type: Used in metadata.json (not sub_type which is internal)
  - allowCommercialUse: ["Image", "Video", "RentCivit", "Rent"]
  - civitai.files/images: Marked as Used by Lora Manager
  - User-editable fields clearly documented (model_name, tags, etc.)
2026-03-16 09:41:58 +08:00
Will Miao
ee466113d5 feat: implement batch import recipe functionality (frontend + backend fixes)
Backend fixes:
- Add missing API route for /api/lm/recipes/batch-import/progress (GET)
- Add missing API route for /api/lm/recipes/batch-import/directory (POST)
- Add missing API route for /api/lm/recipes/browse-directory (POST)
- Register WebSocket endpoint for batch import progress
- Fix skip_no_metadata default value (True -> False) to allow no-LoRA imports
- Add items array to BatchImportProgress.to_dict() for detailed results

Frontend implementation:
- Create BatchImportManager.js with complete batch import workflow
- Add directory browser UI for selecting folders
- Add batch import modal with URL list and directory input modes
- Implement real-time progress tracking (WebSocket + HTTP polling)
- Add results summary with success/failed/skipped statistics
- Add expandable details view showing individual item status
- Auto-refresh recipe list after import completion

UI improvements:
- Add spinner animation for importing status
- Simplify results summary UI to match progress stats styling
- Fix current item text alignment
- Fix dark theme styling for directory browser button
- Fix batch import button styling consistency

Translations:
- Add batch import related i18n keys to all locale files
- Run sync_translation_keys.py to sync all translations

Fixes:
- Batch import now allows images without LoRAs (matches single import behavior)
- Progress endpoint now returns complete items array with status details
- Results view correctly displays skipped items with error messages
2026-03-16 09:41:58 +08:00
Will Miao
f86651652c feat(batch-import): implement backend batch import service with adaptive concurrency
- Add BatchImportService with concurrent execution using asyncio.gather
- Implement AdaptiveConcurrencyController with dynamic adjustment
- Add input validation for URLs and local paths
- Support duplicate detection via skip_duplicates parameter
- Add WebSocket progress broadcasting for real-time updates
- Create comprehensive unit tests for batch import functionality
- Update API handlers and route registrations
- Add i18n translation keys for batch import UI
2026-03-16 09:41:58 +08:00
Will Miao
c89d4dae85 fix(extra-paths): support trigger words for LoRAs in extra folder paths, fixes #860
- Update get_lora_info() to check both loras_roots and extra_loras_roots
- Add fallback logic to return trigger words even if path not in recognized roots
- Ensure Trigger Word Toggle node displays trigger words for LoRAs from extra folder paths

Fixes issue where LoRAs added from extra folder paths would not show their trigger words in connected Trigger Word Toggle nodes.
2026-03-16 09:38:21 +08:00
pixelpaws
55a18d401b Merge pull request #858 from botchedchuckle/patch-1
Fix: Escape HTML in Prompt/NegativePrompt for MetadataPanel
2026-03-14 14:43:46 +08:00
botchedchuckle
7570936c75 Fix: Escape HTML in Prompt/NegativePrompt for MetadataPanel
* Fixed a bug where `prompt` and `negativePrompt` were both being
  added directly to HTML without escaping them. Given prompts are
  allowed to have HTML characters (e.g. `<lora:something:0.75>`), by
  forgetting to escape them some tags were missing in the metadata
  views for example images using those characters.
2026-03-13 01:29:04 -07:00
Will Miao
4fcf641d57 fix(bulk-context-menu): escape special characters in data-filepath selector to support double quotes in filenames (#845) 2026-03-12 08:49:10 +08:00
Will Miao
5c29e26c4e fix(top-menu): add backward compatibility for actionBarButtons API (#853)
- Implement version detection using __COMFYUI_FRONTEND_VERSION__ and /system_stats API
- Add version parsing and comparison utilities
- Dynamically register extension based on frontend version
- Use actionBarButtons API for frontend >= 1.33.9
- Fallback to legacy ComfyButton approach for older versions
- Add comprehensive version detection tests
2026-03-12 07:41:29 +08:00
Will Miao
ee765a6d22 fix(sidebar): escape folder names and paths to support double quotes
- Import and use escapeHtml and escapeAttribute in SidebarManager.js
- Escape data-path and title attributes in folder tree and breadcrumbs
- Use CSS.escape() for attribute selectors in updateTreeSelection
- Fixes issue #843 where folders with double quotes broke navigation
2026-03-11 23:33:11 +08:00
Will Miao
c02f603ed2 fix(autocomplete): add wheel event handler for canvas zoom support
Add @wheel event listener to AutocompleteTextWidget textarea to enable canvas zoom when textarea has no scrollbar.

The onWheel handler:
- Forwards pinch-to-zoom (ctrl+wheel) to canvas
- Passes horizontal scroll to canvas
- When textarea has vertical scrollbar: lets textarea scroll
- When textarea has NO scrollbar: forwards to canvas for zoom

Behavior now matches ComfyUI built-in multiline widget.

Fixes #850
2026-03-11 20:58:01 +08:00
Will Miao
ee84b30023 Fix node selector z-index issue in recipe modal
Change node-selector z-index from 1000 to var(--z-overlay) (2000)
to ensure the model selector UI appears above the recipe modal
when sending checkpoints to workflow with multiple targets.
2026-03-09 19:29:13 +08:00
Will Miao
97979d9e7c fix(send-to-workflow): strip file extension before searching relative paths
Backend _relative_path_matches_tokens() removes extensions from paths
before matching (commit 43f6bfab). This fix ensures frontend also
removes extensions from search terms to avoid matching failures.

Fixes issue where send model to workflow would receive absolute
paths instead of relative paths because the API returned empty
results when searching with file extension.
2026-03-09 15:49:37 +08:00
Will Miao
cda271890a feat(workflow-template): add new tab template workflow with auto-zoom
- Add GET /api/lm/example-workflows endpoint to list available templates
- Add GET /api/lm/example-workflows/{filename} to retrieve specific workflow
- Add 'New Tab Template Workflow' setting in LoRA Manager settings
- Automatically apply 80% zoom level when loading template workflows
- Override workflow's saved view settings to prevent visual zoom flicker

The feature allows users to select a template workflow from example_workflows/
directory to load when creating new workflow tabs, with a hardcoded 0.8 zoom
level for better initial view experience.
2026-03-08 21:03:14 +08:00
Will Miao
2fbe6c8843 fix(autocomplete): fix dropdown width calculation bug
Temporarily remove width constraints when measuring content to prevent
scrollWidth from being limited by narrow container. This fixes the issue
where dropdown width was incorrectly calculated as ~120px.

Also update test to match maxItems default value (100).
2026-03-07 23:23:26 +08:00
Will Miao
4fb07370dd fix(tests): add offset parameter to MockTagFTSIndex.search()
Add missing offset parameter to MockTagFTSIndex to support
pagination changes from commit a802a89.

- Update search() signature to include offset=0
- Implement pagination logic with offset/limit slicing
2026-03-07 23:10:00 +08:00
Will Miao
43f6bfab36 fix(autocomplete): strip file extensions from model names in search suggestions
Remove .safetensors/.ckpt/.pt/.bin extensions from model names in autocomplete
suggestions to improve UX and search relevance:

Frontend (web/comfyui/autocomplete.js):
- Add _getDisplayText() helper to strip extensions from model paths
- Update _matchItem() to match against filename without extension
- Update render() and createItemElement() to display clean names

Backend (py/services/base_model_service.py):
- Add _remove_model_extension() helper method
- Update _relative_path_matches_tokens() to ignore extensions in matching
- Update _relative_path_sort_key() to sort based on names without extensions

Tests (tests/services/test_relative_path_search.py):
- Add tests to verify 's' and 'safe' queries don't match all .safetensors files

Fixes issue where typing 's' would match all .safetensors files and cluttered
suggestions with redundant extension names.
2026-03-07 23:07:10 +08:00
Will Miao
a802a89ff9 feat(autocomplete): implement virtual scrolling and pagination
- Add virtual scrolling with configurable visible items (default: 15)
- Implement pagination with offset/limit for backend APIs
- Support loading more items on scroll
- Fix width calculation for suggestions dropdown
- Update backend services to support offset parameter

Files modified:
- web/comfyui/autocomplete.js (virtual scroll, pagination)
- py/services/base_model_service.py (offset support)
- py/services/custom_words_service.py (offset support)
- py/services/tag_fts_index.py (offset support)
- py/routes/handlers/model_handlers.py (offset param)
- py/routes/handlers/misc_handlers.py (offset param)
2026-03-07 22:17:26 +08:00
Will Miao
343dd91e4b feat(ui): improve clear button UX in autocomplete text widget
Move clear button from top-right to bottom-right to avoid

obscuring text content. Add hover visibility for cleaner UI.

Reserve bottom padding in textarea for button placement.
2026-03-07 21:09:59 +08:00
Will Miao
3756f88368 feat(autocomplete): improve multi-word tag search with query normalization
Implement search query variation generation to improve matching for multi-word tags:
- Generate multiple query forms: original, underscore (spaces->_), no-space, last token
- Execute up to 4 parallel queries with result merging and deduplication
- Add smart matching with symbol-insensitive comparison (blue hair matches blue_hair)
- Sort results with exact matches prioritized over partial matches

This allows users to type natural language queries like 'looking to the side' and
find tags like 'Looking_to_the_side' while maintaining backward compatibility
with continuous typing workflows.
2026-03-07 20:24:35 +08:00
Will Miao
acc625ead3 feat(recipes): add sync changes dropdown menu for recipe refresh
- Add syncChanges() function to recipeApi.js for quick refresh without cache rebuild
- Implement dropdown menu UI in recipes page with quick refresh and full rebuild options
- Add initDropdowns() method to RecipeManager for dropdown interaction handling
- Update AGENTS.md with more precise instruction about running sync_translation_keys.py
- Integrate sync changes functionality as default refresh behavior
2026-03-04 20:31:58 +08:00
Will Miao
f402505f97 i18n: complete TODO translations in locale files
- Add missing translations for modelTypes, recipe refresh, and sync notifications
- Translate for all supported languages (zh-CN, zh-TW, ja, ko, fr, de, es, ru, he)
- Run sync_translation_keys.py to ensure key consistency
2026-03-04 20:27:21 +08:00
Will Miao
4d8113464c perf(recipe_scanner): eliminate event loop blocking during cache rebuild
Refactor force_refresh path to use thread pool execution instead of blocking
the event loop shared with ComfyUI. Key changes:

- Fix 1: Route force_refresh through _initialize_recipe_cache_sync() in thread pool
- Fix 2: Add GIL release points (time.sleep(0)) every 100 files in sync loops
- Fix 3: Move RecipeCache.resort() to thread pool via run_in_executor
- Fix 4: Persist cache automatically after force_refresh
- Fix 5: Increase yield frequency in _enrich_cache_metadata (every recipe)

This eliminates the ~5 minute freeze when rebuilding 30K recipe cache.

Fixes performance issue where ComfyUI became unresponsive during recipe
scanning due to shared Python event loop blocking.
2026-03-04 15:10:46 +08:00
Will Miao
1ed503a6b5 docs: add lazy hash computation to v1.0.0 release notes 2026-03-04 07:41:19 +08:00
Will Miao
d67914e095 docs: update portable package download link to v1.0.0 2026-03-03 22:06:29 +08:00
Will Miao
2c810306fb feat: implement automated supporter recognition in README
- Add scripts/update_supporters.py to generate supporter list from JSON
- Set up GitHub Action to auto-update README.md on supporters.json change
- Update README.md with placeholders and personalized gratitude message
v1.0.0
2026-03-03 21:52:08 +08:00
Will Miao
dd94c6b31a chore: add v1.0.0 release notes and update version in pyproject.toml 2026-03-03 21:19:50 +08:00
Will Miao
1a0edec712 feat: enhance supporters modal with auto-scrolling and visual improvements
- Add auto-scrolling functionality to supporters list with user interaction controls (pause on hover, manual scroll)
- Implement gradient overlays at top/bottom for credits-like appearance
- Style custom scrollbar with subtle hover effects for better UX
- Adjust padding and positioning to ensure all supporters remain visible during scroll
2026-03-03 21:18:12 +08:00
Will Miao
7ba9b998d3 fix(stats): resolve dashboard initialization race condition and test failure
- Refactor StatisticsManager to return promises from initializeVisualizations and initializeLists
- Update fetchAndRenderList to use the fetchData wrapper for consistent mocking
- Update statistics dashboard test to include mock data for paginated model-usage-list endpoint
2026-03-03 15:08:33 +08:00
Will Miao
8c5d5a8ca0 feat(stats): implement infinite scrolling and paginated model usage lists (fixes #812)
- Add get_model_usage_list API endpoint for paginated stats
- Replace static rendering with client-side infinite scroll logic
- Add scrollbars and max-height to model usage lists
2026-03-03 15:00:01 +08:00
Will Miao
672e4cff90 fix(move): reset manual folder selection when using default path (fixes #836) 2026-03-02 23:29:16 +08:00
Will Miao
c2716e3c39 fix(i18n): resolve missing translation keys and complete multi-language support
- Add missing keys 'common.cancel', 'common.confirm', and 'sidebar.dragDrop.noDragState' to en.json
- Synchronize all locale files using sync_translation_keys.py
- Complete translations for zh-CN, zh-TW, ja, ru, de, fr, es, ko, and he
- Implement sidebar drag-and-drop folder creation with visual feedback and input validation
- Optimize MoveManager to use resetAndReload for consistent UI state after moving models
- Fix recursive visibility check for root folder in MoveManager
2026-03-02 22:02:47 +08:00
Will Miao
b72cf7ba98 feat(showcase): optimize CivitAI media URLs for better performance
- Add CivitAI URL utility with optimization strategies for showcase and thumbnail modes
- Replace /original=true with /optimized=true for showcase videos to reduce bandwidth
- Remove redundant crossorigin and referrerpolicy attributes from video elements
- Use media type detection to apply appropriate optimization (image vs video)
- Integrate URL optimization into showcase rendering for improved loading times
2026-03-02 14:05:44 +08:00
Will Miao
bde11b153f fix(preview): resolve CORS error when setting CivitAI remote media as preview
- Add new endpoint POST /api/lm/{prefix}/set-preview-from-url to handle
  remote image downloads server-side, avoiding CORS issues
- Use rewrite_preview_url() to download optimized smaller images (450px width)
- Use Downloader service for reliable downloads with retry logic and proxy support
- Update frontend to call new endpoint instead of fetching images in browser

fixes #837
2026-03-02 13:21:18 +08:00
Will Miao
8b924b1551 feat: add draggable attribute to recipe card elements
- Set draggable=true on recipe card div elements to enable drag-and-drop functionality
- This allows users to drag recipe cards for reordering or other interactions
2026-03-02 10:28:36 +08:00
Will Miao
ce08935b1e fix(showcase): support middle-click and left-click to expand showcase
Fix showcase expansion to work with both left-click and middle-click (drag scroll).

Problem: The scroll-indicator click events were only bound when the carousel
was in expanded state. Initial collapsed state meant no click handlers were
attached, so clicking did nothing.

Solution:
- Extract scroll-indicator event binding into separate bindScrollIndicatorEvents()
- Call bindScrollIndicatorEvents() immediately when showcase loads, regardless
  of collapsed state
- Separate handlers for left-click (click event) and middle-click (mousedown
  event) to avoid double-triggering

Changes:
- Add bindScrollIndicatorEvents() function for early event binding
- Use click event for left mouse button (button 0)
- Use mousedown event for middle mouse button (button 1)
- Update loadExampleImages() to bind events immediately
- Update initShowcaseContent() to use the new function
2026-03-02 08:44:15 +08:00
Will Miao
24fcbeaf76 Skip performance tests by default
- Add 'performance' marker to pytest.ini
- Add pytestmark to test_cache_performance.py
- Use -m 'not performance' by default in addopts
- Allows manual execution with 'pytest -m performance'
2026-02-28 21:46:20 +08:00
Will Miao
c9e5ea42cb Fix null-safety issues and apply code formatting
Bug fixes:
- Add null guards for base_models_roots/embeddings_roots in backup cleanup
- Fix null-safety initialization of extra_unet_roots

Formatting:
- Apply consistent code style across Python files
- Fix line wrapping, quote consistency, and trailing commas
- Add type ignore comments for dynamic/platform-specific code
2026-02-28 21:38:41 +08:00
Will Miao
b005961ee5 feat(ui): improve changelog styling and spacing
- Remove left padding from changelog content container
- Add consistent padding to all changelog items
- Simplify latest changelog item styling by removing redundant padding
- Maintain visual distinction for latest items with background and border
2026-02-28 20:47:44 +08:00
Will Miao
ce03bbbc4e fix(frontend): defer LoadingManager DOM initialization to resolve i18n warning
Delay DOM creation in LoadingManager constructor to first use time,
ensuring window.i18n is ready before translate() is called.

This eliminates the 'i18n not available' console warning during
module initialization while maintaining correct translations
for cancel button and loading status text.
2026-02-28 20:30:16 +08:00
Will Miao
78b55d10ba refactor: move supporters loading to separate API endpoint
- Add SupportersHandler in misc_handlers.py to serve /api/lm/supporters
- Register new endpoint in misc_route_registrar.py
- Remove supporters from page load template context in model_handlers.py
- Create supportersService.js for frontend data fetching
- Update Header.js to fetch supporters when support modal opens
- Modify support_modal.html to use client-side rendering

This change improves page load performance by loading supporters data
on-demand instead of during initial page render.
2026-02-28 20:14:20 +08:00
Will Miao
77a2215e62 Fix lazy hash calculation for checkpoints in extra paths
- Allow empty sha256 when hash_status is 'pending' in cache entry validator
- Add on-demand hash calculation during bulk metadata refresh for checkpoints
  with pending hash status
- Add comprehensive tests for both fixes

Fixes issue where checkpoints in extra paths were not visible in UI and
not processed during bulk metadata refresh due to empty sha256.
2026-02-27 19:19:16 +08:00
pixelpaws
31901f1f0e Merge pull request #829 from willmiao/feature/lazy-hash-checkpoints
feat: lazy hash calculation for checkpoints
2026-02-27 11:02:39 +08:00