Commit Graph

106 Commits

Author SHA1 Message Date
Will Miao
76c15105e6 feat(lora-pool): add regex include/exclude name pattern filtering (#839)
Add name pattern filtering to LoRA Pool node allowing users to filter
LoRAs by filename or model name using either plain text or regex patterns.

Features:
- Include patterns: only show LoRAs matching at least one pattern
- Exclude patterns: exclude LoRAs matching any pattern
- Regex toggle: switch between substring and regex matching
- Case-insensitive matching for both modes
- Invalid regex automatically falls back to substring matching
- Filters apply to both file_name and model_name fields

Backend:
- Update LoraPoolLM._default_config() with namePatterns structure
- Add name pattern filtering to _apply_pool_filters() and _apply_specific_filters()
- Add API parameter parsing for name_pattern_include/exclude/use_regex
- Update LoraPoolConfig type with namePatterns field

Frontend:
- Add NamePatternsSection.vue component with pattern input UI
- Update useLoraPoolState to manage pattern state and API integration
- Update LoraPoolSummaryView to display NamePatternsSection
- Increase LORA_POOL_WIDGET_MIN_HEIGHT to accommodate new UI

Tests:
- Add 7 test cases covering text/regex include, exclude, combined
  filtering, model name fallback, and invalid regex handling

Closes #839
2026-03-19 17:15:05 +08:00
Will Miao
ee466113d5 feat: implement batch import recipe functionality (frontend + backend fixes)
Backend fixes:
- Add missing API route for /api/lm/recipes/batch-import/progress (GET)
- Add missing API route for /api/lm/recipes/batch-import/directory (POST)
- Add missing API route for /api/lm/recipes/browse-directory (POST)
- Register WebSocket endpoint for batch import progress
- Fix skip_no_metadata default value (True -> False) to allow no-LoRA imports
- Add items array to BatchImportProgress.to_dict() for detailed results

Frontend implementation:
- Create BatchImportManager.js with complete batch import workflow
- Add directory browser UI for selecting folders
- Add batch import modal with URL list and directory input modes
- Implement real-time progress tracking (WebSocket + HTTP polling)
- Add results summary with success/failed/skipped statistics
- Add expandable details view showing individual item status
- Auto-refresh recipe list after import completion

UI improvements:
- Add spinner animation for importing status
- Simplify results summary UI to match progress stats styling
- Fix current item text alignment
- Fix dark theme styling for directory browser button
- Fix batch import button styling consistency

Translations:
- Add batch import related i18n keys to all locale files
- Run sync_translation_keys.py to sync all translations

Fixes:
- Batch import now allows images without LoRAs (matches single import behavior)
- Progress endpoint now returns complete items array with status details
- Results view correctly displays skipped items with error messages
2026-03-16 09:41:58 +08:00
Will Miao
f86651652c feat(batch-import): implement backend batch import service with adaptive concurrency
- Add BatchImportService with concurrent execution using asyncio.gather
- Implement AdaptiveConcurrencyController with dynamic adjustment
- Add input validation for URLs and local paths
- Support duplicate detection via skip_duplicates parameter
- Add WebSocket progress broadcasting for real-time updates
- Create comprehensive unit tests for batch import functionality
- Update API handlers and route registrations
- Add i18n translation keys for batch import UI
2026-03-16 09:41:58 +08:00
Will Miao
cda271890a feat(workflow-template): add new tab template workflow with auto-zoom
- Add GET /api/lm/example-workflows endpoint to list available templates
- Add GET /api/lm/example-workflows/{filename} to retrieve specific workflow
- Add 'New Tab Template Workflow' setting in LoRA Manager settings
- Automatically apply 80% zoom level when loading template workflows
- Override workflow's saved view settings to prevent visual zoom flicker

The feature allows users to select a template workflow from example_workflows/
directory to load when creating new workflow tabs, with a hardcoded 0.8 zoom
level for better initial view experience.
2026-03-08 21:03:14 +08:00
Will Miao
a802a89ff9 feat(autocomplete): implement virtual scrolling and pagination
- Add virtual scrolling with configurable visible items (default: 15)
- Implement pagination with offset/limit for backend APIs
- Support loading more items on scroll
- Fix width calculation for suggestions dropdown
- Update backend services to support offset parameter

Files modified:
- web/comfyui/autocomplete.js (virtual scroll, pagination)
- py/services/base_model_service.py (offset support)
- py/services/custom_words_service.py (offset support)
- py/services/tag_fts_index.py (offset support)
- py/routes/handlers/model_handlers.py (offset param)
- py/routes/handlers/misc_handlers.py (offset param)
2026-03-07 22:17:26 +08:00
Will Miao
bde11b153f fix(preview): resolve CORS error when setting CivitAI remote media as preview
- Add new endpoint POST /api/lm/{prefix}/set-preview-from-url to handle
  remote image downloads server-side, avoiding CORS issues
- Use rewrite_preview_url() to download optimized smaller images (450px width)
- Use Downloader service for reliable downloads with retry logic and proxy support
- Update frontend to call new endpoint instead of fetching images in browser

fixes #837
2026-03-02 13:21:18 +08:00
Will Miao
78b55d10ba refactor: move supporters loading to separate API endpoint
- Add SupportersHandler in misc_handlers.py to serve /api/lm/supporters
- Register new endpoint in misc_route_registrar.py
- Remove supporters from page load template context in model_handlers.py
- Create supportersService.js for frontend data fetching
- Update Header.js to fetch supporters when support modal opens
- Modify support_modal.html to use client-side rendering

This change improves page load performance by loading supporters data
on-demand instead of during initial page render.
2026-02-28 20:14:20 +08:00
Will Miao
40d9f8d0aa feat: lazy hash calculation for checkpoints
Checkpoints are typically large (10GB+). This change delays SHA256
hash calculation until metadata fetch from Civitai is requested,
significantly improving initial scan performance.

- Add hash_status field to BaseModelMetadata
- CheckpointScanner skips hash during initial scan
- On-demand hash calculation during Civitai fetch
- Background bulk hash calculation support
2026-02-26 22:41:44 +08:00
pixelpaws
7a04cec08d Merge pull request #825 from RanKaze/main
feat: filter node with mode:0
2026-02-23 16:39:45 +08:00
K1einB1ue
60324c1299 feat: filter node with mode:0 2026-02-22 07:19:08 +08:00
Will Miao
773adb27c9 feat(model_download): add file_params JSON parsing to download handler
- Parse optional file_params query parameter as JSON in ModelDownloadHandler
- Add error handling for invalid JSON with warning log
- Maintain backward compatibility with existing download parameters
2026-02-22 04:26:38 +08:00
Will Miao
879588e252 refactor(settings): invert sync logic from whitelist to blacklist
Replace _SYNC_KEYS (37 keys) with _NO_SYNC_KEYS (5 keys) in SettingsHandler.
New settings automatically sync to frontend unless explicitly excluded.

Changes:
- SettingsHandler now syncs all settings except those in _NO_SYNC_KEYS
- Added keys() method to SettingsManager for iteration
- Updated tests to use new behavior

Benefits:
- No more missing keys when adding new settings
- Reduced maintenance burden
- Explicit exclusions for sensitive/internal settings only

Fixes: #86
2026-02-20 12:14:50 +08:00
Will Miao
67869f19ff feat(early-access): implement EA filtering and UI improvements
Add Early Access version support with filtering and improved UI:

Backend:
- Add is_early_access and early_access_ends_at fields to ModelVersionRecord
- Implement two-phase EA detection (bulk API + single API enrichment)
- Add hide_early_access_updates setting to filter EA updates
- Update has_update() and has_updates_bulk() to respect EA filter setting
- Add _enrich_early_access_details() for precise EA time fetching
- Fix setting propagation through base_model_service and model_update_service

Frontend:
- Add smart relative time display for EA (in Xh, in Xd, or date)
- Replace EA label with clock icon in metadata (fa-clock)
- Show Download button with bolt icon for EA versions (fa-bolt)
- Change EA badge color to #F59F00 (CivitAI Buzz theme)
- Fix toggle UI for hide_early_access_updates setting
- Add translation keys for EA time formatting

Tests:
- Update all tests to pass with new EA functionality
- Add test coverage for EA filtering logic

Closes #815
2026-02-20 10:32:51 +08:00
Will Miao
e8b37365a6 fix: Show all tags in LoRA Pool without limit (#819)
- Backend: Support limit=0 to return all tags in top-tags API
- Frontend: Remove tags limit setting and fetch all tags by default
- UI: Implement virtual scrolling in TagsModal for performance
  - Initial display 200 tags, load more on scroll
  - Show all results when searching
- Remove lora_pool_tags_limit setting to simplify UX

Fixes #819
2026-02-19 09:59:08 +08:00
Will Miao
94edde7744 feat(settings): add metadata_refresh_skip_paths to sync keys for UI update 2026-02-09 10:09:53 +08:00
Will Miao
b4ad03c9bf fix: improve example image upload reliability and error handling, #804
- Sequential per-file upload to avoid client_max_size limits
- Add backend exception handler with proper 500 responses
- Increase standalone server upload limit to 256MB
- Add partial success localization support
2026-02-08 09:17:19 +08:00
Will Miao
fa3625ff72 feat(filter): add tag logic toggle (OR/AND) for include tags filtering
Add a segmented toggle in the Filter Panel to switch between 'Any' (OR)
and 'All' (AND) logic when filtering by multiple include tags.

Changes:
- Backend: Add tag_logic field to FilterCriteria and ModelFilterSet
- Backend: Parse tag_logic parameter in model handlers
- Frontend: Add segmented toggle UI in filter panel header
- Frontend: Add interaction logic and state management for tag logic
- Add translations for all supported languages
- Add comprehensive tests for the new feature

Closes #802
2026-02-05 22:36:30 +08:00
Will Miao
b7e0821f66 feat(duplicates): add filter support for duplicate model finding, #783 2026-02-04 20:47:30 +08:00
Will Miao
7bcf4e4491 feat(config): discover deep symlinks dynamically when accessing previews 2026-02-04 00:16:59 +08:00
Will Miao
c12aefa82a fix(recipes): detect duplicates for remote imports using modelVersionId and Civitai URL, #750
- Use modelVersionId as fallback for all loras in fingerprint calculation (not just deleted)
- Add URL-based duplicate detection using source_path field
- Combine both fingerprint and URL-based duplicate detection in API response
- Fix _download_remote_media return type and unbound variable issue
2026-02-03 21:32:15 +08:00
Will Miao
1da476d858 feat(example-images): add check pending models endpoint and improve async handling
- Add /api/example-images/check-pending endpoint to quickly check models needing downloads
- Improve DownloadManager.start_download() to return immediately without blocking
- Add _handle_download_task_done callback for proper error handling and progress saving
- Add check_pending_models() method for lightweight pre-download validation
- Update frontend ExampleImagesManager to use new check-pending endpoint
- Add comprehensive tests for new functionality
2026-02-02 12:31:07 +08:00
Will Miao
08267cdb48 refactor(filter): extract preset management logic into FilterPresetManager
Move filter preset creation, deletion, application, and storage logic
from FilterManager into a dedicated FilterPresetManager class to
improve separation of concerns and maintainability.

- Add FilterPresetManager with preset CRUD operations
- Update FilterManager to use preset manager via composition
- Handle EMPTY_WILDCARD_MARKER for wildcard base model filters
- Add preset-related translations to all locale files
- Update filter preset UI styling and interactions
2026-01-29 16:32:41 +08:00
Will Miao
58ae6b9de6 fix: persist onboarding and banner dismiss state to backend
Moves onboarding_completed and dismissed_banners from localStorage
to backend settings (settings.json) to survive incognito/private
browser modes.

Fixes #786
2026-01-29 08:48:04 +08:00
Will Miao
7249c9fd4b refactor(autocomplete): remove old CSV fallback, use TagFTSIndex exclusively
Remove all autocomplete.txt parsing logic and fallback code, simplifying
the service to use only TagFTSIndex for Danbooru/e621 tag search
with category filtering.

- Remove WordEntry dataclass and _words_cache, _file_path attributes
- Remove _determine_file_path(), get_file_path(), load_words(), save_words(),
  get_content(), _parse_csv_content() methods
- Simplify search_words() to only use TagFTSIndex, always returning
  enriched results with {tag_name, category, post_count}
- Remove GET/POST /api/lm/custom-words endpoints (unused)
- Keep GET /api/lm/custom-words/search for frontend autocomplete
- Rewrite tests to focus on TagFTSIndex integration

This reduces code by 446 lines and removes untested pysssss plugin
integration. Feature is unreleased so no backward compatibility needed.
2026-01-26 20:36:00 +08:00
pixelpaws
93cd0b54dc Merge branch 'main' into fix-symlink 2026-01-26 17:29:31 +08:00
Will Miao
42f35be9d3 feat(autocomplete): add Danbooru/e621 tag search with category filtering
- Add TagFTSIndex service for fast SQLite FTS5-based tag search (221k+ tags)
- Implement command-mode autocomplete: /char, /artist, /general, /meta, etc.
- Support category filtering via category IDs or names
- Return enriched results with post counts and category badges
- Add UI styling for category badges and command list dropdown
2026-01-26 13:51:45 +08:00
Will Miao
c9e305397c feat: enhance symlink detection and cache invalidation
- Add `_entry_is_symlink` method to detect symlinks and Windows junctions
- Include first-level symlinks in fingerprint for better cache invalidation
- Re-enable preview path validation for security
- Update tests to verify retargeted symlinks trigger rescan
2026-01-25 19:14:16 +08:00
Will Miao
d5a2bd1e24 feat: add custom words autocomplete support for Prompt node
Adds custom words autocomplete functionality similar to comfyui-custom-scripts,
with the following features:

Backend (Python):
- Create CustomWordsService for CSV parsing and priority-based search
- Add API endpoints: GET/POST /api/lm/custom-words and
  GET /api/lm/custom-words/search
- Share storage with pysssss plugin (checks for their user/autocomplete.txt first)
- Fallback to Lora Manager's user directory for storage

Frontend (JavaScript/Vue):
- Add 'custom_words' and 'prompt' model types to autocomplete system
- Prompt node now supports dual-mode autocomplete:
  * Type 'emb:' prefix → search embeddings
  * Type normally → search custom words (no prefix required)
- Add AUTOCOMPLETE_TEXT_PROMPT widget type
- Update Vue component and composable types

Key Features:
- CSV format: word[,priority] compatible with danbooru-tags.txt
- Priority-based sorting: 20% top priority + prefix + include matches
- Preview tooltip for embeddings (not for custom words)
- Dynamic endpoint switching based on prefix detection

Breaking Changes:
- Prompt (LoraManager) node widget type changed from
  AUTOCOMPLETE_TEXT_EMBEDDINGS to AUTOCOMPLETE_TEXT_PROMPT
- Removed standalone web/comfyui/prompt.js (integrated into main widgets)

Fixes comfy_dir path calculation by prioritizing folder_paths.base_path
from ComfyUI when available, with fallback to computed path.
2026-01-25 12:24:32 +08:00
Will Miao
50abd85fae fix(previews): temporarily bypass path validation to restore preview functionality
Temporary workaround for issues #772 and #774 where valid previews
are rejected. Path validation is disabled until proper fix for
preview root path handling is implemented.
2026-01-21 11:33:42 +08:00
Will Miao
839bcbd37f fix(settings): add default_unet_root to SYNC_KEYS for proper frontend sync
The default_unet_root setting was not being synced from backend to frontend
because it was missing from the _SYNC_KEYS tuple in misc_handlers.py. This
caused the "Default Diffusion Model Root" setting to always display "No Default"
even when a valid path was configured in settings.json.
2026-01-18 12:38:46 +08:00
Will Miao
428a2ce420 fix: support multiple include folders in LoRA pool widget
- Add folder_include parameter support in backend API handlers
- Add folder_include to FilterCriteria and implement multi-folder filtering logic
- Update frontend to send all include folders instead of only the first
- Add tests for single/multiple include folders, include with exclude, and non-recursive filtering
2026-01-15 15:17:33 +08:00
Will Miao
4951ff358e feat: add WSL and Docker support for file location opening
- Add WSL detection and Windows path conversion using wslpath
- Add Docker/Kubernetes detection via /.dockerenv and /proc/1/cgroup
- Implement clipboard fallback for containerized environments
- Update open_file_location handler to detect WSL/Docker before POSIX
- Update open_settings_location handler with same detection logic
- Add clipboard API integration with graceful fallback in frontend
- Add translations for clipboard feature across all 10 languages
- Add unit tests for _is_wsl(), _is_docker(), and _wsl_to_windows_path()

Fixes file manager opening failures in WSL and Docker environments.
2026-01-14 15:49:35 +08:00
Will Miao
bc08a45214 feat: improve code formatting and readability in model handlers
- Add blank line after module docstring for better PEP 8 compliance
- Reformat long lines to adhere to 88-character limit using Black-style formatting
- Improve string consistency by using double quotes consistently
- Enhance readability of complex list comprehensions and method calls
- Maintain all existing functionality while improving code structure
2026-01-13 22:57:15 +08:00
Will Miao
ab85ba54a9 feat: Implement recipe repair cancellation with UI support and refactor LoadingManager to a singleton. 2026-01-02 20:03:27 +08:00
Will Miao
837c32c42f feat: implement task cancellation for model scanning and bulk operations 2026-01-02 18:48:28 +08:00
Will Miao
afa5533145 feat: reduce log verbosity by changing info logs to debug
Changed logging level from INFO to DEBUG for performance-related messages in model management service. This reduces noise in production logs while maintaining debugging capability for performance analysis.
2025-12-31 16:14:24 +08:00
Will Miao
102defe29c feat: add open settings location endpoint
- Add `open_settings_location` method to `FileSystemHandler` to open OS file explorer at settings file location
- Register new POST route `/api/lm/settings/open-location` for settings file access
- Inject `SettingsManager` dependency into `FileSystemHandler` constructor
- Add cross-platform support for Windows, macOS, and Linux file explorers
- Include error handling for missing settings files and system exceptions
2025-12-31 16:09:23 +08:00
Will Miao
5d5a2a998a feat: Implement model move, import, and download functionalities with corresponding UI and API updates. 2025-12-28 21:18:27 +08:00
Will Miao
475906a25e perf: Add detailed performance logging to model retrieval, filtering, and sorting operations. see #711 2025-12-25 22:36:24 +08:00
Will Miao
6486107ca2 feat: Introduce recipe management with data models, scanning, enrichment, and repair for generation configurations. 2025-12-24 20:02:20 +08:00
Will Miao
6330c65d41 feat: Add recipe metadata repair functionality with UI, API, and progress tracking. 2025-12-23 21:50:58 +08:00
Will Miao
fc0a834beb feat: Introduce generation parameter merging from request, Civitai, and embedded image metadata, and enhance ComfyUI metadata parsing. 2025-12-23 15:31:04 +08:00
Will Miao
bc9dd317f7 feat: Add prompt search filter for recipes and fix 'Favorites' localization across multiple languages. 2025-12-23 10:52:12 +08:00
Will Miao
db7f09797b feat: Introduce recipe favoriting with star icon toggle and filter options. 2025-12-23 10:07:09 +08:00
Will Miao
6e64f97e2b feat: add bulk move recipes endpoint
Add new move_recipes_bulk endpoint to handle moving multiple recipes simultaneously. This improves efficiency when reorganizing recipe collections by allowing batch operations instead of individual moves.

- Add move_recipes_bulk handler method with proper error handling
- Register new POST /api/lm/recipes/move-bulk route
- Implement bulk move logic in persistence service
- Validate required parameters (recipe_ids and target_path)
- Handle common error cases including validation, not found, and server errors
2025-12-23 09:15:07 +08:00
Will Miao
3f646aa0c9 feat: add recipe root directory and move recipe endpoints
- Add GET /api/lm/recipes/roots endpoint to retrieve recipe root directories
- Add POST /api/lm/recipe/move endpoint to move recipes between directories
- Register new endpoints in route definitions
- Implement error handling for both new endpoints with proper status codes
- Enable recipe management operations for better file organization
2025-12-23 09:13:57 +08:00
Will Miao
67fb205b43 feat: add folder-based recipe organization and navigation
- Add new API endpoints for folder operations: get_folders, get_folder_tree, and get_unified_folder_tree
- Extend recipe listing handler to support folder and recursive filtering parameters
- Register new folder-related routes in route definitions
- Enable users to organize and browse recipes using folder structures
2025-12-23 09:12:27 +08:00
Will Miao
7caca0163e feat: Add support for remote video analysis and preview for recipe imports. see #420 2025-12-21 21:42:28 +08:00
Will Miao
30fd0470de feat: Add support for video recipe previews by conditionally optimizing media during persistence and updating UI components to display videos. 2025-12-21 20:00:44 +08:00
Will Miao
63b087fc80 feat: Implement cache busting for static assets, remove client-side version mismatch banner, and add project overview documentation. 2025-12-19 22:40:36 +08:00