Compare commits

..

1 Commits

Author SHA1 Message Date
Will Miao
68c5f79a67 Refactor showcase and modal components for improved functionality and performance
- Removed unused showcase toggle functionality from ModelCard and ModelModal.
- Simplified metadata panel handling in MediaUtils and MetadataPanel, transitioning to button-based visibility instead of hover.
- Enhanced showcase rendering logic in ShowcaseView to support new layout and navigation features.
- Updated event handling for media controls and thumbnail navigation to streamline user interactions.
- Improved example image import functionality and error handling.
- Cleaned up redundant code and comments across various components for better readability and maintainability.
2025-07-27 15:52:09 +08:00
521 changed files with 19087 additions and 128536 deletions

4
.github/FUNDING.yml vendored
View File

@@ -1,5 +1,5 @@
# These are supported funding model platforms # These are supported funding model platforms
ko_fi: pixelpawsai
patreon: PixelPawsAI patreon: PixelPawsAI
custom: ['paypal.me/pixelpawsai', 'https://afdian.com/a/pixelpawsai'] ko_fi: pixelpawsai
custom: ['paypal.me/pixelpawsai']

View File

@@ -1 +0,0 @@
Always use English for comments.

View File

@@ -1,93 +0,0 @@
name: Backend Tests
on:
push:
branches:
- main
- master
paths:
- 'py/**'
- 'standalone.py'
- 'tests/**'
- 'requirements.txt'
- 'requirements-dev.txt'
- 'pyproject.toml'
- 'pytest.ini'
- '.github/workflows/backend-tests.yml'
pull_request:
paths:
- 'py/**'
- 'standalone.py'
- 'tests/**'
- 'requirements.txt'
- 'requirements-dev.txt'
- 'pyproject.toml'
- 'pytest.ini'
- '.github/workflows/backend-tests.yml'
jobs:
pytest:
name: Run pytest with coverage
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: '3.11'
cache: 'pip'
cache-dependency-path: |
requirements.txt
requirements-dev.txt
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements-dev.txt
- name: Verify symlink support
run: |
python - <<'PY'
import os
import pathlib
import tempfile
root = pathlib.Path(tempfile.mkdtemp(prefix="lm-symlink-check-"))
target = root / "target"
target.mkdir()
link = root / "link"
try:
link.symlink_to(target, target_is_directory=True)
except OSError as exc:
raise SystemExit(f"Failed to create directory symlink in CI: {exc}")
is_link = os.path.islink(link)
is_dir = os.path.isdir(link)
realpath = os.path.realpath(link)
print(f"islink={is_link} isdir={is_dir} realpath={realpath}")
if not (is_link and is_dir and realpath == str(target)):
raise SystemExit("Directory symlink is not functioning correctly in CI; aborting.")
PY
- name: Run pytest with coverage
env:
COVERAGE_FILE: coverage/backend/.coverage
run: |
mkdir -p coverage/backend
python -m pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=html:coverage/backend/html \
--cov-report=json:coverage/backend/coverage.json
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: backend-coverage
path: coverage/backend
if-no-files-found: warn

View File

@@ -1,52 +0,0 @@
name: Frontend Tests
on:
push:
branches:
- main
- master
paths:
- 'package.json'
- 'package-lock.json'
- 'vitest.config.js'
- 'tests/frontend/**'
- 'static/js/**'
- 'scripts/run_frontend_coverage.js'
- '.github/workflows/frontend-tests.yml'
pull_request:
paths:
- 'package.json'
- 'package-lock.json'
- 'vitest.config.js'
- 'tests/frontend/**'
- 'static/js/**'
- 'scripts/run_frontend_coverage.js'
- '.github/workflows/frontend-tests.yml'
jobs:
vitest:
name: Run Vitest with coverage
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4
- name: Use Node.js 20
uses: actions/setup-node@v4
with:
node-version: 20
cache: 'npm'
- name: Install dependencies
run: npm ci
- name: Run frontend tests with coverage
run: npm run test:coverage
- name: Upload coverage artifact
if: always()
uses: actions/upload-artifact@v4
with:
name: frontend-coverage
path: coverage/frontend
if-no-files-found: warn

14
.gitignore vendored
View File

@@ -1,21 +1,7 @@
__pycache__/ __pycache__/
.pytest_cache/
settings.json settings.json
path_mappings.yaml path_mappings.yaml
output/* output/*
py/run_test.py py/run_test.py
.vscode/ .vscode/
cache/ cache/
civitai/
node_modules/
coverage/
.coverage
model_cache/
# agent
.opencode/
# Vue widgets development cache (but keep build output)
vue-widgets/node_modules/
vue-widgets/.vite/
vue-widgets/dist/

192
AGENTS.md
View File

@@ -1,192 +0,0 @@
# AGENTS.md
This file provides guidance for agentic coding assistants working in this repository.
## Development Commands
### Backend Development
```bash
# Install dependencies
pip install -r requirements.txt
pip install -r requirements-dev.txt
# Run standalone server (port 8188 by default)
python standalone.py --port 8188
# Run all backend tests
pytest
# Run specific test file
pytest tests/test_recipes.py
# Run specific test function
pytest tests/test_recipes.py::test_function_name
# Run backend tests with coverage
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
### Frontend Development
```bash
# Install frontend dependencies
npm install
# Run frontend tests
npm test
# Run frontend tests in watch mode
npm run test:watch
# Run frontend tests with coverage
npm run test:coverage
```
## Python Code Style
### Imports
- Use `from __future__ import annotations` for forward references in type hints
- Group imports: standard library, third-party, local (separated by blank lines)
- Use absolute imports within `py/` package: `from ..services import X`
- Mock ComfyUI dependencies in tests using `tests/conftest.py` patterns
### Formatting & Types
- PEP 8 with 4-space indentation
- Type hints required for function signatures and class attributes
- Use `TYPE_CHECKING` guard for type-checking-only imports
- Prefer dataclasses for simple data containers
- Use `Optional[T]` for nullable types, `Union[T, None]` only when necessary
### Naming Conventions
- Files: `snake_case.py` (e.g., `model_scanner.py`, `lora_service.py`)
- Classes: `PascalCase` (e.g., `ModelScanner`, `LoraService`)
- Functions/variables: `snake_case` (e.g., `get_instance`, `model_type`)
- Constants: `UPPER_SNAKE_CASE` (e.g., `VALID_LORA_TYPES`)
- Private members: `_single_underscore` (protected), `__double_underscore` (name-mangled)
### Error Handling
- Use `logging.getLogger(__name__)` for module-level loggers
- Define custom exceptions in `py/services/errors.py`
- Use `asyncio.Lock` for thread-safe singleton patterns
- Raise specific exceptions with descriptive messages
- Log errors at appropriate levels (DEBUG, INFO, WARNING, ERROR, CRITICAL)
### Async Patterns
- Use `async def` for I/O-bound operations
- Mark async tests with `@pytest.mark.asyncio`
- Use `async with` for context managers
- Singleton pattern with class-level locks: see `ModelScanner.get_instance()`
- Use `aiohttp.web.Response` for HTTP responses
### Testing Patterns
- Use `pytest` with `--import-mode=importlib`
- Fixtures in `tests/conftest.py` handle ComfyUI mocking
- Use `@pytest.mark.no_settings_dir_isolation` for tests needing real paths
- Test files: `tests/test_*.py`
- Use `tmp_path_factory` for temporary directory isolation
## JavaScript Code Style
### Imports & Modules
- ES modules with `import`/`export`
- Use `import { app } from "../../scripts/app.js"` for ComfyUI integration
- Export named functions/classes: `export function foo() {}`
- Widget files use `*_widget.js` suffix
### Naming & Formatting
- camelCase for functions, variables, object properties
- PascalCase for classes/constructors
- Constants: `UPPER_SNAKE_CASE` (e.g., `CONVERTED_TYPE`)
- Files: `snake_case.js` or `kebab-case.js`
- 2-space indentation preferred (follow existing file conventions)
### Widget Development
- Use `app.registerExtension()` to register ComfyUI extensions
- Use `node.addDOMWidget(name, type, element, options)` for custom widgets
- Event handlers attached via `addEventListener` or widget callbacks
- See `web/comfyui/utils.js` for shared utilities
## Architecture Patterns
### Service Layer
- Use `ServiceRegistry` singleton for dependency injection
- Services follow singleton pattern via `get_instance()` class method
- Separate scanners (discovery) from services (business logic)
- Handlers in `py/routes/handlers/` implement route logic
### Model Types
- BaseModelService is abstract base for LoRA, Checkpoint, Embedding services
- ModelScanner provides file discovery and hash-based deduplication
- Persistent cache in SQLite via `PersistentModelCache`
- Metadata sync from CivitAI/CivArchive via `MetadataSyncService`
### Routes & Handlers
- Route registrars organize endpoints by domain: `ModelRouteRegistrar`, etc.
- Handlers are pure functions taking dependencies as parameters
- Use `WebSocketManager` for real-time progress updates
- Return `aiohttp.web.json_response` or `web.Response`
### Recipe System
- Base metadata in `py/recipes/base.py`
- Enrichment adds model metadata: `RecipeEnrichmentService`
- Parsers for different formats in `py/recipes/parsers/`
## Important Notes
- Always use English for comments (per copilot-instructions.md)
- Dual mode: ComfyUI plugin (uses folder_paths) vs standalone (reads settings.json)
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
- Settings auto-saved in user directory or portable mode
- WebSocket broadcasts for real-time updates (downloads, scans)
- Symlink handling requires normalized paths
- API endpoints follow `/loras/*`, `/checkpoints/*`, `/embeddings/*` patterns
- Run `python scripts/sync_translation_keys.py` after UI string updates
## Frontend UI Architecture
This project has two distinct UI systems:
### 1. Standalone Lora Manager Web UI
- Location: `./static/` and `./templates/`
- Purpose: Full-featured web application for managing LoRA models
- Tech stack: Vanilla JS + CSS, served by the standalone server
- Development: Uses npm for frontend testing (`npm test`, `npm run test:watch`, etc.)
### 2. ComfyUI Custom Node Widgets
- Location: `./web/comfyui/`
- Purpose: Widgets and UI logic that ComfyUI loads as custom node extensions
- Tech stack: Vanilla JS + Vue.js widgets (in `./vue-widgets/` and built to `./web/comfyui/vue-widgets/`)
- Widget styling: Primary styles in `./web/comfyui/lm_styles.css` (NOT `./static/css/`)
- Development: No npm build step for these widgets (Vue widgets use build system)
### Widget Development Guidelines
- Use `app.registerExtension()` to register ComfyUI extensions (ComfyUI integration layer)
- Use `node.addDOMWidget()` for custom DOM widgets
- Widget styles should follow the patterns in `./web/comfyui/lm_styles.css`
- Selected state: `rgba(66, 153, 225, 0.3)` background, `rgba(66, 153, 225, 0.6)` border
- Hover state: `rgba(66, 153, 225, 0.2)` background
- Color palette matches the Lora Manager accent color (blue #4299e1)
- Use oklch() for color values when possible (defined in `./static/css/base.css`)
- Vue widget components are in `./vue-widgets/src/components/` and built to `./web/comfyui/vue-widgets/`
- When modifying widget styles, check `./web/comfyui/lm_styles.css` for consistency with other ComfyUI widgets

211
CLAUDE.md
View File

@@ -1,211 +0,0 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Overview
ComfyUI LoRA Manager is a comprehensive LoRA management system for ComfyUI that combines a Python backend with browser-based widgets. It provides model organization, downloading from CivitAI/CivArchive, recipe management, and one-click workflow integration.
## Development Commands
### Backend Development
```bash
# Install dependencies
pip install -r requirements.txt
# Install development dependencies (for testing)
pip install -r requirements-dev.txt
# Run standalone server (port 8188 by default)
python standalone.py --port 8188
# Run backend tests with coverage
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
# Run specific test file
pytest tests/test_recipes.py
```
### Frontend Development
```bash
# Install frontend dependencies
npm install
# Run frontend tests
npm test
# Run frontend tests in watch mode
npm run test:watch
# Run frontend tests with coverage
npm run test:coverage
```
### Localization
```bash
# Sync translation keys after UI string updates
python scripts/sync_translation_keys.py
```
## Architecture
### Backend Structure (Python)
**Core Entry Points:**
- `__init__.py` - ComfyUI plugin entry point, registers nodes and routes
- `standalone.py` - Standalone server that mocks ComfyUI dependencies
- `py/lora_manager.py` - Main LoraManager class that registers HTTP routes
**Service Layer** (`py/services/`):
- `ServiceRegistry` - Singleton service registry for dependency management
- `ModelServiceFactory` - Factory for creating model services (LoRA, Checkpoint, Embedding)
- Scanner services (`lora_scanner.py`, `checkpoint_scanner.py`, `embedding_scanner.py`) - Model file discovery and indexing
- `model_scanner.py` - Base scanner with hash-based deduplication and metadata extraction
- `persistent_model_cache.py` - SQLite-based cache for model metadata
- `metadata_sync_service.py` - Syncs metadata from CivitAI/CivArchive APIs
- `civitai_client.py` / `civarchive_client.py` - API clients for external services
- `downloader.py` / `download_manager.py` - Model download orchestration
- `recipe_scanner.py` - Recipe file management and image association
- `settings_manager.py` - Application settings with migration support
- `websocket_manager.py` - WebSocket broadcasting for real-time updates
- `use_cases/` - Business logic orchestration (auto-organize, bulk refresh, downloads)
**Routes Layer** (`py/routes/`):
- Route registrars organize endpoints by domain (models, recipes, previews, example images, updates)
- `handlers/` - Request handlers implementing business logic
- Routes use aiohttp and integrate with ComfyUI's PromptServer
**Recipe System** (`py/recipes/`):
- `base.py` - Base recipe metadata structure
- `enrichment.py` - Enriches recipes with model metadata
- `merger.py` - Merges recipe data from multiple sources
- `parsers/` - Parsers for different recipe formats (PNG, JSON, workflow)
**Custom Nodes** (`py/nodes/`):
- `lora_loader.py` - LoRA loader nodes with preset support
- `save_image.py` - Enhanced save image with pattern-based filenames
- `trigger_word_toggle.py` - Toggle trigger words in prompts
- `lora_stacker.py` - Stack multiple LoRAs
- `prompt.py` - Prompt node with autocomplete
- `wanvideo_lora_select.py` - WanVideo-specific LoRA selection
**Configuration** (`py/config.py`):
- Manages folder paths for models, checkpoints, embeddings
- Handles symlink mappings for complex directory structures
- Auto-saves paths to settings.json in ComfyUI mode
### Frontend Structure (JavaScript)
**ComfyUI Widgets** (`web/comfyui/`):
- Vanilla JavaScript ES modules extending ComfyUI's LiteGraph-based UI
- `loras_widget.js` - Main LoRA selection widget with preview
- `loras_widget_events.js` - Event handling for widget interactions
- `autocomplete.js` - Autocomplete for trigger words and embeddings
- `preview_tooltip.js` - Preview tooltip for model cards
- `top_menu_extension.js` - Adds "Launch LoRA Manager" menu item
- `trigger_word_highlight.js` - Syntax highlighting for trigger words
- `utils.js` - Shared utilities and API helpers
**Widget Development:**
- Widgets use `app.registerExtension` and `getCustomWidgets` hooks
- `node.addDOMWidget(name, type, element, options)` embeds HTML in nodes
- See `docs/dom_widget_dev_guide.md` for complete DOMWidget development guide
**Web Source** (`web-src/`):
- Modern frontend components (if migrating from static)
- `components/` - Reusable UI components
- `styles/` - CSS styling
### Key Patterns
**Dual Mode Operation:**
- ComfyUI plugin mode: Integrates with ComfyUI's PromptServer, uses folder_paths
- Standalone mode: Mocks ComfyUI dependencies via `standalone.py`, reads paths from settings.json
- Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
**Settings Management:**
- Settings stored in user directory (via `platformdirs`) or portable mode (in repo)
- Migration system tracks settings schema version
- Template in `settings.json.example` defines defaults
**Model Scanning Flow:**
1. Scanner walks folder paths, computes file hashes
2. Hash-based deduplication prevents duplicate processing
3. Metadata extracted from safetensors headers
4. Persistent cache stores results in SQLite
5. Background sync fetches CivitAI/CivArchive metadata
6. WebSocket broadcasts updates to connected clients
**Recipe System:**
- Recipes store LoRA combinations with parameters
- Supports import from workflow JSON, PNG metadata
- Images associated with recipes via sibling file detection
- Enrichment adds model metadata for display
**Frontend-Backend Communication:**
- REST API for CRUD operations
- WebSocket for real-time progress updates (downloads, scans)
- API endpoints follow `/loras/*` pattern
## Code Style
**Python:**
- PEP 8 with 4-space indentation
- snake_case for files, functions, variables
- PascalCase for classes
- Type hints preferred
- English comments only (per copilot-instructions.md)
- Loggers via `logging.getLogger(__name__)`
**JavaScript:**
- ES modules with camelCase
- Files use `*_widget.js` suffix for ComfyUI widgets
- Prefer vanilla JS, avoid framework dependencies
## Testing
**Backend Tests:**
- pytest with `--import-mode=importlib`
- Test files: `tests/test_*.py`
- Fixtures in `tests/conftest.py`
- Mock ComfyUI dependencies using standalone.py patterns
- Markers: `@pytest.mark.asyncio` for async tests, `@pytest.mark.no_settings_dir_isolation` for real paths
**Frontend Tests:**
- Vitest with jsdom environment
- Test files: `tests/frontend/**/*.test.js`
- Setup in `tests/frontend/setup.js`
- Coverage via `npm run test:coverage`
## Important Notes
**Settings Location:**
- ComfyUI mode: Auto-saves folder paths to user settings directory
- Standalone mode: Use `settings.json` (copy from `settings.json.example`)
- Portable mode: Set `"use_portable_settings": true` in settings.json
**API Integration:**
- CivitAI API key required for downloads (add to settings)
- CivArchive API used as fallback for deleted models
- Metadata archive database available for offline metadata
**Symlink Handling:**
- Config scans symlinks to map virtual paths to physical locations
- Preview validation uses normalized preview root paths
- Fingerprinting prevents redundant symlink rescans
**ComfyUI Node Development:**
- Nodes defined in `py/nodes/`, registered in `__init__.py`
- Frontend widgets in `web/comfyui/`, matched by node type
- Use `WEB_DIRECTORY = "./web/comfyui"` convention
**Recipe Image Association:**
- Recipes scan for sibling images in same directory
- Supports repair/migration of recipe image paths
- See `py/services/recipe_scanner.py` for implementation details

162
README.md
View File

@@ -34,62 +34,79 @@ Enhance your Civitai browsing experience with our companion browser extension! S
## Release Notes ## Release Notes
### v0.9.12 ### v0.8.20
* **LoRA Randomizer System** - Introduced a comprehensive LoRA randomization system featuring LoRA Pool and LoRA Randomizer nodes for flexible and dynamic generation workflows. * **LM Civitai Extension** - Released [browser extension through Chrome Web Store](https://chromewebstore.google.com/detail/lm-civitai-extension/capigligggeijgmocnaflanlbghnamgm?utm_source=item-share-cb) that works seamlessly with LoRA Manager to enhance Civitai browsing experience, showing which models are already in your local library, enabling one-click downloads, and providing queue and parallel download support
* **LoRA Randomizer Template** - Refer to the new "LoRA Randomizer" template workflow for detailed examples of flexible randomization modes, lock & reuse options, and other features. * **Enhanced Lora Loader** - Added support for nunchaku, improving convenience when working with ComfyUI-nunchaku workflows, plus new template workflows for quick onboarding
* **Recipe Folders** - Introduced a folder system for the Recipes page, allowing users to freely organize recipes just like they do with models. * **WanVideo Integration** - Introduced WanVideo Lora Select (LoraManager) node compatible with ComfyUI-WanVideoWrapper for streamlined lora usage in video workflows, including a template workflow to help you get started quickly
* **Recipe Bulk Operations** - Added bulk mode support for batch moving, deleting, and setting base models for selected recipes with intuitive controls like click-and-drag selection, drag-to-folder, and Ctrl+A (Select All).
* **Prompt Search & Sorting** - Search recipes by prompt content and sort by Recipe Name, Imported Date, or LoRA Count for better browsing.
* **Recipe Favorites** - Mark specific recipes as favorites for quick access.
* **Video Recipe Support** - Enabled support for video recipes (import via LM extension or URL; video file import not supported).
* **Performance Improvements** - Fixed performance issues for dramatically improved startup and loading speed. After first scan, subsequent loads are instant regardless of collection size.
* **ComfyUI Nodes 2.0 Support** - Basic support for ComfyUI Nodes 2.0.
### v0.9.10 ### v0.8.19
* **Smarter Update Matching** - Users can now choose to check and group updates by matching base model only or with no base-model constraint; version lists also support toggling between same-base versions or all versions. * **Analytics Dashboard** - Added new Statistics page providing comprehensive visual analysis of model collection and usage patterns for better library insights
* **Flexible Tag Filtering** - The filter panel now supports tag exclusion: click a tag to include, click again to exclude, and click a third time to clear, enabling stronger and more flexible tag filters. * **Target Node Selection** - Enhanced workflow integration with intelligent target choosing when sending LoRAs/recipes to workflows with multiple loader/stacker nodes; a visual selector now appears showing node color, type, ID, and title for precise targeting
* **License Visibility & Controls** - Model detail headers and ComfyUI preview popups now show Civitai license icons. The filter panel gains license include/exclude options, and a new global context menu action, "Refresh license metadata," fetches missing license data. * **Enhanced NSFW Controls** - Added support for setting NSFW levels on recipes with automatic content blurring based on user preferences
* **Recipe Improvements** - Recipes now allow importing with zero LoRAs, and recipe detail pages show the related checkpoint for easier reference. * **Customizable Card Display** - New display settings allowing users to choose whether card information and action buttons are always visible or only revealed on hover
* **Better ZIP Downloads** - When downloading models packaged in ZIPs, model files are extracted into the target model folder; ZIPs containing multiple model files (e.g., WanVideo high/low LoRA pairs) are added as separate models. * **Expanded Compatibility** - Added support for efficiency-nodes-comfyui in Save Recipe and Save Image nodes, plus fixed compatibility with ComfyUI_Custom_Nodes_AlekPet
* **Template Workflow Update** - Refreshed the "Illustrious Pony Example" template workflow with usage guidance for each LoRA Manager node.
* **Bug Fixes & Stability** - General fixes and stability improvements.
### v0.9.9 ### v0.8.18
* **Check for Updates Feature** - Users can now check for updates for all models or selected models in bulk mode. Models with available updates will display an "update available" badge on their model card, and users can filter to show only models with updates. * **Custom Example Images** - Added ability to import your own example images for LoRAs and checkpoints with automatic metadata extraction from embedded information
* **Model Versions Management** - Added a new Versions tab in the model modal that centralizes all versions of a model, providing download, delete, and ignore update functions. * **Enhanced Example Management** - New action buttons to set specific examples as previews or delete custom examples
* **Send Checkpoint to ComfyUI** - Users can now click the send button on a checkpoint card to send the checkpoint directly to the current workflow's checkpoint or diffusion model loader node in ComfyUI. * **Improved Duplicate Detection** - Enhanced "Find Duplicates" with hash verification feature to eliminate false positives when identifying duplicate models
* **Customizable Model Card Display** - Added a new setting that allows users to choose whether to display the model name or filename on model cards. * **Tag Management** - Added tag editing functionality allowing users to customize and manage model tags
* **New Path Template Placeholders** - Added new path template placeholders: `{model_name}` and `{version_name}` for more flexible organization. * **Advanced Selection Controls** - Implemented Ctrl+A shortcut for quickly selecting all filtered LoRAs, automatically entering bulk mode when needed
* **ComfyUI Auto Path Correction Setting** - Added a new setting within ComfyUI to enable or disable the auto path correction feature. * **Note**: Cache file functionality temporarily disabled pending rework
### v0.9.8 ### v0.8.17
* **Full CivArchive API Support** - Added complete support for the CivArchive API as a fallback metadata source beyond Civitai API. Models deleted from Civitai can now still retrieve metadata through the CivArchive API. * **Duplicate Model Detection** - Added "Find Duplicates" functionality for LoRAs and checkpoints using model file hash detection, enabling convenient viewing and batch deletion of duplicate models
* **Download Models from CivArchive** - Added support for downloading models directly from CivArchive, similar to downloading from Civitai. Simply click the Download button and paste the model URL to download the corresponding model. * **Enhanced URL Recipe Imports** - Optimized import recipe via URL functionality using CivitAI API calls instead of web scraping, now supporting all rated images (including NSFW) for recipe imports
* **Custom Priority Tags** - Introduced Custom Priority Tags feature, allowing users to define custom priority tags. These tags will appear as suggestions when editing tags or during auto organization/download using default paths, providing more precise and controlled folder organization. [Guide](https://github.com/willmiao/ComfyUI-Lora-Manager/wiki/Priority-Tags-Configuration-Guide) * **Improved TriggerWord Control** - Enhanced TriggerWord Toggle node with new default_active switch to set the initial state (active/inactive) when trigger words are added
* **Drag and Drop Tag Reordering** - Added drag and drop functionality to reorder tags in the tags edit mode for improved usability. * **Centralized Example Management** - Added "Migrate Existing Example Images" feature to consolidate downloaded example images from model folders into central storage with customizable naming patterns
* **Download Control in Example Images Panel** - Added stop control in the Download Example Images Panel for better download management. * **Intelligent Word Suggestions** - Implemented smart trigger word suggestions by reading class tokens and tag frequency from safetensors files, displaying recommendations when editing trigger words
* **Prompt (LoraManager) Node with Autocomplete** - Added new Prompt (LoraManager) node with autocomplete feature for adding embeddings. * **Model Version Management** - Added "Re-link to CivitAI" context menu option for connecting models to different CivitAI versions when needed
* **Lora Manager Nodes in Subgraphs** - Lora Manager nodes now support being placed within subgraphs for more flexible workflow organization.
### v0.9.6 ### v0.8.16
* **Metadata Archive Database Support** - Added the ability to download and utilize a metadata archive database, enabling access to metadata for models that have been deleted from CivitAI. * **Dramatic Startup Speed Improvement** - Added cache serialization mechanism for significantly faster loading times, especially beneficial for large model collections
* **App-Level Proxy Settings** - Introduced support for configuring a global proxy within the application, making it easier to use the manager behind network restrictions. * **Enhanced Refresh Options** - Extended functionality with "Full Rebuild (complete)" option alongside "Quick Refresh (incremental)" to fix potential memory cache issues without requiring application restart
* **Bug Fixes** - Various bug fixes for improved stability and reliability. * **Customizable Display Density** - Replaced compact mode with adjustable display density settings for personalized layout customization
* **Model Creator Information** - Added creator details to model information panels for better attribution
* **Improved WebP Support** - Enhanced Save Image node with workflow embedding capability for WebP format images
* **Direct Example Access** - Added "Open Example Images Folder" button to card interfaces for convenient browsing of downloaded model examples
* **Enhanced Compatibility** - Full ComfyUI Desktop support for "Send lora or recipe to workflow" functionality
* **Cache Management** - Added settings to clear existing cache files when needed
* **Bug Fixes & Stability** - Various improvements for overall reliability and performance
### v0.9.2 ### v0.8.15
* **Bulk Auto-Organization Action** - Added a new bulk auto-organization feature. You can now select multiple models and automatically organize them according to your current path template settings for streamlined management. * **Enhanced One-Click Integration** - Replaced copy button with direct send button allowing LoRAs/recipes to be sent directly to your current ComfyUI workflow without needing to paste
* **Bug Fixes** - Addressed several bugs to improve stability and reliability. * **Flexible Workflow Integration** - Click to append LoRAs/recipes to existing loader nodes or Shift+click to replace content, with additional right-click menu options for "Send to Workflow (Append)" or "Send to Workflow (Replace)"
* **Improved LoRA Loader Controls** - Added header drag functionality for proportional strength adjustment of all LoRAs simultaneously (including CLIP strengths when expanded)
* **Keyboard Navigation Support** - Implemented Page Up/Down for page scrolling, Home key to jump to top, and End key to jump to bottom for faster browsing through large collections
### v0.9.1 ### v0.8.14
* **Enhanced Bulk Operations** - Improved bulk operations with Marquee Selection and a bulk operation context menu, providing a more intuitive, desktop-application-like user experience. * **Virtualized Scrolling** - Completely rebuilt rendering mechanism for smooth browsing with no lag or freezing, now supporting virtually unlimited model collections with optimized layouts for large displays, improving space utilization and user experience
* **New Bulk Actions** - Added bulk operations for adding tags and setting base models to multiple models simultaneously. * **Compact Display Mode** - Added space-efficient view option that displays more cards per row (7 on 1080p, 8 on 2K, 10 on 4K)
* **Enhanced LoRA Node Functionality** - Comprehensive improvements to LoRA loader/stacker nodes including real-time trigger word updates (reflecting any change anywhere in the LoRA chain for precise updates) and expanded context menu with "Copy Notes" and "Copy Trigger Words" options for faster workflow
### v0.9.0 ### v0.8.13
* **UI Overhaul for Enhanced Navigation** - Replaced the top flat folder tags with a new folder sidebar and breadcrumb navigation system for a more intuitive folder browsing and selection experience. * **Enhanced Recipe Management** - Added "Find duplicates" feature to identify and batch delete duplicate recipes with duplicate detection notifications during imports
* **Dual-Mode Folder Sidebar** - The new folder sidebar offers two display modes: 'List Mode,' which mirrors the classic folder view, and 'Tree Mode,' which presents a hierarchical folder structure for effortless navigation through nested directories. * **Improved Source Tracking** - Source URLs are now saved with recipes imported via URL, allowing users to view original content with one click or manually edit links
* **Internationalization Support** - Introduced multi-language support, now available in English, Simplified Chinese, Traditional Chinese, Spanish, Japanese, Korean, French, Russian, and German. Feedback from native speakers is welcome to improve the translations. * **Advanced LoRA Control** - Double-click LoRAs in Loader/Stacker nodes to access expanded CLIP strength controls for more precise adjustments of model and CLIP strength separately
* **Automatic Filename Conflict Resolution** - Implemented automatic file renaming (`original name + short hash`) to prevent conflicts when downloading or moving models. * **Lycoris Model Support** - Added compatibility with Lycoris models for expanded creative options
* **Performance Optimizations & Bug Fixes** - Various performance improvements and bug fixes for a more stable and responsive experience. * **Bug Fixes & UX Improvements** - Resolved various issues and enhanced overall user experience with numerous optimizations
### v0.8.12
* **Enhanced Model Discovery** - Added alphabetical navigation bar to LoRAs page for faster browsing through large collections
* **Optimized Example Images** - Improved download logic to automatically refresh stale metadata before fetching example images
* **Model Exclusion System** - New right-click option to exclude specific LoRAs or checkpoints from management
* **Improved Showcase Experience** - Enhanced interaction in LoRA and checkpoint showcase areas for better usability
### v0.8.11
* **Offline Image Support** - Added functionality to download and save all model example images locally, ensuring access even when offline or if images are removed from CivitAI or the site is down
* **Resilient Download System** - Implemented pause/resume capability with checkpoint recovery that persists through restarts or unexpected exits
* **Bug Fixes & Stability** - Resolved various issues to enhance overall reliability and performance
### v0.8.10
* **Standalone Mode** - Run LoRA Manager independently from ComfyUI for a lightweight experience that works even with other stable diffusion interfaces
* **Portable Edition** - New one-click portable version for easy startup and updates in standalone mode
* **Enhanced Metadata Collection** - Added support for SamplerCustomAdvanced node in the metadata collector module
* **Improved UI Organization** - Optimized Lora Loader node height to display up to 5 LoRAs at once with scrolling capability for larger collections
[View Update History](./update_logs.md) [View Update History](./update_logs.md)
@@ -148,12 +165,10 @@ Enhance your Civitai browsing experience with our companion browser extension! S
### Option 2: **Portable Standalone Edition** (No ComfyUI required) ### Option 2: **Portable Standalone Edition** (No ComfyUI required)
1. Download the [Portable Package](https://github.com/willmiao/ComfyUI-Lora-Manager/releases/download/v0.9.8/lora_manager_portable.7z) 1. Download the [Portable Package](https://github.com/willmiao/ComfyUI-Lora-Manager/releases/download/v0.8.15/lora_manager_portable.7z)
2. Copy the provided `settings.json.example` file to create a new file named `settings.json` in `comfyui-lora-manager` folder. 2. Copy the provided `settings.json.example` file to create a new file named `settings.json` in `comfyui-lora-manager` folder
3. Edit the new `settings.json` to include your correct model folder paths and CivitAI API key 3. Edit `settings.json` to include your correct model folder paths and CivitAI API key
- Set `"use_portable_settings": true` if you want the configuration to remain inside the repository folder instead of your user settings directory.
4. Run run.bat 4. Run run.bat
- To change the startup port, edit `run.bat` and modify the parameter (e.g. `--port 9001`)
### Option 3: **Manual Installation** ### Option 3: **Manual Installation**
@@ -219,7 +234,7 @@ You can combine multiple patterns to create detailed, organized filenames for yo
You can now run LoRA Manager independently from ComfyUI: You can now run LoRA Manager independently from ComfyUI:
1. **For ComfyUI users**: 1. **For ComfyUI users**:
- Launch ComfyUI with LoRA Manager at least once to initialize the necessary path information in the `settings.json` file located in your user settings folder (see paths above). - Launch ComfyUI with LoRA Manager at least once to initialize the necessary path information in the `settings.json` file.
- Make sure dependencies are installed: `pip install -r requirements.txt` - Make sure dependencies are installed: `pip install -r requirements.txt`
- From your ComfyUI root directory, run: - From your ComfyUI root directory, run:
```bash ```bash
@@ -232,9 +247,8 @@ You can now run LoRA Manager independently from ComfyUI:
``` ```
2. **For non-ComfyUI users**: 2. **For non-ComfyUI users**:
- Copy the provided `settings.json.example` file to create a new file named `settings.json`. Update the API key, optional language, and folder paths only—the library registry is created automatically when LoRA Manager starts. - Copy the provided `settings.json.example` file to create a new file named `settings.json`
- Edit `settings.json` to include your correct model folder paths and CivitAI API key (you can leave the defaults until ready to configure them) - Edit `settings.json` to include your correct model folder paths and CivitAI API key
- Enable portable mode by setting `"use_portable_settings": true` if you prefer LoRA Manager to read and write the `settings.json` located in the project directory.
- Install required dependencies: `pip install -r requirements.txt` - Install required dependencies: `pip install -r requirements.txt`
- Run standalone mode: - Run standalone mode:
```bash ```bash
@@ -242,37 +256,8 @@ You can now run LoRA Manager independently from ComfyUI:
``` ```
- Access the interface through your browser at: `http://localhost:8188/loras` - Access the interface through your browser at: `http://localhost:8188/loras`
> **Note:** Existing installations automatically migrate the legacy `settings.json` from the plugin folder to the user settings directory the first time you launch this version.
This standalone mode provides a lightweight option for managing your model and recipe collection without needing to run the full ComfyUI environment, making it useful even for users who primarily use other stable diffusion interfaces. This standalone mode provides a lightweight option for managing your model and recipe collection without needing to run the full ComfyUI environment, making it useful even for users who primarily use other stable diffusion interfaces.
## Testing & Coverage
### Backend
Install the development dependencies and run pytest with coverage reports:
```bash
pip install -r requirements-dev.txt
COVERAGE_FILE=coverage/backend/.coverage pytest \
--cov=py \
--cov=standalone \
--cov-report=term-missing \
--cov-report=html:coverage/backend/html \
--cov-report=xml:coverage/backend/coverage.xml \
--cov-report=json:coverage/backend/coverage.json
```
HTML, XML, and JSON artifacts are stored under `coverage/backend/` so you can inspect hot spots locally or from CI artifacts.
### Frontend
Run the Vitest coverage suite to analyze widget hot spots:
```bash
npm run test:coverage
```
--- ---
## Contributing ## Contributing
@@ -313,6 +298,3 @@ Join our Discord community for support, discussions, and updates:
[Discord Server](https://discord.gg/vcqNrWVFvM) [Discord Server](https://discord.gg/vcqNrWVFvM)
--- ---
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=willmiao/ComfyUI-Lora-Manager&type=Date)](https://star-history.com/#willmiao/ComfyUI-Lora-Manager&Date)

View File

@@ -1,91 +1,27 @@
try: # pragma: no cover - import fallback for pytest collection from .py.lora_manager import LoraManager
from .py.lora_manager import LoraManager from .py.nodes.lora_loader import LoraManagerLoader
from .py.nodes.lora_loader import LoraManagerLoader, LoraManagerTextLoader from .py.nodes.trigger_word_toggle import TriggerWordToggle
from .py.nodes.trigger_word_toggle import TriggerWordToggle from .py.nodes.lora_stacker import LoraStacker
from .py.nodes.prompt import PromptLoraManager from .py.nodes.save_image import SaveImage
from .py.nodes.lora_stacker import LoraStacker from .py.nodes.debug_metadata import DebugMetadata
from .py.nodes.save_image import SaveImageLM from .py.nodes.wanvideo_lora_select import WanVideoLoraSelect
from .py.nodes.debug_metadata import DebugMetadata # Import metadata collector to install hooks on startup
from .py.nodes.wanvideo_lora_select import WanVideoLoraSelectLM from .py.metadata_collector import init as init_metadata_collector
from .py.nodes.wanvideo_lora_select_from_text import WanVideoLoraSelectFromText
from .py.nodes.lora_pool import LoraPoolNode
from .py.nodes.lora_randomizer import LoraRandomizerNode
from .py.metadata_collector import init as init_metadata_collector
except (
ImportError
): # pragma: no cover - allows running under pytest without package install
import importlib
import pathlib
import sys
package_root = pathlib.Path(__file__).resolve().parent
if str(package_root) not in sys.path:
sys.path.append(str(package_root))
PromptLoraManager = importlib.import_module("py.nodes.prompt").PromptLoraManager
LoraManager = importlib.import_module("py.lora_manager").LoraManager
LoraManagerLoader = importlib.import_module(
"py.nodes.lora_loader"
).LoraManagerLoader
LoraManagerTextLoader = importlib.import_module(
"py.nodes.lora_loader"
).LoraManagerTextLoader
TriggerWordToggle = importlib.import_module(
"py.nodes.trigger_word_toggle"
).TriggerWordToggle
LoraStacker = importlib.import_module("py.nodes.lora_stacker").LoraStacker
SaveImageLM = importlib.import_module("py.nodes.save_image").SaveImageLM
DebugMetadata = importlib.import_module("py.nodes.debug_metadata").DebugMetadata
WanVideoLoraSelectLM = importlib.import_module(
"py.nodes.wanvideo_lora_select"
).WanVideoLoraSelectLM
WanVideoLoraSelectFromText = importlib.import_module(
"py.nodes.wanvideo_lora_select_from_text"
).WanVideoLoraSelectFromText
LoraPoolNode = importlib.import_module("py.nodes.lora_pool").LoraPoolNode
LoraRandomizerNode = importlib.import_module(
"py.nodes.lora_randomizer"
).LoraRandomizerNode
init_metadata_collector = importlib.import_module("py.metadata_collector").init
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
PromptLoraManager.NAME: PromptLoraManager,
LoraManagerLoader.NAME: LoraManagerLoader, LoraManagerLoader.NAME: LoraManagerLoader,
LoraManagerTextLoader.NAME: LoraManagerTextLoader,
TriggerWordToggle.NAME: TriggerWordToggle, TriggerWordToggle.NAME: TriggerWordToggle,
LoraStacker.NAME: LoraStacker, LoraStacker.NAME: LoraStacker,
SaveImageLM.NAME: SaveImageLM, SaveImage.NAME: SaveImage,
DebugMetadata.NAME: DebugMetadata, DebugMetadata.NAME: DebugMetadata,
WanVideoLoraSelectLM.NAME: WanVideoLoraSelectLM, WanVideoLoraSelect.NAME: WanVideoLoraSelect
WanVideoLoraSelectFromText.NAME: WanVideoLoraSelectFromText,
LoraPoolNode.NAME: LoraPoolNode,
LoraRandomizerNode.NAME: LoraRandomizerNode,
} }
WEB_DIRECTORY = "./web/comfyui" WEB_DIRECTORY = "./web/comfyui"
# Check and build Vue widgets if needed (development mode)
try:
from .py.vue_widget_builder import check_and_build_vue_widgets
# Auto-build in development, warn only if fails
check_and_build_vue_widgets(auto_build=True, warn_only=True)
except ImportError:
# Fallback for pytest
import importlib
check_and_build_vue_widgets = importlib.import_module(
"py.vue_widget_builder"
).check_and_build_vue_widgets
check_and_build_vue_widgets(auto_build=True, warn_only=True)
except Exception as e:
import logging
logging.warning(f"[LoRA Manager] Vue widget build check skipped: {e}")
# Initialize metadata collector # Initialize metadata collector
init_metadata_collector() init_metadata_collector()
# Register routes on import # Register routes on import
LoraManager.add_routes() LoraManager.add_routes()
__all__ = ["NODE_CLASS_MAPPINGS", "WEB_DIRECTORY"] __all__ = ['NODE_CLASS_MAPPINGS', 'WEB_DIRECTORY']

View File

@@ -1,180 +0,0 @@
## Overview
The **LoRA Manager Civitai Extension** is a Browser extension designed to work seamlessly with [LoRA Manager](https://github.com/willmiao/ComfyUI-Lora-Manager) to significantly enhance your browsing experience on [Civitai](https://civitai.com).
It also supports browsing on [CivArchive](https://civarchive.com/) (formerly CivitaiArchive).
With this extension, you can:
✅ Instantly see which models are already present in your local library
✅ Download new models with a single click
✅ Manage downloads efficiently with queue and parallel download support
✅ Keep your downloaded models automatically organized according to your custom settings
![Civitai Models page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-models-page.png)
![CivArchive Models page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civarchive-models-page.png)
---
## Why Are All Features for Supporters Only?
I love building tools for the Stable Diffusion and ComfyUI communities, and LoRA Manager is a passion project that I've poured countless hours into. When I created this companion extension, my hope was to offer its core features for free, as a thank-you to all of you.
Unfortunately, I've reached a point where I need to be realistic. The level of support from the free model has been far lower than what's needed to justify the continuous development and maintenance for both projects. It was a difficult decision, but I've chosen to make the extension's features exclusive to supporters.
This change is crucial for me to be able to continue dedicating my time to improving the free and open-source LoRA Manager, which I'm committed to keeping available for everyone.
Your support does more than just unlock a few features—it allows me to keep innovating and ensures the core LoRA Manager project thrives. I'm incredibly grateful for your understanding and any support you can offer. ❤️
(_For those who previously supported me on Ko-fi with a one-time donation, I'll be sending out license keys individually as a thank-you._)
---
## Installation
### Supported Browsers & Installation Methods
| Browser | Installation Method |
|--------------------|-------------------------------------------------------------------------------------|
| **Google Chrome** | [Chrome Web Store link](https://chromewebstore.google.com/detail/capigligggeijgmocnaflanlbghnamgm?utm_source=item-share-cb) |
| **Microsoft Edge** | Install via Chrome Web Store (compatible) |
| **Brave Browser** | Install via Chrome Web Store (compatible) |
| **Opera** | Install via Chrome Web Store (compatible) |
| **Firefox** | <div id="firefox-install" class="install-ok"><a href="https://github.com/willmiao/lm-civitai-extension-firefox/releases/latest/download/extension.xpi">📦 Install Firefox Extension (reviewed and verified by Mozilla)</a></div> |
For non-Chrome browsers (e.g., Microsoft Edge), you can typically install extensions from the Chrome Web Store by following these steps: open the extensions Chrome Web Store page, click 'Get extension', then click 'Allow' when prompted to enable installations from other stores, and finally click 'Add extension' to complete the installation.
---
## Privacy & Security
I understand concerns around browser extensions and privacy, and I want to be fully transparent about how the **LM Civitai Extension** works:
- **Reviewed and Verified**
This extension has been **manually reviewed and approved by the Chrome Web Store**. The Firefox version uses the **exact same code** (only the packaging format differs) and has passed **Mozillas Add-on review**.
- **Minimal Network Access**
The only external server this extension connects to is:
**`https://willmiao.shop`** — used solely for **license validation**.
It does **not collect, transmit, or store any personal or usage data**.
No browsing history, no user IDs, no analytics, no hidden trackers.
- **Local-Only Model Detection**
Model detection and LoRA Manager communication all happen **locally** within your browser, directly interacting with your local LoRA Manager backend.
I value your trust and are committed to keeping your local setup private and secure. If you have any questions, feel free to reach out!
---
## How to Use
After installing the extension, you'll automatically receive a **7-day trial** to explore all features.
When the extension is correctly installed and your license is valid:
- Open **Civitai**, and you'll see visual indicators added by the extension on model cards, showing:
- ✅ Models already present in your local library
- ⬇️ A download button for models not in your library
Clicking the download button adds the corresponding model version to the download queue, waiting to be downloaded. You can set up to **5 models to download simultaneously**.
### Visual Indicators Appear On:
- **Home Page** — Featured models
- **Models Page**
- **Creator Profiles** — If the creator has set their models to be visible
- **Recommended Resources** — On individual model pages
### Version Buttons on Model Pages
On a specific model page, visual indicators also appear on version buttons, showing which versions are already in your local library.
When switching to a specific version by clicking a version button:
- Clicking the download button will open a dropdown:
- Download via **LoRA Manager**
- Download via **Original Download** (browser download)
You can check **Remember my choice** to set your preferred default. You can change this setting anytime in the extension's settings.
![Civitai Model Page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-model-page.png)
### Resources on Image Pages (2025-08-05) — now shows in-library indicators for image resources. Import image as recipe coming soon!
![Civitai Image Page](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/civitai-image-page.jpg)
---
## Model Download Location & LoRA Manager Settings
To use the **one-click download function**, you must first set:
- Your **Default LoRAs Root**
- Your **Default Checkpoints Root**
These are set within LoRA Manager's settings.
When everything is configured, downloaded model files will be placed in:
`<Default_Models_Root>/<Base_Model_of_the_Model>/<First_Tag_of_the_Model>`
### Update: Default Path Customization (2025-07-21)
A new setting to customize the default download path has been added in the nightly version. You can now personalize where models are saved when downloading via the LM Civitai Extension.
![Default Path Customization](https://github.com/willmiao/ComfyUI-Lora-Manager/blob/main/wiki-images/default-path-customization.png)
The previous YAML path mapping file will be deprecated—settings will now be unified in settings.json to simplify configuration.
---
## Backend Port Configuration
If your **ComfyUI** or **LoRA Manager** backend is running on a port **other than the default 8188**, you must configure the backend port in the extension's settings.
After correctly setting and saving the port, you'll see in the extension's header area:
- A **Healthy** status with the tooltip: `Connected to LoRA Manager on port xxxx`
---
## Advanced Usage
### Connecting to a Remote LoRA Manager
If your LoRA Manager is running on another computer, you can still connect from your browser using port forwarding.
> **Why can't you set a remote IP directly?**
>
> For privacy and security, the extension only requests access to `http://127.0.0.1/*`. Supporting remote IPs would require much broader permissions, which may be rejected by browser stores and could raise user concerns.
**Solution: Port Forwarding with `socat`**
On your browser computer, run:
`socat TCP-LISTEN:8188,bind=127.0.0.1,fork TCP:REMOTE.IP.ADDRESS.HERE:8188`
- Replace `REMOTE.IP.ADDRESS.HERE` with the IP of the machine running LoRA Manager.
- Adjust the port if needed.
This lets the extension connect to `127.0.0.1:8188` as usual, with traffic forwarded to your remote server.
_Thanks to user **Temikus** for sharing this solution!_
---
## Roadmap
The extension will evolve alongside **LoRA Manager** improvements. Planned features include:
- [x] Support for **additional model types** (e.g., embeddings)
- [ ] One-click **Recipe Import**
- [x] Display of in-library status for all resources in the **Resources Used** section of the image page
- [x] One-click **Auto-organize Models**
**Stay tuned — and thank you for your support!**
---

View File

@@ -1,93 +0,0 @@
# Example image route architecture
The example image routing stack mirrors the layered model route stack described in
[`docs/architecture/model_routes.md`](model_routes.md). HTTP wiring, controller setup,
handler orchestration, and long-running workflows now live in clearly separated modules so
we can extend download/import behaviour without touching the entire feature surface.
```mermaid
graph TD
subgraph HTTP
A[ExampleImagesRouteRegistrar] -->|binds| B[ExampleImagesRoutes controller]
end
subgraph Application
B --> C[ExampleImagesHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Download manager / processor / file manager]
end
subgraph Side Effects
F1 --> G1[Filesystem]
F1 --> G2[Model metadata]
F1 --> G3[WebSocket progress]
end
```
## Layer responsibilities
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/example_images_route_registrar.py` | Declarative catalogue of every example image endpoint plus helpers that bind them to an `aiohttp` router. Keeps HTTP concerns symmetrical with the model registrar. |
| Controller | `py/routes/example_images_routes.py` | Lazily constructs `ExampleImagesHandlerSet`, injects defaults for the download manager, processor, and file manager, and exposes the registrar-ready mapping just like `BaseModelRoutes`. |
| Handler set | `py/routes/handlers/example_images_handlers.py` | Groups HTTP adapters by concern (downloads, imports/deletes, filesystem access). Each handler translates domain errors into HTTP responses and defers to a use case or utility service. |
| Use cases | `py/services/use_cases/example_images/*.py` | Encapsulate orchestration for downloads and imports. They validate input, translate concurrency/configuration errors, and keep handler logic declarative. |
| Supporting services | `py/utils/example_images_download_manager.py`, `py/utils/example_images_processor.py`, `py/utils/example_images_file_manager.py` | Execute long-running work: pull assets from Civitai, persist uploads, clean metadata, expose filesystem actions with guardrails, and broadcast progress snapshots. |
## Handler responsibilities & invariants
`ExampleImagesHandlerSet` flattens the handler objects into the `{"handler_name": coroutine}`
mapping consumed by the registrar. The table below outlines how each handler collaborates
with the use cases and utilities.
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `ExampleImagesDownloadHandler` | `/api/lm/download-example-images`, `/api/lm/example-images-status`, `/api/lm/pause-example-images`, `/api/lm/resume-example-images`, `/api/lm/force-download-example-images` | `DownloadExampleImagesUseCase`, `DownloadManager` | Delegates payload validation and concurrency checks to the use case; progress/status endpoints expose the same snapshot used for WebSocket broadcasts; pause/resume surface `DownloadNotRunningError` as HTTP 400 instead of 500. |
| `ExampleImagesManagementHandler` | `/api/lm/import-example-images`, `/api/lm/delete-example-image` | `ImportExampleImagesUseCase`, `ExampleImagesProcessor` | Multipart uploads are streamed to disk via the use case; validation failures return HTTP 400 with no filesystem side effects; deletion funnels through the processor to prune metadata and cached images consistently. |
| `ExampleImagesFileHandler` | `/api/lm/open-example-images-folder`, `/api/lm/example-image-files`, `/api/lm/has-example-images` | `ExampleImagesFileManager` | Centralises filesystem access, enforcing settings-based root paths and returning HTTP 400/404 for missing configuration or folders; responses always include `success`/`has_images` booleans for UI consumption. |
## Use case boundaries
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `DownloadExampleImagesUseCase` | `execute(payload)` | `DownloadManager.start_download`, download configuration errors | Raises `DownloadExampleImagesInProgressError` when the manager reports an active job, rewraps configuration errors into `DownloadExampleImagesConfigurationError`, and lets `ExampleImagesDownloadError` bubble as 500s so handlers do not duplicate logging. |
| `ImportExampleImagesUseCase` | `execute(request)` | `ExampleImagesProcessor.import_images`, temporary file helpers | Supports multipart or JSON payloads, normalises file paths into a single list, cleans up temp files even on failure, and maps validation issues to `ImportExampleImagesValidationError` for HTTP 400 responses. |
## Maintaining critical invariants
* **Shared progress snapshots** - The download handler returns the same snapshot built by
`DownloadManager`, guaranteeing parity between HTTP polling endpoints and WebSocket
progress events.
* **Safe filesystem access** - All folder/file actions flow through
`ExampleImagesFileManager`, which validates the configured example image root and ensures
responses never leak absolute paths outside the allowed directory.
* **Metadata hygiene** - Import/delete operations run through `ExampleImagesProcessor`,
which updates model metadata via `MetadataManager` and notifies the relevant scanners so
cache state stays in sync.
## Migration notes
The refactor brings the example image stack in line with the model/recipe stacks:
1. `ExampleImagesRouteRegistrar` now owns the declarative route list. Downstream projects
should rely on `ExampleImagesRoutes.to_route_mapping()` instead of manually wiring
handler callables.
2. `ExampleImagesRoutes` caches its `ExampleImagesHandlerSet` just like
`BaseModelRoutes`. If you previously instantiated handlers directly, inject custom
collaborators via the controller constructor (`download_manager`, `processor`,
`file_manager`) to keep test seams predictable.
3. Tests that mocked `ExampleImagesRoutes.setup_routes` should switch to patching
`DownloadExampleImagesUseCase`/`ImportExampleImagesUseCase` at import time. The handlers
expect those abstractions to surface validation/concurrency errors, and bypassing them
will skip the HTTP-friendly error mapping.
## Extending the stack
1. Add the endpoint to `ROUTE_DEFINITIONS` with a unique `handler_name`.
2. Expose the coroutine on an existing handler class (or create a new handler and extend
`ExampleImagesHandlerSet`).
3. Wire additional services or factories inside `_build_handler_set` on
`ExampleImagesRoutes`, mirroring how the model stack introduces new use cases.
`tests/routes/test_example_images_routes.py` exercises registrar binding, download pause
flows, and import validations. Use it as a template when introducing new handler
collaborators or error mappings.

View File

@@ -1,100 +0,0 @@
# Base model route architecture
The model routing stack now splits HTTP wiring, orchestration logic, and
business rules into discrete layers. The goal is to make it obvious where a
new collaborator should live and which contract it must honour. The diagram
below captures the end-to-end flow for a typical request:
```mermaid
graph TD
subgraph HTTP
A[ModelRouteRegistrar] -->|binds| B[BaseModelRoutes handler proxy]
end
subgraph Application
B --> C[ModelHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Services / scanners]
end
subgraph Side Effects
F1 --> G1[Cache & metadata]
F1 --> G2[Filesystem]
F1 --> G3[WebSocket state]
end
```
Every box maps to a concrete module:
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/model_route_registrar.py` | Declarative list of routes shared by every model type and helper methods for binding them to an `aiohttp` application. |
| Route controller | `py/routes/base_model_routes.py` | Constructs the handler graph, injects shared services, exposes proxies that surface `503 Service not ready` when the model service has not been attached. |
| Handler set | `py/routes/handlers/model_handlers.py` | Thin HTTP adapters grouped by concern (page rendering, listings, mutations, queries, downloads, CivitAI integration, move operations, auto-organize). |
| Use cases | `py/services/use_cases/*.py` | Encapsulate long-running flows (`DownloadModelUseCase`, `BulkMetadataRefreshUseCase`, `AutoOrganizeUseCase`). They normalise validation errors and concurrency constraints before returning control to the handlers. |
| Services | `py/services/*.py` | Existing services and scanners that mutate caches, write metadata, move files, and broadcast WebSocket updates. |
## Handler responsibilities & contracts
`ModelHandlerSet` flattens the handler objects into the exact callables used by
the registrar. The table below highlights the separation of concerns within
the set and the invariants that must hold after each handler returns.
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `ModelPageView` | `/{prefix}` | `SettingsManager`, `server_i18n`, Jinja environment, `service.scanner` | Template is rendered with `is_initializing` flag when caches are cold; i18n filter is registered exactly once per environment instance. |
| `ModelListingHandler` | `/api/lm/{prefix}/list` | `service.get_paginated_data`, `service.format_response` | Listings respect pagination query parameters and cap `page_size` at 100; every item is formatted before response. |
| `ModelManagementHandler` | Mutations (delete, exclude, metadata, preview, tags, rename, bulk delete, duplicate verification) | `ModelLifecycleService`, `MetadataSyncService`, `PreviewAssetService`, `TagUpdateService`, scanner cache/index | Cache state mirrors filesystem changes: deletes prune cache & hash index, preview replacements synchronise metadata and cache NSFW levels, metadata saves trigger cache resort when names change. |
| `ModelQueryHandler` | Read-only queries (top tags, folders, duplicates, metadata, URLs) | Service query helpers & scanner cache | Outputs always wrapped in `{"success": True}` when no error; duplicate/filename grouping omits empty entries; invalid parameters (e.g. missing `model_root`) return HTTP 400. |
| `ModelDownloadHandler` | `/api/lm/download-model`, `/download-model-get`, `/download-progress/{id}`, `/cancel-download-get` | `DownloadModelUseCase`, `DownloadCoordinator`, `WebSocketManager` | Payload validation errors become HTTP 400 without mutating download progress cache; early-access failures surface as HTTP 401; successful downloads cache progress snapshots that back both WebSocket broadcasts and polling endpoints. |
| `ModelCivitaiHandler` | CivitAI metadata routes | `MetadataSyncService`, metadata provider factory, `BulkMetadataRefreshUseCase` | `fetch_all_civitai` streams progress via `WebSocketBroadcastCallback`; version lookups validate model type before returning; local availability fields derive from hash lookups without mutating cache state. |
| `ModelMoveHandler` | `move_model`, `move_models_bulk` | `ModelMoveService` | Moves execute atomically per request; bulk operations aggregate success/failure per file set. |
| `ModelAutoOrganizeHandler` | `/api/lm/{prefix}/auto-organize` (GET/POST), `/auto-organize-progress` | `AutoOrganizeUseCase`, `WebSocketProgressCallback`, `WebSocketManager` | Enforces single-flight execution using the shared lock; progress broadcasts remain available to polling clients until explicitly cleared; conflicts return HTTP 409 with a descriptive error. |
## Use case boundaries
Each use case exposes a narrow asynchronous API that hides the underlying
services. Their error mapping is essential for predictable HTTP responses.
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `DownloadModelUseCase` | `execute(payload)` | `DownloadCoordinator.schedule_download` | Translates `ValueError` into `DownloadModelValidationError` for HTTP 400, recognises early-access errors (`"401"` in message) and surfaces them as `DownloadModelEarlyAccessError`, forwards success dictionaries untouched. |
| `AutoOrganizeUseCase` | `execute(file_paths, progress_callback)` | `ModelFileService.auto_organize_models`, `WebSocketManager` lock | Guarded by `ws_manager` lock + status checks; raises `AutoOrganizeInProgressError` before invoking the file service when another run is already active. |
| `BulkMetadataRefreshUseCase` | `execute_with_error_handling(progress_callback)` | `MetadataSyncService`, `SettingsManager`, `WebSocketBroadcastCallback` | Iterates through cached models, applies metadata sync, emits progress snapshots that handlers broadcast unchanged. |
## Maintaining legacy contracts
The refactor preserves the invariants called out in the previous architecture
notes. The most critical ones are reiterated here to emphasise the
collaboration points:
1. **Cache mutations** Delete, exclude, rename, and bulk delete operations are
channelled through `ModelManagementHandler`. The handler delegates to
`ModelLifecycleService` or `MetadataSyncService`, and the scanner cache is
mutated in-place before the handler returns. The accompanying tests assert
that `scanner._cache.raw_data` and `scanner._hash_index` stay in sync after
each mutation.
2. **Preview updates** `PreviewAssetService.replace_preview` writes the new
asset, `MetadataSyncService` persists the JSON metadata, and
`scanner.update_preview_in_cache` mirrors the change. The handler returns
the static URL produced by `config.get_preview_static_url`, keeping browser
clients in lockstep with disk state.
3. **Download progress** `DownloadCoordinator.schedule_download` generates the
download identifier, registers a WebSocket progress callback, and caches the
latest numeric progress via `WebSocketManager`. Both `download_model`
responses and `/download-progress/{id}` polling read from the same cache to
guarantee consistent progress reporting across transports.
## Extending the stack
To add a new shared route:
1. Declare it in `COMMON_ROUTE_DEFINITIONS` using a unique handler name.
2. Implement the corresponding coroutine on one of the handlers inside
`ModelHandlerSet` (or introduce a new handler class when the concern does not
fit existing ones).
3. Inject additional dependencies in `BaseModelRoutes._create_handler_set` by
wiring services or use cases through the constructor parameters.
Model-specific routes should continue to be registered inside the subclass
implementation of `setup_specific_routes`, reusing the shared registrar where
possible.

View File

@@ -1,34 +0,0 @@
# Multi-Library Management for Standalone Mode
## Requirements Summary
- **Independent libraries**: In standalone mode, users can maintain multiple libraries, where each library represents a distinct set of model folders (LoRAs, checkpoints, embeddings, etc.). Only one library is active at any given time, but users need a fast way to switch between them.
- **Library-specific settings**: The fields that vary per library are `folder_paths`, `default_lora_root`, `default_checkpoint_root`, and `default_embedding_root` inside `settings.json`.
- **Persistent caches**: Every library must have its own SQLite persistent model cache so that metadata generated for one library does not leak into another.
- **Backward compatibility**: Existing single-library setups should continue to work. When no multi-library configuration is provided, the application should behave exactly as before.
## Proposed Design
1. **Library registry**
- Extend the standalone configuration to hold a list of libraries, each identified by a unique name.
- Each entry stores the folder path configuration plus any library-scoped metadata (e.g. creation time, display name).
- The active library key is stored separately to allow quick switching without rewriting the full config.
2. **Settings management**
- Update `settings_manager` to load and persist the library registry. When a library is activated, hydrate the in-memory settings object with that library's folder configuration.
- Provide helper methods for creating, renaming, and deleting libraries, ensuring validation for duplicate names and path collisions.
- Continue writing the active library settings to `settings.json` for compatibility, while storing the registry in a new section such as `libraries`.
3. **Persistent model cache**
- Derive the SQLite file path from the active library, e.g. `model_cache_<library>.sqlite` or a nested directory structure like `model_cache/<library>/models.sqlite`.
- Update `PersistentModelCache` so it resolves the database path dynamically whenever the active library changes. Ensure connections are closed before switching to avoid locking issues.
- Migrate existing single cache files by treating them as the default library's cache.
4. **Model scanning workflow**
- Modify `ModelScanner` and related services to react to library switches by clearing in-memory caches, re-reading folder paths, and rehydrating metadata from the library-specific SQLite cache.
- Provide API endpoints in standalone mode to list libraries, activate one, and trigger a rescan.
5. **UI/UX considerations**
- In the standalone UI, introduce a library selector component that surfaces available libraries and offers quick switching.
- Offer feedback when switching libraries (e.g. spinner while rescanning) and guard destructive actions with confirmation prompts.
## Implementation Notes
- **Data migration**: On startup, detect if the old `settings.json` structure is present. If so, create a default library entry using the current folder paths and point the active library to it.
- **Thread safety**: Ensure that any long-running scans are cancelled or awaited before switching libraries to prevent race conditions in cache writes.
- **Testing**: Add unit tests for the settings manager to cover library CRUD operations and cache path resolution. Include integration tests that simulate switching libraries and verifying that the correct models are loaded.
- **Documentation**: Update user guides to explain how to define libraries, switch between them, and where the new cache files are stored.
- **Extensibility**: Keep the design open to future per-library settings (e.g. auto-refresh intervals, metadata overrides) by storing library data as objects instead of flat maps.

View File

@@ -1,89 +0,0 @@
# Recipe route architecture
The recipe routing stack now mirrors the modular model route design. HTTP
bindings, controller wiring, handler orchestration, and business rules live in
separate layers so new behaviours can be added without re-threading the entire
feature. The diagram below outlines the flow for a typical request:
```mermaid
graph TD
subgraph HTTP
A[RecipeRouteRegistrar] -->|binds| B[RecipeRoutes controller]
end
subgraph Application
B --> C[RecipeHandlerSet]
C --> D1[Handlers]
D1 --> E1[Use cases]
E1 --> F1[Services / scanners]
end
subgraph Side Effects
F1 --> G1[Cache & fingerprint index]
F1 --> G2[Metadata files]
F1 --> G3[Temporary shares]
end
```
## Layer responsibilities
| Layer | Module(s) | Responsibility |
| --- | --- | --- |
| Registrar | `py/routes/recipe_route_registrar.py` | Declarative list of every recipe endpoint and helper methods that bind them to an `aiohttp` application. |
| Controller | `py/routes/base_recipe_routes.py`, `py/routes/recipe_routes.py` | Lazily resolves scanners/clients from the service registry, wires shared templates/i18n, instantiates `RecipeHandlerSet`, and exposes a `{handler_name: coroutine}` mapping for the registrar. |
| Handler set | `py/routes/handlers/recipe_handlers.py` | Thin HTTP adapters grouped by concern (page view, listings, queries, mutations, sharing). They normalise responses and translate service exceptions into HTTP status codes. |
| Services & scanners | `py/services/recipes/*.py`, `py/services/recipe_scanner.py`, `py/services/service_registry.py` | Concrete business logic: metadata parsing, persistence, sharing, fingerprint/index maintenance, and cache refresh. |
## Handler responsibilities & invariants
`RecipeHandlerSet` flattens purpose-built handler objects into the callables the
registrar binds. Each handler is responsible for a narrow concern and enforces a
set of invariants before returning:
| Handler | Key endpoints | Collaborators | Contracts |
| --- | --- | --- | --- |
| `RecipePageView` | `/loras/recipes` | `SettingsManager`, `server_i18n`, Jinja environment, recipe scanner getter | Template rendered with `is_initializing` flag when caches are still warming; i18n filter registered exactly once per environment instance. |
| `RecipeListingHandler` | `/api/lm/recipes`, `/api/lm/recipe/{id}` | `recipe_scanner.get_paginated_data`, `recipe_scanner.get_recipe_by_id` | Listings respect pagination and search filters; every item receives a `file_url` fallback even when metadata is incomplete; missing recipes become HTTP 404. |
| `RecipeQueryHandler` | Tag/base-model stats, syntax, LoRA lookups | Recipe scanner cache, `format_recipe_file_url` helper | Cache snapshots are reused without forcing refresh; duplicate lookups collapse groups by fingerprint; syntax lookups return helpful errors when LoRAs are absent. |
| `RecipeManagementHandler` | Save, update, reconnect, bulk delete, widget ingest | `RecipePersistenceService`, `RecipeAnalysisService`, recipe scanner | Persistence results propagate HTTP status codes; fingerprint/index updates flow through the scanner before returning; validation errors surface as HTTP 400 without touching disk. |
| `RecipeAnalysisHandler` | Uploaded/local/remote analysis | `RecipeAnalysisService`, `civitai_client`, recipe scanner | Unsupported content types map to HTTP 400; download errors (`RecipeDownloadError`) are not retried; every response includes a `loras` array for client compatibility. |
| `RecipeSharingHandler` | Share + download | `RecipeSharingService`, recipe scanner | Share responses provide a stable download URL and filename; expired shares surface as HTTP 404; downloads stream via `web.FileResponse` with attachment headers. |
## Use case boundaries
The dedicated services encapsulate long-running work so handlers stay thin.
| Use case | Entry point | Dependencies | Guarantees |
| --- | --- | --- | --- |
| `RecipeAnalysisService` | `analyze_uploaded_image`, `analyze_remote_image`, `analyze_local_image`, `analyze_widget_metadata` | `ExifUtils`, `RecipeParserFactory`, downloader factory, optional metadata collector/processor | Normalises missing/invalid payloads into `RecipeValidationError`; generates consistent fingerprint data to keep duplicate detection stable; temporary files are cleaned up after every analysis path. |
| `RecipePersistenceService` | `save_recipe`, `delete_recipe`, `update_recipe`, `reconnect_lora`, `bulk_delete`, `save_recipe_from_widget` | `ExifUtils`, recipe scanner, card preview sizing constants | Writes images/JSON metadata atomically; updates scanner caches and hash indices before returning; recalculates fingerprints whenever LoRA assignments change. |
| `RecipeSharingService` | `share_recipe`, `prepare_download` | `tempfile`, recipe scanner | Copies originals to TTL-managed temp files; metadata lookups re-use the scanner; expired shares trigger cleanup and `RecipeNotFoundError`. |
## Maintaining critical invariants
* **Cache updates** Mutations (`save`, `delete`, `bulk_delete`, `update`) call
back into the recipe scanner to mutate the in-memory cache and fingerprint
index before returning a response. Tests assert that these methods are invoked
even when stubbing persistence.
* **Fingerprint management** `RecipePersistenceService` recomputes
fingerprints whenever LoRA metadata changes and duplicate lookups use those
fingerprints to group recipes. Handlers bubble the resulting IDs so clients
can merge duplicates without an extra fetch.
* **Metadata synchronisation** Saving or reconnecting a recipe updates the
JSON sidecar, refreshes embedded metadata via `ExifUtils`, and instructs the
scanner to resort its cache. Sharing relies on this metadata to generate
filenames and ensure downloads stay in sync with on-disk state.
## Extending the stack
1. Declare the new endpoint in `ROUTE_DEFINITIONS` with a unique handler name.
2. Implement the coroutine on an existing handler or introduce a new handler
class inside `py/routes/handlers/recipe_handlers.py` when the concern does
not fit existing ones.
3. Wire additional collaborators inside
`BaseRecipeRoutes._create_handler_set` (inject new services or factories) and
expose helper getters on the handler owner if the handler needs to share
utilities.
Integration tests in `tests/routes/test_recipe_routes.py` exercise the listing,
mutation, analysis-error, and sharing paths end-to-end, ensuring the controller
and handler wiring remains valid as new capabilities are added.

View File

@@ -1,46 +0,0 @@
# Custom Priority Tag Format Proposal
To support user-defined priority tags with flexible aliasing across different model types, the configuration will be stored as editable strings. The format balances readability with enough structure for parsing on both the backend and frontend.
## Format Overview
- Each model type is declared on its own line: `model_type: entries`.
- Entries are comma-separated and ordered by priority from highest to lowest.
- An entry may be a single canonical tag (e.g., `realistic`) or a canonical tag with aliases.
- Canonical tags define the final folder name that should be used when matching that entry.
- Aliases are enclosed in parentheses and separated by `|` (vertical bar).
- All matching is case-insensitive; stored canonical names preserve the user-specified casing for folder creation and UI suggestions.
### Grammar
```
priority-config := model-config { "\n" model-config }
model-config := model-type ":" entry-list
model-type := <identifier without spaces>
entry-list := entry { "," entry }
entry := canonical [ "(" alias { "|" alias } ")" ]
canonical := <tag text without parentheses or commas>
alias := <tag text without parentheses, commas, or pipes>
```
Examples:
```
lora: celebrity(celeb|celebrity), stylized, character(char)
checkpoint: realistic(realism|realistic), anime(anime-style|toon)
embedding: face, celeb(celebrity|celeb)
```
## Parsing Notes
- Whitespace around separators is ignored to make manual editing more forgiving.
- Duplicate canonical tags within the same model type collapse to a single entry; the first definition wins.
- Aliases map to their canonical tag. When generating folder names, the canonical form is used.
- Tags that do not match any alias or canonical entry fall back to the first tag in the model's tag list, preserving current behavior.
## Usage
- **Backend:** Convert each model type's string into an ordered list of canonical tags with alias sets. During path generation, iterate by priority order and match tags against both canonical names and their aliases.
- **Frontend:** Surface canonical tags as suggestions, optionally displaying aliases in tooltips or secondary text. Input validation should warn about duplicate aliases within the same model type.
This format allows users to customize priority tag handling per model type while keeping editing simple and avoiding proliferation of folder names through alias normalization.

View File

@@ -1,544 +0,0 @@
# DOMWidget Development Guide
This document provides a comprehensive guide for developing custom DOMWidgets in ComfyUI using Vanilla JavaScript. DOMWidgets allow you to embed standard HTML elements (div, video, canvas, input, etc.) into ComfyUI nodes while benefitting from the frontend's automatic layout and zoom management.
## 1. Core Concepts
In ComfyUI, a `DOMWidget` extends the default LiteGraph Canvas rendering logic. It maintains an HTML layer on top of the Canvas, making complex interactions and media displays significantly easier to implement than pure Canvas drawing.
### Key APIs
* **`app.registerExtension`**: The entry point for registering extensions.
* **`getCustomWidgets`**: A hook for defining new widget types associated with specific input types.
* **`node.addDOMWidget`**: The core method to add HTML elements to a node.
---
## 2. Basic Structure
A standard custom DOMWidget extension typically follows this structure:
```javascript
import { app } from "../../scripts/app.js";
app.registerExtension({
name: "My.Custom.Extension",
async getCustomWidgets() {
return {
// Define a new widget type named "MY_WIDGET_TYPE"
MY_WIDGET_TYPE(node, inputName, inputData, app) {
// 1. Create the HTML element
const container = document.createElement("div");
container.innerHTML = "Hello <b>DOMWidget</b>!";
// 2. Setup styles (Optional but recommended)
container.style.color = "white";
container.style.backgroundColor = "#222";
container.style.padding = "5px";
// 3. Add the DOMWidget and return the result
const widget = node.addDOMWidget(inputName, "MY_WIDGET_TYPE", container, {
// Configuration options
getValue() {
return container.innerText;
},
setValue(v) {
container.innerText = v;
}
});
// 4. Return in the standard format
return { widget };
}
};
}
});
```
---
## ComfyUI Dual Rendering Modes
ComfyUI frontend supports two rendering modes:
| Mode | Description | DOM Structure |
| :--- | :--- | :--- |
| **Canvas Mode** | Traditional rendering where widgets are rendered on top of canvas using absolute positioning | Uses `.dom-widget` class on containers |
| **Vue DOM Mode** | New rendering mode where nodes and widgets are rendered as Vue components | Uses `.lg-node-widget` class on containers with dynamic IDs (e.g., `v-1-0`) |
### Mode Switching
The frontend switches between modes via `LiteGraph.vueNodesMode` boolean:
- `LiteGraph.vueNodesMode = true` → Vue DOM Mode
- `LiteGraph.vueNodesMode = false` → Canvas Mode
**Key Behavior**: Mode switching triggers DOM re-rendering WITHOUT page reload. Widget elements are destroyed and recreated, so any event listeners or references to old DOM elements become invalid.
### Testing Mode Switches via Chrome DevTools MCP
```javascript
// Trigger render mode change
LiteGraph.vueNodesMode = !LiteGraph.vueNodesMode;
// Force canvas redraw (optional but helps trigger re-render)
if (app.canvas) {
app.canvas.draw(true, true);
}
```
### Development Notes
When implementing widgets that attach event listeners or maintain external references:
1. **Use `node.onRemoved`** to clean up when node is deleted
2. **Detect DOM changes** by checking if widget input element is still in document: `document.body.contains(inputElement)`
3. **Poll for mode changes** by watching `LiteGraph.vueNodesMode` and re-initializing when it changes
4. **Use `loadedGraphNode` hook** for initial setup (guarantees DOM is fully rendered)
---
## 3. The `addDOMWidget` API
```javascript
node.addDOMWidget(name, type, element, options)
```
### Parameters
1. **`name`**: The internal name of the widget (usually matches the input name).
2. **`type`**: The type identifier for the widget.
3. **`element`**: The actual HTMLElement to embed.
4. **`options`**: (Object) Configuration for lifecycle, sizing, and persistence.
### Common `options` Fields
| Field | Type | Description |
| :--- | :--- | :--- |
| `getValue` | `Function` | Defines how to retrieve the widget's value for serialization. |
| `setValue` | `Function` | Defines how to restore the widget's state from workflow data. |
| `getMinHeight` | `Function` | Returns the minimum height in pixels. |
| `getHeight` | `Function` | Returns the preferred height (supports numbers or percentage strings like `"50%"`). |
| `onResize` | `Function` | Callback triggered when the widget is resized. |
| `hideOnZoom`| `Boolean` | Whether to hide the DOM element when zoomed out to improve performance (default: `true`). |
| `selectOn` | `string[]` | Events on the element that should trigger node selection (default: `['focus', 'click']`). |
---
## 4. Size Control
Custom DOMWidgets must actively inform the parent Node of their size requirements to ensure the Node layout is calculated correctly and connection wires remain aligned.
### 4.1 Core Mechanism
Whether in Canvas Mode or Vue Mode, the underlying logic model (`LGraphNode`) calls the widget's `computeLayoutSize` method to determine dimensions. This logic is used to calculate the Node's total size and the position of input/output slots.
### 4.2 Controlling Height
It is recommended to use the `options` parameter to define height behavior.
**Performance Note:** providing `getMinHeight` and `getHeight` via `options` allows the system to skip expensive DOM measurements (`getComputedStyle`) during rendering loop. This significantly improves performance and prevents FPS drops during node resizing.
**Method 1: Using `options` (Recommended)**
```javascript
const widget = node.addDOMWidget("MyWidget", "custom", element, {
// Specify minimum height in pixels
getMinHeight: () => 150,
// Or specify preferred height (pixels or percentage string)
// getHeight: () => "50%",
});
```
**Method 2: Using CSS Variables**
You can also set specific CSS variables on the root element:
```javascript
element.style.setProperty("--comfy-widget-min-height", "150px");
// or --comfy-widget-height
```
### 4.3 Controlling Width
By default, a DOMWidget's width automatically stretches to fit the Node's width (which is determined by the Title or other Input Slots).
If you must **force the Node to be wider** to accommodate your widget, you need to override the widget instance's `computeLayoutSize` method:
```javascript
const widget = node.addDOMWidget("WideWidget", "custom", element);
// Override the default layout calculation
widget.computeLayoutSize = (targetNode) => {
return {
minHeight: 150, // Must return height
minWidth: 300 // Force the Node to be at least 300px wide
};
};
```
### 4.4 Dynamic Resizing
If your widget's content changes dynamically (e.g., expanding sections, loading images, or CSS changes), the DOM element will resize, but the Canvas-rendered Node background and Slots will not automatically follow. You must manually trigger a synchronization.
**The Update Sequence:**
Whenever the **actual rendering height** of your DOM element changes, execute the following "three-step combo":
```javascript
// 1. Calculate the new optimal size for the node based on current widget requirements
const newSize = node.computeSize();
// 2. Apply the new size to the node model (updates bounding box and slot positions)
node.setSize(newSize);
// 3. Mark the canvas as dirty to trigger a redraw in the next animation frame
node.setDirtyCanvas(true, true);
```
**Common Scenarios:**
| Scenario | Actual Height Change? | Update Required? |
| :--- | :--- | :--- |
| **Expand/Collapse content** | **Yes** | ✅ **Yes**. Prevents widget from overflowing node boundaries. |
| **Image/Video finished loading** | **Yes** | ✅ **Yes**. Initial height might be 0 until the media loads. |
| **Changing `minHeight`** | **Maybe** | ❓ **Only if** the change causes the element's actual height to shift. |
| **Changing font size/styles** | **Yes** | ✅ **Yes**. Text reflow often changes the total height. |
| **User dragging node corner** | **Yes** | ❌ **No**. LiteGraph handles this internally. |
---
## 5. State Persistence (Serialization)
### 5.1 Default Behavior
DOMWidgets have **serialization enabled** by default (`serialize` property is `true`).
* **Saving**: ComfyUI attempts to read the widget's value to save into the Workflow file.
* **Loading**: ComfyUI reads the value from the Workflow file and assigns it to the widget.
### 5.2 Custom Serialization
To make persistence work effectively (saving internal DOM state and restoring it), you must implement `getValue` and `setValue` in the `options`:
* **`getValue`**: Returns the state to be saved (Number, String, or Object).
* **`setValue`**: Receives the restored value and updates the DOM element.
**Example:**
```javascript
const inputEl = document.createElement("input");
const widget = node.addDOMWidget("MyInput", "custom", inputEl, {
// 1. Called during Save
getValue: () => {
return inputEl.value;
},
// 2. Called during Load or Copy/Paste
setValue: (value) => {
inputEl.value = value || "";
}
});
// Optional: Listen for changes to update widget.value immediately
inputEl.addEventListener("change", () => {
widget.value = inputEl.value; // Triggers callbacks
});
```
### 5.3 The Restoration Mechanism (`configure`)
* **`configure(data)`**: When a Workflow is loaded, `LGraphNode` calls its `configure(data)` method.
* **`setValue` Chain**: During `configure`, the Node iterates over the saved `widgets_values` array and assigns each value (`widget.value = savedValue`). For DOMWidgets, this assignment triggers the `setValue` callback defined in your options.
Therefore, `options.setValue` is the critical hook for restoring widget state.
### 5.4 Disabling Serialization
If your widget is purely for display (e.g., a real-time monitor or generated chart) and doesn't need to save state, disable serialization to reduce workflow file size.
**Note**: You cannot set this via `options`. You must modify the widget instance directly.
```javascript
const widget = node.addDOMWidget("DisplayOnly", "custom", element);
widget.serialize = false; // Explicitly disable
```
---
## 6. Lifecycle & Events
### 6.1 `onResize`
When the Node size changes (e.g., user drags the corner), the widget can receive a notification via `options`:
```javascript
const widget = node.addDOMWidget("ResizingWidget", "custom", element, {
onResize: (w) => {
// 'w' is the widget instance
// Adjust internal DOM layout here if necessary
console.log("Widget resized");
}
});
```
### 6.2 Construction & Mounting
* **Construction**: Occurs immediately when `addDOMWidget` is called.
* **Mounting**:
* **Canvas Mode**: Appended to `.dom-widget-container` via `DomWidget.vue`.
* **Vue Mode**: Appended inside the Node component via `WidgetDOM.vue`.
* **Caution**: When `addDOMWidget` returns, the element may not be in the `document.body` yet. If you need to access layout properties like `getBoundingClientRect`, use `setTimeout` or wait for the first `onResize`.
### 6.3 Cleanup
If you create external references (like `setInterval` or global event listeners), ensure you clean them up using `node.onRemoved`:
```javascript
node.onRemoved = function() {
clearInterval(myInterval);
// Call original onRemoved if it existed
};
```
---
## 7. Styling & Best Practices
### 7.1 Styling
Since DOMWidgets are placed in absolute positioned containers or managed by Vue, ensure your container handles sizing gracefully:
```javascript
container.style.width = "100%";
container.style.boxSizing = "border-box";
```
### 7.2 Path References
When importing `app`, adjust the path based on your extension's folder depth. Typically:
`import { app } from "../../scripts/app.js";`
### 7.3 Security
If setting `innerHTML` dynamically, ensure the content is sanitized or trusted to prevent XSS attacks.
### 7.4 UI Constraints for ComfyUI Custom Node Widgets
When developing DOMWidgets as internal UI widgets for ComfyUI custom nodes, keep the following constraints in mind:
#### 7.4.1 Minimize Vertical Space
ComfyUI nodes are often displayed in a compact graph view with many nodes visible simultaneously. Avoid excessive vertical spacing that could clutter the workspace.
- Keep layouts compact and efficient
- Use appropriate padding and margins (4-8px typically)
- Stack related controls vertically but avoid unnecessary spacing
#### 7.4.2 Avoid Dynamic Height Changes
Dynamic height changes (expand/collapse sections, showing/hiding content) can cause node layout recalculations and affect connection wire positioning.
- Prefer static layouts over expandable/collapsible sections
- Use tooltips or overlays for additional information instead
- If dynamic height is unavoidable, manually trigger layout updates (see Section 4.4)
#### 7.4.3 Keep UI Simple and Intuitive
As internal widgets for ComfyUI custom nodes, the UI should be accessible to users without technical implementation details.
- Use clear, user-friendly terminology (avoid "frontend/backend roll" in favor of "fixed/always randomize")
- Focus on user intent rather than implementation details
- Avoid complex interactions that may confuse users
#### 7.4.4 Forward Middle Mouse Events to Canvas
By default, when a DOM widget receives pointer events (e.g., mouse clicks, drags), these events are captured by the widget and not forwarded to the ComfyUI canvas. This prevents users from panning the workflow using the middle mouse button when the cursor is over a DOM widget.
To enable workflow panning over your widget, you should forward middle mouse events (button 1) to the canvas using the `forwardMiddleMouseToCanvas` utility function:
```javascript
import { forwardMiddleMouseToCanvas } from "./utils.js";
// In your widget creation function
const container = document.createElement("div");
container.style.width = "100%";
container.style.height = "100%";
// ... other styles ...
// Forward middle mouse events to canvas for panning
forwardMiddleMouseToCanvas(container);
const widget = node.addDOMWidget(name, type, container, { ... });
```
The `forwardMiddleMouseToCanvas` function:
- Forwards `pointerdown` events with button 1 (middle mouse button) to `app.canvas.processMouseDown`
- Forwards `pointermove` events while middle mouse button is pressed to `app.canvas.processMouseMove`
- Forwards `pointerup` events with button 1 to `app.canvas.processMouseUp`
This allows users to pan the workflow canvas even when their mouse cursor is hovering over your DOM widget.
---
## 8. Event Handling in Vue DOM Render Mode
ComfyUI frontend supports two rendering modes for nodes:
- **Legacy Canvas Mode**: Traditional rendering where widgets are rendered on top of the canvas using absolute positioning
- **Vue DOM Render Mode**: New rendering mode where nodes and widgets are rendered as Vue components
In Vue DOM render mode, event handling works differently. The frontend uses `useCanvasInteractions` composable to manage event forwarding to the canvas. This can cause custom event handlers in your widgets (e.g., mouse wheel for sliders, custom drag operations) to be intercepted by the canvas.
### 8.1 Wheel Event Handling
By default in Vue DOM render mode, wheel events on widgets may be forwarded to the canvas for workflow zoom, overriding your custom wheel handlers (e.g., adjusting slider values with mouse wheel).
To fix this, use the `data-capture-wheel="true"` attribute on elements that should capture wheel events:
```vue
<!-- Vue component template -->
<div class="my-slider" data-capture-wheel="true" @wheel="onWheel">
<!-- Slider content -->
</div>
<script setup lang="ts">
const onWheel = (event: WheelEvent) => {
event.preventDefault()
// Custom wheel handling logic here
}
</script>
```
**How it works:**
- ComfyUI's `useCanvasInteractions.ts` checks `target?.closest('[data-capture-wheel="true"]')` before forwarding wheel events
- If an element (or its ancestor) has this attribute, wheel events are not forwarded to canvas
- Your custom `@wheel` handler will work as expected
**Granular control:**
- Apply `data-capture-wheel="true"` to specific interactive elements (e.g., sliders, scrollable areas)
- Widget container without this attribute will allow workflow zoom when wheel is used elsewhere
- This allows users to both: adjust widget values with wheel, and zoom workflow with wheel in widget's non-interactive areas
**Example from DualRangeSlider.vue:**
```vue
<template>
<div
class="dual-range-slider"
:class="{ disabled, 'is-dragging': dragging !== null }"
data-capture-wheel="true"
@wheel="onWheel"
>
<!-- Slider tracks and handles -->
</div>
</template>
```
### 8.2 Pointer Event Handling
In Vue DOM render mode, pointer events (click, drag, etc.) may also be captured by the canvas system. For custom drag operations:
1. **Use event modifiers to stop propagation:**
```vue
<div
@pointerdown.stop="startDrag"
@pointermove.stop="onDrag"
@pointerup.stop="stopDrag"
>
```
2. **Use pointer capture for reliable drag tracking:**
```javascript
const startDrag = (event: PointerEvent) => {
const target = event.currentTarget as HTMLElement
target.setPointerCapture(event.pointerId)
// ... drag initialization
}
const stopDrag = (event: PointerEvent) => {
const target = event.currentTarget as HTMLElement
target.releasePointerCapture(event.pointerId)
// ... drag cleanup
}
```
3. **Use `touch-action: none` CSS for touch devices:**
```css
.my-draggable {
touch-action: none;
}
```
### 8.3 Compatibility Checklist
Ensure your widget works in both rendering modes:
| Feature | Canvas Mode | Vue DOM Mode | Solution |
|---------|-------------|--------------|----------|
| Mouse wheel on sliders | Works by default | Needs `data-capture-wheel` | Add `data-capture-wheel="true"` to slider elements |
| Custom drag operations | Works with `stopPropagation()` | Needs `stopPropagation()` | Use `.stop` modifier and pointer capture |
| Middle mouse panning | Manual forwarding required | Manual forwarding required | Use `forwardMiddleMouseToCanvas()` |
| Workflow zoom on widget edges | Works by default | Works by default | No action needed (works by default) |
### 8.4 Testing Recommendations
Test your widget in both rendering modes:
1. Toggle between Canvas Mode and Vue DOM Mode in ComfyUI settings
2. Verify custom interactions (wheel, drag, etc.) work in both modes
3. Verify canvas interactions (zoom, pan) still work when cursor is over non-interactive widget areas
4. Test with touch devices if applicable
---
## 9. Complete Example: Text Counter
This example implements a simple widget that displays the character count of another text widget in the same node.
```javascript
import { app } from "../../scripts/app.js";
app.registerExtension({
name: "Comfy.TextCounter",
getCustomWidgets() {
return {
TEXT_COUNTER(node, inputName) {
const el = document.createElement("div");
Object.assign(el.style, {
background: "#222",
border: "1px solid #444",
padding: "8px",
borderRadius: "4px",
fontSize: "12px",
color: "#eee"
});
const label = document.createElement("span");
label.innerText = "Characters: 0";
el.appendChild(label);
const widget = node.addDOMWidget(inputName, "TEXT_COUNTER", el, {
getValue() { return ""; }, // Nothing to save
setValue(v) { }, // Nothing to restore
getMinHeight() { return 40; }
});
// Disable serialization for this display-only widget
widget.serialize = false;
// Custom method to update UI
widget.updateCount = (text) => {
label.innerText = `Characters: ${text.length}`;
};
return { widget };
}
};
},
nodeCreated(node) {
// Logic to link widgets after the node is initialized
if (node.comfyClass === "MyTextNode") {
const counterWidget = node.widgets.find(w => w.type === "TEXT_COUNTER");
const textWidget = node.widgets.find(w => w.name === "text");
if (counterWidget && textWidget) {
// Hook into the text widget's callback
const oldCallback = textWidget.callback;
textWidget.callback = function(v) {
if (oldCallback) oldCallback.apply(this, arguments);
counterWidget.updateCount(v);
};
}
}
}
});
```

View File

@@ -1,51 +0,0 @@
# Frontend DOM Fixture Strategy
This guide outlines how to reproduce the markup emitted by the Django templates while running Vitest in jsdom. The aim is to make it straightforward to write integration-style unit tests for managers and UI helpers without having to duplicate template fragments inline.
## Loading Template Markup
Vitest executes inside Node, so we can read the same HTML templates that ship with the extension:
1. Use the helper utilities from `tests/frontend/utils/domFixtures.js` to read files under the `templates/` directory.
2. Mount the returned markup into `document.body` (or any custom container) before importing the module under test so its query selectors resolve correctly.
```js
import { renderTemplate } from '../utils/domFixtures.js'; // adjust the relative path to your spec
beforeEach(() => {
renderTemplate('loras.html', {
dataset: { page: 'loras' }
});
});
```
The helper ensures the dataset is applied to the container, which mirrors how Django sets `data-page` in production.
## Working with Partial Components
Many features are implemented as template partials located under `templates/components/`. When a test only needs a fragment (for example, the progress panel or context menu markup), load the component file directly:
```js
const container = renderTemplate('components/progress_panel.html');
const progressPanel = container.querySelector('#progress-panel');
```
This pattern avoids hand-written fixture strings and keeps the tests aligned with the actual markup.
## Resetting Between Tests
The shared Vitest setup clears `document.body` and storage APIs before each test. If a suite adds additional DOM nodes outside of the body or needs to reset custom attributes mid-test, use `resetDom()` exported from `domFixtures.js`.
```js
import { resetDom } from '../utils/domFixtures.js';
afterEach(() => {
resetDom();
});
```
## Future Enhancements
- Provide typed helpers for injecting mock script tags (e.g., replicating ComfyUI globals).
- Compose higher-level fixtures that mimic specific pages (loras, checkpoints, recipes) once those managers receive dedicated suites.

View File

@@ -1,44 +0,0 @@
# LoRA & Checkpoints Filtering/Sorting Test Matrix
This matrix captures the scenarios that Phase 3 frontend tests should cover for the LoRA and Checkpoint managers. It focuses on how search, filter, sort, and duplicate badge toggles interact so future specs can share fixtures and expectations.
## Scope
- **Components**: `PageControls`, `FilterManager`, `SearchManager`, and `ModelDuplicatesManager` wiring invoked through `CheckpointsPageManager` and `LorasPageManager`.
- **Templates**: `templates/loras.html` and `templates/checkpoints.html` along with shared filter panel and toolbar partials.
- **APIs**: Requests issued through `baseModelApi.fetchModels` (via `resetAndReload`/`refreshModels`) and duplicates badge updates.
## Shared Setup Considerations
1. Render full page templates using `renderLorasPage` / `renderCheckpointsPage` helpers before importing modules so DOM queries resolve.
2. Stub storage helpers (`getStorageItem`, `setStorageItem`, `getSessionItem`, `setSessionItem`) to observe persistence behavior without mutating real storage.
3. Mock `sidebarManager` to capture refresh calls triggered after sort/filter actions.
4. Provide fake API implementations exposing `resetAndReload`, `refreshModels`, `fetchFromCivitai`, `toggleBulkMode`, and `clearCustomFilter` so control events remain asynchronous but deterministic.
5. Supply a minimal `ModelDuplicatesManager` mock exposing `toggleDuplicateMode`, `checkDuplicatesCount`, and `updateDuplicatesBadgeAfterRefresh` to validate duplicate badge wiring.
## Scenario Matrix
| ID | Feature | Scenario | LoRAs Expectations | Checkpoints Expectations | Notes |
| --- | --- | --- | --- | --- | --- |
| F-01 | Search filter | Typing a query updates `pageState.filters.search`, persists to session, and triggers `resetAndReload` on submit | Validate `SearchManager` writes query and reloads via API stub; confirm LoRA cards pass query downstream | Same as LoRAs | Cover `enter` press and clicking search icon |
| F-02 | Tag filter | Selecting a tag chip cycles include ➜ exclude ➜ clear, updates storage, and reloads results | Tag state stored under `filters.tags[tagName] = 'include'|'exclude'`; `FilterManager.applyFilters` persists and triggers `resetAndReload(true)` | Same; ensure base model tag set is scoped to checkpoints dataset | Include removal path |
| F-03 | Base model filter | Toggling base model checkboxes updates `filters.baseModel`, persists, and reloads | Ensure only LoRA-supported models show; toggle multi-select | Ensure SDXL/Flux base models appear as expected | Capture UI state restored from storage on next init |
| F-04 | Favorites-only | Clicking favorites toggle updates session flag and calls `resetAndReload(true)` | Button gains `.active` class and API called | Same | Verify duplicates badge refresh when active |
| F-05 | Sort selection | Changing sort select saves preference (legacy + new format) and reloads | Confirm `PageControls.saveSortPreference` invoked with option and API called | Same with checkpoints-specific defaults | Cover `convertLegacySortFormat` branch |
| F-06 | Filter persistence | Re-initializing manager loads stored filters/sort and updates DOM | Filters pre-populate chips/checkboxes; favorites state restored | Same | Requires simulating repeated construction |
| F-07 | Combined filters | Applying search + tag + base model yields aggregated query params for fetch | Assert API receives merged filter payload | Same | Validate toast messaging for active filters |
| F-08 | Clearing filters | Using "Clear filters" resets state, storage, and reloads list | `FilterManager.clearFilters` empties `filters`, removes active class, shows toast | Same | Ensure favorites-only toggle unaffected |
| F-09 | Duplicate badge toggle | Pressing "Find duplicates" toggles duplicate mode and updates badge counts post-refresh | `ModelDuplicatesManager.toggleDuplicateMode` invoked and badge refresh called after API rebuild | Same plus checkpoint-specific duplicate badge dataset | Connects to future duplicate-specific specs |
| F-10 | Bulk actions menu | Opening bulk dropdown keeps filters intact and closes on outside click | Validate dropdown class toggling and no unintended reload | Same | Guard against regression when dropdown interacts with filters |
## Automation Coverage Status
- ✅ F-01 Search filter, F-02 Tag filter, F-03 Base model filter, F-04 Favorites-only toggle, F-05 Sort selection, and F-09 Duplicate badge toggle are covered by `tests/frontend/components/pageControls.filtering.test.js` for both LoRA and checkpoint pages.
- ⏳ F-06 Filter persistence, F-07 Combined filters, F-08 Clearing filters, and F-10 Bulk actions remain to be automated alongside upcoming bulk mode refinements.
## Coverage Gaps & Follow-Ups
- Write Vitest suites that exercise the matrix for both managers, sharing fixtures through page helpers to avoid duplication.
- Capture API parameter assertions by inspecting `baseModelApi.fetchModels` mocks rather than relying solely on state mutations.
- Add regression cases for legacy storage migrations (old filter keys) once fixtures exist for older payloads.
- Extend duplicate badge coverage with scenarios where `checkDuplicatesCount` signals zero duplicates versus pending calculations.

View File

@@ -1,33 +0,0 @@
# Frontend Automation Testing Roadmap
This roadmap tracks the planned rollout of automated testing for the ComfyUI LoRA Manager frontend. Each phase builds on the infrastructure introduced in this change set and records progress so future contributors can quickly identify the next tasks.
## Phase Overview
| Phase | Goal | Primary Focus | Status | Notes |
| --- | --- | --- | --- | --- |
| Phase 0 | Establish baseline tooling | Add Node test runner, jsdom environment, and seed smoke tests | ✅ Complete | Vitest + jsdom configured, example state tests committed |
| Phase 1 | Cover state management logic | Unit test selectors, derived data helpers, and storage utilities under `static/js/state` and `static/js/utils` | ✅ Complete | Storage helpers and state selectors now exercised via deterministic suites |
| Phase 2 | Test AppCore orchestration | Simulate page bootstrapping, infinite scroll hooks, and manager registration using JSDOM DOM fixtures | ✅ Complete | AppCore initialization + page feature suites now validate manager wiring, infinite scroll hooks, and onboarding gating |
| Phase 3 | Validate page-specific managers | Add focused suites for `loras`, `checkpoints`, `embeddings`, and `recipes` managers covering filtering, sorting, and bulk actions | ✅ Complete | LoRA/checkpoint suites expanded; embeddings + recipes managers now covered with initialization, filtering, and duplicate workflows |
| Phase 4 | Interaction-level regression tests | Exercise template fragments, modals, and menus to ensure UI wiring remains intact | ✅ Complete | Vitest DOM suites cover NSFW selector, recipe modal editing, and global context menus |
| Phase 5 | Continuous integration & coverage | Integrate frontend tests into CI workflow and track coverage metrics | ✅ Complete | CI workflow runs Vitest and aggregates V8 coverage into `coverage/frontend` via a dedicated script |
## Next Steps Checklist
- [x] Expand unit tests for `storageHelpers` covering migrations and namespace behavior.
- [x] Document DOM fixture strategy for reproducing template structures in tests.
- [x] Prototype AppCore initialization test that verifies manager bootstrapping with stubbed dependencies.
- [x] Add AppCore page feature suite exercising context menu creation and infinite scroll registration via DOM fixtures.
- [x] Extend AppCore orchestration tests to cover manager wiring, bulk menu setup, and onboarding gating scenarios.
- [x] Add interaction regression suites for context menus and recipe modals to complete Phase 4.
- [x] Evaluate integrating coverage reporting once test surface grows (> 20 specs).
- [x] Create shared fixtures for the loras and checkpoints pages once dedicated manager suites are added.
- [x] Draft focused test matrix for loras/checkpoints manager filtering and sorting paths ahead of Phase 3.
- [x] Implement LoRAs manager filtering/sorting specs for scenarios F-01F-05 & F-09; queue remaining edge cases after duplicate/bulk flows stabilize.
- [x] Implement checkpoints manager filtering/sorting specs for scenarios F-01F-05 & F-09; cover remaining paths alongside bulk action work.
- [x] Implement checkpoints page manager smoke tests covering initialization and duplicate badge wiring.
- [x] Outline focused checkpoints scenarios (filtering, sorting, duplicate badge toggles) to feed into the shared test matrix.
- [ ] Add duplicate badge regression coverage for zero/pending states after API refreshes.
Maintaining this roadmap alongside code changes will make it easier to append new automated test tasks and update their progress.

View File

@@ -1,28 +0,0 @@
# Library Switching and Preview Routes
Library switching no longer requires restarting the backend. The preview
thumbnails shown in the UI are now served through a dynamic endpoint that
resolves files against the folders registered for the active library at request
time. This allows the multi-library flow to update model roots without touching
the aiohttp router, so previews remain available immediately after a switch.
## How the dynamic preview endpoint works
* `config.get_preview_static_url()` now returns `/api/lm/previews?path=<encoded>`
for any preview path. The raw filesystem location is URL encoded so that it
can be passed through the query string without leaking directory structure in
the route itself.【F:py/config.py†L398-L404】
* `PreviewRoutes` exposes the `/api/lm/previews` handler which validates the
decoded path against the directories registered for the current library. The
request is rejected if it falls outside those roots or if the file does not
exist.【F:py/routes/preview_routes.py†L5-L21】【F:py/routes/handlers/preview_handlers.py†L9-L48】
* `Config` keeps an up-to-date cache of allowed preview roots. Every time a
library is applied the cache is rebuilt using the declared LoRA, checkpoint
and embedding directories (including symlink targets). The validation logic
checks preview requests against this cache.【F:py/config.py†L51-L68】【F:py/config.py†L180-L248】【F:py/config.py†L332-L346】
Both the ComfyUI runtime (`LoraManager.add_routes`) and the standalone launcher
(`StandaloneLoraManager.add_routes`) register the new preview routes instead of
mounting a static directory per root. Switching libraries therefore works
without restarting the application, and preview URLs generated before or after a
switch continue to resolve correctly.【F:py/lora_manager.py†L21-L82】【F:standalone.py†L302-L315】

View File

@@ -1,71 +0,0 @@
# Priority Tags Configuration Guide
This guide explains how to tailor the tag priority order that powers folder naming and tag suggestions in the LoRA Manager. You only need to edit the comma-separated list of entries shown in the **Priority Tags** field for each model type.
## 1. Pick the Model Type
In the **Priority Tags** dialog you will find one tab per model type (LoRA, Checkpoint, Embedding). Select the tab you want to update; changes on one tab do not affect the others.
## 2. Edit the Entry List
Inside the textarea you will see a line similar to:
```
character, concept, style(toon|toon_style)
```
This entire line is the **entry list**. Replace it with your own ordered list.
### Entry Rules
Each entry is separated by a comma, in order from highest to lowest priority:
- **Canonical tag only:** `realistic`
- **Canonical tag with aliases:** `character(char|chars)`
Aliases live inside `()` and are separated with `|`. The canonical name is what appears in folder names and UI suggestions when any of the aliases are detected. Matching is case-insensitive.
## Use `{first_tag}` in Path Templates
When your path template contains `{first_tag}`, the app picks a folder name based on your priority list and the models own tags:
- It checks the priority list from top to bottom. If a canonical tag or any of its aliases appear in the model tags, that canonical name becomes the folder name.
- If no priority tags are found but the model has tags, the very first model tag is used.
- If the model has no tags at all, the folder falls back to `no tags`.
### Example
With a template like `/{model_type}/{first_tag}` and the priority entry list `character(char|chars), style(anime|toon)`:
| Model Tags | Folder Name | Why |
| --- | --- | --- |
| `["chars", "female"]` | `character` | `chars` matches the `character` alias, so the canonical wins. |
| `["anime", "portrait"]` | `style` | `anime` hits the `style` entry, so its canonical label is used. |
| `["portrait", "bw"]` | `portrait` | No priority match, so the first model tag is used. |
| `[]` | `no tags` | Nothing to match, so the fallback is applied. |
## 3. Save the Settings
After editing the entry list, press **Enter** to save. Use **Shift+Enter** whenever you need a new line. Clicking outside the field also saves automatically. A success toast confirms the update.
## Examples
| Goal | Entry List |
| --- | --- |
| Prefer people over styles | `character, portraits, style(anime\|toon)` |
| Group sci-fi variants | `sci-fi(scifi\|science_fiction), cyberpunk(cyber\|punk)` |
| Alias shorthand tags | `realistic(real\|realisim), photorealistic(photo_real)` |
## Tips
- Keep canonical names short and meaningful—they become folder names.
- Place the most important categories first; the first match wins.
- Avoid duplicate canonical names within the same list; only the first instance is used.
## Troubleshooting
- **Unexpected folder name?** Check that the canonical name you want is placed before other matches.
- **Alias not working?** Ensure the alias is inside parentheses and separated with `|`, e.g. `character(char|chars)`.
- **Validation error?** Look for missing parentheses or stray commas. Each entry must follow the `canonical(alias|alias)` pattern or just `canonical`.
With these basics you can quickly adapt Priority Tags to match your librarys organization style.

View File

@@ -1,26 +0,0 @@
# Backend Test Coverage Notes
## Pytest Execution
- Command: `python -m pytest`
- Result: All 283 collected tests passed in the current environment.
- Coverage tooling (``pytest-cov``/``coverage``) is unavailable in the offline sandbox, so line-level metrics could not be generated. The earlier attempt to install ``pytest-cov`` failed because the package index cannot be reached from the container.
## High-Priority Gaps to Address
### 1. Standalone server bootstrapping
* **Source:** [`standalone.py`](../../standalone.py)
* **Why it matters:** The standalone entry point wires together the aiohttp application, static asset routes, model-route registration, and configuration validation. None of these behaviours are covered by automated tests, leaving regressions in bootstrapping logic undetected.
* **Suggested coverage:** Add integration-style tests that instantiate `StandaloneServer`/`StandaloneLoraManager` with temporary settings and assert that routes (HTTP + websocket) are registered, configuration warnings fire for missing paths, and the mock ComfyUI shims behave as expected.
### 2. Model service registration factory
* **Source:** [`py/services/model_service_factory.py`](../../py/services/model_service_factory.py)
* **Why it matters:** The factory coordinates which model services and routes the API exposes, including error handling when unknown model types are requested. No current tests verify registration, memoization of route instances, or the logging path on failures.
* **Suggested coverage:** Unit tests that exercise `register_model_type`, `get_route_instance`, error branches in `get_service_class`/`get_route_class`, and `setup_all_routes` when a route setup raises. Use lightweight fakes to confirm the logger is called and state is cleared via `clear_registrations`.
### 3. Server-side i18n helper
* **Source:** [`py/services/server_i18n.py`](../../py/services/server_i18n.py)
* **Why it matters:** Template rendering relies on the `ServerI18nManager` to load locale JSON, perform key lookups, and format parameters. The fallback logic (dot-notation lookup, English fallbacks, placeholder substitution) is untested, so malformed locale files or regressions in placeholder handling would slip through.
* **Suggested coverage:** Tests that load fixture locale dictionaries, assert `set_locale` fallbacks, verify nested key resolution and placeholder substitution, and ensure missing keys return the original identifier.
## Next Steps
Prioritize creating focused unit tests around these modules, then re-run pytest once coverage tooling is available to confirm the new tests close the identified gaps.

Binary file not shown.

After

Width:  |  Height:  |  Size: 669 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

After

Width:  |  Height:  |  Size: 669 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

Before

Width:  |  Height:  |  Size: 668 KiB

File diff suppressed because one or more lines are too long

Binary file not shown.

Before

Width:  |  Height:  |  Size: 739 KiB

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

2575
package-lock.json generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,15 +0,0 @@
{
"name": "comfyui-lora-manager-frontend",
"version": "0.1.0",
"private": true,
"type": "module",
"scripts": {
"test": "vitest run",
"test:watch": "vitest",
"test:coverage": "node scripts/run_frontend_coverage.js"
},
"devDependencies": {
"jsdom": "^24.0.0",
"vitest": "^1.6.0"
}
}

View File

@@ -1,12 +0,0 @@
"""Project namespace package."""
# pytest's internal compatibility layer still imports ``py.path.local`` from the
# historical ``py`` dependency. Because this project reuses the ``py`` package
# name, we expose a minimal shim so ``py.path.local`` resolves to ``pathlib.Path``
# during test runs without pulling in the external dependency.
from pathlib import Path
from types import SimpleNamespace
path = SimpleNamespace(local=Path)
__all__ = ["path"]

View File

@@ -1,89 +1,26 @@
import os import os
import platform import platform
import threading
from pathlib import Path
import folder_paths # type: ignore import folder_paths # type: ignore
from typing import Any, Dict, Iterable, List, Mapping, Optional, Set, Tuple from typing import List
import logging import logging
import sys
import json import json
import urllib.parse
import time
from .utils.settings_paths import ensure_settings_file, get_settings_dir, load_settings_template # Check if running in standalone mode
standalone_mode = 'nodes' not in sys.modules
# Use an environment variable to control standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _normalize_folder_paths_for_comparison(
folder_paths: Mapping[str, Iterable[str]]
) -> Dict[str, Set[str]]:
"""Normalize folder paths for comparison across libraries."""
normalized: Dict[str, Set[str]] = {}
for key, values in folder_paths.items():
if isinstance(values, str):
candidate_values: Iterable[str] = [values]
else:
try:
candidate_values = iter(values)
except TypeError:
continue
normalized_values: Set[str] = set()
for value in candidate_values:
if not isinstance(value, str):
continue
stripped = value.strip()
if not stripped:
continue
normalized_values.add(os.path.normcase(os.path.normpath(stripped)))
if normalized_values:
normalized[key] = normalized_values
return normalized
def _normalize_library_folder_paths(
library_payload: Mapping[str, Any]
) -> Dict[str, Set[str]]:
"""Return normalized folder paths extracted from a library payload."""
folder_paths = library_payload.get("folder_paths")
if isinstance(folder_paths, Mapping):
return _normalize_folder_paths_for_comparison(folder_paths)
return {}
def _get_template_folder_paths() -> Dict[str, Set[str]]:
"""Return normalized folder paths defined in the bundled template."""
template_payload = load_settings_template()
if not template_payload:
return {}
folder_paths = template_payload.get("folder_paths")
if isinstance(folder_paths, Mapping):
return _normalize_folder_paths_for_comparison(folder_paths)
return {}
class Config: class Config:
"""Global configuration for LoRA Manager""" """Global configuration for LoRA Manager"""
def __init__(self): def __init__(self):
self.templates_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'templates') self.templates_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'templates')
self.static_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'static') self.static_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'static')
self.i18n_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'locales')
# Path mapping dictionary, target to link mapping # Path mapping dictionary, target to link mapping
self._path_mappings: Dict[str, str] = {} self._path_mappings = {}
# Normalized preview root directories used to validate preview access # Static route mapping dictionary, target to route mapping
self._preview_root_paths: Set[Path] = set() self._route_mappings = {}
# Fingerprint of the symlink layout from the last successful scan
self._cached_fingerprint: Optional[Dict[str, object]] = None
self.loras_roots = self._init_lora_paths() self.loras_roots = self._init_lora_paths()
self.checkpoints_roots = None self.checkpoints_roots = None
self.unet_roots = None self.unet_roots = None
@@ -91,118 +28,43 @@ class Config:
self.base_models_roots = self._init_checkpoint_paths() self.base_models_roots = self._init_checkpoint_paths()
self.embeddings_roots = self._init_embedding_paths() self.embeddings_roots = self._init_embedding_paths()
# Scan symbolic links during initialization # Scan symbolic links during initialization
self._initialize_symlink_mappings() self._scan_symbolic_links()
if not standalone_mode: if not standalone_mode:
# Save the paths to settings.json when running in ComfyUI mode # Save the paths to settings.json when running in ComfyUI mode
self.save_folder_paths_to_settings() self.save_folder_paths_to_settings()
def save_folder_paths_to_settings(self): def save_folder_paths_to_settings(self):
"""Persist ComfyUI-derived folder paths to the multi-library settings.""" """Save folder paths to settings.json for standalone mode to use later"""
try: try:
ensure_settings_file(logger) # Check if we're running in ComfyUI mode (not standalone)
from .services.settings_manager import get_settings_manager # Load existing settings
settings_path = os.path.join(os.path.dirname(os.path.dirname(__file__)), 'settings.json')
settings_service = get_settings_manager() settings = {}
libraries = settings_service.get_libraries() if os.path.exists(settings_path):
comfy_library = libraries.get("comfyui", {}) with open(settings_path, 'r', encoding='utf-8') as f:
default_library = libraries.get("default", {}) settings = json.load(f)
template_folder_paths = _get_template_folder_paths() # Update settings with paths
default_library_paths: Dict[str, Set[str]] = {} settings['folder_paths'] = {
if isinstance(default_library, Mapping): 'loras': self.loras_roots,
default_library_paths = _normalize_library_folder_paths(default_library) 'checkpoints': self.checkpoints_roots,
'unet': self.unet_roots,
libraries_changed = False 'embeddings': self.embeddings_roots,
if (
isinstance(default_library, Mapping)
and template_folder_paths
and default_library_paths == template_folder_paths
):
if "comfyui" in libraries:
try:
settings_service.delete_library("default")
libraries_changed = True
logger.info("Removed template 'default' library entry")
except Exception as delete_error:
logger.debug(
"Failed to delete template 'default' library: %s",
delete_error,
)
else:
try:
settings_service.rename_library("default", "comfyui")
libraries_changed = True
logger.info("Renamed template 'default' library to 'comfyui'")
except Exception as rename_error:
logger.debug(
"Failed to rename template 'default' library: %s",
rename_error,
)
if libraries_changed:
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
default_library = libraries.get("default", {})
target_folder_paths = {
'loras': list(self.loras_roots),
'checkpoints': list(self.checkpoints_roots or []),
'unet': list(self.unet_roots or []),
'embeddings': list(self.embeddings_roots or []),
} }
normalized_target_paths = _normalize_folder_paths_for_comparison(target_folder_paths) # Add default roots if there's only one item and key doesn't exist
if len(self.loras_roots) == 1 and "default_lora_root" not in settings:
normalized_default_paths: Optional[Dict[str, Set[str]]] = None settings["default_lora_root"] = self.loras_roots[0]
if isinstance(default_library, Mapping):
normalized_default_paths = _normalize_library_folder_paths(default_library) if self.checkpoints_roots and len(self.checkpoints_roots) == 1 and "default_checkpoint_root" not in settings:
settings["default_checkpoint_root"] = self.checkpoints_roots[0]
if (
not comfy_library # Save settings
and default_library with open(settings_path, 'w', encoding='utf-8') as f:
and normalized_target_paths json.dump(settings, f, indent=2)
and normalized_default_paths == normalized_target_paths
): logger.info("Saved folder paths to settings.json")
try:
settings_service.rename_library("default", "comfyui")
logger.info("Renamed legacy 'default' library to 'comfyui'")
libraries = settings_service.get_libraries()
comfy_library = libraries.get("comfyui", {})
except Exception as rename_error:
logger.debug(
"Failed to rename legacy 'default' library: %s", rename_error
)
default_lora_root = comfy_library.get("default_lora_root", "")
if not default_lora_root and len(self.loras_roots) == 1:
default_lora_root = self.loras_roots[0]
default_checkpoint_root = comfy_library.get("default_checkpoint_root", "")
if (not default_checkpoint_root and self.checkpoints_roots and
len(self.checkpoints_roots) == 1):
default_checkpoint_root = self.checkpoints_roots[0]
default_embedding_root = comfy_library.get("default_embedding_root", "")
if (not default_embedding_root and self.embeddings_roots and
len(self.embeddings_roots) == 1):
default_embedding_root = self.embeddings_roots[0]
metadata = dict(comfy_library.get("metadata", {}))
metadata.setdefault("display_name", "ComfyUI")
metadata["source"] = "comfyui"
settings_service.upsert_library(
"comfyui",
folder_paths=target_folder_paths,
default_lora_root=default_lora_root,
default_checkpoint_root=default_checkpoint_root,
default_embedding_root=default_embedding_root,
metadata=metadata,
activate=True,
)
logger.info("Updated 'comfyui' library with current folder paths")
except Exception as e: except Exception as e:
logger.warning(f"Failed to save folder paths: {e}") logger.warning(f"Failed to save folder paths: {e}")
@@ -223,398 +85,96 @@ class Config:
logger.error(f"Error checking link status for {path}: {e}") logger.error(f"Error checking link status for {path}: {e}")
return False return False
def _normalize_path(self, path: str) -> str:
return os.path.normpath(path).replace(os.sep, '/')
def _get_symlink_cache_path(self) -> Path:
cache_dir = Path(get_settings_dir(create=True)) / "cache"
cache_dir.mkdir(parents=True, exist_ok=True)
return cache_dir / "symlink_map.json"
def _symlink_roots(self) -> List[str]:
roots: List[str] = []
roots.extend(self.loras_roots or [])
roots.extend(self.base_models_roots or [])
roots.extend(self.embeddings_roots or [])
return roots
def _build_symlink_fingerprint(self) -> Dict[str, object]:
roots = [self._normalize_path(path) for path in self._symlink_roots() if path]
unique_roots = sorted(set(roots))
# Fingerprint now only contains the root paths to avoid sensitivity to folder content changes.
return {"roots": unique_roots}
def _initialize_symlink_mappings(self) -> None:
start = time.perf_counter()
cache_loaded = self._load_persisted_cache_into_mappings()
if cache_loaded:
logger.info(
"Symlink mappings restored from cache in %.2f ms",
(time.perf_counter() - start) * 1000,
)
self._rebuild_preview_roots()
# Only rescan if target roots have changed.
# This is stable across file additions/deletions.
current_fingerprint = self._build_symlink_fingerprint()
cached_fingerprint = self._cached_fingerprint
if cached_fingerprint and current_fingerprint == cached_fingerprint:
return
logger.info("Symlink root paths changed; rescanning symbolic links")
self.rebuild_symlink_cache()
logger.info(
"Symlink mappings rebuilt and cached in %.2f ms",
(time.perf_counter() - start) * 1000,
)
def rebuild_symlink_cache(self) -> None:
"""Force a fresh scan of all symbolic links and update the persistent cache."""
self._scan_symbolic_links()
self._save_symlink_cache()
self._rebuild_preview_roots()
def _load_persisted_cache_into_mappings(self) -> bool:
"""Load the symlink cache and store its fingerprint for comparison."""
cache_path = self._get_symlink_cache_path()
if not cache_path.exists():
return False
try:
with cache_path.open("r", encoding="utf-8") as handle:
payload = json.load(handle)
except Exception as exc:
logger.info("Failed to load symlink cache %s: %s", cache_path, exc)
return False
if not isinstance(payload, dict):
return False
cached_mappings = payload.get("path_mappings")
if not isinstance(cached_mappings, Mapping):
return False
# Store the cached fingerprint for comparison during initialization
self._cached_fingerprint = payload.get("fingerprint")
normalized_mappings: Dict[str, str] = {}
for target, link in cached_mappings.items():
if not isinstance(target, str) or not isinstance(link, str):
continue
normalized_mappings[self._normalize_path(target)] = self._normalize_path(link)
self._path_mappings = normalized_mappings
logger.info("Symlink cache loaded with %d mappings", len(self._path_mappings))
return True
def _save_symlink_cache(self) -> None:
cache_path = self._get_symlink_cache_path()
payload = {
"fingerprint": self._build_symlink_fingerprint(),
"path_mappings": self._path_mappings,
}
try:
with cache_path.open("w", encoding="utf-8") as handle:
json.dump(payload, handle, ensure_ascii=False, indent=2)
logger.debug("Symlink cache saved to %s with %d mappings", cache_path, len(self._path_mappings))
except Exception as exc:
logger.info("Failed to write symlink cache %s: %s", cache_path, exc)
def _scan_symbolic_links(self): def _scan_symbolic_links(self):
"""Scan all symbolic links in LoRA, Checkpoint, and Embedding root directories""" """Scan all symbolic links in LoRA, Checkpoint, and Embedding root directories"""
start = time.perf_counter() for root in self.loras_roots:
self._scan_directory_links(root)
# Reset mappings before rescanning to avoid stale entries for root in self.base_models_roots:
self._path_mappings.clear() self._scan_directory_links(root)
self._seed_root_symlink_mappings()
visited_dirs: Set[str] = set() for root in self.embeddings_roots:
for root in self._symlink_roots(): self._scan_directory_links(root)
self._scan_directory_links(root, visited_dirs)
logger.debug(
"Symlink scan finished in %.2f ms with %d mappings",
(time.perf_counter() - start) * 1000,
len(self._path_mappings),
)
def _scan_directory_links(self, root: str, visited_dirs: Set[str]): def _scan_directory_links(self, root: str):
"""Iteratively scan directory symlinks to avoid deep recursion.""" """Recursively scan symbolic links in a directory"""
try: try:
# Note: We only use realpath for the initial root if it's not already resolved with os.scandir(root) as it:
# to ensure we have a valid entry point. for entry in it:
root_real = self._normalize_path(os.path.realpath(root)) if self._is_link(entry.path):
except OSError: target_path = os.path.realpath(entry.path)
root_real = self._normalize_path(root) if os.path.isdir(target_path):
self.add_path_mapping(entry.path, target_path)
if root_real in visited_dirs: self._scan_directory_links(target_path)
return elif entry.is_dir(follow_symlinks=False):
self._scan_directory_links(entry.path)
visited_dirs.add(root_real) except Exception as e:
# Stack entries: (display_path, real_resolved_path) logger.error(f"Error scanning links in {root}: {e}")
stack: List[Tuple[str, str]] = [(root, root_real)]
while stack:
current_display, current_real = stack.pop()
try:
with os.scandir(current_display) as it:
for entry in it:
try:
# 1. High speed detection using dirent data (is_symlink)
is_link = entry.is_symlink()
# On Windows, is_symlink handles reparse points
if is_link:
# Only resolve realpath when we actually find a link
target_path = os.path.realpath(entry.path)
if not os.path.isdir(target_path):
continue
normalized_target = self._normalize_path(target_path)
self.add_path_mapping(entry.path, target_path)
if normalized_target in visited_dirs:
continue
visited_dirs.add(normalized_target)
stack.append((target_path, normalized_target))
continue
# 2. Process normal directories
if not entry.is_dir(follow_symlinks=False):
continue
# For normal directories, we avoid realpath() call by
# incrementally building the real path relative to current_real.
# This is safe because 'entry' is NOT a symlink.
entry_real = self._normalize_path(os.path.join(current_real, entry.name))
if entry_real in visited_dirs:
continue
visited_dirs.add(entry_real)
stack.append((entry.path, entry_real))
except Exception as inner_exc:
logger.debug(
"Error processing directory entry %s: %s", entry.path, inner_exc
)
except Exception as e:
logger.error(f"Error scanning links in {current_display}: {e}")
def add_path_mapping(self, link_path: str, target_path: str): def add_path_mapping(self, link_path: str, target_path: str):
"""Add a symbolic link path mapping """Add a symbolic link path mapping
target_path: actual target path target_path: actual target path
link_path: symbolic link path link_path: symbolic link path
""" """
normalized_link = self._normalize_path(link_path) normalized_link = os.path.normpath(link_path).replace(os.sep, '/')
normalized_target = self._normalize_path(target_path) normalized_target = os.path.normpath(target_path).replace(os.sep, '/')
# Keep the original mapping: target path -> link path # Keep the original mapping: target path -> link path
self._path_mappings[normalized_target] = normalized_link self._path_mappings[normalized_target] = normalized_link
logger.info(f"Added path mapping: {normalized_target} -> {normalized_link}") logger.info(f"Added path mapping: {normalized_target} -> {normalized_link}")
self._preview_root_paths.update(self._expand_preview_root(normalized_target))
self._preview_root_paths.update(self._expand_preview_root(normalized_link))
def _seed_root_symlink_mappings(self) -> None: def add_route_mapping(self, path: str, route: str):
"""Ensure symlinked root folders are recorded before deep scanning.""" """Add a static route mapping"""
normalized_path = os.path.normpath(path).replace(os.sep, '/')
for root in self._symlink_roots(): self._route_mappings[normalized_path] = route
if not root: # logger.info(f"Added route mapping: {normalized_path} -> {route}")
continue
try:
if not self._is_link(root):
continue
target_path = os.path.realpath(root)
if not os.path.isdir(target_path):
continue
self.add_path_mapping(root, target_path)
except Exception as exc:
logger.debug("Skipping root symlink %s: %s", root, exc)
def _expand_preview_root(self, path: str) -> Set[Path]:
"""Return normalized ``Path`` objects representing a preview root."""
roots: Set[Path] = set()
if not path:
return roots
try:
raw_path = Path(path).expanduser()
except Exception:
return roots
if raw_path.is_absolute():
roots.add(raw_path)
try:
resolved = raw_path.resolve(strict=False)
except RuntimeError:
resolved = raw_path.absolute()
roots.add(resolved)
try:
real_path = raw_path.resolve()
except (FileNotFoundError, RuntimeError):
real_path = resolved
roots.add(real_path)
normalized: Set[Path] = set()
for candidate in roots:
if candidate.is_absolute():
normalized.add(candidate)
else:
try:
normalized.add(candidate.resolve(strict=False))
except RuntimeError:
normalized.add(candidate.absolute())
return normalized
def _rebuild_preview_roots(self) -> None:
"""Recompute the cache of directories permitted for previews."""
preview_roots: Set[Path] = set()
for root in self.loras_roots or []:
preview_roots.update(self._expand_preview_root(root))
for root in self.base_models_roots or []:
preview_roots.update(self._expand_preview_root(root))
for root in self.embeddings_roots or []:
preview_roots.update(self._expand_preview_root(root))
for target, link in self._path_mappings.items():
preview_roots.update(self._expand_preview_root(target))
preview_roots.update(self._expand_preview_root(link))
self._preview_root_paths = {path for path in preview_roots if path.is_absolute()}
logger.debug(
"Preview roots rebuilt: %d paths from %d lora roots, %d checkpoint roots, %d embedding roots, %d symlink mappings",
len(self._preview_root_paths),
len(self.loras_roots or []),
len(self.base_models_roots or []),
len(self.embeddings_roots or []),
len(self._path_mappings),
)
def map_path_to_link(self, path: str) -> str: def map_path_to_link(self, path: str) -> str:
"""Map a target path back to its symbolic link path""" """Map a target path back to its symbolic link path"""
normalized_path = os.path.normpath(path).replace(os.sep, '/') normalized_path = os.path.normpath(path).replace(os.sep, '/')
# Check if the path is contained in any mapped target path # Check if the path is contained in any mapped target path
for target_path, link_path in self._path_mappings.items(): for target_path, link_path in self._path_mappings.items():
# Match whole path components to avoid prefix collisions (e.g., /a/b vs /a/bc) if normalized_path.startswith(target_path):
if normalized_path == target_path:
return link_path
if normalized_path.startswith(target_path + '/'):
# If the path starts with the target path, replace with link path # If the path starts with the target path, replace with link path
mapped_path = normalized_path.replace(target_path, link_path, 1) mapped_path = normalized_path.replace(target_path, link_path, 1)
return mapped_path return mapped_path
return normalized_path return path
def map_link_to_path(self, link_path: str) -> str: def map_link_to_path(self, link_path: str) -> str:
"""Map a symbolic link path back to the actual path""" """Map a symbolic link path back to the actual path"""
normalized_link = os.path.normpath(link_path).replace(os.sep, '/') normalized_link = os.path.normpath(link_path).replace(os.sep, '/')
# Check if the path is contained in any mapped target path # Check if the path is contained in any mapped target path
for target_path, link_path_mapped in self._path_mappings.items(): for target_path, link_path in self._path_mappings.items():
# Match whole path components if normalized_link.startswith(target_path):
if normalized_link == link_path_mapped: # If the path starts with the target path, replace with actual path
return target_path mapped_path = normalized_link.replace(target_path, link_path, 1)
if normalized_link.startswith(link_path_mapped + '/'):
# If the path starts with the link path, replace with actual path
mapped_path = normalized_link.replace(link_path_mapped, target_path, 1)
return mapped_path return mapped_path
return normalized_link return link_path
def _dedupe_existing_paths(self, raw_paths: Iterable[str]) -> Dict[str, str]:
dedup: Dict[str, str] = {}
for path in raw_paths:
if not isinstance(path, str):
continue
if not os.path.exists(path):
continue
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
normalized = os.path.normpath(path).replace(os.sep, '/')
if real_path not in dedup:
dedup[real_path] = normalized
return dedup
def _prepare_lora_paths(self, raw_paths: Iterable[str]) -> List[str]:
path_map = self._dedupe_existing_paths(raw_paths)
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _prepare_checkpoint_paths(
self, checkpoint_paths: Iterable[str], unet_paths: Iterable[str]
) -> List[str]:
checkpoint_map = self._dedupe_existing_paths(checkpoint_paths)
unet_map = self._dedupe_existing_paths(unet_paths)
merged_map: Dict[str, str] = {}
for real_path, original in {**checkpoint_map, **unet_map}.items():
if real_path not in merged_map:
merged_map[real_path] = original
unique_paths = sorted(merged_map.values(), key=lambda p: p.lower())
checkpoint_values = set(checkpoint_map.values())
unet_values = set(unet_map.values())
self.checkpoints_roots = [p for p in unique_paths if p in checkpoint_values]
self.unet_roots = [p for p in unique_paths if p in unet_values]
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _prepare_embedding_paths(self, raw_paths: Iterable[str]) -> List[str]:
path_map = self._dedupe_existing_paths(raw_paths)
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths
def _apply_library_paths(self, folder_paths: Mapping[str, Iterable[str]]) -> None:
self._path_mappings.clear()
self._preview_root_paths = set()
lora_paths = folder_paths.get('loras', []) or []
checkpoint_paths = folder_paths.get('checkpoints', []) or []
unet_paths = folder_paths.get('unet', []) or []
embedding_paths = folder_paths.get('embeddings', []) or []
self.loras_roots = self._prepare_lora_paths(lora_paths)
self.base_models_roots = self._prepare_checkpoint_paths(checkpoint_paths, unet_paths)
self.embeddings_roots = self._prepare_embedding_paths(embedding_paths)
self._initialize_symlink_mappings()
def _init_lora_paths(self) -> List[str]: def _init_lora_paths(self) -> List[str]:
"""Initialize and validate LoRA paths from ComfyUI settings""" """Initialize and validate LoRA paths from ComfyUI settings"""
try: try:
raw_paths = folder_paths.get_folder_paths("loras") raw_paths = folder_paths.get_folder_paths("loras")
unique_paths = self._prepare_lora_paths(raw_paths)
# Normalize and resolve symlinks, store mapping from resolved -> original
path_map = {}
for path in raw_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
path_map[real_path] = path_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Now sort and use only the deduplicated real paths
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
logger.info("Found LoRA roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]")) logger.info("Found LoRA roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]"))
if not unique_paths: if not unique_paths:
logger.warning("No valid loras folders found in ComfyUI configuration") logger.warning("No valid loras folders found in ComfyUI configuration")
return [] return []
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths return unique_paths
except Exception as e: except Exception as e:
logger.warning(f"Error initializing LoRA paths: {e}") logger.warning(f"Error initializing LoRA paths: {e}")
@@ -623,17 +183,48 @@ class Config:
def _init_checkpoint_paths(self) -> List[str]: def _init_checkpoint_paths(self) -> List[str]:
"""Initialize and validate checkpoint paths from ComfyUI settings""" """Initialize and validate checkpoint paths from ComfyUI settings"""
try: try:
# Get checkpoint paths from folder_paths
raw_checkpoint_paths = folder_paths.get_folder_paths("checkpoints") raw_checkpoint_paths = folder_paths.get_folder_paths("checkpoints")
raw_unet_paths = folder_paths.get_folder_paths("unet") raw_unet_paths = folder_paths.get_folder_paths("unet")
unique_paths = self._prepare_checkpoint_paths(raw_checkpoint_paths, raw_unet_paths)
# Normalize and resolve symlinks for checkpoints, store mapping from resolved -> original
logger.info("Found checkpoint roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]")) checkpoint_map = {}
for path in raw_checkpoint_paths:
if not unique_paths: if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
checkpoint_map[real_path] = checkpoint_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Normalize and resolve symlinks for unet, store mapping from resolved -> original
unet_map = {}
for path in raw_unet_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
unet_map[real_path] = unet_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Now sort and use only the deduplicated real paths
unique_checkpoint_paths = sorted(checkpoint_map.values(), key=lambda p: p.lower())
unique_unet_paths = sorted(unet_map.values(), key=lambda p: p.lower())
# Store individual paths in class properties
self.checkpoints_roots = unique_checkpoint_paths
self.unet_roots = unique_unet_paths
# Combine all checkpoint-related paths for return value
all_paths = unique_checkpoint_paths + unique_unet_paths
logger.info("Found checkpoint roots:" + ("\n - " + "\n - ".join(all_paths) if all_paths else "[]"))
if not all_paths:
logger.warning("No valid checkpoint folders found in ComfyUI configuration") logger.warning("No valid checkpoint folders found in ComfyUI configuration")
return [] return []
return unique_paths # Initialize path mappings
for original_path in all_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return all_paths
except Exception as e: except Exception as e:
logger.warning(f"Error initializing checkpoint paths: {e}") logger.warning(f"Error initializing checkpoint paths: {e}")
return [] return []
@@ -642,94 +233,45 @@ class Config:
"""Initialize and validate embedding paths from ComfyUI settings""" """Initialize and validate embedding paths from ComfyUI settings"""
try: try:
raw_paths = folder_paths.get_folder_paths("embeddings") raw_paths = folder_paths.get_folder_paths("embeddings")
unique_paths = self._prepare_embedding_paths(raw_paths)
# Normalize and resolve symlinks, store mapping from resolved -> original
path_map = {}
for path in raw_paths:
if os.path.exists(path):
real_path = os.path.normpath(os.path.realpath(path)).replace(os.sep, '/')
path_map[real_path] = path_map.get(real_path, path.replace(os.sep, "/")) # preserve first seen
# Now sort and use only the deduplicated real paths
unique_paths = sorted(path_map.values(), key=lambda p: p.lower())
logger.info("Found embedding roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]")) logger.info("Found embedding roots:" + ("\n - " + "\n - ".join(unique_paths) if unique_paths else "[]"))
if not unique_paths: if not unique_paths:
logger.warning("No valid embeddings folders found in ComfyUI configuration") logger.warning("No valid embeddings folders found in ComfyUI configuration")
return [] return []
for original_path in unique_paths:
real_path = os.path.normpath(os.path.realpath(original_path)).replace(os.sep, '/')
if real_path != original_path:
self.add_path_mapping(original_path, real_path)
return unique_paths return unique_paths
except Exception as e: except Exception as e:
logger.warning(f"Error initializing embedding paths: {e}") logger.warning(f"Error initializing embedding paths: {e}")
return [] return []
def get_preview_static_url(self, preview_path: str) -> str: def get_preview_static_url(self, preview_path: str) -> str:
"""Convert local preview path to static URL"""
if not preview_path: if not preview_path:
return "" return ""
real_path = os.path.realpath(preview_path).replace(os.sep, '/')
normalized = os.path.normpath(preview_path).replace(os.sep, '/') for path, route in self._route_mappings.items():
encoded_path = urllib.parse.quote(normalized, safe='') if real_path.startswith(path):
return f'/api/lm/previews?path={encoded_path}' relative_path = os.path.relpath(real_path, path)
return f'{route}/{relative_path.replace(os.sep, "/")}'
def is_preview_path_allowed(self, preview_path: str) -> bool: return ""
"""Return ``True`` if ``preview_path`` is within an allowed directory."""
if not preview_path:
return False
try:
candidate = Path(preview_path).expanduser().resolve(strict=False)
except Exception:
return False
# Use os.path.normcase for case-insensitive comparison on Windows.
# On Windows, Path.relative_to() is case-sensitive for drive letters,
# causing paths like 'a:/folder' to not match 'A:/folder'.
candidate_str = os.path.normcase(str(candidate))
for root in self._preview_root_paths:
root_str = os.path.normcase(str(root))
# Check if candidate is equal to or under the root directory
if candidate_str == root_str or candidate_str.startswith(root_str + os.sep):
return True
if self._preview_root_paths:
logger.debug(
"Preview path rejected: %s (candidate=%s, num_roots=%d, first_root=%s)",
preview_path,
candidate_str,
len(self._preview_root_paths),
os.path.normcase(str(next(iter(self._preview_root_paths)))),
)
else:
logger.debug(
"Preview path rejected (no roots configured): %s",
preview_path,
)
return False
def apply_library_settings(self, library_config: Mapping[str, object]) -> None:
"""Update runtime paths to match the provided library configuration."""
folder_paths = library_config.get('folder_paths') if isinstance(library_config, Mapping) else {}
if not isinstance(folder_paths, Mapping):
folder_paths = {}
self._apply_library_paths(folder_paths)
logger.info(
"Applied library settings with %d lora roots, %d checkpoint roots, and %d embedding roots",
len(self.loras_roots or []),
len(self.base_models_roots or []),
len(self.embeddings_roots or []),
)
def get_library_registry_snapshot(self) -> Dict[str, object]:
"""Return the current library registry and active library name."""
try:
from .services.settings_manager import get_settings_manager
settings_service = get_settings_manager()
libraries = settings_service.get_libraries()
active_library = settings_service.get_active_library_name()
return {
"active_library": active_library,
"libraries": libraries,
}
except Exception as exc: # pragma: no cover - defensive logging
logger.debug("Failed to collect library registry snapshot: %s", exc)
return {"active_library": "", "libraries": {}}
# Global config instance # Global config instance
config = Config() config = Config()

View File

@@ -2,15 +2,7 @@ import asyncio
import sys import sys
import os import os
import logging import logging
from .utils.logging_config import setup_logging from pathlib import Path
# Check if we're in standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
# Only setup logging prefix if not in standalone mode
if not standalone_mode:
setup_logging()
from server import PromptServer # type: ignore from server import PromptServer # type: ignore
from .config import config from .config import config
@@ -19,47 +11,16 @@ from .routes.recipe_routes import RecipeRoutes
from .routes.stats_routes import StatsRoutes from .routes.stats_routes import StatsRoutes
from .routes.update_routes import UpdateRoutes from .routes.update_routes import UpdateRoutes
from .routes.misc_routes import MiscRoutes from .routes.misc_routes import MiscRoutes
from .routes.preview_routes import PreviewRoutes
from .routes.example_images_routes import ExampleImagesRoutes from .routes.example_images_routes import ExampleImagesRoutes
from .services.service_registry import ServiceRegistry from .services.service_registry import ServiceRegistry
from .services.settings_manager import get_settings_manager from .services.settings_manager import settings
from .utils.example_images_migration import ExampleImagesMigration from .utils.example_images_migration import ExampleImagesMigration
from .services.websocket_manager import ws_manager from .services.websocket_manager import ws_manager
from .services.example_images_cleanup_service import ExampleImagesCleanupService
from .middleware.csp_middleware import relax_csp_for_remote_media
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
HEADER_SIZE_LIMIT = 16384 # Check if we're in standalone mode
STANDALONE_MODE = 'nodes' not in sys.modules
def _sanitize_size_limit(value):
"""Return a non-negative integer size for ``handler_args`` comparisons."""
try:
coerced = int(value)
except (TypeError, ValueError):
return 0
return coerced if coerced >= 0 else 0
class _SettingsProxy:
def __init__(self):
self._manager = None
def _resolve(self):
if self._manager is None:
self._manager = get_settings_manager()
return self._manager
def get(self, *args, **kwargs):
return self._resolve().get(*args, **kwargs)
def __getattr__(self, item):
return getattr(self._resolve(), item)
settings = _SettingsProxy()
class LoraManager: class LoraManager:
"""Main entry point for LoRA Manager plugin""" """Main entry point for LoRA Manager plugin"""
@@ -69,41 +30,6 @@ class LoraManager:
"""Initialize and register all routes using the new refactored architecture""" """Initialize and register all routes using the new refactored architecture"""
app = PromptServer.instance.app app = PromptServer.instance.app
if relax_csp_for_remote_media not in app.middlewares:
# Ensure CSP relaxer executes after ComfyUI's block_external_middleware so it can
# see and extend the restrictive header instead of being overwritten by it.
block_middleware_index = next(
(
idx
for idx, middleware in enumerate(app.middlewares)
if getattr(middleware, "__name__", "") == "block_external_middleware"
),
None,
)
if block_middleware_index is None:
app.middlewares.append(relax_csp_for_remote_media)
else:
app.middlewares.insert(block_middleware_index, relax_csp_for_remote_media)
# Increase allowed header sizes so browsers with large localhost cookie
# jars (multiple UIs on 127.0.0.1) don't trip aiohttp's 8KB default
# limits. Cookies for unrelated apps are still sent to the plugin and
# may otherwise raise LineTooLong errors when the request parser reads
# them. Preserve any previously configured handler arguments while
# ensuring our minimum sizes are applied.
handler_args = getattr(app, "_handler_args", {}) or {}
updated_handler_args = dict(handler_args)
updated_handler_args["max_field_size"] = max(
_sanitize_size_limit(handler_args.get("max_field_size", 0)),
HEADER_SIZE_LIMIT,
)
updated_handler_args["max_line_size"] = max(
_sanitize_size_limit(handler_args.get("max_line_size", 0)),
HEADER_SIZE_LIMIT,
)
app._handler_args = updated_handler_args
# Configure aiohttp access logger to be less verbose # Configure aiohttp access logger to be less verbose
logging.getLogger('aiohttp.access').setLevel(logging.WARNING) logging.getLogger('aiohttp.access').setLevel(logging.WARNING)
@@ -123,18 +49,103 @@ class LoraManager:
asyncio_logger = logging.getLogger("asyncio") asyncio_logger = logging.getLogger("asyncio")
asyncio_logger.addFilter(ConnectionResetFilter()) asyncio_logger.addFilter(ConnectionResetFilter())
added_targets = set() # Track already added target paths
# Add static route for example images if the path exists in settings # Add static route for example images if the path exists in settings
example_images_path = settings.get('example_images_path') example_images_path = settings.get('example_images_path')
logger.info(f"Example images path: {example_images_path}") logger.info(f"Example images path: {example_images_path}")
if example_images_path and os.path.exists(example_images_path): if example_images_path and os.path.exists(example_images_path):
app.router.add_static('/example_images_static', example_images_path) app.router.add_static('/example_images_static', example_images_path)
logger.info(f"Added static route for example images: /example_images_static -> {example_images_path}") logger.info(f"Added static route for example images: /example_images_static -> {example_images_path}")
# Add static route for locales JSON files # Add static routes for each lora root
if os.path.exists(config.i18n_path): for idx, root in enumerate(config.loras_roots, start=1):
app.router.add_static('/locales', config.i18n_path) preview_path = f'/loras_static/root{idx}/preview'
logger.info(f"Added static route for locales: /locales -> {config.i18n_path}")
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for each checkpoint root
for idx, root in enumerate(config.base_models_roots, start=1):
preview_path = f'/checkpoints_static/root{idx}/preview'
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for each embedding root
for idx, root in enumerate(config.embeddings_roots, start=1):
preview_path = f'/embeddings_static/root{idx}/preview'
real_root = root
if root in config._path_mappings.values():
for target, link in config._path_mappings.items():
if link == root:
real_root = target
break
# Add static route for original path
app.router.add_static(preview_path, real_root)
logger.info(f"Added static route {preview_path} -> {real_root}")
# Record route mapping
config.add_route_mapping(real_root, preview_path)
added_targets.add(real_root)
# Add static routes for symlink target paths
link_idx = {
'lora': 1,
'checkpoint': 1,
'embedding': 1
}
for target_path, link_path in config._path_mappings.items():
if target_path not in added_targets:
# Determine if this is a checkpoint, lora, or embedding link based on path
is_checkpoint = any(cp_root in link_path for cp_root in config.base_models_roots)
is_checkpoint = is_checkpoint or any(cp_root in target_path for cp_root in config.base_models_roots)
is_embedding = any(emb_root in link_path for emb_root in config.embeddings_roots)
is_embedding = is_embedding or any(emb_root in target_path for emb_root in config.embeddings_roots)
if is_checkpoint:
route_path = f'/checkpoints_static/link_{link_idx["checkpoint"]}/preview'
link_idx["checkpoint"] += 1
elif is_embedding:
route_path = f'/embeddings_static/link_{link_idx["embedding"]}/preview'
link_idx["embedding"] += 1
else:
route_path = f'/loras_static/link_{link_idx["lora"]}/preview'
link_idx["lora"] += 1
try:
app.router.add_static(route_path, Path(target_path).resolve(strict=False))
logger.info(f"Added static route for link target {route_path} -> {target_path}")
config.add_route_mapping(target_path, route_path)
added_targets.add(target_path)
except Exception as e:
logger.warning(f"Failed to add static route on initialization for {target_path}: {e}")
continue
# Add static route for plugin assets # Add static route for plugin assets
app.router.add_static('/loras_static', config.static_path) app.router.add_static('/loras_static', config.static_path)
@@ -150,8 +161,7 @@ class LoraManager:
RecipeRoutes.setup_routes(app) RecipeRoutes.setup_routes(app)
UpdateRoutes.setup_routes(app) UpdateRoutes.setup_routes(app)
MiscRoutes.setup_routes(app) MiscRoutes.setup_routes(app)
ExampleImagesRoutes.setup_routes(app, ws_manager=ws_manager) ExampleImagesRoutes.setup_routes(app)
PreviewRoutes.setup_routes(app)
# Setup WebSocket routes that are shared across all model types # Setup WebSocket routes that are shared across all model types
app.router.add_get('/ws/fetch-progress', ws_manager.handle_connection) app.router.add_get('/ws/fetch-progress', ws_manager.handle_connection)
@@ -164,6 +174,8 @@ class LoraManager:
# Add cleanup # Add cleanup
app.on_shutdown.append(cls._cleanup) app.on_shutdown.append(cls._cleanup)
logger.info(f"LoRA Manager: Set up routes for {len(ModelServiceFactory.get_registered_types())} model types: {', '.join(ModelServiceFactory.get_registered_types())}")
@classmethod @classmethod
async def _initialize_services(cls): async def _initialize_services(cls):
"""Initialize all services using the ServiceRegistry""" """Initialize all services using the ServiceRegistry"""
@@ -173,9 +185,6 @@ class LoraManager:
# Register DownloadManager with ServiceRegistry # Register DownloadManager with ServiceRegistry
await ServiceRegistry.get_download_manager() await ServiceRegistry.get_download_manager()
from .services.metadata_service import initialize_metadata_providers
await initialize_metadata_providers()
# Initialize WebSocket manager # Initialize WebSocket manager
await ServiceRegistry.get_websocket_manager() await ServiceRegistry.get_websocket_manager()
@@ -189,188 +198,29 @@ class LoraManager:
recipe_scanner = await ServiceRegistry.get_recipe_scanner() recipe_scanner = await ServiceRegistry.get_recipe_scanner()
# Create low-priority initialization tasks # Create low-priority initialization tasks
init_tasks = [ asyncio.create_task(lora_scanner.initialize_in_background(), name='lora_cache_init')
asyncio.create_task(lora_scanner.initialize_in_background(), name='lora_cache_init'), asyncio.create_task(checkpoint_scanner.initialize_in_background(), name='checkpoint_cache_init')
asyncio.create_task(checkpoint_scanner.initialize_in_background(), name='checkpoint_cache_init'), asyncio.create_task(embedding_scanner.initialize_in_background(), name='embedding_cache_init')
asyncio.create_task(embedding_scanner.initialize_in_background(), name='embedding_cache_init'), asyncio.create_task(recipe_scanner.initialize_in_background(), name='recipe_cache_init')
asyncio.create_task(recipe_scanner.initialize_in_background(), name='recipe_cache_init')
]
await ExampleImagesMigration.check_and_run_migrations() await ExampleImagesMigration.check_and_run_migrations()
# Schedule post-initialization tasks to run after scanners complete logger.info("LoRA Manager: All services initialized and background tasks scheduled")
asyncio.create_task(
cls._run_post_initialization_tasks(init_tasks),
name='post_init_tasks'
)
logger.debug("LoRA Manager: All services initialized and background tasks scheduled")
except Exception as e: except Exception as e:
logger.error(f"LoRA Manager: Error initializing services: {e}", exc_info=True) logger.error(f"LoRA Manager: Error initializing services: {e}", exc_info=True)
@classmethod
async def _run_post_initialization_tasks(cls, init_tasks):
"""Run post-initialization tasks after all scanners complete"""
try:
logger.debug("LoRA Manager: Waiting for scanner initialization to complete...")
# Wait for all scanner initialization tasks to complete
await asyncio.gather(*init_tasks, return_exceptions=True)
logger.debug("LoRA Manager: Scanner initialization completed, starting post-initialization tasks...")
# Run post-initialization tasks
post_tasks = [
asyncio.create_task(cls._cleanup_backup_files(), name='cleanup_bak_files'),
# Add more post-initialization tasks here as needed
# asyncio.create_task(cls._another_post_task(), name='another_task'),
]
# Run all post-initialization tasks
results = await asyncio.gather(*post_tasks, return_exceptions=True)
# Log results
for i, result in enumerate(results):
task_name = post_tasks[i].get_name()
if isinstance(result, Exception):
logger.error(f"Post-initialization task '{task_name}' failed: {result}")
else:
logger.debug(f"Post-initialization task '{task_name}' completed successfully")
logger.debug("LoRA Manager: All post-initialization tasks completed")
except Exception as e:
logger.error(f"LoRA Manager: Error in post-initialization tasks: {e}", exc_info=True)
@classmethod
async def _cleanup_backup_files(cls):
"""Clean up .bak files in all model roots"""
try:
logger.debug("Starting cleanup of .bak files in model directories...")
# Collect all model roots
all_roots = set()
all_roots.update(config.loras_roots)
all_roots.update(config.base_models_roots)
all_roots.update(config.embeddings_roots)
total_deleted = 0
total_size_freed = 0
for root_path in all_roots:
if not os.path.exists(root_path):
continue
try:
deleted_count, size_freed = await cls._cleanup_backup_files_in_directory(root_path)
total_deleted += deleted_count
total_size_freed += size_freed
if deleted_count > 0:
logger.debug(f"Cleaned up {deleted_count} .bak files in {root_path} (freed {size_freed / (1024*1024):.2f} MB)")
except Exception as e:
logger.error(f"Error cleaning up .bak files in {root_path}: {e}")
# Yield control periodically
await asyncio.sleep(0.01)
if total_deleted > 0:
logger.debug(f"Backup cleanup completed: removed {total_deleted} .bak files, freed {total_size_freed / (1024*1024):.2f} MB total")
else:
logger.debug("Backup cleanup completed: no .bak files found")
except Exception as e:
logger.error(f"Error during backup file cleanup: {e}", exc_info=True)
@classmethod
async def _cleanup_backup_files_in_directory(cls, directory_path: str):
"""Clean up .bak files in a specific directory recursively
Args:
directory_path: Path to the directory to clean
Returns:
Tuple[int, int]: (number of files deleted, total size freed in bytes)
"""
deleted_count = 0
size_freed = 0
visited_paths = set()
def cleanup_recursive(path):
nonlocal deleted_count, size_freed
try:
real_path = os.path.realpath(path)
if real_path in visited_paths:
return
visited_paths.add(real_path)
with os.scandir(path) as it:
for entry in it:
try:
if entry.is_file(follow_symlinks=True) and entry.name.endswith('.bak'):
file_size = entry.stat().st_size
os.remove(entry.path)
deleted_count += 1
size_freed += file_size
logger.debug(f"Deleted .bak file: {entry.path}")
elif entry.is_dir(follow_symlinks=True):
cleanup_recursive(entry.path)
except Exception as e:
logger.warning(f"Could not delete .bak file {entry.path}: {e}")
except Exception as e:
logger.error(f"Error scanning directory {path} for .bak files: {e}")
# Run the recursive cleanup in a thread pool to avoid blocking
loop = asyncio.get_event_loop()
await loop.run_in_executor(None, cleanup_recursive, directory_path)
return deleted_count, size_freed
@classmethod
async def _cleanup_example_images_folders(cls):
"""Invoke the example images cleanup service for manual execution."""
try:
service = ExampleImagesCleanupService()
result = await service.cleanup_example_image_folders()
if result.get('success'):
logger.debug(
"Manual example images cleanup completed: moved=%s",
result.get('moved_total'),
)
elif result.get('partial_success'):
logger.warning(
"Manual example images cleanup partially succeeded: moved=%s failures=%s",
result.get('moved_total'),
result.get('move_failures'),
)
else:
logger.debug(
"Manual example images cleanup skipped or failed: %s",
result.get('error', 'no changes'),
)
return result
except Exception as e: # pragma: no cover - defensive guard
logger.error(f"Error during example images cleanup: {e}", exc_info=True)
return {
'success': False,
'error': str(e),
'error_code': 'unexpected_error',
}
@classmethod @classmethod
async def _cleanup(cls, app): async def _cleanup(cls, app):
"""Cleanup resources using ServiceRegistry""" """Cleanup resources using ServiceRegistry"""
try: try:
logger.info("LoRA Manager: Cleaning up services") logger.info("LoRA Manager: Cleaning up services")
# Close CivitaiClient gracefully
civitai_client = await ServiceRegistry.get_service("civitai_client")
if civitai_client:
await civitai_client.close()
logger.info("Closed CivitaiClient connection")
except Exception as e: except Exception as e:
logger.error(f"Error during cleanup: {e}", exc_info=True) logger.error(f"Error during cleanup: {e}", exc_info=True)

View File

@@ -1,7 +1,9 @@
import os import os
import importlib
import sys
# Check if running in standalone mode # Check if running in standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0" standalone_mode = 'nodes' not in sys.modules
if not standalone_mode: if not standalone_mode:
from .metadata_hook import MetadataHook from .metadata_hook import MetadataHook

View File

@@ -146,40 +146,52 @@ class MetadataHook:
# Store the original _async_map_node_over_list function # Store the original _async_map_node_over_list function
original_map_node_over_list = getattr(execution, map_node_func_name) original_map_node_over_list = getattr(execution, map_node_func_name)
# Wrapped async function, compatible with both stable and nightly # Define the wrapped async function - NOTE: Updated signature with prompt_id and unique_id!
async def async_map_node_over_list_with_metadata(prompt_id, unique_id, obj, input_data_all, func, allow_interrupt=False, execution_block_cb=None, pre_execute_cb=None, *args, **kwargs): async def async_map_node_over_list_with_metadata(prompt_id, unique_id, obj, input_data_all, func, allow_interrupt=False, execution_block_cb=None, pre_execute_cb=None):
hidden_inputs = kwargs.get('hidden_inputs', None)
# Only collect metadata when calling the main function of nodes # Only collect metadata when calling the main function of nodes
if func == obj.FUNCTION and hasattr(obj, '__class__'): if func == obj.FUNCTION and hasattr(obj, '__class__'):
try: try:
# Get the current prompt_id from the registry
registry = MetadataRegistry() registry = MetadataRegistry()
# We now have prompt_id directly from the function parameters
if prompt_id is not None: if prompt_id is not None:
# Get node class type
class_type = obj.__class__.__name__ class_type = obj.__class__.__name__
# Use the passed unique_id parameter instead of trying to extract it
node_id = unique_id node_id = unique_id
# Record inputs before execution
if node_id is not None: if node_id is not None:
registry.record_node_execution(node_id, class_type, input_data_all, None) registry.record_node_execution(node_id, class_type, input_data_all, None)
except Exception as e: except Exception as e:
print(f"Error collecting metadata (pre-execution): {str(e)}") print(f"Error collecting metadata (pre-execution): {str(e)}")
# Call original function with all args/kwargs # Execute the original async function with ALL parameters in the correct order
results = await original_map_node_over_list( results = await original_map_node_over_list(prompt_id, unique_id, obj, input_data_all, func, allow_interrupt, execution_block_cb, pre_execute_cb)
prompt_id, unique_id, obj, input_data_all, func,
allow_interrupt, execution_block_cb, pre_execute_cb, *args, **kwargs
)
# After execution, collect outputs for relevant nodes
if func == obj.FUNCTION and hasattr(obj, '__class__'): if func == obj.FUNCTION and hasattr(obj, '__class__'):
try: try:
# Get the current prompt_id from the registry
registry = MetadataRegistry() registry = MetadataRegistry()
if prompt_id is not None: if prompt_id is not None:
# Get node class type
class_type = obj.__class__.__name__ class_type = obj.__class__.__name__
# Use the passed unique_id parameter
node_id = unique_id node_id = unique_id
# Record outputs after execution
if node_id is not None: if node_id is not None:
registry.update_node_execution(node_id, class_type, results) registry.update_node_execution(node_id, class_type, results)
except Exception as e: except Exception as e:
print(f"Error collecting metadata (post-execution): {str(e)}") print(f"Error collecting metadata (post-execution): {str(e)}")
return results return results
# Also hook the execute function to track the current prompt_id # Also hook the execute function to track the current prompt_id
original_execute = execution.execute original_execute = execution.execute

View File

@@ -1,9 +1,9 @@
import json import json
import os import sys
from .constants import IMAGES from .constants import IMAGES
# Check if running in standalone mode # Check if running in standalone mode
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0" standalone_mode = 'nodes' not in sys.modules
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER
@@ -39,39 +39,8 @@ class MetadataProcessor:
if node_id in metadata.get(SAMPLING, {}) and metadata[SAMPLING][node_id].get(IS_SAMPLER, False): if node_id in metadata.get(SAMPLING, {}) and metadata[SAMPLING][node_id].get(IS_SAMPLER, False):
candidate_samplers[node_id] = metadata[SAMPLING][node_id] candidate_samplers[node_id] = metadata[SAMPLING][node_id]
# If we found candidate samplers, apply primary sampler logic to these candidates only # If we found candidate samplers, apply primary sampler logic to these candidates only
if candidate_samplers:
# PRE-PROCESS: Ensure all candidate samplers have their parameters populated
# This is especially important for SamplerCustomAdvanced which needs tracing
prompt = metadata.get("current_prompt")
for node_id in candidate_samplers:
# If a sampler is missing common parameters like steps or denoise,
# try to populate them using tracing before ranking
sampler_info = candidate_samplers[node_id]
params = sampler_info.get("parameters", {})
if prompt and (params.get("steps") is None or params.get("denoise") is None):
# Create a temporary params dict to use the handler
temp_params = {
"steps": params.get("steps"),
"denoise": params.get("denoise"),
"sampler": params.get("sampler_name"),
"scheduler": params.get("scheduler")
}
# Check if it's SamplerCustomAdvanced
if prompt.original_prompt and node_id in prompt.original_prompt:
if prompt.original_prompt[node_id].get("class_type") == "SamplerCustomAdvanced":
MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, node_id, temp_params)
# Update the actual parameters with found values
params["steps"] = temp_params.get("steps")
params["denoise"] = temp_params.get("denoise")
if temp_params.get("sampler"):
params["sampler_name"] = temp_params.get("sampler")
if temp_params.get("scheduler"):
params["scheduler"] = temp_params.get("scheduler")
# Collect potential primary samplers based on different criteria # Collect potential primary samplers based on different criteria
custom_advanced_samplers = [] custom_advanced_samplers = []
advanced_add_noise_samplers = [] advanced_add_noise_samplers = []
@@ -80,6 +49,7 @@ class MetadataProcessor:
high_denoise_id = None high_denoise_id = None
# First, check for SamplerCustomAdvanced among candidates # First, check for SamplerCustomAdvanced among candidates
prompt = metadata.get("current_prompt")
if prompt and prompt.original_prompt: if prompt and prompt.original_prompt:
for node_id in candidate_samplers: for node_id in candidate_samplers:
node_info = prompt.original_prompt.get(node_id, {}) node_info = prompt.original_prompt.get(node_id, {})
@@ -107,16 +77,15 @@ class MetadataProcessor:
# Combine all potential primary samplers # Combine all potential primary samplers
potential_samplers = custom_advanced_samplers + advanced_add_noise_samplers + high_denoise_samplers potential_samplers = custom_advanced_samplers + advanced_add_noise_samplers + high_denoise_samplers
# Find the first potential primary sampler (prefer base sampler over refine) # Find the most recent potential primary sampler (closest to downstream node)
# Use forward search to prioritize the first one in execution order for i in range(downstream_index - 1, -1, -1):
for i in range(downstream_index):
node_id = execution_order[i] node_id = execution_order[i]
if node_id in potential_samplers: if node_id in potential_samplers:
return node_id, candidate_samplers[node_id] return node_id, candidate_samplers[node_id]
# If no potential sampler found from our criteria, return the first sampler # If no potential sampler found from our criteria, return the most recent sampler
if candidate_samplers: if candidate_samplers:
for i in range(downstream_index): for i in range(downstream_index - 1, -1, -1):
node_id = execution_order[i] node_id = execution_order[i]
if node_id in candidate_samplers: if node_id in candidate_samplers:
return node_id, candidate_samplers[node_id] return node_id, candidate_samplers[node_id]
@@ -207,11 +176,8 @@ class MetadataProcessor:
found_node_id = input_value[0] # Connected node_id found_node_id = input_value[0] # Connected node_id
# If we're looking for a specific node class # If we're looking for a specific node class
if target_class: if target_class and prompt.original_prompt[found_node_id].get("class_type") == target_class:
if found_node_id not in prompt.original_prompt: return found_node_id
return None
if prompt.original_prompt[found_node_id].get("class_type") == target_class:
return found_node_id
# If we're not looking for a specific class, update the last valid node # If we're not looking for a specific class, update the last valid node
if not target_class: if not target_class:
@@ -219,19 +185,11 @@ class MetadataProcessor:
# Continue tracing through intermediate nodes # Continue tracing through intermediate nodes
current_node_id = found_node_id current_node_id = found_node_id
# For most conditioning nodes, the input we want to follow is named "conditioning"
# Check if current source node exists if "conditioning" in prompt.original_prompt[current_node_id].get("inputs", {}):
if current_node_id not in prompt.original_prompt:
return found_node_id if not target_class else None
# Determine which input to follow next on the source node
source_node_inputs = prompt.original_prompt[current_node_id].get("inputs", {})
if input_name in source_node_inputs:
current_input = input_name
elif "conditioning" in source_node_inputs:
current_input = "conditioning" current_input = "conditioning"
else: else:
# If there's no suitable input to follow, return the current node # If there's no "conditioning" input, return the current node
# if we're not looking for a specific target_class # if we're not looking for a specific target_class
return found_node_id if not target_class else None return found_node_id if not target_class else None
else: else:
@@ -244,89 +202,12 @@ class MetadataProcessor:
return last_valid_node if not target_class else None return last_valid_node if not target_class else None
@staticmethod @staticmethod
def trace_model_path(metadata, prompt, start_node_id): def find_primary_checkpoint(metadata):
""" """Find the primary checkpoint model in the workflow"""
Trace the model connection path upstream to find the checkpoint
"""
if not prompt or not prompt.original_prompt:
return None
current_node_id = start_node_id
depth = 0
max_depth = 50
while depth < max_depth:
# Check if current node is a registered checkpoint in our metadata
# This handles cached nodes correctly because metadata contains info for all nodes in the graph
if current_node_id in metadata.get(MODELS, {}):
if metadata[MODELS][current_node_id].get("type") == "checkpoint":
return current_node_id
if current_node_id not in prompt.original_prompt:
return None
node = prompt.original_prompt[current_node_id]
inputs = node.get("inputs", {})
class_type = node.get("class_type", "")
# Determine which input to follow next
next_input_name = "model"
# Special handling for initial node
if depth == 0:
if class_type == "SamplerCustomAdvanced":
next_input_name = "guider"
# If the specific input doesn't exist, try generic 'model'
if next_input_name not in inputs:
if "model" in inputs:
next_input_name = "model"
elif "basic_pipe" in inputs:
# Handle pipe nodes like FromBasicPipe by following the pipeline
next_input_name = "basic_pipe"
else:
# Dead end - no model input to follow
return None
# Get connected node
input_val = inputs[next_input_name]
if isinstance(input_val, list) and len(input_val) > 0:
current_node_id = input_val[0]
else:
return None
depth += 1
return None
@staticmethod
def find_primary_checkpoint(metadata, downstream_id=None, primary_sampler_id=None):
"""
Find the primary checkpoint model in the workflow
Parameters:
- metadata: The workflow metadata
- downstream_id: Optional ID of a downstream node to help identify the specific primary sampler
- primary_sampler_id: Optional ID of the primary sampler if already known
"""
if not metadata.get(MODELS): if not metadata.get(MODELS):
return None return None
# Method 1: Topology-based tracing (More accurate for complex workflows)
# First, find the primary sampler if not provided
if not primary_sampler_id:
primary_sampler_id, _ = MetadataProcessor.find_primary_sampler(metadata, downstream_id)
if primary_sampler_id:
prompt = metadata.get("current_prompt")
if prompt:
# Trace back from the sampler to find the checkpoint
checkpoint_id = MetadataProcessor.trace_model_path(metadata, prompt, primary_sampler_id)
if checkpoint_id and checkpoint_id in metadata.get(MODELS, {}):
return metadata[MODELS][checkpoint_id].get("name")
# Method 2: Fallback to the first available checkpoint (Original behavior) # In most workflows, there's only one checkpoint, so we can just take the first one
# In most simple workflows, there's only one checkpoint, so we can just take the first one
for node_id, model_info in metadata.get(MODELS, {}).items(): for node_id, model_info in metadata.get(MODELS, {}).items():
if model_info.get("type") == "checkpoint": if model_info.get("type") == "checkpoint":
return model_info.get("name") return model_info.get("name")
@@ -414,7 +295,7 @@ class MetadataProcessor:
"seed": None, "seed": None,
"steps": None, "steps": None,
"cfg_scale": None, "cfg_scale": None,
# "guidance": None, # Add guidance parameter "guidance": None, # Add guidance parameter
"sampler": None, "sampler": None,
"scheduler": None, "scheduler": None,
"checkpoint": None, "checkpoint": None,
@@ -430,8 +311,7 @@ class MetadataProcessor:
primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id) primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id)
# Directly get checkpoint from metadata instead of tracing # Directly get checkpoint from metadata instead of tracing
# Pass primary_sampler_id to avoid redundant calculation checkpoint = MetadataProcessor.find_primary_checkpoint(metadata)
checkpoint = MetadataProcessor.find_primary_checkpoint(metadata, id, primary_sampler_id)
if checkpoint: if checkpoint:
params["checkpoint"] = checkpoint params["checkpoint"] = checkpoint
@@ -459,8 +339,44 @@ class MetadataProcessor:
is_custom_advanced = prompt.original_prompt[primary_sampler_id].get("class_type") == "SamplerCustomAdvanced" is_custom_advanced = prompt.original_prompt[primary_sampler_id].get("class_type") == "SamplerCustomAdvanced"
if is_custom_advanced: if is_custom_advanced:
# For SamplerCustomAdvanced, use the new handler method # For SamplerCustomAdvanced, trace specific inputs
MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, primary_sampler_id, params)
# 1. Trace sigmas input to find BasicScheduler
scheduler_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "sigmas", "BasicScheduler", max_depth=5)
if scheduler_node_id and scheduler_node_id in metadata.get(SAMPLING, {}):
scheduler_params = metadata[SAMPLING][scheduler_node_id].get("parameters", {})
params["steps"] = scheduler_params.get("steps")
params["scheduler"] = scheduler_params.get("scheduler")
# 2. Trace sampler input to find KSamplerSelect
sampler_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "sampler", "KSamplerSelect", max_depth=5)
if sampler_node_id and sampler_node_id in metadata.get(SAMPLING, {}):
sampler_params = metadata[SAMPLING][sampler_node_id].get("parameters", {})
params["sampler"] = sampler_params.get("sampler_name")
# 3. Trace guider input for CFGGuider and CLIPTextEncode
guider_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "guider", max_depth=5)
if guider_node_id and guider_node_id in prompt.original_prompt:
# Check if the guider node is a CFGGuider
if prompt.original_prompt[guider_node_id].get("class_type") == "CFGGuider":
# Extract cfg value from the CFGGuider
if guider_node_id in metadata.get(SAMPLING, {}):
cfg_params = metadata[SAMPLING][guider_node_id].get("parameters", {})
params["cfg_scale"] = cfg_params.get("cfg")
# Find CLIPTextEncode for positive prompt
positive_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "positive", "CLIPTextEncode", max_depth=10)
if positive_node_id and positive_node_id in metadata.get(PROMPTS, {}):
params["prompt"] = metadata[PROMPTS][positive_node_id].get("text", "")
# Find CLIPTextEncode for negative prompt
negative_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "negative", "CLIPTextEncode", max_depth=10)
if negative_node_id and negative_node_id in metadata.get(PROMPTS, {}):
params["negative_prompt"] = metadata[PROMPTS][negative_node_id].get("text", "")
else:
positive_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "conditioning", max_depth=10)
if positive_node_id and positive_node_id in metadata.get(PROMPTS, {}):
params["prompt"] = metadata[PROMPTS][positive_node_id].get("text", "")
else: else:
# For standard samplers, match conditioning objects to prompts # For standard samplers, match conditioning objects to prompts
@@ -485,9 +401,6 @@ class MetadataProcessor:
negative_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "negative", max_depth=10) negative_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "negative", max_depth=10)
if negative_node_id and negative_node_id in metadata.get(PROMPTS, {}): if negative_node_id and negative_node_id in metadata.get(PROMPTS, {}):
params["negative_prompt"] = metadata[PROMPTS][negative_node_id].get("text", "") params["negative_prompt"] = metadata[PROMPTS][negative_node_id].get("text", "")
# For SamplerCustom, handle any additional parameters
MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, primary_sampler_id, params)
# Size extraction is same for all sampler types # Size extraction is same for all sampler types
# Check if the sampler itself has size information (from latent_image) # Check if the sampler itself has size information (from latent_image)
@@ -541,60 +454,3 @@ class MetadataProcessor:
"""Convert metadata to JSON string""" """Convert metadata to JSON string"""
params = MetadataProcessor.to_dict(metadata, id) params = MetadataProcessor.to_dict(metadata, id)
return json.dumps(params, indent=4) return json.dumps(params, indent=4)
@staticmethod
def handle_custom_advanced_sampler(metadata, prompt, primary_sampler_id, params):
"""
Handle parameter extraction for SamplerCustomAdvanced nodes
Parameters:
- metadata: The workflow metadata
- prompt: The prompt object containing node connections
- primary_sampler_id: ID of the SamplerCustomAdvanced node
- params: Parameters dictionary to update
"""
if not prompt.original_prompt or primary_sampler_id not in prompt.original_prompt:
return
sampler_inputs = prompt.original_prompt[primary_sampler_id].get("inputs", {})
# 1. Trace sigmas input to find BasicScheduler (only if sigmas input exists)
if "sigmas" in sampler_inputs:
scheduler_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "sigmas", None, max_depth=5)
if scheduler_node_id and scheduler_node_id in metadata.get(SAMPLING, {}):
scheduler_params = metadata[SAMPLING][scheduler_node_id].get("parameters", {})
params["steps"] = scheduler_params.get("steps")
params["scheduler"] = scheduler_params.get("scheduler")
params["denoise"] = scheduler_params.get("denoise")
# 2. Trace sampler input to find KSamplerSelect (only if sampler input exists)
if "sampler" in sampler_inputs:
sampler_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "sampler", "KSamplerSelect", max_depth=5)
if sampler_node_id and sampler_node_id in metadata.get(SAMPLING, {}):
sampler_params = metadata[SAMPLING][sampler_node_id].get("parameters", {})
params["sampler"] = sampler_params.get("sampler_name")
# 3. Trace guider input for CFGGuider and CLIPTextEncode
if "guider" in sampler_inputs:
guider_node_id = MetadataProcessor.trace_node_input(prompt, primary_sampler_id, "guider", max_depth=5)
if guider_node_id and guider_node_id in prompt.original_prompt:
# Check if the guider node is a CFGGuider
if prompt.original_prompt[guider_node_id].get("class_type") == "CFGGuider":
# Extract cfg value from the CFGGuider
if guider_node_id in metadata.get(SAMPLING, {}):
cfg_params = metadata[SAMPLING][guider_node_id].get("parameters", {})
params["cfg_scale"] = cfg_params.get("cfg")
# Find CLIPTextEncode for positive prompt
positive_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "positive", "CLIPTextEncode", max_depth=10)
if positive_node_id and positive_node_id in metadata.get(PROMPTS, {}):
params["prompt"] = metadata[PROMPTS][positive_node_id].get("text", "")
# Find CLIPTextEncode for negative prompt
negative_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "negative", "CLIPTextEncode", max_depth=10)
if negative_node_id and negative_node_id in metadata.get(PROMPTS, {}):
params["negative_prompt"] = metadata[PROMPTS][negative_node_id].get("text", "")
else:
positive_node_id = MetadataProcessor.trace_node_input(prompt, guider_node_id, "conditioning", max_depth=10)
if positive_node_id and positive_node_id in metadata.get(PROMPTS, {}):
params["prompt"] = metadata[PROMPTS][positive_node_id].get("text", "")

View File

@@ -196,11 +196,9 @@ class MetadataRegistry:
node_metadata[category] = {} node_metadata[category] = {}
node_metadata[category][node_id] = current_metadata[category][node_id] node_metadata[category][node_id] = current_metadata[category][node_id]
# Save new metadata or clear stale cache entries when metadata is empty # Save to cache if we have any metadata for this node
if any(node_metadata.values()): if any(node_metadata.values()):
self.node_cache[cache_key] = node_metadata self.node_cache[cache_key] = node_metadata
else:
self.node_cache.pop(cache_key, None)
def clear_unused_cache(self): def clear_unused_cache(self):
"""Clean up node_cache entries that are no longer in use""" """Clean up node_cache entries that are no longer in use"""

View File

@@ -3,18 +3,6 @@ import os
from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER
def _store_checkpoint_metadata(metadata, node_id, model_name):
"""Store checkpoint model information when available."""
if not model_name:
return
metadata.setdefault(MODELS, {})
metadata[MODELS][node_id] = {
"name": model_name,
"type": "checkpoint",
"node_id": node_id
}
class NodeMetadataExtractor: class NodeMetadataExtractor:
"""Base class for node-specific metadata extraction""" """Base class for node-specific metadata extraction"""
@@ -41,48 +29,12 @@ class CheckpointLoaderExtractor(NodeMetadataExtractor):
return return
model_name = inputs.get("ckpt_name") model_name = inputs.get("ckpt_name")
_store_checkpoint_metadata(metadata, node_id, model_name) if model_name:
metadata[MODELS][node_id] = {
"name": model_name,
class NunchakuFluxDiTLoaderExtractor(NodeMetadataExtractor): "type": "checkpoint",
@staticmethod "node_id": node_id
def extract(node_id, inputs, outputs, metadata): }
if not inputs or "model_path" not in inputs:
return
model_name = inputs.get("model_path")
_store_checkpoint_metadata(metadata, node_id, model_name)
class NunchakuQwenImageDiTLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "model_name" not in inputs:
return
model_name = inputs.get("model_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class GGUFLoaderExtractor(NodeMetadataExtractor):
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "gguf_name" not in inputs:
return
model_name = inputs.get("gguf_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class KJNodesModelLoaderExtractor(NodeMetadataExtractor):
"""Extract metadata from KJNodes loaders that expose `model_name`."""
@staticmethod
def extract(node_id, inputs, outputs, metadata):
if not inputs or "model_name" not in inputs:
return
model_name = inputs.get("model_name")
_store_checkpoint_metadata(metadata, node_id, model_name)
class TSCCheckpointLoaderExtractor(NodeMetadataExtractor): class TSCCheckpointLoaderExtractor(NodeMetadataExtractor):
@staticmethod @staticmethod
@@ -91,7 +43,12 @@ class TSCCheckpointLoaderExtractor(NodeMetadataExtractor):
return return
model_name = inputs.get("ckpt_name") model_name = inputs.get("ckpt_name")
_store_checkpoint_metadata(metadata, node_id, model_name) if model_name:
metadata[MODELS][node_id] = {
"name": model_name,
"type": "checkpoint",
"node_id": node_id
}
# For loader node has lora_stack input, like Efficient Loader from Efficient Nodes # For loader node has lora_stack input, like Efficient Loader from Efficient Nodes
active_loras = [] active_loras = []
@@ -685,45 +642,31 @@ NODE_EXTRACTORS = {
# Sampling # Sampling
"KSampler": SamplerExtractor, "KSampler": SamplerExtractor,
"KSamplerAdvanced": KSamplerAdvancedExtractor, "KSamplerAdvanced": KSamplerAdvancedExtractor,
"SamplerCustom": KSamplerAdvancedExtractor,
"SamplerCustomAdvanced": SamplerCustomAdvancedExtractor, "SamplerCustomAdvanced": SamplerCustomAdvancedExtractor,
"ClownsharKSampler_Beta": SamplerExtractor,
"TSC_KSampler": TSCKSamplerExtractor, # Efficient Nodes "TSC_KSampler": TSCKSamplerExtractor, # Efficient Nodes
"TSC_KSamplerAdvanced": TSCKSamplerAdvancedExtractor, # Efficient Nodes "TSC_KSamplerAdvanced": TSCKSamplerAdvancedExtractor, # Efficient Nodes
"KSamplerBasicPipe": KSamplerBasicPipeExtractor, # comfyui-impact-pack "KSamplerBasicPipe": KSamplerBasicPipeExtractor, # comfyui-impact-pack
"KSamplerAdvancedBasicPipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-impact-pack "KSamplerAdvancedBasicPipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-impact-pack
"KSampler_inspire_pipe": KSamplerBasicPipeExtractor, # comfyui-inspire-pack "KSampler_inspire_pipe": KSamplerBasicPipeExtractor, # comfyui-inspire-pack
"KSamplerAdvanced_inspire_pipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-inspire-pack "KSamplerAdvanced_inspire_pipe": KSamplerAdvancedBasicPipeExtractor, # comfyui-inspire-pack
"KSampler_inspire": SamplerExtractor, # comfyui-inspire-pack
# Sampling Selectors # Sampling Selectors
"KSamplerSelect": KSamplerSelectExtractor, # Add KSamplerSelect "KSamplerSelect": KSamplerSelectExtractor, # Add KSamplerSelect
"BasicScheduler": BasicSchedulerExtractor, # Add BasicScheduler "BasicScheduler": BasicSchedulerExtractor, # Add BasicScheduler
"AlignYourStepsScheduler": BasicSchedulerExtractor, # Add AlignYourStepsScheduler
# Loaders # Loaders
"CheckpointLoaderSimple": CheckpointLoaderExtractor, "CheckpointLoaderSimple": CheckpointLoaderExtractor,
"comfyLoader": CheckpointLoaderExtractor, # easy comfyLoader "comfyLoader": CheckpointLoaderExtractor, # easy comfyLoader
"CheckpointLoaderSimpleWithImages": CheckpointLoaderExtractor, # CheckpointLoader|pysssss
"TSC_EfficientLoader": TSCCheckpointLoaderExtractor, # Efficient Nodes "TSC_EfficientLoader": TSCCheckpointLoaderExtractor, # Efficient Nodes
"NunchakuFluxDiTLoader": NunchakuFluxDiTLoaderExtractor, # ComfyUI-Nunchaku
"NunchakuQwenImageDiTLoader": NunchakuQwenImageDiTLoaderExtractor, # ComfyUI-Nunchaku
"LoaderGGUF": GGUFLoaderExtractor, # calcuis gguf
"LoaderGGUFAdvanced": GGUFLoaderExtractor, # calcuis gguf
"GGUFLoaderKJ": KJNodesModelLoaderExtractor, # KJNodes
"DiffusionModelLoaderKJ": KJNodesModelLoaderExtractor, # KJNodes
"CheckpointLoaderKJ": CheckpointLoaderExtractor, # KJNodes
"UNETLoader": UNETLoaderExtractor, # Updated to use dedicated extractor "UNETLoader": UNETLoaderExtractor, # Updated to use dedicated extractor
"UnetLoaderGGUF": UNETLoaderExtractor, # Updated to use dedicated extractor "UnetLoaderGGUF": UNETLoaderExtractor, # Updated to use dedicated extractor
"LoraLoader": LoraLoaderExtractor, "LoraLoader": LoraLoaderExtractor,
"LoraManagerLoader": LoraLoaderManagerExtractor, "LoraManagerLoader": LoraLoaderManagerExtractor,
# Conditioning # Conditioning
"CLIPTextEncode": CLIPTextEncodeExtractor, "CLIPTextEncode": CLIPTextEncodeExtractor,
"PromptLoraManager": CLIPTextEncodeExtractor,
"CLIPTextEncodeFlux": CLIPTextEncodeFluxExtractor, # Add CLIPTextEncodeFlux "CLIPTextEncodeFlux": CLIPTextEncodeFluxExtractor, # Add CLIPTextEncodeFlux
"WAS_Text_to_Conditioning": CLIPTextEncodeExtractor, "WAS_Text_to_Conditioning": CLIPTextEncodeExtractor,
"AdvancedCLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb "AdvancedCLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/BlenderNeko/ComfyUI_ADV_CLIP_emb
"smZ_CLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/shiimizu/ComfyUI_smZNodes "smZ_CLIPTextEncode": CLIPTextEncodeExtractor, # From https://github.com/shiimizu/ComfyUI_smZNodes
"CR_ApplyControlNetStack": CR_ApplyControlNetStackExtractor, # Add CR_ApplyControlNetStack "CR_ApplyControlNetStack": CR_ApplyControlNetStackExtractor, # Add CR_ApplyControlNetStack
"PCTextEncode": CLIPTextEncodeExtractor, # From https://github.com/asagi4/comfyui-prompt-control
# Latent # Latent
"EmptyLatentImage": ImageSizeExtractor, "EmptyLatentImage": ImageSizeExtractor,
# Flux # Flux

View File

@@ -1 +0,0 @@
"""Server middleware modules"""

View File

@@ -1,53 +0,0 @@
"""Cache control middleware for ComfyUI server"""
from aiohttp import web
from typing import Callable, Awaitable
# Time in seconds
ONE_HOUR: int = 3600
ONE_DAY: int = 86400
IMG_EXTENSIONS = (
".jpg",
".jpeg",
".png",
".ppm",
".bmp",
".pgm",
".tif",
".tiff",
".webp",
".mp4"
)
@web.middleware
async def cache_control(
request: web.Request, handler: Callable[[web.Request], Awaitable[web.Response]]
) -> web.Response:
"""Cache control middleware that sets appropriate cache headers based on file type and response status"""
response: web.Response = await handler(request)
if (
request.path.endswith(".js")
or request.path.endswith(".css")
or request.path.endswith("index.json")
):
response.headers.setdefault("Cache-Control", "no-cache")
return response
# Early return for non-image files - no cache headers needed
if not request.path.lower().endswith(IMG_EXTENSIONS):
return response
# Handle image files
if response.status == 404:
response.headers.setdefault("Cache-Control", f"public, max-age={ONE_HOUR}")
elif response.status in (200, 201, 202, 203, 204, 205, 206, 301, 308):
# Success responses and permanent redirects - cache for 1 day
response.headers.setdefault("Cache-Control", f"public, max-age={ONE_DAY}")
elif response.status in (302, 303, 307):
# Temporary redirects - no cache
response.headers.setdefault("Cache-Control", "no-cache")
# Note: 304 Not Modified falls through - no cache headers set
return response

View File

@@ -1,65 +0,0 @@
"""Middleware helpers for adjusting Content Security Policy headers."""
from typing import Awaitable, Callable, Dict, List
from aiohttp import web
REMOTE_MEDIA_SOURCES = (
"https://image.civitai.com",
"https://img.genur.art",
)
@web.middleware
async def relax_csp_for_remote_media(
request: web.Request, handler: Callable[[web.Request], Awaitable[web.StreamResponse]]
) -> web.StreamResponse:
"""Allow LoRA Manager media previews to load from trusted remote domains.
When ComfyUI is started with ``--disable-api-nodes`` it injects a restrictive
``Content-Security-Policy`` header that blocks remote images and videos. The
LoRA Manager UI legitimately needs to fetch previews from Civitai and Genur,
so this middleware augments the existing CSP to whitelist those hosts while
preserving all other directives.
"""
response: web.StreamResponse = await handler(request)
header_value = response.headers.get("Content-Security-Policy")
if not header_value:
return response
directive_order: List[str] = []
directives: Dict[str, List[str]] = {}
for raw_directive in header_value.split(";"):
directive = raw_directive.strip()
if not directive:
continue
parts = directive.split()
name, values = parts[0], parts[1:]
if name not in directive_order:
directive_order.append(name)
directives[name] = values
def merge_sources(name: str, sources: List[str], defaults: List[str] | None = None) -> None:
existing = directives.get(name, list(defaults or []))
for source in sources:
if source not in existing:
existing.append(source)
directives[name] = existing
if name not in directive_order:
directive_order.append(name)
merge_sources("img-src", list(REMOTE_MEDIA_SOURCES))
merge_sources("media-src", ["'self'", *REMOTE_MEDIA_SOURCES], defaults=["'self'"])
updated_header = "; ".join(
f"{name} {' '.join(directives[name])}".rstrip() for name in directive_order
)
response.headers["Content-Security-Policy"] = f"{updated_header};"
return response

View File

@@ -1,15 +1,15 @@
import logging import logging
from server import PromptServer # type: ignore
from ..metadata_collector.metadata_processor import MetadataProcessor from ..metadata_collector.metadata_processor import MetadataProcessor
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class DebugMetadata: class DebugMetadata:
NAME = "Debug Metadata (LoraManager)" NAME = "Debug Metadata (LoraManager)"
CATEGORY = "Lora Manager/utils" CATEGORY = "Lora Manager/utils"
DESCRIPTION = "Debug node to verify metadata_processor functionality" DESCRIPTION = "Debug node to verify metadata_processor functionality"
OUTPUT_NODE = True OUTPUT_NODE = True
@classmethod @classmethod
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
@@ -25,37 +25,21 @@ class DebugMetadata:
FUNCTION = "process_metadata" FUNCTION = "process_metadata"
def process_metadata(self, images, id): def process_metadata(self, images, id):
"""
Process metadata from the execution context and return it for UI display.
The metadata is returned via the 'ui' key in the return dict, which triggers
node.onExecuted on the frontend to update the JsonDisplayWidget.
Args:
images: Input images (required for execution flow)
id: Node's unique ID (hidden)
Returns:
Dict with 'result' (empty tuple) and 'ui' (metadata dict for widget display)
"""
try: try:
# Get the current execution context's metadata # Get the current execution context's metadata
from ..metadata_collector import get_metadata from ..metadata_collector import get_metadata
metadata = get_metadata() metadata = get_metadata()
# Use the MetadataProcessor to convert it to dict # Use the MetadataProcessor to convert it to JSON string
metadata_dict = MetadataProcessor.to_dict(metadata, id) metadata_json = MetadataProcessor.to_json(metadata, id)
return { # Send metadata to frontend for display
"result": (), PromptServer.instance.send_sync("metadata_update", {
# ComfyUI expects ui values to be lists, wrap the dict in a list "id": id,
"ui": {"metadata": [metadata_dict]}, "metadata": metadata_json
} })
except Exception as e: except Exception as e:
logger.error(f"Error processing metadata: {e}") logger.error(f"Error processing metadata: {e}")
return {
"result": (), return ()
"ui": {"metadata": [{"error": str(e)}]},
}

View File

@@ -1,6 +1,6 @@
import logging import logging
import re
from nodes import LoraLoader from nodes import LoraLoader
from comfy.comfy_types import IO # type: ignore
from ..utils.utils import get_lora_info from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list, nunchaku_load_lora from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list, nunchaku_load_lora
@@ -16,9 +16,8 @@ class LoraManagerLoader:
"required": { "required": {
"model": ("MODEL",), "model": ("MODEL",),
# "clip": ("CLIP",), # "clip": ("CLIP",),
"text": ("STRING", { "text": (IO.STRING, {
"multiline": True, "multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True, "dynamicPrompts": True,
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation", "tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
"placeholder": "LoRA syntax input: <lora:name:strength>" "placeholder": "LoRA syntax input: <lora:name:strength>"
@@ -27,7 +26,7 @@ class LoraManagerLoader:
"optional": FlexibleOptionalInputType(any_type), "optional": FlexibleOptionalInputType(any_type),
} }
RETURN_TYPES = ("MODEL", "CLIP", "STRING", "STRING") RETURN_TYPES = ("MODEL", "CLIP", IO.STRING, IO.STRING)
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras") RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras" FUNCTION = "load_loras"
@@ -110,143 +109,6 @@ class LoraManagerLoader:
# use ',, ' to separate trigger words for group mode # use ',, ' to separate trigger words for group mode
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else "" trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
# Format loaded_loras with support for both formats
formatted_loras = []
for item in loaded_loras:
parts = item.split(":")
lora_name = parts[0]
strength_parts = parts[1].strip().split(",")
if len(strength_parts) > 1:
# Different model and clip strengths
model_str = strength_parts[0].strip()
clip_str = strength_parts[1].strip()
formatted_loras.append(f"<lora:{lora_name}:{model_str}:{clip_str}>")
else:
# Same strength for both
model_str = strength_parts[0].strip()
formatted_loras.append(f"<lora:{lora_name}:{model_str}>")
formatted_loras_text = " ".join(formatted_loras)
return (model, clip, trigger_words_text, formatted_loras_text)
class LoraManagerTextLoader:
NAME = "LoRA Text Loader (LoraManager)"
CATEGORY = "Lora Manager/loaders"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("MODEL",),
"lora_syntax": ("STRING", {
"forceInput": True,
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation"
}),
},
"optional": {
"clip": ("CLIP",),
"lora_stack": ("LORA_STACK",),
}
}
RETURN_TYPES = ("MODEL", "CLIP", "STRING", "STRING")
RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
FUNCTION = "load_loras_from_text"
def parse_lora_syntax(self, text):
"""Parse LoRA syntax from text input."""
# Pattern to match <lora:name:strength> or <lora:name:model_strength:clip_strength>
pattern = r'<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>'
matches = re.findall(pattern, text, re.IGNORECASE)
loras = []
for match in matches:
lora_name = match[0]
model_strength = float(match[1])
clip_strength = float(match[2]) if match[2] else model_strength
loras.append({
'name': lora_name,
'model_strength': model_strength,
'clip_strength': clip_strength
})
return loras
def load_loras_from_text(self, model, lora_syntax, clip=None, lora_stack=None):
"""Load LoRAs based on text syntax input."""
loaded_loras = []
all_trigger_words = []
# Check if model is a Nunchaku Flux model - simplified approach
is_nunchaku_model = False
try:
model_wrapper = model.model.diffusion_model
# Check if model is a Nunchaku Flux model using only class name
if model_wrapper.__class__.__name__ == "ComfyFluxWrapper":
is_nunchaku_model = True
logger.info("Detected Nunchaku Flux model")
except (AttributeError, TypeError):
# Not a model with the expected structure
pass
# First process lora_stack if available
if lora_stack:
for lora_path, model_strength, clip_strength in lora_stack:
# Apply the LoRA using the appropriate loader
if is_nunchaku_model:
# Use our custom function for Flux models
model = nunchaku_load_lora(model, lora_path, model_strength)
# clip remains unchanged for Nunchaku models
else:
# Use default loader for standard models
model, clip = LoraLoader().load_lora(model, clip, lora_path, model_strength, clip_strength)
# Extract lora name for trigger words lookup
lora_name = extract_lora_name(lora_path)
_, trigger_words = get_lora_info(lora_name)
all_trigger_words.extend(trigger_words)
# Add clip strength to output if different from model strength (except for Nunchaku models)
if not is_nunchaku_model and abs(model_strength - clip_strength) > 0.001:
loaded_loras.append(f"{lora_name}: {model_strength},{clip_strength}")
else:
loaded_loras.append(f"{lora_name}: {model_strength}")
# Parse and process LoRAs from text syntax
parsed_loras = self.parse_lora_syntax(lora_syntax)
for lora in parsed_loras:
lora_name = lora['name']
model_strength = lora['model_strength']
clip_strength = lora['clip_strength']
# Get lora path and trigger words
lora_path, trigger_words = get_lora_info(lora_name)
# Apply the LoRA using the appropriate loader
if is_nunchaku_model:
# For Nunchaku models, use our custom function
model = nunchaku_load_lora(model, lora_path, model_strength)
# clip remains unchanged
else:
# Use default loader for standard models
model, clip = LoraLoader().load_lora(model, clip, lora_path, model_strength, clip_strength)
# Include clip strength in output if different from model strength and not a Nunchaku model
if not is_nunchaku_model and abs(model_strength - clip_strength) > 0.001:
loaded_loras.append(f"{lora_name}: {model_strength},{clip_strength}")
else:
loaded_loras.append(f"{lora_name}: {model_strength}")
# Add trigger words to collection
all_trigger_words.extend(trigger_words)
# use ',, ' to separate trigger words for group mode
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
# Format loaded_loras with support for both formats # Format loaded_loras with support for both formats
formatted_loras = [] formatted_loras = []
for item in loaded_loras: for item in loaded_loras:

View File

@@ -1,87 +0,0 @@
"""
LoRA Pool Node - Defines filter configuration for LoRA selection.
This node provides a visual filter editor that generates a LORA_POOL_CONFIG
object for use by downstream nodes (like LoRA Randomizer).
"""
import logging
logger = logging.getLogger(__name__)
class LoraPoolNode:
"""
A node that defines LoRA filter criteria through a Vue-based widget.
Outputs a LORA_POOL_CONFIG that can be consumed by:
- Frontend: LoRA Randomizer widget reads connected pool's widget value
- Backend: LoRA Randomizer receives config during workflow execution
"""
NAME = "Lora Pool (LoraManager)"
CATEGORY = "Lora Manager/randomizer"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"pool_config": ("LORA_POOL_CONFIG", {}),
},
"hidden": {
# Hidden input to pass through unique node ID for frontend
"unique_id": "UNIQUE_ID",
},
}
RETURN_TYPES = ("POOL_CONFIG",)
RETURN_NAMES = ("POOL_CONFIG",)
FUNCTION = "process"
OUTPUT_NODE = False
def process(self, pool_config, unique_id=None):
"""
Pass through the pool configuration filters.
The config is generated entirely by the frontend widget.
This function validates and returns only the filters field.
Args:
pool_config: Dict containing filter criteria from widget
unique_id: Node's unique ID (hidden)
Returns:
Tuple containing the filters dict from pool_config
"""
# Validate required structure
if not isinstance(pool_config, dict):
logger.warning("Invalid pool_config type, using empty config")
pool_config = self._default_config()
# Ensure version field exists
if "version" not in pool_config:
pool_config["version"] = 1
# Extract filters field
filters = pool_config.get("filters", self._default_config()["filters"])
# Log for debugging
logger.debug(f"[LoraPoolNode] Processing filters: {filters}")
return (filters,)
@staticmethod
def _default_config():
"""Return default empty configuration."""
return {
"version": 1,
"filters": {
"baseModels": [],
"tags": {"include": [], "exclude": []},
"folders": {"include": [], "exclude": []},
"favoritesOnly": False,
"license": {"noCreditRequired": False, "allowSelling": False},
},
"preview": {"matchCount": 0, "lastUpdated": 0},
}

View File

@@ -1,206 +0,0 @@
"""
Lora Randomizer Node - Randomly selects LoRAs from a pool with configurable settings.
This node accepts optional pool_config input to filter available LoRAs, and outputs
a LORA_STACK with randomly selected LoRAs. Returns UI updates with new random LoRAs
and tracks the last used combination for reuse.
"""
import logging
import random
import os
from ..utils.utils import get_lora_info
from .utils import extract_lora_name
logger = logging.getLogger(__name__)
class LoraRandomizerNode:
"""Node that randomly selects LoRAs from a pool"""
NAME = "Lora Randomizer (LoraManager)"
CATEGORY = "Lora Manager/randomizer"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"randomizer_config": ("RANDOMIZER_CONFIG", {}),
"loras": ("LORAS", {}),
},
"optional": {
"pool_config": ("POOL_CONFIG", {}),
},
}
RETURN_TYPES = ("LORA_STACK",)
RETURN_NAMES = ("LORA_STACK",)
FUNCTION = "randomize"
OUTPUT_NODE = False
def _preprocess_loras_input(self, loras):
"""
Preprocess loras input to handle different widget formats.
Args:
loras: Input from widget, either:
- List of LoRA dicts (expected format)
- Dict with '__value__' key containing the list
Returns:
List of LoRA dicts
"""
if isinstance(loras, dict) and "__value__" in loras:
return loras["__value__"]
return loras
async def randomize(self, randomizer_config, loras, pool_config=None):
"""
Randomize LoRAs based on configuration and pool filters.
Args:
randomizer_config: Dict with randomizer settings (count, strength ranges, roll_mode)
loras: List of LoRA dicts from LORAS widget (includes locked state)
pool_config: Optional config from LoRA Pool node for filtering
Returns:
Dictionary with 'result' (LORA_STACK tuple) and 'ui' (for widget display)
"""
from ..services.service_registry import ServiceRegistry
loras = self._preprocess_loras_input(loras)
roll_mode = randomizer_config.get("roll_mode", "always")
logger.debug(f"[LoraRandomizerNode] roll_mode: {roll_mode}")
# Dual seed mechanism for batch queue synchronization
# execution_seed: seed for generating execution_stack (= previous next_seed)
# next_seed: seed for generating ui_loras (= what will be displayed after execution)
execution_seed = randomizer_config.get("execution_seed", None)
next_seed = randomizer_config.get("next_seed", None)
if roll_mode == "fixed":
ui_loras = loras
execution_loras = loras
else:
scanner = await ServiceRegistry.get_lora_scanner()
# Generate execution_loras from execution_seed (if available)
if execution_seed is not None:
# Use execution_seed to regenerate the same loras that were shown to user
execution_loras = await self._generate_random_loras_for_ui(
scanner, randomizer_config, loras, pool_config, seed=execution_seed
)
else:
# First execution: use loras input (what user sees in the widget)
execution_loras = loras
# Generate ui_loras from next_seed (for display after execution)
ui_loras = await self._generate_random_loras_for_ui(
scanner, randomizer_config, loras, pool_config, seed=next_seed
)
execution_stack = self._build_execution_stack_from_input(execution_loras)
return {
"result": (execution_stack,),
"ui": {"loras": ui_loras, "last_used": execution_loras},
}
def _build_execution_stack_from_input(self, loras):
"""
Build LORA_STACK tuple from input loras list for execution.
Args:
loras: List of LoRA dicts with name, strength, clipStrength, active
Returns:
List of tuples (lora_path, model_strength, clip_strength)
"""
lora_stack = []
for lora in loras:
if not lora.get("active", False):
continue
# Get file path
lora_path, trigger_words = get_lora_info(lora["name"])
if not lora_path:
logger.warning(
f"[LoraRandomizerNode] Could not find path for LoRA: {lora['name']}"
)
continue
# Normalize path separators
lora_path = lora_path.replace("/", os.sep)
# Extract strengths (convert to float to prevent string subtraction errors)
model_strength = float(lora.get("strength", 1.0))
clip_strength = float(lora.get("clipStrength", model_strength))
lora_stack.append((lora_path, model_strength, clip_strength))
return lora_stack
async def _generate_random_loras_for_ui(
self, scanner, randomizer_config, input_loras, pool_config=None, seed=None
):
"""
Generate new random loras for UI display.
Args:
scanner: LoraScanner instance
randomizer_config: Dict with randomizer settings
input_loras: Current input loras (for extracting locked loras)
pool_config: Optional pool filters
seed: Optional seed for deterministic randomization
Returns:
List of LoRA dicts for UI display
"""
from ..services.lora_service import LoraService
# Parse randomizer settings (convert numeric values to float to prevent type errors)
count_mode = randomizer_config.get("count_mode", "range")
count_fixed = int(randomizer_config.get("count_fixed", 5))
count_min = int(randomizer_config.get("count_min", 3))
count_max = int(randomizer_config.get("count_max", 7))
model_strength_min = float(randomizer_config.get("model_strength_min", 0.0))
model_strength_max = float(randomizer_config.get("model_strength_max", 1.0))
use_same_clip_strength = randomizer_config.get("use_same_clip_strength", True)
clip_strength_min = float(randomizer_config.get("clip_strength_min", 0.0))
clip_strength_max = float(randomizer_config.get("clip_strength_max", 1.0))
use_recommended_strength = randomizer_config.get(
"use_recommended_strength", False
)
recommended_strength_scale_min = float(
randomizer_config.get("recommended_strength_scale_min", 0.5)
)
recommended_strength_scale_max = float(
randomizer_config.get("recommended_strength_scale_max", 1.0)
)
# Extract locked LoRAs from input
locked_loras = [lora for lora in input_loras if lora.get("locked", False)]
# Use LoraService to generate random LoRAs
lora_service = LoraService(scanner)
result_loras = await lora_service.get_random_loras(
count=count_fixed,
model_strength_min=model_strength_min,
model_strength_max=model_strength_max,
use_same_clip_strength=use_same_clip_strength,
clip_strength_min=clip_strength_min,
clip_strength_max=clip_strength_max,
locked_loras=locked_loras,
pool_config=pool_config,
count_mode=count_mode,
count_min=count_min,
count_max=count_max,
use_recommended_strength=use_recommended_strength,
recommended_strength_scale_min=recommended_strength_scale_min,
recommended_strength_scale_max=recommended_strength_scale_max,
seed=seed,
)
return result_loras

View File

@@ -1,3 +1,4 @@
from comfy.comfy_types import IO # type: ignore
import os import os
from ..utils.utils import get_lora_info from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list from .utils import FlexibleOptionalInputType, any_type, extract_lora_name, get_loras_list
@@ -14,9 +15,8 @@ class LoraStacker:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"text": ("STRING", { "text": (IO.STRING, {
"multiline": True, "multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True, "dynamicPrompts": True,
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation", "tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
"placeholder": "LoRA syntax input: <lora:name:strength>" "placeholder": "LoRA syntax input: <lora:name:strength>"
@@ -25,7 +25,7 @@ class LoraStacker:
"optional": FlexibleOptionalInputType(any_type), "optional": FlexibleOptionalInputType(any_type),
} }
RETURN_TYPES = ("LORA_STACK", "STRING", "STRING") RETURN_TYPES = ("LORA_STACK", IO.STRING, IO.STRING)
RETURN_NAMES = ("LORA_STACK", "trigger_words", "active_loras") RETURN_NAMES = ("LORA_STACK", "trigger_words", "active_loras")
FUNCTION = "stack_loras" FUNCTION = "stack_loras"

View File

@@ -1,59 +0,0 @@
from typing import Any, Optional
class PromptLoraManager:
"""Encodes text (and optional trigger words) into CLIP conditioning."""
NAME = "Prompt (LoraManager)"
CATEGORY = "Lora Manager/conditioning"
DESCRIPTION = (
"Encodes a text prompt using a CLIP model into an embedding that can be used "
"to guide the diffusion model towards generating specific images."
)
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"text": (
'STRING',
{
"multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True,
"tooltip": "The text to be encoded.",
},
),
"clip": (
'CLIP',
{"tooltip": "The CLIP model used for encoding the text."},
),
},
"optional": {
"trigger_words": (
'STRING',
{
"forceInput": True,
"tooltip": (
"Optional trigger words to prepend to the text before "
"encoding."
)
},
)
},
}
RETURN_TYPES = ('CONDITIONING', 'STRING',)
RETURN_NAMES = ('CONDITIONING', 'PROMPT',)
OUTPUT_TOOLTIPS = (
"A conditioning containing the embedded text used to guide the diffusion model.",
)
FUNCTION = "encode"
def encode(self, text: str, clip: Any, trigger_words: Optional[str] = None):
prompt = text
if trigger_words:
prompt = ", ".join([trigger_words, text])
from nodes import CLIPTextEncode # type: ignore
conditioning = CLIPTextEncode().encode(clip, prompt)[0]
return (conditioning, prompt,)

View File

@@ -1,5 +1,6 @@
import json import json
import os import os
import asyncio
import re import re
import numpy as np import numpy as np
import folder_paths # type: ignore import folder_paths # type: ignore
@@ -9,7 +10,7 @@ from ..metadata_collector import get_metadata
from PIL import Image, PngImagePlugin from PIL import Image, PngImagePlugin
import piexif import piexif
class SaveImageLM: class SaveImage:
NAME = "Save Image (LoraManager)" NAME = "Save Image (LoraManager)"
CATEGORY = "Lora Manager/utils" CATEGORY = "Lora Manager/utils"
DESCRIPTION = "Save images with embedded generation metadata in compatible format" DESCRIPTION = "Save images with embedded generation metadata in compatible format"
@@ -273,15 +274,9 @@ class SaveImageLM:
length = int(parts[1]) length = int(parts[1])
prompt = prompt[:length] prompt = prompt[:length]
filename = filename.replace(segment, prompt.strip()) filename = filename.replace(segment, prompt.strip())
elif key == "model": elif key == "model" and 'checkpoint' in metadata_dict:
model_value = metadata_dict.get('checkpoint') model = metadata_dict.get('checkpoint', '')
if isinstance(model_value, (bytes, os.PathLike)): model = os.path.splitext(os.path.basename(model))[0]
model_value = str(model_value)
if not isinstance(model_value, str) or not model_value:
model = "model_unavailable"
else:
model = os.path.splitext(os.path.basename(model_value))[0]
if len(parts) >= 2: if len(parts) >= 2:
length = int(parts[1]) length = int(parts[1])
model = model[:length] model = model[:length]
@@ -424,15 +419,11 @@ class SaveImageLM:
# Make sure the output directory exists # Make sure the output directory exists
os.makedirs(self.output_dir, exist_ok=True) os.makedirs(self.output_dir, exist_ok=True)
# If images is already a list or array of images, do nothing; otherwise, convert to list # Ensure images is always a list of images
if isinstance(images, (list, np.ndarray)): if len(images.shape) == 3: # Single image (height, width, channels)
pass images = [images]
else: else: # Multiple images (batch, height, width, channels)
# Ensure images is always a list of images images = [img for img in images]
if len(images.shape) == 3: # Single image (height, width, channels)
images = [images]
else: # Multiple images (batch, height, width, channels)
images = [img for img in images]
# Save all images # Save all images
results = self.save_images( results = self.save_images(
@@ -448,4 +439,4 @@ class SaveImageLM:
add_counter_to_filename add_counter_to_filename
) )
return (images,) return (images,)

View File

@@ -1,5 +1,6 @@
import json import json
import re import re
from server import PromptServer # type: ignore
from .utils import FlexibleOptionalInputType, any_type from .utils import FlexibleOptionalInputType, any_type
import logging import logging
@@ -23,10 +24,6 @@ class TriggerWordToggle:
"default": True, "default": True,
"tooltip": "Sets the default initial state (active or inactive) when trigger words are added." "tooltip": "Sets the default initial state (active or inactive) when trigger words are added."
}), }),
"allow_strength_adjustment": ("BOOLEAN", {
"default": False,
"tooltip": "Enable mouse wheel adjustment of each trigger word's strength."
}),
}, },
"optional": FlexibleOptionalInputType(any_type), "optional": FlexibleOptionalInputType(any_type),
"hidden": { "hidden": {
@@ -51,14 +48,7 @@ class TriggerWordToggle:
else: else:
return data return data
def process_trigger_words( def process_trigger_words(self, id, group_mode, default_active, **kwargs):
self,
id,
group_mode,
default_active,
allow_strength_adjustment=False,
**kwargs,
):
# Handle both old and new formats for trigger_words # Handle both old and new formats for trigger_words
trigger_words_data = self._get_toggle_data(kwargs, 'orinalMessage') trigger_words_data = self._get_toggle_data(kwargs, 'orinalMessage')
trigger_words = trigger_words_data if isinstance(trigger_words_data, str) else "" trigger_words = trigger_words_data if isinstance(trigger_words_data, str) else ""
@@ -72,81 +62,36 @@ class TriggerWordToggle:
# Convert to list if it's a JSON string # Convert to list if it's a JSON string
if isinstance(trigger_data, str): if isinstance(trigger_data, str):
trigger_data = json.loads(trigger_data) trigger_data = json.loads(trigger_data)
if isinstance(trigger_data, list): # Create dictionaries to track active state of words or groups
if group_mode: active_state = {item['text']: item.get('active', False) for item in trigger_data}
if allow_strength_adjustment:
parsed_items = [ if group_mode:
self._parse_trigger_item(item, allow_strength_adjustment) # Split by two or more consecutive commas to get groups
for item in trigger_data groups = re.split(r',{2,}', trigger_words)
] # Remove leading/trailing whitespace from each group
filtered_groups = [ groups = [group.strip() for group in groups]
self._format_word_output(
item["text"], # Filter groups: keep those not in toggle_trigger_words or those that are active
item["strength"], filtered_groups = [group for group in groups if group not in active_state or active_state[group]]
allow_strength_adjustment,
) if filtered_groups:
for item in parsed_items filtered_triggers = ', '.join(filtered_groups)
if item["text"] and item["active"]
]
else:
filtered_groups = [
(item.get('text') or "").strip()
for item in trigger_data
if (item.get('text') or "").strip() and item.get('active', False)
]
filtered_triggers = ', '.join(filtered_groups) if filtered_groups else ""
else: else:
parsed_items = [ filtered_triggers = ""
self._parse_trigger_item(item, allow_strength_adjustment)
for item in trigger_data
]
filtered_words = [
self._format_word_output(
item["text"],
item["strength"],
allow_strength_adjustment,
)
for item in parsed_items
if item["text"] and item["active"]
]
filtered_triggers = ', '.join(filtered_words) if filtered_words else ""
else: else:
# Fallback to original message parsing if data is not in the expected list format # Original behavior for individual words mode
if group_mode: original_words = [word.strip() for word in trigger_words.split(',')]
groups = re.split(r',{2,}', trigger_words) # Filter out empty strings
groups = [group.strip() for group in groups if group.strip()] original_words = [word for word in original_words if word]
filtered_triggers = ', '.join(groups) filtered_words = [word for word in original_words if word not in active_state or active_state[word]]
if filtered_words:
filtered_triggers = ', '.join(filtered_words)
else: else:
words = [word.strip() for word in trigger_words.split(',') if word.strip()] filtered_triggers = ""
filtered_triggers = ', '.join(words)
except Exception as e: except Exception as e:
logger.error(f"Error processing trigger words: {e}") logger.error(f"Error processing trigger words: {e}")
return (filtered_triggers,) return (filtered_triggers,)
def _parse_trigger_item(self, item, allow_strength_adjustment):
text = (item.get('text') or "").strip()
active = bool(item.get('active', False))
strength = item.get('strength')
strength_match = re.match(r'^\((.+):([\d.]+)\)$', text)
if strength_match:
text = strength_match.group(1).strip()
if strength is None:
try:
strength = float(strength_match.group(2))
except ValueError:
strength = None
return {
"text": text,
"active": active,
"strength": strength if allow_strength_adjustment else None,
}
def _format_word_output(self, base_word, strength, allow_strength_adjustment):
if allow_strength_adjustment and strength is not None:
return f"({base_word}:{strength:.2f})"
return base_word

View File

@@ -36,7 +36,6 @@ any_type = AnyType("*")
import os import os
import logging import logging
import copy import copy
import sys
import folder_paths import folder_paths
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -99,38 +98,22 @@ def to_diffusers(input_lora):
def nunchaku_load_lora(model, lora_name, lora_strength): def nunchaku_load_lora(model, lora_name, lora_strength):
"""Load a Flux LoRA for Nunchaku model""" """Load a Flux LoRA for Nunchaku model"""
# Get full path to the LoRA file. Allow both direct paths and registered LoRA names.
lora_path = lora_name if os.path.isfile(lora_name) else folder_paths.get_full_path("loras", lora_name)
if not lora_path or not os.path.isfile(lora_path):
logger.warning("Skipping LoRA '%s' because it could not be found", lora_name)
return model
model_wrapper = model.model.diffusion_model model_wrapper = model.model.diffusion_model
transformer = model_wrapper.model
# Save the transformer temporarily
model_wrapper.model = None
ret_model = copy.deepcopy(model) # copy everything except the model
ret_model_wrapper = ret_model.model.diffusion_model
# Restore the model and set it for the copy
model_wrapper.model = transformer
ret_model_wrapper.model = transformer
# Get full path to the LoRA file
lora_path = folder_paths.get_full_path("loras", lora_name)
ret_model_wrapper.loras.append((lora_path, lora_strength))
# Try to find copy_with_ctx in the same module as ComfyFluxWrapper
module_name = model_wrapper.__class__.__module__
module = sys.modules.get(module_name)
copy_with_ctx = getattr(module, "copy_with_ctx", None)
if copy_with_ctx is not None:
# New logic using copy_with_ctx from ComfyUI-nunchaku 1.1.0+
ret_model_wrapper, ret_model = copy_with_ctx(model_wrapper)
ret_model_wrapper.loras = [*model_wrapper.loras, (lora_path, lora_strength)]
else:
# Fallback to legacy logic
logger.warning("Please upgrade ComfyUI-nunchaku to 1.1.0 or above for better LoRA support. Falling back to legacy loading logic.")
transformer = model_wrapper.model
# Save the transformer temporarily
model_wrapper.model = None
ret_model = copy.deepcopy(model) # copy everything except the model
ret_model_wrapper = ret_model.model.diffusion_model
# Restore the model and set it for the copy
model_wrapper.model = transformer
ret_model_wrapper.model = transformer
ret_model_wrapper.loras.append((lora_path, lora_strength))
# Convert the LoRA to diffusers format # Convert the LoRA to diffusers format
sd = to_diffusers(lora_path) sd = to_diffusers(lora_path)

View File

@@ -1,3 +1,4 @@
from comfy.comfy_types import IO # type: ignore
import folder_paths # type: ignore import folder_paths # type: ignore
from ..utils.utils import get_lora_info from ..utils.utils import get_lora_info
from .utils import FlexibleOptionalInputType, any_type, get_loras_list from .utils import FlexibleOptionalInputType, any_type, get_loras_list
@@ -5,7 +6,7 @@ import logging
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class WanVideoLoraSelectLM: class WanVideoLoraSelect:
NAME = "WanVideo Lora Select (LoraManager)" NAME = "WanVideo Lora Select (LoraManager)"
CATEGORY = "Lora Manager/stackers" CATEGORY = "Lora Manager/stackers"
@@ -13,11 +14,9 @@ class WanVideoLoraSelectLM:
def INPUT_TYPES(cls): def INPUT_TYPES(cls):
return { return {
"required": { "required": {
"low_mem_load": ("BOOLEAN", {"default": False, "tooltip": "Load LORA models with less VRAM usage, slower loading. This affects ALL LoRAs, not just the current ones. No effect if merge_loras is False"}), "low_mem_load": ("BOOLEAN", {"default": False, "tooltip": "Load the LORA model with less VRAM usage, slower loading"}),
"merge_loras": ("BOOLEAN", {"default": True, "tooltip": "Merge LoRAs into the model, otherwise they are loaded on the fly. Always disabled for GGUF and scaled fp8 models. This affects ALL LoRAs, not just the current one"}), "text": (IO.STRING, {
"text": ("STRING", {
"multiline": True, "multiline": True,
"pysssss.autocomplete": False,
"dynamicPrompts": True, "dynamicPrompts": True,
"tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation", "tooltip": "Format: <lora:lora_name:strength> separated by spaces or punctuation",
"placeholder": "LoRA syntax input: <lora:name:strength>" "placeholder": "LoRA syntax input: <lora:name:strength>"
@@ -26,11 +25,11 @@ class WanVideoLoraSelectLM:
"optional": FlexibleOptionalInputType(any_type), "optional": FlexibleOptionalInputType(any_type),
} }
RETURN_TYPES = ("WANVIDLORA", "STRING", "STRING") RETURN_TYPES = ("WANVIDLORA", IO.STRING, IO.STRING)
RETURN_NAMES = ("lora", "trigger_words", "active_loras") RETURN_NAMES = ("lora", "trigger_words", "active_loras")
FUNCTION = "process_loras" FUNCTION = "process_loras"
def process_loras(self, text, low_mem_load=False, merge_loras=True, **kwargs): def process_loras(self, text, low_mem_load=False, **kwargs):
loras_list = [] loras_list = []
all_trigger_words = [] all_trigger_words = []
active_loras = [] active_loras = []
@@ -39,9 +38,6 @@ class WanVideoLoraSelectLM:
prev_lora = kwargs.get('prev_lora', None) prev_lora = kwargs.get('prev_lora', None)
if prev_lora is not None: if prev_lora is not None:
loras_list.extend(prev_lora) loras_list.extend(prev_lora)
if not merge_loras:
low_mem_load = False # Unmerged LoRAs don't need low_mem_load
# Get blocks if available # Get blocks if available
blocks = kwargs.get('blocks', {}) blocks = kwargs.get('blocks', {})
@@ -69,7 +65,6 @@ class WanVideoLoraSelectLM:
"blocks": selected_blocks, "blocks": selected_blocks,
"layer_filter": layer_filter, "layer_filter": layer_filter,
"low_mem_load": low_mem_load, "low_mem_load": low_mem_load,
"merge_loras": merge_loras,
} }
# Add to list and collect active loras # Add to list and collect active loras

View File

@@ -1,125 +0,0 @@
import folder_paths # type: ignore
from ..utils.utils import get_lora_info
from .utils import any_type
import logging
# 初始化日志记录器
logger = logging.getLogger(__name__)
# 定义新节点的类
class WanVideoLoraSelectFromText:
# 节点在UI中显示的名称
NAME = "WanVideo Lora Select From Text (LoraManager)"
# 节点所属的分类
CATEGORY = "Lora Manager/stackers"
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"low_mem_load": ("BOOLEAN", {"default": False, "tooltip": "Load LORA models with less VRAM usage, slower loading. This affects ALL LoRAs, not just the current ones. No effect if merge_loras is False"}),
"merge_lora": ("BOOLEAN", {"default": True, "tooltip": "Merge LoRAs into the model, otherwise they are loaded on the fly. Always disabled for GGUF and scaled fp8 models. This affects ALL LoRAs, not just the current one"}),
"lora_syntax": ("STRING", {
"multiline": True,
"forceInput": True,
"tooltip": "Connect a TEXT output for LoRA syntax: <lora:name:strength>"
}),
},
"optional": {
"prev_lora": ("WANVIDLORA",),
"blocks": ("BLOCKS",)
}
}
RETURN_TYPES = ("WANVIDLORA", "STRING", "STRING")
RETURN_NAMES = ("lora", "trigger_words", "active_loras")
FUNCTION = "process_loras_from_syntax"
def process_loras_from_syntax(self, lora_syntax, low_mem_load=False, merge_lora=True, **kwargs):
text_to_process = lora_syntax
blocks = kwargs.get('blocks', {})
selected_blocks = blocks.get("selected_blocks", {})
layer_filter = blocks.get("layer_filter", "")
loras_list = []
all_trigger_words = []
active_loras = []
prev_lora = kwargs.get('prev_lora', None)
if prev_lora is not None:
loras_list.extend(prev_lora)
if not merge_lora:
low_mem_load = False
parts = text_to_process.split('<lora:')
for part in parts[1:]:
end_index = part.find('>')
if end_index == -1:
continue
content = part[:end_index]
lora_parts = content.split(':')
lora_name_raw = ""
model_strength = 1.0
clip_strength = 1.0
if len(lora_parts) == 2:
lora_name_raw = lora_parts[0].strip()
try:
model_strength = float(lora_parts[1])
clip_strength = model_strength
except (ValueError, IndexError):
logger.warning(f"Invalid strength for LoRA '{lora_name_raw}'. Skipping.")
continue
elif len(lora_parts) >= 3:
lora_name_raw = lora_parts[0].strip()
try:
model_strength = float(lora_parts[1])
clip_strength = float(lora_parts[2])
except (ValueError, IndexError):
logger.warning(f"Invalid strengths for LoRA '{lora_name_raw}'. Skipping.")
continue
else:
continue
lora_path, trigger_words = get_lora_info(lora_name_raw)
lora_item = {
"path": folder_paths.get_full_path("loras", lora_path),
"strength": model_strength,
"name": lora_path.split(".")[0],
"blocks": selected_blocks,
"layer_filter": layer_filter,
"low_mem_load": low_mem_load,
"merge_loras": merge_lora,
}
loras_list.append(lora_item)
active_loras.append((lora_name_raw, model_strength, clip_strength))
all_trigger_words.extend(trigger_words)
trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
formatted_loras = []
for name, model_strength, clip_strength in active_loras:
if abs(model_strength - clip_strength) > 0.001:
formatted_loras.append(f"<lora:{name}:{str(model_strength).strip()}:{str(clip_strength).strip()}>")
else:
formatted_loras.append(f"<lora:{name}:{str(model_strength).strip()}>")
active_loras_text = " ".join(formatted_loras)
return (loras_list, trigger_words_text, active_loras_text)
NODE_CLASS_MAPPINGS = {
"WanVideoLoraSelectFromText": WanVideoLoraSelectFromText
}
NODE_DISPLAY_NAME_MAPPINGS = {
"WanVideoLoraSelectFromText": "WanVideo Lora Select From Text (LoraManager)"
}

View File

@@ -8,7 +8,6 @@ from typing import Dict, List, Any, Optional, Tuple
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from ..config import config from ..config import config
from ..utils.constants import VALID_LORA_TYPES from ..utils.constants import VALID_LORA_TYPES
from ..utils.civitai_utils import rewrite_preview_url
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -37,8 +36,7 @@ class RecipeMetadataParser(ABC):
""" """
pass pass
@staticmethod async def populate_lora_from_civitai(self, lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any], Optional[str]],
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any], Optional[str]],
recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]: recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]:
""" """
Populate a lora entry with information from Civitai API response Populate a lora entry with information from Civitai API response
@@ -57,7 +55,7 @@ class RecipeMetadataParser(ABC):
# Unpack the tuple to get the actual data # Unpack the tuple to get the actual data
civitai_info, error_msg = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None) civitai_info, error_msg = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
if not civitai_info or error_msg == "Model not found": if not civitai_info or civitai_info.get("error") == "Model not found":
# Model not found or deleted # Model not found or deleted
lora_entry['isDeleted'] = True lora_entry['isDeleted'] = True
lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png' lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
@@ -80,7 +78,7 @@ class RecipeMetadataParser(ABC):
# Update model name if available # Update model name if available
if 'model' in civitai_info and 'name' in civitai_info['model']: if 'model' in civitai_info and 'name' in civitai_info['model']:
lora_entry['name'] = civitai_info['model']['name'] lora_entry['name'] = civitai_info['model']['name']
lora_entry['id'] = civitai_info.get('id') lora_entry['id'] = civitai_info.get('id')
lora_entry['modelId'] = civitai_info.get('modelId') lora_entry['modelId'] = civitai_info.get('modelId')
@@ -90,10 +88,7 @@ class RecipeMetadataParser(ABC):
# Get thumbnail URL from first image # Get thumbnail URL from first image
if 'images' in civitai_info and civitai_info['images']: if 'images' in civitai_info and civitai_info['images']:
image_url = civitai_info['images'][0].get('url') lora_entry['thumbnailUrl'] = civitai_info['images'][0].get('url', '')
if image_url:
rewritten_image_url, _ = rewrite_preview_url(image_url, media_type='image')
lora_entry['thumbnailUrl'] = rewritten_image_url or image_url
# Get base model # Get base model
current_base_model = civitai_info.get('baseModel', '') current_base_model = civitai_info.get('baseModel', '')
@@ -124,10 +119,10 @@ class RecipeMetadataParser(ABC):
# Check if exists locally # Check if exists locally
if recipe_scanner and lora_entry['hash']: if recipe_scanner and lora_entry['hash']:
lora_scanner = recipe_scanner._lora_scanner lora_scanner = recipe_scanner._lora_scanner
exists_locally = lora_scanner.has_hash(lora_entry['hash']) exists_locally = lora_scanner.has_lora_hash(lora_entry['hash'])
if exists_locally: if exists_locally:
try: try:
local_path = lora_scanner.get_path_by_hash(lora_entry['hash']) local_path = lora_scanner.get_lora_path_by_hash(lora_entry['hash'])
lora_entry['existsLocally'] = True lora_entry['existsLocally'] = True
lora_entry['localPath'] = local_path lora_entry['localPath'] = local_path
lora_entry['file_name'] = os.path.splitext(os.path.basename(local_path))[0] lora_entry['file_name'] = os.path.splitext(os.path.basename(local_path))[0]
@@ -149,68 +144,40 @@ class RecipeMetadataParser(ABC):
logger.error(f"Error populating lora from Civitai info: {e}") logger.error(f"Error populating lora from Civitai info: {e}")
return lora_entry return lora_entry
@staticmethod async def populate_checkpoint_from_civitai(self, checkpoint: Dict[str, Any], civitai_info: Dict[str, Any]) -> Dict[str, Any]:
async def populate_checkpoint_from_civitai(checkpoint: Dict[str, Any], civitai_info: Dict[str, Any]) -> Dict[str, Any]:
""" """
Populate checkpoint information from Civitai API response Populate checkpoint information from Civitai API response
Args: Args:
checkpoint: The checkpoint entry to populate checkpoint: The checkpoint entry to populate
civitai_info: The response from Civitai API or a (data, error_msg) tuple civitai_info: The response from Civitai API
Returns: Returns:
The populated checkpoint dict The populated checkpoint dict
""" """
try: try:
civitai_data, error_msg = ( if civitai_info and civitai_info.get("error") != "Model not found":
(civitai_info, None) # Update model name if available
if not isinstance(civitai_info, tuple) if 'model' in civitai_info and 'name' in civitai_info['model']:
else civitai_info checkpoint['name'] = civitai_info['model']['name']
)
# Update version if available
if not civitai_data or error_msg == "Model not found": if 'name' in civitai_info:
checkpoint['version'] = civitai_info.get('name', '')
# Get thumbnail URL from first image
if 'images' in civitai_info and civitai_info['images']:
checkpoint['thumbnailUrl'] = civitai_info['images'][0].get('url', '')
# Get base model
checkpoint['baseModel'] = civitai_info.get('baseModel', '')
# Get download URL
checkpoint['downloadUrl'] = civitai_info.get('downloadUrl', '')
else:
# Model not found or deleted
checkpoint['isDeleted'] = True checkpoint['isDeleted'] = True
return checkpoint
if 'model' in civitai_data and 'name' in civitai_data['model']:
checkpoint['name'] = civitai_data['model']['name']
if 'name' in civitai_data:
checkpoint['version'] = civitai_data.get('name', '')
if 'images' in civitai_data and civitai_data['images']:
image_url = civitai_data['images'][0].get('url')
if image_url:
rewritten_image_url, _ = rewrite_preview_url(image_url, media_type='image')
checkpoint['thumbnailUrl'] = rewritten_image_url or image_url
checkpoint['baseModel'] = civitai_data.get('baseModel', '')
checkpoint['downloadUrl'] = civitai_data.get('downloadUrl', '')
checkpoint['modelId'] = civitai_data.get('modelId', checkpoint.get('modelId', 0))
checkpoint['id'] = civitai_data.get('id', 0)
if 'files' in civitai_data:
model_file = next(
(
file
for file in civitai_data.get('files', [])
if file.get('type') == 'Model'
),
None,
)
if model_file:
checkpoint['size'] = model_file.get('sizeKB', 0) * 1024
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
checkpoint['hash'] = sha256.lower()
file_name = model_file.get('name', '')
if file_name:
checkpoint['file_name'] = os.path.splitext(file_name)[0]
except Exception as e: except Exception as e:
logger.error(f"Error populating checkpoint from Civitai info: {e}") logger.error(f"Error populating checkpoint from Civitai info: {e}")

View File

@@ -1,216 +0,0 @@
import logging
import json
import re
import os
from typing import Any, Dict, Optional
from .merger import GenParamsMerger
from .base import RecipeMetadataParser
from ..services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__)
class RecipeEnricher:
"""Service to enrich recipe metadata from multiple sources (Civitai, Embedded, User)."""
@staticmethod
async def enrich_recipe(
recipe: Dict[str, Any],
civitai_client: Any,
request_params: Optional[Dict[str, Any]] = None
) -> bool:
"""
Enrich a recipe dictionary in-place with metadata from Civitai and embedded params.
Args:
recipe: The recipe dictionary to enrich. Must have 'gen_params' initialized.
civitai_client: Authenticated Civitai client instance.
request_params: (Optional) Parameters from a user request (e.g. import).
Returns:
bool: True if the recipe was modified, False otherwise.
"""
updated = False
gen_params = recipe.get("gen_params", {})
# 1. Fetch Civitai Info if available
civitai_meta = None
model_version_id = None
source_url = recipe.get("source_url") or recipe.get("source_path", "")
# Check if it's a Civitai image URL
image_id_match = re.search(r'civitai\.com/images/(\d+)', str(source_url))
if image_id_match:
image_id = image_id_match.group(1)
try:
image_info = await civitai_client.get_image_info(image_id)
if image_info:
# Handle nested meta often found in Civitai API responses
raw_meta = image_info.get("meta")
if isinstance(raw_meta, dict):
if "meta" in raw_meta and isinstance(raw_meta["meta"], dict):
civitai_meta = raw_meta["meta"]
else:
civitai_meta = raw_meta
model_version_id = image_info.get("modelVersionId")
# If not at top level, check resources in meta
if not model_version_id and civitai_meta:
resources = civitai_meta.get("civitaiResources", [])
for res in resources:
if res.get("type") == "checkpoint":
model_version_id = res.get("modelVersionId")
break
except Exception as e:
logger.warning(f"Failed to fetch Civitai image info: {e}")
# 2. Merge Parameters
# Priority: request_params > civitai_meta > embedded (existing gen_params)
new_gen_params = GenParamsMerger.merge(
request_params=request_params,
civitai_meta=civitai_meta,
embedded_metadata=gen_params
)
if new_gen_params != gen_params:
recipe["gen_params"] = new_gen_params
updated = True
# 3. Checkpoint Enrichment
# If we have a checkpoint entry, or we can find one
# Use 'id' (from Civitai version) as a marker that it's been enriched
checkpoint_entry = recipe.get("checkpoint")
has_full_checkpoint = checkpoint_entry and checkpoint_entry.get("name") and checkpoint_entry.get("id")
if not has_full_checkpoint:
# Helper to look up values in priority order
def start_lookup(keys):
for source in [request_params, civitai_meta, gen_params]:
if source:
if isinstance(keys, list):
for k in keys:
if k in source: return source[k]
else:
if keys in source: return source[keys]
return None
target_version_id = model_version_id or start_lookup("modelVersionId")
# Also check existing checkpoint entry
if not target_version_id and checkpoint_entry:
target_version_id = checkpoint_entry.get("modelVersionId") or checkpoint_entry.get("id")
# Check for version ID in resources (which might be a string in gen_params)
if not target_version_id:
# Look in all sources for "Civitai resources"
resources_val = start_lookup(["Civitai resources", "civitai_resources", "resources"])
if resources_val:
target_version_id = RecipeEnricher._extract_version_id_from_resources({"Civitai resources": resources_val})
target_hash = start_lookup(["Model hash", "checkpoint_hash", "hashes"])
if not target_hash and checkpoint_entry:
target_hash = checkpoint_entry.get("hash") or checkpoint_entry.get("model_hash")
# Look for 'Model' which sometimes is the hash or name
model_val = start_lookup("Model")
# Look for Checkpoint name fallback
checkpoint_val = checkpoint_entry.get("name") if checkpoint_entry else None
if not checkpoint_val:
checkpoint_val = start_lookup(["Checkpoint", "checkpoint"])
checkpoint_updated = await RecipeEnricher._resolve_and_populate_checkpoint(
recipe, target_version_id, target_hash, model_val, checkpoint_val
)
if checkpoint_updated:
updated = True
else:
# Checkpoint exists, no need to sync to gen_params anymore.
pass
# base_model resolution moved to _resolve_and_populate_checkpoint to support strict formatting
return updated
@staticmethod
def _extract_version_id_from_resources(gen_params: Dict[str, Any]) -> Optional[Any]:
"""Try to find modelVersionId in Civitai resources parameter."""
civitai_resources_raw = gen_params.get("Civitai resources")
if not civitai_resources_raw:
return None
resources_list = None
if isinstance(civitai_resources_raw, str):
try:
resources_list = json.loads(civitai_resources_raw)
except Exception:
pass
elif isinstance(civitai_resources_raw, list):
resources_list = civitai_resources_raw
if isinstance(resources_list, list):
for res in resources_list:
if res.get("type") == "checkpoint":
return res.get("modelVersionId")
return None
@staticmethod
async def _resolve_and_populate_checkpoint(
recipe: Dict[str, Any],
target_version_id: Optional[Any],
target_hash: Optional[str],
model_val: Optional[str],
checkpoint_val: Optional[str]
) -> bool:
"""Find checkpoint metadata and populate it in the recipe."""
metadata_provider = await get_default_metadata_provider()
civitai_info = None
if target_version_id:
civitai_info = await metadata_provider.get_model_version_info(str(target_version_id))
elif target_hash:
civitai_info = await metadata_provider.get_model_by_hash(target_hash)
else:
# Look for 'Model' which sometimes is the hash or name
if model_val and len(model_val) == 10: # Likely a short hash
civitai_info = await metadata_provider.get_model_by_hash(model_val)
if civitai_info and not (isinstance(civitai_info, tuple) and civitai_info[1] == "Model not found"):
# If we already have a partial checkpoint, use it as base
existing_cp = recipe.get("checkpoint")
if existing_cp is None:
existing_cp = {}
checkpoint_data = await RecipeMetadataParser.populate_checkpoint_from_civitai(existing_cp, civitai_info)
# 1. First, resolve base_model using full data before we format it away
current_base_model = recipe.get("base_model")
resolved_base_model = checkpoint_data.get("baseModel")
if resolved_base_model:
# Update if empty OR if it matches our generic prefix but is less specific
is_generic = not current_base_model or current_base_model.lower() in ["flux", "sdxl", "sd15"]
if is_generic and resolved_base_model != current_base_model:
recipe["base_model"] = resolved_base_model
# 2. Format according to requirements: type, modelId, modelVersionId, modelName, modelVersionName
formatted_checkpoint = {
"type": "checkpoint",
"modelId": checkpoint_data.get("modelId"),
"modelVersionId": checkpoint_data.get("id") or checkpoint_data.get("modelVersionId"),
"modelName": checkpoint_data.get("name"), # In base.py, 'name' is populated from civitai_data['model']['name']
"modelVersionName": checkpoint_data.get("version") # In base.py, 'version' is populated from civitai_data['name']
}
# Remove None values
recipe["checkpoint"] = {k: v for k, v in formatted_checkpoint.items() if v is not None}
return True
else:
# Fallback to name extraction if we don't already have one
existing_cp = recipe.get("checkpoint")
if not existing_cp or not existing_cp.get("modelName"):
cp_name = checkpoint_val
if cp_name:
recipe["checkpoint"] = {
"type": "checkpoint",
"modelName": cp_name
}
return True
return False

View File

@@ -1,98 +0,0 @@
from typing import Any, Dict, Optional
import logging
logger = logging.getLogger(__name__)
class GenParamsMerger:
"""Utility to merge generation parameters from multiple sources with priority."""
BLACKLISTED_KEYS = {
"id", "url", "userId", "username", "createdAt", "updatedAt", "hash", "meta",
"draft", "extra", "width", "height", "process", "quantity", "workflow",
"baseModel", "resources", "disablePoi", "aspectRatio", "Created Date",
"experimental", "civitaiResources", "civitai_resources", "Civitai resources",
"modelVersionId", "modelId", "hashes", "Model", "Model hash", "checkpoint_hash",
"checkpoint", "checksum", "model_checksum"
}
NORMALIZATION_MAPPING = {
# Civitai specific
"cfgScale": "cfg_scale",
"clipSkip": "clip_skip",
"negativePrompt": "negative_prompt",
# Case variations
"Sampler": "sampler",
"Steps": "steps",
"Seed": "seed",
"Size": "size",
"Prompt": "prompt",
"Negative prompt": "negative_prompt",
"Cfg scale": "cfg_scale",
"Clip skip": "clip_skip",
"Denoising strength": "denoising_strength",
}
@staticmethod
def merge(
request_params: Optional[Dict[str, Any]] = None,
civitai_meta: Optional[Dict[str, Any]] = None,
embedded_metadata: Optional[Dict[str, Any]] = None
) -> Dict[str, Any]:
"""
Merge generation parameters from three sources.
Priority: request_params > civitai_meta > embedded_metadata
Args:
request_params: Params provided directly in the import request
civitai_meta: Params from Civitai Image API 'meta' field
embedded_metadata: Params extracted from image EXIF/embedded metadata
Returns:
Merged parameters dictionary
"""
result = {}
# 1. Start with embedded metadata (lowest priority)
if embedded_metadata:
# If it's a full recipe metadata, we use its gen_params
if "gen_params" in embedded_metadata and isinstance(embedded_metadata["gen_params"], dict):
GenParamsMerger._update_normalized(result, embedded_metadata["gen_params"])
else:
# Otherwise assume the dict itself contains gen_params
GenParamsMerger._update_normalized(result, embedded_metadata)
# 2. Layer Civitai meta (medium priority)
if civitai_meta:
GenParamsMerger._update_normalized(result, civitai_meta)
# 3. Layer request params (highest priority)
if request_params:
GenParamsMerger._update_normalized(result, request_params)
# Filter out blacklisted keys and also the original camelCase keys if they were normalized
final_result = {}
for k, v in result.items():
if k in GenParamsMerger.BLACKLISTED_KEYS:
continue
if k in GenParamsMerger.NORMALIZATION_MAPPING:
continue
final_result[k] = v
return final_result
@staticmethod
def _update_normalized(target: Dict[str, Any], source: Dict[str, Any]) -> None:
"""Update target dict with normalized keys from source."""
for k, v in source.items():
normalized_key = GenParamsMerger.NORMALIZATION_MAPPING.get(k, k)
target[normalized_key] = v
# Also keep the original key for now if it's not the same,
# so we can filter at the end or avoid losing it if it wasn't supposed to be renamed?
# Actually, if we rename it, we should probably NOT keep both in 'target'
# because we want to filter them out at the end anyway.
if normalized_key != k:
# If we are overwriting an existing snake_case key with a camelCase one's value,
# that's fine because of the priority order of calls to _update_normalized.
pass
target[k] = v

View File

@@ -1,13 +1,11 @@
"""Parser for Automatic1111 metadata format.""" """Parser for Automatic1111 metadata format."""
import re import re
import os
import json import json
import logging import logging
from typing import Dict, Any from typing import Dict, Any
from ..base import RecipeMetadataParser from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -23,7 +21,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
CIVITAI_METADATA_REGEX = r', Civitai metadata:\s*(\{.*?\})' CIVITAI_METADATA_REGEX = r', Civitai metadata:\s*(\{.*?\})'
EXTRANETS_REGEX = r'<(lora|hypernet):([^:]+):(-?[0-9.]+)>' EXTRANETS_REGEX = r'<(lora|hypernet):([^:]+):(-?[0-9.]+)>'
MODEL_HASH_PATTERN = r'Model hash: ([a-zA-Z0-9]+)' MODEL_HASH_PATTERN = r'Model hash: ([a-zA-Z0-9]+)'
MODEL_NAME_PATTERN = r'Model: ([^,]+)'
VAE_HASH_PATTERN = r'VAE hash: ([a-zA-Z0-9]+)' VAE_HASH_PATTERN = r'VAE hash: ([a-zA-Z0-9]+)'
def is_metadata_matching(self, user_comment: str) -> bool: def is_metadata_matching(self, user_comment: str) -> bool:
@@ -33,9 +30,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]: async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Automatic1111 format""" """Parse metadata from Automatic1111 format"""
try: try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Split on Negative prompt if it exists # Split on Negative prompt if it exists
if "Negative prompt:" in user_comment: if "Negative prompt:" in user_comment:
parts = user_comment.split('Negative prompt:', 1) parts = user_comment.split('Negative prompt:', 1)
@@ -117,12 +111,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
except json.JSONDecodeError: except json.JSONDecodeError:
logger.error("Error parsing hashes JSON") logger.error("Error parsing hashes JSON")
# Pick up model hash from parsed hashes if available
if "hashes" in metadata and not metadata.get("model_hash"):
model_hash_from_hashes = metadata["hashes"].get("model")
if model_hash_from_hashes:
metadata["model_hash"] = model_hash_from_hashes
# Extract Lora hashes in alternative format # Extract Lora hashes in alternative format
lora_hashes_match = re.search(self.LORA_HASHES_REGEX, params_section) lora_hashes_match = re.search(self.LORA_HASHES_REGEX, params_section)
if not hashes_match and lora_hashes_match: if not hashes_match and lora_hashes_match:
@@ -145,17 +133,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
params_section = params_section.replace(lora_hashes_match.group(0), '') params_section = params_section.replace(lora_hashes_match.group(0), '')
except Exception as e: except Exception as e:
logger.error(f"Error parsing Lora hashes: {e}") logger.error(f"Error parsing Lora hashes: {e}")
# Extract checkpoint model hash/name when provided outside Civitai resources
model_hash_match = re.search(self.MODEL_HASH_PATTERN, params_section)
if model_hash_match:
metadata["model_hash"] = model_hash_match.group(1).strip()
params_section = params_section.replace(model_hash_match.group(0), '')
model_name_match = re.search(self.MODEL_NAME_PATTERN, params_section)
if model_name_match:
metadata["model_name"] = model_name_match.group(1).strip()
params_section = params_section.replace(model_name_match.group(0), '')
# Extract basic parameters # Extract basic parameters
param_pattern = r'([A-Za-z\s]+): ([^,]+)' param_pattern = r'([A-Za-z\s]+): ([^,]+)'
@@ -197,82 +174,20 @@ class AutomaticMetadataParser(RecipeMetadataParser):
metadata["gen_params"] = gen_params metadata["gen_params"] = gen_params
# Extract LoRA and checkpoint information # Extract LoRA information
loras = [] loras = []
base_model_counts = {} base_model_counts = {}
checkpoint = None
# First use Civitai resources if available (more reliable source) # First use Civitai resources if available (more reliable source)
if metadata.get("civitai_resources"): if metadata.get("civitai_resources"):
for resource in metadata.get("civitai_resources", []): for resource in metadata.get("civitai_resources", []):
# --- Added: Parse 'air' field if present ---
air = resource.get("air")
if air:
# Format: urn:air:sdxl:lora:civitai:1221007@1375651
# Or: urn:air:sdxl:checkpoint:civitai:623891@2019115
air_pattern = r"urn:air:[^:]+:(?P<type>[^:]+):civitai:(?P<modelId>\d+)@(?P<modelVersionId>\d+)"
air_match = re.match(air_pattern, air)
if air_match:
air_type = air_match.group("type")
air_modelId = int(air_match.group("modelId"))
air_modelVersionId = int(air_match.group("modelVersionId"))
# checkpoint/lycoris/lora/hypernet
resource["type"] = air_type
resource["modelId"] = air_modelId
resource["modelVersionId"] = air_modelVersionId
# --- End added ---
if resource.get("type") == "checkpoint" and resource.get("modelVersionId"):
version_id = resource.get("modelVersionId")
version_id_str = str(version_id)
checkpoint_entry = {
'id': version_id,
'modelId': resource.get("modelId", 0),
'name': resource.get("modelName", "Unknown Checkpoint"),
'version': resource.get("modelVersionName", resource.get("versionName", "")),
'type': resource.get("type", "checkpoint"),
'existsLocally': False,
'localPath': None,
'file_name': resource.get("modelName", ""),
'hash': resource.get("hash", "") or "",
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = await metadata_provider.get_model_version_info(version_id_str)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(
"Error fetching Civitai info for checkpoint version %s: %s",
version_id,
e,
)
# Prefer the first checkpoint found
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
if checkpoint is None:
checkpoint = checkpoint_entry
continue
if resource.get("type") in ["lora", "lycoris", "hypernet"] and resource.get("modelVersionId"): if resource.get("type") in ["lora", "lycoris", "hypernet"] and resource.get("modelVersionId"):
# Initialize lora entry # Initialize lora entry
lora_entry = { lora_entry = {
'id': resource.get("modelVersionId", 0), 'id': resource.get("modelVersionId", 0),
'modelId': resource.get("modelId", 0), 'modelId': resource.get("modelId", 0),
'name': resource.get("modelName", "Unknown LoRA"), 'name': resource.get("modelName", "Unknown LoRA"),
'version': resource.get("modelVersionName", resource.get("versionName", "")), 'version': resource.get("modelVersionName", ""),
'type': resource.get("type", "lora"), 'type': resource.get("type", "lora"),
'weight': round(float(resource.get("weight", 1.0)), 2), 'weight': round(float(resource.get("weight", 1.0)), 2),
'existsLocally': False, 'existsLocally': False,
@@ -284,9 +199,9 @@ class AutomaticMetadataParser(RecipeMetadataParser):
} }
# Get additional info from Civitai # Get additional info from Civitai
if metadata_provider: if civitai_client:
try: try:
civitai_info = await metadata_provider.get_model_version_info(resource.get("modelVersionId")) civitai_info = await civitai_client.get_model_version_info(resource.get("modelVersionId"))
populated_entry = await self.populate_lora_from_civitai( populated_entry = await self.populate_lora_from_civitai(
lora_entry, lora_entry,
civitai_info, civitai_info,
@@ -301,52 +216,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
loras.append(lora_entry) loras.append(lora_entry)
# Fallback checkpoint parsing from generic "Model" and "Model hash" fields
if checkpoint is None:
model_hash = metadata.get("model_hash")
if not model_hash and metadata.get("hashes"):
model_hash = metadata["hashes"].get("model")
model_name = metadata.get("model_name")
file_name = ""
if model_name:
cleaned_name = re.split(r"[\\\\/]", model_name)[-1]
file_name = os.path.splitext(cleaned_name)[0]
if model_hash or model_name:
checkpoint_entry = {
'id': 0,
'modelId': 0,
'name': model_name or "Unknown Checkpoint",
'version': '',
'type': 'checkpoint',
'hash': model_hash or "",
'existsLocally': False,
'localPath': None,
'file_name': file_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider and model_hash:
try:
civitai_info = await metadata_provider.get_model_by_hash(model_hash)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint hash {model_hash}: {e}")
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
checkpoint = checkpoint_entry
# If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"] # If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"]
if not loras or len(loras) == 0: if not loras or len(loras) == 0:
# Extract lora weights from extranet tags in prompt (for later use) # Extract lora weights from extranet tags in prompt (for later use)
@@ -385,11 +254,11 @@ class AutomaticMetadataParser(RecipeMetadataParser):
} }
# Try to get info from Civitai # Try to get info from Civitai
if metadata_provider: if civitai_client:
try: try:
if lora_hash: if lora_hash:
# If we have hash, use it for lookup # If we have hash, use it for lookup
civitai_info = await metadata_provider.get_model_by_hash(lora_hash) civitai_info = await civitai_client.get_model_by_hash(lora_hash)
else: else:
civitai_info = None civitai_info = None
@@ -410,9 +279,7 @@ class AutomaticMetadataParser(RecipeMetadataParser):
# Try to get base model from resources or make educated guess # Try to get base model from resources or make educated guess
base_model = None base_model = None
if checkpoint and checkpoint.get("baseModel"): if base_model_counts:
base_model = checkpoint.get("baseModel")
elif base_model_counts:
# Use the most common base model from the loras # Use the most common base model from the loras
base_model = max(base_model_counts.items(), key=lambda x: x[1])[0] base_model = max(base_model_counts.items(), key=lambda x: x[1])[0]
@@ -429,10 +296,6 @@ class AutomaticMetadataParser(RecipeMetadataParser):
'gen_params': filtered_gen_params, 'gen_params': filtered_gen_params,
'from_automatic_metadata': True 'from_automatic_metadata': True
} }
if checkpoint:
result['checkpoint'] = checkpoint
result['model'] = checkpoint
return result return result

View File

@@ -5,7 +5,6 @@ import logging
from typing import Dict, Any, Union from typing import Dict, Any, Union
from ..base import RecipeMetadataParser from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -23,48 +22,13 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
""" """
if not metadata or not isinstance(metadata, dict): if not metadata or not isinstance(metadata, dict):
return False return False
def has_markers(payload: Dict[str, Any]) -> bool: # Check for key markers specific to Civitai image metadata
# Check for common CivitAI image metadata fields return any([
civitai_image_fields = ( "resources" in metadata,
"resources", "civitaiResources" in metadata,
"civitaiResources", "additionalResources" in metadata
"additionalResources", ])
"hashes",
"prompt",
"negativePrompt",
"steps",
"sampler",
"cfgScale",
"seed",
"width",
"height",
"Model",
"Model hash"
)
return any(key in payload for key in civitai_image_fields)
# Check the main metadata object
if has_markers(metadata):
return True
# Check for LoRA hash patterns
hashes = metadata.get("hashes")
if isinstance(hashes, dict) and any(str(key).lower().startswith("lora:") for key in hashes):
return True
# Check nested meta object (common in CivitAI image responses)
nested_meta = metadata.get("meta")
if isinstance(nested_meta, dict):
if has_markers(nested_meta):
return True
# Also check for LoRA hash patterns in nested meta
hashes = nested_meta.get("hashes")
if isinstance(hashes, dict) and any(str(key).lower().startswith("lora:") for key in hashes):
return True
return False
async def parse_metadata(self, metadata, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]: async def parse_metadata(self, metadata, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Civitai image format """Parse metadata from Civitai image format
@@ -72,40 +36,16 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
Args: Args:
metadata: The metadata from the image (dict) metadata: The metadata from the image (dict)
recipe_scanner: Optional recipe scanner service recipe_scanner: Optional recipe scanner service
civitai_client: Optional Civitai API client (deprecated, use metadata_provider instead) civitai_client: Optional Civitai API client
Returns: Returns:
Dict containing parsed recipe data Dict containing parsed recipe data
""" """
try: try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Civitai image responses may wrap the actual metadata inside a "meta" key
if (
isinstance(metadata, dict)
and "meta" in metadata
and isinstance(metadata["meta"], dict)
):
inner_meta = metadata["meta"]
if any(
key in inner_meta
for key in (
"resources",
"civitaiResources",
"additionalResources",
"hashes",
"prompt",
"negativePrompt",
)
):
metadata = inner_meta
# Initialize result structure # Initialize result structure
result = { result = {
'base_model': None, 'base_model': None,
'loras': [], 'loras': [],
'model': None,
'gen_params': {}, 'gen_params': {},
'from_civitai_image': True 'from_civitai_image': True
} }
@@ -113,15 +53,6 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Track already added LoRAs to prevent duplicates # Track already added LoRAs to prevent duplicates
added_loras = {} # key: model_version_id or hash, value: index in result["loras"] added_loras = {} # key: model_version_id or hash, value: index in result["loras"]
# Extract hash information from hashes field for LoRA matching
lora_hashes = {}
if "hashes" in metadata and isinstance(metadata["hashes"], dict):
for key, hash_value in metadata["hashes"].items():
key_str = str(key)
if key_str.lower().startswith("lora:"):
lora_name = key_str.split(":", 1)[1]
lora_hashes[lora_name] = hash_value
# Extract prompt and negative prompt # Extract prompt and negative prompt
if "prompt" in metadata: if "prompt" in metadata:
result["gen_params"]["prompt"] = metadata["prompt"] result["gen_params"]["prompt"] = metadata["prompt"]
@@ -146,9 +77,9 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Extract base model information - directly if available # Extract base model information - directly if available
if "baseModel" in metadata: if "baseModel" in metadata:
result["base_model"] = metadata["baseModel"] result["base_model"] = metadata["baseModel"]
elif "Model hash" in metadata and metadata_provider: elif "Model hash" in metadata and civitai_client:
model_hash = metadata["Model hash"] model_hash = metadata["Model hash"]
model_info, error = await metadata_provider.get_model_by_hash(model_hash) model_info = await civitai_client.get_model_by_hash(model_hash)
if model_info: if model_info:
result["base_model"] = model_info.get("baseModel", "") result["base_model"] = model_info.get("baseModel", "")
elif "Model" in metadata and isinstance(metadata.get("resources"), list): elif "Model" in metadata and isinstance(metadata.get("resources"), list):
@@ -156,8 +87,8 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
for resource in metadata.get("resources", []): for resource in metadata.get("resources", []):
if resource.get("type") == "model" and resource.get("name") == metadata.get("Model"): if resource.get("type") == "model" and resource.get("name") == metadata.get("Model"):
# This is likely the checkpoint model # This is likely the checkpoint model
if metadata_provider and resource.get("hash"): if civitai_client and resource.get("hash"):
model_info, error = await metadata_provider.get_model_by_hash(resource.get("hash")) model_info = await civitai_client.get_model_by_hash(resource.get("hash"))
if model_info: if model_info:
result["base_model"] = model_info.get("baseModel", "") result["base_model"] = model_info.get("baseModel", "")
@@ -170,15 +101,6 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
if resource.get("type", "lora") == "lora": if resource.get("type", "lora") == "lora":
lora_hash = resource.get("hash", "") lora_hash = resource.get("hash", "")
# Try to get hash from the hashes field if not present in resource
if not lora_hash and resource.get("name"):
lora_hash = lora_hashes.get(resource["name"], "")
# Skip LoRAs without proper identification (hash or modelVersionId)
if not lora_hash and not resource.get("modelVersionId"):
logger.debug(f"Skipping LoRA resource '{resource.get('name', 'Unknown')}' - no hash or modelVersionId")
continue
# Skip if we've already added this LoRA by hash # Skip if we've already added this LoRA by hash
if lora_hash and lora_hash in added_loras: if lora_hash and lora_hash in added_loras:
continue continue
@@ -199,9 +121,9 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
} }
# Try to get info from Civitai if hash is available # Try to get info from Civitai if hash is available
if lora_entry['hash'] and metadata_provider: if lora_entry['hash'] and civitai_client:
try: try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash) civitai_info = await civitai_client.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai( populated_entry = await self.populate_lora_from_civitai(
lora_entry, lora_entry,
@@ -231,48 +153,17 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
# Process civitaiResources array # Process civitaiResources array
if "civitaiResources" in metadata and isinstance(metadata["civitaiResources"], list): if "civitaiResources" in metadata and isinstance(metadata["civitaiResources"], list):
for resource in metadata["civitaiResources"]: for resource in metadata["civitaiResources"]:
# Get resource type and identifier # Skip resources that aren't LoRAs or LyCORIS
resource_type = str(resource.get("type") or "").lower() if resource.get("type") not in ["lora", "lycoris"] and "type" not in resource:
version_id = str(resource.get("modelVersionId", ""))
if resource_type == "checkpoint":
checkpoint_entry = {
'id': resource.get("modelVersionId", 0),
'modelId': resource.get("modelId", 0),
'name': resource.get("modelName", "Unknown Checkpoint"),
'version': resource.get("modelVersionName", ""),
'type': resource.get("type", "checkpoint"),
'existsLocally': False,
'localPath': None,
'file_name': resource.get("modelName", ""),
'hash': resource.get("hash", "") or "",
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if version_id and metadata_provider:
try:
civitai_info = await metadata_provider.get_model_version_info(version_id)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint version {version_id}: {e}")
if result["model"] is None:
result["model"] = checkpoint_entry
continue continue
# Get unique identifier for deduplication
version_id = str(resource.get("modelVersionId", ""))
# Skip if we've already added this LoRA # Skip if we've already added this LoRA
if version_id and version_id in added_loras: if version_id and version_id in added_loras:
continue continue
# Initialize lora entry # Initialize lora entry
lora_entry = { lora_entry = {
'id': resource.get("modelVersionId", 0), 'id': resource.get("modelVersionId", 0),
@@ -288,31 +179,35 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
'downloadUrl': '', 'downloadUrl': '',
'isDeleted': False 'isDeleted': False
} }
# Try to get info from Civitai if modelVersionId is available # Try to get info from Civitai if modelVersionId is available
if version_id and metadata_provider: if version_id and civitai_client:
try: try:
# Use get_model_version_info instead of get_model_version # Use get_model_version_info instead of get_model_version
civitai_info = await metadata_provider.get_model_version_info(version_id) civitai_info, error = await civitai_client.get_model_version_info(version_id)
if error:
logger.warning(f"Error getting model version info: {error}")
continue
populated_entry = await self.populate_lora_from_civitai( populated_entry = await self.populate_lora_from_civitai(
lora_entry, lora_entry,
civitai_info, civitai_info,
recipe_scanner, recipe_scanner,
base_model_counts base_model_counts
) )
if populated_entry is None: if populated_entry is None:
continue # Skip invalid LoRA types continue # Skip invalid LoRA types
lora_entry = populated_entry lora_entry = populated_entry
except Exception as e: except Exception as e:
logger.error(f"Error fetching Civitai info for model version {version_id}: {e}") logger.error(f"Error fetching Civitai info for model version {version_id}: {e}")
# Track this LoRA in our deduplication dict # Track this LoRA in our deduplication dict
if version_id: if version_id:
added_loras[version_id] = len(result["loras"]) added_loras[version_id] = len(result["loras"])
result["loras"].append(lora_entry) result["loras"].append(lora_entry)
# Process additionalResources array # Process additionalResources array
@@ -351,143 +246,34 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
'isDeleted': False 'isDeleted': False
} }
# If we have a version ID and metadata provider, try to get more info # If we have a version ID and civitai client, try to get more info
if version_id and metadata_provider: if version_id and civitai_client:
try: try:
# Use get_model_version_info with the version ID # Use get_model_version_info with the version ID
civitai_info = await metadata_provider.get_model_version_info(version_id) civitai_info, error = await civitai_client.get_model_version_info(version_id)
populated_entry = await self.populate_lora_from_civitai( if error:
lora_entry, logger.warning(f"Error getting model version info: {error}")
civitai_info, else:
recipe_scanner, populated_entry = await self.populate_lora_from_civitai(
base_model_counts lora_entry,
) civitai_info,
recipe_scanner,
if populated_entry is None: base_model_counts
continue # Skip invalid LoRA types )
lora_entry = populated_entry if populated_entry is None:
continue # Skip invalid LoRA types
# Track this LoRA for deduplication
if version_id: lora_entry = populated_entry
added_loras[version_id] = len(result["loras"])
# Track this LoRA for deduplication
if version_id:
added_loras[version_id] = len(result["loras"])
except Exception as e: except Exception as e:
logger.error(f"Error fetching Civitai info for model ID {version_id}: {e}") logger.error(f"Error fetching Civitai info for model ID {version_id}: {e}")
result["loras"].append(lora_entry) result["loras"].append(lora_entry)
# If we found LoRA hashes in the metadata but haven't already
# populated entries for them, fall back to creating LoRAs from
# the hashes section. Some Civitai image responses only include
# LoRA information here without explicit resources entries.
for lora_name, lora_hash in lora_hashes.items():
if not lora_hash:
continue
# Skip LoRAs we've already added via resources or other fields
if lora_hash in added_loras:
continue
lora_entry = {
'name': lora_name,
'type': "lora",
'weight': 1.0,
'hash': lora_hash,
'existsLocally': False,
'localPath': None,
'file_name': lora_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash
)
if populated_entry is None:
continue
lora_entry = populated_entry
if 'id' in lora_entry and lora_entry['id']:
added_loras[str(lora_entry['id'])] = len(result["loras"])
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA hash {lora_hash}: {e}")
added_loras[lora_hash] = len(result["loras"])
result["loras"].append(lora_entry)
# Check for LoRA info in the format "Lora_0 Model hash", "Lora_0 Model name", etc.
lora_index = 0
while f"Lora_{lora_index} Model hash" in metadata and f"Lora_{lora_index} Model name" in metadata:
lora_hash = metadata[f"Lora_{lora_index} Model hash"]
lora_name = metadata[f"Lora_{lora_index} Model name"]
lora_strength_model = float(metadata.get(f"Lora_{lora_index} Strength model", 1.0))
# Skip if we've already added this LoRA by hash
if lora_hash and lora_hash in added_loras:
lora_index += 1
continue
lora_entry = {
'name': lora_name,
'type': "lora",
'weight': lora_strength_model,
'hash': lora_hash,
'existsLocally': False,
'localPath': None,
'file_name': lora_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
# Try to get info from Civitai if hash is available
if lora_entry['hash'] and metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash
)
if populated_entry is None:
lora_index += 1
continue # Skip invalid LoRA types
lora_entry = populated_entry
# If we have a version ID from Civitai, track it for deduplication
if 'id' in lora_entry and lora_entry['id']:
added_loras[str(lora_entry['id'])] = len(result["loras"])
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA hash {lora_entry['hash']}: {e}")
# Track by hash if we have it
if lora_hash:
added_loras[lora_hash] = len(result["loras"])
result["loras"].append(lora_entry)
lora_index += 1
# If base model wasn't found earlier, use the most common one from LoRAs # If base model wasn't found earlier, use the most common one from LoRAs
if not result["base_model"] and base_model_counts: if not result["base_model"] and base_model_counts:

View File

@@ -6,7 +6,6 @@ import logging
from typing import Dict, Any from typing import Dict, Any
from ..base import RecipeMetadataParser from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -27,15 +26,15 @@ class ComfyMetadataParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]: async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from Civitai ComfyUI metadata format""" """Parse metadata from Civitai ComfyUI metadata format"""
try: try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
data = json.loads(user_comment) data = json.loads(user_comment)
loras = [] loras = []
# Find all LoraLoader nodes # Find all LoraLoader nodes
lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'} lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'}
if not lora_nodes:
return {"error": "No LoRA information found in this ComfyUI workflow", "loras": []}
# Process each LoraLoader node # Process each LoraLoader node
for node_id, node in lora_nodes.items(): for node_id, node in lora_nodes.items():
if 'inputs' not in node or 'lora_name' not in node['inputs']: if 'inputs' not in node or 'lora_name' not in node['inputs']:
@@ -74,10 +73,10 @@ class ComfyMetadataParser(RecipeMetadataParser):
'isDeleted': False 'isDeleted': False
} }
# Get additional info from Civitai if metadata provider is available # Get additional info from Civitai if client is available
if metadata_provider: if civitai_client:
try: try:
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id) civitai_info_tuple = await civitai_client.get_model_version_info(model_version_id)
# Populate lora entry with Civitai info # Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai( populated_entry = await self.populate_lora_from_civitai(
lora_entry, lora_entry,
@@ -117,9 +116,9 @@ class ComfyMetadataParser(RecipeMetadataParser):
} }
# Get additional checkpoint info from Civitai # Get additional checkpoint info from Civitai
if metadata_provider: if civitai_client:
try: try:
civitai_info_tuple = await metadata_provider.get_model_version_info(checkpoint_version_id) civitai_info_tuple = await civitai_client.get_model_version_info(checkpoint_version_id)
civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None) civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
# Populate checkpoint with Civitai info # Populate checkpoint with Civitai info
checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info) checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info)

View File

@@ -1,12 +1,10 @@
"""Parser for meta format (Lora_N Model hash) metadata.""" """Parser for meta format (Lora_N Model hash) metadata."""
import os
import re import re
import logging import logging
from typing import Dict, Any from typing import Dict, Any
from ..base import RecipeMetadataParser from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -20,11 +18,8 @@ class MetaFormatParser(RecipeMetadataParser):
return re.search(self.METADATA_MARKER, user_comment, re.IGNORECASE | re.DOTALL) is not None return re.search(self.METADATA_MARKER, user_comment, re.IGNORECASE | re.DOTALL) is not None
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]: async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from images with meta format metadata (Lora_N Model hash format)""" """Parse metadata from images with meta format metadata"""
try: try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Extract prompt and negative prompt # Extract prompt and negative prompt
parts = user_comment.split('Negative prompt:', 1) parts = user_comment.split('Negative prompt:', 1)
prompt = parts[0].strip() prompt = parts[0].strip()
@@ -127,9 +122,9 @@ class MetaFormatParser(RecipeMetadataParser):
} }
# Get info from Civitai by hash if available # Get info from Civitai by hash if available
if metadata_provider and hash_value: if civitai_client and hash_value:
try: try:
civitai_info = await metadata_provider.get_model_by_hash(hash_value) civitai_info = await civitai_client.get_model_by_hash(hash_value)
# Populate lora entry with Civitai info # Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai( populated_entry = await self.populate_lora_from_civitai(
lora_entry, lora_entry,
@@ -146,53 +141,14 @@ class MetaFormatParser(RecipeMetadataParser):
loras.append(lora_entry) loras.append(lora_entry)
# Extract checkpoint information from generic Model/Model hash fields # Extract model information
checkpoint = None model = None
model_hash = metadata.get("model_hash") if 'model' in metadata:
model_name = metadata.get("model") model = metadata['model']
if model_hash or model_name:
cleaned_name = None
if model_name:
cleaned_name = re.split(r"[\\\\/]", model_name)[-1]
cleaned_name = os.path.splitext(cleaned_name)[0]
checkpoint_entry = {
'id': 0,
'modelId': 0,
'name': model_name or "Unknown Checkpoint",
'version': '',
'type': 'checkpoint',
'hash': model_hash or "",
'existsLocally': False,
'localPath': None,
'file_name': cleaned_name or (model_name or ""),
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider and model_hash:
try:
civitai_info = await metadata_provider.get_model_by_hash(model_hash)
checkpoint_entry = await self.populate_checkpoint_from_civitai(
checkpoint_entry,
civitai_info
)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint hash {model_hash}: {e}")
if checkpoint_entry.get("baseModel"):
base_model_value = checkpoint_entry["baseModel"]
base_model_counts[base_model_value] = base_model_counts.get(base_model_value, 0) + 1
checkpoint = checkpoint_entry
# Set base_model to the most common one from civitai_info or checkpoint # Set base_model to the most common one from civitai_info
base_model = checkpoint["baseModel"] if checkpoint and checkpoint.get("baseModel") else None base_model = None
if not base_model and base_model_counts: if base_model_counts:
base_model = max(base_model_counts.items(), key=lambda x: x[1])[0] base_model = max(base_model_counts.items(), key=lambda x: x[1])[0]
# Extract generation parameters for recipe metadata # Extract generation parameters for recipe metadata
@@ -210,8 +166,7 @@ class MetaFormatParser(RecipeMetadataParser):
'loras': loras, 'loras': loras,
'gen_params': gen_params, 'gen_params': gen_params,
'raw_metadata': metadata, 'raw_metadata': metadata,
'from_meta_format': True, 'from_meta_format': True
**({'checkpoint': checkpoint, 'model': checkpoint} if checkpoint else {})
} }
except Exception as e: except Exception as e:

View File

@@ -3,11 +3,10 @@
import re import re
import json import json
import logging import logging
from typing import Dict, Any, Optional from typing import Dict, Any
from ...config import config from ...config import config
from ..base import RecipeMetadataParser from ..base import RecipeMetadataParser
from ..constants import GEN_PARAM_KEYS from ..constants import GEN_PARAM_KEYS
from ...services.metadata_service import get_default_metadata_provider
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -16,28 +15,6 @@ class RecipeFormatParser(RecipeMetadataParser):
# Regular expression pattern for extracting recipe metadata # Regular expression pattern for extracting recipe metadata
METADATA_MARKER = r'Recipe metadata: (\{.*\})' METADATA_MARKER = r'Recipe metadata: (\{.*\})'
async def _get_lora_from_version_index(self, recipe_scanner, model_version_id: Any) -> Optional[Dict[str, Any]]:
"""Return a cached LoRA entry by modelVersionId if available."""
if not recipe_scanner or not getattr(recipe_scanner, "_lora_scanner", None):
return None
try:
normalized_id = int(model_version_id)
except (TypeError, ValueError):
return None
try:
cache = await recipe_scanner._lora_scanner.get_cached_data()
except Exception as exc: # pragma: no cover - defensive logging
logger.debug("Unable to load lora cache for version lookup: %s", exc)
return None
if not cache or not getattr(cache, "version_index", None):
return None
return cache.version_index.get(normalized_id)
def is_metadata_matching(self, user_comment: str) -> bool: def is_metadata_matching(self, user_comment: str) -> bool:
"""Check if the user comment matches the metadata format""" """Check if the user comment matches the metadata format"""
@@ -46,9 +23,6 @@ class RecipeFormatParser(RecipeMetadataParser):
async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]: async def parse_metadata(self, user_comment: str, recipe_scanner=None, civitai_client=None) -> Dict[str, Any]:
"""Parse metadata from images with dedicated recipe metadata format""" """Parse metadata from images with dedicated recipe metadata format"""
try: try:
# Get metadata provider instead of using civitai_client directly
metadata_provider = await get_default_metadata_provider()
# Extract recipe metadata from user comment # Extract recipe metadata from user comment
try: try:
# Look for recipe metadata section # Look for recipe metadata section
@@ -75,110 +49,49 @@ class RecipeFormatParser(RecipeMetadataParser):
'type': 'lora', 'type': 'lora',
'weight': lora.get('strength', 1.0), 'weight': lora.get('strength', 1.0),
'file_name': lora.get('file_name', ''), 'file_name': lora.get('file_name', ''),
'hash': lora.get('hash', ''), 'hash': lora.get('hash', '')
'existsLocally': False,
'inLibrary': False,
'localPath': None,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'size': 0
} }
# Check if this LoRA exists locally by SHA256 hash # Check if this LoRA exists locally by SHA256 hash
if recipe_scanner: if lora.get('hash') and recipe_scanner:
lora_scanner = recipe_scanner._lora_scanner lora_scanner = recipe_scanner._lora_scanner
exists_locally = lora_scanner.has_lora_hash(lora['hash'])
if lora.get('hash'): if exists_locally:
exists_locally = lora_scanner.has_hash(lora['hash']) lora_cache = await lora_scanner.get_cached_data()
if exists_locally: lora_item = next((item for item in lora_cache.raw_data if item['sha256'].lower() == lora['hash'].lower()), None)
lora_cache = await lora_scanner.get_cached_data() if lora_item:
lora_item = next((item for item in lora_cache.raw_data if item['sha256'].lower() == lora['hash'].lower()), None)
if lora_item:
lora_entry['existsLocally'] = True
lora_entry['inLibrary'] = True
lora_entry['localPath'] = lora_item['file_path']
lora_entry['file_name'] = lora_item['file_name']
lora_entry['size'] = lora_item['size']
lora_entry['thumbnailUrl'] = config.get_preview_static_url(lora_item['preview_url'])
else:
lora_entry['existsLocally'] = False
lora_entry['inLibrary'] = False
lora_entry['localPath'] = None
# If we still don't have a local match, try matching by modelVersionId
if not lora_entry['existsLocally'] and lora.get('modelVersionId') is not None:
cached_lora = await self._get_lora_from_version_index(recipe_scanner, lora.get('modelVersionId'))
if cached_lora:
lora_entry['existsLocally'] = True lora_entry['existsLocally'] = True
lora_entry['inLibrary'] = True lora_entry['localPath'] = lora_item['file_path']
lora_entry['localPath'] = cached_lora.get('file_path') lora_entry['file_name'] = lora_item['file_name']
lora_entry['file_name'] = cached_lora.get('file_name') or lora_entry['file_name'] lora_entry['size'] = lora_item['size']
lora_entry['size'] = cached_lora.get('size', lora_entry['size']) lora_entry['thumbnailUrl'] = config.get_preview_static_url(lora_item['preview_url'])
if cached_lora.get('sha256'):
lora_entry['hash'] = cached_lora['sha256'] else:
preview_url = cached_lora.get('preview_url') lora_entry['existsLocally'] = False
if preview_url: lora_entry['localPath'] = None
lora_entry['thumbnailUrl'] = config.get_preview_static_url(preview_url)
# Try to get additional info from Civitai if we have a model version ID
# Try to get additional info from Civitai if we have a model version ID and still missing locally if lora.get('modelVersionId') and civitai_client:
if not lora_entry['existsLocally'] and lora.get('modelVersionId') and metadata_provider: try:
try: civitai_info_tuple = await civitai_client.get_model_version_info(lora['modelVersionId'])
civitai_info_tuple = await metadata_provider.get_model_version_info(lora['modelVersionId']) # Populate lora entry with Civitai info
# Populate lora entry with Civitai info populated_entry = await self.populate_lora_from_civitai(
populated_entry = await self.populate_lora_from_civitai( lora_entry,
lora_entry, civitai_info_tuple,
civitai_info_tuple, recipe_scanner,
recipe_scanner, None, # No need to track base model counts
None, # No need to track base model counts lora['hash']
lora_entry.get('hash', '') )
) if populated_entry is None:
if populated_entry is None: continue # Skip invalid LoRA types
continue # Skip invalid LoRA types lora_entry = populated_entry
lora_entry = populated_entry except Exception as e:
except Exception as e: logger.error(f"Error fetching Civitai info for LoRA: {e}")
logger.error(f"Error fetching Civitai info for LoRA: {e}") lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
loras.append(lora_entry) loras.append(lora_entry)
logger.info(f"Found {len(loras)} loras in recipe metadata") logger.info(f"Found {len(loras)} loras in recipe metadata")
# Process checkpoint information if present
checkpoint = None
checkpoint_data = recipe_metadata.get('checkpoint') or {}
if isinstance(checkpoint_data, dict) and checkpoint_data:
version_id = checkpoint_data.get('modelVersionId') or checkpoint_data.get('id')
checkpoint_entry = {
'id': version_id or 0,
'modelId': checkpoint_data.get('modelId', 0),
'name': checkpoint_data.get('name', 'Unknown Checkpoint'),
'version': checkpoint_data.get('version', ''),
'type': checkpoint_data.get('type', 'checkpoint'),
'hash': checkpoint_data.get('hash', ''),
'existsLocally': False,
'localPath': None,
'file_name': checkpoint_data.get('file_name', ''),
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if metadata_provider:
try:
civitai_info = None
if version_id:
civitai_info = await metadata_provider.get_model_version_info(str(version_id))
elif checkpoint_entry.get('hash'):
civitai_info = await metadata_provider.get_model_by_hash(checkpoint_entry['hash'])
if civitai_info:
checkpoint_entry = await self.populate_checkpoint_from_civitai(checkpoint_entry, civitai_info)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint in recipe metadata: {e}")
checkpoint = checkpoint_entry
# Filter gen_params to only include recognized keys # Filter gen_params to only include recognized keys
filtered_gen_params = {} filtered_gen_params = {}
@@ -188,13 +101,12 @@ class RecipeFormatParser(RecipeMetadataParser):
filtered_gen_params[key] = value filtered_gen_params[key] = value
return { return {
'base_model': checkpoint['baseModel'] if checkpoint and checkpoint.get('baseModel') else recipe_metadata.get('base_model', ''), 'base_model': recipe_metadata.get('base_model', ''),
'loras': loras, 'loras': loras,
'gen_params': filtered_gen_params, 'gen_params': filtered_gen_params,
'tags': recipe_metadata.get('tags', []), 'tags': recipe_metadata.get('tags', []),
'title': recipe_metadata.get('title', ''), 'title': recipe_metadata.get('title', ''),
'from_recipe_metadata': True, 'from_recipe_metadata': True
**({'checkpoint': checkpoint, 'model': checkpoint} if checkpoint else {})
} }
except Exception as e: except Exception as e:

View File

@@ -1,300 +1,619 @@
from __future__ import annotations
import logging
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Callable, Dict, Mapping import asyncio
import json
import logging
from aiohttp import web
from typing import Dict
import jinja2 import jinja2
from aiohttp import web
from ..utils.routes_common import ModelRouteUtils
from ..services.websocket_manager import ws_manager
from ..services.settings_manager import settings
from ..config import config from ..config import config
from ..services.download_coordinator import DownloadCoordinator
from ..services.downloader import get_downloader
from ..services.metadata_service import get_default_metadata_provider, get_metadata_provider
from ..services.metadata_sync_service import MetadataSyncService
from ..services.model_file_service import ModelFileService, ModelMoveService
from ..services.model_lifecycle_service import ModelLifecycleService
from ..services.preview_asset_service import PreviewAssetService
from ..services.server_i18n import server_i18n as default_server_i18n
from ..services.service_registry import ServiceRegistry
from ..services.settings_manager import get_settings_manager
from ..services.tag_update_service import TagUpdateService
from ..services.websocket_manager import ws_manager as default_ws_manager
from ..services.use_cases import (
AutoOrganizeUseCase,
BulkMetadataRefreshUseCase,
DownloadModelUseCase,
)
from ..services.websocket_progress_callback import (
WebSocketBroadcastCallback,
WebSocketProgressCallback,
)
from ..utils.exif_utils import ExifUtils
from ..utils.metadata_manager import MetadataManager
from .model_route_registrar import COMMON_ROUTE_DEFINITIONS, ModelRouteRegistrar
from .handlers.model_handlers import (
ModelAutoOrganizeHandler,
ModelCivitaiHandler,
ModelDownloadHandler,
ModelHandlerSet,
ModelListingHandler,
ModelManagementHandler,
ModelMoveHandler,
ModelPageView,
ModelQueryHandler,
ModelUpdateHandler,
)
if TYPE_CHECKING:
from ..services.model_update_service import ModelUpdateService
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class BaseModelRoutes(ABC): class BaseModelRoutes(ABC):
"""Base route controller for all model types.""" """Base route controller for all model types"""
template_name: str | None = None def __init__(self, service):
"""Initialize the route controller
def __init__(
self, Args:
service=None, service: Model service instance (LoraService, CheckpointService, etc.)
*, """
settings_service=None,
ws_manager=default_ws_manager,
server_i18n=default_server_i18n,
metadata_provider_factory=get_default_metadata_provider,
) -> None:
self.service = None
self.model_type = ""
self._settings = settings_service or get_settings_manager()
self._ws_manager = ws_manager
self._server_i18n = server_i18n
self._metadata_provider_factory = metadata_provider_factory
self.template_env = jinja2.Environment(
loader=jinja2.FileSystemLoader(config.templates_path),
autoescape=True,
)
self.model_file_service: ModelFileService | None = None
self.model_move_service: ModelMoveService | None = None
self.model_lifecycle_service: ModelLifecycleService | None = None
self.websocket_progress_callback = WebSocketProgressCallback()
self.metadata_progress_callback = WebSocketBroadcastCallback()
self._handler_set: ModelHandlerSet | None = None
self._handler_mapping: Dict[str, Callable[[web.Request], web.StreamResponse]] | None = None
self._preview_service = PreviewAssetService(
metadata_manager=MetadataManager,
downloader_factory=get_downloader,
exif_utils=ExifUtils,
)
self._metadata_sync_service = MetadataSyncService(
metadata_manager=MetadataManager,
preview_service=self._preview_service,
settings=self._settings,
default_metadata_provider_factory=metadata_provider_factory,
metadata_provider_selector=get_metadata_provider,
)
self._tag_update_service = TagUpdateService(metadata_manager=MetadataManager)
self._download_coordinator = DownloadCoordinator(
ws_manager=self._ws_manager,
download_manager_factory=ServiceRegistry.get_download_manager,
)
self._model_update_service: ModelUpdateService | None = None
if service is not None:
self.attach_service(service)
def set_model_update_service(self, service: "ModelUpdateService") -> None:
"""Attach the model update tracking service."""
self._model_update_service = service
self._handler_set = None
self._handler_mapping = None
def attach_service(self, service) -> None:
"""Attach a model service and rebuild handler dependencies."""
self.service = service self.service = service
self.model_type = service.model_type self.model_type = service.model_type
self.model_file_service = ModelFileService(service.scanner, service.model_type) self.template_env = jinja2.Environment(
self.model_move_service = ModelMoveService(service.scanner, service.model_type) loader=jinja2.FileSystemLoader(config.templates_path),
self.model_lifecycle_service = ModelLifecycleService( autoescape=True
scanner=service.scanner,
metadata_manager=MetadataManager,
metadata_loader=self._metadata_sync_service.load_local_metadata,
recipe_scanner_factory=ServiceRegistry.get_recipe_scanner,
update_service=self._model_update_service,
) )
self._handler_set = None
self._handler_mapping = None def setup_routes(self, app: web.Application, prefix: str):
"""Setup common routes for the model type
def _ensure_handler_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
if self._handler_mapping is None: Args:
handler_set = self._create_handler_set() app: aiohttp application
self._handler_set = handler_set prefix: URL prefix (e.g., 'loras', 'checkpoints')
self._handler_mapping = handler_set.to_route_mapping() """
return self._handler_mapping # Common model management routes
app.router.add_get(f'/api/{prefix}', self.get_models)
def _create_handler_set(self) -> ModelHandlerSet: app.router.add_post(f'/api/{prefix}/delete', self.delete_model)
service = self._ensure_service() app.router.add_post(f'/api/{prefix}/exclude', self.exclude_model)
update_service = self._ensure_model_update_service() app.router.add_post(f'/api/{prefix}/fetch-civitai', self.fetch_civitai)
page_view = ModelPageView( app.router.add_post(f'/api/{prefix}/relink-civitai', self.relink_civitai)
template_env=self.template_env, app.router.add_post(f'/api/{prefix}/replace-preview', self.replace_preview)
template_name=self.template_name or "", app.router.add_post(f'/api/{prefix}/save-metadata', self.save_metadata)
service=service, app.router.add_post(f'/api/{prefix}/rename', self.rename_model)
settings_service=self._settings, app.router.add_post(f'/api/{prefix}/bulk-delete', self.bulk_delete_models)
server_i18n=self._server_i18n, app.router.add_post(f'/api/{prefix}/verify-duplicates', self.verify_duplicates)
logger=logger,
) # Common query routes
listing = ModelListingHandler( app.router.add_get(f'/api/{prefix}/top-tags', self.get_top_tags)
service=service, app.router.add_get(f'/api/{prefix}/base-models', self.get_base_models)
parse_specific_params=self._parse_specific_params, app.router.add_get(f'/api/{prefix}/scan', self.scan_models)
logger=logger, app.router.add_get(f'/api/{prefix}/roots', self.get_model_roots)
) app.router.add_get(f'/api/{prefix}/folders', self.get_folders)
management = ModelManagementHandler( app.router.add_get(f'/api/{prefix}/find-duplicates', self.find_duplicate_models)
service=service, app.router.add_get(f'/api/{prefix}/find-filename-conflicts', self.find_filename_conflicts)
logger=logger,
metadata_sync=self._metadata_sync_service,
preview_service=self._preview_service,
tag_update_service=self._tag_update_service,
lifecycle_service=self._ensure_lifecycle_service(),
)
query = ModelQueryHandler(service=service, logger=logger)
download_use_case = DownloadModelUseCase(download_coordinator=self._download_coordinator)
download = ModelDownloadHandler(
ws_manager=self._ws_manager,
logger=logger,
download_use_case=download_use_case,
download_coordinator=self._download_coordinator,
)
metadata_refresh_use_case = BulkMetadataRefreshUseCase(
service=service,
metadata_sync=self._metadata_sync_service,
settings_service=self._settings,
logger=logger,
)
civitai = ModelCivitaiHandler(
service=service,
settings_service=self._settings,
ws_manager=self._ws_manager,
logger=logger,
metadata_provider_factory=self._metadata_provider_factory,
validate_model_type=self._validate_civitai_model_type,
expected_model_types=self._get_expected_model_types,
find_model_file=self._find_model_file,
metadata_sync=self._metadata_sync_service,
metadata_refresh_use_case=metadata_refresh_use_case,
metadata_progress_callback=self.metadata_progress_callback,
)
move = ModelMoveHandler(move_service=self._ensure_move_service(), logger=logger)
auto_organize_use_case = AutoOrganizeUseCase(
file_service=self._ensure_file_service(),
lock_provider=self._ws_manager,
)
auto_organize = ModelAutoOrganizeHandler(
use_case=auto_organize_use_case,
progress_callback=self.websocket_progress_callback,
ws_manager=self._ws_manager,
logger=logger,
)
updates = ModelUpdateHandler(
service=service,
update_service=update_service,
metadata_provider_selector=get_metadata_provider,
logger=logger,
)
return ModelHandlerSet(
page_view=page_view,
listing=listing,
management=management,
query=query,
download=download,
civitai=civitai,
move=move,
auto_organize=auto_organize,
updates=updates,
)
@property
def route_handlers(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
return self._ensure_handler_mapping()
def setup_routes(self, app: web.Application, prefix: str) -> None:
registrar = ModelRouteRegistrar(app)
handler_lookup = {
definition.handler_name: self._make_handler_proxy(definition.handler_name)
for definition in COMMON_ROUTE_DEFINITIONS
}
registrar.register_common_routes(prefix, handler_lookup)
self.setup_specific_routes(registrar, prefix)
# Common Download management
app.router.add_post(f'/api/download-model', self.download_model)
app.router.add_get(f'/api/download-model-get', self.download_model_get)
app.router.add_get(f'/api/cancel-download-get', self.cancel_download_get)
app.router.add_get(f'/api/download-progress/{{download_id}}', self.get_download_progress)
# CivitAI integration routes
app.router.add_post(f'/api/{prefix}/fetch-all-civitai', self.fetch_all_civitai)
# app.router.add_get(f'/api/civitai/versions/{{model_id}}', self.get_civitai_versions)
# Add generic page route
app.router.add_get(f'/{prefix}', self.handle_models_page)
# Setup model-specific routes
self.setup_specific_routes(app, prefix)
@abstractmethod @abstractmethod
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str) -> None: def setup_specific_routes(self, app: web.Application, prefix: str):
"""Setup model-specific routes.""" """Setup model-specific routes - to be implemented by subclasses"""
raise NotImplementedError pass
def _parse_specific_params(self, request: web.Request) -> Dict: async def handle_models_page(self, request: web.Request) -> web.Response:
"""Parse model-specific parameters - to be overridden by subclasses.""" """
return {} Generic handler for model pages (e.g., /loras, /checkpoints).
Subclasses should set self.template_env and template_name.
def _validate_civitai_model_type(self, model_type: str) -> bool: """
"""Validate CivitAI model type - to be overridden by subclasses.""" try:
return True # Check if the scanner is initializing
is_initializing = (
def _get_expected_model_types(self) -> str: self.service.scanner._cache is None or
"""Get expected model types string for error messages - to be overridden by subclasses.""" (hasattr(self.service.scanner, 'is_initializing') and callable(self.service.scanner.is_initializing) and self.service.scanner.is_initializing()) or
return "any model type" (hasattr(self.service.scanner, '_is_initializing') and self.service.scanner._is_initializing)
def _find_model_file(self, files):
"""Find the appropriate model file from the files list - can be overridden by subclasses."""
return next((file for file in files if file.get("type") == "Model" and file.get("primary") is True), None)
def get_handler(self, name: str) -> Callable[[web.Request], web.StreamResponse]:
"""Expose handlers for subclasses or tests."""
return self._ensure_handler_mapping()[name]
def _ensure_service(self):
if self.service is None:
raise RuntimeError("Model service has not been attached")
return self.service
def _ensure_file_service(self) -> ModelFileService:
if self.model_file_service is None:
service = self._ensure_service()
self.model_file_service = ModelFileService(service.scanner, service.model_type)
return self.model_file_service
def _ensure_move_service(self) -> ModelMoveService:
if self.model_move_service is None:
service = self._ensure_service()
self.model_move_service = ModelMoveService(service.scanner, service.model_type)
return self.model_move_service
def _ensure_lifecycle_service(self) -> ModelLifecycleService:
if self.model_lifecycle_service is None:
service = self._ensure_service()
self.model_lifecycle_service = ModelLifecycleService(
scanner=service.scanner,
metadata_manager=MetadataManager,
metadata_loader=self._metadata_sync_service.load_local_metadata,
recipe_scanner_factory=ServiceRegistry.get_recipe_scanner,
) )
return self.model_lifecycle_service
def _make_handler_proxy(self, name: str) -> Callable[[web.Request], web.StreamResponse]: template_name = getattr(self, "template_name", None)
async def proxy(request: web.Request) -> web.StreamResponse: if not self.template_env or not template_name:
return web.Response(text="Template environment or template name not set", status=500)
if is_initializing:
rendered = self.template_env.get_template(template_name).render(
folders=[],
is_initializing=True,
settings=settings,
request=request
)
else:
try:
cache = await self.service.scanner.get_cached_data(force_refresh=False)
rendered = self.template_env.get_template(template_name).render(
folders=getattr(cache, "folders", []),
is_initializing=False,
settings=settings,
request=request
)
except Exception as cache_error:
logger.error(f"Error loading cache data: {cache_error}")
rendered = self.template_env.get_template(template_name).render(
folders=[],
is_initializing=True,
settings=settings,
request=request
)
return web.Response(
text=rendered,
content_type='text/html'
)
except Exception as e:
logger.error(f"Error handling models page: {e}", exc_info=True)
return web.Response(
text="Error loading models page",
status=500
)
async def get_models(self, request: web.Request) -> web.Response:
"""Get paginated model data"""
try:
# Parse common query parameters
params = self._parse_common_params(request)
# Get data from service
result = await self.service.get_paginated_data(**params)
# Format response items
formatted_result = {
'items': [await self.service.format_response(item) for item in result['items']],
'total': result['total'],
'page': result['page'],
'page_size': result['page_size'],
'total_pages': result['total_pages']
}
return web.json_response(formatted_result)
except Exception as e:
logger.error(f"Error in get_{self.model_type}s: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500)
def _parse_common_params(self, request: web.Request) -> Dict:
"""Parse common query parameters"""
# Parse basic pagination and sorting
page = int(request.query.get('page', '1'))
page_size = min(int(request.query.get('page_size', '20')), 100)
sort_by = request.query.get('sort_by', 'name')
folder = request.query.get('folder', None)
search = request.query.get('search', None)
fuzzy_search = request.query.get('fuzzy_search', 'false').lower() == 'true'
# Parse filter arrays
base_models = request.query.getall('base_model', [])
tags = request.query.getall('tag', [])
favorites_only = request.query.get('favorites_only', 'false').lower() == 'true'
# Parse search options
search_options = {
'filename': request.query.get('search_filename', 'true').lower() == 'true',
'modelname': request.query.get('search_modelname', 'true').lower() == 'true',
'tags': request.query.get('search_tags', 'false').lower() == 'true',
'recursive': request.query.get('recursive', 'false').lower() == 'true',
}
# Parse hash filters if provided
hash_filters = {}
if 'hash' in request.query:
hash_filters['single_hash'] = request.query['hash']
elif 'hashes' in request.query:
try: try:
handler = self.get_handler(name) hash_list = json.loads(request.query['hashes'])
except RuntimeError: if isinstance(hash_list, list):
return web.json_response({"success": False, "error": "Service not ready"}, status=503) hash_filters['multiple_hashes'] = hash_list
return await handler(request) except (json.JSONDecodeError, TypeError):
pass
return proxy
return {
def _ensure_model_update_service(self) -> "ModelUpdateService": 'page': page,
if self._model_update_service is None: 'page_size': page_size,
raise RuntimeError("Model update service has not been attached") 'sort_by': sort_by,
return self._model_update_service 'folder': folder,
'search': search,
'fuzzy_search': fuzzy_search,
'base_models': base_models,
'tags': tags,
'search_options': search_options,
'hash_filters': hash_filters,
'favorites_only': favorites_only,
# Add model-specific parameters
**self._parse_specific_params(request)
}
def _parse_specific_params(self, request: web.Request) -> Dict:
"""Parse model-specific parameters - to be overridden by subclasses"""
return {}
# Common route handlers
async def delete_model(self, request: web.Request) -> web.Response:
"""Handle model deletion request"""
return await ModelRouteUtils.handle_delete_model(request, self.service.scanner)
async def exclude_model(self, request: web.Request) -> web.Response:
"""Handle model exclusion request"""
return await ModelRouteUtils.handle_exclude_model(request, self.service.scanner)
async def fetch_civitai(self, request: web.Request) -> web.Response:
"""Handle CivitAI metadata fetch request"""
response = await ModelRouteUtils.handle_fetch_civitai(request, self.service.scanner)
# If successful, format the metadata before returning
if response.status == 200:
data = json.loads(response.body.decode('utf-8'))
if data.get("success") and data.get("metadata"):
formatted_metadata = await self.service.format_response(data["metadata"])
return web.json_response({
"success": True,
"metadata": formatted_metadata
})
return response
async def relink_civitai(self, request: web.Request) -> web.Response:
"""Handle CivitAI metadata re-linking request"""
return await ModelRouteUtils.handle_relink_civitai(request, self.service.scanner)
async def replace_preview(self, request: web.Request) -> web.Response:
"""Handle preview image replacement"""
return await ModelRouteUtils.handle_replace_preview(request, self.service.scanner)
async def save_metadata(self, request: web.Request) -> web.Response:
"""Handle saving metadata updates"""
return await ModelRouteUtils.handle_save_metadata(request, self.service.scanner)
async def rename_model(self, request: web.Request) -> web.Response:
"""Handle renaming a model file and its associated files"""
return await ModelRouteUtils.handle_rename_model(request, self.service.scanner)
async def bulk_delete_models(self, request: web.Request) -> web.Response:
"""Handle bulk deletion of models"""
return await ModelRouteUtils.handle_bulk_delete_models(request, self.service.scanner)
async def verify_duplicates(self, request: web.Request) -> web.Response:
"""Handle verification of duplicate model hashes"""
return await ModelRouteUtils.handle_verify_duplicates(request, self.service.scanner)
async def get_top_tags(self, request: web.Request) -> web.Response:
"""Handle request for top tags sorted by frequency"""
try:
limit = int(request.query.get('limit', '20'))
if limit < 1 or limit > 100:
limit = 20
top_tags = await self.service.get_top_tags(limit)
return web.json_response({
'success': True,
'tags': top_tags
})
except Exception as e:
logger.error(f"Error getting top tags: {str(e)}", exc_info=True)
return web.json_response({
'success': False,
'error': 'Internal server error'
}, status=500)
async def get_base_models(self, request: web.Request) -> web.Response:
"""Get base models used in models"""
try:
limit = int(request.query.get('limit', '20'))
if limit < 1 or limit > 100:
limit = 20
base_models = await self.service.get_base_models(limit)
return web.json_response({
'success': True,
'base_models': base_models
})
except Exception as e:
logger.error(f"Error retrieving base models: {e}")
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def scan_models(self, request: web.Request) -> web.Response:
"""Force a rescan of model files"""
try:
full_rebuild = request.query.get('full_rebuild', 'false').lower() == 'true'
await self.service.scan_models(force_refresh=True, rebuild_cache=full_rebuild)
return web.json_response({
"status": "success",
"message": f"{self.model_type.capitalize()} scan completed"
})
except Exception as e:
logger.error(f"Error in scan_{self.model_type}s: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500)
async def get_model_roots(self, request: web.Request) -> web.Response:
"""Return the model root directories"""
try:
roots = self.service.get_model_roots()
return web.json_response({
"success": True,
"roots": roots
})
except Exception as e:
logger.error(f"Error getting {self.model_type} roots: {e}", exc_info=True)
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
async def get_folders(self, request: web.Request) -> web.Response:
"""Get all folders in the cache"""
try:
cache = await self.service.scanner.get_cached_data()
return web.json_response({
'folders': cache.folders
})
except Exception as e:
logger.error(f"Error getting folders: {e}")
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def find_duplicate_models(self, request: web.Request) -> web.Response:
"""Find models with duplicate SHA256 hashes"""
try:
# Get duplicate hashes from service
duplicates = self.service.find_duplicate_hashes()
# Format the response
result = []
cache = await self.service.scanner.get_cached_data()
for sha256, paths in duplicates.items():
group = {
"hash": sha256,
"models": []
}
# Find matching models for each path
for path in paths:
model = next((m for m in cache.raw_data if m['file_path'] == path), None)
if model:
group["models"].append(await self.service.format_response(model))
# Add the primary model too
primary_path = self.service.get_path_by_hash(sha256)
if primary_path and primary_path not in paths:
primary_model = next((m for m in cache.raw_data if m['file_path'] == primary_path), None)
if primary_model:
group["models"].insert(0, await self.service.format_response(primary_model))
if len(group["models"]) > 1: # Only include if we found multiple models
result.append(group)
return web.json_response({
"success": True,
"duplicates": result,
"count": len(result)
})
except Exception as e:
logger.error(f"Error finding duplicate {self.model_type}s: {e}", exc_info=True)
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
async def find_filename_conflicts(self, request: web.Request) -> web.Response:
"""Find models with conflicting filenames"""
try:
# Get duplicate filenames from service
duplicates = self.service.find_duplicate_filenames()
# Format the response
result = []
cache = await self.service.scanner.get_cached_data()
for filename, paths in duplicates.items():
group = {
"filename": filename,
"models": []
}
# Find matching models for each path
for path in paths:
model = next((m for m in cache.raw_data if m['file_path'] == path), None)
if model:
group["models"].append(await self.service.format_response(model))
# Find the model from the main index too
hash_val = self.service.scanner._hash_index.get_hash_by_filename(filename)
if hash_val:
main_path = self.service.get_path_by_hash(hash_val)
if main_path and main_path not in paths:
main_model = next((m for m in cache.raw_data if m['file_path'] == main_path), None)
if main_model:
group["models"].insert(0, await self.service.format_response(main_model))
if group["models"]:
result.append(group)
return web.json_response({
"success": True,
"conflicts": result,
"count": len(result)
})
except Exception as e:
logger.error(f"Error finding filename conflicts for {self.model_type}s: {e}", exc_info=True)
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
# Download management methods
async def download_model(self, request: web.Request) -> web.Response:
"""Handle model download request"""
return await ModelRouteUtils.handle_download_model(request)
async def download_model_get(self, request: web.Request) -> web.Response:
"""Handle model download request via GET method"""
try:
# Extract query parameters
model_id = request.query.get('model_id')
if not model_id:
return web.Response(
status=400,
text="Missing required parameter: Please provide 'model_id'"
)
# Get optional parameters
model_version_id = request.query.get('model_version_id')
download_id = request.query.get('download_id')
use_default_paths = request.query.get('use_default_paths', 'false').lower() == 'true'
# Create a data dictionary that mimics what would be received from a POST request
data = {
'model_id': model_id
}
# Add optional parameters only if they are provided
if model_version_id:
data['model_version_id'] = model_version_id
if download_id:
data['download_id'] = download_id
data['use_default_paths'] = use_default_paths
# Create a mock request object with the data
future = asyncio.get_event_loop().create_future()
future.set_result(data)
mock_request = type('MockRequest', (), {
'json': lambda self=None: future
})()
# Call the existing download handler
return await ModelRouteUtils.handle_download_model(mock_request)
except Exception as e:
error_message = str(e)
logger.error(f"Error downloading model via GET: {error_message}", exc_info=True)
return web.Response(status=500, text=error_message)
async def cancel_download_get(self, request: web.Request) -> web.Response:
"""Handle GET request for cancelling a download by download_id"""
try:
download_id = request.query.get('download_id')
if not download_id:
return web.json_response({
'success': False,
'error': 'Download ID is required'
}, status=400)
# Create a mock request with match_info for compatibility
mock_request = type('MockRequest', (), {
'match_info': {'download_id': download_id}
})()
return await ModelRouteUtils.handle_cancel_download(mock_request)
except Exception as e:
logger.error(f"Error cancelling download via GET: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def get_download_progress(self, request: web.Request) -> web.Response:
"""Handle request for download progress by download_id"""
try:
# Get download_id from URL path
download_id = request.match_info.get('download_id')
if not download_id:
return web.json_response({
'success': False,
'error': 'Download ID is required'
}, status=400)
progress_data = ws_manager.get_download_progress(download_id)
if progress_data is None:
return web.json_response({
'success': False,
'error': 'Download ID not found'
}, status=404)
return web.json_response({
'success': True,
'progress': progress_data.get('progress', 0)
})
except Exception as e:
logger.error(f"Error getting download progress: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def fetch_all_civitai(self, request: web.Request) -> web.Response:
"""Fetch CivitAI metadata for all models in the background"""
try:
cache = await self.service.scanner.get_cached_data()
total = len(cache.raw_data)
processed = 0
success = 0
needs_resort = False
# Prepare models to process
to_process = [
model for model in cache.raw_data
if model.get('sha256') and (not model.get('civitai') or 'id' not in model.get('civitai')) and model.get('from_civitai', True)
]
total_to_process = len(to_process)
# Send initial progress
await ws_manager.broadcast({
'status': 'started',
'total': total_to_process,
'processed': 0,
'success': 0
})
# Process each model
for model in to_process:
try:
original_name = model.get('model_name')
if await ModelRouteUtils.fetch_and_update_model(
sha256=model['sha256'],
file_path=model['file_path'],
model_data=model,
update_cache_func=self.service.scanner.update_single_model_cache
):
success += 1
if original_name != model.get('model_name'):
needs_resort = True
processed += 1
# Send progress update
await ws_manager.broadcast({
'status': 'processing',
'total': total_to_process,
'processed': processed,
'success': success,
'current_name': model.get('model_name', 'Unknown')
})
except Exception as e:
logger.error(f"Error fetching CivitAI data for {model['file_path']}: {e}")
if needs_resort:
await cache.resort()
# Send completion message
await ws_manager.broadcast({
'status': 'completed',
'total': total_to_process,
'processed': processed,
'success': success
})
return web.json_response({
"success": True,
"message": f"Successfully updated {success} of {processed} processed {self.model_type}s (total: {total})"
})
except Exception as e:
# Send error message
await ws_manager.broadcast({
'status': 'error',
'error': str(e)
})
logger.error(f"Error in fetch_all_civitai for {self.model_type}s: {e}")
return web.Response(text=str(e), status=500)
async def get_civitai_versions(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai model with local availability info"""
# This will be implemented by subclasses as they need CivitAI client access
return web.json_response({
"error": "Not implemented in base class"
}, status=501)

View File

@@ -1,200 +0,0 @@
"""Base infrastructure shared across recipe routes."""
from __future__ import annotations
import logging
import os
from typing import Callable, Mapping
import jinja2
from aiohttp import web
from ..config import config
from ..recipes import RecipeParserFactory
from ..services.downloader import get_downloader
from ..services.recipes import (
RecipeAnalysisService,
RecipePersistenceService,
RecipeSharingService,
)
from ..services.server_i18n import server_i18n
from ..services.service_registry import ServiceRegistry
from ..services.settings_manager import get_settings_manager
from ..utils.constants import CARD_PREVIEW_WIDTH
from ..utils.exif_utils import ExifUtils
from .handlers.recipe_handlers import (
RecipeAnalysisHandler,
RecipeHandlerSet,
RecipeListingHandler,
RecipeManagementHandler,
RecipePageView,
RecipeQueryHandler,
RecipeSharingHandler,
)
from .recipe_route_registrar import ROUTE_DEFINITIONS
logger = logging.getLogger(__name__)
class BaseRecipeRoutes:
"""Common dependency and startup wiring for recipe routes."""
_HANDLER_NAMES: tuple[str, ...] = tuple(
definition.handler_name for definition in ROUTE_DEFINITIONS
)
template_name: str = "recipes.html"
def __init__(self) -> None:
self.recipe_scanner = None
self.lora_scanner = None
self.civitai_client = None
self.settings = get_settings_manager()
self.server_i18n = server_i18n
self.template_env = jinja2.Environment(
loader=jinja2.FileSystemLoader(config.templates_path),
autoescape=True,
)
self._i18n_registered = False
self._startup_hooks_registered = False
self._handler_set: RecipeHandlerSet | None = None
self._handler_mapping: dict[str, Callable] | None = None
async def attach_dependencies(self, app: web.Application | None = None) -> None:
"""Resolve shared services from the registry."""
await self._ensure_services()
self._ensure_i18n_filter()
async def ensure_dependencies_ready(self) -> None:
"""Ensure dependencies are available for request handlers."""
if self.recipe_scanner is None or self.civitai_client is None:
await self.attach_dependencies()
def register_startup_hooks(self, app: web.Application) -> None:
"""Register startup hooks once for dependency wiring."""
if self._startup_hooks_registered:
return
app.on_startup.append(self.attach_dependencies)
self._startup_hooks_registered = True
def to_route_mapping(self) -> Mapping[str, Callable]:
"""Return a mapping of handler name to coroutine for registrar binding."""
if self._handler_mapping is None:
handler_set = self._create_handler_set()
self._handler_set = handler_set
self._handler_mapping = handler_set.to_route_mapping()
return self._handler_mapping
# Internal helpers -------------------------------------------------
async def _ensure_services(self) -> None:
if self.recipe_scanner is None:
self.recipe_scanner = await ServiceRegistry.get_recipe_scanner()
self.lora_scanner = getattr(self.recipe_scanner, "_lora_scanner", None)
if self.civitai_client is None:
self.civitai_client = await ServiceRegistry.get_civitai_client()
def _ensure_i18n_filter(self) -> None:
if not self._i18n_registered:
self.template_env.filters["t"] = self.server_i18n.create_template_filter()
self._i18n_registered = True
def get_handler_owner(self):
"""Return the object supplying bound handler coroutines."""
if self._handler_set is None:
self._handler_set = self._create_handler_set()
return self._handler_set
def _create_handler_set(self) -> RecipeHandlerSet:
recipe_scanner_getter = lambda: self.recipe_scanner
civitai_client_getter = lambda: self.civitai_client
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
if not standalone_mode:
from ..metadata_collector import get_metadata # type: ignore[import-not-found]
from ..metadata_collector.metadata_processor import ( # type: ignore[import-not-found]
MetadataProcessor,
)
from ..metadata_collector.metadata_registry import ( # type: ignore[import-not-found]
MetadataRegistry,
)
else: # pragma: no cover - optional dependency path
get_metadata = None # type: ignore[assignment]
MetadataProcessor = None # type: ignore[assignment]
MetadataRegistry = None # type: ignore[assignment]
analysis_service = RecipeAnalysisService(
exif_utils=ExifUtils,
recipe_parser_factory=RecipeParserFactory,
downloader_factory=get_downloader,
metadata_collector=get_metadata,
metadata_processor_cls=MetadataProcessor,
metadata_registry_cls=MetadataRegistry,
standalone_mode=standalone_mode,
logger=logger,
)
persistence_service = RecipePersistenceService(
exif_utils=ExifUtils,
card_preview_width=CARD_PREVIEW_WIDTH,
logger=logger,
)
sharing_service = RecipeSharingService(logger=logger)
page_view = RecipePageView(
ensure_dependencies_ready=self.ensure_dependencies_ready,
settings_service=self.settings,
server_i18n=self.server_i18n,
template_env=self.template_env,
template_name=self.template_name,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
)
listing = RecipeListingHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
)
query = RecipeQueryHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
format_recipe_file_url=listing.format_recipe_file_url,
logger=logger,
)
management = RecipeManagementHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
persistence_service=persistence_service,
analysis_service=analysis_service,
downloader_factory=get_downloader,
civitai_client_getter=civitai_client_getter,
)
analysis = RecipeAnalysisHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
civitai_client_getter=civitai_client_getter,
logger=logger,
analysis_service=analysis_service,
)
sharing = RecipeSharingHandler(
ensure_dependencies_ready=self.ensure_dependencies_ready,
recipe_scanner_getter=recipe_scanner_getter,
logger=logger,
sharing_service=sharing_service,
)
return RecipeHandlerSet(
page_view=page_view,
listing=listing,
query=query,
management=management,
analysis=analysis,
sharing=sharing,
)

View File

@@ -1,12 +1,9 @@
import logging import logging
from typing import Dict
from aiohttp import web from aiohttp import web
from .base_model_routes import BaseModelRoutes from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.checkpoint_service import CheckpointService from ..services.checkpoint_service import CheckpointService
from ..services.service_registry import ServiceRegistry from ..services.service_registry import ServiceRegistry
from ..config import config
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -15,18 +12,19 @@ class CheckpointRoutes(BaseModelRoutes):
def __init__(self): def __init__(self):
"""Initialize Checkpoint routes with Checkpoint service""" """Initialize Checkpoint routes with Checkpoint service"""
super().__init__() # Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
self.template_name = "checkpoints.html" self.template_name = "checkpoints.html"
async def initialize_services(self): async def initialize_services(self):
"""Initialize services from ServiceRegistry""" """Initialize services from ServiceRegistry"""
checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner() checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
update_service = await ServiceRegistry.get_model_update_service() self.service = CheckpointService(checkpoint_scanner)
self.service = CheckpointService(checkpoint_scanner, update_service=update_service) self.civitai_client = await ServiceRegistry.get_civitai_client()
self.set_model_update_service(update_service)
# Initialize parent with the service
# Attach service dependencies super().__init__(self.service)
self.attach_service(self.service)
def setup_routes(self, app: web.Application): def setup_routes(self, app: web.Application):
"""Setup Checkpoint routes""" """Setup Checkpoint routes"""
@@ -36,35 +34,13 @@ class CheckpointRoutes(BaseModelRoutes):
# Setup common routes with 'checkpoints' prefix (includes page route) # Setup common routes with 'checkpoints' prefix (includes page route)
super().setup_routes(app, 'checkpoints') super().setup_routes(app, 'checkpoints')
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str): def setup_specific_routes(self, app: web.Application, prefix: str):
"""Setup Checkpoint-specific routes""" """Setup Checkpoint-specific routes"""
# Checkpoint-specific CivitAI integration
app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_checkpoint)
# Checkpoint info by name # Checkpoint info by name
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/info/{name}', prefix, self.get_checkpoint_info) app.router.add_get(f'/api/{prefix}/info/{{name}}', self.get_checkpoint_info)
# Checkpoint roots and Unet roots
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/checkpoints_roots', prefix, self.get_checkpoints_roots)
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/unet_roots', prefix, self.get_unet_roots)
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for Checkpoint"""
return model_type.lower() == 'checkpoint'
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "Checkpoint"
def _parse_specific_params(self, request: web.Request) -> Dict:
"""Parse Checkpoint-specific parameters"""
params: Dict = {}
if 'checkpoint_hash' in request.query:
params['hash_filters'] = {'single_hash': request.query['checkpoint_hash'].lower()}
elif 'checkpoint_hashes' in request.query:
params['hash_filters'] = {
'multiple_hashes': [h.lower() for h in request.query['checkpoint_hashes'].split(',')]
}
return params
async def get_checkpoint_info(self, request: web.Request) -> web.Response: async def get_checkpoint_info(self, request: web.Request) -> web.Response:
"""Get detailed information for a specific checkpoint by name""" """Get detailed information for a specific checkpoint by name"""
@@ -81,32 +57,49 @@ class CheckpointRoutes(BaseModelRoutes):
logger.error(f"Error in get_checkpoint_info: {e}", exc_info=True) logger.error(f"Error in get_checkpoint_info: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500) return web.json_response({"error": str(e)}, status=500)
async def get_checkpoints_roots(self, request: web.Request) -> web.Response: async def get_civitai_versions_checkpoint(self, request: web.Request) -> web.Response:
"""Return the list of checkpoint roots from config""" """Get available versions for a Civitai checkpoint model with local availability info"""
try: try:
roots = config.checkpoints_roots model_id = request.match_info['model_id']
return web.json_response({ response = await self.civitai_client.get_model_versions(model_id)
"success": True, if not response or not response.get('modelVersions'):
"roots": roots return web.Response(status=404, text="Model not found")
})
versions = response.get('modelVersions', [])
model_type = response.get('type', '')
# Check model type - should be Checkpoint
if model_type.lower() != 'checkpoint':
return web.json_response({
'error': f"Model type mismatch. Expected Checkpoint, got {model_type}"
}, status=400)
# Check local availability for each version
for version in versions:
# Find the primary model file (type="Model" and primary=true) in the files list
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model' and file.get('primary') == True), None)
# If no primary file found, try to find any model file
if not model_file:
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model'), None)
if model_file:
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
# Set existsLocally and localPath at the version level
version['existsLocally'] = self.service.has_hash(sha256)
if version['existsLocally']:
version['localPath'] = self.service.get_path_by_hash(sha256)
# Also set the model file size at the version level for easier access
version['modelSizeKB'] = model_file.get('sizeKB')
else:
# No model file found in this version
version['existsLocally'] = False
return web.json_response(versions)
except Exception as e: except Exception as e:
logger.error(f"Error getting checkpoint roots: {e}", exc_info=True) logger.error(f"Error fetching checkpoint model versions: {e}")
return web.json_response({ return web.Response(status=500, text=str(e))
"success": False,
"error": str(e)
}, status=500)
async def get_unet_roots(self, request: web.Request) -> web.Response:
"""Return the list of unet roots from config"""
try:
roots = config.unet_roots
return web.json_response({
"success": True,
"roots": roots
})
except Exception as e:
logger.error(f"Error getting unet roots: {e}", exc_info=True)
return web.json_response({
"success": False,
"error": str(e)
}, status=500)

View File

@@ -2,7 +2,6 @@ import logging
from aiohttp import web from aiohttp import web
from .base_model_routes import BaseModelRoutes from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.embedding_service import EmbeddingService from ..services.embedding_service import EmbeddingService
from ..services.service_registry import ServiceRegistry from ..services.service_registry import ServiceRegistry
@@ -13,18 +12,19 @@ class EmbeddingRoutes(BaseModelRoutes):
def __init__(self): def __init__(self):
"""Initialize Embedding routes with Embedding service""" """Initialize Embedding routes with Embedding service"""
super().__init__() # Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
self.template_name = "embeddings.html" self.template_name = "embeddings.html"
async def initialize_services(self): async def initialize_services(self):
"""Initialize services from ServiceRegistry""" """Initialize services from ServiceRegistry"""
embedding_scanner = await ServiceRegistry.get_embedding_scanner() embedding_scanner = await ServiceRegistry.get_embedding_scanner()
update_service = await ServiceRegistry.get_model_update_service() self.service = EmbeddingService(embedding_scanner)
self.service = EmbeddingService(embedding_scanner, update_service=update_service) self.civitai_client = await ServiceRegistry.get_civitai_client()
self.set_model_update_service(update_service)
# Initialize parent with the service
# Attach service dependencies super().__init__(self.service)
self.attach_service(self.service)
def setup_routes(self, app: web.Application): def setup_routes(self, app: web.Application):
"""Setup Embedding routes""" """Setup Embedding routes"""
@@ -34,18 +34,13 @@ class EmbeddingRoutes(BaseModelRoutes):
# Setup common routes with 'embeddings' prefix (includes page route) # Setup common routes with 'embeddings' prefix (includes page route)
super().setup_routes(app, 'embeddings') super().setup_routes(app, 'embeddings')
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str): def setup_specific_routes(self, app: web.Application, prefix: str):
"""Setup Embedding-specific routes""" """Setup Embedding-specific routes"""
# Embedding-specific CivitAI integration
app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_embedding)
# Embedding info by name # Embedding info by name
registrar.add_prefixed_route('GET', '/api/lm/{prefix}/info/{name}', prefix, self.get_embedding_info) app.router.add_get(f'/api/{prefix}/info/{{name}}', self.get_embedding_info)
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for Embedding"""
return model_type.lower() == 'textualinversion'
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "TextualInversion"
async def get_embedding_info(self, request: web.Request) -> web.Response: async def get_embedding_info(self, request: web.Request) -> web.Response:
"""Get detailed information for a specific embedding by name""" """Get detailed information for a specific embedding by name"""
@@ -61,3 +56,50 @@ class EmbeddingRoutes(BaseModelRoutes):
except Exception as e: except Exception as e:
logger.error(f"Error in get_embedding_info: {e}", exc_info=True) logger.error(f"Error in get_embedding_info: {e}", exc_info=True)
return web.json_response({"error": str(e)}, status=500) return web.json_response({"error": str(e)}, status=500)
async def get_civitai_versions_embedding(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai embedding model with local availability info"""
try:
model_id = request.match_info['model_id']
response = await self.civitai_client.get_model_versions(model_id)
if not response or not response.get('modelVersions'):
return web.Response(status=404, text="Model not found")
versions = response.get('modelVersions', [])
model_type = response.get('type', '')
# Check model type - should be TextualInversion (Embedding)
if model_type.lower() not in ['textualinversion', 'embedding']:
return web.json_response({
'error': f"Model type mismatch. Expected TextualInversion/Embedding, got {model_type}"
}, status=400)
# Check local availability for each version
for version in versions:
# Find the primary model file (type="Model" and primary=true) in the files list
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model' and file.get('primary') == True), None)
# If no primary file found, try to find any model file
if not model_file:
model_file = next((file for file in version.get('files', [])
if file.get('type') == 'Model'), None)
if model_file:
sha256 = model_file.get('hashes', {}).get('SHA256')
if sha256:
# Set existsLocally and localPath at the version level
version['existsLocally'] = self.service.has_hash(sha256)
if version['existsLocally']:
version['localPath'] = self.service.get_path_by_hash(sha256)
# Also set the model file size at the version level for easier access
version['modelSizeKB'] = model_file.get('sizeKB')
else:
# No model file found in this version
version['existsLocally'] = False
return web.json_response(versions)
except Exception as e:
logger.error(f"Error fetching embedding model versions: {e}")
return web.Response(status=500, text=str(e))

View File

@@ -1,64 +0,0 @@
"""Route registrar for example image endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative configuration for a HTTP route."""
method: str
path: str
handler_name: str
ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("POST", "/api/lm/download-example-images", "download_example_images"),
RouteDefinition("POST", "/api/lm/import-example-images", "import_example_images"),
RouteDefinition("GET", "/api/lm/example-images-status", "get_example_images_status"),
RouteDefinition("POST", "/api/lm/pause-example-images", "pause_example_images"),
RouteDefinition("POST", "/api/lm/resume-example-images", "resume_example_images"),
RouteDefinition("POST", "/api/lm/stop-example-images", "stop_example_images"),
RouteDefinition("POST", "/api/lm/open-example-images-folder", "open_example_images_folder"),
RouteDefinition("GET", "/api/lm/example-image-files", "get_example_image_files"),
RouteDefinition("GET", "/api/lm/has-example-images", "has_example_images"),
RouteDefinition("POST", "/api/lm/delete-example-image", "delete_example_image"),
RouteDefinition("POST", "/api/lm/force-download-example-images", "force_download_example_images"),
RouteDefinition("POST", "/api/lm/cleanup-example-image-folders", "cleanup_example_image_folders"),
RouteDefinition("POST", "/api/lm/example-images/set-nsfw-level", "set_example_image_nsfw_level"),
)
class ExampleImagesRouteRegistrar:
"""Bind declarative example image routes to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(
self,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = ROUTE_DEFINITIONS,
) -> None:
"""Register each route definition using the supplied handlers."""
for definition in definitions:
handler = handler_lookup[definition.handler_name]
self._bind_route(definition.method, definition.path, handler)
def _bind_route(self, method: str, path: str, handler: Callable[[web.Request], object]) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -1,88 +1,67 @@
from __future__ import annotations
import logging import logging
from typing import Callable, Mapping from ..utils.example_images_download_manager import DownloadManager
from aiohttp import web
from .example_images_route_registrar import ExampleImagesRouteRegistrar
from .handlers.example_images_handlers import (
ExampleImagesDownloadHandler,
ExampleImagesFileHandler,
ExampleImagesHandlerSet,
ExampleImagesManagementHandler,
)
from ..services.use_cases.example_images import (
DownloadExampleImagesUseCase,
ImportExampleImagesUseCase,
)
from ..utils.example_images_download_manager import (
DownloadManager,
get_default_download_manager,
)
from ..utils.example_images_file_manager import ExampleImagesFileManager
from ..utils.example_images_processor import ExampleImagesProcessor from ..utils.example_images_processor import ExampleImagesProcessor
from ..services.example_images_cleanup_service import ExampleImagesCleanupService from ..utils.example_images_file_manager import ExampleImagesFileManager
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class ExampleImagesRoutes: class ExampleImagesRoutes:
"""Route controller for example image endpoints.""" """Routes for example images related functionality"""
@staticmethod
def setup_routes(app):
"""Register example images routes"""
app.router.add_post('/api/download-example-images', ExampleImagesRoutes.download_example_images)
app.router.add_post('/api/import-example-images', ExampleImagesRoutes.import_example_images)
app.router.add_get('/api/example-images-status', ExampleImagesRoutes.get_example_images_status)
app.router.add_post('/api/pause-example-images', ExampleImagesRoutes.pause_example_images)
app.router.add_post('/api/resume-example-images', ExampleImagesRoutes.resume_example_images)
app.router.add_post('/api/open-example-images-folder', ExampleImagesRoutes.open_example_images_folder)
app.router.add_get('/api/example-image-files', ExampleImagesRoutes.get_example_image_files)
app.router.add_get('/api/has-example-images', ExampleImagesRoutes.has_example_images)
app.router.add_post('/api/delete-example-image', ExampleImagesRoutes.delete_example_image)
def __init__( @staticmethod
self, async def download_example_images(request):
*, """Download example images for models from Civitai"""
ws_manager, return await DownloadManager.start_download(request)
download_manager: DownloadManager | None = None,
processor=ExampleImagesProcessor,
file_manager=ExampleImagesFileManager,
cleanup_service: ExampleImagesCleanupService | None = None,
) -> None:
if ws_manager is None:
raise ValueError("ws_manager is required")
self._download_manager = download_manager or get_default_download_manager(ws_manager)
self._processor = processor
self._file_manager = file_manager
self._cleanup_service = cleanup_service or ExampleImagesCleanupService()
self._handler_set: ExampleImagesHandlerSet | None = None
self._handler_mapping: Mapping[str, Callable[[web.Request], web.StreamResponse]] | None = None
@classmethod @staticmethod
def setup_routes(cls, app: web.Application, *, ws_manager) -> None: async def get_example_images_status(request):
"""Register routes on the given aiohttp application using default wiring.""" """Get the current status of example images download"""
return await DownloadManager.get_status(request)
controller = cls(ws_manager=ws_manager) @staticmethod
controller.register(app) async def pause_example_images(request):
"""Pause the example images download"""
return await DownloadManager.pause_download(request)
def register(self, app: web.Application) -> None: @staticmethod
"""Bind the controller's handlers to the aiohttp router.""" async def resume_example_images(request):
"""Resume the example images download"""
return await DownloadManager.resume_download(request)
@staticmethod
async def open_example_images_folder(request):
"""Open the example images folder for a specific model"""
return await ExampleImagesFileManager.open_folder(request)
registrar = ExampleImagesRouteRegistrar(app) @staticmethod
registrar.register_routes(self.to_route_mapping()) async def get_example_image_files(request):
"""Get list of example image files for a specific model"""
return await ExampleImagesFileManager.get_files(request)
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]: @staticmethod
"""Return the registrar-compatible mapping of handler names to callables.""" async def import_example_images(request):
"""Import local example images for a model"""
return await ExampleImagesProcessor.import_images(request)
@staticmethod
async def has_example_images(request):
"""Check if example images folder exists and is not empty for a model"""
return await ExampleImagesFileManager.has_images(request)
if self._handler_mapping is None: @staticmethod
handler_set = self._build_handler_set() async def delete_example_image(request):
self._handler_set = handler_set """Delete a custom example image for a model"""
self._handler_mapping = handler_set.to_route_mapping() return await ExampleImagesProcessor.delete_custom_image(request)
return self._handler_mapping
def _build_handler_set(self) -> ExampleImagesHandlerSet:
logger.debug("Building ExampleImagesHandlerSet with %s, %s, %s", self._download_manager, self._processor, self._file_manager)
download_use_case = DownloadExampleImagesUseCase(download_manager=self._download_manager)
download_handler = ExampleImagesDownloadHandler(download_use_case, self._download_manager)
import_use_case = ImportExampleImagesUseCase(processor=self._processor)
management_handler = ExampleImagesManagementHandler(
import_use_case,
self._processor,
self._cleanup_service,
)
file_handler = ExampleImagesFileHandler(self._file_manager)
return ExampleImagesHandlerSet(
download=download_handler,
management=management_handler,
files=file_handler,
)

View File

@@ -1,171 +0,0 @@
"""Handler set for example image routes."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Mapping
from aiohttp import web
from ...services.use_cases.example_images import (
DownloadExampleImagesConfigurationError,
DownloadExampleImagesInProgressError,
DownloadExampleImagesUseCase,
ImportExampleImagesUseCase,
ImportExampleImagesValidationError,
)
from ...utils.example_images_download_manager import (
DownloadConfigurationError,
DownloadInProgressError,
DownloadNotRunningError,
ExampleImagesDownloadError,
)
from ...utils.example_images_processor import ExampleImagesImportError
class ExampleImagesDownloadHandler:
"""HTTP adapters for download-related example image endpoints."""
def __init__(
self,
download_use_case: DownloadExampleImagesUseCase,
download_manager,
) -> None:
self._download_use_case = download_use_case
self._download_manager = download_manager
async def download_example_images(self, request: web.Request) -> web.StreamResponse:
try:
payload = await request.json()
result = await self._download_use_case.execute(payload)
return web.json_response(result)
except DownloadExampleImagesInProgressError as exc:
response = {
'success': False,
'error': str(exc),
'status': exc.progress,
}
return web.json_response(response, status=400)
except DownloadExampleImagesConfigurationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesDownloadError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
async def get_example_images_status(self, request: web.Request) -> web.StreamResponse:
result = await self._download_manager.get_status(request)
return web.json_response(result)
async def pause_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.pause_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def resume_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.resume_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def stop_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._download_manager.stop_download(request)
return web.json_response(result)
except DownloadNotRunningError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
async def force_download_example_images(self, request: web.Request) -> web.StreamResponse:
try:
payload = await request.json()
result = await self._download_manager.start_force_download(payload)
return web.json_response(result)
except DownloadInProgressError as exc:
response = {
'success': False,
'error': str(exc),
'status': exc.progress_snapshot,
}
return web.json_response(response, status=400)
except DownloadConfigurationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesDownloadError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
class ExampleImagesManagementHandler:
"""HTTP adapters for import/delete endpoints."""
def __init__(self, import_use_case: ImportExampleImagesUseCase, processor, cleanup_service) -> None:
self._import_use_case = import_use_case
self._processor = processor
self._cleanup_service = cleanup_service
async def import_example_images(self, request: web.Request) -> web.StreamResponse:
try:
result = await self._import_use_case.execute(request)
return web.json_response(result)
except ImportExampleImagesValidationError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=400)
except ExampleImagesImportError as exc:
return web.json_response({'success': False, 'error': str(exc)}, status=500)
async def delete_example_image(self, request: web.Request) -> web.StreamResponse:
return await self._processor.delete_custom_image(request)
async def set_example_image_nsfw_level(self, request: web.Request) -> web.StreamResponse:
return await self._processor.set_example_image_nsfw_level(request)
async def cleanup_example_image_folders(self, request: web.Request) -> web.StreamResponse:
result = await self._cleanup_service.cleanup_example_image_folders()
if result.get('success') or result.get('partial_success'):
return web.json_response(result, status=200)
error_code = result.get('error_code')
status = 400 if error_code in {'path_not_configured', 'path_not_found'} else 500
return web.json_response(result, status=status)
class ExampleImagesFileHandler:
"""HTTP adapters for filesystem-centric endpoints."""
def __init__(self, file_manager) -> None:
self._file_manager = file_manager
async def open_example_images_folder(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.open_folder(request)
async def get_example_image_files(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.get_files(request)
async def has_example_images(self, request: web.Request) -> web.StreamResponse:
return await self._file_manager.has_images(request)
@dataclass(frozen=True)
class ExampleImagesHandlerSet:
"""Aggregate of handlers exposed to the registrar."""
download: ExampleImagesDownloadHandler
management: ExampleImagesManagementHandler
files: ExampleImagesFileHandler
def to_route_mapping(self) -> Mapping[str, Callable[[web.Request], web.StreamResponse]]:
"""Flatten handler methods into the registrar mapping."""
return {
"download_example_images": self.download.download_example_images,
"get_example_images_status": self.download.get_example_images_status,
"pause_example_images": self.download.pause_example_images,
"resume_example_images": self.download.resume_example_images,
"stop_example_images": self.download.stop_example_images,
"force_download_example_images": self.download.force_download_example_images,
"import_example_images": self.management.import_example_images,
"delete_example_image": self.management.delete_example_image,
"set_example_image_nsfw_level": self.management.set_example_image_nsfw_level,
"cleanup_example_image_folders": self.management.cleanup_example_image_folders,
"open_example_images_folder": self.files.open_example_images_folder,
"get_example_image_files": self.files.get_example_image_files,
"has_example_images": self.files.has_example_images,
}

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -1,57 +0,0 @@
"""Handlers responsible for serving preview assets dynamically."""
from __future__ import annotations
import logging
import urllib.parse
from pathlib import Path
from aiohttp import web
from ...config import config as global_config
logger = logging.getLogger(__name__)
class PreviewHandler:
"""Serve preview assets for the active library at request time."""
def __init__(self, *, config=global_config) -> None:
self._config = config
async def serve_preview(self, request: web.Request) -> web.StreamResponse:
"""Return the preview file referenced by the encoded ``path`` query."""
raw_path = request.query.get("path", "")
if not raw_path:
raise web.HTTPBadRequest(text="Missing 'path' query parameter")
try:
decoded_path = urllib.parse.unquote(raw_path)
except Exception as exc: # pragma: no cover - defensive guard
logger.debug("Failed to decode preview path %s: %s", raw_path, exc)
raise web.HTTPBadRequest(text="Invalid preview path encoding") from exc
normalized = decoded_path.replace("\\", "/")
candidate = Path(normalized)
try:
resolved = candidate.expanduser().resolve(strict=False)
except Exception as exc:
logger.debug("Failed to resolve preview path %s: %s", normalized, exc)
raise web.HTTPBadRequest(text="Unable to resolve preview path") from exc
resolved_str = str(resolved)
# TODO: Temporarily disabled path validation due to issues #772 and #774
# Re-enable after fixing preview root path handling
# if not self._config.is_preview_path_allowed(resolved_str):
# raise web.HTTPForbidden(text="Preview path is not within an allowed directory")
if not resolved.is_file():
logger.debug("Preview file not found at %s", resolved_str)
raise web.HTTPNotFound(text="Preview file not found")
# aiohttp's FileResponse handles range requests and content headers for us.
return web.FileResponse(path=resolved, chunk_size=256 * 1024)
__all__ = ["PreviewHandler"]

File diff suppressed because it is too large Load Diff

View File

@@ -5,323 +5,479 @@ from typing import Dict
from server import PromptServer # type: ignore from server import PromptServer # type: ignore
from .base_model_routes import BaseModelRoutes from .base_model_routes import BaseModelRoutes
from .model_route_registrar import ModelRouteRegistrar
from ..services.lora_service import LoraService from ..services.lora_service import LoraService
from ..services.service_registry import ServiceRegistry from ..services.service_registry import ServiceRegistry
from ..utils.routes_common import ModelRouteUtils
from ..utils.utils import get_lora_info from ..utils.utils import get_lora_info
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class LoraRoutes(BaseModelRoutes): class LoraRoutes(BaseModelRoutes):
"""LoRA-specific route controller""" """LoRA-specific route controller"""
def __init__(self): def __init__(self):
"""Initialize LoRA routes with LoRA service""" """Initialize LoRA routes with LoRA service"""
super().__init__() # Service will be initialized later via setup_routes
self.service = None
self.civitai_client = None
self.template_name = "loras.html" self.template_name = "loras.html"
async def initialize_services(self): async def initialize_services(self):
"""Initialize services from ServiceRegistry""" """Initialize services from ServiceRegistry"""
lora_scanner = await ServiceRegistry.get_lora_scanner() lora_scanner = await ServiceRegistry.get_lora_scanner()
update_service = await ServiceRegistry.get_model_update_service() self.service = LoraService(lora_scanner)
self.service = LoraService(lora_scanner, update_service=update_service) self.civitai_client = await ServiceRegistry.get_civitai_client()
self.set_model_update_service(update_service)
# Initialize parent with the service
# Attach service dependencies super().__init__(self.service)
self.attach_service(self.service)
def setup_routes(self, app: web.Application): def setup_routes(self, app: web.Application):
"""Setup LoRA routes""" """Setup LoRA routes"""
# Schedule service initialization on app startup # Schedule service initialization on app startup
app.on_startup.append(lambda _: self.initialize_services()) app.on_startup.append(lambda _: self.initialize_services())
# Setup common routes with 'loras' prefix (includes page route) # Setup common routes with 'loras' prefix (includes page route)
super().setup_routes(app, "loras") super().setup_routes(app, 'loras')
def setup_specific_routes(self, registrar: ModelRouteRegistrar, prefix: str): def setup_specific_routes(self, app: web.Application, prefix: str):
"""Setup LoRA-specific routes""" """Setup LoRA-specific routes"""
# LoRA-specific query routes # LoRA-specific query routes
registrar.add_prefixed_route( app.router.add_get(f'/api/{prefix}/letter-counts', self.get_letter_counts)
"GET", "/api/lm/{prefix}/letter-counts", prefix, self.get_letter_counts app.router.add_get(f'/api/{prefix}/get-notes', self.get_lora_notes)
) app.router.add_get(f'/api/{prefix}/get-trigger-words', self.get_lora_trigger_words)
registrar.add_prefixed_route( app.router.add_get(f'/api/{prefix}/preview-url', self.get_lora_preview_url)
"GET", app.router.add_get(f'/api/{prefix}/civitai-url', self.get_lora_civitai_url)
"/api/lm/{prefix}/get-trigger-words", app.router.add_get(f'/api/{prefix}/model-description', self.get_lora_model_description)
prefix,
self.get_lora_trigger_words, # LoRA-specific management routes
) app.router.add_post(f'/api/{prefix}/move_model', self.move_model)
registrar.add_prefixed_route( app.router.add_post(f'/api/{prefix}/move_models_bulk', self.move_models_bulk)
"GET",
"/api/lm/{prefix}/usage-tips-by-path", # CivitAI integration with LoRA-specific validation
prefix, app.router.add_get(f'/api/{prefix}/civitai/versions/{{model_id}}', self.get_civitai_versions_lora)
self.get_lora_usage_tips_by_path, app.router.add_get(f'/api/{prefix}/civitai/model/version/{{modelVersionId}}', self.get_civitai_model_by_version)
) app.router.add_get(f'/api/{prefix}/civitai/model/hash/{{hash}}', self.get_civitai_model_by_hash)
# Randomizer routes
registrar.add_prefixed_route(
"POST", "/api/lm/{prefix}/random-sample", prefix, self.get_random_loras
)
# ComfyUI integration # ComfyUI integration
registrar.add_prefixed_route( app.router.add_post(f'/api/{prefix}/get_trigger_words', self.get_trigger_words)
"POST", "/api/lm/{prefix}/get_trigger_words", prefix, self.get_trigger_words
)
def _parse_specific_params(self, request: web.Request) -> Dict: def _parse_specific_params(self, request: web.Request) -> Dict:
"""Parse LoRA-specific parameters""" """Parse LoRA-specific parameters"""
params = {} params = {}
# LoRA-specific parameters # LoRA-specific parameters
if "first_letter" in request.query: if 'first_letter' in request.query:
params["first_letter"] = request.query.get("first_letter") params['first_letter'] = request.query.get('first_letter')
# Handle fuzzy search parameter name variation # Handle fuzzy search parameter name variation
if request.query.get("fuzzy") == "true": if request.query.get('fuzzy') == 'true':
params["fuzzy_search"] = True params['fuzzy_search'] = True
# Handle additional filter parameters for LoRAs # Handle additional filter parameters for LoRAs
if "lora_hash" in request.query: if 'lora_hash' in request.query:
if not params.get("hash_filters"): if not params.get('hash_filters'):
params["hash_filters"] = {} params['hash_filters'] = {}
params["hash_filters"]["single_hash"] = request.query["lora_hash"].lower() params['hash_filters']['single_hash'] = request.query['lora_hash'].lower()
elif "lora_hashes" in request.query: elif 'lora_hashes' in request.query:
if not params.get("hash_filters"): if not params.get('hash_filters'):
params["hash_filters"] = {} params['hash_filters'] = {}
params["hash_filters"]["multiple_hashes"] = [ params['hash_filters']['multiple_hashes'] = [h.lower() for h in request.query['lora_hashes'].split(',')]
h.lower() for h in request.query["lora_hashes"].split(",")
]
return params return params
def _validate_civitai_model_type(self, model_type: str) -> bool:
"""Validate CivitAI model type for LoRA"""
from ..utils.constants import VALID_LORA_TYPES
return model_type.lower() in VALID_LORA_TYPES
def _get_expected_model_types(self) -> str:
"""Get expected model types string for error messages"""
return "LORA, LoCon, or DORA"
# LoRA-specific route handlers # LoRA-specific route handlers
async def get_letter_counts(self, request: web.Request) -> web.Response: async def get_letter_counts(self, request: web.Request) -> web.Response:
"""Get count of LoRAs for each letter of the alphabet""" """Get count of LoRAs for each letter of the alphabet"""
try: try:
letter_counts = await self.service.get_letter_counts() letter_counts = await self.service.get_letter_counts()
return web.json_response({"success": True, "letter_counts": letter_counts}) return web.json_response({
'success': True,
'letter_counts': letter_counts
})
except Exception as e: except Exception as e:
logger.error(f"Error getting letter counts: {e}") logger.error(f"Error getting letter counts: {e}")
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def get_lora_notes(self, request: web.Request) -> web.Response: async def get_lora_notes(self, request: web.Request) -> web.Response:
"""Get notes for a specific LoRA file""" """Get notes for a specific LoRA file"""
try: try:
lora_name = request.query.get("name") lora_name = request.query.get('name')
if not lora_name: if not lora_name:
return web.Response(text="Lora file name is required", status=400) return web.Response(text='Lora file name is required', status=400)
notes = await self.service.get_lora_notes(lora_name) notes = await self.service.get_lora_notes(lora_name)
if notes is not None: if notes is not None:
return web.json_response({"success": True, "notes": notes}) return web.json_response({
'success': True,
'notes': notes
})
else: else:
return web.json_response( return web.json_response({
{"success": False, "error": "LoRA not found in cache"}, status=404 'success': False,
) 'error': 'LoRA not found in cache'
}, status=404)
except Exception as e: except Exception as e:
logger.error(f"Error getting lora notes: {e}", exc_info=True) logger.error(f"Error getting lora notes: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def get_lora_trigger_words(self, request: web.Request) -> web.Response: async def get_lora_trigger_words(self, request: web.Request) -> web.Response:
"""Get trigger words for a specific LoRA file""" """Get trigger words for a specific LoRA file"""
try: try:
lora_name = request.query.get("name") lora_name = request.query.get('name')
if not lora_name: if not lora_name:
return web.Response(text="Lora file name is required", status=400) return web.Response(text='Lora file name is required', status=400)
trigger_words = await self.service.get_lora_trigger_words(lora_name) trigger_words = await self.service.get_lora_trigger_words(lora_name)
return web.json_response({"success": True, "trigger_words": trigger_words}) return web.json_response({
'success': True,
'trigger_words': trigger_words
})
except Exception as e: except Exception as e:
logger.error(f"Error getting lora trigger words: {e}", exc_info=True) logger.error(f"Error getting lora trigger words: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
'success': False,
async def get_lora_usage_tips_by_path(self, request: web.Request) -> web.Response: 'error': str(e)
"""Get usage tips for a LoRA by its relative path""" }, status=500)
try:
relative_path = request.query.get("relative_path")
if not relative_path:
return web.Response(text="Relative path is required", status=400)
usage_tips = await self.service.get_lora_usage_tips_by_relative_path(
relative_path
)
return web.json_response({"success": True, "usage_tips": usage_tips or ""})
except Exception as e:
logger.error(f"Error getting lora usage tips by path: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500)
async def get_lora_preview_url(self, request: web.Request) -> web.Response: async def get_lora_preview_url(self, request: web.Request) -> web.Response:
"""Get the static preview URL for a LoRA file""" """Get the static preview URL for a LoRA file"""
try: try:
lora_name = request.query.get("name") lora_name = request.query.get('name')
if not lora_name: if not lora_name:
return web.Response(text="Lora file name is required", status=400) return web.Response(text='Lora file name is required', status=400)
preview_url = await self.service.get_lora_preview_url(lora_name) preview_url = await self.service.get_lora_preview_url(lora_name)
if preview_url: if preview_url:
return web.json_response({"success": True, "preview_url": preview_url}) return web.json_response({
'success': True,
'preview_url': preview_url
})
else: else:
return web.json_response( return web.json_response({
{ 'success': False,
"success": False, 'error': 'No preview URL found for the specified lora'
"error": "No preview URL found for the specified lora", }, status=404)
},
status=404,
)
except Exception as e: except Exception as e:
logger.error(f"Error getting lora preview URL: {e}", exc_info=True) logger.error(f"Error getting lora preview URL: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def get_lora_civitai_url(self, request: web.Request) -> web.Response: async def get_lora_civitai_url(self, request: web.Request) -> web.Response:
"""Get the Civitai URL for a LoRA file""" """Get the Civitai URL for a LoRA file"""
try: try:
lora_name = request.query.get("name") lora_name = request.query.get('name')
if not lora_name: if not lora_name:
return web.Response(text="Lora file name is required", status=400) return web.Response(text='Lora file name is required', status=400)
result = await self.service.get_lora_civitai_url(lora_name) result = await self.service.get_lora_civitai_url(lora_name)
if result["civitai_url"]: if result['civitai_url']:
return web.json_response({"success": True, **result}) return web.json_response({
'success': True,
**result
})
else: else:
return web.json_response( return web.json_response({
{ 'success': False,
"success": False, 'error': 'No Civitai data found for the specified lora'
"error": "No Civitai data found for the specified lora", }, status=404)
},
status=404,
)
except Exception as e: except Exception as e:
logger.error(f"Error getting lora Civitai URL: {e}", exc_info=True) logger.error(f"Error getting lora Civitai URL: {e}", exc_info=True)
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
'success': False,
async def get_random_loras(self, request: web.Request) -> web.Response: 'error': str(e)
"""Get random LoRAs based on filters and strength ranges""" }, status=500)
# CivitAI integration methods
async def get_civitai_versions_lora(self, request: web.Request) -> web.Response:
"""Get available versions for a Civitai LoRA model with local availability info"""
try: try:
json_data = await request.json() model_id = request.match_info['model_id']
response = await self.civitai_client.get_model_versions(model_id)
# Parse parameters if not response or not response.get('modelVersions'):
count = json_data.get("count", 5) return web.Response(status=404, text="Model not found")
count_min = json_data.get("count_min")
count_max = json_data.get("count_max") versions = response.get('modelVersions', [])
model_strength_min = float(json_data.get("model_strength_min", 0.0)) model_type = response.get('type', '')
model_strength_max = float(json_data.get("model_strength_max", 1.0))
use_same_clip_strength = json_data.get("use_same_clip_strength", True) # Check model type - should be LORA, LoCon, or DORA
clip_strength_min = float(json_data.get("clip_strength_min", 0.0)) from ..utils.constants import VALID_LORA_TYPES
clip_strength_max = float(json_data.get("clip_strength_max", 1.0)) if model_type.lower() not in VALID_LORA_TYPES:
locked_loras = json_data.get("locked_loras", []) return web.json_response({
pool_config = json_data.get("pool_config") 'error': f"Model type mismatch. Expected LORA or LoCon, got {model_type}"
use_recommended_strength = json_data.get("use_recommended_strength", False) }, status=400)
recommended_strength_scale_min = float(
json_data.get("recommended_strength_scale_min", 0.5) # Check local availability for each version
) for version in versions:
recommended_strength_scale_max = float( # Find the model file (type="Model") in the files list
json_data.get("recommended_strength_scale_max", 1.0) model_file = next((file for file in version.get('files', [])
) if file.get('type') == 'Model'), None)
# Determine target count if model_file:
if count_min is not None and count_max is not None: sha256 = model_file.get('hashes', {}).get('SHA256')
import random if sha256:
# Set existsLocally and localPath at the version level
target_count = random.randint(count_min, count_max) version['existsLocally'] = self.service.has_hash(sha256)
else: if version['existsLocally']:
target_count = count version['localPath'] = self.service.get_path_by_hash(sha256)
# Validate parameters # Also set the model file size at the version level for easier access
if target_count < 1 or target_count > 100: version['modelSizeKB'] = model_file.get('sizeKB')
return web.json_response( else:
{"success": False, "error": "Count must be between 1 and 100"}, # No model file found in this version
status=400, version['existsLocally'] = False
)
return web.json_response(versions)
if model_strength_min < -10 or model_strength_max > 10:
return web.json_response(
{
"success": False,
"error": "Model strength must be between -10 and 10",
},
status=400,
)
# Get random LoRAs from service
result_loras = await self.service.get_random_loras(
count=target_count,
model_strength_min=model_strength_min,
model_strength_max=model_strength_max,
use_same_clip_strength=use_same_clip_strength,
clip_strength_min=clip_strength_min,
clip_strength_max=clip_strength_max,
locked_loras=locked_loras,
pool_config=pool_config,
use_recommended_strength=use_recommended_strength,
recommended_strength_scale_min=recommended_strength_scale_min,
recommended_strength_scale_max=recommended_strength_scale_max,
)
return web.json_response(
{"success": True, "loras": result_loras, "count": len(result_loras)}
)
except ValueError as e:
logger.error(f"Invalid parameter for random LoRAs: {e}")
return web.json_response({"success": False, "error": str(e)}, status=400)
except Exception as e: except Exception as e:
logger.error(f"Error getting random LoRAs: {e}", exc_info=True) logger.error(f"Error fetching LoRA model versions: {e}")
return web.json_response({"success": False, "error": str(e)}, status=500) return web.Response(status=500, text=str(e))
async def get_civitai_model_by_version(self, request: web.Request) -> web.Response:
"""Get CivitAI model details by model version ID"""
try:
model_version_id = request.match_info.get('modelVersionId')
# Get model details from Civitai API
model, error_msg = await self.civitai_client.get_model_version_info(model_version_id)
if not model:
# Log warning for failed model retrieval
logger.warning(f"Failed to fetch model version {model_version_id}: {error_msg}")
# Determine status code based on error message
status_code = 404 if error_msg and "not found" in error_msg.lower() else 500
return web.json_response({
"success": False,
"error": error_msg or "Failed to fetch model information"
}, status=status_code)
return web.json_response(model)
except Exception as e:
logger.error(f"Error fetching model details: {e}")
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
async def get_civitai_model_by_hash(self, request: web.Request) -> web.Response:
"""Get CivitAI model details by hash"""
try:
hash = request.match_info.get('hash')
model = await self.civitai_client.get_model_by_hash(hash)
return web.json_response(model)
except Exception as e:
logger.error(f"Error fetching model details by hash: {e}")
return web.json_response({
"success": False,
"error": str(e)
}, status=500)
# Model management methods
async def move_model(self, request: web.Request) -> web.Response:
"""Handle model move request"""
try:
data = await request.json()
file_path = data.get('file_path') # full path of the model file
target_path = data.get('target_path') # folder path to move the model to
if not file_path or not target_path:
return web.Response(text='File path and target path are required', status=400)
# Check if source and destination are the same
import os
source_dir = os.path.dirname(file_path)
if os.path.normpath(source_dir) == os.path.normpath(target_path):
logger.info(f"Source and target directories are the same: {source_dir}")
return web.json_response({'success': True, 'message': 'Source and target directories are the same'})
# Check if target file already exists
file_name = os.path.basename(file_path)
target_file_path = os.path.join(target_path, file_name).replace(os.sep, '/')
if os.path.exists(target_file_path):
return web.json_response({
'success': False,
'error': f"Target file already exists: {target_file_path}"
}, status=409) # 409 Conflict
# Call scanner to handle the move operation
success = await self.service.scanner.move_model(file_path, target_path)
if success:
return web.json_response({'success': True, 'new_file_path': target_file_path})
else:
return web.Response(text='Failed to move model', status=500)
except Exception as e:
logger.error(f"Error moving model: {e}", exc_info=True)
return web.Response(text=str(e), status=500)
async def move_models_bulk(self, request: web.Request) -> web.Response:
"""Handle bulk model move request"""
try:
data = await request.json()
file_paths = data.get('file_paths', []) # list of full paths of the model files
target_path = data.get('target_path') # folder path to move the models to
if not file_paths or not target_path:
return web.Response(text='File paths and target path are required', status=400)
results = []
import os
for file_path in file_paths:
# Check if source and destination are the same
source_dir = os.path.dirname(file_path)
if os.path.normpath(source_dir) == os.path.normpath(target_path):
results.append({
"path": file_path,
"success": True,
"message": "Source and target directories are the same"
})
continue
# Check if target file already exists
file_name = os.path.basename(file_path)
target_file_path = os.path.join(target_path, file_name).replace(os.sep, '/')
if os.path.exists(target_file_path):
results.append({
"path": file_path,
"success": False,
"message": f"Target file already exists: {target_file_path}"
})
continue
# Try to move the model
success = await self.service.scanner.move_model(file_path, target_path)
results.append({
"path": file_path,
"success": success,
"message": "Success" if success else "Failed to move model"
})
# Count successes and failures
success_count = sum(1 for r in results if r["success"])
failure_count = len(results) - success_count
return web.json_response({
'success': True,
'message': f'Moved {success_count} of {len(file_paths)} models',
'results': results,
'success_count': success_count,
'failure_count': failure_count
})
except Exception as e:
logger.error(f"Error moving models in bulk: {e}", exc_info=True)
return web.Response(text=str(e), status=500)
async def get_lora_model_description(self, request: web.Request) -> web.Response:
"""Get model description for a Lora model"""
try:
# Get parameters
model_id = request.query.get('model_id')
file_path = request.query.get('file_path')
if not model_id:
return web.json_response({
'success': False,
'error': 'Model ID is required'
}, status=400)
# Check if we already have the description stored in metadata
description = None
tags = []
creator = {}
if file_path:
import os
from ..utils.metadata_manager import MetadataManager
metadata_path = os.path.splitext(file_path)[0] + '.metadata.json'
metadata = await ModelRouteUtils.load_local_metadata(metadata_path)
description = metadata.get('modelDescription')
tags = metadata.get('tags', [])
creator = metadata.get('creator', {})
# If description is not in metadata, fetch from CivitAI
if not description:
logger.info(f"Fetching model metadata for model ID: {model_id}")
model_metadata, _ = await self.civitai_client.get_model_metadata(model_id)
if model_metadata:
description = model_metadata.get('description')
tags = model_metadata.get('tags', [])
creator = model_metadata.get('creator', {})
# Save the metadata to file if we have a file path and got metadata
if file_path:
try:
metadata_path = os.path.splitext(file_path)[0] + '.metadata.json'
metadata = await ModelRouteUtils.load_local_metadata(metadata_path)
metadata['modelDescription'] = description
metadata['tags'] = tags
# Ensure the civitai dict exists
if 'civitai' not in metadata:
metadata['civitai'] = {}
# Store creator in the civitai nested structure
metadata['civitai']['creator'] = creator
await MetadataManager.save_metadata(file_path, metadata, True)
except Exception as e:
logger.error(f"Error saving model metadata: {e}")
return web.json_response({
'success': True,
'description': description or "<p>No model description available.</p>",
'tags': tags,
'creator': creator
})
except Exception as e:
logger.error(f"Error getting model metadata: {e}")
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
async def get_trigger_words(self, request: web.Request) -> web.Response: async def get_trigger_words(self, request: web.Request) -> web.Response:
"""Get trigger words for specified LoRA models""" """Get trigger words for specified LoRA models"""
try: try:
json_data = await request.json() json_data = await request.json()
lora_names = json_data.get("lora_names", []) lora_names = json_data.get("lora_names", [])
node_ids = json_data.get("node_ids", []) node_ids = json_data.get("node_ids", [])
all_trigger_words = [] all_trigger_words = []
for lora_name in lora_names: for lora_name in lora_names:
_, trigger_words = get_lora_info(lora_name) _, trigger_words = get_lora_info(lora_name)
all_trigger_words.extend(trigger_words) all_trigger_words.extend(trigger_words)
# Format the trigger words # Format the trigger words
trigger_words_text = ( trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
",, ".join(all_trigger_words) if all_trigger_words else ""
)
# Send update to all connected trigger word toggle nodes # Send update to all connected trigger word toggle nodes
for entry in node_ids: for node_id in node_ids:
node_identifier = entry PromptServer.instance.send_sync("trigger_word_update", {
graph_identifier = None "id": node_id,
if isinstance(entry, dict): "message": trigger_words_text
node_identifier = entry.get("node_id") })
graph_identifier = entry.get("graph_id")
try:
parsed_node_id = int(node_identifier)
except (TypeError, ValueError):
parsed_node_id = node_identifier
payload = {"id": parsed_node_id, "message": trigger_words_text}
if graph_identifier is not None:
payload["graph_id"] = str(graph_identifier)
PromptServer.instance.send_sync("trigger_word_update", payload)
return web.json_response({"success": True}) return web.json_response({"success": True})
except Exception as e: except Exception as e:
logger.error(f"Error getting trigger words: {e}") logger.error(f"Error getting trigger words: {e}")
return web.json_response({"success": False, "error": str(e)}, status=500) return web.json_response({
"success": False,
"error": str(e)
}, status=500)

View File

@@ -1,73 +0,0 @@
"""Route registrar for miscellaneous endpoints.
This module mirrors the model route registrar architecture so that
miscellaneous endpoints share a consistent registration flow.
"""
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a HTTP route."""
method: str
path: str
handler_name: str
MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/settings", "get_settings"),
RouteDefinition("POST", "/api/lm/settings", "update_settings"),
RouteDefinition("GET", "/api/lm/priority-tags", "get_priority_tags"),
RouteDefinition("GET", "/api/lm/settings/libraries", "get_settings_libraries"),
RouteDefinition("POST", "/api/lm/settings/libraries/activate", "activate_library"),
RouteDefinition("GET", "/api/lm/health-check", "health_check"),
RouteDefinition("POST", "/api/lm/open-file-location", "open_file_location"),
RouteDefinition("POST", "/api/lm/update-usage-stats", "update_usage_stats"),
RouteDefinition("GET", "/api/lm/get-usage-stats", "get_usage_stats"),
RouteDefinition("POST", "/api/lm/update-lora-code", "update_lora_code"),
RouteDefinition("GET", "/api/lm/trained-words", "get_trained_words"),
RouteDefinition("GET", "/api/lm/model-example-files", "get_model_example_files"),
RouteDefinition("POST", "/api/lm/register-nodes", "register_nodes"),
RouteDefinition("POST", "/api/lm/update-node-widget", "update_node_widget"),
RouteDefinition("GET", "/api/lm/get-registry", "get_registry"),
RouteDefinition("GET", "/api/lm/check-model-exists", "check_model_exists"),
RouteDefinition("GET", "/api/lm/civitai/user-models", "get_civitai_user_models"),
RouteDefinition("POST", "/api/lm/download-metadata-archive", "download_metadata_archive"),
RouteDefinition("POST", "/api/lm/remove-metadata-archive", "remove_metadata_archive"),
RouteDefinition("GET", "/api/lm/metadata-archive-status", "get_metadata_archive_status"),
RouteDefinition("GET", "/api/lm/model-versions-status", "get_model_versions_status"),
RouteDefinition("POST", "/api/lm/settings/open-location", "open_settings_location"),
)
class MiscRouteRegistrar:
"""Bind miscellaneous route definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(
self,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = MISC_ROUTE_DEFINITIONS,
) -> None:
for definition in definitions:
self._bind(definition.method, definition.path, handler_lookup[definition.handler_name])
def _bind(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -1,135 +1,707 @@
"""Route controller for miscellaneous endpoints.""" import json
from __future__ import annotations
import logging import logging
import os import os
from typing import Awaitable, Callable, Mapping import sys
import threading
import asyncio
from server import PromptServer # type: ignore
from aiohttp import web from aiohttp import web
from server import PromptServer # type: ignore from ..services.settings_manager import settings
from ..services.metadata_service import (
get_metadata_archive_manager,
get_metadata_provider,
update_metadata_providers,
)
from ..services.settings_manager import get_settings_manager
from ..services.downloader import get_downloader
from ..utils.usage_stats import UsageStats from ..utils.usage_stats import UsageStats
from .handlers.misc_handlers import ( from ..utils.lora_metadata import extract_trained_words
FileSystemHandler, from ..config import config
HealthCheckHandler, from ..utils.constants import SUPPORTED_MEDIA_EXTENSIONS, NODE_TYPES, DEFAULT_NODE_COLOR
LoraCodeHandler, from ..services.service_registry import ServiceRegistry
MetadataArchiveHandler, import re
MiscHandlerSet,
ModelExampleFilesHandler,
ModelLibraryHandler,
NodeRegistry,
NodeRegistryHandler,
SettingsHandler,
TrainedWordsHandler,
UsageStatsHandler,
build_service_registry_adapter,
)
from .misc_route_registrar import MiscRouteRegistrar
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get( standalone_mode = 'nodes' not in sys.modules
"HF_HUB_DISABLE_TELEMETRY", "0"
) == "0"
# Node registry for tracking active workflow nodes
class NodeRegistry:
"""Thread-safe registry for tracking Lora nodes in active workflows"""
def __init__(self):
self._lock = threading.RLock()
self._nodes = {} # node_id -> node_info
self._registry_updated = threading.Event()
def register_nodes(self, nodes):
"""Register multiple nodes at once, replacing existing registry"""
with self._lock:
# Clear existing registry
self._nodes.clear()
# Register all new nodes
for node in nodes:
node_id = node['node_id']
node_type = node.get('type', '')
# Convert node type name to integer
type_id = NODE_TYPES.get(node_type, 0) # 0 for unknown types
# Handle null bgcolor with default color
bgcolor = node.get('bgcolor')
if bgcolor is None:
bgcolor = DEFAULT_NODE_COLOR
self._nodes[node_id] = {
'id': node_id,
'bgcolor': bgcolor,
'title': node.get('title'),
'type': type_id,
'type_name': node_type
}
logger.debug(f"Registered {len(nodes)} nodes in registry")
# Signal that registry has been updated
self._registry_updated.set()
def get_registry(self):
"""Get current registry information"""
with self._lock:
return {
'nodes': dict(self._nodes), # Return a copy
'node_count': len(self._nodes)
}
def clear_registry(self):
"""Clear the entire registry"""
with self._lock:
self._nodes.clear()
logger.info("Node registry cleared")
def wait_for_update(self, timeout=1.0):
"""Wait for registry update with timeout"""
self._registry_updated.clear()
return self._registry_updated.wait(timeout)
# Global registry instance
node_registry = NodeRegistry()
class MiscRoutes: class MiscRoutes:
"""Route controller that mirrors the model route architecture.""" """Miscellaneous routes for various utility functions"""
@staticmethod
def setup_routes(app):
"""Register miscellaneous routes"""
app.router.add_post('/api/settings', MiscRoutes.update_settings)
# Add new route for clearing cache
app.router.add_post('/api/clear-cache', MiscRoutes.clear_cache)
def __init__( app.router.add_get('/api/health-check', lambda request: web.json_response({'status': 'ok'}))
self,
*,
settings_service=None,
usage_stats_factory: Callable[[], UsageStats] = UsageStats,
prompt_server: type[PromptServer] = PromptServer,
service_registry_adapter=build_service_registry_adapter(),
metadata_provider_factory=get_metadata_provider,
metadata_archive_manager_factory=get_metadata_archive_manager,
metadata_provider_updater=update_metadata_providers,
downloader_factory=get_downloader,
registrar_factory=MiscRouteRegistrar,
handler_set_factory=MiscHandlerSet,
node_registry: NodeRegistry | None = None,
standalone_mode_flag: bool = standalone_mode,
) -> None:
self._settings = settings_service or get_settings_manager()
self._usage_stats_factory = usage_stats_factory
self._prompt_server = prompt_server
self._service_registry_adapter = service_registry_adapter
self._metadata_provider_factory = metadata_provider_factory
self._metadata_archive_manager_factory = metadata_archive_manager_factory
self._metadata_provider_updater = metadata_provider_updater
self._downloader_factory = downloader_factory
self._registrar_factory = registrar_factory
self._handler_set_factory = handler_set_factory
self._node_registry = node_registry or NodeRegistry()
self._standalone_mode = standalone_mode_flag
self._handler_mapping: Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]] | None = None # Usage stats routes
app.router.add_post('/api/update-usage-stats', MiscRoutes.update_usage_stats)
app.router.add_get('/api/get-usage-stats', MiscRoutes.get_usage_stats)
# Lora code update endpoint
app.router.add_post('/api/update-lora-code', MiscRoutes.update_lora_code)
# Add new route for getting trained words
app.router.add_get('/api/trained-words', MiscRoutes.get_trained_words)
# Add new route for getting model example files
app.router.add_get('/api/model-example-files', MiscRoutes.get_model_example_files)
# Node registry endpoints
app.router.add_post('/api/register-nodes', MiscRoutes.register_nodes)
app.router.add_get('/api/get-registry', MiscRoutes.get_registry)
# Add new route for checking if a model exists in the library
app.router.add_get('/api/check-model-exists', MiscRoutes.check_model_exists)
@staticmethod @staticmethod
def setup_routes(app: web.Application) -> None: async def clear_cache(request):
"""Entry point used by the application bootstrap.""" """Clear all cache files from the cache folder"""
controller = MiscRoutes() try:
controller.bind(app) # Get the cache folder path (relative to project directory)
project_dir = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
cache_folder = os.path.join(project_dir, 'cache')
# Check if cache folder exists
if not os.path.exists(cache_folder):
logger.info("Cache folder does not exist, nothing to clear")
return web.json_response({'success': True, 'message': 'No cache folder found'})
# Get list of cache files before deleting for reporting
cache_files = [f for f in os.listdir(cache_folder) if os.path.isfile(os.path.join(cache_folder, f))]
deleted_files = []
# Delete each .msgpack file in the cache folder
for filename in cache_files:
if filename.endswith('.msgpack'):
file_path = os.path.join(cache_folder, filename)
try:
os.remove(file_path)
deleted_files.append(filename)
logger.info(f"Deleted cache file: {filename}")
except Exception as e:
logger.error(f"Failed to delete {filename}: {e}")
return web.json_response({
'success': False,
'error': f"Failed to delete {filename}: {str(e)}"
}, status=500)
return web.json_response({
'success': True,
'message': f"Successfully cleared {len(deleted_files)} cache files",
'deleted_files': deleted_files
})
except Exception as e:
logger.error(f"Error clearing cache files: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def bind(self, app: web.Application) -> None: @staticmethod
registrar = self._registrar_factory(app) async def update_settings(request):
registrar.register_routes(self._ensure_handler_mapping()) """Update application settings"""
try:
data = await request.json()
# Validate and update settings
for key, value in data.items():
# Special handling for example_images_path - verify path exists
if key == 'example_images_path' and value:
if not os.path.exists(value):
return web.json_response({
'success': False,
'error': f"Path does not exist: {value}"
})
# Path changed - server restart required for new path to take effect
old_path = settings.get('example_images_path')
if old_path != value:
logger.info(f"Example images path changed to {value} - server restart required")
# Special handling for base_model_path_mappings - parse JSON string
if key == 'base_model_path_mappings' and value:
try:
value = json.loads(value)
except json.JSONDecodeError:
return web.json_response({
'success': False,
'error': f"Invalid JSON format for base_model_path_mappings: {value}"
})
# Save to settings
settings.set(key, value)
return web.json_response({'success': True})
except Exception as e:
logger.error(f"Error updating settings: {e}", exc_info=True)
return web.Response(status=500, text=str(e))
@staticmethod
async def update_usage_stats(request):
"""
Update usage statistics based on a prompt_id
Expects a JSON body with:
{
"prompt_id": "string"
}
"""
try:
# Parse the request body
data = await request.json()
prompt_id = data.get('prompt_id')
if not prompt_id:
return web.json_response({
'success': False,
'error': 'Missing prompt_id'
}, status=400)
# Call the UsageStats to process this prompt_id synchronously
usage_stats = UsageStats()
await usage_stats.process_execution(prompt_id)
return web.json_response({
'success': True
})
except Exception as e:
logger.error(f"Failed to update usage stats: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def get_usage_stats(request):
"""Get current usage statistics"""
try:
usage_stats = UsageStats()
stats = await usage_stats.get_stats()
# Add version information to help clients handle format changes
stats_response = {
'success': True,
'data': stats,
'format_version': 2 # Indicate this is the new format with history
}
return web.json_response(stats_response)
except Exception as e:
logger.error(f"Failed to get usage stats: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def update_lora_code(request):
"""
Update Lora code in ComfyUI nodes
Expects a JSON body with:
{
"node_ids": [123, 456], # Optional - List of node IDs to update (for browser mode)
"lora_code": "<lora:modelname:1.0>", # The Lora code to send
"mode": "append" # or "replace" - whether to append or replace existing code
}
"""
try:
# Parse the request body
data = await request.json()
node_ids = data.get('node_ids')
lora_code = data.get('lora_code', '')
mode = data.get('mode', 'append')
if not lora_code:
return web.json_response({
'success': False,
'error': 'Missing lora_code parameter'
}, status=400)
results = []
# Desktop mode: no specific node_ids provided
if node_ids is None:
try:
# Send broadcast message with id=-1 to all Lora Loader nodes
PromptServer.instance.send_sync("lora_code_update", {
"id": -1,
"lora_code": lora_code,
"mode": mode
})
results.append({
'node_id': 'broadcast',
'success': True
})
except Exception as e:
logger.error(f"Error broadcasting lora code: {e}")
results.append({
'node_id': 'broadcast',
'success': False,
'error': str(e)
})
else:
# Browser mode: send to specific nodes
for node_id in node_ids:
try:
# Send the message to the frontend
PromptServer.instance.send_sync("lora_code_update", {
"id": node_id,
"lora_code": lora_code,
"mode": mode
})
results.append({
'node_id': node_id,
'success': True
})
except Exception as e:
logger.error(f"Error sending lora code to node {node_id}: {e}")
results.append({
'node_id': node_id,
'success': False,
'error': str(e)
})
return web.json_response({
'success': True,
'results': results
})
except Exception as e:
logger.error(f"Failed to update lora code: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def _ensure_handler_mapping(self) -> Mapping[str, Callable[[web.Request], Awaitable[web.StreamResponse]]]: @staticmethod
if self._handler_mapping is None: async def get_trained_words(request):
handler_set = self._create_handler_set() """
self._handler_mapping = handler_set.to_route_mapping() Get trained words from a safetensors file, sorted by frequency
return self._handler_mapping
Expects a query parameter:
file_path: Path to the safetensors file
"""
try:
# Get file path from query parameters
file_path = request.query.get('file_path')
if not file_path:
return web.json_response({
'success': False,
'error': 'Missing file_path parameter'
}, status=400)
# Check if file exists and is a safetensors file
if not os.path.exists(file_path):
return web.json_response({
'success': False,
'error': f"File not found: {file_path}"
}, status=404)
if not file_path.lower().endswith('.safetensors'):
return web.json_response({
'success': False,
'error': 'File is not a safetensors file'
}, status=400)
# Extract trained words and class_tokens
trained_words, class_tokens = await extract_trained_words(file_path)
# Return result with both trained words and class tokens
return web.json_response({
'success': True,
'trained_words': trained_words,
'class_tokens': class_tokens
})
except Exception as e:
logger.error(f"Failed to get trained words: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
def _create_handler_set(self) -> MiscHandlerSet: @staticmethod
health = HealthCheckHandler() async def get_model_example_files(request):
settings_handler = SettingsHandler( """
settings_service=self._settings, Get list of example image files for a specific model based on file path
metadata_provider_updater=self._metadata_provider_updater,
downloader_factory=self._downloader_factory, Expects:
) - file_path in query parameters
usage_stats = UsageStatsHandler(usage_stats_factory=self._usage_stats_factory)
lora_code = LoraCodeHandler(prompt_server=self._prompt_server) Returns:
trained_words = TrainedWordsHandler() - List of image files with their paths as static URLs
model_examples = ModelExampleFilesHandler() """
metadata_archive = MetadataArchiveHandler( try:
metadata_archive_manager_factory=self._metadata_archive_manager_factory, # Get the model file path from query parameters
settings_service=self._settings, file_path = request.query.get('file_path')
metadata_provider_updater=self._metadata_provider_updater,
) if not file_path:
filesystem = FileSystemHandler(settings_service=self._settings) return web.json_response({
node_registry_handler = NodeRegistryHandler( 'success': False,
node_registry=self._node_registry, 'error': 'Missing file_path parameter'
prompt_server=self._prompt_server, }, status=400)
standalone_mode=self._standalone_mode,
) # Extract directory and base filename
model_library = ModelLibraryHandler( model_dir = os.path.dirname(file_path)
service_registry=self._service_registry_adapter, model_filename = os.path.basename(file_path)
metadata_provider_factory=self._metadata_provider_factory, model_name = os.path.splitext(model_filename)[0]
)
# Check if the directory exists
if not os.path.exists(model_dir):
return web.json_response({
'success': False,
'error': 'Model directory not found',
'files': []
}, status=404)
# Look for files matching the pattern modelname.example.<index>.<ext>
files = []
pattern = f"{model_name}.example."
for file in os.listdir(model_dir):
file_lower = file.lower()
if file_lower.startswith(pattern.lower()):
file_full_path = os.path.join(model_dir, file)
if os.path.isfile(file_full_path):
# Check if the file is a supported media file
file_ext = os.path.splitext(file)[1].lower()
if (file_ext in SUPPORTED_MEDIA_EXTENSIONS['images'] or
file_ext in SUPPORTED_MEDIA_EXTENSIONS['videos']):
# Extract the index from the filename
try:
# Extract the part after '.example.' and before file extension
index_part = file[len(pattern):].split('.')[0]
# Try to parse it as an integer
index = int(index_part)
except (ValueError, IndexError):
# If we can't parse the index, use infinity to sort at the end
index = float('inf')
# Convert file path to static URL
static_url = config.get_preview_static_url(file_full_path)
files.append({
'name': file,
'path': static_url,
'extension': file_ext,
'is_video': file_ext in SUPPORTED_MEDIA_EXTENSIONS['videos'],
'index': index
})
# Sort files by their index for consistent ordering
files.sort(key=lambda x: x['index'])
# Remove the index field as it's only used for sorting
for file in files:
file.pop('index', None)
return web.json_response({
'success': True,
'files': files
})
except Exception as e:
logger.error(f"Failed to get model example files: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
return self._handler_set_factory( @staticmethod
health=health, async def register_nodes(request):
settings=settings_handler, """
usage_stats=usage_stats, Register multiple Lora nodes at once
lora_code=lora_code,
trained_words=trained_words, Expects a JSON body with:
model_examples=model_examples, {
node_registry=node_registry_handler, "nodes": [
model_library=model_library, {
metadata_archive=metadata_archive, "node_id": 123,
filesystem=filesystem, "bgcolor": "#535",
) "title": "Lora Loader (LoraManager)"
},
...
]
}
"""
try:
data = await request.json()
# Validate required fields
nodes = data.get('nodes', [])
if not isinstance(nodes, list):
return web.json_response({
'success': False,
'error': 'nodes must be a list'
}, status=400)
# Validate each node
for i, node in enumerate(nodes):
if not isinstance(node, dict):
return web.json_response({
'success': False,
'error': f'Node {i} must be an object'
}, status=400)
node_id = node.get('node_id')
if node_id is None:
return web.json_response({
'success': False,
'error': f'Node {i} missing node_id parameter'
}, status=400)
# Validate node_id is an integer
try:
node['node_id'] = int(node_id)
except (ValueError, TypeError):
return web.json_response({
'success': False,
'error': f'Node {i} node_id must be an integer'
}, status=400)
# Register all nodes
node_registry.register_nodes(nodes)
return web.json_response({
'success': True,
'message': f'{len(nodes)} nodes registered successfully'
})
except Exception as e:
logger.error(f"Failed to register nodes: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)
@staticmethod
async def get_registry(request):
"""Get current node registry information by refreshing from frontend"""
try:
# Check if running in standalone mode
if standalone_mode:
logger.warning("Registry refresh not available in standalone mode")
return web.json_response({
'success': False,
'error': 'Standalone Mode Active',
'message': 'Cannot interact with ComfyUI in standalone mode.'
}, status=503)
# Send message to frontend to refresh registry
try:
PromptServer.instance.send_sync("lora_registry_refresh", {})
logger.debug("Sent registry refresh request to frontend")
except Exception as e:
logger.error(f"Failed to send registry refresh message: {e}")
return web.json_response({
'success': False,
'error': 'Communication Error',
'message': f'Failed to communicate with ComfyUI frontend: {str(e)}'
}, status=500)
# Wait for registry update with timeout
def wait_for_registry():
return node_registry.wait_for_update(timeout=1.0)
# Run the wait in a thread to avoid blocking the event loop
loop = asyncio.get_event_loop()
registry_updated = await loop.run_in_executor(None, wait_for_registry)
if not registry_updated:
logger.warning("Registry refresh timeout after 1 second")
return web.json_response({
'success': False,
'error': 'Timeout Error',
'message': 'Registry refresh timeout - ComfyUI frontend may not be responsive'
}, status=408)
# Get updated registry
registry_info = node_registry.get_registry()
return web.json_response({
'success': True,
'data': registry_info
})
except Exception as e:
logger.error(f"Failed to get registry: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': 'Internal Error',
'message': str(e)
}, status=500)
@staticmethod
async def check_model_exists(request):
"""
Check if a model with specified modelId and optionally modelVersionId exists in the library
Expects query parameters:
- modelId: int - Civitai model ID (required)
- modelVersionId: int - Civitai model version ID (optional)
Returns:
- If modelVersionId is provided: JSON with a boolean 'exists' field
- If modelVersionId is not provided: JSON with a list of modelVersionIds that exist in the library
"""
try:
# Get the modelId and modelVersionId from query parameters
model_id_str = request.query.get('modelId')
model_version_id_str = request.query.get('modelVersionId')
# Validate modelId parameter (required)
if not model_id_str:
return web.json_response({
'success': False,
'error': 'Missing required parameter: modelId'
}, status=400)
try:
# Convert modelId to integer
model_id = int(model_id_str)
except ValueError:
return web.json_response({
'success': False,
'error': 'Parameter modelId must be an integer'
}, status=400)
# Get all scanners
lora_scanner = await ServiceRegistry.get_lora_scanner()
checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
embedding_scanner = await ServiceRegistry.get_embedding_scanner()
# If modelVersionId is provided, check for specific version
if model_version_id_str:
try:
model_version_id = int(model_version_id_str)
except ValueError:
return web.json_response({
'success': False,
'error': 'Parameter modelVersionId must be an integer'
}, status=400)
# Check lora scanner first
exists = False
model_type = None
__all__ = ["MiscRoutes"] if await lora_scanner.check_model_version_exists(model_id, model_version_id):
exists = True
model_type = 'lora'
elif checkpoint_scanner and await checkpoint_scanner.check_model_version_exists(model_id, model_version_id):
exists = True
model_type = 'checkpoint'
elif embedding_scanner and await embedding_scanner.check_model_version_exists(model_id, model_version_id):
exists = True
model_type = 'embedding'
return web.json_response({
'success': True,
'exists': exists,
'modelType': model_type if exists else None
})
# If modelVersionId is not provided, return all version IDs for the model
else:
lora_versions = await lora_scanner.get_model_versions_by_id(model_id)
checkpoint_versions = []
embedding_versions = []
# 优先lora其次checkpoint最后embedding
if not lora_versions:
checkpoint_versions = await checkpoint_scanner.get_model_versions_by_id(model_id)
if not lora_versions and not checkpoint_versions:
embedding_versions = await embedding_scanner.get_model_versions_by_id(model_id)
model_type = None
versions = []
if lora_versions:
model_type = 'lora'
versions = lora_versions
elif checkpoint_versions:
model_type = 'checkpoint'
versions = checkpoint_versions
elif embedding_versions:
model_type = 'embedding'
versions = embedding_versions
return web.json_response({
'success': True,
'modelId': model_id,
'modelType': model_type,
'versions': versions
})
except Exception as e:
logger.error(f"Failed to check model existence: {e}", exc_info=True)
return web.json_response({
'success': False,
'error': str(e)
}, status=500)

View File

@@ -1,108 +0,0 @@
"""Route registrar for model endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Iterable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a HTTP route."""
method: str
path_template: str
handler_name: str
def build_path(self, prefix: str) -> str:
return self.path_template.replace("{prefix}", prefix)
COMMON_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/{prefix}/list", "get_models"),
RouteDefinition("POST", "/api/lm/{prefix}/delete", "delete_model"),
RouteDefinition("POST", "/api/lm/{prefix}/exclude", "exclude_model"),
RouteDefinition("POST", "/api/lm/{prefix}/fetch-civitai", "fetch_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/fetch-all-civitai", "fetch_all_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/relink-civitai", "relink_civitai"),
RouteDefinition("POST", "/api/lm/{prefix}/replace-preview", "replace_preview"),
RouteDefinition("POST", "/api/lm/{prefix}/save-metadata", "save_metadata"),
RouteDefinition("POST", "/api/lm/{prefix}/add-tags", "add_tags"),
RouteDefinition("POST", "/api/lm/{prefix}/rename", "rename_model"),
RouteDefinition("POST", "/api/lm/{prefix}/bulk-delete", "bulk_delete_models"),
RouteDefinition("POST", "/api/lm/{prefix}/verify-duplicates", "verify_duplicates"),
RouteDefinition("POST", "/api/lm/{prefix}/move_model", "move_model"),
RouteDefinition("POST", "/api/lm/{prefix}/move_models_bulk", "move_models_bulk"),
RouteDefinition("GET", "/api/lm/{prefix}/auto-organize", "auto_organize_models"),
RouteDefinition("POST", "/api/lm/{prefix}/auto-organize", "auto_organize_models"),
RouteDefinition("GET", "/api/lm/{prefix}/auto-organize-progress", "get_auto_organize_progress"),
RouteDefinition("GET", "/api/lm/{prefix}/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/{prefix}/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/{prefix}/model-types", "get_model_types"),
RouteDefinition("GET", "/api/lm/{prefix}/scan", "scan_models"),
RouteDefinition("GET", "/api/lm/{prefix}/roots", "get_model_roots"),
RouteDefinition("GET", "/api/lm/{prefix}/folders", "get_folders"),
RouteDefinition("GET", "/api/lm/{prefix}/folder-tree", "get_folder_tree"),
RouteDefinition("GET", "/api/lm/{prefix}/unified-folder-tree", "get_unified_folder_tree"),
RouteDefinition("GET", "/api/lm/{prefix}/find-duplicates", "find_duplicate_models"),
RouteDefinition("GET", "/api/lm/{prefix}/find-filename-conflicts", "find_filename_conflicts"),
RouteDefinition("GET", "/api/lm/{prefix}/get-notes", "get_model_notes"),
RouteDefinition("GET", "/api/lm/{prefix}/preview-url", "get_model_preview_url"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai-url", "get_model_civitai_url"),
RouteDefinition("GET", "/api/lm/{prefix}/metadata", "get_model_metadata"),
RouteDefinition("GET", "/api/lm/{prefix}/model-description", "get_model_description"),
RouteDefinition("GET", "/api/lm/{prefix}/relative-paths", "get_relative_paths"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/versions/{model_id}", "get_civitai_versions"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/model/version/{modelVersionId}", "get_civitai_model_by_version"),
RouteDefinition("GET", "/api/lm/{prefix}/civitai/model/hash/{hash}", "get_civitai_model_by_hash"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/refresh", "refresh_model_updates"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/fetch-missing-license", "fetch_missing_civitai_license_data"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/ignore", "set_model_update_ignore"),
RouteDefinition("POST", "/api/lm/{prefix}/updates/ignore-version", "set_version_update_ignore"),
RouteDefinition("GET", "/api/lm/{prefix}/updates/status/{model_id}", "get_model_update_status"),
RouteDefinition("GET", "/api/lm/{prefix}/updates/versions/{model_id}", "get_model_versions"),
RouteDefinition("POST", "/api/lm/download-model", "download_model"),
RouteDefinition("GET", "/api/lm/download-model-get", "download_model_get"),
RouteDefinition("GET", "/api/lm/cancel-download-get", "cancel_download_get"),
RouteDefinition("GET", "/api/lm/pause-download", "pause_download_get"),
RouteDefinition("GET", "/api/lm/resume-download", "resume_download_get"),
RouteDefinition("GET", "/api/lm/download-progress/{download_id}", "get_download_progress"),
RouteDefinition("POST", "/api/lm/{prefix}/cancel-task", "cancel_task"),
RouteDefinition("GET", "/{prefix}", "handle_models_page"),
)
class ModelRouteRegistrar:
"""Bind declarative definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_common_routes(
self,
prefix: str,
handler_lookup: Mapping[str, Callable[[web.Request], object]],
*,
definitions: Iterable[RouteDefinition] = COMMON_ROUTE_DEFINITIONS,
) -> None:
for definition in definitions:
self._bind_route(definition.method, definition.build_path(prefix), handler_lookup[definition.handler_name])
def add_route(self, method: str, path: str, handler: Callable) -> None:
self._bind_route(method, path, handler)
def add_prefixed_route(self, method: str, path_template: str, prefix: str, handler: Callable) -> None:
self._bind_route(method, path_template.replace("{prefix}", prefix), handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

View File

@@ -1,25 +0,0 @@
"""Route controller for preview asset delivery."""
from __future__ import annotations
from aiohttp import web
from .handlers.preview_handlers import PreviewHandler
class PreviewRoutes:
"""Register routes that expose preview assets."""
def __init__(self, *, handler: PreviewHandler | None = None) -> None:
self._handler = handler or PreviewHandler()
@classmethod
def setup_routes(cls, app: web.Application) -> None:
controller = cls()
controller.register(app)
def register(self, app: web.Application) -> None:
app.router.add_get('/api/lm/previews', self._handler.serve_preview)
__all__ = ["PreviewRoutes"]

View File

@@ -1,74 +0,0 @@
"""Route registrar for recipe endpoints."""
from __future__ import annotations
from dataclasses import dataclass
from typing import Callable, Mapping
from aiohttp import web
@dataclass(frozen=True)
class RouteDefinition:
"""Declarative definition for a recipe HTTP route."""
method: str
path: str
handler_name: str
ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/loras/recipes", "render_page"),
RouteDefinition("GET", "/api/lm/recipes", "list_recipes"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}", "get_recipe"),
RouteDefinition("GET", "/api/lm/recipes/import-remote", "import_remote_recipe"),
RouteDefinition("POST", "/api/lm/recipes/analyze-image", "analyze_uploaded_image"),
RouteDefinition("POST", "/api/lm/recipes/analyze-local-image", "analyze_local_image"),
RouteDefinition("POST", "/api/lm/recipes/save", "save_recipe"),
RouteDefinition("DELETE", "/api/lm/recipe/{recipe_id}", "delete_recipe"),
RouteDefinition("GET", "/api/lm/recipes/top-tags", "get_top_tags"),
RouteDefinition("GET", "/api/lm/recipes/base-models", "get_base_models"),
RouteDefinition("GET", "/api/lm/recipes/roots", "get_roots"),
RouteDefinition("GET", "/api/lm/recipes/folders", "get_folders"),
RouteDefinition("GET", "/api/lm/recipes/folder-tree", "get_folder_tree"),
RouteDefinition("GET", "/api/lm/recipes/unified-folder-tree", "get_unified_folder_tree"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/share", "share_recipe"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/share/download", "download_shared_recipe"),
RouteDefinition("GET", "/api/lm/recipe/{recipe_id}/syntax", "get_recipe_syntax"),
RouteDefinition("PUT", "/api/lm/recipe/{recipe_id}/update", "update_recipe"),
RouteDefinition("POST", "/api/lm/recipe/move", "move_recipe"),
RouteDefinition("POST", "/api/lm/recipes/move-bulk", "move_recipes_bulk"),
RouteDefinition("POST", "/api/lm/recipe/lora/reconnect", "reconnect_lora"),
RouteDefinition("GET", "/api/lm/recipes/find-duplicates", "find_duplicates"),
RouteDefinition("POST", "/api/lm/recipes/bulk-delete", "bulk_delete"),
RouteDefinition("POST", "/api/lm/recipes/save-from-widget", "save_recipe_from_widget"),
RouteDefinition("GET", "/api/lm/recipes/for-lora", "get_recipes_for_lora"),
RouteDefinition("GET", "/api/lm/recipes/scan", "scan_recipes"),
RouteDefinition("POST", "/api/lm/recipes/repair", "repair_recipes"),
RouteDefinition("POST", "/api/lm/recipes/cancel-repair", "cancel_repair"),
RouteDefinition("POST", "/api/lm/recipe/{recipe_id}/repair", "repair_recipe"),
RouteDefinition("GET", "/api/lm/recipes/repair-progress", "get_repair_progress"),
)
class RecipeRouteRegistrar:
"""Bind declarative recipe definitions to an aiohttp router."""
_METHOD_MAP = {
"GET": "add_get",
"POST": "add_post",
"PUT": "add_put",
"DELETE": "add_delete",
}
def __init__(self, app: web.Application) -> None:
self._app = app
def register_routes(self, handler_lookup: Mapping[str, Callable[[web.Request], object]]) -> None:
for definition in ROUTE_DEFINITIONS:
handler = handler_lookup[definition.handler_name]
self._bind_route(definition.method, definition.path, handler)
def _bind_route(self, method: str, path: str, handler: Callable) -> None:
add_method_name = self._METHOD_MAP[method.upper()]
add_method = getattr(self._app.router, add_method_name)
add_method(path, handler)

File diff suppressed because it is too large Load Diff

View File

@@ -8,39 +8,18 @@ from collections import defaultdict, Counter
from typing import Dict, List, Any from typing import Dict, List, Any
from ..config import config from ..config import config
from ..services.settings_manager import get_settings_manager from ..services.settings_manager import settings
from ..services.server_i18n import server_i18n
from ..services.service_registry import ServiceRegistry from ..services.service_registry import ServiceRegistry
from ..utils.usage_stats import UsageStats from ..utils.usage_stats import UsageStats
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class _SettingsProxy:
def __init__(self):
self._manager = None
def _resolve(self):
if self._manager is None:
self._manager = get_settings_manager()
return self._manager
def get(self, *args, **kwargs):
return self._resolve().get(*args, **kwargs)
def __getattr__(self, item):
return getattr(self._resolve(), item)
settings = _SettingsProxy()
class StatsRoutes: class StatsRoutes:
"""Route handlers for Statistics page and API endpoints""" """Route handlers for Statistics page and API endpoints"""
def __init__(self): def __init__(self):
self.lora_scanner = None self.lora_scanner = None
self.checkpoint_scanner = None self.checkpoint_scanner = None
self.embedding_scanner = None
self.usage_stats = None self.usage_stats = None
self.template_env = jinja2.Environment( self.template_env = jinja2.Environment(
loader=jinja2.FileSystemLoader(config.templates_path), loader=jinja2.FileSystemLoader(config.templates_path),
@@ -51,14 +30,7 @@ class StatsRoutes:
"""Initialize services from ServiceRegistry""" """Initialize services from ServiceRegistry"""
self.lora_scanner = await ServiceRegistry.get_lora_scanner() self.lora_scanner = await ServiceRegistry.get_lora_scanner()
self.checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner() self.checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
self.embedding_scanner = await ServiceRegistry.get_embedding_scanner() self.usage_stats = UsageStats()
# Only initialize usage stats if we have valid paths configured
try:
self.usage_stats = UsageStats()
except RuntimeError as e:
logger.warning(f"Could not initialize usage statistics: {e}")
self.usage_stats = None
async def handle_stats_page(self, request: web.Request) -> web.Response: async def handle_stats_page(self, request: web.Request) -> web.Response:
"""Handle GET /statistics request""" """Handle GET /statistics request"""
@@ -77,32 +49,13 @@ class StatsRoutes:
(hasattr(self.checkpoint_scanner, '_is_initializing') and self.checkpoint_scanner._is_initializing) (hasattr(self.checkpoint_scanner, '_is_initializing') and self.checkpoint_scanner._is_initializing)
) )
embedding_initializing = ( is_initializing = lora_initializing or checkpoint_initializing
self.embedding_scanner._cache is None or
(hasattr(self.embedding_scanner, 'is_initializing') and self.embedding_scanner.is_initializing())
)
is_initializing = lora_initializing or checkpoint_initializing or embedding_initializing
# 获取用户语言设置
settings_object = settings
user_language = settings_object.get('language', 'en')
settings_manager = settings_object if not isinstance(settings_object, _SettingsProxy) else settings_object._resolve()
# 设置服务端i18n语言
server_i18n.set_locale(user_language)
# 为模板环境添加i18n过滤器
if not hasattr(self.template_env, '_i18n_filter_added'):
self.template_env.filters['t'] = server_i18n.create_template_filter()
self.template_env._i18n_filter_added = True
template = self.template_env.get_template('statistics.html') template = self.template_env.get_template('statistics.html')
rendered = template.render( rendered = template.render(
is_initializing=is_initializing, is_initializing=is_initializing,
settings=settings_manager, settings=settings,
request=request, request=request
t=server_i18n.get_translation,
) )
return web.Response( return web.Response(
@@ -132,29 +85,21 @@ class StatsRoutes:
checkpoint_count = len(checkpoint_cache.raw_data) checkpoint_count = len(checkpoint_cache.raw_data)
checkpoint_size = sum(cp.get('size', 0) for cp in checkpoint_cache.raw_data) checkpoint_size = sum(cp.get('size', 0) for cp in checkpoint_cache.raw_data)
# Get Embedding statistics
embedding_cache = await self.embedding_scanner.get_cached_data()
embedding_count = len(embedding_cache.raw_data)
embedding_size = sum(emb.get('size', 0) for emb in embedding_cache.raw_data)
# Get usage statistics # Get usage statistics
usage_data = await self.usage_stats.get_stats() usage_data = await self.usage_stats.get_stats()
return web.json_response({ return web.json_response({
'success': True, 'success': True,
'data': { 'data': {
'total_models': lora_count + checkpoint_count + embedding_count, 'total_models': lora_count + checkpoint_count,
'lora_count': lora_count, 'lora_count': lora_count,
'checkpoint_count': checkpoint_count, 'checkpoint_count': checkpoint_count,
'embedding_count': embedding_count, 'total_size': lora_size + checkpoint_size,
'total_size': lora_size + checkpoint_size + embedding_size,
'lora_size': lora_size, 'lora_size': lora_size,
'checkpoint_size': checkpoint_size, 'checkpoint_size': checkpoint_size,
'embedding_size': embedding_size,
'total_generations': usage_data.get('total_executions', 0), 'total_generations': usage_data.get('total_executions', 0),
'unused_loras': self._count_unused_models(lora_cache.raw_data, usage_data.get('loras', {})), 'unused_loras': self._count_unused_models(lora_cache.raw_data, usage_data.get('loras', {})),
'unused_checkpoints': self._count_unused_models(checkpoint_cache.raw_data, usage_data.get('checkpoints', {})), 'unused_checkpoints': self._count_unused_models(checkpoint_cache.raw_data, usage_data.get('checkpoints', {}))
'unused_embeddings': self._count_unused_models(embedding_cache.raw_data, usage_data.get('embeddings', {}))
} }
}) })
@@ -176,17 +121,14 @@ class StatsRoutes:
# Get model data for enrichment # Get model data for enrichment
lora_cache = await self.lora_scanner.get_cached_data() lora_cache = await self.lora_scanner.get_cached_data()
checkpoint_cache = await self.checkpoint_scanner.get_cached_data() checkpoint_cache = await self.checkpoint_scanner.get_cached_data()
embedding_cache = await self.embedding_scanner.get_cached_data()
# Create hash to model mapping # Create hash to model mapping
lora_map = {lora['sha256']: lora for lora in lora_cache.raw_data} lora_map = {lora['sha256']: lora for lora in lora_cache.raw_data}
checkpoint_map = {cp['sha256']: cp for cp in checkpoint_cache.raw_data} checkpoint_map = {cp['sha256']: cp for cp in checkpoint_cache.raw_data}
embedding_map = {emb['sha256']: emb for emb in embedding_cache.raw_data}
# Prepare top used models # Prepare top used models
top_loras = self._get_top_used_models(usage_data.get('loras', {}), lora_map, 10) top_loras = self._get_top_used_models(usage_data.get('loras', {}), lora_map, 10)
top_checkpoints = self._get_top_used_models(usage_data.get('checkpoints', {}), checkpoint_map, 10) top_checkpoints = self._get_top_used_models(usage_data.get('checkpoints', {}), checkpoint_map, 10)
top_embeddings = self._get_top_used_models(usage_data.get('embeddings', {}), embedding_map, 10)
# Prepare usage timeline (last 30 days) # Prepare usage timeline (last 30 days)
timeline = self._get_usage_timeline(usage_data, 30) timeline = self._get_usage_timeline(usage_data, 30)
@@ -196,7 +138,6 @@ class StatsRoutes:
'data': { 'data': {
'top_loras': top_loras, 'top_loras': top_loras,
'top_checkpoints': top_checkpoints, 'top_checkpoints': top_checkpoints,
'top_embeddings': top_embeddings,
'usage_timeline': timeline, 'usage_timeline': timeline,
'total_executions': usage_data.get('total_executions', 0) 'total_executions': usage_data.get('total_executions', 0)
} }
@@ -217,19 +158,16 @@ class StatsRoutes:
# Get model data # Get model data
lora_cache = await self.lora_scanner.get_cached_data() lora_cache = await self.lora_scanner.get_cached_data()
checkpoint_cache = await self.checkpoint_scanner.get_cached_data() checkpoint_cache = await self.checkpoint_scanner.get_cached_data()
embedding_cache = await self.embedding_scanner.get_cached_data()
# Count by base model # Count by base model
lora_base_models = Counter(lora.get('base_model', 'Unknown') for lora in lora_cache.raw_data) lora_base_models = Counter(lora.get('base_model', 'Unknown') for lora in lora_cache.raw_data)
checkpoint_base_models = Counter(cp.get('base_model', 'Unknown') for cp in checkpoint_cache.raw_data) checkpoint_base_models = Counter(cp.get('base_model', 'Unknown') for cp in checkpoint_cache.raw_data)
embedding_base_models = Counter(emb.get('base_model', 'Unknown') for emb in embedding_cache.raw_data)
return web.json_response({ return web.json_response({
'success': True, 'success': True,
'data': { 'data': {
'loras': dict(lora_base_models), 'loras': dict(lora_base_models),
'checkpoints': dict(checkpoint_base_models), 'checkpoints': dict(checkpoint_base_models)
'embeddings': dict(embedding_base_models)
} }
}) })
@@ -248,7 +186,6 @@ class StatsRoutes:
# Get model data # Get model data
lora_cache = await self.lora_scanner.get_cached_data() lora_cache = await self.lora_scanner.get_cached_data()
checkpoint_cache = await self.checkpoint_scanner.get_cached_data() checkpoint_cache = await self.checkpoint_scanner.get_cached_data()
embedding_cache = await self.embedding_scanner.get_cached_data()
# Count tag frequencies # Count tag frequencies
all_tags = [] all_tags = []
@@ -256,8 +193,6 @@ class StatsRoutes:
all_tags.extend(lora.get('tags', [])) all_tags.extend(lora.get('tags', []))
for cp in checkpoint_cache.raw_data: for cp in checkpoint_cache.raw_data:
all_tags.extend(cp.get('tags', [])) all_tags.extend(cp.get('tags', []))
for emb in embedding_cache.raw_data:
all_tags.extend(emb.get('tags', []))
tag_counts = Counter(all_tags) tag_counts = Counter(all_tags)
@@ -290,7 +225,6 @@ class StatsRoutes:
# Get model data # Get model data
lora_cache = await self.lora_scanner.get_cached_data() lora_cache = await self.lora_scanner.get_cached_data()
checkpoint_cache = await self.checkpoint_scanner.get_cached_data() checkpoint_cache = await self.checkpoint_scanner.get_cached_data()
embedding_cache = await self.embedding_scanner.get_cached_data()
# Create models with usage data # Create models with usage data
lora_storage = [] lora_storage = []
@@ -321,31 +255,15 @@ class StatsRoutes:
'base_model': cp.get('base_model', 'Unknown') 'base_model': cp.get('base_model', 'Unknown')
}) })
embedding_storage = []
for emb in embedding_cache.raw_data:
usage_count = 0
if emb['sha256'] in usage_data.get('embeddings', {}):
usage_count = usage_data['embeddings'][emb['sha256']].get('total', 0)
embedding_storage.append({
'name': emb['model_name'],
'size': emb.get('size', 0),
'usage_count': usage_count,
'folder': emb.get('folder', ''),
'base_model': emb.get('base_model', 'Unknown')
})
# Sort by size # Sort by size
lora_storage.sort(key=lambda x: x['size'], reverse=True) lora_storage.sort(key=lambda x: x['size'], reverse=True)
checkpoint_storage.sort(key=lambda x: x['size'], reverse=True) checkpoint_storage.sort(key=lambda x: x['size'], reverse=True)
embedding_storage.sort(key=lambda x: x['size'], reverse=True)
return web.json_response({ return web.json_response({
'success': True, 'success': True,
'data': { 'data': {
'loras': lora_storage[:20], # Top 20 by size 'loras': lora_storage[:20], # Top 20 by size
'checkpoints': checkpoint_storage[:20], 'checkpoints': checkpoint_storage[:20]
'embeddings': embedding_storage[:20]
} }
}) })
@@ -367,18 +285,15 @@ class StatsRoutes:
# Get model data # Get model data
lora_cache = await self.lora_scanner.get_cached_data() lora_cache = await self.lora_scanner.get_cached_data()
checkpoint_cache = await self.checkpoint_scanner.get_cached_data() checkpoint_cache = await self.checkpoint_scanner.get_cached_data()
embedding_cache = await self.embedding_scanner.get_cached_data()
insights = [] insights = []
# Calculate unused models # Calculate unused models
unused_loras = self._count_unused_models(lora_cache.raw_data, usage_data.get('loras', {})) unused_loras = self._count_unused_models(lora_cache.raw_data, usage_data.get('loras', {}))
unused_checkpoints = self._count_unused_models(checkpoint_cache.raw_data, usage_data.get('checkpoints', {})) unused_checkpoints = self._count_unused_models(checkpoint_cache.raw_data, usage_data.get('checkpoints', {}))
unused_embeddings = self._count_unused_models(embedding_cache.raw_data, usage_data.get('embeddings', {}))
total_loras = len(lora_cache.raw_data) total_loras = len(lora_cache.raw_data)
total_checkpoints = len(checkpoint_cache.raw_data) total_checkpoints = len(checkpoint_cache.raw_data)
total_embeddings = len(embedding_cache.raw_data)
if total_loras > 0: if total_loras > 0:
unused_lora_percent = (unused_loras / total_loras) * 100 unused_lora_percent = (unused_loras / total_loras) * 100
@@ -400,20 +315,9 @@ class StatsRoutes:
'suggestion': 'Review and consider removing checkpoints you no longer need.' 'suggestion': 'Review and consider removing checkpoints you no longer need.'
}) })
if total_embeddings > 0:
unused_embedding_percent = (unused_embeddings / total_embeddings) * 100
if unused_embedding_percent > 50:
insights.append({
'type': 'warning',
'title': 'High Number of Unused Embeddings',
'description': f'{unused_embedding_percent:.1f}% of your embeddings ({unused_embeddings}/{total_embeddings}) have never been used.',
'suggestion': 'Consider organizing or archiving unused embeddings to optimize your collection.'
})
# Storage insights # Storage insights
total_size = sum(lora.get('size', 0) for lora in lora_cache.raw_data) + \ total_size = sum(lora.get('size', 0) for lora in lora_cache.raw_data) + \
sum(cp.get('size', 0) for cp in checkpoint_cache.raw_data) + \ sum(cp.get('size', 0) for cp in checkpoint_cache.raw_data)
sum(emb.get('size', 0) for emb in embedding_cache.raw_data)
if total_size > 100 * 1024 * 1024 * 1024: # 100GB if total_size > 100 * 1024 * 1024 * 1024: # 100GB
insights.append({ insights.append({
@@ -486,7 +390,6 @@ class StatsRoutes:
lora_usage = 0 lora_usage = 0
checkpoint_usage = 0 checkpoint_usage = 0
embedding_usage = 0
# Count usage for this date # Count usage for this date
for model_usage in usage_data.get('loras', {}).values(): for model_usage in usage_data.get('loras', {}).values():
@@ -497,16 +400,11 @@ class StatsRoutes:
if isinstance(model_usage, dict) and 'history' in model_usage: if isinstance(model_usage, dict) and 'history' in model_usage:
checkpoint_usage += model_usage['history'].get(date_str, 0) checkpoint_usage += model_usage['history'].get(date_str, 0)
for model_usage in usage_data.get('embeddings', {}).values():
if isinstance(model_usage, dict) and 'history' in model_usage:
embedding_usage += model_usage['history'].get(date_str, 0)
timeline.append({ timeline.append({
'date': date_str, 'date': date_str,
'lora_usage': lora_usage, 'lora_usage': lora_usage,
'checkpoint_usage': checkpoint_usage, 'checkpoint_usage': checkpoint_usage,
'embedding_usage': embedding_usage, 'total_usage': lora_usage + checkpoint_usage
'total_usage': lora_usage + checkpoint_usage + embedding_usage
}) })
return list(reversed(timeline)) # Oldest to newest return list(reversed(timeline)) # Oldest to newest
@@ -528,12 +426,12 @@ class StatsRoutes:
app.router.add_get('/statistics', self.handle_stats_page) app.router.add_get('/statistics', self.handle_stats_page)
# Register API routes # Register API routes
app.router.add_get('/api/lm/stats/collection-overview', self.get_collection_overview) app.router.add_get('/api/stats/collection-overview', self.get_collection_overview)
app.router.add_get('/api/lm/stats/usage-analytics', self.get_usage_analytics) app.router.add_get('/api/stats/usage-analytics', self.get_usage_analytics)
app.router.add_get('/api/lm/stats/base-model-distribution', self.get_base_model_distribution) app.router.add_get('/api/stats/base-model-distribution', self.get_base_model_distribution)
app.router.add_get('/api/lm/stats/tag-analytics', self.get_tag_analytics) app.router.add_get('/api/stats/tag-analytics', self.get_tag_analytics)
app.router.add_get('/api/lm/stats/storage-analytics', self.get_storage_analytics) app.router.add_get('/api/stats/storage-analytics', self.get_storage_analytics)
app.router.add_get('/api/lm/stats/insights', self.get_insights) app.router.add_get('/api/stats/insights', self.get_insights)
async def _on_startup(self, app): async def _on_startup(self, app):
"""Initialize services when the app starts""" """Initialize services when the app starts"""

View File

@@ -1,31 +1,25 @@
import os import os
import subprocess
import aiohttp
import logging import logging
import toml import toml
import git import git
import zipfile from datetime import datetime
import shutil from aiohttp import web
import tempfile
import asyncio
from aiohttp import web, ClientError
from typing import Dict, List from typing import Dict, List
from ..utils.settings_paths import ensure_settings_file
from ..services.downloader import get_downloader
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
NETWORK_EXCEPTIONS = (ClientError, OSError, asyncio.TimeoutError)
class UpdateRoutes: class UpdateRoutes:
"""Routes for handling plugin update checks""" """Routes for handling plugin update checks"""
@staticmethod @staticmethod
def setup_routes(app): def setup_routes(app):
"""Register update check routes""" """Register update check routes"""
app.router.add_get('/api/lm/check-updates', UpdateRoutes.check_updates) app.router.add_get('/api/check-updates', UpdateRoutes.check_updates)
app.router.add_get('/api/lm/version-info', UpdateRoutes.get_version_info) app.router.add_get('/api/version-info', UpdateRoutes.get_version_info)
app.router.add_post('/api/lm/perform-update', UpdateRoutes.perform_update) app.router.add_post('/api/perform-update', UpdateRoutes.perform_update)
@staticmethod @staticmethod
async def check_updates(request): async def check_updates(request):
@@ -69,12 +63,6 @@ class UpdateRoutes:
'nightly': nightly 'nightly': nightly
}) })
except NETWORK_EXCEPTIONS as e:
logger.warning("Network unavailable during update check: %s", e)
return web.json_response({
'success': False,
'error': 'Network unavailable for update check'
})
except Exception as e: except Exception as e:
logger.error(f"Failed to check for updates: {e}", exc_info=True) logger.error(f"Failed to check for updates: {e}", exc_info=True)
return web.json_response({ return web.json_response({
@@ -113,36 +101,34 @@ class UpdateRoutes:
@staticmethod @staticmethod
async def perform_update(request): async def perform_update(request):
""" """
Perform Git-based update to latest release tag or main branch. Perform Git-based update to latest release tag or main branch
If .git is missing, fallback to ZIP download.
""" """
try: try:
# Parse request body
body = await request.json() if request.has_body else {} body = await request.json() if request.has_body else {}
nightly = body.get('nightly', False) nightly = body.get('nightly', False)
# Get current plugin directory
current_dir = os.path.dirname(os.path.abspath(__file__)) current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir)) plugin_root = os.path.dirname(os.path.dirname(current_dir))
settings_path = ensure_settings_file(logger) # Backup settings.json if it exists
settings_path = os.path.join(plugin_root, 'settings.json')
settings_backup = None settings_backup = None
if os.path.exists(settings_path): if os.path.exists(settings_path):
with open(settings_path, 'r', encoding='utf-8') as f: with open(settings_path, 'r', encoding='utf-8') as f:
settings_backup = f.read() settings_backup = f.read()
logger.info("Backed up settings.json") logger.info("Backed up settings.json")
git_folder = os.path.join(plugin_root, '.git') # Perform Git update
if os.path.exists(git_folder): success, new_version = await UpdateRoutes._perform_git_update(plugin_root, nightly)
# Git update
success, new_version = await UpdateRoutes._perform_git_update(plugin_root, nightly) # Restore settings.json if we backed it up
else:
# Fallback: Download ZIP and replace files
success, new_version = await UpdateRoutes._download_and_replace_zip(plugin_root)
if settings_backup and success: if settings_backup and success:
with open(settings_path, 'w', encoding='utf-8') as f: with open(settings_path, 'w', encoding='utf-8') as f:
f.write(settings_backup) f.write(settings_backup)
logger.info("Restored settings.json") logger.info("Restored settings.json")
if success: if success:
return web.json_response({ return web.json_response({
'success': True, 'success': True,
@@ -152,118 +138,15 @@ class UpdateRoutes:
else: else:
return web.json_response({ return web.json_response({
'success': False, 'success': False,
'error': 'Failed to complete update' 'error': 'Failed to complete Git update'
}) })
except Exception as e: except Exception as e:
logger.error(f"Failed to perform update: {e}", exc_info=True) logger.error(f"Failed to perform update: {e}", exc_info=True)
return web.json_response({ return web.json_response({
'success': False, 'success': False,
'error': str(e) 'error': str(e)
}) })
@staticmethod
async def _download_and_replace_zip(plugin_root: str) -> tuple[bool, str]:
"""
Download latest release ZIP from GitHub and replace plugin files.
Skips settings.json and civitai folder. Writes extracted file list to .tracking.
"""
repo_owner = "willmiao"
repo_name = "ComfyUI-Lora-Manager"
github_api = f"https://api.github.com/repos/{repo_owner}/{repo_name}/releases/latest"
try:
downloader = await get_downloader()
# Get release info
success, data = await downloader.make_request(
'GET',
github_api,
use_auth=False
)
if not success:
logger.error(f"Failed to fetch release info: {data}")
return False, ""
zip_url = data.get("zipball_url")
version = data.get("tag_name", "unknown")
# Download ZIP to temporary file
with tempfile.NamedTemporaryFile(delete=False, suffix=".zip") as tmp_zip:
tmp_zip_path = tmp_zip.name
success, result = await downloader.download_file(
url=zip_url,
save_path=tmp_zip_path,
use_auth=False,
allow_resume=False
)
if not success:
logger.error(f"Failed to download ZIP: {result}")
return False, ""
zip_path = tmp_zip_path
# Skip both settings.json, civitai and model cache folder
UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json', 'civitai', 'model_cache'])
# Extract ZIP to temp dir
with tempfile.TemporaryDirectory() as tmp_dir:
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
zip_ref.extractall(tmp_dir)
# Find extracted folder (GitHub ZIP contains a root folder)
extracted_root = next(os.scandir(tmp_dir)).path
# Copy files, skipping settings.json and civitai folder
for item in os.listdir(extracted_root):
if item == 'settings.json' or item == 'civitai':
continue
src = os.path.join(extracted_root, item)
dst = os.path.join(plugin_root, item)
if os.path.isdir(src):
if os.path.exists(dst):
shutil.rmtree(dst)
shutil.copytree(src, dst, ignore=shutil.ignore_patterns('settings.json', 'civitai'))
else:
shutil.copy2(src, dst)
# Write .tracking file: list all files under extracted_root, relative to extracted_root
# for ComfyUI Manager to work properly
tracking_info_file = os.path.join(plugin_root, '.tracking')
tracking_files = []
for root, dirs, files in os.walk(extracted_root):
# Skip civitai folder and its contents
rel_root = os.path.relpath(root, extracted_root)
if rel_root == 'civitai' or rel_root.startswith('civitai' + os.sep):
continue
for file in files:
rel_path = os.path.relpath(os.path.join(root, file), extracted_root)
# Skip settings.json and any file under civitai
if rel_path == 'settings.json' or rel_path.startswith('civitai' + os.sep):
continue
tracking_files.append(rel_path.replace("\\", "/"))
with open(tracking_info_file, "w", encoding='utf-8') as file:
file.write('\n'.join(tracking_files))
os.remove(zip_path)
logger.info(f"Updated plugin via ZIP to {version}")
return True, version
except Exception as e:
logger.error(f"ZIP update failed: {e}", exc_info=True)
return False, ""
def _clean_plugin_folder(plugin_root, skip_files=None):
skip_files = skip_files or []
for item in os.listdir(plugin_root):
if item in skip_files:
continue
path = os.path.join(plugin_root, item)
if os.path.isdir(path):
shutil.rmtree(path)
else:
os.remove(path)
@staticmethod @staticmethod
async def _get_nightly_version() -> tuple[str, List[str]]: async def _get_nightly_version() -> tuple[str, List[str]]:
@@ -277,27 +160,24 @@ class UpdateRoutes:
github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/commits/main" github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/commits/main"
try: try:
downloader = await get_downloader() async with aiohttp.ClientSession() as session:
success, data = await downloader.make_request('GET', github_url, custom_headers={'Accept': 'application/vnd.github+json'}) async with session.get(github_url, headers={'Accept': 'application/vnd.github+json'}) as response:
if response.status != 200:
if not success: logger.warning(f"Failed to fetch GitHub commit: {response.status}")
logger.warning(f"Failed to fetch GitHub commit: {data}") return "main", []
return "main", []
data = await response.json()
commit_sha = data.get('sha', '')[:7] # Short hash commit_sha = data.get('sha', '')[:7] # Short hash
commit_message = data.get('commit', {}).get('message', '') commit_message = data.get('commit', {}).get('message', '')
# Format as "main-{short_hash}" # Format as "main-{short_hash}"
version = f"main-{commit_sha}" version = f"main-{commit_sha}"
# Use commit message as changelog # Use commit message as changelog
changelog = [commit_message] if commit_message else [] changelog = [commit_message] if commit_message else []
return version, changelog return version, changelog
except NETWORK_EXCEPTIONS as e:
logger.warning("Unable to reach GitHub for nightly version: %s", e)
return "main", []
except Exception as e: except Exception as e:
logger.error(f"Error fetching nightly version: {e}", exc_info=True) logger.error(f"Error fetching nightly version: {e}", exc_info=True)
return "main", [] return "main", []
@@ -344,11 +224,6 @@ class UpdateRoutes:
origin.fetch() origin.fetch()
if nightly: if nightly:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Switch to main branch and pull latest # Switch to main branch and pull latest
main_branch = 'main' main_branch = 'main'
if main_branch not in [branch.name for branch in repo.branches]: if main_branch not in [branch.name for branch in repo.branches]:
@@ -362,11 +237,6 @@ class UpdateRoutes:
new_version = f"main-{repo.head.commit.hexsha[:7]}" new_version = f"main-{repo.head.commit.hexsha[:7]}"
else: else:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Get latest release tag # Get latest release tag
tags = sorted(repo.tags, key=lambda t: t.commit.committed_datetime, reverse=True) tags = sorted(repo.tags, key=lambda t: t.commit.committed_datetime, reverse=True)
if not tags: if not tags:
@@ -418,28 +288,65 @@ class UpdateRoutes:
"""Get Git repository information""" """Get Git repository information"""
current_dir = os.path.dirname(os.path.abspath(__file__)) current_dir = os.path.dirname(os.path.abspath(__file__))
plugin_root = os.path.dirname(os.path.dirname(current_dir)) plugin_root = os.path.dirname(os.path.dirname(current_dir))
git_info = { git_info = {
'commit_hash': 'unknown', 'commit_hash': 'unknown',
'short_hash': 'stable', 'short_hash': 'unknown',
'branch': 'unknown', 'branch': 'unknown',
'commit_date': 'unknown' 'commit_date': 'unknown'
} }
try: try:
# Check if we're in a git repository # Check if we're in a git repository
if not os.path.exists(os.path.join(plugin_root, '.git')): if not os.path.exists(os.path.join(plugin_root, '.git')):
return git_info return git_info
repo = git.Repo(plugin_root) # Get current commit hash
commit = repo.head.commit result = subprocess.run(
git_info['commit_hash'] = commit.hexsha ['git', 'rev-parse', 'HEAD'],
git_info['short_hash'] = commit.hexsha[:7] cwd=plugin_root,
git_info['branch'] = repo.active_branch.name if not repo.head.is_detached else 'detached' stdout=subprocess.PIPE,
git_info['commit_date'] = commit.committed_datetime.strftime('%Y-%m-%d') stderr=subprocess.PIPE,
text=True,
check=False
)
if result.returncode == 0:
git_info['commit_hash'] = result.stdout.strip()
git_info['short_hash'] = git_info['commit_hash'][:7]
# Get current branch name
result = subprocess.run(
['git', 'rev-parse', '--abbrev-ref', 'HEAD'],
cwd=plugin_root,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False
)
if result.returncode == 0:
git_info['branch'] = result.stdout.strip()
# Get commit date
result = subprocess.run(
['git', 'show', '-s', '--format=%ci', 'HEAD'],
cwd=plugin_root,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False
)
if result.returncode == 0:
commit_date = result.stdout.strip()
# Format the date nicely if possible
try:
date_obj = datetime.strptime(commit_date, '%Y-%m-%d %H:%M:%S %z')
git_info['commit_date'] = date_obj.strftime('%Y-%m-%d')
except:
git_info['commit_date'] = commit_date
except Exception as e: except Exception as e:
logger.warning(f"Error getting git info: {e}") logger.warning(f"Error getting git info: {e}")
return git_info return git_info
@staticmethod @staticmethod
@@ -456,26 +363,23 @@ class UpdateRoutes:
github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/releases/latest" github_url = f"https://api.github.com/repos/{repo_owner}/{repo_name}/releases/latest"
try: try:
downloader = await get_downloader() async with aiohttp.ClientSession() as session:
success, data = await downloader.make_request('GET', github_url, custom_headers={'Accept': 'application/vnd.github+json'}) async with session.get(github_url, headers={'Accept': 'application/vnd.github+json'}) as response:
if response.status != 200:
if not success: logger.warning(f"Failed to fetch GitHub release: {response.status}")
logger.warning(f"Failed to fetch GitHub release: {data}") return "v0.0.0", []
return "v0.0.0", []
data = await response.json()
version = data.get('tag_name', '') version = data.get('tag_name', '')
if not version.startswith('v'): if not version.startswith('v'):
version = f"v{version}" version = f"v{version}"
# Extract changelog from release notes # Extract changelog from release notes
body = data.get('body', '') body = data.get('body', '')
changelog = UpdateRoutes._parse_changelog(body) changelog = UpdateRoutes._parse_changelog(body)
return version, changelog return version, changelog
except NETWORK_EXCEPTIONS as e:
logger.warning("Unable to reach GitHub for release info: %s", e)
return "v0.0.0", []
except Exception as e: except Exception as e:
logger.error(f"Error fetching remote version: {e}", exc_info=True) logger.error(f"Error fetching remote version: {e}", exc_info=True)
return "v0.0.0", [] return "v0.0.0", []

File diff suppressed because it is too large Load Diff

View File

@@ -1,5 +1,5 @@
import logging import logging
from typing import Any, Dict, List, Optional from typing import List
from ..utils.models import CheckpointMetadata from ..utils.models import CheckpointMetadata
from ..config import config from ..config import config
@@ -13,7 +13,7 @@ class CheckpointScanner(ModelScanner):
def __init__(self): def __init__(self):
# Define supported file extensions # Define supported file extensions
file_extensions = {'.ckpt', '.pt', '.pt2', '.bin', '.pth', '.safetensors', '.pkl', '.sft', '.gguf'} file_extensions = {'.safetensors', '.ckpt', '.pt', '.pth', '.sft', '.gguf'}
super().__init__( super().__init__(
model_type="checkpoint", model_type="checkpoint",
model_class=CheckpointMetadata, model_class=CheckpointMetadata,
@@ -21,33 +21,6 @@ class CheckpointScanner(ModelScanner):
hash_index=ModelHashIndex() hash_index=ModelHashIndex()
) )
def _resolve_model_type(self, root_path: Optional[str]) -> Optional[str]:
if not root_path:
return None
if config.checkpoints_roots and root_path in config.checkpoints_roots:
return "checkpoint"
if config.unet_roots and root_path in config.unet_roots:
return "diffusion_model"
return None
def adjust_metadata(self, metadata, file_path, root_path):
if hasattr(metadata, "model_type"):
model_type = self._resolve_model_type(root_path)
if model_type:
metadata.model_type = model_type
return metadata
def adjust_cached_entry(self, entry: Dict[str, Any]) -> Dict[str, Any]:
model_type = self._resolve_model_type(
self._find_root_for_file(entry.get("file_path"))
)
if model_type:
entry["model_type"] = model_type
return entry
def get_model_roots(self) -> List[str]: def get_model_roots(self) -> List[str]:
"""Get checkpoint root directories""" """Get checkpoint root directories"""
return config.base_models_roots return config.base_models_roots

View File

@@ -1,24 +1,24 @@
import os import os
import logging import logging
from typing import Dict from typing import Dict, List, Optional
from .base_model_service import BaseModelService from .base_model_service import BaseModelService
from ..utils.models import CheckpointMetadata from ..utils.models import CheckpointMetadata
from ..config import config from ..config import config
from ..utils.routes_common import ModelRouteUtils
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
class CheckpointService(BaseModelService): class CheckpointService(BaseModelService):
"""Checkpoint-specific service implementation""" """Checkpoint-specific service implementation"""
def __init__(self, scanner, update_service=None): def __init__(self, scanner):
"""Initialize Checkpoint service """Initialize Checkpoint service
Args: Args:
scanner: Checkpoint scanner instance scanner: Checkpoint scanner instance
update_service: Optional service for remote update tracking.
""" """
super().__init__("checkpoint", scanner, CheckpointMetadata, update_service=update_service) super().__init__("checkpoint", scanner, CheckpointMetadata)
async def format_response(self, checkpoint_data: Dict) -> Dict: async def format_response(self, checkpoint_data: Dict) -> Dict:
"""Format Checkpoint data for API response""" """Format Checkpoint data for API response"""
@@ -34,13 +34,12 @@ class CheckpointService(BaseModelService):
"file_size": checkpoint_data.get("size", 0), "file_size": checkpoint_data.get("size", 0),
"modified": checkpoint_data.get("modified", ""), "modified": checkpoint_data.get("modified", ""),
"tags": checkpoint_data.get("tags", []), "tags": checkpoint_data.get("tags", []),
"modelDescription": checkpoint_data.get("modelDescription", ""),
"from_civitai": checkpoint_data.get("from_civitai", True), "from_civitai": checkpoint_data.get("from_civitai", True),
"usage_count": checkpoint_data.get("usage_count", 0),
"notes": checkpoint_data.get("notes", ""), "notes": checkpoint_data.get("notes", ""),
"model_type": checkpoint_data.get("model_type", "checkpoint"), "model_type": checkpoint_data.get("model_type", "checkpoint"),
"favorite": checkpoint_data.get("favorite", False), "favorite": checkpoint_data.get("favorite", False),
"update_available": bool(checkpoint_data.get("update_available", False)), "civitai": ModelRouteUtils.filter_civitai_data(checkpoint_data.get("civitai", {}))
"civitai": self.filter_civitai_data(checkpoint_data.get("civitai", {}), minimal=True)
} }
def find_duplicate_hashes(self) -> Dict: def find_duplicate_hashes(self) -> Dict:
@@ -49,4 +48,4 @@ class CheckpointService(BaseModelService):
def find_duplicate_filenames(self) -> Dict: def find_duplicate_filenames(self) -> Dict:
"""Find Checkpoints with conflicting filenames""" """Find Checkpoints with conflicting filenames"""
return self.scanner._hash_index.get_duplicate_filenames() return self.scanner._hash_index.get_duplicate_filenames()

View File

@@ -1,431 +0,0 @@
import json
import logging
import asyncio
from copy import deepcopy
from typing import Optional, Dict, Tuple, List
from .model_metadata_provider import CivArchiveModelMetadataProvider, ModelMetadataProviderManager
from .downloader import get_downloader
from .errors import RateLimitError
logger = logging.getLogger(__name__)
class CivArchiveClient:
_instance = None
_lock = asyncio.Lock()
@classmethod
async def get_instance(cls):
"""Get singleton instance of CivArchiveClient"""
async with cls._lock:
if cls._instance is None:
cls._instance = cls()
# Register this client as a metadata provider
provider_manager = await ModelMetadataProviderManager.get_instance()
provider_manager.register_provider('civarchive', CivArchiveModelMetadataProvider(cls._instance), False)
return cls._instance
def __init__(self):
# Check if already initialized for singleton pattern
if hasattr(self, '_initialized'):
return
self._initialized = True
self.base_url = "https://civarchive.com/api"
async def _request_json(
self,
path: str,
params: Optional[Dict[str, str]] = None
) -> Tuple[Optional[Dict], Optional[str]]:
"""Call CivArchive API and return JSON payload"""
success, payload = await self._make_request(path, params=params)
if not success:
error = payload if isinstance(payload, str) else "Request failed"
return None, error
if not isinstance(payload, dict):
return None, "Invalid response structure"
return payload, None
async def _make_request(
self,
path: str,
*,
params: Optional[Dict[str, str]] = None,
) -> Tuple[bool, Dict | str]:
"""Wrapper around downloader.make_request that surfaces rate limits."""
downloader = await get_downloader()
kwargs: Dict[str, Dict[str, str]] = {}
if params:
safe_params = {str(key): str(value) for key, value in params.items() if value is not None}
if safe_params:
kwargs["params"] = safe_params
success, payload = await downloader.make_request(
"GET",
f"{self.base_url}{path}",
use_auth=False,
**kwargs,
)
if not success and isinstance(payload, RateLimitError):
if payload.provider is None:
payload.provider = "civarchive_api"
raise payload
return success, payload
@staticmethod
def _normalize_payload(payload: Dict) -> Dict:
"""Unwrap CivArchive responses that wrap content under a data key"""
if not isinstance(payload, dict):
return {}
data = payload.get("data")
if isinstance(data, dict):
return data
return payload
@staticmethod
def _split_context(payload: Dict) -> Tuple[Dict, Dict, List[Dict]]:
"""Separate version payload from surrounding model context"""
data = CivArchiveClient._normalize_payload(payload)
context: Dict = {}
fallback_files: List[Dict] = []
version: Dict = {}
for key, value in data.items():
if key in {"version", "model"}:
continue
context[key] = value
if isinstance(data.get("version"), dict):
version = data["version"]
model_block = data.get("model")
if isinstance(model_block, dict):
for key, value in model_block.items():
if key == "version":
if not version and isinstance(value, dict):
version = value
continue
context.setdefault(key, value)
fallback_files = fallback_files or model_block.get("files") or []
fallback_files = fallback_files or data.get("files") or []
return context, version, fallback_files
@staticmethod
def _ensure_list(value) -> List:
if isinstance(value, list):
return value
if value is None:
return []
return [value]
@staticmethod
def _build_model_info(context: Dict) -> Dict:
tags = context.get("tags")
if not isinstance(tags, list):
tags = list(tags) if isinstance(tags, (set, tuple)) else ([] if tags is None else [tags])
return {
"name": context.get("name"),
"type": context.get("type"),
"nsfw": bool(context.get("is_nsfw", context.get("nsfw", False))),
"description": context.get("description"),
"tags": tags,
}
@staticmethod
def _build_creator_info(context: Dict) -> Dict:
username = context.get("creator_username") or context.get("username") or ""
image = context.get("creator_image") or context.get("creator_avatar") or ""
creator: Dict[str, Optional[str]] = {
"username": username,
"image": image,
}
if context.get("creator_name"):
creator["name"] = context["creator_name"]
if context.get("creator_url"):
creator["url"] = context["creator_url"]
return creator
@staticmethod
def _transform_file_entry(file_data: Dict) -> Dict:
mirrors = file_data.get("mirrors") or []
if not isinstance(mirrors, list):
mirrors = [mirrors]
available_mirror = next(
(mirror for mirror in mirrors if isinstance(mirror, dict) and mirror.get("deletedAt") is None),
None
)
download_url = file_data.get("downloadUrl")
if not download_url and available_mirror:
download_url = available_mirror.get("url")
name = file_data.get("name")
if not name and available_mirror:
name = available_mirror.get("filename")
transformed: Dict = {
"id": file_data.get("id"),
"sizeKB": file_data.get("sizeKB"),
"name": name,
"type": file_data.get("type"),
"downloadUrl": download_url,
"primary": True,
# TODO: for some reason is_primary is false in CivArchive response, need to figure this out,
# "primary": bool(file_data.get("is_primary", file_data.get("primary", False))),
"mirrors": mirrors,
}
sha256 = file_data.get("sha256")
if sha256:
transformed["hashes"] = {"SHA256": str(sha256).upper()}
elif isinstance(file_data.get("hashes"), dict):
transformed["hashes"] = file_data["hashes"]
if "metadata" in file_data:
transformed["metadata"] = file_data["metadata"]
if file_data.get("modelVersionId") is not None:
transformed["modelVersionId"] = file_data.get("modelVersionId")
elif file_data.get("model_version_id") is not None:
transformed["modelVersionId"] = file_data.get("model_version_id")
if file_data.get("modelId") is not None:
transformed["modelId"] = file_data.get("modelId")
elif file_data.get("model_id") is not None:
transformed["modelId"] = file_data.get("model_id")
return transformed
def _transform_files(
self,
files: Optional[List[Dict]],
fallback_files: Optional[List[Dict]] = None
) -> List[Dict]:
candidates: List[Dict] = []
if isinstance(files, list) and files:
candidates = files
elif isinstance(fallback_files, list):
candidates = fallback_files
transformed_files: List[Dict] = []
for file_data in candidates:
if isinstance(file_data, dict):
transformed_files.append(self._transform_file_entry(file_data))
return transformed_files
def _transform_version(
self,
context: Dict,
version: Dict,
fallback_files: Optional[List[Dict]] = None
) -> Optional[Dict]:
if not version:
return None
version_copy = deepcopy(version)
version_copy.pop("model", None)
version_copy.pop("creator", None)
if "trigger" in version_copy:
triggers = version_copy.pop("trigger")
if isinstance(triggers, list):
version_copy["trainedWords"] = triggers
elif triggers is None:
version_copy["trainedWords"] = []
else:
version_copy["trainedWords"] = [triggers]
if "trainedWords" in version_copy and isinstance(version_copy["trainedWords"], str):
version_copy["trainedWords"] = [version_copy["trainedWords"]]
if "nsfw_level" in version_copy:
version_copy["nsfwLevel"] = version_copy.pop("nsfw_level")
elif "nsfwLevel" not in version_copy and context.get("nsfw_level") is not None:
version_copy["nsfwLevel"] = context.get("nsfw_level")
stats_keys = ["downloadCount", "ratingCount", "rating"]
stats = {key: version_copy.pop(key) for key in stats_keys if key in version_copy}
if stats:
version_copy["stats"] = stats
version_copy["files"] = self._transform_files(version_copy.get("files"), fallback_files)
version_copy["images"] = self._ensure_list(version_copy.get("images"))
version_copy["model"] = self._build_model_info(context)
version_copy["creator"] = self._build_creator_info(context)
version_copy["source"] = "civarchive"
version_copy["is_deleted"] = bool(context.get("deletedAt")) or bool(version.get("deletedAt"))
return version_copy
async def _resolve_version_from_files(self, payload: Dict) -> Optional[Dict]:
"""Fallback to fetch version data when only file metadata is available"""
data = self._normalize_payload(payload)
files = data.get("files") or payload.get("files") or []
if not isinstance(files, list):
files = [files]
for file_data in files:
if not isinstance(file_data, dict):
continue
model_id = file_data.get("model_id") or file_data.get("modelId")
version_id = file_data.get("model_version_id") or file_data.get("modelVersionId")
if model_id is None or version_id is None:
continue
resolved = await self.get_model_version(model_id, version_id)
if resolved:
return resolved
return None
async def get_model_by_hash(self, model_hash: str) -> Tuple[Optional[Dict], Optional[str]]:
"""Find model by SHA256 hash value using CivArchive API"""
try:
payload, error = await self._request_json(f"/sha256/{model_hash.lower()}")
if error:
if "not found" in error.lower():
return None, "Model not found"
return None, error
context, version_data, fallback_files = self._split_context(payload)
transformed = self._transform_version(context, version_data, fallback_files)
if transformed:
return transformed, None
resolved = await self._resolve_version_from_files(payload)
if resolved:
return resolved, None
logger.error("Error fetching version of CivArchive model by hash %s", model_hash[:10])
return None, "No version data found"
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model by hash {model_hash[:10]}: {e}")
return None, str(e)
async def get_model_versions(self, model_id: str) -> Optional[Dict]:
"""Get all versions of a model using CivArchive API"""
try:
payload, error = await self._request_json(f"/models/{model_id}")
if error or payload is None:
if error and "not found" in error.lower():
return None
logger.error(f"Error fetching CivArchive model versions for {model_id}: {error}")
return None
data = self._normalize_payload(payload)
context, version_data, fallback_files = self._split_context(payload)
versions_meta = data.get("versions") or []
transformed_versions: List[Dict] = []
for meta in versions_meta:
if not isinstance(meta, dict):
continue
version_id = meta.get("id")
if version_id is None:
continue
target_model_id = meta.get("modelId") or model_id
version = await self.get_model_version(target_model_id, version_id)
if version:
transformed_versions.append(version)
# Ensure the primary version is included even if versions list was empty
primary_version = self._transform_version(context, version_data, fallback_files)
if primary_version:
transformed_versions.insert(0, primary_version)
ordered_versions: List[Dict] = []
seen_ids = set()
for version in transformed_versions:
version_id = version.get("id")
if version_id in seen_ids:
continue
seen_ids.add(version_id)
ordered_versions.append(version)
return {
"modelVersions": ordered_versions,
"type": context.get("type", ""),
"name": context.get("name", ""),
}
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model versions for {model_id}: {e}")
return None
async def get_model_version(self, model_id: int = None, version_id: int = None) -> Optional[Dict]:
"""Get specific model version using CivArchive API
Args:
model_id: The model ID (required)
version_id: Optional specific version ID to filter to
Returns:
Optional[Dict]: The model version data or None if not found
"""
if model_id is None:
return None
try:
params = {"modelVersionId": version_id} if version_id is not None else None
payload, error = await self._request_json(f"/models/{model_id}", params=params)
if error or payload is None:
if error and "not found" in error.lower():
return None
logger.error(f"Error fetching CivArchive model version via API {model_id}/{version_id}: {error}")
return None
context, version_data, fallback_files = self._split_context(payload)
if not version_data:
return await self._resolve_version_from_files(payload)
if version_id is not None:
raw_id = version_data.get("id")
if raw_id != version_id:
logger.warning(
"Requested version %s doesn't match default version %s for model %s",
version_id,
raw_id,
model_id,
)
return None
actual_model_id = version_data.get("modelId")
context_model_id = context.get("id")
# CivArchive can respond with data for a different model id while already
# returning the fully resolved model context. Only follow the redirect when
# the context itself still points to the original (wrong) model.
if (
actual_model_id is not None
and str(actual_model_id) != str(model_id)
and (context_model_id is None or str(context_model_id) != str(actual_model_id))
):
return await self.get_model_version(actual_model_id, version_id)
return self._transform_version(context, version_data, fallback_files)
except RateLimitError:
raise
except Exception as e:
logger.error(f"Error fetching CivArchive model version via API {model_id}/{version_id}: {e}")
return None
async def get_model_version_info(self, version_id: str) -> Tuple[Optional[Dict], Optional[str]]:
""" Fetch model version metadata using a known bogus model lookup
CivArchive lacks a direct version lookup API, this uses a workaround (which we handle in the main model request now)
Args:
version_id: The model version ID
Returns:
Tuple[Optional[Dict], Optional[str]]: (version_data, error_message)
"""
version = await self.get_model_version(1, version_id)
if version is None:
return None, "Model not found"
return version, None

View File

@@ -1,12 +1,11 @@
import asyncio from datetime import datetime
import copy import aiohttp
import logging
import os import os
from typing import Any, Optional, Dict, Tuple, List, Sequence import logging
from .model_metadata_provider import CivitaiModelMetadataProvider, ModelMetadataProviderManager import asyncio
from .downloader import get_downloader from email.parser import Parser
from .errors import RateLimitError, ResourceNotFoundError from typing import Optional, Dict, Tuple, List
from ..utils.civitai_utils import resolve_license_payload from urllib.parse import unquote
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -20,11 +19,6 @@ class CivitaiClient:
async with cls._lock: async with cls._lock:
if cls._instance is None: if cls._instance is None:
cls._instance = cls() cls._instance = cls()
# Register this client as a metadata provider
provider_manager = await ModelMetadataProviderManager.get_instance()
provider_manager.register_provider('civitai', CivitaiModelMetadataProvider(cls._instance), True)
return cls._instance return cls._instance
def __init__(self): def __init__(self):
@@ -34,50 +28,81 @@ class CivitaiClient:
self._initialized = True self._initialized = True
self.base_url = "https://civitai.com/api/v1" self.base_url = "https://civitai.com/api/v1"
self.headers = {
async def _make_request( 'User-Agent': 'ComfyUI-LoRA-Manager/1.0'
self, }
method: str, self._session = None
url: str, self._session_created_at = None
*, # Set default buffer size to 1MB for higher throughput
use_auth: bool = False, self.chunk_size = 1024 * 1024
**kwargs,
) -> Tuple[bool, Dict | str]:
"""Wrapper around downloader.make_request that surfaces rate limits."""
downloader = await get_downloader()
success, result = await downloader.make_request(
method,
url,
use_auth=use_auth,
**kwargs,
)
if not success and isinstance(result, RateLimitError):
if result.provider is None:
result.provider = "civitai_api"
raise result
return success, result
@staticmethod
def _remove_comfy_metadata(model_version: Optional[Dict]) -> None:
"""Remove Comfy-specific metadata from model version images."""
if not isinstance(model_version, dict):
return
images = model_version.get("images")
if not isinstance(images, list):
return
for image in images:
if not isinstance(image, dict):
continue
meta = image.get("meta")
if isinstance(meta, dict) and "comfy" in meta:
meta.pop("comfy", None)
async def download_file(self, url: str, save_dir: str, default_filename: str, progress_callback=None) -> Tuple[bool, str]: @property
"""Download file with resumable downloads and retry mechanism async def session(self) -> aiohttp.ClientSession:
"""Lazy initialize the session"""
if self._session is None:
# Optimize TCP connection parameters
connector = aiohttp.TCPConnector(
ssl=True,
limit=8, # Increase from 3 to 8 for better parallelism
ttl_dns_cache=300, # Enable DNS caching with reasonable timeout
force_close=False, # Keep connections for reuse
enable_cleanup_closed=True
)
trust_env = True # Allow using system environment proxy settings
# Configure timeout parameters - increase read timeout for large files
timeout = aiohttp.ClientTimeout(total=None, connect=60, sock_read=120)
self._session = aiohttp.ClientSession(
connector=connector,
trust_env=trust_env,
timeout=timeout
)
self._session_created_at = datetime.now()
return self._session
async def _ensure_fresh_session(self):
"""Refresh session if it's been open too long"""
if self._session is not None:
if not hasattr(self, '_session_created_at') or \
(datetime.now() - self._session_created_at).total_seconds() > 300: # 5 minutes
await self.close()
self._session = None
return await self.session
def _parse_content_disposition(self, header: str) -> str:
"""Parse filename from content-disposition header"""
if not header:
return None
# Handle quoted filenames
if 'filename="' in header:
start = header.index('filename="') + 10
end = header.index('"', start)
return unquote(header[start:end])
# Fallback to original parsing
disposition = Parser().parsestr(f'Content-Disposition: {header}')
filename = disposition.get_param('filename')
if filename:
return unquote(filename)
return None
def _get_request_headers(self) -> dict:
"""Get request headers with optional API key"""
headers = {
'User-Agent': 'ComfyUI-LoRA-Manager/1.0',
'Content-Type': 'application/json'
}
from .settings_manager import settings
api_key = settings.get('civitai_api_key')
if (api_key):
headers['Authorization'] = f'Bearer {api_key}'
return headers
async def _download_file(self, url: str, save_dir: str, default_filename: str, progress_callback=None) -> Tuple[bool, str]:
"""Download file with content-disposition support and progress tracking
Args: Args:
url: Download URL url: Download URL
@@ -88,335 +113,164 @@ class CivitaiClient:
Returns: Returns:
Tuple[bool, str]: (success, save_path or error message) Tuple[bool, str]: (success, save_path or error message)
""" """
downloader = await get_downloader() logger.debug(f"Resolving DNS for: {url}")
save_path = os.path.join(save_dir, default_filename) session = await self._ensure_fresh_session()
# Use unified downloader with CivitAI authentication
success, result = await downloader.download_file(
url=url,
save_path=save_path,
progress_callback=progress_callback,
use_auth=True, # Enable CivitAI authentication
allow_resume=True
)
return success, result
async def get_model_by_hash(self, model_hash: str) -> Tuple[Optional[Dict], Optional[str]]:
try: try:
success, version = await self._make_request( headers = self._get_request_headers()
'GET',
f"{self.base_url}/model-versions/by-hash/{model_hash}", # Add Range header to allow resumable downloads
use_auth=True headers['Accept-Encoding'] = 'identity' # Disable compression for better chunked downloads
)
if not success: logger.debug(f"Starting download from: {url}")
message = str(version) async with session.get(url, headers=headers, allow_redirects=True) as response:
if "not found" in message.lower(): if response.status != 200:
return None, "Model not found" # Handle 401 unauthorized responses
if response.status == 401:
logger.warning(f"Unauthorized access to resource: {url} (Status 401)")
return False, "Invalid or missing CivitAI API key, or early access restriction."
# Handle other client errors that might be permission-related
if response.status == 403:
logger.warning(f"Forbidden access to resource: {url} (Status 403)")
return False, "Access forbidden: You don't have permission to download this file."
# Generic error response for other status codes
logger.error(f"Download failed for {url} with status {response.status}")
return False, f"Download failed with status {response.status}"
logger.error("Failed to fetch model info for %s: %s", model_hash[:10], message) # Get filename from content-disposition header
return None, message content_disposition = response.headers.get('Content-Disposition')
filename = self._parse_content_disposition(content_disposition)
if not filename:
filename = default_filename
save_path = os.path.join(save_dir, filename)
# Get total file size for progress calculation
total_size = int(response.headers.get('content-length', 0))
current_size = 0
last_progress_report_time = datetime.now()
model_id = version.get('modelId') # Stream download to file with progress updates using larger buffer
if model_id: with open(save_path, 'wb') as f:
model_data = await self._fetch_model_data(model_id) async for chunk in response.content.iter_chunked(self.chunk_size):
if model_data: if chunk:
self._enrich_version_with_model_data(version, model_data) f.write(chunk)
current_size += len(chunk)
# Limit progress update frequency to reduce overhead
now = datetime.now()
time_diff = (now - last_progress_report_time).total_seconds()
if progress_callback and total_size and time_diff >= 1.0:
progress = (current_size / total_size) * 100
await progress_callback(progress)
last_progress_report_time = now
# Ensure 100% progress is reported
if progress_callback:
await progress_callback(100)
return True, save_path
except aiohttp.ClientError as e:
logger.error(f"Network error during download: {e}")
return False, f"Network error: {str(e)}"
except Exception as e:
logger.error(f"Download error: {e}")
return False, str(e)
self._remove_comfy_metadata(version) async def get_model_by_hash(self, model_hash: str) -> Optional[Dict]:
return version, None try:
except RateLimitError: session = await self._ensure_fresh_session()
raise async with session.get(f"{self.base_url}/model-versions/by-hash/{model_hash}") as response:
except Exception as exc: if response.status == 200:
logger.error("API Error: %s", exc) return await response.json()
return None, str(exc) return None
except Exception as e:
logger.error(f"API Error: {str(e)}")
return None
async def download_preview_image(self, image_url: str, save_path: str): async def download_preview_image(self, image_url: str, save_path: str):
try: try:
downloader = await get_downloader() session = await self._ensure_fresh_session()
success, content, headers = await downloader.download_to_memory( async with session.get(image_url) as response:
image_url, if response.status == 200:
use_auth=False # Preview images don't need auth content = await response.read()
) with open(save_path, 'wb') as f:
if success: f.write(content)
# Ensure directory exists return True
os.makedirs(os.path.dirname(save_path), exist_ok=True) return False
with open(save_path, 'wb') as f:
f.write(content)
return True
return False
except Exception as e: except Exception as e:
logger.error(f"Download Error: {str(e)}") print(f"Download Error: {str(e)}")
return False return False
@staticmethod async def get_model_versions(self, model_id: str) -> List[Dict]:
def _extract_error_message(payload: Any) -> str:
"""Return a human-readable error message from an API payload."""
def _from_value(value: Any) -> str:
if isinstance(value, str):
return value
if isinstance(value, dict):
for key in ("message", "error", "detail", "details"):
if key in value:
candidate = _from_value(value[key])
if candidate:
return candidate
if isinstance(value, list):
for item in value:
candidate = _from_value(item)
if candidate:
return candidate
return ""
return _from_value(payload)
async def get_model_versions(self, model_id: str) -> Optional[Dict]:
"""Get all versions of a model with local availability info""" """Get all versions of a model with local availability info"""
try: try:
success, result = await self._make_request( session = await self._ensure_fresh_session() # Use fresh session
'GET', async with session.get(f"{self.base_url}/models/{model_id}") as response:
f"{self.base_url}/models/{model_id}", if response.status != 200:
use_auth=True return None
) data = await response.json()
if success:
# Also return model type along with versions # Also return model type along with versions
return { return {
'modelVersions': result.get('modelVersions', []), 'modelVersions': data.get('modelVersions', []),
'type': result.get('type', ''), 'type': data.get('type', '')
'name': result.get('name', '')
} }
message = self._extract_error_message(result)
if message and 'not found' in message.lower():
raise ResourceNotFoundError(f"Resource not found for model {model_id}")
if message:
raise RuntimeError(message)
return None
except RateLimitError:
raise
except ResourceNotFoundError as exc:
logger.info("Model %s is no longer available on Civitai: %s", model_id, exc)
raise
except Exception as e: except Exception as e:
logger.error("Error fetching model versions: %s", e, exc_info=True) logger.error(f"Error fetching model versions: {e}")
raise
async def get_model_versions_bulk(
self, model_ids: Sequence[int]
) -> Optional[Dict[int, Dict]]:
"""Fetch model metadata for multiple ids using the batch API."""
deduped: Dict[int, None] = {}
for raw_id in model_ids:
try:
normalized = int(raw_id)
except (TypeError, ValueError):
continue
deduped.setdefault(normalized, None)
normalized_ids = [str(model_id) for model_id in deduped.keys()]
if not normalized_ids:
return {}
try:
query = ",".join(normalized_ids)
success, result = await self._make_request(
'GET',
f"{self.base_url}/models",
use_auth=True,
params={'ids': query},
)
if not success:
return None
items = result.get('items') if isinstance(result, dict) else None
if not isinstance(items, list):
return {}
payload: Dict[int, Dict] = {}
for item in items:
if not isinstance(item, dict):
continue
model_id = item.get('id')
try:
normalized_id = int(model_id)
except (TypeError, ValueError):
continue
payload[normalized_id] = {
'modelVersions': item.get('modelVersions', []),
'type': item.get('type', ''),
'name': item.get('name', ''),
'allowNoCredit': item.get('allowNoCredit'),
'allowCommercialUse': item.get('allowCommercialUse'),
'allowDerivatives': item.get('allowDerivatives'),
'allowDifferentLicense': item.get('allowDifferentLicense'),
}
return payload
except RateLimitError:
raise
except Exception as exc:
logger.error(f"Error fetching model versions in bulk: {exc}")
return None return None
async def get_model_version(self, model_id: int = None, version_id: int = None) -> Optional[Dict]: async def get_model_version(self, model_id: int, version_id: int = None) -> Optional[Dict]:
"""Get specific model version with additional metadata.""" """Get specific model version with additional metadata
Args:
model_id: The Civitai model ID
version_id: Optional specific version ID to retrieve
Returns:
Optional[Dict]: The model version data with additional fields or None if not found
"""
try: try:
if model_id is None and version_id is not None: session = await self._ensure_fresh_session()
return await self._get_version_by_id_only(version_id)
# Step 1: Get model data to find version_id if not provided and get additional metadata
if model_id is not None: async with session.get(f"{self.base_url}/models/{model_id}") as response:
return await self._get_version_with_model_id(model_id, version_id) if response.status != 200:
return None
logger.error("Either model_id or version_id must be provided")
return None data = await response.json()
model_versions = data.get('modelVersions', [])
except RateLimitError:
raise # Step 2: Determine the version_id to use
target_version_id = version_id
if target_version_id is None:
target_version_id = model_versions[0].get('id')
# Step 3: Get detailed version info using the version_id
headers = self._get_request_headers()
async with session.get(f"{self.base_url}/model-versions/{target_version_id}", headers=headers) as response:
if response.status != 200:
return None
version = await response.json()
# Step 4: Enrich version_info with model data
# Add description and tags from model data
version['model']['description'] = data.get("description")
version['model']['tags'] = data.get("tags", [])
# Add creator from model data
version['creator'] = data.get("creator")
return version
except Exception as e: except Exception as e:
logger.error(f"Error fetching model version: {e}") logger.error(f"Error fetching model version: {e}")
return None return None
async def _get_version_by_id_only(self, version_id: int) -> Optional[Dict]:
version = await self._fetch_version_by_id(version_id)
if version is None:
return None
model_id = version.get('modelId')
if not model_id:
logger.error(f"No modelId found in version {version_id}")
return None
model_data = await self._fetch_model_data(model_id)
if model_data:
self._enrich_version_with_model_data(version, model_data)
self._remove_comfy_metadata(version)
return version
async def _get_version_with_model_id(self, model_id: int, version_id: Optional[int]) -> Optional[Dict]:
model_data = await self._fetch_model_data(model_id)
if not model_data:
return None
target_version = self._select_target_version(model_data, model_id, version_id)
if target_version is None:
return None
target_version_id = target_version.get('id')
version = await self._fetch_version_by_id(target_version_id) if target_version_id else None
if version is None:
model_hash = self._extract_primary_model_hash(target_version)
if model_hash:
version = await self._fetch_version_by_hash(model_hash)
else:
logger.warning(
f"No primary model hash found for model {model_id} version {target_version_id}"
)
if version is None:
version = self._build_version_from_model_data(target_version, model_id, model_data)
self._enrich_version_with_model_data(version, model_data)
self._remove_comfy_metadata(version)
return version
async def _fetch_model_data(self, model_id: int) -> Optional[Dict]:
success, data = await self._make_request(
'GET',
f"{self.base_url}/models/{model_id}",
use_auth=True
)
if success:
return data
logger.warning(f"Failed to fetch model data for model {model_id}")
return None
async def _fetch_version_by_id(self, version_id: Optional[int]) -> Optional[Dict]:
if version_id is None:
return None
success, version = await self._make_request(
'GET',
f"{self.base_url}/model-versions/{version_id}",
use_auth=True
)
if success:
return version
logger.warning(f"Failed to fetch version by id {version_id}")
return None
async def _fetch_version_by_hash(self, model_hash: Optional[str]) -> Optional[Dict]:
if not model_hash:
return None
success, version = await self._make_request(
'GET',
f"{self.base_url}/model-versions/by-hash/{model_hash}",
use_auth=True
)
if success:
return version
logger.warning(f"Failed to fetch version by hash {model_hash}")
return None
def _select_target_version(self, model_data: Dict, model_id: int, version_id: Optional[int]) -> Optional[Dict]:
model_versions = model_data.get('modelVersions', [])
if not model_versions:
logger.warning(f"No model versions found for model {model_id}")
return None
if version_id is not None:
target_version = next(
(item for item in model_versions if item.get('id') == version_id),
None
)
if target_version is None:
logger.warning(
f"Version {version_id} not found for model {model_id}, defaulting to first version"
)
return model_versions[0]
return target_version
return model_versions[0]
def _extract_primary_model_hash(self, version_entry: Dict) -> Optional[str]:
for file_info in version_entry.get('files', []):
if file_info.get('type') == 'Model' and file_info.get('primary'):
hashes = file_info.get('hashes', {})
model_hash = hashes.get('SHA256')
if model_hash:
return model_hash
return None
def _build_version_from_model_data(self, version_entry: Dict, model_id: int, model_data: Dict) -> Dict:
version = copy.deepcopy(version_entry)
version.pop('index', None)
version['modelId'] = model_id
version['model'] = {
'name': model_data.get('name'),
'type': model_data.get('type'),
'nsfw': model_data.get('nsfw'),
'poi': model_data.get('poi')
}
return version
def _enrich_version_with_model_data(self, version: Dict, model_data: Dict) -> None:
model_info = version.get('model')
if not isinstance(model_info, dict):
model_info = {}
version['model'] = model_info
model_info['description'] = model_data.get("description")
model_info['tags'] = model_data.get("tags", [])
version['creator'] = model_data.get("creator")
license_payload = resolve_license_payload(model_data)
for field, value in license_payload.items():
model_info[field] = value
async def get_model_version_info(self, version_id: str) -> Tuple[Optional[Dict], Optional[str]]: async def get_model_version_info(self, version_id: str) -> Tuple[Optional[Dict], Optional[str]]:
"""Fetch model version metadata from Civitai """Fetch model version metadata from Civitai
@@ -429,39 +283,119 @@ class CivitaiClient:
- An error message if there was an error, or None on success - An error message if there was an error, or None on success
""" """
try: try:
session = await self._ensure_fresh_session()
url = f"{self.base_url}/model-versions/{version_id}" url = f"{self.base_url}/model-versions/{version_id}"
headers = self._get_request_headers()
logger.debug(f"Resolving DNS for model version info: {url}") logger.debug(f"Resolving DNS for model version info: {url}")
success, result = await self._make_request( async with session.get(url, headers=headers) as response:
'GET', if response.status == 200:
url, logger.debug(f"Successfully fetched model version info for: {version_id}")
use_auth=True return await response.json(), None
)
# Handle specific error cases
if success: if response.status == 404:
logger.debug(f"Successfully fetched model version info for: {version_id}") # Try to parse the error message
self._remove_comfy_metadata(result) try:
return result, None error_data = await response.json()
error_msg = error_data.get('error', f"Model not found (status 404)")
# Handle specific error cases logger.warning(f"Model version not found: {version_id} - {error_msg}")
if "not found" in str(result): return None, error_msg
error_msg = f"Model not found" except:
logger.warning(f"Model version not found: {version_id} - {error_msg}") return None, "Model not found (status 404)"
return None, error_msg
# Other error cases
# Other error cases logger.error(f"Failed to fetch model info for {version_id} (status {response.status})")
logger.error(f"Failed to fetch model info for {version_id}: {result}") return None, f"Failed to fetch model info (status {response.status})"
return None, str(result)
except RateLimitError:
raise
except Exception as e: except Exception as e:
error_msg = f"Error fetching model version info: {e}" error_msg = f"Error fetching model version info: {e}"
logger.error(error_msg) logger.error(error_msg)
return None, error_msg return None, error_msg
async def get_model_metadata(self, model_id: str) -> Tuple[Optional[Dict], int]:
"""Fetch model metadata (description, tags, and creator info) from Civitai API
Args:
model_id: The Civitai model ID
Returns:
Tuple[Optional[Dict], int]: A tuple containing:
- A dictionary with model metadata or None if not found
- The HTTP status code from the request
"""
try:
session = await self._ensure_fresh_session()
headers = self._get_request_headers()
url = f"{self.base_url}/models/{model_id}"
async with session.get(url, headers=headers) as response:
status_code = response.status
if status_code != 200:
logger.warning(f"Failed to fetch model metadata: Status {status_code}")
return None, status_code
data = await response.json()
# Extract relevant metadata
metadata = {
"description": data.get("description") or "No model description available",
"tags": data.get("tags", []),
"creator": {
"username": data.get("creator", {}).get("username"),
"image": data.get("creator", {}).get("image")
}
}
if metadata["description"] or metadata["tags"] or metadata["creator"]["username"]:
return metadata, status_code
else:
logger.warning(f"No metadata found for model {model_id}")
return None, status_code
except Exception as e:
logger.error(f"Error fetching model metadata: {e}", exc_info=True)
return None, 0
# Keep old method for backward compatibility, delegating to the new one
async def get_model_description(self, model_id: str) -> Optional[str]:
"""Fetch the model description from Civitai API (Legacy method)"""
metadata, _ = await self.get_model_metadata(model_id)
return metadata.get("description") if metadata else None
async def close(self):
"""Close the session if it exists"""
if self._session is not None:
await self._session.close()
self._session = None
async def _get_hash_from_civitai(self, model_version_id: str) -> Optional[str]:
"""Get hash from Civitai API"""
try:
session = await self._ensure_fresh_session()
if not session:
return None
version_info = await session.get(f"{self.base_url}/model-versions/{model_version_id}")
if not version_info or not version_info.json().get('files'):
return None
# Get hash from the first file
for file_info in version_info.json().get('files', []):
if file_info.get('hashes', {}).get('SHA256'):
# Convert hash to lowercase to standardize
hash_value = file_info['hashes']['SHA256'].lower()
return hash_value
return None
except Exception as e:
logger.error(f"Error getting hash from Civitai: {e}")
return None
async def get_image_info(self, image_id: str) -> Optional[Dict]: async def get_image_info(self, image_id: str) -> Optional[Dict]:
"""Fetch image information from Civitai API """Fetch image information from Civitai API
Args: Args:
image_id: The Civitai image ID image_id: The Civitai image ID
@@ -469,62 +403,23 @@ class CivitaiClient:
Optional[Dict]: The image data or None if not found Optional[Dict]: The image data or None if not found
""" """
try: try:
session = await self._ensure_fresh_session()
headers = self._get_request_headers()
url = f"{self.base_url}/images?imageId={image_id}&nsfw=X" url = f"{self.base_url}/images?imageId={image_id}&nsfw=X"
logger.debug(f"Fetching image info for ID: {image_id}") logger.debug(f"Fetching image info for ID: {image_id}")
success, result = await self._make_request( async with session.get(url, headers=headers) as response:
'GET', if response.status == 200:
url, data = await response.json()
use_auth=True if data and "items" in data and len(data["items"]) > 0:
) logger.debug(f"Successfully fetched image info for ID: {image_id}")
return data["items"][0]
if success: logger.warning(f"No image found with ID: {image_id}")
if result and "items" in result and len(result["items"]) > 0: return None
logger.debug(f"Successfully fetched image info for ID: {image_id}")
return result["items"][0] logger.error(f"Failed to fetch image info for ID: {image_id} (status {response.status})")
logger.warning(f"No image found with ID: {image_id}")
return None return None
logger.error(f"Failed to fetch image info for ID: {image_id}: {result}")
return None
except RateLimitError:
raise
except Exception as e: except Exception as e:
error_msg = f"Error fetching image info: {e}" error_msg = f"Error fetching image info: {e}"
logger.error(error_msg) logger.error(error_msg)
return None return None
async def get_user_models(self, username: str) -> Optional[List[Dict]]:
"""Fetch all models for a specific Civitai user."""
if not username:
return None
try:
url = f"{self.base_url}/models?username={username}"
success, result = await self._make_request(
'GET',
url,
use_auth=True
)
if not success:
logger.error("Failed to fetch models for %s: %s", username, result)
return None
items = result.get("items") if isinstance(result, dict) else None
if not isinstance(items, list):
return []
for model in items:
versions = model.get("modelVersions")
if not isinstance(versions, list):
continue
for version in versions:
self._remove_comfy_metadata(version)
return items
except RateLimitError:
raise
except Exception as exc: # pragma: no cover - defensive logging
logger.error("Error fetching models for %s: %s", username, exc)
return None

View File

@@ -1,178 +0,0 @@
"""Service wrapper for coordinating download lifecycle events."""
from __future__ import annotations
import logging
from typing import Any, Awaitable, Callable, Dict, Optional
from .downloader import DownloadProgress
logger = logging.getLogger(__name__)
class DownloadCoordinator:
"""Manage download scheduling, cancellation and introspection."""
def __init__(
self,
*,
ws_manager,
download_manager_factory: Callable[[], Awaitable],
) -> None:
self._ws_manager = ws_manager
self._download_manager_factory = download_manager_factory
async def schedule_download(self, payload: Dict[str, Any]) -> Dict[str, Any]:
"""Schedule a download using the provided payload."""
download_manager = await self._download_manager_factory()
download_id = payload.get("download_id") or self._ws_manager.generate_download_id()
payload.setdefault("download_id", download_id)
async def progress_callback(progress: Any, snapshot: Optional[DownloadProgress] = None) -> None:
percent = 0.0
metrics: Optional[DownloadProgress] = None
if isinstance(progress, DownloadProgress):
metrics = progress
percent = progress.percent_complete
elif isinstance(snapshot, DownloadProgress):
metrics = snapshot
percent = snapshot.percent_complete
else:
try:
percent = float(progress)
except (TypeError, ValueError):
percent = 0.0
payload: Dict[str, Any] = {
"status": "progress",
"progress": round(percent),
"download_id": download_id,
}
if metrics is not None:
payload.update(
{
"bytes_downloaded": metrics.bytes_downloaded,
"total_bytes": metrics.total_bytes,
"bytes_per_second": metrics.bytes_per_second,
}
)
await self._ws_manager.broadcast_download_progress(
download_id,
payload,
)
model_id = self._parse_optional_int(payload.get("model_id"), "model_id")
model_version_id = self._parse_optional_int(
payload.get("model_version_id"), "model_version_id"
)
if model_id is None and model_version_id is None:
raise ValueError(
"Missing required parameter: Please provide either 'model_id' or 'model_version_id'"
)
result = await download_manager.download_from_civitai(
model_id=model_id,
model_version_id=model_version_id,
save_dir=payload.get("model_root"),
relative_path=payload.get("relative_path", ""),
use_default_paths=payload.get("use_default_paths", False),
progress_callback=progress_callback,
download_id=download_id,
source=payload.get("source"),
)
result["download_id"] = download_id
return result
async def cancel_download(self, download_id: str) -> Dict[str, Any]:
"""Cancel an active download and emit a broadcast event."""
download_manager = await self._download_manager_factory()
result = await download_manager.cancel_download(download_id)
await self._ws_manager.broadcast_download_progress(
download_id,
{
"status": "cancelled",
"progress": 0,
"download_id": download_id,
"message": "Download cancelled by user",
},
)
return result
async def pause_download(self, download_id: str) -> Dict[str, Any]:
"""Pause an active download and notify listeners."""
download_manager = await self._download_manager_factory()
result = await download_manager.pause_download(download_id)
if result.get("success"):
cached_progress = self._ws_manager.get_download_progress(download_id) or {}
payload: Dict[str, Any] = {
"status": "paused",
"progress": cached_progress.get("progress", 0),
"download_id": download_id,
"message": "Download paused by user",
}
for field in ("bytes_downloaded", "total_bytes", "bytes_per_second"):
if field in cached_progress:
payload[field] = cached_progress[field]
payload["bytes_per_second"] = 0.0
await self._ws_manager.broadcast_download_progress(download_id, payload)
return result
async def resume_download(self, download_id: str) -> Dict[str, Any]:
"""Resume a paused download and notify listeners."""
download_manager = await self._download_manager_factory()
result = await download_manager.resume_download(download_id)
if result.get("success"):
cached_progress = self._ws_manager.get_download_progress(download_id) or {}
payload: Dict[str, Any] = {
"status": "downloading",
"progress": cached_progress.get("progress", 0),
"download_id": download_id,
"message": "Download resumed by user",
}
for field in ("bytes_downloaded", "total_bytes"):
if field in cached_progress:
payload[field] = cached_progress[field]
payload["bytes_per_second"] = cached_progress.get("bytes_per_second", 0.0)
await self._ws_manager.broadcast_download_progress(download_id, payload)
return result
async def list_active_downloads(self) -> Dict[str, Any]:
"""Return the active download map from the underlying manager."""
download_manager = await self._download_manager_factory()
return await download_manager.get_active_downloads()
def _parse_optional_int(self, value: Any, field: str) -> Optional[int]:
"""Parse an optional integer from user input."""
if value is None or value == "":
return None
try:
return int(value)
except (TypeError, ValueError) as exc:
raise ValueError(f"Invalid {field}: Must be an integer") from exc

File diff suppressed because it is too large Load Diff

View File

@@ -1,863 +0,0 @@
"""
Unified download manager for all HTTP/HTTPS downloads in the application.
This module provides a centralized download service with:
- Singleton pattern for global session management
- Support for authenticated downloads (e.g., CivitAI API key)
- Resumable downloads with automatic retry
- Progress tracking and callbacks
- Optimized connection pooling and timeouts
- Unified error handling and logging
"""
import os
import logging
import asyncio
import aiohttp
from collections import deque
from dataclasses import dataclass
from datetime import datetime, timedelta
from email.utils import parsedate_to_datetime
from typing import Optional, Dict, Tuple, Callable, Union, Awaitable
from ..services.settings_manager import get_settings_manager
from .errors import RateLimitError
logger = logging.getLogger(__name__)
@dataclass(frozen=True)
class DownloadProgress:
"""Snapshot of a download transfer at a moment in time."""
percent_complete: float
bytes_downloaded: int
total_bytes: Optional[int]
bytes_per_second: float
timestamp: float
class DownloadStreamControl:
"""Synchronize pause/resume requests and reconnect hints for a download."""
def __init__(self, *, stall_timeout: Optional[float] = None) -> None:
self._event = asyncio.Event()
self._event.set()
self._reconnect_requested = False
self.last_progress_timestamp: Optional[float] = None
self.stall_timeout: float = float(stall_timeout) if stall_timeout is not None else 120.0
def is_set(self) -> bool:
return self._event.is_set()
def is_paused(self) -> bool:
return not self._event.is_set()
def set(self) -> None:
self._event.set()
def clear(self) -> None:
self._event.clear()
async def wait(self) -> None:
await self._event.wait()
def pause(self) -> None:
self.clear()
def resume(self, *, force_reconnect: bool = False) -> None:
if force_reconnect:
self._reconnect_requested = True
self.set()
def request_reconnect(self) -> None:
self._reconnect_requested = True
self.set()
def has_reconnect_request(self) -> bool:
return self._reconnect_requested
def consume_reconnect_request(self) -> bool:
reconnect = self._reconnect_requested
self._reconnect_requested = False
return reconnect
def mark_progress(self, timestamp: Optional[float] = None) -> None:
self.last_progress_timestamp = timestamp or datetime.now().timestamp()
self._reconnect_requested = False
def time_since_last_progress(self, *, now: Optional[float] = None) -> Optional[float]:
if self.last_progress_timestamp is None:
return None
reference = now if now is not None else datetime.now().timestamp()
return max(0.0, reference - self.last_progress_timestamp)
def update_stall_timeout(self, stall_timeout: float) -> None:
self.stall_timeout = float(stall_timeout)
class DownloadRestartRequested(Exception):
"""Raised when a caller explicitly requests a fresh HTTP stream."""
class DownloadStalledError(Exception):
"""Raised when download progress stalls beyond the configured timeout."""
class Downloader:
"""Unified downloader for all HTTP/HTTPS downloads in the application."""
_instance = None
_lock = asyncio.Lock()
@classmethod
async def get_instance(cls):
"""Get singleton instance of Downloader"""
async with cls._lock:
if cls._instance is None:
cls._instance = cls()
return cls._instance
def __init__(self):
"""Initialize the downloader with optimal settings"""
# Check if already initialized for singleton pattern
if hasattr(self, '_initialized'):
return
self._initialized = True
# Session management
self._session = None
self._session_created_at = None
self._proxy_url = None # Store proxy URL for current session
self._session_lock = asyncio.Lock()
# Configuration
self.chunk_size = 4 * 1024 * 1024 # 4MB chunks for better throughput
self.max_retries = 5
self.base_delay = 2.0 # Base delay for exponential backoff
self.session_timeout = 300 # 5 minutes
self.stall_timeout = self._resolve_stall_timeout()
# Default headers
self.default_headers = {
'User-Agent': 'ComfyUI-LoRA-Manager/1.0',
# Explicitly request uncompressed payloads so aiohttp doesn't need optional
# decoders (e.g. zstandard) that may be missing in runtime environments.
'Accept-Encoding': 'identity',
}
@property
async def session(self) -> aiohttp.ClientSession:
"""Get or create the global aiohttp session with optimized settings"""
if self._session is None or self._should_refresh_session():
async with self._session_lock:
# Double check after acquiring lock
if self._session is None or self._should_refresh_session():
await self._create_session()
return self._session
@property
def proxy_url(self) -> Optional[str]:
"""Get the current proxy URL (initialize if needed)"""
if not hasattr(self, '_proxy_url'):
self._proxy_url = None
return self._proxy_url
def _resolve_stall_timeout(self) -> float:
"""Determine the stall timeout from settings or environment."""
default_timeout = 120.0
settings_timeout = None
try:
settings_manager = get_settings_manager()
settings_timeout = settings_manager.get('download_stall_timeout_seconds')
except Exception as exc: # pragma: no cover - defensive guard
logger.debug("Failed to read stall timeout from settings: %s", exc)
raw_value = (
settings_timeout
if settings_timeout not in (None, "")
else os.environ.get('COMFYUI_DOWNLOAD_STALL_TIMEOUT')
)
try:
timeout_value = float(raw_value)
except (TypeError, ValueError):
timeout_value = default_timeout
return max(30.0, timeout_value)
def _should_refresh_session(self) -> bool:
"""Check if session should be refreshed"""
if self._session is None:
return True
if not hasattr(self, '_session_created_at') or self._session_created_at is None:
return True
# Refresh if session is older than timeout
if (datetime.now() - self._session_created_at).total_seconds() > self.session_timeout:
return True
return False
async def _create_session(self):
"""Create a new aiohttp session with optimized settings.
Note: This is private and caller MUST hold self._session_lock.
"""
# Close existing session if any
if self._session is not None:
try:
await self._session.close()
except Exception as e: # pragma: no cover
logger.warning(f"Error closing previous session: {e}")
finally:
self._session = None
# Check for app-level proxy settings
proxy_url = None
settings_manager = get_settings_manager()
if settings_manager.get('proxy_enabled', False):
proxy_host = settings_manager.get('proxy_host', '').strip()
proxy_port = settings_manager.get('proxy_port', '').strip()
proxy_type = settings_manager.get('proxy_type', 'http').lower()
proxy_username = settings_manager.get('proxy_username', '').strip()
proxy_password = settings_manager.get('proxy_password', '').strip()
if proxy_host and proxy_port:
# Build proxy URL
if proxy_username and proxy_password:
proxy_url = f"{proxy_type}://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"
else:
proxy_url = f"{proxy_type}://{proxy_host}:{proxy_port}"
logger.debug(f"Using app-level proxy: {proxy_type}://{proxy_host}:{proxy_port}")
logger.debug("Proxy mode: app-level proxy is active.")
else:
logger.debug("Proxy mode: system-level proxy (trust_env) will be used if configured in environment.")
# Optimize TCP connection parameters
connector = aiohttp.TCPConnector(
ssl=True,
limit=8, # Concurrent connections
ttl_dns_cache=300, # DNS cache timeout
force_close=False, # Keep connections for reuse
enable_cleanup_closed=True
)
# Configure timeout parameters
timeout = aiohttp.ClientTimeout(
total=None, # No total timeout for large downloads
connect=60, # Connection timeout
sock_read=300 # 5 minute socket read timeout
)
self._session = aiohttp.ClientSession(
connector=connector,
trust_env=proxy_url is None, # Only use system proxy if no app-level proxy is set
timeout=timeout
)
# Store proxy URL for use in requests
self._proxy_url = proxy_url
self._session_created_at = datetime.now()
logger.debug("Created new HTTP session with proxy settings. App-level proxy: %s, System-level proxy (trust_env): %s", bool(proxy_url), proxy_url is None)
def _get_auth_headers(self, use_auth: bool = False) -> Dict[str, str]:
"""Get headers with optional authentication"""
headers = self.default_headers.copy()
if use_auth:
# Add CivitAI API key if available
settings_manager = get_settings_manager()
api_key = settings_manager.get('civitai_api_key')
if api_key:
headers['Authorization'] = f'Bearer {api_key}'
headers['Content-Type'] = 'application/json'
return headers
async def download_file(
self,
url: str,
save_path: str,
progress_callback: Optional[Callable[..., Awaitable[None]]] = None,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
allow_resume: bool = True,
pause_event: Optional[DownloadStreamControl] = None,
) -> Tuple[bool, str]:
"""
Download a file with resumable downloads and retry mechanism
Args:
url: Download URL
save_path: Full path where the file should be saved
progress_callback: Optional callback for progress updates (0-100)
use_auth: Whether to include authentication headers (e.g., CivitAI API key)
custom_headers: Additional headers to include in request
allow_resume: Whether to support resumable downloads
pause_event: Optional stream control used to pause/resume and request reconnects
Returns:
Tuple[bool, str]: (success, save_path or error message)
"""
retry_count = 0
part_path = save_path + '.part' if allow_resume else save_path
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
# Get existing file size for resume
resume_offset = 0
if allow_resume and os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Resuming download from offset {resume_offset} bytes")
total_size = 0
while retry_count <= self.max_retries:
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[download_file] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[download_file] Using system-level proxy (trust_env) if configured.")
# Add Range header for resume if we have partial data
request_headers = headers.copy()
if allow_resume and resume_offset > 0:
request_headers['Range'] = f'bytes={resume_offset}-'
# Disable compression for better chunked downloads
request_headers['Accept-Encoding'] = 'identity'
logger.debug(f"Download attempt {retry_count + 1}/{self.max_retries + 1} from: {url}")
if resume_offset > 0:
logger.debug(f"Requesting range from byte {resume_offset}")
async with session.get(url, headers=request_headers, allow_redirects=True, proxy=self.proxy_url) as response:
# Handle different response codes
if response.status == 200:
# Full content response
if resume_offset > 0:
# Server doesn't support ranges, restart from beginning
logger.warning("Server doesn't support range requests, restarting download")
resume_offset = 0
if os.path.exists(part_path):
os.remove(part_path)
elif response.status == 206:
# Partial content response (resume successful)
content_range = response.headers.get('Content-Range')
if content_range:
# Parse total size from Content-Range header (e.g., "bytes 1024-2047/2048")
range_parts = content_range.split('/')
if len(range_parts) == 2:
total_size = int(range_parts[1])
logger.info(f"Successfully resumed download from byte {resume_offset}")
elif response.status == 416:
# Range not satisfiable - file might be complete or corrupted
if allow_resume and os.path.exists(part_path):
part_size = os.path.getsize(part_path)
logger.warning(f"Range not satisfiable. Part file size: {part_size}")
# Try to get actual file size
head_response = await session.head(url, headers=headers, proxy=self.proxy_url)
if head_response.status == 200:
actual_size = int(head_response.headers.get('content-length', 0))
if part_size == actual_size:
# File is complete, just rename it
if allow_resume:
os.rename(part_path, save_path)
if progress_callback:
await self._dispatch_progress_callback(
progress_callback,
DownloadProgress(
percent_complete=100.0,
bytes_downloaded=part_size,
total_bytes=actual_size,
bytes_per_second=0.0,
timestamp=datetime.now().timestamp(),
),
)
return True, save_path
# Remove corrupted part file and restart
os.remove(part_path)
resume_offset = 0
continue
elif response.status == 401:
logger.warning(f"Unauthorized access to resource: {url} (Status 401)")
return False, "Invalid or missing API key, or early access restriction."
elif response.status == 403:
logger.warning(f"Forbidden access to resource: {url} (Status 403)")
return False, "Access forbidden: You don't have permission to download this file."
elif response.status == 404:
logger.warning(f"Resource not found: {url} (Status 404)")
return False, "File not found - the download link may be invalid or expired."
else:
logger.error(f"Download failed for {url} with status {response.status}")
return False, f"Download failed with status {response.status}"
# Get total file size for progress calculation (if not set from Content-Range)
if total_size == 0:
total_size = int(response.headers.get('content-length', 0))
if response.status == 206:
# For partial content, add the offset to get total file size
total_size += resume_offset
current_size = resume_offset
last_progress_report_time = datetime.now()
progress_samples: deque[tuple[datetime, int]] = deque()
progress_samples.append((last_progress_report_time, current_size))
# Ensure directory exists
os.makedirs(os.path.dirname(save_path), exist_ok=True)
# Stream download to file with progress updates
loop = asyncio.get_running_loop()
mode = 'ab' if (allow_resume and resume_offset > 0) else 'wb'
control = pause_event
if control is not None:
control.update_stall_timeout(self.stall_timeout)
with open(part_path, mode) as f:
while True:
active_stall_timeout = control.stall_timeout if control else self.stall_timeout
if control is not None:
if control.is_paused():
await control.wait()
resume_time = datetime.now()
last_progress_report_time = resume_time
if control.consume_reconnect_request():
raise DownloadRestartRequested(
"Reconnect requested after resume"
)
elif control.consume_reconnect_request():
raise DownloadRestartRequested("Reconnect requested")
try:
chunk = await asyncio.wait_for(
response.content.read(self.chunk_size),
timeout=active_stall_timeout,
)
except asyncio.TimeoutError as exc:
logger.warning(
"Download stalled for %.1f seconds without progress from %s",
active_stall_timeout,
url,
)
raise DownloadStalledError(
f"No data received for {active_stall_timeout:.1f} seconds"
) from exc
if not chunk:
break
# Run blocking file write in executor
await loop.run_in_executor(None, f.write, chunk)
current_size += len(chunk)
now = datetime.now()
if control is not None:
control.mark_progress(timestamp=now.timestamp())
# Limit progress update frequency to reduce overhead
time_diff = (now - last_progress_report_time).total_seconds()
if progress_callback and time_diff >= 1.0:
progress_samples.append((now, current_size))
cutoff = now - timedelta(seconds=5)
while progress_samples and progress_samples[0][0] < cutoff:
progress_samples.popleft()
percent = (current_size / total_size) * 100 if total_size else 0.0
bytes_per_second = 0.0
if len(progress_samples) >= 2:
first_time, first_bytes = progress_samples[0]
last_time, last_bytes = progress_samples[-1]
elapsed = (last_time - first_time).total_seconds()
if elapsed > 0:
bytes_per_second = (last_bytes - first_bytes) / elapsed
progress_snapshot = DownloadProgress(
percent_complete=percent,
bytes_downloaded=current_size,
total_bytes=total_size or None,
bytes_per_second=bytes_per_second,
timestamp=now.timestamp(),
)
await self._dispatch_progress_callback(progress_callback, progress_snapshot)
last_progress_report_time = now
# Download completed successfully
# Verify file size integrity before finalizing
final_size = os.path.getsize(part_path) if os.path.exists(part_path) else 0
expected_size = total_size if total_size > 0 else None
integrity_error: Optional[str] = None
if final_size <= 0:
integrity_error = "Downloaded file is empty"
elif expected_size is not None and final_size != expected_size:
integrity_error = (
f"File size mismatch. Expected: {expected_size}, Got: {final_size}"
)
if integrity_error is not None:
logger.error(
"Download integrity check failed for %s: %s",
save_path,
integrity_error,
)
# Remove the corrupted payload so future attempts start fresh
if os.path.exists(part_path):
try:
os.remove(part_path)
except OSError as remove_error:
logger.warning(
"Failed to delete corrupted download %s: %s",
part_path,
remove_error,
)
if part_path != save_path and os.path.exists(save_path):
try:
os.remove(save_path)
except OSError as remove_error:
logger.warning(
"Failed to delete target file %s after integrity error: %s",
save_path,
remove_error,
)
retry_count += 1
if retry_count <= self.max_retries:
delay = self.base_delay * (2 ** (retry_count - 1))
logger.info(
"Retrying download in %s seconds due to integrity check failure",
delay,
)
await asyncio.sleep(delay)
resume_offset = 0
total_size = 0
await self._create_session()
continue
return False, integrity_error
# Atomically rename .part to final file (only if using resume)
if allow_resume and part_path != save_path:
max_rename_attempts = 5
rename_attempt = 0
rename_success = False
while rename_attempt < max_rename_attempts and not rename_success:
try:
# If the destination file exists, remove it first (Windows safe)
if os.path.exists(save_path):
os.remove(save_path)
os.rename(part_path, save_path)
rename_success = True
except PermissionError as e:
rename_attempt += 1
if rename_attempt < max_rename_attempts:
logger.info(f"File still in use, retrying rename in 2 seconds (attempt {rename_attempt}/{max_rename_attempts})")
await asyncio.sleep(2)
else:
logger.error(f"Failed to rename file after {max_rename_attempts} attempts: {e}")
return False, f"Failed to finalize download: {str(e)}"
final_size = os.path.getsize(save_path)
# Ensure 100% progress is reported
if progress_callback:
final_snapshot = DownloadProgress(
percent_complete=100.0,
bytes_downloaded=final_size,
total_bytes=total_size or final_size,
bytes_per_second=0.0,
timestamp=datetime.now().timestamp(),
)
await self._dispatch_progress_callback(progress_callback, final_snapshot)
return True, save_path
except (
aiohttp.ClientError,
aiohttp.ClientPayloadError,
aiohttp.ServerDisconnectedError,
asyncio.TimeoutError,
DownloadStalledError,
DownloadRestartRequested,
) as e:
retry_count += 1
logger.warning(f"Network error during download (attempt {retry_count}/{self.max_retries + 1}): {e}")
if retry_count <= self.max_retries:
# Calculate delay with exponential backoff
delay = self.base_delay * (2 ** (retry_count - 1))
logger.info(f"Retrying in {delay} seconds...")
await asyncio.sleep(delay)
# Update resume offset for next attempt
if allow_resume and os.path.exists(part_path):
resume_offset = os.path.getsize(part_path)
logger.info(f"Will resume from byte {resume_offset}")
# Refresh session to get new connection
await self._create_session()
continue
else:
logger.error(f"Max retries exceeded for download: {e}")
return False, f"Network error after {self.max_retries + 1} attempts: {str(e)}"
except Exception as e:
logger.error(f"Unexpected download error: {e}")
return False, str(e)
return False, f"Download failed after {self.max_retries + 1} attempts"
async def _dispatch_progress_callback(
self,
progress_callback: Callable[..., Awaitable[None]],
snapshot: DownloadProgress,
) -> None:
"""Invoke a progress callback while preserving backward compatibility."""
try:
result = progress_callback(snapshot, snapshot)
except TypeError:
result = progress_callback(snapshot.percent_complete)
if asyncio.iscoroutine(result):
await result
elif hasattr(result, "__await__"):
await result
async def download_to_memory(
self,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
return_headers: bool = False
) -> Tuple[bool, Union[bytes, str], Optional[Dict]]:
"""
Download a file to memory (for small files like preview images)
Args:
url: Download URL
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
return_headers: Whether to return response headers along with content
Returns:
Tuple[bool, Union[bytes, str], Optional[Dict]]: (success, content or error message, response headers if requested)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[download_to_memory] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[download_to_memory] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
async with session.get(url, headers=headers, proxy=self.proxy_url) as response:
if response.status == 200:
content = await response.read()
if return_headers:
return True, content, dict(response.headers)
else:
return True, content, None
elif response.status == 401:
error_msg = "Unauthorized access - invalid or missing API key"
return False, error_msg, None
elif response.status == 403:
error_msg = "Access forbidden"
return False, error_msg, None
elif response.status == 404:
error_msg = "File not found"
return False, error_msg, None
else:
error_msg = f"Download failed with status {response.status}"
return False, error_msg, None
except Exception as e:
logger.error(f"Error downloading to memory from {url}: {e}")
return False, str(e), None
async def get_response_headers(
self,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None
) -> Tuple[bool, Union[Dict, str]]:
"""
Get response headers without downloading the full content
Args:
url: URL to check
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
Returns:
Tuple[bool, Union[Dict, str]]: (success, headers dict or error message)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[get_response_headers] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[get_response_headers] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
async with session.head(url, headers=headers, proxy=self.proxy_url) as response:
if response.status == 200:
return True, dict(response.headers)
else:
return False, f"Head request failed with status {response.status}"
except Exception as e:
logger.error(f"Error getting headers from {url}: {e}")
return False, str(e)
async def make_request(
self,
method: str,
url: str,
use_auth: bool = False,
custom_headers: Optional[Dict[str, str]] = None,
**kwargs
) -> Tuple[bool, Union[Dict, str]]:
"""
Make a generic HTTP request and return JSON response
Args:
method: HTTP method (GET, POST, etc.)
url: Request URL
use_auth: Whether to include authentication headers
custom_headers: Additional headers to include in request
**kwargs: Additional arguments for aiohttp request
Returns:
Tuple[bool, Union[Dict, str]]: (success, response data or error message)
"""
try:
session = await self.session
# Debug log for proxy mode at request time
if self.proxy_url:
logger.debug(f"[make_request] Using app-level proxy: {self.proxy_url}")
else:
logger.debug("[make_request] Using system-level proxy (trust_env) if configured.")
# Prepare headers
headers = self._get_auth_headers(use_auth)
if custom_headers:
headers.update(custom_headers)
# Add proxy to kwargs if not already present
if 'proxy' not in kwargs:
kwargs['proxy'] = self.proxy_url
async with session.request(method, url, headers=headers, **kwargs) as response:
if response.status == 200:
# Try to parse as JSON, fall back to text
try:
data = await response.json()
return True, data
except:
text = await response.text()
return True, text
elif response.status == 401:
return False, "Unauthorized access - invalid or missing API key"
elif response.status == 403:
return False, "Access forbidden"
elif response.status == 404:
return False, "Resource not found"
elif response.status == 429:
retry_after = self._extract_retry_after(response.headers)
error_msg = "Request rate limited"
logger.warning(
"Rate limit encountered for %s %s; retry_after=%s",
method,
url,
retry_after,
)
return False, RateLimitError(
error_msg,
retry_after=retry_after,
)
else:
return False, f"Request failed with status {response.status}"
except Exception as e:
logger.error(f"Error making {method} request to {url}: {e}")
return False, str(e)
async def close(self):
"""Close the HTTP session"""
if self._session is not None:
await self._session.close()
self._session = None
self._session_created_at = None
self._proxy_url = None
logger.debug("Closed HTTP session")
async def refresh_session(self):
"""Force refresh the HTTP session (useful when proxy settings change)"""
async with self._session_lock:
await self._create_session()
logger.info("HTTP session refreshed due to settings change")
@staticmethod
def _extract_retry_after(headers) -> Optional[float]:
"""Parse the Retry-After header into seconds."""
if not headers:
return None
header_value = headers.get("Retry-After")
if not header_value:
return None
header_value = header_value.strip()
if not header_value:
return None
if header_value.isdigit():
try:
seconds = float(header_value)
except ValueError:
return None
return max(0.0, seconds)
try:
retry_datetime = parsedate_to_datetime(header_value)
except (TypeError, ValueError):
return None
if retry_datetime.tzinfo is None:
return None
delta = retry_datetime - datetime.now(tz=retry_datetime.tzinfo)
return max(0.0, delta.total_seconds())
# Global instance accessor
async def get_downloader() -> Downloader:
"""Get the global downloader instance"""
return await Downloader.get_instance()

Some files were not shown because too many files have changed in this diff Show More