fix(widget): guard setValue against non-array input to prevent workflow load crash (#1039)

Two bugs prevented type-signature-based fallback from working:

- metadata_hook.py used getattr(obj.__class__, 'RETURN_TYPES')
  which fails when _async_map_node_over_list is called with
  a class (not instance) — obj.__class__ is the metaclass
  'type', which has no RETURN_TYPES. Fixed: getattr(obj, ...).

- metadata_registry.py used type(extractor) is GenericNodeExtractor
  to dispatch return_types. NODE_EXTRACTORS stores class
  references, not instances; type(Class) is always 'type',
  never the class. Fixed: extractor is GenericNodeExtractor.

.docs/batch-import-design.md

@@ -1,153 +0,0 @@
-# Recipe Batch Import Feature Design
-
-## Overview
-Enable users to import multiple images as recipes in a single operation, rather than processing them individually. This feature addresses the need for efficient bulk recipe creation from existing image collections.
-
-## Architecture
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│                        Frontend                                  │
-├─────────────────────────────────────────────────────────────────┤
-│  BatchImportManager.js                                          │
-│  ├── InputCollector (收集URL列表/目录路径)                        │
-│  ├── ConcurrencyController (自适应并发控制)                       │
-│  ├── ProgressTracker (进度追踪)                                   │
-│  └── ResultAggregator (结果汇总)                                  │
-├─────────────────────────────────────────────────────────────────┤
-│  batch_import_modal.html                                        │
-│  └── 批量导入UI组件                                              │
-├─────────────────────────────────────────────────────────────────┤
-│  batch_import_progress.css                                      │
-│  └── 进度显示样式                                                │
-└─────────────────────────────────────────────────────────────────┘
-                              │
-                              ▼
-┌─────────────────────────────────────────────────────────────────┐
-│                        Backend                                   │
-├─────────────────────────────────────────────────────────────────┤
-│  py/routes/handlers/recipe_handlers.py                          │
-│  ├── start_batch_import() - 启动批量导入                         │
-│  ├── get_batch_import_progress() - 查询进度                      │
-│  └── cancel_batch_import() - 取消导入                            │
-├─────────────────────────────────────────────────────────────────┤
-│  py/services/batch_import_service.py                            │
-│  ├── 自适应并发执行                                              │
-│  ├── 结果汇总                                                    │
-│  └── WebSocket进度广播                                           │
-└─────────────────────────────────────────────────────────────────┘
-```
-
-## API Endpoints
-
-| 端点 | 方法 | 说明 |
-|------|------|------|
-| `/api/lm/recipes/batch-import/start` | POST | 启动批量导入，返回 operation_id |
-| `/api/lm/recipes/batch-import/progress` | GET | 查询进度状态 |
-| `/api/lm/recipes/batch-import/cancel` | POST | 取消导入 |
-
-## Backend Implementation Details
-
-### BatchImportService
-
-Location: `py/services/batch_import_service.py`
-
-Key classes:
-- `BatchImportItem`: Dataclass for individual import item
-- `BatchImportProgress`: Dataclass for tracking progress
-- `BatchImportService`: Main service class
-
-Features:
-- Adaptive concurrency control (adjusts based on success/failure rate)
-- WebSocket progress broadcasting
-- Graceful error handling (individual failures don't stop the batch)
-- Result aggregation
-
-### WebSocket Message Format
-
-```json
-{
-    "type": "batch_import_progress",
-    "operation_id": "xxx",
-    "total": 50,
-    "completed": 23,
-    "success": 21,
-    "failed": 2,
-    "skipped": 0,
-    "current_item": "image_024.png",
-    "status": "running"
-}
-```
-
-### Input Types
-
-1. **URL List**: Array of URLs (http/https)
-2. **Local Paths**: Array of local file paths
-3. **Directory**: Path to directory with optional recursive flag
-
-### Error Handling
-
-- Invalid URLs/paths: Skip and record error
-- Download failures: Record error, continue
-- Metadata extraction failures: Mark as "no metadata"
-- Duplicate detection: Option to skip duplicates
-
-## Frontend Implementation Details (TODO)
-
-### UI Components
-
-1. **BatchImportModal**: Main modal with tabs for URLs/Directory input
-2. **ProgressDisplay**: Real-time progress bar and status
-3. **ResultsSummary**: Final results with success/failure breakdown
-
-### Adaptive Concurrency Controller
-
-```javascript
-class AdaptiveConcurrencyController {
-    constructor(options = {}) {
-        this.minConcurrency = options.minConcurrency || 1;
-        this.maxConcurrency = options.maxConcurrency || 5;
-        this.currentConcurrency = options.initialConcurrency || 3;
-    }
-
-    adjustConcurrency(taskDuration, success) {
-        if (success && taskDuration < 1000 && this.currentConcurrency < this.maxConcurrency) {
-            this.currentConcurrency = Math.min(this.currentConcurrency + 1, this.maxConcurrency);
-        }
-        if (!success || taskDuration > 10000) {
-            this.currentConcurrency = Math.max(this.currentConcurrency - 1, this.minConcurrency);
-        }
-        return this.currentConcurrency;
-    }
-}
-```
-
-## File Structure
-
-```
-Backend (implemented):
-├── py/services/batch_import_service.py            # 后端服务
-├── py/routes/handlers/batch_import_handler.py     # API处理器 (added to recipe_handlers.py)
-├── tests/services/test_batch_import_service.py    # 单元测试
-└── tests/routes/test_batch_import_routes.py       # API集成测试
-
-Frontend (TODO):
-├── static/js/managers/BatchImportManager.js       # 主管理器
-├── static/js/managers/batch/                       # 子模块
-│   ├── ConcurrencyController.js                    # 并发控制
-│   ├── ProgressTracker.js                          # 进度追踪
-│   └── ResultAggregator.js                         # 结果汇总
-├── static/css/components/batch-import-modal.css   # 样式
-└── templates/components/batch_import_modal.html    # Modal模板
-```
-
-## Implementation Status
-
-- [x] Backend BatchImportService
-- [x] Backend API handlers
-- [x] WebSocket progress broadcasting
-- [x] Unit tests
-- [x] Integration tests
-- [ ] Frontend BatchImportManager
-- [ ] Frontend UI components
-- [ ] E2E tests

.github/ISSUE_TEMPLATE/feature_request.md

@@ -13,8 +13,5 @@ A clear and concise description of what the problem is. Ex. I'm always frustrate
 **Describe the solution you'd like**
 A clear and concise description of what you want to happen.
 
-**Describe alternatives you've considered**
-A clear and concise description of any alternative solutions or features you've considered.
-
 **Additional context**
 Add any other context or screenshots about the feature request here.

.gitignore

@@ -7,17 +7,24 @@ py/run_test.py
 .vscode/
 cache/
 civitai/
+stats/
+wildcards/
+backups/
+logs/
 node_modules/
 coverage/
 .coverage
 model_cache/
 
-# agent
+# agent / dev tooling
 .opencode/
 .claude/
 .sisyphus/
 .codex
 .omo
+reasonix.toml
+.reasonix/
+.codegraph/
 
 # Vue widgets development cache (but keep build output)
 vue-widgets/node_modules/
@@ -26,3 +33,10 @@ vue-widgets/dist/
 
 # Hypothesis test cache
 .hypothesis/
+
+# Working/research notes (not committed)
+.docs/
+
+# HF enrichment validation baseline snapshots (contain potentially
+# NSFW README content fetched from community model repos)
+tests/enrich_hf_validation/baselines/

.omo/plans/embeddings-hybrid-approach.md

@@ -0,0 +1,181 @@
+# Embeddings Usage Tracking — Hybrid Approach (Plan C)
+
+> **Status**: Reference document for future implementation
+> **Current implementation**: Plan A (prompt text parsing only, see `usage_stats.py:_process_embeddings`)
+> **Next step**: Add Plan B as a supplement when edge-case coverage is needed
+
+## Problem
+
+Embeddings in ComfyUI are not loaded through dedicated ComfyUI nodes like LoRAs or
+Checkpoints. They are resolved during CLIP tokenization when the prompt text contains
+`embedding:<name>` syntax (see `comfy/sd1_clip.py:SDTokenizer.tokenize_with_weights`).
+
+This means the existing metadata_collector hook (which intercepts node execution via
+`_map_node_over_list`) cannot capture embeddings the same way it captures LoRAs and
+checkpoints — there is no "EmbeddingLoader" node to intercept.
+
+## Solution Architecture
+
+The hybrid approach combines **two complementary mechanisms** to capture embedding
+usage from all possible paths.
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                 Plan A (已实现)                          │
+│                                                         │
+│  MetadataRegistry.prompt_metadata["prompts"]            │
+│       │                                                  │
+│       ▼                                                  │
+│  _process_embeddings()                                   │
+│       │                                                  │
+│       ├─ Iterate all prompt node texts                   │
+│       ├─ regex extract "embedding:<name>"                │
+│       ├─ resolve name → sha256 via EmbeddingScanner      │
+│       └─ UsageStats.stats["embeddings"][sha256]++        │
+│                                                         │
+│  Coverage: ~95% — all CLIPTextEncode/Flux/etc nodes     │
+│                                                         │
+│  Gap: Custom nodes that load embeddings programmatically │
+│       without putting embedding:name in prompt text      │
+└─────────────────────────────────────────────────────────┘
+
+                         +
+                         ↓  (future: enable Plan B when needed)
+
+┌─────────────────────────────────────────────────────────┐
+│                 Plan B (未来 — monkey-patch)              │
+│                                                         │
+│  comfy/sd1_clip.py:load_embed()                         │
+│       │                                                  │
+│       ▼                                                  │
+│  Monkey-patch intercepts EVERY embedding file load       │
+│       │                                                  │
+│       ├─ Records embedding_name + success/failure        │
+│       ├─ Associates with current prompt_id (via registry)│
+│       └─ Feeds into UsageStats same as Plan A            │
+│                                                         │
+│  Coverage: 100% — catches ALL embedding loads            │
+│                                                         │
+│  Cost: Requires patching into ComfyUI internals          │
+│        (sd1_clip.py, sdxl_clip.py, some text_encoders)   │
+└─────────────────────────────────────────────────────────┘
+```
+
+## Plan B Detail — Monkey-patch `load_embed`
+
+### Target Function
+
+**`comfy.sd1_clip.load_embed(embedding_name, embedding_directory, embedding_size, embed_key=None)`**
+at line 415 of `sd1_clip.py`.
+
+This is the **single choke point** for all embedding file loads in ComfyUI. Every
+CLIP variant (SD1, SDXL, SD3, Flux) calls this same function.
+
+### Implementation Sketch
+
+```python
+# In metadata_collector/metadata_hook.py (or a new module)
+import comfy.sd1_clip as sd1_clip
+
+_original_load_embed = sd1_clip.load_embed
+
+def _patched_load_embed(embedding_name, embedding_directory, embedding_size, embed_key=None):
+    result = _original_load_embed(
+        embedding_name, embedding_directory, embedding_size, embed_key
+    )
+    if result is not None:
+        _record_embedding_usage(embedding_name)
+    return result
+
+sd1_clip.load_embed = _patched_load_embed
+```
+
+### Prompt ID Association
+
+The challenge is associating the `load_embed` call with the current `prompt_id`.
+Options:
+
+1. **Thread-local / contextvar**: Store current `prompt_id` in a `contextvars.ContextVar`
+   that the metadata_collector sets at the start of each prompt execution.
+
+2. **MetadataRegistry singleton**: The MetadataRegistry already has `current_prompt_id`.
+   The patch can read it directly since both run in the same thread.
+
+3. **Lazy aggregation**: Instead of associating with prompt_id at load time, collect
+   all loaded embedding names in a global set during execution, then flush to
+   UsageStats after the prompt completes.
+
+### Files to Patch
+
+| File | Function | Coverage |
+|------|----------|----------|
+| `comfy/sd1_clip.py:415` | `load_embed()` | Primary — SD1.x, SDXL, SD3, Flux |
+| `comfy/sdxl_clip.py` | Not needed (calls `sd1_clip.SDTokenizer`) | — |
+| `comfy/text_encoders/sd3_clip.py` | Not needed (calls `sd1_clip.SDTokenizer`) | — |
+| `comfy/text_encoders/flux.py` | Not needed (calls `sd1_clip.SDTokenizer`) | — |
+
+The SD1 tokenizer is the base class for all CLIP variants' tokenizers, so patching
+`load_embed` covers them all.
+
+### Edge Cases
+
+| Edge Case | Plan A | Plan B |
+|-----------|--------|--------|
+| `embedding:name` in CLIPTextEncode | ✅ | ✅ |
+| `embedding:name` in CLIPTextEncodeFlux | ✅ | ✅ |
+| `embedding:name` in PromptLM (LoRA Manager) | ✅ | ✅ |
+| `embedding:name` in WAS_Text_to_Conditioning | ✅ | ✅ |
+| Custom node that loads embedding programmatically | ❌ | ✅ |
+| Embedding loaded multiple times in same prompt | ✅ (dedup via set) | ✅ (dedup via set) |
+| Embedding file not found | N/A | ✅ (can log) |
+| Embedding dimension mismatch | N/A | ✅ (can log) |
+| Text encoder with non-standard tokenizer (LLaMA, T5...) | Partial | ✅ (if it calls load_embed) |
+
+## Migration Path: Standalone → Hybrid
+
+### Phase 1 — Plan A (当前状态)
+- Prompt text parsing only
+- No monkey-patching required
+- Covers all standard workflows
+
+### Phase 2 — Enable Plan B (未来工作)
+1. Add monkey-patch of `load_embed` in `metadata_collector/metadata_hook.py` (alongside
+   the existing `_map_node_over_list` hook)
+2. Collect loaded embedding names in a `set()` on the registry
+3. In `UsageStats._process_embeddings()`, merge the Plan A results (from prompt text)
+   with the Plan B results (from the patch)
+4. Add `prompt_data` field on MetadataRegistry to store loaded embeddings per prompt
+
+### Deduplication
+
+```python
+# Merge Plan A + Plan B results in _process_embeddings
+plan_a_names = extract_from_prompt_texts(prompts_data)
+plan_b_names = registry.get_loaded_embeddings(prompt_id)
+
+all_names = plan_a_names | plan_b_names
+```
+
+## Testing the Hybrid
+
+| Scenario | What to verify |
+|----------|---------------|
+| Standard `embedding:name` in prompt | Plan A captures it |
+| Embedding loaded by custom node script | Plan B captures it |
+| Both paths fire for same embedding | No double-counting (dedup) |
+| Embedding name resolves to hash | EmbeddingScanner.get_hash_by_filename works |
+| No embedding scanner available | Graceful skip, no crash |
+| Missing embedding file | Plan B logs warning, Plan A skips gracefully |
+| Empty prompt | No crash, no entries |
+| Standalone mode | Both plans disabled gracefully |
+
+## Key Files Reference
+
+| File | Role |
+|------|------|
+| `py/utils/usage_stats.py` | Core — `_process_embeddings()` for Plan A |
+| `py/metadata_collector/constants.py` | `EMBEDDINGS` category constant |
+| `py/metadata_collector/metadata_hook.py` | Future — monkey-patch for Plan B |
+| `py/services/embedding_scanner.py` | Hash resolution service |
+| `py/routes/stats_routes.py` | Already handles `usage_data.get('embeddings', {})` |
+| `comfy/sd1_clip.py` (ComfyUI) | `load_embed()` — Plan B target |

AGENTS.md

@@ -102,6 +102,7 @@ npm run test:coverage       # Generate coverage report
 - ComfyUI: `app.registerExtension()`, `node.addDOMWidget(name, type, element, options)`
 - Event handlers via `addEventListener` or widget callbacks
 - Shared utilities: `web/comfyui/utils.js`
+- Dual-mode rendering patterns (canvas vs Vue): see `docs/comfyui-dual-mode-widgets.md`
 
 ### Vue Composables Pattern
 
@@ -136,7 +137,13 @@ npm run test:coverage       # Generate coverage report
 - Dual mode: ComfyUI plugin (folder_paths) vs standalone (settings.json)
 - Detection: `os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1"`
 - Run `python scripts/sync_translation_keys.py` after adding UI strings to `locales/en.json`
-- Symlinks require normalized paths
+- Symlinks require normalized paths.
+  **Business paths vs real paths**: All stored paths and operation routing use the
+  original paths as they appear under configured model roots — symlinks are NOT
+  resolved. `os.path.realpath` is only for scanner dedup and the symlink cache.
+  Any path passed to `os.remove`/`os.rename`/`shutil.move` or validated by a
+  containment check MUST use the business path (i.e. `os.path.abspath`, not
+  `realpath`).
 
 ## Git / Commit Messages
 

__init__.py

@@ -15,6 +15,10 @@ try:  # pragma: no cover - import fallback for pytest collection
     from .py.nodes.lora_pool import LoraPoolLM
     from .py.nodes.lora_randomizer import LoraRandomizerLM
     from .py.nodes.lora_cycler import LoraCyclerLM
+    from .py.nodes.lora_info import LoraInfoLM
+    from .py.nodes.lora_syntax_to_path import LoraSyntaxToPath
+    from .py.nodes.create_hook_lora import CreateHookLoraLM
+    from .py.nodes.metadata_overwrite import MetadataOverwriteLM
     from .py.metadata_collector import init as init_metadata_collector
 except (
     ImportError
@@ -56,6 +60,16 @@ except (
         "py.nodes.lora_randomizer"
     ).LoraRandomizerLM
     LoraCyclerLM = importlib.import_module("py.nodes.lora_cycler").LoraCyclerLM
+    LoraInfoLM = importlib.import_module("py.nodes.lora_info").LoraInfoLM
+    LoraSyntaxToPath = importlib.import_module(
+        "py.nodes.lora_syntax_to_path"
+    ).LoraSyntaxToPath
+    CreateHookLoraLM = importlib.import_module(
+        "py.nodes.create_hook_lora"
+    ).CreateHookLoraLM
+    MetadataOverwriteLM = importlib.import_module(
+        "py.nodes.metadata_overwrite"
+    ).MetadataOverwriteLM
     init_metadata_collector = importlib.import_module("py.metadata_collector").init
 
 NODE_CLASS_MAPPINGS = {
@@ -75,6 +89,10 @@ NODE_CLASS_MAPPINGS = {
     LoraPoolLM.NAME: LoraPoolLM,
     LoraRandomizerLM.NAME: LoraRandomizerLM,
     LoraCyclerLM.NAME: LoraCyclerLM,
+    LoraInfoLM.NAME: LoraInfoLM,
+    LoraSyntaxToPath.NAME: LoraSyntaxToPath,
+    CreateHookLoraLM.NAME: CreateHookLoraLM,
+    MetadataOverwriteLM.NAME: MetadataOverwriteLM,
 }
 
 WEB_DIRECTORY = "./web/comfyui"

docs/agent_skills.md

@@ -0,0 +1,208 @@
+# Agent Skills System
+
+The LoRA Manager agent skills system enables LLM-powered metadata enrichment and other AI-driven tasks. Users configure their own LLM provider (BYOK), and skills are executed through right-click context menu actions.
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────┐
+│              LoRA Manager Backend             │
+│                                               │
+│  ┌──────────────┐    ┌────────────────┐       │
+│  │ LLMService    │───▶│ LLM Provider   │       │
+│  │ (BYOK config, │◀───│ (OpenAI/Ollama │       │
+│  │  API calls)   │    │ /custom)       │       │
+│  └───────┬───────┘    └────────────────┘       │
+│          │                                     │
+│  ┌───────▼───────────────────────┐             │
+│  │     AgentService              │             │
+│  │  (orchestration: validate     │             │
+│  │   → LLM call → post-process   │             │
+│  │   → WebSocket broadcast)      │             │
+│  └───────┬───────────────────────┘             │
+│          │                                     │
+│  ┌───────▼───────────────────────┐             │
+│  │     SkillRegistry             │             │
+│  │  ┌─────────────────────────┐  │             │
+│  │  │ enrich_hf_metadata:     │  │             │
+│  │  │  - skill.yaml           │  │             │
+│  │  │  - prompt.md            │  │             │
+│  │  │  - handler.py           │  │             │
+│  │  └─────────────────────────┘  │             │
+│  └───────────────────────────────┘             │
+└──────────────────────────────────────────────┘
+```
+
+### Key Design Principle
+
+**Skills define *what* to do (prompt + post-processing). The AgentService handles *how* (LLM calls, validation, progress).**
+
+Skills never call the LLM directly. This keeps BYOK configuration centralized and provider-agnostic.
+
+## BYOK Configuration
+
+Users configure their LLM provider in **Settings → AI Provider**:
+
+| Setting | Description | Example |
+|---|---|---|
+| `llm_provider` | Provider type | `openai`, `ollama`, or `custom` |
+| `llm_api_key` | API key (not needed for local Ollama) | `sk-...` |
+| `llm_api_base` | Custom API base URL (empty = provider default) | `https://api.openai.com/v1` |
+| `llm_model` | Model name | `gpt-4o-mini` |
+
+Environment variable overrides: `LLM_API_KEY`, `LLM_MODEL`, `LLM_API_BASE`, `LLM_PROVIDER`.
+
+### Supported Providers
+
+- **OpenAI**: Uses `https://api.openai.com/v1` by default
+- **Ollama** (local): Uses `http://localhost:11434/v1`, no API key required
+- **Custom**: Any OpenAI-compatible endpoint (vLLM, LM Studio, etc.) — set `llm_api_base` explicitly
+
+## Available Skills
+
+### enrich_hf_metadata
+
+Enriches HuggingFace-downloaded models with metadata extracted by an LLM from the HF model card.
+
+**Entry point**: Right-click context menu → "Enrich Metadata (Agent)"
+
+**What it does**:
+1. Reads the model's `.metadata.json` to get the `hf_url`
+2. Fetches the README.md from the HuggingFace repository
+3. Sends the README + local metadata to the LLM for structured extraction
+4. Writes extracted fields to `.metadata.json`:
+   - `base_model` — only if current value is empty
+   - `trainedWords` — trigger words (LoRA only, if none exist)
+   - `modelDescription` — concise summary (if none exists)
+   - `tags` — merged with existing tags, deduplicated
+   - `metadata_source` — audit trail: `agent:enrich_hf_metadata`
+   - `llm_enriched_at` — ISO timestamp
+5. Downloads and optimizes preview image (if LLM found one in the README)
+6. Updates the scanner cache
+7. Broadcasts WebSocket progress events
+
+**Model types**: LoRA, Checkpoint, Embedding
+
+## Adding a New Skill
+
+### 1. Create the skill directory
+
+```
+py/services/agent/skills/<skill_name>/
+├── skill.yaml      # Skill metadata and schemas
+├── prompt.md       # LLM prompt template
+└── handler.py      # Pre-processing and post-processing
+```
+
+### 2. Write skill.yaml
+
+```yaml
+name: my_skill
+title: "My Skill"
+description: "What this skill does"
+llm_required: true
+model_type_filter: ["lora"]  # or null for all types
+input_schema:
+  type: object
+  properties:
+    model_paths:
+      type: array
+      items:
+        type: string
+  required:
+    - model_paths
+output_schema:
+  type: object
+  properties:
+    # ... JSON schema for LLM output
+permissions:
+  write_metadata: true
+  write_previews: false
+  network_domains:
+    - "example.com"
+```
+
+### 3. Write prompt.md
+
+Use `{{variable}}` placeholders that will be replaced with data from the `prepare` function:
+
+```markdown
+You are an expert assistant...
+
+Model URL: {{hf_url}}
+README content:
+{{readme_content}}
+
+Current metadata:
+{{current_metadata}}
+```
+
+### 4. Write handler.py
+
+```python
+async def prepare(model_path: str, input_data: dict) -> dict:
+    """Gather context for the LLM prompt. Returns variables for template rendering."""
+    return {
+        "model_path": model_path,
+        # ... other variables used in prompt.md
+    }
+
+async def post_process(context) -> dict:
+    """Apply the LLM-extracted data to the model."""
+    llm_response = context.llm_response
+    # ... write metadata, download previews, update cache
+    return {
+        "success": True,
+        "updated_fields": ["base_model", "tags"],
+        "errors": [],
+    }
+```
+
+**Important**: Use absolute imports (`from py.utils.metadata_manager import MetadataManager`) because skills are loaded via `importlib.util.spec_from_file_location`, which doesn't support relative imports.
+
+### 5. Test
+
+The skill is automatically discovered by `SkillRegistry` on startup. Test with:
+
+```python
+pytest tests/services/test_agent_service.py
+```
+
+## API Endpoints
+
+| Method | Path | Description |
+|---|---|---|
+| GET | `/api/lm/agent/skills` | List available skills |
+| POST | `/api/lm/agent/execute/{skill_name}` | Execute a skill (body: `{"model_paths": [...]}`) |
+| POST | `/api/lm/agent/cancel` | Cancel running skill (stub) |
+
+## WebSocket Events
+
+| Type | When | Key fields |
+|---|---|---|
+| `agent_progress` | Skill started/processing | `skill`, `status`, `total`, `processed`, `success`, `current_path` |
+| `agent_progress` | Skill completed | `skill`, `status`, `updated_models`, `errors`, `summary` |
+| `agent_progress` | Skill error | `skill`, `status`, `error` |
+
+## Security Model
+
+Skills declare permissions in `skill.yaml`:
+- `write_metadata` — can write `.metadata.json` files
+- `write_previews` — can download/replace preview images
+- `network_domains` — allowed domains for HTTP requests
+
+These are declarative constraints checked by `AgentService`. They are defense-in-depth, not a sandbox — the Python process can technically do anything, but the contract is clear and auditable.
+
+## File Locations
+
+| Component | Path |
+|---|---|
+| LLMService | `py/services/llm_service.py` |
+| AgentService | `py/services/agent/agent_service.py` |
+| SkillRegistry | `py/services/agent/skill_registry.py` |
+| SkillDefinition | `py/services/agent/skill_definition.py` |
+| Skills directory | `py/services/agent/skills/` |
+| Route handlers | `py/routes/handlers/agent_handlers.py` |
+| Frontend manager | `static/js/managers/AgentManager.js` |
+| Settings UI | `templates/components/modals/settings_modal.html` |
+| Context menu | `templates/components/context_menu.html` |

docs/comfyui-dual-mode-widgets.md

@@ -0,0 +1,65 @@
+# ComfyUI Dual-Mode Widget Rendering
+
+ComfyUI custom node widgets render in one of two modes. Patterns that work in one often fail silently in the other. Test both.
+
+## Mode Detection
+
+```js
+typeof LiteGraph !== 'undefined' && LiteGraph.vueNodesMode
+```
+
+In Vue SFCs, `window.LiteGraph` is unavailable — pass as a prop from `main.ts`.
+
+## Canvas Mode Layout
+
+Uses `computeLayoutSize()` + `distributeSpace()` to allocate widget height within the node. Widgets with `computeLayoutSize` participate in space distribution; those with `computeSize` have fixed height.
+
+- `getMinHeight()` in `addDOMWidget` options → minimum widget height
+- `widget.computeLayoutSize()` → `{ minHeight, minWidth, maxHeight? }`
+- Avoid `getMaxHeight()` unless the widget genuinely needs a fixed cap (prevents user resize)
+
+## Vue Mode Layout
+
+Uses CSS Grid (`grid-template-rows`) + `ResizeObserver`. The ResizeObserver watches the widget's DOM and feeds back into grid row sizing. This creates a feedback loop: content grows → row resizes → more space for content → content reflows/grows → row resizes again.
+
+### Height Containment
+
+The fix: `contain: layout size` on the widget root. This tells the browser the element's intrinsic size is CSS-determined, not driven by descendant content. The ResizeObserver sees a stable size and the loop is broken.
+
+```css
+.widget-root.lm-vue-node {
+  height: 100%;
+  min-height: var(--comfy-widget-min-height, 200px);
+  contain: layout size;
+}
+```
+
+Existing examples: `.lm-loras-container.lm-vue-node` and `.comfy-tags-container.lm-vue-node` in `web/comfyui/lm_styles.css`.
+
+**Do NOT** fix height issues with `maxHeight`, `getMaxHeight()`, or inline `max-height` — these prevent the user from resizing the node.
+
+## Scroll Wheel Isolation
+
+Both modes need to distinguish "user wants to scroll widget content" from "user wants to zoom canvas".
+
+**Canvas mode:** Add `@wheel` on widget root. Check `event.target.closest(selector)` for scrollable sub-areas. If scrollable → `event.stopPropagation()`. Otherwise → `app.canvas.processMouseWheel(event)`.
+
+**Vue mode:** Add CSS class `lm-wheel-scrollable` to scrollable elements. The global capture-phase hook in `web/comfyui/utils.js` (`enableListWheelScroll`) detects wheel events on marked elements and manually scrolls them via `element.scrollTop`, consuming the event before canvas zoom sees it.
+
+## DOM Structure
+
+`main.ts` creates an outer `<div>` container, then `vueApp.mount(container)`. The Vue app renders its own root element inside.
+
+- `container.id` / `container.style.*` → outer element
+- Vue scoped `<style>` → `[data-v-hash]` applies only to Vue root
+
+Classes needed by scoped Vue CSS must go on the Vue root element. Pass data as props and bind with `:class` rather than manipulating the DOM from `main.ts`.
+
+## Serialization
+
+For stateful widgets that need workflow persistence:
+
+- `serialize: true` in `addDOMWidget` options
+- `serializeValue()` → state snapshot (called on workflow save)
+- `onSetValue(v)` → restore state (called on workflow load)
+- Always handle missing keys in restored value for backward compatibility with old workflows

py/config.py

@@ -8,6 +8,8 @@ from typing import Any, Dict, Iterable, List, Mapping, Optional, Set, Tuple
 import logging
 import json
 import urllib.parse
+import sys as _sys
+import types as _types
 import time
 
 from .utils.cache_paths import CacheType, get_cache_file_path, get_legacy_cache_paths
@@ -175,8 +177,7 @@ class Config:
 
         # Load extra folder paths from active library settings before symlink scan
         # so both primary and extra paths are discovered in a single pass.
-        if not standalone_mode:
-            self._load_extra_paths_from_settings()
+        self._load_extra_paths_from_settings()
 
         # Scan symbolic links during initialization
         self._initialize_symlink_mappings()
@@ -191,7 +192,7 @@ class Config:
         Called during ``Config.__init__`` before the symlink scan so both primary and
         extra paths are discovered in a single pass.  Mirrors the extra-path
         portion of ``_apply_library_paths`` without replacing the primary roots
-        that were already resolved from ComfyUI's ``folder_paths``.
+        that were already resolved via ``folder_paths.get_folder_paths``.
         """
         try:
             from .services.settings_manager import get_settings_manager
@@ -207,6 +208,12 @@ class Config:
             if not isinstance(library_config, dict):
                 return
 
+            # Always read recipes_path — it is independent of extra folder paths
+            # and must be set before any early returns below.
+            recipes_path = library_config.get("recipes_path", "")
+            if isinstance(recipes_path, str) and recipes_path:
+                self.recipes_path = recipes_path
+
             extra_folder_paths = library_config.get("extra_folder_paths")
             if not isinstance(extra_folder_paths, dict):
                 return
@@ -232,10 +239,6 @@ class Config:
                 extra_embedding
             )
 
-            recipes_path = library_config.get("recipes_path", "")
-            if isinstance(recipes_path, str) and recipes_path:
-                self.recipes_path = recipes_path
-
             if self.extra_loras_roots:
                 logger.info(
                     "Found extra LoRA roots:"
@@ -356,6 +359,47 @@ class Config:
                         "Failed to rename legacy 'default' library: %s", rename_error
                     )
 
+            # Clean up a stale "default" library entry that has no meaningful
+            # paths configured (e.g. leftover bootstrap artifact).  This only
+            # fires when "comfyui" already exists so we never delete the last
+            # remaining library.
+            if (
+                "default" in libraries
+                and "comfyui" in libraries
+                and isinstance(default_library, Mapping)
+            ):
+                default_folder_paths = _normalize_library_folder_paths(
+                    default_library
+                )
+                default_extra_paths = default_library.get("extra_folder_paths", {})
+                has_meaningful_paths = bool(default_folder_paths) or bool(
+                    default_extra_paths
+                ) or any(
+                    default_library.get(key)
+                    for key in (
+                        "default_lora_root",
+                        "default_checkpoint_root",
+                        "default_unet_root",
+                        "default_embedding_root",
+                        "recipes_path",
+                    )
+                )
+                if not has_meaningful_paths:
+                    try:
+                        settings_service.delete_library("default")
+                        libraries_changed = True
+                        logger.info(
+                            "Removed stale 'default' library entry "
+                            "with no meaningful paths configured"
+                        )
+                        libraries = settings_service.get_libraries()
+                        comfy_library = libraries.get("comfyui", {})
+                    except Exception as delete_error:
+                        logger.debug(
+                            "Failed to remove stale 'default' library: %s",
+                            delete_error,
+                        )
+
             default_lora_root = _resolve_valid_default_root(
                 comfy_library.get("default_lora_root", ""),
                 list(self.loras_roots or []),
@@ -1380,4 +1424,20 @@ class Config:
 
 
 # Global config instance
-config = Config()
+# NOTE: Guard against re-import.  When ServiceRegistry.get_lora_scanner() triggers
+# a fresh import of lora_scanner → config, we must NOT re-execute Config.__init__()
+# (which re-scans all roots, re-registers libraries, etc.).
+#
+# Strategy: store the config instance in a dedicated sentinel module
+# ('_lm_config_cache') that is NEVER removed from sys.modules (its key does
+# NOT start with 'py.'), so it survives re-imports of py.* modules.
+_CONFIG_SENTINEL = "_lm_config_cache"
+if _CONFIG_SENTINEL in _sys.modules:
+    # Re-import: reuse the existing singleton from the sentinel.
+    config: Config = _sys.modules[_CONFIG_SENTINEL].config  # type: ignore[valid-type]
+else:
+    config: Config = Config()
+    # Register the sentinel so re-imports of py.config find us.
+    _sentinel_mod = _types.ModuleType(_CONFIG_SENTINEL)
+    _sentinel_mod.config = config
+    _sys.modules[_CONFIG_SENTINEL] = _sentinel_mod

py/lora_manager.py

@@ -33,6 +33,7 @@ from .utils.example_images_migration import ExampleImagesMigration
 from .services.websocket_manager import ws_manager
 from .services.example_images_cleanup_service import ExampleImagesCleanupService
 from .middleware.csp_middleware import relax_csp_for_remote_media
+from .middleware.error_middleware import api_json_error
 
 logger = logging.getLogger(__name__)
 
@@ -76,6 +77,11 @@ class LoraManager:
         """Initialize and register all routes using the new refactored architecture"""
         app = PromptServer.instance.app
 
+        # Register JSON error middleware for /api/* routes as the outermost
+        # middleware so it catches errors from all other middlewares.
+        if api_json_error not in app.middlewares:
+            app.middlewares.insert(0, api_json_error)
+
         if relax_csp_for_remote_media not in app.middlewares:
             # Ensure CSP relaxer executes after ComfyUI's block_external_middleware so it can
             # see and extend the restrictive header instead of being overwritten by it.
@@ -189,6 +195,10 @@ class LoraManager:
 
             # Register DownloadManager with ServiceRegistry
             await ServiceRegistry.get_download_manager()
+
+            # Initialize DownloadQueueService for persistent queue/history
+            await ServiceRegistry.get_download_queue_service()
+
             await ServiceRegistry.get_backup_service()
 
             from .services.metadata_service import initialize_metadata_providers
@@ -198,6 +208,10 @@ class LoraManager:
             # Initialize WebSocket manager
             await ServiceRegistry.get_websocket_manager()
 
+            # Preload LLM model catalog (background task, non-blocking)
+            from .services.llm_service import LLMService
+            await LLMService.get_instance()
+
             # Initialize scanners in background
             lora_scanner = await ServiceRegistry.get_lora_scanner()
             checkpoint_scanner = await ServiceRegistry.get_checkpoint_scanner()
@@ -426,5 +440,21 @@ class LoraManager:
         try:
             logger.info("LoRA Manager: Cleaning up services")
 
+            # Cancel any in-flight scanner initialization tasks so thread-pool
+            # workers (e.g. _initialize_cache_sync) can break out of their loops
+            # when the server shuts down (e.g. Ctrl+C on WSL).
+            for name in ("lora_scanner", "checkpoint_scanner", "embedding_scanner"):
+                scanner = ServiceRegistry.get_service_sync(name)
+                if scanner is not None and hasattr(scanner, "cancel_task"):
+                    scanner.cancel_task()
+                    logger.debug("LoRA Manager: Cancelled %s", name)
+
+            # Close shared aiohttp sessions to avoid "Unclosed client session" warnings
+            try:
+                from py.routes.handlers.hf_handlers import close_hf_api_session
+                await close_hf_api_session()
+            except Exception as exc:
+                logger.debug("Error closing HF API session: %s", exc)
+
         except Exception as e:
             logger.error(f"Error during cleanup: {e}", exc_info=True)

py/metadata_collector/constants.py

@@ -5,9 +5,18 @@ MODELS = "models"
 PROMPTS = "prompts"
 SAMPLING = "sampling"
 LORAS = "loras"
+EMBEDDINGS = "embeddings"
 SIZE = "size"
 IMAGES = "images"
 IS_SAMPLER = "is_sampler"  # New constant to mark sampler nodes
+OVERWRITE = "overwrite"  # Manual metadata overwrite from MetadataOverwriteLM node
+
+# Field names that the MetadataOverwriteLM node and its extractor share
+METADATA_OVERWRITE_FIELDS = (
+    "prompt", "negative_prompt", "seed", "steps", "cfg_scale",
+    "sampler", "scheduler", "model", "loras", "size",
+    "clip_skip", "additional_data",
+)
 
 # Complete list of categories to track
-METADATA_CATEGORIES = [MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES]
+METADATA_CATEGORIES = [MODELS, PROMPTS, SAMPLING, LORAS, EMBEDDINGS, SIZE, IMAGES, OVERWRITE]

py/metadata_collector/metadata_hook.py

@@ -83,7 +83,8 @@ class MetadataHook:
                         
                         # Record inputs before execution
                         if node_id is not None:
-                            registry.record_node_execution(node_id, class_type, input_data_all, None)
+                            return_types = getattr(obj, 'RETURN_TYPES', None)
+                            registry.record_node_execution(node_id, class_type, input_data_all, None, return_types=return_types)
                 except Exception as e:
                     logger.error(f"Error collecting metadata (pre-execution): {str(e)}")
             
@@ -114,7 +115,8 @@ class MetadataHook:
                         
                         # Record outputs after execution
                         if node_id is not None:
-                            registry.update_node_execution(node_id, class_type, results)
+                            return_types = getattr(obj, 'RETURN_TYPES', None)
+                            registry.update_node_execution(node_id, class_type, results, return_types=return_types)
                 except Exception as e:
                     logger.error(f"Error collecting metadata (post-execution): {str(e)}")
             
@@ -135,10 +137,13 @@ class MetadataHook:
                 # Store the dynprompt reference for node lookups
                 if hasattr(prompt, 'original_prompt'):
                     registry.set_current_prompt(prompt)
-            
+
+                # Store extra_data for accessing full workflow node properties
+                registry.set_extra_data(extra_data)
+
             # Execute the original function
             return original_execute(*args, **kwargs)
-            
+
         # Replace the functions
         execution._map_node_over_list = map_node_over_list_with_metadata
         execution.execute = execute_with_prompt_tracking
@@ -163,7 +168,8 @@ class MetadataHook:
                         class_type = obj.__class__.__name__
                         node_id = unique_id
                         if node_id is not None:
-                            registry.record_node_execution(node_id, class_type, input_data_all, None)
+                            return_types = getattr(obj, 'RETURN_TYPES', None)
+                            registry.record_node_execution(node_id, class_type, input_data_all, None, return_types=return_types)
                 except Exception as e:
                     logger.error(f"Error collecting metadata (pre-execution): {str(e)}")
 
@@ -180,7 +186,8 @@ class MetadataHook:
                         class_type = obj.__class__.__name__
                         node_id = unique_id
                         if node_id is not None:
-                            registry.update_node_execution(node_id, class_type, results)
+                            return_types = getattr(obj, 'RETURN_TYPES', None)
+                            registry.update_node_execution(node_id, class_type, results, return_types=return_types)
                 except Exception as e:
                     logger.error(f"Error collecting metadata (post-execution): {str(e)}")
 
@@ -202,6 +209,9 @@ class MetadataHook:
                 if hasattr(prompt, 'original_prompt'):
                     registry.set_current_prompt(prompt)
 
+                # Store extra_data for accessing full workflow node properties
+                registry.set_extra_data(extra_data)
+
             # Execute the original function
             return await original_execute(*args, **kwargs)
 

py/metadata_collector/metadata_processor.py

@@ -1,15 +1,68 @@
 import json
+import logging
 import os
 from .constants import IMAGES
 
 # Check if running in standalone mode
 standalone_mode = os.environ.get("LORA_MANAGER_STANDALONE", "0") == "1" or os.environ.get("HF_HUB_DISABLE_TELEMETRY", "0") == "0"
 
-from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER
+from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IS_SAMPLER, OVERWRITE
+from .node_extractors import NODE_EXTRACTORS
+
+logger = logging.getLogger(__name__)
+
+# Keys that identify metadata hint marks stored in node.properties.lm_marker_role
+_META_MARK_PREFIX = "meta_"
+_MARK_PRIMARY_MODEL = "primary_model"
+_MARK_PRIMARY_SAMPLER = "primary_sampler"
+_MARK_POSITIVE_PROMPT = "positive_prompt"
+_MARK_NEGATIVE_PROMPT = "negative_prompt"
 
 class MetadataProcessor:
     """Process and format collected metadata"""
-    
+
+    @staticmethod
+    def _get_user_marks(metadata):
+        """Scan workflow nodes (from extra_data.extra_pnginfo.workflow) for user-assigned
+        metadata hint marks stored in node.properties.lm_marker_role.
+
+        Returns a dict mapping mark type keys to node IDs.
+        Example: {'primary_model': '42', 'primary_sampler': '17'}
+        """
+        marks: dict[str, str] = {}
+
+        # Primary source: extra_data.extra_pnginfo.workflow.nodes (has full properties)
+        extra_data = metadata.get("extra_data")
+        if extra_data and isinstance(extra_data, dict):
+            extra_pnginfo = extra_data.get("extra_pnginfo", {})
+            if isinstance(extra_pnginfo, dict):
+                workflow = extra_pnginfo.get("workflow", {})
+                nodes = workflow.get("nodes", [])
+                for node in nodes:
+                    node_id = str(node.get("id", ""))
+                    role = node.get("properties", {}).get("lm_marker_role", "")
+                    if role.startswith(_META_MARK_PREFIX):
+                        mark_type = role[len(_META_MARK_PREFIX):]
+                        if mark_type in marks:
+                            logger.warning(
+                                "Duplicate meta hint '%s': node %s (previous: %s), "
+                                "last match wins",
+                                mark_type, node_id, marks[mark_type],
+                            )
+                        marks[mark_type] = node_id
+
+        # Fallback: try prompt.original_prompt (API-only submissions may not have workflow)
+        if not marks:
+            prompt = metadata.get("current_prompt")
+            if prompt and getattr(prompt, "original_prompt", None):
+                for node_id, node_data in prompt.original_prompt.items():
+                    role = node_data.get("properties", {}).get("lm_marker_role", "")
+                    if role.startswith(_META_MARK_PREFIX):
+                        mark_type = role[len(_META_MARK_PREFIX):]
+                        marks[mark_type] = node_id
+
+        return marks
+
     @staticmethod
     def find_primary_sampler(metadata, downstream_id=None):
         """
@@ -471,20 +524,57 @@ class MetadataProcessor:
             "checkpoint": None,
             "loras": "",
             "size": None,
-            "clip_skip": None
+            "clip_skip": None,
+            "additional_data": "",
         }
         
         # Get the prompt object for node relationship tracing
         prompt = metadata.get("current_prompt")
-        
-        # Find the primary KSampler node
-        primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id)
-        
-        # Directly get checkpoint from metadata instead of tracing
-        # Pass primary_sampler_id to avoid redundant calculation
-        checkpoint = MetadataProcessor.find_primary_checkpoint(metadata, id, primary_sampler_id)
-        if checkpoint:
-            params["checkpoint"] = checkpoint
+
+        # ---- User marks: override heuristic inference with user-assigned hints ----
+        user_marks = MetadataProcessor._get_user_marks(metadata)
+
+        # Find the primary KSampler node (user mark takes priority)
+        primary_sampler_id = None
+        primary_sampler = None
+        if _MARK_PRIMARY_SAMPLER in user_marks:
+            marked_id = user_marks[_MARK_PRIMARY_SAMPLER]
+            sampler_data = metadata.get(SAMPLING, {}).get(marked_id)
+            if sampler_data and sampler_data.get(IS_SAMPLER):
+                primary_sampler_id = marked_id
+                primary_sampler = sampler_data
+            else:
+                logger.warning(
+                    "User-marked primary sampler %s has no runtime metadata, "
+                    "falling back to heuristic",
+                    marked_id,
+                )
+        if primary_sampler is None:
+            primary_sampler_id, primary_sampler = MetadataProcessor.find_primary_sampler(metadata, id)
+
+        # Resolve checkpoint / model (user mark takes priority)
+        if _MARK_PRIMARY_MODEL in user_marks:
+            marked_id = user_marks[_MARK_PRIMARY_MODEL]
+            if marked_id in metadata.get(MODELS, {}):
+                params["checkpoint"] = metadata[MODELS][marked_id].get("name")
+            else:
+                extra_data = metadata.get("extra_data")
+                extra_pnginfo = extra_data.get("extra_pnginfo", {}) if extra_data and isinstance(extra_data, dict) else {}
+                workflow = extra_pnginfo.get("workflow", {}) if isinstance(extra_pnginfo, dict) else {}
+                node_type = "unknown"
+                for n in workflow.get("nodes", []):
+                    if str(n.get("id", "")) == marked_id:
+                        node_type = n.get("type", "unknown")
+                        break
+                logger.warning(
+                    "User-marked primary model %s (type=%s, registered=%s) has no runtime metadata, "
+                    "falling back to heuristic",
+                    marked_id, node_type, node_type in NODE_EXTRACTORS,
+                )
+        if params["checkpoint"] is None:
+            checkpoint = MetadataProcessor.find_primary_checkpoint(metadata, id, primary_sampler_id)
+            if checkpoint:
+                params["checkpoint"] = checkpoint
         
         # Check if guidance parameter exists in any sampling node
         for node_id, sampler_info in metadata.get(SAMPLING, {}).items():
@@ -539,7 +629,22 @@ class MetadataProcessor:
                     
                     # For SamplerCustom, handle any additional parameters
                     MetadataProcessor.handle_custom_advanced_sampler(metadata, prompt, primary_sampler_id, params)
-                
+
+            # ---- User marks: override prompts with explicitly tagged nodes ----
+            prompts_data = metadata.get(PROMPTS, {})
+            if _MARK_POSITIVE_PROMPT in user_marks:
+                pos_id = user_marks[_MARK_POSITIVE_PROMPT]
+                if pos_id in prompts_data:
+                    prompt_text = prompts_data[pos_id].get("text") or prompts_data[pos_id].get("positive_text")
+                    if prompt_text:
+                        params["prompt"] = prompt_text
+            if _MARK_NEGATIVE_PROMPT in user_marks:
+                neg_id = user_marks[_MARK_NEGATIVE_PROMPT]
+                if neg_id in prompts_data:
+                    prompt_text = prompts_data[neg_id].get("text") or prompts_data[neg_id].get("negative_text")
+                    if prompt_text:
+                        params["negative_prompt"] = prompt_text
+
             # Size extraction is same for all sampler types
             # Check if the sampler itself has size information (from latent_image)
             if primary_sampler_id in metadata.get(SIZE, {}):
@@ -568,7 +673,21 @@ class MetadataProcessor:
                 break
         if params["clip_skip"] is None:
             params["clip_skip"] = "1"
-        
+
+        # ---- Apply manual metadata overwrites ----
+        for overwrite_info in metadata.get(OVERWRITE, {}).values():
+            overwrite_params = overwrite_info.get("parameters", {})
+            for key, value in overwrite_params.items():
+                if value:  # truthy check — only overwrite when user provided a real value
+                    params[key] = value
+
+        # Bridge: the overwrite node exposes the field as "model" (more accurate),
+        # but the internal pipeline key remains "checkpoint" for backward compatibility
+        # with A1111 metadata format and downstream consumers.
+        if params.get("model"):
+            params["checkpoint"] = params["model"]
+            del params["model"]
+
         return params
     
     @staticmethod

py/metadata_collector/metadata_registry.py

@@ -1,7 +1,7 @@
 import time
 from nodes import NODE_CLASS_MAPPINGS  # type: ignore
 from .node_extractors import NODE_EXTRACTORS, GenericNodeExtractor
-from .constants import METADATA_CATEGORIES, IMAGES
+from .constants import METADATA_CATEGORIES, IMAGES, OVERWRITE
 
 
 class MetadataRegistry:
@@ -61,6 +61,7 @@ class MetadataRegistry:
             {
                 "execution_order": [],
                 "current_prompt": None,  # Will store the prompt object
+                "extra_data": None,  # Will store the API extra_data for workflow metadata
                 "timestamp": time.time(),
             }
         )
@@ -75,6 +76,11 @@ class MetadataRegistry:
             # Store the prompt in the metadata for later relationship tracing
             self.prompt_metadata[self.current_prompt_id]["current_prompt"] = prompt
 
+    def set_extra_data(self, extra_data):
+        """Store the API extra_data (contains extra_pnginfo.workflow with node properties)"""
+        if self.current_prompt_id and self.current_prompt_id in self.prompt_metadata:
+            self.prompt_metadata[self.current_prompt_id]["extra_data"] = extra_data
+
     def get_metadata(self, prompt_id=None):
         """Get collected metadata for a prompt"""
         key = prompt_id if prompt_id is not None else self.current_prompt_id
@@ -122,20 +128,28 @@ class MetadataRegistry:
             cache_key = f"{node_id}:{class_type}"
 
             # Check if this node type is relevant for metadata collection
-            if class_type in NODE_EXTRACTORS:
+            if class_type in NODE_EXTRACTORS or cache_key in self.node_cache:
                 # Check if we have cached metadata for this node
                 if cache_key in self.node_cache:
                     cached_data = self.node_cache[cache_key]
 
+                    # Detect bypass (mode=4) / mute (mode=2) — these nodes
+                    # were intentionally disabled and should not contribute
+                    # overwrite values from a previous execution's cache.
+                    node_mode = node_data.get("mode", 0)
+                    node_is_disabled = node_mode in (2, 4)
+
                     # Apply cached metadata to the current metadata
                     for category in self.metadata_categories:
+                        if category == OVERWRITE and node_is_disabled:
+                            continue
                         if category in cached_data and node_id in cached_data[category]:
                             if node_id not in metadata[category]:
                                 metadata[category][node_id] = cached_data[category][
                                     node_id
                                 ]
 
-    def record_node_execution(self, node_id, class_type, inputs, outputs):
+    def record_node_execution(self, node_id, class_type, inputs, outputs, return_types=None):
         """Record information about a node's execution"""
         if not self.current_prompt_id:
             return
@@ -158,17 +172,18 @@ class MetadataRegistry:
 
         # Extract node-specific metadata
         extractor = NODE_EXTRACTORS.get(class_type, GenericNodeExtractor)
-        extractor.extract(
-            node_id,
-            processed_inputs,
-            outputs,
-            self.prompt_metadata[self.current_prompt_id],
-        )
+        if extractor is GenericNodeExtractor:
+            extractor.extract(node_id, processed_inputs, outputs,
+                              self.prompt_metadata[self.current_prompt_id],
+                              return_types=return_types)
+        else:
+            extractor.extract(node_id, processed_inputs, outputs,
+                              self.prompt_metadata[self.current_prompt_id])
 
         # Cache this node's metadata
         self._cache_node_metadata(node_id, class_type)
 
-    def update_node_execution(self, node_id, class_type, outputs):
+    def update_node_execution(self, node_id, class_type, outputs, return_types=None):
         """Update node metadata with output information"""
         if not self.current_prompt_id:
             return
@@ -179,9 +194,17 @@ class MetadataRegistry:
         # Use the same extractor to update with outputs
         extractor = NODE_EXTRACTORS.get(class_type, GenericNodeExtractor)
         if hasattr(extractor, "update"):
-            extractor.update(
-                node_id, processed_outputs, self.prompt_metadata[self.current_prompt_id]
-            )
+            if extractor is GenericNodeExtractor:
+                extractor.update(
+                    node_id, processed_outputs,
+                    self.prompt_metadata[self.current_prompt_id],
+                    return_types=return_types,
+                )
+            else:
+                extractor.update(
+                    node_id, processed_outputs,
+                    self.prompt_metadata[self.current_prompt_id],
+                )
 
         # Update the cached metadata for this node
         self._cache_node_metadata(node_id, class_type)

py/metadata_collector/node_extractors.py

@@ -2,7 +2,7 @@ import json
 import os
 import re
 
-from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER
+from .constants import MODELS, PROMPTS, SAMPLING, LORAS, SIZE, IMAGES, IS_SAMPLER, OVERWRITE, METADATA_OVERWRITE_FIELDS
 
 
 def _store_checkpoint_metadata(metadata, node_id, model_name):
@@ -31,11 +31,78 @@ class NodeMetadataExtractor:
         pass
         
 class GenericNodeExtractor(NodeMetadataExtractor):
-    """Default extractor for nodes without specific handling"""
+    """Fallback extractor with type-signature-based detection.
+
+    When a node is not in the NODE_EXTRACTORS registry, the hook layer
+    passes ``return_types`` from ``obj.RETURN_TYPES``:
+
+    * ``MODEL`` output: common input fields (ckpt_name, unet_name, etc.)
+      are checked for a model file name and stored as checkpoint metadata.
+    * ``CONDITIONING`` output: common text input fields are checked for
+      prompt text and stored as prompt metadata.
+    """
+
+    # Input field names that carry a model path in loader-style nodes.
+    _MODEL_NAME_FIELDS = (
+        "ckpt_name", "unet_name", "model_path", "model_name", "gguf_name",
+    )
+
+    # Extensions used by checkpoint_scanner.py — only record values that look
+    # like real model filenames to avoid capturing unrelated string fields.
+    _MODEL_EXTENSIONS = {
+        ".ckpt", ".pt", ".pt2", ".bin", ".pth", ".safetensors", ".pkl", ".sft", ".gguf",
+    }
+
+    # Input field names that may carry prompt text in encoder-style nodes.
+    _TEXT_FIELDS = ("text", "clip_l", "t5xxl", "prompt", "positive", "negative")
+
     @staticmethod
-    def extract(node_id, inputs, outputs, metadata):
-        pass
-        
+    def extract(node_id, inputs, outputs, metadata, return_types=None):
+        if return_types is None:
+            return
+
+        # — MODEL loader detection (checkpoint / UNET / GGUF) —
+        if "MODEL" in return_types or any("MODEL" in str(t) for t in return_types):
+            for field in GenericNodeExtractor._MODEL_NAME_FIELDS:
+                val = inputs.get(field)
+                if val and isinstance(val, str) and val.strip():
+                    name = val.strip()
+                    if not any(name.lower().endswith(ext) for ext in GenericNodeExtractor._MODEL_EXTENSIONS):
+                        continue
+                    _store_checkpoint_metadata(metadata, node_id, name)
+                    return
+
+        # — CONDITIONING encoder detection (CLIPTextEncode, Flux, custom) —
+        if "CONDITIONING" in return_types or any("CONDITIONING" in str(t) for t in return_types):
+            text = None
+            for field in GenericNodeExtractor._TEXT_FIELDS:
+                val = inputs.get(field)
+                if val and isinstance(val, str) and val.strip():
+                    text = val.strip()
+                    break
+            if text:
+                prompt_data = metadata.setdefault(PROMPTS, {})
+                prompt_data[node_id] = {
+                    "text": text,
+                    "node_id": node_id,
+                }
+
+    @staticmethod
+    def update(node_id, outputs, metadata, return_types=None):
+        if return_types is None:
+            return
+        if "CONDITIONING" not in return_types and not any(
+            "CONDITIONING" in str(t) for t in return_types
+        ):
+            return
+        if node_id not in metadata.get(PROMPTS, {}):
+            return
+        if outputs and isinstance(outputs, list) and len(outputs) > 0:
+            if isinstance(outputs[0], tuple) and len(outputs[0]) > 0:
+                cond = outputs[0][0]
+                if cond is not None:
+                    metadata[PROMPTS][node_id]["conditioning"] = cond
+
 class CheckpointLoaderExtractor(NodeMetadataExtractor):
     @staticmethod
     def extract(node_id, inputs, outputs, metadata):
@@ -901,6 +968,55 @@ class LoraLoaderManagerExtractor(NodeMetadataExtractor):
                 "node_id": node_id
             }
 
+class LoraTextLoaderManagerExtractor(NodeMetadataExtractor):
+    """Extract LoRA metadata from LoraTextLoaderLM (LoRA Text Loader).
+
+    The node accepts a `lora_syntax` STRING containing <lora:name:strength> tags
+    (same format as the ComfyUI prompt), plus an optional `lora_stack`.
+    This extractor parses the syntax string using the same regex as the node.
+    """
+    @staticmethod
+    def extract(node_id, inputs, outputs, metadata):
+        if not inputs:
+            return
+
+        active_loras = []
+
+        # Process lora_stack if available (optional input)
+        if "lora_stack" in inputs:
+            lora_stack = inputs.get("lora_stack", [])
+            for item in lora_stack:
+                # lora_stack entries are (path, model_strength, clip_strength) tuples
+                if isinstance(item, (list, tuple)) and len(item) >= 2:
+                    lora_path = item[0]
+                    model_strength = item[1]
+                    lora_name = os.path.splitext(os.path.basename(lora_path))[0]
+                    active_loras.append({
+                        "name": lora_name,
+                        "strength": round(float(model_strength), 2)
+                    })
+
+        # Process lora_syntax string input
+        if "lora_syntax" in inputs:
+            lora_syntax = inputs.get("lora_syntax", "")
+            if lora_syntax and isinstance(lora_syntax, str):
+                pattern = r"<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>"
+                matches = re.findall(pattern, lora_syntax, re.IGNORECASE)
+                for match in matches:
+                    lora_name = match[0]
+                    model_strength = float(match[1])
+                    active_loras.append({
+                        "name": lora_name,
+                        "strength": round(model_strength, 2)
+                    })
+
+        if active_loras:
+            metadata[LORAS][node_id] = {
+                "lora_list": active_loras,
+                "node_id": node_id
+            }
+
+
 class FluxGuidanceExtractor(NodeMetadataExtractor):
     @staticmethod
     def extract(node_id, inputs, outputs, metadata):
@@ -1105,6 +1221,32 @@ class CR_ApplyControlNetStackExtractor(NodeMetadataExtractor):
                 metadata[PROMPTS][node_id]["positive_encoded"] = transformed_positive
                 metadata[PROMPTS][node_id]["negative_encoded"] = transformed_negative
 
+class MetadataOverwriteExtractor(NodeMetadataExtractor):
+    """Extract manually specified metadata from MetadataOverwriteLM node.
+
+    Stores truthy input values under the OVERWRITE category so that
+    extract_generation_params can merge them over the inferred params.
+    """
+
+    @staticmethod
+    def extract(node_id, inputs, outputs, metadata):
+        if not inputs:
+            return
+
+        overwrite_params = {}
+        for key in METADATA_OVERWRITE_FIELDS:
+            value = inputs.get(key)
+            if value:  # truthy — only overwrite when user provided a real value
+                overwrite_params[key] = value
+
+        if overwrite_params:
+            metadata.setdefault(OVERWRITE, {})
+            metadata[OVERWRITE][node_id] = {
+                "parameters": overwrite_params,
+                "node_id": node_id,
+            }
+
+
 # Registry of node-specific extractors
 # Keys are node class names
 NODE_EXTRACTORS = {
@@ -1146,6 +1288,7 @@ NODE_EXTRACTORS = {
     "UNETLoaderLM": UNETLoaderExtractor,  # LoRA Manager
     "LoraLoader": LoraLoaderExtractor,
     "LoraLoaderLM": LoraLoaderManagerExtractor,
+    "LoraTextLoaderLM": LoraTextLoaderManagerExtractor,
     "RgthreePowerLoraLoader": RgthreePowerLoraLoaderExtractor,
     "TensorRTLoader": TensorRTLoaderExtractor,
     # Conditioning
@@ -1171,5 +1314,7 @@ NODE_EXTRACTORS = {
     "CFGGuider": CFGGuiderExtractor,            # Add CFGGuider
     # Image
     "VAEDecode": VAEDecodeExtractor,  # Added VAEDecode extractor
+    # Metadata overwrite
+    "MetadataOverwriteLM": MetadataOverwriteExtractor,
     # Add other nodes as needed
 }

py/metadata_ops/__init__.py

@@ -0,0 +1,233 @@
+"""Metadata operations — thin in-process wrappers around LoRA Manager internal services.
+
+All functions are simple Python async functions that delegate to the
+appropriate internal service.  They use **relative imports** within the
+``py`` package, so ``sys.modules`` caching works normally and there is no
+risk of double import or circular dependencies.
+
+Usage (in-process, primary)::
+
+    from py.metadata_ops import list_base_models, read_metadata
+
+    models = await list_base_models()
+    meta   = await read_metadata("/path/to/model.safetensors")
+
+Usage (subprocess, debugging / external)::
+
+    python -m py.metadata_ops base-models list
+    python -m py.metadata_ops metadata read /path/to/model.safetensors
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import os
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+SCANNER_TYPE_MAP: dict[str, str] = {
+    "get_lora_scanner": "lora",
+    "get_checkpoint_scanner": "checkpoint",
+    "get_embedding_scanner": "embedding",
+}
+
+SCANNER_GETTER_NAMES = tuple(SCANNER_TYPE_MAP.keys())
+
+
+async def _find_model_entry(
+    model_path: str,
+) -> tuple[object, object, str | None] | tuple[None, None, None]:
+    """Iterate all scanners and return the first (scanner, entry, getter_name)
+    that owns *model_path*.  Returns ``(None, None, None)`` when no scanner
+    claims it.
+    """
+    from ..services.service_registry import ServiceRegistry
+
+    normalized = os.path.normpath(model_path)
+    for getter_name in SCANNER_GETTER_NAMES:
+        getter = getattr(ServiceRegistry, getter_name, None)
+        if getter is None:
+            continue
+        try:
+            scanner = await getter()
+            if scanner is None:
+                continue
+            cache = await scanner.get_cached_data()
+            for entry in cache.raw_data:
+                if os.path.normpath(entry.get("file_path", "")) == normalized:
+                    return scanner, entry, getter_name
+        except Exception as exc:
+            logger.debug(
+                "Scanner %s check failed for %s: %s",
+                getter_name, model_path, exc,
+            )
+    return None, None, None
+
+
+async def _find_scanner_for_model(
+    model_path: str,
+) -> tuple[object, object] | tuple[None, None]:
+    """Find the (scanner, cache_entry) responsible for *model_path*."""
+    scanner, entry, _ = await _find_model_entry(model_path)
+    return scanner, entry
+
+
+async def identify_model_type(model_path: str) -> str:
+    """Determine the model type (``\"lora\"``, ``\"checkpoint\"``, or
+    ``\"embedding\"``) for *model_path*.
+
+    Falls back to ``\"lora\"`` when unknown.
+    """
+    _, _, getter_name = await _find_model_entry(model_path)
+    return SCANNER_TYPE_MAP[getter_name] if getter_name else "lora"
+
+
+# ---------------------------------------------------------------------------
+# Public API
+# ---------------------------------------------------------------------------
+
+
+async def list_base_models(limit: int = 0) -> List[str]:
+    """Return all valid CivitAI base model names.
+
+    Uses ``CivitaiBaseModelService.get_base_models()`` which merges a
+    hardcoded list (``SUPPORTED_DOWNLOAD_SKIP_BASE_MODELS``) with remote
+    models fetched from the CivitAI API.  Never empty — the hardcoded
+    fallback always provides a complete set.
+
+    The result is sorted alphabetically.  Pass *limit* = 0 for all models.
+    """
+    from ..services.civitai_base_model_service import (
+        CivitaiBaseModelService,
+    )
+
+    try:
+        service = await CivitaiBaseModelService.get_instance()
+        response = await service.get_base_models()
+        names: List[str] = response.get("models", [])
+    except Exception as exc:
+        logger.warning("list_base_models failed: %s", exc)
+        names = []
+    if limit > 0:
+        return names[:limit]
+    return names
+
+
+async def read_metadata(model_path: str) -> Dict[str, Any]:
+    """Load the full metadata payload for *model_path* from disk.
+
+    Returns an empty dict when the metadata file does not exist or cannot
+    be parsed — never raises.
+    """
+    from ..utils.metadata_manager import MetadataManager
+
+    try:
+        return await MetadataManager.load_metadata_payload(model_path) or {}
+    except Exception as exc:
+        logger.warning("read_metadata failed for %s: %s", model_path, exc)
+        return {}
+
+
+async def apply_metadata_updates(
+    model_path: str,
+    updates: Dict[str, Any],
+) -> List[str]:
+    """Merge *updates* into the model's on-disk metadata and persist.
+
+    Returns the list of field names that actually changed.
+    """
+    from ..utils.metadata_manager import MetadataManager
+
+    metadata = await read_metadata(model_path)
+    updated_fields: List[str] = []
+    for key, value in updates.items():
+        old = metadata.get(key)
+        if old != value:
+            metadata[key] = value
+            updated_fields.append(key)
+    if updated_fields:
+        await MetadataManager.save_metadata(model_path, metadata)
+    return updated_fields
+
+
+async def download_preview(
+    model_path: str,
+    url: str,
+    *,
+    target_width: int = 480,
+    quality: int = 85,
+) -> str | None:
+    """Download a preview image from *url*, optimise to .webp, and save it.
+
+    The output file is placed alongside the model file with a ``.webp``
+    extension.  Returns the local file path on success, ``None`` on failure.
+    """
+    from ..services.downloader import get_downloader
+    from ..utils.exif_utils import ExifUtils
+
+    if not url or not url.strip():
+        return None
+
+    base_name = os.path.splitext(os.path.basename(model_path))[0]
+    preview_dir = os.path.dirname(model_path)
+    output_path = os.path.join(preview_dir, base_name + ".webp")
+
+    downloader = await get_downloader()
+
+    # Try in-memory download + optimise first
+    success, content, _headers = await downloader.download_to_memory(
+        url, use_auth=False,
+    )
+    if success and content:
+        try:
+            optimized_data, _ = ExifUtils.optimize_image(
+                image_data=content,
+                target_width=target_width,
+                format="webp",
+                quality=quality,
+                preserve_metadata=False,
+            )
+            with open(output_path, "wb") as f:
+                f.write(optimized_data)
+            return output_path
+        except Exception as exc:
+            logger.warning("Preview optimisation failed, saving raw: %s", exc)
+            # Fall through to raw save
+
+    # Fallback: download directly to file
+    try:
+        ok, _ = await downloader.download_file(url, output_path, use_auth=False)
+        if ok:
+            return output_path
+    except Exception as exc:
+        logger.warning("Preview fallback download failed for %s: %s", model_path, exc)
+
+    return None
+
+
+async def refresh_cache(model_path: str) -> bool:
+    """Invalidate and reload the scanner cache entry for *model_path*.
+
+    Returns ``True`` when the model was found and the cache was refreshed.
+    """
+    scanner, entry = await _find_scanner_for_model(model_path)
+    if scanner is None:
+        logger.warning("refresh_cache: no scanner found for %s", model_path)
+        return False
+    try:
+        metadata = await read_metadata(model_path)
+        if not metadata:
+            logger.warning("refresh_cache: no metadata for %s", model_path)
+            return False
+        await scanner.update_single_model_cache(model_path, model_path, metadata)
+        return True
+    except Exception as exc:
+        logger.warning("refresh_cache failed for %s: %s", model_path, exc)
+        return False

py/metadata_ops/__main__.py

@@ -0,0 +1,113 @@
+"""Subprocess entry point for ``metadata_ops`` (debugging / external use).
+
+Usage::
+
+    python -m py.metadata_ops base-models list [--limit N]
+    python -m py.metadata_ops metadata read <path>
+    python -m py.metadata_ops metadata update <path> --json '{...}'
+    python -m py.metadata_ops preview download <path> --url <url>
+    python -m py.metadata_ops cache refresh <path>
+"""
+
+from __future__ import annotations
+
+import argparse
+import asyncio
+import json
+import sys
+from typing import Any, Dict, List
+
+
+def _build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="lmcli", description="LoRA Manager Agent CLI")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    # base-models list
+    base_models = sub.add_parser("base-models", aliases=["bm"])
+    base_models_cmds = base_models.add_subparsers(dest="subcommand", required=True)
+    base_models_list = base_models_cmds.add_parser("list")
+    base_models_list.add_argument(
+        "--limit", type=int, default=0, help="Max number of models (0 = all)"
+    )
+
+    # metadata read
+    meta = sub.add_parser("metadata", aliases=["md"])
+    meta_cmds = meta.add_subparsers(dest="subcommand", required=True)
+    meta_read = meta_cmds.add_parser("read")
+    meta_read.add_argument("path", type=str, help="Model file path")
+
+    # metadata update
+    meta_update = meta_cmds.add_parser("update")
+    meta_update.add_argument("path", type=str, help="Model file path")
+    meta_update.add_argument(
+        "--json",
+        type=str,
+        required=True,
+        help='JSON object of fields to update, e.g. \'{"base_model": "SDXL 1.0"}\'',
+    )
+
+    # preview download
+    prev = sub.add_parser("preview", aliases=["pv"])
+    prev_cmds = prev.add_subparsers(dest="subcommand", required=True)
+    prev_dl = prev_cmds.add_parser("download")
+    prev_dl.add_argument("path", type=str, help="Model file path")
+    prev_dl.add_argument("--url", type=str, required=True, help="Preview image URL")
+
+    # cache refresh
+    cache = sub.add_parser("cache")
+    cache_cmds = cache.add_subparsers(dest="subcommand", required=True)
+    cache_refresh = cache_cmds.add_parser("refresh")
+    cache_refresh.add_argument("path", type=str, help="Model file path")
+
+    return parser
+
+
+async def _run(args: argparse.Namespace) -> Any:
+    from . import (  # lazy import so startup is fast
+        list_base_models,
+        read_metadata,
+        apply_metadata_updates,
+        download_preview,
+        refresh_cache,
+    )
+
+    cmd = args.command
+    sub = args.subcommand
+
+    if cmd in ("base-models", "bm") and sub == "list":
+        return await list_base_models(limit=args.limit)
+
+    if cmd in ("metadata", "md") and sub == "read":
+        return await read_metadata(args.path)
+
+    if cmd in ("metadata", "md") and sub == "update":
+        updates: Dict[str, Any] = json.loads(args.json)
+        return await apply_metadata_updates(args.path, updates)
+
+    if cmd in ("preview", "pv") and sub == "download":
+        return await download_preview(args.path, args.url)
+
+    if cmd == "cache" and sub == "refresh":
+        return await refresh_cache(args.path)
+
+    raise ValueError(f"Unknown command: {cmd} {sub}")
+
+
+def main() -> None:
+    parser = _build_parser()
+    args = parser.parse_args()
+
+    result = asyncio.run(_run(args))
+    # Always print as JSON so callers can parse reliably
+    if isinstance(result, list):
+        for item in result:
+            print(item)
+    elif isinstance(result, dict):
+        json.dump(result, sys.stdout, ensure_ascii=False, indent=2)
+        print()
+    else:
+        print(json.dumps(result))
+
+
+if __name__ == "__main__":
+    main()

py/middleware/cache_middleware.py

@@ -16,6 +16,8 @@ IMG_EXTENSIONS = (
     ".tif",
     ".tiff",
     ".webp",
+    ".avif",
+    ".jxl",
     ".mp4"
 )
 

py/middleware/error_middleware.py

@@ -0,0 +1,76 @@
+"""JSON error middleware for API routes.
+
+Ensures all responses to /api/* requests return valid JSON that the
+browser-extension frontend can JSON.parse() without crashing, even when
+the route does not exist (404) or the handler raises an exception (500).
+
+Extension consumers call response.json() unconditionally — an HTML error
+page causes ``SyntaxError: unexpected end of data`` that leaks into the
+popup UI as a toast notification.
+"""
+
+from __future__ import annotations
+
+import logging
+from typing import Awaitable, Callable
+
+from aiohttp import web
+
+logger = logging.getLogger(__name__)
+
+
+@web.middleware
+async def api_json_error(
+    request: web.Request,
+    handler: Callable[[web.Request], Awaitable[web.Response]],
+) -> web.Response:
+    """Return JSON ``{"success": false, "error": "..."}`` for API errors.
+
+    Only intercepts paths starting with ``/api/`` — all other routes
+    (frontend pages, static files, WebSocket upgrades) pass through
+    unchanged.
+    """
+    if not request.path.startswith("/api/"):
+        return await handler(request)
+
+    try:
+        response = await handler(request)
+        return response
+    except web.HTTPException as exc:
+        # Let redirects (301, 302, 307, 308) propagate — they are not errors.
+        if exc.status < 400:
+            raise
+
+        # Preview 404 is routine (file deleted from disk) — not worth a warning.
+        logger_method = logger.warning
+        if request.path.startswith("/api/lm/previews") and exc.status == 404:
+            logger_method = logger.debug
+
+        logger_method(
+            "API %s %s returned HTTP %d: %s",
+            request.method,
+            request.path,
+            exc.status,
+            exc.reason,
+        )
+
+        return web.json_response(
+            {"success": False, "error": f"{exc.status}: {exc.reason}"},
+            status=exc.status,
+        )
+    except Exception as exc:
+        logger.error(
+            "API %s %s raised unhandled exception: %s",
+            request.method,
+            request.path,
+            exc,
+            exc_info=True,
+        )
+
+        return web.json_response(
+            {
+                "success": False,
+                "error": f"500: Internal Server Error ({type(exc).__name__})",
+            },
+            status=500,
+        )

py/nodes/create_hook_lora.py

@@ -0,0 +1,117 @@
+"""Create Hook LoRA (LoraManager) — multi-LoRA hook node compatible with ComfyUI's built-in hook pipeline.
+
+Produces ``("HOOKS",)`` output that chains seamlessly with downstream hook consumers
+(ConditioningSetProperties, SetHookKeyframes, CombineHooks, SetClipHooks, etc.).
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+
+from ..utils.utils import get_lora_info_absolute
+from .utils import (
+    FlexibleOptionalInputType,
+    any_type,
+    apply_lora_syntax_format,
+    get_loras_list,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class CreateHookLoraLM:
+    NAME = "Create Hook LoRA (LoraManager)"
+    CATEGORY = "Lora Manager/hooks"
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "text": (
+                    "AUTOCOMPLETE_TEXT_LORAS",
+                    {
+                        "placeholder": "Search LoRAs to add...",
+                        "tooltip": (
+                            "Search and select LoRAs. Each LoRA gets its own "
+                            "model/clip strength. Hooks chain with prev_hooks."
+                        ),
+                    },
+                ),
+            },
+            "optional": FlexibleOptionalInputType(any_type),
+        }
+
+    RETURN_TYPES = ("HOOKS", "STRING", "STRING")
+    RETURN_NAMES = ("HOOKS", "trigger_words", "active_loras")
+    FUNCTION = "create_hook"
+
+    def create_hook(self, text: str, **kwargs):
+        """Create a HookGroup from the selected LoRAs, chained with prev_hooks.
+
+        Each active LoRA from the widget is loaded and wrapped in a WeightHook
+        via :func:`comfy.hooks.create_hook_lora`.  All hooks are combined into a
+        single group and returned alongside trigger words and a human-readable
+        summary of the active LoRAs.
+        """
+        del text  # used by the frontend widget only
+
+        # Lazy imports: comfy is not available in CI/test environment at module level
+        import comfy.hooks  # type: ignore  # noqa: C0415
+        import comfy.utils  # type: ignore  # noqa: C0415
+
+        prev_hooks: comfy.hooks.HookGroup | None = kwargs.get("prev_hooks")
+
+        hook_group = prev_hooks.clone() if prev_hooks is not None else comfy.hooks.HookGroup()
+
+        all_trigger_words: list[str] = []
+        active_loras: list[tuple[str, float, float]] = []
+
+        for lora in get_loras_list(kwargs):
+            if not lora.get("active", False):
+                continue
+
+            lora_name = apply_lora_syntax_format(lora["name"])
+            model_strength = float(lora["strength"])
+            clip_strength = float(lora.get("clipStrength", model_strength))
+
+            # Skip useless no-op entries (both strengths are zero)
+            if model_strength == 0.0 and clip_strength == 0.0:
+                continue
+
+            lora_path, trigger_words = get_lora_info_absolute(lora_name)
+            if not lora_path or not os.path.isfile(lora_path):
+                logger.warning("LoRA '%s' not found — skipping", lora_name)
+                continue
+
+            try:
+                lora_weights = comfy.utils.load_torch_file(lora_path, safe_load=True)
+
+                lora_hooks = comfy.hooks.create_hook_lora(
+                    lora=lora_weights,
+                    strength_model=model_strength,
+                    strength_clip=clip_strength,
+                )
+            except Exception:
+                logger.exception("Failed to load LoRA '%s' — skipping", lora_name)
+                continue
+            hook_group = hook_group.clone_and_combine(lora_hooks)
+
+            active_loras.append((lora_name, model_strength, clip_strength))
+            all_trigger_words.extend(trigger_words)
+
+        # Format trigger words (group mode separator)
+        trigger_words_text = ",, ".join(all_trigger_words) if all_trigger_words else ""
+
+        # Format active LoRAs summary
+        formatted_loras = []
+        for name, model_s, clip_s in active_loras:
+            if abs(model_s - clip_s) > 0.001:
+                formatted_loras.append(
+                    f"<lora:{name}:{model_s}:{clip_s}>"
+                )
+            else:
+                formatted_loras.append(f"<lora:{name}:{model_s}>")
+        active_loras_text = " ".join(formatted_loras)
+
+        return (hook_group, trigger_words_text, active_loras_text)

py/nodes/lora_info.py

@@ -0,0 +1,45 @@
+"""Lora Info display node — pure frontend node for showing selected LoRA info.
+
+This node does NOT participate in workflow execution. Its single optional
+"lora_source" input exists solely as a wire-connection anchor so that the
+frontend can traverse the graph and push selection data to connected info nodes.
+"""
+
+from __future__ import annotations
+
+
+class LoraInfoLM:
+    """Display node that shows filename and notes for the selected LoRA."""
+
+    NAME = "Lora Info (LoraManager)"
+    CATEGORY = "Lora Manager/utils"
+    DESCRIPTION = (
+        "Displays information (filename, notes) about the currently selected "
+        "LoRA. Connect any output from a LoRA Loader or Stacker to the "
+        "lora_source input, then select a LoRA in the source widget — the "
+        "info updates automatically. Does not affect workflow execution."
+    )
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {},
+        }
+
+    RETURN_TYPES = ()
+    RETURN_NAMES = ()
+    OUTPUT_NODE = False
+    FUNCTION = "noop"
+
+    def noop(self, **kwargs):
+        # This node is display-only — no workflow execution needed.
+        return ()
+
+
+NODE_CLASS_MAPPINGS = {
+    LoraInfoLM.NAME: LoraInfoLM,
+}
+
+NODE_DISPLAY_NAME_MAPPINGS = {
+    LoraInfoLM.NAME: "Lora Info (LoraManager)",
+}

py/nodes/lora_loader.py

@@ -1,6 +1,5 @@
 import importlib
 import logging
-import re
 
 import comfy.sd  # type: ignore
 import comfy.utils  # type: ignore
@@ -14,6 +13,7 @@ from .utils import (
     extract_lora_name,
     get_loras_list,
     nunchaku_load_lora,
+    parse_lora_syntax,
 )
 
 logger = logging.getLogger(__name__)
@@ -189,25 +189,10 @@ class LoraTextLoaderLM:
     RETURN_NAMES = ("MODEL", "CLIP", "trigger_words", "loaded_loras")
     FUNCTION = "load_loras_from_text"
 
-    def parse_lora_syntax(self, text):
-        """Parse LoRA syntax from text input."""
-        pattern = r"<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>"
-        matches = re.findall(pattern, text, re.IGNORECASE)
-
-        loras = []
-        for match in matches:
-            model_strength = float(match[1])
-            loras.append({
-                "name": match[0],
-                "model_strength": model_strength,
-                "clip_strength": float(match[2]) if match[2] else model_strength,
-            })
-        return loras
-
     def load_loras_from_text(self, model, lora_syntax, clip=None, lora_stack=None):
         """Load LoRAs based on text syntax input."""
         lora_entries = _collect_stack_entries(lora_stack)
-        for lora in self.parse_lora_syntax(lora_syntax):
+        for lora in parse_lora_syntax(lora_syntax):
             lora_path, trigger_words = get_lora_info_absolute(lora["name"])
             lora_entries.append({
                 "name": lora["name"],

py/nodes/lora_syntax_to_path.py

@@ -0,0 +1,62 @@
+"""Node to resolve `<lora:name:strength>` syntax to absolute file system paths.
+
+Takes the loaded_loras / active_loras STRING output from LoraLoaderLM or
+LoraStackerLM and resolves each lora name to its absolute path on disk via
+the scanner cache. Unknown names are returned as-is.
+"""
+
+import logging
+
+from ..utils.utils import get_lora_info_absolute
+from .utils import parse_lora_syntax
+
+logger = logging.getLogger(__name__)
+
+
+class LoraSyntaxToPath:
+    NAME = "LoRA Syntax → Path (LoraManager)"
+    CATEGORY = "Lora Manager/utils"
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "lora_syntax": (
+                    "STRING",
+                    {
+                        "forceInput": True,
+                        "multiline": True,
+                        "tooltip": (
+                            "<lora:name:strength> formatted text from "
+                            "loaded_loras / active_loras output"
+                        ),
+                    },
+                ),
+            },
+        }
+
+    RETURN_TYPES = ("STRING",)
+    RETURN_NAMES = ("paths",)
+    FUNCTION = "resolve"
+
+    def resolve(self, lora_syntax: str) -> tuple[str]:
+        """Parse <lora:...> syntax and resolve each name to its absolute path."""
+        if not lora_syntax or not lora_syntax.strip():
+            logger.info("Received empty lora_syntax input")
+            return ("",)
+
+        parsed = parse_lora_syntax(lora_syntax)
+        if not parsed:
+            logger.info("No valid <lora:...> entries found in input")
+            return ("",)
+
+        paths: list[str] = []
+        for entry in parsed:
+            try:
+                absolute_path, _ = get_lora_info_absolute(entry["name"])
+                paths.append(absolute_path)
+            except Exception:
+                logger.warning("Failed to resolve lora '%s', skipping", entry["name"])
+                continue
+
+        return ("\n".join(paths),)

py/nodes/metadata_overwrite.py

@@ -0,0 +1,157 @@
+"""Metadata Overwrite node — allows users to manually specify generation parameters
+that override the automatically collected/inferred metadata.
+
+All inputs have falsy defaults: only truthy (non-empty / non-zero) values
+will overwrite the corresponding field in the final metadata.
+"""
+
+from typing import Any
+
+from ..metadata_collector.constants import METADATA_OVERWRITE_FIELDS
+
+
+class MetadataOverwriteLM:
+    NAME = "Metadata Overwrite (LoraManager)"
+    CATEGORY = "Lora Manager/utils"
+    DESCRIPTION = (
+        "Manually specify generation parameters to override automatically collected "
+        "metadata. Only filled/connected inputs will take effect — empty defaults "
+        "are ignored."
+    )
+
+    @classmethod
+    def INPUT_TYPES(cls) -> dict[str, Any]:
+        return {
+            "optional": {
+                "prompt": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "multiline": True,
+                        "tooltip": "Positive prompt. Only overwrites when non-empty.",
+                    },
+                ),
+                "negative_prompt": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "multiline": True,
+                        "tooltip": "Negative prompt. Only overwrites when non-empty.",
+                    },
+                ),
+                "seed": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": 0,
+                        "max": 0xFFFFFFFFFFFFFFFF,
+                        "control_after_generate": False,
+                        "tooltip": "Seed value. Only overwrites when > 0.",
+                    },
+                ),
+                "steps": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": 0,
+                        "max": 10000,
+                        "tooltip": "Number of steps. Only overwrites when > 0.",
+                    },
+                ),
+                "cfg_scale": (
+                    "FLOAT",
+                    {
+                        "default": 0.0,
+                        "min": 0.0,
+                        "max": 100.0,
+                        "tooltip": "CFG scale. Only overwrites when > 0.",
+                    },
+                ),
+                "sampler": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Sampler name. Only overwrites when non-empty.",
+                    },
+                ),
+                "scheduler": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": "Scheduler name. Only overwrites when non-empty.",
+                    },
+                ),
+                "model": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": (
+                            "The checkpoint or diffusion model (UNet) used "
+                            "for generation. Only overwrites when non-empty."
+                        ),
+                    },
+                ),
+                "loras": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "multiline": True,
+                        "tooltip": (
+                            "LoRA syntax, e.g. <lora:name:strength> "
+                            "or <lora:name:model_strength:clip_strength>, "
+                            "separated by spaces. Only overwrites when non-empty."
+                        ),
+                    },
+                ),
+                "size": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "tooltip": (
+                            "Image size in WIDTHxHEIGHT format (e.g. 512x768). "
+                            "Only overwrites when non-empty."
+                        ),
+                    },
+                ),
+                "clip_skip": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": -24,
+                        "max": 24,
+                        "tooltip": "Clip skip. Only overwrites when non-zero.",
+                    },
+                ),
+                "additional_data": (
+                    "STRING",
+                    {
+                        "default": "",
+                        "multiline": True,
+                        "tooltip": (
+                            "Additional data to embed in the image metadata. "
+                            "Inserted between Clip skip and Model hash in the "
+                            "A1111-compatible parameters string. "
+                            'Example: "Copyright": "Some license info"'
+                        ),
+                    },
+                ),
+            },
+        }
+
+    RETURN_TYPES = ("METADATA",)
+    RETURN_NAMES = ("metadata",)
+    FUNCTION = "collect_metadata"
+    OUTPUT_NODE = True
+
+    def collect_metadata(self, **kwargs: Any) -> tuple[dict[str, Any]]:
+        """Collect non-falsy input values into a metadata dict.
+
+        Only values that are truthy (non-empty string, non-zero number)
+        are included — matching the overwrite logic in the metadata pipeline.
+        """
+        result: dict[str, Any] = {}
+        for key in METADATA_OVERWRITE_FIELDS:
+            value = kwargs.get(key)
+            if value:
+                result[key] = value
+        return (result,)

py/nodes/save_image.py

@@ -11,11 +11,161 @@ from ..metadata_collector.metadata_processor import MetadataProcessor
 from ..metadata_collector import get_metadata
 from ..utils.constants import CARD_PREVIEW_WIDTH
 from ..utils.exif_utils import ExifUtils
-from ..utils.utils import calculate_recipe_fingerprint
+from ..utils.utils import calculate_recipe_fingerprint, sanitize_folder_name
 from PIL import Image, PngImagePlugin
 import piexif
 import logging
 
+# Civitai-compatible sampler name mapping: ComfyUI internal → A1111 display name
+CIVITAI_SAMPLER_MAP = {
+    "euler": "Euler",
+    "euler_ancestral": "Euler a",
+    "lms": "LMS",
+    "heun": "Heun",
+    "dpm_2": "DPM2",
+    "dpm_2_ancestral": "DPM2 a",
+    "dpmpp_2s_ancestral": "DPM++ 2S a",
+    "dpmpp_2m": "DPM++ 2M",
+    "dpmpp_sde": "DPM++ SDE",
+    "dpmpp_sde_gpu": "DPM++ SDE",
+    "dpmpp_2m_sde": "DPM++ 2M SDE",
+    "dpmpp_2m_sde_gpu": "DPM++ 2M SDE",
+    "dpmpp_3m_sde": "DPM++ 3M SDE",
+    "dpm_fast": "DPM fast",
+    "dpm_adaptive": "DPM adaptive",
+    "ddim": "DDIM",
+    "plms": "PLMS",
+    "uni_pc_bh2": "UniPC",
+    "uni_pc": "UniPC",
+    "lcm": "LCM",
+}
+
+# Base model display name → AIR URN slug
+# Sourced from civitai source: src/shared/constants/basemodel.constants.ts
+BASE_MODEL_AIR_SLUG = {
+    # Stable Diffusion family
+    "SD 1.4": "sd1",
+    "SD 1.5": "sd1",
+    "SD 1.5 LCM": "sd1",
+    "SD 1.5 Hyper": "sd1",
+    "SD 2.0": "sd2",
+    "SD 2.0 768": "sd2",
+    "SD 2.1": "sd2",
+    "SD 2.1 768": "sd2",
+    "SD 2.1 Unclip": "sd2",
+    "SD 3.0": "sd3",
+    "SD 3.5": "sd35",
+    "SD 3.5 Large": "sd35",
+    "SD 3.5 Large Turbo": "sd35",
+    "SD 3.5 Medium": "sd35",
+    "SDXL 0.9": "sdxl",
+    "SDXL 1.0": "sdxl",
+    "SDXL 1.0 LCM": "sdxl",
+    "SDXL Lightning": "sdxl",
+    "SDXL Hyper": "sdxl",
+    "SDXL Turbo": "sdxl",
+    "SDXL Distilled": "sdxldistilled",
+    "Stable Cascade": "scascade",
+    "Stable Video Diffusion": "svd",
+    "SVD": "svd",
+    "SVD XT": "svdxt",
+
+    # SDXL community fine-tunes
+    "Pony": "pony",
+    "Pony Diffusion": "pony",
+    "Illustrious": "illustrious",
+    "NoobAI": "noobai",
+    "Animagine": "illustrious",
+
+    # Flux family
+    "Flux.1": "flux1",
+    "Flux.1 D": "flux1",
+    "Flux.1 S": "flux1",
+    "Flux.1 Krea": "fluxkrea",
+    "Flux.1 Kontext": "flux1kontext",
+    "Flux.2": "flux2",
+    "Flux.2 D": "flux2",
+    "Flux.2 Klein 9B": "flux2klein_9b",
+    "Flux.2 Klein 9B Base": "flux2klein_9b_base",
+    "Flux.2 Klein 4B": "flux2klein_4b",
+    "Flux.2 Klein 4B Base": "flux2klein_4b_base",
+
+    # Other image models (sorted alphabetically)
+    "AuraFlow": "auraflow",
+    "Chroma": "chroma",
+    "HiDream": "hidream",
+    "HiDream-O1": "hidream-o1",
+    "Hunyuan DiT": "hydit1",
+    "Hunyuan Video": "hyv1",
+    "Kolors": "kolors",
+    "Lumina": "lumina",
+    "Mochi": "mochi",
+    "ODOR": "odor",
+    "PixArt Alpha": "pixarta",
+    "PixArt Sigma": "pixarte",
+    "Playground v2": "playgroundv2",
+    "Playground v2.5": "playgroundv2",
+    "Pony Diffusion V7": "ponyv7",
+
+    # Video models
+    "CogVideoX": "cogvideox",
+    "LTX Video": "ltxv",
+    "LTX Video 2": "ltxv2",
+    "LTX Video 2.3": "ltxv23",
+    "Wan Video": "wanvideo",
+    "Wan Video 1.3B T2V": "wanvideo_13b_t2v",
+    "Wan Video 14B T2V": "wanvideo_14b_t2v",
+    "Wan Video 14B I2V 480p": "wanvideo_14b_i2v_480p",
+    "Wan Video 14B I2V 720p": "wanvideo_14b_i2v_720p",
+
+    # Third-party / proprietary image models
+    "Boogu": "boogu",
+    "Ernie": "ernie",
+    "Grok": "grok",
+    "HappyHorse": "happyhorse",
+    "Ideogram": "ideogram",
+    "Ideogram 4.0": "ideogram",
+    "Imagen": "imagen4",
+    "Imagen 4": "imagen4",
+    "Krea": "krea2",
+    "Krea 2": "krea2",
+    "Lens": "lens",
+    "MAI": "mai",
+    "Nano Banana": "nanobanana",
+    "OpenAI": "openai",
+    "Reve": "reve",
+    "Reve 2": "reve",
+    "Reve 2.1": "reve",
+    "Seedream": "seedream",
+    "Sora": "sora2",
+    "Sora 2": "sora2",
+    "Veo": "veo3",
+    "Veo 2": "veo3",
+    "Veo 3": "veo3",
+    "ZImageTurbo": "zimageturbo",
+    "ZImageBase": "zimagebase",
+    "ZImage": "zimagebase",
+
+    # Third-party video models
+    "Hailuo by MiniMax": "minimax",
+    "Haiper": "haiper",
+    "Kling": "kling",
+    "Lightricks": "lightricks",
+    "Seedance": "seedance",
+    "Vidu": "vidu",
+
+    # Qwen family
+    "Qwen": "qwen",
+    "Qwen 2": "qwen2",
+
+    # Anima
+    "Anima": "anima",
+
+    # Special
+    "Upscaler": "upscaler",
+    "Other": "other",
+}
+
 logger = logging.getLogger(__name__)
 
 
@@ -70,11 +220,29 @@ class SaveImageLM:
                         "tooltip": "Compression quality for JPEG and lossy WebP formats (1-100). Higher values mean better quality but larger files.",
                     },
                 ),
+                "webp_method": (
+                    "INT",
+                    {
+                        "default": 6,
+                        "min": 0,
+                        "max": 6,
+                        "tooltip": "WebP compression method (0-6). 0=fastest/largest, 6=slowest/smallest. Only applies when file_format is 'webp'.",
+                    },
+                ),
+                "jpeg_subsampling": (
+                    "INT",
+                    {
+                        "default": 0,
+                        "min": 0,
+                        "max": 2,
+                        "tooltip": "JPEG chroma subsampling level. 0=4:4:4 (best quality), 1=4:2:2, 2=4:2:0 (smallest files). Only applies when file_format is 'jpeg'.",
+                    },
+                ),
                 "embed_workflow": (
                     "BOOLEAN",
                     {
                         "default": False,
-                        "tooltip": "Embeds the complete workflow data into the image metadata. Only works with PNG and WebP formats.",
+                        "tooltip": "When enabled, saved images store the complete workflow. Drag the image back into ComfyUI to restore the original node graph. PNG and WebP only.",
                     },
                 ),
                 "save_with_metadata": (
@@ -142,148 +310,184 @@ class SaveImageLM:
 
         return None
 
-    def format_metadata(self, metadata_dict):
-        """Format metadata in the requested format similar to userComment example"""
-        if not metadata_dict:
-            return ""
+    def _resolve_model_cache_entry(self, scanner_type: str, name: str):
+        """Resolve model hash, civitai metadata, and base_model from scanner cache.
+        Returns (hash_str, civitai_dict, base_model_str). All values are empty defaults when not found."""
+        scanner = ServiceRegistry.get_service_sync(scanner_type)
+        if scanner is None or not name:
+            return "", {}, ""
 
-        # Helper function to only add parameter if value is not None
-        def add_param_if_not_none(param_list, label, value):
-            if value is not None:
-                param_list.append(f"{label}: {value}")
+        entry = self._get_cached_model_by_name(scanner, name)
+        if entry is None:
+            basename = os.path.splitext(os.path.basename(name))[0]
+            hash_val = scanner.get_hash_by_filename(basename)
+            return (hash_val or "").lower(), {}, ""
+
+        hash_val = (entry.get("sha256") or "").lower()
+        civitai = entry.get("civitai") or {}
+        base_model = entry.get("base_model") or ""
+        return hash_val, civitai, base_model
+
+    @staticmethod
+    def _get_civitai_sampler_name(sampler_name: str, scheduler: str) -> str:
+        if sampler_name in CIVITAI_SAMPLER_MAP:
+            civitai_name = CIVITAI_SAMPLER_MAP[sampler_name]
+            if scheduler == "karras":
+                civitai_name += " Karras"
+            elif scheduler == "exponential":
+                civitai_name += " Exponential"
+            return civitai_name
+        else:
+            if scheduler and scheduler != "normal":
+                return f"{sampler_name}_{scheduler}"
+            return sampler_name
+
+    @staticmethod
+    def _build_air_string(base_model: str, model_type: str, model_id: int, version_id: int) -> str:
+        slug = BASE_MODEL_AIR_SLUG.get(base_model, "other")
+        type_lower = model_type.lower() if model_type else "other"
+        return f"urn:air:{slug}:{type_lower}:civitai:{model_id}@{version_id}"
+
+    def format_metadata(self, metadata_dict: dict) -> str:
+        """Format metadata as A1111-compatible parameters string with Hashes JSON and Civitai resources."""
+        if not metadata_dict: return ""
 
-        # Extract the prompt and negative prompt
         prompt = metadata_dict.get("prompt", "")
         negative_prompt = metadata_dict.get("negative_prompt", "")
-
-        # Extract loras from the prompt if present
+        steps = metadata_dict.get("steps")
+        cfg = metadata_dict.get("guidance")
+        if cfg is None:
+            cfg = metadata_dict.get("cfg_scale")
+        if cfg is None:
+            cfg = metadata_dict.get("cfg")
+        seed = metadata_dict.get("seed")
+        size = metadata_dict.get("size")
+        sampler = metadata_dict.get("sampler") or ""
+        scheduler = metadata_dict.get("scheduler") or "normal"
+        checkpoint = metadata_dict.get("checkpoint") or ""
         loras_text = metadata_dict.get("loras", "")
-        lora_hashes = {}
+        clip_skip = metadata_dict.get("clip_skip")
 
-        # If loras are found, add them on a new line after the prompt
+        # Parse LoRA entries from <lora:name:strength> format
+        lora_entries: list[tuple[str, float]] = []
         if loras_text:
-            prompt_with_loras = f"{prompt}\n{loras_text}"
+            for match in re.findall(r"<lora:([^:]+):([^>]+)>", loras_text):
+                lora_name, strength_str = match
+                try:
+                    strength = float(strength_str)
+                except (ValueError, TypeError):
+                    strength = 1.0
+                lora_entries.append((lora_name, strength))
 
-            # Extract lora names from the format <lora:name:strength>
-            lora_matches = re.findall(r"<lora:([^:]+):([^>]+)>", loras_text)
+        # Resolve checkpoint hash and Civitai data from local cache
+        ckpt_hash, ckpt_civitai, ckpt_base_model = "", {}, ""
+        ckpt_display_name = ""
+        if checkpoint:
+            ckpt_hash, ckpt_civitai, ckpt_base_model = self._resolve_model_cache_entry(
+                "checkpoint_scanner", checkpoint
+            )
+            ckpt_display_name = os.path.splitext(os.path.basename(checkpoint))[0]
 
-            # Get hash for each lora
-            for lora_name, strength in lora_matches:
-                hash_value = self.get_lora_hash(lora_name)
-                if hash_value:
-                    lora_hashes[lora_name] = hash_value
-        else:
-            prompt_with_loras = prompt
+        # Resolve LoRA hash and Civitai data from local cache
+        loras_data: list[dict] = []
+        for lora_name, strength in lora_entries:
+            lora_hash, lora_civitai, lora_base_model = self._resolve_model_cache_entry(
+                "lora_scanner", lora_name
+            )
+            loras_data.append({
+                "name": lora_name,
+                "strength": strength,
+                "hash": lora_hash,
+                "civitai": lora_civitai,
+                "base_model": lora_base_model,
+            })
 
-        # Format the first part (prompt and loras)
-        metadata_parts = [prompt_with_loras]
+        # Build Hashes JSON (A1111 / Civitai standard format)
+        hashes: dict[str, str] = {}
+        if ckpt_hash:
+            hashes["model"] = ckpt_hash[:10].upper()
+        for lora in loras_data:
+            if lora["hash"]:
+                hashes[f"LORA:{lora['name']}"] = lora["hash"][:10].upper()
 
-        # Add negative prompt
+        # Build Civitai resources JSON array
+        civitai_resources: list[dict] = []
+        if ckpt_civitai.get("id", 0) > 0:
+            ckpt_resource: dict = {}
+            ckpt_type = (ckpt_civitai.get("model") or {}).get("type", "Checkpoint")
+            model_id = ckpt_civitai.get("modelId", 0)
+            version_id = ckpt_civitai.get("id", 0)
+            if model_id and version_id:
+                ckpt_resource["air"] = self._build_air_string(
+                    ckpt_base_model, ckpt_type, int(model_id), int(version_id)
+                )
+            elif version_id:
+                ckpt_resource["modelVersionId"] = int(version_id)
+            if ckpt_civitai.get("name"):
+                ckpt_resource["versionName"] = ckpt_civitai["name"]
+            if ckpt_resource:
+                civitai_resources.append(ckpt_resource)
+
+        for lora in loras_data:
+            lora_civitai = lora["civitai"]
+            if not lora_civitai or lora_civitai.get("id", 0) <= 0:
+                continue
+            lora_resource: dict = {"weight": lora["strength"]}
+            lora_type = (lora_civitai.get("model") or {}).get("type", "LORA")
+            model_id = lora_civitai.get("modelId", 0)
+            version_id = lora_civitai.get("id", 0)
+            if model_id and version_id:
+                lora_resource["air"] = self._build_air_string(
+                    lora["base_model"], lora_type, int(model_id), int(version_id)
+                )
+            elif version_id:
+                lora_resource["modelVersionId"] = int(version_id)
+            if lora_civitai.get("name"):
+                lora_resource["versionName"] = lora_civitai["name"]
+            civitai_resources.append(lora_resource)
+
+        sampler_display = self._get_civitai_sampler_name(sampler, scheduler)
+
+        # Build output lines
+        lines = [prompt] if prompt else [""]
         if negative_prompt:
-            metadata_parts.append(f"Negative prompt: {negative_prompt}")
+            lines.append(f"Negative prompt: {negative_prompt}")
 
-        # Format the second part (generation parameters)
-        params = []
+        params: list[str] = []
+        if steps is not None:
+            params.append(f"Steps: {steps}")
+        if sampler_display:
+            params.append(f"Sampler: {sampler_display}")
+        if cfg is not None:
+            params.append(f"CFG scale: {cfg}")
+        if seed is not None:
+            params.append(f"Seed: {seed}")
+        if size:
+            params.append(f"Size: {size}")
+        if clip_skip:
+            try:
+                cs = int(clip_skip)
+                if cs != 0:
+                    params.append(f"Clip skip: {abs(cs)}")
+            except (ValueError, TypeError):
+                pass
+        additional_data = metadata_dict.get("additional_data", "")
+        if additional_data:
+            params.append(additional_data)
+        if ckpt_hash:
+            params.append(f"Model hash: {ckpt_hash[:10].upper()}")
+        if ckpt_display_name:
+            params.append(f"Model: {ckpt_display_name}")
+        if hashes:
+            params.append(f"Hashes: {json.dumps(hashes, separators=(',', ':'))}")
+        params.append("Version: ComfyUI")
+        if civitai_resources:
+            params.append(
+                f"Civitai resources: {json.dumps(civitai_resources, separators=(',', ':'))}"
+            )
 
-        # Add standard parameters in the correct order
-        if "steps" in metadata_dict:
-            add_param_if_not_none(params, "Steps", metadata_dict.get("steps"))
-
-        # Combine sampler and scheduler information
-        sampler_name = None
-        scheduler_name = None
-
-        if "sampler" in metadata_dict:
-            sampler = metadata_dict.get("sampler")
-            # Convert ComfyUI sampler names to user-friendly names
-            sampler_mapping = {
-                "euler": "Euler",
-                "euler_ancestral": "Euler a",
-                "dpm_2": "DPM2",
-                "dpm_2_ancestral": "DPM2 a",
-                "heun": "Heun",
-                "dpm_fast": "DPM fast",
-                "dpm_adaptive": "DPM adaptive",
-                "lms": "LMS",
-                "dpmpp_2s_ancestral": "DPM++ 2S a",
-                "dpmpp_sde": "DPM++ SDE",
-                "dpmpp_sde_gpu": "DPM++ SDE",
-                "dpmpp_2m": "DPM++ 2M",
-                "dpmpp_2m_sde": "DPM++ 2M SDE",
-                "dpmpp_2m_sde_gpu": "DPM++ 2M SDE",
-                "ddim": "DDIM",
-            }
-            sampler_name = sampler_mapping.get(sampler, sampler)
-
-        if "scheduler" in metadata_dict:
-            scheduler = metadata_dict.get("scheduler")
-            scheduler_mapping = {
-                "normal": "Simple",
-                "karras": "Karras",
-                "exponential": "Exponential",
-                "sgm_uniform": "SGM Uniform",
-                "sgm_quadratic": "SGM Quadratic",
-            }
-            scheduler_name = scheduler_mapping.get(scheduler, scheduler)
-
-        # Add combined sampler and scheduler information
-        if sampler_name:
-            if scheduler_name:
-                params.append(f"Sampler: {sampler_name} {scheduler_name}")
-            else:
-                params.append(f"Sampler: {sampler_name}")
-
-        # CFG scale (Use guidance if available, otherwise fall back to cfg_scale or cfg)
-        if "guidance" in metadata_dict:
-            add_param_if_not_none(params, "CFG scale", metadata_dict.get("guidance"))
-        elif "cfg_scale" in metadata_dict:
-            add_param_if_not_none(params, "CFG scale", metadata_dict.get("cfg_scale"))
-        elif "cfg" in metadata_dict:
-            add_param_if_not_none(params, "CFG scale", metadata_dict.get("cfg"))
-
-        # Seed
-        if "seed" in metadata_dict:
-            add_param_if_not_none(params, "Seed", metadata_dict.get("seed"))
-
-        # Size
-        if "size" in metadata_dict:
-            add_param_if_not_none(params, "Size", metadata_dict.get("size"))
-
-        # Model info
-        if "checkpoint" in metadata_dict:
-            # Ensure checkpoint is a string before processing
-            checkpoint = metadata_dict.get("checkpoint")
-            if checkpoint is not None:
-                # Get model hash
-                model_hash = self.get_checkpoint_hash(checkpoint)
-
-                # Extract basename without path
-                checkpoint_name = os.path.basename(checkpoint)
-                # Remove extension if present
-                checkpoint_name = os.path.splitext(checkpoint_name)[0]
-
-                # Add model hash if available
-                if model_hash:
-                    params.append(
-                        f"Model hash: {model_hash[:10]}, Model: {checkpoint_name}"
-                    )
-                else:
-                    params.append(f"Model: {checkpoint_name}")
-
-        # Add LoRA hashes if available
-        if lora_hashes:
-            lora_hash_parts = []
-            for lora_name, hash_value in lora_hashes.items():
-                lora_hash_parts.append(f"{lora_name}: {hash_value[:10]}")
-
-            if lora_hash_parts:
-                params.append(f'Lora hashes: "{", ".join(lora_hash_parts)}"')
-
-        # Combine all parameters with commas
-        metadata_parts.append(", ".join(params))
-
-        # Join all parts with a new line
-        return "\n".join(metadata_parts)
+        lines.append(", ".join(params))
+        return "\n".join(lines)
 
     # credit to nkchocoai
     # Add format_filename method to handle pattern substitution
@@ -298,7 +502,12 @@ class SaveImageLM:
             key = parts[0]
 
             if key == "seed" and "seed" in metadata_dict:
-                filename = filename.replace(segment, str(metadata_dict.get("seed", "")))
+                seed_value = metadata_dict.get("seed")
+                if seed_value is not None:
+                    filename = filename.replace(segment, str(seed_value))
+                else:
+                    # Fallback if seed was not captured by metadata collector
+                    filename = filename.replace(segment, "0")
             elif key == "width" and "size" in metadata_dict:
                 size = metadata_dict.get("size", "x")
                 w = size.split("x")[0] if isinstance(size, str) else size[0]
@@ -309,12 +518,14 @@ class SaveImageLM:
                 filename = filename.replace(segment, str(h))
             elif key == "pprompt" and "prompt" in metadata_dict:
                 prompt = metadata_dict.get("prompt", "").replace("\n", " ")
+                prompt = sanitize_folder_name(prompt)
                 if len(parts) >= 2:
                     length = int(parts[1])
                     prompt = prompt[:length]
                 filename = filename.replace(segment, prompt.strip())
             elif key == "nprompt" and "negative_prompt" in metadata_dict:
                 prompt = metadata_dict.get("negative_prompt", "").replace("\n", " ")
+                prompt = sanitize_folder_name(prompt)
                 if len(parts) >= 2:
                     length = int(parts[1])
                     prompt = prompt[:length]
@@ -328,6 +539,7 @@ class SaveImageLM:
                     model = "model_unavailable"
                 else:
                     model = os.path.splitext(os.path.basename(model_value))[0]
+                    model = sanitize_folder_name(model)
                 if len(parts) >= 2:
                     length = int(parts[1])
                     model = model[:length]
@@ -565,6 +777,8 @@ class SaveImageLM:
         extra_pnginfo=None,
         lossless_webp=True,
         quality=100,
+        webp_method=6,
+        jpeg_subsampling=0,
         embed_workflow=False,
         save_with_metadata=True,
         add_counter_to_filename=True,
@@ -600,7 +814,7 @@ class SaveImageLM:
             img = Image.fromarray(np.clip(img, 0, 255).astype(np.uint8))
 
             # Generate filename with counter if needed
-            base_filename = filename
+            base_filename = filename.replace("%batch_num%", str(i))
             if add_counter_to_filename:
                 # Use counter + i to ensure unique filenames for all images in batch
                 current_counter = counter + i
@@ -619,15 +833,14 @@ class SaveImageLM:
             elif file_format == "jpeg":
                 file = base_filename + ".jpg"
                 file_extension = ".jpg"
-                save_kwargs = {"quality": quality, "optimize": True}
+                save_kwargs = {"quality": quality, "optimize": True, "subsampling": jpeg_subsampling}
             elif file_format == "webp":
                 file = base_filename + ".webp"
                 file_extension = ".webp"
-                # Add optimization param to control performance
                 save_kwargs = {
                     "quality": quality,
                     "lossless": lossless_webp,
-                    "method": 0,
+                    "method": webp_method,
                 }
             else:
                 raise ValueError(f"Unsupported file format: {file_format}")
@@ -714,6 +927,8 @@ class SaveImageLM:
         extra_pnginfo=None,
         lossless_webp=True,
         quality=100,
+        webp_method=6,
+        jpeg_subsampling=0,
         embed_workflow=False,
         save_with_metadata=True,
         add_counter_to_filename=True,
@@ -743,6 +958,8 @@ class SaveImageLM:
             extra_pnginfo,
             lossless_webp,
             quality,
+            webp_method,
+            jpeg_subsampling,
             embed_workflow,
             save_with_metadata,
             add_counter_to_filename,

py/nodes/utils.py

@@ -36,6 +36,7 @@ any_type = AnyType("*")
 
 # Common methods extracted from lora_loader.py and lora_stacker.py
 import os
+import re
 import logging
 import copy
 import sys
@@ -69,6 +70,25 @@ def extract_lora_name(lora_path):
     return apply_lora_syntax_format(name_no_ext)
 
 
+def parse_lora_syntax(text: str) -> list[dict]:
+    """Parse <lora:name:strength> syntax from text input into a list of dicts.
+
+    Each entry contains: name, model_strength, clip_strength.
+    Supports both ``<lora:name:strength>`` and ``<lora:name:model_strength:clip_strength>``.
+    """
+    pattern = r"<lora:([^:>]+):([^:>]+)(?::([^:>]+))?>"
+    matches = re.findall(pattern, text, re.IGNORECASE)
+    loras = []
+    for match in matches:
+        model_strength = float(match[1])
+        loras.append({
+            "name": match[0],
+            "model_strength": model_strength,
+            "clip_strength": float(match[2]) if match[2] else model_strength,
+        })
+    return loras
+
+
 def get_loras_list(kwargs):
     """Helper to extract loras list from either old or new kwargs format"""
     if "loras" not in kwargs:

py/recipes/base.py

@@ -7,7 +7,7 @@ import re
 from typing import Dict, List, Any, Optional, Tuple
 from abc import ABC, abstractmethod
 from ..config import config
-from ..utils.constants import VALID_LORA_TYPES
+from ..utils.constants import VALID_LORA_TYPES, VALID_CHECKPOINT_SUB_TYPES
 from ..utils.civitai_utils import rewrite_preview_url
 
 logger = logging.getLogger(__name__)
@@ -58,9 +58,52 @@ class RecipeMetadataParser(ABC):
             civitai_info, error_msg = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
             
             if not civitai_info or error_msg == "Model not found":
-                # Model not found or deleted
-                lora_entry['isDeleted'] = True
-                lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
+                # CivitAI may fail to resolve a hash that is still being
+                # computed (known CivitAI issue). Before marking as deleted,
+                # try to reconcile with a local model that has the same
+                # filename and matching AutoV3 hash.
+                reconciled = False
+                file_name = lora_entry.get("file_name")
+                if file_name and recipe_scanner and hash_value:
+                    lora_scanner = getattr(recipe_scanner, "_lora_scanner", None)
+                    if lora_scanner:
+                        try:
+                            # Local import to avoid circular dependency:
+                            # base.py → file_utils → settings_manager → ...
+                            #  → recipe_scanner → enrichment → base.py
+                            from ..utils.file_utils import calculate_autov3  # fmt: skip
+                            cache = await lora_scanner.get_cached_data()
+                            for item in getattr(cache, "raw_data", []):
+                                if item.get("file_name") == file_name:
+                                    local_path = item.get("file_path")
+                                    if local_path and os.path.exists(local_path):
+                                        local_autov3 = calculate_autov3(local_path)
+                                        if local_autov3 and local_autov3 == hash_value:
+                                            lora_entry["existsLocally"] = True
+                                            lora_entry["localPath"] = local_path
+                                            lora_entry["hash"] = item.get("sha256", hash_value)
+                                            if "preview_url" in item:
+                                                lora_entry["thumbnailUrl"] = config.get_preview_static_url(item["preview_url"])
+                                            civ = item.get("civitai") or {}
+                                            if isinstance(civ, dict):
+                                                if civ.get("id") is not None:
+                                                    lora_entry["id"] = civ["id"]
+                                                if civ.get("modelId") is not None:
+                                                    lora_entry["modelId"] = civ["modelId"]
+                                                if civ.get("name"):
+                                                    lora_entry["version"] = civ["name"]
+                                                # model_name is the CivitAI model display
+                                                # name stored directly in the cache column.
+                                                cached_model_name = item.get("model_name")
+                                                if cached_model_name:
+                                                    lora_entry["name"] = cached_model_name
+                                            reconciled = True
+                                            break
+                        except Exception:
+                            pass
+                if not reconciled:
+                    lora_entry['isDeleted'] = True
+                    lora_entry['thumbnailUrl'] = '/loras_static/images/no-preview.png'
                 return lora_entry
                 
             # Get model type and validate
@@ -173,6 +216,20 @@ class RecipeMetadataParser(ABC):
                 checkpoint['isDeleted'] = True
                 return checkpoint
 
+            # Validate that the model type is actually a checkpoint.
+            # Unlike populate_lora_from_civitai which has this check,
+            # this function was missing type validation — allowing LoRA
+            # version data to be saved as the recipe's checkpoint when the
+            # wrong version ID was passed downstream (fixed in v2.7+).
+            model_type = civitai_data.get('model', {}).get('type', '').lower()
+            if model_type not in VALID_CHECKPOINT_SUB_TYPES:
+                logger.warning(
+                    f"Cannot populate checkpoint: model version {civitai_data.get('id')} "
+                    f"has type '{model_type}', expected one of {VALID_CHECKPOINT_SUB_TYPES}. "
+                    f"Skipping checkpoint enrichment."
+                )
+                return checkpoint
+
             if 'model' in civitai_data and 'name' in civitai_data['model']:
                 checkpoint['name'] = civitai_data['model']['name']
 

py/recipes/enrichment.py

@@ -190,27 +190,42 @@ class RecipeEnricher:
             existing_cp = recipe.get("checkpoint")
             if existing_cp is None:
                 existing_cp = {}
+
+            # Extract baseModel from raw civitai_info before populate_checkpoint_from_civitai
+            # (populate may reject non-checkpoint types and lose this data)
+            base_model_from_civitai: str = ""
+            if isinstance(civitai_info, dict):
+                base_model_from_civitai = civitai_info.get("baseModel", "") or ""
+            elif isinstance(civitai_info, tuple) and len(civitai_info) > 0 and isinstance(civitai_info[0], dict):
+                base_model_from_civitai = civitai_info[0].get("baseModel", "") or ""
+
             checkpoint_data = await RecipeMetadataParser.populate_checkpoint_from_civitai(existing_cp, civitai_info)
-            # 1. First, resolve base_model using full data before we format it away
+
+            # 1. Resolve base_model from checkpoint_data first, then fall back to raw civitai_info
             current_base_model = recipe.get("base_model")
-            resolved_base_model = checkpoint_data.get("baseModel")
+            resolved_base_model = checkpoint_data.get("baseModel") or base_model_from_civitai
             if resolved_base_model:
-                # Update if empty OR if it matches our generic prefix but is less specific
                 is_generic = not current_base_model or current_base_model.lower() in ["flux", "sdxl", "sd15"]
                 if is_generic and resolved_base_model != current_base_model:
                     recipe["base_model"] = resolved_base_model
-            
-            # 2. Format according to requirements: type, modelId, modelVersionId, modelName, modelVersionName
-            formatted_checkpoint = {
-                "type": "checkpoint",
-                "modelId": checkpoint_data.get("modelId"),
-                "modelVersionId": checkpoint_data.get("id") or checkpoint_data.get("modelVersionId"),
-                "modelName": checkpoint_data.get("name"), # In base.py, 'name' is populated from civitai_data['model']['name']
-                "modelVersionName": checkpoint_data.get("version") # In base.py, 'version' is populated from civitai_data['name']
-            }
-            # Remove None values
-            recipe["checkpoint"] = {k: v for k, v in formatted_checkpoint.items() if v is not None}
-            
+
+            # 2. Only format and save checkpoint if it has real data (not just type after type rejection)
+            has_checkpoint_data = any([
+                checkpoint_data.get("modelId"),
+                checkpoint_data.get("id") or checkpoint_data.get("modelVersionId"),
+                checkpoint_data.get("name"),
+                checkpoint_data.get("version"),
+            ])
+            if has_checkpoint_data:
+                formatted_checkpoint = {
+                    "type": "checkpoint",
+                    "modelId": checkpoint_data.get("modelId"),
+                    "modelVersionId": checkpoint_data.get("id") or checkpoint_data.get("modelVersionId"),
+                    "modelName": checkpoint_data.get("name"),
+                    "modelVersionName": checkpoint_data.get("version"),
+                }
+                recipe["checkpoint"] = {k: v for k, v in formatted_checkpoint.items() if v is not None}
+
             return True
         else:
             # Fallback to name extraction if we don't already have one

py/recipes/parsers/automatic.py

@@ -123,24 +123,39 @@ class AutomaticMetadataParser(RecipeMetadataParser):
                         if model_hash_from_hashes:
                             metadata["model_hash"] = model_hash_from_hashes
                     
-                    # Extract Lora hashes in alternative format
+                    # Extract Lora hashes in alternative format.
+                    # Run unconditionally (not just as fallback) so that
+                    # non-empty hashes from Lora hashes fill in the gaps left
+                    # by empty values in the Hashes JSON dict.  Some WebUI
+                    # builds write real hash values only to Lora hashes and
+                    # leave the Hashes JSON values empty.
                     lora_hashes_match = re.search(self.LORA_HASHES_REGEX, params_section)
-                    if not hashes_match and lora_hashes_match:
+                    if lora_hashes_match:
                         try:
                             lora_hashes_str = lora_hashes_match.group(1)
                             lora_hash_entries = lora_hashes_str.split(', ')
-                            
-                            # Initialize hashes dict if it doesn't exist
-                            if "hashes" not in metadata:
-                                metadata["hashes"] = {}
-                                
+
                             # Parse each lora hash entry (format: "name: hash")
                             for entry in lora_hash_entries:
                                 if ': ' in entry:
                                     lora_name, lora_hash = entry.split(': ', 1)
-                                    # Add as lora type in the same format as regular hashes
-                                    metadata["hashes"][f"lora:{lora_name}"] = lora_hash.strip()
-                            
+                                    lora_hash = lora_hash.strip()
+                                    if not lora_hash:
+                                        # Skip entries without a hash value
+                                        continue
+                                    # Initialize hashes dict if it doesn't exist
+                                    if "hashes" not in metadata:
+                                        metadata["hashes"] = {}
+                                    # Add as lora type in the same format as
+                                    # regular hashes.  Only override an
+                                    # existing entry if its value is empty
+                                    # (Lora hashes is the more reliable
+                                    # source when Hashes JSON has blanks).
+                                    key = f"lora:{lora_name}"
+                                    existing = metadata["hashes"].get(key, "")
+                                    if not existing:
+                                        metadata["hashes"][key] = lora_hash
+
                             # Remove lora hashes from params section
                             params_section = params_section.replace(lora_hashes_match.group(0), '')
                         except Exception as e:
@@ -362,6 +377,12 @@ class AutomaticMetadataParser(RecipeMetadataParser):
                         # Only process lora or hypernet types
                         if not hash_key.startswith(("lora:", "hypernet:")):
                             continue
+                        
+                        # Skip entries without a hash value — they can't be
+                        # resolved via CivitAI and would only produce a
+                        # useless "Deleted" entry in the recipe.
+                        if not lora_hash:
+                            continue
                             
                         lora_type, lora_name = hash_key.split(':', 1)
                         
@@ -387,11 +408,7 @@ class AutomaticMetadataParser(RecipeMetadataParser):
                         # Try to get info from Civitai
                         if metadata_provider:
                             try:
-                                if lora_hash:
-                                    # If we have hash, use it for lookup
-                                    civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
-                                else:
-                                    civitai_info = None
+                                civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
                                 
                                 populated_entry = await self.populate_lora_from_civitai(
                                     lora_entry, 

py/recipes/parsers/civitai_image.py

@@ -6,6 +6,7 @@ from typing import Dict, Any, Union
 from ..base import RecipeMetadataParser
 from ..constants import GEN_PARAM_KEYS
 from ...services.metadata_service import get_default_metadata_provider
+from ...config import config
 
 logger = logging.getLogger(__name__)
 
@@ -73,7 +74,8 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
         return False
 
     async def parse_metadata(  # type: ignore[override]
-        self, user_comment, recipe_scanner=None, civitai_client=None
+        self, user_comment, recipe_scanner=None, civitai_client=None,
+        local_cache: dict[str, Any] | None = None,
     ) -> Dict[str, Any]:
         """Parse metadata from Civitai image format
 
@@ -81,6 +83,8 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
             user_comment: The metadata from the image (dict)
             recipe_scanner: Optional recipe scanner service
             civitai_client: Optional Civitai API client (deprecated, use metadata_provider instead)
+            local_cache: Optional dict mapping sha256/autov3 hash → scanner cache item.
+                         When provided, matching models skip CivitAI API calls.
 
         Returns:
             Dict containing parsed recipe data
@@ -185,8 +189,77 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
             # Process standard resources array
             if "resources" in metadata and isinstance(metadata["resources"], list):
                 for resource in metadata["resources"]:
+                    resource_type = resource.get("type", "lora")
+
+                    # Track resources with type "model" — these are checkpoint models.
+                    # The resources array is the most reliable source for checkpoint
+                    # identification because it has an explicit type field and hash,
+                    # unlike modelVersionIds which is a flat list with no type info.
+                    if resource_type == "model":
+                        checkpoint_entry = {
+                            "id": 0,
+                            "modelId": 0,
+                            "name": resource.get("name", "Unknown Model"),
+                            "version": "",
+                            "type": resource.get("type", "model"),
+                            "existsLocally": False,
+                            "localPath": None,
+                            "file_name": resource.get("name", ""),
+                            "hash": resource.get("hash", "") or "",
+                            "thumbnailUrl": "/loras_static/images/no-preview.png",
+                            "baseModel": "",
+                            "size": 0,
+                            "downloadUrl": "",
+                            "isDeleted": False,
+                        }
+
+                        # Try to look up base model from the checkpoint hash
+                        cp_hash = checkpoint_entry.get("hash")
+                        if cp_hash and metadata_provider:
+                            local_cached = local_cache.get(cp_hash) if local_cache else None
+                            if local_cached:
+                                self._populate_entry_from_cache(
+                                    checkpoint_entry, local_cached
+                                )
+                                bm = checkpoint_entry.get("baseModel", "")
+                                if bm and not result["base_model"]:
+                                    result["base_model"] = bm
+                            else:
+                                try:
+                                    civitai_info = (
+                                        await metadata_provider.get_model_by_hash(
+                                            cp_hash
+                                        )
+                                    )
+                                    civitai_data, error_msg = (
+                                        (civitai_info, None)
+                                        if not isinstance(civitai_info, tuple)
+                                        else civitai_info
+                                    )
+                                    if civitai_data and error_msg != "Model not found":
+                                        if 'model' in civitai_data and 'name' in civitai_data['model']:
+                                            checkpoint_entry['name'] = civitai_data['model']['name']
+                                        checkpoint_entry['id'] = civitai_data.get('id', 0)
+                                        checkpoint_entry['modelId'] = civitai_data.get('modelId', 0)
+                                        if 'name' in civitai_data:
+                                            checkpoint_entry['version'] = civitai_data['name']
+                                        base_model = civitai_data.get('baseModel', '')
+                                        if base_model:
+                                            checkpoint_entry['baseModel'] = base_model
+                                            if not result['base_model']:
+                                                result['base_model'] = base_model
+                                except Exception as e:
+                                    logger.error(
+                                        f"Error fetching checkpoint info for hash "
+                                        f"{cp_hash}: {e}"
+                                    )
+
+                        if result["model"] is None:
+                            result["model"] = checkpoint_entry
+                        continue
+
                     # Modified to process resources without a type field as potential LoRAs
-                    if resource.get("type", "lora") == "lora":
+                    if resource_type == "lora":
                         lora_hash = resource.get("hash", "")
 
                         # Try to get hash from the hashes field if not present in resource
@@ -220,34 +293,45 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
                         }
 
                         # Try to get info from Civitai if hash is available
-                        if lora_entry["hash"] and metadata_provider:
-                            try:
-                                civitai_info = (
-                                    await metadata_provider.get_model_by_hash(lora_hash)
+                        if lora_hash and metadata_provider:
+                            local_cached = local_cache.get(lora_hash) if local_cache else None
+                            if local_cached:
+                                self._populate_entry_from_cache(
+                                    lora_entry, local_cached
                                 )
-
-                                populated_entry = await self.populate_lora_from_civitai(
-                                    lora_entry,
-                                    civitai_info,
-                                    recipe_scanner,
-                                    base_model_counts,
-                                    lora_hash,
-                                )
-
-                                if populated_entry is None:
-                                    continue  # Skip invalid LoRA types
-
-                                lora_entry = populated_entry
-
-                                # If we have a version ID from Civitai, track it for deduplication
-                                if "id" in lora_entry and lora_entry["id"]:
+                                # Track by version ID for deduplication
+                                if lora_entry.get("id"):
                                     added_loras[str(lora_entry["id"])] = len(
                                         result["loras"]
                                     )
-                            except Exception as e:
-                                logger.error(
-                                    f"Error fetching Civitai info for LoRA hash {lora_entry['hash']}: {e}"
-                                )
+                            else:
+                                try:
+                                    civitai_info = (
+                                        await metadata_provider.get_model_by_hash(lora_hash)
+                                    )
+
+                                    populated_entry = await self.populate_lora_from_civitai(
+                                        lora_entry,
+                                        civitai_info,
+                                        recipe_scanner,
+                                        base_model_counts,
+                                        lora_hash,
+                                    )
+
+                                    if populated_entry is None:
+                                        continue  # Skip invalid LoRA types
+
+                                    lora_entry = populated_entry
+
+                                    # If we have a version ID from Civitai, track it for deduplication
+                                    if "id" in lora_entry and lora_entry["id"]:
+                                        added_loras[str(lora_entry["id"])] = len(
+                                            result["loras"]
+                                        )
+                                except Exception as e:
+                                    logger.error(
+                                        f"Error fetching Civitai info for LoRA hash {lora_entry['hash']}: {e}"
+                                    )
 
                         # Track by hash if we have it
                         if lora_hash:
@@ -430,11 +514,21 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
 
                         result["loras"].append(lora_entry)
 
-            # Process modelVersionIds from Civitai image API
-            # These are model version IDs returned at root level when meta doesn't contain resources
-            if "modelVersionIds" in metadata and isinstance(
-                metadata["modelVersionIds"], list
+            # Process modelVersionIds from Civitai image API.
+            # These are version IDs returned at root level of the API response.
+            # When resources or civitaiResources are already present in metadata
+            # (which they are when ?withMeta=true is passed), those sections have
+            # complete hash/type information — modelVersionIds is a fallback for
+            # when meta is null and only the flat ID list is available. Skipping
+            # it here avoids duplicates: the same file hash often resolves to
+            # different version IDs via hash lookup (resources) vs the original
+            # version ID in modelVersionIds, and both paths would create entries.
+            if (
+                "modelVersionIds" in metadata
+                and isinstance(metadata["modelVersionIds"], list)
+                and not result.get("loras")
             ):
+
                 for version_id in metadata["modelVersionIds"]:
                     version_id_str = str(version_id)
 
@@ -442,6 +536,13 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
                     if version_id_str in added_loras:
                         continue
 
+                    # Skip if this version ID is already the recipe's checkpoint
+                    # (resolved earlier from embedded resources/Model hash,
+                    # avoiding a duplicate CivitAI API call).
+                    existing_model = result.get("model")
+                    if existing_model and str(existing_model.get("id")) == version_id_str:
+                        continue
+
                     # Initialize lora entry with version ID
                     lora_entry = {
                         "id": version_id,
@@ -475,9 +576,40 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
                             )
 
                             if populated_entry is None:
-                                continue  # Skip invalid LoRA types
+                                # Not a LoRA — try as checkpoint (only if we
+                                # don't already have one).  Reuses the same
+                                # civitai_info from the API call above so no
+                                # extra query is made.
+                                if result["model"] is None:
+                                    checkpoint_entry = {
+                                        "id": version_id,
+                                        "modelId": 0,
+                                        "name": "Unknown Model",
+                                        "version": "",
+                                        "type": "checkpoint",
+                                        "existsLocally": False,
+                                        "localPath": None,
+                                        "file_name": "",
+                                        "hash": "",
+                                        "thumbnailUrl": (
+                                            "/loras_static/images/no-preview.png"
+                                        ),
+                                        "baseModel": "",
+                                        "size": 0,
+                                        "downloadUrl": "",
+                                        "isDeleted": False,
+                                    }
+                                    cp_populated = await (
+                                        self.populate_checkpoint_from_civitai(
+                                            checkpoint_entry, civitai_info
+                                        )
+                                    )
+                                    if cp_populated.get("modelId"):
+                                        result["model"] = cp_populated
+                                continue  # Not a LoRA, don't add to loras
 
                             lora_entry = populated_entry
+
                         except Exception as e:
                             logger.error(
                                 f"Error fetching Civitai info for model version {version_id}: {e}"
@@ -625,3 +757,41 @@ class CivitaiApiMetadataParser(RecipeMetadataParser):
         except Exception as e:
             logger.error(f"Error parsing Civitai image metadata: {e}", exc_info=True)
             return {"error": str(e), "loras": []}
+
+    @staticmethod
+    def _populate_entry_from_cache(
+        entry: dict[str, Any],
+        cache_item: dict[str, Any],
+    ) -> None:
+        """Fill a lora/checkpoint entry from a scanner cache item.
+
+        Avoids CivitAI API calls for models that exist locally.
+        Mirrors the population logic in
+        ``RecipeMetadataParser.populate_lora_from_civitai()`` but operates
+        entirely on cached data.
+        """
+        civ = cache_item.get("civitai") or {}
+        if isinstance(civ, dict):
+            if civ.get("id") is not None:
+                entry["id"] = civ["id"]
+            if civ.get("modelId") is not None:
+                entry["modelId"] = civ["modelId"]
+            if civ.get("name"):
+                entry["version"] = civ["name"]
+            cached_name = cache_item.get("model_name")
+            if cached_name:
+                entry["name"] = cached_name
+        entry["existsLocally"] = True
+        local_path = cache_item.get("file_path")
+        if local_path:
+            entry["localPath"] = local_path
+        sha256 = cache_item.get("sha256")
+        if sha256:
+            entry["hash"] = sha256
+        if "preview_url" in cache_item:
+            entry["thumbnailUrl"] = config.get_preview_static_url(
+                cache_item["preview_url"]
+            )
+        base_model = cache_item.get("base_model", "")
+        if base_model:
+            entry["baseModel"] = base_model

py/routes/handlers/agent_handlers.py

@@ -0,0 +1,165 @@
+"""HTTP route handlers for agent skill endpoints.
+
+These handlers expose the :class:`AgentService` via HTTP, allowing the
+frontend to list available skills and execute them on selected models.
+Progress is reported via WebSocket broadcast.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+from typing import Any, Dict
+
+from aiohttp import web
+
+from ...services.agent import AgentService, AgentProgressReporter
+from ...services.llm_service import LLMNotConfiguredError
+
+logger = logging.getLogger(__name__)
+
+
+class AgentHandler:
+    """HTTP handler for agent skill operations."""
+
+    def __init__(self, agent_service: AgentService | None = None) -> None:
+        self._agent_service = agent_service
+
+    async def _ensure_service(self) -> AgentService:
+        if self._agent_service is None:
+            self._agent_service = await AgentService.get_instance()
+        return self._agent_service
+
+    # ------------------------------------------------------------------
+    # GET /api/lm/agent/skills
+    # ------------------------------------------------------------------
+
+    async def get_agent_skills(self, request: web.Request) -> web.Response:
+        """Return a list of available agent skills."""
+
+        service = await self._ensure_service()
+        skills = await service.list_skills()
+        return web.json_response({"skills": skills})
+
+    # ------------------------------------------------------------------
+    # POST /api/lm/agent/execute/{skill_name}
+    # ------------------------------------------------------------------
+
+    async def execute_agent_skill(self, request: web.Request) -> web.Response:
+        """Execute an agent skill on the provided model paths.
+
+        Request body::
+
+            {"model_paths": ["/path/to/model1.safetensors", ...], "options": {}}
+
+        Returns immediately with a task ID.  Execution runs in the
+        background; progress and completion are pushed via WebSocket
+        events of type ``agent_progress``.
+        """
+
+        skill_name = request.match_info.get("skill_name", "")
+        if not skill_name:
+            return web.json_response(
+                {"error": "Skill name is required"}, status=400
+            )
+
+        try:
+            body = await request.json()
+        except Exception:
+            return web.json_response(
+                {"error": "Invalid JSON body"}, status=400
+            )
+
+        model_paths = body.get("model_paths", [])
+        if not model_paths or not isinstance(model_paths, list):
+            return web.json_response(
+                {"error": "model_paths must be a non-empty array"},
+                status=400,
+            )
+
+        service = await self._ensure_service()
+
+        # Validate LLM configuration early for skills that need it
+        # (fail fast rather than after starting background work)
+        try:
+            from ...services.llm_service import LLMService
+
+            llm = await LLMService.get_instance()
+            if not llm.is_configured():
+                return web.json_response(
+                    {
+                        "error": "LLM provider is not configured. "
+                        "Enable it in Settings → AI Provider.",
+                    },
+                    status=400,
+                )
+        except Exception as exc:
+            logger.error("Failed to check LLM configuration: %s", exc)
+
+        # Launch execution in the background
+        progress_reporter = AgentProgressReporter()
+        logger.info(
+            "LLM enrichment '%s' starting for %d model(s)",
+            skill_name, len(model_paths),
+        )
+
+        async def _run() -> None:
+            try:
+                result = await service.execute_skill(
+                    skill_name=skill_name,
+                    input_data={"model_paths": model_paths},
+                    progress_callback=progress_reporter,
+                )
+                logger.info(
+                    "LLM enrichment '%s' finished: success=%s, summary='%s', errors=%s",
+                    skill_name, result.success, result.summary, result.errors,
+                )
+            except LLMNotConfiguredError as exc:
+                logger.warning("LLM enrichment '%s' not configured: %s", skill_name, exc)
+                await progress_reporter.on_progress(
+                    {
+                        "type": "agent_progress",
+                        "skill": skill_name,
+                        "status": "error",
+                        "error": str(exc),
+                    }
+                )
+            except Exception as exc:
+                logger.error("LLM enrichment '%s' failed: %s", skill_name, exc, exc_info=True)
+                await progress_reporter.on_progress(
+                    {
+                        "type": "agent_progress",
+                        "skill": skill_name,
+                        "status": "error",
+                        "error": str(exc),
+                    }
+                )
+
+        # Fire and forget — progress comes via WebSocket
+        asyncio.create_task(_run())
+
+        return web.json_response(
+            {
+                "status": "started",
+                "skill": skill_name,
+                "model_count": len(model_paths),
+            }
+        )
+
+    # ------------------------------------------------------------------
+    # POST /api/lm/agent/cancel
+    # ------------------------------------------------------------------
+
+    async def cancel_agent_skill(self, request: web.Request) -> web.Response:
+        """Cancel a running agent skill.
+
+        NOTE: Cancellation is a stub for now — the AgentService processes
+        models sequentially and does not yet support mid-execution
+        cancellation.  This endpoint exists for API completeness.
+        """
+
+        # TODO: implement cooperative cancellation in AgentService
+        return web.json_response(
+            {"status": "acknowledged", "note": "Cancellation not yet implemented"},
+            status=200,
+        )

py/routes/handlers/hf_handlers.py

@@ -0,0 +1,508 @@
+"""Handlers for Hugging Face model listing and download.
+
+Minimal MVP implementation — uses direct HTTP to the HF API for file
+listing and the project's existing aiohttp-based Downloader for
+downloading.  No huggingface_hub dependency required.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import re
+from typing import Any
+
+import aiohttp
+from aiohttp import web
+
+from ...config import config
+from ...services.downloader import (
+    DownloadProgress,
+    get_downloader,
+)
+from ...services.aria2_downloader import Aria2Downloader
+from ...services.settings_manager import get_settings_manager
+from ...services.service_registry import ServiceRegistry
+from ...services.websocket_manager import ws_manager
+from ...utils.constants import MODEL_FILE_EXTENSIONS
+from ...utils.metadata_manager import MetadataManager
+from ...utils.models import LoraMetadata, CheckpointMetadata, EmbeddingMetadata
+
+logger = logging.getLogger(__name__)
+
+_DEFAULT_MODEL_CLASS = LoraMetadata
+_DEFAULT_SCANNER_GETTER = "get_lora_scanner"
+
+# Shared aiohttp session for HF API calls (created on first use)
+_hf_api_session: aiohttp.ClientSession | None = None
+
+
+async def _get_hf_api_session() -> aiohttp.ClientSession:
+    """Get or create the shared aiohttp session for HF API calls."""
+    global _hf_api_session  # needed because we reassign the module-level name
+    if _hf_api_session is None or _hf_api_session.closed:
+        _hf_api_session = aiohttp.ClientSession(
+            headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
+            timeout=aiohttp.ClientTimeout(total=30),
+        )
+    return _hf_api_session
+
+
+async def close_hf_api_session() -> None:
+    """Close the shared HF API session, if it was ever created."""
+    global _hf_api_session
+    if _hf_api_session is not None and not _hf_api_session.closed:
+        await _hf_api_session.close()
+        _hf_api_session = None
+
+
+def _infer_model_type(model_root: str) -> tuple[Any, str]:
+    """Determine model class and scanner by matching ``model_root`` against the
+    configured root paths for each model type (from ``Config``).
+
+    The ``model_root`` value comes from the frontend's model-root dropdown,
+    which is populated from the current page's scanner roots.  By checking
+    which scanner's root list it belongs to, we avoid fragile heuristics
+    like substring-matching path names.
+    """
+    norm = os.path.normpath(model_root).replace(os.sep, "/")
+
+    # LoRA roots
+    for p in (config.loras_roots or []) + (config.extra_loras_roots or []):
+        if os.path.normpath(p).replace(os.sep, "/") == norm:
+            return LoraMetadata, "get_lora_scanner"
+
+    # Checkpoint / UNet roots
+    for p in (
+        (config.checkpoints_roots or [])
+        + (config.extra_checkpoints_roots or [])
+        + (config.unet_roots or [])
+        + (config.extra_unet_roots or [])
+    ):
+        if os.path.normpath(p).replace(os.sep, "/") == norm:
+            return CheckpointMetadata, "get_checkpoint_scanner"
+
+    # Embedding roots
+    for p in (config.embeddings_roots or []) + (config.extra_embeddings_roots or []):
+        if os.path.normpath(p).replace(os.sep, "/") == norm:
+            return EmbeddingMetadata, "get_embedding_scanner"
+
+    # Fallback — should not happen in normal use
+    logger.warning(
+        "Could not determine model type for root '%s'; defaulting to LoRA",
+        model_root,
+    )
+    return _DEFAULT_MODEL_CLASS, _DEFAULT_SCANNER_GETTER
+
+
+async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
+    """Create a proper .metadata.json and add the model to the scanner cache.
+
+    Uses ``MetadataManager.create_default_metadata()`` which computes the
+    SHA256 hash, extracts safetensors header metadata (base_model), and
+    produces a fully-populated ``LoraMetadata`` (or ``CheckpointMetadata`` /
+    ``EmbeddingMetadata``) object.  We then overlay HF-specific fields and
+    register the model in the in-memory scanner cache so it appears
+    immediately without a full filesystem walk.
+    """
+    try:
+        hf_url = f"https://huggingface.co/{repo}"
+        model_class, scanner_getter_name = _infer_model_type(model_root)
+
+        # 1. Create proper metadata (computes SHA256, reads safetensors headers)
+        metadata = await MetadataManager.create_default_metadata(
+            dest_path, model_class=model_class
+        )
+        if metadata is None:
+            logger.warning("create_default_metadata returned None for %s", dest_path)
+            return
+
+        # 2. Overlay HF-specific fields
+        metadata._unknown_fields["hf_url"] = hf_url
+        metadata.from_civitai = False  # HF models are not from CivitAI
+
+        metadata_dict = metadata.to_dict()
+        if "trainedWords" in metadata_dict and not metadata_dict["trainedWords"]:
+            del metadata_dict["trainedWords"]
+
+        # 3. Save metadata atomically
+        await MetadataManager.save_metadata(dest_path, metadata_dict)
+        logger.info("Saved HF metadata (with hf_url) for %s", dest_path)
+
+        # 4. Determine relative folder path for cache
+        #    model_root is an absolute path; dest_path is under it
+        folder = ""
+        if os.path.isabs(model_root) and dest_path.startswith(model_root):
+            rel = os.path.relpath(os.path.dirname(dest_path), model_root)
+            folder = rel.replace(os.sep, "/") if rel != "." else ""
+
+        # 5. Add to scanner cache (same as CivitAI's _execute_download does)
+        scanner_getter = getattr(ServiceRegistry, scanner_getter_name, None)
+        if scanner_getter is not None:
+            scanner = await scanner_getter()
+            if scanner is not None:
+                metadata_dict = metadata.to_dict()
+                metadata_dict["hf_url"] = hf_url
+                await scanner.add_model_to_cache(metadata_dict, folder)
+                logger.info("Added %s to scanner cache (folder=%s)", dest_path, folder)
+
+    except Exception as exc:
+        logger.warning("Failed to save HF metadata for %s: %s", dest_path, exc)
+
+
+def _find_matching_root(dest_dir: str) -> str | None:
+    """Walk up *dest_dir* to find which configured scanner root it belongs to."""
+    norm = os.path.normpath(dest_dir).replace(os.sep, "/")
+    all_roots = []
+    for root_list in (
+        config.loras_roots or [],
+        config.extra_loras_roots or [],
+        config.checkpoints_roots or [],
+        config.extra_checkpoints_roots or [],
+        config.unet_roots or [],
+        config.extra_unet_roots or [],
+        config.embeddings_roots or [],
+        config.extra_embeddings_roots or [],
+    ):
+        all_roots.extend([os.path.normpath(p).replace(os.sep, "/") for p in root_list])
+    # Find the longest matching prefix
+    match: str | None = None
+    for root in all_roots:
+        if norm.startswith(root):
+            if match is None or len(root) > len(match):
+                match = root
+    return match
+
+
+async def _add_to_scanner_cache(dest_path: str, metadata: dict[str, Any]) -> None:
+    model_dir = os.path.dirname(dest_path)
+    model_root = _find_matching_root(model_dir)
+    if not model_root:
+        raise ValueError(f"File path {dest_path} is not within any configured scanner root")
+    scanner_getter_name = _infer_model_type(model_root)[1]
+    scanner_getter = getattr(ServiceRegistry, scanner_getter_name, None)
+    if scanner_getter is None:
+        raise RuntimeError(f"Scanner getter '{scanner_getter_name}' not found in ServiceRegistry")
+    scanner = await scanner_getter()
+    if scanner is None:
+        raise RuntimeError(f"Scanner '{scanner_getter_name}' returned None")
+    await scanner.update_single_model_cache(dest_path, dest_path, metadata)
+
+
+class HfHandler:
+    """Handle Hugging Face model browsing and download."""
+
+    async def set_hf_url(self, request: web.Request) -> web.Response:
+        try:
+            payload: dict[str, Any] = await request.json()
+        except json.JSONDecodeError:
+            return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
+
+        file_path = (payload.get("file_path") or "").strip()
+        hf_url = (payload.get("hf_url") or "").strip()
+
+        if not file_path or not hf_url:
+            return web.json_response(
+                {"success": False, "error": "Missing required fields: 'file_path' and 'hf_url'"},
+                status=400,
+            )
+
+        m = re.match(r"^https?://huggingface\.co/([^/]+/[^/]+)/?$", hf_url)
+        if not m:
+            return web.json_response(
+                {
+                    "success": False,
+                    "error": "Invalid HuggingFace URL. Expected format: https://huggingface.co/user/repo",
+                },
+                status=400,
+            )
+
+        if not os.path.isfile(file_path):
+            return web.json_response(
+                {"success": False, "error": f"File not found: {file_path}"},
+                status=404,
+            )
+
+        model_root = _find_matching_root(os.path.dirname(file_path))
+        if not model_root:
+            return web.json_response(
+                {
+                    "success": False,
+                    "error": "File is not within any configured model directory. Cannot link to HuggingFace.",
+                },
+                status=400,
+            )
+
+        try:
+            existing = await MetadataManager.load_metadata_payload(file_path)
+            if existing.get("hf_url") == hf_url:
+                return web.json_response({
+                    "success": True,
+                    "message": "hf_url already set",
+                    "hf_url": hf_url,
+                })
+
+            existing["hf_url"] = hf_url
+            existing["from_civitai"] = False
+            await MetadataManager.save_metadata(file_path, existing)
+
+            await _add_to_scanner_cache(file_path, existing)
+
+            logger.info("Set hf_url=%s for %s", hf_url, file_path)
+            return web.json_response({
+                "success": True,
+                "message": f"hf_url set to {hf_url}",
+                "hf_url": hf_url,
+            })
+        except Exception as exc:
+            logger.error("Failed to set hf_url for %s: %s", file_path, exc)
+            return web.json_response(
+                {"success": False, "error": str(exc)},
+                status=500,
+            )
+
+    async def get_hf_repo_files(self, request: web.Request) -> web.Response:
+        """List model-weight files from a HF repo with real file sizes.
+
+        Uses the HF tree API endpoint which returns accurate file sizes
+        (including LFS-tracked files), unlike the model info endpoint.
+        """
+        repo = request.query.get("repo", "").strip()
+        if not repo or "/" not in repo:
+            return web.json_response(
+                {"error": "Missing or invalid 'repo' parameter (expected user/repo)"},
+                status=400,
+            )
+
+        url = f"https://huggingface.co/api/models/{repo}/tree/main"
+
+        try:
+            session = await _get_hf_api_session()
+            async with session.get(url) as resp:
+                if resp.status == 404:
+                    return web.json_response(
+                        {"error": f"Repo '{repo}' not found"}, status=404
+                    )
+                if resp.status != 200:
+                    text = await resp.text()
+                    return web.json_response(
+                        {"error": f"HF API error {resp.status}: {text[:200]}"},
+                        status=resp.status,
+                    )
+                tree: list[dict[str, Any]] = await resp.json()
+        except Exception as exc:
+            logger.error("Failed to fetch HF repo files: %s", exc)
+            return web.json_response({"error": str(exc)}, status=502)
+
+        files: list[dict[str, Any]] = []
+        for entry in tree:
+            path: str = entry.get("path", "")
+            ext = os.path.splitext(path)[1].lower()
+            if ext not in MODEL_FILE_EXTENSIONS:
+                continue
+            size = entry.get("size", 0) or 0
+            if size == 0 and "lfs" in entry:
+                size = entry["lfs"].get("size", 0) or 0
+            files.append({
+                "filename": path,
+                "size": size,
+            })
+
+        files.sort(key=lambda f: f["size"], reverse=True)
+        return web.json_response(files)
+
+    async def download_hf_model(self, request: web.Request) -> web.Response:
+        """Download a single file from Hugging Face into the model directory.
+
+        POST JSON body::
+
+            {
+              "repo": "dx8152/Flux2-Klein-9B-Consistency",
+              "filename": "Flux2-Klein-9B-consistency-V2.safetensors",
+              "revision": "main",
+              "model_root": "loras",
+              "relative_path": "",
+              "use_default_paths": false,
+              "download_id": "optional-batch-id"
+            }
+
+        If ``download_id`` is provided, real-time progress (bytes, speed,
+        percentage) is broadcast via the WebSocket progress system, matching
+        the CivitAI download experience.
+
+        Respects the ``download_backend`` setting (``aria2`` or ``default``).
+        """
+        try:
+            payload: dict[str, Any] = await request.json()
+        except json.JSONDecodeError:
+            return web.json_response({"error": "Invalid JSON"}, status=400)
+
+        repo = (payload.get("repo") or "").strip()
+        filename = (payload.get("filename") or "").strip()
+        revision = (payload.get("revision") or "main").strip()
+        model_root = (payload.get("model_root") or "").strip()
+        relative_path = (payload.get("relative_path") or "").strip()
+        use_default_paths = bool(payload.get("use_default_paths", False))
+        download_id: str | None = payload.get("download_id")
+
+        logger.info(
+            "download_hf_model: repo=%s file=%s root=%s download_id=%s",
+            repo, filename, model_root, download_id,
+        )
+
+        if not repo or not filename:
+            return web.json_response(
+                {"error": "Missing required fields: 'repo' and 'filename'"}, status=400
+            )
+
+        # Validate repo format — must be user/repo_name
+        if repo.count("/") != 1 or not re.match(r"^[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+$", repo):
+            return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
+        author, repo_name = repo.split("/", 1)
+        if ".." in (author, repo_name) or "." in (author, repo_name):
+            return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
+
+        # Validate filename — must not contain path traversal
+        if ".." in filename:
+            return web.json_response({"error": "Invalid filename"}, status=400)
+
+        # Validate relative_path — must not be absolute or escape base directory
+        if relative_path:
+            if os.path.isabs(relative_path):
+                return web.json_response({"error": "relative_path must not be absolute"}, status=400)
+            if ".." in relative_path.split("/") or "\\" in relative_path:
+                return web.json_response({"error": "Invalid relative_path"}, status=400)
+
+        # Use model_root directly as the base directory — same approach as
+        # CivitAI's download path (download_manager.py).  No realpath, no
+        # allowed-roots validation, no path-traversal check; those are
+        # unnecessary when the frontend sends the path from its own dropdown
+        # (populated from scanner roots).  Using the "business path" directly
+        # keeps dest_path consistent with scanner roots so that later folder
+        # derivation (in _save_hf_metadata) works correctly.
+        if os.path.isabs(model_root):
+            base_dir = os.path.normpath(model_root)
+        else:
+            base_dir = os.path.normpath(os.path.join(os.getcwd(), "models", model_root))
+
+        if use_default_paths:
+            target_dir = os.path.join(base_dir, "huggingface", author, repo_name)
+        elif relative_path:
+            target_dir = os.path.join(base_dir, relative_path)
+        else:
+            target_dir = base_dir
+
+        # Strip HF repo subdirectory — "diffusion_models/xxx.safetensors"
+        # is an HF repo convention, not meaningful for local storage.
+        file_base = os.path.basename(filename)
+
+        os.makedirs(target_dir, exist_ok=True)
+        dest_path = os.path.join(target_dir, file_base)
+
+        # Check if already exists (simple skip)
+        if os.path.exists(dest_path) and os.path.getsize(dest_path) > 0:
+            logger.info("download_hf_model: file already exists, skipping — %s", dest_path)
+            return web.json_response({
+                "success": True,
+                "message": f"File already exists: {dest_path}",
+                "path": dest_path,
+            })
+
+        # Build HF resolve URL
+        resolve_url = (
+            f"https://huggingface.co/{repo}/resolve/{revision}/{filename}"
+        )
+
+        # Set up progress callback if download_id is provided
+        progress_callback = None
+        if download_id:
+
+            async def _progress_callback(
+                progress: float | DownloadProgress,
+                snapshot: DownloadProgress | None = None,
+            ) -> None:
+                percent = 0.0
+                metrics = snapshot if isinstance(snapshot, DownloadProgress) else None
+
+                if isinstance(progress, DownloadProgress):
+                    percent = progress.percent_complete
+                    metrics = progress
+                elif isinstance(snapshot, DownloadProgress):
+                    percent = snapshot.percent_complete
+                else:
+                    percent = float(progress)
+
+                broadcast: dict[str, Any] = {
+                    "status": "progress",
+                    "progress": round(percent),
+                }
+                if metrics:
+                    broadcast["bytes_downloaded"] = metrics.bytes_downloaded
+                    broadcast["total_bytes"] = metrics.total_bytes
+                    broadcast["bytes_per_second"] = metrics.bytes_per_second
+
+                await ws_manager.broadcast_download_progress(download_id, broadcast)
+
+            progress_callback = _progress_callback
+
+        # Respect download backend setting (aria2 vs default)
+        download_backend = (
+            get_settings_manager().get("download_backend", "default")
+        )
+
+        if download_backend == "aria2":
+            aria2 = await Aria2Downloader.get_instance()
+            aid = download_id or f"hf_{repo}_{filename}"
+            try:
+                hf_success, hf_result = await aria2.download_file(
+                    url=resolve_url,
+                    save_path=dest_path,
+                    download_id=aid,
+                    progress_callback=progress_callback,
+                )
+                if hf_success:
+                    await _save_hf_metadata(dest_path, repo, model_root)
+                    return web.json_response({
+                        "success": True,
+                        "message": f"Downloaded to {dest_path}",
+                        "path": dest_path,
+                    })
+                else:
+                    return web.json_response(
+                        {"success": False, "error": hf_result or "aria2 download failed"},
+                        status=500,
+                    )
+            except Exception as exc:
+                logger.error("HF download (aria2) failed: %s", exc)
+                return web.json_response(
+                    {"success": False, "error": str(exc)}, status=500
+                )
+
+        # Default: use built-in aiohttp Downloader
+        downloader = await get_downloader()
+        try:
+            success, result = await downloader.download_file(
+                url=resolve_url,
+                save_path=dest_path,
+                use_auth=False,
+                allow_resume=True,
+                progress_callback=progress_callback,
+            )
+            if success:
+                await _save_hf_metadata(dest_path, repo, model_root)
+                return web.json_response({
+                    "success": True,
+                    "message": f"Downloaded to {result}",
+                    "path": result,
+                })
+            else:
+                return web.json_response(
+                    {"success": False, "error": result or "Download failed"},
+                    status=500,
+                )
+        except Exception as exc:
+            logger.error("HF download failed: %s", exc)
+            return web.json_response(
+                {"success": False, "error": str(exc)}, status=500
+            )

py/routes/handlers/misc_handlers.py

@@ -38,6 +38,12 @@ from ...services.settings_manager import get_settings_manager
 from ...services.websocket_manager import ws_manager
 from ...services.downloader import get_downloader
 from ...services.errors import ResourceNotFoundError
+from ...services.llm_service import (
+    PROVIDER_PRESETS,
+    fetch_ollama_models,
+    get_all_provider_models,
+    get_provider_model_ids,
+)
 from ...services.cache_health_monitor import CacheHealthMonitor, CacheHealthStatus
 from ...utils.models import BaseModelMetadata
 from ...utils.constants import (
@@ -48,8 +54,13 @@ from ...utils.constants import (
     SUPPORTED_MEDIA_EXTENSIONS,
     VALID_LORA_TYPES,
 )
+from .hf_handlers import HfHandler
+from .agent_handlers import AgentHandler
 from ...utils.civitai_utils import rewrite_preview_url
-from ...utils.example_images_paths import is_valid_example_images_root
+from ...utils.example_images_paths import (
+    find_non_compliant_items_in_example_images_root,
+    is_valid_example_images_root,
+)
 from ...utils.lora_metadata import extract_trained_words
 from ...utils.session_logging import get_standalone_session_log_snapshot
 from ...utils.usage_stats import UsageStats
@@ -411,9 +422,10 @@ class PromptServerProtocol(Protocol):
     """Subset of PromptServer used by the handlers."""
 
     instance: "PromptServerProtocol"
+    sockets: dict  # maps clientId (sid) → WebSocketResponse
 
     def send_sync(
-        self, event: str, payload: dict
+        self, event: str, payload: dict | None = None, sid: str | None = None
     ) -> None:  # pragma: no cover - protocol
         ...
 
@@ -468,89 +480,167 @@ class BackupServiceProtocol(Protocol):
 
 
 class NodeRegistry:
-    """Thread-safe registry for tracking LoRA nodes in active workflows."""
+    """Thread-safe registry for tracking LoRA nodes across ComfyUI tabs.
+
+    Each connected ComfyUI browser tab (identified by its ``sid`` / ``clientId``)
+    registers its own set of workflow nodes.  Queries merge all known tabs into
+    a single result so that the calling LM panel always sees *every* available
+    target node, regardless of which tab responded fastest.
+    """
 
     def __init__(self) -> None:
         self._lock = asyncio.Lock()
-        self._nodes: Dict[str, dict] = {}
-        self._registry_updated = asyncio.Event()
+        # sid → {unique_id → node_info}
+        self._tab_nodes: Dict[str, Dict[str, dict]] = {}
+        self._ready = asyncio.Event()
+        self._waiting_clients: set[str] = set()
+
+    @property
+    def pending_client_count(self) -> int:
+        """Number of clients that have not yet responded in the current refresh cycle."""
+        return len(self._waiting_clients)
+
+    # ------------------------------------------------------------------
+    # Helpers to build one node dict (extracted so it's reused for each tab)
+    # ------------------------------------------------------------------
+    @staticmethod
+    def _build_node_dict(node: dict) -> dict:
+        node_id = node["node_id"]
+        graph_id = str(node["graph_id"])
+        unique_id = f"{graph_id}:{node_id}"
+        node_type = node.get("type", "")
+        type_id = NODE_TYPES.get(node_type, 0)
+        bgcolor = node.get("bgcolor") or DEFAULT_NODE_COLOR
+
+        raw_capabilities = node.get("capabilities")
+        capabilities: dict = {}
+        if isinstance(raw_capabilities, dict):
+            capabilities = dict(raw_capabilities)
+
+        raw_widget_names: list | None = node.get("widget_names")
+        if not isinstance(raw_widget_names, list):
+            capability_widget_names = capabilities.get("widget_names")
+            raw_widget_names = (
+                capability_widget_names
+                if isinstance(capability_widget_names, list)
+                else None
+            )
+
+        widget_names: list[str] = []
+        if isinstance(raw_widget_names, list):
+            widget_names = [
+                str(widget_name)
+                for widget_name in raw_widget_names
+                if isinstance(widget_name, str) and widget_name
+            ]
+
+        if widget_names:
+            capabilities["widget_names"] = widget_names
+        else:
+            capabilities.pop("widget_names", None)
+
+        if "supports_lora" in capabilities:
+            capabilities["supports_lora"] = bool(capabilities["supports_lora"])
+
+        comfy_class = node.get("comfy_class")
+        if not isinstance(comfy_class, str) or not comfy_class:
+            comfy_class = node_type if isinstance(node_type, str) else None
+
+        return {
+            "id": node_id,
+            "graph_id": graph_id,
+            "graph_name": node.get("graph_name"),
+            "unique_id": unique_id,
+            "bgcolor": bgcolor,
+            "title": node.get("title"),
+            "type": type_id,
+            "type_name": node_type,
+            "comfy_class": comfy_class,
+            "capabilities": capabilities,
+            "widget_names": widget_names,
+            "mode": node.get("mode"),
+            "marker_role": node.get("marker_role"),
+        }
+
+    # ------------------------------------------------------------------
+    # Public API
+    # ------------------------------------------------------------------
+    async def register_nodes(self, sid: str, nodes: list[dict]) -> None:
+        """Register/replace the node list for a single ComfyUI tab (identified by *sid*)."""
+        tab_nodes: dict[str, dict] = {}
+        for node in nodes:
+            nd = self._build_node_dict(node)
+            tab_nodes[nd["unique_id"]] = nd
 
-    async def register_nodes(self, nodes: list[dict]) -> None:
         async with self._lock:
-            self._nodes.clear()
-            for node in nodes:
-                node_id = node["node_id"]
-                graph_id = str(node["graph_id"])
-                unique_id = f"{graph_id}:{node_id}"
-                node_type = node.get("type", "")
-                type_id = NODE_TYPES.get(node_type, 0)
-                bgcolor = node.get("bgcolor") or DEFAULT_NODE_COLOR
-                raw_capabilities = node.get("capabilities")
-                capabilities: dict = {}
-                if isinstance(raw_capabilities, dict):
-                    capabilities = dict(raw_capabilities)
+            prev_count = len(self._tab_nodes.get(sid, {}))
+            self._tab_nodes[sid] = tab_nodes
+            self._waiting_clients.discard(sid)
+            if not self._waiting_clients:
+                self._ready.set()
+            total_tabs = len(self._tab_nodes)
 
-                raw_widget_names: list | None = node.get("widget_names")
-                if not isinstance(raw_widget_names, list):
-                    capability_widget_names = capabilities.get("widget_names")
-                    raw_widget_names = (
-                        capability_widget_names
-                        if isinstance(capability_widget_names, list)
-                        else None
-                    )
+        if len(nodes) != prev_count or len(nodes) > 0:
+            logger.debug(
+                "[LM:Registry] stored %s nodes (was %s) for client %s (total tabs: %s)",
+                len(nodes), prev_count, sid, total_tabs,
+            )
 
-                widget_names: list[str] = []
-                if isinstance(raw_widget_names, list):
-                    widget_names = [
-                        str(widget_name)
-                        for widget_name in raw_widget_names
-                        if isinstance(widget_name, str) and widget_name
-                    ]
+    def prepare_for_refresh(self, active_sids: list[str]) -> None:
+        """Set the list of client IDs we expect to hear from during the next refresh cycle."""
+        self._ready.clear()
+        self._waiting_clients = set(active_sids)
 
-                if widget_names:
-                    capabilities["widget_names"] = widget_names
-                else:
-                    capabilities.pop("widget_names", None)
-
-                if "supports_lora" in capabilities:
-                    capabilities["supports_lora"] = bool(capabilities["supports_lora"])
-
-                comfy_class = node.get("comfy_class")
-                if not isinstance(comfy_class, str) or not comfy_class:
-                    comfy_class = node_type if isinstance(node_type, str) else None
-
-                self._nodes[unique_id] = {
-                    "id": node_id,
-                    "graph_id": graph_id,
-                    "graph_name": node.get("graph_name"),
-                    "unique_id": unique_id,
-                    "bgcolor": bgcolor,
-                    "title": node.get("title"),
-                    "type": type_id,
-                    "type_name": node_type,
-                    "comfy_class": comfy_class,
-                    "capabilities": capabilities,
-                    "widget_names": widget_names,
-                    "mode": node.get("mode"),
-                }
-            logger.debug("Registered %s nodes in registry", len(nodes))
-            self._registry_updated.set()
-
-    async def get_registry(self) -> dict:
-        async with self._lock:
-            return {
-                "nodes": dict(self._nodes),
-                "node_count": len(self._nodes),
-            }
-
-    async def wait_for_update(self, timeout: float = 1.0) -> bool:
-        self._registry_updated.clear()
+    async def wait_for_all(self, timeout: float = 2.0) -> bool:
+        """Block until every client in the current waiting set has responded
+        (or *timeout* seconds elapse).  Returns ``True`` if all responded."""
+        if not self._waiting_clients:
+            return True
         try:
-            await asyncio.wait_for(self._registry_updated.wait(), timeout=timeout)
+            await asyncio.wait_for(self._ready.wait(), timeout=timeout)
             return True
         except asyncio.TimeoutError:
             return False
 
+    async def get_merged_registry(self, active_sids: set[str] | None = None) -> dict:
+        """Return the union of all known tab nodes, pruning any tab that is no
+        longer connected."""
+        async with self._lock:
+            # Garbage-collect stale entries (disconnected tabs)
+            stale_sids = []
+            if active_sids is not None:
+                for sid in list(self._tab_nodes):
+                    if sid not in active_sids:
+                        stale_sids.append(sid)
+                        del self._tab_nodes[sid]
+            if stale_sids:
+                logger.debug(
+                    "[LM:Registry] GC pruned %s disconnected tabs: %s",
+                    len(stale_sids), stale_sids,
+                )
+
+            merged: dict[str, dict] = {}
+            tab_info: dict[str, dict] = {}
+            for sid, nodes in self._tab_nodes.items():
+                tab_info[sid] = {
+                    "node_count": len(nodes),
+                    "graph_names": list(
+                        {
+                            n.get("graph_name")
+                            for n in nodes.values()
+                            if n.get("graph_name")
+                        }
+                    ),
+                }
+                merged.update(nodes)
+
+            return {
+                "nodes": merged,
+                "node_count": len(merged),
+                "tab_count": len(self._tab_nodes),
+                "tabs": tab_info,
+            }
+
 
 class HealthCheckHandler:
     async def health_check(self, request: web.Request) -> web.Response:
@@ -1328,6 +1418,10 @@ class SettingsHandler:
             "folder_paths",
             "libraries",
             "active_library",
+            # Sensitive — never expose the actual value to the frontend;
+            # frontend receives a boolean instead (*_set).
+            "civitai_api_key",
+            "llm_api_key",
         }
     )
 
@@ -1382,6 +1476,11 @@ class SettingsHandler:
                     value = self._settings.get(key)
                     if value is not None:
                         response_data[key] = value
+            # Sensitive fields: only expose a boolean indicating whether set
+            raw_key = self._settings.get("civitai_api_key")
+            response_data["civitai_api_key_set"] = bool(raw_key)
+            raw_llm_key = self._settings.get("llm_api_key")
+            response_data["llm_api_key_set"] = bool(raw_llm_key)
             settings_file = getattr(self._settings, "settings_file", None)
             if settings_file:
                 response_data["settings_file"] = settings_file
@@ -1471,7 +1570,11 @@ class SettingsHandler:
                 else:
                     self._settings.set(key, value)
 
-                if key == "enable_metadata_archive_db":
+                if key in (
+                    "enable_metadata_archive_db",
+                    "enable_civarchive_api",
+                    "metadata_provider_order",
+                ):
                     await self._metadata_provider_updater()
 
                 if key in self._PROXY_KEYS:
@@ -1486,18 +1589,78 @@ class SettingsHandler:
             logger.error("Error updating settings: %s", exc, exc_info=True)
             return web.Response(status=500, text=str(exc))
 
+    async def get_llm_models(self, request: web.Request) -> web.Response:
+        """Return the model list for a provider.
+
+        For ``ollama`` the list is fetched live from the local Ollama API
+        (only models actually pulled locally are shown).  For all other
+        providers the opencode model catalog is used.
+
+        Query parameters:
+            provider (required): Internal provider id (``openai``, ``ollama``, etc.).
+
+        Returns:
+            ``{"success": true, "models": ["gpt-4o", ...]}``.
+        """
+        provider_id = request.query.get("provider", "").strip()
+        if not provider_id:
+            return web.json_response(
+                {"success": False, "error": "provider query parameter is required", "models": []},
+                status=400,
+            )
+
+        try:
+            if provider_id == "ollama":
+                api_base = request.query.get("api_base", "").strip() or self._settings.get("llm_api_base", "")
+                if not api_base:
+                    api_base = "http://localhost:11434/v1"
+                models = await fetch_ollama_models(api_base)
+            else:
+                models = await get_provider_model_ids(provider_id)
+            return web.json_response({"success": True, "models": models})
+        except Exception as exc:
+            logger.warning("get_llm_models failed for %s: %s", provider_id, exc)
+            return web.json_response(
+                {"success": False, "error": str(exc), "models": []},
+                status=500,
+            )
+
     def _validate_example_images_path(self, folder_path: str) -> str | None:
         if not os.path.exists(folder_path):
             return f"Path does not exist: {folder_path}"
         if not os.path.isdir(folder_path):
             return "Please set a dedicated folder for example images."
         if not self._is_dedicated_example_images_folder(folder_path):
+            offending = find_non_compliant_items_in_example_images_root(folder_path)
+            if offending:
+                items_str = ", ".join(repr(item) for item in offending[:5])
+                if len(offending) > 5:
+                    items_str += f" … and {len(offending) - 5} more"
+                return (
+                    f"The folder contains items that are not valid example image "
+                    f"folders: {items_str}. Please use a dedicated, empty folder "
+                    f"for example images to prevent accidental data loss."
+                )
             return "Please set a dedicated folder for example images."
         return None
 
     def _is_dedicated_example_images_folder(self, folder_path: str) -> bool:
         return is_valid_example_images_root(folder_path)
 
+    async def get_provider_models(self, request: web.Request) -> web.Response:
+        """Return the model catalog for all preset providers.
+
+        This endpoint is called asynchronously by the settings UI so that
+        page rendering never blocks on the remote model catalog fetch.
+        """
+        catalog_provider_ids = [p for p in PROVIDER_PRESETS if p != "custom"]
+        try:
+            provider_models = await get_all_provider_models(catalog_provider_ids)
+            return web.json_response({"success": True, "models": provider_models})
+        except Exception as exc:
+            logger.warning("Failed to fetch provider models: %s", exc)
+            return web.json_response({"success": False, "models": {}, "error": str(exc)})
+
 
 class UsageStatsHandler:
     def __init__(self, usage_stats_factory: UsageStatsFactory = UsageStats) -> None:
@@ -1625,6 +1788,124 @@ class LoraCodeHandler:
             logger.error("Failed to update lora code: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    async def get_update_lora_code(self, request: web.Request) -> web.Response:
+        """GET version of update_lora_code — reads parameters from query string.
+
+        Query params:
+          lora_code (required)  — the LoRA syntax to send
+          mode     (optional)   — "append" (default) or "replace"
+          node_id  (repeatable) — target node id(s), e.g. node_id=3&node_id=5
+          node_ids (optional)   — JSON-encoded array for complex references with graph_id:
+                                   [{"node_id":3,"graph_id":"g1"}, ...]
+        """
+        try:
+            node_ids_raw = request.query.get("node_ids")
+            node_id_list = request.query.getall("node_id", [])
+            lora_code = request.query.get("lora_code", "")
+            mode = request.query.get("mode", "append")
+
+            if not lora_code:
+                return web.json_response(
+                    {"success": False, "error": "Missing lora_code parameter"},
+                    status=400,
+                )
+
+            node_ids = None
+            if node_ids_raw:
+                try:
+                    node_ids = json.loads(node_ids_raw)
+                except (json.JSONDecodeError, TypeError):
+                    return web.json_response(
+                        {"success": False, "error": "node_ids must be a valid JSON array"},
+                        status=400,
+                    )
+                if not isinstance(node_ids, list) or not node_ids:
+                    return web.json_response(
+                        {"success": False, "error": "node_ids must be a non-empty JSON array"},
+                        status=400,
+                    )
+            elif node_id_list:
+                node_ids = node_id_list
+
+            results = []
+            if node_ids is None:
+                try:
+                    self._prompt_server.instance.send_sync(
+                        "lora_code_update",
+                        {"id": -1, "lora_code": lora_code, "mode": mode},
+                    )
+                    results.append({"node_id": "broadcast", "success": True})
+                except Exception as exc:  # pragma: no cover - defensive logging
+                    logger.error("Error broadcasting lora code: %s", exc)
+                    results.append(
+                        {"node_id": "broadcast", "success": False, "error": str(exc)}
+                    )
+            else:
+                for entry in node_ids:
+                    node_identifier = entry
+                    graph_identifier = None
+                    if isinstance(entry, dict):
+                        node_identifier = entry.get("node_id")
+                        graph_identifier = entry.get("graph_id")
+
+                    if node_identifier is None:
+                        results.append(
+                            {
+                                "node_id": node_identifier,
+                                "graph_id": graph_identifier,
+                                "success": False,
+                                "error": "Missing node_id parameter",
+                            }
+                        )
+                        continue
+
+                    try:
+                        parsed_node_id = int(node_identifier)
+                    except (TypeError, ValueError):
+                        parsed_node_id = node_identifier
+
+                    payload = {
+                        "id": parsed_node_id,
+                        "lora_code": lora_code,
+                        "mode": mode,
+                    }
+
+                    if graph_identifier is not None:
+                        payload["graph_id"] = str(graph_identifier)
+
+                    try:
+                        self._prompt_server.instance.send_sync(
+                            "lora_code_update",
+                            payload,
+                        )
+                        results.append(
+                            {
+                                "node_id": parsed_node_id,
+                                "graph_id": payload.get("graph_id"),
+                                "success": True,
+                            }
+                        )
+                    except Exception as exc:  # pragma: no cover - defensive logging
+                        logger.error(
+                            "Error sending lora code to node %s (graph %s): %s",
+                            parsed_node_id,
+                            graph_identifier,
+                            exc,
+                        )
+                        results.append(
+                            {
+                                "node_id": parsed_node_id,
+                                "graph_id": payload.get("graph_id"),
+                                "success": False,
+                                "error": str(exc),
+                            }
+                        )
+
+            return web.json_response({"success": True, "results": results})
+        except Exception as exc:  # pragma: no cover - defensive logging
+            logger.error("Failed to update lora code (GET): %s", exc, exc_info=True)
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
 
 class TrainedWordsHandler:
     async def get_trained_words(self, request: web.Request) -> web.Response:
@@ -2970,15 +3251,28 @@ class NodeRegistryHandler:
         self._node_registry = node_registry
         self._prompt_server = prompt_server
         self._standalone_mode = standalone_mode
+        self._refresh_lock = asyncio.Lock()
+        self._last_slow_path_ts: float = 0.0
 
     async def register_nodes(self, request: web.Request) -> web.Response:
         try:
             data = await request.json()
             nodes = data.get("nodes", [])
+            client_id = data.get("client_id")
             if not isinstance(nodes, list):
                 return web.json_response(
                     {"success": False, "error": "nodes must be a list"}, status=400
                 )
+
+            if not isinstance(client_id, str) or not client_id:
+                return web.json_response(
+                    {
+                        "success": False,
+                        "error": "Missing client_id parameter",
+                    },
+                    status=400,
+                )
+
             for index, node in enumerate(nodes):
                 if not isinstance(node, dict):
                     return web.json_response(
@@ -3005,7 +3299,12 @@ class NodeRegistryHandler:
                     )
                 graph_name = node.get("graph_name")
                 try:
-                    node["node_id"] = int(node_id)
+                    # Handle compound node IDs from expanded group subgraphs,
+                    # e.g. "252:0" → 0 (parent scope is already in graph_id)
+                    if isinstance(node_id, str) and ":" in node_id:
+                        node["node_id"] = int(node_id.rsplit(":", 1)[-1])
+                    else:
+                        node["node_id"] = int(node_id)
                 except (TypeError, ValueError):
                     return web.json_response(
                         {
@@ -3022,7 +3321,7 @@ class NodeRegistryHandler:
                 else:
                     node["graph_name"] = str(graph_name)
 
-            await self._node_registry.register_nodes(nodes)
+            await self._node_registry.register_nodes(client_id, nodes)
             return web.json_response(
                 {
                     "success": True,
@@ -3046,33 +3345,110 @@ class NodeRegistryHandler:
                     status=503,
                 )
 
-            try:
-                self._prompt_server.instance.send_sync("lora_registry_refresh", {})
-                logger.debug("Sent registry refresh request to frontend")
-            except Exception as exc:
-                logger.error("Failed to send registry refresh message: %s", exc)
-                return web.json_response(
-                    {
-                        "success": False,
-                        "error": "Communication Error",
-                        "message": f"Failed to communicate with ComfyUI frontend: {exc}",
-                    },
-                    status=500,
-                )
+            current_sids = set(self._prompt_server.instance.sockets.keys())
 
-            registry_updated = await self._node_registry.wait_for_update(timeout=1.0)
-            if not registry_updated:
-                logger.warning("Registry refresh timeout after 1 second")
+            # Fast path: if the frontend has already pushed node data (via
+            # afterConfigureGraph / graphChanged hooks), return it immediately
+            # without triggering a WebSocket round-trip.
+            registry_info = await self._node_registry.get_merged_registry(
+                active_sids=current_sids
+            )
+            if registry_info["tab_count"] > 0:
+                logger.debug(
+                    "[LM:Registry] fast path: %s nodes across %s tabs %s",
+                    registry_info["node_count"],
+                    registry_info["tab_count"],
+                    dict(registry_info.get("tabs", {})),
+                )
+                return web.json_response({"success": True, "data": registry_info})
+
+            # Slow path: registry is empty — trigger refresh via WebSocket.
+            # Serialize with an async lock so concurrent callers don't all
+            # trigger separate WS refresh cycles.  The second caller will
+            # re-check the fast path and (usually) find populated data.
+            async with self._refresh_lock:
+                # Re-check after acquiring the lock — another concurrent call
+                # may have populated the cache while we were waiting.
+                registry_info = await self._node_registry.get_merged_registry(
+                    active_sids=current_sids
+                )
+                if registry_info["tab_count"] > 0:
+                    logger.debug(
+                        "[LM:Registry] fast path after lock wait: %s nodes across %s tabs",
+                        registry_info["node_count"],
+                        registry_info["tab_count"],
+                    )
+                    return web.json_response({"success": True, "data": registry_info})
+
+                # Cooldown: if the slow path ran recently (< 2 s) and
+                # returned empty, skip another WS round-trip.
+                elapsed = time.monotonic() - self._last_slow_path_ts
+                if elapsed < 2.0:
+                    logger.debug(
+                        "[LM:Registry] slow path cooldown (%.1fs since last refresh), returning empty",
+                        elapsed,
+                    )
+                    return web.json_response(
+                        {
+                            "success": False,
+                            "error": "Empty Registry",
+                            "message": "No workflow nodes found — ensure ComfyUI is open and the extension is loaded.",
+                        },
+                        status=408,
+                    )
+
+                logger.debug(
+                    "[LM:Registry] slow path: cache empty, triggering WS refresh (%s connected tabs: %s)",
+                    len(current_sids), list(current_sids)[:5],
+                )
+                active_sids = list(current_sids)
+                self._node_registry.prepare_for_refresh(active_sids)
+
+                try:
+                    self._prompt_server.instance.send_sync("lora_registry_refresh", {})
+                    logger.debug(
+                        "Sent registry refresh request (expecting %s clients)", len(active_sids)
+                    )
+                except Exception as exc:
+                    logger.error("Failed to send registry refresh message: %s", exc)
+                    return web.json_response(
+                        {
+                            "success": False,
+                            "error": "Communication Error",
+                            "message": f"Failed to communicate with ComfyUI frontend: {exc}",
+                        },
+                        status=500,
+                    )
+
+                if not await self._node_registry.wait_for_all(timeout=0.5):
+                    logger.warning(
+                        "Registry refresh timeout after 0.5s (%s/%s clients responded)",
+                        len(active_sids) - self._node_registry.pending_client_count,
+                        len(active_sids),
+                    )
+
+                # Re-read current sockets after the wait: a tab may have connected
+                # while we were waiting, and we don't want to garbage-collect it.
+                current_sids = set(self._prompt_server.instance.sockets.keys())
+                registry_info = await self._node_registry.get_merged_registry(
+                    active_sids=current_sids
+                )
+                self._last_slow_path_ts = time.monotonic()
+
+            if registry_info["node_count"] == 0:
+                logger.debug(
+                    "[LM:Registry] refresh OK — %s connected tab(s) but 0 compatible nodes found",
+                    registry_info["tab_count"],
+                )
                 return web.json_response(
                     {
                         "success": False,
-                        "error": "Timeout Error",
-                        "message": "Registry refresh timeout - ComfyUI frontend may not be responsive",
+                        "error": "Empty Registry",
+                        "message": "No workflow nodes found — ensure ComfyUI is open and the extension is loaded.",
                     },
                     status=408,
                 )
 
-            registry_info = await self._node_registry.get_registry()
             return web.json_response({"success": True, "data": registry_info})
         except Exception as exc:  # pragma: no cover - defensive logging
             logger.error("Failed to get registry: %s", exc, exc_info=True)
@@ -3085,16 +3461,21 @@ class NodeRegistryHandler:
         try:
             data = await request.json()
             widget_name = data.get("widget_name")
+            action = data.get("action")
             value = data.get("value")
+            mode = data.get("mode", "replace")
             node_ids = data.get("node_ids")
 
-            if not isinstance(widget_name, str) or not widget_name:
+            if not action and (not isinstance(widget_name, str) or not widget_name):
                 return web.json_response(
-                    {"success": False, "error": "Missing widget_name parameter"},
+                    {
+                        "success": False,
+                        "error": "Missing parameter: provide either 'action' or 'widget_name'",
+                    },
                     status=400,
                 )
 
-            if not isinstance(value, str) or not value:
+            if value is None or (isinstance(value, str) and not value):
                 return web.json_response(
                     {"success": False, "error": "Missing value parameter"}, status=400
                 )
@@ -3129,11 +3510,15 @@ class NodeRegistryHandler:
                 except (TypeError, ValueError):
                     parsed_node_id = node_identifier
 
-                payload = {
+                payload: dict = {
                     "id": parsed_node_id,
-                    "widget_name": widget_name,
                     "value": value,
+                    "mode": mode,
                 }
+                if action:
+                    payload["action"] = action
+                if widget_name:
+                    payload["widget_name"] = widget_name
 
                 if graph_identifier is not None:
                     payload["graph_id"] = str(graph_identifier)
@@ -3168,6 +3553,130 @@ class NodeRegistryHandler:
             logger.error("Failed to update node widget: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    async def get_update_node_widget(self, request: web.Request) -> web.Response:
+        """GET version of update_node_widget — reads parameters from query string.
+
+        Query params:
+          widget_name  (optional)   — the widget name to update (required unless action is set)
+          action       (optional)   — alternative action, e.g. "inject_text" (required unless widget_name is set)
+          value        (required)   — the value to set
+          mode         (optional)   — "replace" (default) or "append"
+          node_id      (repeatable) — target node id(s), e.g. node_id=3&node_id=5
+          node_ids     (optional)   — JSON-encoded array for complex references:
+                                       [{"node_id":3,"graph_id":"g1"}, ...]
+        """
+        try:
+            widget_name = request.query.get("widget_name")
+            action = request.query.get("action")
+            value = request.query.get("value")
+            mode = request.query.get("mode", "replace")
+            node_ids_raw = request.query.get("node_ids")
+            node_id_list = request.query.getall("node_id", [])
+
+            if not action and (not isinstance(widget_name, str) or not widget_name):
+                return web.json_response(
+                    {
+                        "success": False,
+                        "error": "Missing parameter: provide either 'action' or 'widget_name'",
+                    },
+                    status=400,
+                )
+
+            if value is None or (isinstance(value, str) and not value):
+                return web.json_response(
+                    {"success": False, "error": "Missing value parameter"}, status=400
+                )
+
+            node_ids = None
+            if node_ids_raw:
+                try:
+                    node_ids = json.loads(node_ids_raw)
+                except (json.JSONDecodeError, TypeError):
+                    return web.json_response(
+                        {"success": False, "error": "node_ids must be a valid JSON array"},
+                        status=400,
+                    )
+                if not isinstance(node_ids, list) or not node_ids:
+                    return web.json_response(
+                        {"success": False, "error": "node_ids must be a non-empty JSON array"},
+                        status=400,
+                    )
+            elif node_id_list:
+                node_ids = node_id_list
+
+            if not isinstance(node_ids, list) or not node_ids:
+                return web.json_response(
+                    {"success": False, "error": "node_ids must be a non-empty list"},
+                    status=400,
+                )
+
+            results = []
+            for entry in node_ids:
+                node_identifier = entry
+                graph_identifier = None
+                if isinstance(entry, dict):
+                    node_identifier = entry.get("node_id")
+                    graph_identifier = entry.get("graph_id")
+
+                if node_identifier is None:
+                    results.append(
+                        {
+                            "node_id": node_identifier,
+                            "graph_id": graph_identifier,
+                            "success": False,
+                            "error": "Missing node_id parameter",
+                        }
+                    )
+                    continue
+
+                try:
+                    parsed_node_id = int(node_identifier)
+                except (TypeError, ValueError):
+                    parsed_node_id = node_identifier
+
+                payload: dict = {
+                    "id": parsed_node_id,
+                    "value": value,
+                    "mode": mode,
+                }
+                if action:
+                    payload["action"] = action
+                if widget_name:
+                    payload["widget_name"] = widget_name
+
+                if graph_identifier is not None:
+                    payload["graph_id"] = str(graph_identifier)
+
+                try:
+                    self._prompt_server.instance.send_sync("lm_widget_update", payload)
+                    results.append(
+                        {
+                            "node_id": parsed_node_id,
+                            "graph_id": payload.get("graph_id"),
+                            "success": True,
+                        }
+                    )
+                except Exception as exc:  # pragma: no cover - defensive logging
+                    logger.error(
+                        "Error sending widget update to node %s (graph %s): %s",
+                        parsed_node_id,
+                        graph_identifier,
+                        exc,
+                    )
+                    results.append(
+                        {
+                            "node_id": parsed_node_id,
+                            "graph_id": payload.get("graph_id"),
+                            "success": False,
+                            "error": str(exc),
+                        }
+                    )
+
+            return web.json_response({"success": True, "results": results})
+        except Exception as exc:  # pragma: no cover - defensive logging
+            logger.error("Failed to update node widget (GET): %s", exc, exc_info=True)
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
 
 class MiscHandlerSet:
     """Aggregate handlers into a lookup compatible with the registrar."""
@@ -3192,6 +3701,8 @@ class MiscHandlerSet:
         doctor: DoctorHandler,
         example_workflows: ExampleWorkflowsHandler,
         base_model: BaseModelHandlerSet,
+        hf_handler: HfHandler | None = None,
+        agent_handler: AgentHandler | None = None,
     ) -> None:
         self.health = health
         self.settings = settings
@@ -3210,6 +3721,8 @@ class MiscHandlerSet:
         self.doctor = doctor
         self.example_workflows = example_workflows
         self.base_model = base_model
+        self.hf_handler = hf_handler
+        self.agent_handler = agent_handler
 
     def to_route_mapping(
         self,
@@ -3225,13 +3738,17 @@ class MiscHandlerSet:
             "get_priority_tags": self.settings.get_priority_tags,
             "get_settings_libraries": self.settings.get_libraries,
             "activate_library": self.settings.activate_library,
+            "get_llm_models": self.settings.get_llm_models,
+            "get_provider_models": self.settings.get_provider_models,
             "update_usage_stats": self.usage_stats.update_usage_stats,
             "get_usage_stats": self.usage_stats.get_usage_stats,
             "update_lora_code": self.lora_code.update_lora_code,
+            "get_update_lora_code": self.lora_code.get_update_lora_code,
             "get_trained_words": self.trained_words.get_trained_words,
             "get_model_example_files": self.model_examples.get_model_example_files,
             "register_nodes": self.node_registry.register_nodes,
             "update_node_widget": self.node_registry.update_node_widget,
+            "get_update_node_widget": self.node_registry.get_update_node_widget,
             "get_registry": self.node_registry.get_registry,
             "check_model_exists": self.model_library.check_model_exists,
             "check_models_exist": self.model_library.check_models_exist,
@@ -3255,6 +3772,14 @@ class MiscHandlerSet:
             "get_supporters": self.supporters.get_supporters,
             "get_example_workflows": self.example_workflows.get_example_workflows,
             "get_example_workflow": self.example_workflows.get_example_workflow,
+            # Hugging Face handlers
+            "get_hf_repo_files": self.hf_handler.get_hf_repo_files,
+            "download_hf_model": self.hf_handler.download_hf_model,
+            "set_hf_url": self.hf_handler.set_hf_url,
+            # Agent skill handlers
+            "get_agent_skills": self.agent_handler.get_agent_skills,
+            "execute_agent_skill": self.agent_handler.execute_agent_skill,
+            "cancel_agent_skill": self.agent_handler.cancel_agent_skill,
             # Base model handlers
             "get_base_models": self.base_model.get_base_models,
             "refresh_base_models": self.base_model.refresh_base_models,

py/routes/handlers/model_handlers.py

@@ -37,6 +37,7 @@ from ...services.use_cases import (
 )
 from ...services.websocket_manager import WebSocketManager
 from ...services.websocket_progress_callback import WebSocketProgressCallback
+from ...services.download_queue_service import DownloadQueueService
 from ...services.errors import RateLimitError, ResourceNotFoundError
 from ...utils.civitai_utils import resolve_license_payload
 from ...utils.file_utils import calculate_sha256
@@ -153,6 +154,14 @@ class ModelPageView:
                 )
                 self._template_env._i18n_filter_added = True  # type: ignore[attr-defined]
 
+            from ...services.llm_service import PROVIDER_PRESETS
+
+            # Provider presets are embedded directly (local, no await needed).
+            # Provider model catalogs are fetched asynchronously by the
+            # frontend via GET /api/lm/llm/provider-models so page rendering
+            # never blocks on the remote model catalog (which can take up to
+            # 30s on cold cache).
+
             template_context = {
                 "is_initializing": is_initializing,
                 "settings": self._settings,
@@ -160,6 +169,8 @@ class ModelPageView:
                 "folders": [],
                 "t": self._server_i18n.get_translation,
                 "version": self._get_app_version(),
+                "provider_presets_json": json.dumps(PROVIDER_PRESETS),
+                "provider_models_json": "{}",
             }
 
             if not is_initializing:
@@ -202,11 +213,17 @@ class ModelListingHandler:
             result = await self._service.get_paginated_data(**params)
 
             format_start = time.perf_counter()
+            formatted_raw = [
+                await self._service.format_response(entry)
+                for entry in result["items"]
+            ]
+            # Filter out None entries returned for corrupted cache rows (issue #730).
+            # Note: "total" intentionally remains the pre-filter count to reflect
+            # the true number of models in the cache; corrupted entries are rare
+            # and adjusting total would cause pagination drift on every page.
+            formatted_items = [item for item in formatted_raw if item is not None]
             formatted_result = {
-                "items": [
-                    await self._service.format_response(item)
-                    for item in result["items"]
-                ],
+                "items": formatted_items,
                 "total": result["total"],
                 "page": result["page"],
                 "page_size": result["page_size"],
@@ -232,14 +249,20 @@ class ModelListingHandler:
         start_time = time.perf_counter()
         try:
             params = self._parse_common_params(request)
+            # group_by_model is meaningless for excluded view; strip it
+            params.pop("group_by_model", None)
             result = await self._service.get_excluded_paginated_data(**params)
 
             format_start = time.perf_counter()
+            formatted_raw = [
+                await self._service.format_response(entry)
+                for entry in result["items"]
+            ]
+            # Filter out None entries returned for corrupted cache rows (issue #730).
+            # "total" stays at the pre-filter count; see get_models for rationale.
+            formatted_items = [item for item in formatted_raw if item is not None]
             formatted_result = {
-                "items": [
-                    await self._service.format_response(item)
-                    for item in result["items"]
-                ],
+                "items": formatted_items,
                 "total": result["total"],
                 "page": result["page"],
                 "page_size": result["page_size"],
@@ -365,6 +388,21 @@ class ModelListingHandler:
             request.query.get("name_pattern_use_regex", "false").lower() == "true"
         )
 
+        # Group-by-model flag: deduplicate versions sharing the same civitai modelId
+        group_by_model = (
+            request.query.get("group_by_model", "false").lower() == "true"
+        )
+
+        # View-local-versions filter: show all local versions of a specific model
+        # Accepts either a CivitAI modelId (int) or a HF group key like "hf:user/repo"
+        civitai_model_id = request.query.get("civitai_model_id")
+        if civitai_model_id is not None:
+            try:
+                civitai_model_id = int(civitai_model_id)
+            except (TypeError, ValueError):
+                # Keep as string — could be an HF group key (e.g. "hf:user/repo")
+                pass
+
         return {
             "page": page,
             "page_size": page_size,
@@ -388,6 +426,8 @@ class ModelListingHandler:
             "name_pattern_include": name_pattern_include,
             "name_pattern_exclude": name_pattern_exclude,
             "name_pattern_use_regex": name_pattern_use_regex,
+            "group_by_model": group_by_model,
+            "civitai_model_id": civitai_model_id,
             **self._parse_specific_params(request),
         }
 
@@ -499,6 +539,7 @@ class ModelManagementHandler:
                     # Update model_data with new hash
                     model_data["sha256"] = sha256
                     model_data["hash_status"] = "completed"
+                    hash_status = "completed"
                 else:
                     return web.json_response(
                         {"success": False, "error": "No SHA256 hash found"}, status=400
@@ -506,6 +547,32 @@ class ModelManagementHandler:
 
             await MetadataManager.hydrate_model_data(model_data)
 
+            # hydrate_model_data replaces model_data with .metadata.json content,
+            # which may lack sha256. Restore from cache and persist the fix.
+            if not model_data.get("sha256"):
+                if sha256:
+                    model_data["sha256"] = sha256
+                    model_data["hash_status"] = model_data.get("hash_status", hash_status)
+                    data_to_save = model_data.copy()
+                    data_to_save.pop("folder", None)
+                    await MetadataManager.save_metadata(file_path, data_to_save)
+                else:
+                    sha256 = await calculate_sha256(file_path)
+                    if sha256:
+                        model_data["sha256"] = sha256.lower()
+                        model_data["hash_status"] = "completed"
+                        data_to_save = model_data.copy()
+                        data_to_save.pop("folder", None)
+                        await MetadataManager.save_metadata(file_path, data_to_save)
+                    else:
+                        return web.json_response(
+                            {
+                                "success": False,
+                                "error": "Failed to compute SHA256 hash for model",
+                            },
+                            status=500,
+                        )
+
             success, error = await self._metadata_sync.fetch_and_update_model(
                 sha256=model_data["sha256"],
                 file_path=file_path,
@@ -515,15 +582,25 @@ class ModelManagementHandler:
             if not success:
                 return web.json_response({"success": False, "error": error})
 
-            formatted_metadata = await self._service.format_response(model_data)
-            return web.json_response({"success": True, "metadata": formatted_metadata})
+            formatted = await self._service.format_response(model_data)
+            if formatted is None:
+                return web.json_response(
+                    {"success": False, "error": "Model entry is corrupted (missing file_path)"},
+                    status=500,
+                )
+            return web.json_response({"success": True, "metadata": formatted})
         except Exception as exc:
             if is_expected_offline_error(str(exc)):
                 return web.json_response(
                     {"success": False, "error": OFFLINE_FRIENDLY_MESSAGE},
                     status=503,
                 )
-            self._logger.error("Error fetching from CivitAI: %s", exc, exc_info=True)
+            self._logger.error(
+                "Error fetching from CivitAI for %s: %s",
+                locals().get("file_path", "unknown"),
+                exc,
+                exc_info=True,
+            )
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
     async def relink_civitai(self, request: web.Request) -> web.Response:
@@ -930,6 +1007,8 @@ class ModelQueryHandler:
             limit = int(request.query.get("limit", "20"))
             if limit < 0:
                 limit = 20
+            elif limit > 200:
+                limit = 20
             top_tags = await self._service.get_top_tags(limit)
             return web.json_response({"success": True, "tags": top_tags})
         except Exception as exc:
@@ -938,6 +1017,22 @@ class ModelQueryHandler:
                 {"success": False, "error": "Internal server error"}, status=500
             )
 
+    async def search_tags(self, request: web.Request) -> web.Response:
+        try:
+            query = request.query.get("q", "")
+            limit = int(request.query.get("limit", "20"))
+            if limit < 0:
+                limit = 20
+            elif limit > 200:
+                limit = 20
+            tags = await self._service.search_tags(query, limit)
+            return web.json_response({"success": True, "tags": tags})
+        except Exception as exc:
+            self._logger.error("Error searching tags: %s", exc, exc_info=True)
+            return web.json_response(
+                {"success": False, "error": "Internal server error"}, status=500
+            )
+
     async def get_base_models(self, request: web.Request) -> web.Response:
         try:
             limit = int(request.query.get("limit", "20"))
@@ -1073,10 +1168,12 @@ class ModelQueryHandler:
                 # Sort: originals first, copies last
                 sorted_models = self._sort_duplicate_group(filtered)
 
-                # Format response
+                # Format response, filtering out corrupted entries (issue #730)
                 group = {"hash": sha256, "models": []}
                 for model in sorted_models:
-                    group["models"].append(await self._service.format_response(model))
+                    formatted = await self._service.format_response(model)
+                    if formatted is not None:
+                        group["models"].append(formatted)
 
                 # Only include groups with 2+ models after filtering
                 if len(group["models"]) > 1:
@@ -1193,9 +1290,9 @@ class ModelQueryHandler:
                         (m for m in cache.raw_data if m["file_path"] == path), None
                     )
                     if model:
-                        group["models"].append(
-                            await self._service.format_response(model)
-                        )
+                        formatted = await self._service.format_response(model)
+                        if formatted is not None:
+                            group["models"].append(formatted)
                 hash_val = self._service.scanner.get_hash_by_filename(filename)
                 if hash_val:
                     main_path = self._service.get_path_by_hash(hash_val)
@@ -1205,9 +1302,9 @@ class ModelQueryHandler:
                             None,
                         )
                         if main_model:
-                            group["models"].insert(
-                                0, await self._service.format_response(main_model)
-                            )
+                            formatted = await self._service.format_response(main_model)
+                            if formatted is not None:
+                                group["models"].insert(0, formatted)
                 if group["models"]:
                     result.append(group)
             return web.json_response(
@@ -1230,9 +1327,13 @@ class ModelQueryHandler:
                     text=f"{self._service.model_type.capitalize()} file name is required",
                     status=400,
                 )
-            notes = await self._service.get_model_notes(model_name)
-            if notes is not None:
-                return web.json_response({"success": True, "notes": notes})
+            result = await self._service.get_model_notes(model_name)
+            if result is not None:
+                return web.json_response({
+                    "success": True,
+                    "notes": result["notes"],
+                    "file_path": result["file_path"],
+                })
             return web.json_response(
                 {
                     "success": False,
@@ -1268,9 +1369,28 @@ class ModelQueryHandler:
                 }
                 if include_license_flags:
                     model_data = await self._service.get_model_info_by_name(model_name)
-                    license_flags = (model_data or {}).get("license_flags")
-                    if license_flags is not None:
-                        response_payload["license_flags"] = int(license_flags)
+                    # Only return license_flags when real CivitAI model license
+                    # data exists. This mirrors ModelModal's guard
+                    # (modelData?.civitai?.model) so the preview tooltip never
+                    # shows misleading license icons for HF or other models
+                    # without actual license metadata.
+                    civitai_data = (model_data or {}).get("civitai") or {}
+                    has_license_data = (
+                        isinstance(civitai_data, dict)
+                        and isinstance(civitai_data.get("model"), dict)
+                    )
+                    if has_license_data:
+                        license_flags = (model_data or {}).get("license_flags")
+                        if license_flags is not None:
+                            response_payload["license_flags"] = int(license_flags)
+                    # Include the user's license icon style preference so the
+                    # ComfyUI tooltip can pick the right set without a separate
+                    # API call.
+                    try:
+                        settings = get_settings_manager()
+                        response_payload["use_new_license_icons"] = settings.get("use_new_license_icons", True)
+                    except Exception:
+                        pass
                 return web.json_response(response_payload)
             return web.json_response(
                 {
@@ -1472,6 +1592,21 @@ class ModelDownloadHandler:
             )
             return web.Response(status=500, text=str(exc))
 
+    async def skip_download_get(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            if not download_id:
+                return web.json_response(
+                    {"success": False, "error": "Download ID is required"}, status=400
+                )
+            result = await self._download_coordinator.skip_download(download_id)
+            return web.json_response(result)
+        except Exception as exc:
+            self._logger.error(
+                "Error skipping download via GET: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
     async def cancel_download_get(self, request: web.Request) -> web.Response:
         try:
             download_id = request.query.get("download_id")
@@ -1552,6 +1687,303 @@ class ModelDownloadHandler:
             )
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    # ------------------------------------------------------------------
+    # Download queue / history handlers
+    # ------------------------------------------------------------------
+
+    async def get_download_queue(self, request: web.Request) -> web.Response:
+        try:
+            service = await DownloadQueueService.get_instance()
+            queue = await service.get_queue()
+            stats = await service.get_stats()
+            return web.json_response({"success": True, "queue": queue, "stats": stats})
+        except Exception as exc:
+            self._logger.error(
+                "Error getting download queue: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def add_to_download_queue(self, request: web.Request) -> web.Response:
+        try:
+            import uuid
+
+            download_id = request.query.get("download_id") or str(uuid.uuid4())
+            model_id_str = request.query.get("model_id")
+            model_version_id_str = request.query.get("model_version_id")
+            model_name = request.query.get("model_name", "")
+            version_name = request.query.get("version_name", "")
+            thumbnail_url = request.query.get("thumbnail_url", "")
+            source = request.query.get("source")
+            file_params_json = request.query.get("file_params")
+
+            model_id = int(model_id_str) if model_id_str else None
+            model_version_id = int(model_version_id_str) if model_version_id_str else None
+            file_params = json.loads(file_params_json) if file_params_json else None
+
+            service = await DownloadQueueService.get_instance()
+            item = await service.add_to_queue(
+                download_id=download_id,
+                model_id=model_id,
+                model_version_id=model_version_id,
+                model_name=model_name,
+                version_name=version_name,
+                thumbnail_url=thumbnail_url,
+                source=source,
+                file_params=file_params,
+            )
+            return web.json_response({"success": True, "item": item})
+        except Exception as exc:
+            self._logger.error(
+                "Error adding to download queue: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def remove_from_download_queue(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            if not download_id:
+                return web.json_response(
+                    {"success": False, "error": "download_id is required"}, status=400
+                )
+
+            service = await DownloadQueueService.get_instance()
+            removed = await service.remove_from_queue(download_id)
+            return web.json_response({"success": removed})
+        except Exception as exc:
+            self._logger.error(
+                "Error removing from download queue: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def move_queue_item_to_top(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            if not download_id:
+                return web.json_response(
+                    {"success": False, "error": "download_id is required"}, status=400
+                )
+
+            service = await DownloadQueueService.get_instance()
+            moved = await service.move_to_top(download_id)
+            return web.json_response({"success": moved})
+        except Exception as exc:
+            self._logger.error(
+                "Error moving queue item to top: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def move_queue_item_to_end(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            if not download_id:
+                return web.json_response(
+                    {"success": False, "error": "download_id is required"}, status=400
+                )
+
+            service = await DownloadQueueService.get_instance()
+            moved = await service.move_to_end(download_id)
+            return web.json_response({"success": moved})
+        except Exception as exc:
+            self._logger.error(
+                "Error moving queue item to end: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def clear_download_queue(self, request: web.Request) -> web.Response:
+        try:
+            status_filter = request.query.get("status") or None
+            service = await DownloadQueueService.get_instance()
+            cleared = await service.clear_queue(status_filter=status_filter)
+            return web.json_response({"success": True, "cleared": cleared})
+        except Exception as exc:
+            self._logger.error(
+                "Error clearing download queue: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def get_download_history(self, request: web.Request) -> web.Response:
+        try:
+            limit = min(int(request.query.get("limit", "50")), 500)
+            offset = int(request.query.get("offset", "0"))
+            status_filter = request.query.get("status") or None
+            service = await DownloadQueueService.get_instance()
+            result = await service.get_history(
+                limit=limit, offset=offset, status_filter=status_filter
+            )
+            return web.json_response(
+                {
+                    "success": True,
+                    "items": result["items"],
+                    "total": result["total"],
+                    "limit": result["limit"],
+                    "offset": result["offset"],
+                }
+            )
+        except Exception as exc:
+            self._logger.error(
+                "Error getting download history: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def clear_download_history(self, request: web.Request) -> web.Response:
+        try:
+            status_filter = request.query.get("status") or None
+            service = await DownloadQueueService.get_instance()
+            cleared = await service.clear_history(status_filter=status_filter)
+            return web.json_response({"success": True, "cleared": cleared})
+        except Exception as exc:
+            self._logger.error(
+                "Error clearing download history: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def delete_download_history_item(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            id_str = request.query.get("id")
+            item_id = int(id_str) if id_str else None
+
+            if not download_id and not item_id:
+                return web.json_response(
+                    {"success": False, "error": "id or download_id is required"},
+                    status=400,
+                )
+
+            service = await DownloadQueueService.get_instance()
+            deleted = await service.delete_history_item(
+                id=item_id, download_id=download_id
+            )
+            return web.json_response({"success": deleted})
+        except Exception as exc:
+            self._logger.error(
+                "Error deleting download history item: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def retry_download_from_history(self, request: web.Request) -> web.Response:
+        try:
+            download_id = request.query.get("download_id")
+            id_str = request.query.get("id")
+            item_id = int(id_str) if id_str else None
+
+            if not download_id and not item_id:
+                return web.json_response(
+                    {"success": False, "error": "id or download_id is required"},
+                    status=400,
+                )
+
+            service = await DownloadQueueService.get_instance()
+            item = await service.retry_from_history(
+                item_id=item_id, download_id=download_id
+            )
+            if item is None:
+                return web.json_response(
+                    {"success": False, "error": "History item not found or not retryable"},
+                    status=404,
+                )
+            return web.json_response({"success": True, "item": item})
+        except Exception as exc:
+            self._logger.error(
+                "Error retrying download from history: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def retry_all_failed_downloads(self, request: web.Request) -> web.Response:
+        try:
+            service = await DownloadQueueService.get_instance()
+            retry_count = await service.retry_all_failed()
+            return web.json_response({"success": True, "retry_count": retry_count})
+        except Exception as exc:
+            self._logger.error(
+                "Error retrying all failed downloads: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def complete_download_in_queue(self, request: web.Request) -> web.Response:
+        """Atomically move a download from queue to history with terminal status."""
+        try:
+            download_id = request.query.get("download_id")
+            if not download_id:
+                return web.json_response(
+                    {"success": False, "error": "download_id is required"}, status=400
+                )
+            status = request.query.get("status", "completed")
+            error = request.query.get("error")
+            file_path = request.query.get("file_path")
+            try:
+                bytes_downloaded = int(request.query.get("bytes_downloaded", "0"))
+            except (TypeError, ValueError):
+                bytes_downloaded = 0
+            total_bytes_raw = request.query.get("total_bytes")
+            total_bytes = int(total_bytes_raw) if total_bytes_raw else None
+            completed_at_raw = request.query.get("completed_at")
+            completed_at = float(completed_at_raw) if completed_at_raw else None
+
+            service = await DownloadQueueService.get_instance()
+            item = await service.complete_download(
+                download_id=download_id,
+                status=status,
+                error=error,
+                file_path=file_path,
+                bytes_downloaded=bytes_downloaded,
+                total_bytes=total_bytes,
+                completed_at=completed_at,
+            )
+            if item is None:
+                return web.json_response(
+                    {"success": False, "error": "Download not found in queue"}, status=404
+                )
+            return web.json_response({"success": True, "item": item})
+        except Exception as exc:
+            self._logger.error(
+                "Error completing download: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def get_download_stats(self, request: web.Request) -> web.Response:
+        try:
+            service = await DownloadQueueService.get_instance()
+            stats = await service.get_stats()
+            return web.json_response({"success": True, "stats": stats})
+        except Exception as exc:
+            self._logger.error(
+                "Error getting download stats: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def update_download_queue_status(self, request: web.Request) -> web.Response:
+        """Update the status of a queue item (non-terminal transitions).
+
+        Supported transitions include ``queued → downloading``,
+        ``downloading → paused``, ``paused → downloading``, etc.
+        Terminal transitions (``completed``, ``failed``, ``canceled``)
+        should use ``complete_download_in_queue`` instead.
+        """
+        try:
+            download_id = request.query.get("download_id")
+            status = request.query.get("status")
+            if not download_id or not status:
+                return web.json_response(
+                    {
+                        "success": False,
+                        "error": "download_id and status are required",
+                    },
+                    status=400,
+                )
+            service = await DownloadQueueService.get_instance()
+            updated = await service.update_status(download_id, status)
+            if not updated:
+                return web.json_response(
+                    {"success": False, "error": "Download not found in queue"},
+                    status=404,
+                )
+            return web.json_response({"success": True})
+        except Exception as exc:
+            self._logger.error(
+                "Error updating download queue status: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
 
 class ModelCivitaiHandler:
     """CivitAI integration endpoints."""
@@ -1593,7 +2025,9 @@ class ModelCivitaiHandler:
             return web.json_response(result)
         except Exception as exc:
             self._logger.error(
-                "Error in fetch_all_civitai for %ss: %s", self._service.model_type, exc
+                "Error in fetch_all_civitai for %ss: %s",
+                self._service.model_type, exc,
+                exc_info=True,
             )
             return web.Response(text=str(exc), status=500)
 
@@ -1960,6 +2394,10 @@ class ModelUpdateHandler:
         if target_model_ids:
             target_model_ids = sorted(set(target_model_ids))
 
+        folder_path: Optional[str] = payload.get("folder_path")
+        if folder_path is not None and not isinstance(folder_path, str):
+            folder_path = None
+
         provider = await self._get_civitai_provider()
         if provider is None:
             return web.json_response(
@@ -1974,6 +2412,7 @@ class ModelUpdateHandler:
                 provider,
                 force_refresh=force_refresh,
                 target_model_ids=target_model_ids or None,
+                folder_path=folder_path,
             )
             if self._service.scanner.is_cancelled():
                 return web.json_response(
@@ -1996,10 +2435,21 @@ class ModelUpdateHandler:
             self._logger.error("Failed to refresh model updates: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+        hide_early_access = False
+        if self._settings is not None:
+            try:
+                hide_early_access = bool(
+                    self._settings.get("hide_early_access_updates", False)
+                )
+            except Exception:
+                pass
+
         serialized_records = []
         for record in records.values():
             has_update_fn = getattr(record, "has_update", None)
-            if callable(has_update_fn) and has_update_fn():
+            if callable(has_update_fn) and has_update_fn(
+                hide_early_access=hide_early_access
+            ):
                 serialized_records.append(self._serialize_record(record))
 
         return web.json_response(
@@ -2549,6 +2999,7 @@ class ModelHandlerSet:
             "bulk_delete_models": self.management.bulk_delete_models,
             "verify_duplicates": self.management.verify_duplicates,
             "get_top_tags": self.query.get_top_tags,
+            "search_tags": self.query.search_tags,
             "get_base_models": self.query.get_base_models,
             "get_model_types": self.query.get_model_types,
             "scan_models": self.query.scan_models,
@@ -2561,9 +3012,24 @@ class ModelHandlerSet:
             "download_model": self.download.download_model,
             "download_model_get": self.download.download_model_get,
             "cancel_download_get": self.download.cancel_download_get,
+            "skip_download_get": self.download.skip_download_get,
             "pause_download_get": self.download.pause_download_get,
             "resume_download_get": self.download.resume_download_get,
             "get_download_progress": self.download.get_download_progress,
+            "get_download_queue": self.download.get_download_queue,
+            "add_to_download_queue": self.download.add_to_download_queue,
+            "remove_from_download_queue": self.download.remove_from_download_queue,
+            "move_queue_item_to_top": self.download.move_queue_item_to_top,
+            "move_queue_item_to_end": self.download.move_queue_item_to_end,
+            "clear_download_queue": self.download.clear_download_queue,
+            "get_download_history": self.download.get_download_history,
+            "clear_download_history": self.download.clear_download_history,
+            "delete_download_history_item": self.download.delete_download_history_item,
+            "retry_download_from_history": self.download.retry_download_from_history,
+            "retry_all_failed_downloads": self.download.retry_all_failed_downloads,
+            "complete_download_in_queue": self.download.complete_download_in_queue,
+            "get_download_stats": self.download.get_download_stats,
+            "update_download_queue_status": self.download.update_download_queue_status,
             "get_civitai_versions": self.civitai.get_civitai_versions,
             "get_civitai_model_by_version": self.civitai.get_civitai_model_by_version,
             "get_civitai_model_by_hash": self.civitai.get_civitai_model_by_hash,

py/routes/handlers/preview_handlers.py

@@ -2,6 +2,7 @@
 
 from __future__ import annotations
 
+import asyncio
 import logging
 import mimetypes
 import urllib.parse
@@ -13,7 +14,7 @@ from ...config import config as global_config
 
 logger = logging.getLogger(__name__)
 
-_CHUNK_SIZE = 256 * 1024  # 256 KB
+_CHUNK_SIZE = 1024 * 1024  # 1 MB — balance between streaming iteration overhead and per-chunk memory
 
 # Video file extensions that bypass native sendfile on Windows
 # to avoid IOCP/ProactorEventLoop crashes during client disconnect.
@@ -53,18 +54,51 @@ class PreviewHandler:
 
         if not resolved.is_file():
             logger.debug("Preview file not found at %s", str(resolved))
+            asyncio.create_task(self._cleanup_stale_preview_url(normalized))
             raise web.HTTPNotFound(text="Preview file not found")
 
-        # Video files: stream manually to avoid Windows native sendfile crash.
-        # aiohttp's FileResponse uses _sendfile_native on Windows (IOCP-based),
-        # which breaks when the client disconnects mid-transfer — this happens
-        # constantly when users scroll through a gallery of animated previews.
-        suffix = resolved.suffix.lower()
-        if suffix in _VIDEO_EXTENSIONS:
-            return await self._stream_file(request, resolved)
+        # aiohttp's FileResponse handles range requests, content headers, and
+        # uses kernel sendfile (zero-copy DMA) on Linux/macOS. On Windows it
+        # uses IOCP-based _sendfile_native which can crash when the client
+        # disconnects mid-transfer during fast scrolling. The _stream_file()
+        # fallback is kept for a future compat toggle.
+        #
+        # Set explicit Cache-Control so the browser can cache video (and image)
+        # previews across VirtualScroller recycling cycles. Without this,
+        # Chrome does not cache 206 Partial Content responses for <video>
+        # elements, causing the same video to be re-downloaded on every scroll.
+        resp = web.FileResponse(path=resolved, chunk_size=_CHUNK_SIZE)
+        resp.headers["Cache-Control"] = "public, max-age=86400"
+        return resp
 
-        # aiohttp's FileResponse handles range requests and content headers for us.
-        return web.FileResponse(path=resolved, chunk_size=_CHUNK_SIZE)
+    async def _cleanup_stale_preview_url(self, normalized_preview_path: str) -> None:
+        """Fire-and-forget: clear stale preview_url from all model caches.
+
+        When a preview file is no longer on disk, remove its reference from
+        every cached entry so subsequent list API responses return an empty
+        ``preview_url``, letting the frontend show the no-preview placeholder.
+        """
+        try:
+            from ...services.service_registry import ServiceRegistry
+
+            for service_name in ("lora_scanner", "checkpoint_scanner", "embedding_scanner"):
+                scanner = ServiceRegistry.get_service_sync(service_name)
+                if scanner is None or not hasattr(scanner, "_cache"):
+                    continue
+                cache = getattr(scanner, "_cache", None)
+                if cache is None or not hasattr(cache, "clear_preview_by_path"):
+                    continue
+                cleared = await cache.clear_preview_by_path(normalized_preview_path)
+                if cleared and hasattr(scanner, "_persist_current_cache"):
+                    await scanner._persist_current_cache()
+                    logger.info(
+                        "Cleared stale preview_url for %d %s entries (%s)",
+                        cleared,
+                        service_name,
+                        normalized_preview_path,
+                    )
+        except Exception as exc:
+            logger.debug("Failed to clean up stale preview_url: %s", exc)
 
     async def _stream_file(
         self, request: web.Request, path: Path
@@ -83,6 +117,10 @@ class PreviewHandler:
         resp.content_type = content_type
         resp.content_length = file_size
 
+        # Allow browser caching: video previews rarely change during a session.
+        # The frontend already appends ?t={version} to bust cache on update.
+        resp.headers["Cache-Control"] = "public, max-age=86400"
+
         await resp.prepare(request)
 
         try:

py/routes/handlers/recipe_handlers.py

@@ -16,7 +16,7 @@ from aiohttp import web
 
 from ...config import config
 from ...services.server_i18n import server_i18n as default_server_i18n
-from ...services.settings_manager import SettingsManager
+from ...services.settings_manager import SettingsManager, get_settings_manager
 from ...services.recipes import (
     RecipeAnalysisService,
     RecipeDownloadError,
@@ -26,7 +26,13 @@ from ...services.recipes import (
     RecipeValidationError,
 )
 from ...services.metadata_service import get_default_metadata_provider
-from ...utils.civitai_utils import extract_civitai_image_id, rewrite_preview_url
+from ...utils.civitai_utils import (
+    build_civitai_image_page_url,
+    extract_civitai_image_id,
+    extract_civitai_image_id_from_cdn_url,
+    rewrite_preview_url,
+)
+from ...utils.constants import NSFW_LEVELS
 from ...utils.exif_utils import ExifUtils
 from ...recipes.merger import GenParamsMerger
 from ...recipes.enrichment import RecipeEnricher
@@ -66,6 +72,7 @@ class RecipeHandlerSet:
             "save_recipe": self.management.save_recipe,
             "delete_recipe": self.management.delete_recipe,
             "get_top_tags": self.query.get_top_tags,
+            "search_tags": self.query.search_tags,
             "get_base_models": self.query.get_base_models,
             "get_roots": self.query.get_roots,
             "get_folders": self.query.get_folders,
@@ -87,6 +94,7 @@ class RecipeHandlerSet:
             "repair_recipes": self.management.repair_recipes,
             "cancel_repair": self.management.cancel_repair,
             "repair_recipe": self.management.repair_recipe,
+            "repair_recipes_bulk": self.management.repair_recipes_bulk,
             "get_repair_progress": self.management.get_repair_progress,
             "start_batch_import": self.batch_import.start_batch_import,
             "get_batch_import_progress": self.batch_import.get_batch_import_progress,
@@ -95,6 +103,8 @@ class RecipeHandlerSet:
             "browse_directory": self.batch_import.browse_directory,
             "check_image_exists": self.management.check_image_exists,
             "import_from_url": self.management.import_from_url,
+            "create_from_example": self.management.create_from_example,
+            "reimport_recipe": self.management.reimport_recipe,
         }
 
 
@@ -308,12 +318,11 @@ class RecipeQueryHandler:
                 raise RuntimeError("Recipe scanner unavailable")
 
             limit = int(request.query.get("limit", "20"))
-            cache = await recipe_scanner.get_cached_data()
-
-            tag_counts: Dict[str, int] = {}
-            for recipe in getattr(cache, "raw_data", []):
-                for tag in recipe.get("tags", []) or []:
-                    tag_counts[tag] = tag_counts.get(tag, 0) + 1
+            if limit < 0:
+                limit = 20
+            elif limit > 200:
+                limit = 20
+            tag_counts = await self._get_recipe_tag_counts(recipe_scanner)
 
             sorted_tags = [
                 {"tag": tag, "count": count} for tag, count in tag_counts.items()
@@ -324,6 +333,55 @@ class RecipeQueryHandler:
             self._logger.error("Error retrieving top tags: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    async def search_tags(self, request: web.Request) -> web.Response:
+        try:
+            await self._ensure_dependencies_ready()
+            recipe_scanner = self._recipe_scanner_getter()
+            if recipe_scanner is None:
+                raise RuntimeError("Recipe scanner unavailable")
+
+            query = request.query.get("q", "")
+            limit = int(request.query.get("limit", "20"))
+            if limit < 0:
+                limit = 20
+            elif limit > 200:
+                limit = 20
+
+            tag_counts = await self._get_recipe_tag_counts(recipe_scanner)
+            normalized_query = (query or "").strip().lower()
+            if not normalized_query:
+                sorted_tags = [
+                    {"tag": tag, "count": count} for tag, count in tag_counts.items()
+                ]
+                sorted_tags.sort(key=lambda entry: entry["count"], reverse=True)
+                return web.json_response(
+                    {"success": True, "tags": sorted_tags[: (limit if limit > 0 else 20)]}
+                )
+
+            matched = [
+                {"tag": tag, "count": count}
+                for tag, count in tag_counts.items()
+                if normalized_query in tag.lower()
+            ]
+            matched.sort(key=lambda entry: entry["count"], reverse=True)
+            if limit == 0:
+                result = matched
+            else:
+                result = matched[:limit]
+            return web.json_response({"success": True, "tags": result})
+        except Exception as exc:
+            self._logger.error("Error searching recipe tags: %s", exc, exc_info=True)
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
+    async def _get_recipe_tag_counts(self, recipe_scanner) -> Dict[str, int]:
+        """Compute tag->count mapping from cached recipe data."""
+        cache = await recipe_scanner.get_cached_data()
+        tag_counts: Dict[str, int] = {}
+        for recipe in getattr(cache, "raw_data", []):
+            for tag in recipe.get("tags", []) or []:
+                tag_counts[tag] = tag_counts.get(tag, 0) + 1
+        return tag_counts
+
     async def get_base_models(self, request: web.Request) -> web.Response:
         try:
             await self._ensure_dependencies_ready()
@@ -460,7 +518,11 @@ class RecipeQueryHandler:
             if recipe_scanner is None:
                 raise RuntimeError("Recipe scanner unavailable")
 
-            self._logger.info("Manually triggering recipe cache rebuild")
+            full_rebuild = request.query.get("full_rebuild", "true").lower() == "true"
+            self._logger.info(
+                "Manually triggering recipe cache %s",
+                "full rebuild" if full_rebuild else "refresh",
+            )
             await recipe_scanner.get_cached_data(force_refresh=True)
             return web.json_response(
                 {"success": True, "message": "Recipe cache refreshed successfully"}
@@ -706,6 +768,69 @@ class RecipeManagementHandler:
             self._logger.error("Error cancelling recipe repair: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    async def repair_recipes_bulk(self, request: web.Request) -> web.Response:
+        """Bulk repair metadata for multiple recipes by their IDs.
+
+        Accepts a JSON body with a "recipe_ids" array and iterates
+        repair_recipe_by_id over each entry, collecting statistics.
+        """
+        try:
+            await self._ensure_dependencies_ready()
+            recipe_scanner = self._recipe_scanner_getter()
+            if recipe_scanner is None:
+                return web.json_response(
+                    {"success": False, "error": "Recipe scanner unavailable"},
+                    status=503,
+                )
+
+            data = await request.json()
+            recipe_ids = data.get("recipe_ids", [])
+            if not recipe_ids:
+                return web.json_response(
+                    {"success": False, "error": "recipe_ids are required"},
+                    status=400,
+                )
+
+            total = len(recipe_ids)
+            repaired = 0
+            skipped = 0
+            errors = 0
+            recipes = []
+
+            for recipe_id in recipe_ids:
+                try:
+                    result = await recipe_scanner.repair_recipe_by_id(recipe_id)
+                    if result.get("success"):
+                        repaired += result.get("repaired", 0)
+                        skipped += result.get("skipped", 0)
+                        if result.get("recipe"):
+                            recipes.append(result["recipe"])
+                    else:
+                        errors += 1
+                except RecipeNotFoundError:
+                    skipped += 1
+                except Exception as exc:
+                    self._logger.error(
+                        "Error repairing recipe %s: %s", recipe_id, exc
+                    )
+                    errors += 1
+
+            return web.json_response({
+                "success": True,
+                "total": total,
+                "repaired": repaired,
+                "skipped": skipped,
+                "errors": errors,
+                "recipes": recipes,
+            })
+        except Exception as exc:
+            self._logger.error(
+                "Error performing bulk repair: %s", exc, exc_info=True
+            )
+            return web.json_response(
+                {"success": False, "error": str(exc)}, status=500
+            )
+
     async def repair_recipe(self, request: web.Request) -> web.Response:
         try:
             await self._ensure_dependencies_ready()
@@ -725,6 +850,126 @@ class RecipeManagementHandler:
             self._logger.error("Error repairing single recipe: %s", exc, exc_info=True)
             return web.json_response({"success": False, "error": str(exc)}, status=500)
 
+    async def reimport_recipe(self, request: web.Request) -> web.Response:
+        """Delete a recipe and re-import it from its source URL.
+
+        This gives the recipe a fresh start — re-downloads the image from
+        CivitAI, re-parses EXIF metadata with the current parser, and
+        re-resolves LoRAs / checkpoint. User edits (title, tags, favorite)
+        are carried over from the old recipe.
+        """
+        try:
+            await self._ensure_dependencies_ready()
+            recipe_scanner = self._recipe_scanner_getter()
+            if recipe_scanner is None:
+                raise RuntimeError("Recipe scanner unavailable")
+
+            recipe_id = request.match_info["recipe_id"]
+            old_recipe = await recipe_scanner.get_recipe_by_id(recipe_id)
+            if not old_recipe:
+                raise RecipeNotFoundError(f"Recipe {recipe_id} not found")
+
+            source_path = old_recipe.get("source_path")
+            if not source_path:
+                return web.json_response(
+                    {
+                        "success": False,
+                        "error": (
+                            "Recipe has no source URL — cannot re-import. "
+                            "Use repair or manual import instead."
+                        ),
+                    },
+                    status=400,
+                )
+
+            user_edits: dict[str, Any] = {}
+            for key in ("title", "tags", "favorite", "preview_nsfw_level"):
+                if key in old_recipe and old_recipe[key] is not None:
+                    user_edits[key] = old_recipe[key]
+            if "tags" in user_edits and not isinstance(user_edits["tags"], list):
+                del user_edits["tags"]
+
+            old_file_path = old_recipe.get("file_path", "")
+            old_folder = os.path.dirname(old_file_path) if old_file_path else None
+
+            image_id = extract_civitai_image_id(source_path)
+            is_local_file = not image_id and os.path.isfile(source_path)
+
+            if not image_id and not is_local_file:
+                return web.json_response(
+                    {
+                        "success": False,
+                        "error": (
+                            "Recipe source is neither a valid CivitAI image URL "
+                            "nor an accessible local file. "
+                            "Use repair or manual import instead."
+                        ),
+                    },
+                    status=400,
+                )
+
+            if is_local_file:
+                return await self._do_reimport_from_local(
+                    source_path,
+                    recipe_scanner,
+                    recipe_id=recipe_id,
+                    target_dir=old_folder,
+                    user_edits=user_edits,
+                    old_title=old_recipe.get("title", ""),
+                )
+
+            async with self._import_semaphore:
+                import_response = await self._do_import_from_url(
+                    source_path,
+                    recipe_scanner,
+                    target_dir=old_folder,
+                )
+
+            await self._persistence_service.delete_recipe(
+                recipe_scanner=recipe_scanner, recipe_id=recipe_id
+            )
+
+            body_bytes = import_response.body
+            if not body_bytes:
+                raise RuntimeError("Re-import returned an empty response")
+            import_body = json.loads(body_bytes.decode())
+            new_recipe_id = import_body.get("recipe_id")
+
+            if new_recipe_id and user_edits:
+                try:
+                    await self._persistence_service.update_recipe(
+                        recipe_scanner=recipe_scanner,
+                        recipe_id=new_recipe_id,
+                        updates=user_edits,
+                    )
+                except Exception as exc:
+                    self._logger.warning(
+                        "Re-import succeeded but failed to carry over "
+                        "user edits for new recipe %s: %s",
+                        new_recipe_id,
+                        exc,
+                    )
+
+            return web.json_response(
+                {
+                    "success": True,
+                    "old_recipe_id": recipe_id,
+                    "recipe_id": new_recipe_id,
+                    "source_path": source_path,
+                }
+            )
+        except RecipeNotFoundError as exc:
+            return web.json_response({"success": False, "error": str(exc)}, status=404)
+        except RecipeValidationError as exc:
+            return web.json_response({"success": False, "error": str(exc)}, status=400)
+        except RecipeDownloadError as exc:
+            return web.json_response({"success": False, "error": str(exc)}, status=400)
+        except Exception as exc:
+            self._logger.error(
+                "Error reimporting recipe: %s", exc, exc_info=True
+            )
+            return web.json_response({"success": False, "error": str(exc)}, status=500)
+
     async def get_repair_progress(self, request: web.Request) -> web.Response:
         try:
             progress = self._ws_manager.get_recipe_repair_progress()
@@ -833,6 +1078,7 @@ class RecipeManagementHandler:
             extension,
             civitai_meta_raw,
             model_version_id,
+            _original_image_url,
         ) = await self._download_remote_media(image_url)
 
         # Extract embedded EXIF metadata (offloaded to thread pool in this call)
@@ -911,6 +1157,9 @@ class RecipeManagementHandler:
             civitai_model = civitai_parsed.get("model")
             if civitai_model and not metadata.get("checkpoint"):
                 metadata["checkpoint"] = civitai_model
+            civitai_base_model = civitai_parsed.get("base_model")
+            if civitai_base_model and not metadata.get("base_model"):
+                metadata["base_model"] = civitai_base_model
         elif parsed_embedded:
             parsed_loras = parsed_embedded.get("loras")
             if parsed_loras and not metadata.get("loras"):
@@ -918,6 +1167,15 @@ class RecipeManagementHandler:
             parsed_model = parsed_embedded.get("model")
             if parsed_model and not metadata.get("checkpoint"):
                 metadata["checkpoint"] = parsed_model
+            if parsed_embedded.get("base_model") and not metadata.get("base_model"):
+                metadata["base_model"] = parsed_embedded["base_model"]
+
+        # Extract preview_nsfw_level from the CivitAI API response
+        # (injected into civitai_meta_raw by _download_remote_media).
+        if isinstance(civitai_meta_raw, dict):
+            bl = civitai_meta_raw.get("browsingLevel")
+            if isinstance(bl, int) and bl > 0:
+                metadata["preview_nsfw_level"] = bl
 
         civitai_client = self._civitai_client_getter()
         await RecipeEnricher.enrich_recipe(
@@ -1240,7 +1498,9 @@ class RecipeManagementHandler:
             "exclude": False,
         }
 
-    async def _download_remote_media(self, image_url: str) -> tuple[bytes, str, Any, Any]:
+    async def _download_remote_media(
+        self, image_url: str
+    ) -> tuple[bytes, str, Any, Any, Optional[str]]:
         civitai_client = self._civitai_client_getter()
         downloader = await self._downloader_factory()
         temp_path = None
@@ -1293,11 +1553,18 @@ class RecipeManagementHandler:
                     image_info.get("meta") if civitai_image_id and image_info else None
                 )
                 if civitai_image_id and image_info:
+                    # modelVersionId (singular) — the primary version for this
+                    # image on CivitAI.  May be absent, or may *not* be the
+                    # checkpoint (e.g. when the image was generated with a LoRA
+                    # as the primary subject).  When absent, DO NOT fall back to
+                    # modelVersionIds[0] — that array mixes checkpoints, LoRAs,
+                    # and other model version IDs without ordering guarantees.
+                    # The downstream enrichment flow will find the real
+                    # checkpoint via meta.resources (type:"model" hash) or
+                    # meta.civitaiResources (type:"checkpoint" version ID), so
+                    # leaving model_ver_id as None is safe and avoids the bug
+                    # where a LoRA version ID was treated as the checkpoint.
                     model_ver_id = image_info.get("modelVersionId")
-                    if not model_ver_id:
-                        ids = image_info.get("modelVersionIds")
-                        if isinstance(ids, list) and ids:
-                            model_ver_id = ids[0]
 
                     # Inject root-level modelVersionIds into meta so downstream
                     # parsers (CivitaiApiMetadataParser) can discover ALL resources
@@ -1305,14 +1572,42 @@ class RecipeManagementHandler:
                     # CivitAI API returns modelVersionIds at the root level of
                     # the image response, NOT inside the meta object.
                     mvids = image_info.get("modelVersionIds")
-                    if mvids and isinstance(civitai_meta_raw, dict):
-                        civitai_meta_raw["modelVersionIds"] = mvids
+                    if mvids:
+                        if isinstance(civitai_meta_raw, dict):
+                            civitai_meta_raw["modelVersionIds"] = mvids
+                        else:
+                            # meta is null but modelVersionIds exists — create a
+                            # minimal dict so downstream parsers can discover
+                            # LoRAs and checkpoints from the API response.
+                            civitai_meta_raw = {"modelVersionIds": mvids}
+
+                    # Inject browsingLevel (canonical integer) so the recipe's
+                    # preview_nsfw_level can be set, enabling proper NSFW blur
+                    # of the preview image.  Fall back to nsfwLevel (string)
+                    # when browsingLevel is absent.
+                    if isinstance(civitai_meta_raw, dict):
+                        browsing_level = image_info.get("browsingLevel")
+                        nsfw_level_str = image_info.get("nsfwLevel")
+                        if isinstance(browsing_level, int) and browsing_level > 0:
+                            civitai_meta_raw["browsingLevel"] = browsing_level
+                        elif (
+                            isinstance(nsfw_level_str, str)
+                            and nsfw_level_str in NSFW_LEVELS
+                        ):
+                            civitai_meta_raw["browsingLevel"] = NSFW_LEVELS[
+                                nsfw_level_str
+                            ]
+
+                original_url = (
+                    image_info.get("url") if civitai_image_id and image_info else None
+                )
 
                 return (
                     file_obj.read(),
                     extension,
                     civitai_meta_raw,
                     model_ver_id,
+                    original_url,
                 )
         except RecipeDownloadError:
             raise
@@ -1382,15 +1677,8 @@ class RecipeManagementHandler:
 
             cache = await recipe_scanner.get_cached_data()
 
-            # Build lookup: image_id -> recipe_id from stored source_path
-            image_to_recipe = {}
-            for recipe in getattr(cache, "raw_data", []):
-                source = recipe.get("source_path")
-                if not source:
-                    continue
-                image_id = extract_civitai_image_id(source)
-                if image_id and image_id not in image_to_recipe:
-                    image_to_recipe[image_id] = recipe.get("id")
+            # Use precomputed image_id_map (built once at cache init)
+            image_to_recipe = getattr(cache, "image_id_map", {})
 
             results = {}
             for img_id in requested_ids:
@@ -1418,25 +1706,30 @@ class RecipeManagementHandler:
             if not image_url:
                 raise RecipeValidationError("Missing required field: image_url")
 
+            force = request.query.get("force", "false").lower() == "true"
+
             image_id = extract_civitai_image_id(image_url)
             if not image_id:
                 raise RecipeValidationError(
                     "Could not extract Civitai image ID from URL"
                 )
 
-            # Check for duplicate (fast, before acquiring semaphore)
-            cache = await recipe_scanner.get_cached_data()
-            for recipe in getattr(cache, "raw_data", []):
-                source = recipe.get("source_path")
-                if source:
-                    existing_id = extract_civitai_image_id(source)
-                    if existing_id == image_id:
-                        return web.json_response({
-                            "success": True,
-                            "recipe_id": recipe.get("id"),
-                            "name": recipe.get("title", ""),
-                            "already_exists": True,
-                        })
+            if not force:
+                cache = await recipe_scanner.get_cached_data()
+                image_to_recipe = getattr(cache, "image_id_map", {})
+                existing_recipe_id = image_to_recipe.get(image_id)
+                if existing_recipe_id:
+                    recipe_name = ""
+                    for recipe in getattr(cache, "raw_data", []):
+                        if str(recipe.get("id", "")) == existing_recipe_id:
+                            recipe_name = recipe.get("title", "") or ""
+                            break
+                    return web.json_response({
+                        "success": True,
+                        "recipe_id": existing_recipe_id,
+                        "name": recipe_name,
+                        "already_exists": True,
+                    })
 
             async with self._import_semaphore:
                 return await self._do_import_from_url(image_url, recipe_scanner)
@@ -1454,6 +1747,9 @@ class RecipeManagementHandler:
         self,
         image_url: str,
         recipe_scanner: Any,
+        *,
+        recipe_id: str | None = None,
+        target_dir: str | None = None,
     ) -> web.Response:
         image_id = extract_civitai_image_id(image_url)
         if not image_id:
@@ -1461,7 +1757,7 @@ class RecipeManagementHandler:
                 "Could not extract Civitai image ID from URL"
             )
 
-        image_bytes, extension, civitai_meta_raw, model_version_id = (
+        image_bytes, extension, civitai_meta_raw, model_version_id, original_image_url = (
             await self._download_remote_media(image_url)
         )
 
@@ -1499,6 +1795,51 @@ class RecipeManagementHandler:
                 "Failed to extract embedded metadata: %s", exc
             )
 
+        if not parsed_embedded and original_image_url:
+            self._logger.debug(
+                "Optimized image has no embedded metadata, "
+                "falling back to original: %s",
+                original_image_url,
+            )
+            try:
+                downloader = await self._downloader_factory()
+                with tempfile.NamedTemporaryFile(
+                    suffix=".png", delete=False
+                ) as tmp:
+                    orig_tmp_path = tmp.name
+                try:
+                    success, _ = await downloader.download_file(
+                        original_image_url, orig_tmp_path, use_auth=False
+                    )
+                    if success:
+                        raw_orig = await asyncio.to_thread(
+                            ExifUtils.extract_image_metadata, orig_tmp_path
+                        )
+                        if raw_orig:
+                            parser = (
+                                self._analysis_service._recipe_parser_factory.create_parser(
+                                    raw_orig
+                                )
+                            )
+                            if parser:
+                                parsed_embedded = await parser.parse_metadata(
+                                    raw_orig, recipe_scanner=recipe_scanner
+                                )
+                                if (
+                                    parsed_embedded
+                                    and "gen_params" in parsed_embedded
+                                ):
+                                    embedded_gen_params = parsed_embedded[
+                                        "gen_params"
+                                    ]
+                finally:
+                    if os.path.exists(orig_tmp_path):
+                        os.unlink(orig_tmp_path)
+            except Exception as exc:
+                self._logger.warning(
+                    "Failed to extract metadata from original image: %s", exc
+                )
+
         # Parse CivitAI API meta to discover all resources from modelVersionIds.
         # Run unconditionally — EXIF parsing succeeds for gen_params but misses
         # LoRAs (modelVersionIds is NOT in the image EXIF).
@@ -1535,6 +1876,13 @@ class RecipeManagementHandler:
             "source_path": image_url,
         }
 
+        # Extract preview_nsfw_level from the CivitAI API response
+        # (injected into civitai_meta_raw by _download_remote_media).
+        if isinstance(civitai_meta_raw, dict):
+            bl = civitai_meta_raw.get("browsingLevel")
+            if isinstance(bl, int) and bl > 0:
+                metadata["preview_nsfw_level"] = bl
+
         if civitai_parsed:
             civitai_loras = civitai_parsed.get("loras", [])
             if civitai_loras and not metadata.get("loras"):
@@ -1542,6 +1890,9 @@ class RecipeManagementHandler:
             civitai_model = civitai_parsed.get("model")
             if civitai_model and not metadata.get("checkpoint"):
                 metadata["checkpoint"] = civitai_model
+            civitai_base_model = civitai_parsed.get("base_model")
+            if civitai_base_model and not metadata.get("base_model"):
+                metadata["base_model"] = civitai_base_model
         elif parsed_embedded:
             parsed_loras = parsed_embedded.get("loras")
             if parsed_loras and not metadata.get("loras"):
@@ -1549,6 +1900,8 @@ class RecipeManagementHandler:
             parsed_model = parsed_embedded.get("model")
             if parsed_model and not metadata.get("checkpoint"):
                 metadata["checkpoint"] = parsed_model
+            if parsed_embedded.get("base_model") and not metadata.get("base_model"):
+                metadata["base_model"] = parsed_embedded["base_model"]
 
         civitai_client = self._civitai_client_getter()
         await RecipeEnricher.enrich_recipe(
@@ -1577,9 +1930,395 @@ class RecipeManagementHandler:
             tags=[],
             metadata=metadata,
             extension=extension,
+            recipe_id=recipe_id,
+            target_dir=target_dir,
         )
         return web.json_response(result.payload, status=result.status)
 
+    async def _do_reimport_from_local(
+        self,
+        file_path: str,
+        recipe_scanner: Any,
+        *,
+        recipe_id: str,
+        target_dir: str | None,
+        user_edits: dict[str, Any],
+        old_title: str,
+    ) -> web.Response:
+        """Re-import a recipe from a local image file.
+
+        Reads the original source file, re-parses its EXIF metadata, saves a
+        fresh recipe, then deletes the old one.
+        """
+        normalized = os.path.normpath(file_path)
+        if not os.path.isfile(normalized):
+            raise RecipeNotFoundError(
+                f"Source file no longer accessible: {normalized}"
+            )
+
+        with open(normalized, "rb") as fh:
+            image_bytes = fh.read()
+
+        extension = os.path.splitext(normalized)[1].lower() or ".png"
+
+        analysis_result = await self._analysis_service.analyze_local_image(
+            file_path=normalized,
+            recipe_scanner=recipe_scanner,
+        )
+        analysis_payload: dict[str, Any] = analysis_result.payload
+
+        gen_params = analysis_payload.get("gen_params") or {}
+        loras = analysis_payload.get("loras") or []
+        checkpoint = analysis_payload.get("checkpoint")
+        base_model = analysis_payload.get("base_model", "")
+
+        metadata: dict[str, Any] = {
+            "base_model": base_model,
+            "loras": loras,
+            "gen_params": gen_params,
+            "source_path": normalized,
+        }
+        if checkpoint:
+            metadata["checkpoint"] = checkpoint
+
+        prompt = (
+            gen_params.get("prompt")
+            or gen_params.get("positivePrompt")
+            or ""
+        )
+        name = " ".join(str(prompt).split()[:10]) if prompt else old_title
+
+        result = await self._persistence_service.save_recipe(
+            recipe_scanner=recipe_scanner,
+            image_bytes=image_bytes,
+            image_base64=analysis_payload.get("image_base64"),
+            name=name,
+            tags=[],
+            metadata=metadata,
+            extension=extension,
+            target_dir=target_dir,
+        )
+
+        await self._persistence_service.delete_recipe(
+            recipe_scanner=recipe_scanner, recipe_id=recipe_id
+        )
+
+        new_recipe_id = result.payload.get("recipe_id")
+        if new_recipe_id and user_edits:
+            try:
+                await self._persistence_service.update_recipe(
+                    recipe_scanner=recipe_scanner,
+                    recipe_id=new_recipe_id,
+                    updates=user_edits,
+                )
+            except Exception as exc:
+                self._logger.warning(
+                    "Re-import (local) succeeded but failed to carry over "
+                    "user edits for recipe %s: %s",
+                    new_recipe_id,
+                    exc,
+                )
+
+        return web.json_response(
+            {
+                "success": True,
+                "old_recipe_id": recipe_id,
+                "recipe_id": new_recipe_id,
+                "source_path": normalized,
+            }
+        )
+
+    async def create_from_example(self, request: web.Request) -> web.Response:
+        """Create a recipe from a model's example image using cached metadata.
+
+        Uses the image's meta data (already cached in .metadata.json from the
+        CivitAI model-versions API) to create a recipe without additional
+        CivitAI API calls.
+
+        If the image metadata doesn't contain any resources of the parent
+        model's type (LoRA-type or Checkpoint), the parent model is
+        auto-populated as a fallback.
+
+        Request body:
+            image_data (dict): The full image object from model-versions API
+                (includes meta, additionalResources, url, etc.)
+            model_hash (str): SHA256 hash of the parent model
+            model_name (str): Filename of the parent model
+            model_type (str): Page type (``"loras"``, ``"checkpoints"``, etc.)
+            local_image_path (str, optional): Local filesystem path to read
+                the image bytes for the recipe preview
+        """
+        try:
+            await self._ensure_dependencies_ready()
+            recipe_scanner = self._recipe_scanner_getter()
+            if recipe_scanner is None:
+                raise RuntimeError("Recipe scanner unavailable")
+
+            data = await request.json()
+            image_data = data.get("image_data")
+            model_hash = data.get("model_hash")
+            model_name = data.get("model_name")
+            model_type = data.get("model_type", "")
+
+            if not image_data or not model_hash or not model_name:
+                raise RecipeValidationError(
+                    "Missing required fields: image_data, model_hash, model_name"
+                )
+
+            # Merge nested meta into top level so the parser finds everything.
+            # CivitaiApiMetadataParser expects prompt, seed, resources, etc.
+            # at the top level or wrapped under a "meta" key.
+            inner_meta = image_data.get("meta") or {}
+            parsed_input = {**image_data, **inner_meta}
+            parsed_input.pop("meta", None)
+
+            # Build a local cache of {hash → cache_item} so the parser can
+            # skip CivitAI API calls for models that exist on disk.
+            local_cache: Dict[str, Dict[str, Any]] = {}
+            lora_scanner = getattr(recipe_scanner, "_lora_scanner", None)
+            if lora_scanner and model_hash:
+                try:
+                    parent_cache_data = await lora_scanner.get_cached_data()
+                    for item in getattr(parent_cache_data, "raw_data", []):
+                        if item.get("sha256", "").lower() == model_hash.lower():
+                            local_cache[model_hash.lower()] = item
+                            # Compute AutoV3 so the parser can also match on
+                            # that hash type (CivitAI metadata resources use
+                            # AutoV3).
+                            file_path = item.get("file_path")
+                            if file_path and os.path.exists(file_path):
+                                try:
+                                    from ...utils.file_utils import (
+                                        calculate_autov3,
+                                    )
+                                    autov3 = calculate_autov3(file_path)
+                                    if autov3:
+                                        local_cache[autov3.lower()] = item
+                                except Exception:
+                                    pass
+                            break
+                except Exception:
+                    pass
+
+            parser = self._analysis_service._recipe_parser_factory.create_parser(
+                parsed_input
+            )
+            if not parser:
+                raise RecipeValidationError("Unable to parse image metadata")
+
+            from ...recipes.parsers.civitai_image import CivitaiApiMetadataParser
+
+            if isinstance(parser, CivitaiApiMetadataParser):
+                parsed = await parser.parse_metadata(
+                    parsed_input,
+                    recipe_scanner=recipe_scanner,
+                    local_cache=local_cache,
+                )
+            else:
+                parsed = await parser.parse_metadata(
+                    parsed_input, recipe_scanner=recipe_scanner
+                )
+
+            loras = list(parsed.get("loras") or [])
+            checkpoint = parsed.get("model")
+            is_lora_type = model_type.startswith("lora")
+            is_ckpt_type = model_type.startswith("checkpoint")
+
+            # Extract parent model metadata from local_cache (used below to
+            # reconcile isDeleted entries and enrich auto-populated ones).
+            parent_civitai_id: int | None = None
+            parent_model_id: int | None = None
+            parent_version_name: str | None = None
+            parent_model_name: str | None = None
+            # Prefer sha256 key; fall back to any cached entry.
+            parent_item = local_cache.get(model_hash.lower()) if model_hash else None
+            if parent_item is None and local_cache:
+                parent_item = next(iter(local_cache.values()))
+            if parent_item:
+                civ = parent_item.get("civitai") or {}
+                if isinstance(civ, dict):
+                    parent_civitai_id = civ.get("id")
+                    parent_model_id = civ.get("modelId")
+                    parent_version_name = civ.get("name")
+                parent_model_name = parent_item.get("model_name")
+
+            # Reconcile isDeleted entries against the parent model.
+            # When the CivitAI hash lookup fails (known issue — hashes not
+            # yet computed), the parser marks the entry isDeleted even though
+            # the model exists locally.
+            if is_lora_type:
+                for lora in loras:
+                    if lora.get("isDeleted") and lora.get("file_name") == model_name:
+                        lora["isDeleted"] = False
+                        lora["existsLocally"] = True
+                        lora["hash"] = model_hash
+                        if parent_civitai_id is not None:
+                            lora["id"] = parent_civitai_id
+                        if parent_model_id is not None:
+                            lora["modelId"] = parent_model_id
+                        if parent_version_name is not None:
+                            lora["version"] = parent_version_name
+                        if parent_model_name is not None:
+                            lora["name"] = parent_model_name
+            elif is_ckpt_type and checkpoint and checkpoint.get("isDeleted"):
+                if checkpoint.get("file_name") == model_name:
+                    checkpoint["isDeleted"] = False
+                    checkpoint["existsLocally"] = True
+                    checkpoint["hash"] = model_hash
+                    if parent_civitai_id is not None:
+                        checkpoint["id"] = parent_civitai_id
+                    if parent_model_id is not None:
+                        checkpoint["modelId"] = parent_model_id
+                    if parent_version_name is not None:
+                        checkpoint["version"] = parent_version_name
+
+            # Auto-populate parent model only when the image metadata didn't
+            # contain any resources of that type.
+            if is_lora_type and not loras:
+                lora_entry = {
+                    "name": model_name,
+                    "type": "lora",
+                    "weight": 1.0,
+                    "hash": model_hash,
+                    "existsLocally": True,
+                    "localPath": None,
+                    "file_name": model_name,
+                    "thumbnailUrl": "/loras_static/images/no-preview.png",
+                    "baseModel": parsed.get("base_model", ""),
+                    "size": 0,
+                    "downloadUrl": "",
+                    "isDeleted": False,
+                }
+                if parent_civitai_id is not None:
+                    lora_entry["id"] = parent_civitai_id
+                if parent_model_id is not None:
+                    lora_entry["modelId"] = parent_model_id
+                if parent_version_name is not None:
+                    lora_entry["version"] = parent_version_name
+                if parent_model_name is not None:
+                    lora_entry["name"] = parent_model_name
+                loras.insert(0, lora_entry)
+            elif is_ckpt_type and not checkpoint:
+                checkpoint = {
+                    "name": model_name,
+                    "type": "checkpoint",
+                    "hash": model_hash,
+                    "file_name": model_name,
+                    "existsLocally": True,
+                    "baseModel": parsed.get("base_model", ""),
+                    "isDeleted": False,
+                }
+                if parent_civitai_id is not None:
+                    checkpoint["id"] = parent_civitai_id
+                if parent_model_id is not None:
+                    checkpoint["modelId"] = parent_model_id
+                if parent_version_name is not None:
+                    checkpoint["version"] = parent_version_name
+                if parent_model_name is not None:
+                    checkpoint["name"] = parent_model_name
+
+            image_url = image_data.get("url") or ""
+            image_id = extract_civitai_image_id_from_cdn_url(image_url)
+            settings_mgr = get_settings_manager()
+            civitai_host = settings_mgr.get("civitai_host") if settings_mgr else None
+            page_url = build_civitai_image_page_url(image_id, host=civitai_host) or image_url
+
+            recipe_metadata: dict[str, Any] = {
+                "base_model": parsed.get("base_model") or "",
+                "loras": loras,
+                "gen_params": parsed.get("gen_params") or {},
+                "source_path": page_url,
+            }
+            nsfw_level = image_data.get("nsfwLevel")
+            if isinstance(nsfw_level, int):
+                recipe_metadata["preview_nsfw_level"] = nsfw_level
+            if checkpoint:
+                recipe_metadata["checkpoint"] = checkpoint
+
+            image_bytes: bytes | None = None
+            extension: str | None = None
+            local_image_path = data.get("local_image_path")
+            if local_image_path and os.path.exists(local_image_path):
+                with open(local_image_path, "rb") as f:
+                    image_bytes = f.read()
+                ext = os.path.splitext(local_image_path)[1].lower()
+                if ext in (".jpg", ".jpeg", ".png", ".webp", ".gif"):
+                    extension = ext
+            elif image_data.get("url"):
+                try:
+                    downloader = await self._downloader_factory()
+                    url = image_data["url"]
+                    tmp = tempfile.NamedTemporaryFile(delete=False)
+                    tmp.close()
+                    success, result = await downloader.download_file(
+                        url, tmp.name, use_auth=False
+                    )
+                    if success:
+                        with open(tmp.name, "rb") as f:
+                            image_bytes = f.read()
+                        url_path = url.split("?")[0].split("#")[0]
+                        ext = os.path.splitext(url_path)[1].lower()
+                        if ext:
+                            extension = ext
+                    if os.path.exists(tmp.name):
+                        os.unlink(tmp.name)
+                except Exception as exc:
+                    self._logger.warning(
+                        "Failed to download image for recipe: %s", exc
+                    )
+
+            # Fallback: try to locate a custom image on disk using model_hash + image id
+            if image_bytes is None:
+                image_id = image_data.get("id") or ""
+                if image_id and model_hash:
+                    from ...utils.example_images_paths import get_model_folder
+                    model_folder = get_model_folder(model_hash)
+                    if model_folder and os.path.exists(model_folder):
+                        for fname in os.listdir(model_folder):
+                            if f"custom_{image_id}" in fname:
+                                ext = os.path.splitext(fname)[1].lower()
+                                if ext not in (".jpg", ".jpeg", ".png", ".webp", ".gif"):
+                                    continue
+                                fpath = os.path.join(model_folder, fname)
+                                if os.path.isfile(fpath):
+                                    try:
+                                        with open(fpath, "rb") as f:
+                                            image_bytes = f.read()
+                                        extension = ext
+                                    except Exception as exc:
+                                        self._logger.warning(
+                                            "Failed to read custom image file %s: %s",
+                                            fpath, exc,
+                                        )
+                                break
+
+            prompt = (
+                (parsed.get("gen_params") or {}).get("prompt") or ""
+            )
+            if prompt:
+                name = " ".join(str(prompt).split()[:10])
+            else:
+                name = f"Recipe from {model_name}"
+
+            save_result = await self._persistence_service.save_recipe(
+                recipe_scanner=recipe_scanner,
+                image_bytes=image_bytes,
+                image_base64=None,
+                name=name,
+                tags=[],
+                metadata=recipe_metadata,
+                extension=extension,
+            )
+            return web.json_response(save_result.payload, status=save_result.status)
+
+        except RecipeValidationError as exc:
+            return web.json_response({"error": str(exc)}, status=400)
+        except Exception as exc:
+            self._logger.error(
+                "Error creating recipe from example: %s", exc, exc_info=True
+            )
+            return web.json_response({"error": str(exc)}, status=500)
+
 
 class RecipeAnalysisHandler:
     """Analyze images to extract recipe metadata."""

py/routes/misc_route_registrar.py

@@ -22,6 +22,8 @@ class RouteDefinition:
 MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition("GET", "/api/lm/settings", "get_settings"),
     RouteDefinition("POST", "/api/lm/settings", "update_settings"),
+    RouteDefinition("GET", "/api/lm/llm/models", "get_llm_models"),
+    RouteDefinition("GET", "/api/lm/llm/provider-models", "get_provider_models"),
     RouteDefinition("GET", "/api/lm/doctor/diagnostics", "get_doctor_diagnostics"),
     RouteDefinition("POST", "/api/lm/doctor/repair-cache", "repair_doctor_cache"),
     RouteDefinition("POST", "/api/lm/doctor/resolve-filename-conflicts", "resolve_doctor_filename_conflicts"),
@@ -37,10 +39,12 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition("POST", "/api/lm/update-usage-stats", "update_usage_stats"),
     RouteDefinition("GET", "/api/lm/get-usage-stats", "get_usage_stats"),
     RouteDefinition("POST", "/api/lm/update-lora-code", "update_lora_code"),
+    RouteDefinition("GET", "/api/lm/update-lora-code", "get_update_lora_code"),
     RouteDefinition("GET", "/api/lm/trained-words", "get_trained_words"),
     RouteDefinition("GET", "/api/lm/model-example-files", "get_model_example_files"),
     RouteDefinition("POST", "/api/lm/register-nodes", "register_nodes"),
     RouteDefinition("POST", "/api/lm/update-node-widget", "update_node_widget"),
+    RouteDefinition("GET", "/api/lm/update-node-widget", "get_update_node_widget"),
     RouteDefinition("GET", "/api/lm/get-registry", "get_registry"),
     RouteDefinition("GET", "/api/lm/check-model-exists", "check_model_exists"),
     RouteDefinition("GET", "/api/lm/check-models-exist", "check_models_exist"),
@@ -94,6 +98,26 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition(
         "GET", "/api/lm/delete-model-version", "delete_model_version"
     ),
+    # Hugging Face model endpoints
+    RouteDefinition(
+        "GET", "/api/lm/hf-repo-files", "get_hf_repo_files"
+    ),
+    RouteDefinition(
+        "POST", "/api/lm/download-hf-model", "download_hf_model"
+    ),
+    RouteDefinition(
+        "POST", "/api/lm/set-hf-url", "set_hf_url"
+    ),
+    # Agent skill endpoints
+    RouteDefinition(
+        "GET", "/api/lm/agent/skills", "get_agent_skills"
+    ),
+    RouteDefinition(
+        "POST", "/api/lm/agent/execute/{skill_name}", "execute_agent_skill"
+    ),
+    RouteDefinition(
+        "POST", "/api/lm/agent/cancel", "cancel_agent_skill"
+    ),
 )
 
 

py/routes/misc_routes.py

@@ -39,6 +39,8 @@ from .handlers.misc_handlers import (
     build_service_registry_adapter,
 )
 from .handlers.base_model_handlers import BaseModelHandlerSet
+from .handlers.hf_handlers import HfHandler
+from .handlers.agent_handlers import AgentHandler
 from .misc_route_registrar import MiscRouteRegistrar
 
 logger = logging.getLogger(__name__)
@@ -136,6 +138,8 @@ class MiscRoutes:
         doctor = DoctorHandler(settings_service=self._settings)
         example_workflows = ExampleWorkflowsHandler()
         base_model = BaseModelHandlerSet()
+        hf_handler = HfHandler()
+        agent_handler = AgentHandler()
 
         return self._handler_set_factory(
             health=health,
@@ -155,6 +159,8 @@ class MiscRoutes:
             doctor=doctor,
             example_workflows=example_workflows,
             base_model=base_model,
+            hf_handler=hf_handler,
+            agent_handler=agent_handler,
         )
 
 

py/routes/model_route_registrar.py

@@ -46,6 +46,7 @@ COMMON_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
         "GET", "/api/lm/{prefix}/auto-organize-progress", "get_auto_organize_progress"
     ),
     RouteDefinition("GET", "/api/lm/{prefix}/top-tags", "get_top_tags"),
+    RouteDefinition("GET", "/api/lm/{prefix}/search-tags", "search_tags"),
     RouteDefinition("GET", "/api/lm/{prefix}/base-models", "get_base_models"),
     RouteDefinition("GET", "/api/lm/{prefix}/model-types", "get_model_types"),
     RouteDefinition("GET", "/api/lm/{prefix}/scan", "scan_models"),
@@ -101,11 +102,46 @@ COMMON_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition("POST", "/api/lm/download-model", "download_model"),
     RouteDefinition("GET", "/api/lm/download-model-get", "download_model_get"),
     RouteDefinition("GET", "/api/lm/cancel-download-get", "cancel_download_get"),
+    RouteDefinition("GET", "/api/lm/skip-download", "skip_download_get"),
     RouteDefinition("GET", "/api/lm/pause-download", "pause_download_get"),
     RouteDefinition("GET", "/api/lm/resume-download", "resume_download_get"),
     RouteDefinition(
         "GET", "/api/lm/download-progress/{download_id}", "get_download_progress"
     ),
+    RouteDefinition("GET", "/api/lm/downloads/queue", "get_download_queue"),
+    RouteDefinition("GET", "/api/lm/downloads/queue/add", "add_to_download_queue"),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/remove", "remove_from_download_queue"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/move-to-top", "move_queue_item_to_top"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/move-to-end", "move_queue_item_to_end"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/clear", "clear_download_queue"
+    ),
+    RouteDefinition("GET", "/api/lm/downloads/history", "get_download_history"),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/history/clear", "clear_download_history"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/history/delete", "delete_download_history_item"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/history/retry", "retry_download_from_history"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/history/retry-all", "retry_all_failed_downloads"
+    ),
+    RouteDefinition("GET", "/api/lm/downloads/stats", "get_download_stats"),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/complete", "complete_download_in_queue"
+    ),
+    RouteDefinition(
+        "GET", "/api/lm/downloads/queue/status", "update_download_queue_status"
+    ),
     RouteDefinition("POST", "/api/lm/{prefix}/cancel-task", "cancel_task"),
     RouteDefinition("GET", "/{prefix}", "handle_models_page"),
 )

py/routes/recipe_route_registrar.py

@@ -29,6 +29,7 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition("POST", "/api/lm/recipes/save", "save_recipe"),
     RouteDefinition("DELETE", "/api/lm/recipe/{recipe_id}", "delete_recipe"),
     RouteDefinition("GET", "/api/lm/recipes/top-tags", "get_top_tags"),
+    RouteDefinition("GET", "/api/lm/recipes/search-tags", "search_tags"),
     RouteDefinition("GET", "/api/lm/recipes/base-models", "get_base_models"),
     RouteDefinition("GET", "/api/lm/recipes/roots", "get_roots"),
     RouteDefinition("GET", "/api/lm/recipes/folders", "get_folders"),
@@ -58,6 +59,7 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
     RouteDefinition("POST", "/api/lm/recipes/repair", "repair_recipes"),
     RouteDefinition("POST", "/api/lm/recipes/cancel-repair", "cancel_repair"),
     RouteDefinition("POST", "/api/lm/recipe/{recipe_id}/repair", "repair_recipe"),
+    RouteDefinition("POST", "/api/lm/recipes/repair-bulk", "repair_recipes_bulk"),
     RouteDefinition("GET", "/api/lm/recipes/repair-progress", "get_repair_progress"),
     RouteDefinition("POST", "/api/lm/recipes/batch-import/start", "start_batch_import"),
     RouteDefinition(
@@ -74,6 +76,12 @@ ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
         "GET", "/api/lm/recipes/check-image-exists", "check_image_exists"
     ),
     RouteDefinition("GET", "/api/lm/recipes/import-from-url", "import_from_url"),
+    RouteDefinition(
+        "POST", "/api/lm/recipes/create-from-example", "create_from_example"
+    ),
+    RouteDefinition(
+        "POST", "/api/lm/recipe/{recipe_id}/reimport", "reimport_recipe"
+    ),
 )
 
 

py/routes/stats_routes.py

@@ -11,6 +11,8 @@ from ..config import config
 from ..services.settings_manager import get_settings_manager
 from ..services.server_i18n import server_i18n
 from ..services.service_registry import ServiceRegistry
+from ..services.model_query import normalize_sub_type, resolve_sub_type
+from ..utils.constants import VALID_LORA_SUB_TYPES, VALID_CHECKPOINT_SUB_TYPES
 from ..utils.usage_stats import UsageStats
 
 logger = logging.getLogger(__name__)
@@ -140,6 +142,21 @@ class StatsRoutes:
             # Get usage statistics
             usage_data = await self.usage_stats.get_stats()
             
+            # CivitAI model type distribution across all model types
+            # Use the same logic as the filter panel: normalize_sub_type(resolve_sub_type(entry))
+            # with sub-type validation per model type
+            model_types_counter: Counter[str] = Counter()
+            for entry in lora_cache.raw_data:
+                ntype = normalize_sub_type(resolve_sub_type(entry))
+                if ntype and ntype in VALID_LORA_SUB_TYPES:
+                    model_types_counter[ntype] += 1
+            for entry in checkpoint_cache.raw_data:
+                ntype = normalize_sub_type(resolve_sub_type(entry))
+                if ntype and ntype in VALID_CHECKPOINT_SUB_TYPES:
+                    model_types_counter[ntype] += 1
+            # Embeddings: always count as "embedding" regardless of CivitAI sub-type
+            model_types_counter['embedding'] = len(embedding_cache.raw_data)
+            
             return web.json_response({
                 'success': True,
                 'data': {
@@ -154,7 +171,8 @@ class StatsRoutes:
                     'total_generations': usage_data.get('total_executions', 0),
                     'unused_loras': self._count_unused_models(lora_cache.raw_data, usage_data.get('loras', {})),
                     'unused_checkpoints': self._count_unused_models(checkpoint_cache.raw_data, usage_data.get('checkpoints', {})),
-                    'unused_embeddings': self._count_unused_models(embedding_cache.raw_data, usage_data.get('embeddings', {}))
+                    'unused_embeddings': self._count_unused_models(embedding_cache.raw_data, usage_data.get('embeddings', {})),
+                    'model_types_distribution': dict(model_types_counter.most_common())
                 }
             })
             
@@ -459,9 +477,12 @@ class StatsRoutes:
                 if unused_lora_percent > 50:
                     insights.append({
                         'type': 'warning',
-                        'title': 'High Number of Unused LoRAs',
-                        'description': f'{unused_lora_percent:.1f}% of your LoRAs ({unused_loras}/{total_loras}) have never been used.',
-                        'suggestion': 'Consider organizing or archiving unused models to free up storage space.'
+                        'key': 'insights.unusedLoras.high',
+                        'params': {
+                            'percent': f'{unused_lora_percent:.1f}',
+                            'count': str(unused_loras),
+                            'total': str(total_loras)
+                        }
                     })
             
             if total_checkpoints > 0:
@@ -469,9 +490,12 @@ class StatsRoutes:
                 if unused_checkpoint_percent > 30:
                     insights.append({
                         'type': 'warning',
-                        'title': 'Unused Checkpoints Detected',
-                        'description': f'{unused_checkpoint_percent:.1f}% of your checkpoints ({unused_checkpoints}/{total_checkpoints}) have never been used.',
-                        'suggestion': 'Review and consider removing checkpoints you no longer need.'
+                        'key': 'insights.unusedCheckpoints.detected',
+                        'params': {
+                            'percent': f'{unused_checkpoint_percent:.1f}',
+                            'count': str(unused_checkpoints),
+                            'total': str(total_checkpoints)
+                        }
                     })
             
             if total_embeddings > 0:
@@ -479,9 +503,12 @@ class StatsRoutes:
                 if unused_embedding_percent > 50:
                     insights.append({
                         'type': 'warning',
-                        'title': 'High Number of Unused Embeddings',
-                        'description': f'{unused_embedding_percent:.1f}% of your embeddings ({unused_embeddings}/{total_embeddings}) have never been used.',
-                        'suggestion': 'Consider organizing or archiving unused embeddings to optimize your collection.'
+                        'key': 'insights.unusedEmbeddings.high',
+                        'params': {
+                            'percent': f'{unused_embedding_percent:.1f}',
+                            'count': str(unused_embeddings),
+                            'total': str(total_embeddings)
+                        }
                     })
             
             # Storage insights
@@ -492,18 +519,20 @@ class StatsRoutes:
             if total_size > 100 * 1024 * 1024 * 1024:  # 100GB
                 insights.append({
                     'type': 'info',
-                    'title': 'Large Collection Detected',
-                    'description': f'Your model collection is using {self._format_size(total_size)} of storage.',
-                    'suggestion': 'Consider using external storage or cloud solutions for better organization.'
+                    'key': 'insights.collection.large',
+                    'params': {
+                        'size': self._format_size(total_size)
+                    }
                 })
             
             # Recent activity insight
             if usage_data.get('total_executions', 0) > 100:
                 insights.append({
                     'type': 'success',
-                    'title': 'Active User',
-                    'description': f'You\'ve completed {usage_data["total_executions"]} generations so far!',
-                    'suggestion': 'Keep exploring and creating amazing content with your models.'
+                    'key': 'insights.activity.active',
+                    'params': {
+                        'count': str(usage_data['total_executions'])
+                    }
                 })
             
             return web.json_response({

py/routes/update_routes.py

@@ -1,7 +1,6 @@
 import os
 import logging
 import toml
-import git
 import zipfile
 import shutil
 import tempfile
@@ -11,11 +10,33 @@ from typing import Dict, List
 
 from ..utils.settings_paths import ensure_settings_file
 from ..services.downloader import get_downloader
+from ..services.service_registry import ServiceRegistry
 
 logger = logging.getLogger(__name__)
 
 NETWORK_EXCEPTIONS = (ClientError, OSError, asyncio.TimeoutError)
 
+# User-managed directories that live inside the plugin folder (portable
+# mode) and must survive a Git-based update. ``git clean -fd`` would
+# otherwise delete them because they are untracked and, in released tags,
+# not listed in ``.gitignore``. ``-e`` excludes a path from cleaning
+# regardless of whether it is ignored.
+_PRESERVE_DIRS = ('settings.json', 'civitai', 'wildcards', 'backups', 'stats', 'logs', 'cache', 'model_cache')
+
+
+def _clean_excludes() -> List[str]:
+    """Build the ``-e`` arguments for ``git clean`` from :data:`_PRESERVE_DIRS`."""
+    excludes: List[str] = []
+    for name in _PRESERVE_DIRS:
+        excludes.append('-e')
+        excludes.append(name)
+        # For directories, also exclude nested matches explicitly
+        # (``-e dir`` alone matches the dir entry; ``-e dir/**`` guards
+        # contents under all git versions as defense-in-depth).
+        excludes.append('-e')
+        excludes.append(f'{name}/**')
+    return excludes
+
 
 class UpdateRoutes:
     """Routes for handling plugin update checks"""
@@ -212,8 +233,19 @@ class UpdateRoutes:
 
             zip_path = tmp_zip_path
 
-            # Skip both settings.json, civitai and model cache folder
-            UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json', 'civitai', 'model_cache'])
+            # Close the downloaded-versions SQLite connection before cleaning,
+            # so that shutil.rmtree() does not fail on Windows (the process
+            # cannot delete a file with an outstanding open handle).
+            try:
+                history_svc = ServiceRegistry._services.get("downloaded_version_history_service")
+                if history_svc is not None:
+                    history_svc.close()
+                    logger.info("Closed downloaded-version history database connection")
+            except Exception:
+                logger.debug("Could not close downloaded-version history database", exc_info=True)
+
+            # Skip settings.json, civitai, model cache and runtime cache folders
+            UpdateRoutes._clean_plugin_folder(plugin_root, skip_files=['settings.json', 'civitai', 'model_cache', 'cache', 'wildcards', 'backups', 'stats'])
 
             # Extract ZIP to temp dir
             with tempfile.TemporaryDirectory() as tmp_dir:
@@ -222,16 +254,17 @@ class UpdateRoutes:
                     # Find extracted folder (GitHub ZIP contains a root folder)
                     extracted_root = next(os.scandir(tmp_dir)).path
 
-                    # Copy files, skipping settings.json and civitai folder
+                    # Copy files, skipping user data that should be preserved
+                    skip_items = {'settings.json', 'civitai', 'wildcards', 'backups', 'stats'}
                     for item in os.listdir(extracted_root):
-                        if item == 'settings.json' or item == 'civitai':
+                        if item in skip_items:
                             continue
                         src = os.path.join(extracted_root, item)
                         dst = os.path.join(plugin_root, item)
                         if os.path.isdir(src):
                             if os.path.exists(dst):
                                 shutil.rmtree(dst)
-                            shutil.copytree(src, dst, ignore=shutil.ignore_patterns('settings.json', 'civitai'))
+                            shutil.copytree(src, dst, ignore=shutil.ignore_patterns(*skip_items))
                         else:
                             shutil.copy2(src, dst)
 
@@ -239,15 +272,17 @@ class UpdateRoutes:
                     # for ComfyUI Manager to work properly
                     tracking_info_file = os.path.join(plugin_root, '.tracking')
                     tracking_files = []
+                    skip_tracked = {'civitai', 'wildcards', 'backups', 'stats'}
                     for root, dirs, files in os.walk(extracted_root):
-                        # Skip civitai folder and its contents
+                        # Skip user data directories and their contents
                         rel_root = os.path.relpath(root, extracted_root)
-                        if rel_root == 'civitai' or rel_root.startswith('civitai' + os.sep):
+                        top_dir = rel_root.split(os.sep)[0] if rel_root != '.' else ''
+                        if top_dir in skip_tracked:
                             continue
                         for file in files:
                             rel_path = os.path.relpath(os.path.join(root, file), extracted_root)
-                            # Skip settings.json and any file under civitai
-                            if rel_path == 'settings.json' or rel_path.startswith('civitai' + os.sep):
+                            # Skip settings.json and any file under user data dirs
+                            if rel_path == 'settings.json' or rel_path.split(os.sep)[0] in skip_tracked:
                                 continue
                             tracking_files.append(rel_path.replace("\\", "/"))
                     with open(tracking_info_file, "w", encoding='utf-8') as file:
@@ -342,6 +377,17 @@ class UpdateRoutes:
         Returns:
             tuple: (success, new_version)
         """
+        try:
+            import git
+        except ImportError:
+            logger.error(
+                "GitPython is not available: the git executable was not found in PATH. "
+                "Install git or set $GIT_PYTHON_GIT_EXECUTABLE to the git binary path."
+            )
+            return False, ""
+
+        clean_excludes = _clean_excludes()
+
         try:
             # Open the Git repository
             repo = git.Repo(plugin_root)
@@ -353,8 +399,9 @@ class UpdateRoutes:
             if nightly:
                 # Reset to discard any local changes
                 repo.git.reset('--hard')
-                # Clean untracked files
-                repo.git.clean('-fd')
+                # Clean untracked files, but preserve user-managed directories
+                # (wildcards, backups, stats, civitai, caches, settings.json).
+                repo.git.clean('-fd', *clean_excludes)
                 
                 # Switch to main branch and pull latest
                 main_branch = 'main'
@@ -371,8 +418,9 @@ class UpdateRoutes:
             else:
                 # Reset to discard any local changes
                 repo.git.reset('--hard')
-                # Clean untracked files
-                repo.git.clean('-fd')
+                # Clean untracked files, but preserve user-managed directories
+                # (wildcards, backups, stats, civitai, caches, settings.json).
+                repo.git.clean('-fd', *clean_excludes)
                 
                 # Get latest release tag
                 tags = sorted(repo.tags, key=lambda t: t.commit.committed_datetime, reverse=True)
@@ -438,6 +486,7 @@ class UpdateRoutes:
             if not os.path.exists(os.path.join(plugin_root, '.git')):
                 return git_info
 
+            import git
             repo = git.Repo(plugin_root)
             commit = repo.head.commit
             git_info['commit_hash'] = commit.hexsha

py/services/agent/__init__.py

@@ -0,0 +1,27 @@
+"""LLM-powered metadata enrichment pipeline infrastructure.
+
+This package provides the orchestration layer for LLM-powered features.
+Skills define *what* to do (prompt template).  The :class:`AgentService`
+handles *how* (LLM calls, context gathering, validation, progress).
+
+NOTE: The current implementation is a code-driven pipeline, not a true
+agent loop.  Future agent orchestration (LLM-driven tool selection) will
+live alongside this package with its own namespace.
+"""
+
+from __future__ import annotations
+
+from .skill_definition import SkillDefinition, SkillPermissions
+from .skill_registry import SkillRegistry
+from .agent_service import AgentService, AgentProgressReporter, SkillResult
+from .post_processor import PostProcessor
+
+__all__ = [
+    "AgentProgressReporter",
+    "AgentService",
+    "PostProcessor",
+    "SkillDefinition",
+    "SkillPermissions",
+    "SkillRegistry",
+    "SkillResult",
+]

py/services/agent/agent_service.py

@@ -0,0 +1,489 @@
+"""Pipeline orchestration service.
+
+The :class:`AgentService` coordinates LLM-powered pipeline execution:
+
+1. Look up the pipeline definition in :class:`SkillRegistry`
+2. Validate input against its ``input_schema``
+3. Prepare context via :mod:`~py.metadata_ops` (read metadata, list base models, fetch HF README)
+4. If ``llm_required``: call :class:`LLMService` with the rendered prompt
+5. Post-process via :class:`PostProcessor` (delegates I/O to :mod:`~py.metadata_ops`)
+6. Broadcast progress and completion via :class:`WebSocketManager`
+
+Pipeline definitions (*skills*) describe *what* to do (prompt template).
+The AgentService handles *how* (LLM calls, context gathering, validation,
+progress).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import re
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional
+
+import aiohttp
+
+import os
+
+from ...config import config
+from ..llm_service import LLMService
+from ..websocket_manager import ws_manager
+from .post_processor import PostProcessor
+from .skill_registry import SkillRegistry
+from .skills.enrich_hf_metadata.readme_processor import (
+    clean_readme_for_llm,
+    extract_relevant_section,
+)
+
+logger = logging.getLogger(__name__)
+
+
+class AgentProgressReporter:
+    """Protocol-compatible progress reporter backed by WebSocket broadcast."""
+
+    async def on_progress(self, payload: Dict[str, Any]) -> None:
+        await ws_manager.broadcast(payload)
+
+
+@dataclass
+class SkillResult:
+    """Outcome of a skill execution."""
+
+    success: bool
+    updated_models: List[Dict[str, Any]] = field(default_factory=list)
+    errors: List[str] = field(default_factory=list)
+    summary: str = ""
+
+
+def _validate_schema(data: Any, schema: Dict[str, Any], path: str = "") -> List[str]:
+    """Minimal JSON schema validator.
+
+    Supports a subset of JSON Schema: ``type``, ``properties``, ``required``,
+    ``items``, ``enum``.  Returns a list of error messages (empty = valid).
+    """
+
+    errors: List[str] = []
+    if not schema:
+        return errors
+
+    expected_type = schema.get("type")
+    if expected_type:
+        type_map = {
+            "string": str,
+            "number": (int, float),
+            "integer": int,
+            "boolean": bool,
+            "array": list,
+            "object": dict,
+            "null": type(None),
+        }
+        expected_py = type_map.get(expected_type)
+        if expected_py is not None and not isinstance(data, expected_py):
+            errors.append(f"{path or 'root'}: expected {expected_type}, got {type(data).__name__}")
+            return errors
+
+    if expected_type == "object" and isinstance(data, dict):
+        properties = schema.get("properties", {})
+        required = schema.get("required", [])
+        for req_key in required:
+            if req_key not in data:
+                errors.append(f"{path or 'root'}: missing required property '{req_key}'")
+        for key, value in data.items():
+            if key in properties:
+                errors.extend(_validate_schema(value, properties[key], f"{path}.{key}"))
+
+    if expected_type == "array" and isinstance(data, list):
+        items_schema = schema.get("items")
+        if items_schema:
+            for i, item in enumerate(data):
+                errors.extend(_validate_schema(item, items_schema, f"{path}[{i}]"))
+
+    if "enum" in schema and data not in schema["enum"]:
+        errors.append(f"{path or 'root'}: value '{data}' not in enum {schema['enum']}")
+
+    return errors
+
+
+# ------------------------------------------------------------------
+# Prompt template rendering
+# ------------------------------------------------------------------
+
+
+def _render_prompt(template: str, variables: Dict[str, Any]) -> str:
+    """Render a prompt template with ``{{variable}}`` placeholders.
+
+    Uses simple regex substitution — no Jinja2 dependency needed.
+    """
+
+    def replace(match: re.Match) -> str:
+        key = match.group(1).strip()
+        value = variables.get(key, "")
+        if isinstance(value, (dict, list)):
+            return json.dumps(value, ensure_ascii=False, indent=2)
+        return str(value)
+
+    return re.sub(r"\{\{(\w+)\}\}", replace, template)
+
+
+class AgentService:
+    """Orchestrate agent skill execution.
+
+    Usage::
+
+        service = await AgentService.get_instance()
+        result = await service.execute_skill(
+            skill_name="enrich_hf_metadata",
+            input_data={"model_paths": ["/path/to/model.safetensors"]},
+            progress_callback=AgentProgressReporter(),
+        )
+    """
+
+    _instance: Optional["AgentService"] = None
+    _lock: asyncio.Lock = asyncio.Lock()
+
+    def __init__(
+        self,
+        *,
+        skill_registry: Optional[SkillRegistry] = None,
+        llm_service: Optional[LLMService] = None,
+    ) -> None:
+        self._registry = skill_registry
+        self._llm_service = llm_service
+
+    @classmethod
+    async def get_instance(cls) -> "AgentService":
+        """Return the lazily-initialised global ``AgentService``."""
+
+        if cls._instance is None:
+            async with cls._lock:
+                if cls._instance is None:
+                    cls._instance = cls(
+                        skill_registry=await SkillRegistry.get_instance(),
+                        llm_service=await LLMService.get_instance(),
+                    )
+        return cls._instance
+
+    @classmethod
+    def reset_instance(cls) -> None:
+        """Reset the cached singleton — primarily for tests."""
+
+        cls._instance = None
+
+    async def _ensure_registry(self) -> SkillRegistry:
+        if self._registry is None:
+            self._registry = await SkillRegistry.get_instance()
+        return self._registry
+
+    async def _ensure_llm(self) -> LLMService:
+        if self._llm_service is None:
+            self._llm_service = await LLMService.get_instance()
+        return self._llm_service
+
+    async def list_skills(self) -> List[Dict[str, Any]]:
+        """Return a JSON-serialisable list of available skills."""
+
+        registry = await self._ensure_registry()
+        return [
+            {
+                "name": s.name,
+                "title": s.title,
+                "description": s.description,
+                "llm_required": s.llm_required,
+                "model_type_filter": s.model_type_filter,
+            }
+            for s in registry.list_skills()
+        ]
+
+    async def execute_skill(
+        self,
+        *,
+        skill_name: str,
+        input_data: Dict[str, Any],
+        progress_callback: Optional[AgentProgressReporter] = None,
+    ) -> SkillResult:
+        """Execute a pipeline (skill) on the given models.
+
+        Args:
+            skill_name: Name of the pipeline to execute
+            input_data: Input validated against the pipeline's ``input_schema``
+            progress_callback: Optional WebSocket progress reporter
+
+        Returns:
+            :class:`SkillResult` with success status and updated model info
+        """
+
+        registry = await self._ensure_registry()
+        skill = registry.get_skill(skill_name)
+        if skill is None:
+            return SkillResult(
+                success=False,
+                errors=[f"Skill not found: {skill_name}"],
+                summary=f"Skill '{skill_name}' does not exist",
+            )
+
+        input_errors = _validate_schema(input_data, skill.input_schema)
+        if input_errors:
+            return SkillResult(
+                success=False,
+                errors=input_errors,
+                summary=f"Invalid input: {'; '.join(input_errors)}",
+            )
+
+        model_paths = input_data.get("model_paths", [])
+        if not model_paths:
+            return SkillResult(
+                success=False,
+                errors=["No model_paths provided"],
+                summary="No models to process",
+            )
+
+        total = len(model_paths)
+        processed = 0
+        success_count = 0
+        skipped_count = 0
+        updated_models: List[Dict[str, Any]] = []
+        errors: List[str] = []
+        post_processor = PostProcessor()
+
+        await self._emit_progress(
+            progress_callback, skill_name, status="started",
+            total=total, processed=0, success=0,
+        )
+
+        llm = await self._ensure_llm()
+        llm_configured = llm.is_configured() if skill.llm_required else True
+
+        for model_path in model_paths:
+            model_filename = os.path.basename(model_path)
+            logger.info(
+                "[%s] [%d/%d] %s",
+                skill_name, processed + 1, total, model_filename,
+            )
+            updated_data: Dict[str, Any] = {}
+            skip_model = False
+            try:
+                from ...metadata_ops import read_metadata
+                metadata = await read_metadata(model_path)
+
+                # Fast-fail: enrich_hf_metadata requires hf_url to have HF README context
+                if skill_name == "enrich_hf_metadata" and not metadata.get("hf_url", ""):
+                    logger.info(
+                        "[%s] SKIP %s — no hf_url in metadata",
+                        skill_name, model_filename,
+                    )
+                    skipped_count += 1
+                    skip_model = True
+
+                if not skip_model:
+                    prompt_vars: Dict[str, Any] = {"model_path": model_path}
+                    if skill.llm_required and llm_configured:
+                        prompt_vars = await self._build_prompt_context(
+                            skill_name, model_path, metadata, registry, llm,
+                        )
+
+                    llm_response: Optional[Dict[str, Any]] = None
+                    if skill.llm_required and llm_configured:
+                        prompt_template = registry.load_prompt(skill_name)
+                        rendered = _render_prompt(prompt_template, prompt_vars)
+                        llm_response = await llm.chat_completion_json(
+                            system_prompt=prompt_vars.get(
+                                "system_prompt",
+                                "You are a helpful assistant that extracts structured metadata.",
+                            ),
+                            user_prompt=rendered,
+                        )
+                        if llm_response:
+                            logger.info(
+                                "[%s] [%d/%d] %s → base_model=%s confidence=%s",
+                                skill_name, processed + 1, total, model_filename,
+                                (llm_response.get("base_model") or "?")[:50],
+                                llm_response.get("confidence", "?"),
+                            )
+
+                    model_result = await post_processor.process(
+                        skill_name=skill_name,
+                        model_path=model_path,
+                        llm_output=llm_response or {},
+                        metadata=metadata,
+                        readme_content=prompt_vars.get("readme_content_full", ""),
+                    )
+
+                    if model_result.get("success", True):
+                        success_count += 1
+                        uf = model_result.get("updated_fields", [])
+                        if uf:
+                            updated_models.append({"path": model_path, "updated_fields": uf})
+                        updated_data = model_result.get("updates", {})
+                        if "preview_url" in updated_data and updated_data["preview_url"]:
+                            updated_data["preview_url"] = config.get_preview_static_url(
+                                updated_data["preview_url"]
+                            )
+                    else:
+                        errors.extend(
+                            model_result.get("errors", [model_result.get("error", "Unknown error")])
+                        )
+
+            except Exception as exc:
+                logger.error("Skill %s failed for %s: %s", skill_name, model_path, exc)
+                errors.append(f"{model_path}: {exc}")
+
+            processed += 1
+            await self._emit_progress(
+                progress_callback, skill_name, status="processing",
+                total=total, processed=processed, success=success_count,
+                skipped=skipped_count,
+                current_path=model_path,
+                updated_data=updated_data,
+            )
+
+        result = SkillResult(
+            success=success_count > 0,
+            updated_models=updated_models,
+            errors=errors,
+            summary=f"Processed {processed}/{total} models, {success_count} succeeded, {skipped_count} skipped",
+        )
+
+        await self._emit_progress(
+            progress_callback, skill_name, status="completed",
+            total=total, processed=processed, success=success_count,
+            skipped=skipped_count,
+            updated_models=updated_models, errors=errors, summary=result.summary,
+        )
+
+        return result
+
+    # ------------------------------------------------------------------
+    # Base model grouping (keeps the prompt compact)
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _format_base_models(models: List[str]) -> str:
+        """Format the base model list as a flat, one-per-line list.
+
+        Attempts to group by family consistently degraded LLM extraction
+        accuracy — the LLM finds individual model names harder to spot
+        in comma-separated groups than in a simple ``- Name`` list.
+        """
+        return "\n".join(f"- {m}" for m in models)
+
+    async def _build_prompt_context(
+        self,
+        skill_name: str,
+        model_path: str,
+        metadata: Dict[str, Any],
+        registry: SkillRegistry,
+        llm: Any,
+    ) -> Dict[str, Any]:
+        """Gather variables for the skill's prompt template.
+
+        Reads metadata, fetches the HF README (if applicable), lists available
+        base models, loads user priority tags, and returns a dict that maps to
+        ``{{variable}}`` placeholders in ``prompt.md``.
+        """
+        from ...metadata_ops import identify_model_type, list_base_models
+        from ..settings_manager import SettingsManager
+
+        context: Dict[str, Any] = {
+            "model_path": model_path,
+            "model_basename": "",
+            "hf_url": "",
+            "repo": "",
+            "readme_content": "",
+            "readme_content_full": "",
+            "current_metadata": {},
+            "base_models": [],
+            "priority_tags": "",
+        }
+
+        # Extract model basename (filename without extension) for the LLM
+        # to use when locating the matching section in collection repos.
+        raw_basename = os.path.splitext(os.path.basename(model_path))[0]
+        context["model_basename"] = raw_basename or ""
+
+        context["current_metadata"] = {
+            "file_name": metadata.get("file_name", ""),
+            "base_model": metadata.get("base_model", ""),
+            "tags": metadata.get("tags", []),
+            "modelDescription": metadata.get("modelDescription", ""),
+            "trainedWords": metadata.get("trainedWords", []),
+            "sha256": (metadata.get("sha256") or "")[:16] + "..." if metadata.get("sha256") else "",
+            "size": metadata.get("size", 0),
+        }
+
+        hf_url = metadata.get("hf_url", "")
+        context["hf_url"] = hf_url
+        repo = self._extract_repo_from_url(hf_url) if hf_url else ""
+        context["repo"] = repo or ""
+        if repo:
+            readme = await self._fetch_readme(repo)
+            # Trim README to the section relevant to this model file
+            # (collection repos often have multiple models in one README).
+            if readme and raw_basename:
+                trimmed = extract_relevant_section(readme, raw_basename)
+                cleaned = clean_readme_for_llm(trimmed) if trimmed else ""
+            else:
+                cleaned = clean_readme_for_llm(readme) if readme else ""
+            context["readme_content"] = cleaned if cleaned else "(README not available)"
+            context["readme_content_full"] = readme or ""
+
+        try:
+            raw_models = await list_base_models()
+            context["base_models"] = self._format_base_models(raw_models)
+        except Exception as exc:
+            logger.debug("Failed to list base models: %s", exc)
+            context["base_models"] = "</not available>"
+
+        # Determine model type and load the corresponding priority_tags
+        try:
+            model_type = await identify_model_type(model_path)
+            context["model_type"] = model_type
+            settings = SettingsManager()
+            priority_config = settings.get_priority_tag_config()
+            context["priority_tags"] = priority_config.get(model_type, "")
+        except Exception as exc:
+            logger.debug("Failed to load priority tags: %s", exc)
+            context["model_type"] = "lora"
+            context["priority_tags"] = ""
+
+        return context
+
+    @staticmethod
+    def _extract_repo_from_url(hf_url: str) -> Optional[str]:
+        """Extract ``user/repo`` from a HuggingFace URL."""
+        if not hf_url:
+            return None
+        m = re.match(r"https?://huggingface\.co/([^/]+/[^/]+)", hf_url)
+        return m.group(1) if m else None
+
+    @staticmethod
+    async def _fetch_readme(repo: str) -> str:
+        """Fetch README.md from HuggingFace (tries ``main``, then ``master``)."""
+        async with aiohttp.ClientSession(
+            headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
+            timeout=aiohttp.ClientTimeout(total=30),
+        ) as session:
+            for branch in ("main", "master"):
+                url = f"https://huggingface.co/{repo}/raw/{branch}/README.md"
+                try:
+                    async with session.get(url) as resp:
+                        if resp.status == 200:
+                            return await resp.text()
+                except Exception as exc:
+                    logger.debug("Failed to fetch README from %s: %s", url, exc)
+        return ""
+
+    async def _emit_progress(
+        self,
+        callback: Optional[AgentProgressReporter],
+        skill_name: str,
+        *,
+        status: str,
+        **extra: Any,
+    ) -> None:
+        """Send a progress update via WebSocket (if callback is set)."""
+        payload: Dict[str, Any] = {"type": "agent_progress", "skill": skill_name, "status": status}
+        payload.update(extra)
+        if callback is not None:
+            await callback.on_progress(payload)

py/services/agent/post_processor.py

@@ -0,0 +1,336 @@
+"""Post-processing engine for skill pipeline outputs.
+
+The :class:`PostProcessor` takes the LLM's structured JSON output and applies
+it to a model's on-disk metadata via the :mod:`~py.metadata_ops` functions.
+
+It handles all the skill-specific business logic — conditions, transformations,
+and orchestration of multiple side-effects (write metadata, download preview,
+refresh cache).  All actual I/O is delegated to :mod:`~py.metadata_ops`.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import re
+from datetime import datetime, timezone
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+
+class PostProcessor:
+    """Deterministic post-processor for skill pipeline outputs.
+
+    Usage (called by :class:`~py.services.agent.agent_service.AgentService`)::
+
+        processor = PostProcessor()
+        result = await processor.process(
+            skill_name="enrich_hf_metadata",
+            model_path="/path/to/model.safetensors",
+            llm_output={...},
+            metadata={...},    # from metadata_ops.read_metadata()
+        )
+    """
+
+    async def process(
+        self,
+        *,
+        skill_name: str,
+        model_path: str,
+        llm_output: Dict[str, Any],
+        metadata: Dict[str, Any],
+        readme_content: str = "",
+    ) -> Dict[str, Any]:
+        """Route *llm_output* to the correct skill post-processor.
+
+        *readme_content* is optional raw markdown content (e.g. HF README)
+        that is converted to HTML and stored as ``modelDescription`` for
+        the description tab.
+
+        Returns a dict with keys ``success`` (bool), ``updated_fields`` (list),
+        ``preview_downloaded`` (bool), and ``errors`` (list).
+        """
+        if skill_name == "enrich_hf_metadata":
+            return await self._process_enrich_hf_metadata(
+                model_path, llm_output, metadata, readme_content,
+            )
+        return {
+            "success": False,
+            "updated_fields": [],
+            "errors": [f"No post-processor registered for skill: {skill_name}"],
+        }
+
+    # ------------------------------------------------------------------
+    # enrich_hf_metadata
+    # ------------------------------------------------------------------
+
+    async def _process_enrich_hf_metadata(
+        self,
+        model_path: str,
+        llm_output: Dict[str, Any],
+        metadata: Dict[str, Any],
+        readme_content: str = "",
+    ) -> Dict[str, Any]:
+        from ...metadata_ops import (
+            apply_metadata_updates,
+            download_preview,
+            refresh_cache,
+        )
+        from .skills.enrich_hf_metadata.readme_processor import (
+            convert_readme_to_html,
+            extract_gallery_images,
+            extract_gallery_table_images,
+            extract_relevant_section,
+            extract_simple_markdown_images,
+            extract_html_img_tags,
+            extract_repo_from_hf_url,
+        )
+
+        updated_fields: List[str] = []
+        preview_downloaded = False
+
+        # -- Determine whether this is an HF-sourced model -----------------
+        is_hf_model = not metadata.get("from_civitai", True)
+
+        # -- Collect updates -----------------------------------------------
+        updates: Dict[str, Any] = {}
+
+        # base_model
+        new_base = (llm_output.get("base_model") or "").strip()
+        current_base = metadata.get("base_model", "") or ""
+        if new_base and self._should_overwrite(current_base, is_hf_model):
+            updates["base_model"] = new_base
+
+        # trigger words → civitai.trainedWords
+        new_triggers = llm_output.get("trigger_words", [])
+        trigger_words_empty = True
+        if isinstance(new_triggers, list):
+            cleaned = [t.strip() for t in new_triggers if t.strip()]
+            cleaned = [t for t in cleaned if t.lower() not in ("none", "null", "n/a")]
+            trigger_words_empty = not cleaned
+            current_civitai = metadata.get("civitai") or {}
+            current_triggers = current_civitai.get("trainedWords") or []
+            if self._should_overwrite_list(current_triggers, is_hf_model):
+                trig_civitai = dict(current_civitai)
+                if "civitai" in updates and isinstance(updates["civitai"], dict):
+                    trig_civitai.update(updates["civitai"])
+                trig_civitai["trainedWords"] = cleaned
+                updates["civitai"] = trig_civitai
+
+        # modelDescription — from raw README content (converted to HTML)
+        if readme_content and is_hf_model:
+            converted = convert_readme_to_html(readme_content)
+            if converted:
+                updates["modelDescription"] = converted
+
+        # short_description → civitai.description (for "About this version")
+        short_desc = (llm_output.get("short_description") or "").strip()
+        if short_desc and is_hf_model:
+            current_civitai = metadata.get("civitai") or {}
+            desc_civitai = dict(current_civitai)
+            if "civitai" in updates and isinstance(updates["civitai"], dict):
+                desc_civitai.update(updates["civitai"])
+            desc_civitai["description"] = short_desc
+            updates["civitai"] = desc_civitai
+
+        # gallery images → civitai.images (from YAML frontmatter widget entries
+        # and Sample Gallery markdown tables in the README body)
+        gallery_images: List[Dict[str, Any]] = []
+        if readme_content and is_hf_model:
+            hf_url = metadata.get("hf_url", "") or ""
+            repo = extract_repo_from_hf_url(hf_url)
+            if repo:
+                rec_w = llm_output.get("recommended_width") or 0
+                rec_h = llm_output.get("recommended_height") or 0
+
+                # 1. Widget images (YAML frontmatter)
+                gallery = extract_gallery_images(
+                    readme_content, repo,
+                    default_width=rec_w, default_height=rec_h,
+                )
+
+                # 2. Sample Gallery table images (markdown body), deduplicated
+                existing_urls = {img["url"] for img in gallery if img.get("url")}
+                table_images = extract_gallery_table_images(
+                    readme_content, repo,
+                    existing_urls=existing_urls,
+                    default_width=rec_w, default_height=rec_h,
+                )
+                existing_urls.update(img["url"] for img in table_images if img.get("url"))
+
+                # 3. Simple markdown images `![alt](url)` in the body
+                simple_images = extract_simple_markdown_images(
+                    readme_content, repo,
+                    existing_urls=existing_urls,
+                    default_width=rec_w, default_height=rec_h,
+                )
+                existing_urls.update(img["url"] for img in simple_images if img.get("url"))
+
+                # 4. HTML `<img>` tags (used by many collection repos)
+                html_images = extract_html_img_tags(
+                    readme_content, repo,
+                    existing_urls=existing_urls,
+                    default_width=rec_w, default_height=rec_h,
+                )
+
+                all_images = gallery + table_images + simple_images + html_images
+                if all_images:
+                    gallery_images = all_images
+                    current_civitai = metadata.get("civitai") or {}
+                    gallery_civitai = dict(current_civitai)
+                    if "civitai" in updates and isinstance(updates["civitai"], dict):
+                        gallery_civitai.update(updates["civitai"])
+                    gallery_civitai["images"] = all_images
+                    updates["civitai"] = gallery_civitai
+
+        # tags
+        new_tags = llm_output.get("tags", [])
+        if isinstance(new_tags, list) and new_tags:
+            existing_tags = metadata.get("tags") or []
+            merged = self._merge_tags(existing_tags, new_tags)
+            if len(merged) > len(existing_tags) or is_hf_model:
+                updates["tags"] = merged
+
+        # metadata_source & llm_enriched_at (always set)
+        updates["metadata_source"] = "agent:enrich_hf_metadata"
+        updates["llm_enriched_at"] = datetime.now(timezone.utc).isoformat()
+
+        # Store LLM confidence in metadata so it's accessible for evaluation
+        raw_confidence = (llm_output.get("confidence") or "").strip()
+        if raw_confidence:
+            updates["_llm_confidence"] = raw_confidence
+
+        # Fallback: extract instance_prompt from YAML frontmatter when the LLM
+        # returned empty trigger words but the README has instance_prompt.
+        if trigger_words_empty:
+            instance_prompt = _extract_yaml_instance_prompt(readme_content)
+            if instance_prompt:
+                current_civitai = metadata.get("civitai") or {}
+                trig_civitai = dict(current_civitai)
+                if "civitai" in updates and isinstance(updates["civitai"], dict):
+                    trig_civitai.update(updates["civitai"])
+                trig_civitai["trainedWords"] = [instance_prompt]
+                updates["civitai"] = trig_civitai
+
+        preview_remote_url = (llm_output.get("preview_url") or "").strip()
+        # Fallback: if the LLM couldn't find a preview image in the cleaned
+        # README, find the first gallery image from the *model-specific
+        # section* of the README (not the repo-wide first image, which
+        # belongs to a different model in collection repos).
+        if not preview_remote_url and readme_content and is_hf_model:
+            model_basename = os.path.splitext(os.path.basename(model_path))[0]
+            relevant_section = extract_relevant_section(
+                readme_content, model_basename,
+            )
+            if relevant_section and relevant_section != readme_content:
+                for img in gallery_images:
+                    img_url = img.get("url", "")
+                    if img_url and img_url in relevant_section:
+                        preview_remote_url = img_url
+                        break
+        # Last resort: use the first gallery image from the full README.
+        if not preview_remote_url and gallery_images:
+            preview_remote_url = gallery_images[0].get("url", "")
+        current_preview = metadata.get("preview_url") or ""
+        if preview_remote_url and not (current_preview and os.path.exists(current_preview)):
+            local_path = await download_preview(model_path, preview_remote_url)
+            if local_path:
+                preview_downloaded = True
+                updates["preview_url"] = local_path
+
+        # notes — plain-text summary of usage info from the LLM
+        new_notes = (llm_output.get("notes") or "").strip()
+        if new_notes:
+            updates["notes"] = new_notes
+
+        # usage_tips — JSON string (e.g. {"strength_min":0.85,"strength_max":1.4})
+        raw_tips = (llm_output.get("usage_tips") or "").strip()
+        if raw_tips and raw_tips != "{}":
+            try:
+                json.loads(raw_tips)
+                updates["usage_tips"] = raw_tips
+            except (json.JSONDecodeError, TypeError):
+                logger.warning(
+                    "LLM returned invalid usage_tips JSON: %s", raw_tips[:200]
+                )
+
+        if updates:
+            updated_fields = await apply_metadata_updates(model_path, updates)
+
+        # -- Refresh scanner cache ------------------------------------------
+        if updated_fields or preview_downloaded:
+            await refresh_cache(model_path)
+
+        return {
+            "success": True,
+            "updated_fields": updated_fields,
+            "preview_downloaded": preview_downloaded,
+            "updates": updates,
+            "errors": [],
+        }
+
+    # ------------------------------------------------------------------
+    # Helpers
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _should_overwrite(current_value: str, is_hf_model: bool) -> bool:
+        """Return ``True`` when a scalar field should be overwritten."""
+        return is_hf_model or not current_value or current_value.lower() in (
+            "", "unknown",
+        )
+
+    @staticmethod
+    def _should_overwrite_list(current_list: List[str], is_hf_model: bool) -> bool:
+        """Return ``True`` when a list field should be overwritten."""
+        return is_hf_model or not current_list
+
+    @staticmethod
+    def _merge_tags(existing: List[str], new: List[str]) -> List[str]:
+        """Merge *new* tags into *existing*, all lowercased.
+
+        This matches the behaviour of :class:`TagUpdateService` which
+        normalises every tag to lowercase for case-insensitive dedup.
+        """
+        merged: List[str] = []
+        seen: set = set()
+        for tag in list(existing) + list(new):
+            t = tag.strip().lower()
+            if t and t not in seen:
+                merged.append(t)
+                seen.add(t)
+        return merged
+
+
+# ------------------------------------------------------------------
+# Module-level helpers
+# ------------------------------------------------------------------
+
+
+def _extract_yaml_instance_prompt(readme_content: str) -> str:
+    """Extract ``instance_prompt`` from the YAML frontmatter of a HF README.
+
+    Returns the prompt text, or empty string if not found.  Handles
+    ``null`` / ``~`` YAML null values by returning empty string.
+    """
+    if not readme_content or not readme_content.startswith("---"):
+        return ""
+
+    # Find end of frontmatter
+    end = readme_content.find("---", 3)
+    if end == -1:
+        return ""
+    frontmatter = readme_content[3:end]
+
+    for line in frontmatter.split("\n"):
+        line = line.strip()
+        m = re.match(r"^instance_prompt:\s*(.*)", line)
+        if m:
+            val = m.group(1).strip().strip('"').strip("'")
+            if val.lower() in ("null", "~", "none", ""):
+                return ""
+            return val
+
+    return ""

py/services/agent/skill_definition.py

@@ -0,0 +1,45 @@
+"""Skill definition data structures.
+
+Each skill is described by a :class:`SkillDefinition` that declares its
+input/output schemas, whether it needs an LLM call, and what permissions
+its post-processor has.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Dict, List, Optional, Tuple
+
+
+@dataclass(frozen=True)
+class SkillPermissions:
+    """Declarative permission scope for a skill's post-processor.
+
+    These are auditable constraints — the :class:`AgentService` checks them
+    before invoking the handler.  They are defense-in-depth, not a sandbox.
+    """
+
+    write_metadata: bool = True
+    write_previews: bool = True
+    network_domains: Tuple[str, ...] = ()
+
+
+@dataclass(frozen=True)
+class SkillDefinition:
+    """Immutable description of an agent skill."""
+
+    name: str
+    title: str
+    description: str
+    llm_required: bool
+    input_schema: Dict[str, Any] = field(default_factory=dict)
+    output_schema: Dict[str, Any] = field(default_factory=dict)
+    model_type_filter: Optional[List[str]] = None
+    permissions: SkillPermissions = field(default_factory=SkillPermissions)
+
+    def applies_to_model_type(self, model_type: str) -> bool:
+        """Return ``True`` if this skill can run on the given model type."""
+
+        if self.model_type_filter is None:
+            return True
+        return model_type in self.model_type_filter

py/services/agent/skill_registry.py

@@ -0,0 +1,210 @@
+"""Discovery and loading of prompt-based skills.
+
+Skills live in ``py/services/agent/skills/<name>/`` directories.  Each
+directory must contain a ``prompt.md`` file with YAML frontmatter::
+
+    ---
+    name: my_skill
+    title: "My Skill"
+    description: "What this skill does"
+    llm_required: true
+    ---
+
+    Prompt template with ``{{variable}}`` placeholders.
+
+Legacy ``SKILL.md`` files are also supported for backward compatibility.
+
+The registry scans the skills directory on first access and caches results.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import logging
+import re
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+import yaml
+
+from .skill_definition import SkillDefinition, SkillPermissions
+
+logger = logging.getLogger(__name__)
+
+# Directory where built-in skills are stored
+_SKILLS_DIR = Path(__file__).parent / "skills"
+
+#: Preferred file names for prompt definition files (tried in order).
+#: ``prompt.md`` is the current convention; ``SKILL.md`` is the legacy name
+#: kept for backward compatibility.
+_PROMPT_FILE_NAMES: tuple[str, ...] = ("prompt.md", "SKILL.md")
+
+
+# ---------------------------------------------------------------------------
+# Frontmatter parser
+# ---------------------------------------------------------------------------
+
+_FRONTMATTER_RE = re.compile(
+    r"^---\s*\n(.*?\n)---\s*\n?(.*)", re.DOTALL
+)
+
+
+def _parse_skill_file(path: Path) -> tuple[dict, str]:
+    """Read a prompt definition file (``prompt.md`` or legacy ``SKILL.md``) and
+    return (frontmatter_dict, body_text).
+
+    Raises ``ValueError`` if the file lacks valid YAML frontmatter.
+    """
+    text = path.read_text(encoding="utf-8")
+    m = _FRONTMATTER_RE.match(text)
+    if not m:
+        raise ValueError(f"Missing or invalid YAML frontmatter in {path}")
+    frontmatter = yaml.safe_load(m.group(1))
+    if not isinstance(frontmatter, dict):
+        raise ValueError(f"Frontmatter in {path} is not a mapping")
+    body = m.group(2).strip()
+    return frontmatter, body
+
+
+class SkillRegistry:
+    """Discover and load agent skills from the filesystem."""
+
+    _instance: Optional["SkillRegistry"] = None
+    _lock: asyncio.Lock = asyncio.Lock()
+
+    def __init__(self, skills_dir: Path = _SKILLS_DIR) -> None:
+        self._skills_dir = skills_dir
+        self._skills: Dict[str, SkillDefinition] = {}
+        self._loaded: bool = False
+
+    # ------------------------------------------------------------------
+    # Singleton access
+    # ------------------------------------------------------------------
+
+    @classmethod
+    async def get_instance(cls) -> "SkillRegistry":
+        """Return the lazily-initialised global ``SkillRegistry``."""
+
+        if cls._instance is None:
+            async with cls._lock:
+                if cls._instance is None:
+                    registry = cls()
+                    registry._discover()
+                    cls._instance = registry
+        return cls._instance
+
+    @classmethod
+    def reset_instance(cls) -> None:
+        """Reset the cached singleton — primarily for tests."""
+
+        cls._instance = None
+
+    # ------------------------------------------------------------------
+    # Discovery
+    # ------------------------------------------------------------------
+
+    @staticmethod
+    def _find_prompt_file(skill_dir: Path) -> Path | None:
+        """Return the first prompt definition file that exists in *skill_dir*.
+
+        Tries ``_PROMPT_FILE_NAMES`` in order so that new conventions
+        (``prompt.md``) take precedence while legacy ``SKILL.md`` files
+        still load without changes.
+        """
+        for name in _PROMPT_FILE_NAMES:
+            candidate = skill_dir / name
+            if candidate.exists():
+                return candidate
+        return None
+
+    def _discover(self) -> None:
+        """Scan the skills directory and load all valid skill definitions."""
+
+        self._skills.clear()
+        if not self._skills_dir.is_dir():
+            logger.warning("Skills directory does not exist: %s", self._skills_dir)
+            self._loaded = True
+            return
+
+        for entry in sorted(self._skills_dir.iterdir()):
+            if not entry.is_dir():
+                continue
+            prompt_file = self._find_prompt_file(entry)
+            if prompt_file is None:
+                continue
+            try:
+                definition = self._load_skill_definition(prompt_file)
+                if definition is not None:
+                    self._skills[definition.name] = definition
+                    logger.debug("Loaded skill: %s", definition.name)
+            except Exception as exc:
+                logger.warning("Failed to load skill from %s: %s", prompt_file, exc)
+
+        self._loaded = True
+        logger.info("Discovered %d prompt-based skills", len(self._skills))
+
+    def _load_skill_definition(self, path: Path) -> Optional[SkillDefinition]:
+        """Parse a prompt definition file's frontmatter into a
+        :class:`SkillDefinition`."""
+
+        try:
+            data, _body = _parse_skill_file(path)
+        except (ValueError, yaml.YAMLError) as exc:
+            logger.warning("Failed to parse prompt file %s: %s", path, exc)
+            return None
+
+        if "name" not in data:
+            logger.warning("Prompt file %s missing required 'name' field", path)
+            return None
+
+        perm_data = data.get("permissions", {})
+        permissions = SkillPermissions(
+            write_metadata=perm_data.get("write_metadata", True),
+            write_previews=perm_data.get("write_previews", True),
+            network_domains=tuple(perm_data.get("network_domains", [])),
+        )
+
+        return SkillDefinition(
+            name=data["name"],
+            title=data.get("title", data["name"]),
+            description=data.get("description", ""),
+            llm_required=data.get("llm_required", False),
+            input_schema=data.get("input_schema", {}),
+            output_schema=data.get("output_schema", {}),
+            model_type_filter=data.get("model_type_filter"),
+            permissions=permissions,
+        )
+
+    # ------------------------------------------------------------------
+    # Public API
+    # ------------------------------------------------------------------
+
+    def list_skills(self) -> List[SkillDefinition]:
+        """Return all discovered skill definitions."""
+
+        if not self._loaded:
+            self._discover()
+        return list(self._skills.values())
+
+    def get_skill(self, name: str) -> Optional[SkillDefinition]:
+        """Return the skill definition for ``name``, or ``None`` if not found."""
+
+        if not self._loaded:
+            self._discover()
+        return self._skills.get(name)
+
+    def load_prompt(self, name: str) -> str:
+        """Load and return the prompt template body for the named skill."""
+
+        skill_dir = self._skills_dir / name
+        skill_path = self._find_prompt_file(skill_dir)
+        if skill_path is None:
+            raise FileNotFoundError(
+                f"Prompt file not found for skill '{name}' in {skill_dir} "
+                f"(tried {list(_PROMPT_FILE_NAMES)})"
+            )
+        try:
+            _frontmatter, body = _parse_skill_file(skill_path)
+            return body
+        except (ValueError, yaml.YAMLError) as exc:
+            raise ValueError(f"Failed to parse prompt from {skill_path}: {exc}") from exc

py/services/agent/skills/enrich_hf_metadata/prompt.md

@@ -0,0 +1,165 @@
+---
+name: enrich_hf_metadata
+title: "Enrich Metadata from HuggingFace"
+description: >
+  Parse the HuggingFace model card via LLM to extract description, trigger
+  words, base model, tags, and preview image URL.
+llm_required: true
+---
+
+You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a HuggingFace model card (README.md).
+
+## Model Information
+
+- **Repository**: {{hf_url}}
+- **Model file path**: {{model_path}}
+- **Model filename**: {{model_basename}}
+- **Repository ID**: {{repo}}
+
+## Current Metadata (may be incomplete)
+
+```json
+{{current_metadata}}
+```
+
+## User Priority Tags Reference
+
+The user has configured the following list of **meaningful tag categories** for this model type (`{{model_type}}`):
+
+```
+{{priority_tags}}
+```
+
+These are the subjects, styles, and concepts the user considers useful for categorization. Use this list as a **reference** when evaluating tags (see the **tags** section below).
+
+## Available Base Models
+
+The following base models are currently valid in this system.  Use the EXACT
+name listed — do not invent aliases or modify variant suffixes.
+
+{{base_models}}
+
+## HuggingFace README Content
+
+```
+{{readme_content}}
+```
+
+## Extraction Instructions
+
+Extract the following information from the README content above:
+
+### base_model
+The base model this model was trained on. Use EXACTLY one of the names from the **Available Base Models** list above. Do not invent new names or use aliases.
+
+Check the YAML frontmatter for ``base_model:`` first. If the frontmatter has no ``base_model:``, look at the **model filename** (``{{model_basename}}``), YAML ``tags:``, README title and first paragraph for clues — the base model family is often embedded in the name
+
+### trigger_words
+The trigger words or activation prompts needed to use this LoRA. Look for:
+- `instance_prompt:` in the YAML frontmatter
+- Phrases like "trigger word:", "trigger:", "use this prompt:", "activation prompt:"
+- In collection repos: the trigger section **specific to this model file** (look near matching download links or anchor IDs)
+- Example prompts at the start (usually the first word or phrase before any description)
+Return as an array of strings. If none found, return an empty array `[]`. **Never** return `["None"]` or any placeholder value — a truly empty list means no trigger words exist.
+
+### short_description
+A concise 1-2 sentence summary of what this model does. Extract from the "Model description" section or the first paragraph.  For collection repos, focus on the **specific model version** matching `{{model_basename}}`, not the repo as a whole.  Return empty string if the README is too minimal.
+
+### tags
+3-8 relevant tags for categorizing this model. **Quality over quantity.**
+
+Sources to consider:
+- The YAML frontmatter `tags:` list (filter out technical ones — see below)
+- The subject, style, character, or concept the model represents
+- The model filename itself may give clues (e.g. "pokemon", "anime", "pixelart")
+
+**Critical filtering rules — apply them strictly:**
+
+1. **Exclude technical/generic tags.** Reject any tag that describes the model's **training methodology, framework, architecture, or modality** rather than its content. Examples to exclude: `text-to-image`, `diffusers`, `lora`, `dreambooth`, `diffusers-training`, `flux`, `sdxl`, `checkpoint`, `pytorch`, `safetensors`, `fine-tuning`, `stable-diffusion`, and any variant of these.
+
+2. **Cross-reference against the priority_tags reference.** Only include a tag if it meaningfully describes what the model actually creates (subject, style, character type) and is semantically close to one of the priority_tags. If none of the README's tags match meaningful categories, prefer returning a smaller set or an empty array over including low-value tags.
+
+3. **All lowercase, no spaces, no hyphens** (use single words like `"photorealistic"`, `"anime"`, `"character"`).
+
+Return empty array if no meaningful content tags remain after filtering.
+
+### recommended_width, recommended_height
+The recommended image generation resolution for this model, in pixels. Look for sections like "Best Dimensions", "Recommended size", "Suggested resolution", or similar phrasing in the README. Prefer the explicitly marked "Best" or default resolution. If the table/list has multiple entries (e.g. "768 x 1024 (Best)" and "1024 x 1024 (Default)"), use the one marked "Best". Return integers. If no resolution can be determined, return 0 for both.
+
+### preview_url
+The URL of the most suitable preview image from the README. Look for:
+- Image tags near the section matching the model filename (`{{model_basename}}`)
+- The YAML frontmatter `widget:` section (which often has `output.url` fields)
+- In collection repos: the sample images listed **under the section** for this specific model version
+- Generic `![alt](url)` in the body
+Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL as `https://huggingface.co/{{repo}}/resolve/main/{filename}`. If no suitable image is found, return an empty string.
+
+### notes
+A plain-text summary of the model card's key practical usage information. Combine trigger words, style modifiers, recommended parameters (steps, CFG, resolution, sampler), and any setup tips into a readable paragraph.  For collection repos, focus on the **specific model version** matching `{{model_basename}}`.  Return empty string if the README has no useful usage info.
+
+### usage_tips
+A JSON string with structured usage recommendations. Extract from the README any explicit ranges or recommended values (e.g. "Set LoRA strength: **0.85 - 1.4**", "CLIP strength: 0.5"). Possible fields (include only those you can determine):
+
+```json
+{
+  "strength_min": 0.85,
+  "strength_max": 1.4,
+  "strength_range": "0.85-1.4",
+  "strength": 0.6,
+  "clip_strength": 0.5,
+  "clip_skip": 2
+}
+```
+
+Return the JSON string (e.g. `'{"strength_min":0.85,"strength_max":1.4}'`). Return `"{}"` if nothing useful is found.
+
+### confidence
+Your confidence level in the extracted data:
+- "high" — most fields were explicitly stated in the README
+- "medium" — some fields were inferred from context
+- "low" — most fields are guesses based on limited information
+
+## Important: Handling Collection Repos (multiple model files)
+
+Many HuggingFace repos contain **multiple model files** in a single repository
+(e.g. a "LoRA collection" with different styles/characters in separate files).
+
+The model file currently being enriched is: **`{{model_basename}}`**
+
+To find the correct section in the README:
+
+1. **Search for download links** containing the filename — the surrounding paragraph is your section.
+2. **Search for anchor IDs** (`<a id="...">`) or section headings whose text matches words from the filename.
+3. **Search for HTML headings** (`<h1>`, `<h2>`, `<span>`) containing parts of the filename.
+4. If no match is found, use the full README as usual — the model may be the only one in the repo.
+
+When a matching section IS found, prefer metadata from that section.
+When no section matches (e.g. single-model repos or repos without per-file sections),
+extract metadata from the full README normally.  Do not return empty data just
+because the filename doesn't appear in the README.
+
+## Output Format
+
+Return ONLY a JSON object with exactly these fields (no markdown fences, no extra text):
+
+```json
+{
+  "model_path": "{{model_path}}",
+  "base_model": "<canonical name or empty string>",
+  "trigger_words": ["<word1>", "<word2>"],
+  "short_description": "<1-2 sentence summary>",
+  "tags": ["<tag1>", "<tag2>"],
+  "recommended_width": 768,
+  "recommended_height": 1024,
+  "preview_url": "<image URL or empty string>",
+  "notes": "<plain-text usage summary or empty string>",
+  "usage_tips": "<JSON string like '{\"strength_min\":0.85,\"strength_max\":1.4}' or '{}'>",
+  "confidence": "<high|medium|low>"
+}
+```
+
+Important:
+- Only include the JSON object, no other text
+- If a field cannot be determined, use an empty string or empty array
+- Do not fabricate information not supported by the README
+- Never use placeholder values like `"None"` or `"unknown"` for missing data — use empty string or empty array

py/services/aria2_downloader.py

@@ -14,12 +14,30 @@ from typing import Any, Dict, Optional, Tuple
 
 import aiohttp
 
-from .downloader import DownloadProgress, get_downloader
+from .downloader import DownloadProgress, get_downloader, is_ssl_cert_verify_error
 from .aria2_transfer_state import Aria2TransferStateStore
 from .settings_manager import get_settings_manager
 
 logger = logging.getLogger(__name__)
 
+def _try_certifi_ca_path() -> str | None:
+    """Return the certifi CA bundle path if available, else None."""
+    try:
+        import certifi  # type: ignore[import-untyped]
+
+        path = certifi.where()
+        if os.path.isfile(path):
+            logger.debug(
+                "aria2 --ca-certificate: using certifi CA bundle at %s", path
+            )
+            return path
+    except ImportError:
+        pass
+
+    logger.debug("aria2 --ca-certificate: certifi not available")
+    return None
+
+
 CIVITAI_DOWNLOAD_URL_PREFIXES = (
     "https://civitai.com/api/download/",
     "https://civitai.red/api/download/",
@@ -66,6 +84,7 @@ class Aria2Downloader:
         self._transfers: Dict[str, Aria2Transfer] = {}
         self._poll_interval = 0.5
         self._state_store = Aria2TransferStateStore()
+        self._stderr_reader_task: Optional[asyncio.Task] = None
 
     @property
     def is_running(self) -> bool:
@@ -97,7 +116,7 @@ class Aria2Downloader:
 
         try:
             while True:
-                status = await self.get_status(download_id)
+                status = await self._get_status_with_retry(download_id)
                 if status is None:
                     return False, "aria2 download not found"
 
@@ -118,6 +137,35 @@ class Aria2Downloader:
         finally:
             self._transfers.pop(download_id, None)
 
+    async def _get_status_with_retry(
+        self, download_id: str, *, max_retries: int = 4, retry_delay: float = 3.0
+    ) -> Optional[Dict[str, Any]]:
+        """Call get_status with retry for transient RPC failures.
+
+        Only retries on :exc:`Aria2Error` (RPC-level failure).  Returns
+        ``None`` immediately when the download_id is not tracked (a missing
+        transfer is not a transient condition, so retrying is pointless).
+
+        A single failed RPC call should not immediately fail the download,
+        because aria2 may be temporarily busy (e.g. finalizing multiple
+        concurrent downloads) and a retry will often succeed.
+        """
+        last_exc: Optional[Exception] = None
+        for attempt in range(max_retries):
+            try:
+                return await self.get_status(download_id)
+            except Aria2Error as exc:
+                last_exc = exc
+                if attempt < max_retries - 1:
+                    logger.warning(
+                        "aria2 get_status transient failure (attempt %d/%d) for %s: %s",
+                        attempt + 1, max_retries, download_id, exc,
+                    )
+                    await asyncio.sleep(retry_delay)
+        raise Aria2Error(
+            f"Failed to query aria2 download status after {max_retries} attempts: {last_exc}"
+        ) from last_exc
+
     async def _schedule_download(
         self,
         url: str,
@@ -153,6 +201,13 @@ class Aria2Downloader:
             "auto-file-renaming": "false",
             "file-allocation": "none",
         }
+
+        # Pass proxy to aria2 so the actual file transfer goes through the
+        # same proxy used by the aiohttp-based URL resolution step above.
+        downloader = await get_downloader()
+        if downloader.proxy_url:
+            options["all-proxy"] = downloader.proxy_url
+
         if request_headers:
             options["header"] = [
                 f"{key}: {value}" for key, value in request_headers.items()
@@ -294,6 +349,16 @@ class Aria2Downloader:
     async def close(self) -> None:
         """Shut down the RPC process and session."""
 
+        # Cancel the background stderr reader first so it stops reading
+        # from the pipe before the subprocess is terminated.
+        if self._stderr_reader_task is not None:
+            self._stderr_reader_task.cancel()
+            try:
+                await asyncio.wait_for(self._stderr_reader_task, timeout=2.0)
+            except (asyncio.CancelledError, asyncio.TimeoutError):
+                pass
+            self._stderr_reader_task = None
+
         if self._rpc_session is not None:
             await self._rpc_session.close()
             self._rpc_session = None
@@ -313,6 +378,23 @@ class Aria2Downloader:
                 process.kill()
                 await process.wait()
 
+    async def _drain_stderr(self) -> None:
+        """Continuously drain aria2's stderr pipe so it never blocks.
+
+        When the 64 KB pipe buffer fills up, aria2's ``write()`` to stderr
+        blocks, which freezes the entire ``aria2c`` process — including its
+        RPC handler.  This background task reads lines from stderr as they
+        arrive and forwards them to Python's logger.
+        """
+        try:
+            assert self._process is not None and self._process.stderr is not None
+            async for line in self._process.stderr:
+                text = line.decode("utf-8", errors="replace").rstrip()
+                if text:
+                    logger.debug("aria2 stderr: %s", text)
+        except Exception:
+            pass
+
     async def _dispatch_progress(self, callback, snapshot: DownloadProgress) -> None:
         try:
             result = callback(snapshot, snapshot)
@@ -391,6 +473,15 @@ class Aria2Downloader:
                     f"Failed to resolve authenticated Civitai redirect: status={response.status} body={body[:300]}"
                 )
         except aiohttp.ClientError as exc:
+            if is_ssl_cert_verify_error(exc):
+                logger.error(
+                    "SSL certificate verification failed during Civitai redirect "
+                    "resolution for %s. This is usually caused by an outdated CA "
+                    "certificate bundle. Recommended fixes:\n"
+                    "  1. pip install --upgrade certifi\n"
+                    "  2. pip install pip-system-certs",
+                    url,
+                )
             raise Aria2Error(
                 f"Failed to resolve authenticated Civitai redirect: {exc}"
             ) from exc
@@ -414,6 +505,11 @@ class Aria2Downloader:
                 f"--rpc-listen-port={self._rpc_port}",
                 f"--rpc-secret={self._rpc_secret}",
                 "--check-certificate=true",
+                # Point aria2 at certifi's CA bundle when available so it uses
+                # the same certificate store as Python downloads.
+                *((
+                    f"--ca-certificate={ca_cert}",
+                ) if (ca_cert := _try_certifi_ca_path()) else ()),
                 "--allow-overwrite=true",
                 "--auto-file-renaming=false",
                 "--file-allocation=none",
@@ -433,6 +529,17 @@ class Aria2Downloader:
 
             await self._wait_until_ready()
 
+            # Drain aria2's stderr in a background task so the pipe buffer
+            # never fills up.  If the pipe blocks, aria2 itself freezes and
+            # cannot respond to RPC — this was the root cause of the
+            # "Failed to query aria2 download status" timeout bug.
+            # Must start AFTER _wait_until_ready to avoid a race where the
+            # drain task consumes aria2's early-exit error message before
+            # _wait_until_ready can read it.
+            self._stderr_reader_task = asyncio.create_task(
+                self._drain_stderr()
+            )
+
     def _resolve_executable(self) -> str:
         settings = get_settings_manager()
         configured_path = (settings.get("aria2c_path") or "").strip()
@@ -552,7 +659,9 @@ class Aria2Downloader:
         if self._rpc_session is None or self._rpc_session.closed:
             async with self._rpc_session_lock:
                 if self._rpc_session is None or self._rpc_session.closed:
-                    timeout = aiohttp.ClientTimeout(total=30)
+                    timeout = aiohttp.ClientTimeout(
+                        total=None, sock_connect=10, sock_read=60
+                    )
                     self._rpc_session = aiohttp.ClientSession(timeout=timeout)
         return self._rpc_session
 

py/services/backup_service.py

@@ -141,6 +141,16 @@ class BackupService:
                     )
                 )
 
+        stats_path = os.path.join(get_settings_dir(create=True), "stats", "lora_manager_stats.json")
+        if os.path.exists(stats_path):
+            targets.append(
+                (
+                    "usage_stats",
+                    "stats/lora_manager_stats.json",
+                    stats_path,
+                )
+            )
+
         return targets
 
     @staticmethod
@@ -348,6 +358,8 @@ class BackupService:
         if kind == "model_update":
             filename = os.path.basename(archive_member)
             return str(Path(get_cache_file_path(CacheType.MODEL_UPDATE, create_dir=True)).parent / filename)
+        if kind == "usage_stats":
+            return os.path.join(get_settings_dir(create=True), "stats", "lora_manager_stats.json")
         return None
 
     async def create_auto_snapshot_if_due(self) -> Optional[dict[str, Any]]:

py/services/base_model_service.py

@@ -1,7 +1,7 @@
 from abc import ABC, abstractmethod
 import asyncio
 import re
-from typing import Any, Dict, List, Optional, Type, TYPE_CHECKING
+from typing import Any, Dict, List, Optional, Type, Union, TYPE_CHECKING
 import logging
 import os
 import time
@@ -104,6 +104,109 @@ class BaseModelService(ABC):
         fetch_duration = time.perf_counter() - t0
         initial_count = len(sorted_data)
 
+        # Optionally filter by civitai model ID (shows all local versions of a specific model)
+        civitai_model_id = kwargs.get("civitai_model_id")
+        if civitai_model_id is not None:
+            sorted_data = [
+                item for item in sorted_data
+                if self._extract_group_key(item) == civitai_model_id
+            ]
+            # VLM mode: always sort by version ID descending (newest version first),
+            # regardless of the current sort_by preference.
+            # Fall back to modified timestamp for non-CivitAI sources.
+            sorted_data.sort(
+                key=lambda x: self._extract_version_id(x)
+                or x.get("modified", 0)
+                or 0,
+                reverse=True,
+            )
+
+        # Optionally group by civitai modelId, showing only the latest version per model
+        dedup_lost = 0
+        if kwargs.get("group_by_model") and civitai_model_id is None:
+            # Determine whether to further sub-group by base model
+            # When version_grouping is "same_base", versions with different
+            # base models are effectively different groups — the dedup key
+            # needs to include base_model so the version count and VLM flow
+            # stay consistent (card shows correct count for its base model).
+            ufs = self.settings.get("version_grouping", "same_base")
+            group_by_base = ufs == "same_base"
+
+            dedup_map = {}  # (modelId [,base_model]) -> (item, version_or_modified)
+            version_counter = {}  # same-key -> count
+            standalone = []
+            for item in sorted_data:
+                mid = self._extract_group_key(item)
+                if mid is None:
+                    standalone.append(item)
+                    continue
+                key = (mid, item.get("base_model") or "") if group_by_base else mid
+                # Count all versions per key
+                version_counter[key] = version_counter.get(key, 0) + 1
+                # Prefer CivitAI version_id; fall back to modified timestamp
+                vid = self._extract_version_id(item)
+                if vid is None:
+                    vid = item.get("modified", 0) or 0
+                if key not in dedup_map or vid > dedup_map[key][1]:
+                    dedup_map[key] = (item, vid)
+            # Attach version_count to each surviving grouped item (shallow copy
+            # to avoid mutating cached dicts — the cache is shared across requests)
+            for key, (item, vid) in dedup_map.items():
+                item = dict(item)
+                item["version_count"] = version_counter[key]
+                dedup_map[key] = (item, vid)
+            dedup_lost = len(sorted_data) - (len(dedup_map) + len(standalone))
+            sorted_data = [entry[0] for entry in dedup_map.values()] + standalone
+
+        # Re-sort by version_count (grouped: after dedup; non-grouped: group internally, sort, expand)
+        if sort_params.key == "versions_count" and civitai_model_id is None:
+            reverse = sort_params.order == "desc"
+            if kwargs.get("group_by_model"):
+                # Grouped mode: items are already dedup'd with version_count attached
+                sorted_data.sort(
+                    key=lambda x: (
+                        x.get("version_count", 0),
+                        (x.get("model_name") or x.get("file_name") or "").lower(),
+                        x.get("file_path", "").lower(),
+                    ),
+                    reverse=reverse,
+                )
+            else:
+                # Non-grouped mode: group internally, sort groups by count, expand
+                # Respect the version_grouping setting (same logic as grouped dedup)
+                ufs = self.settings.get("version_grouping", "same_base")
+                group_by_base = ufs == "same_base"
+
+                model_groups: Dict[Any, List[Dict]] = {}
+                ungrouped_standalone: List[Dict] = []
+                for item in sorted_data:
+                    mid = self._extract_group_key(item)
+                    if mid is None:
+                        ungrouped_standalone.append(item)
+                        continue
+                    key = (mid, item.get("base_model") or "") if group_by_base else mid
+                    model_groups.setdefault(key, []).append(item)
+                # Sort versions within each group by version id (descending);
+                # fall back to modified timestamp for non-CivitAI sources.
+                for items in model_groups.values():
+                    items.sort(
+                        key=lambda x: self._extract_version_id(x)
+                        or x.get("modified", 0)
+                        or 0,
+                        reverse=True,
+                    )
+                # Sort groups by version count
+                sorted_groups = sorted(
+                    model_groups.values(),
+                    key=lambda items: len(items),
+                    reverse=reverse,
+                )
+                # Flatten: grouped items first, standalone items last
+                sorted_data = []
+                for items in sorted_groups:
+                    sorted_data.extend(items)
+                sorted_data.extend(ungrouped_standalone)
+
         t1 = time.perf_counter()
         if hash_filters:
             filtered_data = await self._apply_hash_filters(sorted_data, hash_filters)
@@ -172,7 +275,7 @@ class BaseModelService(ABC):
         overall_duration = time.perf_counter() - overall_start
         logger.debug(
             "%s.get_paginated_data took %.3fs (fetch: %.3fs, filter: %.3fs, update_filter: %.3fs, pagination: %.3fs, annotate: %.3fs). "
-            "Counts: initial=%d, post_filter=%d, final=%d",
+            "Counts: initial=%d, dedup=%d, post_filter=%d, final=%d",
             self.__class__.__name__,
             overall_duration,
             fetch_duration,
@@ -181,6 +284,7 @@ class BaseModelService(ABC):
             pagination_duration,
             annotate_duration,
             initial_count,
+            dedup_lost,
             post_filter_count,
             final_count,
         )
@@ -495,7 +599,7 @@ class BaseModelService(ABC):
         if not ordered_ids:
             return annotated
 
-        strategy_value = self.settings.get("update_flag_strategy")
+        strategy_value = self.settings.get("version_grouping")
         if isinstance(strategy_value, str) and strategy_value.strip():
             strategy = strategy_value.strip().lower()
         else:
@@ -602,6 +706,33 @@ class BaseModelService(ABC):
 
         return annotated
 
+    @staticmethod
+    def _extract_hf_group_key(item: Dict) -> Optional[str]:
+        """Extract `hf:{owner}/{repo}` from item's ``hf_url``, or None."""
+        hf_url = item.get("hf_url") if isinstance(item, dict) else None
+        if not hf_url or not isinstance(hf_url, str):
+            return None
+        m = re.match(
+            r"https?://huggingface\.co/([^/]+/[^/]+)", hf_url.strip()
+        )
+        if not m:
+            return None
+        return f"hf:{m.group(1)}"
+
+    @staticmethod
+    def _extract_group_key(item: Dict) -> Union[int, str, None]:
+        """Return the group identity key: CivitAI modelId (int) or HF repo (str).
+
+        Preference order:
+        1. CivitAI ``modelId`` (int)
+        2. HF repo identity ``hf:{owner}/{repo}`` (str)
+        3. ``None`` (no known grouping source)
+        """
+        mid = BaseModelService._extract_model_id(item)
+        if mid is not None:
+            return mid
+        return BaseModelService._extract_hf_group_key(item)
+
     @staticmethod
     def _extract_model_id(item: Dict) -> Optional[int]:
         civitai = item.get("civitai") if isinstance(item, dict) else None
@@ -696,8 +827,12 @@ class BaseModelService(ABC):
         }
 
     @abstractmethod
-    async def format_response(self, model_data: Dict) -> Dict:
-        """Format model data for API response - must be implemented by subclasses"""
+    async def format_response(self, model_data: Dict) -> Optional[Dict]:
+        """Format model data for API response - must be implemented by subclasses.
+
+        Subclasses should return None for corrupted entries so the handler
+        layer can filter them out. See issue #730.
+        """
         pass
 
     # Common service methods that delegate to scanner
@@ -705,6 +840,12 @@ class BaseModelService(ABC):
         """Get top tags sorted by frequency"""
         return await self.scanner.get_top_tags(limit)
 
+    async def search_tags(
+        self, query: str, limit: int = 50
+    ) -> List[Dict]:
+        """Search tags by substring, sorted by frequency"""
+        return await self.scanner.search_tags(query, limit)
+
     async def get_base_models(self, limit: int = 20) -> List[Dict]:
         """Get base models sorted by frequency"""
         return await self.scanner.get_base_models(limit)
@@ -856,13 +997,21 @@ class BaseModelService(ABC):
 
         return unified_tree
 
-    async def get_model_notes(self, model_name: str) -> Optional[str]:
-        """Get notes for a specific model file"""
+    async def get_model_notes(self, model_name: str) -> Optional[dict]:
+        """Get notes and file_path for a specific model file.
+
+        Supports both simple names (``OWSMianne_ANIMA_V1``) and full-path
+        syntax (``Anima/character/OWSMianne_ANIMA_V1``).
+        """
         cache = await self.scanner.get_cached_data()
 
         for model in cache.raw_data:
-            if model["file_name"] == model_name:
-                return model.get("notes", "")
+            file_name = model.get("file_name", "")
+            if file_name == model_name or model_name.endswith("/" + file_name) or model_name.endswith("\\" + file_name):
+                return {
+                    "notes": model.get("notes", ""),
+                    "file_path": model.get("file_path", ""),
+                }
 
         return None
 
@@ -985,6 +1134,11 @@ class BaseModelService(ABC):
 
         Listing/search endpoints return lightweight cache entries; this method performs
         a lazy read of the on-disk metadata snapshot when callers need full detail.
+
+        As a beneficial side effect, the in-memory and persistent caches are
+        opportunistically synchronised with the on-disk metadata — this keeps the
+        caches fresh even when a ``.metadata.json`` file was edited outside of the
+        normal save path (e.g. manually or by an external script).
         """
         metadata, should_skip = await MetadataManager.load_metadata(
             file_path, self.metadata_class
@@ -1002,6 +1156,19 @@ class BaseModelService(ABC):
                 MetadataManager.save_metadata(file_path, metadata)
             )
 
+        # Opportunistically sync the in-memory + persistent caches.
+        # The .metadata.json disk read is already paid for; the sync only
+        # performs work when the cache is actually stale, and uses targeted,
+        # in-place operations to minimise overhead even with large model sets.
+        #
+        # Fire-and-forget by design: the task is intentionally untracked.
+        # sync_cache_from_metadata handles its own errors internally.
+        asyncio.create_task(
+            self.scanner.sync_cache_from_metadata(
+                file_path, metadata.to_dict()
+            )
+        )
+
         return self.filter_civitai_data(metadata.to_dict().get("civitai", {}))
 
     async def get_model_description(self, file_path: str) -> Optional[str]:

py/services/batch_import_service.py

@@ -523,6 +523,10 @@ class BatchImportService:
                 if payload.get("checkpoint"):
                     metadata["checkpoint"] = payload["checkpoint"]
 
+                nsfw = payload.get("preview_nsfw_level")
+                if isinstance(nsfw, int) and nsfw > 0:
+                    metadata["preview_nsfw_level"] = nsfw
+
                 image_bytes = None
                 image_base64 = payload.get("image_base64")
 

py/services/checkpoint_scanner.py

@@ -114,6 +114,13 @@ class CheckpointScanner(ModelScanner):
                 and metadata.hash_status == "completed"
                 and metadata.sha256
             ):
+                # Ensure the in-memory hash index is populated even when
+                # the hash was already computed and persisted to the metadata
+                # file.  Without this, usage tracking (and any other caller
+                # that queries get_hash_by_filename first) will miss on every
+                # lookup and keep calling back into this method, creating a
+                # tight loop that never populates the index.
+                self._hash_index.add_entry(metadata.sha256.lower(), file_path)
                 return metadata.sha256
 
             async with self._hash_calculation_lock:
@@ -125,6 +132,7 @@ class CheckpointScanner(ModelScanner):
                     and metadata.hash_status == "completed"
                     and metadata.sha256
                 ):
+                    self._hash_index.add_entry(metadata.sha256.lower(), file_path)
                     return metadata.sha256
 
                 task = self._hash_calculation_tasks.get(real_path)
@@ -175,6 +183,9 @@ class CheckpointScanner(ModelScanner):
 
             # Check if hash is already calculated
             if metadata.hash_status == "completed" and metadata.sha256:
+                # Populate the in-memory hash index even for pre-computed
+                # hashes, mirroring the fix in calculate_hash_for_model.
+                self._hash_index.add_entry(metadata.sha256.lower(), file_path)
                 return metadata.sha256
 
             # Update status to calculating
@@ -193,6 +204,20 @@ class CheckpointScanner(ModelScanner):
             # Update hash index
             self._hash_index.add_entry(sha256.lower(), file_path)
 
+            # Update the in-memory cache entry so that subsequent
+            # _persist_current_cache / _save_persistent_cache calls
+            # write the hash back to the SQLite models table.  Without
+            # this the hash only lives in the metadata file and the
+            # in-memory hash index, both of which are lost across
+            # restarts, causing the same re-computation loop on the
+            # next session.
+            if self._cache is not None and self._cache.raw_data:
+                for entry in self._cache.raw_data:
+                    if entry.get("file_path") == file_path:
+                        entry["sha256"] = sha256.lower()
+                        entry["hash_status"] = "completed"
+                        break
+
             logger.info(f"Hash calculated for checkpoint: {file_path}")
             return sha256
 

py/services/checkpoint_service.py

@@ -1,6 +1,6 @@
 import os
 import logging
-from typing import Dict
+from typing import Dict, Optional
 
 from .base_model_service import BaseModelService
 from .auto_tag_service import extract_auto_tags
@@ -21,20 +21,37 @@ class CheckpointService(BaseModelService):
         """
         super().__init__("checkpoint", scanner, CheckpointMetadata, update_service=update_service)
     
-    async def format_response(self, checkpoint_data: Dict) -> Dict:
-        """Format Checkpoint data for API response"""
+    async def format_response(self, checkpoint_data: Dict) -> Optional[Dict]:
+        """Format Checkpoint data for API response.
+
+        Returns None when the entry is missing critical fields (corrupted cache
+        row), so the handler layer can filter it out. See issue #730.
+        """
+        # Guard against corrupted cache entries missing critical fields
+        file_path = checkpoint_data.get("file_path")
+        if not file_path or not isinstance(file_path, str):
+            logger.warning(
+                "Skipping corrupted checkpoint entry (missing file_path): %s",
+                checkpoint_data.get("file_name", "<unknown>"),
+            )
+            return None
+
         # Get sub_type from cache entry (new canonical field)
         sub_type = checkpoint_data.get("sub_type", "checkpoint")
-        
+
+        file_name = checkpoint_data.get("file_name") or ""
+        model_name = checkpoint_data.get("model_name") or file_name
+        folder = checkpoint_data.get("folder") or ""
+
         return {
-            "model_name": checkpoint_data["model_name"],
-            "file_name": checkpoint_data["file_name"],
+            "model_name": model_name,
+            "file_name": file_name,
             "preview_url": config.get_preview_static_url(checkpoint_data.get("preview_url", "")),
             "preview_nsfw_level": checkpoint_data.get("preview_nsfw_level", 0),
             "base_model": checkpoint_data.get("base_model", ""),
-            "folder": checkpoint_data["folder"],
+            "folder": folder,
             "sha256": checkpoint_data.get("sha256", ""),
-            "file_path": checkpoint_data["file_path"].replace(os.sep, "/"),
+            "file_path": file_path.replace(os.sep, "/"),
             "file_size": checkpoint_data.get("size", 0),
             "modified": checkpoint_data.get("modified", ""),
             "tags": checkpoint_data.get("tags", []),
@@ -48,6 +65,8 @@ class CheckpointService(BaseModelService):
             "skip_metadata_refresh": bool(checkpoint_data.get("skip_metadata_refresh", False)),
             "civitai": self.filter_civitai_data(checkpoint_data.get("civitai", {}), minimal=True),
             "auto_tags": checkpoint_data.get("auto_tags") or extract_auto_tags(checkpoint_data),
+            "version_count": checkpoint_data.get("version_count"),
+            "hf_url": checkpoint_data.get("hf_url", ""),
         }
     
     def find_duplicate_hashes(self) -> Dict:

py/services/civarchive_client.py

@@ -186,6 +186,22 @@ class CivArchiveClient:
         if "metadata" in file_data:
             transformed["metadata"] = file_data["metadata"]
 
+        # Infer metadata.format from filename extension
+        name = transformed.get("name")
+        if name and isinstance(name, str):
+            lower_name = name.lower()
+            if lower_name.endswith(".safetensors"):
+                inferred_format = "SafeTensor"
+            elif lower_name.endswith(".ckpt"):
+                inferred_format = "PickleTensor"
+            else:
+                inferred_format = None
+            if inferred_format:
+                if "metadata" not in transformed:
+                    transformed["metadata"] = {}
+                if isinstance(transformed["metadata"], dict):
+                    transformed["metadata"].setdefault("format", inferred_format)
+
         if file_data.get("modelVersionId") is not None:
             transformed["modelVersionId"] = file_data.get("modelVersionId")
         elif file_data.get("model_version_id") is not None:
@@ -213,6 +229,20 @@ class CivArchiveClient:
         for file_data in candidates:
             if isinstance(file_data, dict):
                 transformed_files.append(self._transform_file_entry(file_data))
+
+        # Sort: .safetensors first, .ckpt second, others last
+        # so the backend fallback (no file_params) prefers safetensors
+        def _sort_key(f: Dict) -> int:
+            fname = f.get("name") or ""
+            if isinstance(fname, str):
+                lower = fname.lower()
+                if lower.endswith(".safetensors"):
+                    return 0
+                elif lower.endswith(".ckpt"):
+                    return 1
+            return 2
+
+        transformed_files.sort(key=_sort_key)
         return transformed_files
 
     def _transform_version(
@@ -274,6 +304,20 @@ class CivArchiveClient:
             version_id = file_data.get("model_version_id") or file_data.get("modelVersionId")
             if model_id is None or version_id is None:
                 continue
+            # CivitAI / CivArchive model IDs are small integers (typically ≤ 7
+            # digits).  Reject suspiciously large values that indicate the API
+            # returned a malformed payload (e.g. a hash reinterpreted as an ID)
+            # to avoid pointless HTTP 500 errors from CivArchive.
+            _MAX_VALID_CIVITAI_ID = 100_000_000
+            try:
+                if int(model_id) >= _MAX_VALID_CIVITAI_ID or int(version_id) >= _MAX_VALID_CIVITAI_ID:
+                    logger.debug(
+                        "Skipping implausible CivArchive model_id=%s / version_id=%s",
+                        model_id, version_id,
+                    )
+                    continue
+            except (TypeError, ValueError):
+                continue
             resolved = await self.get_model_version(model_id, version_id)
             if resolved:
                 return resolved
@@ -297,7 +341,7 @@ class CivArchiveClient:
             if resolved:
                 return resolved, None
 
-            logger.error("Error fetching version of CivArchive model by hash %s", model_hash[:10])
+            logger.debug("Error fetching version of CivArchive model by hash %s", model_hash[:10])
             return None, "No version data found"
 
         except RateLimitError:
@@ -387,7 +431,7 @@ class CivArchiveClient:
 
             if version_id is not None:
                 raw_id = version_data.get("id")
-                if raw_id != version_id:
+                if raw_id is not None and str(raw_id) != str(version_id):
                     logger.warning(
                         "Requested version %s doesn't match default version %s for model %s",
                         version_id,

py/services/civitai_base_model_service.py

@@ -196,6 +196,7 @@ class CivitaiBaseModelService:
             "ernie": "ERNI",
             "ernie turbo": "ETRB",
             "nucleus": "NUCL",
+            "krea 2": "KR2",
             "svd": "SVD",
             "ltxv": "LTXV",
             "ltxv2": "LTV2",
@@ -212,6 +213,18 @@ class CivitaiBaseModelService:
             "wan video 2.2 i2v-a14b": "WAN",
             "wan video 2.5 t2v": "WAN",
             "wan video 2.5 i2v": "WAN",
+            "wan video 2.7": "WAN",
+            "wan image 2.7": "WI27",
+            "ace audio": "ACE",
+            "boogu": "BOOG",
+            "grok": "GROK",
+            "happyhorse": "HAPP",
+            "hidream-o1": "HIO1",
+            "lens": "LENS",
+            "mai": "MAI",
+            "upscaler": "UPSC",
+            "ideogram 4.0": "ID40",
+            "qwen 2": "QWN2",
         }
 
         if lower_name in special_cases:
@@ -391,6 +404,7 @@ class CivitaiBaseModelService:
                 "LTXV2",
                 "LTXV 2.3",
                 "CogVideoX",
+                "HappyHorse",
                 "Mochi",
                 "Hunyuan Video",
                 "Wan Video",
@@ -403,15 +417,25 @@ class CivitaiBaseModelService:
                 "Wan Video 2.2 I2V-A14B",
                 "Wan Video 2.5 T2V",
                 "Wan Video 2.5 I2V",
+                "Wan Image 2.7",
+                "Wan Video 2.7",
             ],
             "Other Models": [
+                "ACE Audio",
                 "Illustrious",
                 "Pony",
                 "Pony V7",
+                "Boogu",
                 "HiDream",
+                "HiDream-O1",
+                "Ideogram 4.0",
                 "Qwen",
+                "Qwen 2",
                 "AuraFlow",
                 "Chroma",
+                "Grok",
+                "Lens",
+                "MAI",
                 "ZImageTurbo",
                 "ZImageBase",
                 "PixArt a",
@@ -424,6 +448,8 @@ class CivitaiBaseModelService:
                 "Ernie",
                 "Ernie Turbo",
                 "Nucleus",
+                "Krea 2",
+                "Upscaler",
             ],
         }
 

py/services/civitai_client.py

@@ -56,7 +56,7 @@ class CivitaiClient:
         self._MAX_CACHE_ENTRIES = 500
 
     def _build_image_info_url(self, image_id: str) -> str:
-        return f"{self.base_url}/images?imageId={image_id}&nsfw=X"
+        return f"{self.base_url}/images?imageId={image_id}&nsfw=X&withMeta=true"
 
     async def _make_request(
         self,
@@ -410,6 +410,25 @@ class CivitaiClient:
             return None
 
         target_version = self._select_target_version(model_data, model_id, version_id)
+
+        # If modelVersions is empty (e.g. CivitAI cache lag for newly published
+        # models) but a specific version_id is known, fall back to fetching the
+        # version directly via the individual model-versions endpoint, then
+        # enrich it with the model-level data we already have.
+        if target_version is None and version_id is not None:
+            logger.info(
+                "modelVersions empty for model %s; falling back to direct "
+                "version lookup for %s",
+                model_id,
+                version_id,
+            )
+            version = await self._fetch_version_by_id(version_id)
+            if version:
+                self._enrich_version_with_model_data(version, model_data)
+                self._remove_comfy_metadata(version)
+                return version
+            return None
+
         if target_version is None:
             return None
 

py/services/download_coordinator.py

@@ -110,6 +110,23 @@ class DownloadCoordinator:
 
         return result
 
+    async def skip_download(self, download_id: str) -> Dict[str, Any]:
+        """Skip a download while preserving all partial files on disk."""
+        download_manager = await self._download_manager_factory()
+        result = await download_manager.skip_download(download_id)
+
+        await self._ws_manager.broadcast_download_progress(
+            download_id,
+            {
+                "status": "skipped",
+                "progress": 0,
+                "download_id": download_id,
+                "message": "Download skipped by user (partial files preserved)",
+            },
+        )
+
+        return result
+
     async def pause_download(self, download_id: str) -> Dict[str, Any]:
         """Pause an active download and notify listeners."""
 

py/services/download_manager.py

@@ -29,6 +29,7 @@ from .metadata_service import get_default_metadata_provider, get_metadata_provid
 from .downloader import get_downloader, DownloadProgress, DownloadStreamControl
 from .aria2_downloader import Aria2Error, get_aria2_downloader
 from .aria2_transfer_state import Aria2TransferStateStore
+from .download_queue_service import DownloadQueueService
 
 # Download to temporary file first
 import tempfile
@@ -229,6 +230,12 @@ class DownloadManager:
         Returns:
             Dict with download result
         """
+        logger.debug(
+            "[download] download_from_civitai called: model_id=%s, model_version_id=%s, "
+            "source=%s, file_params=%s",
+            model_id, model_version_id, source, file_params,
+        )
+
         # Validate that at least one identifier is provided
         if not model_id and not model_version_id:
             return {
@@ -249,6 +256,7 @@ class DownloadManager:
             "source": source,
             "file_params": copy.deepcopy(file_params) if file_params is not None else None,
             "progress": 0,
+
             "status": "queued",
             "transfer_backend": self._get_model_download_backend(),
             "bytes_downloaded": 0,
@@ -288,8 +296,8 @@ class DownloadManager:
             return result
         except asyncio.CancelledError:
             return {
-                "success": False,
-                "error": "Download was cancelled",
+                "success": True,
+                "cancelled": True,
                 "download_id": task_id,
             }
         finally:
@@ -360,6 +368,15 @@ class DownloadManager:
                     if self._active_downloads[task_id].get("transfer_backend") == "aria2":
                         await self._persist_aria2_state(task_id)
 
+                # Update SQLite queue status to 'downloading'
+                try:
+                    queue_service = await DownloadQueueService.get_instance()
+                    await queue_service.update_status(task_id, "downloading")
+                except Exception:
+                    logger.warning(
+                        "Failed to update queue status for %s", task_id, exc_info=True
+                    )
+
                 # Use original download implementation
                 try:
                     # Check for cancellation before starting
@@ -396,6 +413,22 @@ class DownloadManager:
                         if self._active_downloads[task_id].get("transfer_backend") == "aria2":
                             await self._persist_aria2_state(task_id)
 
+                    # Move queue item to history on completion
+                    try:
+                        queue_service = await DownloadQueueService.get_instance()
+                        await queue_service.complete_download(
+                            download_id=task_id,
+                            status=result.get("status", "completed") if result.get("success") else "failed",
+                            error=result.get("error") if not result.get("success") else None,
+                            file_path=result.get("file_path"),
+                            bytes_downloaded=self._active_downloads.get(task_id, {}).get("bytes_downloaded", 0),
+                            total_bytes=self._active_downloads.get(task_id, {}).get("total_bytes"),
+                        )
+                    except Exception:
+                        logger.warning(
+                            "Failed to complete queue item for %s", task_id, exc_info=True
+                        )
+
                     return result
                 except asyncio.CancelledError:
                     # Handle cancellation
@@ -404,6 +437,19 @@ class DownloadManager:
                         self._active_downloads[task_id]["bytes_per_second"] = 0.0
                         if self._active_downloads[task_id].get("transfer_backend") == "aria2":
                             await self._persist_aria2_state(task_id)
+
+                    # Move queue item to history as canceled
+                    try:
+                        queue_service = await DownloadQueueService.get_instance()
+                        await queue_service.complete_download(
+                            download_id=task_id,
+                            status="canceled",
+                        )
+                    except Exception:
+                        logger.warning(
+                            "Failed to cancel queue item for %s", task_id, exc_info=True
+                        )
+
                     logger.info(f"Download cancelled for task {task_id}")
                     raise
                 except Exception as e:
@@ -417,6 +463,22 @@ class DownloadManager:
                         self._active_downloads[task_id]["bytes_per_second"] = 0.0
                         if self._active_downloads[task_id].get("transfer_backend") == "aria2":
                             await self._persist_aria2_state(task_id)
+
+                    # Move queue item to history as failed
+                    try:
+                        queue_service = await DownloadQueueService.get_instance()
+                        await queue_service.complete_download(
+                            download_id=task_id,
+                            status="failed",
+                            error=str(e),
+                            bytes_downloaded=self._active_downloads.get(task_id, {}).get("bytes_downloaded", 0),
+                            total_bytes=self._active_downloads.get(task_id, {}).get("total_bytes"),
+                        )
+                    except Exception:
+                        logger.warning(
+                            "Failed to complete queue item for %s", task_id, exc_info=True
+                        )
+
                     return {"success": False, "error": str(e)}
         finally:
             # Schedule cleanup of download record after delay
@@ -620,7 +682,10 @@ class DownloadManager:
                     u for u in download_urls if not u.startswith(CIVITAI_DOWNLOAD_URL_PREFIXES)
                 ]
                 download_urls = non_civitai_urls + civitai_urls
-        else:
+
+        # Fallback: when mirrors is empty or all mirrors have been deleted,
+        # use the file's downloadUrl directly (e.g. CivitAI download endpoint).
+        if not download_urls:
             download_url = file_info.get("downloadUrl")
             if download_url:
                 download_urls.append(normalize_civitai_download_url(download_url))
@@ -1233,10 +1298,24 @@ class DownloadManager:
                     "download_id": download_id,
                 }
 
-            # Check if this checkpoint should be treated as a diffusion model based on baseModel
+            # Check if this checkpoint should be treated as a diffusion model
+            # Priority: (1) any file has type "UNet" or "Diffusion Model",
+            #            (2) baseModel is in DIFFUSION_MODEL_BASE_MODELS
             is_diffusion_model = False
             if model_type == "checkpoint":
-                if base_model_value in DIFFUSION_MODEL_BASE_MODELS:
+                # Check file types first (more direct signal from CivitAI)
+                version_files = version_info.get("files", [])
+                for f in version_files:
+                    f_type = f.get("type", "")
+                    if f_type in ("UNet", "Diffusion Model"):
+                        is_diffusion_model = True
+                        logger.info(
+                            f"File type '{f_type}' detected, routing checkpoint to unet folder"
+                        )
+                        break
+
+                # Fallback to baseModel name check
+                if not is_diffusion_model and base_model_value in DIFFUSION_MODEL_BASE_MODELS:
                     is_diffusion_model = True
                     logger.info(
                         f"baseModel '{base_model_value}' is a known diffusion model, routing to unet folder"
@@ -1310,7 +1389,17 @@ class DownloadManager:
 
             # Update save directory with relative path if provided
             if relative_path:
+                base_save_dir = save_dir
                 save_dir = os.path.join(save_dir, relative_path)
+                # Security: validate path containment after joining
+                resolved_dir = os.path.abspath(os.path.normpath(save_dir))
+                base_dir = os.path.abspath(os.path.normpath(base_save_dir))
+                if not resolved_dir.startswith(base_dir + os.sep) and resolved_dir != base_dir:
+                    logger.warning(
+                        "Path traversal detected: %s escapes %s",
+                        resolved_dir, base_dir,
+                    )
+                    return {"success": False, "error": "Download path is outside allowed directory"}
                 # Create directory if it doesn't exist
                 os.makedirs(save_dir, exist_ok=True)
 
@@ -1352,86 +1441,100 @@ class DownloadManager:
 
             # If file_params is provided, try to find matching file
             if file_params and model_version_id:
+                target_file_id = file_params.get("id")
                 target_type = file_params.get("type", "Model")
-                target_format = file_params.get("format", "SafeTensor")
-                target_size = file_params.get("size", "full")
+                target_format = file_params.get("format")
+                target_size = file_params.get("size")
                 target_fp = file_params.get("fp")
                 is_primary = file_params.get("isPrimary", False)
 
-                if is_primary:
-                    # Find primary file
+                logger.debug(
+                    "[download] file_params received: id=%s, type=%s, format=%s, size=%s, fp=%s, isPrimary=%s, "
+                    "model_version_id=%s, total_files=%d",
+                    target_file_id, target_type, target_format, target_size, target_fp, is_primary,
+                    model_version_id, len(files),
+                )
+
+                if target_file_id:
+                    target_id_str = str(target_file_id)
+                    for f in files:
+                        f_id = f.get("id")
+                        if str(f_id) == target_id_str:
+                            file_info = f
+                            logger.debug(
+                                "[download] MATCH by ID: id=%s name='%s'",
+                                f_id, f.get("name"),
+                            )
+                            break
+                    if not file_info:
+                        logger.debug("[download] No file found with id=%s", target_file_id)
+
+                elif is_primary:
                     file_info = next(
                         (
                             f
                             for f in files
                             if f.get("primary")
-                            and f.get("type") in ("Model", "Negative", "Diffusion Model")
+                            and f.get("type") in ("Model", "Negative", "Diffusion Model", "UNet")
                         ),
                         None,
                     )
                 else:
-                    # Match by metadata
+                    # Lenient metadata match: only compare fields present on both sides
                     for f in files:
                         f_type = f.get("type", "")
-                        f_meta = f.get("metadata", {})
-
-                        # Check type match
                         if f_type != target_type:
                             continue
 
-                        # Check metadata match
-                        if f_meta.get("format") != target_format:
+                        f_meta = f.get("metadata", {})
+                        f_format = f_meta.get("format") or f.get("format")
+                        f_size = f_meta.get("size") or f.get("size")
+                        f_fp = f_meta.get("fp") or f.get("fp")
+
+                        if target_format and f_format != target_format:
                             continue
-                        if f_meta.get("size") != target_size:
+                        if target_size and f_size and f_size != target_size:
                             continue
-                        if target_fp and f_meta.get("fp") != target_fp:
+                        if target_fp and f_fp and f_fp != target_fp:
                             continue
 
                         file_info = f
                         break
 
+                if not file_info:
+                    logger.debug(
+                        "[download] No match found via file_params — falling back to primary file lookup",
+                    )
+            elif not file_params:
+                logger.debug(
+                    "[download] No file_params provided (null/None) — will use primary file lookup. "
+                    "model_version_id=%s, total_files=%d",
+                    model_version_id, len(files),
+                )
+
             # Fallback to primary file if no match found
             if not file_info:
+                logger.debug("[download] Looking for primary file as fallback")
                 file_info = next(
                     (
                         f
                         for f in files
-                        if f.get("primary") and f.get("type") in ("Model", "Negative", "Diffusion Model")
+                        if f.get("primary") and f.get("type") in ("Model", "Negative", "Diffusion Model", "UNet")
                     ),
                     None,
                 )
+                if file_info:
+                    logger.debug(
+                        "[download] Fallback primary file selected: id=%s, name=%s",
+                        file_info.get("id"), file_info.get("name"),
+                    )
+                else:
+                    logger.debug("[download] No primary file found in fallback lookup")
 
             if not file_info:
                 return {"success": False, "error": "No suitable file found in metadata"}
-            mirrors = file_info.get("mirrors") or []
-            download_urls = []
-            if mirrors:
-                for mirror in mirrors:
-                    if mirror.get("deletedAt") is None and mirror.get("url"):
-                        download_urls.append(
-                            normalize_civitai_download_url(mirror["url"])
-                        )
 
-                # When source is 'civarchive', prioritize non-Civitai URLs
-                # This avoids failed downloads from deleted Civitai models
-                if source == "civarchive" and len(download_urls) > 1:
-                    civitai_urls = [
-                        u
-                        for u in download_urls
-                        if u.startswith(CIVITAI_DOWNLOAD_URL_PREFIXES)
-                    ]
-                    non_civitai_urls = [
-                        u
-                        for u in download_urls
-                        if not u.startswith(CIVITAI_DOWNLOAD_URL_PREFIXES)
-                    ]
-                    download_urls = non_civitai_urls + civitai_urls
-            else:
-                download_url = file_info.get("downloadUrl")
-                if download_url:
-                    download_urls.append(
-                        normalize_civitai_download_url(download_url)
-                    )
+            download_urls = self._build_download_urls_from_file_info(file_info, source=source)
 
             if not download_urls:
                 return {"success": False, "error": "No mirror URL found"}
@@ -1734,6 +1837,9 @@ class DownloadManager:
             model_tags, model_type
         )
 
+        if not first_tag:
+            first_tag = "no tags"  # Default if no tags available
+
         # Format the template with available data
         formatted_path = path_template
         formatted_path = formatted_path.replace("{base_model}", mapped_base_model)
@@ -1749,6 +1855,15 @@ class DownloadManager:
         if model_type == "embedding":
             formatted_path = formatted_path.replace(" ", "_")
 
+        # Sanitize the resolved path to prevent path traversal:
+        # - Strip leading slashes (prevents os.path.join from treating path as absolute)
+        # - Collapse double slashes from empty placeholder substitutions
+        # - Strip trailing slashes for cleanliness
+        formatted_path = formatted_path.lstrip("/")
+        while "//" in formatted_path:
+            formatted_path = formatted_path.replace("//", "/")
+        formatted_path = formatted_path.rstrip("/")
+
         return formatted_path
 
     async def _execute_download(
@@ -1974,7 +2089,21 @@ class DownloadManager:
                     break
 
                 last_error = result
-                if os.path.exists(save_path):
+                # For aria2: if the .aria2 control file is missing, aria2 considers
+                # the download complete.  A transient RPC failure may have made us
+                # think the download failed even though the file is fully on disk.
+                # Keep the file so a retry can find it already complete.
+                if (
+                    transfer_backend == "aria2"
+                    and os.path.exists(save_path)
+                    and not os.path.exists(f"{save_path}.aria2")
+                ):
+                    logger.warning(
+                        "aria2 download reported failure but .aria2 file is absent "
+                        "for %s — the file is likely complete.  Preserving it for retry.",
+                        save_path,
+                    )
+                elif os.path.exists(save_path):
                     try:
                         os.remove(save_path)
                     except Exception as e:
@@ -2404,6 +2533,89 @@ class DownloadManager:
                 self._download_tasks.pop(download_id, None)
                 await self._aria2_state_store.remove(download_id)
 
+    async def skip_download(self, download_id: str) -> Dict:
+        """Skip a download while preserving all partial files on disk.
+
+        Removes all in-memory tracking (asyncio task, semaphore, active/pause
+        state) but keeps partial files (.part / .aria2) on disk so that a
+        subsequent download-model-get request for the same save path can
+        auto-resume from the preserved partial download.
+
+        Args:
+            download_id: The unique identifier of the download task
+
+        Returns:
+            Dict: Status of the skip operation
+        """
+        await self._restore_persisted_downloads()
+
+        if download_id not in self._download_tasks and download_id not in self._active_downloads:
+            return {"success": False, "error": "Download task not found"}
+
+        download_info = self._active_downloads.get(download_id)
+        task = self._download_tasks.get(download_id)
+        active_statuses = {"queued", "waiting", "downloading", "paused", "cancelling"}
+        if task is None and (
+            not isinstance(download_info, dict)
+            or download_info.get("status") not in active_statuses
+        ):
+            return {"success": False, "error": "Download task not found"}
+
+        backend = (
+            self._active_downloads.get(download_id, {}).get("transfer_backend")
+            or "python"
+        )
+
+        try:
+            # For aria2: pause the transfer rather than force-removing it, so
+            # the .aria2 control file stays on disk for future resume
+            if backend == "aria2":
+                try:
+                    aria2_downloader = await get_aria2_downloader()
+                    pause_result = await aria2_downloader.pause_download(download_id)
+                    if not pause_result.get("success"):
+                        logger.warning(
+                            "Failed to pause aria2 transfer for %s during skip: %s",
+                            download_id,
+                            pause_result.get("error"),
+                        )
+                except Exception as exc:
+                    logger.warning(
+                        "Failed to pause aria2 transfer for %s during skip: %s",
+                        download_id,
+                        exc,
+                    )
+
+            # Cancel the asyncio task so the semaphore slot is released
+            if task is not None:
+                task.cancel()
+
+            # Resume pause event so the task can exit cleanly
+            pause_control = self._pause_events.get(download_id)
+            if pause_control is not None:
+                pause_control.resume()
+
+            # Wait briefly for task to acknowledge cancellation
+            if task is not None:
+                try:
+                    await asyncio.wait_for(asyncio.shield(task), timeout=2.0)
+                except (asyncio.CancelledError, asyncio.TimeoutError):
+                    pass
+
+            logger.info(f"Download skipped for task {download_id} (partial files preserved)")
+            return {"success": True, "message": "Download skipped successfully"}
+        except Exception as e:
+            logger.error(f"Error skipping download: {e}", exc_info=True)
+            return {"success": False, "error": str(e)}
+        finally:
+            # Clean up local in-memory tracking only - NO file deletion
+            self._pause_events.pop(download_id, None)
+            self._download_tasks.pop(download_id, None)
+            if download_id in self._active_downloads:
+                del self._active_downloads[download_id]
+            # Preserve aria2 state store entry so the partial download
+            # info survives restarts and can be resumed later
+
     async def pause_download(self, download_id: str) -> Dict:
         """Pause an active download without losing progress."""
 

py/services/download_queue_service.py

@@ -0,0 +1,906 @@
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+import os
+import sqlite3
+import time
+from typing import Any, Optional
+
+from ..utils.cache_paths import get_cache_base_dir
+
+logger = logging.getLogger(__name__)
+
+
+def _resolve_database_path() -> str:
+    base_dir = get_cache_base_dir(create=True)
+    history_dir = os.path.join(base_dir, "download_history")
+    os.makedirs(history_dir, exist_ok=True)
+    return os.path.join(history_dir, "download_queue.sqlite")
+
+
+class DownloadQueueService:
+    """Persistent download queue and history manager backed by SQLite.
+
+    Provides a singleton interface for managing a download queue and
+    corresponding history table, both stored in a single SQLite database
+    under the cache directory.
+    """
+
+    _instance: Optional[DownloadQueueService] = None
+    _class_lock: asyncio.Lock = asyncio.Lock()
+
+    _SCHEMA = """
+        CREATE TABLE IF NOT EXISTS download_queue (
+            download_id TEXT PRIMARY KEY,
+            model_id INTEGER,
+            model_version_id INTEGER,
+            model_name TEXT NOT NULL DEFAULT '',
+            version_name TEXT DEFAULT '',
+            thumbnail_url TEXT DEFAULT '',
+            source TEXT,
+            file_params TEXT,
+            status TEXT NOT NULL DEFAULT 'queued',
+            priority INTEGER DEFAULT 0,
+            progress INTEGER DEFAULT 0,
+            bytes_downloaded INTEGER DEFAULT 0,
+            total_bytes INTEGER,
+            bytes_per_second REAL DEFAULT 0.0,
+            error TEXT,
+            file_path TEXT,
+            added_at REAL NOT NULL,
+            started_at REAL,
+            completed_at REAL
+        );
+        CREATE INDEX IF NOT EXISTS idx_dq_status ON download_queue(status);
+        CREATE INDEX IF NOT EXISTS idx_dq_added ON download_queue(added_at);
+
+        CREATE TABLE IF NOT EXISTS download_history (
+            id INTEGER PRIMARY KEY AUTOINCREMENT,
+            download_id TEXT,
+            model_id INTEGER,
+            model_version_id INTEGER,
+            model_name TEXT NOT NULL DEFAULT '',
+            version_name TEXT DEFAULT '',
+            thumbnail_url TEXT DEFAULT '',
+            status TEXT NOT NULL,
+            error TEXT,
+            file_path TEXT,
+            bytes_downloaded INTEGER DEFAULT 0,
+            total_bytes INTEGER,
+            completed_at REAL NOT NULL,
+            is_already_exists INTEGER DEFAULT 0
+        );
+        CREATE INDEX IF NOT EXISTS idx_dh_completed ON download_history(completed_at DESC);
+        CREATE INDEX IF NOT EXISTS idx_dh_status ON download_history(status);
+        CREATE UNIQUE INDEX IF NOT EXISTS idx_dh_download_id
+            ON download_history(download_id) WHERE download_id IS NOT NULL;
+    """
+
+    @classmethod
+    async def get_instance(cls) -> DownloadQueueService:
+        """Return the singleton instance, creating it if necessary."""
+        async with cls._class_lock:
+            if cls._instance is None:
+                cls._instance = cls()
+                await cls._instance.deduplicate()
+            return cls._instance
+
+    def __init__(self, db_path: Optional[str] = None) -> None:
+        self._db_path = db_path or _resolve_database_path()
+        self._lock = asyncio.Lock()
+        self._conn: Optional[sqlite3.Connection] = None
+        self._schema_initialized = False
+        self._ensure_directory()
+        self._initialize_schema()
+
+    def _ensure_directory(self) -> None:
+        directory = os.path.dirname(self._db_path)
+        if directory:
+            os.makedirs(directory, exist_ok=True)
+
+    def _connect(self) -> sqlite3.Connection:
+        conn = sqlite3.connect(self._db_path, check_same_thread=False)
+        conn.row_factory = sqlite3.Row
+        return conn
+
+    def _get_conn(self) -> sqlite3.Connection:
+        if self._conn is None:
+            self._conn = sqlite3.connect(self._db_path, check_same_thread=False)
+            self._conn.row_factory = sqlite3.Row
+        return self._conn
+
+    def _initialize_schema(self) -> None:
+        if self._schema_initialized:
+            return
+        with self._connect() as conn:
+            conn.executescript(self._SCHEMA)
+            conn.commit()
+        self._schema_initialized = True
+
+    def get_database_path(self) -> str:
+        """Return the resolved database file path."""
+        return self._db_path
+
+    def close(self) -> None:
+        """Close the persistent SQLite connection, if open.
+
+        This is called before plugin update operations to release the
+        database file lock on Windows, allowing ``shutil.rmtree()`` to
+        succeed when the cache resides inside the plugin directory.
+        """
+        if self._conn is not None:
+            try:
+                self._conn.close()
+            except Exception:
+                pass
+            finally:
+                self._conn = None
+
+    # ------------------------------------------------------------------
+    # Queue methods
+    # ------------------------------------------------------------------
+
+    async def add_to_queue(
+        self,
+        download_id: str,
+        model_id: Optional[int] = None,
+        model_version_id: Optional[int] = None,
+        model_name: str = "",
+        version_name: str = "",
+        thumbnail_url: str = "",
+        source: Optional[str] = None,
+        file_params: Optional[dict[str, Any]] = None,
+    ) -> dict[str, Any]:
+        """Insert a new download into the queue.
+
+        Returns the inserted row as a dict (or an empty dict if the
+        download_id already exists in the queue or has a terminal
+        record in history).
+        """
+        now = time.time()
+        file_params_json = json.dumps(file_params) if file_params is not None else None
+
+        async with self._lock:
+            conn = self._get_conn()
+
+            # Reject download_ids that already have a terminal record in history.
+            history_row = conn.execute(
+                "SELECT 1 FROM download_history WHERE download_id = ? LIMIT 1",
+                (download_id,),
+            ).fetchone()
+            if history_row is not None:
+                return {}
+
+            conn.execute(
+                """
+                INSERT OR IGNORE INTO download_queue (
+                    download_id, model_id, model_version_id, model_name,
+                    version_name, thumbnail_url, source, file_params,
+                    status, priority, added_at
+                ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'queued', 0, ?)
+                """,
+                (
+                    download_id,
+                    model_id,
+                    model_version_id,
+                    model_name,
+                    version_name,
+                    thumbnail_url,
+                    source,
+                    file_params_json,
+                    now,
+                ),
+            )
+            conn.commit()
+            row = conn.execute(
+                "SELECT * FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            ).fetchone()
+
+        return dict(row) if row else {}
+
+    async def get_queue(self) -> list[dict[str, Any]]:
+        """Return all items in the queue ordered by priority then added time."""
+        async with self._lock:
+            conn = self._get_conn()
+            rows = conn.execute(
+                "SELECT * FROM download_queue ORDER BY priority DESC, added_at ASC"
+            ).fetchall()
+        return [dict(row) for row in rows]
+
+    async def get_queued_count(self) -> int:
+        """Return the number of items with status ``'queued'``."""
+        async with self._lock:
+            conn = self._get_conn()
+            row = conn.execute(
+                "SELECT COUNT(*) AS cnt FROM download_queue WHERE status = 'queued'"
+            ).fetchone()
+        return row["cnt"] if row else 0
+
+    async def update_status(
+        self,
+        download_id: str,
+        status: str,
+        **extra: Any,
+    ) -> bool:
+        """Update the status and/or extra fields of a queue item.
+
+        Accepted extra keyword arguments:
+        ``progress``, ``error``, ``file_path``, ``bytes_downloaded``,
+        ``total_bytes``, ``bytes_per_second``.
+
+        Returns ``True`` if a row was updated.
+        """
+        allowed_extra = {
+            "progress",
+            "error",
+            "file_path",
+            "bytes_downloaded",
+            "total_bytes",
+            "bytes_per_second",
+        }
+
+        set_clauses: list[str] = ["status = ?"]
+        params: list[Any] = [status]
+        now = time.time()
+
+        if status in ("downloading",):
+            set_clauses.append("started_at = COALESCE(started_at, ?)")
+            params.append(now)
+        if status in ("completed", "failed", "canceled"):
+            set_clauses.append("completed_at = ?")
+            params.append(now)
+
+        for key, value in extra.items():
+            if key in allowed_extra:
+                set_clauses.append(f"{key} = ?")
+                params.append(value)
+
+        params.append(download_id)
+
+        async with self._lock:
+            conn = self._get_conn()
+            cursor = conn.execute(
+                f"UPDATE download_queue SET {', '.join(set_clauses)} "
+                "WHERE download_id = ?",
+                params,
+            )
+            conn.commit()
+        return cursor.rowcount > 0
+
+    async def remove_from_queue(self, download_id: str) -> bool:
+        """Remove a single item from the queue by download_id.
+
+        Returns ``True`` if a row was deleted.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            cursor = conn.execute(
+                "DELETE FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            )
+            conn.commit()
+        return cursor.rowcount > 0
+
+    async def move_to_top(self, download_id: str) -> bool:
+        """Move an item to the front of the queue (highest priority).
+
+        Returns ``True`` if the item was found and updated.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            row = conn.execute(
+                "SELECT priority FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            ).fetchone()
+            if row is None:
+                return False
+
+            max_row = conn.execute(
+                "SELECT MAX(priority) AS mx FROM download_queue"
+            ).fetchone()
+            max_priority: int = max_row["mx"] if max_row["mx"] is not None else 0
+
+            conn.execute(
+                "UPDATE download_queue SET priority = ? WHERE download_id = ?",
+                (max_priority + 1, download_id),
+            )
+            conn.commit()
+        return True
+
+    async def move_to_end(self, download_id: str) -> bool:
+        """Move an item to the end of the queue (lowest priority).
+
+        Returns ``True`` if the item was found and updated.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            row = conn.execute(
+                "SELECT priority FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            ).fetchone()
+            if row is None:
+                return False
+
+            min_row = conn.execute(
+                "SELECT MIN(priority) AS mn FROM download_queue"
+            ).fetchone()
+            min_priority: int = min_row["mn"] if min_row["mn"] is not None else 0
+
+            conn.execute(
+                "UPDATE download_queue SET priority = ? WHERE download_id = ?",
+                (min_priority - 1, download_id),
+            )
+            conn.commit()
+        return True
+
+    async def clear_queue(self, status_filter: Optional[str] = None) -> int:
+        """Remove items from the queue.
+
+        When *status_filter* is provided only items with that status are
+        deleted.  Returns the number of deleted rows.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            if status_filter is not None:
+                cursor = conn.execute(
+                    "DELETE FROM download_queue WHERE status = ?",
+                    (status_filter,),
+                )
+            else:
+                cursor = conn.execute("DELETE FROM download_queue")
+            conn.commit()
+        return cursor.rowcount
+
+    async def complete_download(
+        self,
+        download_id: str,
+        status: str = "completed",
+        error: Optional[str] = None,
+        file_path: Optional[str] = None,
+        bytes_downloaded: int = 0,
+        total_bytes: Optional[int] = None,
+        completed_at: Optional[float] = None,
+    ) -> Optional[dict[str, Any]]:
+        """Atomically move a download from the queue into the history table.
+
+        Looks up the queue record by ``download_id``, deletes it from the
+        queue, and inserts a corresponding history entry with the given
+        terminal status (``completed``, ``failed``, or ``canceled``).
+
+        When *completed_at* is provided it is used as the completion
+        timestamp; otherwise ``time.time()`` is used.
+
+        Returns the original queue record (before deletion) on success,
+        or ``None`` if the download was not found in the queue.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            row = conn.execute(
+                "SELECT * FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            ).fetchone()
+            if row is None:
+                return None
+
+            now = completed_at if completed_at is not None else time.time()
+            conn.execute(
+                "DELETE FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            )
+            conn.execute(
+                """
+                INSERT OR IGNORE INTO download_history (
+                    download_id, model_id, model_version_id, model_name,
+                    version_name, thumbnail_url, status, error, file_path,
+                    bytes_downloaded, total_bytes, completed_at
+                ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                """,
+                (
+                    row["download_id"],
+                    row["model_id"],
+                    row["model_version_id"],
+                    row["model_name"],
+                    row["version_name"],
+                    row["thumbnail_url"],
+                    status,
+                    error,
+                    file_path,
+                    bytes_downloaded,
+                    total_bytes,
+                    now,
+                ),
+            )
+            conn.commit()
+        return dict(row)
+
+    async def pop_next_download(self) -> Optional[dict[str, Any]]:
+        """Atomically fetch and mark the next queued item as ``downloading``.
+
+        The item with the highest priority (and earliest ``added_at``
+        among ties) whose status is ``'queued'`` is selected, set to
+        ``'downloading'``, and returned as a dict.  Returns ``None`` if
+        the queue is empty.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            row = conn.execute(
+                """
+                SELECT * FROM download_queue
+                WHERE status = 'queued'
+                ORDER BY priority DESC, added_at ASC
+                LIMIT 1
+                """
+            ).fetchone()
+            if row is None:
+                return None
+
+            download_id = row["download_id"]
+            now = time.time()
+            conn.execute(
+                "UPDATE download_queue SET status = 'downloading', "
+                "started_at = COALESCE(started_at, ?) "
+                "WHERE download_id = ?",
+                (now, download_id),
+            )
+            conn.commit()
+            updated = conn.execute(
+                "SELECT * FROM download_queue WHERE download_id = ?",
+                (download_id,),
+            ).fetchone()
+
+        return dict(updated) if updated else None
+
+    # ------------------------------------------------------------------
+    # History methods
+    # ------------------------------------------------------------------
+
+    async def add_to_history(
+        self,
+        download_id: Optional[str] = None,
+        model_id: Optional[int] = None,
+        model_version_id: Optional[int] = None,
+        model_name: str = "",
+        version_name: str = "",
+        thumbnail_url: str = "",
+        status: str = "completed",
+        error: Optional[str] = None,
+        file_path: Optional[str] = None,
+        bytes_downloaded: int = 0,
+        total_bytes: Optional[int] = None,
+        is_already_exists: int = 0,
+    ) -> int:
+        """Insert a record into the download history.
+
+        Returns the ``id`` (AUTOINCREMENT primary key) of the newly
+        inserted row.
+        """
+        now = time.time()
+
+        async with self._lock:
+            conn = self._get_conn()
+            cursor = conn.execute(
+                """
+                INSERT INTO download_history (
+                    download_id, model_id, model_version_id, model_name,
+                    version_name, thumbnail_url, status, error, file_path,
+                    bytes_downloaded, total_bytes, completed_at, is_already_exists
+                ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+                """,
+                (
+                    download_id,
+                    model_id,
+                    model_version_id,
+                    model_name,
+                    version_name,
+                    thumbnail_url,
+                    status,
+                    error,
+                    file_path,
+                    bytes_downloaded,
+                    total_bytes,
+                    now,
+                    is_already_exists,
+                ),
+            )
+            conn.commit()
+        return cursor.lastrowid or 0
+
+    async def get_history(
+        self,
+        limit: int = 50,
+        offset: int = 0,
+        status_filter: Optional[str] = None,
+    ) -> dict[str, Any]:
+        """Return a page of download history entries.
+
+        Returns a dict with keys ``items``, ``total``, ``limit``, and
+        ``offset``.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+
+            if status_filter is not None:
+                count_row = conn.execute(
+                    "SELECT COUNT(*) AS cnt FROM download_history WHERE status = ?",
+                    (status_filter,),
+                ).fetchone()
+                rows = conn.execute(
+                    "SELECT * FROM download_history WHERE status = ? "
+                    "ORDER BY completed_at DESC LIMIT ? OFFSET ?",
+                    (status_filter, limit, offset),
+                ).fetchall()
+            else:
+                count_row = conn.execute(
+                    "SELECT COUNT(*) AS cnt FROM download_history"
+                ).fetchone()
+                rows = conn.execute(
+                    "SELECT * FROM download_history "
+                    "ORDER BY completed_at DESC LIMIT ? OFFSET ?",
+                    (limit, offset),
+                ).fetchall()
+
+        return {
+            "items": [dict(row) for row in rows],
+            "total": count_row["cnt"] if count_row else 0,
+            "limit": limit,
+            "offset": offset,
+        }
+
+    async def delete_history_item(
+        self, id: Optional[int] = None, download_id: Optional[str] = None
+    ) -> bool:
+        """Delete a single history entry by *download_id* (preferred) or *id*.
+
+        Returns ``True`` if a row was deleted.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            if download_id:
+                cursor = conn.execute(
+                    "DELETE FROM download_history WHERE download_id = ?",
+                    (download_id,),
+                )
+            elif id is not None:
+                cursor = conn.execute(
+                    "DELETE FROM download_history WHERE id = ?",
+                    (id,),
+                )
+            else:
+                return False
+            conn.commit()
+        return cursor.rowcount > 0
+
+    async def clear_history(
+        self,
+        status_filter: Optional[str] = None,
+        before_timestamp: Optional[float] = None,
+    ) -> int:
+        """Remove history entries matching the optional filters.
+
+        Both ``status_filter`` and ``before_timestamp`` can be combined
+        (AND logic).  Returns the number of deleted rows.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+
+            clauses: list[str] = []
+            params: list[Any] = []
+
+            if status_filter is not None:
+                clauses.append("status = ?")
+                params.append(status_filter)
+            if before_timestamp is not None:
+                clauses.append("completed_at < ?")
+                params.append(before_timestamp)
+
+            where = ""
+            if clauses:
+                where = " WHERE " + " AND ".join(clauses)
+
+            cursor = conn.execute(
+                f"DELETE FROM download_history{where}",
+                params,
+            )
+            conn.commit()
+        return cursor.rowcount
+
+    async def get_history_count(self, status_filter: Optional[str] = None) -> int:
+        """Return the number of history entries, optionally filtered by status."""
+        async with self._lock:
+            conn = self._get_conn()
+            if status_filter is not None:
+                row = conn.execute(
+                    "SELECT COUNT(*) AS cnt FROM download_history WHERE status = ?",
+                    (status_filter,),
+                ).fetchone()
+            else:
+                row = conn.execute(
+                    "SELECT COUNT(*) AS cnt FROM download_history"
+                ).fetchone()
+        return row["cnt"] if row else 0
+
+    # ------------------------------------------------------------------
+    # Retry
+    # ------------------------------------------------------------------
+
+    async def retry_from_history(
+        self,
+        item_id: Optional[int] = None,
+        download_id: Optional[str] = None,
+    ) -> Optional[dict[str, Any]]:
+        """Re-queue a failed or canceled download from history.
+
+        Looks up the history record by *download_id* (preferred) or
+        *item_id*.  If the status is ``failed`` or ``canceled`` a new
+        queue entry is created with the same model metadata and a fresh
+        download id, and the original history entry is **deleted** to
+        prevent exponential growth when the retried item is later
+        canceled or fails again and re-retried.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            if download_id:
+                row = conn.execute(
+                    "SELECT * FROM download_history WHERE download_id = ?",
+                    (download_id,),
+                ).fetchone()
+            elif item_id is not None:
+                row = conn.execute(
+                    "SELECT * FROM download_history WHERE id = ?",
+                    (item_id,),
+                ).fetchone()
+            else:
+                return None
+            if row is None:
+                return None
+            status = str(row["status"])
+            if status not in ("failed", "canceled"):
+                return None
+
+            import uuid
+
+            new_id = str(uuid.uuid4())
+            now = time.time()
+            conn.execute(
+                """
+                INSERT INTO download_queue (
+                    download_id, model_id, model_version_id, model_name,
+                    version_name, thumbnail_url, source, file_params,
+                    status, priority, added_at
+                ) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
+                """,
+                (
+                    new_id,
+                    row["model_id"],
+                    row["model_version_id"],
+                    row["model_name"],
+                    row["version_name"],
+                    row["thumbnail_url"],
+                    "retry",
+                    now,
+                ),
+            )
+            conn.execute(
+                "DELETE FROM download_history WHERE id = ?",
+                (row["id"],),
+            )
+            conn.commit()
+            queued = conn.execute(
+                "SELECT * FROM download_queue WHERE download_id = ?",
+                (new_id,),
+            ).fetchone()
+
+        return dict(queued) if queued else None
+
+    async def retry_all_failed(self) -> int:
+        """Re-queue all failed and canceled downloads from history.
+
+        Each history entry is **deleted** after being re-queued so that
+        repeated retry-all calls do not cause exponential growth.
+
+        Returns the number of items that were re-queued.
+        """
+        async with self._lock:
+            conn = self._get_conn()
+            rows = conn.execute(
+                "SELECT * FROM download_history WHERE status IN ('failed', 'canceled')"
+            ).fetchall()
+            if not rows:
+                return 0
+
+            import uuid
+
+            now = time.time()
+            count = 0
+            for row in rows:
+                new_id = str(uuid.uuid4())
+                conn.execute(
+                    """
+                    INSERT INTO download_queue (
+                        download_id, model_id, model_version_id, model_name,
+                        version_name, thumbnail_url, source, file_params,
+                        status, priority, added_at
+                    ) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
+                    """,
+                    (
+                        new_id,
+                        row["model_id"],
+                        row["model_version_id"],
+                        row["model_name"],
+                        row["version_name"],
+                        row["thumbnail_url"],
+                        "retry",
+                        now,
+                    ),
+                )
+                conn.execute(
+                    "DELETE FROM download_history WHERE id = ?",
+                    (row["id"],),
+                )
+                count += 1
+            conn.commit()
+
+        return count
+
+    # ------------------------------------------------------------------
+    # Stats
+    # ------------------------------------------------------------------
+
+    async def get_stats(self) -> dict[str, int]:
+        """Return aggregate counts across both tables.
+
+        Returns a dict with keys ``queued``, ``downloading``, ``paused``
+        (all from the queue table) and ``completed``, ``failed``,
+        ``canceled`` (all from the history table).
+        """
+        async with self._lock:
+            conn = self._get_conn()
+
+            queue_rows = conn.execute(
+                "SELECT status, COUNT(*) AS cnt FROM download_queue GROUP BY status"
+            ).fetchall()
+            queue_stats: dict[str, int] = {}
+            for row in queue_rows:
+                queue_stats[str(row["status"])] = row["cnt"]
+
+            history_rows = conn.execute(
+                "SELECT status, COUNT(*) AS cnt FROM download_history GROUP BY status"
+            ).fetchall()
+            history_stats: dict[str, int] = {}
+            for row in history_rows:
+                history_stats[str(row["status"])] = row["cnt"]
+
+        return {
+            "queued": queue_stats.get("queued", 0),
+            "downloading": queue_stats.get("downloading", 0),
+            "paused": queue_stats.get("paused", 0),
+            "completed": history_stats.get("completed", 0),
+            "failed": history_stats.get("failed", 0),
+            "canceled": history_stats.get("canceled", 0),
+        }
+
+    # ------------------------------------------------------------------
+    # Deduplication (one-time cleanup for bug #980)
+    # ------------------------------------------------------------------
+
+    async def deduplicate(self) -> dict[str, int]:
+        """Remove duplicate entries caused by the retry-amplification bug.
+
+        The bug (issue #980) caused the same download to appear N times in
+        both the queue and history tables when ``retry_all_failed`` was
+        called repeatedly without deleting the original history rows.
+
+        This method is called **once** when the singleton is first created.
+        It is idempotent — after the first run there will be no duplicates
+        to remove, so subsequent calls are a no-op.
+
+        Returns a dict with the count of removed rows per table.
+        """
+        result: dict[str, int] = {
+            "removed_history": 0,
+            "removed_queue": 0,
+            "removed_orphan_queue": 0,
+        }
+
+        async with self._lock:
+            conn = self._get_conn()
+
+            # 1. History: for each (model_id, model_version_id, status) triplet
+            #    keep only the row with the highest id (most recently inserted).
+            conn.execute("""
+                DELETE FROM download_history
+                WHERE id NOT IN (
+                    SELECT MAX(id)
+                    FROM download_history
+                    GROUP BY model_id, model_version_id, status
+                )
+            """)
+            result["removed_history"] = conn.execute(
+                "SELECT changes()"
+            ).fetchone()[0]
+
+            # 2. Cross-status dedup: for each (model_id, model_version_id),
+            #    keep only the entry with the highest-priority terminal status.
+            #    Priority: completed (3) > failed (2) > canceled (1).
+            #    This prevents the same model version from having both a
+            #    'failed' and a 'canceled' entry (or a 'completed' alongside
+            #    either) after the bug-created duplicates are removed.
+            conn.execute("""
+                DELETE FROM download_history
+                WHERE id NOT IN (
+                    SELECT dh.id
+                    FROM download_history dh
+                    INNER JOIN (
+                        SELECT model_id, model_version_id,
+                               MAX(CASE status
+                                   WHEN 'completed' THEN 3
+                                   WHEN 'failed' THEN 2
+                                   WHEN 'canceled' THEN 1
+                                   ELSE 0
+                               END) AS best_prio
+                        FROM download_history
+                        GROUP BY model_id, model_version_id
+                    ) best
+                    ON dh.model_id = best.model_id
+                   AND dh.model_version_id = best.model_version_id
+                   AND CASE dh.status
+                           WHEN 'completed' THEN 3
+                           WHEN 'failed' THEN 2
+                           WHEN 'canceled' THEN 1
+                           ELSE 0
+                       END = best.best_prio
+                    GROUP BY dh.model_id, dh.model_version_id
+                    HAVING dh.id = MAX(dh.id)
+                )
+            """)
+            result["removed_history"] += conn.execute(
+                "SELECT changes()"
+            ).fetchone()[0]
+
+            # 3. Queue: for each (model_id, model_version_id) keep only the
+            #    row with the latest added_at (most recently enqueued).
+            conn.execute("""
+                DELETE FROM download_queue
+                WHERE rowid NOT IN (
+                    SELECT MAX(rowid)
+                    FROM download_queue
+                    WHERE status IN ('queued', 'downloading', 'paused', 'waiting')
+                    GROUP BY model_id, model_version_id
+                )
+                AND status IN ('queued', 'downloading', 'paused', 'waiting')
+            """)
+            result["removed_queue"] = conn.execute(
+                "SELECT changes()"
+            ).fetchone()[0]
+
+            # 4. Remove orphaned queue entries — items that were re-queued
+            #    (source='retry') but whose model version already has a
+            #    terminal history entry. These are artifacts of the buggy
+            #    retry cycle that were never cleaned up.
+            conn.execute("""
+                DELETE FROM download_queue
+                WHERE source = 'retry'
+                AND (model_id, model_version_id) IN (
+                    SELECT model_id, model_version_id
+                    FROM download_history
+                    WHERE status IN ('failed', 'canceled')
+                )
+                AND status IN ('queued', 'waiting')
+            """)
+            result["removed_orphan_queue"] = conn.execute(
+                "SELECT changes()"
+            ).fetchone()[0]
+
+            conn.commit()
+
+        logger.info(
+            "Deduplicate: removed %s history rows, %s queue rows, "
+            "%s orphaned queue rows",
+            result["removed_history"],
+            result["removed_queue"],
+            result["removed_orphan_queue"],
+        )
+        return result

py/services/downloaded_version_history_service.py

@@ -96,6 +96,21 @@ class DownloadedVersionHistoryService:
     def get_database_path(self) -> str:
         return self._db_path
 
+    def close(self) -> None:
+        """Close the persistent SQLite connection, if open.
+
+        This is called before plugin update operations to release the
+        database file lock on Windows, allowing ``shutil.rmtree()`` to
+        succeed when the cache resides inside the plugin directory.
+        """
+        if self._conn is not None:
+            try:
+                self._conn.close()
+            except Exception:
+                pass
+            finally:
+                self._conn = None
+
     def _get_active_library_name(self) -> str | None:
         try:
             value = self._settings.get_active_library_name()

py/services/downloader.py

@@ -13,6 +13,7 @@ This module provides a centralized download service with:
 import os
 import logging
 import asyncio
+import ssl
 import aiohttp
 from collections import deque
 from dataclasses import dataclass
@@ -31,6 +32,44 @@ from .errors import RateLimitError
 logger = logging.getLogger(__name__)
 
 
+def is_ssl_cert_verify_error(exc: BaseException) -> bool:
+    """Check if an exception represents an SSL certificate verification failure.
+
+    Matches ``ssl.SSLCertVerificationError``, ``aiohttp.ClientConnectorCertificateError``
+    (which wraps the former), and falls back to the standard OpenSSL error text.
+    """
+    if isinstance(exc, ssl.SSLCertVerificationError):
+        return True
+    cert_error = getattr(exc, "certificate_error", None)
+    if isinstance(cert_error, ssl.SSLCertVerificationError):
+        return True
+    return "CERTIFICATE_VERIFY_FAILED" in str(exc)
+
+
+def _parse_retry_after(value: str) -> int:
+    """Parse a Retry-After header value into seconds.
+
+    Supports both integer seconds and HTTP-date formats.
+    Returns a default of 60 seconds on invalid/missing input.
+    """
+    if not value or not value.strip():
+        return 60
+
+    value = value.strip()
+    try:
+        return max(1, int(value))
+    except ValueError:
+        pass
+
+    try:
+        parsed = parsedate_to_datetime(value)
+        now = datetime.now().astimezone()
+        delta = (parsed - now).total_seconds()
+        return max(1, int(delta))
+    except (ValueError, OverflowError, OSError):
+        return 60
+
+
 @dataclass(frozen=True)
 class DownloadProgress:
     """Snapshot of a download transfer at a moment in time."""
@@ -231,17 +270,19 @@ class Downloader:
 
         Note: This is private and caller MUST hold self._session_lock.
         """
-        # Close existing session if any
-        if self._session is not None:
-            try:
-                await self._session.close()
-            except Exception as e:  # pragma: no cover
-                logger.warning(f"Error closing previous session: {e}")
-            finally:
-                self._session = None
+        # Snapshot and clear old session reference before creating the new
+        # one.  This ensures self._session is always valid (or None, which
+        # triggers a fresh creation) and avoids a race where concurrent
+        # requests hold a reference to a session whose connector has been
+        # torn down by a premature close() call — the root cause of the
+        # intermittent "NoneType has no attribute connect" crash.
+        old_session = self._session
+        self._session = None
 
         # Check for app-level proxy settings
-        proxy_url = None
+        proxy_url = None  # http(s) proxy, passed via the per-request `proxy=` kwarg
+        socks_proxy_url = None  # SOCKS proxy, handled via aiohttp-socks connector
+        app_proxy_active = False
         settings_manager = get_settings_manager()
         if settings_manager.get("proxy_enabled", False):
             proxy_host = settings_manager.get("proxy_host", "").strip()
@@ -253,9 +294,19 @@ class Downloader:
             if proxy_host and proxy_port:
                 # Build proxy URL
                 if proxy_username and proxy_password:
-                    proxy_url = f"{proxy_type}://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"
+                    full_proxy_url = f"{proxy_type}://{proxy_username}:{proxy_password}@{proxy_host}:{proxy_port}"
                 else:
-                    proxy_url = f"{proxy_type}://{proxy_host}:{proxy_port}"
+                    full_proxy_url = f"{proxy_type}://{proxy_host}:{proxy_port}"
+
+                app_proxy_active = True
+                # aiohttp cannot tunnel SOCKS via the per-request `proxy=` kwarg
+                # (it would send HTTP to the SOCKS port and fail parsing the
+                # SOCKS handshake reply). SOCKS must be handled by an
+                # aiohttp-socks ProxyConnector instead.
+                if proxy_type.startswith("socks"):
+                    socks_proxy_url = full_proxy_url
+                else:
+                    proxy_url = full_proxy_url
 
                 logger.debug(
                     f"Using app-level proxy: {proxy_type}://{proxy_host}:{proxy_port}"
@@ -265,14 +316,41 @@ class Downloader:
             logger.debug(
                 "Proxy mode: system-level proxy (trust_env) will be used if configured in environment."
             )
+        # Build SSL context: prefer certifi's CA bundle for broader
+        # CA coverage across different Python environments (especially
+        # embedded/compatibility Python builds).
+        try:
+            import certifi  # type: ignore[import-untyped]
+
+            ca_path = certifi.where()
+            ssl_context = ssl.create_default_context(cafile=ca_path)
+            logger.debug("SSL: using certifi CA bundle at %s", ca_path)
+        except (ImportError, FileNotFoundError, ValueError, OSError):
+            ssl_context = ssl.create_default_context()
+            logger.debug("SSL: certifi unavailable; using system default CA bundle")
+
         # Optimize TCP connection parameters
-        connector = aiohttp.TCPConnector(
-            ssl=True,
+        connector_kwargs = dict(
+            ssl=ssl_context,
             limit=8,  # Concurrent connections
             ttl_dns_cache=300,  # DNS cache timeout
             force_close=False,  # Keep connections for reuse
             enable_cleanup_closed=True,
         )
+        if socks_proxy_url:
+            # Route all traffic through the SOCKS proxy via aiohttp-socks. The
+            # connector tunnels every connection, so no per-request `proxy=` is
+            # used (and must not be — see self._proxy_url below).
+            try:
+                from aiohttp_socks import ProxyConnector
+            except ImportError as e:  # pragma: no cover
+                raise RuntimeError(
+                    "A SOCKS proxy is configured but the 'aiohttp-socks' package "
+                    "is not installed. Install it with: pip install aiohttp-socks"
+                ) from e
+            connector = ProxyConnector.from_url(socks_proxy_url, **connector_kwargs)
+        else:
+            connector = aiohttp.TCPConnector(**connector_kwargs)
 
         # Configure timeout parameters
         timeout = aiohttp.ClientTimeout(
@@ -283,15 +361,24 @@ class Downloader:
 
         self._session = aiohttp.ClientSession(
             connector=connector,
-            trust_env=proxy_url
-            is None,  # Only use system proxy if no app-level proxy is set
+            # Only fall back to system/env proxy when no app-level proxy is active
+            trust_env=not app_proxy_active,
             timeout=timeout,
         )
 
-        # Store proxy URL for use in requests
+        # Store proxy URL for per-request use. Stays None for SOCKS because the
+        # ProxyConnector already tunnels everything; passing proxy= for SOCKS
+        # would re-trigger the original aiohttp parse error.
         self._proxy_url = proxy_url
         self._session_created_at = datetime.now()
 
+        # Close the previous session now that the replacement is live.
+        if old_session is not None:
+            try:
+                await old_session.close()
+            except Exception as e:  # pragma: no cover
+                logger.warning(f"Error closing previous session: {e}")
+
         logger.debug(
             "Created new HTTP session with proxy settings. App-level proxy: %s, System-level proxy (trust_env): %s",
             bool(proxy_url),
@@ -673,7 +760,8 @@ class Downloader:
                             else:
                                 resume_offset = 0
                                 total_size = 0
-                            await self._create_session()
+                            async with self._session_lock:
+                                await self._create_session()
                             continue
 
                         return False, integrity_error
@@ -736,6 +824,17 @@ class Downloader:
                 DownloadRestartRequested,
             ) as e:
                 retry_count += 1
+
+                if is_ssl_cert_verify_error(e):
+                    logger.error(
+                        "SSL certificate verification failed when connecting to %s. "
+                        "This is usually caused by an outdated CA certificate bundle "
+                        "in the Python environment. Recommended fixes:\n"
+                        "  1. pip install --upgrade certifi\n"
+                        "  2. pip install pip-system-certs",
+                        url,
+                    )
+
                 logger.warning(
                     f"Network error during download (attempt {retry_count}/{self.max_retries + 1}): {e}"
                 )
@@ -752,7 +851,8 @@ class Downloader:
                         logger.info(f"Will resume from byte {resume_offset}")
 
                     # Refresh session to get new connection
-                    await self._create_session()
+                    async with self._session_lock:
+                        await self._create_session()
                     continue
                 else:
                     logger.error(f"Max retries exceeded for download: {e}")
@@ -844,6 +944,19 @@ class Downloader:
                 elif response.status == 404:
                     error_msg = "File not found"
                     return False, error_msg, None
+                elif response.status == 429:
+                    raw_retry_after = response.headers.get("Retry-After")
+                    retry_after = _parse_retry_after(raw_retry_after or "")
+                    if raw_retry_after:
+                        logger.warning(
+                            "Rate limited (429) for %s, Retry-After: %ss", url, retry_after
+                        )
+                    else:
+                        logger.warning(
+                            "Rate limited (429) for %s, no Retry-After header; defaulting to %ss",
+                            url, retry_after,
+                        )
+                    return False, f"Rate limited (429), retry after {retry_after}s", None
                 else:
                     error_msg = f"Download failed with status {response.status}"
                     return False, error_msg, None

py/services/embedding_service.py

@@ -1,6 +1,6 @@
 import os
 import logging
-from typing import Dict
+from typing import Dict, Optional
 
 from .base_model_service import BaseModelService
 from .auto_tag_service import extract_auto_tags
@@ -21,20 +21,37 @@ class EmbeddingService(BaseModelService):
         """
         super().__init__("embedding", scanner, EmbeddingMetadata, update_service=update_service)
     
-    async def format_response(self, embedding_data: Dict) -> Dict:
-        """Format Embedding data for API response"""
+    async def format_response(self, embedding_data: Dict) -> Optional[Dict]:
+        """Format Embedding data for API response.
+
+        Returns None when the entry is missing critical fields (corrupted cache
+        row), so the handler layer can filter it out. See issue #730.
+        """
+        # Guard against corrupted cache entries missing critical fields
+        file_path = embedding_data.get("file_path")
+        if not file_path or not isinstance(file_path, str):
+            logger.warning(
+                "Skipping corrupted embedding entry (missing file_path): %s",
+                embedding_data.get("file_name", "<unknown>"),
+            )
+            return None
+
         # Get sub_type from cache entry (new canonical field)
         sub_type = embedding_data.get("sub_type", "embedding")
-        
+
+        file_name = embedding_data.get("file_name") or ""
+        model_name = embedding_data.get("model_name") or file_name
+        folder = embedding_data.get("folder") or ""
+
         return {
-            "model_name": embedding_data["model_name"],
-            "file_name": embedding_data["file_name"],
+            "model_name": model_name,
+            "file_name": file_name,
             "preview_url": config.get_preview_static_url(embedding_data.get("preview_url", "")),
             "preview_nsfw_level": embedding_data.get("preview_nsfw_level", 0),
             "base_model": embedding_data.get("base_model", ""),
-            "folder": embedding_data["folder"],
+            "folder": folder,
             "sha256": embedding_data.get("sha256", ""),
-            "file_path": embedding_data["file_path"].replace(os.sep, "/"),
+            "file_path": file_path.replace(os.sep, "/"),
             "file_size": embedding_data.get("size", 0),
             "modified": embedding_data.get("modified", ""),
             "tags": embedding_data.get("tags", []),
@@ -48,6 +65,8 @@ class EmbeddingService(BaseModelService):
             "skip_metadata_refresh": bool(embedding_data.get("skip_metadata_refresh", False)),
             "civitai": self.filter_civitai_data(embedding_data.get("civitai", {}), minimal=True),
             "auto_tags": embedding_data.get("auto_tags") or extract_auto_tags(embedding_data),
+            "version_count": embedding_data.get("version_count"),
+            "hf_url": embedding_data.get("hf_url", ""),
         }
     
     def find_duplicate_hashes(self) -> Dict:

py/services/errors.py

@@ -25,3 +25,21 @@ class ResourceNotFoundError(RuntimeError):
 
     pass
 
+
+class LLMNotConfiguredError(RuntimeError):
+    """Raised when an LLM-dependent operation is attempted but no provider is configured."""
+
+    pass
+
+
+class LLMRateLimitError(RateLimitError):
+    """Raised when the LLM provider rejects a request due to rate limiting."""
+
+    pass
+
+
+class LLMResponseError(RuntimeError):
+    """Raised when the LLM returns an unparseable or schema-invalid response."""
+
+    pass
+

py/services/llm_service.py

@@ -0,0 +1,729 @@
+"""Centralized LLM API client with BYOK (bring-your-own-key) provider support.
+
+Reads provider configuration from :class:`SettingsManager` and makes
+OpenAI-compatible ``/chat/completions`` calls.  Supports any provider that
+implements the OpenAI Chat Completions API surface area (OpenAI, Ollama,
+vLLM, LM Studio, etc.).
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import logging
+from typing import Any, Dict, List, Optional
+
+import aiohttp
+
+from .errors import LLMNotConfiguredError, LLMRateLimitError, LLMResponseError
+
+logger = logging.getLogger(__name__)
+
+# ---------------------------------------------------------------------------
+# Model catalog sourced from opencode's maintained model registry.
+# maps provider_id -> list of model IDs.
+# ---------------------------------------------------------------------------
+_MODEL_CATALOG_URL = "https://models.dev/api.json"
+
+# In-memory cache: maps provider slug -> list of model ID strings.
+_catalog_cache: Optional[Dict[str, List[str]]] = None
+
+# Per-model max output token limits parsed from the catalog.
+# ``{provider_id: {model_id: max_output_tokens}}``.
+_model_output_limits: Dict[str, Dict[str, int]] = {}
+
+_CATALOG_TIMEOUT = aiohttp.ClientTimeout(total=30)
+
+
+async def _load_model_catalog() -> Dict[str, List[str]]:
+    """Fetch and parse the model catalog.
+
+    Returns ``{provider_id: [model_id, ...]}`` and also populates
+    :data:`_model_output_limits` with per-model ``limit.output`` values
+    for use by :func:`_get_model_max_output`.
+
+    The JSON at ``_MODEL_CATALOG_URL`` is a dict keyed by provider slug; each
+    value has a ``models`` sub-dict keyed by model ID.  The result is cached
+    in memory after the first successful fetch.
+    Subsequent calls return the cached data immediately.
+    """
+    global _catalog_cache, _model_output_limits
+    if _catalog_cache is not None:
+        return _catalog_cache
+
+    try:
+        async with aiohttp.ClientSession(timeout=_CATALOG_TIMEOUT) as session:
+            async with session.get(_MODEL_CATALOG_URL) as resp:
+                if resp.status != 200:
+                    logger.warning("Model catalog returned HTTP %s", resp.status)
+                    return _catalog_cache or {}
+                data = await resp.json()
+    except (aiohttp.ClientError, asyncio.TimeoutError, json.JSONDecodeError) as exc:
+        logger.warning("Failed to fetch model catalog: %s", exc)
+        return _catalog_cache or {}
+
+    if not isinstance(data, dict):
+        logger.warning("Model catalog is not a dict, got %s", type(data).__name__)
+        return _catalog_cache or {}
+
+    result: Dict[str, List[str]] = {}
+    output_limits: Dict[str, Dict[str, int]] = {}
+    for provider_id, provider_info in data.items():
+        if not isinstance(provider_info, dict):
+            continue
+        models_dict = provider_info.get("models")
+        if not isinstance(models_dict, dict):
+            continue
+        model_ids: List[str] = []
+        provider_limits: Dict[str, int] = {}
+        for mid, model_info in models_dict.items():
+            if not isinstance(mid, str):
+                continue
+            model_ids.append(mid)
+            if isinstance(model_info, dict):
+                limit = model_info.get("limit")
+                if isinstance(limit, dict):
+                    output = limit.get("output")
+                    if isinstance(output, (int, float)) and output > 0:
+                        provider_limits[mid] = int(output)
+        if model_ids:
+            result[provider_id] = model_ids
+        if provider_limits:
+            output_limits[provider_id] = provider_limits
+
+    _catalog_cache = result
+    _model_output_limits = output_limits
+    logger.debug(
+        "Loaded model catalog: %d providers, %d total models "
+        "(%d providers have output limits)",
+        len(result),
+        sum(len(m) for m in result.values()),
+        len(output_limits),
+    )
+    return result
+
+
+def _get_model_max_output(provider: str, model: str) -> Optional[int]:
+    """Return the model's max output token limit from the catalog, or ``None``.
+
+    Returns ``None`` when the provider or model is not found in the catalog
+    (e.g. local Ollama models, custom models, or user-typed model names).
+    Callers should fall back to a safe default.
+    """
+    return _model_output_limits.get(provider, {}).get(model)
+
+
+# Short timeout for Ollama's local API
+_OLLAMA_API_TIMEOUT = aiohttp.ClientTimeout(total=8)
+
+
+async def fetch_ollama_models(api_base: str) -> List[str]:
+    """Fetch locally available models from a running Ollama instance.
+
+    Uses Ollama's OpenAI-compatible ``GET {api_base}/models`` endpoint.
+    Returns an empty list if Ollama is not reachable (not running).
+    """
+    url = f"{api_base.rstrip('/')}/models"
+    try:
+        async with aiohttp.ClientSession(timeout=_OLLAMA_API_TIMEOUT) as session:
+            async with session.get(url) as resp:
+                if resp.status != 200:
+                    logger.debug("Ollama API returned HTTP %s from %s", resp.status, api_base)
+                    return []
+                data = await resp.json()
+    except (aiohttp.ClientError, asyncio.TimeoutError, json.JSONDecodeError) as exc:
+        logger.debug("Ollama not reachable at %s: %s", api_base, exc)
+        return []
+
+    raw = data.get("data") if isinstance(data, dict) else None
+    if not isinstance(raw, list):
+        return []
+
+    return [
+        str(entry["id"]) for entry in raw
+        if isinstance(entry, dict) and isinstance(entry.get("id"), str)
+    ]
+
+
+async def get_provider_model_ids(provider_id: str) -> List[str]:
+    """Return the list of known model IDs for *provider_id* from the catalog.
+
+    The catalog is loaded on first call and cached thereafter.  If the
+    provider is not found an empty list is returned (never raises).
+    """
+    catalog = await _load_model_catalog()
+    return catalog.get(provider_id, [])
+
+
+async def get_all_provider_models(
+    provider_ids: List[str],
+) -> Dict[str, List[str]]:
+    """Return model lists for a subset of providers in one call.
+
+    Loads the catalog (cached) and returns only the requested providers.
+    Handy for embedding lightweight data into the template context.
+    """
+    catalog = await _load_model_catalog()
+    return {
+        pid: catalog.get(pid, [])
+        for pid in provider_ids
+    }
+
+
+# Provider preset definitions.
+# Each entry contains display metadata and defaults for the UI.
+# The key is the internal provider id stored in ``llm_provider``.
+# Models are NOT listed here — they come from the opencode model catalog at
+# runtime (see :func:`get_provider_model_ids`).
+PROVIDER_PRESETS: Dict[str, Dict[str, Any]] = {
+    "openai": {
+        "name": "OpenAI",
+        "api_base": "https://api.openai.com/v1",
+        "requires_key": True,
+    },
+    "ollama": {
+        "name": "Ollama (local)",
+        "api_base": "http://localhost:11434/v1",
+        "requires_key": False,
+    },
+    "deepseek": {
+        "name": "DeepSeek",
+        "api_base": "https://api.deepseek.com/v1",
+        "requires_key": True,
+    },
+    "groq": {
+        "name": "Groq",
+        "api_base": "https://api.groq.com/openai/v1",
+        "requires_key": True,
+    },
+    "openrouter": {
+        "name": "OpenRouter",
+        "api_base": "https://openrouter.ai/api/v1",
+        "requires_key": True,
+    },
+    "opencode-go": {
+        "name": "OpenCode Go",
+        "api_base": "https://opencode.ai/zen/go/v1",
+        "requires_key": True,
+    },
+    # "custom" is handled specially (no preset api_base, requires user input)
+}
+
+# Legacy lookup derived from PROVIDER_PRESETS for backward compat.
+_PROVIDER_DEFAULTS: Dict[str, str] = {
+    pid: info["api_base"]
+    for pid, info in PROVIDER_PRESETS.items()
+    if info.get("api_base")
+}
+
+# Request timeout for LLM calls (seconds)
+_LLM_TIMEOUT = aiohttp.ClientTimeout(total=120)
+
+
+class LLMService:
+    """Centralized LLM API client.
+
+    All LLM-based enrichment features call through this service so
+    that BYOK config, retry logic, and error handling live in one place.
+    """
+
+    _instance: Optional["LLMService"] = None
+    _lock: asyncio.Lock = asyncio.Lock()
+
+    def __init__(self, settings_service) -> None:
+        self._settings = settings_service
+
+    # ------------------------------------------------------------------
+    # Singleton access
+    # ------------------------------------------------------------------
+
+    @classmethod
+    async def get_instance(cls) -> "LLMService":
+        """Return the lazily-initialised global ``LLMService`` instance."""
+
+        if cls._instance is None:
+            async with cls._lock:
+                if cls._instance is None:
+                    from .settings_manager import get_settings_manager
+
+                    cls._instance = cls(get_settings_manager())
+                    # Start preloading the model catalog in the background so
+                    # the settings UI never blocks on it.  The catalog is
+                    # cached after the first fetch (see _load_model_catalog).
+                    asyncio.create_task(_load_model_catalog())
+        return cls._instance
+
+    @classmethod
+    def reset_instance(cls) -> None:
+        """Reset the cached singleton — primarily for tests."""
+
+        cls._instance = None
+
+    # ------------------------------------------------------------------
+    # Configuration helpers
+    # ------------------------------------------------------------------
+
+    def _get_config(self) -> Dict[str, Any]:
+        """Read the current LLM configuration from settings."""
+
+        return {
+            "provider": self._settings.get("llm_provider", "openai"),
+            "api_key": self._settings.get("llm_api_key", ""),
+            "api_base": self._settings.get("llm_api_base", ""),
+            "model": self._settings.get("llm_model", ""),
+        }
+
+    @staticmethod
+    def _provider_requires_key(provider: str) -> bool:
+        """Return ``False`` when the given provider id does not need an API key."""
+        preset = PROVIDER_PRESETS.get(provider, {})
+        return bool(preset.get("requires_key", True))
+
+    def is_configured(self) -> bool:
+        """Return ``True`` when the LLM provider is minimally configured.
+
+        A provider is considered configured when ``llm_model`` is set,
+        an API key is configured for providers that require one (e.g.
+        Ollama does not), and an API base URL is set for providers that
+        have no preset default (e.g. ``custom``).
+        """
+
+        cfg = self._get_config()
+        has_model = bool(cfg["model"])
+        has_key = bool(cfg["api_key"]) or not self._provider_requires_key(cfg["provider"])
+        has_base = bool(cfg["api_base"]) or bool(_PROVIDER_DEFAULTS.get(cfg["provider"]))
+        return has_model and has_key and has_base
+
+    def _resolve_api_base(self, provider: str, api_base: str) -> str:
+        """Resolve the API base URL for the given provider.
+
+        If ``api_base`` is explicitly set (non-empty), it takes priority.
+        Otherwise the default from :data:`PROVIDER_PRESETS` is used.
+        """
+
+        if api_base:
+            return api_base.rstrip("/")
+        return _PROVIDER_DEFAULTS.get(provider, "").rstrip("/")
+
+    def _build_headers(self, api_key: str) -> Dict[str, str]:
+        """Build HTTP headers for the LLM API request."""
+
+        headers = {"Content-Type": "application/json"}
+        if api_key:
+            headers["Authorization"] = f"Bearer {api_key}"
+        return headers
+
+    def _ensure_configured(self) -> Dict[str, Any]:
+        """Validate configuration and return it, or raise.
+
+        A provider is considered configured when ``llm_model`` is set,
+        an API key is configured for providers that require one, and
+        an API base URL is set for providers without a preset default.
+        """
+
+        cfg = self._get_config()
+        has_model = bool(cfg["model"])
+        needs_key = self._provider_requires_key(cfg["provider"])
+        has_key = bool(cfg["api_key"]) or not needs_key
+        has_base = bool(cfg["api_base"]) or bool(_PROVIDER_DEFAULTS.get(cfg["provider"]))
+        if not (has_model and has_key and has_base):
+            parts = []
+            if not has_model:
+                parts.append("No LLM model specified")
+            if not has_key and needs_key:
+                parts.append("No LLM API key configured")
+            if not has_base:
+                parts.append(
+                    f"No API base URL for provider '{cfg['provider']}'"
+                )
+            detail = "; ".join(parts) if parts else "LLM provider is not configured"
+            raise LLMNotConfiguredError(
+                f"{detail}. Configure it in Settings → AI Provider."
+            )
+        return cfg
+
+    # ------------------------------------------------------------------
+    # Core API call
+    # ------------------------------------------------------------------
+
+    async def chat_completion(
+        self,
+        *,
+        messages: List[Dict[str, str]],
+        model: Optional[str] = None,
+        temperature: float = 0.3,
+        response_format: Optional[Dict[str, Any]] = None,
+        max_tokens: Optional[int] = None,
+        retry_on_rate_limit: bool = True,
+    ) -> Dict[str, Any]:
+        """Call the configured LLM provider's ``/chat/completions`` endpoint.
+
+        Args:
+            messages: OpenAI-format message list
+            model: Override the configured model name
+            temperature: Sampling temperature
+            response_format: Optional ``{"type": "json_object"}`` for structured output
+            max_tokens: Optional max output tokens
+            retry_on_rate_limit: Retry once after a 429 with backoff
+
+        Returns:
+            Dict with ``content`` (str), ``usage`` (dict), ``model`` (str)
+
+        Raises:
+            LLMNotConfiguredError: Provider not enabled / missing config
+            LLMRateLimitError: Rate limited and retry exhausted
+            LLMResponseError: Non-200 response or parse failure
+        """
+
+        cfg = self._ensure_configured()
+        api_base = self._resolve_api_base(cfg["provider"], cfg["api_base"])
+        model_name = model or cfg["model"]
+
+        is_ollama = cfg["provider"] == "ollama"
+
+        if is_ollama:
+            # Use Ollama's native /api/chat endpoint which does NOT expose
+            # a separate reasoning/thinking field (the model's full output
+            # lands directly in message.content).  The OpenAI-compatible
+            # endpoint splits thinking into the "reasoning" field, making
+            # content empty when thinking consumes all available tokens.
+            base = api_base.rstrip("/")
+            if base.endswith("/v1"):
+                base = base[:-3]
+            url = f"{base}/api/chat"
+        else:
+            url = f"{api_base}/chat/completions"
+
+        payload: Dict[str, Any]
+        if is_ollama:
+            payload = {
+                "model": model_name,
+                "messages": messages,
+                "stream": False,
+                # Suppress separate thinking trace — thinking still happens
+                # internally (accuracy preserved) but output goes directly to
+                # message.content instead of being split across content +
+                # thinking.  Without this the model can exhaust num_predict
+                # on thinking alone and leave content empty.
+                "think": False,
+                "options": {
+                    "temperature": temperature,
+                    # 8K context is sufficient for metadata enrichment
+                    # (prompt ~2-5K, output ~0.2-1K tokens).  The old 32K
+                    # value was excessive for this use case and increased
+                    # Ollama VRAM usage unnecessarily.
+                    "num_ctx": 8192,
+                },
+            }
+            if response_format is not None:
+                payload["format"] = "json"
+            if max_tokens is not None:
+                payload["options"]["num_predict"] = max_tokens
+        else:
+            payload = {
+                "model": model_name,
+                "messages": messages,
+                "temperature": temperature,
+            }
+            if response_format is not None:
+                payload["response_format"] = response_format
+            if max_tokens is not None:
+                payload["max_tokens"] = max_tokens
+
+        if is_ollama:
+            logger.info(
+                "Ollama request: model=%s num_ctx=%s num_predict=%s format=%s think=%s",
+                payload.get("model"),
+                payload.get("options", {}).get("num_ctx"),
+                payload.get("options", {}).get("num_predict"),
+                payload.get("format", "none"),
+                payload.get("think"),
+            )
+
+        headers = self._build_headers(cfg["api_key"])
+
+        attempt = 0
+        max_attempts = 2 if retry_on_rate_limit else 1
+        while attempt < max_attempts:
+            attempt += 1
+            try:
+                async with aiohttp.ClientSession(timeout=_LLM_TIMEOUT) as session:
+                    async with session.post(
+                        url, json=payload, headers=headers
+                    ) as resp:
+                        if resp.status == 429:
+                            if attempt < max_attempts:
+                                retry_after = float(
+                                    resp.headers.get("Retry-After", "5")
+                                )
+                                logger.warning(
+                                    "LLM rate limited, retrying after %.1fs",
+                                    retry_after,
+                                )
+                                await asyncio.sleep(retry_after)
+                                continue
+                            raise LLMRateLimitError(
+                                f"LLM provider rate limited (HTTP 429)",
+                                provider=cfg["provider"],
+                            )
+
+                        if resp.status != 200:
+                            body = await resp.text()
+                            raise LLMResponseError(
+                                f"LLM API returned HTTP {resp.status}: "
+                                f"{body[:500]}"
+                            )
+
+                        data = await resp.json()
+
+            except aiohttp.ClientError as exc:
+                raise LLMResponseError(f"Network error calling LLM API: {exc}") from exc
+
+            # Parse response
+            try:
+                if is_ollama:
+                    content = (data.get("message") or {}).get("content") or ""
+                    usage = {"completion_tokens": data.get("eval_count", 0)}
+                    finish_reason = data.get("done_reason", "")
+                    if not content:
+                        logger.warning(
+                            "LLM returned empty content. Provider=ollama, "
+                            "done_reason=%s, eval_count=%s",
+                            finish_reason,
+                            data.get("eval_count", 0),
+                        )
+                else:
+                    content = data["choices"][0]["message"].get("content") or ""
+                    usage = data.get("usage", {})
+                    if not content:
+                        logger.warning(
+                            "LLM returned empty content. Full response truncated: %s",
+                            json.dumps(data, ensure_ascii=False)[:1000],
+                        )
+                return {
+                    "content": content,
+                    "usage": usage,
+                    "model": data.get("model", model_name),
+                }
+            except (KeyError, IndexError) as exc:
+                raise LLMResponseError(
+                    f"Unexpected LLM response structure: {json.dumps(data)[:500]}"
+                ) from exc
+
+        # Should not reach here, but satisfy type checker
+        raise LLMRateLimitError("Rate limit retry exhausted", provider=cfg["provider"])
+
+    # ------------------------------------------------------------------
+    # Structured output convenience
+    # ------------------------------------------------------------------
+
+    async def chat_completion_json(
+        self,
+        *,
+        system_prompt: str,
+        user_prompt: str,
+        model: Optional[str] = None,
+        temperature: float = 0.3,
+        max_tokens: Optional[int] = None,
+    ) -> Dict[str, Any]:
+        """Call the LLM with ``response_format=json_object`` and return parsed JSON.
+
+        ``max_tokens`` is resolved in this order:
+        1. Explicit caller-supplied ``max_tokens``
+        2. Per-model ``limit.output`` from the model catalog
+        3. A safe default of 4096 (sufficient for metadata enrichment)
+
+        If the response content is empty or not valid JSON, attempts
+        :func:`_try_salvage_json` before raising.
+
+        Args:
+            system_prompt: System-level instructions
+            user_prompt: User-level query
+            model: Override the configured model name
+            temperature: Sampling temperature
+            max_tokens: Optional max output tokens
+
+        Returns:
+            Parsed JSON dict from the LLM response
+
+        Raises:
+            LLMNotConfiguredError: Provider not configured
+            LLMRateLimitError: Rate limited
+            LLMResponseError: Empty response or JSON parse failure
+        """
+
+        messages = [
+            {"role": "system", "content": system_prompt},
+            {"role": "user", "content": user_prompt},
+        ]
+
+        # Resolve max_tokens: caller override → catalog lookup → safe default
+        if max_tokens is None:
+            cfg = self._get_config()
+            effective_max = _get_model_max_output(cfg["provider"], cfg["model"])
+        else:
+            effective_max = max_tokens
+        if effective_max is None:
+            effective_max = 4096
+
+        # Use json_schema (not json_object) for broader provider compatibility:
+        # LM Studio and some other OpenAI-compatible servers reject
+        # json_object but accept json_schema.  {"type": "object"} is
+        # functionally equivalent — it accepts any JSON object without
+        # constraining specific fields.
+        response_format = {
+            "type": "json_schema",
+            "json_schema": {
+                "name": "metadata",
+                "schema": {"type": "object"},
+            },
+        }
+
+        try:
+            result = await self.chat_completion(
+                messages=messages,
+                model=model,
+                temperature=temperature,
+                response_format=response_format,
+                max_tokens=effective_max,
+            )
+        except LLMResponseError as e:
+            # Only fall back when the provider rejects the response_format
+            # type value (e.g. "'response_format.type' must be...").  Avoid
+            # catching unrelated 400 errors whose body happens to mention
+            # "response_format" (e.g. "model does not support
+            # response_format restrictions on this endpoint").
+            if "'response_format.type'" not in str(e).lower():
+                raise
+            logger.info(
+                "Provider rejected response_format, retrying without it. "
+                "Falling back to prompt-only JSON mode. Error: %s",
+                e,
+            )
+            result = await self.chat_completion(
+                messages=messages,
+                model=model,
+                temperature=temperature,
+                response_format=None,
+                max_tokens=effective_max,
+            )
+
+        content = result.get("content", "") or ""
+        if not content:
+            raise LLMResponseError(
+                "LLM returned empty content. "
+                f"Raw response: {json.dumps(result)[:500]}"
+            )
+
+        try:
+            parsed = json.loads(content)
+            logger.debug(
+                "LLM raw content: %s",
+                json.dumps(parsed, ensure_ascii=False)[:2000],
+            )
+            return parsed
+        except (json.JSONDecodeError, TypeError) as exc:
+            logger.info(
+                "LLM raw response (first 800 chars): %s",
+                content[:800],
+            )
+
+        # Last resort: attempt to salvage partial/truncated JSON
+        salvaged = _try_salvage_json(content)
+        if salvaged is not None:
+            logger.warning(
+                "LLM JSON salvaged from partial content (%d chars raw)",
+                len(content),
+            )
+            return salvaged
+
+        raise LLMResponseError(
+            f"LLM response could not be parsed as JSON: {content[:200]}"
+        )
+
+
+def _try_salvage_json(raw: str) -> Dict[str, Any] | None:
+    """Attempt to repair and parse a truncated JSON string.
+
+    Handles common truncation patterns:
+
+    * Incomplete string value at the end (``"foo`` → ``"foo"``)
+    * Missing closing ``}`` or ``]`` (respecting nesting order)
+    * Trailing comma before closing bracket
+    * Extra text after the JSON object (e.g. markdown fences)
+
+    Returns the parsed dict on success, ``None`` if repair is impossible.
+    """
+    if not raw:
+        return None
+
+    text = raw.strip()
+
+    # Strip markdown fences if the LLM wrapped the JSON
+    if text.startswith("```"):
+        end = text.find("\n")
+        text = text[end + 1:] if end != -1 else text[3:]
+    if text.endswith("```"):
+        text = text[:-3].rstrip()
+
+    # Find the first '{' and strip everything before it
+    start = text.find("{")
+    if start == -1:
+        return None
+    text = text[start:]
+
+    # Try to close an incomplete string at the end (e.g. ``"https://huggingf``)
+    # Pattern: ends mid-string (last quote is open)
+    if text.count('"') % 2 == 1:
+        text += '"'
+
+    # Ensure trailing commas before closing braces work
+    text = _strip_trailing_commas(text)
+
+    # Walk through the text character by character to find unclosed
+    # brackets and close them in the correct (LIFO) order.
+    # We ignore brackets inside quoted strings.
+    stack: list[str] = []
+    in_string = False
+    escape = False
+    for ch in text:
+        if escape:
+            escape = False
+            continue
+        if ch == "\\":
+            escape = True
+            continue
+        if ch == '"':
+            in_string = not in_string
+            continue
+        if in_string:
+            continue
+        if ch in ("{", "["):
+            stack.append(ch)
+        elif ch == "}":
+            if stack and stack[-1] == "{":
+                stack.pop()
+            else:
+                return None  # Unmatched closer — unrecoverable
+        elif ch == "]":
+            if stack and stack[-1] == "[":
+                stack.pop()
+            else:
+                return None
+
+    # Close remaining open brackets in reverse order
+    for opener in reversed(stack):
+        text += "}" if opener == "{" else "]"
+
+    try:
+        return json.loads(text)
+    except (json.JSONDecodeError, ValueError):
+        return None
+
+
+def _strip_trailing_commas(text: str) -> str:
+    """Remove commas that appear before a closing brace/bracket."""
+    import re as _re
+    text = _re.sub(r",\s*}", "}", text)
+    text = _re.sub(r",\s*]", "]", text)
+    return text

py/services/lora_service.py

@@ -24,23 +24,41 @@ class LoraService(BaseModelService):
         """
         super().__init__("lora", scanner, LoraMetadata, update_service=update_service)
 
-    async def format_response(self, lora_data: Dict) -> Dict:
-        """Format LoRA data for API response"""
+    async def format_response(self, lora_data: Dict) -> Optional[Dict]:
+        """Format LoRA data for API response.
+
+        Returns None when the entry is missing critical fields (corrupted cache
+        row), so the handler layer can filter it out instead of crashing the
+        whole listing request. See issue #730.
+        """
+        # Guard against corrupted cache entries missing critical fields
+        file_path = lora_data.get("file_path")
+        if not file_path or not isinstance(file_path, str):
+            logger.warning(
+                "Skipping corrupted LoRA entry (missing file_path): %s",
+                lora_data.get("file_name", "<unknown>"),
+            )
+            return None
+
         # Resolve sub_type using priority: sub_type > model_type > civitai.model.type > default
         # Normalize to lowercase for consistent API responses
         sub_type = resolve_sub_type(lora_data).lower()
 
+        file_name = lora_data.get("file_name") or ""
+        model_name = lora_data.get("model_name") or file_name
+        folder = lora_data.get("folder") or ""
+
         return {
-            "model_name": lora_data["model_name"],
-            "file_name": lora_data["file_name"],
+            "model_name": model_name,
+            "file_name": file_name,
             "preview_url": config.get_preview_static_url(
                 lora_data.get("preview_url", "")
             ),
             "preview_nsfw_level": lora_data.get("preview_nsfw_level", 0),
             "base_model": lora_data.get("base_model", ""),
-            "folder": lora_data["folder"],
+            "folder": folder,
             "sha256": lora_data.get("sha256", ""),
-            "file_path": lora_data["file_path"].replace(os.sep, "/"),
+            "file_path": file_path.replace(os.sep, "/"),
             "file_size": lora_data.get("size", 0),
             "modified": lora_data.get("modified", ""),
             "tags": lora_data.get("tags", []),
@@ -59,6 +77,8 @@ class LoraService(BaseModelService):
                 lora_data.get("civitai", {}), minimal=True
             ),
             "auto_tags": lora_data.get("auto_tags") or extract_auto_tags(lora_data),
+            "version_count": lora_data.get("version_count"),
+            "hf_url": lora_data.get("hf_url", ""),
         }
 
     async def _apply_specific_filters(self, data: List[Dict], **kwargs) -> List[Dict]:
@@ -251,12 +271,16 @@ class LoraService(BaseModelService):
         return letters
 
     async def get_lora_trigger_words(self, lora_name: str) -> List[str]:
-        """Get trigger words for a specific LoRA file"""
+        """Get trigger words for a specific LoRA file.
+
+        Supports both simple names and full-path syntax.
+        """
         cache = await self.scanner.get_cached_data()
 
         for lora in cache.raw_data:
-            if lora["file_name"] == lora_name:
-                civitai_data = lora.get("civitai", {})
+            file_name = lora.get("file_name", "")
+            if file_name == lora_name or lora_name.endswith("/" + file_name) or lora_name.endswith("\\" + file_name):
+                civitai_data = lora.get("civitai") or {}
                 return civitai_data.get("trainedWords", [])
 
         return []

py/services/metadata_service.py

@@ -15,6 +15,17 @@ from .service_registry import ServiceRegistry
 
 logger = logging.getLogger(__name__)
 
+_PROVIDER_DISPLAY_NAMES = {
+    "civitai_api": "CivitAI",
+    "civarchive_api": "CivArchive",
+    "sqlite": "Archive DB",
+}
+
+_PRESET_PROVIDER_ORDERS = {
+    "civitai_archive_sqlite": ["civitai_api", "civarchive_api", "sqlite"],
+    "civitai_sqlite_archive": ["civitai_api", "sqlite", "civarchive_api"],
+}
+
 async def initialize_metadata_providers():
     """Initialize and configure all metadata providers based on settings"""
     provider_manager = await ModelMetadataProviderManager.get_instance()
@@ -26,7 +37,9 @@ async def initialize_metadata_providers():
     # Get settings
     settings_manager = get_settings_manager()
     enable_archive_db = settings_manager.get('enable_metadata_archive_db', False)
-    
+    enable_civarchive_api = settings_manager.get('enable_civarchive_api', True)
+    provider_order = settings_manager.get('metadata_provider_order', 'civitai_archive_sqlite')
+
     providers = []
     
     # Initialize archive database provider if enabled
@@ -59,27 +72,48 @@ async def initialize_metadata_providers():
     except Exception as e:
         logger.error(f"Failed to initialize Civitai API metadata provider: {e}")
 
-    # Register CivArchive provider, and all add to fallback providers
-    try:
-        civarchive_client = await ServiceRegistry.get_civarchive_client()
-        civarchive_provider = CivArchiveModelMetadataProvider(civarchive_client)
-        provider_manager.register_provider('civarchive_api', civarchive_provider)
-        providers.append(('civarchive_api', civarchive_provider))
-        logger.debug("CivArchive metadata provider registered (also included in fallback)")
-    except Exception as e:
-        logger.error(f"Failed to initialize CivArchive metadata provider: {e}")
+    # Register CivArchive provider when enabled. Civitai API is always
+    # preferred (better metadata); CivArchive mainly recovers metadata for
+    # models deleted from Civitai, so it can be turned off to avoid its long
+    # rate-limit windows entirely.
+    if enable_civarchive_api:
+        try:
+            civarchive_client = await ServiceRegistry.get_civarchive_client()
+            civarchive_provider = CivArchiveModelMetadataProvider(civarchive_client)
+            provider_manager.register_provider('civarchive_api', civarchive_provider)
+            providers.append(('civarchive_api', civarchive_provider))
+            logger.debug("CivArchive metadata provider registered (also included in fallback)")
+        except Exception as e:
+            logger.error(f"Failed to initialize CivArchive metadata provider: {e}")
+    else:
+        logger.debug("CivArchive metadata provider disabled by setting 'enable_civarchive_api'")
+
+    # Preset fallback orderings (see module-level _PRESET_PROVIDER_ORDERS).
+    # civitai_api is always first (better metadata); the remaining providers
+    # are arranged by the configured preset.  Providers that are not
+    # registered (disabled/unavailable) are simply skipped, so each preset
+    # degrades gracefully.
+    desired_order = _PRESET_PROVIDER_ORDERS.get(
+        provider_order, _PRESET_PROVIDER_ORDERS["civitai_archive_sqlite"]
+    )
 
     # Set up fallback provider based on available providers
     if len(providers) > 1:
-        # Always use Civitai API (it has better metadata), then CivArchive API, then Archive DB
         ordered_providers: list[tuple[str, ModelMetadataProvider]] = []
-        ordered_providers.extend([p for p in providers if p[0] == 'civitai_api'])
-        ordered_providers.extend([p for p in providers if p[0] == 'civarchive_api'])
-        ordered_providers.extend([p for p in providers if p[0] == 'sqlite'])
-        
+        for name in desired_order:
+            ordered_providers.extend([p for p in providers if p[0] == name])
+        # Include any provider not covered by the preset (defensive) at the end
+        for p in providers:
+            if p not in ordered_providers:
+                ordered_providers.append(p)
+
         if ordered_providers:
             fallback_provider = FallbackMetadataProvider(ordered_providers)
             provider_manager.register_provider('fallback', fallback_provider, is_default=True)
+            logger.debug(
+                "Metadata fallback provider order: %s",
+                ", ".join(name for name, _ in ordered_providers),
+            )
     elif len(providers) == 1:
         # Only one provider available, set it as default
         provider_name, provider = providers[0]
@@ -96,11 +130,30 @@ async def update_metadata_providers():
         # Get current settings
         settings_manager = get_settings_manager()
         enable_archive_db = settings_manager.get('enable_metadata_archive_db', False)
+        enable_civarchive_api = settings_manager.get('enable_civarchive_api', True)
+        provider_order = settings_manager.get('metadata_provider_order', 'civitai_archive_sqlite')
         
         # Reinitialize all providers with new settings
         provider_manager = await initialize_metadata_providers()
         
-        logger.info(f"Updated metadata providers, archive_db enabled: {enable_archive_db}")
+        # Build effective provider chain for logging (use actually-registered
+        # providers, not just settings, so a failed init is reflected correctly)
+        registered = set(provider_manager.providers.keys())
+        desired = _PRESET_PROVIDER_ORDERS.get(
+            provider_order, _PRESET_PROVIDER_ORDERS["civitai_archive_sqlite"]
+        )
+        chain = " → ".join(
+            _PROVIDER_DISPLAY_NAMES[p]
+            for p in desired
+            if p in registered and p in _PROVIDER_DISPLAY_NAMES
+        )
+        
+        logger.info(
+            "Updated metadata providers: archive_db=%s, civarchive_api=%s, chain=%s",
+            enable_archive_db,
+            enable_civarchive_api,
+            chain,
+        )
         return provider_manager
     except Exception as e:
         logger.error(f"Failed to update metadata providers: {e}")

py/services/metadata_sync_service.py

@@ -209,20 +209,40 @@ class MetadataSyncService:
                         error_msg = "CivitAI model is deleted and no archive provider is available"
                     return False, error_msg
             else:
-                provider_attempts.append((None, await self._get_default_provider()))
+                is_hf_source = bool(model_data.get("hf_url"))
+                if is_hf_source:
+                    # HF-sourced model: only check CivitAI API directly.
+                    # CivArchive is almost guaranteed to have no record, and
+                    # hitting it wastes rate-limit budget.
+                    # Use a distinct provider name ("civitai_api" not None) so
+                    # downstream code does NOT interpret a "Model not found"
+                    # response as civitai_api_not_found — which would mark the
+                    # model civitai_deleted=True when it was never on CivitAI.
+                    try:
+                        provider_attempts.append(("civitai_api", await self._get_provider("civitai_api")))
+                    except Exception as exc:  # pragma: no cover - provider resolution fault
+                        logger.debug("Unable to resolve civitai_api provider: %s", exc)
+                if not provider_attempts:
+                    provider_attempts.append((None, await self._get_default_provider()))
 
             civitai_metadata: Optional[Dict[str, Any]] = None
             metadata_provider: Optional[MetadataProviderProtocol] = None
             provider_used: Optional[str] = None
             last_error: Optional[str] = None
             civitai_api_not_found = False
+            any_rate_limited = False
 
             for provider_name, provider in provider_attempts:
                 try:
                     civitai_metadata_candidate, error = await provider.get_model_by_hash(sha256)
                 except RateLimitError as exc:
-                    exc.provider = exc.provider or (provider_name or provider.__class__.__name__)
-                    raise
+                    logger.warning(
+                        "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                        provider_name or provider.__class__.__name__,
+                        exc.retry_after or 0,
+                    )
+                    any_rate_limited = True
+                    continue
                 except Exception as exc:  # pragma: no cover - defensive logging
                     logger.error("Provider %s failed for hash %s: %s", provider_name, sha256, exc)
                     civitai_metadata_candidate, error = None, str(exc)
@@ -258,6 +278,14 @@ class MetadataSyncService:
                     model_data["last_checked_at"] = datetime.now().timestamp()
                     needs_save = True
 
+                # When the model was already classified as "not on CivitAI" via
+                # .metadata.json (civitai_deleted=True) but the SQLite cache is
+                # stale (because the pre-fix code never persisted these flags),
+                # ensure the flags are written to the scanner cache + SQLite.
+                if not needs_save and model_data.get("civitai_deleted") is True:
+                    model_data["last_checked_at"] = datetime.now().timestamp()
+                    needs_save = True
+
                 # Save metadata if any state was updated
                 if needs_save:
                     data_to_save = model_data.copy()
@@ -266,6 +294,7 @@ class MetadataSyncService:
                     if "last_checked_at" not in data_to_save:
                         data_to_save["last_checked_at"] = datetime.now().timestamp()
                     await self._metadata_manager.save_metadata(file_path, data_to_save)
+                    await update_cache_func(file_path, file_path, data_to_save)
 
                 default_error = (
                     "CivitAI model is deleted and metadata archive DB is not enabled"
@@ -276,17 +305,18 @@ class MetadataSyncService:
                 )
 
                 resolved_error = last_error or default_error
+                if any_rate_limited and "Rate limited" not in resolved_error:
+                    resolved_error = "Rate limited"
                 if is_expected_offline_error(resolved_error):
                     resolved_error = OFFLINE_FRIENDLY_MESSAGE
 
                 error_msg = (
                     f"Error fetching metadata: {resolved_error} "
-                    f"(model_name={model_data.get('model_name', '')})"
+                    f"(file={os.path.basename(file_path)}, sha256={sha256})"
                 )
-                if is_expected_offline_error(resolved_error):
-                    logger.info(error_msg)
-                else:
-                    logger.error(error_msg)
+                # Use case layer (BulkMetadataRefreshUseCase) logs failed models at WARNING level,
+                # so this level is demoted to DEBUG to avoid duplicate user-visible logging.
+                logger.debug(error_msg)
                 return False, error_msg
 
             model_data["from_civitai"] = True
@@ -411,7 +441,18 @@ class MetadataSyncService:
         metadata = await metadata_loader(metadata_path)
 
         for key, value in updates.items():
-            if isinstance(value, dict) and isinstance(metadata.get(key), dict):
+            if key == "tags" and isinstance(value, list):
+                # Normalize tags: trim, lowercase, deduplicate
+                normalized = []
+                seen = set()
+                for tag in value:
+                    if isinstance(tag, str):
+                        t = tag.strip().lower()
+                        if t and t not in seen:
+                            normalized.append(t)
+                            seen.add(t)
+                metadata[key] = normalized
+            elif isinstance(value, dict) and isinstance(metadata.get(key), dict):
                 metadata[key].update(value)
             else:
                 metadata[key] = value

py/services/model_cache.py

@@ -18,6 +18,8 @@ SUPPORTED_SORT_MODES = [
     ('size', 'desc'),
     ('usage', 'asc'),
     ('usage', 'desc'),
+    ('versions_count', 'asc'),
+    ('versions_count', 'desc'),
 ]
 # Is this in use?
 
@@ -263,6 +265,17 @@ class ModelCache:
                 ),
                 reverse=reverse
             )
+        elif sort_key == 'versions_count':
+            # Pre-dedup sort: fall back to name sort.
+            # Actual re-sort by version_count happens in get_paginated_data after dedup.
+            result = natsorted(
+                data,
+                key=lambda x: (
+                    self._get_display_name(x).lower(),
+                    x.get('file_path', '').lower()
+                ),
+                reverse=reverse
+            )
         else:
             # Fallback: no sort
             result = list(data)
@@ -324,4 +337,25 @@ class ModelCache:
             else:
                 return False  # Model not found
                     
-            return True
+            return True
+
+    async def clear_preview_by_path(self, preview_file_path: str) -> int:
+        """Clear ``preview_url`` for every cached entry referencing a file path.
+
+        When a preview file has been deleted from disk, this removes its
+        reference from all matching cache entries so the next list-API
+        response returns an empty ``preview_url`` instead of a stale URL
+        that produces 404s.
+
+        Returns the number of entries that were updated.
+        """
+        normalized = preview_file_path.replace("\\", "/")
+        cleared = 0
+        async with self._lock:
+            for item in self.raw_data:
+                cached_url = item.get("preview_url", "")
+                if cached_url.replace("\\", "/") == normalized:
+                    item["preview_url"] = ""
+                    item["preview_nsfw_level"] = 0
+                    cleared += 1
+        return cleared

py/services/model_file_service.py

@@ -8,6 +8,7 @@ from abc import ABC, abstractmethod
 from ..utils.utils import calculate_relative_path_for_model, remove_empty_dirs
 from ..utils.constants import AUTO_ORGANIZE_BATCH_SIZE
 from ..services.settings_manager import get_settings_manager
+from ..services.model_lifecycle_service import _require_path_in_library_roots
 
 logger = logging.getLogger(__name__)
 
@@ -493,6 +494,9 @@ class ModelMoveService:
             Dictionary with move result
         """
         try:
+            _require_path_in_library_roots(file_path, self.scanner, label="Source path")
+            _require_path_in_library_roots(target_path, self.scanner, label="Target path")
+
             if use_default_paths:
                 # Find the model in cache to get metadata
                 cache = await self.scanner.get_cached_data()

py/services/model_hash_index.py

@@ -7,6 +7,7 @@ class ModelHashIndex:
     def __init__(self):
         self._hash_to_path: Dict[str, str] = {}
         self._filename_to_hash: Dict[str, str] = {}
+        self._autov2_to_path: Dict[str, str] = {}
         # New data structures for tracking duplicates
         self._duplicate_hashes: Dict[str, List[str]] = {}  # sha256 -> list of paths
         self._duplicate_filenames: Dict[str, List[str]] = {}  # filename -> list of paths
@@ -63,6 +64,9 @@ class ModelHashIndex:
         # Add new mappings
         self._hash_to_path[sha256] = file_path
         self._filename_to_hash[filename] = sha256
+        # AutoV2 = first 10 chars of SHA256
+        if len(sha256) >= 10:
+            self._autov2_to_path[sha256[:10]] = file_path
     
     def _get_filename_from_path(self, file_path: str) -> str:
         """Extract filename without extension from path"""
@@ -157,7 +161,12 @@ class ModelHashIndex:
                 del self._duplicate_filenames[filename]
                 if filename in self._filename_to_hash:
                     del self._filename_to_hash[filename]
-    
+
+        # Remove from AutoV2 index
+        autov2_keys_to_remove = [k for k, v in self._autov2_to_path.items() if v == file_path]
+        for k in autov2_keys_to_remove:
+            del self._autov2_to_path[k]
+
     def remove_by_hash(self, sha256: str) -> None:
         """Remove entry by hash"""
         sha256 = sha256.lower()
@@ -177,6 +186,10 @@ class ModelHashIndex:
         # Remove hash-to-path mapping
         del self._hash_to_path[sha256]
         
+        autov2_key = sha256[:10]
+        if autov2_key in self._autov2_to_path:
+            del self._autov2_to_path[autov2_key]
+        
         # Update filename-to-hash and duplicate filenames for all paths
         for path_to_remove in paths_to_remove:
             fname = self._get_filename_from_path(path_to_remove)
@@ -195,13 +208,24 @@ class ModelHashIndex:
                     # If only one entry remains, it's no longer a duplicate
                     del self._duplicate_filenames[fname]
     
-    def has_hash(self, sha256: str) -> bool:
-        """Check if hash exists in index"""
-        return sha256.lower() in self._hash_to_path
-    
-    def get_path(self, sha256: str) -> Optional[str]:
-        """Get file path for a hash"""
-        return self._hash_to_path.get(sha256.lower())
+    def has_hash(self, hash_value: str) -> bool:
+        """Check if hash exists in index (SHA256 or AutoV2)"""
+        normalized = hash_value.lower()
+        if normalized in self._hash_to_path:
+            return True
+        if len(normalized) == 10:
+            return normalized in self._autov2_to_path
+        return False
+
+    def get_path(self, hash_value: str) -> Optional[str]:
+        """Get file path for a hash (SHA256 or AutoV2)"""
+        normalized = hash_value.lower()
+        path = self._hash_to_path.get(normalized)
+        if path is not None:
+            return path
+        if len(normalized) == 10:
+            return self._autov2_to_path.get(normalized)
+        return None
     
     def get_hash(self, file_path: str) -> Optional[str]:
         """Get hash for a file path"""
@@ -218,6 +242,7 @@ class ModelHashIndex:
         """Clear all entries"""
         self._hash_to_path.clear()
         self._filename_to_hash.clear()
+        self._autov2_to_path.clear()
         self._duplicate_hashes.clear()
         self._duplicate_filenames.clear()
     

py/services/model_lifecycle_service.py

@@ -48,6 +48,36 @@ async def delete_model_artifacts(
     return deleted
 
 
+def _require_path_in_library_roots(file_path: str, scanner, *, label: str = "path") -> None:
+    """Raise ``ValueError`` if *file_path* is not inside a configured model root.
+
+    Uses ``os.path.abspath()`` (NOT ``realpath``) to resolve ``..`` and ``.``
+    while preserving symlinks — this keeps the check in business-path space.
+    Skips when the scanner does not expose ``get_model_roots`` or the list
+    is empty.
+    """
+
+    roots = None
+    if hasattr(scanner, "get_model_roots"):
+        try:
+            roots = scanner.get_model_roots()
+        except NotImplementedError:
+            roots = None
+    if not roots:
+        return
+
+    resolved = os.path.abspath(os.path.normpath(file_path))
+
+    for root in roots:
+        root_resolved = os.path.abspath(os.path.normpath(root))
+        if resolved == root_resolved or resolved.startswith(root_resolved + os.sep):
+            return
+
+    raise ValueError(
+        f"{label} '{file_path}' is outside configured library directories"
+    )
+
+
 class ModelLifecycleService:
     """Co-ordinate destructive and mutating model operations."""
 
@@ -74,6 +104,8 @@ class ModelLifecycleService:
         if not file_path:
             raise ValueError("Model path is required")
 
+        _require_path_in_library_roots(file_path, self._scanner, label="File path")
+
         cache = await self._scanner.get_cached_data()
 
         cached_entry = None
@@ -182,6 +214,8 @@ class ModelLifecycleService:
         if not file_path:
             raise ValueError("Model path is required")
 
+        _require_path_in_library_roots(file_path, self._scanner, label="File path")
+
         metadata_path = os.path.splitext(file_path)[0] + ".metadata.json"
         metadata = await self._metadata_loader(metadata_path)
         metadata["exclude"] = True
@@ -229,6 +263,8 @@ class ModelLifecycleService:
         if not file_path:
             raise ValueError("Model path is required")
 
+        _require_path_in_library_roots(file_path, self._scanner, label="File path")
+
         if not os.path.exists(file_path):
             raise ValueError("Model file does not exist")
 
@@ -270,6 +306,9 @@ class ModelLifecycleService:
         if not file_paths:
             raise ValueError("No file paths provided for deletion")
 
+        for path in file_paths:
+            _require_path_in_library_roots(path, self._scanner, label="File path")
+
         return await self._scanner.bulk_delete_models(file_paths)
 
     async def rename_model(
@@ -280,6 +319,8 @@ class ModelLifecycleService:
         if not file_path or not new_file_name:
             raise ValueError("File path and new file name are required")
 
+        _require_path_in_library_roots(file_path, self._scanner, label="File path")
+
         invalid_chars = {"/", "\\", ":", "*", "?", '"', "<", ">", "|"}
         if any(char in new_file_name for char in invalid_chars):
             raise ValueError("Invalid characters in file name")

py/services/model_metadata_provider.py

@@ -5,7 +5,7 @@ import logging
 import random
 from typing import Optional, Dict, Tuple, Any, List, Sequence
 from .downloader import get_downloader
-from .errors import RateLimitError
+from .errors import RateLimitError, ResourceNotFoundError
 
 try:
     from bs4 import BeautifulSoup
@@ -65,7 +65,14 @@ class _RateLimitRetryHelper:
                 return await func(*args, **kwargs)
             except RateLimitError as exc:
                 attempt += 1
-                if attempt >= self._retry_limit:
+
+                # Determine effective retry limit based on rate-limit magnitude
+                effective_retry_limit = self._retry_limit  # default: 3
+                if exc.retry_after is not None and exc.retry_after >= 120.0:
+                    # Long rate-limit window (>=2 min) — retries are futile
+                    effective_retry_limit = 1  # total 1 attempt = 0 retries
+
+                if attempt >= effective_retry_limit:
                     exc.provider = exc.provider or label
                     raise
 
@@ -81,7 +88,11 @@ class _RateLimitRetryHelper:
 
     def _calculate_delay(self, retry_after: Optional[float], attempt: int) -> float:
         if retry_after is not None:
-            return min(self._max_delay, max(0.0, retry_after))
+            # Cap at 1800s (30 min) as a safety ceiling. The old 30s cap was
+            # too low — CivArchive can return retry_after ~1500s, causing all
+            # retries to fail. A generous ceiling protects against pathological
+            # server values while still respecting the server's guidance.
+            return min(1800.0, max(0.0, retry_after))
 
         base_delay = self._base_delay * (2 ** max(0, attempt - 1))
         jitter_span = base_delay * self._jitter_ratio
@@ -474,14 +485,19 @@ class FallbackMetadataProvider(ModelMetadataProvider):
                 if result:
                     return result, error
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
             except Exception as e:
                 logger.debug("Provider %s failed for get_model_by_hash: %s", label, e)
                 continue
         return None, "Model not found"
 
     async def get_model_versions(self, model_id: str) -> Optional[Dict]:
+        not_found_confirmed = False
         for provider, label in self._iter_providers():
             try:
                 result = await self._call_with_rate_limit(
@@ -492,8 +508,20 @@ class FallbackMetadataProvider(ModelMetadataProvider):
                 if result:
                     return result
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
+            except ResourceNotFoundError:
+                not_found_confirmed = True
+                logger.debug(
+                    "Provider %s reports model %s as not found",
+                    label,
+                    model_id,
+                )
+                continue
             except Exception as e:
                 logger.debug("Provider %s failed for get_model_versions: %s", label, e)
                 continue
@@ -511,8 +539,12 @@ class FallbackMetadataProvider(ModelMetadataProvider):
                 if result:
                     return result
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
             except Exception as e:
                 logger.debug("Provider %s failed for get_model_version: %s", label, e)
                 continue
@@ -529,8 +561,12 @@ class FallbackMetadataProvider(ModelMetadataProvider):
                 if result:
                     return result, error
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
             except Exception as e:
                 logger.debug("Provider %s failed for get_model_version_info: %s", label, e)
                 continue
@@ -551,8 +587,12 @@ class FallbackMetadataProvider(ModelMetadataProvider):
             except NotImplementedError:
                 continue
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
             except Exception as e:
                 logger.debug(
                     "Provider %s failed for get_model_versions_by_hashes: %s",
@@ -573,8 +613,12 @@ class FallbackMetadataProvider(ModelMetadataProvider):
                 if result is not None:
                     return result
             except RateLimitError as exc:
-                exc.provider = exc.provider or label
-                raise exc
+                logger.warning(
+                    "Provider %s is rate-limited (retry_after=%.0fs); skipping to next provider",
+                    label,
+                    exc.retry_after or 0,
+                )
+                continue
             except Exception as e:
                 logger.debug("Provider %s failed for get_user_models: %s", label, e)
                 continue

py/services/model_query.py

@@ -294,12 +294,14 @@ class ModelFilterSet:
                 for tag, state in tag_filters.items():
                     if not tag:
                         continue
+                    # Normalize to lowercase for case-insensitive matching
+                    normalized = tag.strip().lower()
                     if state == "exclude":
-                        exclude_tags.add(tag)
+                        exclude_tags.add(normalized)
                     else:
-                        include_tags.add(tag)
+                        include_tags.add(normalized)
             else:
-                include_tags = {tag for tag in tag_filters if tag}
+                include_tags = {tag.strip().lower() for tag in tag_filters if tag}
 
             if include_tags:
                 tag_logic = criteria.tag_logic.lower() if criteria.tag_logic else "any"
@@ -318,13 +320,17 @@ class ModelFilterSet:
                                 return True
                             # Otherwise, check if all non-special tags match
                             if non_special_tags:
-                                return all(tag in (item_tags or []) for tag in non_special_tags)
+                                # Case-insensitive: normalize item tags too
+                                normalized_item_tags = {t.strip().lower() for t in (item_tags or []) if isinstance(t, str)}
+                                return all(tag in normalized_item_tags for tag in non_special_tags)
                             return True
-                        # Normal case: all tags must match
-                        return all(tag in (item_tags or []) for tag in non_special_tags)
+                        # Normal case: all tags must match (case-insensitive)
+                        normalized_item_tags = {t.strip().lower() for t in (item_tags or []) if isinstance(t, str)}
+                        return all(tag in normalized_item_tags for tag in non_special_tags)
                     else:
-                        # OR logic (default): item must have ANY include tag
-                        return any(tag in include_tags for tag in (item_tags or []))
+                        # OR logic (default): item must have ANY include tag (case-insensitive)
+                        normalized_item_tags = {t.strip().lower() for t in (item_tags or []) if isinstance(t, str)}
+                        return bool(normalized_item_tags & include_tags)
 
                 items = [item for item in items if matches_include(item.get("tags"))]
 
@@ -333,7 +339,9 @@ class ModelFilterSet:
                 def matches_exclude(item_tags):
                     if not item_tags and "__no_tags__" in exclude_tags:
                         return True
-                    return any(tag in exclude_tags for tag in (item_tags or []))
+                    # Case-insensitive: normalize item tags
+                    normalized_item_tags = {t.strip().lower() for t in (item_tags or []) if isinstance(t, str)}
+                    return bool(normalized_item_tags & exclude_tags)
 
                 items = [
                     item for item in items if not matches_exclude(item.get("tags"))

py/services/model_scanner.py

@@ -14,7 +14,7 @@ from ..utils.metadata_manager import MetadataManager
 from ..utils.civitai_utils import resolve_license_info
 from .model_cache import ModelCache
 from .model_hash_index import ModelHashIndex
-from .model_lifecycle_service import delete_model_artifacts
+from .model_lifecycle_service import delete_model_artifacts, _require_path_in_library_roots
 from .service_registry import ServiceRegistry
 from .websocket_manager import ws_manager
 from .persistent_model_cache import get_persistent_cache
@@ -227,6 +227,11 @@ class ModelScanner:
 
         entry: Dict[str, Any] = {
             'file_path': normalized_path,
+            # file_name is always stored WITHOUT extension (e.g. "OWSMianne_ANIMA_V1",
+            # not "OWSMianne_ANIMA_V1.safetensors"). All upstream population points
+            # (MetadataManager, from_civitai_info, download manager, etc.) strip the
+            # extension via os.path.splitext before writing. Code consuming this field
+            # should match against names that are likewise extension-free.
             'file_name': get_value('file_name', '') or '',
             'model_name': get_value('model_name', '') or '',
             'folder': normalized_folder,
@@ -248,6 +253,7 @@ class ModelScanner:
             'civitai': civitai_slim,
             'civitai_deleted': bool(get_value('civitai_deleted', False)),
             'skip_metadata_refresh': bool(get_value('skip_metadata_refresh', False)),
+            'hf_url': get_value('hf_url', '') or '',
         }
 
         license_source: Dict[str, Any] = {}
@@ -476,11 +482,20 @@ class ModelScanner:
             for tag in adjusted_item.get('tags') or []:
                 tags_count[tag] = tags_count.get(tag, 0) + 1
 
-        # Validate cache entries and check health
+        # Validate cache entries and check health.
+        # Always use the validated/repaired entries — even when there are no
+        # invalid entries, auto_repair may have filled in missing optional
+        # fields (model_name, file_name, folder) with safe defaults on a copied
+        # working_entry.  Without this unconditional replacement the repaired
+        # copies are discarded and None values propagate to format_response.
+        # See issue #730.
         valid_entries, invalid_entries = CacheEntryValidator.validate_batch(
             adjusted_raw_data, auto_repair=True
         )
 
+        # Always use the validated entries (repaired copies)
+        adjusted_raw_data = valid_entries
+
         if invalid_entries:
             monitor = CacheHealthMonitor()
             report = monitor.check_health(adjusted_raw_data, auto_repair=True)
@@ -532,6 +547,13 @@ class ModelScanner:
         if not scan_result or not getattr(self, '_persistent_cache', None):
             return
 
+        if self.is_cancelled():
+            logger.info(
+                f"{self.model_type.capitalize()} Scanner: Skipping _save_persistent_cache "
+                "after cancellation"
+            )
+            return
+
         hash_snapshot = self._build_hash_index_snapshot(scan_result.hash_index)
         loop = asyncio.get_event_loop()
         try:
@@ -705,14 +727,20 @@ class ModelScanner:
             # Determine the page type based on model type
             # Scan for new data
             scan_result = await self._gather_model_data()
-            await self._apply_scan_result(scan_result)
-            await self._save_persistent_cache(scan_result)
-            await self._sync_download_history(scan_result.raw_data, source='scan')
+            if not self.is_cancelled():
+                await self._apply_scan_result(scan_result)
+                await self._save_persistent_cache(scan_result)
+                await self._sync_download_history(scan_result.raw_data, source='scan')
 
-            logger.info(
-                f"{self.model_type.capitalize()} Scanner: Cache initialization completed in {time.time() - start_time:.2f} seconds, "
-                f"found {len(scan_result.raw_data)} models"
-            )
+                logger.info(
+                    f"{self.model_type.capitalize()} Scanner: Cache initialization completed in {time.time() - start_time:.2f} seconds, "
+                    f"found {len(scan_result.raw_data)} models"
+                )
+            else:
+                logger.info(
+                    f"{self.model_type.capitalize()} Scanner: Cache initialization cancelled "
+                    f"after {time.time() - start_time:.2f} seconds"
+                )
         except Exception as e:
             logger.error(f"{self.model_type.capitalize()} Scanner: Error initializing cache: {e}")
             # Ensure cache is at least an empty structure on error
@@ -1067,8 +1095,11 @@ class ModelScanner:
 
         model_data = self._build_cache_entry(metadata, folder=normalized_folder)
 
-        # Compute SHA256 hash when metadata provided none (e.g., CivitAI API response has empty hashes)
-        if not model_data.get('sha256') and file_path:
+        # Compute SHA256 hash when metadata provided none (e.g., CivitAI API response has empty hashes).
+        # Respect hash_status='pending' (set by CheckpointScanner for large models) to defer
+        # hash calculation until on-demand — avoids reading entire checkpoint files at startup.
+        hash_status = model_data.get('hash_status', '')
+        if not model_data.get('sha256') and hash_status != 'pending' and file_path:
             try:
                 logger.info(f"Computing SHA256 hash for {file_path} (was empty from metadata)")
                 sha256 = await calculate_sha256(file_path)
@@ -1093,6 +1124,13 @@ class ModelScanner:
         if scan_result is None:
             return
 
+        if self.is_cancelled():
+            logger.info(
+                f"{self.model_type.capitalize()} Scanner: Skipping _apply_scan_result "
+                "after cancellation"
+            )
+            return
+
         self._hash_index = scan_result.hash_index
         self._tags_count = dict(scan_result.tags_count)
         self._excluded_models = list(scan_result.excluded_models)
@@ -1356,6 +1394,9 @@ class ModelScanner:
                 
             base_name = os.path.splitext(os.path.basename(source_path))[0]
             source_dir = os.path.dirname(source_path)
+
+            _require_path_in_library_roots(source_path, self, label="Source path")
+            _require_path_in_library_roots(target_path, self, label="Target path")
             
             os.makedirs(target_path, exist_ok=True)
             
@@ -1528,6 +1569,218 @@ class ModelScanner:
 
         return cache_entry if metadata else True
         
+    async def sync_cache_from_metadata(
+        self, file_path: str, metadata_dict: Dict[str, Any]
+    ) -> bool:
+        """Opportunistically sync in-memory and persistent caches from metadata.
+
+        Builds a prospective cache entry from *metadata_dict* (deserialized
+        ``.metadata.json`` content) and compares it against the current cache
+        entry.  When the two are already identical this method returns
+        ``False`` without touching anything — avoiding the overhead of
+        ``update_single_model_cache``, which always removes and re-inserts
+        the entry, triggers a full resort, and persists via the heavyweight
+        ``save_cache()``.
+
+        When differences are detected the update is applied **in-place** with
+        targeted operations:
+
+        * The existing ``raw_data`` entry is modified rather than removed and
+          re-appended (O(1) instead of O(n)).
+        * Tag counts and the hash index are updated incrementally.
+        * The version index is rebuilt only for the affected entry.
+        * ``resort()`` is called **only** when a sort-relevant field changed
+          (``model_name`` / ``file_name`` for name-sort, ``modified`` for
+          date-sort, ``size`` for size-sort).
+        * The persistent (SQLite) cache receives a targeted single-row update
+          via :meth:`PersistentModelCache.update_single_model` rather than a
+          full-table ``save_cache()``.
+
+        Returns:
+            ``True`` if any cache update was performed, ``False`` if the
+            caches were already in sync.
+
+        .. note::
+
+            This is a **best-effort** operation.  Failures are logged but
+            never propagated — callers should fire-and-forget via
+            :func:`asyncio.create_task`.
+        """
+        try:
+            return await self._sync_cache_from_metadata_impl(
+                file_path, metadata_dict
+            )
+        except Exception:
+            logger.warning(
+                "sync_cache_from_metadata failed for %s",
+                file_path,
+                exc_info=True,
+            )
+            return False
+
+    async def _sync_cache_from_metadata_impl(
+        self, file_path: str, metadata_dict: Dict[str, Any]
+    ) -> bool:
+        cache = await self.get_cached_data()
+
+        # Locate the existing cache entry -----------------------------------
+        existing_idx: Optional[int] = None
+        existing_entry: Optional[Dict[str, Any]] = None
+        for i, item in enumerate(cache.raw_data):
+            if item.get("file_path") == file_path:
+                existing_entry = item
+                existing_idx = i
+                break
+
+        # Build the desired entry from metadata ------------------------------
+        folder_value = (
+            existing_entry.get("folder", "")
+            if existing_entry
+            else self._calculate_folder(file_path)
+        )
+        desired_entry = self._build_cache_entry(
+            metadata_dict,
+            folder=folder_value,
+            file_path_override=file_path,
+        )
+
+        # Ensure sha256 is populated (defensive — metadata should have it)
+        if (
+            not desired_entry.get("sha256")
+            and file_path
+            and os.path.exists(file_path)
+        ):
+            try:
+                sha256 = await calculate_sha256(file_path)
+                if sha256:
+                    desired_entry["sha256"] = sha256.lower()
+            except Exception:
+                pass
+
+        # Not in cache at all — delegate to the full update path ------------
+        if existing_entry is None:
+            result = await self.update_single_model_cache(
+                file_path, file_path, metadata_dict
+            )
+            return bool(result)
+
+        # Compare — skip everything if already in sync -----------------------
+        if not self._cache_entries_differ(existing_entry, desired_entry):
+            return False
+
+        # Re-validate: the cache may have been replaced concurrently
+        # (e.g. by _apply_scan_result).  Use identity check, not equality,
+        # so we detect when the raw_data list was swapped out from under us.
+        if self._cache is None or not any(
+            item is existing_entry for item in self._cache.raw_data
+        ):
+            return False
+
+        # ---- Differences detected: apply targeted, in-place updates --------
+
+        # Snapshot old values for delta computations
+        old_tags = list(existing_entry.get("tags") or [])
+        old_sha256: str = existing_entry.get("sha256", "") or ""
+        old_model_name: str = existing_entry.get("model_name", "") or ""
+        old_file_name: str = existing_entry.get("file_name", "") or ""
+        old_modified: float = float(existing_entry.get("modified", 0.0) or 0.0)
+        old_size: int = int(existing_entry.get("size", 0) or 0)
+        old_civitai = existing_entry.get("civitai")
+
+        # ---- In-place update of the cache entry ----
+        existing_entry.clear()
+        existing_entry.update(desired_entry)
+
+        # ---- Incremental tag count update ----
+        new_tags: set = set(desired_entry.get("tags") or [])
+        old_tag_set: set = set(old_tags)
+        for tag in old_tag_set - new_tags:
+            current = self._tags_count.get(tag, 0)
+            if current <= 1:
+                self._tags_count.pop(tag, None)
+            else:
+                self._tags_count[tag] = current - 1
+        for tag in new_tags - old_tag_set:
+            self._tags_count[tag] = self._tags_count.get(tag, 0) + 1
+
+        # ---- Incremental hash index update ----
+        new_sha = (desired_entry.get("sha256", "") or "").lower()
+        old_sha = (old_sha256 or "").lower()
+        if new_sha != old_sha:
+            if old_sha:
+                self._hash_index.remove_by_path(file_path)
+            if new_sha:
+                self._hash_index.add_entry(new_sha, file_path)
+
+        # ---- Incremental version index update ----
+        new_civitai = desired_entry.get("civitai")
+        if old_civitai != new_civitai:
+            temp_old = {
+                "file_path": file_path,
+                "file_name": old_file_name,
+                "civitai": old_civitai,
+            }
+            cache.remove_from_version_index(temp_old)
+            cache.add_to_version_index(existing_entry)
+
+        # ---- Conditional resort (only when sort-key fields changed) ----
+        need_resort = False
+        _last = cache._last_sort
+        sort_key: Optional[str] = _last[0] if _last != (None, None) else None
+        if sort_key == "name":
+            if (
+                old_model_name != desired_entry.get("model_name", "")
+                or old_file_name != desired_entry.get("file_name", "")
+            ):
+                need_resort = True
+        elif sort_key == "date":
+            if old_modified != float(desired_entry.get("modified", 0.0) or 0.0):
+                need_resort = True
+        elif sort_key == "size":
+            if old_size != int(desired_entry.get("size", 0) or 0):
+                need_resort = True
+
+        if need_resort:
+            await cache.resort()
+
+        # ---- Targeted SQL update (single row, not full save_cache) ----
+        persistent = getattr(self, "_persistent_cache", None)
+        if persistent is not None:
+            old_item_for_sql: Dict[str, Any] = {
+                "file_path": file_path,
+                "tags": old_tags,
+                "sha256": old_sha256,
+            }
+            await asyncio.get_event_loop().run_in_executor(
+                None,
+                persistent.update_single_model,
+                self.model_type,
+                desired_entry,
+                old_item_for_sql,
+            )
+
+        return True
+
+    @staticmethod
+    def _cache_entries_differ(a: Dict[str, Any], b: Dict[str, Any]) -> bool:
+        """Return ``True`` when two cache-entry dicts differ in any field.
+
+        Tag lists are compared order-insensitively; all other keys use
+        standard equality.
+        """
+        a_tags = sorted(a.get("tags") or [])
+        b_tags = sorted(b.get("tags") or [])
+        if a_tags != b_tags:
+            return True
+
+        all_keys = set(a.keys()) | set(b.keys())
+        for key in all_keys:
+            if key == "tags":
+                continue
+            if a.get(key) != b.get(key):
+                return True
+        return False
+
     def has_hash(self, sha256: str) -> bool:
         """Check if a model with given hash exists"""
         return self._hash_index.has_hash(sha256.lower())
@@ -1580,7 +1833,32 @@ class ModelScanner:
         if limit == 0:
             return sorted_tags
         return sorted_tags[:limit]
-        
+
+    async def search_tags(
+        self, query: str, limit: int = 50
+    ) -> List[Dict[str, any]]:
+        """Search tags by case-insensitive substring match, sorted by count.
+
+        If query is empty, behaves like get_top_tags (returns top ``limit``
+        tags). If limit is 0, all matching tags are returned.
+        """
+        await self.get_cached_data()
+
+        normalized_query = (query or "").strip().lower()
+        if not normalized_query:
+            return await self.get_top_tags(limit if limit > 0 else 20)
+
+        matched = [
+            {"tag": tag, "count": count}
+            for tag, count in self._tags_count.items()
+            if normalized_query in tag.lower()
+        ]
+        matched.sort(key=lambda x: x["count"], reverse=True)
+
+        if limit == 0:
+            return matched
+        return matched[:limit]
+
     async def get_base_models(self, limit: int = 20) -> List[Dict[str, any]]:
         """Get base models sorted by count. If limit is 0, return all."""
         cache = await self.get_cached_data()
@@ -1696,6 +1974,8 @@ class ModelScanner:
                     break
 
                 try:
+                    _require_path_in_library_roots(file_path, self, label="File path")
+
                     target_dir = os.path.dirname(file_path)
                     base_name = os.path.basename(file_path)
                     file_name, main_extension = os.path.splitext(base_name)
@@ -1761,6 +2041,13 @@ class ModelScanner:
         """
         if not file_paths or self._cache is None:
             return False
+
+        if self.is_cancelled():
+            logger.info(
+                f"{self.model_type.capitalize()} Scanner: Skipping cache update "
+                "after cancelled bulk delete"
+            )
+            return False
             
         try:
             # Get all models that need to be removed from cache

py/services/model_update_service.py

@@ -689,6 +689,7 @@ class ModelUpdateService:
         *,
         force_refresh: bool = False,
         target_model_ids: Optional[Sequence[int]] = None,
+        folder_path: Optional[str] = None,
     ) -> Dict[int, ModelUpdateRecord]:
         """Refresh update information for every model present in the cache."""
         scanner.reset_cancellation()
@@ -703,6 +704,7 @@ class ModelUpdateService:
         local_versions = await self._collect_local_versions(
             scanner,
             target_model_ids=target_filter,
+            folder_path=folder_path,
         )
         total_models = len(local_versions)
         if total_models == 0:
@@ -722,6 +724,16 @@ class ModelUpdateService:
             "Refreshing update metadata for %d %s models", total_models, model_type
         )
 
+        # When filtering by folder, also collect the cross-folder version set
+        # so that versions already present in other folders are not reported
+        # as available updates. See issue #997.
+        all_local_versions: Optional[Dict[int, List[int]]] = None
+        if folder_path is not None:
+            all_local_versions = await self._collect_local_versions(
+                scanner,
+                target_model_ids=target_filter,
+            )
+
         results: Dict[int, ModelUpdateRecord] = {}
         prefetched: Dict[int, Mapping] = {}
 
@@ -760,6 +772,12 @@ class ModelUpdateService:
         for index, (model_id, version_ids) in enumerate(
             local_versions.items(), start=1
         ):
+            # Use cross-folder version IDs for is_in_library if available
+            all_vids: Sequence[int] = (
+                all_local_versions.get(model_id, [])
+                if all_local_versions is not None
+                else version_ids
+            )
             record = await self._refresh_single_model(
                 model_type,
                 model_id,
@@ -767,6 +785,7 @@ class ModelUpdateService:
                 metadata_provider,
                 force_refresh=force_refresh,
                 prefetched_response=prefetched.get(model_id),
+                all_local_version_ids=all_vids,
             )
             if scanner.is_cancelled():
                 logger.info(f"{model_type.capitalize()} Update Service: Refresh cancelled by user")
@@ -962,8 +981,16 @@ class ModelUpdateService:
         *,
         force_refresh: bool = False,
         prefetched_response: Optional[Mapping] = None,
+        all_local_version_ids: Optional[Sequence[int]] = None,
     ) -> Optional[ModelUpdateRecord]:
         normalized_local = self._normalize_sequence(local_versions)
+        # When folder-filtering, this carries the cross-folder version set
+        # for is_in_library; otherwise it falls back to normalized_local.
+        normalized_all = (
+            self._normalize_sequence(all_local_version_ids)
+            if all_local_version_ids is not None
+            else normalized_local
+        )
         now = time.time()
         async with self._lock:
             existing = self._get_record(model_type, model_id)
@@ -971,6 +998,7 @@ class ModelUpdateService:
                 record = self._merge_with_local_versions(
                     existing,
                     normalized_local,
+                    all_local_version_ids=normalized_all,
                 )
                 self._upsert_record(record)
                 return record
@@ -1046,6 +1074,7 @@ class ModelUpdateService:
                 record = self._merge_with_local_versions(
                     existing,
                     normalized_local,
+                    all_local_version_ids=normalized_all,
                 )
                 self._upsert_record(record)
                 return record
@@ -1057,6 +1086,7 @@ class ModelUpdateService:
                     model_type=model_type,
                     model_id=model_id,
                     last_checked_at=now,
+                    all_local_version_ids=normalized_all,
                 )
                 record = replace(record, should_ignore_model=True)
                 self._upsert_record(record)
@@ -1075,6 +1105,7 @@ class ModelUpdateService:
                     fetched_versions,
                     existing,
                     now,
+                    all_local_version_ids=normalized_all,
                 )
             else:
                 record = self._merge_with_local_versions(
@@ -1083,6 +1114,7 @@ class ModelUpdateService:
                     model_type=model_type,
                     model_id=model_id,
                     last_checked_at=existing.last_checked_at if existing else None,
+                    all_local_version_ids=normalized_all,
                 )
             self._upsert_record(record)
             return record
@@ -1276,6 +1308,7 @@ class ModelUpdateService:
         scanner,
         *,
         target_model_ids: Optional[Sequence[int]] = None,
+        folder_path: Optional[str] = None,
     ) -> Dict[int, List[int]]:
         cache = await scanner.get_cached_data()
         mapping: Dict[int, set[int]] = {}
@@ -1288,7 +1321,19 @@ class ModelUpdateService:
             if not target_set:
                 return {}
 
+        normalized_folder = None
+        if folder_path is not None:
+            normalized_folder = folder_path.replace("\\", "/").strip("/")
+
         for item in cache.raw_data:
+            # Apply folder filter first (cheapest check)
+            if normalized_folder is not None:
+                if not isinstance(item, dict):
+                    continue
+                item_folder = (item.get("folder") or "").replace("\\", "/").strip("/")
+                if item_folder != normalized_folder and not item_folder.startswith(normalized_folder + "/"):
+                    continue
+
             civitai = item.get("civitai") if isinstance(item, dict) else None
             if not isinstance(civitai, dict):
                 continue
@@ -1307,12 +1352,20 @@ class ModelUpdateService:
         existing: Optional[ModelUpdateRecord],
         normalized_local: Sequence[int],
         *,
+        all_local_version_ids: Optional[Sequence[int]] = None,
         model_type: Optional[str] = None,
         model_id: Optional[int] = None,
         last_checked_at: Optional[float] = None,
         version_info: Optional[Mapping] = None,
     ) -> ModelUpdateRecord:
         local_set = set(normalized_local)
+        # When folder-filtering, also consider versions in other folders
+        # as in-library so they are not reported as available updates.
+        effective_local_set: set[int] = (
+            local_set | set(all_local_version_ids)
+            if all_local_version_ids is not None
+            else local_set
+        )
         versions: List[ModelVersionRecord] = []
         ignore_map: Dict[int, bool] = {}
         if existing:
@@ -1324,7 +1377,7 @@ class ModelUpdateService:
                 versions.append(
                     replace(
                         version,
-                        is_in_library=version.version_id in local_set,
+                        is_in_library=version.version_id in effective_local_set,
                     )
                 )
         elif model_type is None or model_id is None:
@@ -1371,8 +1424,17 @@ class ModelUpdateService:
         remote_versions: Sequence[ModelVersionRecord],
         existing: Optional[ModelUpdateRecord],
         timestamp: float,
+        *,
+        all_local_version_ids: Optional[Sequence[int]] = None,
     ) -> ModelUpdateRecord:
         local_set = set(local_versions)
+        # When folder-filtering, also consider versions in other folders
+        # as in-library so they are not reported as available updates.
+        effective_local_set: set[int] = (
+            local_set | set(all_local_version_ids)
+            if all_local_version_ids is not None
+            else local_set
+        )
         ignore_map = {version.version_id: version.should_ignore for version in existing.versions} if existing else {}
         preview_map = {version.version_id: version.preview_url for version in existing.versions} if existing else {}
         sort_map = {version.version_id: version.sort_index for version in existing.versions} if existing else {}
@@ -1391,7 +1453,7 @@ class ModelUpdateService:
                     released_at=remote_version.released_at,
                     size_bytes=remote_version.size_bytes,
                     preview_url=remote_version.preview_url or preview_map.get(version_id),
-                    is_in_library=version_id in local_set,
+                    is_in_library=version_id in effective_local_set,
                     should_ignore=ignore_map.get(version_id, remote_version.should_ignore),
                     sort_index=sort_map.get(version_id, index),
                     early_access_ends_at=remote_version.early_access_ends_at,

py/services/persistent_model_cache.py

@@ -57,6 +57,7 @@ class PersistentModelCache:
         "db_checked",
         "last_checked_at",
         "hash_status",
+        "hf_url",
     )
     _MODEL_UPDATE_COLUMNS: Tuple[str, ...] = _MODEL_COLUMNS[2:]
     _instances: Dict[str, "PersistentModelCache"] = {}
@@ -165,8 +166,8 @@ class PersistentModelCache:
 
             item = {
                 "file_path": file_path,
-                "file_name": row["file_name"],
-                "model_name": row["model_name"],
+                "file_name": row["file_name"] or "",
+                "model_name": row["model_name"] or "",
                 "folder": row["folder"] or "",
                 "size": row["size"] or 0,
                 "modified": row["modified"] or 0.0,
@@ -188,6 +189,7 @@ class PersistentModelCache:
                 "skip_metadata_refresh": bool(row["skip_metadata_refresh"]),
                 "license_flags": int(license_value),
                 "hash_status": row["hash_status"] or "completed",
+                "hf_url": row["hf_url"] or "",
             }
             raw_data.append(item)
 
@@ -452,6 +454,7 @@ class PersistentModelCache:
                             db_checked INTEGER,
                             last_checked_at REAL,
                             hash_status TEXT,
+                            hf_url TEXT DEFAULT '',
                             PRIMARY KEY (model_type, file_path)
                         );
 
@@ -500,6 +503,7 @@ class PersistentModelCache:
             # Persisting without explicit flags should assume CivitAI's documented defaults (0b111001 == 57).
             "license_flags": f"INTEGER DEFAULT {DEFAULT_LICENSE_FLAGS}",
             "hash_status": "TEXT DEFAULT 'completed'",
+            "hf_url": "TEXT DEFAULT ''",
         }
 
         for column, definition in required_columns.items():
@@ -548,19 +552,19 @@ class PersistentModelCache:
         return (
             model_type,
             item.get("file_path"),
-            item.get("file_name"),
-            item.get("model_name"),
-            item.get("folder"),
+            item.get("file_name") or "",
+            item.get("model_name") or "",
+            item.get("folder") or "",
             int(item.get("size") or 0),
             float(item.get("modified") or 0.0),
             (item.get("sha256") or "").lower() or None,
-            item.get("base_model"),
-            item.get("preview_url"),
+            item.get("base_model") or "",
+            item.get("preview_url") or "",
             int(item.get("preview_nsfw_level") or 0),
             1 if item.get("from_civitai", True) else 0,
             1 if item.get("favorite") else 0,
-            item.get("notes"),
-            item.get("usage_tips"),
+            item.get("notes") or "",
+            item.get("usage_tips") or "",
             metadata_source,
             civitai.get("id"),
             civitai.get("modelId"),
@@ -575,6 +579,7 @@ class PersistentModelCache:
             1 if item.get("db_checked") else 0,
             float(item.get("last_checked_at") or 0.0),
             item.get("hash_status", "completed"),
+            item.get("hf_url") or "",
         )
 
     def _insert_model_sql(self) -> str:
@@ -582,6 +587,95 @@ class PersistentModelCache:
         placeholders = ", ".join(["?"] * len(self._MODEL_COLUMNS))
         return f"INSERT INTO models ({columns}) VALUES ({placeholders})"
 
+    def update_single_model(
+        self,
+        model_type: str,
+        new_item: Dict,
+        old_item: Optional[Dict] = None,
+    ) -> None:
+        """Update a single model row in the persistent cache.
+
+        A lightweight alternative to :meth:`save_cache` that performs a targeted
+        DELETE + INSERT for the model row and computes incremental tag / hash-index
+        deltas from *old_item*.  When *old_item* is omitted the previous tags and
+        hash are not cleaned up (callers should only omit it for brand-new entries).
+
+        All operations run inside a single transaction so readers see a consistent
+        view.
+        """
+        if not self.is_enabled():
+            return
+        if not self._schema_initialized:
+            self._initialize_schema()
+        if not self._schema_initialized:
+            return
+
+        file_path: Optional[str] = new_item.get("file_path")
+        if not file_path:
+            return
+
+        try:
+            with self._db_lock:
+                conn = self._connect()
+                try:
+                    conn.execute("PRAGMA foreign_keys = ON")
+                    conn.execute("BEGIN")
+
+                    # --- model row (DELETE + INSERT = upsert) ---
+                    conn.execute(
+                        "DELETE FROM models WHERE model_type = ? AND file_path = ?",
+                        (model_type, file_path),
+                    )
+                    row = self._prepare_model_row(model_type, new_item)
+                    conn.execute(self._insert_model_sql(), row)
+
+                    # --- tags ---
+                    new_tags: set = set(new_item.get("tags") or [])
+                    old_tags: set = set(old_item.get("tags") or []) if old_item else set()
+                    tags_to_delete = old_tags - new_tags
+                    tags_to_insert = new_tags - old_tags
+
+                    if tags_to_delete:
+                        conn.executemany(
+                            "DELETE FROM model_tags WHERE model_type = ? AND file_path = ? AND tag = ?",
+                            [(model_type, file_path, t) for t in tags_to_delete],
+                        )
+                    if tags_to_insert:
+                        conn.executemany(
+                            "INSERT INTO model_tags (model_type, file_path, tag) VALUES (?, ?, ?)",
+                            [(model_type, file_path, t) for t in tags_to_insert],
+                        )
+
+                    # --- hash_index ---
+                    new_sha: Optional[str] = (new_item.get("sha256") or "").lower() or None
+                    old_sha: Optional[str] = (
+                        (old_item.get("sha256") or "").lower() or None
+                    ) if old_item else None
+                    if new_sha != old_sha:
+                        if old_sha:
+                            conn.execute(
+                                "DELETE FROM hash_index WHERE model_type = ? AND sha256 = ? AND file_path = ?",
+                                (model_type, old_sha, file_path),
+                            )
+                        if new_sha:
+                            conn.execute(
+                                "INSERT OR IGNORE INTO hash_index (model_type, sha256, file_path) VALUES (?, ?, ?)",
+                                (model_type, new_sha, file_path),
+                            )
+
+                    conn.execute("COMMIT")
+                except Exception:
+                    conn.execute("ROLLBACK")
+                    raise
+                finally:
+                    conn.close()
+        except Exception as exc:
+            logger.warning(
+                "Failed to update single model in persistent cache (%s): %s",
+                file_path,
+                exc,
+            )
+
     def _load_tags(self, conn: sqlite3.Connection, model_type: str) -> Dict[str, List[str]]:
         tag_rows = conn.execute(
             "SELECT file_path, tag FROM model_tags WHERE model_type = ?",

py/services/persistent_recipe_cache.py

@@ -12,7 +12,7 @@ import logging
 import os
 import sqlite3
 import threading
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from typing import Dict, List, Optional, Set, Tuple
 
 from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration
@@ -26,6 +26,8 @@ class PersistedRecipeData:
 
     raw_data: List[Dict]
     file_stats: Dict[str, Tuple[float, int]]  # json_path -> (mtime, size)
+    image_id_map: Dict[str, str] = field(default_factory=dict)
+    """Precomputed mapping of civitai image_id → recipe_id."""
 
 
 class PersistentRecipeCache:
@@ -116,6 +118,20 @@ class PersistentRecipeCache:
                     if not rows:
                         return None
 
+                    # Restore precomputed image_id_map if available
+                    image_id_map: Dict[str, str] = {}
+                    try:
+                        meta_row = conn.execute(
+                            "SELECT value FROM cache_metadata WHERE key = ?",
+                            ("image_id_map",),
+                        ).fetchone()
+                        if meta_row:
+                            parsed = json.loads(meta_row["value"])
+                            if isinstance(parsed, dict):
+                                image_id_map = parsed
+                    except Exception:
+                        pass  # missing or corrupt — rebuilt on next cache refresh
+
                 finally:
                     conn.close()
         except FileNotFoundError:
@@ -138,14 +154,24 @@ class PersistentRecipeCache:
                     row["file_size"] or 0,
                 )
 
-        return PersistedRecipeData(raw_data=raw_data, file_stats=file_stats)
+        return PersistedRecipeData(
+            raw_data=raw_data,
+            file_stats=file_stats,
+            image_id_map=image_id_map,
+        )
 
-    def save_cache(self, recipes: List[Dict], json_paths: Optional[Dict[str, str]] = None) -> None:
+    def save_cache(
+        self,
+        recipes: List[Dict],
+        json_paths: Optional[Dict[str, str]] = None,
+        image_id_map: Optional[Dict[str, str]] = None,
+    ) -> None:
         """Save all recipes to SQLite cache.
 
         Args:
             recipes: List of recipe dictionaries to persist.
             json_paths: Optional mapping of recipe_id -> json_path for file stats.
+            image_id_map: Optional precomputed civitai image_id → recipe_id mapping.
         """
         if not self.is_enabled():
             return
@@ -186,6 +212,12 @@ class PersistentRecipeCache:
                             recipe_rows,
                         )
 
+                    # Persist image_id_map for O(1) lookups on cache load
+                    conn.execute(
+                        "INSERT OR REPLACE INTO cache_metadata (key, value) VALUES (?, ?)",
+                        ("image_id_map", json.dumps(image_id_map or {})),
+                    )
+
                     conn.commit()
                     logger.debug("Persisted %d recipes to cache", len(recipe_rows))
                 finally:
@@ -273,6 +305,29 @@ class PersistentRecipeCache:
         except Exception as exc:
             logger.debug("Failed to remove recipe %s from cache: %s", recipe_id, exc)
 
+    def save_image_id_map(self, image_id_map: Dict[str, str]) -> None:
+        """Persist the image_id_map to cache_metadata without rewriting the full cache.
+
+        This is called after ``add_recipe`` / ``remove_recipe`` mutations so
+        the persistent copy does not go stale between full ``save_cache`` calls.
+        """
+        if not self.is_enabled() or not self._schema_initialized:
+            return
+
+        try:
+            with self._db_lock:
+                conn = self._connect()
+                try:
+                    conn.execute(
+                        "INSERT OR REPLACE INTO cache_metadata (key, value) VALUES (?, ?)",
+                        ("image_id_map", json.dumps(image_id_map)),
+                    )
+                    conn.commit()
+                finally:
+                    conn.close()
+        except Exception as exc:
+            logger.debug("Failed to persist image_id_map: %s", exc)
+
     def get_indexed_recipe_ids(self) -> Set[str]:
         """Return all recipe IDs in the cache.
 

py/services/recipe_cache.py

@@ -1,7 +1,6 @@
 import asyncio
 from typing import Iterable, List, Dict, Optional
-from dataclasses import dataclass
-from operator import itemgetter
+from dataclasses import dataclass, field
 from natsort import natsorted
 
 
@@ -14,6 +13,15 @@ class RecipeCache:
     sorted_by_date: List[Dict]
     folders: List[str] | None = None
     folder_tree: Dict | None = None
+    image_id_map: Dict[str, str] = field(default_factory=dict)
+    """Mapping of civitai image_id → recipe_id, precomputed at cache build time.
+
+    Built once during cache initialization (O(n)) so that
+    ``check_image_exists`` and ``import_from_url`` duplicate checks
+    can look up image_id in O(1) instead of scanning all recipes.
+    Recipes imported from local files have no valid civitai image_id
+    and are naturally excluded from this map.
+    """
 
     def __post_init__(self):
         self._lock = asyncio.Lock()
@@ -140,5 +148,10 @@ class RecipeCache:
         )
         if not name_only:
             self.sorted_by_date = sorted(
-                self.raw_data, key=itemgetter("created_date", "file_path"), reverse=True
+                self.raw_data,
+                key=lambda x: (
+                    x.get("modified", x.get("created_date", 0)),
+                    x.get("file_path", ""),
+                ),
+                reverse=True,
             )

py/services/recipe_scanner.py

@@ -20,7 +20,8 @@ from .metadata_service import get_default_metadata_provider
 from .checkpoint_scanner import CheckpointScanner
 from .settings_manager import get_settings_manager
 from .recipes.errors import RecipeNotFoundError
-from ..utils.utils import calculate_recipe_fingerprint, fuzzy_match
+from ..utils.civitai_utils import extract_civitai_image_id
+from ..utils.utils import calculate_recipe_fingerprint
 from natsort import natsorted
 import sys
 import re
@@ -65,7 +66,7 @@ class RecipeScanner:
             cls._instance._civitai_client = None  # Will be lazily initialized
         return cls._instance
 
-    REPAIR_VERSION = 3
+    REPAIR_VERSION = 4
 
     def __init__(
         self,
@@ -292,6 +293,32 @@ class RecipeScanner:
         if recipe.get("repair_version", 0) >= self.REPAIR_VERSION:
             return False
 
+        # 1.5 Detect and clear corrupted checkpoint (LoRA data saved as checkpoint).
+        #     A checkpoint whose modelVersionId also appears in a LoRA entry is
+        #     definitely wrong — the CivitAI import code used to pick
+        #     modelVersionIds[0] as the checkpoint, which was often a LoRA.
+        #     Clearing it lets the enrichment flow re-resolve the correct
+        #     checkpoint from CivitAI image metadata.
+        cp = recipe.get("checkpoint")
+        lora_mvids = {
+            l.get("modelVersionId")
+            for l in recipe.get("loras", [])
+            if l.get("modelVersionId")
+        }
+        if cp and cp.get("modelVersionId") and cp["modelVersionId"] in lora_mvids:
+            cp_mvid = cp["modelVersionId"]
+            logger.info(
+                "Recipe %s: checkpoint modelVersionId %s matches a LoRA — "
+                "clearing corrupted checkpoint and removing matching LoRA entry",
+                recipe.get("id"),
+                cp_mvid,
+            )
+            recipe["checkpoint"] = None
+            recipe["loras"] = [
+                l for l in recipe.get("loras", [])
+                if l.get("modelVersionId") != cp_mvid
+            ]
+
         # 2. Identification: Is repair needed?
         has_checkpoint = (
             "checkpoint" in recipe
@@ -506,7 +533,21 @@ class RecipeScanner:
                     self._sort_cache_sync()
                     # Backfill source_path from JSON files if missing (schema migration)
                     if self._backfill_source_path_if_needed(recipes, json_paths):
-                        self._persistent_cache.save_cache(recipes, json_paths)
+                        self._cache.image_id_map = self._build_image_id_map()
+                        self._persistent_cache.save_cache(
+                            recipes, json_paths, self._cache.image_id_map
+                        )
+                    else:
+                        # Use persisted map, or rebuild if empty (e.g. first startup
+                        # after deploying the image_id_map feature).
+                        if persisted.image_id_map:
+                            self._cache.image_id_map = dict(persisted.image_id_map)
+                        else:
+                            self._cache.image_id_map = self._build_image_id_map()
+                            if self._cache.image_id_map:
+                                self._persistent_cache.save_image_id_map(
+                                    self._cache.image_id_map
+                                )
                     return self._cache
                 else:
                     # Partial update: some files changed
@@ -519,8 +560,11 @@ class RecipeScanner:
                     self._sort_cache_sync()
                     # Backfill source_path from JSON files if missing (schema migration)
                     self._backfill_source_path_if_needed(recipes, json_paths)
+                    self._cache.image_id_map = self._build_image_id_map()
                     # Persist updated cache
-                    self._persistent_cache.save_cache(recipes, json_paths)
+                    self._persistent_cache.save_cache(
+                        recipes, json_paths, self._cache.image_id_map
+                    )
                     return self._cache
 
             # Fall back to full directory scan
@@ -532,9 +576,12 @@ class RecipeScanner:
             self._cache.raw_data = recipes
             self._update_folder_metadata(self._cache)
             self._sort_cache_sync()
+            self._cache.image_id_map = self._build_image_id_map()
 
             # Persist for next startup
-            self._persistent_cache.save_cache(recipes, json_paths)
+            self._persistent_cache.save_cache(
+                recipes, json_paths, self._cache.image_id_map
+            )
 
             return self._cache
         except Exception as e:
@@ -806,6 +853,28 @@ class RecipeScanner:
         except Exception as e:
             logger.error(f"Error sorting recipe cache: {e}")
 
+    def _build_image_id_map(self) -> Dict[str, str]:
+        """Build civitai image_id → recipe_id mapping from cached recipes.
+
+        Only recipes with a valid CivitAI image URL source_path produce an
+        entry.  Recipes imported from local files are naturally excluded.
+        """
+        mapping: Dict[str, str] = {}
+        if not self._cache:
+            return mapping
+        for recipe in getattr(self._cache, "raw_data", []):
+            if not isinstance(recipe, dict):
+                continue
+            source = recipe.get("source_path")
+            if not source:
+                continue
+            image_id = extract_civitai_image_id(source)
+            if image_id and image_id not in mapping:
+                recipe_id = recipe.get("id")
+                if recipe_id is not None:
+                    mapping[image_id] = str(recipe_id)
+        return mapping
+
     async def _wait_for_lora_scanner(self) -> None:
         """Ensure the LoRA scanner has initialized before recipe enrichment."""
 
@@ -951,13 +1020,16 @@ class RecipeScanner:
 
         try:
             result = self._fts_index.search(search, fields)
-            # Return None if empty to trigger fuzzy fallback
-            # Empty FTS results may indicate query syntax issues or need for fuzzy matching
+            # Return empty set for empty FTS results — do NOT fall back to
+            # Python fuzzy matching, which freezes the server with 10k+ recipes.
+            # FTS5 prefix matching with unicode61 tokenizer correctly handles
+            # compound tokens (e.g. "illustrious" matches "path/illustrious/model").
+            # If FTS returns nothing, there are genuinely no matching recipes.
             if not result:
-                return None
+                return set()
             return result
         except Exception as exc:
-            logger.debug("FTS search failed, falling back to fuzzy search: %s", exc)
+            logger.debug("FTS search failed, falling back to title-only search: %s", exc)
             return None
 
     def _update_fts_index_for_recipe(
@@ -1270,11 +1342,20 @@ class RecipeScanner:
         # Update FTS index
         self._update_fts_index_for_recipe(recipe_data, "add")
 
+        source = recipe_data.get("source_path")
+        if source:
+            image_id = extract_civitai_image_id(source)
+            if image_id:
+                recipe_id_value = recipe_data.get("id")
+                if recipe_id_value is not None:
+                    cache.image_id_map[image_id] = str(recipe_id_value)
+
         # Persist to SQLite cache
         if self._persistent_cache:
             recipe_id = str(recipe_data.get("id", ""))
             json_path = self._json_path_map.get(recipe_id, "")
             self._persistent_cache.update_recipe(recipe_data, json_path)
+            self._persistent_cache.save_image_id_map(cache.image_id_map)
 
     async def remove_recipe(self, recipe_id: str) -> bool:
         """Remove a recipe from the cache by ID."""
@@ -1293,9 +1374,15 @@ class RecipeScanner:
         # Update FTS index
         self._update_fts_index_for_recipe(recipe_id, "remove")
 
+        # Remove any image_id entry pointing to this recipe
+        stale = [k for k, v in cache.image_id_map.items() if v == recipe_id]
+        for k in stale:
+            del cache.image_id_map[k]
+
         # Remove from SQLite cache
         if self._persistent_cache:
             self._persistent_cache.remove_recipe(recipe_id)
+            self._persistent_cache.save_image_id_map(cache.image_id_map)
             self._json_path_map.pop(recipe_id, None)
 
         return True
@@ -1306,14 +1393,21 @@ class RecipeScanner:
         cache = await self.get_cached_data()
         removed = await cache.bulk_remove(recipe_ids, resort=False)
         if removed:
+            removed_ids = {str(r.get("id", "")) for r in removed}
+            stale = [k for k, v in cache.image_id_map.items() if v in removed_ids]
+            for k in stale:
+                del cache.image_id_map[k]
+
             self._schedule_resort()
-            # Update FTS index and persistent cache for each removed recipe
             for recipe in removed:
                 recipe_id = str(recipe.get("id", ""))
                 self._update_fts_index_for_recipe(recipe_id, "remove")
                 if self._persistent_cache:
                     self._persistent_cache.remove_recipe(recipe_id)
                     self._json_path_map.pop(recipe_id, None)
+
+            if self._persistent_cache:
+                self._persistent_cache.save_image_id_map(cache.image_id_map)
         return len(removed)
 
     async def scan_all_recipes(self) -> List[Dict]:
@@ -1988,49 +2082,14 @@ class RecipeScanner:
                         if str(item.get("id", "")) in fts_matching_ids
                     ]
                 else:
-                    # Fallback to fuzzy_match (slower but always available)
-                    # Build the search predicate based on search options
-                    def matches_search(item):
-                        # Search in title if enabled
-                        if search_options.get("title", True):
-                            if fuzzy_match(str(item.get("title", "")), search):
-                                return True
-
-                        # Search in tags if enabled
-                        if search_options.get("tags", True) and "tags" in item:
-                            for tag in item["tags"]:
-                                if fuzzy_match(tag, search):
-                                    return True
-
-                        # Search in lora file names if enabled
-                        if search_options.get("lora_name", True) and "loras" in item:
-                            for lora in item["loras"]:
-                                if fuzzy_match(str(lora.get("file_name", "")), search):
-                                    return True
-
-                        # Search in lora model names if enabled
-                        if search_options.get("lora_model", True) and "loras" in item:
-                            for lora in item["loras"]:
-                                if fuzzy_match(str(lora.get("modelName", "")), search):
-                                    return True
-
-                        # Search in prompt and negative_prompt if enabled
-                        if search_options.get("prompt", True) and "gen_params" in item:
-                            gen_params = item["gen_params"]
-                            if fuzzy_match(str(gen_params.get("prompt", "")), search):
-                                return True
-                            if fuzzy_match(
-                                str(gen_params.get("negative_prompt", "")), search
-                            ):
-                                return True
-
-                        # No match found
-                        return False
-
-                    # Filter the data using the search predicate
-                    filtered_data = [
-                        item for item in filtered_data if matches_search(item)
-                    ]
+                    # FTS index not yet built — return empty rather than
+                    # scanning 42k+ items in Python. The FTS background build
+                    # finishes in seconds; by the time a user navigates here
+                    # and types a search, it is already available.
+                    logger.debug(
+                        "FTS index not ready — search '%s' returning empty", search
+                    )
+                    filtered_data = []
 
             # Apply additional filters
             if filters:

py/services/recipes/analysis_service.py

@@ -146,11 +146,38 @@ class RecipeAnalysisService:
                 ):
                     metadata = metadata["meta"]
 
-                # Include modelVersionIds from root level if available
-                # Civitai API returns modelVersionIds at root level, not in meta
+                # Include modelVersionIds from root level if available.
+                # CivitAI API returns modelVersionIds at root level, not in meta.
+                # When meta is null (None), create a minimal dict so downstream
+                # parsers can still discover LoRAs and checkpoints.
                 model_version_ids = image_info.get("modelVersionIds")
-                if model_version_ids and isinstance(metadata, dict):
-                    metadata["modelVersionIds"] = model_version_ids
+                if model_version_ids:
+                    if isinstance(metadata, dict):
+                        metadata["modelVersionIds"] = model_version_ids
+                    else:
+                        metadata = {"modelVersionIds": model_version_ids}
+
+                # Inject browsingLevel (canonical integer) so the recipe's
+                # preview_nsfw_level can be set, enabling proper NSFW blur
+                # of the preview image.  Fall back to nsfwLevel (string)
+                # when browsingLevel is absent.
+                if isinstance(metadata, dict):
+                    browsing_level = image_info.get("browsingLevel")
+                    nsfw_level_str = image_info.get("nsfwLevel")
+                    if isinstance(browsing_level, int) and browsing_level > 0:
+                        metadata["browsingLevel"] = browsing_level
+                    elif (
+                        isinstance(nsfw_level_str, str)
+                        and nsfw_level_str
+                        in (
+                            "PG", "PG13", "R", "X", "XXX", "Blocked",
+                        )
+                    ):
+                        from ...utils.constants import NSFW_LEVELS
+
+                        metadata["browsingLevel"] = NSFW_LEVELS.get(
+                            nsfw_level_str, 0
+                        )
 
                 # Validate that metadata contains meaningful recipe fields
                 # If not, treat as None to trigger EXIF extraction from downloaded image
@@ -171,13 +198,61 @@ class RecipeAnalysisService:
                 temp_path = self._create_temp_path(suffix=extension)
                 await self._download_image(url, temp_path)
 
-            if metadata is None and not is_video:
-                metadata = await asyncio.to_thread(
+            # Always extract EXIF from the downloaded image for generation
+            # params (prompt, negative prompt, sampler, steps, etc.).
+            # Previously this was gated on ``metadata is None``, but that
+            # skipped EXIF entirely when API metadata (modelVersionIds,
+            # browsingLevel) is present, losing all generation parameters.
+            exif_metadata = None
+            if not is_video:
+                exif_metadata = await asyncio.to_thread(
                     self._exif_utils.extract_image_metadata, temp_path
                 )
 
+            # Fallback: try the original (non-optimized) image for EXIF data
+            if not exif_metadata and civitai_image_id and image_info:
+                original_url = image_info.get("url")
+                if original_url:
+                    self._logger.debug(
+                        "Optimized image lacks embedded metadata, "
+                        "falling back to original image: %s",
+                        original_url,
+                    )
+                    orig_temp_path = self._create_temp_path(suffix=".png")
+                    try:
+                        await self._download_image(original_url, orig_temp_path)
+                        exif_metadata = await asyncio.to_thread(
+                            self._exif_utils.extract_image_metadata,
+                            orig_temp_path,
+                        )
+                    finally:
+                        self._safe_cleanup(orig_temp_path)
+
+            # Parse EXIF data (typically a string like parameters/prompt/workflow)
+            # and API metadata (dict with modelVersionIds, browsingLevel) separately,
+            # then merge: API loras/checkpoint override, EXIF gen_params fill in gaps.
+            # This mirrors the two-pass approach in _do_import_from_url.
+            exif_parsed_result = None
+            if isinstance(exif_metadata, str):
+                exif_parser = self._recipe_parser_factory.create_parser(exif_metadata)
+                if exif_parser:
+                    exif_data = await exif_parser.parse_metadata(
+                        exif_metadata, recipe_scanner=recipe_scanner,
+                    )
+                    if exif_data and not exif_data.get("error"):
+                        exif_parsed_result = exif_data
+
+            # Merge API metadata (dict) with EXIF data (if dict) for the
+            # CivitaiApiMetadataParser.  If EXIF data is a string it was
+            # parsed above — don't try to merge a string into a dict.
+            merged = {}
+            if isinstance(exif_metadata, dict):
+                merged.update(exif_metadata)
+            if isinstance(metadata, dict):
+                merged.update(metadata)
+
             result = await self._parse_metadata(
-                metadata or {},
+                merged,
                 recipe_scanner=recipe_scanner,
                 image_path=temp_path,
                 include_image_base64=True,
@@ -185,13 +260,23 @@ class RecipeAnalysisService:
                 extension=extension,
             )
 
-            if civitai_image_id and image_info and not result.payload.get("error"):
-                mvid = image_info.get("modelVersionId")
-                if not mvid:
-                    mvids = image_info.get("modelVersionIds")
-                    if isinstance(mvids, list) and mvids:
-                        mvid = mvids[0]
+            # Merge EXIF string-parsed gen_params into the API result.
+            # API gen_params take priority (they come later via update).
+            if exif_parsed_result and not result.payload.get("error"):
+                exif_gp = exif_parsed_result.get("gen_params") or {}
+                result_gp = result.payload.get("gen_params") or {}
+                merged_gp = {**exif_gp, **result_gp}
+                if merged_gp:
+                    result.payload["gen_params"] = merged_gp
 
+            if civitai_image_id and image_info and not result.payload.get("error"):
+                # Use the metadata dict we built (may contain modelVersionIds
+                # and browsingLevel from the API root level).  Do NOT pass
+                # image_info.get("meta") — it is null for images whose meta
+                # lives at the root level only.  Also do NOT derive
+                # model_version_id from modelVersionIds[0] — that array mixes
+                # checkpoints, LoRAs, and other types without ordering
+                # guarantees; the parser already resolved them correctly.
                 recipe_for_enrich = {
                     "gen_params": result.payload.get("gen_params", {}),
                     "loras": result.payload.get("loras", []),
@@ -204,8 +289,10 @@ class RecipeAnalysisService:
                     recipe=recipe_for_enrich,
                     civitai_client=civitai_client,
                     request_params=None,
-                    prefetched_civitai_meta_raw=image_info.get("meta"),
-                    prefetched_model_version_id=mvid,
+                    prefetched_civitai_meta_raw=(
+                        metadata if isinstance(metadata, dict) else None
+                    ),
+                    prefetched_model_version_id=None,
                 )
 
                 result.payload["gen_params"] = recipe_for_enrich["gen_params"]
@@ -214,6 +301,12 @@ class RecipeAnalysisService:
                 if recipe_for_enrich.get("base_model"):
                     result.payload["base_model"] = recipe_for_enrich["base_model"]
 
+                # Extract browsingLevel from our constructed metadata for NSFW blur
+                if isinstance(metadata, dict):
+                    bl = metadata.get("browsingLevel")
+                    if isinstance(bl, int) and bl > 0:
+                        result.payload["preview_nsfw_level"] = bl
+
             return result
         finally:
             if temp_path:
@@ -296,6 +389,10 @@ class RecipeAnalysisService:
             "prompt_type",
             "positive",
             "negative",
+            # modelVersionIds is injected at the root level by CivitAI's image
+            # API when meta is null. It carries the version IDs of ALL models
+            # (checkpoint + LoRAs) used to generate the image.
+            "modelVersionIds",
         }
         return any(field in metadata for field in recipe_fields)
 

py/services/recipes/persistence_service.py

@@ -49,8 +49,18 @@ class RecipePersistenceService:
         tags: Iterable[str],
         metadata: Optional[dict[str, Any]],
         extension: str | None = None,
+        recipe_id: str | None = None,
+        target_dir: str | None = None,
     ) -> PersistenceResult:
-        """Persist a user uploaded recipe."""
+        """Persist a user uploaded recipe.
+
+        Args:
+            recipe_id: If provided, reuse this ID instead of generating a new
+                UUID. Used by re-import to preserve the original recipe identity.
+            target_dir: If provided, save recipe files to this directory instead
+                of the default recipes_dir. Used by re-import to preserve the
+                original folder location.
+        """
 
         missing_fields = []
         if not name:
@@ -63,10 +73,10 @@ class RecipePersistenceService:
             )
 
         resolved_image_bytes = self._resolve_image_bytes(image_bytes, image_base64)
-        recipes_dir = recipe_scanner.recipes_dir
+        recipes_dir = target_dir or recipe_scanner.recipes_dir
         os.makedirs(recipes_dir, exist_ok=True)
 
-        recipe_id = str(uuid.uuid4())
+        recipe_id = recipe_id or str(uuid.uuid4())
         
         # Handle video formats by bypassing optimization and metadata embedding
         is_video = extension in [".mp4", ".webm"]
@@ -115,6 +125,22 @@ class RecipePersistenceService:
         if metadata.get("source_path"):
             recipe_data["source_path"] = metadata.get("source_path")
 
+        nsfw_level = metadata.get("preview_nsfw_level")
+        if nsfw_level is not None and isinstance(nsfw_level, int):
+            recipe_data["preview_nsfw_level"] = nsfw_level
+
+        # Compute recipe folder relative to recipes root, mirroring
+        # RecipeScanner._calculate_folder() which is only called during scan/load.
+        if recipe_scanner.recipes_dir:
+            recipe_file_dir = os.path.dirname(normalized_image_path)
+            try:
+                relative_folder = os.path.relpath(recipe_file_dir, recipe_scanner.recipes_dir)
+                if relative_folder in (".", ""):
+                    relative_folder = ""
+                recipe_data["folder"] = relative_folder.replace(os.path.sep, "/")
+            except Exception:
+                recipe_data["folder"] = ""
+
         json_filename = f"{recipe_id}.recipe.json"
         json_path = os.path.join(recipes_dir, json_filename)
         json_path = os.path.normpath(json_path)
@@ -190,11 +216,12 @@ class RecipePersistenceService:
             "preview_nsfw_level",
             "favorite",
             "gen_params",
+            "base_model",
         )
 
         if not any(key in updates for key in allowed_fields):
             raise RecipeValidationError(
-                "At least one field to update must be provided (title or tags or source_path or preview_nsfw_level or favorite or gen_params)"
+                "At least one field to update must be provided (title or tags or source_path or preview_nsfw_level or favorite or gen_params or base_model)"
             )
 
         if "gen_params" in updates and not isinstance(updates["gen_params"], dict):

py/services/service_registry.py

@@ -188,6 +188,25 @@ class ServiceRegistry:
             logger.debug(f"Created and registered {service_name}")
             return service
 
+    @classmethod
+    async def get_download_queue_service(cls):
+        """Get or create the download queue service."""
+        service_name = "download_queue_service"
+
+        if service_name in cls._services:
+            return cls._services[service_name]
+
+        async with cls._get_lock(service_name):
+            if service_name in cls._services:
+                return cls._services[service_name]
+
+            from .download_queue_service import DownloadQueueService
+
+            service = await DownloadQueueService.get_instance()
+            cls._services[service_name] = service
+            logger.debug(f"Created and registered {service_name}")
+            return service
+
     @classmethod
     async def get_backup_service(cls):
         """Get or create the backup service."""

py/services/settings_manager.py

@@ -65,6 +65,8 @@ DEFAULT_SETTINGS: Dict[str, Any] = {
     "onboarding_completed": False,
     "dismissed_banners": [],
     "enable_metadata_archive_db": False,
+    "enable_civarchive_api": True,
+    "metadata_provider_order": "civitai_archive_sqlite",
     "proxy_enabled": False,
     "proxy_host": "",
     "proxy_port": "",
@@ -91,7 +93,6 @@ DEFAULT_SETTINGS: Dict[str, Any] = {
     "autoplay_on_hover": False,
     "display_density": "default",
     "card_info_display": "always",
-    "show_folder_sidebar": True,
     "include_trigger_words": False,
     "compact_mode": False,
     "priority_tags": DEFAULT_PRIORITY_TAG_CONFIG.copy(),
@@ -99,13 +100,20 @@ DEFAULT_SETTINGS: Dict[str, Any] = {
     "lora_syntax_format": "legacy",
     "model_card_footer_action": "replace_preview",
     "show_version_on_card": True,
-    "update_flag_strategy": "same_base",
+    "version_grouping": "same_base",
     "auto_organize_exclusions": [],
     "metadata_refresh_skip_paths": [],
     "skip_previously_downloaded_model_versions": False,
     "download_skip_base_models": [],
     "backup_auto_enabled": True,
     "backup_retention_count": 5,
+    "use_new_license_icons": True,
+    "group_by_model": False,
+    # AI / LLM provider configuration (BYOK)
+    "llm_provider": "openai",  # "openai" | "ollama" | "custom"
+    "llm_api_key": "",
+    "llm_api_base": "",  # empty = provider default
+    "llm_model": "",  # e.g. "gpt-4o-mini"
 }
 
 
@@ -134,6 +142,9 @@ class SettingsManager:
         self._template_path = (
             Path(__file__).resolve().parents[2] / "settings.json.example"
         )
+        # Known placeholder value in settings.json.example; any file containing
+        # this value should be treated as "not configured".
+        self._TEMPLATE_PLACEHOLDER_API_KEY = "your_civitai_api_key_here"
         self.settings = self._load_settings()
         self._migrate_setting_keys()
         self._ensure_default_settings()
@@ -143,6 +154,11 @@ class SettingsManager:
         self._check_environment_variables()
         self._collect_configuration_warnings()
 
+        if os.environ.get("LORA_MANAGER_PORTABLE", "0") == "1":
+            if not self.settings.get("use_portable_settings"):
+                self.settings["use_portable_settings"] = True
+                self._save_settings()
+
         if self._needs_initial_save:
             self._save_settings()
             self._needs_initial_save = False
@@ -165,6 +181,12 @@ class SettingsManager:
                     self._original_disk_payload = copy.deepcopy(data)
                     if self._matches_template_payload(data):
                         self._preserve_disk_template = True
+                    # Clean up the template placeholder so it is not treated
+                    # as a real key (affects both the frontend boolean and
+                    # the downloader's Authorization header).
+                    placeholder = self._TEMPLATE_PLACEHOLDER_API_KEY
+                    if data.get("civitai_api_key") == placeholder:
+                        data["civitai_api_key"] = ""
                 return data
             except json.JSONDecodeError as exc:
                 logger.error("Failed to parse settings.json: %s", exc)
@@ -610,12 +632,37 @@ class SettingsManager:
 
         return False
 
+    @staticmethod
+    def _normalize_path_set(paths: Iterable[str]) -> set[str]:
+        """Normalize an iterable of paths for set-based overlap comparison.
+
+        Resolves symlinks via ``os.path.realpath`` when the path exists on disk,
+        then applies ``os.path.normcase`` + ``os.path.normpath`` for consistent
+        cross-platform comparison.  Non-string / empty entries are skipped.
+        """
+        result: set[str] = set()
+        for p in paths:
+            if not isinstance(p, str):
+                continue
+            stripped = p.strip()
+            if not stripped:
+                continue
+            if os.path.exists(stripped):
+                stripped = os.path.normpath(os.path.realpath(stripped))
+            result.add(os.path.normcase(stripped))
+        return result
+
     def _validate_folder_paths(
         self,
         library_name: str,
         folder_paths: Mapping[str, Iterable[str]],
     ) -> None:
-        """Ensure folder paths do not overlap with other libraries."""
+        """Ensure folder paths do not overlap with other libraries.
+
+        Also detects checkpoints ↔ unet path overlap within the same library
+        (including via symlink resolution), which is a configuration error since
+        these model types must use separate physical folders.
+        """
         libraries = self.settings.get("libraries", {})
         normalized_new: Dict[str, Dict[str, str]] = {}
         for key, values in folder_paths.items():
@@ -653,6 +700,22 @@ class SettingsManager:
                         f"Folder path(s) {collisions} already assigned to library '{other_name}'"
                     )
 
+        # Checkpoints ↔ unet overlap within the same library
+        ckpt_paths = folder_paths.get("checkpoints", []) or []
+        unet_paths = folder_paths.get("unet", []) or []
+        if ckpt_paths and unet_paths:
+            ckpt_real = self._normalize_path_set(ckpt_paths)
+            unet_real = self._normalize_path_set(unet_paths)
+            overlap = ckpt_real & unet_real
+            if overlap:
+                collisions = ", ".join(sorted(overlap))
+                raise ValueError(
+                    f"Path(s) {collisions} are configured for both "
+                    f"'checkpoints' and 'unet' (diffusion models). "
+                    f"These model types must use separate physical folders. "
+                    f"Please remove one of the conflicting entries."
+                )
+
     def _update_active_library_entry(
         self,
         *,
@@ -735,6 +798,7 @@ class SettingsManager:
             "includeTriggerWords": "include_trigger_words",
             "compactMode": "compact_mode",
             "modelCardFooterAction": "model_card_footer_action",
+            "update_flag_strategy": "version_grouping",
         }
 
         updated = False
@@ -862,6 +926,23 @@ class SettingsManager:
             self.settings["civitai_api_key"] = env_api_key
             self._save_settings()
 
+        # LLM provider overrides
+        llm_env_map = {
+            "LLM_API_KEY": "llm_api_key",
+            "LLM_MODEL": "llm_model",
+            "LLM_API_BASE": "llm_api_base",
+            "LLM_PROVIDER": "llm_provider",
+        }
+        llm_changed = False
+        for env_var, settings_key in llm_env_map.items():
+            env_val = os.environ.get(env_var)
+            if env_val:
+                logger.info("Found %s environment variable", env_var)
+                self.settings[settings_key] = env_val
+                llm_changed = True
+        if llm_changed:
+            self._save_settings()
+
     def _default_settings_actions(self) -> List[Dict[str, Any]]:
         return [
             {
@@ -1509,8 +1590,12 @@ class SettingsManager:
             portable_switch_pending = True
             self._prepare_portable_switch(value)
         if key == "folder_paths" and isinstance(value, Mapping):
+            active_name = self.get_active_library_name()
+            self._validate_folder_paths(active_name, value)
             self._update_active_library_entry(folder_paths=value)  # type: ignore[arg-type]
         elif key == "extra_folder_paths" and isinstance(value, Mapping):
+            active_name = self.get_active_library_name()
+            self._validate_folder_paths(active_name, value)
             self._update_active_library_entry(extra_folder_paths=value)  # type: ignore[arg-type]
         elif key == "default_lora_root":
             self._update_active_library_entry(default_lora_root=str(value))
@@ -1557,7 +1642,7 @@ class SettingsManager:
         previous_dir = os.path.dirname(previous_path) or target_dir
 
         if os.path.abspath(previous_path) != os.path.abspath(target_path):
-            self._copy_model_cache_directory(previous_dir, target_dir)
+            self._migrate_settings_directory_content(previous_dir, target_dir)
             logger.info("Switching settings file to: %s", target_path)
 
         self._pending_portable_switch = {"other_path": other_path}
@@ -1592,46 +1677,52 @@ class SettingsManager:
         finally:
             self._pending_portable_switch = None
 
-    def _copy_model_cache_directory(self, source_dir: str, target_dir: str) -> None:
-        """Copy model_cache artifacts when switching storage locations."""
+    def _migrate_settings_directory_content(
+        self, source_dir: str, target_dir: str
+    ) -> None:
+        """Migrate settings directory subdirectories when switching storage locations.
+
+        Copies the canonical subdirectories (cache, backups, logs, stats, wildcards)
+        from the old settings directory to the new one. Legacy cache artifacts
+        (model_cache, recipe_cache, etc.) are migrated lazily by
+        ``resolve_cache_path_with_migration`` on first access.
+
+        Args:
+            source_dir: The previous settings directory path.
+            target_dir: The new settings directory path.
+        """
 
         if not source_dir or not target_dir:
             return
 
-        source_cache_dir = os.path.join(source_dir, "model_cache")
-        target_cache_dir = os.path.join(target_dir, "model_cache")
-        if os.path.isdir(source_cache_dir) and os.path.abspath(
-            source_cache_dir
-        ) != os.path.abspath(target_cache_dir):
-            try:
-                shutil.copytree(
-                    source_cache_dir,
-                    target_cache_dir,
-                    dirs_exist_ok=True,
-                    ignore=shutil.ignore_patterns("*.sqlite-shm", "*.sqlite-wal"),
-                )
-            except Exception as exc:
-                logger.warning(
-                    "Failed to copy model_cache directory from %s to %s: %s",
-                    source_cache_dir,
-                    target_cache_dir,
-                    exc,
-                )
+        def _copy_dir(name: str) -> None:
+            source = os.path.join(source_dir, name)
+            target = os.path.join(target_dir, name)
+            if os.path.isdir(source) and os.path.abspath(source) != os.path.abspath(
+                target
+            ):
+                try:
+                    shutil.copytree(
+                        source,
+                        target,
+                        dirs_exist_ok=True,
+                        ignore=shutil.ignore_patterns("*.sqlite-shm", "*.sqlite-wal"),
+                    )
+                except Exception as exc:
+                    logger.warning(
+                        "Failed to copy directory %s from %s to %s: %s",
+                        name,
+                        source,
+                        target,
+                        exc,
+                    )
 
-        source_cache_file = os.path.join(source_dir, "model_cache.sqlite")
-        target_cache_file = os.path.join(target_dir, "model_cache.sqlite")
-        if os.path.isfile(source_cache_file) and os.path.abspath(
-            source_cache_file
-        ) != os.path.abspath(target_cache_file):
-            try:
-                shutil.copy2(source_cache_file, target_cache_file)
-            except Exception as exc:
-                logger.warning(
-                    "Failed to copy model_cache.sqlite from %s to %s: %s",
-                    source_cache_file,
-                    target_cache_file,
-                    exc,
-                )
+        # Managed subdirectories under settings_dir
+        _copy_dir("cache")
+        _copy_dir("backups")
+        _copy_dir("logs")
+        _copy_dir("stats")
+        _copy_dir("wildcards")
 
     def _get_user_config_directory(self) -> str:
         """Return the user configuration directory, falling back to ~/.config."""
@@ -1758,6 +1849,9 @@ class SettingsManager:
                 if key in self.settings:
                     minimal[key] = copy.deepcopy(self.settings[key])
 
+            if self.settings.get("use_portable_settings"):
+                minimal["use_portable_settings"] = True
+
             if self._seed_template:
                 for key, value in self._seed_template.items():
                     minimal.setdefault(key, copy.deepcopy(value))

py/services/tag_update_service.py

@@ -36,9 +36,9 @@ class TagUpdateService:
             if isinstance(tag, str) and tag.strip():
                 # Convert all tags to lowercase to avoid case sensitivity issues on Windows
                 normalized = tag.strip().lower()
-                if normalized.lower() not in existing_lower:
+                if normalized not in existing_lower:
                     existing_tags.append(normalized)
-                    existing_lower.append(normalized.lower())
+                    existing_lower.append(normalized)
                     tags_added.append(normalized)
 
         metadata["tags"] = existing_tags

py/services/use_cases/bulk_metadata_refresh_use_case.py

@@ -3,6 +3,7 @@
 from __future__ import annotations
 
 import logging
+import time
 from typing import Any, Dict, List, Optional, Protocol, Sequence
 
 from ..metadata_sync_service import MetadataSyncService
@@ -50,6 +51,10 @@ class BulkMetadataRefreshUseCase:
             if not model.get("skip_metadata_refresh", False)
             and not self._is_in_skip_path(model.get("folder", ""), skip_paths)
             and (not model.get("civitai") or not model["civitai"].get("id"))
+            # Skip models downloaded from Hugging Face — they are not on
+            # CivitAI / CivArchive.  Users can still refresh them individually
+            # via the right-click context menu.
+            and not model.get("hf_url", "")
             and not (
                 # Skip models confirmed not on CivitAI when no need to retry
                 model.get("from_civitai") is False
@@ -62,26 +67,48 @@ class BulkMetadataRefreshUseCase:
         ]
 
         total_to_process = len(to_process)
+        initial_skipped = total_models - total_to_process  # models excluded from fetch queue
         processed = 0
         success = 0
+        skipped_count = initial_skipped
+        handled_count = initial_skipped
         needs_resort = False
+        start_time = time.monotonic()
+        failures: List[Dict[str, str]] = []
 
         self._service.scanner.reset_cancellation()
 
         async def emit(status: str, **extra: Any) -> None:
             if progress_callback is None:
                 return
-            payload = {"status": status, "total": total_to_process, "processed": processed, "success": success}
+            payload = {
+                "status": status,
+                "total": total_models,
+                "processed": processed,
+                "success": success,
+                "failure_count": len(failures),
+                "skipped_count": skipped_count,
+                "handled": handled_count,
+                "elapsed_seconds": int(time.monotonic() - start_time),
+            }
+            # Only include full failure details in terminal emits (completed,
+            # cancelled, rate_limited) to avoid serializing the list on every
+            # per-model progress update.
+            if failures and status in ("completed", "cancelled", "rate_limited"):
+                payload["failures"] = failures
             payload.update(extra)
             await progress_callback.on_progress(payload)
 
         await emit("started")
 
+        RATE_LIMIT_ABORT_THRESHOLD = 3
+        consecutive_rate_limits = 0
+
         for model in to_process:
             if self._service.scanner.is_cancelled():
                 self._logger.info("Bulk metadata refresh cancelled by user")
                 await emit("cancelled", processed=processed, success=success)
-                return {"success": False, "message": "Operation cancelled", "processed": processed, "updated": success, "total": total_models}
+                return {"success": False, "message": "Operation cancelled", "processed": processed, "updated": success, "total": total_models, "failures": failures, "failure_count": len(failures), "skipped_count": skipped_count, "elapsed_seconds": int(time.monotonic() - start_time)}
             try:
                 original_name = model.get("model_name")
 
@@ -99,33 +126,89 @@ class BulkMetadataRefreshUseCase:
                         if sha256:
                             model["sha256"] = sha256
                             model["hash_status"] = "completed"
+                            hash_status = "completed"
                         else:
                             self._logger.error(f"Failed to calculate hash for {file_path}")
+                            failures.append({"name": model.get("model_name", file_path or "Unknown"), "error": "Failed to calculate hash"})
                             processed += 1
+                            handled_count += 1
                             continue
                     else:
                         self._logger.warning(f"Scanner does not support lazy hash calculation for {file_path}")
+                        skipped_count += 1
                         processed += 1
+                        handled_count += 1
                         continue
 
                 # Skip models without valid hash
                 if not model.get("sha256"):
                     self._logger.warning(f"Skipping model without hash: {file_path}")
+                    skipped_count += 1
                     processed += 1
+                    handled_count += 1
                     continue
 
                 await MetadataManager.hydrate_model_data(model)
-                result, _ = await self._metadata_sync.fetch_and_update_model(
+
+                # hydrate_model_data replaces model with .metadata.json content,
+                # which may lack sha256. Restore from cache and persist the fix.
+                if not model.get("sha256"):
+                    model["sha256"] = sha256
+                    model["hash_status"] = model.get("hash_status", hash_status)
+                    data_to_save = model.copy()
+                    data_to_save.pop("folder", None)
+                    await MetadataManager.save_metadata(file_path, data_to_save)
+
+                result, error_msg = await self._metadata_sync.fetch_and_update_model(
                     sha256=model["sha256"],
                     file_path=model["file_path"],
                     model_data=model,
                     update_cache_func=self._service.scanner.update_single_model_cache,
                 )
+
+                if not result and error_msg and "Rate limited" in error_msg:
+                    consecutive_rate_limits += 1
+                else:
+                    consecutive_rate_limits = 0
+
+                if not result:
+                    current_name = model.get("model_name", file_path or "Unknown")
+                    failures.append({"name": current_name, "error": error_msg or "Unknown error"})
+                    self._logger.warning("Failed to fetch metadata for %s: %s", current_name, error_msg)
+
+                if consecutive_rate_limits >= RATE_LIMIT_ABORT_THRESHOLD:
+                    # The current model was attempted and failed due to rate limiting;
+                    # count it before aborting so the summary is consistent.
+                    processed += 1
+                    handled_count += 1
+                    self._logger.warning(
+                        "Bulk metadata refresh aborted: %d consecutive rate limits detected. "
+                        "Processed %d/%d models.",
+                        consecutive_rate_limits,
+                        processed,
+                        total_to_process,
+                    )
+                    await emit(
+                        "rate_limited",
+                    )
+                    return {
+                        "success": False,
+                        "message": f"Rate limit detected; {total_to_process - processed} models skipped",
+                        "processed": processed,
+                        "updated": success,
+                        "total": total_models,
+                        "failures": failures,
+                        "failure_count": len(failures),
+                        "skipped_count": skipped_count,
+                        "elapsed_seconds": int(time.monotonic() - start_time),
+                    }
+
                 if result:
                     success += 1
                     if original_name != model.get("model_name"):
                         needs_resort = True
                 processed += 1
+                handled_count += 1
                 await emit(
                     "processing",
                     processed=processed,
@@ -134,6 +217,9 @@ class BulkMetadataRefreshUseCase:
                 )
             except Exception as exc:  # pragma: no cover - logging path
                 processed += 1
+                handled_count += 1
+                current_name = model.get("model_name", model.get("file_path", "Unknown"))
+                failures.append({"name": current_name, "error": str(exc)})
                 self._logger.error(
                     "Error fetching CivitAI data for %s: %s",
                     model.get("file_path"),
@@ -150,7 +236,7 @@ class BulkMetadataRefreshUseCase:
             f"{success} of {processed} processed {self._service.model_type}s (total: {total_models})"
         )
 
-        return {"success": True, "message": message, "processed": processed, "updated": success, "total": total_models}
+        return {"success": True, "message": message, "processed": processed, "updated": success, "total": total_models, "failures": failures, "failure_count": len(failures), "skipped_count": skipped_count, "elapsed_seconds": int(time.monotonic() - start_time)}
 
     @staticmethod
     def _is_in_skip_path(folder: str, skip_paths: List[str]) -> bool: