Compare commits

...

9 Commits

Author SHA1 Message Date
Will Miao e57e11897e refactor(services): share weight-file extension set between rematch and find_matching_models 2026-08-19 21:54:22 +08:00
Will Miao 8a16034135 refactor(services): unify local model name matching with uniqueness and base-model guards (#1065)
Consolidate the duplicate name-matching logic into ModelScanner:
find_matching_models is now the single core, using each scanner's own
file_extensions for suffix stripping. get_model_info_by_name gains
require_unique/base_model kwargs while legacy route behavior is kept
byte-identical. reconnect_lora passes the recipe base model as a guard
and distinguishes ambiguous, base-model-mismatched, and missing LoRAs
in its error messages.
2026-08-19 21:02:04 +08:00
Will Miao 7fc3b7e5be docs: fix star history chart with official token-based embed (#1066) 2026-08-19 20:51:23 +08:00
Aaalice b0c7a1baae Fix recipe parsing for metadata-free local LoRAs (#1065)
* fix(recipes): resolve metadata-free local LoRAs

* fix(recipes): prioritize LoRA hashes over names
2026-08-19 19:07:17 +08:00
Will Miao 6411d83d46 fix(i18n): translate remaining untranslated UI strings 2026-08-19 18:59:55 +08:00
Will Miao 74a063b0e5 fix(i18n): complete translations for per-file download UI (#1058) 2026-08-19 18:53:31 +08:00
Will Miao 96376e5cce fix(download): hide URL step when file dialog opens from versions tab (#1058) 2026-08-19 18:35:16 +08:00
Will Miao e7c26bf722 feat(download): per-file download status and multi-file selection (#1058) 2026-08-19 17:51:31 +08:00
Will Miao cef4129fc9 fix(download): allow downloading additional files of an in-library model version (#1058) 2026-08-19 16:29:59 +08:00
39 changed files with 24409 additions and 21259 deletions
+1 -1
View File
@@ -350,4 +350,4 @@ Join our Discord community for support, discussions, and updates:
---
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=willmiao/ComfyUI-Lora-Manager&type=Date)](https://star-history.com/#willmiao/ComfyUI-Lora-Manager&Date)
[![Star History Chart](https://api.star-history.com/chart?repos=willmiao/ComfyUI-Lora-Manager&type=date&legend=top-left&sealed_token=Ly6HDvUHnRrM7jWpL67I1aYGnmRiLyxEMcrPz8tNEHpaIAzxVsJVO6trhjWB2gG333EyqRL8ygW0pETjKgRHzoePaQXtaw3xIt6ODjo_5zER74xU51xLVruGOi9kkb16G7Mv8Unq5PkBtQW88tPa53b2jzwZYbDa7GP_k3K8aiA49uk1Vd4yU4uG1q-0)](https://www.star-history.com/?repos=willmiao%2FComfyUI-Lora-Manager&type=date&legend=top-left)
@@ -0,0 +1,206 @@
# Plan: Multi-File Downloads Within a Single CivitAI Model Version
**Issue:** [#1058 — Cannot download multiple file variants from the same model version](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1058)
**Status:** v2 — revised after adversarial review (backend correctness + frontend/tests)
**Scope:** CivitAI/CivArchive downloads of `lora`, `checkpoint`, `embedding` model types. HuggingFace downloads are out of scope (already per-file).
> v2 changelog: incorporated 18 review findings. Key changes vs v1:
> shared file resolver + `resolved_version_id` for the gate (R1); `file_params` normalization at API boundary (R2); D2 hash-matching rule fixed for empty-hash cases (R6/R7); D3 extended to re-point `version_index` on removal (R4); D4 replaced with a child table (R3); `delete_model_version` interaction documented (R5); `ModelVersionsTab` surface added to phase 2 (F6); phase-2 multi-file loop requires a reload-deferred download variant (F7); queue-retry `file_params=NULL` known issue recorded (R9); test-fixture gaps and revised estimates (F10).
---
## 1. Problem Statement
A CivitAI model version can contain multiple downloadable weight files (e.g. fp16/fp32, safetensors/ckpt, different sizes). LoRA Manager already has a working file-selection pipeline (frontend file dialog → `fileParams` → backend file matching), but downloaded state is tracked at the **model-version** level. After any single file of a version is downloaded:
1. The version is marked **In Library** and the file-selection entry point disappears.
2. The backend rejects further download attempts for that version.
There is no way to download the remaining files of the same version through LoRA Manager.
## 2. Current State (verified against code; all references confirmed by review)
### 2.1 Download gating — backend (`py/services/download_manager.py`)
`_execute_original_download` enforces two version-level gates:
- **Library gate, early** (lines 11571184, before metadata fetch, fires when `model_version_id` given) and **late** (lines 13501376, fires only when `model_version_id is None`): `scanner.check_model_version_exists(version_id)` across lora/checkpoint/embedding scanners → hard error `"Model version already exists in ... library"`.
- **History gate** (lines 12381279): when `skip_previously_downloaded_model_versions` setting is on, `_has_been_downloaded(model_type, version_id)` → silent skip. History DB primary key is `(model_type, version_id)` (`py/services/downloaded_version_history_service.py:61`).
File selection works: `file_params {id, type, format, size, fp}` is matched against `version_info.files` (lines 14981569), **but only under `if file_params and model_version_id:` (line 1499)** — with `model_id`-only requests the selection silently falls back to the primary file (15711619). `file_params` currently carries no file `name` or hash.
### 2.2 Downloaded-state surfacing — backend (`py/routes/handlers/model_handlers.py`)
`get_civitai_versions` (lines 21482188) sets per-version `existsLocally` via `cache.version_index.get(version_id)` (plus a single `localPath` from that entry) and `hasBeenDownloaded` via the history service. No per-file granularity.
### 2.3 Frontend blockers (`static/js/managers/DownloadManager.js`)
Three independent gates prevent re-entering the file dialog:
1. **Line 598:** file-select badge rendered only when `modelFiles.length > 1 && !existsLocally`.
2. **Lines 666681 (`updateNextButtonState`):** Next button disabled with "Already in Library" when `currentVersion.existsLocally`.
3. **Lines 784787 (`proceedToLocation`):** toast + abort when `currentVersion.existsLocally`.
The badge path (`confirmFileSelection` lines 737759 → `proceedToLocationContent``startDownload` single mode → `executeDownloadWithProgress` → POST `file_params`, `static/js/api/baseModelApi.js:12361250`) has **zero** `existsLocally` guards (all 12 occurrences enumerated; none on this path; `import/DownloadManager.js` has none either). The `.exists-locally` CSS class is purely visual (`download-modal.css:496499`). **Making the badge visible again is sufficient to unlock the flow** for phase 1.
Post-download refresh is clean: the modal closes and `resetAndReload(true)` performs a full library refetch (`DownloadManager.js:1063`); dialog reopen resets state and refetches versions with no client-side cache. No same-session staleness.
### 2.4 Local identity of the downloaded file
`LoraMetadata/CheckpointMetadata/EmbeddingMetadata.from_civitai_info(version_info, file_info, ...)` (`py/utils/models.py:245369`) persists:
- `sha256` = `file_info.hashes.SHA256` (lowercased, defaults to `""`) — a stable per-file identity;
- `civitai` = the full `version_info` payload (including the `files` list).
Metadata refresh (`metadata_sync_service.py:104105`) replaces the `civitai` blob wholesale but never overwrites top-level `sha256`; `verify_duplicate_hashes` (481526) corrects it to the on-disk hash. Top-level-sha256 matching is refresh-robust.
**Caveats (review R6/R7):**
- SHA256 is not guaranteed: CivArchive's transform only sets `hashes` when source data carries it (`civarchive_client.py:185189`); `from_civitai_info` defaults to `""`.
- Name fallback is unreliable exactly when it matters: local `file_name` is extension-less (`models.py:264`) and `generate_unique_filename` rewrites it with a hash suffix on conflict (`download_manager.py:11251136`); checkpoints with `hash_status='pending'` keep empty sha256 until on-demand hashing (`model_scanner.py:12321240`).
### 2.5 Version index collision (pre-existing hazard)
`ModelCache.version_index` is single-valued (`model_cache.py:133`: `version_index[version_id] = item`). Two files of the same version in the library → second entry overwrites the first; `remove_from_version_index` (lines 151181) drops the whole version key when the indexed entry is removed, even if a sibling file remains. ~10 read sites depend on this index (48 grep touch points total; readers include `recipe_scanner.py:26822726`, `recipe_format.py:3740`, `misc_handlers.py:24402444`, `model_handlers.py`, `model_scanner.check_model_version_exists:2444`).
Review correction (F3): bulk paths `remove_models` (`model_scanner.py:2376`) and `update_single_model_cache` (`:1689`) call `rebuild_version_index()` right after, so a sibling re-enters the index in those flows — the hazard is narrower than v1 stated, but direct `remove_from_version_index` callers (e.g. `model_scanner.py:1018`) still drop the key, and the user-visible artifact in phase 1 is real: `localPath` in the dialog flips to whichever file was indexed last.
### 2.6 Entry points that send / don't send `file_params` (fully enumerated by review)
**Send `file_params` (user-initiated dialog flows only):** `DownloadManager.js:16111639` (single mode). API surface accepting arbitrary JSON `file_params`: GET `/api/lm/download-model-get` (`model_handlers.py:16341686`), POST `/api/lm/downloads/queue/add` (`model_handlers.py:17991832`).
**Never send `file_params` (keep version-level semantics):** batch download (`DownloadManager.js:17561766`; batch also filters out in-library versions at `:1648`), `downloadVersionWithDefaults` (`:18101830`), recipe import (`import/DownloadManager.js:269276`), bulk missing-LoRA (`BulkMissingLoraDownloadManager.js:292299`), `RecipeModal.js:17281736`, `ModelVersionsTab.js:1427`. `web/comfyui/` and `vue-widgets/src` contain **no** download triggers at all (grep-verified). `py/services/use_cases/` has only `download_model_use_case.py` (pass-through).
### 2.7 Paths that do NOT need changes (verified)
- **aria2 pause/resume** (`_resume_restored_aria2_download`, line 754+): resumes from persisted `resume_context`; never re-runs existence gates.
- **`download_coordinator.py:90`**: pure pass-through of `file_params`.
- **Update checker / plugin self-update** (`update_routes.py:496501`): only closes the history DB handle.
- **History delete semantics**: `mark_as_deleted` sets `is_deleted_override=1` and `has_been_downloaded` then returns False (`downloaded_version_history_service.py:276`) — LM-initiated deletes already reset the history skip.
### 2.8 Related pre-existing issues (record, not necessarily fix)
- **Queue retry drops file selection** (R9): `download_queue_service.retry_from_history` / `retry_all_failed` re-queue with `file_params=NULL` (`download_queue_service.py:705, 758`) although the queue table has a `file_params` column (`:43`) — a retried non-primary download silently reverts to the primary file. Fix alongside phase 1 (small: persist and reuse the column).
- **`delete_model_version`** (`misc_handlers.py:24102487`): resolves the file via the single-valued `version_index` (24402444), deletes only that one file, and `mark_as_deleted` flags the **entire version** as deleted in history (2479) even when a sibling file remains in the library. See phase 2 item 6.1.5.
## 3. Goals / Non-Goals
**Goals**
- G1: A user can download any not-yet-downloaded file of a version already partially in the library (issue repro steps 68).
- G2: True duplicates stay blocked: downloading the *same* file of the same version twice is rejected.
- G3: Per-file downloaded state visible in the file dialog; multiple files selectable and downloadable in one pass.
- G4: No regression for version-level semantics relied on by batch download, recipe missing-LoRA detection, and `skip_previously_downloaded_model_versions`.
**Non-Goals**
- No change to recipe `inLibrary` semantics ("any file of the version present" remains sufficient).
- No change to the update-checker (version-level comparison).
- No primary-key rebuild of the history database.
- HuggingFace download flow untouched.
## 4. Design Decisions
- **D1 — Explicit file selection bypasses the history gate, version-level gates stay for everyone else.** The history skip exists to dedupe automated flows. A user explicitly picking a file is unambiguous intent; the file-level library gate (G2) still prevents real duplicates. **Guard conditions use normalized truthiness** (see D1a). All confirmed `file_params` senders are user-initiated dialog flows (2.6), and LM-initiated deletes already reset history (2.7), so the bypass only affects "downloaded but not LM-deleted" versions with the setting on — intended.
- **D1a — `file_params` normalization at the boundary (R2).** `download-model-get` and `downloads/queue/add` accept arbitrary JSON; `{}` is `not None` but falsy and would bypass gates while downloading the primary file. Normalize `file_params = file_params or None` in the coordinator/handlers, and treat the bypass as active only when a target file id is resolvable.
- **D2 — File identity matching rule (R6/R7):** hash-compare **only when both sides are non-empty** (lowercase SHA256 equality); name-compare when either side is empty. Never let `"" == ""` match. Name fallback caveats from 2.4 apply (renamed files, pending checkpoint hashes) — acceptable residual risk, worst case is a blocked re-download the user can retry after hashing completes.
- **D3 — Cache indexes: additive multi-index + removal re-pointing (R4).** Add `version_files_index: Dict[int, List[dict]]` maintained alongside `version_index` by the same add/remove/rebuild methods; existing readers of `version_index` untouched. Additionally fix `remove_from_version_index`: when the popped entry has a surviving sibling (per the multi-index), re-point `version_index[version_id]` to the sibling instead of dropping the key; same for the `model_id_index` descriptor. This closes the 2.5 hazard for existing readers (`check_model_version_exists`, `existsLocally`, recipe matching) without restructuring anything.
- **D4 — Per-file history via a child table (R3).** v1's additive-column approach is structurally impossible on a `(model_type, version_id)` PK (`ON CONFLICT DO UPDATE` would keep only the last file). Instead add `downloaded_version_files(model_type, version_id, file_id, file_name, downloaded_at, PRIMARY KEY(model_type, version_id, file_id))` — additive, no PK rebuild, honors the Non-Goal. Existing version-level table and queries unchanged. New per-file queries are opt-in. `_initialize_schema` uses `CREATE TABLE IF NOT EXISTS`, so the new table is created for existing DBs without any ALTER.
- **D5 — UI flow reuse, with an extracted inner download function for multi-file (F7).** Phase 1 unlocks the existing badge → file dialog → location → download pipeline. Phase 2 upgrades the dialog to multi-select; iterating `executeDownloadWithProgress` as-is would produce N full library reloads, N toasts, and competing failure-summary modals — so phase 2 extracts a reload-deferred, failure-aggregating inner variant and runs one reload + one summary at the end.
## 5. Implementation — Phase 1 (fix the issue; independently shippable)
### 5.1 Backend — `py/services/download_manager.py`
1. **Normalize `file_params`** at the boundary (D1a): `download_coordinator.schedule_download` and the two API handlers (`model_handlers.py:16491666`, `18101832`) apply `file_params = file_params or None`.
2. **Extract a shared file resolver** (R1): pull the matching logic at 14981569 into `_resolve_target_file(version_info, file_params) -> Optional[dict]`, used by **both** the new gate and the download-selection path. The selection path's condition (line 1499) switches from `model_version_id` to `resolved_version_id` (already computed at 12301236 from `version_info.id`), so gate and download always agree on the target file — including the `model_id`-only case.
3. **New helper** `_find_local_file_entry(version_id, target_file) -> Optional[dict]`: iterate the three scanners' cached `raw_data` (NOT `version_index` — single-valued); candidates = entries whose `civitai.id` normalizes to `version_id`; match per D2.
4. **Gate restructure in `_execute_original_download`**:
- Early scanner gate (11571184): add `file_params is None` guard; with normalized `file_params`, defer (file identity not resolvable before metadata fetch).
- After `version_info` fetch + `resolved_version_id` (~1229): when `file_params` present, resolve target file via the shared resolver; unresolvable → hard error "No matching file" (fail closed, prevents empty-dict bypass). Resolvable → `_find_local_file_entry`; hit → same hard error shape as today with the file name in the message.
- History gate (12381279): add `file_params is None` (D1). Base-model skip (12811324) unchanged — still applies.
- Late gate (13501376): add `file_params is None` guard (F2) — the post-fetch file-level check above already covers this case.
- Nothing between the early gate and the post-fetch point assumes the version is absent (review task 6: only provider selection + metadata fetch; no DB writes; `_persist_aria2_state` runs only when actually downloading at 1659).
5. **Queue retry fix** (2.8, small): persist `file_params` into the queue table on enqueue and reuse it in `retry_from_history` / `retry_all_failed`.
6. Logging: `[download]` lines for file-level allow/block, consistent with existing style.
**Estimated:** ~150220 LOC + resolver extraction.
### 5.2 Frontend — `static/js/managers/DownloadManager.js`
1. Line 598: drop `&& !existsLocally` from the badge condition (badge shows whenever `modelFiles.length > 1`).
2. `fileParams` construction (16111616): add `name: this.selectedFile.name`.
3. Surface the backend "file already in library" hard error as a toast instead of only the batch-summary modal (R10/F12 nit; reuse existing error message field).
4. No changes to `updateNextButtonState` / `proceedToLocation` in phase 1; no template or CSS changes.
**Known phase-1 UX limitations (acknowledged, fixed in phase 2):** with all files downloaded the badge still renders and re-picking a downloaded file fails late (backend error after the location step); `localPath` may point at a sibling file; batch-preview "In Library" badge stays version-level and gives no hint of remaining files.
**Estimated:** ~1030 LOC (confirmed realistic by review).
### 5.3 Phase 1 tests
Backend — extend `tests/services/test_download_manager_basic.py` (1694 lines; all fixture patterns exist):
- **Fixture gaps to add (F10):** `DummyScanner.get_cached_data()`/`raw_data` stub (~10 lines); `hashes.SHA256` in the metadata-provider payload's `files`.
- Cases: same version + different SHA256 in library + `file_params` → proceeds; same SHA256 → hard error; `file_params=None` + version in library → hard error (unchanged); history-skip on + `file_params` → not skipped; without → skipped (unchanged); empty-dict `file_params` normalized → version-level behavior; `model_id`-only + `file_params` → gate and selection resolve the same file; legacy metadata (empty local sha256) matched by name; target file with empty SHA256 → name fallback, no `""==""` false positive.
- Queue retry: `file_params` survives retry.
- Assert proceed/abort via the existing `_execute_download` mock pattern.
Frontend (`tests/frontend/`): badge renders for multi-file version with `existsLocally=true` (pattern from `downloadManager.history.test.js`).
**Estimated:** ~150250 LOC (confirmed realistic).
## 6. Implementation — Phase 2 (per-file status + multi-select + index hardening)
### 6.1 Backend
1. **`py/services/model_cache.py`** (D3): add `version_files_index`; maintain in `add_to_version_index` / `remove_from_version_index` / `rebuild_version_index`; removal re-points `version_index[version_id]` (and the `model_id_index` descriptor) to a surviving sibling instead of dropping the key.
2. **`py/services/model_scanner.py`**: expose `get_files_for_version(version_id) -> List[dict]`.
3. **`py/routes/handlers/model_handlers.py` `get_civitai_versions`**: annotate each version with `downloadedFiles: [{fileId, fileName, filePath}]` via `version_files_index` + D2 matching against `version.files`.
4. **`py/services/downloaded_version_history_service.py`** (D4): new child table `downloaded_version_files`; `mark_downloaded` also upserts the child row when `file_id` known; `mark_as_deleted` clears the version's child rows only when no sibling remains in the library; new `get_downloaded_file_ids(model_type, version_id) -> set[int]`. `_record_downloaded_version_history` passes `file_info` through.
5. **`delete_model_version`** (`misc_handlers.py:24102487`, R5): resolve **all** local files of the version via `version_files_index`; delete all (current endpoint semantics are version-level) or — if kept per-file — only `mark_as_deleted` when no sibling remains. Decide at implementation time; minimum is documenting current behavior.
6. **`ModelVersionsTab` backend support**: none needed beyond item 3 (`downloadedFiles`); the tab consumes the same versions payload.
### 6.2 Frontend
1. **File dialog multi-select** — change surface (F8): option markup (`DownloadManager.js:712724`), the single-select click handler (`727734`), the `input[type="radio"]:checked` selector in `confirmFileSelection` (`738`); template `templates/components/modals/download_modal.html:4860` (confirm-button label only); CSS `download-modal.css` — checkbox variant of `.file-option-radio input` (595604) and a **new** `.file-option.disabled` style (does not exist). Files whose id ∈ `downloadedFiles` render disabled with an "In Library" tag.
2. **Mixed-type guard (F8):** multi-select is restricted to files sharing the same routing target (`_isDiffusionModel` is computed once from a single `selectedFile` at 798803; e.g. "Model" + "UNet" files route to different roots). Disallow mixed-type multi-select (simplest, predictable); single-file selection unchanged.
3. **Multi-file download loop (D5/F7):** extract from `executeDownloadWithProgress` a reload-deferred, no-toast inner function; iterate per selected file with per-file progress; one `resetAndReload(true)` + one aggregated success/failure summary at the end (reuse `showDownloadBatchSummary`).
4. **`updateNextButtonState` / `proceedToLocation`:** for multi-file versions, Next routes into the file dialog; hard block only when *every* weight file is downloaded.
5. **`ModelVersionsTab.js` (F6):** the Download action (`:576` hidden when `isInLibrary`) — for multi-file versions with remaining files, show it and route into the download modal's file dialog; keep hidden when all files present.
6. **Batch preview (F5):** `batch-preview-local-badge` (`:1320`) gains a "partially downloaded" hint for multi-file versions with remaining files.
7. New i18n keys (`modals.download.fileSelection.inLibrary`, `downloadSelected`, partial-download tooltip, etc.) → run `python scripts/sync_translation_keys.py`.
### 6.3 Phase 2 tests
- `model_cache` (`tests/services/test_model_cache.py` already covers add/remove at 4455): multi-valued index; sibling re-point on removal; rebuild.
- `get_civitai_versions`: `downloadedFiles` correctness (hash match, name fallback, no match, CivArchive no-hash payload).
- History service (`tests/services/test_downloaded_version_history_service.py` uses real SQLite on tmp_path): child-table creation on a legacy DB; per-file record/query; `mark_as_deleted` sibling semantics.
- Frontend: dialog checkbox rendering/disabled state and multi-file confirm — **greenfield behavior coverage** (F10: no existing test exercises `showFileSelectionStep`/`confirmFileSelection`; infra exists, patterns must be built).
## 7. Risks and Mitigations
| Risk | Impact | Mitigation |
|---|---|---|
| History-gate bypass (D1) causes unwanted re-downloads in automated flows | Large checkpoint files re-downloaded | Bypass only with normalized, resolvable `file_params` (D1a); all such senders are user-initiated dialog flows (2.6, verified); tests pin batch/recipe/bulk behavior. |
| Empty-hash matching edge cases (R6) | Duplicate download of the same file, or false block | D2 rule: hash only when both non-empty; name otherwise; never `""==""`. Residual risk documented (2.4). |
| Phase-1 late-failure UX (F12) | User picks a downloaded file, fails only after location step | Toast surfacing (5.2.3); phase 2 disables downloaded files up front. |
| Phase-2 index change corrupts existing behavior | Recipe matching, delete flows | Additive index + re-point only; `version_index` read semantics unchanged; `remove_models`/`update_single_model_cache` already rebuild (F3); tests. |
| `delete_model_version` marks whole version deleted while sibling remains (R5) | History wrongly suppresses re-download of the surviving sibling's version | Phase 2 item 6.1.5; documented until then. |
| History child-table migration failure on user installs | Service init crash | `CREATE TABLE IF NOT EXISTS` in `_initialize_schema`; failure degrades to version-level behavior (per-file queries return empty). |
| Batch-preview badge misleading for partial versions (F5) | Minor UX confusion | Acknowledged in phase 1; fixed in phase 2 item 6.2.6. |
| UI confusion: version shows "In Library" while files remain downloadable | Support burden | Phase 2: per-file disabled state + partial-download tooltip. |
| Hash-identical sibling files (repacked content) | Second file blocked | Acceptable: scanner hash dedup already collapses them. |
## 8. Rollout
1. **Commit 1**`fix(download): allow downloading additional files of an in-library model version (#1058)` → Phase 1 (5.15.3).
2. **Commit 2**`feat(download): per-file download status and multi-file selection (#1058)` → Phase 2 (6.16.3).
Phase 1 alone resolves the issue as reported; phase 2 can ship in a later release if review prefers smaller increments.
## 9. Effort Estimate (revised after review)
| Phase | Backend | Frontend | Tests | Risk |
|---|---|---|---|---|
| 1 | ~150220 LOC (+ queue-retry fix ~30) | ~1030 LOC | ~150250 LOC | Low |
| 2 | ~250350 LOC | ~250350 LOC (multi-file loop refactor + ModelVersionsTab + batch badge) | ~250350 LOC (dialog tests greenfield) | Medium |
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+5 -1
View File
@@ -1244,11 +1244,13 @@
"downloaded": "Downloaded",
"downloadedTooltip": "Previously downloaded, but it is not currently in your library.",
"alreadyInLibrary": "Already in Library",
"partiallyDownloaded": "Partially downloaded",
"autoOrganizedPath": "[Auto-organized by path template]",
"fileSelection": {
"title": "Select File Format",
"files": "files",
"select": "Select File"
"select": "Select File",
"inLibrary": "In Library"
},
"errors": {
"invalidUrl": "Invalid Civitai URL format",
@@ -1594,6 +1596,7 @@
"actions": {
"download": "Download",
"downloadTooltip": "Download this version",
"downloadRemainingTooltip": "Download remaining files of this version",
"downloadEarlyAccessTooltip": "Download this early access version from Civitai",
"downloadPaidTooltip": "Download this paid version from Civitai",
"downloadNotAllowedTooltip": "This version is only available for on-site generation on Civitai",
@@ -1942,6 +1945,7 @@
"downloadPartialSuccess": "Downloaded {completed} of {total} LoRAs",
"downloadPartialWithAccess": "Downloaded {completed} of {total} LoRAs. {accessFailures} failed due to access restrictions. Check your API key in settings or early access status.",
"pleaseSelectVersion": "Please select a version",
"pleaseSelectFile": "Please select at least one file",
"versionExists": "This version already exists in your library",
"downloadCompleted": "Download completed successfully",
"downloadSkippedByBaseModel": "Skipped download because base model {baseModel} is excluded",
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+2331 -2327
View File
File diff suppressed because it is too large Load Diff
+34
View File
@@ -41,6 +41,40 @@ class RecipeMetadataParser(ABC):
"""
pass
@staticmethod
def populate_lora_from_local(lora_entry: Dict[str, Any], local_lora: Dict[str, Any], base_model_counts=None) -> Dict[str, Any]:
"""Populate a recipe LoRA entry from the local scanner cache."""
local_path = local_lora.get('file_path') or ''
file_name = local_lora.get('file_name') or os.path.splitext(os.path.basename(local_path))[0]
base_model = local_lora.get('base_model') or ''
lora_entry['name'] = local_lora.get('model_name') or file_name or lora_entry.get('name', '')
lora_entry['file_name'] = file_name
lora_entry['hash'] = (local_lora.get('sha256') or lora_entry.get('hash') or '').lower()
lora_entry['localPath'] = local_path or None
lora_entry['size'] = local_lora.get('size', 0) or 0
lora_entry['baseModel'] = base_model
lora_entry['existsLocally'] = True
lora_entry['isDeleted'] = False
preview_url = local_lora.get('preview_url')
if preview_url:
lora_entry['thumbnailUrl'] = config.get_preview_static_url(preview_url)
civitai_info = local_lora.get('civitai') or {}
if isinstance(civitai_info, dict):
if civitai_info.get('id') is not None:
lora_entry['id'] = civitai_info['id']
if civitai_info.get('modelId') is not None:
lora_entry['modelId'] = civitai_info['modelId']
if civitai_info.get('name'):
lora_entry['version'] = civitai_info['name']
if base_model_counts is not None and base_model:
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
return lora_entry
@staticmethod
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any] | None, str | None] | Dict[str, Any],
recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]:
+201 -61
View File
@@ -362,68 +362,208 @@ class AutomaticMetadataParser(RecipeMetadataParser):
checkpoint = checkpoint_entry
# If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"]
if not loras or len(loras) == 0:
# Extract lora weights from extranet tags in prompt (for later use)
lora_weights = {}
lora_matches = re.findall(self.EXTRANETS_REGEX, prompt)
for lora_type, lora_name, lora_weight in lora_matches:
key = f"{lora_type}:{lora_name}"
lora_weights[key] = round(float(lora_weight), 2)
# Use hashes from metadata as the primary source
if metadata.get("hashes"):
for hash_key, lora_hash in metadata.get("hashes", {}).items():
# Only process lora or hypernet types
if not hash_key.startswith(("lora:", "hypernet:")):
def normalize_lora_name(name, basename=False):
normalized = str(name or '').replace('\\', '/')
if normalized.casefold().endswith('.safetensors'):
normalized = normalized[:-12]
if basename:
normalized = normalized.rsplit('/', 1)[-1]
return normalized.casefold()
def get_version_id(lora):
version_id = lora.get('id')
if version_id in (None, '', 0, '0'):
version_id = lora.get('modelVersionId')
if version_id in (None, '', 0, '0'):
return None
return str(version_id)
prompt_loras = {}
for match in re.findall(self.EXTRANETS_REGEX, prompt):
lora_type, lora_name, _ = match
prompt_loras[(lora_type, normalize_lora_name(lora_name))] = match
prompt_by_basename = {}
for lora_type, lora_name, lora_weight in prompt_loras.values():
key = (lora_type, normalize_lora_name(lora_name, True))
prompt_by_basename.setdefault(key, []).append((lora_name, round(float(lora_weight), 2)))
hash_basenames = {
(hash_key.split(':', 1)[0], normalize_lora_name(hash_key.split(':', 1)[1], True))
for hash_key, hash_value in metadata.get("hashes", {}).items()
if hash_value and hash_key.startswith(("lora:", "hypernet:"))
}
recipe_base_model = checkpoint.get("baseModel") if checkpoint else None
if not recipe_base_model and len(base_model_counts) == 1:
recipe_base_model = next(iter(base_model_counts))
resource_lora_count = len(loras)
def make_lora_entry(lora_type, lora_name, weight, lora_hash=''):
return {
'name': lora_name,
'type': lora_type,
'weight': weight,
'hash': lora_hash,
'existsLocally': False,
'localPath': None,
'file_name': lora_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
def merge_or_append_civitai(civitai_entry, preserve_existing_weight=False):
civitai_id = get_version_id(civitai_entry)
civitai_hash = (civitai_entry.get('hash') or '').lower()
for index, existing in enumerate(loras):
existing_id = get_version_id(existing)
existing_hash = (existing.get('hash') or '').lower()
if not (
(civitai_id and existing_id == civitai_id)
or (civitai_hash and existing_hash == civitai_hash)
):
continue
if preserve_existing_weight:
civitai_entry['weight'] = existing.get('weight', civitai_entry['weight'])
existing_base = existing.get('baseModel')
if not civitai_entry.get('baseModel'):
civitai_entry['baseModel'] = existing_base or ''
elif existing_base:
remaining = base_model_counts.get(existing_base, 0) - 1
if remaining > 0:
base_model_counts[existing_base] = remaining
else:
base_model_counts.pop(existing_base, None)
loras[index] = civitai_entry
return
loras.append(civitai_entry)
def merge_or_append_local(local_entry):
local_id = get_version_id(local_entry)
local_hash = (local_entry.get('hash') or '').lower()
for existing in loras:
existing_id = get_version_id(existing)
existing_hash = (existing.get('hash') or '').lower()
if not (
(local_id and existing_id == local_id)
or (local_hash and existing_hash == local_hash)
):
continue
existing['weight'] = local_entry['weight']
existing['hash'] = local_entry['hash']
existing['file_name'] = local_entry['file_name']
existing['existsLocally'] = True
existing['localPath'] = local_entry['localPath']
existing['size'] = local_entry['size']
existing['isDeleted'] = False
if not existing.get('modelId') and local_entry.get('modelId'):
existing['modelId'] = local_entry['modelId']
if not existing.get('baseModel') and local_entry.get('baseModel'):
existing['baseModel'] = local_entry['baseModel']
base_model_counts[local_entry['baseModel']] = base_model_counts.get(local_entry['baseModel'], 0) + 1
thumbnail_url = local_entry.get('thumbnailUrl')
if thumbnail_url and not thumbnail_url.endswith('/images/no-preview.png'):
existing['thumbnailUrl'] = thumbnail_url
return
if local_entry.get('baseModel'):
base_model = local_entry['baseModel']
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
loras.append(local_entry)
resolved_prompt_basenames = set()
queried_local_basenames = set()
for lora_type, lora_name, lora_weight in prompt_loras.values():
weight = round(float(lora_weight), 2)
basename_key = (lora_type, normalize_lora_name(lora_name, True))
matching_resources = [
lora
for lora in loras[:resource_lora_count]
if lora.get('file_name')
and normalize_lora_name(lora['file_name'], True) == basename_key[1]
and (
(lora_type == 'hypernet' and str(lora.get('type', '')).casefold() in ('hypernet', 'hypernetwork'))
or (lora_type == 'lora' and str(lora.get('type', '')).casefold() not in ('hypernet', 'hypernetwork'))
)
]
if len(prompt_by_basename[basename_key]) == 1 and len(matching_resources) == 1:
matching_resources[0]['weight'] = weight
if basename_key not in hash_basenames:
resolved_prompt_basenames.add(basename_key)
continue
if basename_key in hash_basenames:
continue
if not recipe_scanner or lora_type != 'lora':
continue
queried_local_basenames.add(basename_key)
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if not local_lora:
continue
local_entry = self.populate_lora_from_local(
make_lora_entry(lora_type, lora_name, weight),
local_lora,
)
merge_or_append_local(local_entry)
resolved_prompt_basenames.add(basename_key)
for hash_key, lora_hash in metadata.get("hashes", {}).items():
if not hash_key.startswith(("lora:", "hypernet:")):
continue
lora_type, lora_name = hash_key.split(':', 1)
basename_key = (lora_type, normalize_lora_name(lora_name, True))
if basename_key in resolved_prompt_basenames:
continue
prompt_entries = prompt_by_basename.get(basename_key, [])
weight = prompt_entries[0][1] if len(prompt_entries) == 1 else 1.0
lora_entry = make_lora_entry(lora_type, lora_name, weight, lora_hash)
if lora_hash and recipe_scanner and lora_type == 'lora':
local_lora = await recipe_scanner.get_local_lora_by_hash(lora_hash)
if local_lora:
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
merge_or_append_local(local_entry)
continue
hash_resolved = False
if lora_hash and metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash,
)
if populated_entry is None:
continue
# Skip entries without a hash value — they can't be
# resolved via CivitAI and would only produce a
# useless "Deleted" entry in the recipe.
if not lora_hash:
continue
lora_type, lora_name = hash_key.split(':', 1)
# Get weight from extranet tags if available, else default to 1.0
weight = lora_weights.get(hash_key, 1.0)
# Initialize lora entry
lora_entry = {
'name': lora_name,
'type': lora_type, # 'lora' or 'hypernet'
'weight': weight,
'hash': lora_hash,
'existsLocally': False,
'localPath': None,
'file_name': lora_name,
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
# Try to get info from Civitai
if metadata_provider:
try:
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info,
recipe_scanner,
base_model_counts,
lora_hash
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA {lora_name}: {e}")
loras.append(lora_entry)
lora_entry = populated_entry
hash_resolved = not lora_entry.get('isDeleted')
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA {lora_name}: {e}")
if hash_resolved:
merge_or_append_civitai(lora_entry, preserve_existing_weight=not prompt_entries)
continue
if recipe_scanner and lora_type == 'lora' and basename_key not in queried_local_basenames:
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if local_lora:
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
merge_or_append_local(local_entry)
continue
if lora_hash and not resource_lora_count:
loras.append(lora_entry)
# Try to get base model from resources or make educated guess
base_model = None
+95 -68
View File
@@ -31,79 +31,15 @@ class ComfyMetadataParser(RecipeMetadataParser):
metadata_provider = await get_default_metadata_provider()
data = json.loads(user_comment)
loras = []
# Find all LoraLoader nodes
lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'}
# Process each LoraLoader node
for node_id, node in lora_nodes.items():
if 'inputs' not in node or 'lora_name' not in node['inputs']:
continue
lora_name = node['inputs'].get('lora_name', '')
# Parse the URN to extract model ID and version ID
# Format: "urn:air:sdxl:lora:civitai:1107767@1253442"
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
if not lora_id_match:
continue
model_id = lora_id_match.group(1)
model_version_id = lora_id_match.group(2)
# Get strength from node inputs
weight = node['inputs'].get('strength_model', 1.0)
# Initialize lora entry with default values
lora_entry = {
'id': model_version_id,
'modelId': model_id,
'name': f"Lora {model_id}", # Default name
'version': '',
'type': 'lora',
'weight': weight,
'existsLocally': False,
'localPath': None,
'file_name': '',
'hash': '',
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
# Get additional info from Civitai if metadata provider is available
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
# Populate lora entry with Civitai info
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner
)
if populated_entry is None:
continue # Skip invalid LoRA types
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
loras.append(lora_entry)
# Find checkpoint info
checkpoint_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'CheckpointLoaderSimple'}
checkpoint = None
checkpoint_id = None
checkpoint_version_id = None
if checkpoint_nodes:
# Get the first checkpoint node
checkpoint_node = next(iter(checkpoint_nodes.values()))
if 'inputs' in checkpoint_node and 'ckpt_name' in checkpoint_node['inputs']:
checkpoint_name = checkpoint_node['inputs']['ckpt_name']
# Parse checkpoint URN
checkpoint_match = re.search(r'civitai:(\d+)@(\d+)', checkpoint_name)
if checkpoint_match:
checkpoint_id = checkpoint_match.group(1)
@@ -115,16 +51,107 @@ class ComfyMetadataParser(RecipeMetadataParser):
'version': '',
'type': 'checkpoint'
}
# Get additional checkpoint info from Civitai
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(checkpoint_version_id)
civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
# Populate checkpoint with Civitai info
checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info)
except Exception as e:
logger.error(f"Error fetching Civitai info for checkpoint: {e}")
recipe_base_model = checkpoint.get('baseModel') if checkpoint else None
loras = []
lora_candidates = []
for node in data.values():
if not isinstance(node, dict):
continue
inputs = node.get('inputs')
if not isinstance(inputs, dict):
continue
if node.get('class_type') == 'LoraLoader':
lora_name = inputs.get('lora_name', '')
if isinstance(lora_name, str) and lora_name:
lora_candidates.append((lora_name, inputs.get('strength_model', 1.0)))
continue
if node.get('class_type') != 'LoraLoaderLM':
continue
loras_data = inputs.get('loras', [])
if isinstance(loras_data, dict):
loras_data = loras_data.get('__value__', [])
if isinstance(loras_data, list) and len(loras_data) == 1 and isinstance(loras_data[0], list):
loras_data = loras_data[0]
if not isinstance(loras_data, list):
continue
for lora in loras_data:
if not isinstance(lora, dict) or not lora.get('active', False) or lora.get('_isDummy', False):
continue
lora_name = lora.get('name', '')
if isinstance(lora_name, str) and lora_name:
lora_candidates.append((lora_name, lora.get('strength', 1.0)))
for lora_name, weight in lora_candidates:
if isinstance(weight, str):
try:
weight = float(weight)
except ValueError:
weight = 1.0
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
if lora_id_match:
model_id = lora_id_match.group(1)
model_version_id = lora_id_match.group(2)
entry_name = f"Lora {model_id}"
else:
model_id = 0
model_version_id = 0
entry_name = re.split(r'[\\/]', lora_name)[-1]
entry_name = re.sub(r'\.[^.]+$', '', entry_name)
lora_entry = {
'id': model_version_id,
'modelId': model_id,
'name': entry_name,
'version': '',
'type': 'lora',
'weight': weight,
'existsLocally': False,
'localPath': None,
'file_name': entry_name,
'hash': '',
'thumbnailUrl': '/loras_static/images/no-preview.png',
'baseModel': '',
'size': 0,
'downloadUrl': '',
'isDeleted': False
}
if lora_id_match:
if metadata_provider:
try:
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
populated_entry = await self.populate_lora_from_civitai(
lora_entry,
civitai_info_tuple,
recipe_scanner
)
if populated_entry is None:
continue
lora_entry = populated_entry
except Exception as e:
logger.error(f"Error fetching Civitai info for LoRA: {e}")
else:
if not recipe_scanner:
continue
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
if not local_lora:
continue
lora_entry = self.populate_lora_from_local(lora_entry, local_lora)
loras.append(lora_entry)
# Extract generation parameters
gen_params = {}
+26 -9
View File
@@ -2428,8 +2428,8 @@ class ModelLibraryHandler:
embedding_scanner = await self._service_registry.get_embedding_scanner()
found_type = None
file_path = None
found_cache = None
entries: list = []
for model_type, scanner in (
("lora", lora_scanner),
@@ -2440,27 +2440,43 @@ class ModelLibraryHandler:
if cache and model_version_id in cache.version_index:
found_type = model_type
found_cache = cache
entry = cache.version_index[model_version_id]
file_path = entry.get("file_path")
# A version can have several local files (#1058); collect
# them all so the delete below covers every file.
files_getter = getattr(cache, "get_files_by_version_id", None)
if files_getter is not None:
entries = files_getter(model_version_id)
else:
entries = [cache.version_index[model_version_id]]
break
if not file_path:
file_paths = [
entry.get("file_path")
for entry in entries
if isinstance(entry, dict) and entry.get("file_path")
]
if not file_paths:
return web.json_response(
{"success": False, "error": "Model version not found in any scanner cache"},
status=404,
)
target_dir = os.path.dirname(file_path)
base_name = os.path.basename(file_path)
file_name, extension = os.path.splitext(base_name)
await delete_model_artifacts(target_dir, file_name, main_extension=extension)
for file_path in file_paths:
target_dir = os.path.dirname(file_path)
base_name = os.path.basename(file_path)
file_name, extension = os.path.splitext(base_name)
await delete_model_artifacts(target_dir, file_name, main_extension=extension)
if found_cache:
removed_paths = set(file_paths)
found_cache.raw_data = [
item
for item in found_cache.raw_data
if item.get("file_path") != file_path
if item.get("file_path") not in removed_paths
]
rebuild = getattr(found_cache, "rebuild_version_index", None)
if rebuild is not None:
rebuild()
await found_cache.resort()
scanner_map = {
@@ -2483,6 +2499,7 @@ class ModelLibraryHandler:
"success": True,
"modelType": found_type,
"modelVersionId": model_version_id,
"deletedFiles": len(file_paths),
}
)
except Exception as exc:
+75 -2
View File
@@ -1659,7 +1659,8 @@ class ModelDownloadHandler:
import json
try:
data["file_params"] = json.loads(file_params_json)
# Normalize falsy payloads (e.g. {}) to None (#1058)
data["file_params"] = json.loads(file_params_json) or None
except json.JSONDecodeError:
self._logger.warning(
"Invalid file_params JSON: %s", file_params_json
@@ -1811,7 +1812,8 @@ class ModelDownloadHandler:
model_id = int(model_id_str) if model_id_str else None
model_version_id = int(model_version_id_str) if model_version_id_str else None
file_params = json.loads(file_params_json) if file_params_json else None
# Normalize falsy payloads (e.g. {}) to None (#1058)
file_params = (json.loads(file_params_json) if file_params_json else None) or None
service = await DownloadQueueService.get_instance()
item = await service.add_to_queue(
@@ -2187,6 +2189,19 @@ class ModelCivitaiHandler:
else:
version.pop("localPath", None)
# Per-file downloaded state so multi-file versions can show
# which individual files are already in the library (#1058)
local_entries: List[Any] = []
if version_id is not None and cache:
files_getter = getattr(cache, "get_files_by_version_id", None)
if files_getter is not None:
local_entries = files_getter(version_id)
elif cache_entry is not None:
local_entries = [cache_entry]
version["downloadedFiles"] = self._match_downloaded_files(
version, local_entries
)
model_file = (
self._find_model_file(version.get("files", []))
if isinstance(version.get("files"), Iterable)
@@ -2201,6 +2216,64 @@ class ModelCivitaiHandler:
)
return web.Response(status=500, text=str(exc))
@staticmethod
def _match_downloaded_files(
version: Mapping[str, Any], local_entries: List[Any]
) -> List[Dict[str, Any]]:
"""Map local library entries back to individual files of a version.
Matching follows rule D2 (#1058): SHA256 is authoritative when the
local entry carries one; otherwise fall back to extension-less file
name equality. Returns ``[{fileId, fileName, filePath}]``.
"""
files = version.get("files")
if not isinstance(files, list) or not local_entries:
return []
by_hash: Dict[str, Mapping[str, Any]] = {}
by_name: Dict[str, Mapping[str, Any]] = {}
for file_info in files:
if not isinstance(file_info, Mapping):
continue
sha = str(
(file_info.get("hashes") or {}).get("SHA256") or ""
).strip().lower()
if sha:
by_hash.setdefault(sha, file_info)
name = str(file_info.get("name") or "").strip()
if name:
by_name.setdefault(os.path.splitext(name)[0], file_info)
downloaded: List[Dict[str, Any]] = []
seen_keys: set = set()
for entry in local_entries:
if not isinstance(entry, Mapping):
continue
matched: Optional[Mapping[str, Any]] = None
local_hash = str(entry.get("sha256") or "").strip().lower()
if local_hash:
matched = by_hash.get(local_hash)
if matched is None:
local_name = str(entry.get("file_name") or "").strip()
if local_name:
matched = by_name.get(local_name)
if matched is None:
continue
file_id = matched.get("id")
dedupe_key = file_id if file_id is not None else matched.get("name")
if dedupe_key in seen_keys:
continue
seen_keys.add(dedupe_key)
downloaded.append(
{
"fileId": file_id,
"fileName": matched.get("name"),
"filePath": entry.get("file_path"),
}
)
return downloaded
async def get_civitai_model_by_version(self, request: web.Request) -> web.Response:
try:
model_version_id = request.match_info.get("modelVersionId")
+3 -1
View File
@@ -87,7 +87,9 @@ class DownloadCoordinator:
progress_callback=progress_callback,
download_id=download_id,
source=payload.get("source"),
file_params=payload.get("file_params"),
# Normalize falsy file_params (e.g. {}) to None so download gates
# treat it as "no explicit file selection" (#1058).
file_params=payload.get("file_params") or None,
)
result["download_id"] = download_id
+237 -69
View File
@@ -213,6 +213,162 @@ class DownloadManager:
)
return False
async def _get_scanner_for_model_type(self, model_type: str):
"""Return the scanner responsible for the given model type."""
if model_type == "checkpoint":
return await self._get_checkpoint_scanner()
if model_type == "embedding":
return await ServiceRegistry.get_embedding_scanner()
return await self._get_lora_scanner()
@staticmethod
def _resolve_target_file(
files: Any, file_params: Dict[str, Any] | None
) -> Optional[Dict[str, Any]]:
"""Resolve the target file within a version's file list from file_params.
Shared by the existence gate and the actual file selection so both
always agree on which file a download refers to (#1058). Returns None
when file_params is None or no file matches.
"""
if not file_params or not isinstance(files, list):
return None
target_file_id = file_params.get("id")
target_type = file_params.get("type", "Model")
target_format = file_params.get("format")
target_size = file_params.get("size")
target_fp = file_params.get("fp")
is_primary = file_params.get("isPrimary", False)
logger.debug(
"[download] file_params received: id=%s, type=%s, format=%s, size=%s, fp=%s, "
"isPrimary=%s, total_files=%d",
target_file_id, target_type, target_format, target_size, target_fp,
is_primary, len(files),
)
file_info: Optional[Dict[str, Any]] = None
if target_file_id:
target_id_str = str(target_file_id)
for f in files:
if not isinstance(f, dict):
continue
f_id = f.get("id")
if str(f_id) == target_id_str:
file_info = f
logger.debug(
"[download] MATCH by ID: id=%s name='%s'",
f_id, f.get("name"),
)
break
if not file_info:
logger.debug("[download] No file found with id=%s", target_file_id)
elif is_primary:
file_info = next(
(
f
for f in files
if isinstance(f, dict)
and f.get("primary")
and f.get("type") in MODEL_WEIGHT_FILE_TYPES
),
None,
)
else:
# Lenient metadata match: only compare fields present on both sides
for f in files:
if not isinstance(f, dict):
continue
f_type = f.get("type", "")
if f_type != target_type:
continue
f_meta = f.get("metadata", {})
f_format = f_meta.get("format") or f.get("format")
f_size = f_meta.get("size") or f.get("size")
f_fp = f_meta.get("fp") or f.get("fp")
if target_format and f_format != target_format:
continue
if target_size and f_size and f_size != target_size:
continue
if target_fp and f_fp and f_fp != target_fp:
continue
file_info = f
break
return file_info
async def _find_local_file_entry(
self,
model_type: str,
model_version_id: int,
target_file: Dict[str, Any],
) -> Optional[Dict[str, Any]]:
"""Find a local library entry for a specific file of a model version.
Matches per design rule D2 (#1058): SHA256 is only compared when both
sides carry a non-empty hash; otherwise fall back to (extension-less)
file name equality. Never let two empty hashes compare equal.
"""
try:
normalized_version_id = int(model_version_id)
except (TypeError, ValueError):
return None
try:
scanner = await self._get_scanner_for_model_type(model_type)
cache = await scanner.get_cached_data()
except Exception as exc:
logger.debug(
"Failed to scan local entries for version %s file check: %s",
model_version_id,
exc,
)
return None
raw_data = getattr(cache, "raw_data", None) if cache else None
if not raw_data:
return None
target_hash = str(
(target_file.get("hashes") or {}).get("SHA256") or ""
).strip().lower()
target_name = str(target_file.get("name") or "").strip()
target_base = os.path.splitext(target_name)[0] if target_name else ""
for item in raw_data:
if not isinstance(item, dict):
continue
civitai_data = item.get("civitai")
if not isinstance(civitai_data, dict):
continue
try:
item_version_id = int(civitai_data.get("id"))
except (TypeError, ValueError):
continue
if item_version_id != normalized_version_id:
continue
local_hash = str(item.get("sha256") or "").strip().lower()
if target_hash and local_hash:
if local_hash == target_hash:
return item
# Both sides carry hashes that differ: this is a different
# file of the same version — do not fall back to name match.
continue
if target_base:
local_name = str(item.get("file_name") or "").strip()
if local_name == target_base:
return item
return None
async def download_from_civitai(
self,
model_id: int | None = None,
@@ -242,6 +398,10 @@ class DownloadManager:
Returns:
Dict with download result
"""
# Normalize falsy file_params (e.g. an empty dict from API JSON
# parsing) to None so gate conditions behave consistently (#1058).
file_params = file_params or None
logger.debug(
"[download] download_from_civitai called: model_id=%s, model_version_id=%s, "
"source=%s, file_params=%s",
@@ -816,6 +976,7 @@ class DownloadManager:
version_info,
record.get("model_version_id"),
record.get("save_path") or record.get("file_path"),
file_info=file_info,
)
await self._sync_downloaded_version(
model_type,
@@ -1152,9 +1313,13 @@ class DownloadManager:
use_save_dir_as_root: bool = False,
) -> Dict[str, Any]:
"""Wrapper for original download_from_civitai implementation"""
file_params = file_params or None
try:
# Check if model version already exists in library
if model_version_id is not None:
# Check if model version already exists in library.
# With an explicit file selection (file_params) the version-level
# check is deferred until after the metadata fetch, when the target
# file can be resolved and checked individually (#1058).
if model_version_id is not None and file_params is None:
# Check both scanners
lora_scanner = await self._get_lora_scanner()
checkpoint_scanner = await self._get_checkpoint_scanner()
@@ -1235,8 +1400,26 @@ class DownloadManager:
except (TypeError, ValueError):
resolved_version_id = None
# Resolve the explicitly selected file (if any) up front so the
# existence gates and the actual file selection below always agree
# on the target file (#1058).
target_file: Optional[Dict[str, Any]] = None
if file_params is not None:
target_file = self._resolve_target_file(
version_info.get("files") or [], file_params
)
if target_file is None:
logger.warning(
"[download] file_params provided but no file matched; "
"falling back to version-level checks and primary file "
"selection (model_version_id=%s)",
resolved_version_id,
)
explicit_file = target_file is not None
if (
get_settings_manager().get_skip_previously_downloaded_model_versions()
not explicit_file
and get_settings_manager().get_skip_previously_downloaded_model_versions()
and resolved_version_id is not None
and await self._has_been_downloaded(model_type, resolved_version_id)
):
@@ -1346,9 +1529,38 @@ class DownloadManager:
f"baseModel '{base_model_value}' is a known diffusion model, routing to unet folder"
)
# Case 2: model_version_id was None, check after getting version_info
if model_version_id is None:
version_id = version_info.get("id")
# Existence check after the metadata fetch (#1058):
# - An explicit file selection only blocks when THIS file is
# already in the library; other files of the same version
# remain downloadable.
# - Without file_params (or when file_params failed to resolve),
# keep version-level protection. The case "model_version_id
# given + no file_params" was already covered by the early
# gate above.
if explicit_file and resolved_version_id is not None:
existing_entry = await self._find_local_file_entry(
model_type, resolved_version_id, target_file
)
if existing_entry is not None:
error_message = (
f"File '{target_file.get('name')}' from model version "
f"{resolved_version_id} already exists in {model_type} library"
)
logger.info("[download] %s", error_message)
return {"success": False, "error": error_message}
logger.info(
"[download] File '%s' of model version %s not in %s library — "
"download allowed (other files of this version may exist locally)",
target_file.get("name"), resolved_version_id, model_type,
)
elif file_params is not None or model_version_id is None:
# Case 2: model_version_id was None, or file_params did not
# resolve to a concrete file — check at version level.
version_id = (
resolved_version_id
if resolved_version_id is not None
else version_info.get("id")
)
if model_type == "lora":
# Check lora scanner
@@ -1495,73 +1707,16 @@ class DownloadManager:
files = version_info.get("files", [])
file_info = None
# If file_params is provided, try to find matching file
if file_params and model_version_id:
target_file_id = file_params.get("id")
target_type = file_params.get("type", "Model")
target_format = file_params.get("format")
target_size = file_params.get("size")
target_fp = file_params.get("fp")
is_primary = file_params.get("isPrimary", False)
logger.debug(
"[download] file_params received: id=%s, type=%s, format=%s, size=%s, fp=%s, isPrimary=%s, "
"model_version_id=%s, total_files=%d",
target_file_id, target_type, target_format, target_size, target_fp, is_primary,
model_version_id, len(files),
)
if target_file_id:
target_id_str = str(target_file_id)
for f in files:
f_id = f.get("id")
if str(f_id) == target_id_str:
file_info = f
logger.debug(
"[download] MATCH by ID: id=%s name='%s'",
f_id, f.get("name"),
)
break
if not file_info:
logger.debug("[download] No file found with id=%s", target_file_id)
elif is_primary:
file_info = next(
(
f
for f in files
if f.get("primary")
and f.get("type") in MODEL_WEIGHT_FILE_TYPES
),
None,
)
else:
# Lenient metadata match: only compare fields present on both sides
for f in files:
f_type = f.get("type", "")
if f_type != target_type:
continue
f_meta = f.get("metadata", {})
f_format = f_meta.get("format") or f.get("format")
f_size = f_meta.get("size") or f.get("size")
f_fp = f_meta.get("fp") or f.get("fp")
if target_format and f_format != target_format:
continue
if target_size and f_size and f_size != target_size:
continue
if target_fp and f_fp and f_fp != target_fp:
continue
file_info = f
break
# If file_params is provided, reuse the file resolved right after
# the metadata fetch so the existence gate and this selection
# always agree on the target file (#1058).
if file_params is not None:
file_info = target_file
if not file_info:
logger.debug(
"[download] No match found via file_params — falling back to primary file lookup",
)
elif not file_params:
else:
logger.debug(
"[download] No file_params provided (null/None) — will use primary file lookup. "
"model_version_id=%s, total_files=%d",
@@ -1706,6 +1861,7 @@ class DownloadManager:
version_info,
model_version_id,
save_path,
file_info=file_info,
)
await self._sync_downloaded_version(
model_type,
@@ -1748,6 +1904,7 @@ class DownloadManager:
version_info: Dict[str, Any],
fallback_version_id=None,
file_path: str | None = None,
file_info: Dict[str, Any] | None = None,
) -> None:
try:
history_service = await ServiceRegistry.get_downloaded_version_history_service()
@@ -1773,6 +1930,15 @@ class DownloadManager:
if version_id is None:
version_id = fallback_version_id
# Per-file identity for multi-file versions (#1058)
file_id = None
file_name = None
if isinstance(file_info, dict):
file_id = file_info.get("id")
raw_file_name = file_info.get("name")
if isinstance(raw_file_name, str) and raw_file_name.strip():
file_name = raw_file_name.strip()
try:
await history_service.mark_downloaded(
model_type,
@@ -1780,6 +1946,8 @@ class DownloadManager:
model_id=int(cast(Any, resolved_model_id)) if resolved_model_id is not None else None,
source="download",
file_path=file_path,
file_id=file_id,
file_name=file_name,
)
except (TypeError, ValueError):
logger.debug(
+34 -8
View File
@@ -64,6 +64,7 @@ class DownloadQueueService:
model_name TEXT NOT NULL DEFAULT '',
version_name TEXT DEFAULT '',
thumbnail_url TEXT DEFAULT '',
file_params TEXT,
status TEXT NOT NULL,
error TEXT,
file_path TEXT,
@@ -120,6 +121,18 @@ class DownloadQueueService:
with self._connect() as conn:
conn.executescript(self._SCHEMA_TABLES)
# Databases created by older versions lack
# download_history.file_params; add it so retry-from-history can
# restore the originally selected file (#1058).
history_columns = {
row["name"]
for row in conn.execute("PRAGMA table_info(download_history)")
}
if "file_params" not in history_columns:
conn.execute(
"ALTER TABLE download_history ADD COLUMN file_params TEXT"
)
# Creating the unique index on download_history.download_id can
# fail if pre-existing rows have duplicate values (e.g. from a
# previous version that lacked the index). Deduplicate first so
@@ -418,6 +431,12 @@ class DownloadQueueService:
return None
now = completed_at if completed_at is not None else time.time()
# Guard against legacy databases whose download_queue table
# predates the file_params column.
queue_columns = set(row.keys())
file_params_json = (
row["file_params"] if "file_params" in queue_columns else None
)
conn.execute(
"DELETE FROM download_queue WHERE download_id = ?",
(download_id,),
@@ -426,9 +445,9 @@ class DownloadQueueService:
"""
INSERT OR IGNORE INTO download_history (
download_id, model_id, model_version_id, model_name,
version_name, thumbnail_url, status, error, file_path,
bytes_downloaded, total_bytes, completed_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
version_name, thumbnail_url, file_params, status, error,
file_path, bytes_downloaded, total_bytes, completed_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
row["download_id"],
@@ -437,6 +456,7 @@ class DownloadQueueService:
row["model_name"],
row["version_name"],
row["thumbnail_url"],
file_params_json,
status,
error,
file_path,
@@ -503,6 +523,7 @@ class DownloadQueueService:
bytes_downloaded: int = 0,
total_bytes: Optional[int] = None,
is_already_exists: int = 0,
file_params: Optional[dict[str, Any]] = None,
) -> int:
"""Insert a record into the download history.
@@ -510,6 +531,7 @@ class DownloadQueueService:
inserted row.
"""
now = time.time()
file_params_json = json.dumps(file_params) if file_params is not None else None
async with self._lock:
conn = self._get_conn()
@@ -517,9 +539,10 @@ class DownloadQueueService:
"""
INSERT INTO download_history (
download_id, model_id, model_version_id, model_name,
version_name, thumbnail_url, status, error, file_path,
bytes_downloaded, total_bytes, completed_at, is_already_exists
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
version_name, thumbnail_url, file_params, status, error,
file_path, bytes_downloaded, total_bytes, completed_at,
is_already_exists
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
download_id,
@@ -528,6 +551,7 @@ class DownloadQueueService:
model_name,
version_name,
thumbnail_url,
file_params_json,
status,
error,
file_path,
@@ -702,7 +726,7 @@ class DownloadQueueService:
download_id, model_id, model_version_id, model_name,
version_name, thumbnail_url, source, file_params,
status, priority, added_at
) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'queued', 0, ?)
""",
(
new_id,
@@ -712,6 +736,7 @@ class DownloadQueueService:
row["version_name"],
row["thumbnail_url"],
"retry",
row["file_params"],
now,
),
)
@@ -755,7 +780,7 @@ class DownloadQueueService:
download_id, model_id, model_version_id, model_name,
version_name, thumbnail_url, source, file_params,
status, priority, added_at
) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'queued', 0, ?)
""",
(
new_id,
@@ -765,6 +790,7 @@ class DownloadQueueService:
row["version_name"],
row["thumbnail_url"],
"retry",
row["file_params"],
now,
),
)
@@ -62,6 +62,14 @@ class DownloadedVersionHistoryService:
);
CREATE INDEX IF NOT EXISTS idx_downloaded_model_versions_model
ON downloaded_model_versions(model_type, model_id);
CREATE TABLE IF NOT EXISTS downloaded_version_files (
model_type TEXT NOT NULL,
version_id INTEGER NOT NULL,
file_id INTEGER NOT NULL,
file_name TEXT,
downloaded_at REAL NOT NULL,
PRIMARY KEY (model_type, version_id, file_id)
);
"""
def __init__(self, db_path: str | None = None, *, settings_manager=None) -> None:
@@ -131,10 +139,13 @@ class DownloadedVersionHistoryService:
source: str = "manual",
file_path: str | None = None,
library_name: str | None = None,
file_id: int | None = None,
file_name: str | None = None,
) -> None:
normalized_type = _normalize_model_type(model_type)
normalized_version_id = _normalize_int(version_id)
normalized_model_id = _normalize_int(model_id)
normalized_file_id = _normalize_int(file_id)
if normalized_type is None or normalized_version_id is None:
return
@@ -168,6 +179,25 @@ class DownloadedVersionHistoryService:
active_library_name,
),
)
if normalized_file_id is not None:
# Per-file history for multi-file versions (#1058)
conn.execute(
"""
INSERT INTO downloaded_version_files (
model_type, version_id, file_id, file_name, downloaded_at
) VALUES (?, ?, ?, ?, ?)
ON CONFLICT(model_type, version_id, file_id) DO UPDATE SET
file_name = COALESCE(excluded.file_name, downloaded_version_files.file_name),
downloaded_at = excluded.downloaded_at
""",
(
normalized_type,
normalized_version_id,
normalized_file_id,
file_name,
timestamp,
),
)
conn.commit()
async def mark_downloaded_bulk(
@@ -255,8 +285,63 @@ class DownloadedVersionHistoryService:
self._get_active_library_name(),
),
)
# Whole-version deletion also clears the per-file records (#1058)
conn.execute(
"""
DELETE FROM downloaded_version_files
WHERE model_type = ? AND version_id = ?
""",
(normalized_type, normalized_version_id),
)
conn.commit()
async def mark_file_deleted(
self, model_type: str, version_id: int, file_id: int
) -> None:
"""Drop a single file record of a version, keeping siblings (#1058)."""
normalized_type = _normalize_model_type(model_type)
normalized_version_id = _normalize_int(version_id)
normalized_file_id = _normalize_int(file_id)
if (
normalized_type is None
or normalized_version_id is None
or normalized_file_id is None
):
return
async with self._lock:
conn = self._get_conn()
conn.execute(
"""
DELETE FROM downloaded_version_files
WHERE model_type = ? AND version_id = ? AND file_id = ?
""",
(normalized_type, normalized_version_id, normalized_file_id),
)
conn.commit()
async def get_downloaded_file_ids(
self, model_type: str, version_id: int
) -> list[int]:
"""Return the CivitAI file ids recorded as downloaded for a version."""
normalized_type = _normalize_model_type(model_type)
normalized_version_id = _normalize_int(version_id)
if normalized_type is None or normalized_version_id is None:
return []
async with self._lock:
conn = self._get_conn()
rows = conn.execute(
"""
SELECT file_id
FROM downloaded_version_files
WHERE model_type = ? AND version_id = ?
ORDER BY file_id ASC
""",
(normalized_type, normalized_version_id),
).fetchall()
return [int(row["file_id"]) for row in rows]
async def has_been_downloaded(self, model_type: str, version_id: int) -> bool:
normalized_type = _normalize_model_type(model_type)
normalized_version_id = _normalize_int(version_id)
+65 -1
View File
@@ -35,6 +35,10 @@ class ModelCache:
folders: List[str]
version_index: Dict[int, Dict[str, Any]] = field(default_factory=dict)
model_id_index: Dict[int, List[Dict[str, Any]]] = field(default_factory=dict)
# Multi-valued companion to version_index: every local file entry of a
# CivitAI model version, so versions with several downloaded files stay
# consistent (#1058).
version_files_index: Dict[int, List[Dict[str, Any]]] = field(default_factory=dict)
name_display_mode: str = "model_name"
_lock: Any = field(init=False, repr=False, default=None)
# Cache for last sort: (sort_key, order, seed) -> sorted list
@@ -116,6 +120,7 @@ class ModelCache:
self.version_index = {}
self.model_id_index = {}
self.version_files_index = {}
for item in self.raw_data:
self.add_to_version_index(item)
@@ -132,6 +137,17 @@ class ModelCache:
self.version_index[version_id] = item
# Register in the multi-valued index, deduplicated by file_path (#1058)
files = self.version_files_index.setdefault(version_id, [])
for entry in files:
if entry is item or (
isinstance(entry, dict)
and entry.get('file_path') == item.get('file_path')
):
break
else:
files.append(item)
model_id = self._normalize_version_id(civitai_data.get('modelId'))
if model_id is None:
return
@@ -159,12 +175,37 @@ class ModelCache:
if version_id is None:
return
# Drop only this file's entry from the multi-valued index (#1058)
files = self.version_files_index.get(version_id)
if files:
remaining = [
entry
for entry in files
if not (
entry is item
or (
isinstance(entry, dict)
and entry.get('file_path') == item.get('file_path')
)
)
]
if remaining:
self.version_files_index[version_id] = remaining
else:
self.version_files_index.pop(version_id, None)
# A surviving sibling file keeps the version present in the indexes
sibling = (self.version_files_index.get(version_id) or [None])[0]
existing = self.version_index.get(version_id)
if existing is item or (
isinstance(existing, dict)
and existing.get('file_path') == item.get('file_path')
):
self.version_index.pop(version_id, None)
if sibling is not None:
self.version_index[version_id] = sibling
else:
self.version_index.pop(version_id, None)
model_id = self._normalize_version_id(civitai_data.get('modelId'))
if model_id is None:
@@ -174,6 +215,20 @@ class ModelCache:
if not versions:
return
if sibling is not None:
# Update the descriptor to reflect the surviving sibling file
descriptor = self._build_version_descriptor(
sibling,
sibling.get('civitai') if isinstance(sibling, dict) else {},
version_id,
)
for index, existing_desc in enumerate(versions):
if existing_desc.get('versionId') == version_id:
if descriptor is not None:
versions[index] = descriptor
break
return
filtered = [v for v in versions if v.get('versionId') != version_id]
if filtered:
self.model_id_index[model_id] = filtered
@@ -206,6 +261,15 @@ class ModelCache:
versions = self.model_id_index.get(normalized_id, [])
return [dict(version) for version in versions]
def get_files_by_version_id(self, version_id: Any) -> List[Dict[str, Any]]:
"""Return every local file entry for a CivitAI model version (#1058)."""
normalized_id = self._normalize_version_id(version_id)
if normalized_id is None:
return []
return list(self.version_files_index.get(normalized_id, []))
async def resort(self):
"""Resort cached data according to last sort mode if set"""
async with self._lock:
+147 -2
View File
@@ -25,6 +25,28 @@ from .cache_health_monitor import CacheHealthMonitor, CacheHealthStatus
logger = logging.getLogger(__name__)
# Canonical set of weight-file extensions stripped when normalizing model
# names for matching (ModelScanner.find_matching_models and the recipe rematch
# filename key share this set). It is the union of the LoRA scanner set
# ({".safetensors"}) and the Checkpoint scanner set (ComfyUI's
# supported_pt_extensions plus ".gguf") so type-blind lookups (lora +
# checkpoint merged) cover every format either scanner indexes. ".safebin"
# is deliberately absent — no scanner indexes it, so a recipe entry
# "model.safebin" must not be bound to a local "model.safetensors".
WEIGHT_FILE_EXTENSIONS = frozenset(
{
".safetensors",
".ckpt",
".pt",
".pt2",
".bin",
".pth",
".pkl",
".sft",
".gguf",
}
)
def _is_excluded_dir(name: str) -> bool:
"""Return True when a directory entry must be skipped during model walks.
@@ -2140,8 +2162,98 @@ class ModelScanner:
return sorted_models
return sorted_models[:limit]
async def get_model_info_by_name(self, name):
"""Get model information by name"""
@staticmethod
def find_matching_models(
raw_data: List[Dict[str, Any]],
name: str,
*,
base_model: Optional[str] = None,
extensions: Optional[Set[str]] = None,
) -> List[Dict[str, Any]]:
"""Return all cached models matching ``name`` (case-insensitive).
A name containing a path separator must equal the model's
folder-relative path; a bare name matches on basename. When
``base_model`` is given, confident mismatches are rejected while
unknowns on either side stay eligible (lenient guard).
``extensions`` should be the scanner's own ``file_extensions`` so
suffix stripping only covers formats the scanner actually indexes;
when omitted, the shared :data:`WEIGHT_FILE_EXTENSIONS` set is used.
"""
# Longest first so overlapping suffixes strip correctly.
exts = sorted(extensions or WEIGHT_FILE_EXTENSIONS, key=len, reverse=True)
normalized_name = str(name).replace("\\", "/").casefold()
for ext in exts:
if normalized_name.endswith(ext):
normalized_name = normalized_name[: -len(ext)]
break
has_path = "/" in normalized_name
basename = normalized_name.rsplit("/", 1)[-1]
matches = []
for model in raw_data:
file_name = str(model.get("file_name") or "").replace("\\", "/")
folder = str(model.get("folder") or "").replace("\\", "/").strip("/")
model_path = f"{folder}/{file_name}" if folder else file_name
for ext in exts:
if model_path.casefold().endswith(ext):
model_path = model_path[: -len(ext)]
break
if (has_path and model_path.casefold() == normalized_name) or (
not has_path and model_path.rsplit("/", 1)[-1].casefold() == basename
):
matches.append(model)
expected_base = str(base_model or "").strip().casefold()
if expected_base and expected_base != "unknown":
matches = [
model
for model in matches
if str(model.get("base_model") or "").strip().casefold()
in ("", "unknown", expected_base)
]
return matches
async def find_models_by_name(
self, name: str, *, base_model: Optional[str] = None
) -> List[Dict[str, Any]]:
"""Return every cached model matching ``name`` (see ``find_matching_models``)."""
try:
cache = await self.get_cached_data()
return self.find_matching_models(
cache.raw_data,
name,
base_model=base_model,
extensions=self.file_extensions,
)
except Exception as e:
logger.error(f"Error finding models by name: {e}", exc_info=True)
return []
async def get_model_info_by_name(
self,
name: str,
*,
require_unique: bool = False,
base_model: Optional[str] = None,
):
"""Get model information by name.
Default mode keeps the legacy first-match/fallback semantics. With
``require_unique`` an ambiguous name is a miss, and ``base_model``
rejects confident base-model mismatches (unknowns stay eligible).
"""
if require_unique or base_model:
try:
matches = await self.find_models_by_name(name, base_model=base_model)
if require_unique and len(matches) != 1:
return None
return matches[0] if matches else None
except Exception as e:
logger.error(f"Error getting model info by name: {e}", exc_info=True)
return None
try:
cache = await self.get_cached_data()
@@ -2446,6 +2558,39 @@ class ModelScanner:
logger.error(f"Error checking model version existence: {e}")
return False
async def get_files_for_version(self, model_version_id: int) -> List[Dict[str, Any]]:
"""Get all local file entries for a specific model version (#1058).
A Civitai model version can have several weight files downloaded;
unlike the single-valued version_index this returns every entry.
Args:
model_version_id: Civitai model version ID
Returns:
List[Dict]: Cache entries (may be empty)
"""
try:
normalized_id = int(model_version_id)
except (TypeError, ValueError):
return []
try:
cache = await self.get_cached_data()
if not cache:
return []
getter = getattr(cache, "get_files_by_version_id", None)
if getter is not None:
return getter(normalized_id)
# Fallback for cache implementations without the multi-file index
entry = cache.version_index.get(normalized_id)
return [entry] if entry is not None else []
except Exception as e:
logger.error(f"Error getting files for model version: {e}")
return []
async def get_model_versions_by_id(self, model_id: int) -> List[Dict[str, Any]]:
"""Get all versions of a model by its ID
+40 -10
View File
@@ -15,6 +15,7 @@ from ..config import config
from ..utils.constants import VALID_CHECKPOINT_SUB_TYPES, VALID_LORA_TYPES
from ..utils.file_utils import calculate_autov3
from ..utils.recipe_open_stats import RecipeOpenStats
from .model_scanner import WEIGHT_FILE_EXTENSIONS
from .recipe_cache import RecipeCache
from .recipes.errors import RecipeNotFoundError, RecipePersistenceError
from natsort import natsorted
@@ -36,11 +37,6 @@ logger = logging.getLogger(__name__)
# explicitly to "diffusion_model" (mirrors Oracle R2-F1).
_CHECKPOINT_MODEL_TYPE_ALIASES = {"diffusionmodel": "diffusion_model"}
# Known weight-file extensions stripped by _normalize_filename_key. Names are
# stored extensionless on both sides, so splitext would misread dotted stems
# ("my.mix" -> "my") and silently collide distinct models.
_WEIGHT_FILE_EXTS = (".safetensors", ".ckpt", ".pt", ".pth", ".gguf", ".bin", ".safebin", ".sft")
class RecipeScanner:
"""Service for scanning and managing recipe images"""
@@ -179,13 +175,15 @@ class RecipeScanner:
Only known weight-file extensions are stripped names are stored
extensionless on both sides, so splitext would misread dotted stems
("my.mix" -> "my") and collide distinct models.
("my.mix" -> "my") and collide distinct models. The extension set is
shared with ModelScanner.find_matching_models, and is iterated longest
first to keep the strip ordering identical to that function.
"""
if not name:
return ""
basename = os.path.basename(name.replace("\\", "/"))
lower = basename.lower()
for ext in _WEIGHT_FILE_EXTS:
for ext in sorted(WEIGHT_FILE_EXTENSIONS, key=len, reverse=True):
if lower.endswith(ext):
basename = basename[: -len(ext)]
break
@@ -2926,13 +2924,45 @@ class RecipeScanner:
return normalized
async def get_local_lora(self, name: str) -> Optional[Dict[str, Any]]:
"""Lookup a local LoRA model by name."""
async def get_local_lora(
self, name: str, base_model: Optional[str] = None
) -> Optional[Dict[str, Any]]:
"""Lookup an unambiguous local LoRA by name and optional base model."""
if not self._lora_scanner or not name:
return None
return await self._lora_scanner.get_model_info_by_name(name)
return await self._lora_scanner.get_model_info_by_name(
name, require_unique=True, base_model=base_model
)
async def find_local_loras_by_name(
self, name: str, base_model: Optional[str] = None
) -> List[Dict[str, Any]]:
"""Return every local LoRA matching ``name`` (used to explain lookup misses)."""
if not self._lora_scanner or not name:
return []
return await self._lora_scanner.find_models_by_name(name, base_model=base_model)
async def get_local_lora_by_hash(self, hash_value: str) -> Optional[Dict[str, Any]]:
"""Lookup a local LoRA through the scanner's hash index."""
if not self._lora_scanner or not hash_value:
return None
file_path = self._lora_scanner.get_path_by_hash(hash_value)
if not file_path:
return None
target_path = os.path.normcase(os.path.abspath(file_path))
cached_data = await self._lora_scanner.get_cached_data()
for model in cached_data.raw_data:
model_path = model.get("file_path")
if model_path and os.path.normcase(os.path.abspath(model_path)) == target_path:
return model
return None
async def get_local_checkpoint(self, name: str) -> Optional[Dict[str, Any]]:
"""Lookup a local checkpoint model by name."""
+14 -1
View File
@@ -426,8 +426,21 @@ class RecipePersistenceService:
if not recipe_path or not os.path.exists(recipe_path):
raise RecipeNotFoundError("Recipe not found")
target_lora = await recipe_scanner.get_local_lora(target_name)
with open(recipe_path, "r", encoding="utf-8") as file_obj:
recipe_base_model = json.load(file_obj).get("base_model", "")
target_lora = await recipe_scanner.get_local_lora(target_name, recipe_base_model)
if not target_lora:
matches = await recipe_scanner.find_local_loras_by_name(target_name)
if len(matches) > 1:
raise RecipeValidationError(
f"Multiple local LoRAs match '{target_name}'; "
"include the folder path to disambiguate"
)
if len(matches) == 1:
raise RecipeValidationError(
f"Local LoRA '{target_name}' has a different base model than the recipe"
)
raise RecipeNotFoundError(f"Local LoRA not found with name: {target_name}")
recipe_data, updated_lora = await recipe_scanner.update_lora_entry(
@@ -603,6 +603,51 @@
cursor: pointer;
}
.file-option-radio input[type="checkbox"] {
width: 16px;
height: 16px;
accent-color: var(--lora-accent);
cursor: pointer;
}
/* Files already in the library are greyed out and not clickable */
.file-option.disabled {
opacity: 0.55;
cursor: not-allowed;
}
.file-option.disabled:hover {
border-color: var(--border-color);
box-shadow: none;
transform: none;
}
.file-option.disabled input[type="checkbox"] {
cursor: not-allowed;
}
/* Options of the other routing group are temporarily disabled once a
selection is made (mixed-type multi-select is not allowed) */
.file-option.group-disabled {
opacity: 0.6;
cursor: not-allowed;
}
.file-option.group-disabled:hover {
border-color: var(--border-color);
box-shadow: none;
transform: none;
}
.file-option.group-disabled input[type="checkbox"] {
cursor: not-allowed;
}
.file-tag.in-library {
background: oklch(var(--lora-accent) / 0.15);
color: var(--lora-accent);
}
.file-option-info {
flex: 1;
min-width: 0;
+55 -39
View File
@@ -573,46 +573,53 @@ function renderRow(version, options) {
);
const actions = [];
if (!version.isInLibrary) {
const canDownload = isDownloadAllowed(version);
const downloadIcon = isEarlyAccess ? '<i class="fas fa-bolt"></i> ' : '';
let downloadTitle;
if (!canDownload) {
downloadTitle = translate(
'modals.model.versions.actions.downloadNotAllowedTooltip',
{},
'This version is only available for on-site generation on Civitai'
);
} else if (isPaidPermanent(version)) {
downloadTitle = translate(
'modals.model.versions.actions.downloadPaidTooltip',
{},
'Download this paid version from Civitai'
);
} else if (isEarlyAccess) {
downloadTitle = translate(
'modals.model.versions.actions.downloadEarlyAccessTooltip',
{},
'Download this early access version from Civitai'
);
} else {
downloadTitle = translate(
'modals.model.versions.actions.downloadTooltip',
{},
'Download this version'
);
const canDownload = isDownloadAllowed(version);
const downloadIcon = isEarlyAccess ? '<i class="fas fa-bolt"></i> ' : '';
let downloadTitle;
if (!canDownload) {
downloadTitle = translate(
'modals.model.versions.actions.downloadNotAllowedTooltip',
{},
'This version is only available for on-site generation on Civitai'
);
} else if (version.isInLibrary) {
// In-library versions may still have undownloaded weight files; the
// download modal's file dialog decides what remains (#1058).
downloadTitle = translate(
'modals.model.versions.actions.downloadRemainingTooltip',
{},
'Download remaining files of this version'
);
} else if (isPaidPermanent(version)) {
downloadTitle = translate(
'modals.model.versions.actions.downloadPaidTooltip',
{},
'Download this paid version from Civitai'
);
} else if (isEarlyAccess) {
downloadTitle = translate(
'modals.model.versions.actions.downloadEarlyAccessTooltip',
{},
'Download this early access version from Civitai'
);
} else {
downloadTitle = translate(
'modals.model.versions.actions.downloadTooltip',
{},
'Download this version'
);
}
actions.push(buildActionButton(
downloadLabel,
canDownload ? 'version-action-primary' : 'version-action-disabled',
canDownload ? 'download' : '',
{
title: downloadTitle,
iconMarkup: downloadIcon,
disabled: !canDownload,
}
actions.push(buildActionButton(
downloadLabel,
canDownload ? 'version-action-primary' : 'version-action-disabled',
canDownload ? 'download' : '',
{
title: downloadTitle,
iconMarkup: downloadIcon,
disabled: !canDownload,
}
));
} else if (version.filePath) {
));
if (version.isInLibrary && version.filePath) {
actions.push(buildActionButton(
deleteLabel,
'version-action-danger',
@@ -1422,6 +1429,15 @@ export function initVersionsTab({
button.disabled = true;
try {
// In-library versions may still have undownloaded weight files
// (#1058). The tab payload has no per-file state, so open the
// download modal's file dialog, which refetches the full version
// payload and shows what remains.
if (version.isInLibrary) {
await downloadManager.openFileSelectionForVersion(modelType, modelId, versionId);
return;
}
const pathInfo = await resolveDownloadPathFromCurrentVersion();
const resolveTemplatePath = shouldResolveTemplatePath(version, pathInfo);
const success = await downloadManager.downloadVersionWithDefaults(modelType, modelId, versionId, {
+332 -41
View File
@@ -25,6 +25,12 @@ export class DownloadManager {
this.apiClient = null;
this.useDefaultPath = false;
// Multi-file selection state: selectedFile stays the first selected
// file for backward compatibility with single-file flows (#1058).
this.selectedFile = null;
this.selectedFiles = [];
this._lastDownloadError = null;
// Batch mode state
this.batchModels = [];
this.isBatchMode = false;
@@ -160,6 +166,8 @@ export class DownloadManager {
this.modelVersionId = null;
this.source = null;
this.selectedFile = null;
this.selectedFiles = [];
this._lastDownloadError = null;
this._isDiffusionModel = false;
this.selectedFolder = '';
@@ -546,6 +554,64 @@ export class DownloadManager {
await this.fetchVersionsForCurrentModel();
}
/**
* Open the download modal directly on the file-selection step for a
* specific model version (#1058). Used by entry points (e.g.
* ModelVersionsTab) whose version payloads lack per-file downloaded
* state, so the full versions payload is fetched here first.
*/
async openFileSelectionForVersion(modelType, modelId, versionId, { source = null } = {}) {
try {
this.apiClient = getModelApiClient(modelType);
} catch (error) {
this.apiClient = getModelApiClient();
}
this.showDownloadModal();
this.modelId = modelId ? modelId.toString() : null;
this.modelVersionId = versionId ? versionId.toString() : null;
this.source = source;
if (!this.modelId) {
return;
}
try {
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingVersions'));
await this.retrieveVersionsForModel(this.modelId, this.source);
} catch (error) {
showToast('toast.downloads.loadError', { message: error.message }, 'error');
return;
} finally {
this.loadingManager.hide();
}
const version = this.versions.find(v => v.id.toString() === this.modelVersionId);
if (!version) {
console.warn('[download] openFileSelectionForVersion: version %s not found for model %s',
this.modelVersionId, this.modelId);
this.showVersionStep();
return;
}
const hasRemainingFiles = this._getWeightFiles(version).length > 1
&& this._getRemainingFiles(version).length > 0;
if (hasRemainingFiles) {
this.showFileSelectionStep(version.id);
return;
}
// Nothing left to download for this version (single file or all
// files already in the library) — fall back to the version step.
if (version.existsLocally) {
showToast('toast.loras.versionExists', {}, 'info');
}
this.currentVersion = version;
this.showVersionStep();
}
showVersionStep() {
document.getElementById('urlStep').style.display = 'none';
document.getElementById('versionStep').style.display = 'block';
@@ -595,7 +661,10 @@ export class DownloadManager {
</div>`;
}
const fileBadge = modelFiles.length > 1 && !existsLocally
// Always offer the file-selection entry for multi-file versions,
// even when the version is already (partially) in the library, so
// remaining files can still be downloaded (#1058).
const fileBadge = modelFiles.length > 1
? `<span class="file-select-badge" data-version-id="${version.id}">
<i class="fas fa-th-list"></i> ${modelFiles.length} ${translate('modals.download.fileSelection.files')} <i class="fas fa-chevron-right badge-arrow"></i>
</span>`
@@ -667,9 +736,14 @@ export class DownloadManager {
const nextButton = document.getElementById('nextFromVersion');
if (!nextButton) return;
const existsLocally = this.currentVersion?.existsLocally;
const version = this.currentVersion;
const existsLocally = version?.existsLocally;
// A partially downloaded multi-file version still has downloadable
// files, so Next routes into the file dialog instead of blocking (#1058).
const hasRemainingFiles = this._getWeightFiles(version).length > 1
&& this._getRemainingFiles(version).length > 0;
if (existsLocally) {
if (existsLocally && !hasRemainingFiles) {
nextButton.disabled = true;
nextButton.classList.add('disabled');
nextButton.textContent = translate('modals.download.alreadyInLibrary');
@@ -680,14 +754,41 @@ export class DownloadManager {
}
}
_getWeightFiles(version) {
return (version?.files || []).filter(f => isModelWeightFile(f.type));
}
_getRemainingFiles(version) {
const downloadedIds = new Set(
(version?.downloadedFiles || []).map(f => String(f.fileId))
);
return this._getWeightFiles(version).filter(f => !downloadedIds.has(String(f.id)));
}
// Files of type UNet / Diffusion Model are routed to the diffusion_model
// root while regular files go to the model-type root, so a single
// multi-file selection session must stay within one routing group.
_getFileRoutingGroup(file) {
return (file.type === 'UNet' || file.type === 'Diffusion Model') ? 'diffusion' : 'model';
}
showFileSelectionStep(versionId) {
const version = this.versions.find(v => v.id.toString() === versionId.toString());
if (!version) return;
this.currentVersion = version;
const modelFiles = (version.files || []).filter(f => isModelWeightFile(f.type));
// Start each file-selection session with a clean selection
this.selectedFiles = [];
this.selectedFile = null;
const modelFiles = this._getWeightFiles(version);
const downloadedIds = new Set(
(version.downloadedFiles || []).map(f => String(f.fileId))
);
document.getElementById('versionStep').style.display = 'none';
// Hide every other step — this dialog can be entered directly from
// entry points like ModelVersionsTab, where the URL step would
// otherwise remain visible (#1058).
document.querySelectorAll('.download-step').forEach(step => step.style.display = 'none');
document.getElementById('fileSelectionStep').style.display = 'block';
const nameEl = document.getElementById('fileSelectionVersionName');
@@ -699,9 +800,12 @@ export class DownloadManager {
container.innerHTML = modelFiles.map(file => {
const meta = file.metadata || {};
const sizeGB = file.sizeKB ? (file.sizeKB / (1024 * 1024)).toFixed(2) : '--';
const isSelected = this.selectedFile?.id === file.id;
const isDownloaded = downloadedIds.has(String(file.id));
const tags = [];
if (isDownloaded) {
tags.push(`<span class="file-tag in-library">${translate('modals.download.fileSelection.inLibrary', {}, 'In Library')}</span>`);
}
if (meta.size) tags.push(`<span class="file-tag size">${meta.size}</span>`);
if (meta.format) tags.push(`<span class="file-tag format">${meta.format}</span>`);
if (meta.fp) tags.push(`<span class="file-tag fp">${meta.fp}</span>`);
@@ -709,9 +813,9 @@ export class DownloadManager {
const fileName = file.name || '';
return `
<div class="file-option ${isSelected ? 'selected' : ''}" data-file-id="${file.id}">
<div class="file-option ${isDownloaded ? 'disabled' : ''}" data-file-id="${file.id}">
<div class="file-option-radio">
<input type="radio" name="fileSelection" value="${file.id}" ${isSelected ? 'checked' : ''}>
<input type="checkbox" name="fileSelection" value="${file.id}" ${isDownloaded ? 'disabled' : ''}>
</div>
<div class="file-option-info">
<div class="file-option-tags">
@@ -725,33 +829,80 @@ export class DownloadManager {
}).join('');
container.querySelectorAll('.file-option').forEach(el => {
el.addEventListener('click', () => {
container.querySelectorAll('.file-option').forEach(o => o.classList.remove('selected'));
el.classList.add('selected');
const radio = el.querySelector('input[type="radio"]');
if (radio) radio.checked = true;
el.addEventListener('click', (event) => {
// Already-downloaded files stay disabled regardless
if (el.classList.contains('disabled')) {
event.preventDefault();
return;
}
const checkbox = el.querySelector('input[type="checkbox"]');
if (!checkbox || checkbox.disabled) {
event.preventDefault();
return;
}
// Clicking the checkbox directly toggles natively; clicking
// anywhere else on the option toggles it programmatically.
if (event.target !== checkbox) {
checkbox.checked = !checkbox.checked;
}
this._syncFileSelectionState();
});
});
}
confirmFileSelection() {
const selectedRadio = document.querySelector('#fileSelectionList input[type="radio"]:checked');
if (!selectedRadio) {
console.warn('[download] confirmFileSelection: no radio button checked');
return;
}
// Sync this.selectedFiles with the DOM checkboxes and enforce the
// mixed-type routing guard by disabling the other routing group.
_syncFileSelectionState() {
const container = document.getElementById('fileSelectionList');
if (!container || !this.currentVersion) return;
const checkedValues = new Set(
Array.from(container.querySelectorAll('input[type="checkbox"]:checked'))
.map(cb => cb.value)
);
const modelFiles = this._getWeightFiles(this.currentVersion);
this.selectedFiles = modelFiles.filter(f => checkedValues.has(f.id.toString()));
this.selectedFile = this.selectedFiles[0] || null;
const activeGroup = this.selectedFiles.length > 0
? this._getFileRoutingGroup(this.selectedFiles[0])
: null;
container.querySelectorAll('.file-option').forEach(el => {
const checkbox = el.querySelector('input[type="checkbox"]');
if (!checkbox || el.classList.contains('disabled')) return;
const file = modelFiles.find(f => f.id.toString() === el.dataset.fileId);
const groupBlocked = activeGroup !== null
&& file
&& this._getFileRoutingGroup(file) !== activeGroup
&& !checkbox.checked;
el.classList.toggle('selected', checkbox.checked);
el.classList.toggle('group-disabled', groupBlocked);
checkbox.disabled = groupBlocked;
});
}
confirmFileSelection() {
const version = this.currentVersion;
if (!version) {
console.warn('[download] confirmFileSelection: no currentVersion set');
return;
}
const modelFiles = (version.files || []).filter(f => isModelWeightFile(f.type));
this.selectedFile = modelFiles.find(f => f.id.toString() === selectedRadio.value);
// Sync from the DOM first so programmatically checked boxes count too
this._syncFileSelectionState();
console.log('[download] confirmFileSelection: selected file id=%s, name="%s", type="%s", metadata=%o',
this.selectedFile?.id, this.selectedFile?.name, this.selectedFile?.type, this.selectedFile?.metadata);
if (this.selectedFiles.length === 0) {
console.warn('[download] confirmFileSelection: no file selected');
showToast('toast.loras.pleaseSelectFile', {}, 'error');
return;
}
console.log('[download] confirmFileSelection: %d file(s) selected — %o',
this.selectedFiles.length,
this.selectedFiles.map(f => ({ id: f.id, name: f.name, type: f.type })));
document.getElementById('fileSelectionStep').style.display = 'none';
document.getElementById('downloadLocationStep').style.display = 'block';
@@ -782,6 +933,13 @@ export class DownloadManager {
return;
}
if (this.currentVersion.existsLocally) {
// Multi-file versions with remaining undownloaded files route
// into the file dialog instead of being blocked outright (#1058).
if (this._getWeightFiles(this.currentVersion).length > 1
&& this._getRemainingFiles(this.currentVersion).length > 0) {
this.showFileSelectionStep(this.currentVersion.id);
return;
}
showToast('toast.loras.versionExists', {}, 'info');
return;
}
@@ -916,6 +1074,9 @@ export class DownloadManager {
source = null,
fileParams = null,
closeModal = false,
deferReload = false,
suppressSuccessToast = false,
suppressFailureSummary = false,
}) {
const config = this.apiClient?.apiConfig?.config;
@@ -924,7 +1085,8 @@ export class DownloadManager {
}
const displayName = versionName || `#${versionId}`;
const retryParams = { modelId, versionId, versionName, modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot, source, fileParams, closeModal: false };
const retryParams = { modelId, versionId, versionName, modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot, source, fileParams, closeModal: false, deferReload, suppressSuccessToast, suppressFailureSummary };
this._lastDownloadError = null;
let ws = null;
let updateProgress = () => { };
let cancelled = false;
@@ -1007,7 +1169,9 @@ export class DownloadManager {
if (response?.skipped) {
this.loadingManager.setStatus(translate('modals.download.status.finalizing'));
updateProgress(100, 0, displayName);
showToast('toast.loras.downloadSkippedByBaseModel', { baseModel: response.base_model || 'Unknown' }, 'warning');
if (!suppressSuccessToast) {
showToast('toast.loras.downloadSkippedByBaseModel', { baseModel: response.base_model || 'Unknown' }, 'warning');
}
if (closeModal) {
modalManager.closeModal('downloadModal');
}
@@ -1016,6 +1180,22 @@ export class DownloadManager {
if (!response?.success) {
this.loadingManager.setStatus(translate('modals.download.status.finalizing'));
const errorMessage = response?.error || 'Unknown error';
// When the caller aggregates failures itself (multi-file
// loop), just record the error and return (#1058).
if (suppressFailureSummary) {
this._lastDownloadError = errorMessage;
return false;
}
// A file-level "already in library" rejection is an expected
// outcome when browsing files of a partially downloaded
// version — surface it as a lightweight toast instead of the
// failure summary modal so the user can simply go back and
// pick another file (#1058).
if (typeof errorMessage === 'string' && errorMessage.includes('already exists in')) {
showToast(errorMessage, {}, 'info');
return false;
}
showDownloadBatchSummary({
total: 1,
completed: 0,
@@ -1026,7 +1206,7 @@ export class DownloadManager {
source,
url: this._buildSingleItemUrl({ modelId, versionId, source }),
},
error: response?.error || 'Unknown error',
error: errorMessage,
name: displayName,
}],
onRetry: () => this.executeDownloadWithProgress(retryParams),
@@ -1034,7 +1214,9 @@ export class DownloadManager {
return false;
}
showToast('toast.loras.downloadCompleted', {}, 'success');
if (!suppressSuccessToast) {
showToast('toast.loras.downloadCompleted', {}, 'success');
}
if (closeModal) {
modalManager.closeModal('downloadModal');
@@ -1045,29 +1227,35 @@ export class DownloadManager {
ws = null;
}
const pageState = this.apiClient.getPageState();
if (!deferReload) {
const pageState = this.apiClient.getPageState();
if (!useDefaultPaths && targetFolder) {
pageState.activeFolder = targetFolder;
setStorageItem(`${this.apiClient.modelType}_activeFolder`, targetFolder);
if (!useDefaultPaths && targetFolder) {
pageState.activeFolder = targetFolder;
setStorageItem(`${this.apiClient.modelType}_activeFolder`, targetFolder);
document.querySelectorAll('.folder-tags .tag').forEach(tag => {
const isActive = tag.dataset.folder === targetFolder;
tag.classList.toggle('active', isActive);
if (isActive && !tag.parentNode.classList.contains('collapsed')) {
tag.scrollIntoView({ behavior: 'smooth', block: 'nearest' });
}
});
document.querySelectorAll('.folder-tags .tag').forEach(tag => {
const isActive = tag.dataset.folder === targetFolder;
tag.classList.toggle('active', isActive);
if (isActive && !tag.parentNode.classList.contains('collapsed')) {
tag.scrollIntoView({ behavior: 'smooth', block: 'nearest' });
}
});
}
await resetAndReload(true);
}
await resetAndReload(true);
return true;
} catch (error) {
if (cancelled) {
console.log('Download cancelled by user:', downloadId);
} else {
console.error('Failed to download model version:', error);
if (suppressFailureSummary) {
this._lastDownloadError = error?.message || 'Unknown error';
return false;
}
showDownloadBatchSummary({
total: 1,
completed: 0,
@@ -1097,6 +1285,89 @@ export class DownloadManager {
}
}
/**
* Download multiple selected files of the same version sequentially,
* reusing the location-step choices for every file. Per-file toasts,
* reloads and failure modals are suppressed; a single aggregated result
* is shown at the end (design decision D5, #1058).
*/
async _downloadSelectedFilesSequentially({ modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot = false, files = null }) {
const filesToDownload = files || this.selectedFiles;
const totalFiles = filesToDownload.length;
const failedItems = [];
let completedDownloads = 0;
for (const file of filesToDownload) {
const fileParams = {
id: file.id,
name: file.name || null,
type: file.type || 'Model',
format: file.metadata?.format || null,
size: file.metadata?.size || null,
fp: file.metadata?.fp || null,
};
console.log('[download] multi-file loop: downloading file id=%s, name="%s" (%d/%d)',
fileParams.id, fileParams.name, completedDownloads + failedItems.length + 1, totalFiles);
const success = await this.executeDownloadWithProgress({
modelId: this.modelId,
versionId: this.currentVersion.id,
versionName: file.name || `${this.currentVersion.name} #${file.id}`,
modelRoot,
targetFolder,
useDefaultPaths,
useSaveDirAsRoot,
source: this.source,
fileParams,
closeModal: false,
deferReload: true,
suppressSuccessToast: true,
suppressFailureSummary: true,
});
if (success) {
completedDownloads++;
} else {
failedItems.push({
item: {
modelId: this.modelId,
versionId: this.currentVersion.id,
source: this.source,
file,
url: this._buildSingleItemUrl({
modelId: this.modelId,
versionId: this.currentVersion.id,
source: this.source,
}),
},
error: this._lastDownloadError || 'Unknown error',
name: file.name || `#${file.id}`,
});
}
}
if (failedItems.length === 0) {
showToast('toast.loras.allDownloadSuccessful', { count: completedDownloads }, 'success');
} else {
showDownloadBatchSummary({
total: totalFiles,
completed: completedDownloads,
failedItems,
onRetry: () => this._downloadSelectedFilesSequentially({
modelRoot,
targetFolder,
useDefaultPaths,
useSaveDirAsRoot,
files: failedItems.map(f => f.item.file),
}),
});
}
await resetAndReload(true);
return failedItems.length === 0;
}
async _downloadHfSingle({ modelRoot, targetFolder, useDefaultPaths, files = null }) {
modalManager.closeModal('downloadModal');
this.loadingManager.restoreProgressBar();
@@ -1307,6 +1578,14 @@ export class DownloadManager {
? (ver.modelSizeKB / 1024).toFixed(1)
: (ver?.files?.[0]?.sizeKB ? (ver.files[0].sizeKB / 1024).toFixed(1) : '?');
const existsLocally = ver?.existsLocally;
// Multi-file versions that are only partially downloaded get a
// distinct hint instead of the plain in-library badge (#1058).
const isPartiallyDownloaded = existsLocally
&& this._getWeightFiles(ver).length > 1
&& this._getRemainingFiles(ver).length > 0;
const localBadgeLabel = isPartiallyDownloaded
? translate('modals.download.partiallyDownloaded', {}, 'Partially downloaded')
: translate('modals.download.inLibrary');
return `
<div class="batch-preview-item ${existsLocally ? 'batch-preview-local' : ''}" data-index="${index}">
<div class="batch-preview-thumbnail">
@@ -1317,7 +1596,7 @@ export class DownloadManager {
<div class="batch-preview-meta">
${ver?.baseModel ? `<span>${ver.baseModel}</span>` : ''}
<span>${fileSize} MB</span>
${existsLocally ? `<span class="batch-preview-local-badge"><i class="fas fa-check"></i> ${translate('modals.download.inLibrary')}</span>` : ''}
${existsLocally ? `<span class="batch-preview-local-badge"><i class="fas fa-check"></i> ${localBadgeLabel}</span>` : ''}
</div>
</div>
${item.versions.length > 1 ? `
@@ -1608,8 +1887,20 @@ export class DownloadManager {
});
}
// Multi-file selection: download all selected files sequentially,
// reusing the chosen location for every file (#1058).
if (this.selectedFiles.length > 1) {
modalManager.closeModal('downloadModal');
return this._downloadSelectedFilesSequentially({
modelRoot,
targetFolder,
useDefaultPaths,
});
}
const fileParams = this.selectedFile ? {
id: this.selectedFile.id,
name: this.selectedFile.name || null,
type: this.selectedFile.type || 'Model',
format: this.selectedFile.metadata?.format || null,
size: this.selectedFile.metadata?.size || null,
@@ -0,0 +1,164 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const {
DOWNLOAD_MANAGER_MODULE,
MODAL_MANAGER_MODULE,
UI_HELPERS_MODULE,
STATE_MODULE,
LOADING_MANAGER_MODULE,
API_FACTORY_MODULE,
STORAGE_HELPERS_MODULE,
FOLDER_TREE_MANAGER_MODULE,
I18N_HELPERS_MODULE,
} = vi.hoisted(() => ({
DOWNLOAD_MANAGER_MODULE: new URL('../../../static/js/managers/DownloadManager.js', import.meta.url).pathname,
MODAL_MANAGER_MODULE: new URL('../../../static/js/managers/ModalManager.js', import.meta.url).pathname,
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
STATE_MODULE: new URL('../../../static/js/state/index.js', import.meta.url).pathname,
LOADING_MANAGER_MODULE: new URL('../../../static/js/managers/LoadingManager.js', import.meta.url).pathname,
API_FACTORY_MODULE: new URL('../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
STORAGE_HELPERS_MODULE: new URL('../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
FOLDER_TREE_MANAGER_MODULE: new URL('../../../static/js/components/FolderTreeManager.js', import.meta.url).pathname,
I18N_HELPERS_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
}));
vi.mock(MODAL_MANAGER_MODULE, () => ({
modalManager: {
showModal: vi.fn(),
closeModal: vi.fn(),
},
}));
vi.mock(UI_HELPERS_MODULE, () => ({
showToast: vi.fn(),
}));
vi.mock(STATE_MODULE, () => ({
state: {
global: {
settings: {},
},
},
}));
vi.mock(LOADING_MANAGER_MODULE, () => ({
LoadingManager: vi.fn(() => ({
showSimpleLoading: vi.fn(),
hide: vi.fn(),
restoreProgressBar: vi.fn(),
showDownloadProgress: vi.fn(() => vi.fn()),
setStatus: vi.fn(),
})),
}));
vi.mock(API_FACTORY_MODULE, () => ({
getModelApiClient: vi.fn(() => ({
apiConfig: {
config: {
displayName: 'LoRA',
singularName: 'lora',
},
},
})),
resetAndReload: vi.fn(),
}));
vi.mock(STORAGE_HELPERS_MODULE, () => ({
getStorageItem: vi.fn((_key, defaultValue) => defaultValue),
setStorageItem: vi.fn(),
}));
vi.mock(FOLDER_TREE_MANAGER_MODULE, () => ({
FolderTreeManager: vi.fn(() => ({
clearSelection: vi.fn(),
init: vi.fn(),
})),
}));
vi.mock(I18N_HELPERS_MODULE, () => ({
translate: vi.fn((_, __, fallback) => fallback ?? ''),
}));
const MULTI_FILE_VERSION = {
id: 201,
name: 'Multi-file version',
images: [],
files: [
{ id: 1001, type: 'Model', sizeKB: 2048, name: 'file-a.safetensors' },
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
],
createdAt: '2026-01-01T00:00:00Z',
};
const SINGLE_FILE_VERSION = {
id: 202,
name: 'Single-file version',
images: [],
files: [{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' }],
createdAt: '2026-01-01T00:00:00Z',
};
describe('DownloadManager multi-file version badge (#1058)', () => {
let DownloadManager;
beforeEach(async () => {
vi.resetModules();
document.body.innerHTML = `
<div id="urlStep"></div>
<div id="versionStep"></div>
<div id="versionList"></div>
<button id="nextFromVersion"></button>
`;
({ DownloadManager } = await import(DOWNLOAD_MANAGER_MODULE));
});
afterEach(() => {
document.body.innerHTML = '';
});
it('shows the file-select badge for a multi-file version already in library', () => {
const manager = new DownloadManager();
manager.versions = [
{
...MULTI_FILE_VERSION,
existsLocally: true,
localPath: '/models/loras/file-a.safetensors',
},
];
manager.showVersionStep();
const badge = document.querySelector('.file-select-badge');
expect(badge).not.toBeNull();
expect(badge.dataset.versionId).toBe('201');
expect(badge.textContent).toContain('2');
// The in-library badge is still shown alongside
expect(document.querySelector('.local-badge')).not.toBeNull();
});
it('still shows the file-select badge for a multi-file version not in library', () => {
const manager = new DownloadManager();
manager.versions = [{ ...MULTI_FILE_VERSION, existsLocally: false }];
manager.showVersionStep();
const badge = document.querySelector('.file-select-badge');
expect(badge).not.toBeNull();
expect(badge.dataset.versionId).toBe('201');
});
it('does not show the file-select badge for a single-file version in library', () => {
const manager = new DownloadManager();
manager.versions = [
{
...SINGLE_FILE_VERSION,
existsLocally: true,
localPath: '/models/loras/file-c.safetensors',
},
];
manager.showVersionStep();
expect(document.querySelector('.file-select-badge')).toBeNull();
});
});
@@ -0,0 +1,332 @@
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
const {
DOWNLOAD_MANAGER_MODULE,
MODAL_MANAGER_MODULE,
UI_HELPERS_MODULE,
STATE_MODULE,
LOADING_MANAGER_MODULE,
API_FACTORY_MODULE,
STORAGE_HELPERS_MODULE,
FOLDER_TREE_MANAGER_MODULE,
I18N_HELPERS_MODULE,
SUMMARY_MODULE,
mockApiClient,
mockLoadingManager,
showToastMock,
showDownloadBatchSummaryMock,
resetAndReloadMock,
} = vi.hoisted(() => {
// Shared API client returned by the mocked getModelApiClient factory.
const mockApiClient = {
modelType: 'loras',
apiConfig: {
config: {
displayName: 'LoRA',
singularName: 'lora',
},
},
fetchCivitaiVersions: vi.fn(),
fetchModelRoots: vi.fn(async () => ({ roots: ['/models/loras'] })),
fetchUnifiedFolderTree: vi.fn(async () => ({ success: false })),
downloadModel: vi.fn(),
cancelDownload: vi.fn(),
getPageState: vi.fn(() => ({})),
};
const mockLoadingManager = {
showSimpleLoading: vi.fn(),
setStatus: vi.fn(),
hide: vi.fn(),
restoreProgressBar: vi.fn(),
showDownloadProgress: vi.fn(() => vi.fn()),
showCancelButton: vi.fn(),
};
return {
DOWNLOAD_MANAGER_MODULE: new URL('../../../static/js/managers/DownloadManager.js', import.meta.url).pathname,
MODAL_MANAGER_MODULE: new URL('../../../static/js/managers/ModalManager.js', import.meta.url).pathname,
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
STATE_MODULE: new URL('../../../static/js/state/index.js', import.meta.url).pathname,
LOADING_MANAGER_MODULE: new URL('../../../static/js/managers/LoadingManager.js', import.meta.url).pathname,
API_FACTORY_MODULE: new URL('../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
STORAGE_HELPERS_MODULE: new URL('../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
FOLDER_TREE_MANAGER_MODULE: new URL('../../../static/js/components/FolderTreeManager.js', import.meta.url).pathname,
I18N_HELPERS_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
SUMMARY_MODULE: new URL('../../../static/js/components/DownloadBatchSummaryModal.js', import.meta.url).pathname,
mockApiClient,
mockLoadingManager,
showToastMock: vi.fn(),
showDownloadBatchSummaryMock: vi.fn(),
resetAndReloadMock: vi.fn(),
};
});
vi.mock(MODAL_MANAGER_MODULE, () => ({
modalManager: {
showModal: vi.fn(),
closeModal: vi.fn(),
},
}));
vi.mock(UI_HELPERS_MODULE, () => ({
showToast: showToastMock,
setupAutoNewlineOnPaste: vi.fn(),
}));
vi.mock(STATE_MODULE, () => ({
state: {
global: {
settings: {},
},
loadingManager: mockLoadingManager,
},
}));
vi.mock(LOADING_MANAGER_MODULE, () => ({
LoadingManager: vi.fn(() => mockLoadingManager),
}));
vi.mock(API_FACTORY_MODULE, () => ({
getModelApiClient: vi.fn(() => mockApiClient),
resetAndReload: resetAndReloadMock,
}));
vi.mock(STORAGE_HELPERS_MODULE, () => ({
getStorageItem: vi.fn((_key, defaultValue) => defaultValue),
setStorageItem: vi.fn(),
}));
vi.mock(FOLDER_TREE_MANAGER_MODULE, () => ({
FolderTreeManager: vi.fn(() => ({
clearSelection: vi.fn(),
init: vi.fn(),
getSelectedPath: vi.fn(() => ''),
})),
}));
vi.mock(I18N_HELPERS_MODULE, () => ({
translate: vi.fn((_, __, fallback) => fallback ?? ''),
}));
vi.mock(SUMMARY_MODULE, () => ({
showDownloadBatchSummary: showDownloadBatchSummaryMock,
}));
/** DOM covering the file-selection, version and location steps. */
function setupDownloadDom() {
document.body.innerHTML = `
<div id="downloadModal">
<div class="download-step" id="urlStep"></div>
<div class="download-step" id="versionStep"></div>
<div class="download-step" id="fileSelectionStep"></div>
<div class="download-step" id="downloadLocationStep"></div>
<div id="fileSelectionList"></div>
<div id="fileSelectionVersionName"></div>
<button id="nextFromVersion"></button>
<div id="downloadModalTitle"></div>
<select id="modelRoot"></select>
<input id="folderPath" />
<div id="targetPathDisplay"></div>
<input id="useDefaultPath" type="checkbox" />
<div id="manualPathSelection"></div>
</div>
`;
}
function makeMultiFileVersion(overrides = {}) {
return {
id: 201,
name: 'Multi-file version',
baseModel: 'SDXL',
images: [],
files: [
{ id: 1001, type: 'Model', sizeKB: 2048, name: 'file-a.safetensors' },
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' },
],
createdAt: '2026-01-01T00:00:00Z',
existsLocally: true,
...overrides,
};
}
function getFileOption(fileId) {
return document.querySelector(`.file-option[data-file-id="${fileId}"]`);
}
describe('DownloadManager multi-select file dialog (#1058)', () => {
let DownloadManager;
beforeEach(async () => {
vi.resetModules();
vi.clearAllMocks();
setupDownloadDom();
({ DownloadManager } = await import(DOWNLOAD_MANAGER_MODULE));
});
afterEach(() => {
document.body.innerHTML = '';
});
it('renders downloaded files disabled with an In Library tag', () => {
const manager = new DownloadManager();
manager.versions = [makeMultiFileVersion({
downloadedFiles: [
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
],
})];
manager.showFileSelectionStep('201');
const downloadedOption = getFileOption('1001');
expect(downloadedOption.classList.contains('disabled')).toBe(true);
expect(downloadedOption.querySelector('input[type="checkbox"]').disabled).toBe(true);
expect(downloadedOption.querySelector('.file-tag.in-library').textContent).toBe('In Library');
// Remaining files stay selectable
const otherOption = getFileOption('1002');
expect(otherOption.classList.contains('disabled')).toBe(false);
expect(otherOption.querySelector('input[type="checkbox"]').disabled).toBe(false);
});
it('ignores clicks on already-downloaded options', () => {
const manager = new DownloadManager();
manager.versions = [makeMultiFileVersion({
downloadedFiles: [
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
],
})];
manager.showFileSelectionStep('201');
getFileOption('1001').click();
expect(getFileOption('1001').querySelector('input[type="checkbox"]').checked).toBe(false);
expect(manager.selectedFiles).toHaveLength(0);
});
it('confirmFileSelection collects multiple checked files into selectedFiles', () => {
const manager = new DownloadManager();
manager.apiClient = mockApiClient;
manager.versions = [makeMultiFileVersion({ downloadedFiles: [] })];
manager.showFileSelectionStep('201');
getFileOption('1001').click();
getFileOption('1003').click();
manager.confirmFileSelection();
expect(manager.selectedFiles.map(f => f.id)).toEqual([1001, 1003]);
// selectedFile stays the first selected file for single-file flows
expect(manager.selectedFile?.id).toBe(1001);
expect(document.getElementById('fileSelectionStep').style.display).toBe('none');
expect(document.getElementById('downloadLocationStep').style.display).toBe('block');
});
it('confirmFileSelection warns when nothing is selected', () => {
const manager = new DownloadManager();
manager.versions = [makeMultiFileVersion({ downloadedFiles: [] })];
manager.showFileSelectionStep('201');
manager.confirmFileSelection();
expect(showToastMock).toHaveBeenCalledWith('toast.loras.pleaseSelectFile', {}, 'error');
expect(manager.selectedFiles).toHaveLength(0);
expect(document.getElementById('downloadLocationStep').style.display).not.toBe('block');
});
it('disables the other routing group once a file is checked and re-enables when unchecked', () => {
const manager = new DownloadManager();
manager.versions = [makeMultiFileVersion({
downloadedFiles: [],
files: [
{ id: 1001, type: 'UNet', sizeKB: 2048, name: 'unet-a.safetensors' },
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' },
],
})];
manager.showFileSelectionStep('201');
// Checking a regular Model file disables the UNet option
getFileOption('1002').click();
expect(getFileOption('1001').classList.contains('group-disabled')).toBe(true);
expect(getFileOption('1001').querySelector('input[type="checkbox"]').disabled).toBe(true);
expect(getFileOption('1003').classList.contains('group-disabled')).toBe(false);
// Clicking a group-disabled option does nothing
getFileOption('1001').click();
expect(manager.selectedFiles.map(f => f.id)).toEqual([1002]);
// Unchecking everything re-enables the other group
getFileOption('1002').click();
expect(manager.selectedFiles).toHaveLength(0);
expect(getFileOption('1001').classList.contains('group-disabled')).toBe(false);
expect(getFileOption('1001').querySelector('input[type="checkbox"]').disabled).toBe(false);
});
it('keeps Next enabled for a partially downloaded multi-file version', () => {
const manager = new DownloadManager();
manager.currentVersion = makeMultiFileVersion({
downloadedFiles: [
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
],
});
manager.updateNextButtonState();
const nextButton = document.getElementById('nextFromVersion');
expect(nextButton.disabled).toBe(false);
expect(nextButton.classList.contains('disabled')).toBe(false);
});
it('disables Next when every weight file is already downloaded', () => {
const manager = new DownloadManager();
manager.currentVersion = makeMultiFileVersion({
downloadedFiles: [
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
{ fileId: 1002, fileName: 'file-b.safetensors', filePath: '/models/loras/file-b.safetensors' },
{ fileId: 1003, fileName: 'file-c.safetensors', filePath: '/models/loras/file-c.safetensors' },
],
});
manager.updateNextButtonState();
const nextButton = document.getElementById('nextFromVersion');
expect(nextButton.disabled).toBe(true);
expect(nextButton.classList.contains('disabled')).toBe(true);
});
it('disables Next for an in-library single-file version', () => {
const manager = new DownloadManager();
manager.currentVersion = {
id: 202,
name: 'Single-file version',
files: [{ id: 1004, type: 'Model', sizeKB: 2048, name: 'file-d.safetensors' }],
existsLocally: true,
downloadedFiles: [],
};
manager.updateNextButtonState();
const nextButton = document.getElementById('nextFromVersion');
expect(nextButton.disabled).toBe(true);
});
it('hides every other step (including the URL step) when the file dialog shows', () => {
// Regression: entering via openFileSelectionForVersion (ModelVersionsTab)
// left the URL step visible alongside the file selection step (#1058).
const manager = new DownloadManager();
manager.versions = [makeMultiFileVersion()];
document.getElementById('urlStep').style.display = 'block';
document.getElementById('versionStep').style.display = 'block';
manager.showFileSelectionStep('201');
expect(document.getElementById('urlStep').style.display).toBe('none');
expect(document.getElementById('versionStep').style.display).toBe('none');
expect(document.getElementById('fileSelectionStep').style.display).toBe('block');
});
});
@@ -0,0 +1,77 @@
"""Unit tests for per-file downloaded-state matching (#1058)."""
from py.routes.handlers.model_handlers import ModelCivitaiHandler
VERSION = {
"id": 42,
"files": [
{
"id": 1001,
"name": "file-a.safetensors",
"hashes": {"SHA256": "AAA111"},
},
{
"id": 1002,
"name": "file-b.safetensors",
"hashes": {"SHA256": "BBB222"},
},
],
}
def _entry(file_name: str, sha256: str = "", file_path: str | None = None):
return {
"file_name": file_name,
"file_path": file_path or f"/models/{file_name}.safetensors",
"sha256": sha256,
}
def test_matches_by_sha256():
entries = [_entry("renamed-locally", "bbb222")]
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
assert result == [
{
"fileId": 1002,
"fileName": "file-b.safetensors",
"filePath": "/models/renamed-locally.safetensors",
}
]
def test_falls_back_to_name_when_hash_missing():
entries = [_entry("file-a", "")]
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
assert [r["fileId"] for r in result] == [1001]
def test_hash_takes_precedence_over_name():
# Hash points at file-b while the name points at file-a: hash wins.
entries = [_entry("file-a", "bbb222")]
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
assert [r["fileId"] for r in result] == [1002]
def test_unmatched_entries_are_skipped():
entries = [
_entry("unrelated", "ccc333"),
_entry("file-b", ""), # name match
]
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
assert [r["fileId"] for r in result] == [1002]
def test_multiple_files_of_same_version():
entries = [
_entry("file-a", "aaa111"),
_entry("file-b", "bbb222"),
]
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
assert [r["fileId"] for r in result] == [1001, 1002]
def test_empty_inputs():
assert ModelCivitaiHandler._match_downloaded_files(VERSION, []) == []
assert ModelCivitaiHandler._match_downloaded_files({"id": 1}, [_entry("x")]) == []
assert ModelCivitaiHandler._match_downloaded_files(VERSION, None) == []
@@ -3,6 +3,40 @@ import pytest
from py.recipes.parsers.automatic import AutomaticMetadataParser
class LocalRecipeScanner:
class LoraScanner:
@staticmethod
def has_hash(model_hash):
return False
def __init__(self, models):
self.models = models
self.queries = []
self.hash_queries = []
self._lora_scanner = self.LoraScanner()
async def get_local_lora(self, name, base_model=None):
self.queries.append(name)
return self.models.get(name)
async def get_local_lora_by_hash(self, hash_value):
self.hash_queries.append(hash_value)
return next((model for model in self.models.values() if model.get("sha256") == hash_value), None)
def local_lora(file_name="local_only"):
return {
"file_path": f"/models/loras/styles/{file_name}.safetensors",
"file_name": file_name,
"model_name": "Local Only",
"sha256": "a" * 64,
"size": 123456,
"base_model": "Flux.1 D",
"preview_url": f"/models/loras/styles/{file_name}.preview.png",
"civitai": None,
}
@pytest.mark.asyncio
async def test_parse_metadata_extracts_checkpoint_from_civitai_resources(monkeypatch):
checkpoint_info = {
@@ -132,6 +166,218 @@ async def test_parse_metadata_merges_lora_hashes_over_empty_hashes_json(monkeypa
assert "UnusedLora" not in lora_names, "UnusedLora should have been skipped"
@pytest.mark.asyncio
async def test_parse_metadata_resolves_local_lora_with_empty_hash(monkeypatch):
async def fake_metadata_provider():
class Provider:
async def get_model_by_hash(self, model_hash):
raise AssertionError("Local and empty-hash LoRAs must not query Civitai")
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local_only": local_lora()})
metadata_text = (
"portrait <lora:local_only:0.65> <lora:missing:0.4>\n"
"Steps: 20, Sampler: Euler, CFG scale: 7, Seed: 1, "
'Hashes: {"lora:local_only": "", "lora:missing": ""}'
)
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert len(result["loras"]) == 1
entry = result["loras"][0]
assert entry["name"] == "Local Only"
assert entry["file_name"] == "local_only"
assert entry["weight"] == 0.65
assert entry["hash"] == "a" * 64
assert entry["localPath"].endswith("local_only.safetensors")
assert entry["size"] == 123456
assert entry["baseModel"] == "Flux.1 D"
assert entry["existsLocally"] is True
assert entry["isDeleted"] is False
assert scanner.queries == ["local_only", "missing"]
@pytest.mark.asyncio
async def test_parse_metadata_resolves_prompt_lora_without_hashes(monkeypatch):
async def fake_metadata_provider():
return None
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
model = local_lora()
scanner = LocalRecipeScanner({"styles/local_only": model})
metadata_text = "portrait <lora:styles/local_only:0.7>\nSteps: 20, Seed: 1"
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert len(result["loras"]) == 1
assert result["loras"][0]["weight"] == 0.7
assert result["loras"][0]["hash"] == "a" * 64
assert scanner.queries == ["styles/local_only"]
@pytest.mark.asyncio
async def test_parse_metadata_prefers_hash_over_colliding_local_name(monkeypatch):
remote_info = {
"id": 100,
"modelId": 200,
"model": {"name": "Hash Match", "type": "LORA"},
"name": "v1",
"files": [{"type": "Model", "primary": True, "name": "hash_match.safetensors", "hashes": {"SHA256": "b" * 64}}],
}
async def fake_metadata_provider():
class Provider:
async def get_model_by_hash(self, model_hash):
assert model_hash == "deadbeef00"
return remote_info, None
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local_only": local_lora()})
metadata_text = (
"portrait <lora:local_only:0.8>\n"
"Steps: 20, Seed: 1, "
'Hashes: {"lora:local_only": "deadbeef00"}'
)
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert len(result["loras"]) == 1
assert result["loras"][0]["id"] == 100
assert result["loras"][0]["weight"] == 0.8
assert scanner.queries == []
assert scanner.hash_queries == ["deadbeef00"]
@pytest.mark.asyncio
async def test_parse_metadata_falls_back_to_name_when_hash_is_unresolved(monkeypatch):
async def fake_metadata_provider():
class Provider:
async def get_model_by_hash(self, model_hash):
return None, "Model not found"
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local_only": local_lora()})
metadata_text = (
"portrait <lora:local_only:0.8>\nSteps: 20, Seed: 1, "
'Hashes: {"lora:local_only": "deadbeef00"}'
)
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert result["loras"][0]["hash"] == "a" * 64
assert scanner.queries == ["local_only"]
@pytest.mark.asyncio
async def test_parse_metadata_uses_prompt_weight_for_civitai_resource(monkeypatch):
remote_info = {
"id": 100,
"modelId": 200,
"model": {"name": "local_only", "type": "LORA"},
"name": "v1",
"files": [
{
"type": "Model",
"primary": True,
"name": "remote_file.safetensors",
"hashes": {"SHA256": "b" * 64},
}
],
}
async def fake_metadata_provider():
class Provider:
async def get_model_version_info(self, version_id):
assert version_id == 100
return remote_info, None
async def get_model_by_hash(self, model_hash):
raise AssertionError("The Civitai resource should not be fetched again by hash")
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local_only": local_lora()})
metadata_text = (
"portrait <lora:remote_file:0.35> <lora:local_only:0.6>\n"
"Steps: 20, Seed: 1, "
'Civitai resources: [{"type":"lora","modelVersionId":100,"modelName":"local_only"}]'
)
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert len(result["loras"]) == 2
assert [entry["file_name"] for entry in result["loras"]] == ["remote_file", "local_only"]
assert [entry["weight"] for entry in result["loras"]] == [0.35, 0.6]
assert result["loras"][1]["existsLocally"] is True
assert scanner.queries == ["local_only"]
@pytest.mark.asyncio
async def test_parse_metadata_keeps_mixed_local_and_civitai_loras(monkeypatch):
remote_info = {
"id": 100,
"modelId": 200,
"model": {"name": "Remote LoRA", "type": "LORA"},
"name": "v1",
"files": [
{
"type": "Model",
"primary": True,
"name": "remote.safetensors",
"hashes": {"SHA256": "b" * 64},
}
],
}
async def fake_metadata_provider():
class Provider:
async def get_model_by_hash(self, model_hash):
assert model_hash == "bbbbbbbbbb"
return remote_info, None
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.automatic.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local_only": local_lora()})
metadata_text = (
"portrait <lora:local_only:0.6> <lora:remote:0.9>\n"
"Steps: 20, Seed: 1, "
'Hashes: {"lora:local_only": "", "lora:remote": "bbbbbbbbbb"}'
)
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
assert [entry["name"] for entry in result["loras"]] == ["Local Only", "Remote LoRA"]
assert [entry["weight"] for entry in result["loras"]] == [0.6, 0.9]
assert result["loras"][0]["existsLocally"] is True
assert result["loras"][1]["existsLocally"] is False
@pytest.mark.asyncio
async def test_parse_metadata_extracts_checkpoint_from_model_hash(monkeypatch):
checkpoint_info = {
@@ -2,6 +2,38 @@ import pytest
import json
from py.recipes.parsers.comfy import ComfyMetadataParser
class LocalRecipeScanner:
class LoraScanner:
@staticmethod
def has_hash(model_hash):
return False
def __init__(self, models):
self.models = models
self.queries = []
self.base_models = []
self._lora_scanner = self.LoraScanner()
async def get_local_lora(self, name, base_model=None):
self.queries.append(name)
self.base_models.append(base_model)
return self.models.get(name)
def local_lora(file_name):
return {
"file_path": f"/models/loras/{file_name}.safetensors",
"file_name": file_name.rsplit("/", 1)[-1],
"model_name": file_name.rsplit("/", 1)[-1],
"sha256": file_name[0] * 64,
"size": 4096,
"base_model": "SDXL 1.0",
"preview_url": "",
"civitai": None,
}
@pytest.mark.asyncio
async def test_parse_metadata_without_loras(monkeypatch):
checkpoint_info = {
@@ -84,6 +116,140 @@ async def test_parse_metadata_without_loras(monkeypatch):
assert result["gen_params"]["size"] == "1024x1024"
assert result["from_comfy_metadata"] is True
@pytest.mark.asyncio
async def test_parse_metadata_resolves_standard_and_manager_local_loras(monkeypatch):
async def fake_metadata_provider():
class Provider:
async def get_model_version_info(self, version_id):
raise AssertionError("Local LoRAs must not query Civitai")
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.comfy.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({
"styles/standard.safetensors": local_lora("standard"),
"manager": local_lora("manager"),
})
metadata_json = {
"1": {
"class_type": "LoraLoader",
"inputs": {
"lora_name": "styles/standard.safetensors",
"strength_model": 0.55,
},
},
"2": {
"class_type": "LoraLoaderLM",
"inputs": {
"loras": {
"__value__": [
{"name": "manager", "strength": "0.80", "active": True},
{"name": "disabled", "strength": 1.0, "active": False},
{"name": "dummy", "strength": 1.0, "active": True, "_isDummy": True},
]
}
},
},
}
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
assert [entry["file_name"] for entry in result["loras"]] == ["standard", "manager"]
assert [entry["weight"] for entry in result["loras"]] == [0.55, 0.8]
assert all(isinstance(entry["weight"], float) for entry in result["loras"])
assert all(entry["existsLocally"] is True for entry in result["loras"])
assert all(entry["isDeleted"] is False for entry in result["loras"])
assert scanner.queries == ["styles/standard.safetensors", "manager"]
@pytest.mark.asyncio
async def test_parse_metadata_defaults_malformed_weight_and_passes_checkpoint_base_model(monkeypatch):
checkpoint_info = {
"id": 456,
"modelId": 123,
"model": {"name": "Checkpoint", "type": "checkpoint"},
"name": "v1",
"baseModel": "SDXL 1.0",
}
async def fake_metadata_provider():
class Provider:
async def get_model_version_info(self, version_id):
return checkpoint_info, None
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.comfy.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"style": local_lora("style")})
metadata_json = {
"1": {"class_type": "LoraLoader", "inputs": {"lora_name": "style", "strength_model": "invalid"}},
"2": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "civitai:123@456"}},
}
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
assert result["loras"][0]["weight"] == 1.0
assert scanner.base_models == ["SDXL 1.0"]
@pytest.mark.asyncio
async def test_parse_metadata_keeps_civitai_urn_with_local_lora(monkeypatch):
remote_info = {
"id": 456,
"modelId": 123,
"model": {"name": "Remote LoRA", "type": "LORA"},
"name": "v1",
"files": [
{
"type": "Model",
"primary": True,
"name": "remote.safetensors",
"hashes": {"SHA256": "c" * 64},
}
],
}
async def fake_metadata_provider():
class Provider:
async def get_model_version_info(self, version_id):
assert version_id == "456"
return remote_info, None
return Provider()
monkeypatch.setattr(
"py.recipes.parsers.comfy.get_default_metadata_provider",
fake_metadata_provider,
)
scanner = LocalRecipeScanner({"local": local_lora("local")})
metadata_json = {
"1": {
"class_type": "LoraLoader",
"inputs": {"lora_name": "local", "strength_model": 0.4},
},
"2": {
"class_type": "LoraLoader",
"inputs": {
"lora_name": "urn:air:sdxl:lora:civitai:123@456",
"strength_model": 0.9,
},
},
}
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
assert [entry["name"] for entry in result["loras"]] == ["local", "Remote LoRA"]
assert [entry["weight"] for entry in result["loras"]] == [0.4, 0.9]
assert result["loras"][0]["existsLocally"] is True
assert result["loras"][1]["id"] == 456
@pytest.mark.asyncio
async def test_parse_metadata_without_extra_metadata(monkeypatch):
async def fake_metadata_provider():
+312 -1
View File
@@ -82,14 +82,18 @@ def stub_metadata(monkeypatch):
class DummyScanner:
def __init__(self, exists: bool = False):
def __init__(self, exists: bool = False, raw_data=None):
self.exists = exists
self.calls = []
self._cache = SimpleNamespace(raw_data=list(raw_data or []))
async def check_model_version_exists(self, version_id):
self.calls.append(version_id)
return self.exists
async def get_cached_data(self):
return self._cache
@pytest.fixture
def scanners(monkeypatch):
@@ -1692,3 +1696,310 @@ async def test_download_proceeds_when_history_skip_disabled(monkeypatch, scanner
assert result.get("skipped") is not True
history_service.has_been_downloaded.assert_not_called()
execute_download.assert_awaited_once()
# ---------------------------------------------------------------------------
# Multi-file downloads within a single model version (#1058)
# ---------------------------------------------------------------------------
def _multi_file_payload(include_hashes: bool = True):
"""Version payload with two weight files under the same version."""
files = [
{
"id": 1001,
"type": "Model",
"primary": True,
"downloadUrl": "https://example.invalid/file-a.safetensors",
"name": "file-a.safetensors",
},
{
"id": 1002,
"type": "Model",
"primary": False,
"downloadUrl": "https://example.invalid/file-b.safetensors",
"name": "file-b.safetensors",
},
]
if include_hashes:
files[0]["hashes"] = {"SHA256": "AAA111"}
files[1]["hashes"] = {"SHA256": "BBB222"}
return {
"id": 42,
"modelId": 7,
"model": {"type": "LoRA", "tags": ["fantasy"]},
"baseModel": "BaseModel",
"creator": {"username": "Author"},
"files": files,
}
def _local_entry(file_name: str, sha256: str, version_id: int = 42):
"""A library cache entry for one already-downloaded file of a version."""
return {
"file_name": file_name,
"file_path": f"/tmp/{file_name}.safetensors",
"sha256": sha256,
"civitai": {"id": version_id, "modelId": 7},
}
def _stub_history_service(monkeypatch, has_been_downloaded: bool = False):
history_service = AsyncMock()
history_service.has_been_downloaded = AsyncMock(
return_value=has_been_downloaded
)
history_service.mark_downloaded = AsyncMock()
monkeypatch.setattr(
ServiceRegistry,
"get_downloaded_version_history_service",
AsyncMock(return_value=history_service),
)
return history_service
@pytest.mark.asyncio
async def test_download_allows_other_file_of_in_library_version(
monkeypatch, scanners, metadata_provider
):
"""A different file of an in-library version must still download (#1058)."""
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 1002, "type": "Model", "name": "file-b.safetensors"},
)
assert result["success"] is True
execute_download.assert_awaited_once()
# Version-level gates must not run for an explicit file selection
assert scanners.lora.calls == []
@pytest.mark.asyncio
async def test_download_blocks_same_file_of_in_library_version(
monkeypatch, scanners, metadata_provider
):
"""Re-downloading the SAME file of an in-library version stays blocked."""
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 1001, "type": "Model", "name": "file-a.safetensors"},
)
assert result["success"] is False
assert "file-a.safetensors" in result["error"]
assert "already exists in lora library" in result["error"]
execute_download.assert_not_called()
@pytest.mark.asyncio
async def test_download_unresolvable_file_params_falls_back_to_version_gate(
monkeypatch, scanners, metadata_provider
):
"""file_params that match no file must not bypass version-level gates."""
scanners.lora.exists = True
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 9999, "type": "Model"},
)
assert result["success"] is False
assert result["error"] == "Model version already exists in lora library"
execute_download.assert_not_called()
@pytest.mark.asyncio
async def test_download_empty_file_params_treated_as_no_selection(
monkeypatch, scanners, metadata_provider
):
"""An empty file_params dict is normalized to None (version-level gates)."""
scanners.lora.exists = True
execute_download = AsyncMock(return_value={"success": True})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={},
)
assert result["success"] is False
assert result["error"] == "Model version already exists in lora library"
# Early version-level gate fired (file_params normalized to None)
assert scanners.lora.calls == [42]
execute_download.assert_not_called()
@pytest.mark.asyncio
async def test_download_explicit_file_bypasses_history_skip(
monkeypatch, scanners, metadata_provider
):
"""Explicit file selection bypasses the previously-downloaded skip."""
get_settings_manager().settings[
"skip_previously_downloaded_model_versions"
] = True
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
history_service = _stub_history_service(monkeypatch, has_been_downloaded=True)
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 1002, "type": "Model", "name": "file-b.safetensors"},
)
assert result["success"] is True
execute_download.assert_awaited_once()
# History gate is bypassed before it even queries the service
history_service.has_been_downloaded.assert_not_called()
@pytest.mark.asyncio
async def test_download_file_match_by_name_when_local_hash_missing(
monkeypatch, scanners, metadata_provider
):
"""Legacy local metadata without sha256 falls back to name matching."""
scanners.lora._cache.raw_data.append(_local_entry("file-a", ""))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 1001, "type": "Model"},
)
assert result["success"] is False
assert "already exists in lora library" in result["error"]
execute_download.assert_not_called()
@pytest.mark.asyncio
async def test_download_no_false_positive_when_both_hashes_empty(
monkeypatch, scanners, metadata_provider
):
"""Two empty hashes must never compare equal; name decides instead."""
# Local entry for a DIFFERENT file of the same version, no hash stored
scanners.lora._cache.raw_data.append(_local_entry("file-b", ""))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload(include_hashes=False)
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_version_id=42,
save_dir="/tmp",
file_params={"id": 1001, "type": "Model"},
)
assert result["success"] is True
execute_download.assert_awaited_once()
@pytest.mark.asyncio
async def test_download_model_id_only_with_file_params_resolves_file(
monkeypatch, scanners, metadata_provider
):
"""model_id-only requests with file_params resolve the file post-fetch."""
scanners.lora.exists = True # version-level index says "in library"
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
metadata_provider.get_model_version = AsyncMock(
return_value=_multi_file_payload()
)
_stub_history_service(monkeypatch)
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
monkeypatch.setattr(
DownloadManager, "_execute_download", execute_download, raising=False
)
manager = DownloadManager()
result = await manager.download_from_civitai(
model_id=7,
save_dir="/tmp",
file_params={"id": 1002, "type": "Model"},
)
assert result["success"] is True
execute_download.assert_awaited_once()
def test_resolve_target_file_by_id():
files = _multi_file_payload()["files"]
resolved = DownloadManager._resolve_target_file(files, {"id": 1002})
assert resolved is files[1]
def test_resolve_target_file_by_primary_flag():
files = _multi_file_payload()["files"]
resolved = DownloadManager._resolve_target_file(files, {"isPrimary": True})
assert resolved is files[0]
def test_resolve_target_file_returns_none_for_no_match():
files = _multi_file_payload()["files"]
assert DownloadManager._resolve_target_file(files, {"id": 9999}) is None
assert DownloadManager._resolve_target_file(files, None) is None
assert DownloadManager._resolve_target_file(files, {}) is None
@@ -5,6 +5,8 @@ Covers the new ``download_id``-based code paths in
compatibility with ``id``.
"""
import json
import sqlite3
from pathlib import Path
import pytest
@@ -191,3 +193,131 @@ async def test_retry_unknown_download_id(tmp_path: Path) -> None:
item = await svc.retry_from_history(download_id="dl-nope")
assert item is None
# ---------------------------------------------------------------------------
# file_params persistence across queue -> history -> retry (#1058)
# ---------------------------------------------------------------------------
@pytest.mark.asyncio
async def test_file_params_survive_queue_to_history(tmp_path: Path) -> None:
"""complete_download copies the queue row's file_params into history."""
svc = _make_service(tmp_path)
await svc.add_to_queue(
download_id="dl-fp",
model_id=1,
model_version_id=100,
file_params={"id": 1002, "type": "Model"},
)
await svc.complete_download("dl-fp", status="failed", error="boom")
history = await svc.get_history()
assert len(history["items"]) == 1
assert json.loads(history["items"][0]["file_params"]) == {
"id": 1002,
"type": "Model",
}
@pytest.mark.asyncio
async def test_retry_restores_file_params(tmp_path: Path) -> None:
"""retry_from_history re-queues with the originally selected file."""
svc = _make_service(tmp_path)
await svc.add_to_history(
download_id="dl-fail-fp",
model_id=1,
model_version_id=100,
status="failed",
file_params={"id": 1002, "type": "Model"},
)
item = await svc.retry_from_history(download_id="dl-fail-fp")
assert item is not None
assert item["status"] == "queued"
assert json.loads(item["file_params"]) == {"id": 1002, "type": "Model"}
@pytest.mark.asyncio
async def test_retry_all_restores_file_params(tmp_path: Path) -> None:
"""retry_all_failed preserves file_params for every re-queued item."""
svc = _make_service(tmp_path)
await svc.add_to_history(
download_id="dl-f1", status="failed", file_params={"id": 1001}
)
await svc.add_to_history(download_id="dl-f2", status="canceled")
count = await svc.retry_all_failed()
assert count == 2
queue = await svc.get_queue()
restored = sorted(
(q["file_params"] or "") for q in queue
)
assert restored[0] == "" # dl-f2 never had file_params
assert json.loads(restored[1]) == {"id": 1001}
@pytest.mark.asyncio
async def test_legacy_history_db_gains_file_params_column(tmp_path: Path) -> None:
"""Databases created before the file_params column get migrated (#1058)."""
db_path = tmp_path / "queue.sqlite"
conn = sqlite3.connect(db_path)
conn.executescript(
"""
CREATE TABLE download_queue (
download_id TEXT PRIMARY KEY,
model_id INTEGER,
model_version_id INTEGER,
model_name TEXT NOT NULL DEFAULT '',
version_name TEXT DEFAULT '',
thumbnail_url TEXT DEFAULT '',
source TEXT,
file_params TEXT,
status TEXT NOT NULL DEFAULT 'queued',
priority INTEGER DEFAULT 0,
progress INTEGER DEFAULT 0,
bytes_downloaded INTEGER DEFAULT 0,
total_bytes INTEGER,
bytes_per_second REAL DEFAULT 0.0,
error TEXT,
file_path TEXT,
added_at REAL NOT NULL,
started_at REAL,
completed_at REAL
);
CREATE TABLE download_history (
id INTEGER PRIMARY KEY AUTOINCREMENT,
download_id TEXT,
model_id INTEGER,
model_version_id INTEGER,
model_name TEXT NOT NULL DEFAULT '',
version_name TEXT DEFAULT '',
thumbnail_url TEXT DEFAULT '',
status TEXT NOT NULL,
error TEXT,
file_path TEXT,
bytes_downloaded INTEGER DEFAULT 0,
total_bytes INTEGER,
completed_at REAL NOT NULL,
is_already_exists INTEGER DEFAULT 0
);
"""
)
conn.close()
svc = DownloadQueueService(db_path=str(db_path))
# The migrated table accepts file_params writes
await svc.add_to_history(
download_id="dl-legacy-fp", status="failed", file_params={"id": 5}
)
history = await svc.get_history()
assert json.loads(history["items"][0]["file_params"]) == {"id": 5}
# And retry restores them
item = await svc.retry_from_history(download_id="dl-legacy-fp")
assert item is not None
assert json.loads(item["file_params"]) == {"id": 5}
@@ -68,3 +68,94 @@ async def test_download_history_bulk_lookup(tmp_path: Path) -> None:
5: {501, 502},
6: {601},
}
@pytest.mark.asyncio
async def test_per_file_history_tracking(tmp_path: Path) -> None:
"""Per-file records coexist with the version-level row (#1058)."""
db_path = tmp_path / "download-history.sqlite"
service = DownloadedVersionHistoryService(
str(db_path),
settings_manager=DummySettings(),
)
await service.mark_downloaded(
"lora", 101, model_id=11, source="download",
file_path="/models/a.safetensors", file_id=1001, file_name="a.safetensors",
)
await service.mark_downloaded(
"lora", 101, model_id=11, source="download",
file_path="/models/b.safetensors", file_id=1002, file_name="b.safetensors",
)
assert await service.get_downloaded_file_ids("lora", 101) == [1001, 1002]
# Version-level tracking remains single-row per version
assert await service.get_downloaded_version_ids("lora", 11) == [101]
# Re-downloading the same file updates in place, no duplicate
await service.mark_downloaded(
"lora", 101, source="download", file_id=1001, file_name="a.safetensors",
)
assert await service.get_downloaded_file_ids("lora", 101) == [1001, 1002]
# Single-file deletion keeps the sibling record
await service.mark_file_deleted("lora", 101, 1001)
assert await service.get_downloaded_file_ids("lora", 101) == [1002]
# Whole-version deletion clears per-file records
await service.mark_as_deleted("lora", 101)
assert await service.get_downloaded_file_ids("lora", 101) == []
assert await service.has_been_downloaded("lora", 101) is False
@pytest.mark.asyncio
async def test_per_file_history_ignores_invalid_ids(tmp_path: Path) -> None:
service = DownloadedVersionHistoryService(
str(tmp_path / "download-history.sqlite"),
settings_manager=DummySettings(),
)
# mark_downloaded without a file id only touches the version-level table
await service.mark_downloaded("lora", 201, model_id=21, source="scan")
assert await service.get_downloaded_file_ids("lora", 201) == []
# Invalid inputs are no-ops
await service.mark_file_deleted("lora", 201, None) # type: ignore[arg-type]
assert await service.get_downloaded_file_ids("unknown-type", 201) == []
@pytest.mark.asyncio
async def test_file_history_table_created_for_legacy_db(tmp_path: Path) -> None:
"""Existing databases gain the per-file table via CREATE IF NOT EXISTS."""
import sqlite3
db_path = tmp_path / "download-history.sqlite"
conn = sqlite3.connect(db_path)
conn.executescript(
"""
CREATE TABLE downloaded_model_versions (
model_type TEXT NOT NULL,
version_id INTEGER NOT NULL,
model_id INTEGER,
first_seen_at REAL NOT NULL,
last_seen_at REAL NOT NULL,
source TEXT NOT NULL,
last_file_path TEXT,
last_library_name TEXT,
is_deleted_override INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (model_type, version_id)
);
"""
)
conn.close()
service = DownloadedVersionHistoryService(
str(db_path),
settings_manager=DummySettings(),
)
await service.mark_downloaded(
"lora", 301, model_id=31, source="download",
file_id=9001, file_name="file.safetensors",
)
assert await service.get_downloaded_file_ids("lora", 301) == [9001]
assert await service.has_been_downloaded("lora", 301) is True
+84
View File
@@ -61,3 +61,87 @@ async def test_model_cache_tracks_versions_by_model_id():
assert cache.get_versions_by_model_id(2) == [
{'versionId': 201, 'name': 'Gamma', 'fileName': 'model-b'},
]
@pytest.mark.asyncio
async def test_version_files_index_tracks_multiple_files_per_version():
"""Two downloaded files of the same version both stay indexed (#1058)."""
item_a = {
'file_path': '/models/v1-a.safetensors',
'file_name': 'model-v1-a',
'folder': '',
'civitai': {'id': 301, 'modelId': 3, 'name': 'Multi'},
}
item_b = {
'file_path': '/models/v1-b.safetensors',
'file_name': 'model-v1-b',
'folder': '',
'civitai': {'id': 301, 'modelId': 3, 'name': 'Multi'},
}
cache = ModelCache(
raw_data=[item_a, item_b],
folders=[],
name_display_mode='model_name',
)
files = cache.get_files_by_version_id(301)
assert {f['file_path'] for f in files} == {
'/models/v1-a.safetensors',
'/models/v1-b.safetensors',
}
# Re-adding an existing entry must not duplicate it
cache.add_to_version_index(item_a)
assert len(cache.get_files_by_version_id(301)) == 2
# Removing the indexed file re-points version_index to the sibling
indexed = cache.version_index[301]
sibling = item_b if indexed is item_a else item_a
cache.remove_from_version_index(indexed)
assert 301 in cache.version_index
assert cache.version_index[301]['file_path'] == sibling['file_path']
assert cache.get_versions_by_model_id(3) == [
{'versionId': 301, 'name': 'Multi', 'fileName': sibling['file_name']},
]
remaining = cache.get_files_by_version_id(301)
assert [f['file_path'] for f in remaining] == [sibling['file_path']]
# Removing the last file drops the version from all indexes
cache.remove_from_version_index(sibling)
assert 301 not in cache.version_index
assert cache.get_files_by_version_id(301) == []
assert cache.get_versions_by_model_id(3) == []
assert 3 not in cache.model_id_index
@pytest.mark.asyncio
async def test_version_files_index_rebuild_from_raw_data():
"""rebuild_version_index reconstructs the multi-valued index (#1058)."""
item_a = {
'file_path': '/models/v1-a.safetensors',
'file_name': 'model-v1-a',
'folder': '',
'civitai': {'id': 401, 'modelId': 4, 'name': 'Multi'},
}
item_b = {
'file_path': '/models/v1-b.safetensors',
'file_name': 'model-v1-b',
'folder': '',
'civitai': {'id': 401, 'modelId': 4, 'name': 'Multi'},
}
cache = ModelCache(
raw_data=[item_a, item_b],
folders=[],
name_display_mode='model_name',
)
cache.version_files_index = {}
cache.rebuild_version_index()
assert len(cache.get_files_by_version_id(401)) == 2
# Invalid ids normalize to empty results
assert cache.get_files_by_version_id('not-an-int') == []
assert cache.get_files_by_version_id(None) == []
+58 -1
View File
@@ -57,9 +57,33 @@ class StubLoraScanner:
meta = self._hash_meta.get(hash_value.lower())
return meta.get("path") if meta else None
async def get_model_info_by_name(self, name: str):
async def get_model_info_by_name(
self,
name: str,
*,
require_unique: bool = False,
base_model: str | None = None,
):
if require_unique or base_model:
matches = ModelScanner.find_matching_models(
self._cache.raw_data,
name,
base_model=base_model,
extensions={".safetensors"},
)
if require_unique and len(matches) != 1:
return None
return matches[0] if matches else None
return self._models_by_name.get(name)
async def find_models_by_name(self, name: str, *, base_model: str | None = None):
return ModelScanner.find_matching_models(
self._cache.raw_data,
name,
base_model=base_model,
extensions={".safetensors"},
)
def register_model(self, name: str, info: Dict[str, Any]) -> None:
self._models_by_name[name] = info
hash_value = (info.get("sha256") or "").lower()
@@ -107,6 +131,39 @@ def recipe_scanner(tmp_path: Path, monkeypatch):
settings_manager_module.reset_settings_manager()
@pytest.mark.asyncio
async def test_local_lora_lookup_requires_unambiguous_name_and_matching_base_model(recipe_scanner):
scanner, stub = recipe_scanner
models = [
{
"file_name": "style.safetensors",
"folder": "sd15",
"file_path": "/models/loras/sd15/style.safetensors",
"sha256": "a" * 64,
"base_model": "SD 1.5",
},
{
"file_name": "style.safetensors",
"folder": "sdxl",
"file_path": "/models/loras/sdxl/style.safetensors",
"sha256": "b" * 64,
"base_model": "SDXL 1.0",
},
]
stub._cache.raw_data = models
stub._hash_meta["b" * 64] = {"path": models[1]["file_path"]}
assert await scanner.get_local_lora("style") is None
assert await scanner.get_local_lora("style", "SDXL 1.0") is models[1]
assert await scanner.get_local_lora("sdxl/style.safetensors", "SDXL 1.0") is models[1]
# The lora scanner only indexes .safetensors, so a .pt name must not be
# stripped into a cross-extension match.
assert await scanner.get_local_lora("sdxl/style.pt", "SDXL 1.0") is None
assert await scanner.get_local_lora("sdxl/style.safetensors", "SD 1.5") is None
assert await scanner.get_local_lora("other/style.safetensors") is None
assert await scanner.get_local_lora_by_hash("b" * 64) is models[1]
def test_recipes_dir_uses_custom_settings_path(tmp_path: Path, monkeypatch):
RecipeScanner._instance = None
settings_manager_module.reset_settings_manager()
+70
View File
@@ -17,6 +17,7 @@ from py.services.recipes.errors import (
RecipeValidationError,
)
from py.services.recipes.persistence_service import RecipePersistenceService
from py.services.model_scanner import ModelScanner
from py.recipes.parsers.civitai_image import CivitaiApiMetadataParser
from py.utils.exif_utils import ExifUtils
@@ -1156,3 +1157,72 @@ async def test_analyze_local_image_fingerprint_uses_sha256_normalized_hash(tmp_p
assert result.payload["loras"][0]["hash"] == sha256
assert result.payload["fingerprint"] == f"{sha256}:1.0"
@pytest.mark.asyncio
async def test_reconnect_lora_distinguishes_ambiguous_mismatched_and_missing(tmp_path):
service = RecipePersistenceService(
exif_utils=DummyExifUtils(),
card_preview_width=512,
logger=logging.getLogger("test"),
)
models = [
{
"file_name": "style.safetensors",
"folder": "sd15",
"file_path": "/models/loras/sd15/style.safetensors",
"base_model": "SD 1.5",
},
{
"file_name": "style.safetensors",
"folder": "sdxl",
"file_path": "/models/loras/sdxl/style.safetensors",
"base_model": "SDXL 1.0",
},
]
class DummyScanner:
def __init__(self, recipe_path):
self._recipe_path = recipe_path
async def get_recipe_json_path(self, recipe_id):
return str(self._recipe_path)
async def get_local_lora(self, name, base_model=None):
matches = ModelScanner.find_matching_models(models, name, base_model=base_model)
return matches[0] if len(matches) == 1 else None
async def find_local_loras_by_name(self, name, base_model=None):
return ModelScanner.find_matching_models(models, name, base_model=base_model)
def write_recipe(base_model):
recipe_path = tmp_path / "recipe.json"
recipe_path.write_text(
json.dumps({"id": "r1", "base_model": base_model, "loras": []})
)
return DummyScanner(recipe_path)
# Ambiguous bare name: two candidates survive (recipe base model unknown)
scanner = write_recipe("")
with pytest.raises(RecipeValidationError, match="include the folder path"):
await service.reconnect_lora(
recipe_scanner=scanner, recipe_id="r1", lora_index=0, target_name="style"
)
# Confident base-model mismatch: the only candidate belongs to another family
scanner = write_recipe("SD 1.5")
with pytest.raises(RecipeValidationError, match="different base model"):
await service.reconnect_lora(
recipe_scanner=scanner,
recipe_id="r1",
lora_index=0,
target_name="sdxl/style",
)
# No candidate at all
scanner = write_recipe("SDXL 1.0")
with pytest.raises(RecipeNotFoundError, match="not found"):
await service.reconnect_lora(
recipe_scanner=scanner, recipe_id="r1", lora_index=0, target_name="missing"
)