mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-09-21 11:11:26 -03:00
Compare commits
9 Commits
0a28500848
...
e57e11897e
| Author | SHA1 | Date | |
|---|---|---|---|
| e57e11897e | |||
| 8a16034135 | |||
| 7fc3b7e5be | |||
| b0c7a1baae | |||
| 6411d83d46 | |||
| 74a063b0e5 | |||
| 96376e5cce | |||
| e7c26bf722 | |||
| cef4129fc9 |
@@ -350,4 +350,4 @@ Join our Discord community for support, discussions, and updates:
|
||||
---
|
||||
## Star History
|
||||
|
||||
[](https://star-history.com/#willmiao/ComfyUI-Lora-Manager&Date)
|
||||
[](https://www.star-history.com/?repos=willmiao%2FComfyUI-Lora-Manager&type=date&legend=top-left)
|
||||
|
||||
@@ -0,0 +1,206 @@
|
||||
# Plan: Multi-File Downloads Within a Single CivitAI Model Version
|
||||
|
||||
**Issue:** [#1058 — Cannot download multiple file variants from the same model version](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1058)
|
||||
**Status:** v2 — revised after adversarial review (backend correctness + frontend/tests)
|
||||
**Scope:** CivitAI/CivArchive downloads of `lora`, `checkpoint`, `embedding` model types. HuggingFace downloads are out of scope (already per-file).
|
||||
|
||||
> v2 changelog: incorporated 18 review findings. Key changes vs v1:
|
||||
> shared file resolver + `resolved_version_id` for the gate (R1); `file_params` normalization at API boundary (R2); D2 hash-matching rule fixed for empty-hash cases (R6/R7); D3 extended to re-point `version_index` on removal (R4); D4 replaced with a child table (R3); `delete_model_version` interaction documented (R5); `ModelVersionsTab` surface added to phase 2 (F6); phase-2 multi-file loop requires a reload-deferred download variant (F7); queue-retry `file_params=NULL` known issue recorded (R9); test-fixture gaps and revised estimates (F10).
|
||||
|
||||
---
|
||||
|
||||
## 1. Problem Statement
|
||||
|
||||
A CivitAI model version can contain multiple downloadable weight files (e.g. fp16/fp32, safetensors/ckpt, different sizes). LoRA Manager already has a working file-selection pipeline (frontend file dialog → `fileParams` → backend file matching), but downloaded state is tracked at the **model-version** level. After any single file of a version is downloaded:
|
||||
|
||||
1. The version is marked **In Library** and the file-selection entry point disappears.
|
||||
2. The backend rejects further download attempts for that version.
|
||||
|
||||
There is no way to download the remaining files of the same version through LoRA Manager.
|
||||
|
||||
## 2. Current State (verified against code; all references confirmed by review)
|
||||
|
||||
### 2.1 Download gating — backend (`py/services/download_manager.py`)
|
||||
|
||||
`_execute_original_download` enforces two version-level gates:
|
||||
|
||||
- **Library gate, early** (lines 1157–1184, before metadata fetch, fires when `model_version_id` given) and **late** (lines 1350–1376, fires only when `model_version_id is None`): `scanner.check_model_version_exists(version_id)` across lora/checkpoint/embedding scanners → hard error `"Model version already exists in ... library"`.
|
||||
- **History gate** (lines 1238–1279): when `skip_previously_downloaded_model_versions` setting is on, `_has_been_downloaded(model_type, version_id)` → silent skip. History DB primary key is `(model_type, version_id)` (`py/services/downloaded_version_history_service.py:61`).
|
||||
|
||||
File selection works: `file_params {id, type, format, size, fp}` is matched against `version_info.files` (lines 1498–1569), **but only under `if file_params and model_version_id:` (line 1499)** — with `model_id`-only requests the selection silently falls back to the primary file (1571–1619). `file_params` currently carries no file `name` or hash.
|
||||
|
||||
### 2.2 Downloaded-state surfacing — backend (`py/routes/handlers/model_handlers.py`)
|
||||
|
||||
`get_civitai_versions` (lines 2148–2188) sets per-version `existsLocally` via `cache.version_index.get(version_id)` (plus a single `localPath` from that entry) and `hasBeenDownloaded` via the history service. No per-file granularity.
|
||||
|
||||
### 2.3 Frontend blockers (`static/js/managers/DownloadManager.js`)
|
||||
|
||||
Three independent gates prevent re-entering the file dialog:
|
||||
|
||||
1. **Line 598:** file-select badge rendered only when `modelFiles.length > 1 && !existsLocally`.
|
||||
2. **Lines 666–681 (`updateNextButtonState`):** Next button disabled with "Already in Library" when `currentVersion.existsLocally`.
|
||||
3. **Lines 784–787 (`proceedToLocation`):** toast + abort when `currentVersion.existsLocally`.
|
||||
|
||||
The badge path (`confirmFileSelection` lines 737–759 → `proceedToLocationContent` → `startDownload` single mode → `executeDownloadWithProgress` → POST `file_params`, `static/js/api/baseModelApi.js:1236–1250`) has **zero** `existsLocally` guards (all 12 occurrences enumerated; none on this path; `import/DownloadManager.js` has none either). The `.exists-locally` CSS class is purely visual (`download-modal.css:496–499`). **Making the badge visible again is sufficient to unlock the flow** for phase 1.
|
||||
|
||||
Post-download refresh is clean: the modal closes and `resetAndReload(true)` performs a full library refetch (`DownloadManager.js:1063`); dialog reopen resets state and refetches versions with no client-side cache. No same-session staleness.
|
||||
|
||||
### 2.4 Local identity of the downloaded file
|
||||
|
||||
`LoraMetadata/CheckpointMetadata/EmbeddingMetadata.from_civitai_info(version_info, file_info, ...)` (`py/utils/models.py:245–369`) persists:
|
||||
|
||||
- `sha256` = `file_info.hashes.SHA256` (lowercased, defaults to `""`) — a stable per-file identity;
|
||||
- `civitai` = the full `version_info` payload (including the `files` list).
|
||||
|
||||
Metadata refresh (`metadata_sync_service.py:104–105`) replaces the `civitai` blob wholesale but never overwrites top-level `sha256`; `verify_duplicate_hashes` (481–526) corrects it to the on-disk hash. Top-level-sha256 matching is refresh-robust.
|
||||
|
||||
**Caveats (review R6/R7):**
|
||||
- SHA256 is not guaranteed: CivArchive's transform only sets `hashes` when source data carries it (`civarchive_client.py:185–189`); `from_civitai_info` defaults to `""`.
|
||||
- Name fallback is unreliable exactly when it matters: local `file_name` is extension-less (`models.py:264`) and `generate_unique_filename` rewrites it with a hash suffix on conflict (`download_manager.py:1125–1136`); checkpoints with `hash_status='pending'` keep empty sha256 until on-demand hashing (`model_scanner.py:1232–1240`).
|
||||
|
||||
### 2.5 Version index collision (pre-existing hazard)
|
||||
|
||||
`ModelCache.version_index` is single-valued (`model_cache.py:133`: `version_index[version_id] = item`). Two files of the same version in the library → second entry overwrites the first; `remove_from_version_index` (lines 151–181) drops the whole version key when the indexed entry is removed, even if a sibling file remains. ~10 read sites depend on this index (48 grep touch points total; readers include `recipe_scanner.py:2682–2726`, `recipe_format.py:37–40`, `misc_handlers.py:2440–2444`, `model_handlers.py`, `model_scanner.check_model_version_exists:2444`).
|
||||
|
||||
Review correction (F3): bulk paths `remove_models` (`model_scanner.py:2376`) and `update_single_model_cache` (`:1689`) call `rebuild_version_index()` right after, so a sibling re-enters the index in those flows — the hazard is narrower than v1 stated, but direct `remove_from_version_index` callers (e.g. `model_scanner.py:1018`) still drop the key, and the user-visible artifact in phase 1 is real: `localPath` in the dialog flips to whichever file was indexed last.
|
||||
|
||||
### 2.6 Entry points that send / don't send `file_params` (fully enumerated by review)
|
||||
|
||||
**Send `file_params` (user-initiated dialog flows only):** `DownloadManager.js:1611–1639` (single mode). API surface accepting arbitrary JSON `file_params`: GET `/api/lm/download-model-get` (`model_handlers.py:1634–1686`), POST `/api/lm/downloads/queue/add` (`model_handlers.py:1799–1832`).
|
||||
|
||||
**Never send `file_params` (keep version-level semantics):** batch download (`DownloadManager.js:1756–1766`; batch also filters out in-library versions at `:1648`), `downloadVersionWithDefaults` (`:1810–1830`), recipe import (`import/DownloadManager.js:269–276`), bulk missing-LoRA (`BulkMissingLoraDownloadManager.js:292–299`), `RecipeModal.js:1728–1736`, `ModelVersionsTab.js:1427`. `web/comfyui/` and `vue-widgets/src` contain **no** download triggers at all (grep-verified). `py/services/use_cases/` has only `download_model_use_case.py` (pass-through).
|
||||
|
||||
### 2.7 Paths that do NOT need changes (verified)
|
||||
|
||||
- **aria2 pause/resume** (`_resume_restored_aria2_download`, line 754+): resumes from persisted `resume_context`; never re-runs existence gates.
|
||||
- **`download_coordinator.py:90`**: pure pass-through of `file_params`.
|
||||
- **Update checker / plugin self-update** (`update_routes.py:496–501`): only closes the history DB handle.
|
||||
- **History delete semantics**: `mark_as_deleted` sets `is_deleted_override=1` and `has_been_downloaded` then returns False (`downloaded_version_history_service.py:276`) — LM-initiated deletes already reset the history skip.
|
||||
|
||||
### 2.8 Related pre-existing issues (record, not necessarily fix)
|
||||
|
||||
- **Queue retry drops file selection** (R9): `download_queue_service.retry_from_history` / `retry_all_failed` re-queue with `file_params=NULL` (`download_queue_service.py:705, 758`) although the queue table has a `file_params` column (`:43`) — a retried non-primary download silently reverts to the primary file. Fix alongside phase 1 (small: persist and reuse the column).
|
||||
- **`delete_model_version`** (`misc_handlers.py:2410–2487`): resolves the file via the single-valued `version_index` (2440–2444), deletes only that one file, and `mark_as_deleted` flags the **entire version** as deleted in history (2479) even when a sibling file remains in the library. See phase 2 item 6.1.5.
|
||||
|
||||
## 3. Goals / Non-Goals
|
||||
|
||||
**Goals**
|
||||
|
||||
- G1: A user can download any not-yet-downloaded file of a version already partially in the library (issue repro steps 6–8).
|
||||
- G2: True duplicates stay blocked: downloading the *same* file of the same version twice is rejected.
|
||||
- G3: Per-file downloaded state visible in the file dialog; multiple files selectable and downloadable in one pass.
|
||||
- G4: No regression for version-level semantics relied on by batch download, recipe missing-LoRA detection, and `skip_previously_downloaded_model_versions`.
|
||||
|
||||
**Non-Goals**
|
||||
|
||||
- No change to recipe `inLibrary` semantics ("any file of the version present" remains sufficient).
|
||||
- No change to the update-checker (version-level comparison).
|
||||
- No primary-key rebuild of the history database.
|
||||
- HuggingFace download flow untouched.
|
||||
|
||||
## 4. Design Decisions
|
||||
|
||||
- **D1 — Explicit file selection bypasses the history gate, version-level gates stay for everyone else.** The history skip exists to dedupe automated flows. A user explicitly picking a file is unambiguous intent; the file-level library gate (G2) still prevents real duplicates. **Guard conditions use normalized truthiness** (see D1a). All confirmed `file_params` senders are user-initiated dialog flows (2.6), and LM-initiated deletes already reset history (2.7), so the bypass only affects "downloaded but not LM-deleted" versions with the setting on — intended.
|
||||
- **D1a — `file_params` normalization at the boundary (R2).** `download-model-get` and `downloads/queue/add` accept arbitrary JSON; `{}` is `not None` but falsy and would bypass gates while downloading the primary file. Normalize `file_params = file_params or None` in the coordinator/handlers, and treat the bypass as active only when a target file id is resolvable.
|
||||
- **D2 — File identity matching rule (R6/R7):** hash-compare **only when both sides are non-empty** (lowercase SHA256 equality); name-compare when either side is empty. Never let `"" == ""` match. Name fallback caveats from 2.4 apply (renamed files, pending checkpoint hashes) — acceptable residual risk, worst case is a blocked re-download the user can retry after hashing completes.
|
||||
- **D3 — Cache indexes: additive multi-index + removal re-pointing (R4).** Add `version_files_index: Dict[int, List[dict]]` maintained alongside `version_index` by the same add/remove/rebuild methods; existing readers of `version_index` untouched. Additionally fix `remove_from_version_index`: when the popped entry has a surviving sibling (per the multi-index), re-point `version_index[version_id]` to the sibling instead of dropping the key; same for the `model_id_index` descriptor. This closes the 2.5 hazard for existing readers (`check_model_version_exists`, `existsLocally`, recipe matching) without restructuring anything.
|
||||
- **D4 — Per-file history via a child table (R3).** v1's additive-column approach is structurally impossible on a `(model_type, version_id)` PK (`ON CONFLICT DO UPDATE` would keep only the last file). Instead add `downloaded_version_files(model_type, version_id, file_id, file_name, downloaded_at, PRIMARY KEY(model_type, version_id, file_id))` — additive, no PK rebuild, honors the Non-Goal. Existing version-level table and queries unchanged. New per-file queries are opt-in. `_initialize_schema` uses `CREATE TABLE IF NOT EXISTS`, so the new table is created for existing DBs without any ALTER.
|
||||
- **D5 — UI flow reuse, with an extracted inner download function for multi-file (F7).** Phase 1 unlocks the existing badge → file dialog → location → download pipeline. Phase 2 upgrades the dialog to multi-select; iterating `executeDownloadWithProgress` as-is would produce N full library reloads, N toasts, and competing failure-summary modals — so phase 2 extracts a reload-deferred, failure-aggregating inner variant and runs one reload + one summary at the end.
|
||||
|
||||
## 5. Implementation — Phase 1 (fix the issue; independently shippable)
|
||||
|
||||
### 5.1 Backend — `py/services/download_manager.py`
|
||||
|
||||
1. **Normalize `file_params`** at the boundary (D1a): `download_coordinator.schedule_download` and the two API handlers (`model_handlers.py:1649–1666`, `1810–1832`) apply `file_params = file_params or None`.
|
||||
2. **Extract a shared file resolver** (R1): pull the matching logic at 1498–1569 into `_resolve_target_file(version_info, file_params) -> Optional[dict]`, used by **both** the new gate and the download-selection path. The selection path's condition (line 1499) switches from `model_version_id` to `resolved_version_id` (already computed at 1230–1236 from `version_info.id`), so gate and download always agree on the target file — including the `model_id`-only case.
|
||||
3. **New helper** `_find_local_file_entry(version_id, target_file) -> Optional[dict]`: iterate the three scanners' cached `raw_data` (NOT `version_index` — single-valued); candidates = entries whose `civitai.id` normalizes to `version_id`; match per D2.
|
||||
4. **Gate restructure in `_execute_original_download`**:
|
||||
- Early scanner gate (1157–1184): add `file_params is None` guard; with normalized `file_params`, defer (file identity not resolvable before metadata fetch).
|
||||
- After `version_info` fetch + `resolved_version_id` (~1229): when `file_params` present, resolve target file via the shared resolver; unresolvable → hard error "No matching file" (fail closed, prevents empty-dict bypass). Resolvable → `_find_local_file_entry`; hit → same hard error shape as today with the file name in the message.
|
||||
- History gate (1238–1279): add `file_params is None` (D1). Base-model skip (1281–1324) unchanged — still applies.
|
||||
- Late gate (1350–1376): add `file_params is None` guard (F2) — the post-fetch file-level check above already covers this case.
|
||||
- Nothing between the early gate and the post-fetch point assumes the version is absent (review task 6: only provider selection + metadata fetch; no DB writes; `_persist_aria2_state` runs only when actually downloading at 1659).
|
||||
5. **Queue retry fix** (2.8, small): persist `file_params` into the queue table on enqueue and reuse it in `retry_from_history` / `retry_all_failed`.
|
||||
6. Logging: `[download]` lines for file-level allow/block, consistent with existing style.
|
||||
|
||||
**Estimated:** ~150–220 LOC + resolver extraction.
|
||||
|
||||
### 5.2 Frontend — `static/js/managers/DownloadManager.js`
|
||||
|
||||
1. Line 598: drop `&& !existsLocally` from the badge condition (badge shows whenever `modelFiles.length > 1`).
|
||||
2. `fileParams` construction (1611–1616): add `name: this.selectedFile.name`.
|
||||
3. Surface the backend "file already in library" hard error as a toast instead of only the batch-summary modal (R10/F12 nit; reuse existing error message field).
|
||||
4. No changes to `updateNextButtonState` / `proceedToLocation` in phase 1; no template or CSS changes.
|
||||
|
||||
**Known phase-1 UX limitations (acknowledged, fixed in phase 2):** with all files downloaded the badge still renders and re-picking a downloaded file fails late (backend error after the location step); `localPath` may point at a sibling file; batch-preview "In Library" badge stays version-level and gives no hint of remaining files.
|
||||
|
||||
**Estimated:** ~10–30 LOC (confirmed realistic by review).
|
||||
|
||||
### 5.3 Phase 1 tests
|
||||
|
||||
Backend — extend `tests/services/test_download_manager_basic.py` (1694 lines; all fixture patterns exist):
|
||||
|
||||
- **Fixture gaps to add (F10):** `DummyScanner.get_cached_data()`/`raw_data` stub (~10 lines); `hashes.SHA256` in the metadata-provider payload's `files`.
|
||||
- Cases: same version + different SHA256 in library + `file_params` → proceeds; same SHA256 → hard error; `file_params=None` + version in library → hard error (unchanged); history-skip on + `file_params` → not skipped; without → skipped (unchanged); empty-dict `file_params` normalized → version-level behavior; `model_id`-only + `file_params` → gate and selection resolve the same file; legacy metadata (empty local sha256) matched by name; target file with empty SHA256 → name fallback, no `""==""` false positive.
|
||||
- Queue retry: `file_params` survives retry.
|
||||
- Assert proceed/abort via the existing `_execute_download` mock pattern.
|
||||
|
||||
Frontend (`tests/frontend/`): badge renders for multi-file version with `existsLocally=true` (pattern from `downloadManager.history.test.js`).
|
||||
|
||||
**Estimated:** ~150–250 LOC (confirmed realistic).
|
||||
|
||||
## 6. Implementation — Phase 2 (per-file status + multi-select + index hardening)
|
||||
|
||||
### 6.1 Backend
|
||||
|
||||
1. **`py/services/model_cache.py`** (D3): add `version_files_index`; maintain in `add_to_version_index` / `remove_from_version_index` / `rebuild_version_index`; removal re-points `version_index[version_id]` (and the `model_id_index` descriptor) to a surviving sibling instead of dropping the key.
|
||||
2. **`py/services/model_scanner.py`**: expose `get_files_for_version(version_id) -> List[dict]`.
|
||||
3. **`py/routes/handlers/model_handlers.py` `get_civitai_versions`**: annotate each version with `downloadedFiles: [{fileId, fileName, filePath}]` via `version_files_index` + D2 matching against `version.files`.
|
||||
4. **`py/services/downloaded_version_history_service.py`** (D4): new child table `downloaded_version_files`; `mark_downloaded` also upserts the child row when `file_id` known; `mark_as_deleted` clears the version's child rows only when no sibling remains in the library; new `get_downloaded_file_ids(model_type, version_id) -> set[int]`. `_record_downloaded_version_history` passes `file_info` through.
|
||||
5. **`delete_model_version`** (`misc_handlers.py:2410–2487`, R5): resolve **all** local files of the version via `version_files_index`; delete all (current endpoint semantics are version-level) or — if kept per-file — only `mark_as_deleted` when no sibling remains. Decide at implementation time; minimum is documenting current behavior.
|
||||
6. **`ModelVersionsTab` backend support**: none needed beyond item 3 (`downloadedFiles`); the tab consumes the same versions payload.
|
||||
|
||||
### 6.2 Frontend
|
||||
|
||||
1. **File dialog multi-select** — change surface (F8): option markup (`DownloadManager.js:712–724`), the single-select click handler (`727–734`), the `input[type="radio"]:checked` selector in `confirmFileSelection` (`738`); template `templates/components/modals/download_modal.html:48–60` (confirm-button label only); CSS `download-modal.css` — checkbox variant of `.file-option-radio input` (595–604) and a **new** `.file-option.disabled` style (does not exist). Files whose id ∈ `downloadedFiles` render disabled with an "In Library" tag.
|
||||
2. **Mixed-type guard (F8):** multi-select is restricted to files sharing the same routing target (`_isDiffusionModel` is computed once from a single `selectedFile` at 798–803; e.g. "Model" + "UNet" files route to different roots). Disallow mixed-type multi-select (simplest, predictable); single-file selection unchanged.
|
||||
3. **Multi-file download loop (D5/F7):** extract from `executeDownloadWithProgress` a reload-deferred, no-toast inner function; iterate per selected file with per-file progress; one `resetAndReload(true)` + one aggregated success/failure summary at the end (reuse `showDownloadBatchSummary`).
|
||||
4. **`updateNextButtonState` / `proceedToLocation`:** for multi-file versions, Next routes into the file dialog; hard block only when *every* weight file is downloaded.
|
||||
5. **`ModelVersionsTab.js` (F6):** the Download action (`:576` hidden when `isInLibrary`) — for multi-file versions with remaining files, show it and route into the download modal's file dialog; keep hidden when all files present.
|
||||
6. **Batch preview (F5):** `batch-preview-local-badge` (`:1320`) gains a "partially downloaded" hint for multi-file versions with remaining files.
|
||||
7. New i18n keys (`modals.download.fileSelection.inLibrary`, `downloadSelected`, partial-download tooltip, etc.) → run `python scripts/sync_translation_keys.py`.
|
||||
|
||||
### 6.3 Phase 2 tests
|
||||
|
||||
- `model_cache` (`tests/services/test_model_cache.py` already covers add/remove at 44–55): multi-valued index; sibling re-point on removal; rebuild.
|
||||
- `get_civitai_versions`: `downloadedFiles` correctness (hash match, name fallback, no match, CivArchive no-hash payload).
|
||||
- History service (`tests/services/test_downloaded_version_history_service.py` uses real SQLite on tmp_path): child-table creation on a legacy DB; per-file record/query; `mark_as_deleted` sibling semantics.
|
||||
- Frontend: dialog checkbox rendering/disabled state and multi-file confirm — **greenfield behavior coverage** (F10: no existing test exercises `showFileSelectionStep`/`confirmFileSelection`; infra exists, patterns must be built).
|
||||
|
||||
## 7. Risks and Mitigations
|
||||
|
||||
| Risk | Impact | Mitigation |
|
||||
|---|---|---|
|
||||
| History-gate bypass (D1) causes unwanted re-downloads in automated flows | Large checkpoint files re-downloaded | Bypass only with normalized, resolvable `file_params` (D1a); all such senders are user-initiated dialog flows (2.6, verified); tests pin batch/recipe/bulk behavior. |
|
||||
| Empty-hash matching edge cases (R6) | Duplicate download of the same file, or false block | D2 rule: hash only when both non-empty; name otherwise; never `""==""`. Residual risk documented (2.4). |
|
||||
| Phase-1 late-failure UX (F12) | User picks a downloaded file, fails only after location step | Toast surfacing (5.2.3); phase 2 disables downloaded files up front. |
|
||||
| Phase-2 index change corrupts existing behavior | Recipe matching, delete flows | Additive index + re-point only; `version_index` read semantics unchanged; `remove_models`/`update_single_model_cache` already rebuild (F3); tests. |
|
||||
| `delete_model_version` marks whole version deleted while sibling remains (R5) | History wrongly suppresses re-download of the surviving sibling's version | Phase 2 item 6.1.5; documented until then. |
|
||||
| History child-table migration failure on user installs | Service init crash | `CREATE TABLE IF NOT EXISTS` in `_initialize_schema`; failure degrades to version-level behavior (per-file queries return empty). |
|
||||
| Batch-preview badge misleading for partial versions (F5) | Minor UX confusion | Acknowledged in phase 1; fixed in phase 2 item 6.2.6. |
|
||||
| UI confusion: version shows "In Library" while files remain downloadable | Support burden | Phase 2: per-file disabled state + partial-download tooltip. |
|
||||
| Hash-identical sibling files (repacked content) | Second file blocked | Acceptable: scanner hash dedup already collapses them. |
|
||||
|
||||
## 8. Rollout
|
||||
|
||||
1. **Commit 1** — `fix(download): allow downloading additional files of an in-library model version (#1058)` → Phase 1 (5.1–5.3).
|
||||
2. **Commit 2** — `feat(download): per-file download status and multi-file selection (#1058)` → Phase 2 (6.1–6.3).
|
||||
|
||||
Phase 1 alone resolves the issue as reported; phase 2 can ship in a later release if review prefers smaller increments.
|
||||
|
||||
## 9. Effort Estimate (revised after review)
|
||||
|
||||
| Phase | Backend | Frontend | Tests | Risk |
|
||||
|---|---|---|---|---|
|
||||
| 1 | ~150–220 LOC (+ queue-retry fix ~30) | ~10–30 LOC | ~150–250 LOC | Low |
|
||||
| 2 | ~250–350 LOC | ~250–350 LOC (multi-file loop refactor + ModelVersionsTab + batch badge) | ~250–350 LOC (dialog tests greenfield) | Medium |
|
||||
+2331
-2327
File diff suppressed because it is too large
Load Diff
+5
-1
@@ -1244,11 +1244,13 @@
|
||||
"downloaded": "Downloaded",
|
||||
"downloadedTooltip": "Previously downloaded, but it is not currently in your library.",
|
||||
"alreadyInLibrary": "Already in Library",
|
||||
"partiallyDownloaded": "Partially downloaded",
|
||||
"autoOrganizedPath": "[Auto-organized by path template]",
|
||||
"fileSelection": {
|
||||
"title": "Select File Format",
|
||||
"files": "files",
|
||||
"select": "Select File"
|
||||
"select": "Select File",
|
||||
"inLibrary": "In Library"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Invalid Civitai URL format",
|
||||
@@ -1594,6 +1596,7 @@
|
||||
"actions": {
|
||||
"download": "Download",
|
||||
"downloadTooltip": "Download this version",
|
||||
"downloadRemainingTooltip": "Download remaining files of this version",
|
||||
"downloadEarlyAccessTooltip": "Download this early access version from Civitai",
|
||||
"downloadPaidTooltip": "Download this paid version from Civitai",
|
||||
"downloadNotAllowedTooltip": "This version is only available for on-site generation on Civitai",
|
||||
@@ -1942,6 +1945,7 @@
|
||||
"downloadPartialSuccess": "Downloaded {completed} of {total} LoRAs",
|
||||
"downloadPartialWithAccess": "Downloaded {completed} of {total} LoRAs. {accessFailures} failed due to access restrictions. Check your API key in settings or early access status.",
|
||||
"pleaseSelectVersion": "Please select a version",
|
||||
"pleaseSelectFile": "Please select at least one file",
|
||||
"versionExists": "This version already exists in your library",
|
||||
"downloadCompleted": "Download completed successfully",
|
||||
"downloadSkippedByBaseModel": "Skipped download because base model {baseModel} is excluded",
|
||||
|
||||
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
+2331
-2327
File diff suppressed because it is too large
Load Diff
@@ -41,6 +41,40 @@ class RecipeMetadataParser(ABC):
|
||||
"""
|
||||
pass
|
||||
|
||||
@staticmethod
|
||||
def populate_lora_from_local(lora_entry: Dict[str, Any], local_lora: Dict[str, Any], base_model_counts=None) -> Dict[str, Any]:
|
||||
"""Populate a recipe LoRA entry from the local scanner cache."""
|
||||
local_path = local_lora.get('file_path') or ''
|
||||
file_name = local_lora.get('file_name') or os.path.splitext(os.path.basename(local_path))[0]
|
||||
base_model = local_lora.get('base_model') or ''
|
||||
|
||||
lora_entry['name'] = local_lora.get('model_name') or file_name or lora_entry.get('name', '')
|
||||
lora_entry['file_name'] = file_name
|
||||
lora_entry['hash'] = (local_lora.get('sha256') or lora_entry.get('hash') or '').lower()
|
||||
lora_entry['localPath'] = local_path or None
|
||||
lora_entry['size'] = local_lora.get('size', 0) or 0
|
||||
lora_entry['baseModel'] = base_model
|
||||
lora_entry['existsLocally'] = True
|
||||
lora_entry['isDeleted'] = False
|
||||
|
||||
preview_url = local_lora.get('preview_url')
|
||||
if preview_url:
|
||||
lora_entry['thumbnailUrl'] = config.get_preview_static_url(preview_url)
|
||||
|
||||
civitai_info = local_lora.get('civitai') or {}
|
||||
if isinstance(civitai_info, dict):
|
||||
if civitai_info.get('id') is not None:
|
||||
lora_entry['id'] = civitai_info['id']
|
||||
if civitai_info.get('modelId') is not None:
|
||||
lora_entry['modelId'] = civitai_info['modelId']
|
||||
if civitai_info.get('name'):
|
||||
lora_entry['version'] = civitai_info['name']
|
||||
|
||||
if base_model_counts is not None and base_model:
|
||||
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
|
||||
|
||||
return lora_entry
|
||||
|
||||
@staticmethod
|
||||
async def populate_lora_from_civitai(lora_entry: Dict[str, Any], civitai_info_tuple: Tuple[Dict[str, Any] | None, str | None] | Dict[str, Any],
|
||||
recipe_scanner=None, base_model_counts=None, hash_value=None) -> Optional[Dict[str, Any]]:
|
||||
|
||||
+201
-61
@@ -362,68 +362,208 @@ class AutomaticMetadataParser(RecipeMetadataParser):
|
||||
|
||||
checkpoint = checkpoint_entry
|
||||
|
||||
# If no LoRAs from Civitai resources or to supplement, extract from metadata["hashes"]
|
||||
if not loras or len(loras) == 0:
|
||||
# Extract lora weights from extranet tags in prompt (for later use)
|
||||
lora_weights = {}
|
||||
lora_matches = re.findall(self.EXTRANETS_REGEX, prompt)
|
||||
for lora_type, lora_name, lora_weight in lora_matches:
|
||||
key = f"{lora_type}:{lora_name}"
|
||||
lora_weights[key] = round(float(lora_weight), 2)
|
||||
|
||||
# Use hashes from metadata as the primary source
|
||||
if metadata.get("hashes"):
|
||||
for hash_key, lora_hash in metadata.get("hashes", {}).items():
|
||||
# Only process lora or hypernet types
|
||||
if not hash_key.startswith(("lora:", "hypernet:")):
|
||||
def normalize_lora_name(name, basename=False):
|
||||
normalized = str(name or '').replace('\\', '/')
|
||||
if normalized.casefold().endswith('.safetensors'):
|
||||
normalized = normalized[:-12]
|
||||
if basename:
|
||||
normalized = normalized.rsplit('/', 1)[-1]
|
||||
return normalized.casefold()
|
||||
|
||||
def get_version_id(lora):
|
||||
version_id = lora.get('id')
|
||||
if version_id in (None, '', 0, '0'):
|
||||
version_id = lora.get('modelVersionId')
|
||||
if version_id in (None, '', 0, '0'):
|
||||
return None
|
||||
return str(version_id)
|
||||
|
||||
prompt_loras = {}
|
||||
for match in re.findall(self.EXTRANETS_REGEX, prompt):
|
||||
lora_type, lora_name, _ = match
|
||||
prompt_loras[(lora_type, normalize_lora_name(lora_name))] = match
|
||||
|
||||
prompt_by_basename = {}
|
||||
for lora_type, lora_name, lora_weight in prompt_loras.values():
|
||||
key = (lora_type, normalize_lora_name(lora_name, True))
|
||||
prompt_by_basename.setdefault(key, []).append((lora_name, round(float(lora_weight), 2)))
|
||||
|
||||
hash_basenames = {
|
||||
(hash_key.split(':', 1)[0], normalize_lora_name(hash_key.split(':', 1)[1], True))
|
||||
for hash_key, hash_value in metadata.get("hashes", {}).items()
|
||||
if hash_value and hash_key.startswith(("lora:", "hypernet:"))
|
||||
}
|
||||
recipe_base_model = checkpoint.get("baseModel") if checkpoint else None
|
||||
if not recipe_base_model and len(base_model_counts) == 1:
|
||||
recipe_base_model = next(iter(base_model_counts))
|
||||
|
||||
resource_lora_count = len(loras)
|
||||
|
||||
def make_lora_entry(lora_type, lora_name, weight, lora_hash=''):
|
||||
return {
|
||||
'name': lora_name,
|
||||
'type': lora_type,
|
||||
'weight': weight,
|
||||
'hash': lora_hash,
|
||||
'existsLocally': False,
|
||||
'localPath': None,
|
||||
'file_name': lora_name,
|
||||
'thumbnailUrl': '/loras_static/images/no-preview.png',
|
||||
'baseModel': '',
|
||||
'size': 0,
|
||||
'downloadUrl': '',
|
||||
'isDeleted': False
|
||||
}
|
||||
|
||||
def merge_or_append_civitai(civitai_entry, preserve_existing_weight=False):
|
||||
civitai_id = get_version_id(civitai_entry)
|
||||
civitai_hash = (civitai_entry.get('hash') or '').lower()
|
||||
for index, existing in enumerate(loras):
|
||||
existing_id = get_version_id(existing)
|
||||
existing_hash = (existing.get('hash') or '').lower()
|
||||
if not (
|
||||
(civitai_id and existing_id == civitai_id)
|
||||
or (civitai_hash and existing_hash == civitai_hash)
|
||||
):
|
||||
continue
|
||||
|
||||
if preserve_existing_weight:
|
||||
civitai_entry['weight'] = existing.get('weight', civitai_entry['weight'])
|
||||
existing_base = existing.get('baseModel')
|
||||
if not civitai_entry.get('baseModel'):
|
||||
civitai_entry['baseModel'] = existing_base or ''
|
||||
elif existing_base:
|
||||
remaining = base_model_counts.get(existing_base, 0) - 1
|
||||
if remaining > 0:
|
||||
base_model_counts[existing_base] = remaining
|
||||
else:
|
||||
base_model_counts.pop(existing_base, None)
|
||||
loras[index] = civitai_entry
|
||||
return
|
||||
loras.append(civitai_entry)
|
||||
|
||||
def merge_or_append_local(local_entry):
|
||||
local_id = get_version_id(local_entry)
|
||||
local_hash = (local_entry.get('hash') or '').lower()
|
||||
for existing in loras:
|
||||
existing_id = get_version_id(existing)
|
||||
existing_hash = (existing.get('hash') or '').lower()
|
||||
if not (
|
||||
(local_id and existing_id == local_id)
|
||||
or (local_hash and existing_hash == local_hash)
|
||||
):
|
||||
continue
|
||||
|
||||
existing['weight'] = local_entry['weight']
|
||||
existing['hash'] = local_entry['hash']
|
||||
existing['file_name'] = local_entry['file_name']
|
||||
existing['existsLocally'] = True
|
||||
existing['localPath'] = local_entry['localPath']
|
||||
existing['size'] = local_entry['size']
|
||||
existing['isDeleted'] = False
|
||||
if not existing.get('modelId') and local_entry.get('modelId'):
|
||||
existing['modelId'] = local_entry['modelId']
|
||||
if not existing.get('baseModel') and local_entry.get('baseModel'):
|
||||
existing['baseModel'] = local_entry['baseModel']
|
||||
base_model_counts[local_entry['baseModel']] = base_model_counts.get(local_entry['baseModel'], 0) + 1
|
||||
thumbnail_url = local_entry.get('thumbnailUrl')
|
||||
if thumbnail_url and not thumbnail_url.endswith('/images/no-preview.png'):
|
||||
existing['thumbnailUrl'] = thumbnail_url
|
||||
return
|
||||
|
||||
if local_entry.get('baseModel'):
|
||||
base_model = local_entry['baseModel']
|
||||
base_model_counts[base_model] = base_model_counts.get(base_model, 0) + 1
|
||||
loras.append(local_entry)
|
||||
|
||||
resolved_prompt_basenames = set()
|
||||
queried_local_basenames = set()
|
||||
for lora_type, lora_name, lora_weight in prompt_loras.values():
|
||||
weight = round(float(lora_weight), 2)
|
||||
basename_key = (lora_type, normalize_lora_name(lora_name, True))
|
||||
matching_resources = [
|
||||
lora
|
||||
for lora in loras[:resource_lora_count]
|
||||
if lora.get('file_name')
|
||||
and normalize_lora_name(lora['file_name'], True) == basename_key[1]
|
||||
and (
|
||||
(lora_type == 'hypernet' and str(lora.get('type', '')).casefold() in ('hypernet', 'hypernetwork'))
|
||||
or (lora_type == 'lora' and str(lora.get('type', '')).casefold() not in ('hypernet', 'hypernetwork'))
|
||||
)
|
||||
]
|
||||
if len(prompt_by_basename[basename_key]) == 1 and len(matching_resources) == 1:
|
||||
matching_resources[0]['weight'] = weight
|
||||
if basename_key not in hash_basenames:
|
||||
resolved_prompt_basenames.add(basename_key)
|
||||
continue
|
||||
|
||||
if basename_key in hash_basenames:
|
||||
continue
|
||||
|
||||
if not recipe_scanner or lora_type != 'lora':
|
||||
continue
|
||||
queried_local_basenames.add(basename_key)
|
||||
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
|
||||
if not local_lora:
|
||||
continue
|
||||
|
||||
local_entry = self.populate_lora_from_local(
|
||||
make_lora_entry(lora_type, lora_name, weight),
|
||||
local_lora,
|
||||
)
|
||||
merge_or_append_local(local_entry)
|
||||
resolved_prompt_basenames.add(basename_key)
|
||||
|
||||
for hash_key, lora_hash in metadata.get("hashes", {}).items():
|
||||
if not hash_key.startswith(("lora:", "hypernet:")):
|
||||
continue
|
||||
lora_type, lora_name = hash_key.split(':', 1)
|
||||
basename_key = (lora_type, normalize_lora_name(lora_name, True))
|
||||
if basename_key in resolved_prompt_basenames:
|
||||
continue
|
||||
|
||||
prompt_entries = prompt_by_basename.get(basename_key, [])
|
||||
weight = prompt_entries[0][1] if len(prompt_entries) == 1 else 1.0
|
||||
lora_entry = make_lora_entry(lora_type, lora_name, weight, lora_hash)
|
||||
|
||||
if lora_hash and recipe_scanner and lora_type == 'lora':
|
||||
local_lora = await recipe_scanner.get_local_lora_by_hash(lora_hash)
|
||||
if local_lora:
|
||||
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
|
||||
merge_or_append_local(local_entry)
|
||||
continue
|
||||
|
||||
hash_resolved = False
|
||||
if lora_hash and metadata_provider:
|
||||
try:
|
||||
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
|
||||
populated_entry = await self.populate_lora_from_civitai(
|
||||
lora_entry,
|
||||
civitai_info,
|
||||
recipe_scanner,
|
||||
base_model_counts,
|
||||
lora_hash,
|
||||
)
|
||||
if populated_entry is None:
|
||||
continue
|
||||
|
||||
# Skip entries without a hash value — they can't be
|
||||
# resolved via CivitAI and would only produce a
|
||||
# useless "Deleted" entry in the recipe.
|
||||
if not lora_hash:
|
||||
continue
|
||||
|
||||
lora_type, lora_name = hash_key.split(':', 1)
|
||||
|
||||
# Get weight from extranet tags if available, else default to 1.0
|
||||
weight = lora_weights.get(hash_key, 1.0)
|
||||
|
||||
# Initialize lora entry
|
||||
lora_entry = {
|
||||
'name': lora_name,
|
||||
'type': lora_type, # 'lora' or 'hypernet'
|
||||
'weight': weight,
|
||||
'hash': lora_hash,
|
||||
'existsLocally': False,
|
||||
'localPath': None,
|
||||
'file_name': lora_name,
|
||||
'thumbnailUrl': '/loras_static/images/no-preview.png',
|
||||
'baseModel': '',
|
||||
'size': 0,
|
||||
'downloadUrl': '',
|
||||
'isDeleted': False
|
||||
}
|
||||
|
||||
# Try to get info from Civitai
|
||||
if metadata_provider:
|
||||
try:
|
||||
civitai_info = await metadata_provider.get_model_by_hash(lora_hash)
|
||||
|
||||
populated_entry = await self.populate_lora_from_civitai(
|
||||
lora_entry,
|
||||
civitai_info,
|
||||
recipe_scanner,
|
||||
base_model_counts,
|
||||
lora_hash
|
||||
)
|
||||
if populated_entry is None:
|
||||
continue # Skip invalid LoRA types
|
||||
lora_entry = populated_entry
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching Civitai info for LoRA {lora_name}: {e}")
|
||||
|
||||
loras.append(lora_entry)
|
||||
lora_entry = populated_entry
|
||||
hash_resolved = not lora_entry.get('isDeleted')
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching Civitai info for LoRA {lora_name}: {e}")
|
||||
|
||||
if hash_resolved:
|
||||
merge_or_append_civitai(lora_entry, preserve_existing_weight=not prompt_entries)
|
||||
continue
|
||||
|
||||
if recipe_scanner and lora_type == 'lora' and basename_key not in queried_local_basenames:
|
||||
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
|
||||
if local_lora:
|
||||
local_entry = self.populate_lora_from_local(lora_entry, local_lora)
|
||||
merge_or_append_local(local_entry)
|
||||
continue
|
||||
|
||||
if lora_hash and not resource_lora_count:
|
||||
loras.append(lora_entry)
|
||||
|
||||
# Try to get base model from resources or make educated guess
|
||||
base_model = None
|
||||
|
||||
+95
-68
@@ -31,79 +31,15 @@ class ComfyMetadataParser(RecipeMetadataParser):
|
||||
metadata_provider = await get_default_metadata_provider()
|
||||
|
||||
data = json.loads(user_comment)
|
||||
loras = []
|
||||
|
||||
# Find all LoraLoader nodes
|
||||
lora_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'LoraLoader'}
|
||||
|
||||
# Process each LoraLoader node
|
||||
for node_id, node in lora_nodes.items():
|
||||
if 'inputs' not in node or 'lora_name' not in node['inputs']:
|
||||
continue
|
||||
|
||||
lora_name = node['inputs'].get('lora_name', '')
|
||||
|
||||
# Parse the URN to extract model ID and version ID
|
||||
# Format: "urn:air:sdxl:lora:civitai:1107767@1253442"
|
||||
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
|
||||
if not lora_id_match:
|
||||
continue
|
||||
|
||||
model_id = lora_id_match.group(1)
|
||||
model_version_id = lora_id_match.group(2)
|
||||
|
||||
# Get strength from node inputs
|
||||
weight = node['inputs'].get('strength_model', 1.0)
|
||||
|
||||
# Initialize lora entry with default values
|
||||
lora_entry = {
|
||||
'id': model_version_id,
|
||||
'modelId': model_id,
|
||||
'name': f"Lora {model_id}", # Default name
|
||||
'version': '',
|
||||
'type': 'lora',
|
||||
'weight': weight,
|
||||
'existsLocally': False,
|
||||
'localPath': None,
|
||||
'file_name': '',
|
||||
'hash': '',
|
||||
'thumbnailUrl': '/loras_static/images/no-preview.png',
|
||||
'baseModel': '',
|
||||
'size': 0,
|
||||
'downloadUrl': '',
|
||||
'isDeleted': False
|
||||
}
|
||||
|
||||
# Get additional info from Civitai if metadata provider is available
|
||||
if metadata_provider:
|
||||
try:
|
||||
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
|
||||
# Populate lora entry with Civitai info
|
||||
populated_entry = await self.populate_lora_from_civitai(
|
||||
lora_entry,
|
||||
civitai_info_tuple,
|
||||
recipe_scanner
|
||||
)
|
||||
if populated_entry is None:
|
||||
continue # Skip invalid LoRA types
|
||||
lora_entry = populated_entry
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching Civitai info for LoRA: {e}")
|
||||
|
||||
loras.append(lora_entry)
|
||||
|
||||
# Find checkpoint info
|
||||
|
||||
checkpoint_nodes = {k: v for k, v in data.items() if isinstance(v, dict) and v.get('class_type') == 'CheckpointLoaderSimple'}
|
||||
checkpoint = None
|
||||
checkpoint_id = None
|
||||
checkpoint_version_id = None
|
||||
|
||||
if checkpoint_nodes:
|
||||
# Get the first checkpoint node
|
||||
checkpoint_node = next(iter(checkpoint_nodes.values()))
|
||||
if 'inputs' in checkpoint_node and 'ckpt_name' in checkpoint_node['inputs']:
|
||||
checkpoint_name = checkpoint_node['inputs']['ckpt_name']
|
||||
# Parse checkpoint URN
|
||||
checkpoint_match = re.search(r'civitai:(\d+)@(\d+)', checkpoint_name)
|
||||
if checkpoint_match:
|
||||
checkpoint_id = checkpoint_match.group(1)
|
||||
@@ -115,16 +51,107 @@ class ComfyMetadataParser(RecipeMetadataParser):
|
||||
'version': '',
|
||||
'type': 'checkpoint'
|
||||
}
|
||||
|
||||
# Get additional checkpoint info from Civitai
|
||||
if metadata_provider:
|
||||
try:
|
||||
civitai_info_tuple = await metadata_provider.get_model_version_info(checkpoint_version_id)
|
||||
civitai_info, _ = civitai_info_tuple if isinstance(civitai_info_tuple, tuple) else (civitai_info_tuple, None)
|
||||
# Populate checkpoint with Civitai info
|
||||
checkpoint = await self.populate_checkpoint_from_civitai(checkpoint, civitai_info)
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching Civitai info for checkpoint: {e}")
|
||||
|
||||
recipe_base_model = checkpoint.get('baseModel') if checkpoint else None
|
||||
loras = []
|
||||
lora_candidates = []
|
||||
for node in data.values():
|
||||
if not isinstance(node, dict):
|
||||
continue
|
||||
|
||||
inputs = node.get('inputs')
|
||||
if not isinstance(inputs, dict):
|
||||
continue
|
||||
|
||||
if node.get('class_type') == 'LoraLoader':
|
||||
lora_name = inputs.get('lora_name', '')
|
||||
if isinstance(lora_name, str) and lora_name:
|
||||
lora_candidates.append((lora_name, inputs.get('strength_model', 1.0)))
|
||||
continue
|
||||
|
||||
if node.get('class_type') != 'LoraLoaderLM':
|
||||
continue
|
||||
|
||||
loras_data = inputs.get('loras', [])
|
||||
if isinstance(loras_data, dict):
|
||||
loras_data = loras_data.get('__value__', [])
|
||||
if isinstance(loras_data, list) and len(loras_data) == 1 and isinstance(loras_data[0], list):
|
||||
loras_data = loras_data[0]
|
||||
if not isinstance(loras_data, list):
|
||||
continue
|
||||
|
||||
for lora in loras_data:
|
||||
if not isinstance(lora, dict) or not lora.get('active', False) or lora.get('_isDummy', False):
|
||||
continue
|
||||
lora_name = lora.get('name', '')
|
||||
if isinstance(lora_name, str) and lora_name:
|
||||
lora_candidates.append((lora_name, lora.get('strength', 1.0)))
|
||||
|
||||
for lora_name, weight in lora_candidates:
|
||||
if isinstance(weight, str):
|
||||
try:
|
||||
weight = float(weight)
|
||||
except ValueError:
|
||||
weight = 1.0
|
||||
lora_id_match = re.search(r'civitai:(\d+)@(\d+)', lora_name)
|
||||
if lora_id_match:
|
||||
model_id = lora_id_match.group(1)
|
||||
model_version_id = lora_id_match.group(2)
|
||||
entry_name = f"Lora {model_id}"
|
||||
else:
|
||||
model_id = 0
|
||||
model_version_id = 0
|
||||
entry_name = re.split(r'[\\/]', lora_name)[-1]
|
||||
entry_name = re.sub(r'\.[^.]+$', '', entry_name)
|
||||
|
||||
lora_entry = {
|
||||
'id': model_version_id,
|
||||
'modelId': model_id,
|
||||
'name': entry_name,
|
||||
'version': '',
|
||||
'type': 'lora',
|
||||
'weight': weight,
|
||||
'existsLocally': False,
|
||||
'localPath': None,
|
||||
'file_name': entry_name,
|
||||
'hash': '',
|
||||
'thumbnailUrl': '/loras_static/images/no-preview.png',
|
||||
'baseModel': '',
|
||||
'size': 0,
|
||||
'downloadUrl': '',
|
||||
'isDeleted': False
|
||||
}
|
||||
|
||||
if lora_id_match:
|
||||
if metadata_provider:
|
||||
try:
|
||||
civitai_info_tuple = await metadata_provider.get_model_version_info(model_version_id)
|
||||
populated_entry = await self.populate_lora_from_civitai(
|
||||
lora_entry,
|
||||
civitai_info_tuple,
|
||||
recipe_scanner
|
||||
)
|
||||
if populated_entry is None:
|
||||
continue
|
||||
lora_entry = populated_entry
|
||||
except Exception as e:
|
||||
logger.error(f"Error fetching Civitai info for LoRA: {e}")
|
||||
else:
|
||||
if not recipe_scanner:
|
||||
continue
|
||||
local_lora = await recipe_scanner.get_local_lora(lora_name, recipe_base_model)
|
||||
if not local_lora:
|
||||
continue
|
||||
lora_entry = self.populate_lora_from_local(lora_entry, local_lora)
|
||||
|
||||
loras.append(lora_entry)
|
||||
|
||||
# Extract generation parameters
|
||||
gen_params = {}
|
||||
|
||||
@@ -2428,8 +2428,8 @@ class ModelLibraryHandler:
|
||||
embedding_scanner = await self._service_registry.get_embedding_scanner()
|
||||
|
||||
found_type = None
|
||||
file_path = None
|
||||
found_cache = None
|
||||
entries: list = []
|
||||
|
||||
for model_type, scanner in (
|
||||
("lora", lora_scanner),
|
||||
@@ -2440,27 +2440,43 @@ class ModelLibraryHandler:
|
||||
if cache and model_version_id in cache.version_index:
|
||||
found_type = model_type
|
||||
found_cache = cache
|
||||
entry = cache.version_index[model_version_id]
|
||||
file_path = entry.get("file_path")
|
||||
# A version can have several local files (#1058); collect
|
||||
# them all so the delete below covers every file.
|
||||
files_getter = getattr(cache, "get_files_by_version_id", None)
|
||||
if files_getter is not None:
|
||||
entries = files_getter(model_version_id)
|
||||
else:
|
||||
entries = [cache.version_index[model_version_id]]
|
||||
break
|
||||
|
||||
if not file_path:
|
||||
file_paths = [
|
||||
entry.get("file_path")
|
||||
for entry in entries
|
||||
if isinstance(entry, dict) and entry.get("file_path")
|
||||
]
|
||||
|
||||
if not file_paths:
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Model version not found in any scanner cache"},
|
||||
status=404,
|
||||
)
|
||||
|
||||
target_dir = os.path.dirname(file_path)
|
||||
base_name = os.path.basename(file_path)
|
||||
file_name, extension = os.path.splitext(base_name)
|
||||
await delete_model_artifacts(target_dir, file_name, main_extension=extension)
|
||||
for file_path in file_paths:
|
||||
target_dir = os.path.dirname(file_path)
|
||||
base_name = os.path.basename(file_path)
|
||||
file_name, extension = os.path.splitext(base_name)
|
||||
await delete_model_artifacts(target_dir, file_name, main_extension=extension)
|
||||
|
||||
if found_cache:
|
||||
removed_paths = set(file_paths)
|
||||
found_cache.raw_data = [
|
||||
item
|
||||
for item in found_cache.raw_data
|
||||
if item.get("file_path") != file_path
|
||||
if item.get("file_path") not in removed_paths
|
||||
]
|
||||
rebuild = getattr(found_cache, "rebuild_version_index", None)
|
||||
if rebuild is not None:
|
||||
rebuild()
|
||||
await found_cache.resort()
|
||||
|
||||
scanner_map = {
|
||||
@@ -2483,6 +2499,7 @@ class ModelLibraryHandler:
|
||||
"success": True,
|
||||
"modelType": found_type,
|
||||
"modelVersionId": model_version_id,
|
||||
"deletedFiles": len(file_paths),
|
||||
}
|
||||
)
|
||||
except Exception as exc:
|
||||
|
||||
@@ -1659,7 +1659,8 @@ class ModelDownloadHandler:
|
||||
import json
|
||||
|
||||
try:
|
||||
data["file_params"] = json.loads(file_params_json)
|
||||
# Normalize falsy payloads (e.g. {}) to None (#1058)
|
||||
data["file_params"] = json.loads(file_params_json) or None
|
||||
except json.JSONDecodeError:
|
||||
self._logger.warning(
|
||||
"Invalid file_params JSON: %s", file_params_json
|
||||
@@ -1811,7 +1812,8 @@ class ModelDownloadHandler:
|
||||
|
||||
model_id = int(model_id_str) if model_id_str else None
|
||||
model_version_id = int(model_version_id_str) if model_version_id_str else None
|
||||
file_params = json.loads(file_params_json) if file_params_json else None
|
||||
# Normalize falsy payloads (e.g. {}) to None (#1058)
|
||||
file_params = (json.loads(file_params_json) if file_params_json else None) or None
|
||||
|
||||
service = await DownloadQueueService.get_instance()
|
||||
item = await service.add_to_queue(
|
||||
@@ -2187,6 +2189,19 @@ class ModelCivitaiHandler:
|
||||
else:
|
||||
version.pop("localPath", None)
|
||||
|
||||
# Per-file downloaded state so multi-file versions can show
|
||||
# which individual files are already in the library (#1058)
|
||||
local_entries: List[Any] = []
|
||||
if version_id is not None and cache:
|
||||
files_getter = getattr(cache, "get_files_by_version_id", None)
|
||||
if files_getter is not None:
|
||||
local_entries = files_getter(version_id)
|
||||
elif cache_entry is not None:
|
||||
local_entries = [cache_entry]
|
||||
version["downloadedFiles"] = self._match_downloaded_files(
|
||||
version, local_entries
|
||||
)
|
||||
|
||||
model_file = (
|
||||
self._find_model_file(version.get("files", []))
|
||||
if isinstance(version.get("files"), Iterable)
|
||||
@@ -2201,6 +2216,64 @@ class ModelCivitaiHandler:
|
||||
)
|
||||
return web.Response(status=500, text=str(exc))
|
||||
|
||||
@staticmethod
|
||||
def _match_downloaded_files(
|
||||
version: Mapping[str, Any], local_entries: List[Any]
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Map local library entries back to individual files of a version.
|
||||
|
||||
Matching follows rule D2 (#1058): SHA256 is authoritative when the
|
||||
local entry carries one; otherwise fall back to extension-less file
|
||||
name equality. Returns ``[{fileId, fileName, filePath}]``.
|
||||
"""
|
||||
files = version.get("files")
|
||||
if not isinstance(files, list) or not local_entries:
|
||||
return []
|
||||
|
||||
by_hash: Dict[str, Mapping[str, Any]] = {}
|
||||
by_name: Dict[str, Mapping[str, Any]] = {}
|
||||
for file_info in files:
|
||||
if not isinstance(file_info, Mapping):
|
||||
continue
|
||||
sha = str(
|
||||
(file_info.get("hashes") or {}).get("SHA256") or ""
|
||||
).strip().lower()
|
||||
if sha:
|
||||
by_hash.setdefault(sha, file_info)
|
||||
name = str(file_info.get("name") or "").strip()
|
||||
if name:
|
||||
by_name.setdefault(os.path.splitext(name)[0], file_info)
|
||||
|
||||
downloaded: List[Dict[str, Any]] = []
|
||||
seen_keys: set = set()
|
||||
for entry in local_entries:
|
||||
if not isinstance(entry, Mapping):
|
||||
continue
|
||||
matched: Optional[Mapping[str, Any]] = None
|
||||
local_hash = str(entry.get("sha256") or "").strip().lower()
|
||||
if local_hash:
|
||||
matched = by_hash.get(local_hash)
|
||||
if matched is None:
|
||||
local_name = str(entry.get("file_name") or "").strip()
|
||||
if local_name:
|
||||
matched = by_name.get(local_name)
|
||||
if matched is None:
|
||||
continue
|
||||
|
||||
file_id = matched.get("id")
|
||||
dedupe_key = file_id if file_id is not None else matched.get("name")
|
||||
if dedupe_key in seen_keys:
|
||||
continue
|
||||
seen_keys.add(dedupe_key)
|
||||
downloaded.append(
|
||||
{
|
||||
"fileId": file_id,
|
||||
"fileName": matched.get("name"),
|
||||
"filePath": entry.get("file_path"),
|
||||
}
|
||||
)
|
||||
return downloaded
|
||||
|
||||
async def get_civitai_model_by_version(self, request: web.Request) -> web.Response:
|
||||
try:
|
||||
model_version_id = request.match_info.get("modelVersionId")
|
||||
|
||||
@@ -87,7 +87,9 @@ class DownloadCoordinator:
|
||||
progress_callback=progress_callback,
|
||||
download_id=download_id,
|
||||
source=payload.get("source"),
|
||||
file_params=payload.get("file_params"),
|
||||
# Normalize falsy file_params (e.g. {}) to None so download gates
|
||||
# treat it as "no explicit file selection" (#1058).
|
||||
file_params=payload.get("file_params") or None,
|
||||
)
|
||||
|
||||
result["download_id"] = download_id
|
||||
|
||||
+237
-69
@@ -213,6 +213,162 @@ class DownloadManager:
|
||||
)
|
||||
return False
|
||||
|
||||
async def _get_scanner_for_model_type(self, model_type: str):
|
||||
"""Return the scanner responsible for the given model type."""
|
||||
if model_type == "checkpoint":
|
||||
return await self._get_checkpoint_scanner()
|
||||
if model_type == "embedding":
|
||||
return await ServiceRegistry.get_embedding_scanner()
|
||||
return await self._get_lora_scanner()
|
||||
|
||||
@staticmethod
|
||||
def _resolve_target_file(
|
||||
files: Any, file_params: Dict[str, Any] | None
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Resolve the target file within a version's file list from file_params.
|
||||
|
||||
Shared by the existence gate and the actual file selection so both
|
||||
always agree on which file a download refers to (#1058). Returns None
|
||||
when file_params is None or no file matches.
|
||||
"""
|
||||
if not file_params or not isinstance(files, list):
|
||||
return None
|
||||
|
||||
target_file_id = file_params.get("id")
|
||||
target_type = file_params.get("type", "Model")
|
||||
target_format = file_params.get("format")
|
||||
target_size = file_params.get("size")
|
||||
target_fp = file_params.get("fp")
|
||||
is_primary = file_params.get("isPrimary", False)
|
||||
|
||||
logger.debug(
|
||||
"[download] file_params received: id=%s, type=%s, format=%s, size=%s, fp=%s, "
|
||||
"isPrimary=%s, total_files=%d",
|
||||
target_file_id, target_type, target_format, target_size, target_fp,
|
||||
is_primary, len(files),
|
||||
)
|
||||
|
||||
file_info: Optional[Dict[str, Any]] = None
|
||||
|
||||
if target_file_id:
|
||||
target_id_str = str(target_file_id)
|
||||
for f in files:
|
||||
if not isinstance(f, dict):
|
||||
continue
|
||||
f_id = f.get("id")
|
||||
if str(f_id) == target_id_str:
|
||||
file_info = f
|
||||
logger.debug(
|
||||
"[download] MATCH by ID: id=%s name='%s'",
|
||||
f_id, f.get("name"),
|
||||
)
|
||||
break
|
||||
if not file_info:
|
||||
logger.debug("[download] No file found with id=%s", target_file_id)
|
||||
|
||||
elif is_primary:
|
||||
file_info = next(
|
||||
(
|
||||
f
|
||||
for f in files
|
||||
if isinstance(f, dict)
|
||||
and f.get("primary")
|
||||
and f.get("type") in MODEL_WEIGHT_FILE_TYPES
|
||||
),
|
||||
None,
|
||||
)
|
||||
else:
|
||||
# Lenient metadata match: only compare fields present on both sides
|
||||
for f in files:
|
||||
if not isinstance(f, dict):
|
||||
continue
|
||||
f_type = f.get("type", "")
|
||||
if f_type != target_type:
|
||||
continue
|
||||
|
||||
f_meta = f.get("metadata", {})
|
||||
f_format = f_meta.get("format") or f.get("format")
|
||||
f_size = f_meta.get("size") or f.get("size")
|
||||
f_fp = f_meta.get("fp") or f.get("fp")
|
||||
|
||||
if target_format and f_format != target_format:
|
||||
continue
|
||||
if target_size and f_size and f_size != target_size:
|
||||
continue
|
||||
if target_fp and f_fp and f_fp != target_fp:
|
||||
continue
|
||||
|
||||
file_info = f
|
||||
break
|
||||
|
||||
return file_info
|
||||
|
||||
async def _find_local_file_entry(
|
||||
self,
|
||||
model_type: str,
|
||||
model_version_id: int,
|
||||
target_file: Dict[str, Any],
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Find a local library entry for a specific file of a model version.
|
||||
|
||||
Matches per design rule D2 (#1058): SHA256 is only compared when both
|
||||
sides carry a non-empty hash; otherwise fall back to (extension-less)
|
||||
file name equality. Never let two empty hashes compare equal.
|
||||
"""
|
||||
try:
|
||||
normalized_version_id = int(model_version_id)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
try:
|
||||
scanner = await self._get_scanner_for_model_type(model_type)
|
||||
cache = await scanner.get_cached_data()
|
||||
except Exception as exc:
|
||||
logger.debug(
|
||||
"Failed to scan local entries for version %s file check: %s",
|
||||
model_version_id,
|
||||
exc,
|
||||
)
|
||||
return None
|
||||
|
||||
raw_data = getattr(cache, "raw_data", None) if cache else None
|
||||
if not raw_data:
|
||||
return None
|
||||
|
||||
target_hash = str(
|
||||
(target_file.get("hashes") or {}).get("SHA256") or ""
|
||||
).strip().lower()
|
||||
target_name = str(target_file.get("name") or "").strip()
|
||||
target_base = os.path.splitext(target_name)[0] if target_name else ""
|
||||
|
||||
for item in raw_data:
|
||||
if not isinstance(item, dict):
|
||||
continue
|
||||
civitai_data = item.get("civitai")
|
||||
if not isinstance(civitai_data, dict):
|
||||
continue
|
||||
try:
|
||||
item_version_id = int(civitai_data.get("id"))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if item_version_id != normalized_version_id:
|
||||
continue
|
||||
|
||||
local_hash = str(item.get("sha256") or "").strip().lower()
|
||||
if target_hash and local_hash:
|
||||
if local_hash == target_hash:
|
||||
return item
|
||||
# Both sides carry hashes that differ: this is a different
|
||||
# file of the same version — do not fall back to name match.
|
||||
continue
|
||||
|
||||
if target_base:
|
||||
local_name = str(item.get("file_name") or "").strip()
|
||||
if local_name == target_base:
|
||||
return item
|
||||
|
||||
return None
|
||||
|
||||
async def download_from_civitai(
|
||||
self,
|
||||
model_id: int | None = None,
|
||||
@@ -242,6 +398,10 @@ class DownloadManager:
|
||||
Returns:
|
||||
Dict with download result
|
||||
"""
|
||||
# Normalize falsy file_params (e.g. an empty dict from API JSON
|
||||
# parsing) to None so gate conditions behave consistently (#1058).
|
||||
file_params = file_params or None
|
||||
|
||||
logger.debug(
|
||||
"[download] download_from_civitai called: model_id=%s, model_version_id=%s, "
|
||||
"source=%s, file_params=%s",
|
||||
@@ -816,6 +976,7 @@ class DownloadManager:
|
||||
version_info,
|
||||
record.get("model_version_id"),
|
||||
record.get("save_path") or record.get("file_path"),
|
||||
file_info=file_info,
|
||||
)
|
||||
await self._sync_downloaded_version(
|
||||
model_type,
|
||||
@@ -1152,9 +1313,13 @@ class DownloadManager:
|
||||
use_save_dir_as_root: bool = False,
|
||||
) -> Dict[str, Any]:
|
||||
"""Wrapper for original download_from_civitai implementation"""
|
||||
file_params = file_params or None
|
||||
try:
|
||||
# Check if model version already exists in library
|
||||
if model_version_id is not None:
|
||||
# Check if model version already exists in library.
|
||||
# With an explicit file selection (file_params) the version-level
|
||||
# check is deferred until after the metadata fetch, when the target
|
||||
# file can be resolved and checked individually (#1058).
|
||||
if model_version_id is not None and file_params is None:
|
||||
# Check both scanners
|
||||
lora_scanner = await self._get_lora_scanner()
|
||||
checkpoint_scanner = await self._get_checkpoint_scanner()
|
||||
@@ -1235,8 +1400,26 @@ class DownloadManager:
|
||||
except (TypeError, ValueError):
|
||||
resolved_version_id = None
|
||||
|
||||
# Resolve the explicitly selected file (if any) up front so the
|
||||
# existence gates and the actual file selection below always agree
|
||||
# on the target file (#1058).
|
||||
target_file: Optional[Dict[str, Any]] = None
|
||||
if file_params is not None:
|
||||
target_file = self._resolve_target_file(
|
||||
version_info.get("files") or [], file_params
|
||||
)
|
||||
if target_file is None:
|
||||
logger.warning(
|
||||
"[download] file_params provided but no file matched; "
|
||||
"falling back to version-level checks and primary file "
|
||||
"selection (model_version_id=%s)",
|
||||
resolved_version_id,
|
||||
)
|
||||
explicit_file = target_file is not None
|
||||
|
||||
if (
|
||||
get_settings_manager().get_skip_previously_downloaded_model_versions()
|
||||
not explicit_file
|
||||
and get_settings_manager().get_skip_previously_downloaded_model_versions()
|
||||
and resolved_version_id is not None
|
||||
and await self._has_been_downloaded(model_type, resolved_version_id)
|
||||
):
|
||||
@@ -1346,9 +1529,38 @@ class DownloadManager:
|
||||
f"baseModel '{base_model_value}' is a known diffusion model, routing to unet folder"
|
||||
)
|
||||
|
||||
# Case 2: model_version_id was None, check after getting version_info
|
||||
if model_version_id is None:
|
||||
version_id = version_info.get("id")
|
||||
# Existence check after the metadata fetch (#1058):
|
||||
# - An explicit file selection only blocks when THIS file is
|
||||
# already in the library; other files of the same version
|
||||
# remain downloadable.
|
||||
# - Without file_params (or when file_params failed to resolve),
|
||||
# keep version-level protection. The case "model_version_id
|
||||
# given + no file_params" was already covered by the early
|
||||
# gate above.
|
||||
if explicit_file and resolved_version_id is not None:
|
||||
existing_entry = await self._find_local_file_entry(
|
||||
model_type, resolved_version_id, target_file
|
||||
)
|
||||
if existing_entry is not None:
|
||||
error_message = (
|
||||
f"File '{target_file.get('name')}' from model version "
|
||||
f"{resolved_version_id} already exists in {model_type} library"
|
||||
)
|
||||
logger.info("[download] %s", error_message)
|
||||
return {"success": False, "error": error_message}
|
||||
logger.info(
|
||||
"[download] File '%s' of model version %s not in %s library — "
|
||||
"download allowed (other files of this version may exist locally)",
|
||||
target_file.get("name"), resolved_version_id, model_type,
|
||||
)
|
||||
elif file_params is not None or model_version_id is None:
|
||||
# Case 2: model_version_id was None, or file_params did not
|
||||
# resolve to a concrete file — check at version level.
|
||||
version_id = (
|
||||
resolved_version_id
|
||||
if resolved_version_id is not None
|
||||
else version_info.get("id")
|
||||
)
|
||||
|
||||
if model_type == "lora":
|
||||
# Check lora scanner
|
||||
@@ -1495,73 +1707,16 @@ class DownloadManager:
|
||||
files = version_info.get("files", [])
|
||||
file_info = None
|
||||
|
||||
# If file_params is provided, try to find matching file
|
||||
if file_params and model_version_id:
|
||||
target_file_id = file_params.get("id")
|
||||
target_type = file_params.get("type", "Model")
|
||||
target_format = file_params.get("format")
|
||||
target_size = file_params.get("size")
|
||||
target_fp = file_params.get("fp")
|
||||
is_primary = file_params.get("isPrimary", False)
|
||||
|
||||
logger.debug(
|
||||
"[download] file_params received: id=%s, type=%s, format=%s, size=%s, fp=%s, isPrimary=%s, "
|
||||
"model_version_id=%s, total_files=%d",
|
||||
target_file_id, target_type, target_format, target_size, target_fp, is_primary,
|
||||
model_version_id, len(files),
|
||||
)
|
||||
|
||||
if target_file_id:
|
||||
target_id_str = str(target_file_id)
|
||||
for f in files:
|
||||
f_id = f.get("id")
|
||||
if str(f_id) == target_id_str:
|
||||
file_info = f
|
||||
logger.debug(
|
||||
"[download] MATCH by ID: id=%s name='%s'",
|
||||
f_id, f.get("name"),
|
||||
)
|
||||
break
|
||||
if not file_info:
|
||||
logger.debug("[download] No file found with id=%s", target_file_id)
|
||||
|
||||
elif is_primary:
|
||||
file_info = next(
|
||||
(
|
||||
f
|
||||
for f in files
|
||||
if f.get("primary")
|
||||
and f.get("type") in MODEL_WEIGHT_FILE_TYPES
|
||||
),
|
||||
None,
|
||||
)
|
||||
else:
|
||||
# Lenient metadata match: only compare fields present on both sides
|
||||
for f in files:
|
||||
f_type = f.get("type", "")
|
||||
if f_type != target_type:
|
||||
continue
|
||||
|
||||
f_meta = f.get("metadata", {})
|
||||
f_format = f_meta.get("format") or f.get("format")
|
||||
f_size = f_meta.get("size") or f.get("size")
|
||||
f_fp = f_meta.get("fp") or f.get("fp")
|
||||
|
||||
if target_format and f_format != target_format:
|
||||
continue
|
||||
if target_size and f_size and f_size != target_size:
|
||||
continue
|
||||
if target_fp and f_fp and f_fp != target_fp:
|
||||
continue
|
||||
|
||||
file_info = f
|
||||
break
|
||||
|
||||
# If file_params is provided, reuse the file resolved right after
|
||||
# the metadata fetch so the existence gate and this selection
|
||||
# always agree on the target file (#1058).
|
||||
if file_params is not None:
|
||||
file_info = target_file
|
||||
if not file_info:
|
||||
logger.debug(
|
||||
"[download] No match found via file_params — falling back to primary file lookup",
|
||||
)
|
||||
elif not file_params:
|
||||
else:
|
||||
logger.debug(
|
||||
"[download] No file_params provided (null/None) — will use primary file lookup. "
|
||||
"model_version_id=%s, total_files=%d",
|
||||
@@ -1706,6 +1861,7 @@ class DownloadManager:
|
||||
version_info,
|
||||
model_version_id,
|
||||
save_path,
|
||||
file_info=file_info,
|
||||
)
|
||||
await self._sync_downloaded_version(
|
||||
model_type,
|
||||
@@ -1748,6 +1904,7 @@ class DownloadManager:
|
||||
version_info: Dict[str, Any],
|
||||
fallback_version_id=None,
|
||||
file_path: str | None = None,
|
||||
file_info: Dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
try:
|
||||
history_service = await ServiceRegistry.get_downloaded_version_history_service()
|
||||
@@ -1773,6 +1930,15 @@ class DownloadManager:
|
||||
if version_id is None:
|
||||
version_id = fallback_version_id
|
||||
|
||||
# Per-file identity for multi-file versions (#1058)
|
||||
file_id = None
|
||||
file_name = None
|
||||
if isinstance(file_info, dict):
|
||||
file_id = file_info.get("id")
|
||||
raw_file_name = file_info.get("name")
|
||||
if isinstance(raw_file_name, str) and raw_file_name.strip():
|
||||
file_name = raw_file_name.strip()
|
||||
|
||||
try:
|
||||
await history_service.mark_downloaded(
|
||||
model_type,
|
||||
@@ -1780,6 +1946,8 @@ class DownloadManager:
|
||||
model_id=int(cast(Any, resolved_model_id)) if resolved_model_id is not None else None,
|
||||
source="download",
|
||||
file_path=file_path,
|
||||
file_id=file_id,
|
||||
file_name=file_name,
|
||||
)
|
||||
except (TypeError, ValueError):
|
||||
logger.debug(
|
||||
|
||||
@@ -64,6 +64,7 @@ class DownloadQueueService:
|
||||
model_name TEXT NOT NULL DEFAULT '',
|
||||
version_name TEXT DEFAULT '',
|
||||
thumbnail_url TEXT DEFAULT '',
|
||||
file_params TEXT,
|
||||
status TEXT NOT NULL,
|
||||
error TEXT,
|
||||
file_path TEXT,
|
||||
@@ -120,6 +121,18 @@ class DownloadQueueService:
|
||||
with self._connect() as conn:
|
||||
conn.executescript(self._SCHEMA_TABLES)
|
||||
|
||||
# Databases created by older versions lack
|
||||
# download_history.file_params; add it so retry-from-history can
|
||||
# restore the originally selected file (#1058).
|
||||
history_columns = {
|
||||
row["name"]
|
||||
for row in conn.execute("PRAGMA table_info(download_history)")
|
||||
}
|
||||
if "file_params" not in history_columns:
|
||||
conn.execute(
|
||||
"ALTER TABLE download_history ADD COLUMN file_params TEXT"
|
||||
)
|
||||
|
||||
# Creating the unique index on download_history.download_id can
|
||||
# fail if pre-existing rows have duplicate values (e.g. from a
|
||||
# previous version that lacked the index). Deduplicate first so
|
||||
@@ -418,6 +431,12 @@ class DownloadQueueService:
|
||||
return None
|
||||
|
||||
now = completed_at if completed_at is not None else time.time()
|
||||
# Guard against legacy databases whose download_queue table
|
||||
# predates the file_params column.
|
||||
queue_columns = set(row.keys())
|
||||
file_params_json = (
|
||||
row["file_params"] if "file_params" in queue_columns else None
|
||||
)
|
||||
conn.execute(
|
||||
"DELETE FROM download_queue WHERE download_id = ?",
|
||||
(download_id,),
|
||||
@@ -426,9 +445,9 @@ class DownloadQueueService:
|
||||
"""
|
||||
INSERT OR IGNORE INTO download_history (
|
||||
download_id, model_id, model_version_id, model_name,
|
||||
version_name, thumbnail_url, status, error, file_path,
|
||||
bytes_downloaded, total_bytes, completed_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
version_name, thumbnail_url, file_params, status, error,
|
||||
file_path, bytes_downloaded, total_bytes, completed_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
row["download_id"],
|
||||
@@ -437,6 +456,7 @@ class DownloadQueueService:
|
||||
row["model_name"],
|
||||
row["version_name"],
|
||||
row["thumbnail_url"],
|
||||
file_params_json,
|
||||
status,
|
||||
error,
|
||||
file_path,
|
||||
@@ -503,6 +523,7 @@ class DownloadQueueService:
|
||||
bytes_downloaded: int = 0,
|
||||
total_bytes: Optional[int] = None,
|
||||
is_already_exists: int = 0,
|
||||
file_params: Optional[dict[str, Any]] = None,
|
||||
) -> int:
|
||||
"""Insert a record into the download history.
|
||||
|
||||
@@ -510,6 +531,7 @@ class DownloadQueueService:
|
||||
inserted row.
|
||||
"""
|
||||
now = time.time()
|
||||
file_params_json = json.dumps(file_params) if file_params is not None else None
|
||||
|
||||
async with self._lock:
|
||||
conn = self._get_conn()
|
||||
@@ -517,9 +539,10 @@ class DownloadQueueService:
|
||||
"""
|
||||
INSERT INTO download_history (
|
||||
download_id, model_id, model_version_id, model_name,
|
||||
version_name, thumbnail_url, status, error, file_path,
|
||||
bytes_downloaded, total_bytes, completed_at, is_already_exists
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
version_name, thumbnail_url, file_params, status, error,
|
||||
file_path, bytes_downloaded, total_bytes, completed_at,
|
||||
is_already_exists
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(
|
||||
download_id,
|
||||
@@ -528,6 +551,7 @@ class DownloadQueueService:
|
||||
model_name,
|
||||
version_name,
|
||||
thumbnail_url,
|
||||
file_params_json,
|
||||
status,
|
||||
error,
|
||||
file_path,
|
||||
@@ -702,7 +726,7 @@ class DownloadQueueService:
|
||||
download_id, model_id, model_version_id, model_name,
|
||||
version_name, thumbnail_url, source, file_params,
|
||||
status, priority, added_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'queued', 0, ?)
|
||||
""",
|
||||
(
|
||||
new_id,
|
||||
@@ -712,6 +736,7 @@ class DownloadQueueService:
|
||||
row["version_name"],
|
||||
row["thumbnail_url"],
|
||||
"retry",
|
||||
row["file_params"],
|
||||
now,
|
||||
),
|
||||
)
|
||||
@@ -755,7 +780,7 @@ class DownloadQueueService:
|
||||
download_id, model_id, model_version_id, model_name,
|
||||
version_name, thumbnail_url, source, file_params,
|
||||
status, priority, added_at
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, NULL, 'queued', 0, ?)
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, 'queued', 0, ?)
|
||||
""",
|
||||
(
|
||||
new_id,
|
||||
@@ -765,6 +790,7 @@ class DownloadQueueService:
|
||||
row["version_name"],
|
||||
row["thumbnail_url"],
|
||||
"retry",
|
||||
row["file_params"],
|
||||
now,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -62,6 +62,14 @@ class DownloadedVersionHistoryService:
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_downloaded_model_versions_model
|
||||
ON downloaded_model_versions(model_type, model_id);
|
||||
CREATE TABLE IF NOT EXISTS downloaded_version_files (
|
||||
model_type TEXT NOT NULL,
|
||||
version_id INTEGER NOT NULL,
|
||||
file_id INTEGER NOT NULL,
|
||||
file_name TEXT,
|
||||
downloaded_at REAL NOT NULL,
|
||||
PRIMARY KEY (model_type, version_id, file_id)
|
||||
);
|
||||
"""
|
||||
|
||||
def __init__(self, db_path: str | None = None, *, settings_manager=None) -> None:
|
||||
@@ -131,10 +139,13 @@ class DownloadedVersionHistoryService:
|
||||
source: str = "manual",
|
||||
file_path: str | None = None,
|
||||
library_name: str | None = None,
|
||||
file_id: int | None = None,
|
||||
file_name: str | None = None,
|
||||
) -> None:
|
||||
normalized_type = _normalize_model_type(model_type)
|
||||
normalized_version_id = _normalize_int(version_id)
|
||||
normalized_model_id = _normalize_int(model_id)
|
||||
normalized_file_id = _normalize_int(file_id)
|
||||
if normalized_type is None or normalized_version_id is None:
|
||||
return
|
||||
|
||||
@@ -168,6 +179,25 @@ class DownloadedVersionHistoryService:
|
||||
active_library_name,
|
||||
),
|
||||
)
|
||||
if normalized_file_id is not None:
|
||||
# Per-file history for multi-file versions (#1058)
|
||||
conn.execute(
|
||||
"""
|
||||
INSERT INTO downloaded_version_files (
|
||||
model_type, version_id, file_id, file_name, downloaded_at
|
||||
) VALUES (?, ?, ?, ?, ?)
|
||||
ON CONFLICT(model_type, version_id, file_id) DO UPDATE SET
|
||||
file_name = COALESCE(excluded.file_name, downloaded_version_files.file_name),
|
||||
downloaded_at = excluded.downloaded_at
|
||||
""",
|
||||
(
|
||||
normalized_type,
|
||||
normalized_version_id,
|
||||
normalized_file_id,
|
||||
file_name,
|
||||
timestamp,
|
||||
),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
async def mark_downloaded_bulk(
|
||||
@@ -255,8 +285,63 @@ class DownloadedVersionHistoryService:
|
||||
self._get_active_library_name(),
|
||||
),
|
||||
)
|
||||
# Whole-version deletion also clears the per-file records (#1058)
|
||||
conn.execute(
|
||||
"""
|
||||
DELETE FROM downloaded_version_files
|
||||
WHERE model_type = ? AND version_id = ?
|
||||
""",
|
||||
(normalized_type, normalized_version_id),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
async def mark_file_deleted(
|
||||
self, model_type: str, version_id: int, file_id: int
|
||||
) -> None:
|
||||
"""Drop a single file record of a version, keeping siblings (#1058)."""
|
||||
normalized_type = _normalize_model_type(model_type)
|
||||
normalized_version_id = _normalize_int(version_id)
|
||||
normalized_file_id = _normalize_int(file_id)
|
||||
if (
|
||||
normalized_type is None
|
||||
or normalized_version_id is None
|
||||
or normalized_file_id is None
|
||||
):
|
||||
return
|
||||
|
||||
async with self._lock:
|
||||
conn = self._get_conn()
|
||||
conn.execute(
|
||||
"""
|
||||
DELETE FROM downloaded_version_files
|
||||
WHERE model_type = ? AND version_id = ? AND file_id = ?
|
||||
""",
|
||||
(normalized_type, normalized_version_id, normalized_file_id),
|
||||
)
|
||||
conn.commit()
|
||||
|
||||
async def get_downloaded_file_ids(
|
||||
self, model_type: str, version_id: int
|
||||
) -> list[int]:
|
||||
"""Return the CivitAI file ids recorded as downloaded for a version."""
|
||||
normalized_type = _normalize_model_type(model_type)
|
||||
normalized_version_id = _normalize_int(version_id)
|
||||
if normalized_type is None or normalized_version_id is None:
|
||||
return []
|
||||
|
||||
async with self._lock:
|
||||
conn = self._get_conn()
|
||||
rows = conn.execute(
|
||||
"""
|
||||
SELECT file_id
|
||||
FROM downloaded_version_files
|
||||
WHERE model_type = ? AND version_id = ?
|
||||
ORDER BY file_id ASC
|
||||
""",
|
||||
(normalized_type, normalized_version_id),
|
||||
).fetchall()
|
||||
return [int(row["file_id"]) for row in rows]
|
||||
|
||||
async def has_been_downloaded(self, model_type: str, version_id: int) -> bool:
|
||||
normalized_type = _normalize_model_type(model_type)
|
||||
normalized_version_id = _normalize_int(version_id)
|
||||
|
||||
@@ -35,6 +35,10 @@ class ModelCache:
|
||||
folders: List[str]
|
||||
version_index: Dict[int, Dict[str, Any]] = field(default_factory=dict)
|
||||
model_id_index: Dict[int, List[Dict[str, Any]]] = field(default_factory=dict)
|
||||
# Multi-valued companion to version_index: every local file entry of a
|
||||
# CivitAI model version, so versions with several downloaded files stay
|
||||
# consistent (#1058).
|
||||
version_files_index: Dict[int, List[Dict[str, Any]]] = field(default_factory=dict)
|
||||
name_display_mode: str = "model_name"
|
||||
_lock: Any = field(init=False, repr=False, default=None)
|
||||
# Cache for last sort: (sort_key, order, seed) -> sorted list
|
||||
@@ -116,6 +120,7 @@ class ModelCache:
|
||||
|
||||
self.version_index = {}
|
||||
self.model_id_index = {}
|
||||
self.version_files_index = {}
|
||||
for item in self.raw_data:
|
||||
self.add_to_version_index(item)
|
||||
|
||||
@@ -132,6 +137,17 @@ class ModelCache:
|
||||
|
||||
self.version_index[version_id] = item
|
||||
|
||||
# Register in the multi-valued index, deduplicated by file_path (#1058)
|
||||
files = self.version_files_index.setdefault(version_id, [])
|
||||
for entry in files:
|
||||
if entry is item or (
|
||||
isinstance(entry, dict)
|
||||
and entry.get('file_path') == item.get('file_path')
|
||||
):
|
||||
break
|
||||
else:
|
||||
files.append(item)
|
||||
|
||||
model_id = self._normalize_version_id(civitai_data.get('modelId'))
|
||||
if model_id is None:
|
||||
return
|
||||
@@ -159,12 +175,37 @@ class ModelCache:
|
||||
if version_id is None:
|
||||
return
|
||||
|
||||
# Drop only this file's entry from the multi-valued index (#1058)
|
||||
files = self.version_files_index.get(version_id)
|
||||
if files:
|
||||
remaining = [
|
||||
entry
|
||||
for entry in files
|
||||
if not (
|
||||
entry is item
|
||||
or (
|
||||
isinstance(entry, dict)
|
||||
and entry.get('file_path') == item.get('file_path')
|
||||
)
|
||||
)
|
||||
]
|
||||
if remaining:
|
||||
self.version_files_index[version_id] = remaining
|
||||
else:
|
||||
self.version_files_index.pop(version_id, None)
|
||||
|
||||
# A surviving sibling file keeps the version present in the indexes
|
||||
sibling = (self.version_files_index.get(version_id) or [None])[0]
|
||||
|
||||
existing = self.version_index.get(version_id)
|
||||
if existing is item or (
|
||||
isinstance(existing, dict)
|
||||
and existing.get('file_path') == item.get('file_path')
|
||||
):
|
||||
self.version_index.pop(version_id, None)
|
||||
if sibling is not None:
|
||||
self.version_index[version_id] = sibling
|
||||
else:
|
||||
self.version_index.pop(version_id, None)
|
||||
|
||||
model_id = self._normalize_version_id(civitai_data.get('modelId'))
|
||||
if model_id is None:
|
||||
@@ -174,6 +215,20 @@ class ModelCache:
|
||||
if not versions:
|
||||
return
|
||||
|
||||
if sibling is not None:
|
||||
# Update the descriptor to reflect the surviving sibling file
|
||||
descriptor = self._build_version_descriptor(
|
||||
sibling,
|
||||
sibling.get('civitai') if isinstance(sibling, dict) else {},
|
||||
version_id,
|
||||
)
|
||||
for index, existing_desc in enumerate(versions):
|
||||
if existing_desc.get('versionId') == version_id:
|
||||
if descriptor is not None:
|
||||
versions[index] = descriptor
|
||||
break
|
||||
return
|
||||
|
||||
filtered = [v for v in versions if v.get('versionId') != version_id]
|
||||
if filtered:
|
||||
self.model_id_index[model_id] = filtered
|
||||
@@ -206,6 +261,15 @@ class ModelCache:
|
||||
versions = self.model_id_index.get(normalized_id, [])
|
||||
return [dict(version) for version in versions]
|
||||
|
||||
def get_files_by_version_id(self, version_id: Any) -> List[Dict[str, Any]]:
|
||||
"""Return every local file entry for a CivitAI model version (#1058)."""
|
||||
|
||||
normalized_id = self._normalize_version_id(version_id)
|
||||
if normalized_id is None:
|
||||
return []
|
||||
|
||||
return list(self.version_files_index.get(normalized_id, []))
|
||||
|
||||
async def resort(self):
|
||||
"""Resort cached data according to last sort mode if set"""
|
||||
async with self._lock:
|
||||
|
||||
@@ -25,6 +25,28 @@ from .cache_health_monitor import CacheHealthMonitor, CacheHealthStatus
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Canonical set of weight-file extensions stripped when normalizing model
|
||||
# names for matching (ModelScanner.find_matching_models and the recipe rematch
|
||||
# filename key share this set). It is the union of the LoRA scanner set
|
||||
# ({".safetensors"}) and the Checkpoint scanner set (ComfyUI's
|
||||
# supported_pt_extensions plus ".gguf") so type-blind lookups (lora +
|
||||
# checkpoint merged) cover every format either scanner indexes. ".safebin"
|
||||
# is deliberately absent — no scanner indexes it, so a recipe entry
|
||||
# "model.safebin" must not be bound to a local "model.safetensors".
|
||||
WEIGHT_FILE_EXTENSIONS = frozenset(
|
||||
{
|
||||
".safetensors",
|
||||
".ckpt",
|
||||
".pt",
|
||||
".pt2",
|
||||
".bin",
|
||||
".pth",
|
||||
".pkl",
|
||||
".sft",
|
||||
".gguf",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _is_excluded_dir(name: str) -> bool:
|
||||
"""Return True when a directory entry must be skipped during model walks.
|
||||
@@ -2140,8 +2162,98 @@ class ModelScanner:
|
||||
return sorted_models
|
||||
return sorted_models[:limit]
|
||||
|
||||
async def get_model_info_by_name(self, name):
|
||||
"""Get model information by name"""
|
||||
@staticmethod
|
||||
def find_matching_models(
|
||||
raw_data: List[Dict[str, Any]],
|
||||
name: str,
|
||||
*,
|
||||
base_model: Optional[str] = None,
|
||||
extensions: Optional[Set[str]] = None,
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return all cached models matching ``name`` (case-insensitive).
|
||||
|
||||
A name containing a path separator must equal the model's
|
||||
folder-relative path; a bare name matches on basename. When
|
||||
``base_model`` is given, confident mismatches are rejected while
|
||||
unknowns on either side stay eligible (lenient guard).
|
||||
``extensions`` should be the scanner's own ``file_extensions`` so
|
||||
suffix stripping only covers formats the scanner actually indexes;
|
||||
when omitted, the shared :data:`WEIGHT_FILE_EXTENSIONS` set is used.
|
||||
"""
|
||||
# Longest first so overlapping suffixes strip correctly.
|
||||
exts = sorted(extensions or WEIGHT_FILE_EXTENSIONS, key=len, reverse=True)
|
||||
|
||||
normalized_name = str(name).replace("\\", "/").casefold()
|
||||
for ext in exts:
|
||||
if normalized_name.endswith(ext):
|
||||
normalized_name = normalized_name[: -len(ext)]
|
||||
break
|
||||
has_path = "/" in normalized_name
|
||||
basename = normalized_name.rsplit("/", 1)[-1]
|
||||
|
||||
matches = []
|
||||
for model in raw_data:
|
||||
file_name = str(model.get("file_name") or "").replace("\\", "/")
|
||||
folder = str(model.get("folder") or "").replace("\\", "/").strip("/")
|
||||
model_path = f"{folder}/{file_name}" if folder else file_name
|
||||
for ext in exts:
|
||||
if model_path.casefold().endswith(ext):
|
||||
model_path = model_path[: -len(ext)]
|
||||
break
|
||||
if (has_path and model_path.casefold() == normalized_name) or (
|
||||
not has_path and model_path.rsplit("/", 1)[-1].casefold() == basename
|
||||
):
|
||||
matches.append(model)
|
||||
|
||||
expected_base = str(base_model or "").strip().casefold()
|
||||
if expected_base and expected_base != "unknown":
|
||||
matches = [
|
||||
model
|
||||
for model in matches
|
||||
if str(model.get("base_model") or "").strip().casefold()
|
||||
in ("", "unknown", expected_base)
|
||||
]
|
||||
return matches
|
||||
|
||||
async def find_models_by_name(
|
||||
self, name: str, *, base_model: Optional[str] = None
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return every cached model matching ``name`` (see ``find_matching_models``)."""
|
||||
try:
|
||||
cache = await self.get_cached_data()
|
||||
return self.find_matching_models(
|
||||
cache.raw_data,
|
||||
name,
|
||||
base_model=base_model,
|
||||
extensions=self.file_extensions,
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Error finding models by name: {e}", exc_info=True)
|
||||
return []
|
||||
|
||||
async def get_model_info_by_name(
|
||||
self,
|
||||
name: str,
|
||||
*,
|
||||
require_unique: bool = False,
|
||||
base_model: Optional[str] = None,
|
||||
):
|
||||
"""Get model information by name.
|
||||
|
||||
Default mode keeps the legacy first-match/fallback semantics. With
|
||||
``require_unique`` an ambiguous name is a miss, and ``base_model``
|
||||
rejects confident base-model mismatches (unknowns stay eligible).
|
||||
"""
|
||||
if require_unique or base_model:
|
||||
try:
|
||||
matches = await self.find_models_by_name(name, base_model=base_model)
|
||||
if require_unique and len(matches) != 1:
|
||||
return None
|
||||
return matches[0] if matches else None
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting model info by name: {e}", exc_info=True)
|
||||
return None
|
||||
|
||||
try:
|
||||
cache = await self.get_cached_data()
|
||||
|
||||
@@ -2446,6 +2558,39 @@ class ModelScanner:
|
||||
logger.error(f"Error checking model version existence: {e}")
|
||||
return False
|
||||
|
||||
async def get_files_for_version(self, model_version_id: int) -> List[Dict[str, Any]]:
|
||||
"""Get all local file entries for a specific model version (#1058).
|
||||
|
||||
A Civitai model version can have several weight files downloaded;
|
||||
unlike the single-valued version_index this returns every entry.
|
||||
|
||||
Args:
|
||||
model_version_id: Civitai model version ID
|
||||
|
||||
Returns:
|
||||
List[Dict]: Cache entries (may be empty)
|
||||
"""
|
||||
try:
|
||||
normalized_id = int(model_version_id)
|
||||
except (TypeError, ValueError):
|
||||
return []
|
||||
|
||||
try:
|
||||
cache = await self.get_cached_data()
|
||||
if not cache:
|
||||
return []
|
||||
|
||||
getter = getattr(cache, "get_files_by_version_id", None)
|
||||
if getter is not None:
|
||||
return getter(normalized_id)
|
||||
|
||||
# Fallback for cache implementations without the multi-file index
|
||||
entry = cache.version_index.get(normalized_id)
|
||||
return [entry] if entry is not None else []
|
||||
except Exception as e:
|
||||
logger.error(f"Error getting files for model version: {e}")
|
||||
return []
|
||||
|
||||
async def get_model_versions_by_id(self, model_id: int) -> List[Dict[str, Any]]:
|
||||
"""Get all versions of a model by its ID
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ from ..config import config
|
||||
from ..utils.constants import VALID_CHECKPOINT_SUB_TYPES, VALID_LORA_TYPES
|
||||
from ..utils.file_utils import calculate_autov3
|
||||
from ..utils.recipe_open_stats import RecipeOpenStats
|
||||
from .model_scanner import WEIGHT_FILE_EXTENSIONS
|
||||
from .recipe_cache import RecipeCache
|
||||
from .recipes.errors import RecipeNotFoundError, RecipePersistenceError
|
||||
from natsort import natsorted
|
||||
@@ -36,11 +37,6 @@ logger = logging.getLogger(__name__)
|
||||
# explicitly to "diffusion_model" (mirrors Oracle R2-F1).
|
||||
_CHECKPOINT_MODEL_TYPE_ALIASES = {"diffusionmodel": "diffusion_model"}
|
||||
|
||||
# Known weight-file extensions stripped by _normalize_filename_key. Names are
|
||||
# stored extensionless on both sides, so splitext would misread dotted stems
|
||||
# ("my.mix" -> "my") and silently collide distinct models.
|
||||
_WEIGHT_FILE_EXTS = (".safetensors", ".ckpt", ".pt", ".pth", ".gguf", ".bin", ".safebin", ".sft")
|
||||
|
||||
|
||||
class RecipeScanner:
|
||||
"""Service for scanning and managing recipe images"""
|
||||
@@ -179,13 +175,15 @@ class RecipeScanner:
|
||||
|
||||
Only known weight-file extensions are stripped — names are stored
|
||||
extensionless on both sides, so splitext would misread dotted stems
|
||||
("my.mix" -> "my") and collide distinct models.
|
||||
("my.mix" -> "my") and collide distinct models. The extension set is
|
||||
shared with ModelScanner.find_matching_models, and is iterated longest
|
||||
first to keep the strip ordering identical to that function.
|
||||
"""
|
||||
if not name:
|
||||
return ""
|
||||
basename = os.path.basename(name.replace("\\", "/"))
|
||||
lower = basename.lower()
|
||||
for ext in _WEIGHT_FILE_EXTS:
|
||||
for ext in sorted(WEIGHT_FILE_EXTENSIONS, key=len, reverse=True):
|
||||
if lower.endswith(ext):
|
||||
basename = basename[: -len(ext)]
|
||||
break
|
||||
@@ -2926,13 +2924,45 @@ class RecipeScanner:
|
||||
|
||||
return normalized
|
||||
|
||||
async def get_local_lora(self, name: str) -> Optional[Dict[str, Any]]:
|
||||
"""Lookup a local LoRA model by name."""
|
||||
async def get_local_lora(
|
||||
self, name: str, base_model: Optional[str] = None
|
||||
) -> Optional[Dict[str, Any]]:
|
||||
"""Lookup an unambiguous local LoRA by name and optional base model."""
|
||||
|
||||
if not self._lora_scanner or not name:
|
||||
return None
|
||||
|
||||
return await self._lora_scanner.get_model_info_by_name(name)
|
||||
return await self._lora_scanner.get_model_info_by_name(
|
||||
name, require_unique=True, base_model=base_model
|
||||
)
|
||||
|
||||
async def find_local_loras_by_name(
|
||||
self, name: str, base_model: Optional[str] = None
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return every local LoRA matching ``name`` (used to explain lookup misses)."""
|
||||
|
||||
if not self._lora_scanner or not name:
|
||||
return []
|
||||
|
||||
return await self._lora_scanner.find_models_by_name(name, base_model=base_model)
|
||||
|
||||
async def get_local_lora_by_hash(self, hash_value: str) -> Optional[Dict[str, Any]]:
|
||||
"""Lookup a local LoRA through the scanner's hash index."""
|
||||
|
||||
if not self._lora_scanner or not hash_value:
|
||||
return None
|
||||
|
||||
file_path = self._lora_scanner.get_path_by_hash(hash_value)
|
||||
if not file_path:
|
||||
return None
|
||||
|
||||
target_path = os.path.normcase(os.path.abspath(file_path))
|
||||
cached_data = await self._lora_scanner.get_cached_data()
|
||||
for model in cached_data.raw_data:
|
||||
model_path = model.get("file_path")
|
||||
if model_path and os.path.normcase(os.path.abspath(model_path)) == target_path:
|
||||
return model
|
||||
return None
|
||||
|
||||
async def get_local_checkpoint(self, name: str) -> Optional[Dict[str, Any]]:
|
||||
"""Lookup a local checkpoint model by name."""
|
||||
|
||||
@@ -426,8 +426,21 @@ class RecipePersistenceService:
|
||||
if not recipe_path or not os.path.exists(recipe_path):
|
||||
raise RecipeNotFoundError("Recipe not found")
|
||||
|
||||
target_lora = await recipe_scanner.get_local_lora(target_name)
|
||||
with open(recipe_path, "r", encoding="utf-8") as file_obj:
|
||||
recipe_base_model = json.load(file_obj).get("base_model", "")
|
||||
|
||||
target_lora = await recipe_scanner.get_local_lora(target_name, recipe_base_model)
|
||||
if not target_lora:
|
||||
matches = await recipe_scanner.find_local_loras_by_name(target_name)
|
||||
if len(matches) > 1:
|
||||
raise RecipeValidationError(
|
||||
f"Multiple local LoRAs match '{target_name}'; "
|
||||
"include the folder path to disambiguate"
|
||||
)
|
||||
if len(matches) == 1:
|
||||
raise RecipeValidationError(
|
||||
f"Local LoRA '{target_name}' has a different base model than the recipe"
|
||||
)
|
||||
raise RecipeNotFoundError(f"Local LoRA not found with name: {target_name}")
|
||||
|
||||
recipe_data, updated_lora = await recipe_scanner.update_lora_entry(
|
||||
|
||||
@@ -603,6 +603,51 @@
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.file-option-radio input[type="checkbox"] {
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
accent-color: var(--lora-accent);
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
/* Files already in the library are greyed out and not clickable */
|
||||
.file-option.disabled {
|
||||
opacity: 0.55;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.file-option.disabled:hover {
|
||||
border-color: var(--border-color);
|
||||
box-shadow: none;
|
||||
transform: none;
|
||||
}
|
||||
|
||||
.file-option.disabled input[type="checkbox"] {
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
/* Options of the other routing group are temporarily disabled once a
|
||||
selection is made (mixed-type multi-select is not allowed) */
|
||||
.file-option.group-disabled {
|
||||
opacity: 0.6;
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.file-option.group-disabled:hover {
|
||||
border-color: var(--border-color);
|
||||
box-shadow: none;
|
||||
transform: none;
|
||||
}
|
||||
|
||||
.file-option.group-disabled input[type="checkbox"] {
|
||||
cursor: not-allowed;
|
||||
}
|
||||
|
||||
.file-tag.in-library {
|
||||
background: oklch(var(--lora-accent) / 0.15);
|
||||
color: var(--lora-accent);
|
||||
}
|
||||
|
||||
.file-option-info {
|
||||
flex: 1;
|
||||
min-width: 0;
|
||||
|
||||
@@ -573,46 +573,53 @@ function renderRow(version, options) {
|
||||
);
|
||||
|
||||
const actions = [];
|
||||
if (!version.isInLibrary) {
|
||||
const canDownload = isDownloadAllowed(version);
|
||||
const downloadIcon = isEarlyAccess ? '<i class="fas fa-bolt"></i> ' : '';
|
||||
let downloadTitle;
|
||||
if (!canDownload) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadNotAllowedTooltip',
|
||||
{},
|
||||
'This version is only available for on-site generation on Civitai'
|
||||
);
|
||||
} else if (isPaidPermanent(version)) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadPaidTooltip',
|
||||
{},
|
||||
'Download this paid version from Civitai'
|
||||
);
|
||||
} else if (isEarlyAccess) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadEarlyAccessTooltip',
|
||||
{},
|
||||
'Download this early access version from Civitai'
|
||||
);
|
||||
} else {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadTooltip',
|
||||
{},
|
||||
'Download this version'
|
||||
);
|
||||
const canDownload = isDownloadAllowed(version);
|
||||
const downloadIcon = isEarlyAccess ? '<i class="fas fa-bolt"></i> ' : '';
|
||||
let downloadTitle;
|
||||
if (!canDownload) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadNotAllowedTooltip',
|
||||
{},
|
||||
'This version is only available for on-site generation on Civitai'
|
||||
);
|
||||
} else if (version.isInLibrary) {
|
||||
// In-library versions may still have undownloaded weight files; the
|
||||
// download modal's file dialog decides what remains (#1058).
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadRemainingTooltip',
|
||||
{},
|
||||
'Download remaining files of this version'
|
||||
);
|
||||
} else if (isPaidPermanent(version)) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadPaidTooltip',
|
||||
{},
|
||||
'Download this paid version from Civitai'
|
||||
);
|
||||
} else if (isEarlyAccess) {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadEarlyAccessTooltip',
|
||||
{},
|
||||
'Download this early access version from Civitai'
|
||||
);
|
||||
} else {
|
||||
downloadTitle = translate(
|
||||
'modals.model.versions.actions.downloadTooltip',
|
||||
{},
|
||||
'Download this version'
|
||||
);
|
||||
}
|
||||
actions.push(buildActionButton(
|
||||
downloadLabel,
|
||||
canDownload ? 'version-action-primary' : 'version-action-disabled',
|
||||
canDownload ? 'download' : '',
|
||||
{
|
||||
title: downloadTitle,
|
||||
iconMarkup: downloadIcon,
|
||||
disabled: !canDownload,
|
||||
}
|
||||
actions.push(buildActionButton(
|
||||
downloadLabel,
|
||||
canDownload ? 'version-action-primary' : 'version-action-disabled',
|
||||
canDownload ? 'download' : '',
|
||||
{
|
||||
title: downloadTitle,
|
||||
iconMarkup: downloadIcon,
|
||||
disabled: !canDownload,
|
||||
}
|
||||
));
|
||||
} else if (version.filePath) {
|
||||
));
|
||||
if (version.isInLibrary && version.filePath) {
|
||||
actions.push(buildActionButton(
|
||||
deleteLabel,
|
||||
'version-action-danger',
|
||||
@@ -1422,6 +1429,15 @@ export function initVersionsTab({
|
||||
button.disabled = true;
|
||||
|
||||
try {
|
||||
// In-library versions may still have undownloaded weight files
|
||||
// (#1058). The tab payload has no per-file state, so open the
|
||||
// download modal's file dialog, which refetches the full version
|
||||
// payload and shows what remains.
|
||||
if (version.isInLibrary) {
|
||||
await downloadManager.openFileSelectionForVersion(modelType, modelId, versionId);
|
||||
return;
|
||||
}
|
||||
|
||||
const pathInfo = await resolveDownloadPathFromCurrentVersion();
|
||||
const resolveTemplatePath = shouldResolveTemplatePath(version, pathInfo);
|
||||
const success = await downloadManager.downloadVersionWithDefaults(modelType, modelId, versionId, {
|
||||
|
||||
@@ -25,6 +25,12 @@ export class DownloadManager {
|
||||
this.apiClient = null;
|
||||
this.useDefaultPath = false;
|
||||
|
||||
// Multi-file selection state: selectedFile stays the first selected
|
||||
// file for backward compatibility with single-file flows (#1058).
|
||||
this.selectedFile = null;
|
||||
this.selectedFiles = [];
|
||||
this._lastDownloadError = null;
|
||||
|
||||
// Batch mode state
|
||||
this.batchModels = [];
|
||||
this.isBatchMode = false;
|
||||
@@ -160,6 +166,8 @@ export class DownloadManager {
|
||||
this.modelVersionId = null;
|
||||
this.source = null;
|
||||
this.selectedFile = null;
|
||||
this.selectedFiles = [];
|
||||
this._lastDownloadError = null;
|
||||
this._isDiffusionModel = false;
|
||||
|
||||
this.selectedFolder = '';
|
||||
@@ -546,6 +554,64 @@ export class DownloadManager {
|
||||
await this.fetchVersionsForCurrentModel();
|
||||
}
|
||||
|
||||
/**
|
||||
* Open the download modal directly on the file-selection step for a
|
||||
* specific model version (#1058). Used by entry points (e.g.
|
||||
* ModelVersionsTab) whose version payloads lack per-file downloaded
|
||||
* state, so the full versions payload is fetched here first.
|
||||
*/
|
||||
async openFileSelectionForVersion(modelType, modelId, versionId, { source = null } = {}) {
|
||||
try {
|
||||
this.apiClient = getModelApiClient(modelType);
|
||||
} catch (error) {
|
||||
this.apiClient = getModelApiClient();
|
||||
}
|
||||
|
||||
this.showDownloadModal();
|
||||
|
||||
this.modelId = modelId ? modelId.toString() : null;
|
||||
this.modelVersionId = versionId ? versionId.toString() : null;
|
||||
this.source = source;
|
||||
|
||||
if (!this.modelId) {
|
||||
return;
|
||||
}
|
||||
|
||||
try {
|
||||
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingVersions'));
|
||||
await this.retrieveVersionsForModel(this.modelId, this.source);
|
||||
} catch (error) {
|
||||
showToast('toast.downloads.loadError', { message: error.message }, 'error');
|
||||
return;
|
||||
} finally {
|
||||
this.loadingManager.hide();
|
||||
}
|
||||
|
||||
const version = this.versions.find(v => v.id.toString() === this.modelVersionId);
|
||||
if (!version) {
|
||||
console.warn('[download] openFileSelectionForVersion: version %s not found for model %s',
|
||||
this.modelVersionId, this.modelId);
|
||||
this.showVersionStep();
|
||||
return;
|
||||
}
|
||||
|
||||
const hasRemainingFiles = this._getWeightFiles(version).length > 1
|
||||
&& this._getRemainingFiles(version).length > 0;
|
||||
|
||||
if (hasRemainingFiles) {
|
||||
this.showFileSelectionStep(version.id);
|
||||
return;
|
||||
}
|
||||
|
||||
// Nothing left to download for this version (single file or all
|
||||
// files already in the library) — fall back to the version step.
|
||||
if (version.existsLocally) {
|
||||
showToast('toast.loras.versionExists', {}, 'info');
|
||||
}
|
||||
this.currentVersion = version;
|
||||
this.showVersionStep();
|
||||
}
|
||||
|
||||
showVersionStep() {
|
||||
document.getElementById('urlStep').style.display = 'none';
|
||||
document.getElementById('versionStep').style.display = 'block';
|
||||
@@ -595,7 +661,10 @@ export class DownloadManager {
|
||||
</div>`;
|
||||
}
|
||||
|
||||
const fileBadge = modelFiles.length > 1 && !existsLocally
|
||||
// Always offer the file-selection entry for multi-file versions,
|
||||
// even when the version is already (partially) in the library, so
|
||||
// remaining files can still be downloaded (#1058).
|
||||
const fileBadge = modelFiles.length > 1
|
||||
? `<span class="file-select-badge" data-version-id="${version.id}">
|
||||
<i class="fas fa-th-list"></i> ${modelFiles.length} ${translate('modals.download.fileSelection.files')} <i class="fas fa-chevron-right badge-arrow"></i>
|
||||
</span>`
|
||||
@@ -667,9 +736,14 @@ export class DownloadManager {
|
||||
const nextButton = document.getElementById('nextFromVersion');
|
||||
if (!nextButton) return;
|
||||
|
||||
const existsLocally = this.currentVersion?.existsLocally;
|
||||
const version = this.currentVersion;
|
||||
const existsLocally = version?.existsLocally;
|
||||
// A partially downloaded multi-file version still has downloadable
|
||||
// files, so Next routes into the file dialog instead of blocking (#1058).
|
||||
const hasRemainingFiles = this._getWeightFiles(version).length > 1
|
||||
&& this._getRemainingFiles(version).length > 0;
|
||||
|
||||
if (existsLocally) {
|
||||
if (existsLocally && !hasRemainingFiles) {
|
||||
nextButton.disabled = true;
|
||||
nextButton.classList.add('disabled');
|
||||
nextButton.textContent = translate('modals.download.alreadyInLibrary');
|
||||
@@ -680,14 +754,41 @@ export class DownloadManager {
|
||||
}
|
||||
}
|
||||
|
||||
_getWeightFiles(version) {
|
||||
return (version?.files || []).filter(f => isModelWeightFile(f.type));
|
||||
}
|
||||
|
||||
_getRemainingFiles(version) {
|
||||
const downloadedIds = new Set(
|
||||
(version?.downloadedFiles || []).map(f => String(f.fileId))
|
||||
);
|
||||
return this._getWeightFiles(version).filter(f => !downloadedIds.has(String(f.id)));
|
||||
}
|
||||
|
||||
// Files of type UNet / Diffusion Model are routed to the diffusion_model
|
||||
// root while regular files go to the model-type root, so a single
|
||||
// multi-file selection session must stay within one routing group.
|
||||
_getFileRoutingGroup(file) {
|
||||
return (file.type === 'UNet' || file.type === 'Diffusion Model') ? 'diffusion' : 'model';
|
||||
}
|
||||
|
||||
showFileSelectionStep(versionId) {
|
||||
const version = this.versions.find(v => v.id.toString() === versionId.toString());
|
||||
if (!version) return;
|
||||
|
||||
this.currentVersion = version;
|
||||
const modelFiles = (version.files || []).filter(f => isModelWeightFile(f.type));
|
||||
// Start each file-selection session with a clean selection
|
||||
this.selectedFiles = [];
|
||||
this.selectedFile = null;
|
||||
const modelFiles = this._getWeightFiles(version);
|
||||
const downloadedIds = new Set(
|
||||
(version.downloadedFiles || []).map(f => String(f.fileId))
|
||||
);
|
||||
|
||||
document.getElementById('versionStep').style.display = 'none';
|
||||
// Hide every other step — this dialog can be entered directly from
|
||||
// entry points like ModelVersionsTab, where the URL step would
|
||||
// otherwise remain visible (#1058).
|
||||
document.querySelectorAll('.download-step').forEach(step => step.style.display = 'none');
|
||||
document.getElementById('fileSelectionStep').style.display = 'block';
|
||||
|
||||
const nameEl = document.getElementById('fileSelectionVersionName');
|
||||
@@ -699,9 +800,12 @@ export class DownloadManager {
|
||||
container.innerHTML = modelFiles.map(file => {
|
||||
const meta = file.metadata || {};
|
||||
const sizeGB = file.sizeKB ? (file.sizeKB / (1024 * 1024)).toFixed(2) : '--';
|
||||
const isSelected = this.selectedFile?.id === file.id;
|
||||
const isDownloaded = downloadedIds.has(String(file.id));
|
||||
|
||||
const tags = [];
|
||||
if (isDownloaded) {
|
||||
tags.push(`<span class="file-tag in-library">${translate('modals.download.fileSelection.inLibrary', {}, 'In Library')}</span>`);
|
||||
}
|
||||
if (meta.size) tags.push(`<span class="file-tag size">${meta.size}</span>`);
|
||||
if (meta.format) tags.push(`<span class="file-tag format">${meta.format}</span>`);
|
||||
if (meta.fp) tags.push(`<span class="file-tag fp">${meta.fp}</span>`);
|
||||
@@ -709,9 +813,9 @@ export class DownloadManager {
|
||||
const fileName = file.name || '';
|
||||
|
||||
return `
|
||||
<div class="file-option ${isSelected ? 'selected' : ''}" data-file-id="${file.id}">
|
||||
<div class="file-option ${isDownloaded ? 'disabled' : ''}" data-file-id="${file.id}">
|
||||
<div class="file-option-radio">
|
||||
<input type="radio" name="fileSelection" value="${file.id}" ${isSelected ? 'checked' : ''}>
|
||||
<input type="checkbox" name="fileSelection" value="${file.id}" ${isDownloaded ? 'disabled' : ''}>
|
||||
</div>
|
||||
<div class="file-option-info">
|
||||
<div class="file-option-tags">
|
||||
@@ -725,33 +829,80 @@ export class DownloadManager {
|
||||
}).join('');
|
||||
|
||||
container.querySelectorAll('.file-option').forEach(el => {
|
||||
el.addEventListener('click', () => {
|
||||
container.querySelectorAll('.file-option').forEach(o => o.classList.remove('selected'));
|
||||
el.classList.add('selected');
|
||||
const radio = el.querySelector('input[type="radio"]');
|
||||
if (radio) radio.checked = true;
|
||||
el.addEventListener('click', (event) => {
|
||||
// Already-downloaded files stay disabled regardless
|
||||
if (el.classList.contains('disabled')) {
|
||||
event.preventDefault();
|
||||
return;
|
||||
}
|
||||
const checkbox = el.querySelector('input[type="checkbox"]');
|
||||
if (!checkbox || checkbox.disabled) {
|
||||
event.preventDefault();
|
||||
return;
|
||||
}
|
||||
// Clicking the checkbox directly toggles natively; clicking
|
||||
// anywhere else on the option toggles it programmatically.
|
||||
if (event.target !== checkbox) {
|
||||
checkbox.checked = !checkbox.checked;
|
||||
}
|
||||
this._syncFileSelectionState();
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
confirmFileSelection() {
|
||||
const selectedRadio = document.querySelector('#fileSelectionList input[type="radio"]:checked');
|
||||
if (!selectedRadio) {
|
||||
console.warn('[download] confirmFileSelection: no radio button checked');
|
||||
return;
|
||||
}
|
||||
// Sync this.selectedFiles with the DOM checkboxes and enforce the
|
||||
// mixed-type routing guard by disabling the other routing group.
|
||||
_syncFileSelectionState() {
|
||||
const container = document.getElementById('fileSelectionList');
|
||||
if (!container || !this.currentVersion) return;
|
||||
|
||||
const checkedValues = new Set(
|
||||
Array.from(container.querySelectorAll('input[type="checkbox"]:checked'))
|
||||
.map(cb => cb.value)
|
||||
);
|
||||
const modelFiles = this._getWeightFiles(this.currentVersion);
|
||||
this.selectedFiles = modelFiles.filter(f => checkedValues.has(f.id.toString()));
|
||||
this.selectedFile = this.selectedFiles[0] || null;
|
||||
|
||||
const activeGroup = this.selectedFiles.length > 0
|
||||
? this._getFileRoutingGroup(this.selectedFiles[0])
|
||||
: null;
|
||||
|
||||
container.querySelectorAll('.file-option').forEach(el => {
|
||||
const checkbox = el.querySelector('input[type="checkbox"]');
|
||||
if (!checkbox || el.classList.contains('disabled')) return;
|
||||
|
||||
const file = modelFiles.find(f => f.id.toString() === el.dataset.fileId);
|
||||
const groupBlocked = activeGroup !== null
|
||||
&& file
|
||||
&& this._getFileRoutingGroup(file) !== activeGroup
|
||||
&& !checkbox.checked;
|
||||
|
||||
el.classList.toggle('selected', checkbox.checked);
|
||||
el.classList.toggle('group-disabled', groupBlocked);
|
||||
checkbox.disabled = groupBlocked;
|
||||
});
|
||||
}
|
||||
|
||||
confirmFileSelection() {
|
||||
const version = this.currentVersion;
|
||||
if (!version) {
|
||||
console.warn('[download] confirmFileSelection: no currentVersion set');
|
||||
return;
|
||||
}
|
||||
|
||||
const modelFiles = (version.files || []).filter(f => isModelWeightFile(f.type));
|
||||
this.selectedFile = modelFiles.find(f => f.id.toString() === selectedRadio.value);
|
||||
// Sync from the DOM first so programmatically checked boxes count too
|
||||
this._syncFileSelectionState();
|
||||
|
||||
console.log('[download] confirmFileSelection: selected file id=%s, name="%s", type="%s", metadata=%o',
|
||||
this.selectedFile?.id, this.selectedFile?.name, this.selectedFile?.type, this.selectedFile?.metadata);
|
||||
if (this.selectedFiles.length === 0) {
|
||||
console.warn('[download] confirmFileSelection: no file selected');
|
||||
showToast('toast.loras.pleaseSelectFile', {}, 'error');
|
||||
return;
|
||||
}
|
||||
|
||||
console.log('[download] confirmFileSelection: %d file(s) selected — %o',
|
||||
this.selectedFiles.length,
|
||||
this.selectedFiles.map(f => ({ id: f.id, name: f.name, type: f.type })));
|
||||
|
||||
document.getElementById('fileSelectionStep').style.display = 'none';
|
||||
document.getElementById('downloadLocationStep').style.display = 'block';
|
||||
@@ -782,6 +933,13 @@ export class DownloadManager {
|
||||
return;
|
||||
}
|
||||
if (this.currentVersion.existsLocally) {
|
||||
// Multi-file versions with remaining undownloaded files route
|
||||
// into the file dialog instead of being blocked outright (#1058).
|
||||
if (this._getWeightFiles(this.currentVersion).length > 1
|
||||
&& this._getRemainingFiles(this.currentVersion).length > 0) {
|
||||
this.showFileSelectionStep(this.currentVersion.id);
|
||||
return;
|
||||
}
|
||||
showToast('toast.loras.versionExists', {}, 'info');
|
||||
return;
|
||||
}
|
||||
@@ -916,6 +1074,9 @@ export class DownloadManager {
|
||||
source = null,
|
||||
fileParams = null,
|
||||
closeModal = false,
|
||||
deferReload = false,
|
||||
suppressSuccessToast = false,
|
||||
suppressFailureSummary = false,
|
||||
}) {
|
||||
const config = this.apiClient?.apiConfig?.config;
|
||||
|
||||
@@ -924,7 +1085,8 @@ export class DownloadManager {
|
||||
}
|
||||
|
||||
const displayName = versionName || `#${versionId}`;
|
||||
const retryParams = { modelId, versionId, versionName, modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot, source, fileParams, closeModal: false };
|
||||
const retryParams = { modelId, versionId, versionName, modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot, source, fileParams, closeModal: false, deferReload, suppressSuccessToast, suppressFailureSummary };
|
||||
this._lastDownloadError = null;
|
||||
let ws = null;
|
||||
let updateProgress = () => { };
|
||||
let cancelled = false;
|
||||
@@ -1007,7 +1169,9 @@ export class DownloadManager {
|
||||
if (response?.skipped) {
|
||||
this.loadingManager.setStatus(translate('modals.download.status.finalizing'));
|
||||
updateProgress(100, 0, displayName);
|
||||
showToast('toast.loras.downloadSkippedByBaseModel', { baseModel: response.base_model || 'Unknown' }, 'warning');
|
||||
if (!suppressSuccessToast) {
|
||||
showToast('toast.loras.downloadSkippedByBaseModel', { baseModel: response.base_model || 'Unknown' }, 'warning');
|
||||
}
|
||||
if (closeModal) {
|
||||
modalManager.closeModal('downloadModal');
|
||||
}
|
||||
@@ -1016,6 +1180,22 @@ export class DownloadManager {
|
||||
|
||||
if (!response?.success) {
|
||||
this.loadingManager.setStatus(translate('modals.download.status.finalizing'));
|
||||
const errorMessage = response?.error || 'Unknown error';
|
||||
// When the caller aggregates failures itself (multi-file
|
||||
// loop), just record the error and return (#1058).
|
||||
if (suppressFailureSummary) {
|
||||
this._lastDownloadError = errorMessage;
|
||||
return false;
|
||||
}
|
||||
// A file-level "already in library" rejection is an expected
|
||||
// outcome when browsing files of a partially downloaded
|
||||
// version — surface it as a lightweight toast instead of the
|
||||
// failure summary modal so the user can simply go back and
|
||||
// pick another file (#1058).
|
||||
if (typeof errorMessage === 'string' && errorMessage.includes('already exists in')) {
|
||||
showToast(errorMessage, {}, 'info');
|
||||
return false;
|
||||
}
|
||||
showDownloadBatchSummary({
|
||||
total: 1,
|
||||
completed: 0,
|
||||
@@ -1026,7 +1206,7 @@ export class DownloadManager {
|
||||
source,
|
||||
url: this._buildSingleItemUrl({ modelId, versionId, source }),
|
||||
},
|
||||
error: response?.error || 'Unknown error',
|
||||
error: errorMessage,
|
||||
name: displayName,
|
||||
}],
|
||||
onRetry: () => this.executeDownloadWithProgress(retryParams),
|
||||
@@ -1034,7 +1214,9 @@ export class DownloadManager {
|
||||
return false;
|
||||
}
|
||||
|
||||
showToast('toast.loras.downloadCompleted', {}, 'success');
|
||||
if (!suppressSuccessToast) {
|
||||
showToast('toast.loras.downloadCompleted', {}, 'success');
|
||||
}
|
||||
|
||||
if (closeModal) {
|
||||
modalManager.closeModal('downloadModal');
|
||||
@@ -1045,29 +1227,35 @@ export class DownloadManager {
|
||||
ws = null;
|
||||
}
|
||||
|
||||
const pageState = this.apiClient.getPageState();
|
||||
if (!deferReload) {
|
||||
const pageState = this.apiClient.getPageState();
|
||||
|
||||
if (!useDefaultPaths && targetFolder) {
|
||||
pageState.activeFolder = targetFolder;
|
||||
setStorageItem(`${this.apiClient.modelType}_activeFolder`, targetFolder);
|
||||
if (!useDefaultPaths && targetFolder) {
|
||||
pageState.activeFolder = targetFolder;
|
||||
setStorageItem(`${this.apiClient.modelType}_activeFolder`, targetFolder);
|
||||
|
||||
document.querySelectorAll('.folder-tags .tag').forEach(tag => {
|
||||
const isActive = tag.dataset.folder === targetFolder;
|
||||
tag.classList.toggle('active', isActive);
|
||||
if (isActive && !tag.parentNode.classList.contains('collapsed')) {
|
||||
tag.scrollIntoView({ behavior: 'smooth', block: 'nearest' });
|
||||
}
|
||||
});
|
||||
document.querySelectorAll('.folder-tags .tag').forEach(tag => {
|
||||
const isActive = tag.dataset.folder === targetFolder;
|
||||
tag.classList.toggle('active', isActive);
|
||||
if (isActive && !tag.parentNode.classList.contains('collapsed')) {
|
||||
tag.scrollIntoView({ behavior: 'smooth', block: 'nearest' });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
await resetAndReload(true);
|
||||
}
|
||||
|
||||
await resetAndReload(true);
|
||||
|
||||
return true;
|
||||
} catch (error) {
|
||||
if (cancelled) {
|
||||
console.log('Download cancelled by user:', downloadId);
|
||||
} else {
|
||||
console.error('Failed to download model version:', error);
|
||||
if (suppressFailureSummary) {
|
||||
this._lastDownloadError = error?.message || 'Unknown error';
|
||||
return false;
|
||||
}
|
||||
showDownloadBatchSummary({
|
||||
total: 1,
|
||||
completed: 0,
|
||||
@@ -1097,6 +1285,89 @@ export class DownloadManager {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Download multiple selected files of the same version sequentially,
|
||||
* reusing the location-step choices for every file. Per-file toasts,
|
||||
* reloads and failure modals are suppressed; a single aggregated result
|
||||
* is shown at the end (design decision D5, #1058).
|
||||
*/
|
||||
async _downloadSelectedFilesSequentially({ modelRoot, targetFolder, useDefaultPaths, useSaveDirAsRoot = false, files = null }) {
|
||||
const filesToDownload = files || this.selectedFiles;
|
||||
const totalFiles = filesToDownload.length;
|
||||
const failedItems = [];
|
||||
let completedDownloads = 0;
|
||||
|
||||
for (const file of filesToDownload) {
|
||||
const fileParams = {
|
||||
id: file.id,
|
||||
name: file.name || null,
|
||||
type: file.type || 'Model',
|
||||
format: file.metadata?.format || null,
|
||||
size: file.metadata?.size || null,
|
||||
fp: file.metadata?.fp || null,
|
||||
};
|
||||
|
||||
console.log('[download] multi-file loop: downloading file id=%s, name="%s" (%d/%d)',
|
||||
fileParams.id, fileParams.name, completedDownloads + failedItems.length + 1, totalFiles);
|
||||
|
||||
const success = await this.executeDownloadWithProgress({
|
||||
modelId: this.modelId,
|
||||
versionId: this.currentVersion.id,
|
||||
versionName: file.name || `${this.currentVersion.name} #${file.id}`,
|
||||
modelRoot,
|
||||
targetFolder,
|
||||
useDefaultPaths,
|
||||
useSaveDirAsRoot,
|
||||
source: this.source,
|
||||
fileParams,
|
||||
closeModal: false,
|
||||
deferReload: true,
|
||||
suppressSuccessToast: true,
|
||||
suppressFailureSummary: true,
|
||||
});
|
||||
|
||||
if (success) {
|
||||
completedDownloads++;
|
||||
} else {
|
||||
failedItems.push({
|
||||
item: {
|
||||
modelId: this.modelId,
|
||||
versionId: this.currentVersion.id,
|
||||
source: this.source,
|
||||
file,
|
||||
url: this._buildSingleItemUrl({
|
||||
modelId: this.modelId,
|
||||
versionId: this.currentVersion.id,
|
||||
source: this.source,
|
||||
}),
|
||||
},
|
||||
error: this._lastDownloadError || 'Unknown error',
|
||||
name: file.name || `#${file.id}`,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if (failedItems.length === 0) {
|
||||
showToast('toast.loras.allDownloadSuccessful', { count: completedDownloads }, 'success');
|
||||
} else {
|
||||
showDownloadBatchSummary({
|
||||
total: totalFiles,
|
||||
completed: completedDownloads,
|
||||
failedItems,
|
||||
onRetry: () => this._downloadSelectedFilesSequentially({
|
||||
modelRoot,
|
||||
targetFolder,
|
||||
useDefaultPaths,
|
||||
useSaveDirAsRoot,
|
||||
files: failedItems.map(f => f.item.file),
|
||||
}),
|
||||
});
|
||||
}
|
||||
|
||||
await resetAndReload(true);
|
||||
return failedItems.length === 0;
|
||||
}
|
||||
|
||||
async _downloadHfSingle({ modelRoot, targetFolder, useDefaultPaths, files = null }) {
|
||||
modalManager.closeModal('downloadModal');
|
||||
this.loadingManager.restoreProgressBar();
|
||||
@@ -1307,6 +1578,14 @@ export class DownloadManager {
|
||||
? (ver.modelSizeKB / 1024).toFixed(1)
|
||||
: (ver?.files?.[0]?.sizeKB ? (ver.files[0].sizeKB / 1024).toFixed(1) : '?');
|
||||
const existsLocally = ver?.existsLocally;
|
||||
// Multi-file versions that are only partially downloaded get a
|
||||
// distinct hint instead of the plain in-library badge (#1058).
|
||||
const isPartiallyDownloaded = existsLocally
|
||||
&& this._getWeightFiles(ver).length > 1
|
||||
&& this._getRemainingFiles(ver).length > 0;
|
||||
const localBadgeLabel = isPartiallyDownloaded
|
||||
? translate('modals.download.partiallyDownloaded', {}, 'Partially downloaded')
|
||||
: translate('modals.download.inLibrary');
|
||||
return `
|
||||
<div class="batch-preview-item ${existsLocally ? 'batch-preview-local' : ''}" data-index="${index}">
|
||||
<div class="batch-preview-thumbnail">
|
||||
@@ -1317,7 +1596,7 @@ export class DownloadManager {
|
||||
<div class="batch-preview-meta">
|
||||
${ver?.baseModel ? `<span>${ver.baseModel}</span>` : ''}
|
||||
<span>${fileSize} MB</span>
|
||||
${existsLocally ? `<span class="batch-preview-local-badge"><i class="fas fa-check"></i> ${translate('modals.download.inLibrary')}</span>` : ''}
|
||||
${existsLocally ? `<span class="batch-preview-local-badge"><i class="fas fa-check"></i> ${localBadgeLabel}</span>` : ''}
|
||||
</div>
|
||||
</div>
|
||||
${item.versions.length > 1 ? `
|
||||
@@ -1608,8 +1887,20 @@ export class DownloadManager {
|
||||
});
|
||||
}
|
||||
|
||||
// Multi-file selection: download all selected files sequentially,
|
||||
// reusing the chosen location for every file (#1058).
|
||||
if (this.selectedFiles.length > 1) {
|
||||
modalManager.closeModal('downloadModal');
|
||||
return this._downloadSelectedFilesSequentially({
|
||||
modelRoot,
|
||||
targetFolder,
|
||||
useDefaultPaths,
|
||||
});
|
||||
}
|
||||
|
||||
const fileParams = this.selectedFile ? {
|
||||
id: this.selectedFile.id,
|
||||
name: this.selectedFile.name || null,
|
||||
type: this.selectedFile.type || 'Model',
|
||||
format: this.selectedFile.metadata?.format || null,
|
||||
size: this.selectedFile.metadata?.size || null,
|
||||
|
||||
@@ -0,0 +1,164 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const {
|
||||
DOWNLOAD_MANAGER_MODULE,
|
||||
MODAL_MANAGER_MODULE,
|
||||
UI_HELPERS_MODULE,
|
||||
STATE_MODULE,
|
||||
LOADING_MANAGER_MODULE,
|
||||
API_FACTORY_MODULE,
|
||||
STORAGE_HELPERS_MODULE,
|
||||
FOLDER_TREE_MANAGER_MODULE,
|
||||
I18N_HELPERS_MODULE,
|
||||
} = vi.hoisted(() => ({
|
||||
DOWNLOAD_MANAGER_MODULE: new URL('../../../static/js/managers/DownloadManager.js', import.meta.url).pathname,
|
||||
MODAL_MANAGER_MODULE: new URL('../../../static/js/managers/ModalManager.js', import.meta.url).pathname,
|
||||
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
|
||||
STATE_MODULE: new URL('../../../static/js/state/index.js', import.meta.url).pathname,
|
||||
LOADING_MANAGER_MODULE: new URL('../../../static/js/managers/LoadingManager.js', import.meta.url).pathname,
|
||||
API_FACTORY_MODULE: new URL('../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
|
||||
STORAGE_HELPERS_MODULE: new URL('../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
|
||||
FOLDER_TREE_MANAGER_MODULE: new URL('../../../static/js/components/FolderTreeManager.js', import.meta.url).pathname,
|
||||
I18N_HELPERS_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
|
||||
}));
|
||||
|
||||
vi.mock(MODAL_MANAGER_MODULE, () => ({
|
||||
modalManager: {
|
||||
showModal: vi.fn(),
|
||||
closeModal: vi.fn(),
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock(UI_HELPERS_MODULE, () => ({
|
||||
showToast: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(STATE_MODULE, () => ({
|
||||
state: {
|
||||
global: {
|
||||
settings: {},
|
||||
},
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock(LOADING_MANAGER_MODULE, () => ({
|
||||
LoadingManager: vi.fn(() => ({
|
||||
showSimpleLoading: vi.fn(),
|
||||
hide: vi.fn(),
|
||||
restoreProgressBar: vi.fn(),
|
||||
showDownloadProgress: vi.fn(() => vi.fn()),
|
||||
setStatus: vi.fn(),
|
||||
})),
|
||||
}));
|
||||
|
||||
vi.mock(API_FACTORY_MODULE, () => ({
|
||||
getModelApiClient: vi.fn(() => ({
|
||||
apiConfig: {
|
||||
config: {
|
||||
displayName: 'LoRA',
|
||||
singularName: 'lora',
|
||||
},
|
||||
},
|
||||
})),
|
||||
resetAndReload: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(STORAGE_HELPERS_MODULE, () => ({
|
||||
getStorageItem: vi.fn((_key, defaultValue) => defaultValue),
|
||||
setStorageItem: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(FOLDER_TREE_MANAGER_MODULE, () => ({
|
||||
FolderTreeManager: vi.fn(() => ({
|
||||
clearSelection: vi.fn(),
|
||||
init: vi.fn(),
|
||||
})),
|
||||
}));
|
||||
|
||||
vi.mock(I18N_HELPERS_MODULE, () => ({
|
||||
translate: vi.fn((_, __, fallback) => fallback ?? ''),
|
||||
}));
|
||||
|
||||
const MULTI_FILE_VERSION = {
|
||||
id: 201,
|
||||
name: 'Multi-file version',
|
||||
images: [],
|
||||
files: [
|
||||
{ id: 1001, type: 'Model', sizeKB: 2048, name: 'file-a.safetensors' },
|
||||
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
|
||||
],
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
};
|
||||
|
||||
const SINGLE_FILE_VERSION = {
|
||||
id: 202,
|
||||
name: 'Single-file version',
|
||||
images: [],
|
||||
files: [{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' }],
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
};
|
||||
|
||||
describe('DownloadManager multi-file version badge (#1058)', () => {
|
||||
let DownloadManager;
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.resetModules();
|
||||
document.body.innerHTML = `
|
||||
<div id="urlStep"></div>
|
||||
<div id="versionStep"></div>
|
||||
<div id="versionList"></div>
|
||||
<button id="nextFromVersion"></button>
|
||||
`;
|
||||
({ DownloadManager } = await import(DOWNLOAD_MANAGER_MODULE));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
document.body.innerHTML = '';
|
||||
});
|
||||
|
||||
it('shows the file-select badge for a multi-file version already in library', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [
|
||||
{
|
||||
...MULTI_FILE_VERSION,
|
||||
existsLocally: true,
|
||||
localPath: '/models/loras/file-a.safetensors',
|
||||
},
|
||||
];
|
||||
|
||||
manager.showVersionStep();
|
||||
|
||||
const badge = document.querySelector('.file-select-badge');
|
||||
expect(badge).not.toBeNull();
|
||||
expect(badge.dataset.versionId).toBe('201');
|
||||
expect(badge.textContent).toContain('2');
|
||||
// The in-library badge is still shown alongside
|
||||
expect(document.querySelector('.local-badge')).not.toBeNull();
|
||||
});
|
||||
|
||||
it('still shows the file-select badge for a multi-file version not in library', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [{ ...MULTI_FILE_VERSION, existsLocally: false }];
|
||||
|
||||
manager.showVersionStep();
|
||||
|
||||
const badge = document.querySelector('.file-select-badge');
|
||||
expect(badge).not.toBeNull();
|
||||
expect(badge.dataset.versionId).toBe('201');
|
||||
});
|
||||
|
||||
it('does not show the file-select badge for a single-file version in library', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [
|
||||
{
|
||||
...SINGLE_FILE_VERSION,
|
||||
existsLocally: true,
|
||||
localPath: '/models/loras/file-c.safetensors',
|
||||
},
|
||||
];
|
||||
|
||||
manager.showVersionStep();
|
||||
|
||||
expect(document.querySelector('.file-select-badge')).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,332 @@
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const {
|
||||
DOWNLOAD_MANAGER_MODULE,
|
||||
MODAL_MANAGER_MODULE,
|
||||
UI_HELPERS_MODULE,
|
||||
STATE_MODULE,
|
||||
LOADING_MANAGER_MODULE,
|
||||
API_FACTORY_MODULE,
|
||||
STORAGE_HELPERS_MODULE,
|
||||
FOLDER_TREE_MANAGER_MODULE,
|
||||
I18N_HELPERS_MODULE,
|
||||
SUMMARY_MODULE,
|
||||
mockApiClient,
|
||||
mockLoadingManager,
|
||||
showToastMock,
|
||||
showDownloadBatchSummaryMock,
|
||||
resetAndReloadMock,
|
||||
} = vi.hoisted(() => {
|
||||
// Shared API client returned by the mocked getModelApiClient factory.
|
||||
const mockApiClient = {
|
||||
modelType: 'loras',
|
||||
apiConfig: {
|
||||
config: {
|
||||
displayName: 'LoRA',
|
||||
singularName: 'lora',
|
||||
},
|
||||
},
|
||||
fetchCivitaiVersions: vi.fn(),
|
||||
fetchModelRoots: vi.fn(async () => ({ roots: ['/models/loras'] })),
|
||||
fetchUnifiedFolderTree: vi.fn(async () => ({ success: false })),
|
||||
downloadModel: vi.fn(),
|
||||
cancelDownload: vi.fn(),
|
||||
getPageState: vi.fn(() => ({})),
|
||||
};
|
||||
|
||||
const mockLoadingManager = {
|
||||
showSimpleLoading: vi.fn(),
|
||||
setStatus: vi.fn(),
|
||||
hide: vi.fn(),
|
||||
restoreProgressBar: vi.fn(),
|
||||
showDownloadProgress: vi.fn(() => vi.fn()),
|
||||
showCancelButton: vi.fn(),
|
||||
};
|
||||
|
||||
return {
|
||||
DOWNLOAD_MANAGER_MODULE: new URL('../../../static/js/managers/DownloadManager.js', import.meta.url).pathname,
|
||||
MODAL_MANAGER_MODULE: new URL('../../../static/js/managers/ModalManager.js', import.meta.url).pathname,
|
||||
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
|
||||
STATE_MODULE: new URL('../../../static/js/state/index.js', import.meta.url).pathname,
|
||||
LOADING_MANAGER_MODULE: new URL('../../../static/js/managers/LoadingManager.js', import.meta.url).pathname,
|
||||
API_FACTORY_MODULE: new URL('../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
|
||||
STORAGE_HELPERS_MODULE: new URL('../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
|
||||
FOLDER_TREE_MANAGER_MODULE: new URL('../../../static/js/components/FolderTreeManager.js', import.meta.url).pathname,
|
||||
I18N_HELPERS_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
|
||||
SUMMARY_MODULE: new URL('../../../static/js/components/DownloadBatchSummaryModal.js', import.meta.url).pathname,
|
||||
mockApiClient,
|
||||
mockLoadingManager,
|
||||
showToastMock: vi.fn(),
|
||||
showDownloadBatchSummaryMock: vi.fn(),
|
||||
resetAndReloadMock: vi.fn(),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock(MODAL_MANAGER_MODULE, () => ({
|
||||
modalManager: {
|
||||
showModal: vi.fn(),
|
||||
closeModal: vi.fn(),
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock(UI_HELPERS_MODULE, () => ({
|
||||
showToast: showToastMock,
|
||||
setupAutoNewlineOnPaste: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(STATE_MODULE, () => ({
|
||||
state: {
|
||||
global: {
|
||||
settings: {},
|
||||
},
|
||||
loadingManager: mockLoadingManager,
|
||||
},
|
||||
}));
|
||||
|
||||
vi.mock(LOADING_MANAGER_MODULE, () => ({
|
||||
LoadingManager: vi.fn(() => mockLoadingManager),
|
||||
}));
|
||||
|
||||
vi.mock(API_FACTORY_MODULE, () => ({
|
||||
getModelApiClient: vi.fn(() => mockApiClient),
|
||||
resetAndReload: resetAndReloadMock,
|
||||
}));
|
||||
|
||||
vi.mock(STORAGE_HELPERS_MODULE, () => ({
|
||||
getStorageItem: vi.fn((_key, defaultValue) => defaultValue),
|
||||
setStorageItem: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(FOLDER_TREE_MANAGER_MODULE, () => ({
|
||||
FolderTreeManager: vi.fn(() => ({
|
||||
clearSelection: vi.fn(),
|
||||
init: vi.fn(),
|
||||
getSelectedPath: vi.fn(() => ''),
|
||||
})),
|
||||
}));
|
||||
|
||||
vi.mock(I18N_HELPERS_MODULE, () => ({
|
||||
translate: vi.fn((_, __, fallback) => fallback ?? ''),
|
||||
}));
|
||||
|
||||
vi.mock(SUMMARY_MODULE, () => ({
|
||||
showDownloadBatchSummary: showDownloadBatchSummaryMock,
|
||||
}));
|
||||
|
||||
/** DOM covering the file-selection, version and location steps. */
|
||||
function setupDownloadDom() {
|
||||
document.body.innerHTML = `
|
||||
<div id="downloadModal">
|
||||
<div class="download-step" id="urlStep"></div>
|
||||
<div class="download-step" id="versionStep"></div>
|
||||
<div class="download-step" id="fileSelectionStep"></div>
|
||||
<div class="download-step" id="downloadLocationStep"></div>
|
||||
<div id="fileSelectionList"></div>
|
||||
<div id="fileSelectionVersionName"></div>
|
||||
<button id="nextFromVersion"></button>
|
||||
<div id="downloadModalTitle"></div>
|
||||
<select id="modelRoot"></select>
|
||||
<input id="folderPath" />
|
||||
<div id="targetPathDisplay"></div>
|
||||
<input id="useDefaultPath" type="checkbox" />
|
||||
<div id="manualPathSelection"></div>
|
||||
</div>
|
||||
`;
|
||||
}
|
||||
|
||||
function makeMultiFileVersion(overrides = {}) {
|
||||
return {
|
||||
id: 201,
|
||||
name: 'Multi-file version',
|
||||
baseModel: 'SDXL',
|
||||
images: [],
|
||||
files: [
|
||||
{ id: 1001, type: 'Model', sizeKB: 2048, name: 'file-a.safetensors' },
|
||||
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
|
||||
{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' },
|
||||
],
|
||||
createdAt: '2026-01-01T00:00:00Z',
|
||||
existsLocally: true,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function getFileOption(fileId) {
|
||||
return document.querySelector(`.file-option[data-file-id="${fileId}"]`);
|
||||
}
|
||||
|
||||
describe('DownloadManager multi-select file dialog (#1058)', () => {
|
||||
let DownloadManager;
|
||||
|
||||
beforeEach(async () => {
|
||||
vi.resetModules();
|
||||
vi.clearAllMocks();
|
||||
setupDownloadDom();
|
||||
({ DownloadManager } = await import(DOWNLOAD_MANAGER_MODULE));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
document.body.innerHTML = '';
|
||||
});
|
||||
|
||||
it('renders downloaded files disabled with an In Library tag', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [makeMultiFileVersion({
|
||||
downloadedFiles: [
|
||||
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
|
||||
],
|
||||
})];
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
|
||||
const downloadedOption = getFileOption('1001');
|
||||
expect(downloadedOption.classList.contains('disabled')).toBe(true);
|
||||
expect(downloadedOption.querySelector('input[type="checkbox"]').disabled).toBe(true);
|
||||
expect(downloadedOption.querySelector('.file-tag.in-library').textContent).toBe('In Library');
|
||||
|
||||
// Remaining files stay selectable
|
||||
const otherOption = getFileOption('1002');
|
||||
expect(otherOption.classList.contains('disabled')).toBe(false);
|
||||
expect(otherOption.querySelector('input[type="checkbox"]').disabled).toBe(false);
|
||||
});
|
||||
|
||||
it('ignores clicks on already-downloaded options', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [makeMultiFileVersion({
|
||||
downloadedFiles: [
|
||||
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
|
||||
],
|
||||
})];
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
|
||||
getFileOption('1001').click();
|
||||
|
||||
expect(getFileOption('1001').querySelector('input[type="checkbox"]').checked).toBe(false);
|
||||
expect(manager.selectedFiles).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('confirmFileSelection collects multiple checked files into selectedFiles', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.apiClient = mockApiClient;
|
||||
manager.versions = [makeMultiFileVersion({ downloadedFiles: [] })];
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
|
||||
getFileOption('1001').click();
|
||||
getFileOption('1003').click();
|
||||
|
||||
manager.confirmFileSelection();
|
||||
|
||||
expect(manager.selectedFiles.map(f => f.id)).toEqual([1001, 1003]);
|
||||
// selectedFile stays the first selected file for single-file flows
|
||||
expect(manager.selectedFile?.id).toBe(1001);
|
||||
expect(document.getElementById('fileSelectionStep').style.display).toBe('none');
|
||||
expect(document.getElementById('downloadLocationStep').style.display).toBe('block');
|
||||
});
|
||||
|
||||
it('confirmFileSelection warns when nothing is selected', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [makeMultiFileVersion({ downloadedFiles: [] })];
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
manager.confirmFileSelection();
|
||||
|
||||
expect(showToastMock).toHaveBeenCalledWith('toast.loras.pleaseSelectFile', {}, 'error');
|
||||
expect(manager.selectedFiles).toHaveLength(0);
|
||||
expect(document.getElementById('downloadLocationStep').style.display).not.toBe('block');
|
||||
});
|
||||
|
||||
it('disables the other routing group once a file is checked and re-enables when unchecked', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [makeMultiFileVersion({
|
||||
downloadedFiles: [],
|
||||
files: [
|
||||
{ id: 1001, type: 'UNet', sizeKB: 2048, name: 'unet-a.safetensors' },
|
||||
{ id: 1002, type: 'Model', sizeKB: 2048, name: 'file-b.safetensors' },
|
||||
{ id: 1003, type: 'Model', sizeKB: 2048, name: 'file-c.safetensors' },
|
||||
],
|
||||
})];
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
|
||||
// Checking a regular Model file disables the UNet option
|
||||
getFileOption('1002').click();
|
||||
expect(getFileOption('1001').classList.contains('group-disabled')).toBe(true);
|
||||
expect(getFileOption('1001').querySelector('input[type="checkbox"]').disabled).toBe(true);
|
||||
expect(getFileOption('1003').classList.contains('group-disabled')).toBe(false);
|
||||
|
||||
// Clicking a group-disabled option does nothing
|
||||
getFileOption('1001').click();
|
||||
expect(manager.selectedFiles.map(f => f.id)).toEqual([1002]);
|
||||
|
||||
// Unchecking everything re-enables the other group
|
||||
getFileOption('1002').click();
|
||||
expect(manager.selectedFiles).toHaveLength(0);
|
||||
expect(getFileOption('1001').classList.contains('group-disabled')).toBe(false);
|
||||
expect(getFileOption('1001').querySelector('input[type="checkbox"]').disabled).toBe(false);
|
||||
});
|
||||
|
||||
it('keeps Next enabled for a partially downloaded multi-file version', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.currentVersion = makeMultiFileVersion({
|
||||
downloadedFiles: [
|
||||
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
|
||||
],
|
||||
});
|
||||
|
||||
manager.updateNextButtonState();
|
||||
|
||||
const nextButton = document.getElementById('nextFromVersion');
|
||||
expect(nextButton.disabled).toBe(false);
|
||||
expect(nextButton.classList.contains('disabled')).toBe(false);
|
||||
});
|
||||
|
||||
it('disables Next when every weight file is already downloaded', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.currentVersion = makeMultiFileVersion({
|
||||
downloadedFiles: [
|
||||
{ fileId: 1001, fileName: 'file-a.safetensors', filePath: '/models/loras/file-a.safetensors' },
|
||||
{ fileId: 1002, fileName: 'file-b.safetensors', filePath: '/models/loras/file-b.safetensors' },
|
||||
{ fileId: 1003, fileName: 'file-c.safetensors', filePath: '/models/loras/file-c.safetensors' },
|
||||
],
|
||||
});
|
||||
|
||||
manager.updateNextButtonState();
|
||||
|
||||
const nextButton = document.getElementById('nextFromVersion');
|
||||
expect(nextButton.disabled).toBe(true);
|
||||
expect(nextButton.classList.contains('disabled')).toBe(true);
|
||||
});
|
||||
|
||||
it('disables Next for an in-library single-file version', () => {
|
||||
const manager = new DownloadManager();
|
||||
manager.currentVersion = {
|
||||
id: 202,
|
||||
name: 'Single-file version',
|
||||
files: [{ id: 1004, type: 'Model', sizeKB: 2048, name: 'file-d.safetensors' }],
|
||||
existsLocally: true,
|
||||
downloadedFiles: [],
|
||||
};
|
||||
|
||||
manager.updateNextButtonState();
|
||||
|
||||
const nextButton = document.getElementById('nextFromVersion');
|
||||
expect(nextButton.disabled).toBe(true);
|
||||
});
|
||||
|
||||
it('hides every other step (including the URL step) when the file dialog shows', () => {
|
||||
// Regression: entering via openFileSelectionForVersion (ModelVersionsTab)
|
||||
// left the URL step visible alongside the file selection step (#1058).
|
||||
const manager = new DownloadManager();
|
||||
manager.versions = [makeMultiFileVersion()];
|
||||
document.getElementById('urlStep').style.display = 'block';
|
||||
document.getElementById('versionStep').style.display = 'block';
|
||||
|
||||
manager.showFileSelectionStep('201');
|
||||
|
||||
expect(document.getElementById('urlStep').style.display).toBe('none');
|
||||
expect(document.getElementById('versionStep').style.display).toBe('none');
|
||||
expect(document.getElementById('fileSelectionStep').style.display).toBe('block');
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
"""Unit tests for per-file downloaded-state matching (#1058)."""
|
||||
|
||||
from py.routes.handlers.model_handlers import ModelCivitaiHandler
|
||||
|
||||
|
||||
VERSION = {
|
||||
"id": 42,
|
||||
"files": [
|
||||
{
|
||||
"id": 1001,
|
||||
"name": "file-a.safetensors",
|
||||
"hashes": {"SHA256": "AAA111"},
|
||||
},
|
||||
{
|
||||
"id": 1002,
|
||||
"name": "file-b.safetensors",
|
||||
"hashes": {"SHA256": "BBB222"},
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _entry(file_name: str, sha256: str = "", file_path: str | None = None):
|
||||
return {
|
||||
"file_name": file_name,
|
||||
"file_path": file_path or f"/models/{file_name}.safetensors",
|
||||
"sha256": sha256,
|
||||
}
|
||||
|
||||
|
||||
def test_matches_by_sha256():
|
||||
entries = [_entry("renamed-locally", "bbb222")]
|
||||
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
|
||||
assert result == [
|
||||
{
|
||||
"fileId": 1002,
|
||||
"fileName": "file-b.safetensors",
|
||||
"filePath": "/models/renamed-locally.safetensors",
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_falls_back_to_name_when_hash_missing():
|
||||
entries = [_entry("file-a", "")]
|
||||
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
|
||||
assert [r["fileId"] for r in result] == [1001]
|
||||
|
||||
|
||||
def test_hash_takes_precedence_over_name():
|
||||
# Hash points at file-b while the name points at file-a: hash wins.
|
||||
entries = [_entry("file-a", "bbb222")]
|
||||
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
|
||||
assert [r["fileId"] for r in result] == [1002]
|
||||
|
||||
|
||||
def test_unmatched_entries_are_skipped():
|
||||
entries = [
|
||||
_entry("unrelated", "ccc333"),
|
||||
_entry("file-b", ""), # name match
|
||||
]
|
||||
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
|
||||
assert [r["fileId"] for r in result] == [1002]
|
||||
|
||||
|
||||
def test_multiple_files_of_same_version():
|
||||
entries = [
|
||||
_entry("file-a", "aaa111"),
|
||||
_entry("file-b", "bbb222"),
|
||||
]
|
||||
result = ModelCivitaiHandler._match_downloaded_files(VERSION, entries)
|
||||
assert [r["fileId"] for r in result] == [1001, 1002]
|
||||
|
||||
|
||||
def test_empty_inputs():
|
||||
assert ModelCivitaiHandler._match_downloaded_files(VERSION, []) == []
|
||||
assert ModelCivitaiHandler._match_downloaded_files({"id": 1}, [_entry("x")]) == []
|
||||
assert ModelCivitaiHandler._match_downloaded_files(VERSION, None) == []
|
||||
@@ -3,6 +3,40 @@ import pytest
|
||||
from py.recipes.parsers.automatic import AutomaticMetadataParser
|
||||
|
||||
|
||||
class LocalRecipeScanner:
|
||||
class LoraScanner:
|
||||
@staticmethod
|
||||
def has_hash(model_hash):
|
||||
return False
|
||||
|
||||
def __init__(self, models):
|
||||
self.models = models
|
||||
self.queries = []
|
||||
self.hash_queries = []
|
||||
self._lora_scanner = self.LoraScanner()
|
||||
|
||||
async def get_local_lora(self, name, base_model=None):
|
||||
self.queries.append(name)
|
||||
return self.models.get(name)
|
||||
|
||||
async def get_local_lora_by_hash(self, hash_value):
|
||||
self.hash_queries.append(hash_value)
|
||||
return next((model for model in self.models.values() if model.get("sha256") == hash_value), None)
|
||||
|
||||
|
||||
def local_lora(file_name="local_only"):
|
||||
return {
|
||||
"file_path": f"/models/loras/styles/{file_name}.safetensors",
|
||||
"file_name": file_name,
|
||||
"model_name": "Local Only",
|
||||
"sha256": "a" * 64,
|
||||
"size": 123456,
|
||||
"base_model": "Flux.1 D",
|
||||
"preview_url": f"/models/loras/styles/{file_name}.preview.png",
|
||||
"civitai": None,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_extracts_checkpoint_from_civitai_resources(monkeypatch):
|
||||
checkpoint_info = {
|
||||
@@ -132,6 +166,218 @@ async def test_parse_metadata_merges_lora_hashes_over_empty_hashes_json(monkeypa
|
||||
assert "UnusedLora" not in lora_names, "UnusedLora should have been skipped"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_resolves_local_lora_with_empty_hash(monkeypatch):
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_by_hash(self, model_hash):
|
||||
raise AssertionError("Local and empty-hash LoRAs must not query Civitai")
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local_only": local_lora()})
|
||||
metadata_text = (
|
||||
"portrait <lora:local_only:0.65> <lora:missing:0.4>\n"
|
||||
"Steps: 20, Sampler: Euler, CFG scale: 7, Seed: 1, "
|
||||
'Hashes: {"lora:local_only": "", "lora:missing": ""}'
|
||||
)
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert len(result["loras"]) == 1
|
||||
entry = result["loras"][0]
|
||||
assert entry["name"] == "Local Only"
|
||||
assert entry["file_name"] == "local_only"
|
||||
assert entry["weight"] == 0.65
|
||||
assert entry["hash"] == "a" * 64
|
||||
assert entry["localPath"].endswith("local_only.safetensors")
|
||||
assert entry["size"] == 123456
|
||||
assert entry["baseModel"] == "Flux.1 D"
|
||||
assert entry["existsLocally"] is True
|
||||
assert entry["isDeleted"] is False
|
||||
assert scanner.queries == ["local_only", "missing"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_resolves_prompt_lora_without_hashes(monkeypatch):
|
||||
async def fake_metadata_provider():
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
model = local_lora()
|
||||
scanner = LocalRecipeScanner({"styles/local_only": model})
|
||||
metadata_text = "portrait <lora:styles/local_only:0.7>\nSteps: 20, Seed: 1"
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert len(result["loras"]) == 1
|
||||
assert result["loras"][0]["weight"] == 0.7
|
||||
assert result["loras"][0]["hash"] == "a" * 64
|
||||
assert scanner.queries == ["styles/local_only"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_prefers_hash_over_colliding_local_name(monkeypatch):
|
||||
remote_info = {
|
||||
"id": 100,
|
||||
"modelId": 200,
|
||||
"model": {"name": "Hash Match", "type": "LORA"},
|
||||
"name": "v1",
|
||||
"files": [{"type": "Model", "primary": True, "name": "hash_match.safetensors", "hashes": {"SHA256": "b" * 64}}],
|
||||
}
|
||||
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_by_hash(self, model_hash):
|
||||
assert model_hash == "deadbeef00"
|
||||
return remote_info, None
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local_only": local_lora()})
|
||||
metadata_text = (
|
||||
"portrait <lora:local_only:0.8>\n"
|
||||
"Steps: 20, Seed: 1, "
|
||||
'Hashes: {"lora:local_only": "deadbeef00"}'
|
||||
)
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert len(result["loras"]) == 1
|
||||
assert result["loras"][0]["id"] == 100
|
||||
assert result["loras"][0]["weight"] == 0.8
|
||||
assert scanner.queries == []
|
||||
assert scanner.hash_queries == ["deadbeef00"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_falls_back_to_name_when_hash_is_unresolved(monkeypatch):
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_by_hash(self, model_hash):
|
||||
return None, "Model not found"
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local_only": local_lora()})
|
||||
metadata_text = (
|
||||
"portrait <lora:local_only:0.8>\nSteps: 20, Seed: 1, "
|
||||
'Hashes: {"lora:local_only": "deadbeef00"}'
|
||||
)
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert result["loras"][0]["hash"] == "a" * 64
|
||||
assert scanner.queries == ["local_only"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_uses_prompt_weight_for_civitai_resource(monkeypatch):
|
||||
remote_info = {
|
||||
"id": 100,
|
||||
"modelId": 200,
|
||||
"model": {"name": "local_only", "type": "LORA"},
|
||||
"name": "v1",
|
||||
"files": [
|
||||
{
|
||||
"type": "Model",
|
||||
"primary": True,
|
||||
"name": "remote_file.safetensors",
|
||||
"hashes": {"SHA256": "b" * 64},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_version_info(self, version_id):
|
||||
assert version_id == 100
|
||||
return remote_info, None
|
||||
|
||||
async def get_model_by_hash(self, model_hash):
|
||||
raise AssertionError("The Civitai resource should not be fetched again by hash")
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local_only": local_lora()})
|
||||
metadata_text = (
|
||||
"portrait <lora:remote_file:0.35> <lora:local_only:0.6>\n"
|
||||
"Steps: 20, Seed: 1, "
|
||||
'Civitai resources: [{"type":"lora","modelVersionId":100,"modelName":"local_only"}]'
|
||||
)
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert len(result["loras"]) == 2
|
||||
assert [entry["file_name"] for entry in result["loras"]] == ["remote_file", "local_only"]
|
||||
assert [entry["weight"] for entry in result["loras"]] == [0.35, 0.6]
|
||||
assert result["loras"][1]["existsLocally"] is True
|
||||
assert scanner.queries == ["local_only"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_keeps_mixed_local_and_civitai_loras(monkeypatch):
|
||||
remote_info = {
|
||||
"id": 100,
|
||||
"modelId": 200,
|
||||
"model": {"name": "Remote LoRA", "type": "LORA"},
|
||||
"name": "v1",
|
||||
"files": [
|
||||
{
|
||||
"type": "Model",
|
||||
"primary": True,
|
||||
"name": "remote.safetensors",
|
||||
"hashes": {"SHA256": "b" * 64},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_by_hash(self, model_hash):
|
||||
assert model_hash == "bbbbbbbbbb"
|
||||
return remote_info, None
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.automatic.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local_only": local_lora()})
|
||||
metadata_text = (
|
||||
"portrait <lora:local_only:0.6> <lora:remote:0.9>\n"
|
||||
"Steps: 20, Seed: 1, "
|
||||
'Hashes: {"lora:local_only": "", "lora:remote": "bbbbbbbbbb"}'
|
||||
)
|
||||
|
||||
result = await AutomaticMetadataParser().parse_metadata(metadata_text, scanner)
|
||||
|
||||
assert [entry["name"] for entry in result["loras"]] == ["Local Only", "Remote LoRA"]
|
||||
assert [entry["weight"] for entry in result["loras"]] == [0.6, 0.9]
|
||||
assert result["loras"][0]["existsLocally"] is True
|
||||
assert result["loras"][1]["existsLocally"] is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_extracts_checkpoint_from_model_hash(monkeypatch):
|
||||
checkpoint_info = {
|
||||
|
||||
@@ -2,6 +2,38 @@ import pytest
|
||||
import json
|
||||
from py.recipes.parsers.comfy import ComfyMetadataParser
|
||||
|
||||
|
||||
class LocalRecipeScanner:
|
||||
class LoraScanner:
|
||||
@staticmethod
|
||||
def has_hash(model_hash):
|
||||
return False
|
||||
|
||||
def __init__(self, models):
|
||||
self.models = models
|
||||
self.queries = []
|
||||
self.base_models = []
|
||||
self._lora_scanner = self.LoraScanner()
|
||||
|
||||
async def get_local_lora(self, name, base_model=None):
|
||||
self.queries.append(name)
|
||||
self.base_models.append(base_model)
|
||||
return self.models.get(name)
|
||||
|
||||
|
||||
def local_lora(file_name):
|
||||
return {
|
||||
"file_path": f"/models/loras/{file_name}.safetensors",
|
||||
"file_name": file_name.rsplit("/", 1)[-1],
|
||||
"model_name": file_name.rsplit("/", 1)[-1],
|
||||
"sha256": file_name[0] * 64,
|
||||
"size": 4096,
|
||||
"base_model": "SDXL 1.0",
|
||||
"preview_url": "",
|
||||
"civitai": None,
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_without_loras(monkeypatch):
|
||||
checkpoint_info = {
|
||||
@@ -84,6 +116,140 @@ async def test_parse_metadata_without_loras(monkeypatch):
|
||||
assert result["gen_params"]["size"] == "1024x1024"
|
||||
assert result["from_comfy_metadata"] is True
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_resolves_standard_and_manager_local_loras(monkeypatch):
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_version_info(self, version_id):
|
||||
raise AssertionError("Local LoRAs must not query Civitai")
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.comfy.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({
|
||||
"styles/standard.safetensors": local_lora("standard"),
|
||||
"manager": local_lora("manager"),
|
||||
})
|
||||
metadata_json = {
|
||||
"1": {
|
||||
"class_type": "LoraLoader",
|
||||
"inputs": {
|
||||
"lora_name": "styles/standard.safetensors",
|
||||
"strength_model": 0.55,
|
||||
},
|
||||
},
|
||||
"2": {
|
||||
"class_type": "LoraLoaderLM",
|
||||
"inputs": {
|
||||
"loras": {
|
||||
"__value__": [
|
||||
{"name": "manager", "strength": "0.80", "active": True},
|
||||
{"name": "disabled", "strength": 1.0, "active": False},
|
||||
{"name": "dummy", "strength": 1.0, "active": True, "_isDummy": True},
|
||||
]
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
|
||||
|
||||
assert [entry["file_name"] for entry in result["loras"]] == ["standard", "manager"]
|
||||
assert [entry["weight"] for entry in result["loras"]] == [0.55, 0.8]
|
||||
assert all(isinstance(entry["weight"], float) for entry in result["loras"])
|
||||
assert all(entry["existsLocally"] is True for entry in result["loras"])
|
||||
assert all(entry["isDeleted"] is False for entry in result["loras"])
|
||||
assert scanner.queries == ["styles/standard.safetensors", "manager"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_defaults_malformed_weight_and_passes_checkpoint_base_model(monkeypatch):
|
||||
checkpoint_info = {
|
||||
"id": 456,
|
||||
"modelId": 123,
|
||||
"model": {"name": "Checkpoint", "type": "checkpoint"},
|
||||
"name": "v1",
|
||||
"baseModel": "SDXL 1.0",
|
||||
}
|
||||
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_version_info(self, version_id):
|
||||
return checkpoint_info, None
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.comfy.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"style": local_lora("style")})
|
||||
metadata_json = {
|
||||
"1": {"class_type": "LoraLoader", "inputs": {"lora_name": "style", "strength_model": "invalid"}},
|
||||
"2": {"class_type": "CheckpointLoaderSimple", "inputs": {"ckpt_name": "civitai:123@456"}},
|
||||
}
|
||||
|
||||
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
|
||||
|
||||
assert result["loras"][0]["weight"] == 1.0
|
||||
assert scanner.base_models == ["SDXL 1.0"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_keeps_civitai_urn_with_local_lora(monkeypatch):
|
||||
remote_info = {
|
||||
"id": 456,
|
||||
"modelId": 123,
|
||||
"model": {"name": "Remote LoRA", "type": "LORA"},
|
||||
"name": "v1",
|
||||
"files": [
|
||||
{
|
||||
"type": "Model",
|
||||
"primary": True,
|
||||
"name": "remote.safetensors",
|
||||
"hashes": {"SHA256": "c" * 64},
|
||||
}
|
||||
],
|
||||
}
|
||||
|
||||
async def fake_metadata_provider():
|
||||
class Provider:
|
||||
async def get_model_version_info(self, version_id):
|
||||
assert version_id == "456"
|
||||
return remote_info, None
|
||||
|
||||
return Provider()
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.recipes.parsers.comfy.get_default_metadata_provider",
|
||||
fake_metadata_provider,
|
||||
)
|
||||
scanner = LocalRecipeScanner({"local": local_lora("local")})
|
||||
metadata_json = {
|
||||
"1": {
|
||||
"class_type": "LoraLoader",
|
||||
"inputs": {"lora_name": "local", "strength_model": 0.4},
|
||||
},
|
||||
"2": {
|
||||
"class_type": "LoraLoader",
|
||||
"inputs": {
|
||||
"lora_name": "urn:air:sdxl:lora:civitai:123@456",
|
||||
"strength_model": 0.9,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
result = await ComfyMetadataParser().parse_metadata(json.dumps(metadata_json), scanner)
|
||||
|
||||
assert [entry["name"] for entry in result["loras"]] == ["local", "Remote LoRA"]
|
||||
assert [entry["weight"] for entry in result["loras"]] == [0.4, 0.9]
|
||||
assert result["loras"][0]["existsLocally"] is True
|
||||
assert result["loras"][1]["id"] == 456
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_parse_metadata_without_extra_metadata(monkeypatch):
|
||||
async def fake_metadata_provider():
|
||||
|
||||
@@ -82,14 +82,18 @@ def stub_metadata(monkeypatch):
|
||||
|
||||
|
||||
class DummyScanner:
|
||||
def __init__(self, exists: bool = False):
|
||||
def __init__(self, exists: bool = False, raw_data=None):
|
||||
self.exists = exists
|
||||
self.calls = []
|
||||
self._cache = SimpleNamespace(raw_data=list(raw_data or []))
|
||||
|
||||
async def check_model_version_exists(self, version_id):
|
||||
self.calls.append(version_id)
|
||||
return self.exists
|
||||
|
||||
async def get_cached_data(self):
|
||||
return self._cache
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def scanners(monkeypatch):
|
||||
@@ -1692,3 +1696,310 @@ async def test_download_proceeds_when_history_skip_disabled(monkeypatch, scanner
|
||||
assert result.get("skipped") is not True
|
||||
history_service.has_been_downloaded.assert_not_called()
|
||||
execute_download.assert_awaited_once()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multi-file downloads within a single model version (#1058)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _multi_file_payload(include_hashes: bool = True):
|
||||
"""Version payload with two weight files under the same version."""
|
||||
files = [
|
||||
{
|
||||
"id": 1001,
|
||||
"type": "Model",
|
||||
"primary": True,
|
||||
"downloadUrl": "https://example.invalid/file-a.safetensors",
|
||||
"name": "file-a.safetensors",
|
||||
},
|
||||
{
|
||||
"id": 1002,
|
||||
"type": "Model",
|
||||
"primary": False,
|
||||
"downloadUrl": "https://example.invalid/file-b.safetensors",
|
||||
"name": "file-b.safetensors",
|
||||
},
|
||||
]
|
||||
if include_hashes:
|
||||
files[0]["hashes"] = {"SHA256": "AAA111"}
|
||||
files[1]["hashes"] = {"SHA256": "BBB222"}
|
||||
return {
|
||||
"id": 42,
|
||||
"modelId": 7,
|
||||
"model": {"type": "LoRA", "tags": ["fantasy"]},
|
||||
"baseModel": "BaseModel",
|
||||
"creator": {"username": "Author"},
|
||||
"files": files,
|
||||
}
|
||||
|
||||
|
||||
def _local_entry(file_name: str, sha256: str, version_id: int = 42):
|
||||
"""A library cache entry for one already-downloaded file of a version."""
|
||||
return {
|
||||
"file_name": file_name,
|
||||
"file_path": f"/tmp/{file_name}.safetensors",
|
||||
"sha256": sha256,
|
||||
"civitai": {"id": version_id, "modelId": 7},
|
||||
}
|
||||
|
||||
|
||||
def _stub_history_service(monkeypatch, has_been_downloaded: bool = False):
|
||||
history_service = AsyncMock()
|
||||
history_service.has_been_downloaded = AsyncMock(
|
||||
return_value=has_been_downloaded
|
||||
)
|
||||
history_service.mark_downloaded = AsyncMock()
|
||||
monkeypatch.setattr(
|
||||
ServiceRegistry,
|
||||
"get_downloaded_version_history_service",
|
||||
AsyncMock(return_value=history_service),
|
||||
)
|
||||
return history_service
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_allows_other_file_of_in_library_version(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""A different file of an in-library version must still download (#1058)."""
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1002, "type": "Model", "name": "file-b.safetensors"},
|
||||
)
|
||||
|
||||
assert result["success"] is True
|
||||
execute_download.assert_awaited_once()
|
||||
# Version-level gates must not run for an explicit file selection
|
||||
assert scanners.lora.calls == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_blocks_same_file_of_in_library_version(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""Re-downloading the SAME file of an in-library version stays blocked."""
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1001, "type": "Model", "name": "file-a.safetensors"},
|
||||
)
|
||||
|
||||
assert result["success"] is False
|
||||
assert "file-a.safetensors" in result["error"]
|
||||
assert "already exists in lora library" in result["error"]
|
||||
execute_download.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_unresolvable_file_params_falls_back_to_version_gate(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""file_params that match no file must not bypass version-level gates."""
|
||||
scanners.lora.exists = True
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 9999, "type": "Model"},
|
||||
)
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["error"] == "Model version already exists in lora library"
|
||||
execute_download.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_empty_file_params_treated_as_no_selection(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""An empty file_params dict is normalized to None (version-level gates)."""
|
||||
scanners.lora.exists = True
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={},
|
||||
)
|
||||
|
||||
assert result["success"] is False
|
||||
assert result["error"] == "Model version already exists in lora library"
|
||||
# Early version-level gate fired (file_params normalized to None)
|
||||
assert scanners.lora.calls == [42]
|
||||
execute_download.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_explicit_file_bypasses_history_skip(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""Explicit file selection bypasses the previously-downloaded skip."""
|
||||
get_settings_manager().settings[
|
||||
"skip_previously_downloaded_model_versions"
|
||||
] = True
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
history_service = _stub_history_service(monkeypatch, has_been_downloaded=True)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1002, "type": "Model", "name": "file-b.safetensors"},
|
||||
)
|
||||
|
||||
assert result["success"] is True
|
||||
execute_download.assert_awaited_once()
|
||||
# History gate is bypassed before it even queries the service
|
||||
history_service.has_been_downloaded.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_file_match_by_name_when_local_hash_missing(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""Legacy local metadata without sha256 falls back to name matching."""
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-a", ""))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1001, "type": "Model"},
|
||||
)
|
||||
|
||||
assert result["success"] is False
|
||||
assert "already exists in lora library" in result["error"]
|
||||
execute_download.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_no_false_positive_when_both_hashes_empty(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""Two empty hashes must never compare equal; name decides instead."""
|
||||
# Local entry for a DIFFERENT file of the same version, no hash stored
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-b", ""))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload(include_hashes=False)
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_version_id=42,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1001, "type": "Model"},
|
||||
)
|
||||
|
||||
assert result["success"] is True
|
||||
execute_download.assert_awaited_once()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_id_only_with_file_params_resolves_file(
|
||||
monkeypatch, scanners, metadata_provider
|
||||
):
|
||||
"""model_id-only requests with file_params resolve the file post-fetch."""
|
||||
scanners.lora.exists = True # version-level index says "in library"
|
||||
scanners.lora._cache.raw_data.append(_local_entry("file-a", "aaa111"))
|
||||
metadata_provider.get_model_version = AsyncMock(
|
||||
return_value=_multi_file_payload()
|
||||
)
|
||||
_stub_history_service(monkeypatch)
|
||||
|
||||
execute_download = AsyncMock(return_value={"success": True, "download_id": "done"})
|
||||
monkeypatch.setattr(
|
||||
DownloadManager, "_execute_download", execute_download, raising=False
|
||||
)
|
||||
|
||||
manager = DownloadManager()
|
||||
result = await manager.download_from_civitai(
|
||||
model_id=7,
|
||||
save_dir="/tmp",
|
||||
file_params={"id": 1002, "type": "Model"},
|
||||
)
|
||||
|
||||
assert result["success"] is True
|
||||
execute_download.assert_awaited_once()
|
||||
|
||||
|
||||
def test_resolve_target_file_by_id():
|
||||
files = _multi_file_payload()["files"]
|
||||
resolved = DownloadManager._resolve_target_file(files, {"id": 1002})
|
||||
assert resolved is files[1]
|
||||
|
||||
|
||||
def test_resolve_target_file_by_primary_flag():
|
||||
files = _multi_file_payload()["files"]
|
||||
resolved = DownloadManager._resolve_target_file(files, {"isPrimary": True})
|
||||
assert resolved is files[0]
|
||||
|
||||
|
||||
def test_resolve_target_file_returns_none_for_no_match():
|
||||
files = _multi_file_payload()["files"]
|
||||
assert DownloadManager._resolve_target_file(files, {"id": 9999}) is None
|
||||
assert DownloadManager._resolve_target_file(files, None) is None
|
||||
assert DownloadManager._resolve_target_file(files, {}) is None
|
||||
|
||||
@@ -5,6 +5,8 @@ Covers the new ``download_id``-based code paths in
|
||||
compatibility with ``id``.
|
||||
"""
|
||||
|
||||
import json
|
||||
import sqlite3
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
@@ -191,3 +193,131 @@ async def test_retry_unknown_download_id(tmp_path: Path) -> None:
|
||||
|
||||
item = await svc.retry_from_history(download_id="dl-nope")
|
||||
assert item is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# file_params persistence across queue -> history -> retry (#1058)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_params_survive_queue_to_history(tmp_path: Path) -> None:
|
||||
"""complete_download copies the queue row's file_params into history."""
|
||||
svc = _make_service(tmp_path)
|
||||
await svc.add_to_queue(
|
||||
download_id="dl-fp",
|
||||
model_id=1,
|
||||
model_version_id=100,
|
||||
file_params={"id": 1002, "type": "Model"},
|
||||
)
|
||||
|
||||
await svc.complete_download("dl-fp", status="failed", error="boom")
|
||||
|
||||
history = await svc.get_history()
|
||||
assert len(history["items"]) == 1
|
||||
assert json.loads(history["items"][0]["file_params"]) == {
|
||||
"id": 1002,
|
||||
"type": "Model",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_restores_file_params(tmp_path: Path) -> None:
|
||||
"""retry_from_history re-queues with the originally selected file."""
|
||||
svc = _make_service(tmp_path)
|
||||
await svc.add_to_history(
|
||||
download_id="dl-fail-fp",
|
||||
model_id=1,
|
||||
model_version_id=100,
|
||||
status="failed",
|
||||
file_params={"id": 1002, "type": "Model"},
|
||||
)
|
||||
|
||||
item = await svc.retry_from_history(download_id="dl-fail-fp")
|
||||
|
||||
assert item is not None
|
||||
assert item["status"] == "queued"
|
||||
assert json.loads(item["file_params"]) == {"id": 1002, "type": "Model"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_retry_all_restores_file_params(tmp_path: Path) -> None:
|
||||
"""retry_all_failed preserves file_params for every re-queued item."""
|
||||
svc = _make_service(tmp_path)
|
||||
await svc.add_to_history(
|
||||
download_id="dl-f1", status="failed", file_params={"id": 1001}
|
||||
)
|
||||
await svc.add_to_history(download_id="dl-f2", status="canceled")
|
||||
|
||||
count = await svc.retry_all_failed()
|
||||
assert count == 2
|
||||
|
||||
queue = await svc.get_queue()
|
||||
restored = sorted(
|
||||
(q["file_params"] or "") for q in queue
|
||||
)
|
||||
assert restored[0] == "" # dl-f2 never had file_params
|
||||
assert json.loads(restored[1]) == {"id": 1001}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_legacy_history_db_gains_file_params_column(tmp_path: Path) -> None:
|
||||
"""Databases created before the file_params column get migrated (#1058)."""
|
||||
db_path = tmp_path / "queue.sqlite"
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.executescript(
|
||||
"""
|
||||
CREATE TABLE download_queue (
|
||||
download_id TEXT PRIMARY KEY,
|
||||
model_id INTEGER,
|
||||
model_version_id INTEGER,
|
||||
model_name TEXT NOT NULL DEFAULT '',
|
||||
version_name TEXT DEFAULT '',
|
||||
thumbnail_url TEXT DEFAULT '',
|
||||
source TEXT,
|
||||
file_params TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'queued',
|
||||
priority INTEGER DEFAULT 0,
|
||||
progress INTEGER DEFAULT 0,
|
||||
bytes_downloaded INTEGER DEFAULT 0,
|
||||
total_bytes INTEGER,
|
||||
bytes_per_second REAL DEFAULT 0.0,
|
||||
error TEXT,
|
||||
file_path TEXT,
|
||||
added_at REAL NOT NULL,
|
||||
started_at REAL,
|
||||
completed_at REAL
|
||||
);
|
||||
CREATE TABLE download_history (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
download_id TEXT,
|
||||
model_id INTEGER,
|
||||
model_version_id INTEGER,
|
||||
model_name TEXT NOT NULL DEFAULT '',
|
||||
version_name TEXT DEFAULT '',
|
||||
thumbnail_url TEXT DEFAULT '',
|
||||
status TEXT NOT NULL,
|
||||
error TEXT,
|
||||
file_path TEXT,
|
||||
bytes_downloaded INTEGER DEFAULT 0,
|
||||
total_bytes INTEGER,
|
||||
completed_at REAL NOT NULL,
|
||||
is_already_exists INTEGER DEFAULT 0
|
||||
);
|
||||
"""
|
||||
)
|
||||
conn.close()
|
||||
|
||||
svc = DownloadQueueService(db_path=str(db_path))
|
||||
|
||||
# The migrated table accepts file_params writes
|
||||
await svc.add_to_history(
|
||||
download_id="dl-legacy-fp", status="failed", file_params={"id": 5}
|
||||
)
|
||||
history = await svc.get_history()
|
||||
assert json.loads(history["items"][0]["file_params"]) == {"id": 5}
|
||||
|
||||
# And retry restores them
|
||||
item = await svc.retry_from_history(download_id="dl-legacy-fp")
|
||||
assert item is not None
|
||||
assert json.loads(item["file_params"]) == {"id": 5}
|
||||
|
||||
@@ -68,3 +68,94 @@ async def test_download_history_bulk_lookup(tmp_path: Path) -> None:
|
||||
5: {501, 502},
|
||||
6: {601},
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_per_file_history_tracking(tmp_path: Path) -> None:
|
||||
"""Per-file records coexist with the version-level row (#1058)."""
|
||||
db_path = tmp_path / "download-history.sqlite"
|
||||
service = DownloadedVersionHistoryService(
|
||||
str(db_path),
|
||||
settings_manager=DummySettings(),
|
||||
)
|
||||
|
||||
await service.mark_downloaded(
|
||||
"lora", 101, model_id=11, source="download",
|
||||
file_path="/models/a.safetensors", file_id=1001, file_name="a.safetensors",
|
||||
)
|
||||
await service.mark_downloaded(
|
||||
"lora", 101, model_id=11, source="download",
|
||||
file_path="/models/b.safetensors", file_id=1002, file_name="b.safetensors",
|
||||
)
|
||||
|
||||
assert await service.get_downloaded_file_ids("lora", 101) == [1001, 1002]
|
||||
# Version-level tracking remains single-row per version
|
||||
assert await service.get_downloaded_version_ids("lora", 11) == [101]
|
||||
|
||||
# Re-downloading the same file updates in place, no duplicate
|
||||
await service.mark_downloaded(
|
||||
"lora", 101, source="download", file_id=1001, file_name="a.safetensors",
|
||||
)
|
||||
assert await service.get_downloaded_file_ids("lora", 101) == [1001, 1002]
|
||||
|
||||
# Single-file deletion keeps the sibling record
|
||||
await service.mark_file_deleted("lora", 101, 1001)
|
||||
assert await service.get_downloaded_file_ids("lora", 101) == [1002]
|
||||
|
||||
# Whole-version deletion clears per-file records
|
||||
await service.mark_as_deleted("lora", 101)
|
||||
assert await service.get_downloaded_file_ids("lora", 101) == []
|
||||
assert await service.has_been_downloaded("lora", 101) is False
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_per_file_history_ignores_invalid_ids(tmp_path: Path) -> None:
|
||||
service = DownloadedVersionHistoryService(
|
||||
str(tmp_path / "download-history.sqlite"),
|
||||
settings_manager=DummySettings(),
|
||||
)
|
||||
|
||||
# mark_downloaded without a file id only touches the version-level table
|
||||
await service.mark_downloaded("lora", 201, model_id=21, source="scan")
|
||||
assert await service.get_downloaded_file_ids("lora", 201) == []
|
||||
|
||||
# Invalid inputs are no-ops
|
||||
await service.mark_file_deleted("lora", 201, None) # type: ignore[arg-type]
|
||||
assert await service.get_downloaded_file_ids("unknown-type", 201) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_file_history_table_created_for_legacy_db(tmp_path: Path) -> None:
|
||||
"""Existing databases gain the per-file table via CREATE IF NOT EXISTS."""
|
||||
import sqlite3
|
||||
|
||||
db_path = tmp_path / "download-history.sqlite"
|
||||
conn = sqlite3.connect(db_path)
|
||||
conn.executescript(
|
||||
"""
|
||||
CREATE TABLE downloaded_model_versions (
|
||||
model_type TEXT NOT NULL,
|
||||
version_id INTEGER NOT NULL,
|
||||
model_id INTEGER,
|
||||
first_seen_at REAL NOT NULL,
|
||||
last_seen_at REAL NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
last_file_path TEXT,
|
||||
last_library_name TEXT,
|
||||
is_deleted_override INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (model_type, version_id)
|
||||
);
|
||||
"""
|
||||
)
|
||||
conn.close()
|
||||
|
||||
service = DownloadedVersionHistoryService(
|
||||
str(db_path),
|
||||
settings_manager=DummySettings(),
|
||||
)
|
||||
await service.mark_downloaded(
|
||||
"lora", 301, model_id=31, source="download",
|
||||
file_id=9001, file_name="file.safetensors",
|
||||
)
|
||||
assert await service.get_downloaded_file_ids("lora", 301) == [9001]
|
||||
assert await service.has_been_downloaded("lora", 301) is True
|
||||
|
||||
@@ -61,3 +61,87 @@ async def test_model_cache_tracks_versions_by_model_id():
|
||||
assert cache.get_versions_by_model_id(2) == [
|
||||
{'versionId': 201, 'name': 'Gamma', 'fileName': 'model-b'},
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_version_files_index_tracks_multiple_files_per_version():
|
||||
"""Two downloaded files of the same version both stay indexed (#1058)."""
|
||||
item_a = {
|
||||
'file_path': '/models/v1-a.safetensors',
|
||||
'file_name': 'model-v1-a',
|
||||
'folder': '',
|
||||
'civitai': {'id': 301, 'modelId': 3, 'name': 'Multi'},
|
||||
}
|
||||
item_b = {
|
||||
'file_path': '/models/v1-b.safetensors',
|
||||
'file_name': 'model-v1-b',
|
||||
'folder': '',
|
||||
'civitai': {'id': 301, 'modelId': 3, 'name': 'Multi'},
|
||||
}
|
||||
|
||||
cache = ModelCache(
|
||||
raw_data=[item_a, item_b],
|
||||
folders=[],
|
||||
name_display_mode='model_name',
|
||||
)
|
||||
|
||||
files = cache.get_files_by_version_id(301)
|
||||
assert {f['file_path'] for f in files} == {
|
||||
'/models/v1-a.safetensors',
|
||||
'/models/v1-b.safetensors',
|
||||
}
|
||||
|
||||
# Re-adding an existing entry must not duplicate it
|
||||
cache.add_to_version_index(item_a)
|
||||
assert len(cache.get_files_by_version_id(301)) == 2
|
||||
|
||||
# Removing the indexed file re-points version_index to the sibling
|
||||
indexed = cache.version_index[301]
|
||||
sibling = item_b if indexed is item_a else item_a
|
||||
cache.remove_from_version_index(indexed)
|
||||
|
||||
assert 301 in cache.version_index
|
||||
assert cache.version_index[301]['file_path'] == sibling['file_path']
|
||||
assert cache.get_versions_by_model_id(3) == [
|
||||
{'versionId': 301, 'name': 'Multi', 'fileName': sibling['file_name']},
|
||||
]
|
||||
remaining = cache.get_files_by_version_id(301)
|
||||
assert [f['file_path'] for f in remaining] == [sibling['file_path']]
|
||||
|
||||
# Removing the last file drops the version from all indexes
|
||||
cache.remove_from_version_index(sibling)
|
||||
assert 301 not in cache.version_index
|
||||
assert cache.get_files_by_version_id(301) == []
|
||||
assert cache.get_versions_by_model_id(3) == []
|
||||
assert 3 not in cache.model_id_index
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_version_files_index_rebuild_from_raw_data():
|
||||
"""rebuild_version_index reconstructs the multi-valued index (#1058)."""
|
||||
item_a = {
|
||||
'file_path': '/models/v1-a.safetensors',
|
||||
'file_name': 'model-v1-a',
|
||||
'folder': '',
|
||||
'civitai': {'id': 401, 'modelId': 4, 'name': 'Multi'},
|
||||
}
|
||||
item_b = {
|
||||
'file_path': '/models/v1-b.safetensors',
|
||||
'file_name': 'model-v1-b',
|
||||
'folder': '',
|
||||
'civitai': {'id': 401, 'modelId': 4, 'name': 'Multi'},
|
||||
}
|
||||
|
||||
cache = ModelCache(
|
||||
raw_data=[item_a, item_b],
|
||||
folders=[],
|
||||
name_display_mode='model_name',
|
||||
)
|
||||
|
||||
cache.version_files_index = {}
|
||||
cache.rebuild_version_index()
|
||||
|
||||
assert len(cache.get_files_by_version_id(401)) == 2
|
||||
# Invalid ids normalize to empty results
|
||||
assert cache.get_files_by_version_id('not-an-int') == []
|
||||
assert cache.get_files_by_version_id(None) == []
|
||||
|
||||
@@ -57,9 +57,33 @@ class StubLoraScanner:
|
||||
meta = self._hash_meta.get(hash_value.lower())
|
||||
return meta.get("path") if meta else None
|
||||
|
||||
async def get_model_info_by_name(self, name: str):
|
||||
async def get_model_info_by_name(
|
||||
self,
|
||||
name: str,
|
||||
*,
|
||||
require_unique: bool = False,
|
||||
base_model: str | None = None,
|
||||
):
|
||||
if require_unique or base_model:
|
||||
matches = ModelScanner.find_matching_models(
|
||||
self._cache.raw_data,
|
||||
name,
|
||||
base_model=base_model,
|
||||
extensions={".safetensors"},
|
||||
)
|
||||
if require_unique and len(matches) != 1:
|
||||
return None
|
||||
return matches[0] if matches else None
|
||||
return self._models_by_name.get(name)
|
||||
|
||||
async def find_models_by_name(self, name: str, *, base_model: str | None = None):
|
||||
return ModelScanner.find_matching_models(
|
||||
self._cache.raw_data,
|
||||
name,
|
||||
base_model=base_model,
|
||||
extensions={".safetensors"},
|
||||
)
|
||||
|
||||
def register_model(self, name: str, info: Dict[str, Any]) -> None:
|
||||
self._models_by_name[name] = info
|
||||
hash_value = (info.get("sha256") or "").lower()
|
||||
@@ -107,6 +131,39 @@ def recipe_scanner(tmp_path: Path, monkeypatch):
|
||||
settings_manager_module.reset_settings_manager()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_local_lora_lookup_requires_unambiguous_name_and_matching_base_model(recipe_scanner):
|
||||
scanner, stub = recipe_scanner
|
||||
models = [
|
||||
{
|
||||
"file_name": "style.safetensors",
|
||||
"folder": "sd15",
|
||||
"file_path": "/models/loras/sd15/style.safetensors",
|
||||
"sha256": "a" * 64,
|
||||
"base_model": "SD 1.5",
|
||||
},
|
||||
{
|
||||
"file_name": "style.safetensors",
|
||||
"folder": "sdxl",
|
||||
"file_path": "/models/loras/sdxl/style.safetensors",
|
||||
"sha256": "b" * 64,
|
||||
"base_model": "SDXL 1.0",
|
||||
},
|
||||
]
|
||||
stub._cache.raw_data = models
|
||||
stub._hash_meta["b" * 64] = {"path": models[1]["file_path"]}
|
||||
|
||||
assert await scanner.get_local_lora("style") is None
|
||||
assert await scanner.get_local_lora("style", "SDXL 1.0") is models[1]
|
||||
assert await scanner.get_local_lora("sdxl/style.safetensors", "SDXL 1.0") is models[1]
|
||||
# The lora scanner only indexes .safetensors, so a .pt name must not be
|
||||
# stripped into a cross-extension match.
|
||||
assert await scanner.get_local_lora("sdxl/style.pt", "SDXL 1.0") is None
|
||||
assert await scanner.get_local_lora("sdxl/style.safetensors", "SD 1.5") is None
|
||||
assert await scanner.get_local_lora("other/style.safetensors") is None
|
||||
assert await scanner.get_local_lora_by_hash("b" * 64) is models[1]
|
||||
|
||||
|
||||
def test_recipes_dir_uses_custom_settings_path(tmp_path: Path, monkeypatch):
|
||||
RecipeScanner._instance = None
|
||||
settings_manager_module.reset_settings_manager()
|
||||
|
||||
@@ -17,6 +17,7 @@ from py.services.recipes.errors import (
|
||||
RecipeValidationError,
|
||||
)
|
||||
from py.services.recipes.persistence_service import RecipePersistenceService
|
||||
from py.services.model_scanner import ModelScanner
|
||||
from py.recipes.parsers.civitai_image import CivitaiApiMetadataParser
|
||||
from py.utils.exif_utils import ExifUtils
|
||||
|
||||
@@ -1156,3 +1157,72 @@ async def test_analyze_local_image_fingerprint_uses_sha256_normalized_hash(tmp_p
|
||||
|
||||
assert result.payload["loras"][0]["hash"] == sha256
|
||||
assert result.payload["fingerprint"] == f"{sha256}:1.0"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_reconnect_lora_distinguishes_ambiguous_mismatched_and_missing(tmp_path):
|
||||
service = RecipePersistenceService(
|
||||
exif_utils=DummyExifUtils(),
|
||||
card_preview_width=512,
|
||||
logger=logging.getLogger("test"),
|
||||
)
|
||||
|
||||
models = [
|
||||
{
|
||||
"file_name": "style.safetensors",
|
||||
"folder": "sd15",
|
||||
"file_path": "/models/loras/sd15/style.safetensors",
|
||||
"base_model": "SD 1.5",
|
||||
},
|
||||
{
|
||||
"file_name": "style.safetensors",
|
||||
"folder": "sdxl",
|
||||
"file_path": "/models/loras/sdxl/style.safetensors",
|
||||
"base_model": "SDXL 1.0",
|
||||
},
|
||||
]
|
||||
|
||||
class DummyScanner:
|
||||
def __init__(self, recipe_path):
|
||||
self._recipe_path = recipe_path
|
||||
|
||||
async def get_recipe_json_path(self, recipe_id):
|
||||
return str(self._recipe_path)
|
||||
|
||||
async def get_local_lora(self, name, base_model=None):
|
||||
matches = ModelScanner.find_matching_models(models, name, base_model=base_model)
|
||||
return matches[0] if len(matches) == 1 else None
|
||||
|
||||
async def find_local_loras_by_name(self, name, base_model=None):
|
||||
return ModelScanner.find_matching_models(models, name, base_model=base_model)
|
||||
|
||||
def write_recipe(base_model):
|
||||
recipe_path = tmp_path / "recipe.json"
|
||||
recipe_path.write_text(
|
||||
json.dumps({"id": "r1", "base_model": base_model, "loras": []})
|
||||
)
|
||||
return DummyScanner(recipe_path)
|
||||
|
||||
# Ambiguous bare name: two candidates survive (recipe base model unknown)
|
||||
scanner = write_recipe("")
|
||||
with pytest.raises(RecipeValidationError, match="include the folder path"):
|
||||
await service.reconnect_lora(
|
||||
recipe_scanner=scanner, recipe_id="r1", lora_index=0, target_name="style"
|
||||
)
|
||||
|
||||
# Confident base-model mismatch: the only candidate belongs to another family
|
||||
scanner = write_recipe("SD 1.5")
|
||||
with pytest.raises(RecipeValidationError, match="different base model"):
|
||||
await service.reconnect_lora(
|
||||
recipe_scanner=scanner,
|
||||
recipe_id="r1",
|
||||
lora_index=0,
|
||||
target_name="sdxl/style",
|
||||
)
|
||||
|
||||
# No candidate at all
|
||||
scanner = write_recipe("SDXL 1.0")
|
||||
with pytest.raises(RecipeNotFoundError, match="not found"):
|
||||
await service.reconnect_lora(
|
||||
recipe_scanner=scanner, recipe_id="r1", lora_index=0, target_name="missing"
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user