mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-09-21 03:01:27 -03:00
Compare commits
5 Commits
84146b62fd
...
326df32933
| Author | SHA1 | Date | |
|---|---|---|---|
 Will Miao
|
326df32933 | ||
|
Will Miao
|
31ef9ffa06 | ||
|
Will Miao
|
38d4c59b4c | ||
|
Will Miao
|
b9bf006998 | ||
|
Will Miao
|
5ab0e88abc |
+17
-5
@@ -62,13 +62,23 @@ Environment variable overrides: `LLM_API_KEY`, `LLM_MODEL`, `LLM_API_BASE`, `LLM
|
||||
|
||||
### enrich_hf_metadata
|
||||
|
||||
Enriches HuggingFace-downloaded models with metadata extracted by an LLM from the HF model card.
|
||||
Enriches models linked to an external model site with metadata extracted by an LLM from the site's model card (README).
|
||||
|
||||
**Entry point**: Right-click context menu → "Enrich Metadata (Agent)"
|
||||
**Entry point**: Right-click context menu → "Enrich Metadata with AI"
|
||||
|
||||
**Supported model sources**:
|
||||
|
||||
| Platform | Link | AI enrichment | Direct download |
|
||||
| --- | --- | --- | --- |
|
||||
| Hugging Face | yes | yes | yes |
|
||||
| ModelScope | yes | yes | yes |
|
||||
| TensorArt | yes | no (see below) | no |
|
||||
|
||||
TensorArt is link-only: `tensor.art` sits behind a Cloudflare managed challenge and its internal API requires session authorization, so the backend cannot read its model pages. Linking still stores the canonical page URL and the "View on TensorArt" link works.
|
||||
|
||||
**What it does**:
|
||||
1. Reads the model's `.metadata.json` to get the `hf_url`
|
||||
2. Fetches the README.md from the HuggingFace repository
|
||||
1. Reads the model's `.metadata.json` to get the source (`source_platform` + `source_url`, or the legacy `hf_url`)
|
||||
2. Fetches the model card through the provider in `py/services/model_sources/`
|
||||
3. Sends the README + local metadata to the LLM for structured extraction
|
||||
4. Writes extracted fields to `.metadata.json`:
|
||||
- `base_model` — only if current value is empty
|
||||
@@ -81,6 +91,8 @@ Enriches HuggingFace-downloaded models with metadata extracted by an LLM from th
|
||||
6. Updates the scanner cache
|
||||
7. Broadcasts WebSocket progress events
|
||||
|
||||
Models with no source, an unknown source, or a source without model-card access (TensorArt) are skipped with an explicit reason and counted in the run summary.
|
||||
|
||||
**Model types**: LoRA, Checkpoint, Embedding
|
||||
|
||||
## Adding a New Skill
|
||||
@@ -129,7 +141,7 @@ Use `{{variable}}` placeholders that will be replaced with data from the `prepar
|
||||
```markdown
|
||||
You are an expert assistant...
|
||||
|
||||
Model URL: {{hf_url}}
|
||||
Model URL: {{source_url}}
|
||||
README content:
|
||||
{{readme_content}}
|
||||
|
||||
|
||||
@@ -33,6 +33,15 @@ Locales: `en`, `zh-CN`, `zh-TW`, `ja`, `ko`, `fr`, `de`, `es`, `ru`, `he` (RTL).
|
||||
> in the same pass. The `folder_paths` JSON snippet shown in that state lives in
|
||||
> `templates/other.html`, **not** in the locale files, so it is never translated — only the
|
||||
> surrounding prose is. Terminology added in §2.
|
||||
>
|
||||
> **Status (2026-09, model sources):** models can now be linked to ModelScope and TensorArt
|
||||
> alongside Hugging Face, which added 15 keys (`modelCard.actions.viewOnSource`,
|
||||
> `loras.contextMenu.linkModelSource`, `modals.linkModelSource.*`,
|
||||
> `modals.model.versions.sourceGroupInfo`, `toast.contextMenu.enrichNeedsSource`,
|
||||
> `toast.contextMenu.enrichUnsupportedSource`) and refreshed the two `enrichHfAgent` labels,
|
||||
> which had hardcoded "HF" for a button that now also enriches ModelScope models. The
|
||||
> `modals.linkModelSource.urlPlaceholder` value stays byte-identical to `en.json` (it is a URL,
|
||||
> the §6 exception). Terminology in §2, "Model source feature".
|
||||
|
||||
---
|
||||
|
||||
@@ -292,6 +301,35 @@ physically exist:
|
||||
`settings.json` and `ComfyUI` stay verbatim in every locale; "reload this page" / "restart
|
||||
LoRA Manager" reuse each locale's existing restart wording (`settings.extraFolderPaths.*`).
|
||||
|
||||
### Model source feature (Hugging Face / ModelScope / TensorArt)
|
||||
|
||||
A model file can be linked to the page of an external model site. **Hugging Face**,
|
||||
**ModelScope** and **TensorArt** are brand names and stay Latin in every locale (R3); the
|
||||
generic nouns around them are translated:
|
||||
|
||||
| Term | Rendering |
|
||||
|---|---|
|
||||
| model source | zh-CN 模型来源 · zh-TW 模型來源 · ja モデルソース · ko 모델 소스 · fr source de modèle · de Modellquelle · es fuente de modelo · ru источник модели · he מקור מודל |
|
||||
| model page | zh-CN 模型页面 · zh-TW 模型頁面 · ja モデルページ · ko 모델 페이지 · fr page du modèle · de Modellseite · es página del modelo · ru страница модели · he עמוד המודל |
|
||||
| model card | zh-CN 模型卡 · zh-TW 模型卡 · ja モデルカード · ko 모델 카드 · fr fiche de modèle · de Modellkarte · es ficha de modelo · ru карточка модели · he כרטיס מודל |
|
||||
| AI enrichment (noun) | reuse the existing pair per locale: zh-CN 增强 · zh-TW 增強 · ja 補完 · ko 보강 · fr enrichissement (par IA) · de Anreicherung (KI-) · es enriquecimiento (con IA) · ru обогащение (с помощью ИИ) · he העשרה (AI) |
|
||||
|
||||
`modelCard.actions.viewOnSource` ("View on {source}") follows each locale's existing
|
||||
`viewOnHuggingFace` pattern — de `Auf … ansehen`, ru `Открыть …`, he `צפייה ב-…`,
|
||||
ja `… で見る`, ko `…에서 보기`, zh `在 … 查看`, fr `Voir sur …`, es `Ver en …`. `{source}` is
|
||||
replaced at runtime with the untranslated platform name, so the brand never appears inside the
|
||||
translated text.
|
||||
|
||||
`modals.linkModelSource.enrichNote` states the rule that only sites exposing a readable model
|
||||
card can be enriched and names TensorArt as the current exception. Keep the parenthetical
|
||||
exception in sync if another link-only source is ever added — the sentence is deliberately
|
||||
phrased as a rule, not as an apology for one site.
|
||||
|
||||
The context-menu and bulk-operation enrichment entry points read **"Enrich Metadata with AI"**
|
||||
in `en`, not "Enrich HF Metadata": they cover ModelScope as well, so no locale may reintroduce
|
||||
an `HF` qualifier in `loras.contextMenu.enrichHfAgent` / `loras.bulkOperations.enrichHfAgent`
|
||||
(the key names keep the historical `Hf`; only the values changed).
|
||||
|
||||
---
|
||||
|
||||
## 3. Cross-cutting confusion hot-spots (must-fix list)
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "Auf CivitAI anzeigen",
|
||||
"notAvailableFromCivitai": "Nicht auf CivitAI verfügbar",
|
||||
"viewOnHuggingFace": "Auf Hugging Face ansehen",
|
||||
"viewOnSource": "Auf {source} ansehen",
|
||||
"sendToWorkflow": "An ComfyUI senden (Klick: Anhängen, Shift+Klick: Ersetzen)",
|
||||
"copyLoRASyntax": "LoRA-Syntax kopieren",
|
||||
"checkpointNameCopied": "Checkpoint-Name kopiert",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "Automatische Organisation abgeschlossen",
|
||||
"error": "Fehler: {error}"
|
||||
},
|
||||
"enrichHfAgent": "HF-Metadaten mit KI anreichern"
|
||||
"enrichHfAgent": "Metadaten mit KI anreichern"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "CivitAI-Daten aktualisieren",
|
||||
"checkUpdates": "Updates prüfen",
|
||||
"linkModel": "Modell verknüpfen",
|
||||
"linkCivitai": "Mit CivitAI neu verknüpfen",
|
||||
"linkHuggingFace": "Mit HuggingFace verknüpfen",
|
||||
"linkModelSource": "Mit Modellquelle verknüpfen",
|
||||
"copySyntax": "LoRA-Syntax kopieren",
|
||||
"copyFilename": "Modell-Dateiname kopieren",
|
||||
"copyRecipeSyntax": "Rezept-Syntax kopieren",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "Alle LoRAs anzeigen",
|
||||
"downloadMissingLoras": "Fehlende LoRAs herunterladen",
|
||||
"deleteRecipe": "Rezept löschen",
|
||||
"enrichHfAgent": "HF-Metadaten mit KI anreichern"
|
||||
"enrichHfAgent": "Metadaten mit KI anreichern"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "Modell von URL herunterladen",
|
||||
"titleWithType": "{type} von URL herunterladen",
|
||||
"civitaiUrl": "CivitAI URL:",
|
||||
"civitaiUrl": "Modell-URL:",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "Geben Sie eine CivitAI-, CivArchive- oder Hugging Face-URL pro Zeile ein. Unterstützt mehrere URLs für den Batch-Download.",
|
||||
"urlHint": "Geben Sie eine CivitAI-, CivArchive-, Hugging Face- oder ModelScope-URL pro Zeile ein. Unterstützt mehrere URLs für den Batch-Download.",
|
||||
"selectHfFiles": "Datei(en) zum Herunterladen aus diesem Repository auswählen:",
|
||||
"selectAll": "Alle auswählen",
|
||||
"fetchingRepoFiles": "Repository-Dateien werden abgerufen...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "In Bibliothek"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Ungültiges CivitAI URL-Format",
|
||||
"invalidUrl": "Ungültiges Modell-URL-Format",
|
||||
"noVersions": "Keine Versionen für dieses Modell verfügbar",
|
||||
"mixedSources": "CivitAI- und Hugging Face-URLs können nicht in derselben Charge gemischt werden.",
|
||||
"mixedSources": "CivitAI- und Hugging Face-/ModelScope-URLs können nicht in derselben Charge gemischt werden.",
|
||||
"noModelFiles": "In diesem Repository wurden keine Modelldateien gefunden."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "Ordnerpfad eingeben oder aus Baum unten auswählen...",
|
||||
"root": "Stammverzeichnis"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "Mit HuggingFace verknüpfen",
|
||||
"infoText": "Fügen Sie die HuggingFace-Repository-URL ein, um dieses Modell zuzuordnen. Dies ermöglicht die KI-gestützte Metadatenanreicherung.",
|
||||
"urlLabel": "HuggingFace-Repository-URL:",
|
||||
"linkModelSource": {
|
||||
"title": "Mit Modellquelle verknüpfen",
|
||||
"infoText": "Fügen Sie die URL der Modellseite ein, um dieses Modell seiner Quelle zuzuordnen. Die Verknüpfung ermöglicht die KI-gestützte Metadatenanreicherung für Modelle von Hugging Face und ModelScope.",
|
||||
"urlLabel": "URL der Modellseite:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "Geben Sie die vollständige URL des HuggingFace-Repositorys ein.",
|
||||
"helpText": "Geben Sie die vollständige URL der Modellseite ein. Unterstützte Websites:",
|
||||
"enrichNote": "Die KI-Anreicherung benötigt eine lesbare Modellkarte. Websites, die keine bereitstellen (derzeit TensorArt), können nur verknüpft werden.",
|
||||
"urlRequired": "Bitte geben Sie die URL der Modellseite ein.",
|
||||
"invalidUrl": "Nicht unterstützte URL. Unterstützte Websites: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "Modellquelle wird verknüpft...",
|
||||
"confirmAction": "Speichern & Verknüpfen"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "Noch keine Versionshistorie für dieses Modell vorhanden.",
|
||||
"error": "Versionen konnten nicht geladen werden.",
|
||||
"missingModelId": "Für dieses Modell ist keine CivitAI-Model-ID vorhanden.",
|
||||
"hfGroupInfo": "Dies ist eine HuggingFace-Modellgruppe. Öffnen Sie die Bibliothek, um alle Versionen im Raster zu sehen.",
|
||||
"sourceGroupInfo": "Dies ist eine {source}-Modellgruppe. Öffnen Sie die Bibliothek, um alle Versionen im Raster zu sehen.",
|
||||
"confirm": {
|
||||
"delete": "Diese Version aus Ihrer Bibliothek löschen?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "Modell erfolgreich über CivitArchive neu verknüpft",
|
||||
"fetchMetadataFirst": "Bitte rufen Sie zuerst Metadaten von CivitAI ab",
|
||||
"noCivitaiInfo": "Keine CivitAI-Informationen verfügbar",
|
||||
"missingHash": "Modell-Hash nicht verfügbar"
|
||||
"missingHash": "Modell-Hash nicht verfügbar",
|
||||
"enrichNeedsSource": "Verknüpfen Sie dieses Modell zuerst mit einer Modellquelle (Modell verknüpfen → Mit Modellquelle verknüpfen)",
|
||||
"enrichUnsupportedSource": "Die KI-Anreicherung ist für {source}-Modelle nicht verfügbar"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "Beispielbilder-Pfad erfolgreich aktualisiert",
|
||||
|
||||
+22
-15
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "View on CivitAI",
|
||||
"notAvailableFromCivitai": "Not available from CivitAI",
|
||||
"viewOnHuggingFace": "View on Hugging Face",
|
||||
"viewOnSource": "View on {source}",
|
||||
"sendToWorkflow": "Send to ComfyUI (Click: Append, Shift+Click: Replace)",
|
||||
"copyLoRASyntax": "Copy LoRA Syntax",
|
||||
"checkpointNameCopied": "Checkpoint name copied",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "Auto-organize complete",
|
||||
"error": "Error: {error}"
|
||||
},
|
||||
"enrichHfAgent": "Enrich HF Metadata (AI)"
|
||||
"enrichHfAgent": "Enrich Metadata with AI"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "Refresh CivitAI Data",
|
||||
"checkUpdates": "Check Updates",
|
||||
"linkModel": "Link Model",
|
||||
"linkCivitai": "Link to CivitAI",
|
||||
"linkHuggingFace": "Link to HuggingFace",
|
||||
"linkModelSource": "Link to Model Source",
|
||||
"copySyntax": "Copy LoRA Syntax",
|
||||
"copyFilename": "Copy Model Filename",
|
||||
"copyRecipeSyntax": "Copy Recipe Syntax",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "View All LoRAs",
|
||||
"downloadMissingLoras": "Download Missing LoRAs",
|
||||
"deleteRecipe": "Delete Recipe",
|
||||
"enrichHfAgent": "Enrich HF Metadata (AI)"
|
||||
"enrichHfAgent": "Enrich Metadata with AI"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "Download Model from URL",
|
||||
"titleWithType": "Download {type} from URL",
|
||||
"civitaiUrl": "CivitAI URL(s):",
|
||||
"civitaiUrl": "Model URL(s):",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "Enter one CivitAI, CivArchive, or Hugging Face URL per line. Supports multiple URLs for batch download.",
|
||||
"urlHint": "Enter one CivitAI, CivArchive, Hugging Face, or ModelScope URL per line. Supports multiple URLs for batch download.",
|
||||
"selectHfFiles": "Select file(s) to download from this repository:",
|
||||
"selectAll": "Select All",
|
||||
"fetchingRepoFiles": "Fetching repository files...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "In Library"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Invalid CivitAI URL format",
|
||||
"invalidUrl": "Invalid model URL format",
|
||||
"noVersions": "No versions available for this model",
|
||||
"mixedSources": "Cannot mix CivitAI and Hugging Face URLs in the same batch.",
|
||||
"mixedSources": "Cannot mix CivitAI and Hugging Face / ModelScope URLs in the same batch.",
|
||||
"noModelFiles": "No model files found in this repository."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "Type folder path or select from tree below...",
|
||||
"root": "Root"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "Link to HuggingFace",
|
||||
"infoText": "Paste the HuggingFace repository URL to associate this model with its source. This enables AI-powered metadata enrichment.",
|
||||
"urlLabel": "HuggingFace Repository URL:",
|
||||
"linkModelSource": {
|
||||
"title": "Link to Model Source",
|
||||
"infoText": "Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||
"urlLabel": "Model Page URL:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "Enter the full URL of the HuggingFace repository.",
|
||||
"helpText": "Enter the full URL of the model page. Supported sites:",
|
||||
"enrichNote": "AI enrichment needs a readable model card. Sites that don't expose one (currently TensorArt) can only be linked.",
|
||||
"urlRequired": "Please enter a model page URL.",
|
||||
"invalidUrl": "Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "Linking model source...",
|
||||
"confirmAction": "Save & Link"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "No version history available for this model yet.",
|
||||
"error": "Failed to load versions.",
|
||||
"missingModelId": "This model is missing a CivitAI model id.",
|
||||
"hfGroupInfo": "This is a HuggingFace model group. Open the library to see all versions in the grid.",
|
||||
"sourceGroupInfo": "This is a {source} model group. Open the library to see all versions in the grid.",
|
||||
"confirm": {
|
||||
"delete": "Delete this version from your library?"
|
||||
},
|
||||
@@ -2478,12 +2483,14 @@
|
||||
"contentRatingFailed": "Failed to set content rating: {message}",
|
||||
"relinkSuccess": "Model successfully re-linked to CivitAI",
|
||||
"relinkFailed": "Error: {message}",
|
||||
"linkHfSuccess": "Model successfully linked to HuggingFace",
|
||||
"linkHfSuccess": "Model successfully linked to its model source",
|
||||
"linkHfFailed": "Error: {message}",
|
||||
"linkCivArchSuccess": "Model successfully re-linked via CivitArchive",
|
||||
"fetchMetadataFirst": "Please fetch metadata from CivitAI first",
|
||||
"noCivitaiInfo": "No CivitAI information available",
|
||||
"missingHash": "Model hash not available"
|
||||
"missingHash": "Model hash not available",
|
||||
"enrichNeedsSource": "Link this model to a model source first (Link Model → Link to Model Source)",
|
||||
"enrichUnsupportedSource": "AI enrichment is not available for {source} models"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "Example images path updated successfully",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "Ver en CivitAI",
|
||||
"notAvailableFromCivitai": "No disponible en CivitAI",
|
||||
"viewOnHuggingFace": "Ver en Hugging Face",
|
||||
"viewOnSource": "Ver en {source}",
|
||||
"sendToWorkflow": "Enviar a ComfyUI (Clic: Añadir, Shift+Clic: Reemplazar)",
|
||||
"copyLoRASyntax": "Copiar sintaxis de LoRA",
|
||||
"checkpointNameCopied": "Nombre del checkpoint copiado",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "Auto-organización completada",
|
||||
"error": "Error: {error}"
|
||||
},
|
||||
"enrichHfAgent": "Enriquecer metadatos HF (IA)"
|
||||
"enrichHfAgent": "Enriquecer metadatos con IA"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "Actualizar datos de CivitAI",
|
||||
"checkUpdates": "Comprobar actualizaciones",
|
||||
"linkModel": "Vincular modelo",
|
||||
"linkCivitai": "Re-vincular a CivitAI",
|
||||
"linkHuggingFace": "Vincular a HuggingFace",
|
||||
"linkModelSource": "Vincular a una fuente de modelo",
|
||||
"copySyntax": "Copiar sintaxis de LoRA",
|
||||
"copyFilename": "Copiar nombre de archivo del modelo",
|
||||
"copyRecipeSyntax": "Copiar sintaxis de receta",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "Ver todos los LoRAs",
|
||||
"downloadMissingLoras": "Descargar LoRAs faltantes",
|
||||
"deleteRecipe": "Eliminar receta",
|
||||
"enrichHfAgent": "Enriquecer metadatos HF (IA)"
|
||||
"enrichHfAgent": "Enriquecer metadatos con IA"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "Descargar modelo desde URL",
|
||||
"titleWithType": "Descargar {type} desde URL",
|
||||
"civitaiUrl": "URL de CivitAI:",
|
||||
"civitaiUrl": "URL del modelo:",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "Ingrese una URL de CivitAI, CivArchive o Hugging Face por línea. Admite múltiples URLs para descarga por lotes.",
|
||||
"urlHint": "Ingrese una URL de CivitAI, CivArchive, Hugging Face o ModelScope por línea. Admite múltiples URLs para descarga por lotes.",
|
||||
"selectHfFiles": "Seleccione el/los archivo(s) para descargar de este repositorio:",
|
||||
"selectAll": "Seleccionar todo",
|
||||
"fetchingRepoFiles": "Obteniendo archivos del repositorio...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "En la biblioteca"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Formato de URL de CivitAI inválido",
|
||||
"invalidUrl": "Formato de URL de modelo inválido",
|
||||
"noVersions": "No hay versiones disponibles para este modelo",
|
||||
"mixedSources": "No se pueden mezclar URL de CivitAI y Hugging Face en el mismo lote.",
|
||||
"mixedSources": "No se pueden mezclar URL de CivitAI y Hugging Face / ModelScope en el mismo lote.",
|
||||
"noModelFiles": "No se encontraron archivos de modelo en este repositorio."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "Escribe la ruta de la carpeta o selecciona del árbol de abajo...",
|
||||
"root": "Raíz"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "Vincular a HuggingFace",
|
||||
"infoText": "Pegue la URL del repositorio de HuggingFace para asociar este modelo. Esto permite el enriquecimiento de metadatos con IA.",
|
||||
"urlLabel": "URL del repositorio de HuggingFace:",
|
||||
"linkModelSource": {
|
||||
"title": "Vincular a una fuente de modelo",
|
||||
"infoText": "Pegue la URL de la página del modelo para asociar este modelo con su fuente. La vinculación permite el enriquecimiento de metadatos con IA para modelos de Hugging Face y ModelScope.",
|
||||
"urlLabel": "URL de la página del modelo:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "Ingrese la URL completa del repositorio de HuggingFace.",
|
||||
"helpText": "Ingrese la URL completa de la página del modelo. Sitios soportados:",
|
||||
"enrichNote": "El enriquecimiento con IA necesita una ficha de modelo legible. Los sitios que no la exponen (actualmente TensorArt) solo se pueden vincular.",
|
||||
"urlRequired": "Ingrese la URL de la página del modelo.",
|
||||
"invalidUrl": "URL no soportada. Sitios soportados: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "Vinculando la fuente del modelo...",
|
||||
"confirmAction": "Guardar y vincular"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "Aún no hay historial de versiones para este modelo.",
|
||||
"error": "No se pudieron cargar las versiones.",
|
||||
"missingModelId": "Este modelo no tiene un ID de modelo de CivitAI.",
|
||||
"hfGroupInfo": "Este es un grupo de modelos de HuggingFace. Abra la biblioteca para ver todas las versiones en la cuadrícula.",
|
||||
"sourceGroupInfo": "Este es un grupo de modelos de {source}. Abra la biblioteca para ver todas las versiones en la cuadrícula.",
|
||||
"confirm": {
|
||||
"delete": "¿Eliminar esta versión de tu biblioteca?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "Modelo re-vinculado exitosamente mediante CivitArchive",
|
||||
"fetchMetadataFirst": "Por favor obtén metadatos de CivitAI primero",
|
||||
"noCivitaiInfo": "No hay información de CivitAI disponible",
|
||||
"missingHash": "Hash del modelo no disponible"
|
||||
"missingHash": "Hash del modelo no disponible",
|
||||
"enrichNeedsSource": "Vincule este modelo a una fuente de modelo primero (Vincular modelo → Vincular a una fuente de modelo)",
|
||||
"enrichUnsupportedSource": "El enriquecimiento con IA no está disponible para modelos de {source}"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "Ruta de imágenes de ejemplo actualizada exitosamente",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "Voir sur CivitAI",
|
||||
"notAvailableFromCivitai": "Non disponible sur CivitAI",
|
||||
"viewOnHuggingFace": "Voir sur Hugging Face",
|
||||
"viewOnSource": "Voir sur {source}",
|
||||
"sendToWorkflow": "Envoyer vers ComfyUI (Clic: Ajouter, Maj+Clic: Remplacer)",
|
||||
"copyLoRASyntax": "Copier la syntaxe LoRA",
|
||||
"checkpointNameCopied": "Nom du checkpoint copié",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "Auto-organisation terminée",
|
||||
"error": "Erreur : {error}"
|
||||
},
|
||||
"enrichHfAgent": "Enrichir les métadonnées HF (IA)"
|
||||
"enrichHfAgent": "Enrichir les métadonnées avec l'IA"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "Actualiser les données CivitAI",
|
||||
"checkUpdates": "Vérifier les mises à jour",
|
||||
"linkModel": "Lier le modèle",
|
||||
"linkCivitai": "Relier à nouveau à CivitAI",
|
||||
"linkHuggingFace": "Lier à HuggingFace",
|
||||
"linkModelSource": "Lier à une source de modèle",
|
||||
"copySyntax": "Copier la syntaxe LoRA",
|
||||
"copyFilename": "Copier le nom de fichier du modèle",
|
||||
"copyRecipeSyntax": "Copier la syntaxe de la recipe",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "Voir tous les LoRAs",
|
||||
"downloadMissingLoras": "Télécharger les LoRAs manquants",
|
||||
"deleteRecipe": "Supprimer la recipe",
|
||||
"enrichHfAgent": "Enrichir les métadonnées HF (IA)"
|
||||
"enrichHfAgent": "Enrichir les métadonnées avec l'IA"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "Télécharger un modèle depuis une URL",
|
||||
"titleWithType": "Télécharger {type} depuis une URL",
|
||||
"civitaiUrl": "URL CivitAI :",
|
||||
"civitaiUrl": "URL du modèle :",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "Entrez une URL CivitAI, CivArchive ou Hugging Face par ligne. Prend en charge plusieurs URL pour le téléchargement par lot.",
|
||||
"urlHint": "Entrez une URL CivitAI, CivArchive, Hugging Face ou ModelScope par ligne. Prend en charge plusieurs URL pour le téléchargement par lot.",
|
||||
"selectHfFiles": "Sélectionnez le(s) fichier(s) à télécharger depuis ce dépôt :",
|
||||
"selectAll": "Tout sélectionner",
|
||||
"fetchingRepoFiles": "Récupération des fichiers du dépôt...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "Dans la bibliothèque"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Format d'URL CivitAI invalide",
|
||||
"invalidUrl": "Format d'URL de modèle invalide",
|
||||
"noVersions": "Aucune version disponible pour ce modèle",
|
||||
"mixedSources": "Impossible de mélanger les URL CivitAI et Hugging Face dans le même lot.",
|
||||
"mixedSources": "Impossible de mélanger les URL CivitAI et Hugging Face / ModelScope dans le même lot.",
|
||||
"noModelFiles": "Aucun fichier de modèle trouvé dans ce dépôt."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "Tapez le chemin du dossier ou sélectionnez dans l'arbre ci-dessous...",
|
||||
"root": "Racine"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "Lier à HuggingFace",
|
||||
"infoText": "Collez l'URL du dépôt HuggingFace pour associer ce modèle à sa source. Cela permet l'enrichissement des métadonnées par IA.",
|
||||
"urlLabel": "URL du dépôt HuggingFace :",
|
||||
"linkModelSource": {
|
||||
"title": "Lier à une source de modèle",
|
||||
"infoText": "Collez l'URL de la page du modèle pour associer ce modèle à sa source. La liaison permet l'enrichissement des métadonnées par IA pour les modèles Hugging Face et ModelScope.",
|
||||
"urlLabel": "URL de la page du modèle :",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "Entrez l'URL complète du dépôt HuggingFace.",
|
||||
"helpText": "Entrez l'URL complète de la page du modèle. Sites pris en charge :",
|
||||
"enrichNote": "L'enrichissement par IA nécessite une fiche de modèle lisible. Les sites qui n'en exposent pas (actuellement TensorArt) ne peuvent être que liés.",
|
||||
"urlRequired": "Veuillez saisir l'URL de la page du modèle.",
|
||||
"invalidUrl": "URL non prise en charge. Sites pris en charge : Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "Liaison de la source du modèle...",
|
||||
"confirmAction": "Enregistrer & lier"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "Aucun historique de versions n'est disponible pour ce modèle pour le moment.",
|
||||
"error": "Échec du chargement des versions.",
|
||||
"missingModelId": "Ce modèle ne possède pas d'identifiant de modèle CivitAI.",
|
||||
"hfGroupInfo": "Ceci est un groupe de modèles HuggingFace. Ouvrez la bibliothèque pour voir toutes les versions dans la grille.",
|
||||
"sourceGroupInfo": "Ceci est un groupe de modèles {source}. Ouvrez la bibliothèque pour voir toutes les versions dans la grille.",
|
||||
"confirm": {
|
||||
"delete": "Supprimer cette version de votre bibliothèque ?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "Modèle relié via CivitArchive avec succès",
|
||||
"fetchMetadataFirst": "Veuillez d'abord récupérer les métadonnées depuis CivitAI",
|
||||
"noCivitaiInfo": "Aucune information CivitAI disponible",
|
||||
"missingHash": "Hash du modèle non disponible"
|
||||
"missingHash": "Hash du modèle non disponible",
|
||||
"enrichNeedsSource": "Liez d'abord ce modèle à une source de modèle (Lier le modèle → Lier à une source de modèle)",
|
||||
"enrichUnsupportedSource": "L'enrichissement par IA n'est pas disponible pour les modèles {source}"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "Chemin des images d'exemple mis à jour avec succès",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "הצג ב-CivitAI",
|
||||
"notAvailableFromCivitai": "לא זמין מ-CivitAI",
|
||||
"viewOnHuggingFace": "צפייה ב-Hugging Face",
|
||||
"viewOnSource": "צפייה ב-{source}",
|
||||
"sendToWorkflow": "שלח ל-ComfyUI (לחיצה: הוסף, Shift+לחיצה: החלף)",
|
||||
"copyLoRASyntax": "העתק תחביר LoRA",
|
||||
"checkpointNameCopied": "שם Checkpoint הועתק",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "ארגון אוטומטי הושלם",
|
||||
"error": "שגיאה: {error}"
|
||||
},
|
||||
"enrichHfAgent": "העשרת HF מטא-נתונים (AI)"
|
||||
"enrichHfAgent": "העשרת מטא-נתונים ב-AI"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "רענן נתוני CivitAI",
|
||||
"checkUpdates": "בדוק עדכונים",
|
||||
"linkModel": "קישור מודל",
|
||||
"linkCivitai": "קשר מחדש ל-CivitAI",
|
||||
"linkHuggingFace": "קישור ל-HuggingFace",
|
||||
"linkModelSource": "קישור למקור מודל",
|
||||
"copySyntax": "העתק תחביר LoRA",
|
||||
"copyFilename": "העתק שם קובץ מודל",
|
||||
"copyRecipeSyntax": "העתק תחביר מתכון",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "הצג את כל ה-LoRAs",
|
||||
"downloadMissingLoras": "הורד LoRAs חסרים",
|
||||
"deleteRecipe": "מחק מתכון",
|
||||
"enrichHfAgent": "העשרת HF מטא-נתונים (AI)"
|
||||
"enrichHfAgent": "העשרת מטא-נתונים ב-AI"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "הורד מודל מכתובת URL",
|
||||
"titleWithType": "הורד {type} מכתובת URL",
|
||||
"civitaiUrl": "כתובת URL של CivitAI:",
|
||||
"civitaiUrl": "כתובת URL של מודל:",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "יש להזין כתובת URL אחת של CivitAI, CivArchive או Hugging Face בכל שורה. תומך במספר כתובות URL להורדה בקבוצה.",
|
||||
"urlHint": "יש להזין כתובת URL אחת של CivitAI, CivArchive, Hugging Face או ModelScope בכל שורה. תומך במספר כתובות URL להורדה בקבוצה.",
|
||||
"selectHfFiles": "בחר קבצים להורדה ממאגר זה:",
|
||||
"selectAll": "בחר הכל",
|
||||
"fetchingRepoFiles": "מביא קבצים מהמאגר...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "בספרייה"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "פורמט URL של CivitAI לא חוקי",
|
||||
"invalidUrl": "פורמט URL של מודל לא חוקי",
|
||||
"noVersions": "אין גרסאות זמינות למודל זה",
|
||||
"mixedSources": "לא ניתן לערבב כתובות URL של CivitAI ו-Hugging Face באותה קבוצה.",
|
||||
"mixedSources": "לא ניתן לערבב כתובות URL של CivitAI ו-Hugging Face / ModelScope באותה קבוצה.",
|
||||
"noModelFiles": "לא נמצאו קבצי מודל במאגר זה."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "הקלד נתיב תיקייה או בחר מהעץ למטה...",
|
||||
"root": "שורש"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "קישור ל-HuggingFace",
|
||||
"infoText": "הדבק את כתובת ה-URL של מאגר HuggingFace כדי לשייך מודל זה למקורו. פעולה זו מאפשרת העשרת מטא-נתונים באמצעות AI.",
|
||||
"urlLabel": "כתובת URL של מאגר HuggingFace:",
|
||||
"linkModelSource": {
|
||||
"title": "קישור למקור מודל",
|
||||
"infoText": "הדבק את כתובת ה-URL של עמוד המודל כדי לשייך מודל זה למקורו. הקישור מאפשר העשרת מטא-נתונים באמצעות AI עבור מודלים של Hugging Face ו-ModelScope.",
|
||||
"urlLabel": "כתובת URL של עמוד המודל:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "הזן את כתובת ה-URL המלאה של מאגר HuggingFace.",
|
||||
"helpText": "הזן את כתובת ה-URL המלאה של עמוד המודל. אתרים נתמכים:",
|
||||
"enrichNote": "העשרת AI דורשת כרטיס מודל קריא. אתרים שאינם חושפים אותו (נכון להיום TensorArt) ניתנים לקישור בלבד.",
|
||||
"urlRequired": "הזן כתובת URL של עמוד המודל.",
|
||||
"invalidUrl": "כתובת URL לא נתמכת. אתרים נתמכים: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "מקשר את מקור המודל...",
|
||||
"confirmAction": "שמור וקשר"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "אין עדיין היסטוריית גרסאות למודל זה.",
|
||||
"error": "טעינת הגרסאות נכשלה.",
|
||||
"missingModelId": "למודל זה אין מזהה מודל של CivitAI.",
|
||||
"hfGroupInfo": "זוהי קבוצת מודלים של HuggingFace. פתח את הספרייה כדי לראות את כל הגרסאות ברשת.",
|
||||
"sourceGroupInfo": "זוהי קבוצת מודלים של {source}. פתח את הספרייה כדי לראות את כל הגרסאות ברשת.",
|
||||
"confirm": {
|
||||
"delete": "למחוק גרסה זו מהספרייה שלך?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "המודל קושר מחדש דרך CivitArchive בהצלחה",
|
||||
"fetchMetadataFirst": "אנא אחזר מטא-נתונים מ-CivitAI תחילה",
|
||||
"noCivitaiInfo": "אין מידע מ-CivitAI זמין",
|
||||
"missingHash": "ה-hash של המודל אינו זמין"
|
||||
"missingHash": "ה-hash של המודל אינו זמין",
|
||||
"enrichNeedsSource": "קשר מודל זה למקור מודל תחילה (קישור מודל → קישור למקור מודל)",
|
||||
"enrichUnsupportedSource": "העשרת AI אינה זמינה עבור מודלים של {source}"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "נתיב תמונות הדוגמה עודכן בהצלחה",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "CivitAIで表示",
|
||||
"notAvailableFromCivitai": "CivitAIでは利用できません",
|
||||
"viewOnHuggingFace": "Hugging Face で見る",
|
||||
"viewOnSource": "{source} で見る",
|
||||
"sendToWorkflow": "ComfyUIに送信（クリック：追加、Shift+クリック：置換）",
|
||||
"copyLoRASyntax": "LoRA構文をコピー",
|
||||
"checkpointNameCopied": "Checkpointの名前をコピーしました",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "自動整理が完了しました",
|
||||
"error": "エラー：{error}"
|
||||
},
|
||||
"enrichHfAgent": "HF メタデータをAIで補完"
|
||||
"enrichHfAgent": "メタデータをAIで補完"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "CivitAIデータを更新",
|
||||
"checkUpdates": "更新確認",
|
||||
"linkModel": "モデルをリンク",
|
||||
"linkCivitai": "CivitAI にリンク",
|
||||
"linkHuggingFace": "HuggingFace にリンク",
|
||||
"linkModelSource": "モデルソースにリンク",
|
||||
"copySyntax": "LoRA構文をコピー",
|
||||
"copyFilename": "モデルファイル名をコピー",
|
||||
"copyRecipeSyntax": "レシピ構文をコピー",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "すべてのLoRAを表示",
|
||||
"downloadMissingLoras": "不足しているLoRAをダウンロード",
|
||||
"deleteRecipe": "レシピを削除",
|
||||
"enrichHfAgent": "HF メタデータをAIで補完"
|
||||
"enrichHfAgent": "メタデータをAIで補完"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "URLからモデルをダウンロード",
|
||||
"titleWithType": "URLから{type}をダウンロード",
|
||||
"civitaiUrl": "CivitAI URL：",
|
||||
"civitaiUrl": "モデル URL：",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "1行に1つのCivitAI、CivArchive、またはHugging Face URLを入力してください。複数のURLを一括ダウンロードできます。",
|
||||
"urlHint": "1行に1つのCivitAI、CivArchive、Hugging Face、またはModelScope URLを入力してください。複数のURLを一括ダウンロードできます。",
|
||||
"selectHfFiles": "このリポジトリからダウンロードするファイルを選択してください：",
|
||||
"selectAll": "すべて選択",
|
||||
"fetchingRepoFiles": "リポジトリのファイルを取得中...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "ライブラリ内"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "無効なCivitAI URL形式",
|
||||
"invalidUrl": "無効なモデル URL 形式",
|
||||
"noVersions": "このモデルの利用可能なバージョンがありません",
|
||||
"mixedSources": "同じバッチ内でCivitAIとHugging FaceのURLを混在させることはできません。",
|
||||
"mixedSources": "同じバッチ内でCivitAIとHugging Face / ModelScopeのURLを混在させることはできません。",
|
||||
"noModelFiles": "このリポジトリにモデルファイルが見つかりませんでした。"
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "フォルダパスを入力するか、下のツリーから選択...",
|
||||
"root": "ルート"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "HuggingFace にリンク",
|
||||
"infoText": "HuggingFace リポジトリの URL を貼り付けてモデルを関連付けます。AI によるメタデータ補完が有効になります。",
|
||||
"urlLabel": "HuggingFace リポジトリ URL：",
|
||||
"linkModelSource": {
|
||||
"title": "モデルソースにリンク",
|
||||
"infoText": "モデルページの URL を貼り付けて、このモデルをソースに関連付けます。リンクすると、Hugging Face と ModelScope のモデルで AI によるメタデータ補完が有効になります。",
|
||||
"urlLabel": "モデルページ URL：",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "完全な HuggingFace リポジトリ URL を入力してください。",
|
||||
"helpText": "完全なモデルページ URL を入力してください。対応サイト：",
|
||||
"enrichNote": "AI 補完には読み取り可能なモデルカードが必要です。モデルカードを公開していないサイト（現在は TensorArt）はリンクのみ可能です。",
|
||||
"urlRequired": "モデルページの URL を入力してください。",
|
||||
"invalidUrl": "サポートされていない URL です。対応サイト：Hugging Face、ModelScope、TensorArt。",
|
||||
"linking": "モデルソースをリンクしています...",
|
||||
"confirmAction": "保存＆リンク"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "このモデルにはまだバージョン履歴がありません。",
|
||||
"error": "バージョンの読み込みに失敗しました。",
|
||||
"missingModelId": "このモデルにはCivitAIのモデルIDがありません。",
|
||||
"hfGroupInfo": "これは HuggingFace モデルグループです。ライブラリを開いてグリッドですべてのバージョンを表示してください。",
|
||||
"sourceGroupInfo": "これは {source} モデルグループです。ライブラリを開いてグリッドですべてのバージョンを表示してください。",
|
||||
"confirm": {
|
||||
"delete": "このバージョンをライブラリから削除しますか？"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "モデルがCivitArchive経由で正常に再リンクされました",
|
||||
"fetchMetadataFirst": "最初にCivitAIからメタデータを取得してください",
|
||||
"noCivitaiInfo": "CivitAI情報が利用できません",
|
||||
"missingHash": "モデルハッシュが利用できません"
|
||||
"missingHash": "モデルハッシュが利用できません",
|
||||
"enrichNeedsSource": "まずこのモデルをモデルソースにリンクしてください（モデルをリンク → モデルソースにリンク）",
|
||||
"enrichUnsupportedSource": "{source} モデルでは AI 補完を利用できません"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "例画像パスが正常に更新されました",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "CivitAI에서 보기",
|
||||
"notAvailableFromCivitai": "CivitAI에서 사용할 수 없음",
|
||||
"viewOnHuggingFace": "Hugging Face에서 보기",
|
||||
"viewOnSource": "{source}에서 보기",
|
||||
"sendToWorkflow": "ComfyUI로 전송 (클릭: 추가, Shift+클릭: 교체)",
|
||||
"copyLoRASyntax": "LoRA 문법 복사",
|
||||
"checkpointNameCopied": "Checkpoint 이름 복사됨",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "자동 정리 완료",
|
||||
"error": "오류: {error}"
|
||||
},
|
||||
"enrichHfAgent": "HF AI로 메타데이터 보강"
|
||||
"enrichHfAgent": "AI로 메타데이터 보강"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "CivitAI 데이터 새로고침",
|
||||
"checkUpdates": "업데이트 확인",
|
||||
"linkModel": "모델 연결",
|
||||
"linkCivitai": "CivitAI에 연결",
|
||||
"linkHuggingFace": "HuggingFace에 연결",
|
||||
"linkModelSource": "모델 소스에 연결",
|
||||
"copySyntax": "LoRA 문법 복사",
|
||||
"copyFilename": "모델 파일명 복사",
|
||||
"copyRecipeSyntax": "레시피 문법 복사",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "모든 LoRA 보기",
|
||||
"downloadMissingLoras": "누락된 LoRA 다운로드",
|
||||
"deleteRecipe": "레시피 삭제",
|
||||
"enrichHfAgent": "HF AI로 메타데이터 보강"
|
||||
"enrichHfAgent": "AI로 메타데이터 보강"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "URL에서 모델 다운로드",
|
||||
"titleWithType": "URL에서 {type} 다운로드",
|
||||
"civitaiUrl": "CivitAI URL:",
|
||||
"civitaiUrl": "모델 URL:",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "한 줄에 하나의 CivitAI, CivArchive 또는 Hugging Face URL을 입력하세요. 여러 URL을 일괄 다운로드할 수 있습니다.",
|
||||
"urlHint": "한 줄에 하나의 CivitAI, CivArchive, Hugging Face 또는 ModelScope URL을 입력하세요. 여러 URL을 일괄 다운로드할 수 있습니다.",
|
||||
"selectHfFiles": "이 저장소에서 다운로드할 파일을 선택하세요：",
|
||||
"selectAll": "모두 선택",
|
||||
"fetchingRepoFiles": "저장소 파일을 가져오는 중...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "라이브러리에 있음"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "잘못된 CivitAI URL 형식",
|
||||
"invalidUrl": "잘못된 모델 URL 형식",
|
||||
"noVersions": "이 모델에 사용 가능한 버전이 없습니다",
|
||||
"mixedSources": "동일한 배치에서 CivitAI와 Hugging Face URL을 혼합할 수 없습니다.",
|
||||
"mixedSources": "동일한 배치에서 CivitAI와 Hugging Face / ModelScope URL을 혼합할 수 없습니다.",
|
||||
"noModelFiles": "이 저장소에서 모델 파일을 찾을 수 없습니다."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "폴더 경로를 입력하거나 아래 트리에서 선택하세요...",
|
||||
"root": "루트"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "HuggingFace에 연결",
|
||||
"infoText": "HuggingFace 저장소 URL을 붙여넣어 모델을 연결합니다. AI 메타데이터 보강 기능을 사용할 수 있습니다.",
|
||||
"urlLabel": "HuggingFace 저장소 URL：",
|
||||
"linkModelSource": {
|
||||
"title": "모델 소스에 연결",
|
||||
"infoText": "모델 페이지 URL을 붙여넣어 이 모델을 소스에 연결합니다. 연결하면 Hugging Face 및 ModelScope 모델에 AI 메타데이터 보강을 사용할 수 있습니다.",
|
||||
"urlLabel": "모델 페이지 URL:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "전체 HuggingFace 저장소 URL을 입력하세요.",
|
||||
"helpText": "전체 모델 페이지 URL을 입력하세요. 지원 사이트:",
|
||||
"enrichNote": "AI 보강에는 읽을 수 있는 모델 카드가 필요합니다. 모델 카드를 제공하지 않는 사이트(현재 TensorArt)는 연결만 가능합니다.",
|
||||
"urlRequired": "모델 페이지 URL을 입력하세요.",
|
||||
"invalidUrl": "지원되지 않는 URL입니다. 지원 사이트: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "모델 소스를 연결하는 중...",
|
||||
"confirmAction": "저장 및 연결"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "이 모델에는 아직 버전 기록이 없습니다.",
|
||||
"error": "버전을 불러오지 못했습니다.",
|
||||
"missingModelId": "이 모델에는 CivitAI 모델 ID가 없습니다.",
|
||||
"hfGroupInfo": "HuggingFace 모델 그룹입니다. 라이브러리를 열어 그리드에서 모든 버전을 확인하세요.",
|
||||
"sourceGroupInfo": "{source} 모델 그룹입니다. 라이브러리를 열어 그리드에서 모든 버전을 확인하세요.",
|
||||
"confirm": {
|
||||
"delete": "이 버전을 라이브러리에서 삭제하시겠습니까?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "모델이 CivitArchive을 통해 성공적으로 다시 연결되었습니다",
|
||||
"fetchMetadataFirst": "먼저 CivitAI에서 메타데이터를 가져와주세요",
|
||||
"noCivitaiInfo": "사용 가능한 CivitAI 정보가 없습니다",
|
||||
"missingHash": "모델 해시를 사용할 수 없습니다"
|
||||
"missingHash": "모델 해시를 사용할 수 없습니다",
|
||||
"enrichNeedsSource": "먼저 이 모델을 모델 소스에 연결하세요 (모델 연결 → 모델 소스에 연결)",
|
||||
"enrichUnsupportedSource": "{source} 모델에서는 AI 보강을 사용할 수 없습니다"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "예시 이미지 경로가 성공적으로 업데이트되었습니다",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "Посмотреть на CivitAI",
|
||||
"notAvailableFromCivitai": "Недоступно на CivitAI",
|
||||
"viewOnHuggingFace": "Открыть Hugging Face",
|
||||
"viewOnSource": "Открыть {source}",
|
||||
"sendToWorkflow": "Отправить в ComfyUI (Клик: Добавить, Shift+Клик: Заменить)",
|
||||
"copyLoRASyntax": "Копировать синтаксис LoRA",
|
||||
"checkpointNameCopied": "Имя checkpoint скопировано",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "Автоматическая организация завершена",
|
||||
"error": "Ошибка: {error}"
|
||||
},
|
||||
"enrichHfAgent": "Обогатить HF метаданные (ИИ)"
|
||||
"enrichHfAgent": "Обогатить метаданные с помощью ИИ"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "Обновить данные CivitAI",
|
||||
"checkUpdates": "Проверить обновления",
|
||||
"linkModel": "Связать модель",
|
||||
"linkCivitai": "Пересвязать с CivitAI",
|
||||
"linkHuggingFace": "Связать с HuggingFace",
|
||||
"linkModelSource": "Связать с источником модели",
|
||||
"copySyntax": "Копировать синтаксис LoRA",
|
||||
"copyFilename": "Копировать имя файла модели",
|
||||
"copyRecipeSyntax": "Копировать синтаксис рецепта",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "Посмотреть все LoRAs",
|
||||
"downloadMissingLoras": "Загрузить отсутствующие LoRAs",
|
||||
"deleteRecipe": "Удалить рецепт",
|
||||
"enrichHfAgent": "Обогатить HF метаданные (ИИ)"
|
||||
"enrichHfAgent": "Обогатить метаданные с помощью ИИ"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "Скачать модель по URL",
|
||||
"titleWithType": "Скачать {type} по URL",
|
||||
"civitaiUrl": "CivitAI URL:",
|
||||
"civitaiUrl": "URL модели:",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "Введите один URL CivitAI, CivArchive или Hugging Face в каждой строке. Поддерживает несколько URL для пакетной загрузки.",
|
||||
"urlHint": "Введите один URL CivitAI, CivArchive, Hugging Face или ModelScope в каждой строке. Поддерживает несколько URL для пакетной загрузки.",
|
||||
"selectHfFiles": "Выберите файл(ы) для загрузки из этого репозитория:",
|
||||
"selectAll": "Выбрать все",
|
||||
"fetchingRepoFiles": "Получение файлов репозитория...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "В библиотеке"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "Неверный формат URL CivitAI",
|
||||
"invalidUrl": "Неверный формат URL модели",
|
||||
"noVersions": "Нет доступных версий для этой модели",
|
||||
"mixedSources": "Нельзя смешивать URL-адреса CivitAI и Hugging Face в одном пакете.",
|
||||
"mixedSources": "Нельзя смешивать URL-адреса CivitAI и Hugging Face / ModelScope в одном пакете.",
|
||||
"noModelFiles": "В этом репозитории не найдено файлов моделей."
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "Введите путь к папке или выберите из дерева ниже...",
|
||||
"root": "Корень"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "Связать с HuggingFace",
|
||||
"infoText": "Вставьте URL репозитория HuggingFace, чтобы связать эту модель с её источником. Это позволит обогащать метаданные с помощью ИИ.",
|
||||
"urlLabel": "URL репозитория HuggingFace:",
|
||||
"linkModelSource": {
|
||||
"title": "Связать с источником модели",
|
||||
"infoText": "Вставьте URL страницы модели, чтобы связать эту модель с её источником. Связывание включает обогащение метаданных с помощью ИИ для моделей Hugging Face и ModelScope.",
|
||||
"urlLabel": "URL страницы модели:",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "Введите полный URL репозитория HuggingFace.",
|
||||
"helpText": "Введите полный URL страницы модели. Поддерживаемые сайты:",
|
||||
"enrichNote": "Для обогащения с помощью ИИ нужна читаемая карточка модели. Сайты, которые её не предоставляют (сейчас TensorArt), можно только связать.",
|
||||
"urlRequired": "Введите URL страницы модели.",
|
||||
"invalidUrl": "Неподдерживаемый URL. Поддерживаемые сайты: Hugging Face, ModelScope, TensorArt.",
|
||||
"linking": "Связывание с источником модели...",
|
||||
"confirmAction": "Сохранить и связать"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "Для этой модели пока нет истории версий.",
|
||||
"error": "Не удалось загрузить версии.",
|
||||
"missingModelId": "У этой модели отсутствует идентификатор модели CivitAI.",
|
||||
"hfGroupInfo": "Это группа моделей HuggingFace. Откройте библиотеку, чтобы увидеть все версии в сетке.",
|
||||
"sourceGroupInfo": "Это группа моделей {source}. Откройте библиотеку, чтобы увидеть все версии в сетке.",
|
||||
"confirm": {
|
||||
"delete": "Удалить эту версию из библиотеки?"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "Модель успешно пересвязана через CivitArchive",
|
||||
"fetchMetadataFirst": "Пожалуйста, сначала получите метаданные с CivitAI",
|
||||
"noCivitaiInfo": "Информация CivitAI недоступна",
|
||||
"missingHash": "Хеш модели недоступен"
|
||||
"missingHash": "Хеш модели недоступен",
|
||||
"enrichNeedsSource": "Сначала свяжите эту модель с источником модели (Связать модель → Связать с источником модели)",
|
||||
"enrichUnsupportedSource": "Обогащение с помощью ИИ недоступно для моделей {source}"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "Путь к примерам изображений успешно обновлен",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "在 CivitAI 查看",
|
||||
"notAvailableFromCivitai": "CivitAI 上不可用",
|
||||
"viewOnHuggingFace": "在 Hugging Face 查看",
|
||||
"viewOnSource": "在 {source} 查看",
|
||||
"sendToWorkflow": "发送到 ComfyUI（点击：追加，Shift+点击：替换）",
|
||||
"copyLoRASyntax": "复制 LoRA 语法",
|
||||
"checkpointNameCopied": "Checkpoint 名称已复制",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "自动整理已完成",
|
||||
"error": "错误：{error}"
|
||||
},
|
||||
"enrichHfAgent": "AI HF 元数据增强"
|
||||
"enrichHfAgent": "AI 元数据增强"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "刷新 CivitAI 数据",
|
||||
"checkUpdates": "检查更新",
|
||||
"linkModel": "链接模型",
|
||||
"linkCivitai": "链接到 CivitAI",
|
||||
"linkHuggingFace": "链接到 HuggingFace",
|
||||
"linkModelSource": "链接到模型来源",
|
||||
"copySyntax": "复制 LoRA 语法",
|
||||
"copyFilename": "复制模型文件名",
|
||||
"copyRecipeSyntax": "复制配方语法",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "查看所有 LoRA",
|
||||
"downloadMissingLoras": "下载缺失的 LoRA",
|
||||
"deleteRecipe": "删除配方",
|
||||
"enrichHfAgent": "AI HF 元数据增强"
|
||||
"enrichHfAgent": "AI 元数据增强"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "从 URL 下载模型",
|
||||
"titleWithType": "从 URL 下载 {type}",
|
||||
"civitaiUrl": "CivitAI URL:",
|
||||
"civitaiUrl": "模型 URL：",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "每行输入一个 CivitAI、CivArchive 或 Hugging Face URL。支持批量下载多个 URL。",
|
||||
"urlHint": "每行输入一个 CivitAI、CivArchive、Hugging Face 或 ModelScope URL。支持批量下载多个 URL。",
|
||||
"selectHfFiles": "选择从此仓库下载的文件：",
|
||||
"selectAll": "全选",
|
||||
"fetchingRepoFiles": "正在获取仓库文件...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "已在库中"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "无效的 CivitAI URL 格式",
|
||||
"invalidUrl": "无效的模型 URL 格式",
|
||||
"noVersions": "此模型没有可用版本",
|
||||
"mixedSources": "无法在同一批次中混合使用 CivitAI 和 Hugging Face URL。",
|
||||
"mixedSources": "无法在同一批次中混合使用 CivitAI 和 Hugging Face / ModelScope URL。",
|
||||
"noModelFiles": "在此仓库中未找到模型文件。"
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "输入文件夹路径或从下方树中选择...",
|
||||
"root": "根目录"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "链接到 HuggingFace",
|
||||
"infoText": "粘贴 HuggingFace 仓库 URL 以关联此模型。关联后可启用 AI 元数据增强功能。",
|
||||
"urlLabel": "HuggingFace 仓库 URL：",
|
||||
"linkModelSource": {
|
||||
"title": "链接到模型来源",
|
||||
"infoText": "粘贴模型页面 URL 以关联此模型与其来源。关联后可对 Hugging Face 和 ModelScope 模型启用 AI 元数据增强。",
|
||||
"urlLabel": "模型页面 URL：",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "请输入完整的 HuggingFace 仓库 URL。",
|
||||
"helpText": "请输入完整的模型页面 URL。支持的站点：",
|
||||
"enrichNote": "AI 增强需要可读取的模型卡。未提供模型卡的站点（目前为 TensorArt）只能建立链接。",
|
||||
"urlRequired": "请输入模型页面 URL。",
|
||||
"invalidUrl": "URL 不受支持。支持的站点：Hugging Face、ModelScope、TensorArt。",
|
||||
"linking": "正在链接模型来源...",
|
||||
"confirmAction": "保存并链接"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "该模型还没有版本历史。",
|
||||
"error": "加载版本失败。",
|
||||
"missingModelId": "该模型缺少 CivitAI 模型 ID。",
|
||||
"hfGroupInfo": "这是一个 HuggingFace 模型组。打开库页面即可在网格中查看所有版本。",
|
||||
"sourceGroupInfo": "这是一个 {source} 模型组。打开库页面即可在网格中查看所有版本。",
|
||||
"confirm": {
|
||||
"delete": "从库中删除此版本？"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "模型已成功通过 CivitArchive 重新关联",
|
||||
"fetchMetadataFirst": "请先从 CivitAI 获取元数据",
|
||||
"noCivitaiInfo": "无 CivitAI 信息",
|
||||
"missingHash": "模型哈希不可用"
|
||||
"missingHash": "模型哈希不可用",
|
||||
"enrichNeedsSource": "请先将此模型链接到模型来源（链接模型 → 链接到模型来源）",
|
||||
"enrichUnsupportedSource": "{source} 模型不支持 AI 增强"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "示例图片路径更新成功",
|
||||
|
||||
+21
-14
@@ -139,6 +139,7 @@
|
||||
"viewOnCivitai": "在 CivitAI 查看",
|
||||
"notAvailableFromCivitai": "CivitAI 不提供",
|
||||
"viewOnHuggingFace": "在 Hugging Face 查看",
|
||||
"viewOnSource": "在 {source} 查看",
|
||||
"sendToWorkflow": "傳送到 ComfyUI（點擊：附加，Shift+點擊：取代）",
|
||||
"copyLoRASyntax": "複製 LoRA 語法",
|
||||
"checkpointNameCopied": "Checkpoint 名稱已複製",
|
||||
@@ -867,14 +868,14 @@
|
||||
"complete": "自動整理完成",
|
||||
"error": "錯誤：{error}"
|
||||
},
|
||||
"enrichHfAgent": "AI HF 中繼資料增強"
|
||||
"enrichHfAgent": "AI 中繼資料增強"
|
||||
},
|
||||
"contextMenu": {
|
||||
"refreshMetadata": "刷新 CivitAI 資料",
|
||||
"checkUpdates": "檢查更新",
|
||||
"linkModel": "連結模型",
|
||||
"linkCivitai": "連結到 CivitAI",
|
||||
"linkHuggingFace": "連結到 HuggingFace",
|
||||
"linkModelSource": "連結到模型來源",
|
||||
"copySyntax": "複製 LoRA 語法",
|
||||
"copyFilename": "複製模型檔名",
|
||||
"copyRecipeSyntax": "複製配方語法",
|
||||
@@ -896,7 +897,7 @@
|
||||
"viewAllLoras": "檢視全部 LoRA",
|
||||
"downloadMissingLoras": "下載缺少的 LoRA",
|
||||
"deleteRecipe": "刪除配方",
|
||||
"enrichHfAgent": "AI HF 中繼資料增強"
|
||||
"enrichHfAgent": "AI 中繼資料增強"
|
||||
}
|
||||
},
|
||||
"recipes": {
|
||||
@@ -1394,9 +1395,9 @@
|
||||
"download": {
|
||||
"title": "從網址下載模型",
|
||||
"titleWithType": "從網址下載 {type}",
|
||||
"civitaiUrl": "CivitAI 網址：",
|
||||
"civitaiUrl": "模型網址：",
|
||||
"placeholder": "https://civitai.com/models/...",
|
||||
"urlHint": "每行輸入一個 CivitAI、CivArchive 或 Hugging Face URL。支援批量下載多個 URL。",
|
||||
"urlHint": "每行輸入一個 CivitAI、CivArchive、Hugging Face 或 ModelScope URL。支援批量下載多個 URL。",
|
||||
"selectHfFiles": "選擇從此倉庫下載的檔案：",
|
||||
"selectAll": "全選",
|
||||
"fetchingRepoFiles": "正在獲取倉庫檔案...",
|
||||
@@ -1429,9 +1430,9 @@
|
||||
"inLibrary": "已在庫中"
|
||||
},
|
||||
"errors": {
|
||||
"invalidUrl": "CivitAI 網址格式無效",
|
||||
"invalidUrl": "模型網址格式無效",
|
||||
"noVersions": "此模型無可用版本",
|
||||
"mixedSources": "無法在同一批次中混合使用 CivitAI 和 Hugging Face URL。",
|
||||
"mixedSources": "無法在同一批次中混合使用 CivitAI 和 Hugging Face / ModelScope URL。",
|
||||
"noModelFiles": "在此倉庫中未找到模型檔案。"
|
||||
},
|
||||
"status": {
|
||||
@@ -1596,12 +1597,16 @@
|
||||
"pathPlaceholder": "輸入資料夾路徑或從下方樹狀結構選擇...",
|
||||
"root": "根目錄"
|
||||
},
|
||||
"linkHuggingFace": {
|
||||
"title": "連結到 HuggingFace",
|
||||
"infoText": "貼上 HuggingFace 倉庫 URL 以關聯此模型。關聯後可啟用 AI 中繼資料增強功能。",
|
||||
"urlLabel": "HuggingFace 倉庫 URL：",
|
||||
"linkModelSource": {
|
||||
"title": "連結到模型來源",
|
||||
"infoText": "貼上模型頁面 URL 以關聯此模型與其來源。關聯後可對 Hugging Face 和 ModelScope 模型啟用 AI 中繼資料增強。",
|
||||
"urlLabel": "模型頁面 URL：",
|
||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||
"helpText": "請輸入完整的 HuggingFace 倉庫 URL。",
|
||||
"helpText": "請輸入完整的模型頁面 URL。支援的站點：",
|
||||
"enrichNote": "AI 增強需要可讀取的模型卡。未提供模型卡的站點（目前為 TensorArt）只能建立連結。",
|
||||
"urlRequired": "請輸入模型頁面 URL。",
|
||||
"invalidUrl": "URL 不受支援。支援的站點：Hugging Face、ModelScope、TensorArt。",
|
||||
"linking": "正在連結模型來源...",
|
||||
"confirmAction": "儲存並連結"
|
||||
},
|
||||
"relinkCivitai": {
|
||||
@@ -1847,7 +1852,7 @@
|
||||
"empty": "此模型尚無版本歷史。",
|
||||
"error": "載入版本失敗。",
|
||||
"missingModelId": "此模型缺少 CivitAI 模型 ID。",
|
||||
"hfGroupInfo": "這是一個 HuggingFace 模型組。打開庫頁面即可在網格中查看所有版本。",
|
||||
"sourceGroupInfo": "這是一個 {source} 模型組。打開庫頁面即可在網格中查看所有版本。",
|
||||
"confirm": {
|
||||
"delete": "要從庫中刪除此版本嗎？"
|
||||
},
|
||||
@@ -2483,7 +2488,9 @@
|
||||
"linkCivArchSuccess": "模型已成功透過 CivitArchive 重新連結",
|
||||
"fetchMetadataFirst": "請先從 CivitAI 取得 metadata",
|
||||
"noCivitaiInfo": "無 CivitAI 資訊",
|
||||
"missingHash": "模型雜湊不可用"
|
||||
"missingHash": "模型雜湊不可用",
|
||||
"enrichNeedsSource": "請先將此模型連結到模型來源（連結模型 → 連結到模型來源）",
|
||||
"enrichUnsupportedSource": "{source} 模型不支援 AI 增強"
|
||||
},
|
||||
"exampleImages": {
|
||||
"pathUpdated": "範例圖片路徑已更新",
|
||||
|
||||
@@ -472,12 +472,5 @@ class LoraManager:
|
||||
scanner.cancel_task()
|
||||
logger.debug("LoRA Manager: Cancelled %s", name)
|
||||
|
||||
# Close shared aiohttp sessions to avoid "Unclosed client session" warnings
|
||||
try:
|
||||
from py.routes.handlers.hf_handlers import close_hf_api_session
|
||||
await close_hf_api_session()
|
||||
except Exception as exc:
|
||||
logger.debug("Error closing HF API session: %s", exc)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error during cleanup: {e}", exc_info=True)
|
||||
|
||||
@@ -55,7 +55,7 @@ from ...utils.constants import (
|
||||
VALID_LORA_TYPES,
|
||||
VALID_OTHER_CIVITAI_TYPES,
|
||||
)
|
||||
from .hf_handlers import HfHandler
|
||||
from .model_source_handlers import ModelSourceHandler
|
||||
from .agent_handlers import AgentHandler
|
||||
from .download_routing_handlers import DownloadRoutingHandler
|
||||
from .model_handlers import ModelCivitaiHandler
|
||||
@@ -4001,7 +4001,7 @@ class MiscHandlerSet:
|
||||
doctor: DoctorHandler,
|
||||
example_workflows: ExampleWorkflowsHandler,
|
||||
base_model: BaseModelHandlerSet,
|
||||
hf_handler: Any = None,
|
||||
model_source_handler: Any = None,
|
||||
agent_handler: Any = None,
|
||||
download_routing: Any = None,
|
||||
) -> None:
|
||||
@@ -4022,7 +4022,7 @@ class MiscHandlerSet:
|
||||
self.doctor = doctor
|
||||
self.example_workflows = example_workflows
|
||||
self.base_model = base_model
|
||||
self.hf_handler = hf_handler
|
||||
self.model_source_handler = model_source_handler
|
||||
self.agent_handler = agent_handler
|
||||
self.download_routing = download_routing
|
||||
|
||||
@@ -4076,9 +4076,13 @@ class MiscHandlerSet:
|
||||
"get_example_workflows": self.example_workflows.get_example_workflows,
|
||||
"get_example_workflow": self.example_workflows.get_example_workflow,
|
||||
# Hugging Face handlers
|
||||
"get_hf_repo_files": self.hf_handler.get_hf_repo_files,
|
||||
"download_hf_model": self.hf_handler.download_hf_model,
|
||||
"set_hf_url": self.hf_handler.set_hf_url,
|
||||
# External model sources (Hugging Face / ModelScope)
|
||||
"list_model_source_files": self.model_source_handler.list_model_source_files,
|
||||
"download_model_source": self.model_source_handler.download_model_source,
|
||||
"get_hf_repo_files": self.model_source_handler.list_model_source_files,
|
||||
"download_hf_model": self.model_source_handler.download_model_source,
|
||||
"set_hf_url": self.model_source_handler.set_hf_url,
|
||||
"get_model_sources": self.model_source_handler.get_model_sources,
|
||||
# Agent skill handlers
|
||||
"get_agent_skills": self.agent_handler.get_agent_skills,
|
||||
"execute_agent_skill": self.agent_handler.execute_agent_skill,
|
||||
|
||||
@@ -1,8 +1,13 @@
|
||||
"""Handlers for Hugging Face model listing and download.
|
||||
"""Handlers for external model sources: linking, file listing and downloads.
|
||||
|
||||
Minimal MVP implementation — uses direct HTTP to the HF API for file
|
||||
listing and the project's existing aiohttp-based Downloader for
|
||||
downloading. No huggingface_hub dependency required.
|
||||
Covers every site registered in :mod:`py.services.model_sources`. The module
|
||||
was Hugging Face only (``hf_handlers.py`` / ``HfHandler``) until ModelScope
|
||||
downloads were added; the per-site differences now live in the providers, so
|
||||
this file has no platform branches beyond the capability lookups.
|
||||
|
||||
The historical route paths (``/api/lm/set-hf-url``, ``/api/lm/hf-repo-files``,
|
||||
``/api/lm/download-hf-model``) are still registered as aliases of the generic
|
||||
handlers, so existing callers keep working.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -10,10 +15,8 @@ from __future__ import annotations
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
import aiohttp
|
||||
from aiohttp import web
|
||||
|
||||
from ...config import config
|
||||
@@ -22,10 +25,18 @@ from ...services.downloader import (
|
||||
get_downloader,
|
||||
)
|
||||
from ...services.aria2_downloader import Aria2Downloader
|
||||
from ...services.model_sources import (
|
||||
ModelSourceError,
|
||||
SourceRef,
|
||||
detect_source,
|
||||
get_download_source,
|
||||
is_valid_source_id,
|
||||
list_sources,
|
||||
normalize_metadata_source,
|
||||
)
|
||||
from ...services.settings_manager import get_settings_manager
|
||||
from ...services.service_registry import ServiceRegistry
|
||||
from ...services.websocket_manager import ws_manager
|
||||
from ...utils.constants import MODEL_FILE_EXTENSIONS
|
||||
from ...utils.metadata_manager import MetadataManager
|
||||
from ...utils.models import LoraMetadata, CheckpointMetadata, EmbeddingMetadata
|
||||
|
||||
@@ -34,28 +45,6 @@ logger = logging.getLogger(__name__)
|
||||
_DEFAULT_MODEL_CLASS = LoraMetadata
|
||||
_DEFAULT_SCANNER_GETTER = "get_lora_scanner"
|
||||
|
||||
# Shared aiohttp session for HF API calls (created on first use)
|
||||
_hf_api_session: aiohttp.ClientSession | None = None
|
||||
|
||||
|
||||
async def _get_hf_api_session() -> aiohttp.ClientSession:
|
||||
"""Get or create the shared aiohttp session for HF API calls."""
|
||||
global _hf_api_session # needed because we reassign the module-level name
|
||||
if _hf_api_session is None or _hf_api_session.closed:
|
||||
_hf_api_session = aiohttp.ClientSession(
|
||||
headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
)
|
||||
return _hf_api_session
|
||||
|
||||
|
||||
async def close_hf_api_session() -> None:
|
||||
"""Close the shared HF API session, if it was ever created."""
|
||||
global _hf_api_session
|
||||
if _hf_api_session is not None and not _hf_api_session.closed:
|
||||
await _hf_api_session.close()
|
||||
_hf_api_session = None
|
||||
|
||||
|
||||
def _infer_model_type(model_root: str) -> tuple[Any, str]:
|
||||
"""Determine model class and scanner by matching ``model_root`` against the
|
||||
@@ -96,18 +85,19 @@ def _infer_model_type(model_root: str) -> tuple[Any, str]:
|
||||
return _DEFAULT_MODEL_CLASS, _DEFAULT_SCANNER_GETTER
|
||||
|
||||
|
||||
async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
|
||||
async def _save_source_metadata(
|
||||
dest_path: str, ref: SourceRef, model_root: str
|
||||
) -> None:
|
||||
"""Create a proper .metadata.json and add the model to the scanner cache.
|
||||
|
||||
Uses ``MetadataManager.create_default_metadata()`` which computes the
|
||||
SHA256 hash, extracts safetensors header metadata (base_model), and
|
||||
produces a fully-populated ``LoraMetadata`` (or ``CheckpointMetadata`` /
|
||||
``EmbeddingMetadata``) object. We then overlay HF-specific fields and
|
||||
register the model in the in-memory scanner cache so it appears
|
||||
``EmbeddingMetadata``) object. We then overlay the external-source fields
|
||||
and register the model in the in-memory scanner cache so it appears
|
||||
immediately without a full filesystem walk.
|
||||
"""
|
||||
try:
|
||||
hf_url = f"https://huggingface.co/{repo}"
|
||||
model_class, scanner_getter_name = _infer_model_type(model_root)
|
||||
|
||||
# 1. Create proper metadata (computes SHA256, reads safetensors headers)
|
||||
@@ -118,13 +108,21 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
|
||||
logger.warning("create_default_metadata returned None for %s", dest_path)
|
||||
return
|
||||
|
||||
# 2. Overlay HF-specific fields
|
||||
metadata._unknown_fields["hf_url"] = hf_url
|
||||
metadata.from_civitai = False # HF models are not from CivitAI
|
||||
# 2. Overlay the external-source fields (`hf_url` is written by
|
||||
# normalisation for Hugging Face only)
|
||||
fields = metadata._unknown_fields
|
||||
fields["source_url"] = ref.url
|
||||
fields["source_platform"] = ref.platform
|
||||
if ref.platform == "huggingface":
|
||||
fields["hf_url"] = ref.url
|
||||
metadata.from_civitai = False # externally-sourced models are not from CivitAI
|
||||
|
||||
# 3. Save metadata atomically
|
||||
await MetadataManager.save_metadata(dest_path, metadata)
|
||||
logger.info("Saved HF metadata (with hf_url) for %s", dest_path)
|
||||
logger.info(
|
||||
"Saved %s metadata (source=%s) for %s",
|
||||
ref.platform, ref.url, dest_path,
|
||||
)
|
||||
|
||||
# 4. Determine relative folder path for cache
|
||||
# model_root is an absolute path; dest_path is under it
|
||||
@@ -138,13 +136,12 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
|
||||
if scanner_getter is not None:
|
||||
scanner = await scanner_getter()
|
||||
if scanner is not None:
|
||||
metadata_dict = metadata.to_dict()
|
||||
metadata_dict["hf_url"] = hf_url
|
||||
metadata_dict = normalize_metadata_source(metadata.to_dict())
|
||||
await scanner.add_model_to_cache(metadata_dict, folder)
|
||||
logger.info("Added %s to scanner cache (folder=%s)", dest_path, folder)
|
||||
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to save HF metadata for %s: %s", dest_path, exc)
|
||||
logger.warning("Failed to save source metadata for %s: %s", dest_path, exc)
|
||||
|
||||
|
||||
def _find_matching_root(dest_dir: str) -> str | None:
|
||||
@@ -186,30 +183,87 @@ async def _add_to_scanner_cache(dest_path: str, metadata: dict[str, Any]) -> Non
|
||||
await scanner.update_single_model_cache(dest_path, dest_path, metadata)
|
||||
|
||||
|
||||
class HfHandler:
|
||||
"""Handle Hugging Face model browsing and download."""
|
||||
def _unsupported_platform_error(platform: str) -> web.Response:
|
||||
supported = ", ".join(source.label for source in list_sources() if source.supports_download)
|
||||
return web.json_response(
|
||||
{"error": f"'{platform}' does not support downloads. Supported: {supported}"},
|
||||
status=400,
|
||||
)
|
||||
|
||||
|
||||
class ModelSourceHandler:
|
||||
"""Handle external model browsing, linking and downloads."""
|
||||
|
||||
async def get_model_sources(self, request: web.Request) -> web.Response:
|
||||
"""List the external model sites the UI can link a model to.
|
||||
|
||||
Used by the "Link Model" dialog to validate URLs client-side, to
|
||||
explain which sites support AI metadata enrichment, and to pick the
|
||||
right download endpoint/revision.
|
||||
"""
|
||||
|
||||
return web.json_response([
|
||||
{
|
||||
"platform": source.platform,
|
||||
"label": source.label,
|
||||
"supports_enrichment": source.supports_enrichment,
|
||||
"supports_download": source.supports_download,
|
||||
"default_revision": source.default_revision,
|
||||
"example_url": source.canonical_url(
|
||||
"user/repo" if source.platform != "tensorart" else "827823520299086029"
|
||||
),
|
||||
}
|
||||
for source in list_sources()
|
||||
])
|
||||
|
||||
async def set_hf_url(self, request: web.Request) -> web.Response:
|
||||
"""Link a model file to its page on an external model site.
|
||||
|
||||
Accepts ``source_url`` (preferred) or the legacy ``hf_url`` / ``url``
|
||||
payload key. Every registered site is recognised and the platform is
|
||||
stored alongside the canonical URL. TensorArt models can be linked and
|
||||
browsed, but not AI-enriched.
|
||||
|
||||
The route path keeps its historical ``set-hf-url`` name.
|
||||
"""
|
||||
|
||||
try:
|
||||
payload: dict[str, Any] = await request.json()
|
||||
except json.JSONDecodeError:
|
||||
return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
|
||||
|
||||
file_path = (payload.get("file_path") or "").strip()
|
||||
hf_url = (payload.get("hf_url") or "").strip()
|
||||
raw_url = (
|
||||
payload.get("source_url")
|
||||
or payload.get("hf_url")
|
||||
or payload.get("url")
|
||||
or ""
|
||||
)
|
||||
source_url = raw_url.strip() if isinstance(raw_url, str) else ""
|
||||
|
||||
if not file_path or not hf_url:
|
||||
return web.json_response(
|
||||
{"success": False, "error": "Missing required fields: 'file_path' and 'hf_url'"},
|
||||
status=400,
|
||||
)
|
||||
|
||||
m = re.match(r"^https?://huggingface\.co/([^/]+/[^/]+)/?$", hf_url)
|
||||
if not m:
|
||||
if not file_path or not source_url:
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"error": "Invalid HuggingFace URL. Expected format: https://huggingface.co/user/repo",
|
||||
"error": "Missing required fields: 'file_path' and 'source_url'",
|
||||
},
|
||||
status=400,
|
||||
)
|
||||
|
||||
ref = detect_source(source_url, strict=True)
|
||||
if ref is None:
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"error": (
|
||||
"Unsupported model URL. Supported formats: "
|
||||
+ ", ".join(
|
||||
f"{s.label} ({s.canonical_url('user/repo')})"
|
||||
if s.platform != "tensorart"
|
||||
else f"{s.label} (https://tensor.art/models/<id>)"
|
||||
for s in list_sources()
|
||||
)
|
||||
),
|
||||
},
|
||||
status=400,
|
||||
)
|
||||
@@ -225,110 +279,120 @@ class HfHandler:
|
||||
return web.json_response(
|
||||
{
|
||||
"success": False,
|
||||
"error": "File is not within any configured model directory. Cannot link to HuggingFace.",
|
||||
"error": "File is not within any configured model directory. Cannot link to a model source.",
|
||||
},
|
||||
status=400,
|
||||
)
|
||||
|
||||
try:
|
||||
existing = await MetadataManager.load_metadata_payload(file_path)
|
||||
if existing.get("hf_url") == hf_url:
|
||||
|
||||
already_linked = (
|
||||
(existing.get("source_url") or "").strip() == ref.url
|
||||
and (existing.get("source_platform") or "").strip().lower()
|
||||
== ref.platform
|
||||
) or (
|
||||
not existing.get("source_url")
|
||||
and ref.platform == "huggingface"
|
||||
and (existing.get("hf_url") or "").strip() == ref.url
|
||||
)
|
||||
if already_linked:
|
||||
return web.json_response({
|
||||
"success": True,
|
||||
"message": "hf_url already set",
|
||||
"hf_url": hf_url,
|
||||
"message": "source_url already set",
|
||||
"source_url": ref.url,
|
||||
"source_platform": ref.platform,
|
||||
"hf_url": ref.url if ref.platform == "huggingface" else "",
|
||||
})
|
||||
|
||||
existing["hf_url"] = hf_url
|
||||
existing["source_url"] = ref.url
|
||||
existing["source_platform"] = ref.platform
|
||||
if ref.platform == "huggingface":
|
||||
existing["hf_url"] = ref.url
|
||||
else:
|
||||
existing.pop("hf_url", None)
|
||||
normalize_metadata_source(existing)
|
||||
|
||||
# NOTE: deliberately do NOT touch `from_civitai` here. It records
|
||||
# where the metadata came from, and the UI must show the CivitAI
|
||||
# link whenever CivitAI data is present — linking HuggingFace must
|
||||
# not hide it (#1094). HF provenance is tracked via `hf_url`.
|
||||
# link whenever CivitAI data is present — linking an external
|
||||
# source must not hide it (#1094). Source provenance is tracked
|
||||
# via `source_platform` / `source_url`.
|
||||
await MetadataManager.save_metadata(file_path, existing)
|
||||
|
||||
await _add_to_scanner_cache(file_path, existing)
|
||||
|
||||
logger.info("Set hf_url=%s for %s", hf_url, file_path)
|
||||
logger.info(
|
||||
"Linked %s to %s source (%s)", file_path, ref.platform, ref.url
|
||||
)
|
||||
return web.json_response({
|
||||
"success": True,
|
||||
"message": f"hf_url set to {hf_url}",
|
||||
"hf_url": hf_url,
|
||||
"message": f"Linked to {ref.url}",
|
||||
"source_url": ref.url,
|
||||
"source_platform": ref.platform,
|
||||
"hf_url": existing.get("hf_url", ""),
|
||||
})
|
||||
except Exception as exc:
|
||||
logger.error("Failed to set hf_url for %s: %s", file_path, exc)
|
||||
logger.error("Failed to link %s to a model source: %s", file_path, exc)
|
||||
return web.json_response(
|
||||
{"success": False, "error": str(exc)},
|
||||
status=500,
|
||||
)
|
||||
|
||||
async def get_hf_repo_files(self, request: web.Request) -> web.Response:
|
||||
"""List model-weight files from a HF repo with real file sizes.
|
||||
async def list_model_source_files(self, request: web.Request) -> web.Response:
|
||||
"""List the downloadable weight files of an external repository.
|
||||
|
||||
Uses the HF tree API endpoint which returns accurate file sizes
|
||||
(including LFS-tracked files), unlike the model info endpoint.
|
||||
Query params: ``platform``, ``repo`` (``owner/name``), ``revision``
|
||||
(optional; each site has its own default branch).
|
||||
|
||||
Returns a JSON array of ``{"filename", "size"}``, largest first —
|
||||
the same shape the Hugging Face endpoint has always returned.
|
||||
"""
|
||||
repo = request.query.get("repo", "").strip()
|
||||
if not repo or "/" not in repo:
|
||||
|
||||
platform = (request.query.get("platform") or "").strip()
|
||||
repo = (request.query.get("repo") or "").strip()
|
||||
revision = (request.query.get("revision") or "").strip()
|
||||
|
||||
source = get_download_source(platform)
|
||||
if source is None:
|
||||
return _unsupported_platform_error(platform)
|
||||
if not is_valid_source_id(repo):
|
||||
return web.json_response(
|
||||
{"error": "Missing or invalid 'repo' parameter (expected user/repo)"},
|
||||
{"error": "Missing or invalid 'repo' parameter (expected owner/name)"},
|
||||
status=400,
|
||||
)
|
||||
|
||||
url = f"https://huggingface.co/api/models/{repo}/tree/main"
|
||||
|
||||
try:
|
||||
session = await _get_hf_api_session()
|
||||
async with session.get(url) as resp:
|
||||
if resp.status == 404:
|
||||
return web.json_response(
|
||||
{"error": f"Repo '{repo}' not found"}, status=404
|
||||
)
|
||||
if resp.status != 200:
|
||||
text = await resp.text()
|
||||
return web.json_response(
|
||||
{"error": f"HF API error {resp.status}: {text[:200]}"},
|
||||
status=resp.status,
|
||||
)
|
||||
tree: list[dict[str, Any]] = await resp.json()
|
||||
files = await source.list_files(repo, revision)
|
||||
except ModelSourceError as exc:
|
||||
return web.json_response({"error": str(exc)}, status=exc.status)
|
||||
except Exception as exc:
|
||||
logger.error("Failed to fetch HF repo files: %s", exc)
|
||||
logger.error("Failed to list %s files in %s: %s", platform, repo, exc)
|
||||
return web.json_response({"error": str(exc)}, status=502)
|
||||
|
||||
files: list[dict[str, Any]] = []
|
||||
for entry in tree:
|
||||
path: str = entry.get("path", "")
|
||||
ext = os.path.splitext(path)[1].lower()
|
||||
if ext not in MODEL_FILE_EXTENSIONS:
|
||||
continue
|
||||
size = entry.get("size", 0) or 0
|
||||
if size == 0 and "lfs" in entry:
|
||||
size = entry["lfs"].get("size", 0) or 0
|
||||
files.append({
|
||||
"filename": path,
|
||||
"size": size,
|
||||
})
|
||||
|
||||
files.sort(key=lambda f: f["size"], reverse=True)
|
||||
return web.json_response(files)
|
||||
|
||||
async def download_hf_model(self, request: web.Request) -> web.Response:
|
||||
"""Download a single file from Hugging Face into the model directory.
|
||||
async def download_model_source(self, request: web.Request) -> web.Response:
|
||||
"""Download a single file from an external repository.
|
||||
|
||||
POST JSON body::
|
||||
|
||||
{
|
||||
"repo": "dx8152/Flux2-Klein-9B-Consistency",
|
||||
"filename": "Flux2-Klein-9B-consistency-V2.safetensors",
|
||||
"revision": "main",
|
||||
"platform": "modelscope",
|
||||
"repo": "owner/name",
|
||||
"filename": "subdir/model.safetensors",
|
||||
"revision": "master",
|
||||
"model_root": "loras",
|
||||
"relative_path": "",
|
||||
"use_default_paths": false,
|
||||
"download_id": "optional-batch-id"
|
||||
}
|
||||
|
||||
``platform`` defaults to ``huggingface`` when omitted, which keeps the
|
||||
legacy ``/api/lm/download-hf-model`` payload working unchanged.
|
||||
|
||||
If ``download_id`` is provided, real-time progress (bytes, speed,
|
||||
percentage) is broadcast via the WebSocket progress system, matching
|
||||
the CivitAI download experience.
|
||||
percentage) is broadcast via the WebSocket progress system.
|
||||
|
||||
Respects the ``download_backend`` setting (``aria2`` or ``default``).
|
||||
"""
|
||||
@@ -337,30 +401,33 @@ class HfHandler:
|
||||
except json.JSONDecodeError:
|
||||
return web.json_response({"error": "Invalid JSON"}, status=400)
|
||||
|
||||
platform = (payload.get("platform") or "huggingface").strip()
|
||||
repo = (payload.get("repo") or "").strip()
|
||||
filename = (payload.get("filename") or "").strip()
|
||||
revision = (payload.get("revision") or "main").strip()
|
||||
revision = (payload.get("revision") or "").strip()
|
||||
model_root = (payload.get("model_root") or "").strip()
|
||||
relative_path = (payload.get("relative_path") or "").strip()
|
||||
use_default_paths = bool(payload.get("use_default_paths", False))
|
||||
download_id: str | None = payload.get("download_id")
|
||||
|
||||
logger.info(
|
||||
"download_hf_model: repo=%s file=%s root=%s download_id=%s",
|
||||
repo, filename, model_root, download_id,
|
||||
"download_model_source: platform=%s repo=%s file=%s root=%s download_id=%s",
|
||||
platform, repo, filename, model_root, download_id,
|
||||
)
|
||||
|
||||
source = get_download_source(platform)
|
||||
if source is None:
|
||||
return _unsupported_platform_error(platform)
|
||||
|
||||
if not repo or not filename:
|
||||
return web.json_response(
|
||||
{"error": "Missing required fields: 'repo' and 'filename'"}, status=400
|
||||
)
|
||||
|
||||
# Validate repo format — must be user/repo_name
|
||||
if repo.count("/") != 1 or not re.match(r"^[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+$", repo):
|
||||
return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
|
||||
author, repo_name = repo.split("/", 1)
|
||||
if ".." in (author, repo_name) or "." in (author, repo_name):
|
||||
# `owner/name` only; the components become path segments below.
|
||||
if not is_valid_source_id(repo):
|
||||
return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
|
||||
owner, repo_name = repo.split("/", 1)
|
||||
|
||||
# Validate filename — must not contain path traversal
|
||||
if ".." in filename:
|
||||
@@ -379,21 +446,21 @@ class HfHandler:
|
||||
# unnecessary when the frontend sends the path from its own dropdown
|
||||
# (populated from scanner roots). Using the "business path" directly
|
||||
# keeps dest_path consistent with scanner roots so that later folder
|
||||
# derivation (in _save_hf_metadata) works correctly.
|
||||
# derivation (in _save_source_metadata) works correctly.
|
||||
if os.path.isabs(model_root):
|
||||
base_dir = os.path.normpath(model_root)
|
||||
else:
|
||||
base_dir = os.path.normpath(os.path.join(os.getcwd(), "models", model_root))
|
||||
|
||||
if use_default_paths:
|
||||
target_dir = os.path.join(base_dir, "huggingface", author, repo_name)
|
||||
target_dir = os.path.join(base_dir, source.default_subdir, owner, repo_name)
|
||||
elif relative_path:
|
||||
target_dir = os.path.join(base_dir, relative_path)
|
||||
else:
|
||||
target_dir = base_dir
|
||||
|
||||
# Strip HF repo subdirectory — "diffusion_models/xxx.safetensors"
|
||||
# is an HF repo convention, not meaningful for local storage.
|
||||
# Strip the repository sub-directory — "diffusion_models/xxx.safetensors"
|
||||
# is a repository convention, not meaningful for local storage.
|
||||
file_base = os.path.basename(filename)
|
||||
|
||||
os.makedirs(target_dir, exist_ok=True)
|
||||
@@ -401,16 +468,18 @@ class HfHandler:
|
||||
|
||||
# Check if already exists (simple skip)
|
||||
if os.path.exists(dest_path) and os.path.getsize(dest_path) > 0:
|
||||
logger.info("download_hf_model: file already exists, skipping — %s", dest_path)
|
||||
logger.info("download_model_source: file already exists, skipping — %s", dest_path)
|
||||
return web.json_response({
|
||||
"success": True,
|
||||
"message": f"File already exists: {dest_path}",
|
||||
"path": dest_path,
|
||||
})
|
||||
|
||||
# Build HF resolve URL
|
||||
resolve_url = (
|
||||
f"https://huggingface.co/{repo}/resolve/{revision}/{filename}"
|
||||
# Built per request: sites that redirect to a CDN hand out a
|
||||
# time-limited token in the redirect, so the URL must never be cached.
|
||||
resolve_url = source.file_download_url(repo, filename, revision)
|
||||
ref = SourceRef(
|
||||
platform=source.platform, source_id=repo, url=source.canonical_url(repo)
|
||||
)
|
||||
|
||||
# Set up progress callback if download_id is provided
|
||||
@@ -452,28 +521,27 @@ class HfHandler:
|
||||
|
||||
if download_backend == "aria2":
|
||||
aria2 = await Aria2Downloader.get_instance()
|
||||
aid = download_id or f"hf_{repo}_{filename}"
|
||||
aid = download_id or f"{source.platform}_{repo}_{filename}"
|
||||
try:
|
||||
hf_success, hf_result = await aria2.download_file(
|
||||
ok, result = await aria2.download_file(
|
||||
url=resolve_url,
|
||||
save_path=dest_path,
|
||||
download_id=aid,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
if hf_success:
|
||||
await _save_hf_metadata(dest_path, repo, model_root)
|
||||
if ok:
|
||||
await _save_source_metadata(dest_path, ref, model_root)
|
||||
return web.json_response({
|
||||
"success": True,
|
||||
"message": f"Downloaded to {dest_path}",
|
||||
"path": dest_path,
|
||||
})
|
||||
else:
|
||||
return web.json_response(
|
||||
{"success": False, "error": hf_result or "aria2 download failed"},
|
||||
status=500,
|
||||
)
|
||||
return web.json_response(
|
||||
{"success": False, "error": result or "aria2 download failed"},
|
||||
status=500,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("HF download (aria2) failed: %s", exc)
|
||||
logger.error("%s download (aria2) failed: %s", platform, exc)
|
||||
return web.json_response(
|
||||
{"success": False, "error": str(exc)}, status=500
|
||||
)
|
||||
@@ -489,19 +557,18 @@ class HfHandler:
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
if success:
|
||||
await _save_hf_metadata(dest_path, repo, model_root)
|
||||
await _save_source_metadata(dest_path, ref, model_root)
|
||||
return web.json_response({
|
||||
"success": True,
|
||||
"message": f"Downloaded to {result}",
|
||||
"path": result,
|
||||
})
|
||||
else:
|
||||
return web.json_response(
|
||||
{"success": False, "error": result or "Download failed"},
|
||||
status=500,
|
||||
)
|
||||
return web.json_response(
|
||||
{"success": False, "error": result or "Download failed"},
|
||||
status=500,
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.error("HF download failed: %s", exc)
|
||||
logger.error("%s download failed: %s", platform, exc)
|
||||
return web.json_response(
|
||||
{"success": False, "error": str(exc)}, status=500
|
||||
)
|
||||
@@ -99,7 +99,11 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
|
||||
RouteDefinition(
|
||||
"GET", "/api/lm/delete-model-version", "delete_model_version"
|
||||
),
|
||||
# Hugging Face model endpoints
|
||||
# External model source endpoints (Hugging Face / ModelScope).
|
||||
# The hf-* paths are the historical names, kept as aliases.
|
||||
RouteDefinition(
|
||||
"GET", "/api/lm/model-source-files", "list_model_source_files"
|
||||
),
|
||||
RouteDefinition(
|
||||
"GET", "/api/lm/hf-repo-files", "get_hf_repo_files"
|
||||
),
|
||||
@@ -107,12 +111,19 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
|
||||
RouteDefinition(
|
||||
"POST", "/api/lm/download/routing", "get_download_routing"
|
||||
),
|
||||
RouteDefinition(
|
||||
"POST", "/api/lm/download-model-source", "download_model_source"
|
||||
),
|
||||
RouteDefinition(
|
||||
"POST", "/api/lm/download-hf-model", "download_hf_model"
|
||||
),
|
||||
RouteDefinition(
|
||||
"POST", "/api/lm/set-hf-url", "set_hf_url"
|
||||
),
|
||||
# Supported external model sites (Hugging Face / ModelScope / TensorArt)
|
||||
RouteDefinition(
|
||||
"GET", "/api/lm/model-sources", "get_model_sources"
|
||||
),
|
||||
# Agent skill endpoints
|
||||
RouteDefinition(
|
||||
"GET", "/api/lm/agent/skills", "get_agent_skills"
|
||||
|
||||
@@ -39,7 +39,7 @@ from .handlers.misc_handlers import (
|
||||
build_service_registry_adapter,
|
||||
)
|
||||
from .handlers.base_model_handlers import BaseModelHandlerSet
|
||||
from .handlers.hf_handlers import HfHandler
|
||||
from .handlers.model_source_handlers import ModelSourceHandler
|
||||
from .handlers.agent_handlers import AgentHandler
|
||||
from .handlers.download_routing_handlers import DownloadRoutingHandler
|
||||
from .misc_route_registrar import MiscRouteRegistrar
|
||||
@@ -139,7 +139,7 @@ class MiscRoutes:
|
||||
doctor = DoctorHandler(settings_service=self._settings)
|
||||
example_workflows = ExampleWorkflowsHandler()
|
||||
base_model = BaseModelHandlerSet()
|
||||
hf_handler = HfHandler()
|
||||
model_source_handler = ModelSourceHandler()
|
||||
agent_handler = AgentHandler()
|
||||
download_routing = DownloadRoutingHandler()
|
||||
|
||||
@@ -161,7 +161,7 @@ class MiscRoutes:
|
||||
doctor=doctor,
|
||||
example_workflows=example_workflows,
|
||||
base_model=base_model,
|
||||
hf_handler=hf_handler,
|
||||
model_source_handler=model_source_handler,
|
||||
agent_handler=agent_handler,
|
||||
download_routing=download_routing,
|
||||
)
|
||||
|
||||
@@ -19,16 +19,18 @@ from __future__ import annotations
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
import aiohttp
|
||||
|
||||
import os
|
||||
|
||||
from ...config import config
|
||||
from ..llm_service import LLMService
|
||||
from ..model_sources import (
|
||||
get_source,
|
||||
resolve_source_ref,
|
||||
source_label,
|
||||
)
|
||||
from ..websocket_manager import ws_manager
|
||||
from .post_processor import PostProcessor
|
||||
from .skill_registry import SkillRegistry
|
||||
@@ -267,14 +269,17 @@ class AgentService:
|
||||
from ...metadata_ops import read_metadata
|
||||
metadata = await read_metadata(model_path)
|
||||
|
||||
# Fast-fail: enrich_hf_metadata requires hf_url to have HF README context
|
||||
if skill_name == "enrich_hf_metadata" and not metadata.get("hf_url", ""):
|
||||
logger.info(
|
||||
"[%s] SKIP %s — no hf_url in metadata",
|
||||
skill_name, model_filename,
|
||||
)
|
||||
skipped_count += 1
|
||||
skip_model = True
|
||||
# Fast-fail: enrich_hf_metadata needs an external model source
|
||||
# that exposes an accessible model card.
|
||||
if skill_name == "enrich_hf_metadata":
|
||||
skip_reason = self._enrichment_skip_reason(metadata)
|
||||
if skip_reason:
|
||||
logger.info(
|
||||
"[%s] SKIP %s — %s",
|
||||
skill_name, model_filename, skip_reason,
|
||||
)
|
||||
skipped_count += 1
|
||||
skip_model = True
|
||||
|
||||
if not skip_model:
|
||||
prompt_vars: Dict[str, Any] = {"model_path": model_path}
|
||||
@@ -358,6 +363,28 @@ class AgentService:
|
||||
# Base model grouping (keeps the prompt compact)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _enrichment_skip_reason(metadata: Dict[str, Any]) -> str:
|
||||
"""Return why ``enrich_hf_metadata`` cannot run, or ``""`` if it can.
|
||||
|
||||
Distinguishes the three cases the user can act on: no source linked,
|
||||
a source we don't know, and a known source whose model card is not
|
||||
reachable from the backend (TensorArt).
|
||||
"""
|
||||
|
||||
ref = resolve_source_ref(metadata)
|
||||
if ref is None:
|
||||
return "no model source linked (source_url missing)"
|
||||
source = get_source(ref.platform)
|
||||
if source is None:
|
||||
return f"unsupported model source platform '{ref.platform}'"
|
||||
if not source.supports_enrichment:
|
||||
return (
|
||||
f"{source.label} does not expose a model card to the backend; "
|
||||
"AI metadata enrichment is not available for this source"
|
||||
)
|
||||
return ""
|
||||
|
||||
@staticmethod
|
||||
def _format_base_models(models: List[str]) -> str:
|
||||
"""Format the base model list as a flat, one-per-line list.
|
||||
@@ -388,6 +415,14 @@ class AgentService:
|
||||
context: Dict[str, Any] = {
|
||||
"model_path": model_path,
|
||||
"model_basename": "",
|
||||
# Canonical external-source variables
|
||||
"source_url": "",
|
||||
"source_id": "",
|
||||
"source_platform": "",
|
||||
"source_label": "",
|
||||
"asset_base_url": "",
|
||||
# Legacy Hugging Face aliases (kept so older prompt templates and
|
||||
# third-party skills keep rendering)
|
||||
"hf_url": "",
|
||||
"repo": "",
|
||||
"readme_content": "",
|
||||
@@ -411,12 +446,20 @@ class AgentService:
|
||||
"size": metadata.get("size", 0),
|
||||
}
|
||||
|
||||
hf_url = metadata.get("hf_url", "")
|
||||
context["hf_url"] = hf_url
|
||||
repo = self._extract_repo_from_url(hf_url) if hf_url else ""
|
||||
context["repo"] = repo or ""
|
||||
if repo:
|
||||
readme = await self._fetch_readme(repo)
|
||||
ref = resolve_source_ref(metadata)
|
||||
if ref is not None:
|
||||
context["source_url"] = ref.url
|
||||
context["source_id"] = ref.source_id
|
||||
context["source_platform"] = ref.platform
|
||||
context["source_label"] = source_label(ref.platform, ref.platform)
|
||||
if ref.platform == "huggingface":
|
||||
context["hf_url"] = ref.url
|
||||
context["repo"] = ref.source_id
|
||||
|
||||
source = get_source(ref.platform) if ref is not None else None
|
||||
if ref is not None and source is not None and source.supports_enrichment:
|
||||
context["asset_base_url"] = source.asset_base_url(ref.source_id)
|
||||
readme = await source.fetch_model_card(ref.source_id)
|
||||
# Trim README to the section relevant to this model file
|
||||
# (collection repos often have multiple models in one README).
|
||||
if readme and raw_basename:
|
||||
@@ -458,20 +501,14 @@ class AgentService:
|
||||
|
||||
@staticmethod
|
||||
async def _fetch_readme(repo: str) -> str:
|
||||
"""Fetch README.md from HuggingFace (tries ``main``, then ``master``)."""
|
||||
async with aiohttp.ClientSession(
|
||||
headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
|
||||
timeout=aiohttp.ClientTimeout(total=30),
|
||||
) as session:
|
||||
for branch in ("main", "master"):
|
||||
url = f"https://huggingface.co/{repo}/raw/{branch}/README.md"
|
||||
try:
|
||||
async with session.get(url) as resp:
|
||||
if resp.status == 200:
|
||||
return await resp.text()
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to fetch README from %s: %s", url, exc)
|
||||
return ""
|
||||
"""Fetch a Hugging Face README (tries ``main``, then ``master``).
|
||||
|
||||
Kept for backward compatibility; new code should go through the
|
||||
model-source registry so every supported site works.
|
||||
"""
|
||||
from ..model_sources import HuggingFaceSource
|
||||
|
||||
return await HuggingFaceSource().fetch_model_card(repo)
|
||||
|
||||
async def _emit_progress(
|
||||
self,
|
||||
|
||||
@@ -78,6 +78,7 @@ class PostProcessor:
|
||||
download_preview,
|
||||
refresh_cache,
|
||||
)
|
||||
from ..model_sources import get_source, has_external_source, resolve_source_ref
|
||||
from .skills.enrich_hf_metadata.readme_processor import (
|
||||
convert_readme_to_html,
|
||||
extract_gallery_images,
|
||||
@@ -85,17 +86,25 @@ class PostProcessor:
|
||||
extract_relevant_section,
|
||||
extract_simple_markdown_images,
|
||||
extract_html_img_tags,
|
||||
extract_repo_from_hf_url,
|
||||
)
|
||||
|
||||
updated_fields: List[str] = []
|
||||
preview_downloaded = False
|
||||
|
||||
# -- Determine whether this is an HF-sourced model -----------------
|
||||
# Key off `hf_url` directly: `from_civitai` records provenance and can
|
||||
# be true for a model that is also linked to HuggingFace (both sources
|
||||
# coexist, see #1094), so it must not gate HF enrichment.
|
||||
is_hf_model = bool(metadata.get("hf_url", ""))
|
||||
# -- Determine whether this is an externally-sourced model ---------
|
||||
# Key off the source fields directly: `from_civitai` records provenance
|
||||
# and can be true for a model that is also linked to an external site
|
||||
# (both sources coexist, see #1094), so it must not gate enrichment.
|
||||
is_source_model = has_external_source(metadata)
|
||||
|
||||
source_ref = resolve_source_ref(metadata)
|
||||
source = get_source(source_ref.platform) if source_ref else None
|
||||
source_id = source_ref.source_id if source_ref else ""
|
||||
asset_base_url = (
|
||||
source.asset_base_url(source_id)
|
||||
if source is not None and source_id
|
||||
else None
|
||||
)
|
||||
|
||||
# -- Collect updates -----------------------------------------------
|
||||
updates: Dict[str, Any] = {}
|
||||
@@ -103,7 +112,7 @@ class PostProcessor:
|
||||
# base_model
|
||||
new_base = (llm_output.get("base_model") or "").strip()
|
||||
current_base = metadata.get("base_model", "") or ""
|
||||
if new_base and self._should_overwrite(current_base, is_hf_model):
|
||||
if new_base and self._should_overwrite(current_base, is_source_model):
|
||||
updates["base_model"] = new_base
|
||||
|
||||
# trigger words → civitai.trainedWords
|
||||
@@ -115,7 +124,7 @@ class PostProcessor:
|
||||
trigger_words_empty = not cleaned
|
||||
current_civitai = metadata.get("civitai") or {}
|
||||
current_triggers = current_civitai.get("trainedWords") or []
|
||||
if self._should_overwrite_list(current_triggers, is_hf_model):
|
||||
if self._should_overwrite_list(current_triggers, is_source_model):
|
||||
trig_civitai = dict(current_civitai)
|
||||
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
||||
trig_civitai.update(updates["civitai"])
|
||||
@@ -123,14 +132,14 @@ class PostProcessor:
|
||||
updates["civitai"] = trig_civitai
|
||||
|
||||
# modelDescription — from raw README content (converted to HTML)
|
||||
if readme_content and is_hf_model:
|
||||
if readme_content and is_source_model:
|
||||
converted = convert_readme_to_html(readme_content)
|
||||
if converted:
|
||||
updates["modelDescription"] = converted
|
||||
|
||||
# short_description → civitai.description (for "About this version")
|
||||
short_desc = (llm_output.get("short_description") or "").strip()
|
||||
if short_desc and is_hf_model:
|
||||
if short_desc and is_source_model:
|
||||
current_civitai = metadata.get("civitai") or {}
|
||||
desc_civitai = dict(current_civitai)
|
||||
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
||||
@@ -141,9 +150,8 @@ class PostProcessor:
|
||||
# gallery images → civitai.images (from YAML frontmatter widget entries
|
||||
# and Sample Gallery markdown tables in the README body)
|
||||
gallery_images: List[Dict[str, Any]] = []
|
||||
if readme_content and is_hf_model:
|
||||
hf_url = metadata.get("hf_url", "") or ""
|
||||
repo = extract_repo_from_hf_url(hf_url)
|
||||
if readme_content and is_source_model:
|
||||
repo = source_id
|
||||
if repo:
|
||||
rec_w = llm_output.get("recommended_width") or 0
|
||||
rec_h = llm_output.get("recommended_height") or 0
|
||||
@@ -152,6 +160,7 @@ class PostProcessor:
|
||||
gallery = extract_gallery_images(
|
||||
readme_content, repo,
|
||||
default_width=rec_w, default_height=rec_h,
|
||||
base_url=asset_base_url,
|
||||
)
|
||||
|
||||
# 2. Sample Gallery table images (markdown body), deduplicated
|
||||
@@ -160,6 +169,7 @@ class PostProcessor:
|
||||
readme_content, repo,
|
||||
existing_urls=existing_urls,
|
||||
default_width=rec_w, default_height=rec_h,
|
||||
base_url=asset_base_url,
|
||||
)
|
||||
existing_urls.update(img["url"] for img in table_images if img.get("url"))
|
||||
|
||||
@@ -168,6 +178,7 @@ class PostProcessor:
|
||||
readme_content, repo,
|
||||
existing_urls=existing_urls,
|
||||
default_width=rec_w, default_height=rec_h,
|
||||
base_url=asset_base_url,
|
||||
)
|
||||
existing_urls.update(img["url"] for img in simple_images if img.get("url"))
|
||||
|
||||
@@ -176,6 +187,7 @@ class PostProcessor:
|
||||
readme_content, repo,
|
||||
existing_urls=existing_urls,
|
||||
default_width=rec_w, default_height=rec_h,
|
||||
base_url=asset_base_url,
|
||||
)
|
||||
|
||||
all_images = gallery + table_images + simple_images + html_images
|
||||
@@ -193,7 +205,7 @@ class PostProcessor:
|
||||
if isinstance(new_tags, list) and new_tags:
|
||||
existing_tags = metadata.get("tags") or []
|
||||
merged = self._merge_tags(existing_tags, new_tags)
|
||||
if len(merged) > len(existing_tags) or is_hf_model:
|
||||
if len(merged) > len(existing_tags) or is_source_model:
|
||||
updates["tags"] = merged
|
||||
|
||||
# metadata_source & llm_enriched_at (always set)
|
||||
@@ -222,7 +234,7 @@ class PostProcessor:
|
||||
# README, find the first gallery image from the *model-specific
|
||||
# section* of the README (not the repo-wide first image, which
|
||||
# belongs to a different model in collection repos).
|
||||
if not preview_remote_url and readme_content and is_hf_model:
|
||||
if not preview_remote_url and readme_content and is_source_model:
|
||||
model_basename = os.path.splitext(os.path.basename(model_path))[0]
|
||||
relevant_section = extract_relevant_section(
|
||||
readme_content, model_basename,
|
||||
@@ -279,16 +291,16 @@ class PostProcessor:
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _should_overwrite(current_value: str, is_hf_model: bool) -> bool:
|
||||
def _should_overwrite(current_value: str, is_source_model: bool) -> bool:
|
||||
"""Return ``True`` when a scalar field should be overwritten."""
|
||||
return is_hf_model or not current_value or current_value.lower() in (
|
||||
return is_source_model or not current_value or current_value.lower() in (
|
||||
"", "unknown",
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _should_overwrite_list(current_list: List[str], is_hf_model: bool) -> bool:
|
||||
def _should_overwrite_list(current_list: List[str], is_source_model: bool) -> bool:
|
||||
"""Return ``True`` when a list field should be overwritten."""
|
||||
return is_hf_model or not current_list
|
||||
return is_source_model or not current_list
|
||||
|
||||
@staticmethod
|
||||
def _merge_tags(existing: List[str], new: List[str]) -> List[str]:
|
||||
|
||||
@@ -1,20 +1,23 @@
|
||||
---
|
||||
name: enrich_hf_metadata
|
||||
title: "Enrich Metadata from HuggingFace"
|
||||
title: "Enrich Metadata from Model Card"
|
||||
description: >
|
||||
Parse the HuggingFace model card via LLM to extract description, trigger
|
||||
words, base model, tags, and preview image URL.
|
||||
Parse the model card (README) from HuggingFace, ModelScope, or any other
|
||||
supported model site via LLM to extract description, trigger words, base
|
||||
model, tags, and preview image URL.
|
||||
llm_required: true
|
||||
---
|
||||
|
||||
You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a HuggingFace model card (README.md).
|
||||
You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a model card (README).
|
||||
|
||||
## Model Information
|
||||
|
||||
- **Repository**: {{hf_url}}
|
||||
- **Source site**: {{source_label}} ({{source_platform}})
|
||||
- **Model page**: {{source_url}}
|
||||
- **Model file path**: {{model_path}}
|
||||
- **Model filename**: {{model_basename}}
|
||||
- **Repository ID**: {{repo}}
|
||||
- **Repository ID**: {{source_id}}
|
||||
- **Repository raw-file base URL**: {{asset_base_url}}
|
||||
|
||||
## Current Metadata (may be incomplete)
|
||||
|
||||
@@ -39,7 +42,7 @@ name listed — do not invent aliases or modify variant suffixes.
|
||||
|
||||
{{base_models}}
|
||||
|
||||
## HuggingFace README Content
|
||||
## Model Card Content
|
||||
|
||||
```
|
||||
{{readme_content}}
|
||||
@@ -92,7 +95,7 @@ The URL of the most suitable preview image from the README. Look for:
|
||||
- The YAML frontmatter `widget:` section (which often has `output.url` fields)
|
||||
- In collection repos: the sample images listed **under the section** for this specific model version
|
||||
- Generic `` in the body
|
||||
Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL as `https://huggingface.co/{{repo}}/resolve/main/{filename}`. If no suitable image is found, return an empty string.
|
||||
Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL from the repository raw-file base URL (`{{asset_base_url}}`) plus the relative path. If no suitable image is found, return an empty string.
|
||||
|
||||
### notes
|
||||
A plain-text summary of the model card's key practical usage information. Combine trigger words, style modifiers, recommended parameters (steps, CFG, resolution, sampler), and any setup tips into a readable paragraph. For collection repos, focus on the **specific model version** matching `{{model_basename}}`. Return empty string if the README has no useful usage info.
|
||||
@@ -121,7 +124,7 @@ Your confidence level in the extracted data:
|
||||
|
||||
## Important: Handling Collection Repos (multiple model files)
|
||||
|
||||
Many HuggingFace repos contain **multiple model files** in a single repository
|
||||
Many model repositories contain **multiple model files** in a single repository
|
||||
(e.g. a "LoRA collection" with different styles/characters in separate files).
|
||||
|
||||
The model file currently being enriched is: **`{{model_basename}}`**
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
"""HF README processing for the ``enrich_hf_metadata`` skill.
|
||||
"""Model card (README) processing for the ``enrich_hf_metadata`` skill.
|
||||
|
||||
Provides README cleaning for LLM injection, gallery/image extraction from
|
||||
multiple formats (YAML widget, markdown, HTML ``<img>``, gallery tables),
|
||||
and section-based README trimming for collection repos.
|
||||
|
||||
The extractors default to Hugging Face asset URLs, but every one of them
|
||||
accepts an explicit ``base_url`` so the same parsing works for any model
|
||||
source (ModelScope, ...). See :mod:`py.services.model_sources`.
|
||||
|
||||
This module deliberately has no package-relative imports: it is also loaded
|
||||
standalone by the README-processing test harness.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -15,12 +22,25 @@ from typing import Any, List, Tuple
|
||||
_REPO_URL_PATTERN = re.compile(r"https?://huggingface\.co/([^/]+/[^/]+)")
|
||||
|
||||
|
||||
def resolve_asset_base_url(repo: str, base_url: str | None = None) -> str:
|
||||
"""Return the base URL used to resolve repository-relative assets.
|
||||
|
||||
Falls back to the historical Hugging Face layout when *base_url* is not
|
||||
supplied, so existing callers keep their behaviour.
|
||||
"""
|
||||
|
||||
if base_url:
|
||||
return base_url.rstrip("/")
|
||||
return f"https://huggingface.co/{repo}/resolve/main"
|
||||
|
||||
|
||||
def extract_simple_markdown_images(
|
||||
markdown_text: str,
|
||||
repo: str,
|
||||
existing_urls: set[str] | None = None,
|
||||
default_width: int = 512,
|
||||
default_height: int = 512,
|
||||
base_url: str | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Extract standalone markdown images from the README body.
|
||||
|
||||
@@ -32,10 +52,10 @@ def extract_simple_markdown_images(
|
||||
Returns a list of dicts in the same ``civitai.images`` format as
|
||||
:func:`extract_gallery_images`.
|
||||
"""
|
||||
if not markdown_text or not repo:
|
||||
if not markdown_text or not (repo or base_url):
|
||||
return []
|
||||
|
||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
||||
base_url = resolve_asset_base_url(repo, base_url)
|
||||
images: list[dict[str, Any]] = []
|
||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||
|
||||
@@ -89,20 +109,21 @@ def extract_html_img_tags(
|
||||
existing_urls: set[str] | None = None,
|
||||
default_width: int = 512,
|
||||
default_height: int = 512,
|
||||
base_url: str | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Extract image URLs from HTML ``<img src=\"...\">`` tags in the README.
|
||||
|
||||
Many HF collection repos (e.g. ``deadman44/Z-Image_LoRA``) use raw HTML
|
||||
``<img>`` tags exclusively for their sample images, with no markdown
|
||||
``![]()`` equivalents. This function finds those tags and constructs
|
||||
resolvable HF URLs.
|
||||
resolvable URLs.
|
||||
|
||||
Returns a list of dicts in the ``civitai.images`` format.
|
||||
"""
|
||||
if not markdown_text or not repo:
|
||||
if not markdown_text or not (repo or base_url):
|
||||
return []
|
||||
|
||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
||||
base_url = resolve_asset_base_url(repo, base_url)
|
||||
images: list[dict[str, Any]] = []
|
||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||
|
||||
@@ -166,7 +187,7 @@ def extract_html_img_tags(
|
||||
|
||||
def extract_repo_from_hf_url(hf_url: str) -> str:
|
||||
"""Extract ``user/repo`` from a HuggingFace URL."""
|
||||
m = _REPO_URL_PATTERN.match(hf_url)
|
||||
m = _REPO_URL_PATTERN.match(hf_url or "")
|
||||
return m.group(1) if m else ""
|
||||
|
||||
|
||||
@@ -175,21 +196,23 @@ def extract_gallery_images(
|
||||
repo: str,
|
||||
default_width: int = 512,
|
||||
default_height: int = 512,
|
||||
base_url: str | None = None,
|
||||
) -> List[dict[str, Any]]:
|
||||
"""Extract widget/gallery images from the YAML frontmatter of a HF README.
|
||||
"""Extract widget/gallery images from the YAML frontmatter of a README.
|
||||
|
||||
Args:
|
||||
markdown_text: Raw README content.
|
||||
repo: HF repo identifier (``user/repo``).
|
||||
repo: Repository identifier (``user/repo``).
|
||||
default_width: Fallback width when the README provides no dimension.
|
||||
default_height: Fallback height when the README provides no dimension.
|
||||
base_url: Overrides the asset base URL (defaults to Hugging Face).
|
||||
|
||||
Returns a list of dicts compatible with the ``civitai.images`` metadata
|
||||
format, each containing ``url`` (absolute HF URL), ``meta.prompt``,
|
||||
format, each containing ``url`` (absolute), ``meta.prompt``,
|
||||
``width``, ``height``, and ``type``. Returns an empty list when no
|
||||
widget entries are found or when *repo* is empty.
|
||||
"""
|
||||
if not markdown_text or not repo:
|
||||
if not markdown_text or not (repo or base_url):
|
||||
return []
|
||||
|
||||
frontmatter = _extract_frontmatter(markdown_text)
|
||||
@@ -197,7 +220,7 @@ def extract_gallery_images(
|
||||
return []
|
||||
|
||||
images: List[dict[str, Any]] = []
|
||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
||||
base_url = resolve_asset_base_url(repo, base_url)
|
||||
w = default_width or 512
|
||||
h = default_height or 512
|
||||
|
||||
@@ -279,10 +302,11 @@ def extract_gallery_table_images(
|
||||
existing_urls: set[str] | None = None,
|
||||
default_width: int = 512,
|
||||
default_height: int = 512,
|
||||
base_url: str | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Extract images from ``| Preview | Prompt |`` markdown gallery tables.
|
||||
|
||||
Many HF READMEs include a sample-gallery table in the body (outside
|
||||
Many READMEs include a sample-gallery table in the body (outside
|
||||
the YAML frontmatter) that shows generation examples with their
|
||||
prompts. This function parses those tables and merges results with
|
||||
the widget-sourced images from :func:`extract_gallery_images`.
|
||||
@@ -291,10 +315,10 @@ def extract_gallery_table_images(
|
||||
:func:`extract_gallery_images`. Already-seen URLs (from *existing_urls*)
|
||||
are skipped.
|
||||
"""
|
||||
if not markdown_text or not repo:
|
||||
if not markdown_text or not (repo or base_url):
|
||||
return []
|
||||
|
||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
||||
base_url = resolve_asset_base_url(repo, base_url)
|
||||
images: list[dict[str, Any]] = []
|
||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||
lines = markdown_text.split("\n")
|
||||
|
||||
@@ -21,6 +21,7 @@ from .model_query import (
|
||||
resolve_sub_type,
|
||||
)
|
||||
from .settings_manager import get_settings_manager
|
||||
from .model_sources import source_group_key
|
||||
from ..utils.civitai_utils import build_civitai_model_page_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -742,29 +743,32 @@ class BaseModelService(ABC):
|
||||
@staticmethod
|
||||
def _extract_hf_group_key(item: Dict[str, Any]) -> Optional[str]:
|
||||
"""Extract `hf:{owner}/{repo}` from item's ``hf_url``, or None."""
|
||||
hf_url = item.get("hf_url") if isinstance(item, dict) else None
|
||||
if not hf_url or not isinstance(hf_url, str):
|
||||
return None
|
||||
m = re.match(
|
||||
r"https?://huggingface\.co/([^/]+/[^/]+)", hf_url.strip()
|
||||
)
|
||||
if not m:
|
||||
return None
|
||||
return f"hf:{m.group(1)}"
|
||||
key = BaseModelService._extract_source_group_key(item)
|
||||
return key if key and key.startswith("hf:") else None
|
||||
|
||||
@staticmethod
|
||||
def _extract_source_group_key(item: Dict[str, Any]) -> Optional[str]:
|
||||
"""Return the external-source group key for *item*, or None.
|
||||
|
||||
Hugging Face keeps the historical ``hf:{owner}/{repo}`` shape; other
|
||||
platforms use their own short prefix (``ms:`` / ``ta:``).
|
||||
"""
|
||||
return source_group_key(item)
|
||||
|
||||
@staticmethod
|
||||
def _extract_group_key(item: Dict[str, Any]) -> Union[int, str, None]:
|
||||
"""Return the group identity key: CivitAI modelId (int) or HF repo (str).
|
||||
"""Return the group identity key.
|
||||
|
||||
Preference order:
|
||||
1. CivitAI ``modelId`` (int)
|
||||
2. HF repo identity ``hf:{owner}/{repo}`` (str)
|
||||
2. External model source identity, e.g. ``hf:{owner}/{repo}``,
|
||||
``ms:{owner}/{repo}``, ``ta:{model_id}`` (str)
|
||||
3. ``None`` (no known grouping source)
|
||||
"""
|
||||
mid = BaseModelService._extract_model_id(item)
|
||||
if mid is not None:
|
||||
return mid
|
||||
return BaseModelService._extract_hf_group_key(item)
|
||||
return BaseModelService._extract_source_group_key(item)
|
||||
|
||||
@staticmethod
|
||||
def _extract_model_id(item: Dict[str, Any]) -> Optional[int]:
|
||||
|
||||
@@ -67,6 +67,8 @@ class CheckpointService(BaseModelService):
|
||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||
"version_count": model_data.get("version_count"),
|
||||
"source_platform": model_data.get("source_platform", ""),
|
||||
"source_url": model_data.get("source_url", ""),
|
||||
"hf_url": model_data.get("hf_url", ""),
|
||||
}
|
||||
|
||||
|
||||
@@ -67,6 +67,8 @@ class EmbeddingService(BaseModelService):
|
||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||
"version_count": model_data.get("version_count"),
|
||||
"source_platform": model_data.get("source_platform", ""),
|
||||
"source_url": model_data.get("source_url", ""),
|
||||
"hf_url": model_data.get("hf_url", ""),
|
||||
}
|
||||
|
||||
|
||||
+58
-34
@@ -267,6 +267,16 @@ _PROVIDER_DEFAULTS: Dict[str, str] = {
|
||||
# Request timeout for LLM calls (seconds)
|
||||
_LLM_TIMEOUT = aiohttp.ClientTimeout(total=120)
|
||||
|
||||
# Providers that do NOT implement ``response_format: {"type": "json_schema"}``
|
||||
# and reject it with HTTP 400. For these the weaker, widely supported
|
||||
# ``json_object`` mode is used instead (the prompt already specifies the
|
||||
# expected JSON shape, and ``_try_salvage_json`` repairs imperfect output).
|
||||
# DeepSeek answers a json_schema request with
|
||||
# ``{"error":{"message":"This response_format type is unavailable now"}}``.
|
||||
# LM Studio and some other local OpenAI-compatible servers reject
|
||||
# ``json_object`` but accept ``json_schema``, so they are not listed here.
|
||||
_JSON_OBJECT_ONLY_PROVIDERS = frozenset({"deepseek"})
|
||||
|
||||
|
||||
class LLMService:
|
||||
"""Centralized LLM API client.
|
||||
@@ -614,47 +624,61 @@ class LLMService:
|
||||
if effective_max is None:
|
||||
effective_max = 4096
|
||||
|
||||
# Use json_schema (not json_object) for broader provider compatibility:
|
||||
# LM Studio and some other OpenAI-compatible servers reject
|
||||
# json_object but accept json_schema. {"type": "object"} is
|
||||
# functionally equivalent — it accepts any JSON object without
|
||||
# constraining specific fields.
|
||||
response_format = {
|
||||
# Structured-output format. ``json_schema`` is preferred because LM
|
||||
# Studio and other local OpenAI-compatible servers reject
|
||||
# ``json_object`` but accept ``json_schema``; ``{"type": "object"}``
|
||||
# accepts any JSON object without constraining specific fields, so the
|
||||
# two modes are functionally equivalent here. Providers known to
|
||||
# reject json_schema (see _JSON_OBJECT_ONLY_PROVIDERS) get
|
||||
# ``json_object`` instead.
|
||||
schema_format: Dict[str, Any] = {
|
||||
"type": "json_schema",
|
||||
"json_schema": {
|
||||
"name": "metadata",
|
||||
"schema": {"type": "object"},
|
||||
},
|
||||
}
|
||||
json_object_format: Dict[str, Any] = {"type": "json_object"}
|
||||
|
||||
try:
|
||||
result = await self.chat_completion(
|
||||
messages=messages,
|
||||
model=model,
|
||||
temperature=temperature,
|
||||
response_format=response_format,
|
||||
max_tokens=effective_max,
|
||||
)
|
||||
except LLMResponseError as e:
|
||||
# Only fall back when the provider rejects the response_format
|
||||
# type value (e.g. "'response_format.type' must be..."). Avoid
|
||||
# catching unrelated 400 errors whose body happens to mention
|
||||
# "response_format" (e.g. "model does not support
|
||||
# response_format restrictions on this endpoint").
|
||||
if "'response_format.type'" not in str(e).lower():
|
||||
raise
|
||||
logger.info(
|
||||
"Provider rejected response_format, retrying without it. "
|
||||
"Falling back to prompt-only JSON mode. Error: %s",
|
||||
e,
|
||||
)
|
||||
result = await self.chat_completion(
|
||||
messages=messages,
|
||||
model=model,
|
||||
temperature=temperature,
|
||||
response_format=None,
|
||||
max_tokens=effective_max,
|
||||
)
|
||||
if self._get_config()["provider"] in _JSON_OBJECT_ONLY_PROVIDERS:
|
||||
format_chain: List[Optional[Dict[str, Any]]] = [
|
||||
json_object_format,
|
||||
None,
|
||||
]
|
||||
else:
|
||||
format_chain = [schema_format, json_object_format, None]
|
||||
|
||||
result: Optional[Dict[str, Any]] = None
|
||||
for index, fmt in enumerate(format_chain):
|
||||
try:
|
||||
result = await self.chat_completion(
|
||||
messages=messages,
|
||||
model=model,
|
||||
temperature=temperature,
|
||||
response_format=fmt,
|
||||
max_tokens=effective_max,
|
||||
)
|
||||
break
|
||||
except LLMResponseError as e:
|
||||
message = str(e).lower()
|
||||
if index + 1 >= len(format_chain):
|
||||
raise
|
||||
# Only downgrade when the failure is about ``response_format``.
|
||||
# Everything else (auth, unknown model, rate limits) must
|
||||
# surface unchanged. Matching on the bare parameter name also
|
||||
# covers variants such as DeepSeek's "This response_format
|
||||
# type is unavailable now" without swallowing unrelated 400s.
|
||||
if "response_format" not in message:
|
||||
raise
|
||||
logger.info(
|
||||
"Provider rejected response_format=%s, retrying with %s. "
|
||||
"Error: %s",
|
||||
(fmt or {}).get("type", "none"),
|
||||
(format_chain[index + 1] or {}).get("type", "none"),
|
||||
e,
|
||||
)
|
||||
|
||||
assert result is not None # non-empty chain always sets or raises
|
||||
|
||||
content = result.get("content", "") or ""
|
||||
if not content:
|
||||
|
||||
@@ -79,6 +79,8 @@ class LoraService(BaseModelService):
|
||||
),
|
||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||
"version_count": model_data.get("version_count"),
|
||||
"source_platform": model_data.get("source_platform", ""),
|
||||
"source_url": model_data.get("source_url", ""),
|
||||
"hf_url": model_data.get("hf_url", ""),
|
||||
}
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ from ..utils.model_utils import determine_base_model
|
||||
from ..utils.models import autov3_from_civitai_files
|
||||
from .connectivity_guard import OFFLINE_FRIENDLY_MESSAGE, is_expected_offline_error
|
||||
from .errors import RateLimitError
|
||||
from .model_sources import has_external_source
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -222,9 +223,10 @@ class MetadataSyncService:
|
||||
error_msg = "CivitAI model is deleted and no archive provider is available"
|
||||
return False, error_msg
|
||||
else:
|
||||
is_hf_source = bool(model_data.get("hf_url"))
|
||||
is_hf_source = has_external_source(model_data)
|
||||
if is_hf_source:
|
||||
# HF-sourced model: only check CivitAI API directly.
|
||||
# External-source model (Hugging Face / ModelScope /
|
||||
# TensorArt): only check CivitAI API directly.
|
||||
# CivArchive is almost guaranteed to have no record, and
|
||||
# hitting it wastes rate-limit budget.
|
||||
# Use a distinct provider name ("civitai_api" not None) so
|
||||
|
||||
@@ -15,6 +15,7 @@ from ..utils.civitai_utils import resolve_license_info
|
||||
from .model_cache import ModelCache
|
||||
from .model_hash_index import ModelHashIndex
|
||||
from .model_lifecycle_service import delete_model_artifacts, _require_path_in_library_roots
|
||||
from .model_sources import normalize_metadata_source
|
||||
from .service_registry import ServiceRegistry
|
||||
from .websocket_manager import ws_manager
|
||||
from .persistent_model_cache import get_persistent_cache
|
||||
@@ -387,8 +388,14 @@ class ModelScanner:
|
||||
'civitai': civitai_slim,
|
||||
'civitai_deleted': bool(get_value('civitai_deleted', False)),
|
||||
'skip_metadata_refresh': bool(get_value('skip_metadata_refresh', False)),
|
||||
# External model source (Hugging Face / ModelScope / TensorArt).
|
||||
# `source_url` + `source_platform` are canonical; `hf_url` stays in
|
||||
# sync as a legacy alias (normalised below).
|
||||
'source_platform': get_value('source_platform', '') or '',
|
||||
'source_url': get_value('source_url', '') or '',
|
||||
'hf_url': get_value('hf_url', '') or '',
|
||||
}
|
||||
normalize_metadata_source(entry)
|
||||
|
||||
license_source: Dict[str, Any] = {}
|
||||
if isinstance(civitai_full, Mapping):
|
||||
|
||||
@@ -0,0 +1,73 @@
|
||||
"""External model-source providers (Hugging Face, ModelScope, TensorArt).
|
||||
|
||||
This package is the single abstraction over "a site that hosts models and
|
||||
a model card". See :mod:`py.services.model_sources.base` for the provider
|
||||
protocol and :mod:`py.services.model_sources.registry` for the lookup and
|
||||
metadata-normalisation helpers used across the codebase.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .base import (
|
||||
GROUP_PREFIXES,
|
||||
HTTP_TIMEOUT,
|
||||
ModelSource,
|
||||
ModelSourceError,
|
||||
SourceRef,
|
||||
USER_AGENT,
|
||||
clean_source_url,
|
||||
fetch_json,
|
||||
fetch_text,
|
||||
filter_weight_files,
|
||||
is_valid_source_id,
|
||||
)
|
||||
from .huggingface import HuggingFaceSource
|
||||
from .modelscope import ModelScopeSource
|
||||
from .registry import (
|
||||
LEGACY_HF_URL_FIELD,
|
||||
SOURCE_PLATFORM_FIELD,
|
||||
SOURCE_URL_FIELD,
|
||||
detect_source,
|
||||
downloadable_sources,
|
||||
get_download_source,
|
||||
get_source,
|
||||
get_source_platform,
|
||||
has_external_source,
|
||||
list_sources,
|
||||
normalize_metadata_source,
|
||||
resolve_source_ref,
|
||||
source_group_key,
|
||||
source_label,
|
||||
)
|
||||
from .tensorart import TensorArtSource
|
||||
|
||||
__all__ = [
|
||||
"GROUP_PREFIXES",
|
||||
"HTTP_TIMEOUT",
|
||||
"LEGACY_HF_URL_FIELD",
|
||||
"ModelSource",
|
||||
"ModelSourceError",
|
||||
"HuggingFaceSource",
|
||||
"ModelScopeSource",
|
||||
"SOURCE_PLATFORM_FIELD",
|
||||
"SOURCE_URL_FIELD",
|
||||
"SourceRef",
|
||||
"TensorArtSource",
|
||||
"USER_AGENT",
|
||||
"clean_source_url",
|
||||
"detect_source",
|
||||
"downloadable_sources",
|
||||
"fetch_json",
|
||||
"fetch_text",
|
||||
"filter_weight_files",
|
||||
"get_download_source",
|
||||
"get_source",
|
||||
"get_source_platform",
|
||||
"has_external_source",
|
||||
"is_valid_source_id",
|
||||
"list_sources",
|
||||
"normalize_metadata_source",
|
||||
"resolve_source_ref",
|
||||
"source_group_key",
|
||||
"source_label",
|
||||
]
|
||||
@@ -0,0 +1,313 @@
|
||||
"""Base types for the external model-source provider abstraction.
|
||||
|
||||
A *model source* is a third-party site that hosts model files and a model
|
||||
card (README) describing them — Hugging Face, ModelScope, TensorArt, and
|
||||
whatever gets added later. Everything the rest of the codebase needs to
|
||||
know about such a site is expressed by :class:`ModelSource`:
|
||||
|
||||
* how to recognise one of its URLs (:meth:`ModelSource.parse`)
|
||||
* the canonical page URL for a source id (:meth:`ModelSource.canonical_url`)
|
||||
* how to fetch the model card (:meth:`ModelSource.fetch_model_card`)
|
||||
* how to turn repository-relative asset paths into absolute URLs
|
||||
(:meth:`ModelSource.asset_base_url`)
|
||||
* which capabilities the site actually supports
|
||||
(``supports_enrichment`` / ``supports_download``)
|
||||
|
||||
Keeping this in one place means the agent pipeline, the scanners, and the
|
||||
HTTP handlers never need site-specific branching.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, Iterable, Optional
|
||||
|
||||
import aiohttp
|
||||
|
||||
from ...utils.constants import MODEL_FILE_EXTENSIONS
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Shared HTTP timeout for model-card fetches.
|
||||
HTTP_TIMEOUT = 30
|
||||
|
||||
#: User agent used for all model-source HTTP requests.
|
||||
USER_AGENT = "ComfyUI-LoRA-Manager/1.0"
|
||||
|
||||
#: Platform → short prefix used when building version-group keys.
|
||||
#: ``huggingface`` keeps the historical ``hf:`` prefix for backward
|
||||
#: compatibility with already-cached group keys.
|
||||
GROUP_PREFIXES: dict[str, str] = {
|
||||
"huggingface": "hf",
|
||||
"modelscope": "ms",
|
||||
"tensorart": "ta",
|
||||
}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceRef:
|
||||
"""A parsed reference to a model hosted on an external site."""
|
||||
|
||||
platform: str
|
||||
"""Canonical platform id, e.g. ``"huggingface"``."""
|
||||
|
||||
source_id: str
|
||||
"""Site-specific identity, e.g. ``"user/repo"`` or ``"827823520299086029"``."""
|
||||
|
||||
url: str
|
||||
"""Canonical URL of the model page."""
|
||||
|
||||
|
||||
class ModelSourceError(Exception):
|
||||
"""Raised when a model source cannot satisfy a request.
|
||||
|
||||
Carries the HTTP status the API handler should answer with, so the
|
||||
handlers stay free of per-site error mapping.
|
||||
"""
|
||||
|
||||
def __init__(self, message: str, status: int = 502) -> None:
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
|
||||
|
||||
#: Repository ids are always exactly ``owner/name``. Components may contain
|
||||
#: dots (``black-forest-labs/FLUX.1-dev``) but must not be empty, ``.`` / ``..``,
|
||||
#: or start with a dot - the id is used as a path segment on disk.
|
||||
_SOURCE_ID_COMPONENT = re.compile(r"^[A-Za-z0-9_][A-Za-z0-9_.\-]*$")
|
||||
|
||||
|
||||
def is_valid_source_id(source_id: str) -> bool:
|
||||
"""Return ``True`` when *source_id* is a safe ``owner/name`` repository id."""
|
||||
|
||||
if not source_id or not isinstance(source_id, str) or source_id.count("/") != 1:
|
||||
return False
|
||||
owner, name = source_id.split("/", 1)
|
||||
return all(
|
||||
part and part not in (".", "..") and _SOURCE_ID_COMPONENT.match(part)
|
||||
for part in (owner, name)
|
||||
)
|
||||
|
||||
|
||||
async def fetch_text(url: str, *, timeout: int = HTTP_TIMEOUT) -> str:
|
||||
"""Fetch *url* and return its body as text, or ``""`` on any failure.
|
||||
|
||||
Network problems are expected (offline installs, rate limits, dead
|
||||
repos) and must never bubble up into the pipeline, so every error is
|
||||
logged at debug level and normalised to an empty string.
|
||||
"""
|
||||
|
||||
try:
|
||||
async with aiohttp.ClientSession(
|
||||
headers={"User-Agent": USER_AGENT},
|
||||
timeout=aiohttp.ClientTimeout(total=timeout),
|
||||
) as session:
|
||||
async with session.get(url) as resp:
|
||||
if resp.status == 200:
|
||||
return await resp.text()
|
||||
logger.debug("Fetch %s returned HTTP %s", url, resp.status)
|
||||
except Exception as exc: # pragma: no cover - network dependent
|
||||
logger.debug("Failed to fetch %s: %s", url, exc)
|
||||
return ""
|
||||
|
||||
|
||||
async def fetch_json(
|
||||
url: str, *, timeout: int = HTTP_TIMEOUT
|
||||
) -> tuple[int, Any]:
|
||||
"""Fetch *url* and return ``(status, parsed_body)``.
|
||||
|
||||
Unlike :func:`fetch_text` this reports the status, because callers such as
|
||||
the file-listing endpoints need to distinguish "repo not found" (404) from
|
||||
a transport failure. ``parsed_body`` is ``None`` when the response is not
|
||||
JSON or the request failed outright (status ``0``).
|
||||
"""
|
||||
|
||||
try:
|
||||
async with aiohttp.ClientSession(
|
||||
headers={"User-Agent": USER_AGENT},
|
||||
timeout=aiohttp.ClientTimeout(total=timeout),
|
||||
) as session:
|
||||
async with session.get(url) as resp:
|
||||
if resp.status != 200:
|
||||
return resp.status, None
|
||||
try:
|
||||
return resp.status, await resp.json(content_type=None)
|
||||
except Exception:
|
||||
return resp.status, None
|
||||
except Exception as exc: # pragma: no cover - network dependent
|
||||
logger.debug("Failed to fetch %s: %s", url, exc)
|
||||
return 0, None
|
||||
|
||||
|
||||
class ModelSource:
|
||||
"""Description and I/O for one external model hosting site."""
|
||||
|
||||
#: Canonical platform id stored in metadata.
|
||||
platform: str = ""
|
||||
|
||||
#: Human-readable name used in UI copy and prompts.
|
||||
label: str = ""
|
||||
|
||||
#: Whether the agent skill can fetch a model card and run AI extraction.
|
||||
supports_enrichment: bool = False
|
||||
|
||||
#: Whether models can be downloaded directly from this site.
|
||||
supports_download: bool = False
|
||||
|
||||
#: Branch used when the caller does not pass an explicit revision.
|
||||
default_revision: str = ""
|
||||
|
||||
#: Sub-directory the "use default paths" template places downloads in.
|
||||
default_subdir: str = ""
|
||||
|
||||
#: Lenient pattern used to recognise URLs already stored in metadata.
|
||||
#: Captures the site-specific source id in group ``id``.
|
||||
url_pattern: re.Pattern[str] | None = None
|
||||
|
||||
#: Strict pattern used to validate user input. Must match the whole URL.
|
||||
strict_url_pattern: re.Pattern[str] | None = None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Parsing
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def parse(self, url: str, *, strict: bool = False) -> Optional[str]:
|
||||
"""Return the source id contained in *url*, or ``None``.
|
||||
|
||||
With ``strict=True`` the URL must match this site's canonical shape
|
||||
exactly (used when validating what a user pasted); with
|
||||
``strict=False`` sub-paths such as ``/resolve/main/file.bin`` are
|
||||
tolerated (used when normalising already-stored values).
|
||||
"""
|
||||
|
||||
if not url or not isinstance(url, str):
|
||||
return None
|
||||
candidate = url.strip()
|
||||
if not candidate:
|
||||
return None
|
||||
pattern = self.strict_url_pattern if strict else self.url_pattern
|
||||
if pattern is None:
|
||||
return None
|
||||
match = pattern.match(candidate)
|
||||
return match.group("id") if match else None
|
||||
|
||||
def ref(self, url: str, *, strict: bool = False) -> Optional[SourceRef]:
|
||||
"""Return a :class:`SourceRef` for *url*, or ``None`` if not ours."""
|
||||
|
||||
source_id = self.parse(url, strict=strict)
|
||||
if not source_id:
|
||||
return None
|
||||
return SourceRef(
|
||||
platform=self.platform,
|
||||
source_id=source_id,
|
||||
url=self.canonical_url(source_id),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# URLs and content
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def canonical_url(self, source_id: str) -> str:
|
||||
"""Return the canonical model-page URL for *source_id*."""
|
||||
|
||||
raise NotImplementedError
|
||||
|
||||
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||
"""Base URL used to resolve repository-relative asset paths."""
|
||||
|
||||
return ""
|
||||
|
||||
def group_key(self, source_id: str) -> str:
|
||||
"""Return the version-group key for *source_id*."""
|
||||
|
||||
prefix = GROUP_PREFIXES.get(self.platform, self.platform)
|
||||
return f"{prefix}:{source_id}"
|
||||
|
||||
async def fetch_model_card(self, source_id: str) -> str:
|
||||
"""Fetch the raw model card (README) markdown for *source_id*."""
|
||||
|
||||
return ""
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Download support
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
async def list_files(
|
||||
self, source_id: str, revision: str = ""
|
||||
) -> list[dict[str, Any]]:
|
||||
"""List downloadable weight files in *source_id*.
|
||||
|
||||
Returns ``[{"filename": <repo-relative path>, "size": <bytes>}]``,
|
||||
largest first, filtered to :data:`MODEL_FILE_EXTENSIONS`. Sites
|
||||
without download support return an empty list.
|
||||
|
||||
Raises :class:`ModelSourceError` when the repository cannot be read,
|
||||
so the handler can surface "not found" separately from a transport
|
||||
failure.
|
||||
"""
|
||||
|
||||
return []
|
||||
|
||||
def file_download_url(
|
||||
self, source_id: str, filename: str, revision: str = ""
|
||||
) -> str:
|
||||
"""Return the direct (redirecting) download URL for one file."""
|
||||
|
||||
raise ModelSourceError(
|
||||
f"{self.label or self.platform} does not support downloads", status=400
|
||||
)
|
||||
|
||||
def resolve_revision(self, revision: str = "") -> str:
|
||||
"""Return *revision*, falling back to this site's default branch."""
|
||||
|
||||
return revision or self.default_revision
|
||||
|
||||
def page_url_for_file(self, source_id: str, filename: str) -> str:
|
||||
"""Return the human-facing page for *filename* inside *source_id*."""
|
||||
|
||||
return self.canonical_url(source_id)
|
||||
|
||||
def __repr__(self) -> str: # pragma: no cover - debugging aid
|
||||
return f"<ModelSource {self.platform}>"
|
||||
|
||||
|
||||
def clean_source_url(url: Any) -> str:
|
||||
"""Normalise a stored source URL value into a stripped string."""
|
||||
|
||||
if not isinstance(url, str):
|
||||
return ""
|
||||
return url.strip()
|
||||
|
||||
|
||||
def filter_weight_files(entries: Iterable[tuple[str, int]]) -> list[dict[str, Any]]:
|
||||
"""Keep model-weight files from ``(path, size)`` pairs, largest first.
|
||||
|
||||
Every site lists a lot more than weights (READMEs, configs, tokenizers,
|
||||
…); the download picker only ever wants the files ComfyUI can load, which
|
||||
is exactly :data:`MODEL_FILE_EXTENSIONS`.
|
||||
"""
|
||||
|
||||
files = [
|
||||
{"filename": path, "size": int(size or 0)}
|
||||
for path, size in entries
|
||||
if path and os.path.splitext(path)[1].lower() in MODEL_FILE_EXTENSIONS
|
||||
]
|
||||
files.sort(key=lambda entry: entry["size"], reverse=True)
|
||||
return files
|
||||
|
||||
|
||||
__all__ = [
|
||||
"GROUP_PREFIXES",
|
||||
"HTTP_TIMEOUT",
|
||||
"ModelSource",
|
||||
"ModelSourceError",
|
||||
"SourceRef",
|
||||
"USER_AGENT",
|
||||
"clean_source_url",
|
||||
"fetch_json",
|
||||
"fetch_text",
|
||||
"filter_weight_files",
|
||||
"is_valid_source_id",
|
||||
]
|
||||
@@ -0,0 +1,106 @@
|
||||
"""Hugging Face model source."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
|
||||
from .base import (
|
||||
ModelSource,
|
||||
ModelSourceError,
|
||||
fetch_json,
|
||||
fetch_text,
|
||||
filter_weight_files,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Lenient — used to normalise URLs already stored in metadata; tolerates
|
||||
#: sub-paths such as ``/resolve/main/model.safetensors``.
|
||||
_URL_PATTERN = re.compile(
|
||||
r"https?://(?:www\.)?huggingface\.co/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||
)
|
||||
|
||||
#: Strict — validates what the user pasted into the "link model" dialog.
|
||||
_STRICT_URL_PATTERN = re.compile(
|
||||
r"https?://(?:www\.)?huggingface\.co/(?P<id>[^/?#\s]+/[^/?#\s]+)/?$"
|
||||
)
|
||||
|
||||
|
||||
class HuggingFaceSource(ModelSource):
|
||||
"""Hugging Face Hub (``huggingface.co``)."""
|
||||
|
||||
platform = "huggingface"
|
||||
label = "Hugging Face"
|
||||
supports_enrichment = True
|
||||
supports_download = True
|
||||
default_revision = "main"
|
||||
default_subdir = "huggingface"
|
||||
url_pattern = _URL_PATTERN
|
||||
strict_url_pattern = _STRICT_URL_PATTERN
|
||||
|
||||
def canonical_url(self, source_id: str) -> str:
|
||||
return f"https://huggingface.co/{source_id}"
|
||||
|
||||
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||
return f"https://huggingface.co/{source_id}/resolve/{self.resolve_revision(revision)}"
|
||||
|
||||
async def fetch_model_card(self, source_id: str) -> str:
|
||||
"""Fetch ``README.md`` from Hugging Face (tries ``main``, then ``master``)."""
|
||||
|
||||
for branch in ("main", "master"):
|
||||
text = await fetch_text(
|
||||
f"https://huggingface.co/{source_id}/raw/{branch}/README.md"
|
||||
)
|
||||
if text:
|
||||
return text
|
||||
return ""
|
||||
|
||||
async def list_files(
|
||||
self, source_id: str, revision: str = ""
|
||||
) -> list[dict]:
|
||||
"""List weight files via the Hub tree API.
|
||||
|
||||
The tree endpoint (rather than the model-info endpoint) is used
|
||||
because it reports accurate sizes for LFS-tracked files.
|
||||
"""
|
||||
|
||||
revision = self.resolve_revision(revision)
|
||||
status, payload = await fetch_json(
|
||||
f"https://huggingface.co/api/models/{source_id}/tree/{revision}"
|
||||
)
|
||||
|
||||
if status == 404:
|
||||
raise ModelSourceError(f"Repository '{source_id}' not found", status=404)
|
||||
if status != 200 or not isinstance(payload, list):
|
||||
raise ModelSourceError(
|
||||
f"Hugging Face API error while listing '{source_id}' (HTTP {status})"
|
||||
)
|
||||
|
||||
entries = []
|
||||
for entry in payload:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
path = entry.get("path", "")
|
||||
size = entry.get("size", 0) or 0
|
||||
if not size and isinstance(entry.get("lfs"), dict):
|
||||
size = entry["lfs"].get("size", 0) or 0
|
||||
entries.append((path, size))
|
||||
|
||||
return filter_weight_files(entries)
|
||||
|
||||
def file_download_url(
|
||||
self, source_id: str, filename: str, revision: str = ""
|
||||
) -> str:
|
||||
return (
|
||||
f"https://huggingface.co/{source_id}/resolve/"
|
||||
f"{self.resolve_revision(revision)}/{filename}"
|
||||
)
|
||||
|
||||
def page_url_for_file(self, source_id: str, filename: str) -> str:
|
||||
return (
|
||||
f"https://huggingface.co/{source_id}/blob/{self.default_revision}/{filename}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["HuggingFaceSource"]
|
||||
@@ -0,0 +1,144 @@
|
||||
"""ModelScope (魔搭社区) model source.
|
||||
|
||||
ModelScope exposes the same "model card as README.md" convention as
|
||||
Hugging Face, including a YAML frontmatter block that often carries
|
||||
``base_model:`` and ``trigger_words:``. Three public endpoints are used,
|
||||
none of which requires an API key for public models:
|
||||
|
||||
* ``/models/{owner}/{name}/resolve/{revision}/README.md`` — raw model card
|
||||
* ``/api/v1/models/{owner}/{name}/repo?Revision=..&FilePath=README.md`` —
|
||||
the same content through the API, used as a fallback when the resolve
|
||||
URL is unavailable.
|
||||
* ``/api/v1/models/{owner}/{name}/repo/files?Revision=..`` — the file
|
||||
listing backing the download picker. It reports real sizes for LFS
|
||||
files (not the pointer size), so no extra HEAD request is needed.
|
||||
|
||||
Downloads go through ``/models/{owner}/{name}/resolve/{revision}/{path}``,
|
||||
which redirects to a CDN URL carrying a time-limited ``auth_key``.
|
||||
Requesting the resolve URL fresh on every attempt (which the shared
|
||||
downloader does, including for resumable Range requests) keeps that key
|
||||
valid; the CDN URL must never be cached.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
|
||||
from .base import (
|
||||
ModelSource,
|
||||
ModelSourceError,
|
||||
fetch_json,
|
||||
fetch_text,
|
||||
filter_weight_files,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_URL_PATTERN = re.compile(
|
||||
r"https?://(?:www\.)?modelscope\.(?:cn|com)/models/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||
)
|
||||
|
||||
#: Trailing view segments the site appends to a model URL; accepted verbatim
|
||||
#: when the user pastes a browser tab URL.
|
||||
_VIEW_SEGMENTS = r"(?:summary|files|model-file|readme|community|evaluation)?"
|
||||
|
||||
_STRICT_URL_PATTERN = re.compile(
|
||||
r"https?://(?:www\.)?modelscope\.(?:cn|com)/models/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||
rf"/?{_VIEW_SEGMENTS}/?$"
|
||||
)
|
||||
|
||||
#: ``master`` is ModelScope's default branch; ``main`` is tried as a fallback
|
||||
#: for repos imported from Hugging Face.
|
||||
_REVISIONS = ("master", "main")
|
||||
|
||||
|
||||
class ModelScopeSource(ModelSource):
|
||||
"""ModelScope (``modelscope.cn``)."""
|
||||
|
||||
platform = "modelscope"
|
||||
label = "ModelScope"
|
||||
supports_enrichment = True
|
||||
supports_download = True
|
||||
default_revision = "master"
|
||||
default_subdir = "modelscope"
|
||||
url_pattern = _URL_PATTERN
|
||||
strict_url_pattern = _STRICT_URL_PATTERN
|
||||
|
||||
def canonical_url(self, source_id: str) -> str:
|
||||
return f"https://modelscope.cn/models/{source_id}"
|
||||
|
||||
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||
return (
|
||||
f"https://modelscope.cn/models/{source_id}/resolve/"
|
||||
f"{self.resolve_revision(revision)}"
|
||||
)
|
||||
|
||||
async def fetch_model_card(self, source_id: str) -> str:
|
||||
"""Fetch the model card, preferring the raw resolve URL."""
|
||||
|
||||
for revision in _REVISIONS:
|
||||
text = await fetch_text(
|
||||
f"https://modelscope.cn/models/{source_id}/resolve/{revision}/README.md"
|
||||
)
|
||||
if text:
|
||||
return text
|
||||
|
||||
# Fallback: the repo API proxies the same file and is reachable in
|
||||
# environments where the CDN resolve host is blocked.
|
||||
for revision in _REVISIONS:
|
||||
text = await fetch_text(
|
||||
"https://modelscope.cn/api/v1/models/"
|
||||
f"{source_id}/repo?Revision={revision}&FilePath=README.md"
|
||||
)
|
||||
if text:
|
||||
return text
|
||||
return ""
|
||||
|
||||
async def list_files(
|
||||
self, source_id: str, revision: str = ""
|
||||
) -> list[dict]:
|
||||
"""List weight files via the repo files API.
|
||||
|
||||
``master`` is the only branch name the API accepts — even repos
|
||||
imported from Hugging Face are addressed as ``master`` (``main``
|
||||
returns 404) — so no fallback probing is done here.
|
||||
"""
|
||||
|
||||
revision = self.resolve_revision(revision)
|
||||
status, payload = await fetch_json(
|
||||
"https://modelscope.cn/api/v1/models/"
|
||||
f"{source_id}/repo/files?Revision={revision}"
|
||||
)
|
||||
|
||||
if status == 404:
|
||||
raise ModelSourceError(f"Repository '{source_id}' not found", status=404)
|
||||
if status != 200 or not isinstance(payload, dict):
|
||||
raise ModelSourceError(
|
||||
f"ModelScope API error while listing '{source_id}' (HTTP {status})"
|
||||
)
|
||||
|
||||
entries = []
|
||||
for entry in (payload.get("Data") or {}).get("Files") or []:
|
||||
if not isinstance(entry, dict) or entry.get("Type") != "blob":
|
||||
continue
|
||||
entries.append((entry.get("Path", ""), entry.get("Size", 0) or 0))
|
||||
|
||||
return filter_weight_files(entries)
|
||||
|
||||
def file_download_url(
|
||||
self, source_id: str, filename: str, revision: str = ""
|
||||
) -> str:
|
||||
return (
|
||||
f"https://modelscope.cn/models/{source_id}/resolve/"
|
||||
f"{self.resolve_revision(revision)}/{filename}"
|
||||
)
|
||||
|
||||
def page_url_for_file(self, source_id: str, filename: str) -> str:
|
||||
return (
|
||||
f"https://modelscope.cn/models/{source_id}/file/view/"
|
||||
f"{self.default_revision}/{filename}"
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["ModelScopeSource"]
|
||||
@@ -0,0 +1,225 @@
|
||||
"""Registry and metadata helpers for external model sources.
|
||||
|
||||
The registry is the single place the rest of the codebase asks "which site
|
||||
is this URL from?", "what is this model's source?", and "can we enrich it?".
|
||||
Import from :mod:`py.services.model_sources` rather than this module
|
||||
directly.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Dict, Mapping, Optional
|
||||
|
||||
from .base import GROUP_PREFIXES, ModelSource, SourceRef, clean_source_url
|
||||
from .huggingface import HuggingFaceSource
|
||||
from .modelscope import ModelScopeSource
|
||||
from .tensorart import TensorArtSource
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
#: Order matters only for disambiguation; the URL patterns are disjoint.
|
||||
_SOURCES: tuple[ModelSource, ...] = (
|
||||
HuggingFaceSource(),
|
||||
ModelScopeSource(),
|
||||
TensorArtSource(),
|
||||
)
|
||||
|
||||
_BY_PLATFORM: Dict[str, ModelSource] = {s.platform: s for s in _SOURCES}
|
||||
|
||||
#: Metadata keys that carry the canonical external-source identity.
|
||||
SOURCE_PLATFORM_FIELD = "source_platform"
|
||||
SOURCE_URL_FIELD = "source_url"
|
||||
#: Legacy field kept as a read/write alias for Hugging Face models so that
|
||||
#: older sidecars, cached rows, and third-party consumers keep working.
|
||||
LEGACY_HF_URL_FIELD = "hf_url"
|
||||
|
||||
|
||||
def list_sources() -> list[ModelSource]:
|
||||
"""Return every known model source."""
|
||||
|
||||
return list(_SOURCES)
|
||||
|
||||
|
||||
def get_source(platform: Optional[str]) -> Optional[ModelSource]:
|
||||
"""Return the source registered for *platform*, or ``None``."""
|
||||
|
||||
if not platform or not isinstance(platform, str):
|
||||
return None
|
||||
return _BY_PLATFORM.get(platform.strip().lower())
|
||||
|
||||
|
||||
def source_label(platform: Optional[str], default: str = "") -> str:
|
||||
"""Return the human-readable label for *platform*."""
|
||||
|
||||
source = get_source(platform)
|
||||
return source.label if source else default
|
||||
|
||||
|
||||
def downloadable_sources() -> list[ModelSource]:
|
||||
"""Return the sources whose repositories can be downloaded directly."""
|
||||
|
||||
return [source for source in _SOURCES if source.supports_download]
|
||||
|
||||
|
||||
def get_download_source(platform: Optional[str]) -> Optional[ModelSource]:
|
||||
"""Return the source for *platform*, but only when it supports downloads."""
|
||||
|
||||
source = get_source(platform)
|
||||
if source is None or not source.supports_download:
|
||||
return None
|
||||
return source
|
||||
|
||||
|
||||
def detect_source(url: Optional[str], *, strict: bool = False) -> Optional[SourceRef]:
|
||||
"""Return the :class:`SourceRef` for *url*, or ``None`` if unsupported."""
|
||||
|
||||
if not url or not isinstance(url, str):
|
||||
return None
|
||||
for source in _SOURCES:
|
||||
ref = source.ref(url, strict=strict)
|
||||
if ref is not None:
|
||||
return ref
|
||||
return None
|
||||
|
||||
|
||||
def resolve_source_ref(metadata: Mapping[str, Any]) -> Optional[SourceRef]:
|
||||
"""Return the source reference described by a model's metadata.
|
||||
|
||||
Handles all three storage states found in the wild:
|
||||
|
||||
1. ``source_url`` + ``source_platform`` (current format)
|
||||
2. ``hf_url`` only (legacy Hugging Face storage)
|
||||
3. ``hf_url`` plus a newer ``source_url`` (both written by older builds)
|
||||
"""
|
||||
|
||||
if not isinstance(metadata, Mapping):
|
||||
return None
|
||||
|
||||
platform = clean_source_url(metadata.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||
url = clean_source_url(metadata.get(SOURCE_URL_FIELD))
|
||||
legacy = clean_source_url(metadata.get(LEGACY_HF_URL_FIELD))
|
||||
|
||||
source = get_source(platform)
|
||||
if url:
|
||||
if source is not None:
|
||||
ref = source.ref(url)
|
||||
if ref is not None:
|
||||
return ref
|
||||
ref = detect_source(url)
|
||||
if ref is not None:
|
||||
return ref
|
||||
# Unknown platform but a URL is present: keep it addressable.
|
||||
return SourceRef(platform=platform or "unknown", source_id="", url=url)
|
||||
|
||||
if legacy:
|
||||
return detect_source(legacy)
|
||||
return None
|
||||
|
||||
|
||||
def normalize_metadata_source(metadata: Dict[str, Any]) -> Dict[str, Any]:
|
||||
"""Normalise the external-source fields on *metadata* in place.
|
||||
|
||||
Guarantees that ``source_url``/``source_platform`` are present and
|
||||
consistent, and that ``hf_url`` mirrors ``source_url`` for Hugging Face
|
||||
models (never for other platforms, so a stale alias can't make a
|
||||
ModelScope model look like a Hugging Face one).
|
||||
|
||||
Returns the same dict for convenient chaining.
|
||||
"""
|
||||
|
||||
if not isinstance(metadata, dict):
|
||||
return metadata
|
||||
|
||||
platform = clean_source_url(metadata.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||
url = clean_source_url(metadata.get(SOURCE_URL_FIELD))
|
||||
legacy = clean_source_url(metadata.get(LEGACY_HF_URL_FIELD))
|
||||
|
||||
source = get_source(platform)
|
||||
ref: Optional[SourceRef] = None
|
||||
|
||||
if url:
|
||||
ref = source.ref(url) if source is not None else None
|
||||
if ref is None:
|
||||
ref = detect_source(url)
|
||||
elif legacy:
|
||||
ref = detect_source(legacy)
|
||||
|
||||
if ref is not None and ref.source_id:
|
||||
platform = ref.platform
|
||||
url = ref.url or url
|
||||
|
||||
if platform:
|
||||
metadata[SOURCE_PLATFORM_FIELD] = platform
|
||||
else:
|
||||
metadata.setdefault(SOURCE_PLATFORM_FIELD, "")
|
||||
|
||||
metadata[SOURCE_URL_FIELD] = url
|
||||
|
||||
# Keep the legacy alias in sync, but only for Hugging Face.
|
||||
if url and platform == "huggingface":
|
||||
metadata[LEGACY_HF_URL_FIELD] = url
|
||||
elif LEGACY_HF_URL_FIELD in metadata and platform and platform != "huggingface":
|
||||
metadata[LEGACY_HF_URL_FIELD] = ""
|
||||
elif legacy and not url:
|
||||
metadata[LEGACY_HF_URL_FIELD] = legacy
|
||||
|
||||
return metadata
|
||||
|
||||
|
||||
def has_external_source(item: Mapping[str, Any]) -> bool:
|
||||
"""Return ``True`` when *item* is linked to any external model site."""
|
||||
|
||||
if not isinstance(item, Mapping):
|
||||
return False
|
||||
return bool(
|
||||
clean_source_url(item.get(SOURCE_URL_FIELD))
|
||||
or clean_source_url(item.get(LEGACY_HF_URL_FIELD))
|
||||
)
|
||||
|
||||
|
||||
def get_source_platform(item: Mapping[str, Any]) -> str:
|
||||
"""Return the platform id stored on *item* (may be empty)."""
|
||||
|
||||
if not isinstance(item, Mapping):
|
||||
return ""
|
||||
platform = clean_source_url(item.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||
if platform:
|
||||
return platform
|
||||
ref = resolve_source_ref(item)
|
||||
return ref.platform if ref else ""
|
||||
|
||||
|
||||
def source_group_key(item: Mapping[str, Any]) -> Optional[str]:
|
||||
"""Return the version-group key for *item*, or ``None``.
|
||||
|
||||
Hugging Face keeps the historical ``hf:{owner}/{repo}`` shape; other
|
||||
platforms use their own short prefix (see :data:`GROUP_PREFIXES`).
|
||||
"""
|
||||
|
||||
ref = resolve_source_ref(item)
|
||||
if ref is None or not ref.source_id:
|
||||
return None
|
||||
source = get_source(ref.platform)
|
||||
if source is None:
|
||||
return None
|
||||
return source.group_key(ref.source_id)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"GROUP_PREFIXES",
|
||||
"LEGACY_HF_URL_FIELD",
|
||||
"SOURCE_PLATFORM_FIELD",
|
||||
"SOURCE_URL_FIELD",
|
||||
"detect_source",
|
||||
"downloadable_sources",
|
||||
"get_download_source",
|
||||
"get_source",
|
||||
"get_source_platform",
|
||||
"has_external_source",
|
||||
"list_sources",
|
||||
"normalize_metadata_source",
|
||||
"resolve_source_ref",
|
||||
"source_group_key",
|
||||
"source_label",
|
||||
]
|
||||
@@ -0,0 +1,56 @@
|
||||
"""TensorArt model source (link / provenance only).
|
||||
|
||||
TensorArt support is intentionally limited to *linking* a model to its
|
||||
TensorArt page. Automatic metadata extraction is not possible without a
|
||||
user session:
|
||||
|
||||
* ``tensor.art`` sits behind a Cloudflare managed challenge, so plain
|
||||
HTTP clients (aiohttp, requests, curl) receive ``403 "Just a moment..."``.
|
||||
* Its internal API (``ap-east-1.tensorart.cloud`` / ``cn.tensorart.net``)
|
||||
answers every ``/v1/model/*`` route with
|
||||
``{"code":100002,"message":"invalid authorization header"}``.
|
||||
* The official TAMS API requires an AccessKey/SecretKey pair and request
|
||||
signatures, which is a poor fit for a "paste a URL" workflow.
|
||||
|
||||
``supports_enrichment`` is therefore ``False``: the agent pipeline skips
|
||||
these models with an explicit reason instead of failing silently, and the
|
||||
UI keeps showing the "View on TensorArt" link. ``tusi.cn`` is TensorArt's
|
||||
Chinese mirror and is accepted as the same platform.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from .base import ModelSource
|
||||
|
||||
_DOMAINS = r"(?:tensor\.art|tusi\.cn)"
|
||||
|
||||
_URL_PATTERN = re.compile(
|
||||
rf"https?://(?:www\.)?{_DOMAINS}/models/(?P<id>\d+)"
|
||||
)
|
||||
|
||||
_STRICT_URL_PATTERN = re.compile(
|
||||
rf"https?://(?:www\.)?{_DOMAINS}/models/(?P<id>\d+)(?:/[^/?#\s]+)?/?$"
|
||||
)
|
||||
|
||||
|
||||
class TensorArtSource(ModelSource):
|
||||
"""TensorArt (``tensor.art``)."""
|
||||
|
||||
platform = "tensorart"
|
||||
label = "TensorArt"
|
||||
supports_enrichment = False
|
||||
supports_download = False
|
||||
url_pattern = _URL_PATTERN
|
||||
strict_url_pattern = _STRICT_URL_PATTERN
|
||||
|
||||
def canonical_url(self, source_id: str) -> str:
|
||||
return f"https://tensor.art/models/{source_id}"
|
||||
|
||||
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||
# Unreachable today: enrichment is disabled for this platform.
|
||||
return f"https://tensor.art/models/{source_id}"
|
||||
|
||||
|
||||
__all__ = ["TensorArtSource"]
|
||||
@@ -67,6 +67,8 @@ class OtherModelService(BaseModelService):
|
||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||
"version_count": model_data.get("version_count"),
|
||||
"source_platform": model_data.get("source_platform", ""),
|
||||
"source_url": model_data.get("source_url", ""),
|
||||
"hf_url": model_data.get("hf_url", ""),
|
||||
}
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ from dataclasses import dataclass, field
|
||||
from typing import Any, Dict, List, Mapping, Optional, Sequence, Tuple
|
||||
|
||||
from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration
|
||||
from .model_sources import normalize_metadata_source
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -62,6 +63,8 @@ class PersistentModelCache:
|
||||
"db_checked",
|
||||
"last_checked_at",
|
||||
"hash_status",
|
||||
"source_platform",
|
||||
"source_url",
|
||||
"hf_url",
|
||||
)
|
||||
_MODEL_UPDATE_COLUMNS: Tuple[str, ...] = _MODEL_COLUMNS[2:]
|
||||
@@ -206,8 +209,13 @@ class PersistentModelCache:
|
||||
"skip_metadata_refresh": bool(row["skip_metadata_refresh"]),
|
||||
"license_flags": int(license_value),
|
||||
"hash_status": row["hash_status"] or "completed",
|
||||
"source_platform": row["source_platform"] or "",
|
||||
"source_url": row["source_url"] or "",
|
||||
"hf_url": row["hf_url"] or "",
|
||||
}
|
||||
# Legacy rows only carry `hf_url`; derive the canonical pair so
|
||||
# every consumer sees the same shape.
|
||||
normalize_metadata_source(item)
|
||||
if row["autov3"] is not None:
|
||||
item["autov3"] = (row["autov3"] or "").lower()
|
||||
raw_data.append(item)
|
||||
@@ -562,6 +570,8 @@ class PersistentModelCache:
|
||||
db_checked INTEGER,
|
||||
last_checked_at REAL,
|
||||
hash_status TEXT,
|
||||
source_platform TEXT DEFAULT '',
|
||||
source_url TEXT DEFAULT '',
|
||||
hf_url TEXT DEFAULT '',
|
||||
PRIMARY KEY (model_type, file_path)
|
||||
);
|
||||
@@ -629,6 +639,8 @@ class PersistentModelCache:
|
||||
# Persisting without explicit flags should assume CivitAI's documented defaults (0b111001 == 57).
|
||||
"license_flags": f"INTEGER DEFAULT {DEFAULT_LICENSE_FLAGS}",
|
||||
"hash_status": "TEXT DEFAULT 'completed'",
|
||||
"source_platform": "TEXT DEFAULT ''",
|
||||
"source_url": "TEXT DEFAULT ''",
|
||||
"hf_url": "TEXT DEFAULT ''",
|
||||
"autov3": "TEXT",
|
||||
}
|
||||
@@ -650,6 +662,9 @@ class PersistentModelCache:
|
||||
return conn
|
||||
|
||||
def _prepare_model_row(self, model_type: str, item: Dict[str, Any]) -> Tuple[Any, ...]:
|
||||
# Keep `source_*` and the legacy `hf_url` alias consistent no matter
|
||||
# which caller populated the item.
|
||||
normalize_metadata_source(item)
|
||||
civitai = item.get("civitai") or {}
|
||||
trained_words = civitai.get("trainedWords")
|
||||
if isinstance(trained_words, str):
|
||||
@@ -713,6 +728,8 @@ class PersistentModelCache:
|
||||
1 if item.get("db_checked") else 0,
|
||||
float(item.get("last_checked_at") or 0.0),
|
||||
item.get("hash_status", "completed"),
|
||||
item.get("source_platform") or "",
|
||||
item.get("source_url") or "",
|
||||
item.get("hf_url") or "",
|
||||
)
|
||||
|
||||
|
||||
@@ -7,6 +7,7 @@ import time
|
||||
from typing import Any, Dict, List, Optional, Protocol, Sequence
|
||||
|
||||
from ..metadata_sync_service import MetadataSyncService
|
||||
from ..model_sources import has_external_source
|
||||
from ...utils.metadata_manager import MetadataManager
|
||||
|
||||
|
||||
@@ -51,10 +52,11 @@ class BulkMetadataRefreshUseCase:
|
||||
if not model.get("skip_metadata_refresh", False)
|
||||
and not self._is_in_skip_path(model.get("folder", ""), skip_paths)
|
||||
and (not model.get("civitai") or not model["civitai"].get("id"))
|
||||
# Skip models downloaded from Hugging Face — they are not on
|
||||
# CivitAI / CivArchive. Users can still refresh them individually
|
||||
# via the right-click context menu.
|
||||
and not model.get("hf_url", "")
|
||||
# Skip models linked to an external model site (Hugging Face /
|
||||
# ModelScope / TensorArt) — they are not on CivitAI / CivArchive.
|
||||
# Users can still refresh them individually via the right-click
|
||||
# context menu.
|
||||
and not has_external_source(model)
|
||||
and not (
|
||||
# Skip models confirmed not on CivitAI when no need to retry
|
||||
model.get("from_civitai") is False
|
||||
|
||||
@@ -203,10 +203,18 @@ export const DOWNLOAD_ENDPOINTS = {
|
||||
exampleImagesMissing: '/api/lm/download-example-images' // Download only missing example images
|
||||
};
|
||||
|
||||
// Hugging Face API endpoints
|
||||
// External model source endpoints (Hugging Face / ModelScope).
|
||||
// The hf-* paths are the historical names, kept as server-side aliases.
|
||||
export const MODEL_SOURCE_ENDPOINTS = {
|
||||
repoFiles: '/api/lm/model-source-files',
|
||||
download: '/api/lm/download-model-source',
|
||||
sources: '/api/lm/model-sources',
|
||||
};
|
||||
|
||||
/** @deprecated use MODEL_SOURCE_ENDPOINTS */
|
||||
export const HF_ENDPOINTS = {
|
||||
repoFiles: '/api/lm/hf-repo-files',
|
||||
download: '/api/lm/download-hf-model',
|
||||
repoFiles: MODEL_SOURCE_ENDPOINTS.repoFiles,
|
||||
download: MODEL_SOURCE_ENDPOINTS.download,
|
||||
};
|
||||
|
||||
// WebSocket endpoints
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
isValidModelType,
|
||||
DOWNLOAD_ENDPOINTS,
|
||||
HF_ENDPOINTS,
|
||||
MODEL_SOURCE_ENDPOINTS,
|
||||
WS_ENDPOINTS
|
||||
} from './apiConfig.js';
|
||||
import { resetAndReload } from './modelApiFactory.js';
|
||||
@@ -1367,30 +1368,52 @@ export class BaseModelApiClient {
|
||||
}
|
||||
}
|
||||
|
||||
async fetchHfRepoFiles(repo, revision = 'main') {
|
||||
/**
|
||||
* List the downloadable weight files of an external repository.
|
||||
* @param {string} repo - `owner/name`
|
||||
* @param {string} [platform] - `huggingface` (default) or `modelscope`
|
||||
* @param {string} [revision] - branch; each site has its own default
|
||||
*/
|
||||
async fetchModelSourceFiles(repo, platform = 'huggingface', revision = '') {
|
||||
try {
|
||||
const params = new URLSearchParams({ repo, revision });
|
||||
const response = await fetch(`${HF_ENDPOINTS.repoFiles}?${params}`);
|
||||
const params = new URLSearchParams({ repo, platform });
|
||||
if (revision) params.set('revision', revision);
|
||||
const response = await fetch(`${MODEL_SOURCE_ENDPOINTS.repoFiles}?${params}`);
|
||||
if (!response.ok) {
|
||||
const err = await response.json().catch(() => ({}));
|
||||
throw new Error(err.error || 'Failed to fetch HF repo files');
|
||||
throw new Error(err.error || 'Failed to fetch repository files');
|
||||
}
|
||||
return await response.json();
|
||||
} catch (error) {
|
||||
console.error('Error fetching HF repo files:', error);
|
||||
console.error('Error fetching repository files:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
async downloadHfModel({ repo, filename, revision, modelRoot, relativePath, useDefaultPaths, download_id }) {
|
||||
/** Backwards-compatible Hugging Face wrapper. */
|
||||
async fetchHfRepoFiles(repo, revision = 'main') {
|
||||
return this.fetchModelSourceFiles(repo, 'huggingface', revision);
|
||||
}
|
||||
|
||||
async downloadModelSource({
|
||||
platform = 'huggingface',
|
||||
repo,
|
||||
filename,
|
||||
revision,
|
||||
modelRoot,
|
||||
relativePath,
|
||||
useDefaultPaths,
|
||||
download_id,
|
||||
}) {
|
||||
try {
|
||||
const response = await fetch(HF_ENDPOINTS.download, {
|
||||
const response = await fetch(MODEL_SOURCE_ENDPOINTS.download, {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
platform,
|
||||
repo,
|
||||
filename,
|
||||
revision: revision || 'main',
|
||||
revision: revision || '',
|
||||
model_root: modelRoot,
|
||||
relative_path: relativePath || '',
|
||||
use_default_paths: useDefaultPaths || false,
|
||||
@@ -1404,11 +1427,25 @@ export class BaseModelApiClient {
|
||||
|
||||
return await response.json();
|
||||
} catch (error) {
|
||||
console.error('Error downloading HF model:', error);
|
||||
console.error('Error downloading model:', error);
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
/** Backwards-compatible Hugging Face wrapper. */
|
||||
async downloadHfModel({ repo, filename, revision, modelRoot, relativePath, useDefaultPaths, download_id }) {
|
||||
return this.downloadModelSource({
|
||||
platform: 'huggingface',
|
||||
repo,
|
||||
filename,
|
||||
revision: revision || 'main',
|
||||
modelRoot,
|
||||
relativePath,
|
||||
useDefaultPaths,
|
||||
download_id,
|
||||
});
|
||||
}
|
||||
|
||||
_buildQueryParams(baseParams, pageState) {
|
||||
const params = new URLSearchParams(baseParams);
|
||||
const isExcludedView = pageState.viewMode === 'excluded';
|
||||
|
||||
@@ -7,6 +7,8 @@ import { MODEL_CONFIG } from '../../api/apiConfig.js';
|
||||
import { translate } from '../../utils/i18nHelpers.js';
|
||||
import { getNsfwLevelSelector } from '../shared/NsfwLevelSelector.js';
|
||||
import { classifyModelRelinkUrl } from '../../utils/civitaiUtils.js';
|
||||
import { parseModelSourceUrl, getModelSourceInfo } from '../../utils/modelSourceHelpers.js';
|
||||
import { escapeHtml } from '../shared/utils.js';
|
||||
|
||||
// Mixin with shared functionality for LoraContextMenu and CheckpointContextMenu
|
||||
export const ModelContextMenuMixin = {
|
||||
@@ -211,7 +213,7 @@ export const ModelContextMenuMixin = {
|
||||
setTimeout(() => urlInput.focus(), 50);
|
||||
},
|
||||
|
||||
// HuggingFace linking methods
|
||||
// External model source linking (Hugging Face / ModelScope / TensorArt)
|
||||
showLinkHfModal() {
|
||||
const filePath = this.currentCard.dataset.filepath;
|
||||
if (!filePath) return;
|
||||
@@ -225,15 +227,23 @@ export const ModelContextMenuMixin = {
|
||||
}
|
||||
|
||||
this._boundLinkHfHandler = async () => {
|
||||
const hfUrl = urlInput.value.trim();
|
||||
if (!hfUrl) {
|
||||
errorDiv.textContent = 'Please enter a HuggingFace repository URL.';
|
||||
const rawUrl = urlInput.value.trim();
|
||||
if (!rawUrl) {
|
||||
errorDiv.textContent = translate(
|
||||
'modals.linkModelSource.urlRequired',
|
||||
{},
|
||||
'Please enter a model page URL.'
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
const hfPattern = /^https?:\/\/huggingface\.co\/([^/]+\/[^/]+)\/?$/;
|
||||
if (!hfPattern.test(hfUrl)) {
|
||||
errorDiv.textContent = 'Invalid URL format. Expected: https://huggingface.co/user/repo';
|
||||
const sourceInfo = parseModelSourceUrl(rawUrl);
|
||||
if (!sourceInfo) {
|
||||
errorDiv.textContent = translate(
|
||||
'modals.linkModelSource.invalidUrl',
|
||||
{},
|
||||
'Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.'
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -241,12 +251,14 @@ export const ModelContextMenuMixin = {
|
||||
modalManager.closeModal('linkHfModal');
|
||||
|
||||
try {
|
||||
state.loadingManager.showSimpleLoading('Linking to HuggingFace...');
|
||||
state.loadingManager.showSimpleLoading(
|
||||
translate('modals.linkModelSource.linking', {}, 'Linking model source...')
|
||||
);
|
||||
|
||||
const response = await fetch('/api/lm/set-hf-url', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify({ file_path: filePath, hf_url: hfUrl }),
|
||||
body: JSON.stringify({ file_path: filePath, source_url: sourceInfo.url }),
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
@@ -262,7 +274,7 @@ export const ModelContextMenuMixin = {
|
||||
throw new Error(data.error || 'Failed to link model');
|
||||
}
|
||||
} catch (error) {
|
||||
console.error('Error linking model to HuggingFace:', error);
|
||||
console.error('Error linking model source:', error);
|
||||
showToast('toast.contextMenu.linkHfFailed', { message: error.message }, 'error');
|
||||
} finally {
|
||||
state.loadingManager.hide();
|
||||
@@ -276,18 +288,68 @@ export const ModelContextMenuMixin = {
|
||||
|
||||
modalManager.showModal('linkHfModal');
|
||||
|
||||
this._renderSupportedSources();
|
||||
|
||||
setTimeout(() => urlInput.focus(), 50);
|
||||
},
|
||||
|
||||
// HF metadata enrichment (AI agent) methods
|
||||
/**
|
||||
* Refresh the supported-site hints from the server so the dialog reflects
|
||||
* whatever sources this backend build actually knows about. Falls back to
|
||||
* the static markup in the template when the request fails.
|
||||
*/
|
||||
async _renderSupportedSources() {
|
||||
const container = document.getElementById('hfSupportedSources');
|
||||
if (!container) return;
|
||||
|
||||
try {
|
||||
const response = await fetch('/api/lm/model-sources');
|
||||
if (!response.ok) return;
|
||||
const sources = await response.json();
|
||||
if (!Array.isArray(sources) || sources.length === 0) return;
|
||||
|
||||
const examples = sources
|
||||
.map((source) => source?.example_url)
|
||||
.filter((url) => typeof url === 'string' && url);
|
||||
if (examples.length === 0) return;
|
||||
|
||||
container.innerHTML = examples
|
||||
.map((url) => `<strong>${escapeHtml(url)}</strong>`)
|
||||
.join('<br>');
|
||||
} catch (error) {
|
||||
console.debug('Failed to load supported model sources:', error);
|
||||
}
|
||||
},
|
||||
|
||||
// Model metadata enrichment (AI agent) methods
|
||||
updateEnrichMenuItem(card) {
|
||||
const enrichItem = this.menu?.querySelector('[data-action="enrich-hf-llm"]');
|
||||
if (!enrichItem) return;
|
||||
const hasHfUrl = !!card.dataset.hf_url;
|
||||
enrichItem.classList.toggle('disabled', !hasHfUrl);
|
||||
enrichItem.title = hasHfUrl
|
||||
? ''
|
||||
: 'Link this model to a HuggingFace repo first (Link Model → Link to HuggingFace)';
|
||||
|
||||
const model = {
|
||||
source_url: card.dataset.source_url || '',
|
||||
source_platform: card.dataset.source_platform || '',
|
||||
hf_url: card.dataset.hf_url || '',
|
||||
};
|
||||
const sourceInfo = getModelSourceInfo(model);
|
||||
const canEnrich = Boolean(sourceInfo && sourceInfo.supportsEnrichment);
|
||||
|
||||
enrichItem.classList.toggle('disabled', !canEnrich);
|
||||
if (canEnrich) {
|
||||
enrichItem.title = '';
|
||||
} else if (!sourceInfo) {
|
||||
enrichItem.title = translate(
|
||||
'toast.contextMenu.enrichNeedsSource',
|
||||
{},
|
||||
'Link this model to a model source first (Link Model → Link to Model Source)'
|
||||
);
|
||||
} else {
|
||||
enrichItem.title = translate(
|
||||
'toast.contextMenu.enrichUnsupportedSource',
|
||||
{ source: sourceInfo.label },
|
||||
`AI enrichment is not available for ${sourceInfo.label} models`
|
||||
);
|
||||
}
|
||||
},
|
||||
|
||||
async enrichWithAgent(filePath) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { showToast, openCivitai, openHuggingFace, copyToClipboard, copyLoraSyntax, sendLoraToWorkflow, sendEmbeddingToWorkflow, openExampleImagesFolder, buildLoraSyntax, sendModelPathToWorkflow } from '../../utils/uiHelpers.js';
|
||||
import { getModelSourceInfo, getModelSourceGroupKey, getModelSourceViewTitle, openModelSource } from '../../utils/modelSourceHelpers.js';
|
||||
import { state, getCurrentPageState } from '../../state/index.js';
|
||||
import { showModelModal } from './ModelModal.js';
|
||||
import { hasCivitaiSource } from './utils.js';
|
||||
@@ -65,12 +66,15 @@ function handleModelCardEvent_internal(event, modelType) {
|
||||
if (event.target.closest('.fa-globe')) {
|
||||
event.stopPropagation();
|
||||
// CivitAI wins when the model actually has CivitAI data; otherwise fall
|
||||
// back to HuggingFace. Relying on `from_civitai` here made the two
|
||||
// sources mutually exclusive whenever one of them was (re)linked (#1094).
|
||||
// back to the linked external source. Relying on `from_civitai` here
|
||||
// made the two sources mutually exclusive whenever one of them was
|
||||
// (re)linked (#1094).
|
||||
if (card.dataset.has_civitai === 'true') {
|
||||
openCivitai(card.dataset.filepath);
|
||||
} else if (card.dataset.hf_url) {
|
||||
} else if (card.dataset.source_platform === 'huggingface' && card.dataset.hf_url) {
|
||||
openHuggingFace(card.dataset.hf_url);
|
||||
} else if (card.dataset.source_url) {
|
||||
openModelSource(card.dataset.source_url);
|
||||
}
|
||||
return true; // Stop propagation
|
||||
}
|
||||
@@ -337,6 +341,8 @@ async function showModelModalFromCard(card, modelType) {
|
||||
modified: card.dataset.modified,
|
||||
file_size: parseInt(card.dataset.file_size || '0'),
|
||||
from_civitai: card.dataset.from_civitai === 'true',
|
||||
source_platform: card.dataset.source_platform || '',
|
||||
source_url: card.dataset.source_url || '',
|
||||
hf_url: card.dataset.hf_url || '',
|
||||
base_model: card.dataset.base_model,
|
||||
notes: card.dataset.notes || '',
|
||||
@@ -428,6 +434,8 @@ function showExampleAccessModal(card, modelType) {
|
||||
modified: card.dataset.modified,
|
||||
file_size: card.dataset.file_size,
|
||||
from_civitai: card.dataset.from_civitai === 'true',
|
||||
source_platform: card.dataset.source_platform || '',
|
||||
source_url: card.dataset.source_url || '',
|
||||
hf_url: card.dataset.hf_url || '',
|
||||
base_model: card.dataset.base_model,
|
||||
notes: card.dataset.notes,
|
||||
@@ -490,7 +498,11 @@ export function createModelCard(model, modelType) {
|
||||
card.dataset.base_model = model.base_model || 'Unknown';
|
||||
card.dataset.favorite = model.favorite ? 'true' : 'false';
|
||||
card.dataset.exclude = model.exclude ? 'true' : 'false';
|
||||
card.dataset.hf_url = model.hf_url || '';
|
||||
const modelSourceInfo = getModelSourceInfo(model);
|
||||
card.dataset.source_url = modelSourceInfo?.url || '';
|
||||
card.dataset.source_platform = modelSourceInfo?.platform || '';
|
||||
// Legacy alias: only Hugging Face models expose `hf_url`.
|
||||
card.dataset.hf_url = modelSourceInfo?.platform === 'huggingface' ? modelSourceInfo.url : '';
|
||||
const hasUpdateAvailable = Boolean(model.update_available);
|
||||
card.dataset.update_available = hasUpdateAvailable ? 'true' : 'false';
|
||||
card.dataset.skip_metadata_refresh = model.skip_metadata_refresh ? 'true' : 'false';
|
||||
@@ -508,11 +520,12 @@ export function createModelCard(model, modelType) {
|
||||
const modelId = civitaiData?.modelId ?? civitaiData?.model_id;
|
||||
if (modelId !== undefined && modelId !== null && modelId !== '') {
|
||||
card.dataset.modelId = modelId;
|
||||
} else if (model.hf_url) {
|
||||
// For HF-only models, derive a group key from hf_url for version grouping
|
||||
const match = model.hf_url.match(/https?:\/\/huggingface\.co\/([^/]+\/[^/]+)/);
|
||||
if (match) {
|
||||
card.dataset.modelId = 'hf:' + match[1];
|
||||
} else {
|
||||
// For externally-sourced models, derive a group key from the source
|
||||
// URL for version grouping (hf:user/repo, ms:user/repo, ta:<id>).
|
||||
const sourceGroupKey = getModelSourceGroupKey(model);
|
||||
if (sourceGroupKey) {
|
||||
card.dataset.modelId = sourceGroupKey;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -610,10 +623,10 @@ export function createModelCard(model, modelType) {
|
||||
const hasCivitai = hasCivitaiSource(model.civitai);
|
||||
const globeTitle = hasCivitai ?
|
||||
translate('modelCard.actions.viewOnCivitai', {}, 'View on Civitai') :
|
||||
model.hf_url ?
|
||||
translate('modelCard.actions.viewOnHuggingFace', {}, 'View on Hugging Face') :
|
||||
modelSourceInfo ?
|
||||
getModelSourceViewTitle(modelSourceInfo) :
|
||||
translate('modelCard.actions.notAvailableFromCivitai', {}, 'Not available from Civitai');
|
||||
const globeEnabled = hasCivitai || !!model.hf_url;
|
||||
const globeEnabled = hasCivitai || !!modelSourceInfo;
|
||||
let sendTitle;
|
||||
let copyTitle;
|
||||
if (modelType === MODEL_TYPES.LORA) {
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { showToast, openCivitai, sendLoraToWorkflow, sendEmbeddingToWorkflow, sendModelPathToWorkflow, buildLoraSyntax, copyToClipboard } from '../../utils/uiHelpers.js';
|
||||
import { getModelSourceInfo, getModelSourceGroupKey, getModelSourceViewTitle, openModelSource } from '../../utils/modelSourceHelpers.js';
|
||||
import { modalManager } from '../../managers/ModalManager.js';
|
||||
import { MODEL_TYPES } from '../../api/apiConfig.js';
|
||||
import {
|
||||
@@ -397,10 +398,13 @@ export async function showModelModal(model, modelType) {
|
||||
<div class="civitai-view" title="${translate('modals.model.actions.viewOnCivitai', {}, 'View on Civitai')}" data-action="view-civitai" data-filepath="${escapedFilePathAttr}">
|
||||
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnCivitaiText', {}, 'View on Civitai')}
|
||||
</div>`.trim() : '';
|
||||
const escapedHfUrl = modelWithFullData.hf_url ? escapeAttribute(modelWithFullData.hf_url) : '';
|
||||
const viewOnHuggingFaceAction = escapedHfUrl ? `
|
||||
<div class="civitai-view" title="${translate('modals.model.actions.viewOnHuggingFace', {}, 'View on Hugging Face')}" data-action="view-huggingface" data-hf-url="${escapedHfUrl}">
|
||||
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnHuggingFaceText', {}, 'View on Hugging Face')}
|
||||
const sourceInfo = getModelSourceInfo(modelWithFullData);
|
||||
const escapedSourceUrl = sourceInfo?.url ? escapeAttribute(sourceInfo.url) : '';
|
||||
const isHuggingFaceSource = sourceInfo?.platform === 'huggingface';
|
||||
const sourceTitle = sourceInfo ? getModelSourceViewTitle(sourceInfo) : '';
|
||||
const viewOnHuggingFaceAction = escapedSourceUrl ? `
|
||||
<div class="civitai-view" title="${escapeAttribute(sourceTitle)}" data-action="${isHuggingFaceSource ? 'view-huggingface' : 'view-model-source'}" ${isHuggingFaceSource ? 'data-hf-url' : 'data-source-url'}="${escapedSourceUrl}">
|
||||
<i class="fas fa-globe"></i> ${escapeHtml(sourceTitle)}
|
||||
</div>`.trim() : '';
|
||||
const creatorInfoAction = modelWithFullData.civitai?.creator ? `
|
||||
<div class="creator-info" data-username="${modelWithFullData.civitai.creator.username}" data-action="view-creator" title="${translate('modals.model.actions.viewCreatorProfile', {}, 'View Creator Profile')}">
|
||||
@@ -520,12 +524,12 @@ export async function showModelModal(model, modelType) {
|
||||
const loadingExamplesText = translate('modals.model.loading.examples', {}, 'Loading examples...');
|
||||
|
||||
const loadingVersionsText = translate('modals.model.loading.versions', {}, 'Loading versions...');
|
||||
// Use CivitAI modelId, or derive HF group key for HF-only models
|
||||
// Use CivitAI modelId, or derive a source group key for externally-linked models
|
||||
let civitaiModelId = modelWithFullData.civitai?.modelId || '';
|
||||
if (!civitaiModelId && modelWithFullData.hf_url) {
|
||||
const match = modelWithFullData.hf_url.match(/https?:\/\/huggingface\.co\/([^/]+\/[^/]+)/);
|
||||
if (match) {
|
||||
civitaiModelId = 'hf:' + match[1];
|
||||
if (!civitaiModelId) {
|
||||
const sourceGroupKey = getModelSourceGroupKey(modelWithFullData);
|
||||
if (sourceGroupKey) {
|
||||
civitaiModelId = sourceGroupKey;
|
||||
}
|
||||
}
|
||||
const civitaiVersionId = modelWithFullData.civitai?.id || '';
|
||||
@@ -939,6 +943,11 @@ function setupEventHandlers(filePath, modelType) {
|
||||
window.open(target.dataset.hfUrl, '_blank', 'noopener,noreferrer');
|
||||
}
|
||||
break;
|
||||
case 'view-model-source':
|
||||
if (target.dataset.sourceUrl) {
|
||||
openModelSource(target.dataset.sourceUrl);
|
||||
}
|
||||
break;
|
||||
case 'view-creator':
|
||||
const username = target.dataset.username;
|
||||
if (username) {
|
||||
|
||||
@@ -5,6 +5,7 @@ import { openCivitaiUrl, showToast } from '../../utils/uiHelpers.js';
|
||||
import { translate } from '../../utils/i18nHelpers.js';
|
||||
import { state } from '../../state/index.js';
|
||||
import { buildCivitaiModelUrl } from '../../utils/civitaiUtils.js';
|
||||
import { parseModelSourceGroupKey } from '../../utils/modelSourceHelpers.js';
|
||||
import { formatFileSize } from './utils.js';
|
||||
import { setSessionItem, removeSessionItem } from '../../utils/storageHelpers.js';
|
||||
|
||||
@@ -993,22 +994,23 @@ export function initVersionsTab({
|
||||
renderErrorState(container, translate('modals.model.versions.missingModelId', {}, 'This model is missing a Civitai model id.'));
|
||||
return;
|
||||
}
|
||||
// HF group keys (e.g. "hf:user/repo") are not real CivitAI model IDs —
|
||||
// skip the remote API call and show a helpful message instead.
|
||||
const isHfGroupKey = typeof modelId === 'string' && modelId.startsWith('hf:');
|
||||
if (isHfGroupKey) {
|
||||
// External source group keys (e.g. "hf:user/repo", "ms:user/repo",
|
||||
// "ta:8278...") are not real CivitAI model IDs — skip the remote API
|
||||
// call and show a helpful message instead.
|
||||
const sourceGroup = parseModelSourceGroupKey(modelId);
|
||||
if (sourceGroup) {
|
||||
controller.isLoading = false;
|
||||
controller.hasLoaded = true;
|
||||
controller.record = null;
|
||||
const hfMsg = translate(
|
||||
'modals.model.versions.hfGroupInfo',
|
||||
{},
|
||||
'This is a HuggingFace model group. Open the library to see all versions in the grid.'
|
||||
const sourceMsg = translate(
|
||||
'modals.model.versions.sourceGroupInfo',
|
||||
{ source: sourceGroup.label },
|
||||
`This is a ${sourceGroup.label} model group. Open the library to see all versions in the grid.`
|
||||
);
|
||||
container.innerHTML = `
|
||||
<div class="versions-empty-state">
|
||||
<i class="fas fa-info-circle"></i>
|
||||
<p>${escapeHtml(hfMsg)}</p>
|
||||
<p>${escapeHtml(sourceMsg)}</p>
|
||||
</div>
|
||||
`;
|
||||
return;
|
||||
|
||||
@@ -13,6 +13,13 @@ import { buildCivitaiUrl, extractCivitaiModelUrlParts, normalizeCivitaiPageHost
|
||||
import { formatFileSize } from '../utils/formatters.js';
|
||||
import { showDownloadBatchSummary } from '../components/DownloadBatchSummaryModal.js';
|
||||
import { openOtherModelsSettings } from '../utils/otherModels.js';
|
||||
import {
|
||||
buildModelSourceFilePage,
|
||||
detectModelSourceDownloadUrl,
|
||||
getModelSource,
|
||||
isExternalModelSource,
|
||||
isValidRepoId,
|
||||
} from '../utils/modelSourceHelpers.js';
|
||||
|
||||
export class DownloadManager {
|
||||
constructor() {
|
||||
@@ -39,10 +46,11 @@ export class DownloadManager {
|
||||
this.isBatchMode = false;
|
||||
this.editingBatchIndex = -1;
|
||||
|
||||
// HF download state
|
||||
this.hfRepoId = null;
|
||||
this.hfSelectedFiles = [];
|
||||
this.hfRepoCollapsed = {};
|
||||
// External repository download state (Hugging Face / ModelScope)
|
||||
this.sourcePlatform = 'huggingface';
|
||||
this.sourceRepoId = null;
|
||||
this.sourceSelectedFiles = [];
|
||||
this.sourceRepoCollapsed = {};
|
||||
|
||||
this.loadingManager = new LoadingManager();
|
||||
this.folderTreeManager = new FolderTreeManager();
|
||||
@@ -186,10 +194,11 @@ export class DownloadManager {
|
||||
// Reset default path toggle
|
||||
this.loadDefaultPathSetting();
|
||||
|
||||
// Reset HF state
|
||||
this.hfRepoId = null;
|
||||
this.hfSelectedFiles = [];
|
||||
this.hfRepoCollapsed = {};
|
||||
// Reset external repository state
|
||||
this.sourcePlatform = 'huggingface';
|
||||
this.sourceRepoId = null;
|
||||
this.sourceSelectedFiles = [];
|
||||
this.sourceRepoCollapsed = {};
|
||||
}
|
||||
|
||||
async retrieveVersionsForModel(modelId, source = null) {
|
||||
@@ -212,10 +221,12 @@ export class DownloadManager {
|
||||
|
||||
// Detect URL types — all URLs must share the same source type
|
||||
const urlTypes = urls.map(u => DownloadManager.detectUrlType(u));
|
||||
const isHf = urlTypes.every(t => t && (t.type === 'hf-resolve' || t.type === 'hf-repo'));
|
||||
const isExternalSource = urlTypes.every(
|
||||
t => t && (t.type === 'model-source-repo' || t.type === 'model-source-file')
|
||||
);
|
||||
const isCivitai = urlTypes.every(t => t && t.type === 'civitai');
|
||||
|
||||
if (!isHf && !isCivitai) {
|
||||
if (!isExternalSource && !isCivitai) {
|
||||
const allValid = urlTypes.every(t => t !== null);
|
||||
if (!allValid) {
|
||||
errorElement.textContent = translate('modals.download.errors.invalidUrl');
|
||||
@@ -228,8 +239,8 @@ export class DownloadManager {
|
||||
}
|
||||
}
|
||||
|
||||
if (isHf) {
|
||||
return this._validateAndFetchHf(urls, errorElement);
|
||||
if (isExternalSource) {
|
||||
return this._validateAndFetchExternalRepo(urls, errorElement);
|
||||
}
|
||||
|
||||
// --- Original CivitAI flow below ---
|
||||
@@ -327,45 +338,66 @@ export class DownloadManager {
|
||||
this.showBatchPreviewStep();
|
||||
}
|
||||
|
||||
// ---- Hugging Face download flow ----
|
||||
// ---- External repository download flow (Hugging Face / ModelScope) ----
|
||||
|
||||
async _validateAndFetchHf(urls, errorElement) {
|
||||
/** Rendering group key: the same repo on two sites is two groups. */
|
||||
_externalGroupKey(item) {
|
||||
return `${item.source}:${item.repo || 'unknown'}`;
|
||||
}
|
||||
|
||||
_defaultRevisionFor(platform) {
|
||||
const source = getModelSource(platform);
|
||||
return (source && source.defaultRevision) || '';
|
||||
}
|
||||
|
||||
_makeExternalItem(url, info, file) {
|
||||
return {
|
||||
url,
|
||||
source: info.platform,
|
||||
platform: info.platform,
|
||||
repo: info.repo,
|
||||
revision: file.revision || this._defaultRevisionFor(info.platform),
|
||||
filename: file.filename,
|
||||
displayName: file.filename,
|
||||
fileSizeBytes: file.size,
|
||||
selectedVersion: true,
|
||||
versions: [],
|
||||
checked: false,
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
|
||||
/** Fetch a repository's weight files as flat batch items. */
|
||||
async _fetchExternalRepoItems(url, info) {
|
||||
const revision = this._defaultRevisionFor(info.platform);
|
||||
const files = await this.apiClient.fetchModelSourceFiles(
|
||||
info.repo, info.platform, revision
|
||||
);
|
||||
if (!files || files.length === 0) {
|
||||
throw new Error(translate('modals.download.errors.noModelFiles'));
|
||||
}
|
||||
return files.map(file => this._makeExternalItem(url, info, { ...file, revision }));
|
||||
}
|
||||
|
||||
async _validateAndFetchExternalRepo(urls, errorElement) {
|
||||
if (urls.length === 1) {
|
||||
const info = DownloadManager.detectUrlType(urls[0]);
|
||||
// Direct file resolve URL → skip file selection, go to location
|
||||
if (info.type === 'hf-resolve') {
|
||||
// Direct file URL → skip file selection, go to location
|
||||
if (info.type === 'model-source-file') {
|
||||
this.isBatchMode = false;
|
||||
this.hfRepoId = info.repo;
|
||||
this.hfSelectedFiles = [info.filename];
|
||||
this.source = 'huggingface';
|
||||
this.sourcePlatform = info.platform;
|
||||
this.sourceRepoId = info.repo;
|
||||
this.sourceSelectedFiles = [info.filename];
|
||||
this.source = info.platform;
|
||||
this.proceedToLocation();
|
||||
return;
|
||||
}
|
||||
// Repo URL → fetch file list and convert to batch items
|
||||
try {
|
||||
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingRepoFiles'));
|
||||
const files = await this.apiClient.fetchHfRepoFiles(info.repo);
|
||||
if (!files || files.length === 0) {
|
||||
throw new Error(translate('modals.download.errors.noModelFiles'));
|
||||
}
|
||||
this.isBatchMode = true;
|
||||
this.batchModels = [];
|
||||
this.source = 'huggingface';
|
||||
for (const file of files) {
|
||||
this.batchModels.push({
|
||||
url: urls[0],
|
||||
source: 'huggingface',
|
||||
repo: info.repo,
|
||||
filename: file.filename,
|
||||
revision: 'main',
|
||||
displayName: file.filename,
|
||||
fileSizeBytes: file.size,
|
||||
selectedVersion: true,
|
||||
versions: [],
|
||||
checked: false,
|
||||
error: null,
|
||||
});
|
||||
}
|
||||
this.batchModels = await this._fetchExternalRepoItems(urls[0], info);
|
||||
this.source = info.platform;
|
||||
this.showBatchPreviewStep();
|
||||
} catch (err) {
|
||||
errorElement.textContent = err.message;
|
||||
@@ -375,10 +407,9 @@ export class DownloadManager {
|
||||
return;
|
||||
}
|
||||
|
||||
// Multiple HF URLs → batch mode: flatten all files from all repos
|
||||
// Multiple URLs → batch mode: flatten all files from all repos
|
||||
this.isBatchMode = true;
|
||||
this.batchModels = [];
|
||||
this.source = 'huggingface';
|
||||
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingRepoFiles'));
|
||||
|
||||
for (const url of urls) {
|
||||
@@ -387,42 +418,15 @@ export class DownloadManager {
|
||||
this.batchModels.push({ url, error: 'Invalid URL', versions: [], selectedVersion: null });
|
||||
continue;
|
||||
}
|
||||
if (info.type === 'hf-resolve') {
|
||||
this.batchModels.push({
|
||||
url,
|
||||
source: 'huggingface',
|
||||
repo: info.repo,
|
||||
this.source = info.platform;
|
||||
if (info.type === 'model-source-file') {
|
||||
this.batchModels.push(this._makeExternalItem(url, info, {
|
||||
filename: info.filename,
|
||||
revision: info.revision || 'main',
|
||||
displayName: info.filename,
|
||||
selectedVersion: true,
|
||||
versions: [],
|
||||
checked: false,
|
||||
error: null,
|
||||
});
|
||||
} else if (info.type === 'hf-repo') {
|
||||
revision: info.revision,
|
||||
}));
|
||||
} else if (info.type === 'model-source-repo') {
|
||||
try {
|
||||
const files = await this.apiClient.fetchHfRepoFiles(info.repo);
|
||||
if (!files || files.length === 0) {
|
||||
this.batchModels.push({ url, error: 'No model files found', versions: [], selectedVersion: null });
|
||||
continue;
|
||||
}
|
||||
// Flatten: create one batch item per file, all checked by default
|
||||
for (const file of files) {
|
||||
this.batchModels.push({
|
||||
url,
|
||||
source: 'huggingface',
|
||||
repo: info.repo,
|
||||
filename: file.filename,
|
||||
revision: 'main',
|
||||
displayName: file.filename,
|
||||
fileSizeBytes: file.size,
|
||||
selectedVersion: true,
|
||||
versions: [],
|
||||
checked: false,
|
||||
error: null,
|
||||
});
|
||||
}
|
||||
this.batchModels.push(...await this._fetchExternalRepoItems(url, info));
|
||||
} catch (err) {
|
||||
this.batchModels.push({ url, error: err.message, versions: [], selectedVersion: null });
|
||||
}
|
||||
@@ -480,7 +484,8 @@ export class DownloadManager {
|
||||
* Detect the source type of a download URL.
|
||||
* @param {string} url
|
||||
* @returns {{ type: string, repo?: string, filename?: string, revision?: string } | null}
|
||||
* type: 'civitai' | 'civarchive' | 'hf-resolve' | 'hf-repo' | 'direct-http'
|
||||
* type: 'civitai' | 'civarchive' | 'model-source-file' | 'model-source-repo'
|
||||
* | 'direct-http'
|
||||
*/
|
||||
static detectUrlType(url) {
|
||||
const trimmed = url.trim();
|
||||
@@ -492,38 +497,27 @@ export class DownloadManager {
|
||||
return { type: 'civitai' };
|
||||
}
|
||||
|
||||
// Hugging Face resolve/blob URL → direct file
|
||||
// "blob" is the web preview page; it maps 1:1 to the "resolve" download URL
|
||||
const hfResolveMatch = trimmed.match(/huggingface\.co\/([^/\s]+\/[^/\s]+)\/(?:resolve|blob)\/([^/\s]+)\/(.+)/i);
|
||||
if (hfResolveMatch) {
|
||||
return {
|
||||
type: 'hf-resolve',
|
||||
repo: hfResolveMatch[1],
|
||||
revision: hfResolveMatch[2],
|
||||
filename: hfResolveMatch[3],
|
||||
};
|
||||
}
|
||||
|
||||
// Hugging Face repo URL (huggingface.co/user/repo or bare user/repo path)
|
||||
// Require huggingface.co prefix for full URLs; bare user/repo only without ://
|
||||
const hfRepoMatch = trimmed.match(
|
||||
trimmed.includes('://')
|
||||
? /^https?:\/\/huggingface\.co\/([a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+)(?:\/?$|$)/
|
||||
: /^([a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+)$/
|
||||
);
|
||||
if (hfRepoMatch) {
|
||||
// External model sources (Hugging Face / ModelScope). Repository URLs
|
||||
// list every weight file; resolve URLs point at one file. Both are
|
||||
// recognised through the shared registry, so adding a site is a
|
||||
// registry change rather than a change here.
|
||||
const sourceInfo = detectModelSourceDownloadUrl(trimmed);
|
||||
if (sourceInfo) {
|
||||
// Reject path-traversal patterns like "../.." or "user/.."
|
||||
const parts = hfRepoMatch[1].split('/');
|
||||
if (parts.some(p => p === '.' || p === '..')) {
|
||||
if (!isValidRepoId(sourceInfo.repo)) {
|
||||
return null;
|
||||
}
|
||||
return {
|
||||
type: 'hf-repo',
|
||||
repo: hfRepoMatch[1],
|
||||
type: sourceInfo.kind === 'file' ? 'model-source-file' : 'model-source-repo',
|
||||
platform: sourceInfo.platform,
|
||||
repo: sourceInfo.repo,
|
||||
...(sourceInfo.kind === 'file'
|
||||
? { revision: sourceInfo.revision, filename: sourceInfo.filename }
|
||||
: {}),
|
||||
};
|
||||
}
|
||||
|
||||
// Direct HTTP(S) URL (non-HF)
|
||||
// Direct HTTP(S) URL (non model-source)
|
||||
if (/^https?:\/\//i.test(trimmed)) {
|
||||
return { type: 'direct-http' };
|
||||
}
|
||||
@@ -931,7 +925,7 @@ export class DownloadManager {
|
||||
}
|
||||
|
||||
// In single-URL mode, validate version selection (skip for HF)
|
||||
if (!this.isBatchMode && this.source !== 'huggingface') {
|
||||
if (!this.isBatchMode && !isExternalModelSource(this.source)) {
|
||||
if (!this.currentVersion) {
|
||||
showToast('toast.loras.pleaseSelectVersion', {}, 'error');
|
||||
return;
|
||||
@@ -1164,12 +1158,15 @@ export class DownloadManager {
|
||||
/**
|
||||
* Synthesize a clickable URL for a single-download failure entry.
|
||||
* Single downloads have no pasted URL, so the modal link is derived from
|
||||
* the model/version ids (CivitAI) or the HF repo/file (HuggingFace).
|
||||
* the model/version ids (CivitAI) or the external repo/file.
|
||||
*/
|
||||
_buildSingleItemUrl({ modelId, versionId, source, repo = null, filename = null }) {
|
||||
if (source === 'huggingface' && repo) {
|
||||
const base = `https://huggingface.co/${encodeURI(repo)}`;
|
||||
return filename ? `${base}/blob/${encodeURI('main')}/${encodeURI(filename)}` : base;
|
||||
if (isExternalModelSource(source) && repo) {
|
||||
return buildModelSourceFilePage({
|
||||
platform: source,
|
||||
repo,
|
||||
filename,
|
||||
}) || getModelSource(source).canonical(repo);
|
||||
}
|
||||
if (modelId) {
|
||||
return buildCivitaiUrl({
|
||||
@@ -1490,8 +1487,8 @@ export class DownloadManager {
|
||||
* matched card-by-card via `_reconcileViewAfterDownload`; HF
|
||||
* downloads (no CivitAI identity to match) keep the legacy reload.
|
||||
*/
|
||||
async _reconcileBatchViewAfterDownload(completedCivitaiItems = [], hfCompletedCount = 0) {
|
||||
if (hfCompletedCount > 0) {
|
||||
async _reconcileBatchViewAfterDownload(completedCivitaiItems = [], externalCompletedCount = 0) {
|
||||
if (externalCompletedCount > 0) {
|
||||
await resetAndReload(true);
|
||||
return;
|
||||
}
|
||||
@@ -1661,10 +1658,11 @@ export class DownloadManager {
|
||||
return failedItems.length === 0;
|
||||
}
|
||||
|
||||
async _downloadHfSingle({ modelRoot, targetFolder, useDefaultPaths, files = null }) {
|
||||
async _downloadExternalRepoFiles({ modelRoot, targetFolder, useDefaultPaths, files = null }) {
|
||||
modalManager.closeModal('downloadModal');
|
||||
this.loadingManager.restoreProgressBar();
|
||||
const filesToDownload = files || this.hfSelectedFiles;
|
||||
const platform = this.sourcePlatform;
|
||||
const filesToDownload = files || this.sourceSelectedFiles;
|
||||
const totalFiles = filesToDownload.length;
|
||||
const updateProgress = this.loadingManager.showDownloadProgress(totalFiles);
|
||||
|
||||
@@ -1720,10 +1718,11 @@ export class DownloadManager {
|
||||
}
|
||||
};
|
||||
|
||||
const response = await this.apiClient.downloadHfModel({
|
||||
repo: this.hfRepoId,
|
||||
const response = await this.apiClient.downloadModelSource({
|
||||
platform,
|
||||
repo: this.sourceRepoId,
|
||||
filename,
|
||||
revision: 'main',
|
||||
revision: this._defaultRevisionFor(platform),
|
||||
modelRoot,
|
||||
relativePath: targetFolder,
|
||||
useDefaultPaths,
|
||||
@@ -1738,10 +1737,10 @@ export class DownloadManager {
|
||||
} else {
|
||||
failedFiles.push({
|
||||
item: {
|
||||
source: 'huggingface',
|
||||
repo: this.hfRepoId,
|
||||
source: platform,
|
||||
repo: this.sourceRepoId,
|
||||
filename,
|
||||
url: this._buildSingleItemUrl({ source: 'huggingface', repo: this.hfRepoId, filename }),
|
||||
url: this._buildSingleItemUrl({ source: platform, repo: this.sourceRepoId, filename }),
|
||||
},
|
||||
error: response?.error || 'Unknown error',
|
||||
name: filename,
|
||||
@@ -1749,13 +1748,13 @@ export class DownloadManager {
|
||||
}
|
||||
} catch (err) {
|
||||
if (!cancelled) {
|
||||
console.error(`Failed to download HF file ${filename}:`, err);
|
||||
console.error(`Failed to download repo file ${filename}:`, err);
|
||||
failedFiles.push({
|
||||
item: {
|
||||
source: 'huggingface',
|
||||
repo: this.hfRepoId,
|
||||
source: platform,
|
||||
repo: this.sourceRepoId,
|
||||
filename,
|
||||
url: this._buildSingleItemUrl({ source: 'huggingface', repo: this.hfRepoId, filename }),
|
||||
url: this._buildSingleItemUrl({ source: platform, repo: this.sourceRepoId, filename }),
|
||||
},
|
||||
error: err?.message || 'Unknown error',
|
||||
name: filename,
|
||||
@@ -1781,7 +1780,7 @@ export class DownloadManager {
|
||||
total: totalFiles,
|
||||
completed: completedDownloads,
|
||||
failedItems: failedFiles,
|
||||
onRetry: () => this._downloadHfSingle({
|
||||
onRetry: () => this._downloadExternalRepoFiles({
|
||||
modelRoot,
|
||||
targetFolder,
|
||||
useDefaultPaths,
|
||||
@@ -1831,7 +1830,7 @@ export class DownloadManager {
|
||||
|
||||
const validCount = this.batchModels.filter(m => {
|
||||
if (m.error) return false;
|
||||
if (m.source === 'huggingface') return m.checked !== false;
|
||||
if (isExternalModelSource(m.source)) return m.checked !== false;
|
||||
return m.selectedVersion;
|
||||
}).length;
|
||||
document.getElementById('downloadModalTitle').textContent =
|
||||
@@ -1839,7 +1838,9 @@ export class DownloadManager {
|
||||
` (${validCount})`;
|
||||
|
||||
const list = document.getElementById('batchPreviewList');
|
||||
const hasHfItems = this.batchModels.some(m => m.source === 'huggingface' && !m.error);
|
||||
const hasExternalItems = this.batchModels.some(
|
||||
m => isExternalModelSource(m.source) && !m.error
|
||||
);
|
||||
|
||||
// Error items render flat, outside any group
|
||||
const errorItemsHtml = this.batchModels.map((item, index) => {
|
||||
@@ -1863,7 +1864,7 @@ export class DownloadManager {
|
||||
// CivitAI items render flat, outside any group (unchanged)
|
||||
const civitaiItemsHtml = this.batchModels.map((item, index) => {
|
||||
if (item.error) return null;
|
||||
if (item.source === 'huggingface') return null;
|
||||
if (isExternalModelSource(item.source)) return null;
|
||||
const ver = item.selectedVersion;
|
||||
const firstImage = ver?.images?.find(img => !img.url.endsWith('.mp4'));
|
||||
const thumbnailUrl = firstImage ? firstImage.url : '/loras_static/images/no-preview.png';
|
||||
@@ -1901,25 +1902,30 @@ export class DownloadManager {
|
||||
`;
|
||||
}).filter(Boolean).join('');
|
||||
|
||||
// Group HF items by repo (data model stays flat — only rendering groups)
|
||||
const hfGroups = {};
|
||||
// Group external-repository items by platform + repo so that the same
|
||||
// `owner/name` on two sites stays in two groups (data model stays flat
|
||||
// — only rendering groups).
|
||||
const externalGroups = {};
|
||||
this.batchModels.forEach((item, index) => {
|
||||
if (item.error || item.source !== 'huggingface') return;
|
||||
const repo = item.repo || 'unknown';
|
||||
if (!hfGroups[repo]) hfGroups[repo] = [];
|
||||
hfGroups[repo].push({ item, index });
|
||||
if (item.error || !isExternalModelSource(item.source)) return;
|
||||
const groupKey = this._externalGroupKey(item);
|
||||
if (!externalGroups[groupKey]) {
|
||||
externalGroups[groupKey] = { repo: item.repo || 'unknown', items: [] };
|
||||
}
|
||||
externalGroups[groupKey].items.push({ item, index });
|
||||
});
|
||||
|
||||
const renderHfItem = ({ item, index }) => {
|
||||
const hfSize = item.fileSizeBytes ? formatFileSize(item.fileSizeBytes) : '?';
|
||||
const renderExternalItem = ({ item, index }) => {
|
||||
const fileSize = item.fileSizeBytes ? formatFileSize(item.fileSizeBytes) : '?';
|
||||
const badge = getModelSource(item.source)?.label || item.source;
|
||||
return `
|
||||
<div class="batch-preview-item" data-index="${index}">
|
||||
<input type="checkbox" class="batch-preview-checkbox"
|
||||
data-index="${index}" ${item.checked !== false ? 'checked' : ''} />
|
||||
<div class="batch-preview-info">
|
||||
<div class="batch-preview-name">${item.displayName || item.filename || `HF #${index}`} <span class="hf-badge">HF</span></div>
|
||||
<div class="batch-preview-name">${item.displayName || item.filename || `${badge} #${index}`} <span class="hf-badge">${badge}</span></div>
|
||||
<div class="batch-preview-meta">
|
||||
<span>${hfSize}</span>
|
||||
<span>${fileSize}</span>
|
||||
<span>${item.repo || ''}</span>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1930,32 +1936,32 @@ export class DownloadManager {
|
||||
`;
|
||||
};
|
||||
|
||||
const hfGroupsHtml = Object.keys(hfGroups).map(repo => {
|
||||
const items = hfGroups[repo];
|
||||
const isCollapsed = this.hfRepoCollapsed[repo] === true;
|
||||
const externalGroupsHtml = Object.keys(externalGroups).map(groupKey => {
|
||||
const { repo, items } = externalGroups[groupKey];
|
||||
const isCollapsed = this.sourceRepoCollapsed[groupKey] === true;
|
||||
const allChecked = items.every(({ item }) => item.checked !== false);
|
||||
const fileCount = items.length;
|
||||
return `
|
||||
<div class="batch-preview-group" data-repo="${repo}">
|
||||
<div class="batch-preview-group" data-repo="${groupKey}">
|
||||
<div class="batch-preview-group-header">
|
||||
<i class="fas fa-chevron-right batch-preview-group-toggle ${isCollapsed ? '' : 'expanded'}"></i>
|
||||
<span class="batch-preview-group-name">${repo}</span>
|
||||
<span class="batch-preview-group-count">${fileCount} ${translate('modals.download.fileSelection.files', {}, 'files')}</span>
|
||||
<input type="checkbox" class="batch-preview-group-select-all" data-repo="${repo}" ${allChecked ? 'checked' : ''} />
|
||||
<input type="checkbox" class="batch-preview-group-select-all" data-repo="${groupKey}" ${allChecked ? 'checked' : ''} />
|
||||
</div>
|
||||
<div class="batch-preview-group-body ${isCollapsed ? '' : 'expanded'}">
|
||||
${items.map(renderHfItem).join('')}
|
||||
${items.map(renderExternalItem).join('')}
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
|
||||
let itemsHtml = errorItemsHtml + civitaiItemsHtml + hfGroupsHtml;
|
||||
let itemsHtml = errorItemsHtml + civitaiItemsHtml + externalGroupsHtml;
|
||||
|
||||
// Prepend select-all toolbar if there are HF items with checkboxes
|
||||
if (hasHfItems) {
|
||||
// Prepend select-all toolbar if there are external items with checkboxes
|
||||
if (hasExternalItems) {
|
||||
const allChecked = this.batchModels
|
||||
.filter(m => m.source === 'huggingface' && !m.error)
|
||||
.filter(m => isExternalModelSource(m.source) && !m.error)
|
||||
.every(m => m.checked !== false);
|
||||
itemsHtml = `
|
||||
<div class="batch-preview-select-all">
|
||||
@@ -1980,13 +1986,18 @@ export class DownloadManager {
|
||||
// Global select-all
|
||||
const selectAll = document.getElementById('batchSelectAll');
|
||||
if (selectAll) {
|
||||
const hfItems = this.batchModels.filter(m => m.source === 'huggingface' && !m.error);
|
||||
selectAll.checked = hfItems.length > 0 && hfItems.every(m => m.checked !== false);
|
||||
const externalItems = this.batchModels.filter(
|
||||
m => isExternalModelSource(m.source) && !m.error
|
||||
);
|
||||
selectAll.checked = externalItems.length > 0
|
||||
&& externalItems.every(m => m.checked !== false);
|
||||
}
|
||||
// Per-group select-all
|
||||
list.querySelectorAll('.batch-preview-group-select-all').forEach(gsa => {
|
||||
const repo = gsa.dataset.repo;
|
||||
const repoItems = this.batchModels.filter(m => m.source === 'huggingface' && !m.error && m.repo === repo);
|
||||
const repoItems = this.batchModels.filter(
|
||||
m => isExternalModelSource(m.source) && !m.error && this._externalGroupKey(m) === repo
|
||||
);
|
||||
gsa.checked = repoItems.length > 0 && repoItems.every(m => m.checked !== false);
|
||||
});
|
||||
};
|
||||
@@ -1998,7 +2009,7 @@ export class DownloadManager {
|
||||
const repo = groupSelectAll.dataset.repo;
|
||||
const checked = groupSelectAll.checked;
|
||||
this.batchModels.forEach((m, idx) => {
|
||||
if (m.source === 'huggingface' && !m.error && m.repo === repo) {
|
||||
if (isExternalModelSource(m.source) && !m.error && this._externalGroupKey(m) === repo) {
|
||||
m.checked = checked;
|
||||
const cb = list.querySelector(`.batch-preview-checkbox[data-index="${idx}"]`);
|
||||
if (cb) cb.checked = checked;
|
||||
@@ -2014,9 +2025,9 @@ export class DownloadManager {
|
||||
const repo = group.dataset.repo;
|
||||
const body = group.querySelector('.batch-preview-group-body');
|
||||
const toggle = group.querySelector('.batch-preview-group-toggle');
|
||||
const isCollapsed = this.hfRepoCollapsed[repo];
|
||||
const isCollapsed = this.sourceRepoCollapsed[repo];
|
||||
if (isCollapsed) {
|
||||
this.hfRepoCollapsed[repo] = false;
|
||||
this.sourceRepoCollapsed[repo] = false;
|
||||
body.style.transition = ''; // restore in case collapse was interrupted
|
||||
body.classList.add('expanded');
|
||||
toggle.classList.add('expanded');
|
||||
@@ -2025,13 +2036,13 @@ export class DownloadManager {
|
||||
body.style.maxHeight = body.scrollHeight + 'px';
|
||||
const onEnd = (e) => {
|
||||
if (e.propertyName !== 'max-height') return;
|
||||
if (this.hfRepoCollapsed[repo] !== false) return;
|
||||
if (this.sourceRepoCollapsed[repo] !== false) return;
|
||||
body.style.maxHeight = ''; // fall back to .expanded's 9999px
|
||||
body.removeEventListener('transitionend', onEnd);
|
||||
};
|
||||
body.addEventListener('transitionend', onEnd);
|
||||
} else {
|
||||
this.hfRepoCollapsed[repo] = true;
|
||||
this.sourceRepoCollapsed[repo] = true;
|
||||
body.style.maxHeight = body.scrollHeight + 'px';
|
||||
requestAnimationFrame(() => {
|
||||
// animate only max-height; keep expanded so opacity stays 1
|
||||
@@ -2040,7 +2051,7 @@ export class DownloadManager {
|
||||
toggle.classList.remove('expanded');
|
||||
const onEnd = (e) => {
|
||||
if (e.propertyName !== 'max-height') return;
|
||||
if (this.hfRepoCollapsed[repo] !== true) return; // state changed since
|
||||
if (this.sourceRepoCollapsed[repo] !== true) return; // state changed since
|
||||
body.classList.remove('expanded');
|
||||
body.style.transition = '';
|
||||
body.removeEventListener('transitionend', onEnd);
|
||||
@@ -2119,7 +2130,7 @@ export class DownloadManager {
|
||||
// For HF items, respect the checked flag; for CivitAI items, use selectedVersion
|
||||
const validModels = this.batchModels.filter(m => {
|
||||
if (m.error) return false;
|
||||
if (m.source === 'huggingface') return m.checked !== false;
|
||||
if (isExternalModelSource(m.source)) return m.checked !== false;
|
||||
return m.selectedVersion;
|
||||
});
|
||||
if (validModels.length === 0) return;
|
||||
@@ -2172,8 +2183,8 @@ export class DownloadManager {
|
||||
}
|
||||
if (!this.isBatchMode) {
|
||||
// Single-item download
|
||||
if (this.source === 'huggingface') {
|
||||
return this._downloadHfSingle({
|
||||
if (isExternalModelSource(this.source)) {
|
||||
return this._downloadExternalRepoFiles({
|
||||
modelRoot,
|
||||
targetFolder,
|
||||
useDefaultPaths,
|
||||
@@ -2228,7 +2239,7 @@ export class DownloadManager {
|
||||
if (m.error) return false;
|
||||
if (!m.selectedVersion) return false;
|
||||
// HF items have selectedVersion as a boolean marker + checked flag
|
||||
if (m.source === 'huggingface') return m.checked !== false;
|
||||
if (isExternalModelSource(m.source)) return m.checked !== false;
|
||||
return !m.selectedVersion.existsLocally;
|
||||
});
|
||||
if (downloadItems.length === 0) {
|
||||
@@ -2255,10 +2266,11 @@ export class DownloadManager {
|
||||
let cancelled = false;
|
||||
const failedItems = [];
|
||||
// Successful CivitAI items are reconciled in place afterwards
|
||||
// (their cards can be matched by model id); HF items keep the
|
||||
// legacy full reload because they have no CivitAI identity (#1078).
|
||||
// (their cards can be matched by model id); externally-sourced items
|
||||
// keep the legacy full reload because they have no CivitAI identity
|
||||
// (#1078).
|
||||
const completedCivitaiItems = [];
|
||||
let hfCompletedCount = 0;
|
||||
let externalCompletedCount = 0;
|
||||
|
||||
loadingManager.showCancelButton(async () => {
|
||||
if (cancelled) return;
|
||||
@@ -2301,15 +2313,15 @@ export class DownloadManager {
|
||||
|
||||
const item = downloadItems[i];
|
||||
const name = item.displayName || item.filename || (item.selectedVersion?.name || `Model #${item.modelId}`);
|
||||
const isHf = item.source === 'huggingface';
|
||||
const isExternal = isExternalModelSource(item.source);
|
||||
|
||||
updateProgress(0, completedDownloads, name);
|
||||
loadingManager.setStatus(`${i + 1}/${downloadItems.length}: ${name}`);
|
||||
|
||||
try {
|
||||
let response;
|
||||
if (isHf) {
|
||||
const downloadId = Date.now().toString() + '_hf_' + i;
|
||||
if (isExternal) {
|
||||
const downloadId = Date.now().toString() + '_src_' + i;
|
||||
const wsHf = new WebSocket(`${wsProtocol}${window.location.host}/ws/download-progress?id=${downloadId}`);
|
||||
try {
|
||||
await new Promise((resolve, reject) => {
|
||||
@@ -2329,10 +2341,11 @@ export class DownloadManager {
|
||||
}
|
||||
};
|
||||
|
||||
response = await this.apiClient.downloadHfModel({
|
||||
response = await this.apiClient.downloadModelSource({
|
||||
platform: item.platform || item.source,
|
||||
repo: item.repo,
|
||||
filename: item.filename,
|
||||
revision: item.revision || 'main',
|
||||
revision: item.revision || this._defaultRevisionFor(item.platform || item.source),
|
||||
modelRoot,
|
||||
relativePath: targetFolder,
|
||||
useDefaultPaths,
|
||||
@@ -2363,8 +2376,8 @@ export class DownloadManager {
|
||||
} else {
|
||||
completedDownloads++;
|
||||
updateProgress(100, completedDownloads, '');
|
||||
if (isHf) {
|
||||
hfCompletedCount++;
|
||||
if (isExternal) {
|
||||
externalCompletedCount++;
|
||||
} else {
|
||||
completedCivitaiItems.push(item);
|
||||
}
|
||||
@@ -2398,7 +2411,7 @@ export class DownloadManager {
|
||||
});
|
||||
}
|
||||
|
||||
await this._reconcileBatchViewAfterDownload(completedCivitaiItems, hfCompletedCount);
|
||||
await this._reconcileBatchViewAfterDownload(completedCivitaiItems, externalCompletedCount);
|
||||
}
|
||||
|
||||
async downloadVersionWithDefaults(modelType, modelId, versionId, {
|
||||
|
||||
@@ -0,0 +1,291 @@
|
||||
/**
|
||||
* External model source helpers (Hugging Face / ModelScope / TensorArt).
|
||||
*
|
||||
* Mirrors `py/services/model_sources/registry.py` so the frontend and the
|
||||
* backend agree on URL recognition, version-group keys, and which sites
|
||||
* support AI metadata enrichment.
|
||||
*
|
||||
* Models loaded from an older cache may only carry the legacy `hf_url`
|
||||
* field; every helper here falls back to it, and to the legacy
|
||||
* `hf:user/repo` group key shape.
|
||||
*/
|
||||
|
||||
import { translate } from './i18nHelpers.js';
|
||||
|
||||
export const MODEL_SOURCES = [
|
||||
{
|
||||
platform: 'huggingface',
|
||||
label: 'Hugging Face',
|
||||
groupPrefix: 'hf',
|
||||
supportsEnrichment: true,
|
||||
supportsDownload: true,
|
||||
defaultRevision: 'main',
|
||||
defaultSubdir: 'huggingface',
|
||||
exampleUrl: 'https://huggingface.co/user/repo',
|
||||
placeholder: 'https://huggingface.co/user/repo',
|
||||
pattern: /^https?:\/\/(?:www\.)?huggingface\.co\/([^/?#\s]+\/[^/?#\s]+)/i,
|
||||
// `blob` is the web preview page; it maps 1:1 to the `resolve` download URL.
|
||||
filePattern:
|
||||
/^https?:\/\/(?:www\.)?huggingface\.co\/([^/?#\s]+\/[^/?#\s]+)\/(?:resolve|blob)\/([^/?#\s]+)\/(.+)$/i,
|
||||
canonical: (id) => `https://huggingface.co/${id}`,
|
||||
filePage: (id, filename) => `https://huggingface.co/${id}/blob/main/${filename}`,
|
||||
// Bare `user/repo` has always meant Hugging Face; keep that meaning.
|
||||
bareRepoPattern: /^([A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+)$/,
|
||||
},
|
||||
{
|
||||
platform: 'modelscope',
|
||||
label: 'ModelScope',
|
||||
groupPrefix: 'ms',
|
||||
supportsEnrichment: true,
|
||||
supportsDownload: true,
|
||||
defaultRevision: 'master',
|
||||
defaultSubdir: 'modelscope',
|
||||
exampleUrl: 'https://modelscope.cn/models/user/repo',
|
||||
placeholder: 'https://modelscope.cn/models/user/repo',
|
||||
pattern: /^https?:\/\/(?:www\.)?modelscope\.(?:cn|com)\/models\/([^/?#\s]+\/[^/?#\s]+)/i,
|
||||
filePattern:
|
||||
/^https?:\/\/(?:www\.)?modelscope\.(?:cn|com)\/models\/([^/?#\s]+\/[^/?#\s]+)\/resolve\/([^/?#\s]+)\/(.+)$/i,
|
||||
canonical: (id) => `https://modelscope.cn/models/${id}`,
|
||||
filePage: (id, filename) =>
|
||||
`https://modelscope.cn/models/${id}/file/view/master/${filename}`,
|
||||
},
|
||||
{
|
||||
platform: 'tensorart',
|
||||
label: 'TensorArt',
|
||||
groupPrefix: 'ta',
|
||||
supportsEnrichment: false,
|
||||
supportsDownload: false,
|
||||
defaultRevision: '',
|
||||
defaultSubdir: '',
|
||||
exampleUrl: 'https://tensor.art/models/827823520299086029',
|
||||
placeholder: 'https://tensor.art/models/827823520299086029',
|
||||
pattern: /^https?:\/\/(?:www\.)?(?:tensor\.art|tusi\.cn)\/models\/(\d+)/i,
|
||||
filePattern: null,
|
||||
canonical: (id) => `https://tensor.art/models/${id}`,
|
||||
filePage: null,
|
||||
},
|
||||
];
|
||||
|
||||
/** Return the source descriptor for a platform id, or null. */
|
||||
export function getModelSource(platform) {
|
||||
if (!platform || typeof platform !== 'string') return null;
|
||||
const needle = platform.trim().toLowerCase();
|
||||
return MODEL_SOURCES.find((source) => source.platform === needle) || null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse any supported model URL.
|
||||
* @returns {{platform: string, label: string, groupPrefix: string,
|
||||
* supportsEnrichment: boolean, supportsDownload: boolean,
|
||||
* sourceId: string, url: string}|null}
|
||||
*/
|
||||
export function parseModelSourceUrl(url) {
|
||||
if (!url || typeof url !== 'string') return null;
|
||||
const candidate = url.trim();
|
||||
if (!candidate) return null;
|
||||
for (const source of MODEL_SOURCES) {
|
||||
const match = candidate.match(source.pattern);
|
||||
if (match) {
|
||||
return {
|
||||
...source,
|
||||
sourceId: match[1],
|
||||
url: source.canonical(match[1]),
|
||||
};
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Return the stored source URL of a model (new field, then legacy). */
|
||||
export function getModelSourceUrl(model) {
|
||||
if (!model) return '';
|
||||
const value = model.source_url || model.hf_url || '';
|
||||
return typeof value === 'string' ? value.trim() : '';
|
||||
}
|
||||
|
||||
/** Return the stored source platform of a model. */
|
||||
export function getModelSourcePlatform(model) {
|
||||
if (!model) return '';
|
||||
const value = model.source_platform || '';
|
||||
return typeof value === 'string' ? value.trim().toLowerCase() : '';
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the full source descriptor for a model, tolerating models that
|
||||
* predate the `source_*` fields.
|
||||
*/
|
||||
export function getModelSourceInfo(model) {
|
||||
if (!model) return null;
|
||||
const url = getModelSourceUrl(model);
|
||||
const declared = getModelSource(getModelSourcePlatform(model));
|
||||
const parsed = parseModelSourceUrl(url);
|
||||
|
||||
if (declared) {
|
||||
return {
|
||||
...declared,
|
||||
sourceId: parsed ? parsed.sourceId : '',
|
||||
url: parsed ? parsed.url : url,
|
||||
};
|
||||
}
|
||||
return parsed;
|
||||
}
|
||||
|
||||
/**
|
||||
* Version-group key for a model, matching the backend's `_extract_group_key`.
|
||||
* Returns `''` when the model has no external source.
|
||||
*/
|
||||
export function getModelSourceGroupKey(model) {
|
||||
const info = getModelSourceInfo(model);
|
||||
if (!info || !info.sourceId) return '';
|
||||
return `${info.groupPrefix}:${info.sourceId}`;
|
||||
}
|
||||
|
||||
/** Whether AI metadata enrichment can run for this model's source. */
|
||||
export function canEnrichModelSource(model) {
|
||||
const info = getModelSourceInfo(model);
|
||||
return Boolean(info && info.supportsEnrichment);
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse a version-group key such as `hf:user/repo`, `ms:user/repo`, or
|
||||
* `ta:827823520299086029` back into its source descriptor.
|
||||
*
|
||||
* These keys are NOT CivitAI model ids, so callers must not send them to the
|
||||
* CivitAI API.
|
||||
*
|
||||
* @returns {{platform: string, label: string, sourceId: string}|null}
|
||||
*/
|
||||
export function parseModelSourceGroupKey(groupKey) {
|
||||
if (!groupKey || typeof groupKey !== 'string') return null;
|
||||
const separator = groupKey.indexOf(':');
|
||||
if (separator <= 0) return null;
|
||||
|
||||
const prefix = groupKey.slice(0, separator);
|
||||
const source = MODEL_SOURCES.find((candidate) => candidate.groupPrefix === prefix);
|
||||
if (!source) return null;
|
||||
|
||||
return {
|
||||
platform: source.platform,
|
||||
label: source.label,
|
||||
sourceId: groupKey.slice(separator + 1),
|
||||
};
|
||||
}
|
||||
|
||||
/** Localised "View on X" title for the source globe icon. */
|
||||
export function getModelSourceViewTitle(info) {
|
||||
if (!info) return '';
|
||||
if (info.platform === 'huggingface') {
|
||||
return translate('modelCard.actions.viewOnHuggingFace', {}, 'View on Hugging Face');
|
||||
}
|
||||
return translate(
|
||||
'modelCard.actions.viewOnSource',
|
||||
{ source: info.label },
|
||||
`View on ${info.label}`
|
||||
);
|
||||
}
|
||||
|
||||
/** Open a model page on its external site in a new tab. */
|
||||
export function openModelSource(url) {
|
||||
if (!url) return;
|
||||
window.open(url, '_blank', 'noopener,noreferrer');
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Download support
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/** Sources whose repositories the backend can download from. */
|
||||
export const DOWNLOADABLE_SOURCES = MODEL_SOURCES.filter((s) => s.supportsDownload);
|
||||
|
||||
/**
|
||||
* Whether a DownloadManager `source` value refers to an external repository
|
||||
* download (as opposed to a CivitAI/CivArchive version or a direct link).
|
||||
*/
|
||||
export function isExternalModelSource(source) {
|
||||
return DOWNLOADABLE_SOURCES.some((s) => s.platform === source);
|
||||
}
|
||||
|
||||
/** Return the downloadable source descriptor for a platform, or null. */
|
||||
export function getDownloadSource(platform) {
|
||||
const source = getModelSource(platform);
|
||||
return source && source.supportsDownload ? source : null;
|
||||
}
|
||||
|
||||
/** Normalise a repository id: reject traversal, exactly one slash. */
|
||||
export function isValidRepoId(repo) {
|
||||
if (!repo || typeof repo !== 'string' || repo.split('/').length !== 2) return false;
|
||||
return repo
|
||||
.split('/')
|
||||
.every((part) => part && part !== '.' && part !== '..' && /^[A-Za-z0-9_][\w.-]*$/.test(part));
|
||||
}
|
||||
|
||||
/**
|
||||
* Recognise a downloadable model-source URL.
|
||||
*
|
||||
* Handles both a repository page and a direct file (resolve) URL for every
|
||||
* source that supports downloads, plus the historical bare `owner/name`
|
||||
* shorthand, which only ever meant Hugging Face.
|
||||
*
|
||||
* @returns {{kind: 'repo'|'file', platform: string, label: string,
|
||||
* repo: string, revision?: string, filename?: string}|null}
|
||||
*/
|
||||
export function detectModelSourceDownloadUrl(url) {
|
||||
if (!url || typeof url !== 'string') return null;
|
||||
const candidate = url.trim();
|
||||
if (!candidate) return null;
|
||||
|
||||
// Direct file URLs first: the repo pattern would match their prefix and
|
||||
// lose the revision/filename.
|
||||
for (const source of DOWNLOADABLE_SOURCES) {
|
||||
if (!source.filePattern) continue;
|
||||
const match = candidate.match(source.filePattern);
|
||||
if (match) {
|
||||
return {
|
||||
kind: 'file',
|
||||
platform: source.platform,
|
||||
label: source.label,
|
||||
repo: match[1],
|
||||
revision: match[2],
|
||||
filename: match[3],
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
for (const source of DOWNLOADABLE_SOURCES) {
|
||||
const match = candidate.match(source.pattern);
|
||||
if (match) {
|
||||
return {
|
||||
kind: 'repo',
|
||||
platform: source.platform,
|
||||
label: source.label,
|
||||
repo: match[1],
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
if (!candidate.includes('://')) {
|
||||
for (const source of DOWNLOADABLE_SOURCES) {
|
||||
if (!source.bareRepoPattern) continue;
|
||||
const match = candidate.match(source.bareRepoPattern);
|
||||
if (match && isValidRepoId(match[1])) {
|
||||
return {
|
||||
kind: 'repo',
|
||||
platform: source.platform,
|
||||
label: source.label,
|
||||
repo: match[1],
|
||||
};
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/** Human-facing page for one file of an external repository. */
|
||||
export function buildModelSourceFilePage({ platform, repo, filename }) {
|
||||
const source = getModelSource(platform);
|
||||
if (!source || !source.filePage || !filename) {
|
||||
return source ? source.canonical(repo) : null;
|
||||
}
|
||||
return source.filePage(repo, filename);
|
||||
}
|
||||
@@ -21,7 +21,7 @@
|
||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||
</div>
|
||||
<div class="context-menu-item" data-action="link-hf">
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -21,7 +21,7 @@
|
||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||
</div>
|
||||
<div class="context-menu-item" data-action="link-hf">
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -1,24 +1,29 @@
|
||||
<!-- Link to HuggingFace Modal -->
|
||||
<!-- Link to Model Source Modal -->
|
||||
<div id="linkHfModal" class="modal">
|
||||
<div class="modal-content">
|
||||
<button class="close" onclick="modalManager.closeModal('linkHfModal')">×</button>
|
||||
<h2>{{ t('modals.linkHuggingFace.title') }}</h2>
|
||||
<h2>{{ t('modals.linkModelSource.title') }}</h2>
|
||||
<div class="warning-box">
|
||||
<i class="fas fa-info-circle"></i>
|
||||
<p>{{ t('modals.linkHuggingFace.infoText') }}</p>
|
||||
<p>{{ t('modals.linkModelSource.infoText') }}</p>
|
||||
</div>
|
||||
<div class="input-group">
|
||||
<label for="hfModelUrl">{{ t('modals.linkHuggingFace.urlLabel') }}</label>
|
||||
<input type="text" id="hfModelUrl" placeholder="{{ t('modals.linkHuggingFace.urlPlaceholder') }}" />
|
||||
<label for="hfModelUrl">{{ t('modals.linkModelSource.urlLabel') }}</label>
|
||||
<input type="text" id="hfModelUrl" placeholder="{{ t('modals.linkModelSource.urlPlaceholder') }}" />
|
||||
<div class="input-error" id="hfModelUrlError"></div>
|
||||
<div class="input-help">
|
||||
{{ t('modals.linkHuggingFace.helpText') }}<br>
|
||||
<strong>https://huggingface.co/user/repo</strong>
|
||||
{{ t('modals.linkModelSource.helpText') }}
|
||||
<div id="hfSupportedSources">
|
||||
<strong>https://huggingface.co/user/repo</strong><br>
|
||||
<strong>https://modelscope.cn/models/user/repo</strong><br>
|
||||
<strong>https://tensor.art/models/827823520299086029</strong>
|
||||
</div>
|
||||
{{ t('modals.linkModelSource.enrichNote') }}
|
||||
</div>
|
||||
</div>
|
||||
<div class="modal-actions">
|
||||
<button class="cancel-btn" onclick="modalManager.closeModal('linkHfModal')">{{ t('common.actions.cancel') }}</button>
|
||||
<button class="confirm-btn" id="confirmLinkHfBtn">{{ t('modals.linkHuggingFace.confirmAction') }}</button>
|
||||
<button class="confirm-btn" id="confirmLinkHfBtn">{{ t('modals.linkModelSource.confirmAction') }}</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -21,7 +21,7 @@
|
||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||
</div>
|
||||
<div class="context-menu-item" data-action="link-hf">
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -80,7 +80,7 @@
|
||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||
</div>
|
||||
<div class="context-menu-item" data-action="link-hf">
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
@@ -191,4 +191,60 @@ describe('ModelCard source globe (#1094)', () => {
|
||||
expect(openHuggingFace).toHaveBeenCalledWith('https://huggingface.co/user/repo');
|
||||
expect(openCivitai).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('points the globe at ModelScope for a ModelScope-linked model', () => {
|
||||
const card = mountCard(
|
||||
createModelCard,
|
||||
makeModel({
|
||||
from_civitai: false,
|
||||
civitai: {},
|
||||
source_platform: 'modelscope',
|
||||
source_url: 'https://modelscope.cn/models/user/repo',
|
||||
})
|
||||
);
|
||||
|
||||
expect(card.dataset.has_civitai).toBe('false');
|
||||
expect(card.dataset.source_platform).toBe('modelscope');
|
||||
expect(card.dataset.hf_url).toBe('');
|
||||
expect(card.querySelector('.fa-globe').getAttribute('title')).toBe('View on ModelScope');
|
||||
});
|
||||
|
||||
it('opens the ModelScope page when the globe is clicked', () => {
|
||||
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||
const card = mountCard(
|
||||
createModelCard,
|
||||
makeModel({
|
||||
from_civitai: false,
|
||||
civitai: {},
|
||||
source_platform: 'modelscope',
|
||||
source_url: 'https://modelscope.cn/models/user/repo',
|
||||
})
|
||||
);
|
||||
setupModelCardEventDelegation('loras');
|
||||
|
||||
card.querySelector('.fa-globe').dispatchEvent(new MouseEvent('click', { bubbles: true }));
|
||||
|
||||
expect(openSpy).toHaveBeenCalledWith(
|
||||
'https://modelscope.cn/models/user/repo',
|
||||
'_blank',
|
||||
'noopener,noreferrer'
|
||||
);
|
||||
expect(openCivitai).not.toHaveBeenCalled();
|
||||
expect(openHuggingFace).not.toHaveBeenCalled();
|
||||
openSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('points the globe at TensorArt for a TensorArt-linked model', () => {
|
||||
const card = mountCard(
|
||||
createModelCard,
|
||||
makeModel({
|
||||
from_civitai: false,
|
||||
civitai: {},
|
||||
source_platform: 'tensorart',
|
||||
source_url: 'https://tensor.art/models/827823520299086029',
|
||||
})
|
||||
);
|
||||
|
||||
expect(card.querySelector('.fa-globe').getAttribute('title')).toBe('View on TensorArt');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
import { ModelContextMenuMixin } from '../../../static/js/components/ContextMenu/ModelContextMenuMixin.js';
|
||||
|
||||
@@ -14,3 +14,115 @@ describe('ModelContextMenuMixin.getModelTypePrefix', () => {
|
||||
expect(ModelContextMenuMixin.getModelTypePrefix.call({})).toBe('loras');
|
||||
});
|
||||
});
|
||||
|
||||
describe('ModelContextMenuMixin.updateEnrichMenuItem', () => {
|
||||
function setupMenu() {
|
||||
document.body.innerHTML = '<div id="menu"><div data-action="enrich-hf-llm"></div></div>';
|
||||
return { menu: document.getElementById('menu') };
|
||||
}
|
||||
|
||||
function cardWith(dataset) {
|
||||
return { dataset };
|
||||
}
|
||||
|
||||
it('enables enrichment for Hugging Face links', () => {
|
||||
const context = setupMenu();
|
||||
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||
context,
|
||||
cardWith({ hf_url: 'https://huggingface.co/user/repo' })
|
||||
);
|
||||
|
||||
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||
expect(item.classList.contains('disabled')).toBe(false);
|
||||
expect(item.title).toBe('');
|
||||
});
|
||||
|
||||
it('enables enrichment for ModelScope links', () => {
|
||||
const context = setupMenu();
|
||||
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||
context,
|
||||
cardWith({
|
||||
source_platform: 'modelscope',
|
||||
source_url: 'https://modelscope.cn/models/user/repo',
|
||||
})
|
||||
);
|
||||
|
||||
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||
expect(item.classList.contains('disabled')).toBe(false);
|
||||
});
|
||||
|
||||
it('disables enrichment for TensorArt and explains why', () => {
|
||||
const context = setupMenu();
|
||||
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||
context,
|
||||
cardWith({
|
||||
source_platform: 'tensorart',
|
||||
source_url: 'https://tensor.art/models/827823520299086029',
|
||||
})
|
||||
);
|
||||
|
||||
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||
expect(item.classList.contains('disabled')).toBe(true);
|
||||
expect(item.title).toContain('TensorArt');
|
||||
});
|
||||
|
||||
it('disables enrichment when no source is linked', () => {
|
||||
const context = setupMenu();
|
||||
ModelContextMenuMixin.updateEnrichMenuItem.call(context, cardWith({}));
|
||||
|
||||
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||
expect(item.classList.contains('disabled')).toBe(true);
|
||||
expect(item.title).toContain('Link this model to a model source');
|
||||
});
|
||||
});
|
||||
|
||||
describe('ModelContextMenuMixin._renderSupportedSources', () => {
|
||||
const originalFetch = global.fetch;
|
||||
|
||||
beforeEach(() => {
|
||||
document.body.innerHTML = '<div id="hfSupportedSources">static fallback</div>';
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
global.fetch = originalFetch;
|
||||
});
|
||||
|
||||
it('renders the server-provided example URLs', async () => {
|
||||
global.fetch = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => [
|
||||
{ platform: 'huggingface', example_url: 'https://huggingface.co/user/repo' },
|
||||
{ platform: 'modelscope', example_url: 'https://modelscope.cn/models/user/repo' },
|
||||
{ platform: 'tensorart', example_url: 'https://tensor.art/models/123' },
|
||||
],
|
||||
});
|
||||
|
||||
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||
|
||||
const html = document.getElementById('hfSupportedSources').innerHTML;
|
||||
expect(html).toContain('https://huggingface.co/user/repo');
|
||||
expect(html).toContain('https://modelscope.cn/models/user/repo');
|
||||
expect(html).toContain('https://tensor.art/models/123');
|
||||
});
|
||||
|
||||
it('keeps the static fallback when the request fails', async () => {
|
||||
global.fetch = vi.fn().mockRejectedValue(new Error('offline'));
|
||||
|
||||
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||
|
||||
expect(document.getElementById('hfSupportedSources').innerHTML).toBe('static fallback');
|
||||
});
|
||||
|
||||
it('escapes markup from the server payload', async () => {
|
||||
global.fetch = vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => [{ example_url: '<img src=x onerror=alert(1)>' }],
|
||||
});
|
||||
|
||||
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||
|
||||
const html = document.getElementById('hfSupportedSources').innerHTML;
|
||||
expect(html).not.toContain('<img');
|
||||
expect(html).toContain('<img');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -26,6 +26,7 @@ const {
|
||||
},
|
||||
},
|
||||
downloadModel: vi.fn(),
|
||||
downloadModelSource: vi.fn(),
|
||||
downloadHfModel: vi.fn(),
|
||||
cancelDownload: vi.fn(),
|
||||
getPageState: vi.fn(() => ({})),
|
||||
@@ -158,7 +159,7 @@ describe('DownloadManager batch download summary flow', () => {
|
||||
// Reset the shared mocks so mockResolvedValueOnce queues and call
|
||||
// history never leak between tests.
|
||||
mockApiClient.downloadModel.mockReset();
|
||||
mockApiClient.downloadHfModel.mockReset();
|
||||
mockApiClient.downloadModelSource.mockReset();
|
||||
mockApiClient.cancelDownload.mockReset();
|
||||
showToastMock.mockClear();
|
||||
showDownloadBatchSummaryMock.mockClear();
|
||||
@@ -406,14 +407,15 @@ describe('DownloadManager batch download summary flow', () => {
|
||||
expect(showToastMock).toHaveBeenCalledWith('toast.loras.downloadCompleted', {}, 'success');
|
||||
});
|
||||
|
||||
it('shows a summary for HF partial failure and retries only the failed files', async () => {
|
||||
manager.hfRepoId = 'user/repo';
|
||||
manager.hfSelectedFiles = ['a.safetensors', 'b.safetensors'];
|
||||
mockApiClient.downloadHfModel
|
||||
it('shows a summary for external repo partial failure and retries only the failed files', async () => {
|
||||
manager.sourcePlatform = 'huggingface';
|
||||
manager.sourceRepoId = 'user/repo';
|
||||
manager.sourceSelectedFiles = ['a.safetensors', 'b.safetensors'];
|
||||
mockApiClient.downloadModelSource
|
||||
.mockResolvedValueOnce({ success: true })
|
||||
.mockResolvedValueOnce({ success: false, error: 'denied' });
|
||||
|
||||
const result = await manager._downloadHfSingle({ modelRoot: '/m', useDefaultPaths: true });
|
||||
const result = await manager._downloadExternalRepoFiles({ modelRoot: '/m', useDefaultPaths: true });
|
||||
|
||||
expect(result).toBe(false);
|
||||
expect(showDownloadBatchSummaryMock).toHaveBeenCalledTimes(1);
|
||||
@@ -428,7 +430,9 @@ describe('DownloadManager batch download summary flow', () => {
|
||||
|
||||
await summary.onRetry();
|
||||
|
||||
expect(mockApiClient.downloadHfModel).toHaveBeenCalledTimes(3);
|
||||
expect(mockApiClient.downloadHfModel.mock.calls[2][0].filename).toBe('b.safetensors');
|
||||
expect(mockApiClient.downloadModelSource).toHaveBeenCalledTimes(3);
|
||||
const retryArgs = mockApiClient.downloadModelSource.mock.calls[2][0];
|
||||
expect(retryArgs.filename).toBe('b.safetensors');
|
||||
expect(retryArgs.platform).toBe('huggingface');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -0,0 +1,269 @@
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
|
||||
const {
|
||||
DOWNLOAD_MANAGER_MODULE,
|
||||
MODAL_MANAGER_MODULE,
|
||||
UI_HELPERS_MODULE,
|
||||
STATE_MODULE,
|
||||
LOADING_MANAGER_MODULE,
|
||||
API_FACTORY_MODULE,
|
||||
STORAGE_HELPERS_MODULE,
|
||||
FOLDER_TREE_MANAGER_MODULE,
|
||||
I18N_HELPERS_MODULE,
|
||||
SUMMARY_MODULE,
|
||||
mockApiClient,
|
||||
mockLoadingManager,
|
||||
showDownloadBatchSummaryMock,
|
||||
} = vi.hoisted(() => {
|
||||
const mockApiClient = {
|
||||
apiConfig: { config: { displayName: 'LoRA', singularName: 'lora' } },
|
||||
downloadModel: vi.fn(),
|
||||
downloadModelSource: vi.fn(),
|
||||
fetchModelSourceFiles: vi.fn(),
|
||||
cancelDownload: vi.fn(),
|
||||
getPageState: vi.fn(() => ({})),
|
||||
};
|
||||
|
||||
const mockLoadingManager = {
|
||||
showSimpleLoading: vi.fn(),
|
||||
hide: vi.fn(),
|
||||
restoreProgressBar: vi.fn(),
|
||||
showDownloadProgress: vi.fn(() => vi.fn()),
|
||||
setStatus: vi.fn(),
|
||||
showCancelButton: vi.fn(),
|
||||
};
|
||||
|
||||
return {
|
||||
DOWNLOAD_MANAGER_MODULE: new URL('../../../static/js/managers/DownloadManager.js', import.meta.url).pathname,
|
||||
MODAL_MANAGER_MODULE: new URL('../../../static/js/managers/ModalManager.js', import.meta.url).pathname,
|
||||
UI_HELPERS_MODULE: new URL('../../../static/js/utils/uiHelpers.js', import.meta.url).pathname,
|
||||
STATE_MODULE: new URL('../../../static/js/state/index.js', import.meta.url).pathname,
|
||||
LOADING_MANAGER_MODULE: new URL('../../../static/js/managers/LoadingManager.js', import.meta.url).pathname,
|
||||
API_FACTORY_MODULE: new URL('../../../static/js/api/modelApiFactory.js', import.meta.url).pathname,
|
||||
STORAGE_HELPERS_MODULE: new URL('../../../static/js/utils/storageHelpers.js', import.meta.url).pathname,
|
||||
FOLDER_TREE_MANAGER_MODULE: new URL('../../../static/js/components/FolderTreeManager.js', import.meta.url).pathname,
|
||||
I18N_HELPERS_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
|
||||
SUMMARY_MODULE: new URL('../../../static/js/components/DownloadBatchSummaryModal.js', import.meta.url).pathname,
|
||||
mockApiClient,
|
||||
mockLoadingManager,
|
||||
showDownloadBatchSummaryMock: vi.fn(),
|
||||
};
|
||||
});
|
||||
|
||||
vi.mock(MODAL_MANAGER_MODULE, () => ({
|
||||
modalManager: { showModal: vi.fn(), closeModal: vi.fn() },
|
||||
}));
|
||||
|
||||
vi.mock(UI_HELPERS_MODULE, () => ({ showToast: vi.fn() }));
|
||||
|
||||
vi.mock(STATE_MODULE, () => ({
|
||||
state: { global: { settings: {} }, loadingManager: mockLoadingManager },
|
||||
}));
|
||||
|
||||
vi.mock(LOADING_MANAGER_MODULE, () => ({
|
||||
LoadingManager: vi.fn(() => mockLoadingManager),
|
||||
}));
|
||||
|
||||
vi.mock(API_FACTORY_MODULE, () => ({
|
||||
getModelApiClient: vi.fn(() => mockApiClient),
|
||||
resetAndReload: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(STORAGE_HELPERS_MODULE, () => ({
|
||||
getStorageItem: vi.fn((_key, defaultValue) => defaultValue),
|
||||
setStorageItem: vi.fn(),
|
||||
}));
|
||||
|
||||
vi.mock(FOLDER_TREE_MANAGER_MODULE, () => ({
|
||||
FolderTreeManager: vi.fn(() => ({ clearSelection: vi.fn(), init: vi.fn() })),
|
||||
}));
|
||||
|
||||
vi.mock(I18N_HELPERS_MODULE, () => ({
|
||||
translate: vi.fn((_, __, fallback) => fallback ?? ''),
|
||||
}));
|
||||
|
||||
vi.mock(SUMMARY_MODULE, () => ({
|
||||
showDownloadBatchSummary: showDownloadBatchSummaryMock,
|
||||
}));
|
||||
|
||||
class FakeWebSocket {
|
||||
constructor(url) {
|
||||
this.url = url;
|
||||
this.onopen = null;
|
||||
this.onmessage = null;
|
||||
this.onerror = null;
|
||||
this.close = vi.fn();
|
||||
queueMicrotask(() => {
|
||||
if (this.onopen) this.onopen();
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const MS_REPO_URL = 'https://modelscope.cn/models/jj3550945163/Krea-2-LORA';
|
||||
|
||||
describe('DownloadManager external model source downloads', () => {
|
||||
let DownloadManager;
|
||||
let manager;
|
||||
|
||||
beforeEach(async () => {
|
||||
document.body.innerHTML = '';
|
||||
vi.stubGlobal('WebSocket', FakeWebSocket);
|
||||
mockApiClient.downloadModelSource.mockReset();
|
||||
mockApiClient.fetchModelSourceFiles.mockReset();
|
||||
mockLoadingManager.showSimpleLoading.mockReset();
|
||||
|
||||
({ DownloadManager } = await import(DOWNLOAD_MANAGER_MODULE));
|
||||
manager = new DownloadManager();
|
||||
manager.apiClient = mockApiClient;
|
||||
manager.showBatchPreviewStep = vi.fn();
|
||||
manager.proceedToLocation = vi.fn();
|
||||
});
|
||||
|
||||
it('loads a ModelScope repo as batch items on the master revision', async () => {
|
||||
mockApiClient.fetchModelSourceFiles.mockResolvedValue([
|
||||
{ filename: 'a.safetensors', size: 10 },
|
||||
{ filename: 'sub/b.safetensors', size: 20 },
|
||||
]);
|
||||
const errorElement = { textContent: '' };
|
||||
|
||||
await manager._validateAndFetchExternalRepo([MS_REPO_URL], errorElement);
|
||||
|
||||
expect(mockApiClient.fetchModelSourceFiles).toHaveBeenCalledWith(
|
||||
'jj3550945163/Krea-2-LORA',
|
||||
'modelscope',
|
||||
'master'
|
||||
);
|
||||
expect(errorElement.textContent).toBe('');
|
||||
expect(manager.source).toBe('modelscope');
|
||||
expect(manager.isBatchMode).toBe(true);
|
||||
expect(manager.batchModels).toHaveLength(2);
|
||||
expect(manager.batchModels[0]).toMatchObject({
|
||||
source: 'modelscope',
|
||||
platform: 'modelscope',
|
||||
repo: 'jj3550945163/Krea-2-LORA',
|
||||
revision: 'master',
|
||||
filename: 'a.safetensors',
|
||||
fileSizeBytes: 10,
|
||||
displayName: 'a.safetensors',
|
||||
});
|
||||
expect(manager.showBatchPreviewStep).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps Hugging Face on its own revision', async () => {
|
||||
mockApiClient.fetchModelSourceFiles.mockResolvedValue([
|
||||
{ filename: 'a.safetensors', size: 10 },
|
||||
]);
|
||||
|
||||
await manager._validateAndFetchExternalRepo(
|
||||
['https://huggingface.co/user/repo'],
|
||||
{ textContent: '' }
|
||||
);
|
||||
|
||||
expect(mockApiClient.fetchModelSourceFiles).toHaveBeenCalledWith(
|
||||
'user/repo',
|
||||
'huggingface',
|
||||
'main'
|
||||
);
|
||||
expect(manager.batchModels[0].revision).toBe('main');
|
||||
});
|
||||
|
||||
it('surfaces a listing failure on the URL field', async () => {
|
||||
mockApiClient.fetchModelSourceFiles.mockRejectedValue(new Error('Repository not found'));
|
||||
const errorElement = { textContent: '' };
|
||||
|
||||
await manager._validateAndFetchExternalRepo([MS_REPO_URL], errorElement);
|
||||
|
||||
expect(errorElement.textContent).toBe('Repository not found');
|
||||
expect(manager.showBatchPreviewStep).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('skips file selection for a direct ModelScope file URL', async () => {
|
||||
await manager._validateAndFetchExternalRepo(
|
||||
[`${MS_REPO_URL}/resolve/master/Krea-2-LORA_c1-st1000.safetensors`],
|
||||
{ textContent: '' }
|
||||
);
|
||||
|
||||
expect(manager.isBatchMode).toBe(false);
|
||||
expect(manager.sourcePlatform).toBe('modelscope');
|
||||
expect(manager.sourceRepoId).toBe('jj3550945163/Krea-2-LORA');
|
||||
expect(manager.sourceSelectedFiles).toEqual(['Krea-2-LORA_c1-st1000.safetensors']);
|
||||
expect(manager.proceedToLocation).toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('downloads a single ModelScope file through the generic endpoint', async () => {
|
||||
mockApiClient.downloadModelSource.mockResolvedValue({ success: true });
|
||||
manager.sourcePlatform = 'modelscope';
|
||||
manager.sourceRepoId = 'jj3550945163/Krea-2-LORA';
|
||||
manager.sourceSelectedFiles = ['Krea-2-LORA_c1-st1000.safetensors'];
|
||||
|
||||
await manager._downloadExternalRepoFiles({
|
||||
modelRoot: '/models/loras',
|
||||
targetFolder: '',
|
||||
useDefaultPaths: true,
|
||||
});
|
||||
|
||||
expect(mockApiClient.downloadModelSource).toHaveBeenCalledTimes(1);
|
||||
expect(mockApiClient.downloadModelSource.mock.calls[0][0]).toMatchObject({
|
||||
platform: 'modelscope',
|
||||
repo: 'jj3550945163/Krea-2-LORA',
|
||||
filename: 'Krea-2-LORA_c1-st1000.safetensors',
|
||||
revision: 'master',
|
||||
});
|
||||
});
|
||||
|
||||
it('carries the platform through a batch download', async () => {
|
||||
mockApiClient.downloadModelSource.mockResolvedValue({ success: true });
|
||||
manager.showBatchPreviewStep = vi.fn();
|
||||
|
||||
await manager.executeBatchDownload(
|
||||
[
|
||||
{
|
||||
source: 'modelscope',
|
||||
platform: 'modelscope',
|
||||
repo: 'u/r',
|
||||
filename: 'f.safetensors',
|
||||
revision: 'master',
|
||||
displayName: 'f.safetensors',
|
||||
checked: true,
|
||||
},
|
||||
],
|
||||
{ modelRoot: '/models/loras', targetFolder: '', useDefaultPaths: true }
|
||||
);
|
||||
|
||||
expect(mockApiClient.downloadModelSource).toHaveBeenCalledTimes(1);
|
||||
expect(mockApiClient.downloadModelSource.mock.calls[0][0]).toMatchObject({
|
||||
platform: 'modelscope',
|
||||
repo: 'u/r',
|
||||
filename: 'f.safetensors',
|
||||
revision: 'master',
|
||||
});
|
||||
expect(mockApiClient.downloadModel).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('links failures to the ModelScope file page', async () => {
|
||||
expect(
|
||||
manager._buildSingleItemUrl({
|
||||
source: 'modelscope',
|
||||
repo: 'u/r',
|
||||
filename: 'sub/f.safetensors',
|
||||
})
|
||||
).toBe('https://modelscope.cn/models/u/r/file/view/master/sub/f.safetensors');
|
||||
|
||||
expect(
|
||||
manager._buildSingleItemUrl({
|
||||
source: 'huggingface',
|
||||
repo: 'u/r',
|
||||
filename: 'f.safetensors',
|
||||
})
|
||||
).toBe('https://huggingface.co/u/r/blob/main/f.safetensors');
|
||||
});
|
||||
|
||||
it('groups the same repo name on two platforms separately', () => {
|
||||
const hf = { source: 'huggingface', repo: 'u/r' };
|
||||
const ms = { source: 'modelscope', repo: 'u/r' };
|
||||
|
||||
expect(manager._externalGroupKey(hf)).toBe('huggingface:u/r');
|
||||
expect(manager._externalGroupKey(ms)).toBe('modelscope:u/r');
|
||||
expect(manager._externalGroupKey(hf)).not.toBe(manager._externalGroupKey(ms));
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,177 @@
|
||||
import { describe, it, expect, vi } from 'vitest';
|
||||
|
||||
const { I18N_MODULE } = vi.hoisted(() => ({
|
||||
I18N_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
|
||||
}));
|
||||
|
||||
vi.mock(I18N_MODULE, () => ({
|
||||
translate: vi.fn((key, params, fallback) => (typeof fallback === 'string' ? fallback : key)),
|
||||
}));
|
||||
|
||||
const {
|
||||
MODEL_SOURCES,
|
||||
parseModelSourceUrl,
|
||||
getModelSource,
|
||||
getModelSourceInfo,
|
||||
getModelSourceUrl,
|
||||
getModelSourceGroupKey,
|
||||
canEnrichModelSource,
|
||||
getModelSourceViewTitle,
|
||||
parseModelSourceGroupKey,
|
||||
openModelSource,
|
||||
} = await import('../../../static/js/utils/modelSourceHelpers.js');
|
||||
|
||||
describe('modelSourceHelpers', () => {
|
||||
it('exposes one descriptor per supported platform', () => {
|
||||
expect(MODEL_SOURCES.map((s) => s.platform)).toEqual([
|
||||
'huggingface',
|
||||
'modelscope',
|
||||
'tensorart',
|
||||
]);
|
||||
});
|
||||
|
||||
describe('parseModelSourceUrl', () => {
|
||||
it('recognises Hugging Face URLs', () => {
|
||||
const info = parseModelSourceUrl('https://huggingface.co/user/repo');
|
||||
expect(info.platform).toBe('huggingface');
|
||||
expect(info.sourceId).toBe('user/repo');
|
||||
});
|
||||
|
||||
it('recognises ModelScope URLs with view sub-paths', () => {
|
||||
const info = parseModelSourceUrl('https://modelscope.cn/models/user/repo/summary');
|
||||
expect(info.platform).toBe('modelscope');
|
||||
expect(info.sourceId).toBe('user/repo');
|
||||
expect(info.url).toBe('https://modelscope.cn/models/user/repo');
|
||||
});
|
||||
|
||||
it('recognises TensorArt URLs and keeps only the numeric id', () => {
|
||||
const info = parseModelSourceUrl(
|
||||
'https://tensor.art/models/827823520299086029/Vivid-Impressions-Storybook-Sstyle-V1.0'
|
||||
);
|
||||
expect(info.platform).toBe('tensorart');
|
||||
expect(info.sourceId).toBe('827823520299086029');
|
||||
expect(info.url).toBe('https://tensor.art/models/827823520299086029');
|
||||
});
|
||||
|
||||
it('rejects unsupported URLs', () => {
|
||||
expect(parseModelSourceUrl('https://example.com/x')).toBeNull();
|
||||
expect(parseModelSourceUrl('')).toBeNull();
|
||||
expect(parseModelSourceUrl(null)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('getModelSourceInfo', () => {
|
||||
it('falls back to the legacy hf_url field', () => {
|
||||
const info = getModelSourceInfo({ hf_url: 'https://huggingface.co/user/repo' });
|
||||
expect(info.platform).toBe('huggingface');
|
||||
expect(info.sourceId).toBe('user/repo');
|
||||
});
|
||||
|
||||
it('prefers the canonical source fields', () => {
|
||||
const info = getModelSourceInfo({
|
||||
source_platform: 'modelscope',
|
||||
source_url: 'https://modelscope.cn/models/user/repo',
|
||||
hf_url: 'https://huggingface.co/old/repo',
|
||||
});
|
||||
expect(info.platform).toBe('modelscope');
|
||||
});
|
||||
|
||||
it('returns null when there is no source', () => {
|
||||
expect(getModelSourceInfo({})).toBeNull();
|
||||
expect(getModelSourceInfo({ hf_url: '' })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('getModelSourceUrl', () => {
|
||||
it('reads source_url then hf_url', () => {
|
||||
expect(getModelSourceUrl({ source_url: 'https://a.example/1' })).toBe('https://a.example/1');
|
||||
expect(getModelSourceUrl({ hf_url: 'https://huggingface.co/u/r' })).toBe(
|
||||
'https://huggingface.co/u/r'
|
||||
);
|
||||
expect(getModelSourceUrl({})).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('getModelSourceGroupKey', () => {
|
||||
it('matches the backend group-key shapes', () => {
|
||||
expect(getModelSourceGroupKey({ hf_url: 'https://huggingface.co/u/r' })).toBe('hf:u/r');
|
||||
expect(
|
||||
getModelSourceGroupKey({ source_url: 'https://modelscope.cn/models/u/r' })
|
||||
).toBe('ms:u/r');
|
||||
expect(getModelSourceGroupKey({ source_url: 'https://tensor.art/models/123' })).toBe(
|
||||
'ta:123'
|
||||
);
|
||||
});
|
||||
|
||||
it('returns an empty string without a source', () => {
|
||||
expect(getModelSourceGroupKey({})).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('canEnrichModelSource', () => {
|
||||
it('allows Hugging Face and ModelScope', () => {
|
||||
expect(canEnrichModelSource({ hf_url: 'https://huggingface.co/u/r' })).toBe(true);
|
||||
expect(
|
||||
canEnrichModelSource({ source_url: 'https://modelscope.cn/models/u/r' })
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it('disallows TensorArt and unlinked models', () => {
|
||||
expect(canEnrichModelSource({ source_url: 'https://tensor.art/models/123' })).toBe(false);
|
||||
expect(canEnrichModelSource({})).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('getModelSourceViewTitle', () => {
|
||||
it('uses the branded label for non-HF sources', () => {
|
||||
expect(getModelSourceViewTitle(getModelSource('modelscope'))).toBe('View on ModelScope');
|
||||
expect(getModelSourceViewTitle(getModelSource('tensorart'))).toBe('View on TensorArt');
|
||||
});
|
||||
|
||||
it('keeps the historical Hugging Face title', () => {
|
||||
expect(getModelSourceViewTitle(getModelSource('huggingface'))).toBe(
|
||||
'View on Hugging Face'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseModelSourceGroupKey', () => {
|
||||
it('parses every external group-key prefix', () => {
|
||||
expect(parseModelSourceGroupKey('hf:user/repo')).toEqual({
|
||||
platform: 'huggingface',
|
||||
label: 'Hugging Face',
|
||||
sourceId: 'user/repo',
|
||||
});
|
||||
expect(parseModelSourceGroupKey('ms:user/repo').platform).toBe('modelscope');
|
||||
expect(parseModelSourceGroupKey('ta:123').platform).toBe('tensorart');
|
||||
});
|
||||
|
||||
it('rejects numeric CivitAI model ids and unknown prefixes', () => {
|
||||
expect(parseModelSourceGroupKey(222)).toBeNull();
|
||||
expect(parseModelSourceGroupKey('222')).toBeNull();
|
||||
expect(parseModelSourceGroupKey('unknown:1')).toBeNull();
|
||||
expect(parseModelSourceGroupKey('')).toBeNull();
|
||||
expect(parseModelSourceGroupKey(null)).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('openModelSource', () => {
|
||||
it('opens the URL in a new tab', () => {
|
||||
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||
openModelSource('https://modelscope.cn/models/u/r');
|
||||
expect(openSpy).toHaveBeenCalledWith(
|
||||
'https://modelscope.cn/models/u/r',
|
||||
'_blank',
|
||||
'noopener,noreferrer'
|
||||
);
|
||||
openSpy.mockRestore();
|
||||
});
|
||||
|
||||
it('does nothing without a URL', () => {
|
||||
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||
openModelSource('');
|
||||
expect(openSpy).not.toHaveBeenCalled();
|
||||
openSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
});
|
||||
+61
-8
@@ -1,14 +1,15 @@
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { DownloadManager } from '../../../static/js/managers/DownloadManager.js';
|
||||
|
||||
describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
describe('DownloadManager.detectUrlType — external model source URLs', () => {
|
||||
|
||||
it('detects HF resolve URL with file', () => {
|
||||
const result = DownloadManager.detectUrlType(
|
||||
'https://huggingface.co/dx8152/Flux2-Klein-9B-Consistency/resolve/main/Flux2-Klein-9B-consistency-V2.safetensors'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'hf-resolve',
|
||||
type: 'model-source-file',
|
||||
platform: 'huggingface',
|
||||
repo: 'dx8152/Flux2-Klein-9B-Consistency',
|
||||
revision: 'main',
|
||||
filename: 'Flux2-Klein-9B-consistency-V2.safetensors',
|
||||
@@ -20,7 +21,8 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
'https://huggingface.co/user/repo/resolve/main/subdir/model.safetensors'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'hf-resolve',
|
||||
type: 'model-source-file',
|
||||
platform: 'huggingface',
|
||||
repo: 'user/repo',
|
||||
revision: 'main',
|
||||
filename: 'subdir/model.safetensors',
|
||||
@@ -32,7 +34,8 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
'https://huggingface.co/dx8152/Flux2-Klein-9B-Consistency'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'hf-repo',
|
||||
type: 'model-source-repo',
|
||||
platform: 'huggingface',
|
||||
repo: 'dx8152/Flux2-Klein-9B-Consistency',
|
||||
});
|
||||
});
|
||||
@@ -40,7 +43,8 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
it('detects HF repo URL (bare user/repo)', () => {
|
||||
const result = DownloadManager.detectUrlType('dx8152/Flux2-Klein-9B-Consistency');
|
||||
expect(result).toEqual({
|
||||
type: 'hf-repo',
|
||||
type: 'model-source-repo',
|
||||
platform: 'huggingface',
|
||||
repo: 'dx8152/Flux2-Klein-9B-Consistency',
|
||||
});
|
||||
});
|
||||
@@ -50,7 +54,8 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
'https://huggingface.co/user/repo/'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'hf-repo',
|
||||
type: 'model-source-repo',
|
||||
platform: 'huggingface',
|
||||
repo: 'user/repo',
|
||||
});
|
||||
});
|
||||
@@ -60,7 +65,8 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
'https://huggingface.co/Comfy-Org/z_image_turbo/blob/main/split_files/diffusion_models/z_image_turbo_bf16.safetensors'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'hf-resolve',
|
||||
type: 'model-source-file',
|
||||
platform: 'huggingface',
|
||||
repo: 'Comfy-Org/z_image_turbo',
|
||||
revision: 'main',
|
||||
filename: 'split_files/diffusion_models/z_image_turbo_bf16.safetensors',
|
||||
@@ -115,7 +121,7 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
const result = DownloadManager.detectUrlType(
|
||||
'https://huggingface.co/user/repo/resolve/main/file.safetensors'
|
||||
);
|
||||
expect(result?.type).toBe('hf-resolve');
|
||||
expect(result?.type).toBe('model-source-file');
|
||||
});
|
||||
|
||||
it('prefers CivitAI over HF when both match', () => {
|
||||
@@ -126,4 +132,51 @@ describe('DownloadManager.detectUrlType — HF URL detection', () => {
|
||||
);
|
||||
expect(result?.type).toBe('civitai');
|
||||
});
|
||||
|
||||
it('detects a ModelScope repo URL', () => {
|
||||
const result = DownloadManager.detectUrlType(
|
||||
'https://modelscope.cn/models/jj3550945163/Krea-2-LORA'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'model-source-repo',
|
||||
platform: 'modelscope',
|
||||
repo: 'jj3550945163/Krea-2-LORA',
|
||||
});
|
||||
});
|
||||
|
||||
it('detects a ModelScope file URL with revision and subdirectory', () => {
|
||||
const result = DownloadManager.detectUrlType(
|
||||
'https://modelscope.cn/models/AI-ModelScope/stable-diffusion-v1-5/resolve/master/vae/diffusion_pytorch_model.bin'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'model-source-file',
|
||||
platform: 'modelscope',
|
||||
repo: 'AI-ModelScope/stable-diffusion-v1-5',
|
||||
revision: 'master',
|
||||
filename: 'vae/diffusion_pytorch_model.bin',
|
||||
});
|
||||
});
|
||||
|
||||
it('detects a ModelScope view sub-page as a repo URL', () => {
|
||||
const result = DownloadManager.detectUrlType(
|
||||
'https://www.modelscope.cn/models/user/repo/summary'
|
||||
);
|
||||
expect(result).toEqual({
|
||||
type: 'model-source-repo',
|
||||
platform: 'modelscope',
|
||||
repo: 'user/repo',
|
||||
});
|
||||
});
|
||||
|
||||
it('does not treat a bare owner/name as ModelScope', () => {
|
||||
// The shorthand has always meant Hugging Face; ModelScope needs its host.
|
||||
const result = DownloadManager.detectUrlType('user/repo');
|
||||
expect(result.platform).toBe('huggingface');
|
||||
});
|
||||
|
||||
it('rejects path traversal in either platform', () => {
|
||||
expect(
|
||||
DownloadManager.detectUrlType('https://modelscope.cn/models/../etc/passwd')
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -1,156 +0,0 @@
|
||||
"""Tests for the HuggingFace link handler (``set_hf_url``).
|
||||
|
||||
Regression coverage for issue #1094: linking a model to HuggingFace must not
|
||||
clear its CivitAI provenance or metadata, so both "View on CivitAI" and
|
||||
"View on Hugging Face" can coexist.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from typing import Any
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from py.routes.handlers import hf_handlers
|
||||
from py.routes.handlers.hf_handlers import HfHandler
|
||||
from py.utils.metadata_manager import MetadataManager
|
||||
|
||||
|
||||
def _json_payload(response) -> dict[str, Any]:
|
||||
assert response.text is not None
|
||||
return json.loads(response.text)
|
||||
|
||||
|
||||
class FakeRequest:
|
||||
def __init__(self, *, json_data=None):
|
||||
self._json_data = json_data or {}
|
||||
|
||||
async def json(self):
|
||||
return self._json_data
|
||||
|
||||
|
||||
def _sidecar_path(model_path) -> str:
|
||||
return f"{os.path.splitext(str(model_path))[0]}.metadata.json"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def hf_env(tmp_path, monkeypatch):
|
||||
"""Point HF linking at *tmp_path* and stub the scanner cache write."""
|
||||
monkeypatch.setattr(hf_handlers, "_find_matching_root", lambda _dir: str(tmp_path))
|
||||
cache_write = AsyncMock()
|
||||
monkeypatch.setattr(hf_handlers, "_add_to_scanner_cache", cache_write)
|
||||
return {"root": tmp_path, "cache_write": cache_write}
|
||||
|
||||
|
||||
async def _write_model(model_path, payload: dict[str, Any]) -> None:
|
||||
model_path.write_bytes(b"x" * 32)
|
||||
await MetadataManager.save_metadata(str(model_path), payload)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_keeps_civitai_metadata_and_provenance(tmp_path, hf_env):
|
||||
model_path = tmp_path / "civitai_model.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "civitai_model",
|
||||
"model_name": "CivitAI Model",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "a" * 64,
|
||||
"base_model": "SDXL 1.0",
|
||||
"preview_url": "",
|
||||
"from_civitai": True,
|
||||
"civitai": {"id": 111, "modelId": 222, "name": "v1", "trainedWords": []},
|
||||
},
|
||||
)
|
||||
|
||||
response = await HfHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"hf_url": "https://huggingface.co/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert _json_payload(response)["success"] is True
|
||||
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["hf_url"] == "https://huggingface.co/user/repo"
|
||||
# Linking HF must not erase the model's CivitAI provenance or data.
|
||||
assert saved["from_civitai"] is True
|
||||
assert saved["civitai"]["modelId"] == 222
|
||||
assert saved["civitai"]["id"] == 111
|
||||
|
||||
hf_env["cache_write"].assert_awaited_once()
|
||||
cached_metadata = hf_env["cache_write"].await_args.args[1]
|
||||
assert cached_metadata["hf_url"] == "https://huggingface.co/user/repo"
|
||||
assert cached_metadata["from_civitai"] is True
|
||||
assert cached_metadata["civitai"]["modelId"] == 222
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_does_not_force_from_civitai_false(tmp_path, hf_env):
|
||||
"""A model without CivitAI data keeps its existing provenance flag."""
|
||||
model_path = tmp_path / "hf_only.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "hf_only",
|
||||
"model_name": "HF Only",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "b" * 64,
|
||||
"base_model": "Unknown",
|
||||
"preview_url": "",
|
||||
"from_civitai": True,
|
||||
},
|
||||
)
|
||||
|
||||
response = await HfHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"hf_url": "https://huggingface.co/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["hf_url"] == "https://huggingface.co/user/repo"
|
||||
assert saved["from_civitai"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_rejects_non_repo_url(tmp_path, hf_env):
|
||||
model_path = tmp_path / "model.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "model",
|
||||
"model_name": "model",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "c" * 64,
|
||||
"base_model": "Unknown",
|
||||
"preview_url": "",
|
||||
},
|
||||
)
|
||||
|
||||
response = await HfHandler().set_hf_url(
|
||||
FakeRequest(json_data={"file_path": str(model_path), "hf_url": "https://example.com/x"})
|
||||
)
|
||||
|
||||
assert response.status == 400
|
||||
payload = _json_payload(response)
|
||||
assert payload["success"] is False
|
||||
hf_env["cache_write"].assert_not_awaited()
|
||||
@@ -0,0 +1,635 @@
|
||||
"""Tests for the external model-source handlers.
|
||||
|
||||
Covers linking (``set_hf_url``), file listing and downloads across the
|
||||
registered platforms (Hugging Face / ModelScope).
|
||||
|
||||
Regression coverage for issue #1094: linking a model to HuggingFace must not
|
||||
clear its CivitAI provenance or metadata, so both "View on CivitAI" and
|
||||
"View on Hugging Face" can coexist.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import AsyncMock
|
||||
|
||||
import pytest
|
||||
|
||||
from py.routes.handlers import model_source_handlers
|
||||
from py.routes.handlers.model_source_handlers import ModelSourceHandler
|
||||
from py.services.model_sources import ModelSourceError, SourceRef
|
||||
from py.services.service_registry import ServiceRegistry
|
||||
from py.utils.models import LoraMetadata
|
||||
from py.utils.metadata_manager import MetadataManager
|
||||
|
||||
|
||||
def _json_payload(response) -> dict[str, Any]:
|
||||
assert response.text is not None
|
||||
return json.loads(response.text)
|
||||
|
||||
|
||||
class FakeRequest:
|
||||
def __init__(self, *, json_data=None, query=None):
|
||||
self._json_data = json_data or {}
|
||||
self.query = query or {}
|
||||
|
||||
async def json(self):
|
||||
return self._json_data
|
||||
|
||||
|
||||
def _sidecar_path(model_path) -> str:
|
||||
return f"{os.path.splitext(str(model_path))[0]}.metadata.json"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def source_env(tmp_path, monkeypatch):
|
||||
"""Point HF linking at *tmp_path* and stub the scanner cache write."""
|
||||
monkeypatch.setattr(model_source_handlers, "_find_matching_root", lambda _dir: str(tmp_path))
|
||||
cache_write = AsyncMock()
|
||||
monkeypatch.setattr(model_source_handlers, "_add_to_scanner_cache", cache_write)
|
||||
return {"root": tmp_path, "cache_write": cache_write}
|
||||
|
||||
|
||||
async def _write_model(model_path, payload: dict[str, Any]) -> None:
|
||||
model_path.write_bytes(b"x" * 32)
|
||||
await MetadataManager.save_metadata(str(model_path), payload)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_keeps_civitai_metadata_and_provenance(tmp_path, source_env):
|
||||
model_path = tmp_path / "civitai_model.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "civitai_model",
|
||||
"model_name": "CivitAI Model",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "a" * 64,
|
||||
"base_model": "SDXL 1.0",
|
||||
"preview_url": "",
|
||||
"from_civitai": True,
|
||||
"civitai": {"id": 111, "modelId": 222, "name": "v1", "trainedWords": []},
|
||||
},
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"hf_url": "https://huggingface.co/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert _json_payload(response)["success"] is True
|
||||
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["hf_url"] == "https://huggingface.co/user/repo"
|
||||
# Linking HF must not erase the model's CivitAI provenance or data.
|
||||
assert saved["from_civitai"] is True
|
||||
assert saved["civitai"]["modelId"] == 222
|
||||
assert saved["civitai"]["id"] == 111
|
||||
|
||||
source_env["cache_write"].assert_awaited_once()
|
||||
cached_metadata = source_env["cache_write"].await_args.args[1]
|
||||
assert cached_metadata["hf_url"] == "https://huggingface.co/user/repo"
|
||||
assert cached_metadata["from_civitai"] is True
|
||||
assert cached_metadata["civitai"]["modelId"] == 222
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_does_not_force_from_civitai_false(tmp_path, source_env):
|
||||
"""A model without CivitAI data keeps its existing provenance flag."""
|
||||
model_path = tmp_path / "hf_only.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "hf_only",
|
||||
"model_name": "HF Only",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "b" * 64,
|
||||
"base_model": "Unknown",
|
||||
"preview_url": "",
|
||||
"from_civitai": True,
|
||||
},
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"hf_url": "https://huggingface.co/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["hf_url"] == "https://huggingface.co/user/repo"
|
||||
assert saved["from_civitai"] is True
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_rejects_non_repo_url(tmp_path, source_env):
|
||||
model_path = tmp_path / "model.safetensors"
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "model",
|
||||
"model_name": "model",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": "c" * 64,
|
||||
"base_model": "Unknown",
|
||||
"preview_url": "",
|
||||
},
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(json_data={"file_path": str(model_path), "hf_url": "https://example.com/x"})
|
||||
)
|
||||
|
||||
assert response.status == 400
|
||||
payload = _json_payload(response)
|
||||
assert payload["success"] is False
|
||||
source_env["cache_write"].assert_not_awaited()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Multi-source linking (ModelScope / TensorArt)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
async def _write_plain_model(model_path, sha: str = "d" * 64) -> None:
|
||||
await _write_model(
|
||||
model_path,
|
||||
{
|
||||
"file_name": "model",
|
||||
"model_name": "model",
|
||||
"file_path": str(model_path),
|
||||
"size": 32,
|
||||
"modified": 1.0,
|
||||
"sha256": sha,
|
||||
"base_model": "Unknown",
|
||||
"preview_url": "",
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_accepts_modelscope_and_stores_source_fields(tmp_path, source_env):
|
||||
model_path = tmp_path / "ms_model.safetensors"
|
||||
await _write_plain_model(model_path)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": "https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
payload = _json_payload(response)
|
||||
assert payload["source_platform"] == "modelscope"
|
||||
assert payload["source_url"] == "https://modelscope.cn/models/jj3550945163/Krea-2-LORA"
|
||||
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["source_platform"] == "modelscope"
|
||||
assert saved["source_url"] == "https://modelscope.cn/models/jj3550945163/Krea-2-LORA"
|
||||
# No stale Hugging Face alias for a ModelScope model.
|
||||
assert saved.get("hf_url", "") == ""
|
||||
|
||||
cached_metadata = source_env["cache_write"].await_args.args[1]
|
||||
assert cached_metadata["source_platform"] == "modelscope"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_accepts_tensorart_url(tmp_path, source_env):
|
||||
model_path = tmp_path / "ta_model.safetensors"
|
||||
await _write_plain_model(model_path, sha="e" * 64)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": (
|
||||
"https://tensor.art/models/827823520299086029/"
|
||||
"Vivid-Impressions-Storybook-Sstyle-V1.0"
|
||||
),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
payload = _json_payload(response)
|
||||
assert payload["source_platform"] == "tensorart"
|
||||
# The canonical page URL is stored, without the slug.
|
||||
assert payload["source_url"] == "https://tensor.art/models/827823520299086029"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_canonicalises_modelscope_subpage(tmp_path, source_env):
|
||||
model_path = tmp_path / "ms_sub.safetensors"
|
||||
await _write_plain_model(model_path, sha="f" * 64)
|
||||
|
||||
response = await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": "https://modelscope.cn/models/user/repo/summary",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert _json_payload(response)["source_url"] == "https://modelscope.cn/models/user/repo"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_is_idempotent_for_modelscope(tmp_path, source_env):
|
||||
model_path = tmp_path / "ms_twice.safetensors"
|
||||
await _write_plain_model(model_path, sha="1" * 64)
|
||||
|
||||
request = FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": "https://modelscope.cn/models/user/repo",
|
||||
}
|
||||
)
|
||||
await ModelSourceHandler().set_hf_url(request)
|
||||
await ModelSourceHandler().set_hf_url(request)
|
||||
|
||||
# The second call short-circuits without rewriting the cache entry.
|
||||
assert source_env["cache_write"].await_count == 1
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_set_hf_url_switching_source_clears_hf_alias(tmp_path, source_env):
|
||||
model_path = tmp_path / "switch.safetensors"
|
||||
await _write_plain_model(model_path, sha="2" * 64)
|
||||
|
||||
await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": "https://huggingface.co/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
await ModelSourceHandler().set_hf_url(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"file_path": str(model_path),
|
||||
"source_url": "https://modelscope.cn/models/user/repo",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["source_platform"] == "modelscope"
|
||||
assert saved.get("hf_url", "") == ""
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_model_sources_lists_capabilities():
|
||||
response = await ModelSourceHandler().get_model_sources(FakeRequest())
|
||||
sources = _json_payload(response)
|
||||
|
||||
by_platform = {s["platform"]: s for s in sources}
|
||||
assert set(by_platform) == {"huggingface", "modelscope", "tensorart"}
|
||||
assert by_platform["huggingface"]["supports_enrichment"] is True
|
||||
assert by_platform["modelscope"]["supports_enrichment"] is True
|
||||
# TensorArt is link-only: no accessible model card for the backend.
|
||||
assert by_platform["tensorart"]["supports_enrichment"] is False
|
||||
assert by_platform["modelscope"]["supports_download"] is True
|
||||
assert by_platform["modelscope"]["default_revision"] == "master"
|
||||
assert by_platform["tensorart"]["supports_download"] is False
|
||||
assert all(s["example_url"] for s in sources)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# File listing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_model_source_files_returns_provider_result(monkeypatch):
|
||||
captured: dict = {}
|
||||
|
||||
async def fake_list_files(self, source_id, revision=""):
|
||||
captured["source_id"] = source_id
|
||||
captured["revision"] = revision
|
||||
return [{"filename": "a.safetensors", "size": 10}]
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.modelscope.ModelScopeSource.list_files",
|
||||
fake_list_files,
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().list_model_source_files(
|
||||
FakeRequest(
|
||||
query={
|
||||
"platform": "modelscope",
|
||||
"repo": "jj3550945163/Krea-2-LORA",
|
||||
"revision": "v1",
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert _json_payload(response) == [{"filename": "a.safetensors", "size": 10}]
|
||||
assert captured == {"source_id": "jj3550945163/Krea-2-LORA", "revision": "v1"}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_model_source_files_rejects_link_only_platform():
|
||||
response = await ModelSourceHandler().list_model_source_files(
|
||||
FakeRequest(query={"platform": "tensorart", "repo": "u/r"})
|
||||
)
|
||||
|
||||
assert response.status == 400
|
||||
assert "does not support downloads" in _json_payload(response)["error"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_model_source_files_rejects_unsafe_repo():
|
||||
for repo in ("noslash", "../etc/passwd", "u/.."):
|
||||
response = await ModelSourceHandler().list_model_source_files(
|
||||
FakeRequest(query={"platform": "modelscope", "repo": repo})
|
||||
)
|
||||
assert response.status == 400, repo
|
||||
assert "repo" in _json_payload(response)["error"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_model_source_files_maps_missing_repo_to_404(monkeypatch):
|
||||
async def fake_list_files(self, source_id, revision=""):
|
||||
raise ModelSourceError(f"Repository '{source_id}' not found", status=404)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.modelscope.ModelScopeSource.list_files",
|
||||
fake_list_files,
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().list_model_source_files(
|
||||
FakeRequest(query={"platform": "modelscope", "repo": "u/r"})
|
||||
)
|
||||
|
||||
assert response.status == 404
|
||||
assert "not found" in _json_payload(response)["error"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_list_model_source_files_maps_transport_failure_to_502(monkeypatch):
|
||||
async def fake_list_files(self, source_id, revision=""):
|
||||
raise ModelSourceError("upstream exploded", status=502)
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.modelscope.ModelScopeSource.list_files",
|
||||
fake_list_files,
|
||||
)
|
||||
|
||||
response = await ModelSourceHandler().list_model_source_files(
|
||||
FakeRequest(query={"platform": "modelscope", "repo": "u/r"})
|
||||
)
|
||||
|
||||
assert response.status == 502
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Downloads
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _stub_download_backend(monkeypatch) -> dict:
|
||||
"""Replace the downloader/settings plumbing with a recording stub."""
|
||||
captured: dict = {}
|
||||
|
||||
async def fake_download_file(**kwargs):
|
||||
captured.update(kwargs)
|
||||
return True, kwargs["save_path"]
|
||||
|
||||
class _Downloader:
|
||||
download_file = staticmethod(fake_download_file)
|
||||
|
||||
async def fake_get_downloader():
|
||||
return _Downloader()
|
||||
|
||||
class _Settings:
|
||||
def get(self, key, default=None):
|
||||
return default
|
||||
|
||||
monkeypatch.setattr(model_source_handlers, "get_downloader", fake_get_downloader)
|
||||
monkeypatch.setattr(
|
||||
model_source_handlers, "get_settings_manager", lambda: _Settings()
|
||||
)
|
||||
return captured
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_modelscope_uses_resolve_url(tmp_path, monkeypatch):
|
||||
captured = _stub_download_backend(monkeypatch)
|
||||
saved = AsyncMock()
|
||||
monkeypatch.setattr(model_source_handlers, "_save_source_metadata", saved)
|
||||
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"platform": "modelscope",
|
||||
"repo": "jj3550945163/Krea-2-LORA",
|
||||
"filename": "Krea-2-LORA_c1-st1000.safetensors",
|
||||
"model_root": str(tmp_path),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert captured["url"] == (
|
||||
"https://modelscope.cn/models/jj3550945163/Krea-2-LORA/resolve/master/"
|
||||
"Krea-2-LORA_c1-st1000.safetensors"
|
||||
)
|
||||
assert captured["save_path"] == str(tmp_path / "Krea-2-LORA_c1-st1000.safetensors")
|
||||
|
||||
ref = saved.await_args.args[1]
|
||||
assert ref.platform == "modelscope"
|
||||
assert ref.source_id == "jj3550945163/Krea-2-LORA"
|
||||
assert ref.url == "https://modelscope.cn/models/jj3550945163/Krea-2-LORA"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_modelscope_default_paths(tmp_path, monkeypatch):
|
||||
captured = _stub_download_backend(monkeypatch)
|
||||
saved = AsyncMock()
|
||||
monkeypatch.setattr(model_source_handlers, "_save_source_metadata", saved)
|
||||
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"platform": "modelscope",
|
||||
"repo": "owner/name",
|
||||
"filename": "nested/model.safetensors",
|
||||
"model_root": str(tmp_path),
|
||||
"use_default_paths": True,
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
# The site gets its own sub-directory, mirroring `huggingface/<owner>/<repo>`.
|
||||
assert captured["save_path"] == str(
|
||||
tmp_path / "modelscope" / "owner" / "name" / "model.safetensors"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_defaults_to_huggingface(tmp_path, monkeypatch):
|
||||
"""The legacy /api/lm/download-hf-model payload has no `platform` key."""
|
||||
captured = _stub_download_backend(monkeypatch)
|
||||
monkeypatch.setattr(model_source_handlers, "_save_source_metadata", AsyncMock())
|
||||
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"repo": "user/repo",
|
||||
"filename": "f.safetensors",
|
||||
"revision": "main",
|
||||
"model_root": str(tmp_path),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert captured["url"] == (
|
||||
"https://huggingface.co/user/repo/resolve/main/f.safetensors"
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_rejects_link_only_platform(tmp_path):
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"platform": "tensorart",
|
||||
"repo": "u/r",
|
||||
"filename": "f.safetensors",
|
||||
"model_root": str(tmp_path),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 400
|
||||
assert "does not support downloads" in _json_payload(response)["error"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_rejects_unsafe_input(tmp_path, monkeypatch):
|
||||
_stub_download_backend(monkeypatch)
|
||||
|
||||
cases = [
|
||||
({"repo": "noslash", "filename": "f.safetensors"}, "repo format"),
|
||||
({"repo": "u/r", "filename": "../../etc/passwd"}, "Invalid filename"),
|
||||
(
|
||||
{"repo": "u/r", "filename": "f.safetensors", "relative_path": "/abs"},
|
||||
"relative_path must not be absolute",
|
||||
),
|
||||
(
|
||||
{"repo": "u/r", "filename": "f.safetensors", "relative_path": "../up"},
|
||||
"Invalid relative_path",
|
||||
),
|
||||
]
|
||||
for extra, expected in cases:
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"platform": "modelscope",
|
||||
"model_root": str(tmp_path),
|
||||
**extra,
|
||||
}
|
||||
)
|
||||
)
|
||||
assert response.status == 400, extra
|
||||
assert expected in _json_payload(response)["error"], extra
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_download_model_source_skips_existing_file(tmp_path, monkeypatch):
|
||||
captured = _stub_download_backend(monkeypatch)
|
||||
monkeypatch.setattr(model_source_handlers, "_save_source_metadata", AsyncMock())
|
||||
(tmp_path / "f.safetensors").write_bytes(b"already here")
|
||||
|
||||
response = await ModelSourceHandler().download_model_source(
|
||||
FakeRequest(
|
||||
json_data={
|
||||
"platform": "modelscope",
|
||||
"repo": "u/r",
|
||||
"filename": "f.safetensors",
|
||||
"model_root": str(tmp_path),
|
||||
}
|
||||
)
|
||||
)
|
||||
|
||||
assert response.status == 200
|
||||
assert "already exists" in _json_payload(response)["message"]
|
||||
assert captured == {}
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.parametrize(
|
||||
("platform", "url", "expect_hf_alias"),
|
||||
[
|
||||
("modelscope", "https://modelscope.cn/models/u/r", False),
|
||||
("huggingface", "https://huggingface.co/u/r", True),
|
||||
],
|
||||
)
|
||||
async def test_save_source_metadata_writes_platform_fields(
|
||||
tmp_path, monkeypatch, platform, url, expect_hf_alias
|
||||
):
|
||||
"""A download's sidecar must record its own platform (and no stale HF alias)."""
|
||||
model_path = tmp_path / "downloaded.safetensors"
|
||||
model_path.write_bytes(b"x" * 32)
|
||||
|
||||
metadata = LoraMetadata(
|
||||
file_name="downloaded",
|
||||
model_name="Downloaded",
|
||||
file_path=str(model_path),
|
||||
size=32,
|
||||
modified=1.0,
|
||||
sha256="a" * 64,
|
||||
base_model="SDXL 1.0",
|
||||
preview_url="",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
model_source_handlers.MetadataManager,
|
||||
"create_default_metadata",
|
||||
AsyncMock(return_value=metadata),
|
||||
)
|
||||
scanner = SimpleNamespace(add_model_to_cache=AsyncMock())
|
||||
monkeypatch.setattr(
|
||||
ServiceRegistry, "get_lora_scanner", AsyncMock(return_value=scanner)
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
model_source_handlers, "_infer_model_type", lambda _root: (LoraMetadata, "get_lora_scanner")
|
||||
)
|
||||
|
||||
ref = SourceRef(platform=platform, source_id="u/r", url=url)
|
||||
await model_source_handlers._save_source_metadata(str(model_path), ref, str(tmp_path))
|
||||
|
||||
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||
assert saved["source_platform"] == platform
|
||||
assert saved["source_url"] == url
|
||||
assert bool(saved.get("hf_url", "")) is expect_hf_alias
|
||||
|
||||
cached = scanner.add_model_to_cache.await_args.args[0]
|
||||
assert cached["source_platform"] == platform
|
||||
assert cached["source_url"] == url
|
||||
@@ -0,0 +1,182 @@
|
||||
"""Tests for source-aware AI enrichment orchestration.
|
||||
|
||||
Covers the fast-fail gate (:meth:`AgentService._enrichment_skip_reason`) and
|
||||
the prompt-context builder for non-Hugging Face model sources.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from py.services.agent.agent_service import AgentService
|
||||
|
||||
|
||||
class TestEnrichmentSkipReason:
|
||||
def test_skips_when_no_source_linked(self):
|
||||
reason = AgentService._enrichment_skip_reason({})
|
||||
assert "source_url" in reason
|
||||
|
||||
def test_allows_huggingface(self):
|
||||
assert (
|
||||
AgentService._enrichment_skip_reason(
|
||||
{"hf_url": "https://huggingface.co/user/repo"}
|
||||
)
|
||||
== ""
|
||||
)
|
||||
|
||||
def test_allows_modelscope(self):
|
||||
assert (
|
||||
AgentService._enrichment_skip_reason(
|
||||
{
|
||||
"source_platform": "modelscope",
|
||||
"source_url": "https://modelscope.cn/models/user/repo",
|
||||
}
|
||||
)
|
||||
== ""
|
||||
)
|
||||
|
||||
def test_skips_tensorart_with_reason(self):
|
||||
reason = AgentService._enrichment_skip_reason(
|
||||
{
|
||||
"source_platform": "tensorart",
|
||||
"source_url": "https://tensor.art/models/827823520299086029",
|
||||
}
|
||||
)
|
||||
assert "TensorArt" in reason
|
||||
assert "not available" in reason
|
||||
|
||||
def test_skips_unknown_platform(self):
|
||||
reason = AgentService._enrichment_skip_reason(
|
||||
{"source_platform": "somewhere", "source_url": "https://somewhere.example/m/1"}
|
||||
)
|
||||
assert "somewhere" in reason
|
||||
|
||||
|
||||
class TestBuildPromptContext:
|
||||
@pytest.mark.asyncio
|
||||
async def test_modelscope_card_populates_source_variables(self):
|
||||
service = AgentService()
|
||||
readme = "---\nbase_model: krea/Krea-2-Turbo\n---\n# krea\n"
|
||||
|
||||
with (
|
||||
mock.patch(
|
||||
"py.services.model_sources.modelscope.ModelScopeSource.fetch_model_card",
|
||||
new=mock.AsyncMock(return_value=readme),
|
||||
) as mock_fetch,
|
||||
mock.patch(
|
||||
"py.metadata_ops.list_base_models",
|
||||
new=mock.AsyncMock(return_value=["Krea 2 Turbo"]),
|
||||
),
|
||||
mock.patch(
|
||||
"py.metadata_ops.identify_model_type",
|
||||
new=mock.AsyncMock(return_value="lora"),
|
||||
),
|
||||
mock.patch(
|
||||
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||
return_value={"lora": "style, subject"},
|
||||
),
|
||||
):
|
||||
context = await service._build_prompt_context(
|
||||
skill_name="enrich_hf_metadata",
|
||||
model_path="/models/loras/krea.safetensors",
|
||||
metadata={
|
||||
"source_platform": "modelscope",
|
||||
"source_url": "https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||
"file_name": "krea",
|
||||
},
|
||||
registry=mock.Mock(),
|
||||
llm=mock.Mock(),
|
||||
)
|
||||
|
||||
mock_fetch.assert_awaited_once_with("jj3550945163/Krea-2-LORA")
|
||||
assert context["source_platform"] == "modelscope"
|
||||
assert context["source_id"] == "jj3550945163/Krea-2-LORA"
|
||||
assert context["source_label"] == "ModelScope"
|
||||
assert (
|
||||
context["asset_base_url"]
|
||||
== "https://modelscope.cn/models/jj3550945163/Krea-2-LORA/resolve/master"
|
||||
)
|
||||
assert readme in context["readme_content_full"]
|
||||
# Hugging Face aliases stay empty for a non-HF source.
|
||||
assert context["hf_url"] == ""
|
||||
assert context["repo"] == "jj3550945163/Krea-2-LORA"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_huggingface_keeps_legacy_aliases(self):
|
||||
service = AgentService()
|
||||
readme = "# card\n"
|
||||
|
||||
with (
|
||||
mock.patch(
|
||||
"py.services.model_sources.huggingface.HuggingFaceSource.fetch_model_card",
|
||||
new=mock.AsyncMock(return_value=readme),
|
||||
) as mock_fetch,
|
||||
mock.patch(
|
||||
"py.metadata_ops.list_base_models",
|
||||
new=mock.AsyncMock(return_value=[]),
|
||||
),
|
||||
mock.patch(
|
||||
"py.metadata_ops.identify_model_type",
|
||||
new=mock.AsyncMock(return_value="lora"),
|
||||
),
|
||||
mock.patch(
|
||||
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||
return_value={},
|
||||
),
|
||||
):
|
||||
context = await service._build_prompt_context(
|
||||
skill_name="enrich_hf_metadata",
|
||||
model_path="/models/loras/thing.safetensors",
|
||||
metadata={"hf_url": "https://huggingface.co/user/repo"},
|
||||
registry=mock.Mock(),
|
||||
llm=mock.Mock(),
|
||||
)
|
||||
|
||||
mock_fetch.assert_awaited_once_with("user/repo")
|
||||
assert context["source_platform"] == "huggingface"
|
||||
assert context["hf_url"] == "https://huggingface.co/user/repo"
|
||||
assert context["repo"] == "user/repo"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tensorart_never_fetches_a_card(self):
|
||||
service = AgentService()
|
||||
|
||||
with (
|
||||
mock.patch(
|
||||
"py.services.model_sources.huggingface.HuggingFaceSource.fetch_model_card",
|
||||
new=mock.AsyncMock(),
|
||||
) as hf_fetch,
|
||||
mock.patch(
|
||||
"py.services.model_sources.modelscope.ModelScopeSource.fetch_model_card",
|
||||
new=mock.AsyncMock(),
|
||||
) as ms_fetch,
|
||||
mock.patch(
|
||||
"py.metadata_ops.list_base_models",
|
||||
new=mock.AsyncMock(return_value=[]),
|
||||
),
|
||||
mock.patch(
|
||||
"py.metadata_ops.identify_model_type",
|
||||
new=mock.AsyncMock(return_value="lora"),
|
||||
),
|
||||
mock.patch(
|
||||
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||
return_value={},
|
||||
),
|
||||
):
|
||||
context = await service._build_prompt_context(
|
||||
skill_name="enrich_hf_metadata",
|
||||
model_path="/models/loras/thing.safetensors",
|
||||
metadata={
|
||||
"source_platform": "tensorart",
|
||||
"source_url": "https://tensor.art/models/827823520299086029",
|
||||
},
|
||||
registry=mock.Mock(),
|
||||
llm=mock.Mock(),
|
||||
)
|
||||
|
||||
hf_fetch.assert_not_awaited()
|
||||
ms_fetch.assert_not_awaited()
|
||||
assert context["readme_content"] == ""
|
||||
assert context["source_platform"] == "tensorart"
|
||||
@@ -76,6 +76,25 @@ class MockSession:
|
||||
pass
|
||||
|
||||
|
||||
class RecordingSession:
|
||||
"""Mock session that records each request payload and replays responses."""
|
||||
|
||||
def __init__(self, responses):
|
||||
self._responses = list(responses)
|
||||
self.payloads = []
|
||||
|
||||
def post(self, url, json=None, headers=None):
|
||||
self.payloads.append(json)
|
||||
index = min(len(self.payloads) - 1, len(self._responses) - 1)
|
||||
return self._responses[index]
|
||||
|
||||
async def __aenter__(self):
|
||||
return self
|
||||
|
||||
async def __aexit__(self, *args):
|
||||
pass
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def llm_service():
|
||||
"""Create an LLMService with mock settings."""
|
||||
@@ -298,6 +317,107 @@ class TestLLMServiceChatCompletionJson:
|
||||
assert result == {"key": "value"}
|
||||
assert call_index == 2
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_chat_completion_json_prefers_json_object_for_deepseek(self):
|
||||
"""DeepSeek rejects json_schema, so json_object is used first.
|
||||
|
||||
Regression: DeepSeek answers json_schema with
|
||||
"This response_format type is unavailable now", which the old
|
||||
substring check did not recognise, so enrichment failed outright.
|
||||
"""
|
||||
settings = MockSettings(
|
||||
llm_enabled=True,
|
||||
llm_provider="deepseek",
|
||||
llm_api_key="sk-test-key",
|
||||
llm_api_base="https://api.deepseek.com/v1",
|
||||
llm_model="deepseek-v4-flash",
|
||||
)
|
||||
service = LLMService(settings)
|
||||
|
||||
session = RecordingSession(
|
||||
[
|
||||
MockResponse(
|
||||
200,
|
||||
json_data={
|
||||
"choices": [{"message": {"content": '{"key": "value"}'}}],
|
||||
"usage": {},
|
||||
},
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
with mock.patch("aiohttp.ClientSession", return_value=session):
|
||||
result = await service.chat_completion_json(
|
||||
system_prompt="You are helpful.",
|
||||
user_prompt="Return JSON.",
|
||||
)
|
||||
|
||||
assert result == {"key": "value"}
|
||||
assert len(session.payloads) == 1
|
||||
assert session.payloads[0]["response_format"] == {"type": "json_object"}
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_chat_completion_json_downgrades_from_json_schema(
|
||||
self, llm_service,
|
||||
):
|
||||
"""json_schema → json_object when the provider rejects json_schema."""
|
||||
session = RecordingSession(
|
||||
[
|
||||
MockResponse(
|
||||
400,
|
||||
text_data=(
|
||||
'{"error":{"message":"This response_format type is '
|
||||
'unavailable now","type":"invalid_request_error"}}'
|
||||
),
|
||||
),
|
||||
MockResponse(
|
||||
200,
|
||||
json_data={
|
||||
"choices": [{"message": {"content": '{"key": "value"}'}}],
|
||||
"usage": {},
|
||||
},
|
||||
),
|
||||
]
|
||||
)
|
||||
|
||||
with mock.patch("aiohttp.ClientSession", return_value=session):
|
||||
result = await llm_service.chat_completion_json(
|
||||
system_prompt="You are helpful.",
|
||||
user_prompt="Return JSON.",
|
||||
)
|
||||
|
||||
assert result == {"key": "value"}
|
||||
assert [p.get("response_format") for p in session.payloads] == [
|
||||
{
|
||||
"type": "json_schema",
|
||||
"json_schema": {"name": "metadata", "schema": {"type": "object"}},
|
||||
},
|
||||
{"type": "json_object"},
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_chat_completion_json_does_not_retry_unrelated_errors(
|
||||
self, llm_service,
|
||||
):
|
||||
"""Unrelated 400s are surfaced unchanged, without format downgrades."""
|
||||
session = RecordingSession(
|
||||
[
|
||||
MockResponse(
|
||||
400,
|
||||
text_data='{"error":{"message":"Model not found"}}',
|
||||
)
|
||||
]
|
||||
)
|
||||
|
||||
with mock.patch("aiohttp.ClientSession", return_value=session):
|
||||
with pytest.raises(LLMResponseError, match="HTTP 400"):
|
||||
await llm_service.chat_completion_json(
|
||||
system_prompt="You are helpful.",
|
||||
user_prompt="Return JSON.",
|
||||
)
|
||||
|
||||
assert len(session.payloads) == 1
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_chat_completion_json_raises_on_non_json(self, llm_service):
|
||||
# Non-JSON content raises LLMResponseError (salvage also fails)
|
||||
|
||||
@@ -964,6 +964,8 @@ def _make_cache_entry(**overrides) -> Dict[str, Any]:
|
||||
"civitai": {"id": 111, "modelId": 222, "name": "v1"},
|
||||
"civitai_deleted": False,
|
||||
"skip_metadata_refresh": False,
|
||||
"source_platform": "",
|
||||
"source_url": "",
|
||||
"hf_url": "",
|
||||
"license_flags": 113,
|
||||
"hash_status": "completed",
|
||||
|
||||
@@ -0,0 +1,506 @@
|
||||
"""Tests for the external model-source provider registry.
|
||||
|
||||
Covers URL recognition for Hugging Face / ModelScope / TensorArt, the
|
||||
legacy ``hf_url`` → ``source_url`` normalisation, version-group keys, and
|
||||
each provider's model-card fetching and capability flags.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from py.services.model_sources import (
|
||||
HuggingFaceSource,
|
||||
ModelScopeSource,
|
||||
TensorArtSource,
|
||||
detect_source,
|
||||
downloadable_sources,
|
||||
get_download_source,
|
||||
get_source,
|
||||
get_source_platform,
|
||||
has_external_source,
|
||||
is_valid_source_id,
|
||||
list_sources,
|
||||
ModelSourceError,
|
||||
normalize_metadata_source,
|
||||
resolve_source_ref,
|
||||
source_group_key,
|
||||
source_label,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# URL recognition
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestDetectSource:
|
||||
@pytest.mark.parametrize(
|
||||
("url", "platform", "source_id"),
|
||||
[
|
||||
("https://huggingface.co/user/repo", "huggingface", "user/repo"),
|
||||
("https://www.huggingface.co/user/repo", "huggingface", "user/repo"),
|
||||
(
|
||||
"https://huggingface.co/user/repo/resolve/main/model.safetensors",
|
||||
"huggingface",
|
||||
"user/repo",
|
||||
),
|
||||
(
|
||||
"https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||
"modelscope",
|
||||
"jj3550945163/Krea-2-LORA",
|
||||
),
|
||||
(
|
||||
"https://www.modelscope.cn/models/jj3550945163/Krea-2-LORA/summary",
|
||||
"modelscope",
|
||||
"jj3550945163/Krea-2-LORA",
|
||||
),
|
||||
(
|
||||
"https://tensor.art/models/827823520299086029/Vivid-Impressions-Storybook-Sstyle-V1.0",
|
||||
"tensorart",
|
||||
"827823520299086029",
|
||||
),
|
||||
("https://tusi.cn/models/827823520299086029", "tensorart", "827823520299086029"),
|
||||
],
|
||||
)
|
||||
def test_recognises_supported_urls(self, url, platform, source_id):
|
||||
ref = detect_source(url)
|
||||
assert ref is not None
|
||||
assert ref.platform == platform
|
||||
assert ref.source_id == source_id
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"url",
|
||||
[
|
||||
"",
|
||||
None,
|
||||
"not-a-url",
|
||||
"https://example.com/x",
|
||||
"https://civitai.com/models/123",
|
||||
],
|
||||
)
|
||||
def test_ignores_unsupported_urls(self, url):
|
||||
assert detect_source(url) is None
|
||||
|
||||
def test_canonical_url_is_stable(self):
|
||||
assert detect_source("https://huggingface.co/u/r").url == "https://huggingface.co/u/r"
|
||||
assert (
|
||||
detect_source("https://modelscope.cn/models/u/r/summary").url
|
||||
== "https://modelscope.cn/models/u/r"
|
||||
)
|
||||
assert (
|
||||
detect_source("https://tensor.art/models/123/some-slug").url
|
||||
== "https://tensor.art/models/123"
|
||||
)
|
||||
|
||||
|
||||
class TestStrictParsing:
|
||||
@pytest.mark.parametrize(
|
||||
"url",
|
||||
[
|
||||
"https://huggingface.co/user/repo",
|
||||
"https://huggingface.co/user/repo/",
|
||||
"https://modelscope.cn/models/user/repo",
|
||||
"https://modelscope.cn/models/user/repo/summary",
|
||||
"https://tensor.art/models/827823520299086029",
|
||||
"https://tensor.art/models/827823520299086029/Vivid-Impressions",
|
||||
],
|
||||
)
|
||||
def test_accepts_user_facing_urls(self, url):
|
||||
assert detect_source(url, strict=True) is not None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"url",
|
||||
[
|
||||
"https://huggingface.co/user/repo/resolve/main/model.safetensors",
|
||||
"https://example.com/x",
|
||||
"https://tensor.art/models/not-a-number",
|
||||
],
|
||||
)
|
||||
def test_rejects_non_page_urls(self, url):
|
||||
assert detect_source(url, strict=True) is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Capabilities
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestCapabilities:
|
||||
def test_huggingface_supports_everything(self):
|
||||
source = get_source("huggingface")
|
||||
assert source.supports_enrichment is True
|
||||
assert source.supports_download is True
|
||||
assert source.default_revision == "main"
|
||||
assert source.default_subdir == "huggingface"
|
||||
|
||||
def test_modelscope_supports_enrichment_and_download(self):
|
||||
source = get_source("modelscope")
|
||||
assert source.supports_enrichment is True
|
||||
assert source.supports_download is True
|
||||
assert source.default_revision == "master"
|
||||
assert source.default_subdir == "modelscope"
|
||||
|
||||
def test_tensorart_is_link_only(self):
|
||||
source = get_source("tensorart")
|
||||
assert source.supports_enrichment is False
|
||||
assert source.supports_download is False
|
||||
|
||||
def test_registry_lists_every_source(self):
|
||||
platforms = {s.platform for s in list_sources()}
|
||||
assert platforms == {"huggingface", "modelscope", "tensorart"}
|
||||
|
||||
def test_labels_are_brand_names(self):
|
||||
assert source_label("huggingface") == "Hugging Face"
|
||||
assert source_label("modelscope") == "ModelScope"
|
||||
assert source_label("tensorart") == "TensorArt"
|
||||
assert source_label("unknown", "fallback") == "fallback"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Metadata normalisation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestNormalizeMetadataSource:
|
||||
def test_derives_source_fields_from_legacy_hf_url(self):
|
||||
metadata = {"hf_url": "https://huggingface.co/user/repo"}
|
||||
normalize_metadata_source(metadata)
|
||||
assert metadata["source_platform"] == "huggingface"
|
||||
assert metadata["source_url"] == "https://huggingface.co/user/repo"
|
||||
assert metadata["hf_url"] == "https://huggingface.co/user/repo"
|
||||
|
||||
def test_canonicalises_modelscope_url_and_clears_hf_alias(self):
|
||||
metadata = {
|
||||
"source_platform": "modelscope",
|
||||
"source_url": "https://modelscope.cn/models/user/repo/summary",
|
||||
"hf_url": "https://huggingface.co/old/repo",
|
||||
}
|
||||
normalize_metadata_source(metadata)
|
||||
assert metadata["source_platform"] == "modelscope"
|
||||
assert metadata["source_url"] == "https://modelscope.cn/models/user/repo"
|
||||
# A stale HF alias must not make a ModelScope model look like HF.
|
||||
assert metadata["hf_url"] == ""
|
||||
|
||||
def test_preserves_unknown_url_for_unknown_platform(self):
|
||||
metadata = {"source_url": "https://example.com/model/1", "source_platform": "other"}
|
||||
normalize_metadata_source(metadata)
|
||||
assert metadata["source_url"] == "https://example.com/model/1"
|
||||
assert metadata["source_platform"] == "other"
|
||||
|
||||
def test_empty_metadata_gets_default_fields(self):
|
||||
metadata: dict = {}
|
||||
normalize_metadata_source(metadata)
|
||||
assert metadata["source_platform"] == ""
|
||||
assert metadata["source_url"] == ""
|
||||
|
||||
def test_infers_platform_from_url_when_missing(self):
|
||||
metadata = {"source_url": "https://modelscope.cn/models/user/repo"}
|
||||
normalize_metadata_source(metadata)
|
||||
assert metadata["source_platform"] == "modelscope"
|
||||
|
||||
|
||||
class TestResolveSourceRef:
|
||||
def test_resolves_from_canonical_fields(self):
|
||||
ref = resolve_source_ref(
|
||||
{"source_platform": "modelscope", "source_url": "https://modelscope.cn/models/u/r"}
|
||||
)
|
||||
assert ref is not None
|
||||
assert ref.platform == "modelscope"
|
||||
assert ref.source_id == "u/r"
|
||||
|
||||
def test_resolves_from_legacy_hf_url(self):
|
||||
ref = resolve_source_ref({"hf_url": "https://huggingface.co/u/r"})
|
||||
assert ref is not None
|
||||
assert ref.platform == "huggingface"
|
||||
|
||||
def test_returns_none_without_any_source(self):
|
||||
assert resolve_source_ref({}) is None
|
||||
assert resolve_source_ref({"hf_url": ""}) is None
|
||||
|
||||
|
||||
class TestHelpers:
|
||||
def test_has_external_source_accepts_both_field_shapes(self):
|
||||
assert has_external_source({"hf_url": "https://huggingface.co/u/r"}) is True
|
||||
assert has_external_source({"source_url": "https://modelscope.cn/models/u/r"}) is True
|
||||
assert has_external_source({"source_url": ""}) is False
|
||||
assert has_external_source({}) is False
|
||||
|
||||
def test_get_source_platform_infers_from_url(self):
|
||||
assert get_source_platform({"hf_url": "https://huggingface.co/u/r"}) == "huggingface"
|
||||
assert get_source_platform({"source_platform": "tensorart"}) == "tensorart"
|
||||
assert get_source_platform({}) == ""
|
||||
|
||||
def test_group_keys_match_legacy_hf_shape(self):
|
||||
assert source_group_key({"hf_url": "https://huggingface.co/u/r"}) == "hf:u/r"
|
||||
assert (
|
||||
source_group_key({"source_url": "https://modelscope.cn/models/u/r"}) == "ms:u/r"
|
||||
)
|
||||
assert (
|
||||
source_group_key({"source_url": "https://tensor.art/models/123"}) == "ta:123"
|
||||
)
|
||||
|
||||
def test_group_key_is_none_without_source(self):
|
||||
assert source_group_key({}) is None
|
||||
assert source_group_key({"hf_url": "https://example.com/x"}) is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model card fetching
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestFetchModelCard:
|
||||
@pytest.mark.asyncio
|
||||
async def test_huggingface_tries_main_then_master(self, monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||
calls.append(url)
|
||||
if url.endswith("/master/README.md"):
|
||||
return "# card"
|
||||
return ""
|
||||
|
||||
monkeypatch.setattr("py.services.model_sources.huggingface.fetch_text", fake_fetch_text)
|
||||
|
||||
card = await HuggingFaceSource().fetch_model_card("user/repo")
|
||||
|
||||
assert card == "# card"
|
||||
assert calls == [
|
||||
"https://huggingface.co/user/repo/raw/main/README.md",
|
||||
"https://huggingface.co/user/repo/raw/master/README.md",
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_modelscope_prefers_resolve_url(self, monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||
calls.append(url)
|
||||
return "---\nbase_model: krea/Krea-2-Turbo\n---\n# krea"
|
||||
|
||||
monkeypatch.setattr("py.services.model_sources.modelscope.fetch_text", fake_fetch_text)
|
||||
|
||||
card = await ModelScopeSource().fetch_model_card("u/r")
|
||||
|
||||
assert card.startswith("---")
|
||||
assert calls == ["https://modelscope.cn/models/u/r/resolve/master/README.md"]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_modelscope_falls_back_to_repo_api(self, monkeypatch):
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||
calls.append(url)
|
||||
if "/api/v1/models/" in url:
|
||||
return "# from api"
|
||||
return ""
|
||||
|
||||
monkeypatch.setattr("py.services.model_sources.modelscope.fetch_text", fake_fetch_text)
|
||||
|
||||
card = await ModelScopeSource().fetch_model_card("u/r")
|
||||
|
||||
assert card == "# from api"
|
||||
assert "resolve/master/README.md" in calls[0]
|
||||
assert (
|
||||
"https://modelscope.cn/api/v1/models/u/r/repo?Revision=master&FilePath=README.md"
|
||||
in calls
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tensorart_never_fetches(self):
|
||||
# TensorArt enrichment is disabled: the provider must not issue any
|
||||
# HTTP request, so it deliberately does not import `fetch_text`.
|
||||
import importlib
|
||||
|
||||
module = importlib.import_module("py.services.model_sources.tensorart")
|
||||
assert not hasattr(module, "fetch_text")
|
||||
assert await TensorArtSource().fetch_model_card("123") == ""
|
||||
|
||||
|
||||
class TestAssetBaseUrl:
|
||||
def test_huggingface_uses_main_revision(self):
|
||||
assert (
|
||||
HuggingFaceSource().asset_base_url("u/r")
|
||||
== "https://huggingface.co/u/r/resolve/main"
|
||||
)
|
||||
|
||||
def test_modelscope_uses_master_revision(self):
|
||||
assert (
|
||||
ModelScopeSource().asset_base_url("u/r")
|
||||
== "https://modelscope.cn/models/u/r/resolve/master"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Download support
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class TestListFiles:
|
||||
@pytest.mark.asyncio
|
||||
async def test_huggingface_reads_tree_api_with_lfs_sizes(self, monkeypatch):
|
||||
captured: dict = {}
|
||||
|
||||
async def fake_fetch_json(url, **_kwargs):
|
||||
captured["url"] = url
|
||||
return 200, [
|
||||
{"path": "README.md", "size": 120},
|
||||
{"path": "a/model.safetensors", "size": 300},
|
||||
{"path": "b.safetensors", "size": 0, "lfs": {"size": 200}},
|
||||
]
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.huggingface.fetch_json", fake_fetch_json
|
||||
)
|
||||
|
||||
files = await HuggingFaceSource().list_files("u/r")
|
||||
|
||||
assert captured["url"] == "https://huggingface.co/api/models/u/r/tree/main"
|
||||
assert files == [
|
||||
{"filename": "a/model.safetensors", "size": 300},
|
||||
{"filename": "b.safetensors", "size": 200},
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_huggingface_honours_explicit_revision(self, monkeypatch):
|
||||
captured: dict = {}
|
||||
|
||||
async def fake_fetch_json(url, **_kwargs):
|
||||
captured["url"] = url
|
||||
return 200, []
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.huggingface.fetch_json", fake_fetch_json
|
||||
)
|
||||
|
||||
await HuggingFaceSource().list_files("u/r", "v2.0")
|
||||
|
||||
assert captured["url"].endswith("/tree/v2.0")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_modelscope_reads_repo_files_api(self, monkeypatch):
|
||||
captured: dict = {}
|
||||
|
||||
async def fake_fetch_json(url, **_kwargs):
|
||||
captured["url"] = url
|
||||
return 200, {
|
||||
"Data": {
|
||||
"Files": [
|
||||
# directories are listed too and must be dropped
|
||||
{"Type": "tree", "Path": "vae", "Size": 0},
|
||||
{"Type": "blob", "Path": "README.md", "Size": 100},
|
||||
{"Type": "blob", "Path": "sub/model.safetensors", "Size": 500},
|
||||
{"Type": "blob", "Path": "model.ckpt", "Size": 200},
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.modelscope.fetch_json", fake_fetch_json
|
||||
)
|
||||
|
||||
files = await ModelScopeSource().list_files("u/r")
|
||||
|
||||
assert captured["url"] == (
|
||||
"https://modelscope.cn/api/v1/models/u/r/repo/files?Revision=master"
|
||||
)
|
||||
assert files == [
|
||||
{"filename": "sub/model.safetensors", "size": 500},
|
||||
{"filename": "model.ckpt", "size": 200},
|
||||
]
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_missing_repo_is_reported_as_not_found(self, monkeypatch):
|
||||
async def fake_fetch_json(url, **_kwargs):
|
||||
return 404, None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.modelscope.fetch_json", fake_fetch_json
|
||||
)
|
||||
|
||||
with pytest.raises(ModelSourceError) as excinfo:
|
||||
await ModelScopeSource().list_files("u/r")
|
||||
|
||||
assert excinfo.value.status == 404
|
||||
assert "not found" in str(excinfo.value)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_transport_failure_is_reported_as_bad_gateway(self, monkeypatch):
|
||||
async def fake_fetch_json(url, **_kwargs):
|
||||
return 0, None
|
||||
|
||||
monkeypatch.setattr(
|
||||
"py.services.model_sources.huggingface.fetch_json", fake_fetch_json
|
||||
)
|
||||
|
||||
with pytest.raises(ModelSourceError) as excinfo:
|
||||
await HuggingFaceSource().list_files("u/r")
|
||||
|
||||
assert excinfo.value.status == 502
|
||||
|
||||
|
||||
class TestDownloadUrls:
|
||||
def test_huggingface_resolve_url(self):
|
||||
assert HuggingFaceSource().file_download_url("u/r", "sub/f.safetensors") == (
|
||||
"https://huggingface.co/u/r/resolve/main/sub/f.safetensors"
|
||||
)
|
||||
|
||||
def test_modelscope_resolve_url_defaults_to_master(self):
|
||||
assert ModelScopeSource().file_download_url("u/r", "sub/f.safetensors") == (
|
||||
"https://modelscope.cn/models/u/r/resolve/master/sub/f.safetensors"
|
||||
)
|
||||
|
||||
def test_explicit_revision_wins(self):
|
||||
assert ModelScopeSource().file_download_url("u/r", "f.bin", "v1") == (
|
||||
"https://modelscope.cn/models/u/r/resolve/v1/f.bin"
|
||||
)
|
||||
|
||||
def test_tensorart_refuses_to_build_a_download_url(self):
|
||||
source = TensorArtSource()
|
||||
assert source.supports_download is False
|
||||
with pytest.raises(ModelSourceError):
|
||||
source.file_download_url("123", "f.safetensors")
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_tensorart_lists_nothing(self):
|
||||
assert await TensorArtSource().list_files("123") == []
|
||||
|
||||
|
||||
class TestSourceIdValidation:
|
||||
@pytest.mark.parametrize(
|
||||
"source_id",
|
||||
["u/r", "black-forest-labs/FLUX.1-dev", "AI-ModelScope/stable-diffusion-v1-5"],
|
||||
)
|
||||
def test_accepts_repo_ids(self, source_id):
|
||||
assert is_valid_source_id(source_id) is True
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"source_id",
|
||||
[
|
||||
"",
|
||||
"noslash",
|
||||
"a/b/c",
|
||||
"../etc/passwd",
|
||||
"u/..",
|
||||
"u/.",
|
||||
".hidden/r",
|
||||
"u/r with space",
|
||||
"/r",
|
||||
"u/",
|
||||
],
|
||||
)
|
||||
def test_rejects_unsafe_ids(self, source_id):
|
||||
assert is_valid_source_id(source_id) is False
|
||||
|
||||
|
||||
class TestDownloadSourceRegistry:
|
||||
def test_downloadable_sources_excludes_link_only_sites(self):
|
||||
platforms = {source.platform for source in downloadable_sources()}
|
||||
assert platforms == {"huggingface", "modelscope"}
|
||||
|
||||
def test_get_download_source_rejects_link_only_platform(self):
|
||||
assert get_download_source("tensorart") is None
|
||||
assert get_download_source("nope") is None
|
||||
assert get_download_source("modelscope").platform == "modelscope"
|
||||
assert get_download_source("huggingface").platform == "huggingface"
|
||||
@@ -292,6 +292,61 @@ Content
|
||||
)
|
||||
assert images[0]["meta"]["prompt"] == "a cat"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gallery_images_use_modelscope_asset_base_url(self, processor):
|
||||
"""A ModelScope-linked model resolves relative images against ModelScope."""
|
||||
readme = """---
|
||||
widget:
|
||||
- text: "a cat"
|
||||
output:
|
||||
url: images/cat.png
|
||||
---
|
||||
Content
|
||||
"""
|
||||
with (
|
||||
mock.patch("py.metadata_ops.apply_metadata_updates") as mock_apply,
|
||||
mock.patch("py.metadata_ops.download_preview", return_value=None),
|
||||
mock.patch("py.metadata_ops.refresh_cache"),
|
||||
):
|
||||
await processor.process(
|
||||
skill_name="enrich_hf_metadata",
|
||||
model_path="/p.safetensors",
|
||||
llm_output=self.MIN_LLM_OUTPUT,
|
||||
metadata={
|
||||
"from_civitai": False,
|
||||
"source_platform": "modelscope",
|
||||
"source_url": "https://modelscope.cn/models/user/repo",
|
||||
},
|
||||
readme_content=readme,
|
||||
)
|
||||
applied = mock_apply.call_args[0][1]
|
||||
images = applied.get("civitai", {}).get("images", [])
|
||||
assert len(images) == 1
|
||||
assert images[0]["url"] == (
|
||||
"https://modelscope.cn/models/user/repo/resolve/master/images/cat.png"
|
||||
)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_base_model_overwrites_existing_modelscope_model(self, processor):
|
||||
"""ModelScope is an external source, so the LLM may overwrite base_model."""
|
||||
llm = {**self.MIN_LLM_OUTPUT, "base_model": "Flux.1 D"}
|
||||
with (
|
||||
mock.patch("py.metadata_ops.apply_metadata_updates") as mock_apply,
|
||||
mock.patch("py.metadata_ops.download_preview", return_value=False),
|
||||
mock.patch("py.metadata_ops.refresh_cache"),
|
||||
):
|
||||
await processor.process(
|
||||
skill_name="enrich_hf_metadata",
|
||||
model_path="/p.safetensors",
|
||||
llm_output=llm,
|
||||
metadata={
|
||||
"base_model": "SD 1.5",
|
||||
"source_platform": "modelscope",
|
||||
"source_url": "https://modelscope.cn/models/user/repo",
|
||||
},
|
||||
)
|
||||
assert mock_apply.call_args[0][1]["base_model"] == "Flux.1 D"
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_gallery_images_skipped_without_hf_url(self, processor):
|
||||
"""Gallery images NOT extracted when the model has no HF source."""
|
||||
|
||||
Reference in New Issue
Block a user