mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-09-21 11:11:26 -03:00
feat(links): support ModelScope and TensorArt as model sources
A model file could only ever be linked to huggingface.co: `set_hf_url` validated the URL with a huggingface-only regex, the agent fetched the card from a hardcoded HF URL, and the readme processor built every relative image path off `https://huggingface.co/{repo}/resolve/main`. ModelScope publishes the same model-card convention (README.md + YAML frontmatter, often carrying `base_model:` and `trigger_words:`) behind a public, key-less API, so the enrichment pipeline could already serve it - it was the plumbing that was HF-shaped, not the idea. Make the external source a first-class, provider-driven concept: - New `py/services/model_sources/` registry. A `ModelSource` owns URL recognition (lenient for stored values, strict for user input), the canonical page URL, model-card fetching, the asset base URL and the capability flags. `HuggingFaceSource` is the previous logic relocated; `ModelScopeSource` reads `/models/{o}/{n}/resolve/{master|main}/README.md` and falls back to `/api/v1/models/{o}/{n}/repo`. `TensorArtSource` is link-only on purpose: tensor.art answers plain HTTP clients with a Cloudflare challenge and its internal API (ap-east-1.tensorart.cloud / cn.tensorart.net) rejects every /v1/model/* route with "invalid authorization header", so it declares supports_enrichment=False rather than failing silently later. - Metadata gains `source_platform` + `source_url`; `hf_url` stays as a read/write alias, written only for Hugging Face, so existing sidecars, cached rows and third-party consumers keep working. Normalisation runs at the scanner, the persistent cache (both directions, plus two new columns behind an ALTER migration) and the linking handler - which is what stops a user who switches sources from leaving a stale `hf_url` on a ModelScope model. - The agent pipeline keys off the provider instead of `hf_url`: the fast-fail gate now explains *why* a model is skipped (no source / unknown source / source without a reachable card), the prompt context exposes source_url/source_id/source_label/asset_base_url while still filling the legacy hf_url/repo aliases, and the four README image extractors take a base_url (defaulting to HF) so relative paths resolve against the right site. Version grouping generalises to hf: / ms: / ta: keys. - `POST /api/lm/set-hf-url` keeps its path and its legacy payload keys but accepts `source_url`, validates against every provider and returns the platform. `GET /api/lm/model-sources` lets the UI render the supported-site list from the server. - Frontend: a `modelSourceHelpers` mirror of the registry drives the link dialog, the card/modal globe (branded "View on ModelScope/TensorArt"), the version-group key and the enrichment gate; the versions tab no longer sends ms:/ta: keys to the CivitAI API. TensorArt stays in the list because provenance is worth keeping even when the card is unreadable - the dialog says so plainly ("Sites that don't expose one (currently TensorArt) can only be linked") and the context menu disables enrichment with a matching tooltip, instead of the user getting "Unsupported URL". Verified against the real ModelScope API: jj3550945163/Krea-2-LORA returns a 1882-byte card whose frontmatter carries base_model/tags/trigger_words, and relative images resolve to .../resolve/master/.... Tests: backend 2815 passed; frontend 1130 JS + 91 Vue passed; pytest tests/i18n and a Jinja compile pass over templates/. The nine locales carry [TODO: Translate] for the new strings, completed in the next commit.
This commit is contained in:
+17
-5
@@ -62,13 +62,23 @@ Environment variable overrides: `LLM_API_KEY`, `LLM_MODEL`, `LLM_API_BASE`, `LLM
|
|||||||
|
|
||||||
### enrich_hf_metadata
|
### enrich_hf_metadata
|
||||||
|
|
||||||
Enriches HuggingFace-downloaded models with metadata extracted by an LLM from the HF model card.
|
Enriches models linked to an external model site with metadata extracted by an LLM from the site's model card (README).
|
||||||
|
|
||||||
**Entry point**: Right-click context menu → "Enrich Metadata (Agent)"
|
**Entry point**: Right-click context menu → "Enrich Metadata with AI"
|
||||||
|
|
||||||
|
**Supported model sources**:
|
||||||
|
|
||||||
|
| Platform | Link | AI enrichment | Direct download |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Hugging Face | yes | yes | yes |
|
||||||
|
| ModelScope | yes | yes | no |
|
||||||
|
| TensorArt | yes | no (see below) | no |
|
||||||
|
|
||||||
|
TensorArt is link-only: `tensor.art` sits behind a Cloudflare managed challenge and its internal API requires session authorization, so the backend cannot read its model pages. Linking still stores the canonical page URL and the "View on TensorArt" link works.
|
||||||
|
|
||||||
**What it does**:
|
**What it does**:
|
||||||
1. Reads the model's `.metadata.json` to get the `hf_url`
|
1. Reads the model's `.metadata.json` to get the source (`source_platform` + `source_url`, or the legacy `hf_url`)
|
||||||
2. Fetches the README.md from the HuggingFace repository
|
2. Fetches the model card through the provider in `py/services/model_sources/`
|
||||||
3. Sends the README + local metadata to the LLM for structured extraction
|
3. Sends the README + local metadata to the LLM for structured extraction
|
||||||
4. Writes extracted fields to `.metadata.json`:
|
4. Writes extracted fields to `.metadata.json`:
|
||||||
- `base_model` — only if current value is empty
|
- `base_model` — only if current value is empty
|
||||||
@@ -81,6 +91,8 @@ Enriches HuggingFace-downloaded models with metadata extracted by an LLM from th
|
|||||||
6. Updates the scanner cache
|
6. Updates the scanner cache
|
||||||
7. Broadcasts WebSocket progress events
|
7. Broadcasts WebSocket progress events
|
||||||
|
|
||||||
|
Models with no source, an unknown source, or a source without model-card access (TensorArt) are skipped with an explicit reason and counted in the run summary.
|
||||||
|
|
||||||
**Model types**: LoRA, Checkpoint, Embedding
|
**Model types**: LoRA, Checkpoint, Embedding
|
||||||
|
|
||||||
## Adding a New Skill
|
## Adding a New Skill
|
||||||
@@ -129,7 +141,7 @@ Use `{{variable}}` placeholders that will be replaced with data from the `prepar
|
|||||||
```markdown
|
```markdown
|
||||||
You are an expert assistant...
|
You are an expert assistant...
|
||||||
|
|
||||||
Model URL: {{hf_url}}
|
Model URL: {{source_url}}
|
||||||
README content:
|
README content:
|
||||||
{{readme_content}}
|
{{readme_content}}
|
||||||
|
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "Auf CivitAI anzeigen",
|
"viewOnCivitai": "Auf CivitAI anzeigen",
|
||||||
"notAvailableFromCivitai": "Nicht auf CivitAI verfügbar",
|
"notAvailableFromCivitai": "Nicht auf CivitAI verfügbar",
|
||||||
"viewOnHuggingFace": "Auf Hugging Face ansehen",
|
"viewOnHuggingFace": "Auf Hugging Face ansehen",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "An ComfyUI senden (Klick: Anhängen, Shift+Klick: Ersetzen)",
|
"sendToWorkflow": "An ComfyUI senden (Klick: Anhängen, Shift+Klick: Ersetzen)",
|
||||||
"copyLoRASyntax": "LoRA-Syntax kopieren",
|
"copyLoRASyntax": "LoRA-Syntax kopieren",
|
||||||
"checkpointNameCopied": "Checkpoint-Name kopiert",
|
"checkpointNameCopied": "Checkpoint-Name kopiert",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "Updates prüfen",
|
"checkUpdates": "Updates prüfen",
|
||||||
"linkModel": "Modell verknüpfen",
|
"linkModel": "Modell verknüpfen",
|
||||||
"linkCivitai": "Mit CivitAI neu verknüpfen",
|
"linkCivitai": "Mit CivitAI neu verknüpfen",
|
||||||
"linkHuggingFace": "Mit HuggingFace verknüpfen",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "LoRA-Syntax kopieren",
|
"copySyntax": "LoRA-Syntax kopieren",
|
||||||
"copyFilename": "Modell-Dateiname kopieren",
|
"copyFilename": "Modell-Dateiname kopieren",
|
||||||
"copyRecipeSyntax": "Rezept-Syntax kopieren",
|
"copyRecipeSyntax": "Rezept-Syntax kopieren",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "Ordnerpfad eingeben oder aus Baum unten auswählen...",
|
"pathPlaceholder": "Ordnerpfad eingeben oder aus Baum unten auswählen...",
|
||||||
"root": "Stammverzeichnis"
|
"root": "Stammverzeichnis"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "Mit HuggingFace verknüpfen",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "Fügen Sie die HuggingFace-Repository-URL ein, um dieses Modell zuzuordnen. Dies ermöglicht die KI-gestützte Metadatenanreicherung.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace-Repository-URL:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "Geben Sie die vollständige URL des HuggingFace-Repositorys ein.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "Speichern & Verknüpfen"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "Mit CivitAI neu verknüpfen",
|
"title": "Mit CivitAI neu verknüpfen",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "Noch keine Versionshistorie für dieses Modell vorhanden.",
|
"empty": "Noch keine Versionshistorie für dieses Modell vorhanden.",
|
||||||
"error": "Versionen konnten nicht geladen werden.",
|
"error": "Versionen konnten nicht geladen werden.",
|
||||||
"missingModelId": "Für dieses Modell ist keine CivitAI-Model-ID vorhanden.",
|
"missingModelId": "Für dieses Modell ist keine CivitAI-Model-ID vorhanden.",
|
||||||
"hfGroupInfo": "Dies ist eine HuggingFace-Modellgruppe. Öffnen Sie die Bibliothek, um alle Versionen im Raster zu sehen.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "Diese Version aus Ihrer Bibliothek löschen?"
|
"delete": "Diese Version aus Ihrer Bibliothek löschen?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "Modell erfolgreich über CivitArchive neu verknüpft",
|
"linkCivArchSuccess": "Modell erfolgreich über CivitArchive neu verknüpft",
|
||||||
"fetchMetadataFirst": "Bitte rufen Sie zuerst Metadaten von CivitAI ab",
|
"fetchMetadataFirst": "Bitte rufen Sie zuerst Metadaten von CivitAI ab",
|
||||||
"noCivitaiInfo": "Keine CivitAI-Informationen verfügbar",
|
"noCivitaiInfo": "Keine CivitAI-Informationen verfügbar",
|
||||||
"missingHash": "Modell-Hash nicht verfügbar"
|
"missingHash": "Modell-Hash nicht verfügbar",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "Beispielbilder-Pfad erfolgreich aktualisiert",
|
"pathUpdated": "Beispielbilder-Pfad erfolgreich aktualisiert",
|
||||||
|
|||||||
+18
-11
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "View on CivitAI",
|
"viewOnCivitai": "View on CivitAI",
|
||||||
"notAvailableFromCivitai": "Not available from CivitAI",
|
"notAvailableFromCivitai": "Not available from CivitAI",
|
||||||
"viewOnHuggingFace": "View on Hugging Face",
|
"viewOnHuggingFace": "View on Hugging Face",
|
||||||
|
"viewOnSource": "View on {source}",
|
||||||
"sendToWorkflow": "Send to ComfyUI (Click: Append, Shift+Click: Replace)",
|
"sendToWorkflow": "Send to ComfyUI (Click: Append, Shift+Click: Replace)",
|
||||||
"copyLoRASyntax": "Copy LoRA Syntax",
|
"copyLoRASyntax": "Copy LoRA Syntax",
|
||||||
"checkpointNameCopied": "Checkpoint name copied",
|
"checkpointNameCopied": "Checkpoint name copied",
|
||||||
@@ -867,14 +868,14 @@
|
|||||||
"complete": "Auto-organize complete",
|
"complete": "Auto-organize complete",
|
||||||
"error": "Error: {error}"
|
"error": "Error: {error}"
|
||||||
},
|
},
|
||||||
"enrichHfAgent": "Enrich HF Metadata (AI)"
|
"enrichHfAgent": "Enrich Metadata with AI"
|
||||||
},
|
},
|
||||||
"contextMenu": {
|
"contextMenu": {
|
||||||
"refreshMetadata": "Refresh CivitAI Data",
|
"refreshMetadata": "Refresh CivitAI Data",
|
||||||
"checkUpdates": "Check Updates",
|
"checkUpdates": "Check Updates",
|
||||||
"linkModel": "Link Model",
|
"linkModel": "Link Model",
|
||||||
"linkCivitai": "Link to CivitAI",
|
"linkCivitai": "Link to CivitAI",
|
||||||
"linkHuggingFace": "Link to HuggingFace",
|
"linkModelSource": "Link to Model Source",
|
||||||
"copySyntax": "Copy LoRA Syntax",
|
"copySyntax": "Copy LoRA Syntax",
|
||||||
"copyFilename": "Copy Model Filename",
|
"copyFilename": "Copy Model Filename",
|
||||||
"copyRecipeSyntax": "Copy Recipe Syntax",
|
"copyRecipeSyntax": "Copy Recipe Syntax",
|
||||||
@@ -896,7 +897,7 @@
|
|||||||
"viewAllLoras": "View All LoRAs",
|
"viewAllLoras": "View All LoRAs",
|
||||||
"downloadMissingLoras": "Download Missing LoRAs",
|
"downloadMissingLoras": "Download Missing LoRAs",
|
||||||
"deleteRecipe": "Delete Recipe",
|
"deleteRecipe": "Delete Recipe",
|
||||||
"enrichHfAgent": "Enrich HF Metadata (AI)"
|
"enrichHfAgent": "Enrich Metadata with AI"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"recipes": {
|
"recipes": {
|
||||||
@@ -1596,12 +1597,16 @@
|
|||||||
"pathPlaceholder": "Type folder path or select from tree below...",
|
"pathPlaceholder": "Type folder path or select from tree below...",
|
||||||
"root": "Root"
|
"root": "Root"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "Link to HuggingFace",
|
"title": "Link to Model Source",
|
||||||
"infoText": "Paste the HuggingFace repository URL to associate this model with its source. This enables AI-powered metadata enrichment.",
|
"infoText": "Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace Repository URL:",
|
"urlLabel": "Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "https://huggingface.co/user/repo",
|
||||||
"helpText": "Enter the full URL of the HuggingFace repository.",
|
"helpText": "Enter the full URL of the model page. Supported sites:",
|
||||||
|
"enrichNote": "AI enrichment needs a readable model card. Sites that don't expose one (currently TensorArt) can only be linked.",
|
||||||
|
"urlRequired": "Please enter a model page URL.",
|
||||||
|
"invalidUrl": "Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "Linking model source...",
|
||||||
"confirmAction": "Save & Link"
|
"confirmAction": "Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "No version history available for this model yet.",
|
"empty": "No version history available for this model yet.",
|
||||||
"error": "Failed to load versions.",
|
"error": "Failed to load versions.",
|
||||||
"missingModelId": "This model is missing a CivitAI model id.",
|
"missingModelId": "This model is missing a CivitAI model id.",
|
||||||
"hfGroupInfo": "This is a HuggingFace model group. Open the library to see all versions in the grid.",
|
"sourceGroupInfo": "This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "Delete this version from your library?"
|
"delete": "Delete this version from your library?"
|
||||||
},
|
},
|
||||||
@@ -2478,12 +2483,14 @@
|
|||||||
"contentRatingFailed": "Failed to set content rating: {message}",
|
"contentRatingFailed": "Failed to set content rating: {message}",
|
||||||
"relinkSuccess": "Model successfully re-linked to CivitAI",
|
"relinkSuccess": "Model successfully re-linked to CivitAI",
|
||||||
"relinkFailed": "Error: {message}",
|
"relinkFailed": "Error: {message}",
|
||||||
"linkHfSuccess": "Model successfully linked to HuggingFace",
|
"linkHfSuccess": "Model successfully linked to its model source",
|
||||||
"linkHfFailed": "Error: {message}",
|
"linkHfFailed": "Error: {message}",
|
||||||
"linkCivArchSuccess": "Model successfully re-linked via CivitArchive",
|
"linkCivArchSuccess": "Model successfully re-linked via CivitArchive",
|
||||||
"fetchMetadataFirst": "Please fetch metadata from CivitAI first",
|
"fetchMetadataFirst": "Please fetch metadata from CivitAI first",
|
||||||
"noCivitaiInfo": "No CivitAI information available",
|
"noCivitaiInfo": "No CivitAI information available",
|
||||||
"missingHash": "Model hash not available"
|
"missingHash": "Model hash not available",
|
||||||
|
"enrichNeedsSource": "Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "Example images path updated successfully",
|
"pathUpdated": "Example images path updated successfully",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "Ver en CivitAI",
|
"viewOnCivitai": "Ver en CivitAI",
|
||||||
"notAvailableFromCivitai": "No disponible en CivitAI",
|
"notAvailableFromCivitai": "No disponible en CivitAI",
|
||||||
"viewOnHuggingFace": "Ver en Hugging Face",
|
"viewOnHuggingFace": "Ver en Hugging Face",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "Enviar a ComfyUI (Clic: Añadir, Shift+Clic: Reemplazar)",
|
"sendToWorkflow": "Enviar a ComfyUI (Clic: Añadir, Shift+Clic: Reemplazar)",
|
||||||
"copyLoRASyntax": "Copiar sintaxis de LoRA",
|
"copyLoRASyntax": "Copiar sintaxis de LoRA",
|
||||||
"checkpointNameCopied": "Nombre del checkpoint copiado",
|
"checkpointNameCopied": "Nombre del checkpoint copiado",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "Comprobar actualizaciones",
|
"checkUpdates": "Comprobar actualizaciones",
|
||||||
"linkModel": "Vincular modelo",
|
"linkModel": "Vincular modelo",
|
||||||
"linkCivitai": "Re-vincular a CivitAI",
|
"linkCivitai": "Re-vincular a CivitAI",
|
||||||
"linkHuggingFace": "Vincular a HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "Copiar sintaxis de LoRA",
|
"copySyntax": "Copiar sintaxis de LoRA",
|
||||||
"copyFilename": "Copiar nombre de archivo del modelo",
|
"copyFilename": "Copiar nombre de archivo del modelo",
|
||||||
"copyRecipeSyntax": "Copiar sintaxis de receta",
|
"copyRecipeSyntax": "Copiar sintaxis de receta",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "Escribe la ruta de la carpeta o selecciona del árbol de abajo...",
|
"pathPlaceholder": "Escribe la ruta de la carpeta o selecciona del árbol de abajo...",
|
||||||
"root": "Raíz"
|
"root": "Raíz"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "Vincular a HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "Pegue la URL del repositorio de HuggingFace para asociar este modelo. Esto permite el enriquecimiento de metadatos con IA.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "URL del repositorio de HuggingFace:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "Ingrese la URL completa del repositorio de HuggingFace.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "Guardar y vincular"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "Re-vincular a CivitAI",
|
"title": "Re-vincular a CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "Aún no hay historial de versiones para este modelo.",
|
"empty": "Aún no hay historial de versiones para este modelo.",
|
||||||
"error": "No se pudieron cargar las versiones.",
|
"error": "No se pudieron cargar las versiones.",
|
||||||
"missingModelId": "Este modelo no tiene un ID de modelo de CivitAI.",
|
"missingModelId": "Este modelo no tiene un ID de modelo de CivitAI.",
|
||||||
"hfGroupInfo": "Este es un grupo de modelos de HuggingFace. Abra la biblioteca para ver todas las versiones en la cuadrícula.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "¿Eliminar esta versión de tu biblioteca?"
|
"delete": "¿Eliminar esta versión de tu biblioteca?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "Modelo re-vinculado exitosamente mediante CivitArchive",
|
"linkCivArchSuccess": "Modelo re-vinculado exitosamente mediante CivitArchive",
|
||||||
"fetchMetadataFirst": "Por favor obtén metadatos de CivitAI primero",
|
"fetchMetadataFirst": "Por favor obtén metadatos de CivitAI primero",
|
||||||
"noCivitaiInfo": "No hay información de CivitAI disponible",
|
"noCivitaiInfo": "No hay información de CivitAI disponible",
|
||||||
"missingHash": "Hash del modelo no disponible"
|
"missingHash": "Hash del modelo no disponible",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "Ruta de imágenes de ejemplo actualizada exitosamente",
|
"pathUpdated": "Ruta de imágenes de ejemplo actualizada exitosamente",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "Voir sur CivitAI",
|
"viewOnCivitai": "Voir sur CivitAI",
|
||||||
"notAvailableFromCivitai": "Non disponible sur CivitAI",
|
"notAvailableFromCivitai": "Non disponible sur CivitAI",
|
||||||
"viewOnHuggingFace": "Voir sur Hugging Face",
|
"viewOnHuggingFace": "Voir sur Hugging Face",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "Envoyer vers ComfyUI (Clic: Ajouter, Maj+Clic: Remplacer)",
|
"sendToWorkflow": "Envoyer vers ComfyUI (Clic: Ajouter, Maj+Clic: Remplacer)",
|
||||||
"copyLoRASyntax": "Copier la syntaxe LoRA",
|
"copyLoRASyntax": "Copier la syntaxe LoRA",
|
||||||
"checkpointNameCopied": "Nom du checkpoint copié",
|
"checkpointNameCopied": "Nom du checkpoint copié",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "Vérifier les mises à jour",
|
"checkUpdates": "Vérifier les mises à jour",
|
||||||
"linkModel": "Lier le modèle",
|
"linkModel": "Lier le modèle",
|
||||||
"linkCivitai": "Relier à nouveau à CivitAI",
|
"linkCivitai": "Relier à nouveau à CivitAI",
|
||||||
"linkHuggingFace": "Lier à HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "Copier la syntaxe LoRA",
|
"copySyntax": "Copier la syntaxe LoRA",
|
||||||
"copyFilename": "Copier le nom de fichier du modèle",
|
"copyFilename": "Copier le nom de fichier du modèle",
|
||||||
"copyRecipeSyntax": "Copier la syntaxe de la recipe",
|
"copyRecipeSyntax": "Copier la syntaxe de la recipe",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "Tapez le chemin du dossier ou sélectionnez dans l'arbre ci-dessous...",
|
"pathPlaceholder": "Tapez le chemin du dossier ou sélectionnez dans l'arbre ci-dessous...",
|
||||||
"root": "Racine"
|
"root": "Racine"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "Lier à HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "Collez l'URL du dépôt HuggingFace pour associer ce modèle à sa source. Cela permet l'enrichissement des métadonnées par IA.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "URL du dépôt HuggingFace :",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "Entrez l'URL complète du dépôt HuggingFace.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "Enregistrer & lier"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "Relier à nouveau à CivitAI",
|
"title": "Relier à nouveau à CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "Aucun historique de versions n'est disponible pour ce modèle pour le moment.",
|
"empty": "Aucun historique de versions n'est disponible pour ce modèle pour le moment.",
|
||||||
"error": "Échec du chargement des versions.",
|
"error": "Échec du chargement des versions.",
|
||||||
"missingModelId": "Ce modèle ne possède pas d'identifiant de modèle CivitAI.",
|
"missingModelId": "Ce modèle ne possède pas d'identifiant de modèle CivitAI.",
|
||||||
"hfGroupInfo": "Ceci est un groupe de modèles HuggingFace. Ouvrez la bibliothèque pour voir toutes les versions dans la grille.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "Supprimer cette version de votre bibliothèque ?"
|
"delete": "Supprimer cette version de votre bibliothèque ?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "Modèle relié via CivitArchive avec succès",
|
"linkCivArchSuccess": "Modèle relié via CivitArchive avec succès",
|
||||||
"fetchMetadataFirst": "Veuillez d'abord récupérer les métadonnées depuis CivitAI",
|
"fetchMetadataFirst": "Veuillez d'abord récupérer les métadonnées depuis CivitAI",
|
||||||
"noCivitaiInfo": "Aucune information CivitAI disponible",
|
"noCivitaiInfo": "Aucune information CivitAI disponible",
|
||||||
"missingHash": "Hash du modèle non disponible"
|
"missingHash": "Hash du modèle non disponible",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "Chemin des images d'exemple mis à jour avec succès",
|
"pathUpdated": "Chemin des images d'exemple mis à jour avec succès",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "הצג ב-CivitAI",
|
"viewOnCivitai": "הצג ב-CivitAI",
|
||||||
"notAvailableFromCivitai": "לא זמין מ-CivitAI",
|
"notAvailableFromCivitai": "לא זמין מ-CivitAI",
|
||||||
"viewOnHuggingFace": "צפייה ב-Hugging Face",
|
"viewOnHuggingFace": "צפייה ב-Hugging Face",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "שלח ל-ComfyUI (לחיצה: הוסף, Shift+לחיצה: החלף)",
|
"sendToWorkflow": "שלח ל-ComfyUI (לחיצה: הוסף, Shift+לחיצה: החלף)",
|
||||||
"copyLoRASyntax": "העתק תחביר LoRA",
|
"copyLoRASyntax": "העתק תחביר LoRA",
|
||||||
"checkpointNameCopied": "שם Checkpoint הועתק",
|
"checkpointNameCopied": "שם Checkpoint הועתק",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "בדוק עדכונים",
|
"checkUpdates": "בדוק עדכונים",
|
||||||
"linkModel": "קישור מודל",
|
"linkModel": "קישור מודל",
|
||||||
"linkCivitai": "קשר מחדש ל-CivitAI",
|
"linkCivitai": "קשר מחדש ל-CivitAI",
|
||||||
"linkHuggingFace": "קישור ל-HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "העתק תחביר LoRA",
|
"copySyntax": "העתק תחביר LoRA",
|
||||||
"copyFilename": "העתק שם קובץ מודל",
|
"copyFilename": "העתק שם קובץ מודל",
|
||||||
"copyRecipeSyntax": "העתק תחביר מתכון",
|
"copyRecipeSyntax": "העתק תחביר מתכון",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "הקלד נתיב תיקייה או בחר מהעץ למטה...",
|
"pathPlaceholder": "הקלד נתיב תיקייה או בחר מהעץ למטה...",
|
||||||
"root": "שורש"
|
"root": "שורש"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "קישור ל-HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "הדבק את כתובת ה-URL של מאגר HuggingFace כדי לשייך מודל זה למקורו. פעולה זו מאפשרת העשרת מטא-נתונים באמצעות AI.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "כתובת URL של מאגר HuggingFace:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "הזן את כתובת ה-URL המלאה של מאגר HuggingFace.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "שמור וקשר"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "קשר מחדש ל-CivitAI",
|
"title": "קשר מחדש ל-CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "אין עדיין היסטוריית גרסאות למודל זה.",
|
"empty": "אין עדיין היסטוריית גרסאות למודל זה.",
|
||||||
"error": "טעינת הגרסאות נכשלה.",
|
"error": "טעינת הגרסאות נכשלה.",
|
||||||
"missingModelId": "למודל זה אין מזהה מודל של CivitAI.",
|
"missingModelId": "למודל זה אין מזהה מודל של CivitAI.",
|
||||||
"hfGroupInfo": "זוהי קבוצת מודלים של HuggingFace. פתח את הספרייה כדי לראות את כל הגרסאות ברשת.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "למחוק גרסה זו מהספרייה שלך?"
|
"delete": "למחוק גרסה זו מהספרייה שלך?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "המודל קושר מחדש דרך CivitArchive בהצלחה",
|
"linkCivArchSuccess": "המודל קושר מחדש דרך CivitArchive בהצלחה",
|
||||||
"fetchMetadataFirst": "אנא אחזר מטא-נתונים מ-CivitAI תחילה",
|
"fetchMetadataFirst": "אנא אחזר מטא-נתונים מ-CivitAI תחילה",
|
||||||
"noCivitaiInfo": "אין מידע מ-CivitAI זמין",
|
"noCivitaiInfo": "אין מידע מ-CivitAI זמין",
|
||||||
"missingHash": "ה-hash של המודל אינו זמין"
|
"missingHash": "ה-hash של המודל אינו זמין",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "נתיב תמונות הדוגמה עודכן בהצלחה",
|
"pathUpdated": "נתיב תמונות הדוגמה עודכן בהצלחה",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "CivitAIで表示",
|
"viewOnCivitai": "CivitAIで表示",
|
||||||
"notAvailableFromCivitai": "CivitAIでは利用できません",
|
"notAvailableFromCivitai": "CivitAIでは利用できません",
|
||||||
"viewOnHuggingFace": "Hugging Face で見る",
|
"viewOnHuggingFace": "Hugging Face で見る",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "ComfyUIに送信(クリック:追加、Shift+クリック:置換)",
|
"sendToWorkflow": "ComfyUIに送信(クリック:追加、Shift+クリック:置換)",
|
||||||
"copyLoRASyntax": "LoRA構文をコピー",
|
"copyLoRASyntax": "LoRA構文をコピー",
|
||||||
"checkpointNameCopied": "Checkpointの名前をコピーしました",
|
"checkpointNameCopied": "Checkpointの名前をコピーしました",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "更新確認",
|
"checkUpdates": "更新確認",
|
||||||
"linkModel": "モデルをリンク",
|
"linkModel": "モデルをリンク",
|
||||||
"linkCivitai": "CivitAI にリンク",
|
"linkCivitai": "CivitAI にリンク",
|
||||||
"linkHuggingFace": "HuggingFace にリンク",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "LoRA構文をコピー",
|
"copySyntax": "LoRA構文をコピー",
|
||||||
"copyFilename": "モデルファイル名をコピー",
|
"copyFilename": "モデルファイル名をコピー",
|
||||||
"copyRecipeSyntax": "レシピ構文をコピー",
|
"copyRecipeSyntax": "レシピ構文をコピー",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "フォルダパスを入力するか、下のツリーから選択...",
|
"pathPlaceholder": "フォルダパスを入力するか、下のツリーから選択...",
|
||||||
"root": "ルート"
|
"root": "ルート"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "HuggingFace にリンク",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "HuggingFace リポジトリの URL を貼り付けてモデルを関連付けます。AI によるメタデータ補完が有効になります。",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace リポジトリ URL:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "完全な HuggingFace リポジトリ URL を入力してください。",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "保存&リンク"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "CivitAIに再リンク",
|
"title": "CivitAIに再リンク",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "このモデルにはまだバージョン履歴がありません。",
|
"empty": "このモデルにはまだバージョン履歴がありません。",
|
||||||
"error": "バージョンの読み込みに失敗しました。",
|
"error": "バージョンの読み込みに失敗しました。",
|
||||||
"missingModelId": "このモデルにはCivitAIのモデルIDがありません。",
|
"missingModelId": "このモデルにはCivitAIのモデルIDがありません。",
|
||||||
"hfGroupInfo": "これは HuggingFace モデルグループです。ライブラリを開いてグリッドですべてのバージョンを表示してください。",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "このバージョンをライブラリから削除しますか?"
|
"delete": "このバージョンをライブラリから削除しますか?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "モデルがCivitArchive経由で正常に再リンクされました",
|
"linkCivArchSuccess": "モデルがCivitArchive経由で正常に再リンクされました",
|
||||||
"fetchMetadataFirst": "最初にCivitAIからメタデータを取得してください",
|
"fetchMetadataFirst": "最初にCivitAIからメタデータを取得してください",
|
||||||
"noCivitaiInfo": "CivitAI情報が利用できません",
|
"noCivitaiInfo": "CivitAI情報が利用できません",
|
||||||
"missingHash": "モデルハッシュが利用できません"
|
"missingHash": "モデルハッシュが利用できません",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "例画像パスが正常に更新されました",
|
"pathUpdated": "例画像パスが正常に更新されました",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "CivitAI에서 보기",
|
"viewOnCivitai": "CivitAI에서 보기",
|
||||||
"notAvailableFromCivitai": "CivitAI에서 사용할 수 없음",
|
"notAvailableFromCivitai": "CivitAI에서 사용할 수 없음",
|
||||||
"viewOnHuggingFace": "Hugging Face에서 보기",
|
"viewOnHuggingFace": "Hugging Face에서 보기",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "ComfyUI로 전송 (클릭: 추가, Shift+클릭: 교체)",
|
"sendToWorkflow": "ComfyUI로 전송 (클릭: 추가, Shift+클릭: 교체)",
|
||||||
"copyLoRASyntax": "LoRA 문법 복사",
|
"copyLoRASyntax": "LoRA 문법 복사",
|
||||||
"checkpointNameCopied": "Checkpoint 이름 복사됨",
|
"checkpointNameCopied": "Checkpoint 이름 복사됨",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "업데이트 확인",
|
"checkUpdates": "업데이트 확인",
|
||||||
"linkModel": "모델 연결",
|
"linkModel": "모델 연결",
|
||||||
"linkCivitai": "CivitAI에 연결",
|
"linkCivitai": "CivitAI에 연결",
|
||||||
"linkHuggingFace": "HuggingFace에 연결",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "LoRA 문법 복사",
|
"copySyntax": "LoRA 문법 복사",
|
||||||
"copyFilename": "모델 파일명 복사",
|
"copyFilename": "모델 파일명 복사",
|
||||||
"copyRecipeSyntax": "레시피 문법 복사",
|
"copyRecipeSyntax": "레시피 문법 복사",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "폴더 경로를 입력하거나 아래 트리에서 선택하세요...",
|
"pathPlaceholder": "폴더 경로를 입력하거나 아래 트리에서 선택하세요...",
|
||||||
"root": "루트"
|
"root": "루트"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "HuggingFace에 연결",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "HuggingFace 저장소 URL을 붙여넣어 모델을 연결합니다. AI 메타데이터 보강 기능을 사용할 수 있습니다.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace 저장소 URL:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "전체 HuggingFace 저장소 URL을 입력하세요.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "저장 및 연결"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "CivitAI에 다시 연결",
|
"title": "CivitAI에 다시 연결",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "이 모델에는 아직 버전 기록이 없습니다.",
|
"empty": "이 모델에는 아직 버전 기록이 없습니다.",
|
||||||
"error": "버전을 불러오지 못했습니다.",
|
"error": "버전을 불러오지 못했습니다.",
|
||||||
"missingModelId": "이 모델에는 CivitAI 모델 ID가 없습니다.",
|
"missingModelId": "이 모델에는 CivitAI 모델 ID가 없습니다.",
|
||||||
"hfGroupInfo": "HuggingFace 모델 그룹입니다. 라이브러리를 열어 그리드에서 모든 버전을 확인하세요.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "이 버전을 라이브러리에서 삭제하시겠습니까?"
|
"delete": "이 버전을 라이브러리에서 삭제하시겠습니까?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "모델이 CivitArchive을 통해 성공적으로 다시 연결되었습니다",
|
"linkCivArchSuccess": "모델이 CivitArchive을 통해 성공적으로 다시 연결되었습니다",
|
||||||
"fetchMetadataFirst": "먼저 CivitAI에서 메타데이터를 가져와주세요",
|
"fetchMetadataFirst": "먼저 CivitAI에서 메타데이터를 가져와주세요",
|
||||||
"noCivitaiInfo": "사용 가능한 CivitAI 정보가 없습니다",
|
"noCivitaiInfo": "사용 가능한 CivitAI 정보가 없습니다",
|
||||||
"missingHash": "모델 해시를 사용할 수 없습니다"
|
"missingHash": "모델 해시를 사용할 수 없습니다",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "예시 이미지 경로가 성공적으로 업데이트되었습니다",
|
"pathUpdated": "예시 이미지 경로가 성공적으로 업데이트되었습니다",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "Посмотреть на CivitAI",
|
"viewOnCivitai": "Посмотреть на CivitAI",
|
||||||
"notAvailableFromCivitai": "Недоступно на CivitAI",
|
"notAvailableFromCivitai": "Недоступно на CivitAI",
|
||||||
"viewOnHuggingFace": "Открыть Hugging Face",
|
"viewOnHuggingFace": "Открыть Hugging Face",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "Отправить в ComfyUI (Клик: Добавить, Shift+Клик: Заменить)",
|
"sendToWorkflow": "Отправить в ComfyUI (Клик: Добавить, Shift+Клик: Заменить)",
|
||||||
"copyLoRASyntax": "Копировать синтаксис LoRA",
|
"copyLoRASyntax": "Копировать синтаксис LoRA",
|
||||||
"checkpointNameCopied": "Имя checkpoint скопировано",
|
"checkpointNameCopied": "Имя checkpoint скопировано",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "Проверить обновления",
|
"checkUpdates": "Проверить обновления",
|
||||||
"linkModel": "Связать модель",
|
"linkModel": "Связать модель",
|
||||||
"linkCivitai": "Пересвязать с CivitAI",
|
"linkCivitai": "Пересвязать с CivitAI",
|
||||||
"linkHuggingFace": "Связать с HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "Копировать синтаксис LoRA",
|
"copySyntax": "Копировать синтаксис LoRA",
|
||||||
"copyFilename": "Копировать имя файла модели",
|
"copyFilename": "Копировать имя файла модели",
|
||||||
"copyRecipeSyntax": "Копировать синтаксис рецепта",
|
"copyRecipeSyntax": "Копировать синтаксис рецепта",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "Введите путь к папке или выберите из дерева ниже...",
|
"pathPlaceholder": "Введите путь к папке или выберите из дерева ниже...",
|
||||||
"root": "Корень"
|
"root": "Корень"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "Связать с HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "Вставьте URL репозитория HuggingFace, чтобы связать эту модель с её источником. Это позволит обогащать метаданные с помощью ИИ.",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "URL репозитория HuggingFace:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "Введите полный URL репозитория HuggingFace.",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "Сохранить и связать"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "Пересвязать с CivitAI",
|
"title": "Пересвязать с CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "Для этой модели пока нет истории версий.",
|
"empty": "Для этой модели пока нет истории версий.",
|
||||||
"error": "Не удалось загрузить версии.",
|
"error": "Не удалось загрузить версии.",
|
||||||
"missingModelId": "У этой модели отсутствует идентификатор модели CivitAI.",
|
"missingModelId": "У этой модели отсутствует идентификатор модели CivitAI.",
|
||||||
"hfGroupInfo": "Это группа моделей HuggingFace. Откройте библиотеку, чтобы увидеть все версии в сетке.",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "Удалить эту версию из библиотеки?"
|
"delete": "Удалить эту версию из библиотеки?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "Модель успешно пересвязана через CivitArchive",
|
"linkCivArchSuccess": "Модель успешно пересвязана через CivitArchive",
|
||||||
"fetchMetadataFirst": "Пожалуйста, сначала получите метаданные с CivitAI",
|
"fetchMetadataFirst": "Пожалуйста, сначала получите метаданные с CivitAI",
|
||||||
"noCivitaiInfo": "Информация CivitAI недоступна",
|
"noCivitaiInfo": "Информация CivitAI недоступна",
|
||||||
"missingHash": "Хеш модели недоступен"
|
"missingHash": "Хеш модели недоступен",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "Путь к примерам изображений успешно обновлен",
|
"pathUpdated": "Путь к примерам изображений успешно обновлен",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "在 CivitAI 查看",
|
"viewOnCivitai": "在 CivitAI 查看",
|
||||||
"notAvailableFromCivitai": "CivitAI 上不可用",
|
"notAvailableFromCivitai": "CivitAI 上不可用",
|
||||||
"viewOnHuggingFace": "在 Hugging Face 查看",
|
"viewOnHuggingFace": "在 Hugging Face 查看",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "发送到 ComfyUI(点击:追加,Shift+点击:替换)",
|
"sendToWorkflow": "发送到 ComfyUI(点击:追加,Shift+点击:替换)",
|
||||||
"copyLoRASyntax": "复制 LoRA 语法",
|
"copyLoRASyntax": "复制 LoRA 语法",
|
||||||
"checkpointNameCopied": "Checkpoint 名称已复制",
|
"checkpointNameCopied": "Checkpoint 名称已复制",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "检查更新",
|
"checkUpdates": "检查更新",
|
||||||
"linkModel": "链接模型",
|
"linkModel": "链接模型",
|
||||||
"linkCivitai": "链接到 CivitAI",
|
"linkCivitai": "链接到 CivitAI",
|
||||||
"linkHuggingFace": "链接到 HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "复制 LoRA 语法",
|
"copySyntax": "复制 LoRA 语法",
|
||||||
"copyFilename": "复制模型文件名",
|
"copyFilename": "复制模型文件名",
|
||||||
"copyRecipeSyntax": "复制配方语法",
|
"copyRecipeSyntax": "复制配方语法",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "输入文件夹路径或从下方树中选择...",
|
"pathPlaceholder": "输入文件夹路径或从下方树中选择...",
|
||||||
"root": "根目录"
|
"root": "根目录"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "链接到 HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "粘贴 HuggingFace 仓库 URL 以关联此模型。关联后可启用 AI 元数据增强功能。",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace 仓库 URL:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "请输入完整的 HuggingFace 仓库 URL。",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "保存并链接"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "重新关联到 CivitAI",
|
"title": "重新关联到 CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "该模型还没有版本历史。",
|
"empty": "该模型还没有版本历史。",
|
||||||
"error": "加载版本失败。",
|
"error": "加载版本失败。",
|
||||||
"missingModelId": "该模型缺少 CivitAI 模型 ID。",
|
"missingModelId": "该模型缺少 CivitAI 模型 ID。",
|
||||||
"hfGroupInfo": "这是一个 HuggingFace 模型组。打开库页面即可在网格中查看所有版本。",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "从库中删除此版本?"
|
"delete": "从库中删除此版本?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "模型已成功通过 CivitArchive 重新关联",
|
"linkCivArchSuccess": "模型已成功通过 CivitArchive 重新关联",
|
||||||
"fetchMetadataFirst": "请先从 CivitAI 获取元数据",
|
"fetchMetadataFirst": "请先从 CivitAI 获取元数据",
|
||||||
"noCivitaiInfo": "无 CivitAI 信息",
|
"noCivitaiInfo": "无 CivitAI 信息",
|
||||||
"missingHash": "模型哈希不可用"
|
"missingHash": "模型哈希不可用",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "示例图片路径更新成功",
|
"pathUpdated": "示例图片路径更新成功",
|
||||||
|
|||||||
+17
-10
@@ -139,6 +139,7 @@
|
|||||||
"viewOnCivitai": "在 CivitAI 查看",
|
"viewOnCivitai": "在 CivitAI 查看",
|
||||||
"notAvailableFromCivitai": "CivitAI 不提供",
|
"notAvailableFromCivitai": "CivitAI 不提供",
|
||||||
"viewOnHuggingFace": "在 Hugging Face 查看",
|
"viewOnHuggingFace": "在 Hugging Face 查看",
|
||||||
|
"viewOnSource": "[TODO: Translate] View on {source}",
|
||||||
"sendToWorkflow": "傳送到 ComfyUI(點擊:附加,Shift+點擊:取代)",
|
"sendToWorkflow": "傳送到 ComfyUI(點擊:附加,Shift+點擊:取代)",
|
||||||
"copyLoRASyntax": "複製 LoRA 語法",
|
"copyLoRASyntax": "複製 LoRA 語法",
|
||||||
"checkpointNameCopied": "Checkpoint 名稱已複製",
|
"checkpointNameCopied": "Checkpoint 名稱已複製",
|
||||||
@@ -874,7 +875,7 @@
|
|||||||
"checkUpdates": "檢查更新",
|
"checkUpdates": "檢查更新",
|
||||||
"linkModel": "連結模型",
|
"linkModel": "連結模型",
|
||||||
"linkCivitai": "連結到 CivitAI",
|
"linkCivitai": "連結到 CivitAI",
|
||||||
"linkHuggingFace": "連結到 HuggingFace",
|
"linkModelSource": "[TODO: Translate] Link to Model Source",
|
||||||
"copySyntax": "複製 LoRA 語法",
|
"copySyntax": "複製 LoRA 語法",
|
||||||
"copyFilename": "複製模型檔名",
|
"copyFilename": "複製模型檔名",
|
||||||
"copyRecipeSyntax": "複製配方語法",
|
"copyRecipeSyntax": "複製配方語法",
|
||||||
@@ -1596,13 +1597,17 @@
|
|||||||
"pathPlaceholder": "輸入資料夾路徑或從下方樹狀結構選擇...",
|
"pathPlaceholder": "輸入資料夾路徑或從下方樹狀結構選擇...",
|
||||||
"root": "根目錄"
|
"root": "根目錄"
|
||||||
},
|
},
|
||||||
"linkHuggingFace": {
|
"linkModelSource": {
|
||||||
"title": "連結到 HuggingFace",
|
"title": "[TODO: Translate] Link to Model Source",
|
||||||
"infoText": "貼上 HuggingFace 倉庫 URL 以關聯此模型。關聯後可啟用 AI 中繼資料增強功能。",
|
"infoText": "[TODO: Translate] Paste the model page URL to associate this model with its source. Linking enables AI-powered metadata enrichment for Hugging Face and ModelScope models.",
|
||||||
"urlLabel": "HuggingFace 倉庫 URL:",
|
"urlLabel": "[TODO: Translate] Model Page URL:",
|
||||||
"urlPlaceholder": "https://huggingface.co/user/repo",
|
"urlPlaceholder": "[TODO: Translate] https://huggingface.co/user/repo",
|
||||||
"helpText": "請輸入完整的 HuggingFace 倉庫 URL。",
|
"helpText": "[TODO: Translate] Enter the full URL of the model page. Supported sites:",
|
||||||
"confirmAction": "儲存並連結"
|
"enrichNote": "[TODO: Translate] Note: AI metadata enrichment requires an accessible model card. TensorArt pages are not readable by the backend, so only the link is stored.",
|
||||||
|
"urlRequired": "[TODO: Translate] Please enter a model page URL.",
|
||||||
|
"invalidUrl": "[TODO: Translate] Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.",
|
||||||
|
"linking": "[TODO: Translate] Linking model source...",
|
||||||
|
"confirmAction": "[TODO: Translate] Save & Link"
|
||||||
},
|
},
|
||||||
"relinkCivitai": {
|
"relinkCivitai": {
|
||||||
"title": "重新連結至 CivitAI",
|
"title": "重新連結至 CivitAI",
|
||||||
@@ -1847,7 +1852,7 @@
|
|||||||
"empty": "此模型尚無版本歷史。",
|
"empty": "此模型尚無版本歷史。",
|
||||||
"error": "載入版本失敗。",
|
"error": "載入版本失敗。",
|
||||||
"missingModelId": "此模型缺少 CivitAI 模型 ID。",
|
"missingModelId": "此模型缺少 CivitAI 模型 ID。",
|
||||||
"hfGroupInfo": "這是一個 HuggingFace 模型組。打開庫頁面即可在網格中查看所有版本。",
|
"sourceGroupInfo": "[TODO: Translate] This is a {source} model group. Open the library to see all versions in the grid.",
|
||||||
"confirm": {
|
"confirm": {
|
||||||
"delete": "要從庫中刪除此版本嗎?"
|
"delete": "要從庫中刪除此版本嗎?"
|
||||||
},
|
},
|
||||||
@@ -2483,7 +2488,9 @@
|
|||||||
"linkCivArchSuccess": "模型已成功透過 CivitArchive 重新連結",
|
"linkCivArchSuccess": "模型已成功透過 CivitArchive 重新連結",
|
||||||
"fetchMetadataFirst": "請先從 CivitAI 取得 metadata",
|
"fetchMetadataFirst": "請先從 CivitAI 取得 metadata",
|
||||||
"noCivitaiInfo": "無 CivitAI 資訊",
|
"noCivitaiInfo": "無 CivitAI 資訊",
|
||||||
"missingHash": "模型雜湊不可用"
|
"missingHash": "模型雜湊不可用",
|
||||||
|
"enrichNeedsSource": "[TODO: Translate] Link this model to a model source first (Link Model → Link to Model Source)",
|
||||||
|
"enrichUnsupportedSource": "[TODO: Translate] AI enrichment is not available for {source} models"
|
||||||
},
|
},
|
||||||
"exampleImages": {
|
"exampleImages": {
|
||||||
"pathUpdated": "範例圖片路徑已更新",
|
"pathUpdated": "範例圖片路徑已更新",
|
||||||
|
|||||||
@@ -22,6 +22,11 @@ from ...services.downloader import (
|
|||||||
get_downloader,
|
get_downloader,
|
||||||
)
|
)
|
||||||
from ...services.aria2_downloader import Aria2Downloader
|
from ...services.aria2_downloader import Aria2Downloader
|
||||||
|
from ...services.model_sources import (
|
||||||
|
detect_source,
|
||||||
|
list_sources,
|
||||||
|
normalize_metadata_source,
|
||||||
|
)
|
||||||
from ...services.settings_manager import get_settings_manager
|
from ...services.settings_manager import get_settings_manager
|
||||||
from ...services.service_registry import ServiceRegistry
|
from ...services.service_registry import ServiceRegistry
|
||||||
from ...services.websocket_manager import ws_manager
|
from ...services.websocket_manager import ws_manager
|
||||||
@@ -120,6 +125,8 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
|
|||||||
|
|
||||||
# 2. Overlay HF-specific fields
|
# 2. Overlay HF-specific fields
|
||||||
metadata._unknown_fields["hf_url"] = hf_url
|
metadata._unknown_fields["hf_url"] = hf_url
|
||||||
|
metadata._unknown_fields["source_url"] = hf_url
|
||||||
|
metadata._unknown_fields["source_platform"] = "huggingface"
|
||||||
metadata.from_civitai = False # HF models are not from CivitAI
|
metadata.from_civitai = False # HF models are not from CivitAI
|
||||||
|
|
||||||
# 3. Save metadata atomically
|
# 3. Save metadata atomically
|
||||||
@@ -189,27 +196,72 @@ async def _add_to_scanner_cache(dest_path: str, metadata: dict[str, Any]) -> Non
|
|||||||
class HfHandler:
|
class HfHandler:
|
||||||
"""Handle Hugging Face model browsing and download."""
|
"""Handle Hugging Face model browsing and download."""
|
||||||
|
|
||||||
|
async def get_model_sources(self, request: web.Request) -> web.Response:
|
||||||
|
"""List the external model sites the UI can link a model to.
|
||||||
|
|
||||||
|
Used by the "Link Model" dialog to validate URLs client-side and to
|
||||||
|
explain which sites support AI metadata enrichment.
|
||||||
|
"""
|
||||||
|
|
||||||
|
return web.json_response([
|
||||||
|
{
|
||||||
|
"platform": source.platform,
|
||||||
|
"label": source.label,
|
||||||
|
"supports_enrichment": source.supports_enrichment,
|
||||||
|
"supports_download": source.supports_download,
|
||||||
|
"example_url": source.canonical_url(
|
||||||
|
"user/repo" if source.platform != "tensorart" else "827823520299086029"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
for source in list_sources()
|
||||||
|
])
|
||||||
|
|
||||||
async def set_hf_url(self, request: web.Request) -> web.Response:
|
async def set_hf_url(self, request: web.Request) -> web.Response:
|
||||||
|
"""Link a model file to its page on an external model site.
|
||||||
|
|
||||||
|
Accepts ``source_url`` (preferred) or the legacy ``hf_url`` /
|
||||||
|
``url`` payload key. Hugging Face, ModelScope, and TensorArt URLs
|
||||||
|
are recognised; the platform is stored alongside the canonical URL.
|
||||||
|
TensorArt models can be linked and browsed, but not AI-enriched.
|
||||||
|
"""
|
||||||
|
|
||||||
try:
|
try:
|
||||||
payload: dict[str, Any] = await request.json()
|
payload: dict[str, Any] = await request.json()
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
|
return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
|
||||||
|
|
||||||
file_path = (payload.get("file_path") or "").strip()
|
file_path = (payload.get("file_path") or "").strip()
|
||||||
hf_url = (payload.get("hf_url") or "").strip()
|
raw_url = (
|
||||||
|
payload.get("source_url")
|
||||||
if not file_path or not hf_url:
|
or payload.get("hf_url")
|
||||||
return web.json_response(
|
or payload.get("url")
|
||||||
{"success": False, "error": "Missing required fields: 'file_path' and 'hf_url'"},
|
or ""
|
||||||
status=400,
|
|
||||||
)
|
)
|
||||||
|
source_url = raw_url.strip() if isinstance(raw_url, str) else ""
|
||||||
|
|
||||||
m = re.match(r"^https?://huggingface\.co/([^/]+/[^/]+)/?$", hf_url)
|
if not file_path or not source_url:
|
||||||
if not m:
|
|
||||||
return web.json_response(
|
return web.json_response(
|
||||||
{
|
{
|
||||||
"success": False,
|
"success": False,
|
||||||
"error": "Invalid HuggingFace URL. Expected format: https://huggingface.co/user/repo",
|
"error": "Missing required fields: 'file_path' and 'source_url'",
|
||||||
|
},
|
||||||
|
status=400,
|
||||||
|
)
|
||||||
|
|
||||||
|
ref = detect_source(source_url, strict=True)
|
||||||
|
if ref is None:
|
||||||
|
return web.json_response(
|
||||||
|
{
|
||||||
|
"success": False,
|
||||||
|
"error": (
|
||||||
|
"Unsupported model URL. Supported formats: "
|
||||||
|
+ ", ".join(
|
||||||
|
f"{s.label} ({s.canonical_url('user/repo')})"
|
||||||
|
if s.platform != "tensorart"
|
||||||
|
else f"{s.label} (https://tensor.art/models/<id>)"
|
||||||
|
for s in list_sources()
|
||||||
|
)
|
||||||
|
),
|
||||||
},
|
},
|
||||||
status=400,
|
status=400,
|
||||||
)
|
)
|
||||||
@@ -225,37 +277,61 @@ class HfHandler:
|
|||||||
return web.json_response(
|
return web.json_response(
|
||||||
{
|
{
|
||||||
"success": False,
|
"success": False,
|
||||||
"error": "File is not within any configured model directory. Cannot link to HuggingFace.",
|
"error": "File is not within any configured model directory. Cannot link to a model source.",
|
||||||
},
|
},
|
||||||
status=400,
|
status=400,
|
||||||
)
|
)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
existing = await MetadataManager.load_metadata_payload(file_path)
|
existing = await MetadataManager.load_metadata_payload(file_path)
|
||||||
if existing.get("hf_url") == hf_url:
|
|
||||||
|
already_linked = (
|
||||||
|
(existing.get("source_url") or "").strip() == ref.url
|
||||||
|
and (existing.get("source_platform") or "").strip().lower()
|
||||||
|
== ref.platform
|
||||||
|
) or (
|
||||||
|
not existing.get("source_url")
|
||||||
|
and ref.platform == "huggingface"
|
||||||
|
and (existing.get("hf_url") or "").strip() == ref.url
|
||||||
|
)
|
||||||
|
if already_linked:
|
||||||
return web.json_response({
|
return web.json_response({
|
||||||
"success": True,
|
"success": True,
|
||||||
"message": "hf_url already set",
|
"message": "source_url already set",
|
||||||
"hf_url": hf_url,
|
"source_url": ref.url,
|
||||||
|
"source_platform": ref.platform,
|
||||||
|
"hf_url": ref.url if ref.platform == "huggingface" else "",
|
||||||
})
|
})
|
||||||
|
|
||||||
existing["hf_url"] = hf_url
|
existing["source_url"] = ref.url
|
||||||
|
existing["source_platform"] = ref.platform
|
||||||
|
if ref.platform == "huggingface":
|
||||||
|
existing["hf_url"] = ref.url
|
||||||
|
else:
|
||||||
|
existing.pop("hf_url", None)
|
||||||
|
normalize_metadata_source(existing)
|
||||||
|
|
||||||
# NOTE: deliberately do NOT touch `from_civitai` here. It records
|
# NOTE: deliberately do NOT touch `from_civitai` here. It records
|
||||||
# where the metadata came from, and the UI must show the CivitAI
|
# where the metadata came from, and the UI must show the CivitAI
|
||||||
# link whenever CivitAI data is present — linking HuggingFace must
|
# link whenever CivitAI data is present — linking an external
|
||||||
# not hide it (#1094). HF provenance is tracked via `hf_url`.
|
# source must not hide it (#1094). Source provenance is tracked
|
||||||
|
# via `source_platform` / `source_url`.
|
||||||
await MetadataManager.save_metadata(file_path, existing)
|
await MetadataManager.save_metadata(file_path, existing)
|
||||||
|
|
||||||
await _add_to_scanner_cache(file_path, existing)
|
await _add_to_scanner_cache(file_path, existing)
|
||||||
|
|
||||||
logger.info("Set hf_url=%s for %s", hf_url, file_path)
|
logger.info(
|
||||||
|
"Linked %s to %s source (%s)", file_path, ref.platform, ref.url
|
||||||
|
)
|
||||||
return web.json_response({
|
return web.json_response({
|
||||||
"success": True,
|
"success": True,
|
||||||
"message": f"hf_url set to {hf_url}",
|
"message": f"Linked to {ref.url}",
|
||||||
"hf_url": hf_url,
|
"source_url": ref.url,
|
||||||
|
"source_platform": ref.platform,
|
||||||
|
"hf_url": existing.get("hf_url", ""),
|
||||||
})
|
})
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.error("Failed to set hf_url for %s: %s", file_path, exc)
|
logger.error("Failed to link %s to a model source: %s", file_path, exc)
|
||||||
return web.json_response(
|
return web.json_response(
|
||||||
{"success": False, "error": str(exc)},
|
{"success": False, "error": str(exc)},
|
||||||
status=500,
|
status=500,
|
||||||
|
|||||||
@@ -4079,6 +4079,7 @@ class MiscHandlerSet:
|
|||||||
"get_hf_repo_files": self.hf_handler.get_hf_repo_files,
|
"get_hf_repo_files": self.hf_handler.get_hf_repo_files,
|
||||||
"download_hf_model": self.hf_handler.download_hf_model,
|
"download_hf_model": self.hf_handler.download_hf_model,
|
||||||
"set_hf_url": self.hf_handler.set_hf_url,
|
"set_hf_url": self.hf_handler.set_hf_url,
|
||||||
|
"get_model_sources": self.hf_handler.get_model_sources,
|
||||||
# Agent skill handlers
|
# Agent skill handlers
|
||||||
"get_agent_skills": self.agent_handler.get_agent_skills,
|
"get_agent_skills": self.agent_handler.get_agent_skills,
|
||||||
"execute_agent_skill": self.agent_handler.execute_agent_skill,
|
"execute_agent_skill": self.agent_handler.execute_agent_skill,
|
||||||
|
|||||||
@@ -113,6 +113,10 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
|
|||||||
RouteDefinition(
|
RouteDefinition(
|
||||||
"POST", "/api/lm/set-hf-url", "set_hf_url"
|
"POST", "/api/lm/set-hf-url", "set_hf_url"
|
||||||
),
|
),
|
||||||
|
# Supported external model sites (Hugging Face / ModelScope / TensorArt)
|
||||||
|
RouteDefinition(
|
||||||
|
"GET", "/api/lm/model-sources", "get_model_sources"
|
||||||
|
),
|
||||||
# Agent skill endpoints
|
# Agent skill endpoints
|
||||||
RouteDefinition(
|
RouteDefinition(
|
||||||
"GET", "/api/lm/agent/skills", "get_agent_skills"
|
"GET", "/api/lm/agent/skills", "get_agent_skills"
|
||||||
|
|||||||
@@ -19,16 +19,18 @@ from __future__ import annotations
|
|||||||
import asyncio
|
import asyncio
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
import re
|
import re
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from typing import Any, Dict, List, Optional
|
from typing import Any, Dict, List, Optional
|
||||||
|
|
||||||
import aiohttp
|
|
||||||
|
|
||||||
import os
|
|
||||||
|
|
||||||
from ...config import config
|
from ...config import config
|
||||||
from ..llm_service import LLMService
|
from ..llm_service import LLMService
|
||||||
|
from ..model_sources import (
|
||||||
|
get_source,
|
||||||
|
resolve_source_ref,
|
||||||
|
source_label,
|
||||||
|
)
|
||||||
from ..websocket_manager import ws_manager
|
from ..websocket_manager import ws_manager
|
||||||
from .post_processor import PostProcessor
|
from .post_processor import PostProcessor
|
||||||
from .skill_registry import SkillRegistry
|
from .skill_registry import SkillRegistry
|
||||||
@@ -267,11 +269,14 @@ class AgentService:
|
|||||||
from ...metadata_ops import read_metadata
|
from ...metadata_ops import read_metadata
|
||||||
metadata = await read_metadata(model_path)
|
metadata = await read_metadata(model_path)
|
||||||
|
|
||||||
# Fast-fail: enrich_hf_metadata requires hf_url to have HF README context
|
# Fast-fail: enrich_hf_metadata needs an external model source
|
||||||
if skill_name == "enrich_hf_metadata" and not metadata.get("hf_url", ""):
|
# that exposes an accessible model card.
|
||||||
|
if skill_name == "enrich_hf_metadata":
|
||||||
|
skip_reason = self._enrichment_skip_reason(metadata)
|
||||||
|
if skip_reason:
|
||||||
logger.info(
|
logger.info(
|
||||||
"[%s] SKIP %s — no hf_url in metadata",
|
"[%s] SKIP %s — %s",
|
||||||
skill_name, model_filename,
|
skill_name, model_filename, skip_reason,
|
||||||
)
|
)
|
||||||
skipped_count += 1
|
skipped_count += 1
|
||||||
skip_model = True
|
skip_model = True
|
||||||
@@ -358,6 +363,28 @@ class AgentService:
|
|||||||
# Base model grouping (keeps the prompt compact)
|
# Base model grouping (keeps the prompt compact)
|
||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _enrichment_skip_reason(metadata: Dict[str, Any]) -> str:
|
||||||
|
"""Return why ``enrich_hf_metadata`` cannot run, or ``""`` if it can.
|
||||||
|
|
||||||
|
Distinguishes the three cases the user can act on: no source linked,
|
||||||
|
a source we don't know, and a known source whose model card is not
|
||||||
|
reachable from the backend (TensorArt).
|
||||||
|
"""
|
||||||
|
|
||||||
|
ref = resolve_source_ref(metadata)
|
||||||
|
if ref is None:
|
||||||
|
return "no model source linked (source_url missing)"
|
||||||
|
source = get_source(ref.platform)
|
||||||
|
if source is None:
|
||||||
|
return f"unsupported model source platform '{ref.platform}'"
|
||||||
|
if not source.supports_enrichment:
|
||||||
|
return (
|
||||||
|
f"{source.label} does not expose a model card to the backend; "
|
||||||
|
"AI metadata enrichment is not available for this source"
|
||||||
|
)
|
||||||
|
return ""
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _format_base_models(models: List[str]) -> str:
|
def _format_base_models(models: List[str]) -> str:
|
||||||
"""Format the base model list as a flat, one-per-line list.
|
"""Format the base model list as a flat, one-per-line list.
|
||||||
@@ -388,6 +415,14 @@ class AgentService:
|
|||||||
context: Dict[str, Any] = {
|
context: Dict[str, Any] = {
|
||||||
"model_path": model_path,
|
"model_path": model_path,
|
||||||
"model_basename": "",
|
"model_basename": "",
|
||||||
|
# Canonical external-source variables
|
||||||
|
"source_url": "",
|
||||||
|
"source_id": "",
|
||||||
|
"source_platform": "",
|
||||||
|
"source_label": "",
|
||||||
|
"asset_base_url": "",
|
||||||
|
# Legacy Hugging Face aliases (kept so older prompt templates and
|
||||||
|
# third-party skills keep rendering)
|
||||||
"hf_url": "",
|
"hf_url": "",
|
||||||
"repo": "",
|
"repo": "",
|
||||||
"readme_content": "",
|
"readme_content": "",
|
||||||
@@ -411,12 +446,20 @@ class AgentService:
|
|||||||
"size": metadata.get("size", 0),
|
"size": metadata.get("size", 0),
|
||||||
}
|
}
|
||||||
|
|
||||||
hf_url = metadata.get("hf_url", "")
|
ref = resolve_source_ref(metadata)
|
||||||
context["hf_url"] = hf_url
|
if ref is not None:
|
||||||
repo = self._extract_repo_from_url(hf_url) if hf_url else ""
|
context["source_url"] = ref.url
|
||||||
context["repo"] = repo or ""
|
context["source_id"] = ref.source_id
|
||||||
if repo:
|
context["source_platform"] = ref.platform
|
||||||
readme = await self._fetch_readme(repo)
|
context["source_label"] = source_label(ref.platform, ref.platform)
|
||||||
|
if ref.platform == "huggingface":
|
||||||
|
context["hf_url"] = ref.url
|
||||||
|
context["repo"] = ref.source_id
|
||||||
|
|
||||||
|
source = get_source(ref.platform) if ref is not None else None
|
||||||
|
if ref is not None and source is not None and source.supports_enrichment:
|
||||||
|
context["asset_base_url"] = source.asset_base_url(ref.source_id)
|
||||||
|
readme = await source.fetch_model_card(ref.source_id)
|
||||||
# Trim README to the section relevant to this model file
|
# Trim README to the section relevant to this model file
|
||||||
# (collection repos often have multiple models in one README).
|
# (collection repos often have multiple models in one README).
|
||||||
if readme and raw_basename:
|
if readme and raw_basename:
|
||||||
@@ -458,20 +501,14 @@ class AgentService:
|
|||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
async def _fetch_readme(repo: str) -> str:
|
async def _fetch_readme(repo: str) -> str:
|
||||||
"""Fetch README.md from HuggingFace (tries ``main``, then ``master``)."""
|
"""Fetch a Hugging Face README (tries ``main``, then ``master``).
|
||||||
async with aiohttp.ClientSession(
|
|
||||||
headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
|
Kept for backward compatibility; new code should go through the
|
||||||
timeout=aiohttp.ClientTimeout(total=30),
|
model-source registry so every supported site works.
|
||||||
) as session:
|
"""
|
||||||
for branch in ("main", "master"):
|
from ..model_sources import HuggingFaceSource
|
||||||
url = f"https://huggingface.co/{repo}/raw/{branch}/README.md"
|
|
||||||
try:
|
return await HuggingFaceSource().fetch_model_card(repo)
|
||||||
async with session.get(url) as resp:
|
|
||||||
if resp.status == 200:
|
|
||||||
return await resp.text()
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug("Failed to fetch README from %s: %s", url, exc)
|
|
||||||
return ""
|
|
||||||
|
|
||||||
async def _emit_progress(
|
async def _emit_progress(
|
||||||
self,
|
self,
|
||||||
|
|||||||
@@ -78,6 +78,7 @@ class PostProcessor:
|
|||||||
download_preview,
|
download_preview,
|
||||||
refresh_cache,
|
refresh_cache,
|
||||||
)
|
)
|
||||||
|
from ..model_sources import get_source, has_external_source, resolve_source_ref
|
||||||
from .skills.enrich_hf_metadata.readme_processor import (
|
from .skills.enrich_hf_metadata.readme_processor import (
|
||||||
convert_readme_to_html,
|
convert_readme_to_html,
|
||||||
extract_gallery_images,
|
extract_gallery_images,
|
||||||
@@ -85,17 +86,25 @@ class PostProcessor:
|
|||||||
extract_relevant_section,
|
extract_relevant_section,
|
||||||
extract_simple_markdown_images,
|
extract_simple_markdown_images,
|
||||||
extract_html_img_tags,
|
extract_html_img_tags,
|
||||||
extract_repo_from_hf_url,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
updated_fields: List[str] = []
|
updated_fields: List[str] = []
|
||||||
preview_downloaded = False
|
preview_downloaded = False
|
||||||
|
|
||||||
# -- Determine whether this is an HF-sourced model -----------------
|
# -- Determine whether this is an externally-sourced model ---------
|
||||||
# Key off `hf_url` directly: `from_civitai` records provenance and can
|
# Key off the source fields directly: `from_civitai` records provenance
|
||||||
# be true for a model that is also linked to HuggingFace (both sources
|
# and can be true for a model that is also linked to an external site
|
||||||
# coexist, see #1094), so it must not gate HF enrichment.
|
# (both sources coexist, see #1094), so it must not gate enrichment.
|
||||||
is_hf_model = bool(metadata.get("hf_url", ""))
|
is_source_model = has_external_source(metadata)
|
||||||
|
|
||||||
|
source_ref = resolve_source_ref(metadata)
|
||||||
|
source = get_source(source_ref.platform) if source_ref else None
|
||||||
|
source_id = source_ref.source_id if source_ref else ""
|
||||||
|
asset_base_url = (
|
||||||
|
source.asset_base_url(source_id)
|
||||||
|
if source is not None and source_id
|
||||||
|
else None
|
||||||
|
)
|
||||||
|
|
||||||
# -- Collect updates -----------------------------------------------
|
# -- Collect updates -----------------------------------------------
|
||||||
updates: Dict[str, Any] = {}
|
updates: Dict[str, Any] = {}
|
||||||
@@ -103,7 +112,7 @@ class PostProcessor:
|
|||||||
# base_model
|
# base_model
|
||||||
new_base = (llm_output.get("base_model") or "").strip()
|
new_base = (llm_output.get("base_model") or "").strip()
|
||||||
current_base = metadata.get("base_model", "") or ""
|
current_base = metadata.get("base_model", "") or ""
|
||||||
if new_base and self._should_overwrite(current_base, is_hf_model):
|
if new_base and self._should_overwrite(current_base, is_source_model):
|
||||||
updates["base_model"] = new_base
|
updates["base_model"] = new_base
|
||||||
|
|
||||||
# trigger words → civitai.trainedWords
|
# trigger words → civitai.trainedWords
|
||||||
@@ -115,7 +124,7 @@ class PostProcessor:
|
|||||||
trigger_words_empty = not cleaned
|
trigger_words_empty = not cleaned
|
||||||
current_civitai = metadata.get("civitai") or {}
|
current_civitai = metadata.get("civitai") or {}
|
||||||
current_triggers = current_civitai.get("trainedWords") or []
|
current_triggers = current_civitai.get("trainedWords") or []
|
||||||
if self._should_overwrite_list(current_triggers, is_hf_model):
|
if self._should_overwrite_list(current_triggers, is_source_model):
|
||||||
trig_civitai = dict(current_civitai)
|
trig_civitai = dict(current_civitai)
|
||||||
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
||||||
trig_civitai.update(updates["civitai"])
|
trig_civitai.update(updates["civitai"])
|
||||||
@@ -123,14 +132,14 @@ class PostProcessor:
|
|||||||
updates["civitai"] = trig_civitai
|
updates["civitai"] = trig_civitai
|
||||||
|
|
||||||
# modelDescription — from raw README content (converted to HTML)
|
# modelDescription — from raw README content (converted to HTML)
|
||||||
if readme_content and is_hf_model:
|
if readme_content and is_source_model:
|
||||||
converted = convert_readme_to_html(readme_content)
|
converted = convert_readme_to_html(readme_content)
|
||||||
if converted:
|
if converted:
|
||||||
updates["modelDescription"] = converted
|
updates["modelDescription"] = converted
|
||||||
|
|
||||||
# short_description → civitai.description (for "About this version")
|
# short_description → civitai.description (for "About this version")
|
||||||
short_desc = (llm_output.get("short_description") or "").strip()
|
short_desc = (llm_output.get("short_description") or "").strip()
|
||||||
if short_desc and is_hf_model:
|
if short_desc and is_source_model:
|
||||||
current_civitai = metadata.get("civitai") or {}
|
current_civitai = metadata.get("civitai") or {}
|
||||||
desc_civitai = dict(current_civitai)
|
desc_civitai = dict(current_civitai)
|
||||||
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
if "civitai" in updates and isinstance(updates["civitai"], dict):
|
||||||
@@ -141,9 +150,8 @@ class PostProcessor:
|
|||||||
# gallery images → civitai.images (from YAML frontmatter widget entries
|
# gallery images → civitai.images (from YAML frontmatter widget entries
|
||||||
# and Sample Gallery markdown tables in the README body)
|
# and Sample Gallery markdown tables in the README body)
|
||||||
gallery_images: List[Dict[str, Any]] = []
|
gallery_images: List[Dict[str, Any]] = []
|
||||||
if readme_content and is_hf_model:
|
if readme_content and is_source_model:
|
||||||
hf_url = metadata.get("hf_url", "") or ""
|
repo = source_id
|
||||||
repo = extract_repo_from_hf_url(hf_url)
|
|
||||||
if repo:
|
if repo:
|
||||||
rec_w = llm_output.get("recommended_width") or 0
|
rec_w = llm_output.get("recommended_width") or 0
|
||||||
rec_h = llm_output.get("recommended_height") or 0
|
rec_h = llm_output.get("recommended_height") or 0
|
||||||
@@ -152,6 +160,7 @@ class PostProcessor:
|
|||||||
gallery = extract_gallery_images(
|
gallery = extract_gallery_images(
|
||||||
readme_content, repo,
|
readme_content, repo,
|
||||||
default_width=rec_w, default_height=rec_h,
|
default_width=rec_w, default_height=rec_h,
|
||||||
|
base_url=asset_base_url,
|
||||||
)
|
)
|
||||||
|
|
||||||
# 2. Sample Gallery table images (markdown body), deduplicated
|
# 2. Sample Gallery table images (markdown body), deduplicated
|
||||||
@@ -160,6 +169,7 @@ class PostProcessor:
|
|||||||
readme_content, repo,
|
readme_content, repo,
|
||||||
existing_urls=existing_urls,
|
existing_urls=existing_urls,
|
||||||
default_width=rec_w, default_height=rec_h,
|
default_width=rec_w, default_height=rec_h,
|
||||||
|
base_url=asset_base_url,
|
||||||
)
|
)
|
||||||
existing_urls.update(img["url"] for img in table_images if img.get("url"))
|
existing_urls.update(img["url"] for img in table_images if img.get("url"))
|
||||||
|
|
||||||
@@ -168,6 +178,7 @@ class PostProcessor:
|
|||||||
readme_content, repo,
|
readme_content, repo,
|
||||||
existing_urls=existing_urls,
|
existing_urls=existing_urls,
|
||||||
default_width=rec_w, default_height=rec_h,
|
default_width=rec_w, default_height=rec_h,
|
||||||
|
base_url=asset_base_url,
|
||||||
)
|
)
|
||||||
existing_urls.update(img["url"] for img in simple_images if img.get("url"))
|
existing_urls.update(img["url"] for img in simple_images if img.get("url"))
|
||||||
|
|
||||||
@@ -176,6 +187,7 @@ class PostProcessor:
|
|||||||
readme_content, repo,
|
readme_content, repo,
|
||||||
existing_urls=existing_urls,
|
existing_urls=existing_urls,
|
||||||
default_width=rec_w, default_height=rec_h,
|
default_width=rec_w, default_height=rec_h,
|
||||||
|
base_url=asset_base_url,
|
||||||
)
|
)
|
||||||
|
|
||||||
all_images = gallery + table_images + simple_images + html_images
|
all_images = gallery + table_images + simple_images + html_images
|
||||||
@@ -193,7 +205,7 @@ class PostProcessor:
|
|||||||
if isinstance(new_tags, list) and new_tags:
|
if isinstance(new_tags, list) and new_tags:
|
||||||
existing_tags = metadata.get("tags") or []
|
existing_tags = metadata.get("tags") or []
|
||||||
merged = self._merge_tags(existing_tags, new_tags)
|
merged = self._merge_tags(existing_tags, new_tags)
|
||||||
if len(merged) > len(existing_tags) or is_hf_model:
|
if len(merged) > len(existing_tags) or is_source_model:
|
||||||
updates["tags"] = merged
|
updates["tags"] = merged
|
||||||
|
|
||||||
# metadata_source & llm_enriched_at (always set)
|
# metadata_source & llm_enriched_at (always set)
|
||||||
@@ -222,7 +234,7 @@ class PostProcessor:
|
|||||||
# README, find the first gallery image from the *model-specific
|
# README, find the first gallery image from the *model-specific
|
||||||
# section* of the README (not the repo-wide first image, which
|
# section* of the README (not the repo-wide first image, which
|
||||||
# belongs to a different model in collection repos).
|
# belongs to a different model in collection repos).
|
||||||
if not preview_remote_url and readme_content and is_hf_model:
|
if not preview_remote_url and readme_content and is_source_model:
|
||||||
model_basename = os.path.splitext(os.path.basename(model_path))[0]
|
model_basename = os.path.splitext(os.path.basename(model_path))[0]
|
||||||
relevant_section = extract_relevant_section(
|
relevant_section = extract_relevant_section(
|
||||||
readme_content, model_basename,
|
readme_content, model_basename,
|
||||||
@@ -279,16 +291,16 @@ class PostProcessor:
|
|||||||
# ------------------------------------------------------------------
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _should_overwrite(current_value: str, is_hf_model: bool) -> bool:
|
def _should_overwrite(current_value: str, is_source_model: bool) -> bool:
|
||||||
"""Return ``True`` when a scalar field should be overwritten."""
|
"""Return ``True`` when a scalar field should be overwritten."""
|
||||||
return is_hf_model or not current_value or current_value.lower() in (
|
return is_source_model or not current_value or current_value.lower() in (
|
||||||
"", "unknown",
|
"", "unknown",
|
||||||
)
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _should_overwrite_list(current_list: List[str], is_hf_model: bool) -> bool:
|
def _should_overwrite_list(current_list: List[str], is_source_model: bool) -> bool:
|
||||||
"""Return ``True`` when a list field should be overwritten."""
|
"""Return ``True`` when a list field should be overwritten."""
|
||||||
return is_hf_model or not current_list
|
return is_source_model or not current_list
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _merge_tags(existing: List[str], new: List[str]) -> List[str]:
|
def _merge_tags(existing: List[str], new: List[str]) -> List[str]:
|
||||||
|
|||||||
@@ -1,20 +1,23 @@
|
|||||||
---
|
---
|
||||||
name: enrich_hf_metadata
|
name: enrich_hf_metadata
|
||||||
title: "Enrich Metadata from HuggingFace"
|
title: "Enrich Metadata from Model Card"
|
||||||
description: >
|
description: >
|
||||||
Parse the HuggingFace model card via LLM to extract description, trigger
|
Parse the model card (README) from HuggingFace, ModelScope, or any other
|
||||||
words, base model, tags, and preview image URL.
|
supported model site via LLM to extract description, trigger words, base
|
||||||
|
model, tags, and preview image URL.
|
||||||
llm_required: true
|
llm_required: true
|
||||||
---
|
---
|
||||||
|
|
||||||
You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a HuggingFace model card (README.md).
|
You are an expert assistant for AI image generation models. Your task is to extract structured metadata from a model card (README).
|
||||||
|
|
||||||
## Model Information
|
## Model Information
|
||||||
|
|
||||||
- **Repository**: {{hf_url}}
|
- **Source site**: {{source_label}} ({{source_platform}})
|
||||||
|
- **Model page**: {{source_url}}
|
||||||
- **Model file path**: {{model_path}}
|
- **Model file path**: {{model_path}}
|
||||||
- **Model filename**: {{model_basename}}
|
- **Model filename**: {{model_basename}}
|
||||||
- **Repository ID**: {{repo}}
|
- **Repository ID**: {{source_id}}
|
||||||
|
- **Repository raw-file base URL**: {{asset_base_url}}
|
||||||
|
|
||||||
## Current Metadata (may be incomplete)
|
## Current Metadata (may be incomplete)
|
||||||
|
|
||||||
@@ -39,7 +42,7 @@ name listed — do not invent aliases or modify variant suffixes.
|
|||||||
|
|
||||||
{{base_models}}
|
{{base_models}}
|
||||||
|
|
||||||
## HuggingFace README Content
|
## Model Card Content
|
||||||
|
|
||||||
```
|
```
|
||||||
{{readme_content}}
|
{{readme_content}}
|
||||||
@@ -92,7 +95,7 @@ The URL of the most suitable preview image from the README. Look for:
|
|||||||
- The YAML frontmatter `widget:` section (which often has `output.url` fields)
|
- The YAML frontmatter `widget:` section (which often has `output.url` fields)
|
||||||
- In collection repos: the sample images listed **under the section** for this specific model version
|
- In collection repos: the sample images listed **under the section** for this specific model version
|
||||||
- Generic `` in the body
|
- Generic `` in the body
|
||||||
Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL as `https://huggingface.co/{{repo}}/resolve/main/{filename}`. If no suitable image is found, return an empty string.
|
Choose the first image that appears to be a generation example (not a logo or diagram). Construct the absolute URL from the repository raw-file base URL (`{{asset_base_url}}`) plus the relative path. If no suitable image is found, return an empty string.
|
||||||
|
|
||||||
### notes
|
### notes
|
||||||
A plain-text summary of the model card's key practical usage information. Combine trigger words, style modifiers, recommended parameters (steps, CFG, resolution, sampler), and any setup tips into a readable paragraph. For collection repos, focus on the **specific model version** matching `{{model_basename}}`. Return empty string if the README has no useful usage info.
|
A plain-text summary of the model card's key practical usage information. Combine trigger words, style modifiers, recommended parameters (steps, CFG, resolution, sampler), and any setup tips into a readable paragraph. For collection repos, focus on the **specific model version** matching `{{model_basename}}`. Return empty string if the README has no useful usage info.
|
||||||
@@ -121,7 +124,7 @@ Your confidence level in the extracted data:
|
|||||||
|
|
||||||
## Important: Handling Collection Repos (multiple model files)
|
## Important: Handling Collection Repos (multiple model files)
|
||||||
|
|
||||||
Many HuggingFace repos contain **multiple model files** in a single repository
|
Many model repositories contain **multiple model files** in a single repository
|
||||||
(e.g. a "LoRA collection" with different styles/characters in separate files).
|
(e.g. a "LoRA collection" with different styles/characters in separate files).
|
||||||
|
|
||||||
The model file currently being enriched is: **`{{model_basename}}`**
|
The model file currently being enriched is: **`{{model_basename}}`**
|
||||||
|
|||||||
@@ -1,8 +1,15 @@
|
|||||||
"""HF README processing for the ``enrich_hf_metadata`` skill.
|
"""Model card (README) processing for the ``enrich_hf_metadata`` skill.
|
||||||
|
|
||||||
Provides README cleaning for LLM injection, gallery/image extraction from
|
Provides README cleaning for LLM injection, gallery/image extraction from
|
||||||
multiple formats (YAML widget, markdown, HTML ``<img>``, gallery tables),
|
multiple formats (YAML widget, markdown, HTML ``<img>``, gallery tables),
|
||||||
and section-based README trimming for collection repos.
|
and section-based README trimming for collection repos.
|
||||||
|
|
||||||
|
The extractors default to Hugging Face asset URLs, but every one of them
|
||||||
|
accepts an explicit ``base_url`` so the same parsing works for any model
|
||||||
|
source (ModelScope, ...). See :mod:`py.services.model_sources`.
|
||||||
|
|
||||||
|
This module deliberately has no package-relative imports: it is also loaded
|
||||||
|
standalone by the README-processing test harness.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -15,12 +22,25 @@ from typing import Any, List, Tuple
|
|||||||
_REPO_URL_PATTERN = re.compile(r"https?://huggingface\.co/([^/]+/[^/]+)")
|
_REPO_URL_PATTERN = re.compile(r"https?://huggingface\.co/([^/]+/[^/]+)")
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_asset_base_url(repo: str, base_url: str | None = None) -> str:
|
||||||
|
"""Return the base URL used to resolve repository-relative assets.
|
||||||
|
|
||||||
|
Falls back to the historical Hugging Face layout when *base_url* is not
|
||||||
|
supplied, so existing callers keep their behaviour.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if base_url:
|
||||||
|
return base_url.rstrip("/")
|
||||||
|
return f"https://huggingface.co/{repo}/resolve/main"
|
||||||
|
|
||||||
|
|
||||||
def extract_simple_markdown_images(
|
def extract_simple_markdown_images(
|
||||||
markdown_text: str,
|
markdown_text: str,
|
||||||
repo: str,
|
repo: str,
|
||||||
existing_urls: set[str] | None = None,
|
existing_urls: set[str] | None = None,
|
||||||
default_width: int = 512,
|
default_width: int = 512,
|
||||||
default_height: int = 512,
|
default_height: int = 512,
|
||||||
|
base_url: str | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Extract standalone markdown images from the README body.
|
"""Extract standalone markdown images from the README body.
|
||||||
|
|
||||||
@@ -32,10 +52,10 @@ def extract_simple_markdown_images(
|
|||||||
Returns a list of dicts in the same ``civitai.images`` format as
|
Returns a list of dicts in the same ``civitai.images`` format as
|
||||||
:func:`extract_gallery_images`.
|
:func:`extract_gallery_images`.
|
||||||
"""
|
"""
|
||||||
if not markdown_text or not repo:
|
if not markdown_text or not (repo or base_url):
|
||||||
return []
|
return []
|
||||||
|
|
||||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
base_url = resolve_asset_base_url(repo, base_url)
|
||||||
images: list[dict[str, Any]] = []
|
images: list[dict[str, Any]] = []
|
||||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||||
|
|
||||||
@@ -89,20 +109,21 @@ def extract_html_img_tags(
|
|||||||
existing_urls: set[str] | None = None,
|
existing_urls: set[str] | None = None,
|
||||||
default_width: int = 512,
|
default_width: int = 512,
|
||||||
default_height: int = 512,
|
default_height: int = 512,
|
||||||
|
base_url: str | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Extract image URLs from HTML ``<img src=\"...\">`` tags in the README.
|
"""Extract image URLs from HTML ``<img src=\"...\">`` tags in the README.
|
||||||
|
|
||||||
Many HF collection repos (e.g. ``deadman44/Z-Image_LoRA``) use raw HTML
|
Many HF collection repos (e.g. ``deadman44/Z-Image_LoRA``) use raw HTML
|
||||||
``<img>`` tags exclusively for their sample images, with no markdown
|
``<img>`` tags exclusively for their sample images, with no markdown
|
||||||
``![]()`` equivalents. This function finds those tags and constructs
|
``![]()`` equivalents. This function finds those tags and constructs
|
||||||
resolvable HF URLs.
|
resolvable URLs.
|
||||||
|
|
||||||
Returns a list of dicts in the ``civitai.images`` format.
|
Returns a list of dicts in the ``civitai.images`` format.
|
||||||
"""
|
"""
|
||||||
if not markdown_text or not repo:
|
if not markdown_text or not (repo or base_url):
|
||||||
return []
|
return []
|
||||||
|
|
||||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
base_url = resolve_asset_base_url(repo, base_url)
|
||||||
images: list[dict[str, Any]] = []
|
images: list[dict[str, Any]] = []
|
||||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||||
|
|
||||||
@@ -166,7 +187,7 @@ def extract_html_img_tags(
|
|||||||
|
|
||||||
def extract_repo_from_hf_url(hf_url: str) -> str:
|
def extract_repo_from_hf_url(hf_url: str) -> str:
|
||||||
"""Extract ``user/repo`` from a HuggingFace URL."""
|
"""Extract ``user/repo`` from a HuggingFace URL."""
|
||||||
m = _REPO_URL_PATTERN.match(hf_url)
|
m = _REPO_URL_PATTERN.match(hf_url or "")
|
||||||
return m.group(1) if m else ""
|
return m.group(1) if m else ""
|
||||||
|
|
||||||
|
|
||||||
@@ -175,21 +196,23 @@ def extract_gallery_images(
|
|||||||
repo: str,
|
repo: str,
|
||||||
default_width: int = 512,
|
default_width: int = 512,
|
||||||
default_height: int = 512,
|
default_height: int = 512,
|
||||||
|
base_url: str | None = None,
|
||||||
) -> List[dict[str, Any]]:
|
) -> List[dict[str, Any]]:
|
||||||
"""Extract widget/gallery images from the YAML frontmatter of a HF README.
|
"""Extract widget/gallery images from the YAML frontmatter of a README.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
markdown_text: Raw README content.
|
markdown_text: Raw README content.
|
||||||
repo: HF repo identifier (``user/repo``).
|
repo: Repository identifier (``user/repo``).
|
||||||
default_width: Fallback width when the README provides no dimension.
|
default_width: Fallback width when the README provides no dimension.
|
||||||
default_height: Fallback height when the README provides no dimension.
|
default_height: Fallback height when the README provides no dimension.
|
||||||
|
base_url: Overrides the asset base URL (defaults to Hugging Face).
|
||||||
|
|
||||||
Returns a list of dicts compatible with the ``civitai.images`` metadata
|
Returns a list of dicts compatible with the ``civitai.images`` metadata
|
||||||
format, each containing ``url`` (absolute HF URL), ``meta.prompt``,
|
format, each containing ``url`` (absolute), ``meta.prompt``,
|
||||||
``width``, ``height``, and ``type``. Returns an empty list when no
|
``width``, ``height``, and ``type``. Returns an empty list when no
|
||||||
widget entries are found or when *repo* is empty.
|
widget entries are found or when *repo* is empty.
|
||||||
"""
|
"""
|
||||||
if not markdown_text or not repo:
|
if not markdown_text or not (repo or base_url):
|
||||||
return []
|
return []
|
||||||
|
|
||||||
frontmatter = _extract_frontmatter(markdown_text)
|
frontmatter = _extract_frontmatter(markdown_text)
|
||||||
@@ -197,7 +220,7 @@ def extract_gallery_images(
|
|||||||
return []
|
return []
|
||||||
|
|
||||||
images: List[dict[str, Any]] = []
|
images: List[dict[str, Any]] = []
|
||||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
base_url = resolve_asset_base_url(repo, base_url)
|
||||||
w = default_width or 512
|
w = default_width or 512
|
||||||
h = default_height or 512
|
h = default_height or 512
|
||||||
|
|
||||||
@@ -279,10 +302,11 @@ def extract_gallery_table_images(
|
|||||||
existing_urls: set[str] | None = None,
|
existing_urls: set[str] | None = None,
|
||||||
default_width: int = 512,
|
default_width: int = 512,
|
||||||
default_height: int = 512,
|
default_height: int = 512,
|
||||||
|
base_url: str | None = None,
|
||||||
) -> list[dict[str, Any]]:
|
) -> list[dict[str, Any]]:
|
||||||
"""Extract images from ``| Preview | Prompt |`` markdown gallery tables.
|
"""Extract images from ``| Preview | Prompt |`` markdown gallery tables.
|
||||||
|
|
||||||
Many HF READMEs include a sample-gallery table in the body (outside
|
Many READMEs include a sample-gallery table in the body (outside
|
||||||
the YAML frontmatter) that shows generation examples with their
|
the YAML frontmatter) that shows generation examples with their
|
||||||
prompts. This function parses those tables and merges results with
|
prompts. This function parses those tables and merges results with
|
||||||
the widget-sourced images from :func:`extract_gallery_images`.
|
the widget-sourced images from :func:`extract_gallery_images`.
|
||||||
@@ -291,10 +315,10 @@ def extract_gallery_table_images(
|
|||||||
:func:`extract_gallery_images`. Already-seen URLs (from *existing_urls*)
|
:func:`extract_gallery_images`. Already-seen URLs (from *existing_urls*)
|
||||||
are skipped.
|
are skipped.
|
||||||
"""
|
"""
|
||||||
if not markdown_text or not repo:
|
if not markdown_text or not (repo or base_url):
|
||||||
return []
|
return []
|
||||||
|
|
||||||
base_url = f"https://huggingface.co/{repo}/resolve/main"
|
base_url = resolve_asset_base_url(repo, base_url)
|
||||||
images: list[dict[str, Any]] = []
|
images: list[dict[str, Any]] = []
|
||||||
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
seen_urls: set[str] = set(existing_urls) if existing_urls else set()
|
||||||
lines = markdown_text.split("\n")
|
lines = markdown_text.split("\n")
|
||||||
|
|||||||
@@ -21,6 +21,7 @@ from .model_query import (
|
|||||||
resolve_sub_type,
|
resolve_sub_type,
|
||||||
)
|
)
|
||||||
from .settings_manager import get_settings_manager
|
from .settings_manager import get_settings_manager
|
||||||
|
from .model_sources import source_group_key
|
||||||
from ..utils.civitai_utils import build_civitai_model_page_url
|
from ..utils.civitai_utils import build_civitai_model_page_url
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
@@ -742,29 +743,32 @@ class BaseModelService(ABC):
|
|||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_hf_group_key(item: Dict[str, Any]) -> Optional[str]:
|
def _extract_hf_group_key(item: Dict[str, Any]) -> Optional[str]:
|
||||||
"""Extract `hf:{owner}/{repo}` from item's ``hf_url``, or None."""
|
"""Extract `hf:{owner}/{repo}` from item's ``hf_url``, or None."""
|
||||||
hf_url = item.get("hf_url") if isinstance(item, dict) else None
|
key = BaseModelService._extract_source_group_key(item)
|
||||||
if not hf_url or not isinstance(hf_url, str):
|
return key if key and key.startswith("hf:") else None
|
||||||
return None
|
|
||||||
m = re.match(
|
@staticmethod
|
||||||
r"https?://huggingface\.co/([^/]+/[^/]+)", hf_url.strip()
|
def _extract_source_group_key(item: Dict[str, Any]) -> Optional[str]:
|
||||||
)
|
"""Return the external-source group key for *item*, or None.
|
||||||
if not m:
|
|
||||||
return None
|
Hugging Face keeps the historical ``hf:{owner}/{repo}`` shape; other
|
||||||
return f"hf:{m.group(1)}"
|
platforms use their own short prefix (``ms:`` / ``ta:``).
|
||||||
|
"""
|
||||||
|
return source_group_key(item)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_group_key(item: Dict[str, Any]) -> Union[int, str, None]:
|
def _extract_group_key(item: Dict[str, Any]) -> Union[int, str, None]:
|
||||||
"""Return the group identity key: CivitAI modelId (int) or HF repo (str).
|
"""Return the group identity key.
|
||||||
|
|
||||||
Preference order:
|
Preference order:
|
||||||
1. CivitAI ``modelId`` (int)
|
1. CivitAI ``modelId`` (int)
|
||||||
2. HF repo identity ``hf:{owner}/{repo}`` (str)
|
2. External model source identity, e.g. ``hf:{owner}/{repo}``,
|
||||||
|
``ms:{owner}/{repo}``, ``ta:{model_id}`` (str)
|
||||||
3. ``None`` (no known grouping source)
|
3. ``None`` (no known grouping source)
|
||||||
"""
|
"""
|
||||||
mid = BaseModelService._extract_model_id(item)
|
mid = BaseModelService._extract_model_id(item)
|
||||||
if mid is not None:
|
if mid is not None:
|
||||||
return mid
|
return mid
|
||||||
return BaseModelService._extract_hf_group_key(item)
|
return BaseModelService._extract_source_group_key(item)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_model_id(item: Dict[str, Any]) -> Optional[int]:
|
def _extract_model_id(item: Dict[str, Any]) -> Optional[int]:
|
||||||
|
|||||||
@@ -67,6 +67,8 @@ class CheckpointService(BaseModelService):
|
|||||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||||
"version_count": model_data.get("version_count"),
|
"version_count": model_data.get("version_count"),
|
||||||
|
"source_platform": model_data.get("source_platform", ""),
|
||||||
|
"source_url": model_data.get("source_url", ""),
|
||||||
"hf_url": model_data.get("hf_url", ""),
|
"hf_url": model_data.get("hf_url", ""),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -67,6 +67,8 @@ class EmbeddingService(BaseModelService):
|
|||||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||||
"version_count": model_data.get("version_count"),
|
"version_count": model_data.get("version_count"),
|
||||||
|
"source_platform": model_data.get("source_platform", ""),
|
||||||
|
"source_url": model_data.get("source_url", ""),
|
||||||
"hf_url": model_data.get("hf_url", ""),
|
"hf_url": model_data.get("hf_url", ""),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -79,6 +79,8 @@ class LoraService(BaseModelService):
|
|||||||
),
|
),
|
||||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||||
"version_count": model_data.get("version_count"),
|
"version_count": model_data.get("version_count"),
|
||||||
|
"source_platform": model_data.get("source_platform", ""),
|
||||||
|
"source_url": model_data.get("source_url", ""),
|
||||||
"hf_url": model_data.get("hf_url", ""),
|
"hf_url": model_data.get("hf_url", ""),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ from ..utils.model_utils import determine_base_model
|
|||||||
from ..utils.models import autov3_from_civitai_files
|
from ..utils.models import autov3_from_civitai_files
|
||||||
from .connectivity_guard import OFFLINE_FRIENDLY_MESSAGE, is_expected_offline_error
|
from .connectivity_guard import OFFLINE_FRIENDLY_MESSAGE, is_expected_offline_error
|
||||||
from .errors import RateLimitError
|
from .errors import RateLimitError
|
||||||
|
from .model_sources import has_external_source
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -222,9 +223,10 @@ class MetadataSyncService:
|
|||||||
error_msg = "CivitAI model is deleted and no archive provider is available"
|
error_msg = "CivitAI model is deleted and no archive provider is available"
|
||||||
return False, error_msg
|
return False, error_msg
|
||||||
else:
|
else:
|
||||||
is_hf_source = bool(model_data.get("hf_url"))
|
is_hf_source = has_external_source(model_data)
|
||||||
if is_hf_source:
|
if is_hf_source:
|
||||||
# HF-sourced model: only check CivitAI API directly.
|
# External-source model (Hugging Face / ModelScope /
|
||||||
|
# TensorArt): only check CivitAI API directly.
|
||||||
# CivArchive is almost guaranteed to have no record, and
|
# CivArchive is almost guaranteed to have no record, and
|
||||||
# hitting it wastes rate-limit budget.
|
# hitting it wastes rate-limit budget.
|
||||||
# Use a distinct provider name ("civitai_api" not None) so
|
# Use a distinct provider name ("civitai_api" not None) so
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ from ..utils.civitai_utils import resolve_license_info
|
|||||||
from .model_cache import ModelCache
|
from .model_cache import ModelCache
|
||||||
from .model_hash_index import ModelHashIndex
|
from .model_hash_index import ModelHashIndex
|
||||||
from .model_lifecycle_service import delete_model_artifacts, _require_path_in_library_roots
|
from .model_lifecycle_service import delete_model_artifacts, _require_path_in_library_roots
|
||||||
|
from .model_sources import normalize_metadata_source
|
||||||
from .service_registry import ServiceRegistry
|
from .service_registry import ServiceRegistry
|
||||||
from .websocket_manager import ws_manager
|
from .websocket_manager import ws_manager
|
||||||
from .persistent_model_cache import get_persistent_cache
|
from .persistent_model_cache import get_persistent_cache
|
||||||
@@ -387,8 +388,14 @@ class ModelScanner:
|
|||||||
'civitai': civitai_slim,
|
'civitai': civitai_slim,
|
||||||
'civitai_deleted': bool(get_value('civitai_deleted', False)),
|
'civitai_deleted': bool(get_value('civitai_deleted', False)),
|
||||||
'skip_metadata_refresh': bool(get_value('skip_metadata_refresh', False)),
|
'skip_metadata_refresh': bool(get_value('skip_metadata_refresh', False)),
|
||||||
|
# External model source (Hugging Face / ModelScope / TensorArt).
|
||||||
|
# `source_url` + `source_platform` are canonical; `hf_url` stays in
|
||||||
|
# sync as a legacy alias (normalised below).
|
||||||
|
'source_platform': get_value('source_platform', '') or '',
|
||||||
|
'source_url': get_value('source_url', '') or '',
|
||||||
'hf_url': get_value('hf_url', '') or '',
|
'hf_url': get_value('hf_url', '') or '',
|
||||||
}
|
}
|
||||||
|
normalize_metadata_source(entry)
|
||||||
|
|
||||||
license_source: Dict[str, Any] = {}
|
license_source: Dict[str, Any] = {}
|
||||||
if isinstance(civitai_full, Mapping):
|
if isinstance(civitai_full, Mapping):
|
||||||
|
|||||||
@@ -0,0 +1,61 @@
|
|||||||
|
"""External model-source providers (Hugging Face, ModelScope, TensorArt).
|
||||||
|
|
||||||
|
This package is the single abstraction over "a site that hosts models and
|
||||||
|
a model card". See :mod:`py.services.model_sources.base` for the provider
|
||||||
|
protocol and :mod:`py.services.model_sources.registry` for the lookup and
|
||||||
|
metadata-normalisation helpers used across the codebase.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .base import (
|
||||||
|
GROUP_PREFIXES,
|
||||||
|
HTTP_TIMEOUT,
|
||||||
|
ModelSource,
|
||||||
|
SourceRef,
|
||||||
|
USER_AGENT,
|
||||||
|
clean_source_url,
|
||||||
|
fetch_text,
|
||||||
|
)
|
||||||
|
from .huggingface import HuggingFaceSource
|
||||||
|
from .modelscope import ModelScopeSource
|
||||||
|
from .registry import (
|
||||||
|
LEGACY_HF_URL_FIELD,
|
||||||
|
SOURCE_PLATFORM_FIELD,
|
||||||
|
SOURCE_URL_FIELD,
|
||||||
|
detect_source,
|
||||||
|
get_source,
|
||||||
|
get_source_platform,
|
||||||
|
has_external_source,
|
||||||
|
list_sources,
|
||||||
|
normalize_metadata_source,
|
||||||
|
resolve_source_ref,
|
||||||
|
source_group_key,
|
||||||
|
source_label,
|
||||||
|
)
|
||||||
|
from .tensorart import TensorArtSource
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"GROUP_PREFIXES",
|
||||||
|
"HTTP_TIMEOUT",
|
||||||
|
"LEGACY_HF_URL_FIELD",
|
||||||
|
"ModelSource",
|
||||||
|
"HuggingFaceSource",
|
||||||
|
"ModelScopeSource",
|
||||||
|
"SOURCE_PLATFORM_FIELD",
|
||||||
|
"SOURCE_URL_FIELD",
|
||||||
|
"SourceRef",
|
||||||
|
"TensorArtSource",
|
||||||
|
"USER_AGENT",
|
||||||
|
"clean_source_url",
|
||||||
|
"detect_source",
|
||||||
|
"fetch_text",
|
||||||
|
"get_source",
|
||||||
|
"get_source_platform",
|
||||||
|
"has_external_source",
|
||||||
|
"list_sources",
|
||||||
|
"normalize_metadata_source",
|
||||||
|
"resolve_source_ref",
|
||||||
|
"source_group_key",
|
||||||
|
"source_label",
|
||||||
|
]
|
||||||
@@ -0,0 +1,186 @@
|
|||||||
|
"""Base types for the external model-source provider abstraction.
|
||||||
|
|
||||||
|
A *model source* is a third-party site that hosts model files and a model
|
||||||
|
card (README) describing them — Hugging Face, ModelScope, TensorArt, and
|
||||||
|
whatever gets added later. Everything the rest of the codebase needs to
|
||||||
|
know about such a site is expressed by :class:`ModelSource`:
|
||||||
|
|
||||||
|
* how to recognise one of its URLs (:meth:`ModelSource.parse`)
|
||||||
|
* the canonical page URL for a source id (:meth:`ModelSource.canonical_url`)
|
||||||
|
* how to fetch the model card (:meth:`ModelSource.fetch_model_card`)
|
||||||
|
* how to turn repository-relative asset paths into absolute URLs
|
||||||
|
(:meth:`ModelSource.asset_base_url`)
|
||||||
|
* which capabilities the site actually supports
|
||||||
|
(``supports_enrichment`` / ``supports_download``)
|
||||||
|
|
||||||
|
Keeping this in one place means the agent pipeline, the scanners, and the
|
||||||
|
HTTP handlers never need site-specific branching.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
import aiohttp
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#: Shared HTTP timeout for model-card fetches.
|
||||||
|
HTTP_TIMEOUT = 30
|
||||||
|
|
||||||
|
#: User agent used for all model-source HTTP requests.
|
||||||
|
USER_AGENT = "ComfyUI-LoRA-Manager/1.0"
|
||||||
|
|
||||||
|
#: Platform → short prefix used when building version-group keys.
|
||||||
|
#: ``huggingface`` keeps the historical ``hf:`` prefix for backward
|
||||||
|
#: compatibility with already-cached group keys.
|
||||||
|
GROUP_PREFIXES: dict[str, str] = {
|
||||||
|
"huggingface": "hf",
|
||||||
|
"modelscope": "ms",
|
||||||
|
"tensorart": "ta",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SourceRef:
|
||||||
|
"""A parsed reference to a model hosted on an external site."""
|
||||||
|
|
||||||
|
platform: str
|
||||||
|
"""Canonical platform id, e.g. ``"huggingface"``."""
|
||||||
|
|
||||||
|
source_id: str
|
||||||
|
"""Site-specific identity, e.g. ``"user/repo"`` or ``"827823520299086029"``."""
|
||||||
|
|
||||||
|
url: str
|
||||||
|
"""Canonical URL of the model page."""
|
||||||
|
|
||||||
|
|
||||||
|
async def fetch_text(url: str, *, timeout: int = HTTP_TIMEOUT) -> str:
|
||||||
|
"""Fetch *url* and return its body as text, or ``""`` on any failure.
|
||||||
|
|
||||||
|
Network problems are expected (offline installs, rate limits, dead
|
||||||
|
repos) and must never bubble up into the pipeline, so every error is
|
||||||
|
logged at debug level and normalised to an empty string.
|
||||||
|
"""
|
||||||
|
|
||||||
|
try:
|
||||||
|
async with aiohttp.ClientSession(
|
||||||
|
headers={"User-Agent": USER_AGENT},
|
||||||
|
timeout=aiohttp.ClientTimeout(total=timeout),
|
||||||
|
) as session:
|
||||||
|
async with session.get(url) as resp:
|
||||||
|
if resp.status == 200:
|
||||||
|
return await resp.text()
|
||||||
|
logger.debug("Fetch %s returned HTTP %s", url, resp.status)
|
||||||
|
except Exception as exc: # pragma: no cover - network dependent
|
||||||
|
logger.debug("Failed to fetch %s: %s", url, exc)
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
class ModelSource:
|
||||||
|
"""Description and I/O for one external model hosting site."""
|
||||||
|
|
||||||
|
#: Canonical platform id stored in metadata.
|
||||||
|
platform: str = ""
|
||||||
|
|
||||||
|
#: Human-readable name used in UI copy and prompts.
|
||||||
|
label: str = ""
|
||||||
|
|
||||||
|
#: Whether the agent skill can fetch a model card and run AI extraction.
|
||||||
|
supports_enrichment: bool = False
|
||||||
|
|
||||||
|
#: Whether models can be downloaded directly from this site.
|
||||||
|
supports_download: bool = False
|
||||||
|
|
||||||
|
#: Lenient pattern used to recognise URLs already stored in metadata.
|
||||||
|
#: Captures the site-specific source id in group ``id``.
|
||||||
|
url_pattern: re.Pattern[str] | None = None
|
||||||
|
|
||||||
|
#: Strict pattern used to validate user input. Must match the whole URL.
|
||||||
|
strict_url_pattern: re.Pattern[str] | None = None
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Parsing
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def parse(self, url: str, *, strict: bool = False) -> Optional[str]:
|
||||||
|
"""Return the source id contained in *url*, or ``None``.
|
||||||
|
|
||||||
|
With ``strict=True`` the URL must match this site's canonical shape
|
||||||
|
exactly (used when validating what a user pasted); with
|
||||||
|
``strict=False`` sub-paths such as ``/resolve/main/file.bin`` are
|
||||||
|
tolerated (used when normalising already-stored values).
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not url or not isinstance(url, str):
|
||||||
|
return None
|
||||||
|
candidate = url.strip()
|
||||||
|
if not candidate:
|
||||||
|
return None
|
||||||
|
pattern = self.strict_url_pattern if strict else self.url_pattern
|
||||||
|
if pattern is None:
|
||||||
|
return None
|
||||||
|
match = pattern.match(candidate)
|
||||||
|
return match.group("id") if match else None
|
||||||
|
|
||||||
|
def ref(self, url: str, *, strict: bool = False) -> Optional[SourceRef]:
|
||||||
|
"""Return a :class:`SourceRef` for *url*, or ``None`` if not ours."""
|
||||||
|
|
||||||
|
source_id = self.parse(url, strict=strict)
|
||||||
|
if not source_id:
|
||||||
|
return None
|
||||||
|
return SourceRef(
|
||||||
|
platform=self.platform,
|
||||||
|
source_id=source_id,
|
||||||
|
url=self.canonical_url(source_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# URLs and content
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def canonical_url(self, source_id: str) -> str:
|
||||||
|
"""Return the canonical model-page URL for *source_id*."""
|
||||||
|
|
||||||
|
raise NotImplementedError
|
||||||
|
|
||||||
|
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||||
|
"""Base URL used to resolve repository-relative asset paths."""
|
||||||
|
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def group_key(self, source_id: str) -> str:
|
||||||
|
"""Return the version-group key for *source_id*."""
|
||||||
|
|
||||||
|
prefix = GROUP_PREFIXES.get(self.platform, self.platform)
|
||||||
|
return f"{prefix}:{source_id}"
|
||||||
|
|
||||||
|
async def fetch_model_card(self, source_id: str) -> str:
|
||||||
|
"""Fetch the raw model card (README) markdown for *source_id*."""
|
||||||
|
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def __repr__(self) -> str: # pragma: no cover - debugging aid
|
||||||
|
return f"<ModelSource {self.platform}>"
|
||||||
|
|
||||||
|
|
||||||
|
def clean_source_url(url: Any) -> str:
|
||||||
|
"""Normalise a stored source URL value into a stripped string."""
|
||||||
|
|
||||||
|
if not isinstance(url, str):
|
||||||
|
return ""
|
||||||
|
return url.strip()
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"GROUP_PREFIXES",
|
||||||
|
"HTTP_TIMEOUT",
|
||||||
|
"ModelSource",
|
||||||
|
"SourceRef",
|
||||||
|
"USER_AGENT",
|
||||||
|
"clean_source_url",
|
||||||
|
"fetch_text",
|
||||||
|
]
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
"""Hugging Face model source."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
from .base import ModelSource, fetch_text
|
||||||
|
|
||||||
|
#: Lenient — used to normalise URLs already stored in metadata; tolerates
|
||||||
|
#: sub-paths such as ``/resolve/main/model.safetensors``.
|
||||||
|
_URL_PATTERN = re.compile(
|
||||||
|
r"https?://(?:www\.)?huggingface\.co/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||||
|
)
|
||||||
|
|
||||||
|
#: Strict — validates what the user pasted into the "link model" dialog.
|
||||||
|
_STRICT_URL_PATTERN = re.compile(
|
||||||
|
r"https?://(?:www\.)?huggingface\.co/(?P<id>[^/?#\s]+/[^/?#\s]+)/?$"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class HuggingFaceSource(ModelSource):
|
||||||
|
"""Hugging Face Hub (``huggingface.co``)."""
|
||||||
|
|
||||||
|
platform = "huggingface"
|
||||||
|
label = "Hugging Face"
|
||||||
|
supports_enrichment = True
|
||||||
|
supports_download = True
|
||||||
|
url_pattern = _URL_PATTERN
|
||||||
|
strict_url_pattern = _STRICT_URL_PATTERN
|
||||||
|
|
||||||
|
def canonical_url(self, source_id: str) -> str:
|
||||||
|
return f"https://huggingface.co/{source_id}"
|
||||||
|
|
||||||
|
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||||
|
return f"https://huggingface.co/{source_id}/resolve/{revision or 'main'}"
|
||||||
|
|
||||||
|
async def fetch_model_card(self, source_id: str) -> str:
|
||||||
|
"""Fetch ``README.md`` from Hugging Face (tries ``main``, then ``master``)."""
|
||||||
|
|
||||||
|
for branch in ("main", "master"):
|
||||||
|
text = await fetch_text(
|
||||||
|
f"https://huggingface.co/{source_id}/raw/{branch}/README.md"
|
||||||
|
)
|
||||||
|
if text:
|
||||||
|
return text
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["HuggingFaceSource"]
|
||||||
@@ -0,0 +1,76 @@
|
|||||||
|
"""ModelScope (魔搭社区) model source.
|
||||||
|
|
||||||
|
ModelScope exposes the same "model card as README.md" convention as
|
||||||
|
Hugging Face, including a YAML frontmatter block that often carries
|
||||||
|
``base_model:`` and ``trigger_words:``. Two public endpoints are used,
|
||||||
|
neither of which requires an API key for public models:
|
||||||
|
|
||||||
|
* ``/models/{owner}/{name}/resolve/{revision}/README.md`` — raw model card
|
||||||
|
* ``/api/v1/models/{owner}/{name}/repo?Revision=..&FilePath=README.md`` —
|
||||||
|
the same content through the API, used as a fallback when the resolve
|
||||||
|
URL is unavailable.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
from .base import ModelSource, fetch_text
|
||||||
|
|
||||||
|
_URL_PATTERN = re.compile(
|
||||||
|
r"https?://(?:www\.)?modelscope\.(?:cn|com)/models/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||||
|
)
|
||||||
|
|
||||||
|
#: Trailing view segments the site appends to a model URL; accepted verbatim
|
||||||
|
#: when the user pastes a browser tab URL.
|
||||||
|
_VIEW_SEGMENTS = r"(?:summary|files|model-file|readme|community|evaluation)?"
|
||||||
|
|
||||||
|
_STRICT_URL_PATTERN = re.compile(
|
||||||
|
r"https?://(?:www\.)?modelscope\.(?:cn|com)/models/(?P<id>[^/?#\s]+/[^/?#\s]+)"
|
||||||
|
rf"/?{_VIEW_SEGMENTS}/?$"
|
||||||
|
)
|
||||||
|
|
||||||
|
#: ``master`` is ModelScope's default branch; ``main`` is tried as a fallback
|
||||||
|
#: for repos imported from Hugging Face.
|
||||||
|
_REVISIONS = ("master", "main")
|
||||||
|
|
||||||
|
|
||||||
|
class ModelScopeSource(ModelSource):
|
||||||
|
"""ModelScope (``modelscope.cn``)."""
|
||||||
|
|
||||||
|
platform = "modelscope"
|
||||||
|
label = "ModelScope"
|
||||||
|
supports_enrichment = True
|
||||||
|
supports_download = False
|
||||||
|
url_pattern = _URL_PATTERN
|
||||||
|
strict_url_pattern = _STRICT_URL_PATTERN
|
||||||
|
|
||||||
|
def canonical_url(self, source_id: str) -> str:
|
||||||
|
return f"https://modelscope.cn/models/{source_id}"
|
||||||
|
|
||||||
|
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||||
|
return f"https://modelscope.cn/models/{source_id}/resolve/{revision or 'master'}"
|
||||||
|
|
||||||
|
async def fetch_model_card(self, source_id: str) -> str:
|
||||||
|
"""Fetch the model card, preferring the raw resolve URL."""
|
||||||
|
|
||||||
|
for revision in _REVISIONS:
|
||||||
|
text = await fetch_text(
|
||||||
|
f"https://modelscope.cn/models/{source_id}/resolve/{revision}/README.md"
|
||||||
|
)
|
||||||
|
if text:
|
||||||
|
return text
|
||||||
|
|
||||||
|
# Fallback: the repo API proxies the same file and is reachable in
|
||||||
|
# environments where the CDN resolve host is blocked.
|
||||||
|
for revision in _REVISIONS:
|
||||||
|
text = await fetch_text(
|
||||||
|
"https://modelscope.cn/api/v1/models/"
|
||||||
|
f"{source_id}/repo?Revision={revision}&FilePath=README.md"
|
||||||
|
)
|
||||||
|
if text:
|
||||||
|
return text
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["ModelScopeSource"]
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
"""Registry and metadata helpers for external model sources.
|
||||||
|
|
||||||
|
The registry is the single place the rest of the codebase asks "which site
|
||||||
|
is this URL from?", "what is this model's source?", and "can we enrich it?".
|
||||||
|
Import from :mod:`py.services.model_sources` rather than this module
|
||||||
|
directly.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from typing import Any, Dict, Mapping, Optional
|
||||||
|
|
||||||
|
from .base import GROUP_PREFIXES, ModelSource, SourceRef, clean_source_url
|
||||||
|
from .huggingface import HuggingFaceSource
|
||||||
|
from .modelscope import ModelScopeSource
|
||||||
|
from .tensorart import TensorArtSource
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#: Order matters only for disambiguation; the URL patterns are disjoint.
|
||||||
|
_SOURCES: tuple[ModelSource, ...] = (
|
||||||
|
HuggingFaceSource(),
|
||||||
|
ModelScopeSource(),
|
||||||
|
TensorArtSource(),
|
||||||
|
)
|
||||||
|
|
||||||
|
_BY_PLATFORM: Dict[str, ModelSource] = {s.platform: s for s in _SOURCES}
|
||||||
|
|
||||||
|
#: Metadata keys that carry the canonical external-source identity.
|
||||||
|
SOURCE_PLATFORM_FIELD = "source_platform"
|
||||||
|
SOURCE_URL_FIELD = "source_url"
|
||||||
|
#: Legacy field kept as a read/write alias for Hugging Face models so that
|
||||||
|
#: older sidecars, cached rows, and third-party consumers keep working.
|
||||||
|
LEGACY_HF_URL_FIELD = "hf_url"
|
||||||
|
|
||||||
|
|
||||||
|
def list_sources() -> list[ModelSource]:
|
||||||
|
"""Return every known model source."""
|
||||||
|
|
||||||
|
return list(_SOURCES)
|
||||||
|
|
||||||
|
|
||||||
|
def get_source(platform: Optional[str]) -> Optional[ModelSource]:
|
||||||
|
"""Return the source registered for *platform*, or ``None``."""
|
||||||
|
|
||||||
|
if not platform or not isinstance(platform, str):
|
||||||
|
return None
|
||||||
|
return _BY_PLATFORM.get(platform.strip().lower())
|
||||||
|
|
||||||
|
|
||||||
|
def source_label(platform: Optional[str], default: str = "") -> str:
|
||||||
|
"""Return the human-readable label for *platform*."""
|
||||||
|
|
||||||
|
source = get_source(platform)
|
||||||
|
return source.label if source else default
|
||||||
|
|
||||||
|
|
||||||
|
def detect_source(url: Optional[str], *, strict: bool = False) -> Optional[SourceRef]:
|
||||||
|
"""Return the :class:`SourceRef` for *url*, or ``None`` if unsupported."""
|
||||||
|
|
||||||
|
if not url or not isinstance(url, str):
|
||||||
|
return None
|
||||||
|
for source in _SOURCES:
|
||||||
|
ref = source.ref(url, strict=strict)
|
||||||
|
if ref is not None:
|
||||||
|
return ref
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_source_ref(metadata: Mapping[str, Any]) -> Optional[SourceRef]:
|
||||||
|
"""Return the source reference described by a model's metadata.
|
||||||
|
|
||||||
|
Handles all three storage states found in the wild:
|
||||||
|
|
||||||
|
1. ``source_url`` + ``source_platform`` (current format)
|
||||||
|
2. ``hf_url`` only (legacy Hugging Face storage)
|
||||||
|
3. ``hf_url`` plus a newer ``source_url`` (both written by older builds)
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not isinstance(metadata, Mapping):
|
||||||
|
return None
|
||||||
|
|
||||||
|
platform = clean_source_url(metadata.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||||
|
url = clean_source_url(metadata.get(SOURCE_URL_FIELD))
|
||||||
|
legacy = clean_source_url(metadata.get(LEGACY_HF_URL_FIELD))
|
||||||
|
|
||||||
|
source = get_source(platform)
|
||||||
|
if url:
|
||||||
|
if source is not None:
|
||||||
|
ref = source.ref(url)
|
||||||
|
if ref is not None:
|
||||||
|
return ref
|
||||||
|
ref = detect_source(url)
|
||||||
|
if ref is not None:
|
||||||
|
return ref
|
||||||
|
# Unknown platform but a URL is present: keep it addressable.
|
||||||
|
return SourceRef(platform=platform or "unknown", source_id="", url=url)
|
||||||
|
|
||||||
|
if legacy:
|
||||||
|
return detect_source(legacy)
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_metadata_source(metadata: Dict[str, Any]) -> Dict[str, Any]:
|
||||||
|
"""Normalise the external-source fields on *metadata* in place.
|
||||||
|
|
||||||
|
Guarantees that ``source_url``/``source_platform`` are present and
|
||||||
|
consistent, and that ``hf_url`` mirrors ``source_url`` for Hugging Face
|
||||||
|
models (never for other platforms, so a stale alias can't make a
|
||||||
|
ModelScope model look like a Hugging Face one).
|
||||||
|
|
||||||
|
Returns the same dict for convenient chaining.
|
||||||
|
"""
|
||||||
|
|
||||||
|
if not isinstance(metadata, dict):
|
||||||
|
return metadata
|
||||||
|
|
||||||
|
platform = clean_source_url(metadata.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||||
|
url = clean_source_url(metadata.get(SOURCE_URL_FIELD))
|
||||||
|
legacy = clean_source_url(metadata.get(LEGACY_HF_URL_FIELD))
|
||||||
|
|
||||||
|
source = get_source(platform)
|
||||||
|
ref: Optional[SourceRef] = None
|
||||||
|
|
||||||
|
if url:
|
||||||
|
ref = source.ref(url) if source is not None else None
|
||||||
|
if ref is None:
|
||||||
|
ref = detect_source(url)
|
||||||
|
elif legacy:
|
||||||
|
ref = detect_source(legacy)
|
||||||
|
|
||||||
|
if ref is not None and ref.source_id:
|
||||||
|
platform = ref.platform
|
||||||
|
url = ref.url or url
|
||||||
|
|
||||||
|
if platform:
|
||||||
|
metadata[SOURCE_PLATFORM_FIELD] = platform
|
||||||
|
else:
|
||||||
|
metadata.setdefault(SOURCE_PLATFORM_FIELD, "")
|
||||||
|
|
||||||
|
metadata[SOURCE_URL_FIELD] = url
|
||||||
|
|
||||||
|
# Keep the legacy alias in sync, but only for Hugging Face.
|
||||||
|
if url and platform == "huggingface":
|
||||||
|
metadata[LEGACY_HF_URL_FIELD] = url
|
||||||
|
elif LEGACY_HF_URL_FIELD in metadata and platform and platform != "huggingface":
|
||||||
|
metadata[LEGACY_HF_URL_FIELD] = ""
|
||||||
|
elif legacy and not url:
|
||||||
|
metadata[LEGACY_HF_URL_FIELD] = legacy
|
||||||
|
|
||||||
|
return metadata
|
||||||
|
|
||||||
|
|
||||||
|
def has_external_source(item: Mapping[str, Any]) -> bool:
|
||||||
|
"""Return ``True`` when *item* is linked to any external model site."""
|
||||||
|
|
||||||
|
if not isinstance(item, Mapping):
|
||||||
|
return False
|
||||||
|
return bool(
|
||||||
|
clean_source_url(item.get(SOURCE_URL_FIELD))
|
||||||
|
or clean_source_url(item.get(LEGACY_HF_URL_FIELD))
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def get_source_platform(item: Mapping[str, Any]) -> str:
|
||||||
|
"""Return the platform id stored on *item* (may be empty)."""
|
||||||
|
|
||||||
|
if not isinstance(item, Mapping):
|
||||||
|
return ""
|
||||||
|
platform = clean_source_url(item.get(SOURCE_PLATFORM_FIELD)).lower()
|
||||||
|
if platform:
|
||||||
|
return platform
|
||||||
|
ref = resolve_source_ref(item)
|
||||||
|
return ref.platform if ref else ""
|
||||||
|
|
||||||
|
|
||||||
|
def source_group_key(item: Mapping[str, Any]) -> Optional[str]:
|
||||||
|
"""Return the version-group key for *item*, or ``None``.
|
||||||
|
|
||||||
|
Hugging Face keeps the historical ``hf:{owner}/{repo}`` shape; other
|
||||||
|
platforms use their own short prefix (see :data:`GROUP_PREFIXES`).
|
||||||
|
"""
|
||||||
|
|
||||||
|
ref = resolve_source_ref(item)
|
||||||
|
if ref is None or not ref.source_id:
|
||||||
|
return None
|
||||||
|
source = get_source(ref.platform)
|
||||||
|
if source is None:
|
||||||
|
return None
|
||||||
|
return source.group_key(ref.source_id)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"GROUP_PREFIXES",
|
||||||
|
"LEGACY_HF_URL_FIELD",
|
||||||
|
"SOURCE_PLATFORM_FIELD",
|
||||||
|
"SOURCE_URL_FIELD",
|
||||||
|
"detect_source",
|
||||||
|
"get_source",
|
||||||
|
"get_source_platform",
|
||||||
|
"has_external_source",
|
||||||
|
"list_sources",
|
||||||
|
"normalize_metadata_source",
|
||||||
|
"resolve_source_ref",
|
||||||
|
"source_group_key",
|
||||||
|
"source_label",
|
||||||
|
]
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
"""TensorArt model source (link / provenance only).
|
||||||
|
|
||||||
|
TensorArt support is intentionally limited to *linking* a model to its
|
||||||
|
TensorArt page. Automatic metadata extraction is not possible without a
|
||||||
|
user session:
|
||||||
|
|
||||||
|
* ``tensor.art`` sits behind a Cloudflare managed challenge, so plain
|
||||||
|
HTTP clients (aiohttp, requests, curl) receive ``403 "Just a moment..."``.
|
||||||
|
* Its internal API (``ap-east-1.tensorart.cloud`` / ``cn.tensorart.net``)
|
||||||
|
answers every ``/v1/model/*`` route with
|
||||||
|
``{"code":100002,"message":"invalid authorization header"}``.
|
||||||
|
* The official TAMS API requires an AccessKey/SecretKey pair and request
|
||||||
|
signatures, which is a poor fit for a "paste a URL" workflow.
|
||||||
|
|
||||||
|
``supports_enrichment`` is therefore ``False``: the agent pipeline skips
|
||||||
|
these models with an explicit reason instead of failing silently, and the
|
||||||
|
UI keeps showing the "View on TensorArt" link. ``tusi.cn`` is TensorArt's
|
||||||
|
Chinese mirror and is accepted as the same platform.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
|
||||||
|
from .base import ModelSource
|
||||||
|
|
||||||
|
_DOMAINS = r"(?:tensor\.art|tusi\.cn)"
|
||||||
|
|
||||||
|
_URL_PATTERN = re.compile(
|
||||||
|
rf"https?://(?:www\.)?{_DOMAINS}/models/(?P<id>\d+)"
|
||||||
|
)
|
||||||
|
|
||||||
|
_STRICT_URL_PATTERN = re.compile(
|
||||||
|
rf"https?://(?:www\.)?{_DOMAINS}/models/(?P<id>\d+)(?:/[^/?#\s]+)?/?$"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TensorArtSource(ModelSource):
|
||||||
|
"""TensorArt (``tensor.art``)."""
|
||||||
|
|
||||||
|
platform = "tensorart"
|
||||||
|
label = "TensorArt"
|
||||||
|
supports_enrichment = False
|
||||||
|
supports_download = False
|
||||||
|
url_pattern = _URL_PATTERN
|
||||||
|
strict_url_pattern = _STRICT_URL_PATTERN
|
||||||
|
|
||||||
|
def canonical_url(self, source_id: str) -> str:
|
||||||
|
return f"https://tensor.art/models/{source_id}"
|
||||||
|
|
||||||
|
def asset_base_url(self, source_id: str, revision: str = "") -> str:
|
||||||
|
# Unreachable today: enrichment is disabled for this platform.
|
||||||
|
return f"https://tensor.art/models/{source_id}"
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["TensorArtSource"]
|
||||||
@@ -67,6 +67,8 @@ class OtherModelService(BaseModelService):
|
|||||||
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
"civitai": self.filter_civitai_data(model_data.get("civitai", {}), minimal=True),
|
||||||
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
"auto_tags": model_data.get("auto_tags") or extract_auto_tags(model_data),
|
||||||
"version_count": model_data.get("version_count"),
|
"version_count": model_data.get("version_count"),
|
||||||
|
"source_platform": model_data.get("source_platform", ""),
|
||||||
|
"source_url": model_data.get("source_url", ""),
|
||||||
"hf_url": model_data.get("hf_url", ""),
|
"hf_url": model_data.get("hf_url", ""),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ from dataclasses import dataclass, field
|
|||||||
from typing import Any, Dict, List, Mapping, Optional, Sequence, Tuple
|
from typing import Any, Dict, List, Mapping, Optional, Sequence, Tuple
|
||||||
|
|
||||||
from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration
|
from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration
|
||||||
|
from .model_sources import normalize_metadata_source
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -62,6 +63,8 @@ class PersistentModelCache:
|
|||||||
"db_checked",
|
"db_checked",
|
||||||
"last_checked_at",
|
"last_checked_at",
|
||||||
"hash_status",
|
"hash_status",
|
||||||
|
"source_platform",
|
||||||
|
"source_url",
|
||||||
"hf_url",
|
"hf_url",
|
||||||
)
|
)
|
||||||
_MODEL_UPDATE_COLUMNS: Tuple[str, ...] = _MODEL_COLUMNS[2:]
|
_MODEL_UPDATE_COLUMNS: Tuple[str, ...] = _MODEL_COLUMNS[2:]
|
||||||
@@ -206,8 +209,13 @@ class PersistentModelCache:
|
|||||||
"skip_metadata_refresh": bool(row["skip_metadata_refresh"]),
|
"skip_metadata_refresh": bool(row["skip_metadata_refresh"]),
|
||||||
"license_flags": int(license_value),
|
"license_flags": int(license_value),
|
||||||
"hash_status": row["hash_status"] or "completed",
|
"hash_status": row["hash_status"] or "completed",
|
||||||
|
"source_platform": row["source_platform"] or "",
|
||||||
|
"source_url": row["source_url"] or "",
|
||||||
"hf_url": row["hf_url"] or "",
|
"hf_url": row["hf_url"] or "",
|
||||||
}
|
}
|
||||||
|
# Legacy rows only carry `hf_url`; derive the canonical pair so
|
||||||
|
# every consumer sees the same shape.
|
||||||
|
normalize_metadata_source(item)
|
||||||
if row["autov3"] is not None:
|
if row["autov3"] is not None:
|
||||||
item["autov3"] = (row["autov3"] or "").lower()
|
item["autov3"] = (row["autov3"] or "").lower()
|
||||||
raw_data.append(item)
|
raw_data.append(item)
|
||||||
@@ -562,6 +570,8 @@ class PersistentModelCache:
|
|||||||
db_checked INTEGER,
|
db_checked INTEGER,
|
||||||
last_checked_at REAL,
|
last_checked_at REAL,
|
||||||
hash_status TEXT,
|
hash_status TEXT,
|
||||||
|
source_platform TEXT DEFAULT '',
|
||||||
|
source_url TEXT DEFAULT '',
|
||||||
hf_url TEXT DEFAULT '',
|
hf_url TEXT DEFAULT '',
|
||||||
PRIMARY KEY (model_type, file_path)
|
PRIMARY KEY (model_type, file_path)
|
||||||
);
|
);
|
||||||
@@ -629,6 +639,8 @@ class PersistentModelCache:
|
|||||||
# Persisting without explicit flags should assume CivitAI's documented defaults (0b111001 == 57).
|
# Persisting without explicit flags should assume CivitAI's documented defaults (0b111001 == 57).
|
||||||
"license_flags": f"INTEGER DEFAULT {DEFAULT_LICENSE_FLAGS}",
|
"license_flags": f"INTEGER DEFAULT {DEFAULT_LICENSE_FLAGS}",
|
||||||
"hash_status": "TEXT DEFAULT 'completed'",
|
"hash_status": "TEXT DEFAULT 'completed'",
|
||||||
|
"source_platform": "TEXT DEFAULT ''",
|
||||||
|
"source_url": "TEXT DEFAULT ''",
|
||||||
"hf_url": "TEXT DEFAULT ''",
|
"hf_url": "TEXT DEFAULT ''",
|
||||||
"autov3": "TEXT",
|
"autov3": "TEXT",
|
||||||
}
|
}
|
||||||
@@ -650,6 +662,9 @@ class PersistentModelCache:
|
|||||||
return conn
|
return conn
|
||||||
|
|
||||||
def _prepare_model_row(self, model_type: str, item: Dict[str, Any]) -> Tuple[Any, ...]:
|
def _prepare_model_row(self, model_type: str, item: Dict[str, Any]) -> Tuple[Any, ...]:
|
||||||
|
# Keep `source_*` and the legacy `hf_url` alias consistent no matter
|
||||||
|
# which caller populated the item.
|
||||||
|
normalize_metadata_source(item)
|
||||||
civitai = item.get("civitai") or {}
|
civitai = item.get("civitai") or {}
|
||||||
trained_words = civitai.get("trainedWords")
|
trained_words = civitai.get("trainedWords")
|
||||||
if isinstance(trained_words, str):
|
if isinstance(trained_words, str):
|
||||||
@@ -713,6 +728,8 @@ class PersistentModelCache:
|
|||||||
1 if item.get("db_checked") else 0,
|
1 if item.get("db_checked") else 0,
|
||||||
float(item.get("last_checked_at") or 0.0),
|
float(item.get("last_checked_at") or 0.0),
|
||||||
item.get("hash_status", "completed"),
|
item.get("hash_status", "completed"),
|
||||||
|
item.get("source_platform") or "",
|
||||||
|
item.get("source_url") or "",
|
||||||
item.get("hf_url") or "",
|
item.get("hf_url") or "",
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ import time
|
|||||||
from typing import Any, Dict, List, Optional, Protocol, Sequence
|
from typing import Any, Dict, List, Optional, Protocol, Sequence
|
||||||
|
|
||||||
from ..metadata_sync_service import MetadataSyncService
|
from ..metadata_sync_service import MetadataSyncService
|
||||||
|
from ..model_sources import has_external_source
|
||||||
from ...utils.metadata_manager import MetadataManager
|
from ...utils.metadata_manager import MetadataManager
|
||||||
|
|
||||||
|
|
||||||
@@ -51,10 +52,11 @@ class BulkMetadataRefreshUseCase:
|
|||||||
if not model.get("skip_metadata_refresh", False)
|
if not model.get("skip_metadata_refresh", False)
|
||||||
and not self._is_in_skip_path(model.get("folder", ""), skip_paths)
|
and not self._is_in_skip_path(model.get("folder", ""), skip_paths)
|
||||||
and (not model.get("civitai") or not model["civitai"].get("id"))
|
and (not model.get("civitai") or not model["civitai"].get("id"))
|
||||||
# Skip models downloaded from Hugging Face — they are not on
|
# Skip models linked to an external model site (Hugging Face /
|
||||||
# CivitAI / CivArchive. Users can still refresh them individually
|
# ModelScope / TensorArt) — they are not on CivitAI / CivArchive.
|
||||||
# via the right-click context menu.
|
# Users can still refresh them individually via the right-click
|
||||||
and not model.get("hf_url", "")
|
# context menu.
|
||||||
|
and not has_external_source(model)
|
||||||
and not (
|
and not (
|
||||||
# Skip models confirmed not on CivitAI when no need to retry
|
# Skip models confirmed not on CivitAI when no need to retry
|
||||||
model.get("from_civitai") is False
|
model.get("from_civitai") is False
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ import { MODEL_CONFIG } from '../../api/apiConfig.js';
|
|||||||
import { translate } from '../../utils/i18nHelpers.js';
|
import { translate } from '../../utils/i18nHelpers.js';
|
||||||
import { getNsfwLevelSelector } from '../shared/NsfwLevelSelector.js';
|
import { getNsfwLevelSelector } from '../shared/NsfwLevelSelector.js';
|
||||||
import { classifyModelRelinkUrl } from '../../utils/civitaiUtils.js';
|
import { classifyModelRelinkUrl } from '../../utils/civitaiUtils.js';
|
||||||
|
import { parseModelSourceUrl, getModelSourceInfo } from '../../utils/modelSourceHelpers.js';
|
||||||
|
import { escapeHtml } from '../shared/utils.js';
|
||||||
|
|
||||||
// Mixin with shared functionality for LoraContextMenu and CheckpointContextMenu
|
// Mixin with shared functionality for LoraContextMenu and CheckpointContextMenu
|
||||||
export const ModelContextMenuMixin = {
|
export const ModelContextMenuMixin = {
|
||||||
@@ -211,7 +213,7 @@ export const ModelContextMenuMixin = {
|
|||||||
setTimeout(() => urlInput.focus(), 50);
|
setTimeout(() => urlInput.focus(), 50);
|
||||||
},
|
},
|
||||||
|
|
||||||
// HuggingFace linking methods
|
// External model source linking (Hugging Face / ModelScope / TensorArt)
|
||||||
showLinkHfModal() {
|
showLinkHfModal() {
|
||||||
const filePath = this.currentCard.dataset.filepath;
|
const filePath = this.currentCard.dataset.filepath;
|
||||||
if (!filePath) return;
|
if (!filePath) return;
|
||||||
@@ -225,15 +227,23 @@ export const ModelContextMenuMixin = {
|
|||||||
}
|
}
|
||||||
|
|
||||||
this._boundLinkHfHandler = async () => {
|
this._boundLinkHfHandler = async () => {
|
||||||
const hfUrl = urlInput.value.trim();
|
const rawUrl = urlInput.value.trim();
|
||||||
if (!hfUrl) {
|
if (!rawUrl) {
|
||||||
errorDiv.textContent = 'Please enter a HuggingFace repository URL.';
|
errorDiv.textContent = translate(
|
||||||
|
'modals.linkModelSource.urlRequired',
|
||||||
|
{},
|
||||||
|
'Please enter a model page URL.'
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
const hfPattern = /^https?:\/\/huggingface\.co\/([^/]+\/[^/]+)\/?$/;
|
const sourceInfo = parseModelSourceUrl(rawUrl);
|
||||||
if (!hfPattern.test(hfUrl)) {
|
if (!sourceInfo) {
|
||||||
errorDiv.textContent = 'Invalid URL format. Expected: https://huggingface.co/user/repo';
|
errorDiv.textContent = translate(
|
||||||
|
'modals.linkModelSource.invalidUrl',
|
||||||
|
{},
|
||||||
|
'Unsupported URL. Supported sites: Hugging Face, ModelScope, TensorArt.'
|
||||||
|
);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -241,12 +251,14 @@ export const ModelContextMenuMixin = {
|
|||||||
modalManager.closeModal('linkHfModal');
|
modalManager.closeModal('linkHfModal');
|
||||||
|
|
||||||
try {
|
try {
|
||||||
state.loadingManager.showSimpleLoading('Linking to HuggingFace...');
|
state.loadingManager.showSimpleLoading(
|
||||||
|
translate('modals.linkModelSource.linking', {}, 'Linking model source...')
|
||||||
|
);
|
||||||
|
|
||||||
const response = await fetch('/api/lm/set-hf-url', {
|
const response = await fetch('/api/lm/set-hf-url', {
|
||||||
method: 'POST',
|
method: 'POST',
|
||||||
headers: { 'Content-Type': 'application/json' },
|
headers: { 'Content-Type': 'application/json' },
|
||||||
body: JSON.stringify({ file_path: filePath, hf_url: hfUrl }),
|
body: JSON.stringify({ file_path: filePath, source_url: sourceInfo.url }),
|
||||||
});
|
});
|
||||||
|
|
||||||
if (!response.ok) {
|
if (!response.ok) {
|
||||||
@@ -262,7 +274,7 @@ export const ModelContextMenuMixin = {
|
|||||||
throw new Error(data.error || 'Failed to link model');
|
throw new Error(data.error || 'Failed to link model');
|
||||||
}
|
}
|
||||||
} catch (error) {
|
} catch (error) {
|
||||||
console.error('Error linking model to HuggingFace:', error);
|
console.error('Error linking model source:', error);
|
||||||
showToast('toast.contextMenu.linkHfFailed', { message: error.message }, 'error');
|
showToast('toast.contextMenu.linkHfFailed', { message: error.message }, 'error');
|
||||||
} finally {
|
} finally {
|
||||||
state.loadingManager.hide();
|
state.loadingManager.hide();
|
||||||
@@ -276,18 +288,68 @@ export const ModelContextMenuMixin = {
|
|||||||
|
|
||||||
modalManager.showModal('linkHfModal');
|
modalManager.showModal('linkHfModal');
|
||||||
|
|
||||||
|
this._renderSupportedSources();
|
||||||
|
|
||||||
setTimeout(() => urlInput.focus(), 50);
|
setTimeout(() => urlInput.focus(), 50);
|
||||||
},
|
},
|
||||||
|
|
||||||
// HF metadata enrichment (AI agent) methods
|
/**
|
||||||
|
* Refresh the supported-site hints from the server so the dialog reflects
|
||||||
|
* whatever sources this backend build actually knows about. Falls back to
|
||||||
|
* the static markup in the template when the request fails.
|
||||||
|
*/
|
||||||
|
async _renderSupportedSources() {
|
||||||
|
const container = document.getElementById('hfSupportedSources');
|
||||||
|
if (!container) return;
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await fetch('/api/lm/model-sources');
|
||||||
|
if (!response.ok) return;
|
||||||
|
const sources = await response.json();
|
||||||
|
if (!Array.isArray(sources) || sources.length === 0) return;
|
||||||
|
|
||||||
|
const examples = sources
|
||||||
|
.map((source) => source?.example_url)
|
||||||
|
.filter((url) => typeof url === 'string' && url);
|
||||||
|
if (examples.length === 0) return;
|
||||||
|
|
||||||
|
container.innerHTML = examples
|
||||||
|
.map((url) => `<strong>${escapeHtml(url)}</strong>`)
|
||||||
|
.join('<br>');
|
||||||
|
} catch (error) {
|
||||||
|
console.debug('Failed to load supported model sources:', error);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|
||||||
|
// Model metadata enrichment (AI agent) methods
|
||||||
updateEnrichMenuItem(card) {
|
updateEnrichMenuItem(card) {
|
||||||
const enrichItem = this.menu?.querySelector('[data-action="enrich-hf-llm"]');
|
const enrichItem = this.menu?.querySelector('[data-action="enrich-hf-llm"]');
|
||||||
if (!enrichItem) return;
|
if (!enrichItem) return;
|
||||||
const hasHfUrl = !!card.dataset.hf_url;
|
|
||||||
enrichItem.classList.toggle('disabled', !hasHfUrl);
|
const model = {
|
||||||
enrichItem.title = hasHfUrl
|
source_url: card.dataset.source_url || '',
|
||||||
? ''
|
source_platform: card.dataset.source_platform || '',
|
||||||
: 'Link this model to a HuggingFace repo first (Link Model → Link to HuggingFace)';
|
hf_url: card.dataset.hf_url || '',
|
||||||
|
};
|
||||||
|
const sourceInfo = getModelSourceInfo(model);
|
||||||
|
const canEnrich = Boolean(sourceInfo && sourceInfo.supportsEnrichment);
|
||||||
|
|
||||||
|
enrichItem.classList.toggle('disabled', !canEnrich);
|
||||||
|
if (canEnrich) {
|
||||||
|
enrichItem.title = '';
|
||||||
|
} else if (!sourceInfo) {
|
||||||
|
enrichItem.title = translate(
|
||||||
|
'toast.contextMenu.enrichNeedsSource',
|
||||||
|
{},
|
||||||
|
'Link this model to a model source first (Link Model → Link to Model Source)'
|
||||||
|
);
|
||||||
|
} else {
|
||||||
|
enrichItem.title = translate(
|
||||||
|
'toast.contextMenu.enrichUnsupportedSource',
|
||||||
|
{ source: sourceInfo.label },
|
||||||
|
`AI enrichment is not available for ${sourceInfo.label} models`
|
||||||
|
);
|
||||||
|
}
|
||||||
},
|
},
|
||||||
|
|
||||||
async enrichWithAgent(filePath) {
|
async enrichWithAgent(filePath) {
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { showToast, openCivitai, openHuggingFace, copyToClipboard, copyLoraSyntax, sendLoraToWorkflow, sendEmbeddingToWorkflow, openExampleImagesFolder, buildLoraSyntax, sendModelPathToWorkflow } from '../../utils/uiHelpers.js';
|
import { showToast, openCivitai, openHuggingFace, copyToClipboard, copyLoraSyntax, sendLoraToWorkflow, sendEmbeddingToWorkflow, openExampleImagesFolder, buildLoraSyntax, sendModelPathToWorkflow } from '../../utils/uiHelpers.js';
|
||||||
|
import { getModelSourceInfo, getModelSourceGroupKey, getModelSourceViewTitle, openModelSource } from '../../utils/modelSourceHelpers.js';
|
||||||
import { state, getCurrentPageState } from '../../state/index.js';
|
import { state, getCurrentPageState } from '../../state/index.js';
|
||||||
import { showModelModal } from './ModelModal.js';
|
import { showModelModal } from './ModelModal.js';
|
||||||
import { hasCivitaiSource } from './utils.js';
|
import { hasCivitaiSource } from './utils.js';
|
||||||
@@ -65,12 +66,15 @@ function handleModelCardEvent_internal(event, modelType) {
|
|||||||
if (event.target.closest('.fa-globe')) {
|
if (event.target.closest('.fa-globe')) {
|
||||||
event.stopPropagation();
|
event.stopPropagation();
|
||||||
// CivitAI wins when the model actually has CivitAI data; otherwise fall
|
// CivitAI wins when the model actually has CivitAI data; otherwise fall
|
||||||
// back to HuggingFace. Relying on `from_civitai` here made the two
|
// back to the linked external source. Relying on `from_civitai` here
|
||||||
// sources mutually exclusive whenever one of them was (re)linked (#1094).
|
// made the two sources mutually exclusive whenever one of them was
|
||||||
|
// (re)linked (#1094).
|
||||||
if (card.dataset.has_civitai === 'true') {
|
if (card.dataset.has_civitai === 'true') {
|
||||||
openCivitai(card.dataset.filepath);
|
openCivitai(card.dataset.filepath);
|
||||||
} else if (card.dataset.hf_url) {
|
} else if (card.dataset.source_platform === 'huggingface' && card.dataset.hf_url) {
|
||||||
openHuggingFace(card.dataset.hf_url);
|
openHuggingFace(card.dataset.hf_url);
|
||||||
|
} else if (card.dataset.source_url) {
|
||||||
|
openModelSource(card.dataset.source_url);
|
||||||
}
|
}
|
||||||
return true; // Stop propagation
|
return true; // Stop propagation
|
||||||
}
|
}
|
||||||
@@ -337,6 +341,8 @@ async function showModelModalFromCard(card, modelType) {
|
|||||||
modified: card.dataset.modified,
|
modified: card.dataset.modified,
|
||||||
file_size: parseInt(card.dataset.file_size || '0'),
|
file_size: parseInt(card.dataset.file_size || '0'),
|
||||||
from_civitai: card.dataset.from_civitai === 'true',
|
from_civitai: card.dataset.from_civitai === 'true',
|
||||||
|
source_platform: card.dataset.source_platform || '',
|
||||||
|
source_url: card.dataset.source_url || '',
|
||||||
hf_url: card.dataset.hf_url || '',
|
hf_url: card.dataset.hf_url || '',
|
||||||
base_model: card.dataset.base_model,
|
base_model: card.dataset.base_model,
|
||||||
notes: card.dataset.notes || '',
|
notes: card.dataset.notes || '',
|
||||||
@@ -428,6 +434,8 @@ function showExampleAccessModal(card, modelType) {
|
|||||||
modified: card.dataset.modified,
|
modified: card.dataset.modified,
|
||||||
file_size: card.dataset.file_size,
|
file_size: card.dataset.file_size,
|
||||||
from_civitai: card.dataset.from_civitai === 'true',
|
from_civitai: card.dataset.from_civitai === 'true',
|
||||||
|
source_platform: card.dataset.source_platform || '',
|
||||||
|
source_url: card.dataset.source_url || '',
|
||||||
hf_url: card.dataset.hf_url || '',
|
hf_url: card.dataset.hf_url || '',
|
||||||
base_model: card.dataset.base_model,
|
base_model: card.dataset.base_model,
|
||||||
notes: card.dataset.notes,
|
notes: card.dataset.notes,
|
||||||
@@ -490,7 +498,11 @@ export function createModelCard(model, modelType) {
|
|||||||
card.dataset.base_model = model.base_model || 'Unknown';
|
card.dataset.base_model = model.base_model || 'Unknown';
|
||||||
card.dataset.favorite = model.favorite ? 'true' : 'false';
|
card.dataset.favorite = model.favorite ? 'true' : 'false';
|
||||||
card.dataset.exclude = model.exclude ? 'true' : 'false';
|
card.dataset.exclude = model.exclude ? 'true' : 'false';
|
||||||
card.dataset.hf_url = model.hf_url || '';
|
const modelSourceInfo = getModelSourceInfo(model);
|
||||||
|
card.dataset.source_url = modelSourceInfo?.url || '';
|
||||||
|
card.dataset.source_platform = modelSourceInfo?.platform || '';
|
||||||
|
// Legacy alias: only Hugging Face models expose `hf_url`.
|
||||||
|
card.dataset.hf_url = modelSourceInfo?.platform === 'huggingface' ? modelSourceInfo.url : '';
|
||||||
const hasUpdateAvailable = Boolean(model.update_available);
|
const hasUpdateAvailable = Boolean(model.update_available);
|
||||||
card.dataset.update_available = hasUpdateAvailable ? 'true' : 'false';
|
card.dataset.update_available = hasUpdateAvailable ? 'true' : 'false';
|
||||||
card.dataset.skip_metadata_refresh = model.skip_metadata_refresh ? 'true' : 'false';
|
card.dataset.skip_metadata_refresh = model.skip_metadata_refresh ? 'true' : 'false';
|
||||||
@@ -508,11 +520,12 @@ export function createModelCard(model, modelType) {
|
|||||||
const modelId = civitaiData?.modelId ?? civitaiData?.model_id;
|
const modelId = civitaiData?.modelId ?? civitaiData?.model_id;
|
||||||
if (modelId !== undefined && modelId !== null && modelId !== '') {
|
if (modelId !== undefined && modelId !== null && modelId !== '') {
|
||||||
card.dataset.modelId = modelId;
|
card.dataset.modelId = modelId;
|
||||||
} else if (model.hf_url) {
|
} else {
|
||||||
// For HF-only models, derive a group key from hf_url for version grouping
|
// For externally-sourced models, derive a group key from the source
|
||||||
const match = model.hf_url.match(/https?:\/\/huggingface\.co\/([^/]+\/[^/]+)/);
|
// URL for version grouping (hf:user/repo, ms:user/repo, ta:<id>).
|
||||||
if (match) {
|
const sourceGroupKey = getModelSourceGroupKey(model);
|
||||||
card.dataset.modelId = 'hf:' + match[1];
|
if (sourceGroupKey) {
|
||||||
|
card.dataset.modelId = sourceGroupKey;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -610,10 +623,10 @@ export function createModelCard(model, modelType) {
|
|||||||
const hasCivitai = hasCivitaiSource(model.civitai);
|
const hasCivitai = hasCivitaiSource(model.civitai);
|
||||||
const globeTitle = hasCivitai ?
|
const globeTitle = hasCivitai ?
|
||||||
translate('modelCard.actions.viewOnCivitai', {}, 'View on Civitai') :
|
translate('modelCard.actions.viewOnCivitai', {}, 'View on Civitai') :
|
||||||
model.hf_url ?
|
modelSourceInfo ?
|
||||||
translate('modelCard.actions.viewOnHuggingFace', {}, 'View on Hugging Face') :
|
getModelSourceViewTitle(modelSourceInfo) :
|
||||||
translate('modelCard.actions.notAvailableFromCivitai', {}, 'Not available from Civitai');
|
translate('modelCard.actions.notAvailableFromCivitai', {}, 'Not available from Civitai');
|
||||||
const globeEnabled = hasCivitai || !!model.hf_url;
|
const globeEnabled = hasCivitai || !!modelSourceInfo;
|
||||||
let sendTitle;
|
let sendTitle;
|
||||||
let copyTitle;
|
let copyTitle;
|
||||||
if (modelType === MODEL_TYPES.LORA) {
|
if (modelType === MODEL_TYPES.LORA) {
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { showToast, openCivitai, sendLoraToWorkflow, sendEmbeddingToWorkflow, sendModelPathToWorkflow, buildLoraSyntax, copyToClipboard } from '../../utils/uiHelpers.js';
|
import { showToast, openCivitai, sendLoraToWorkflow, sendEmbeddingToWorkflow, sendModelPathToWorkflow, buildLoraSyntax, copyToClipboard } from '../../utils/uiHelpers.js';
|
||||||
|
import { getModelSourceInfo, getModelSourceGroupKey, getModelSourceViewTitle, openModelSource } from '../../utils/modelSourceHelpers.js';
|
||||||
import { modalManager } from '../../managers/ModalManager.js';
|
import { modalManager } from '../../managers/ModalManager.js';
|
||||||
import { MODEL_TYPES } from '../../api/apiConfig.js';
|
import { MODEL_TYPES } from '../../api/apiConfig.js';
|
||||||
import {
|
import {
|
||||||
@@ -397,10 +398,13 @@ export async function showModelModal(model, modelType) {
|
|||||||
<div class="civitai-view" title="${translate('modals.model.actions.viewOnCivitai', {}, 'View on Civitai')}" data-action="view-civitai" data-filepath="${escapedFilePathAttr}">
|
<div class="civitai-view" title="${translate('modals.model.actions.viewOnCivitai', {}, 'View on Civitai')}" data-action="view-civitai" data-filepath="${escapedFilePathAttr}">
|
||||||
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnCivitaiText', {}, 'View on Civitai')}
|
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnCivitaiText', {}, 'View on Civitai')}
|
||||||
</div>`.trim() : '';
|
</div>`.trim() : '';
|
||||||
const escapedHfUrl = modelWithFullData.hf_url ? escapeAttribute(modelWithFullData.hf_url) : '';
|
const sourceInfo = getModelSourceInfo(modelWithFullData);
|
||||||
const viewOnHuggingFaceAction = escapedHfUrl ? `
|
const escapedSourceUrl = sourceInfo?.url ? escapeAttribute(sourceInfo.url) : '';
|
||||||
<div class="civitai-view" title="${translate('modals.model.actions.viewOnHuggingFace', {}, 'View on Hugging Face')}" data-action="view-huggingface" data-hf-url="${escapedHfUrl}">
|
const isHuggingFaceSource = sourceInfo?.platform === 'huggingface';
|
||||||
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnHuggingFaceText', {}, 'View on Hugging Face')}
|
const sourceTitle = sourceInfo ? getModelSourceViewTitle(sourceInfo) : '';
|
||||||
|
const viewOnHuggingFaceAction = escapedSourceUrl ? `
|
||||||
|
<div class="civitai-view" title="${escapeAttribute(sourceTitle)}" data-action="${isHuggingFaceSource ? 'view-huggingface' : 'view-model-source'}" ${isHuggingFaceSource ? 'data-hf-url' : 'data-source-url'}="${escapedSourceUrl}">
|
||||||
|
<i class="fas fa-globe"></i> ${escapeHtml(sourceTitle)}
|
||||||
</div>`.trim() : '';
|
</div>`.trim() : '';
|
||||||
const creatorInfoAction = modelWithFullData.civitai?.creator ? `
|
const creatorInfoAction = modelWithFullData.civitai?.creator ? `
|
||||||
<div class="creator-info" data-username="${modelWithFullData.civitai.creator.username}" data-action="view-creator" title="${translate('modals.model.actions.viewCreatorProfile', {}, 'View Creator Profile')}">
|
<div class="creator-info" data-username="${modelWithFullData.civitai.creator.username}" data-action="view-creator" title="${translate('modals.model.actions.viewCreatorProfile', {}, 'View Creator Profile')}">
|
||||||
@@ -520,12 +524,12 @@ export async function showModelModal(model, modelType) {
|
|||||||
const loadingExamplesText = translate('modals.model.loading.examples', {}, 'Loading examples...');
|
const loadingExamplesText = translate('modals.model.loading.examples', {}, 'Loading examples...');
|
||||||
|
|
||||||
const loadingVersionsText = translate('modals.model.loading.versions', {}, 'Loading versions...');
|
const loadingVersionsText = translate('modals.model.loading.versions', {}, 'Loading versions...');
|
||||||
// Use CivitAI modelId, or derive HF group key for HF-only models
|
// Use CivitAI modelId, or derive a source group key for externally-linked models
|
||||||
let civitaiModelId = modelWithFullData.civitai?.modelId || '';
|
let civitaiModelId = modelWithFullData.civitai?.modelId || '';
|
||||||
if (!civitaiModelId && modelWithFullData.hf_url) {
|
if (!civitaiModelId) {
|
||||||
const match = modelWithFullData.hf_url.match(/https?:\/\/huggingface\.co\/([^/]+\/[^/]+)/);
|
const sourceGroupKey = getModelSourceGroupKey(modelWithFullData);
|
||||||
if (match) {
|
if (sourceGroupKey) {
|
||||||
civitaiModelId = 'hf:' + match[1];
|
civitaiModelId = sourceGroupKey;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
const civitaiVersionId = modelWithFullData.civitai?.id || '';
|
const civitaiVersionId = modelWithFullData.civitai?.id || '';
|
||||||
@@ -939,6 +943,11 @@ function setupEventHandlers(filePath, modelType) {
|
|||||||
window.open(target.dataset.hfUrl, '_blank', 'noopener,noreferrer');
|
window.open(target.dataset.hfUrl, '_blank', 'noopener,noreferrer');
|
||||||
}
|
}
|
||||||
break;
|
break;
|
||||||
|
case 'view-model-source':
|
||||||
|
if (target.dataset.sourceUrl) {
|
||||||
|
openModelSource(target.dataset.sourceUrl);
|
||||||
|
}
|
||||||
|
break;
|
||||||
case 'view-creator':
|
case 'view-creator':
|
||||||
const username = target.dataset.username;
|
const username = target.dataset.username;
|
||||||
if (username) {
|
if (username) {
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import { openCivitaiUrl, showToast } from '../../utils/uiHelpers.js';
|
|||||||
import { translate } from '../../utils/i18nHelpers.js';
|
import { translate } from '../../utils/i18nHelpers.js';
|
||||||
import { state } from '../../state/index.js';
|
import { state } from '../../state/index.js';
|
||||||
import { buildCivitaiModelUrl } from '../../utils/civitaiUtils.js';
|
import { buildCivitaiModelUrl } from '../../utils/civitaiUtils.js';
|
||||||
|
import { parseModelSourceGroupKey } from '../../utils/modelSourceHelpers.js';
|
||||||
import { formatFileSize } from './utils.js';
|
import { formatFileSize } from './utils.js';
|
||||||
import { setSessionItem, removeSessionItem } from '../../utils/storageHelpers.js';
|
import { setSessionItem, removeSessionItem } from '../../utils/storageHelpers.js';
|
||||||
|
|
||||||
@@ -993,22 +994,23 @@ export function initVersionsTab({
|
|||||||
renderErrorState(container, translate('modals.model.versions.missingModelId', {}, 'This model is missing a Civitai model id.'));
|
renderErrorState(container, translate('modals.model.versions.missingModelId', {}, 'This model is missing a Civitai model id.'));
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
// HF group keys (e.g. "hf:user/repo") are not real CivitAI model IDs —
|
// External source group keys (e.g. "hf:user/repo", "ms:user/repo",
|
||||||
// skip the remote API call and show a helpful message instead.
|
// "ta:8278...") are not real CivitAI model IDs — skip the remote API
|
||||||
const isHfGroupKey = typeof modelId === 'string' && modelId.startsWith('hf:');
|
// call and show a helpful message instead.
|
||||||
if (isHfGroupKey) {
|
const sourceGroup = parseModelSourceGroupKey(modelId);
|
||||||
|
if (sourceGroup) {
|
||||||
controller.isLoading = false;
|
controller.isLoading = false;
|
||||||
controller.hasLoaded = true;
|
controller.hasLoaded = true;
|
||||||
controller.record = null;
|
controller.record = null;
|
||||||
const hfMsg = translate(
|
const sourceMsg = translate(
|
||||||
'modals.model.versions.hfGroupInfo',
|
'modals.model.versions.sourceGroupInfo',
|
||||||
{},
|
{ source: sourceGroup.label },
|
||||||
'This is a HuggingFace model group. Open the library to see all versions in the grid.'
|
`This is a ${sourceGroup.label} model group. Open the library to see all versions in the grid.`
|
||||||
);
|
);
|
||||||
container.innerHTML = `
|
container.innerHTML = `
|
||||||
<div class="versions-empty-state">
|
<div class="versions-empty-state">
|
||||||
<i class="fas fa-info-circle"></i>
|
<i class="fas fa-info-circle"></i>
|
||||||
<p>${escapeHtml(hfMsg)}</p>
|
<p>${escapeHtml(sourceMsg)}</p>
|
||||||
</div>
|
</div>
|
||||||
`;
|
`;
|
||||||
return;
|
return;
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
/**
|
||||||
|
* External model source helpers (Hugging Face / ModelScope / TensorArt).
|
||||||
|
*
|
||||||
|
* Mirrors `py/services/model_sources/registry.py` so the frontend and the
|
||||||
|
* backend agree on URL recognition, version-group keys, and which sites
|
||||||
|
* support AI metadata enrichment.
|
||||||
|
*
|
||||||
|
* Models loaded from an older cache may only carry the legacy `hf_url`
|
||||||
|
* field; every helper here falls back to it, and to the legacy
|
||||||
|
* `hf:user/repo` group key shape.
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { translate } from './i18nHelpers.js';
|
||||||
|
|
||||||
|
export const MODEL_SOURCES = [
|
||||||
|
{
|
||||||
|
platform: 'huggingface',
|
||||||
|
label: 'Hugging Face',
|
||||||
|
groupPrefix: 'hf',
|
||||||
|
supportsEnrichment: true,
|
||||||
|
supportsDownload: true,
|
||||||
|
exampleUrl: 'https://huggingface.co/user/repo',
|
||||||
|
placeholder: 'https://huggingface.co/user/repo',
|
||||||
|
pattern: /^https?:\/\/(?:www\.)?huggingface\.co\/([^/?#\s]+\/[^/?#\s]+)/i,
|
||||||
|
canonical: (id) => `https://huggingface.co/${id}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
platform: 'modelscope',
|
||||||
|
label: 'ModelScope',
|
||||||
|
groupPrefix: 'ms',
|
||||||
|
supportsEnrichment: true,
|
||||||
|
supportsDownload: false,
|
||||||
|
exampleUrl: 'https://modelscope.cn/models/user/repo',
|
||||||
|
placeholder: 'https://modelscope.cn/models/user/repo',
|
||||||
|
pattern: /^https?:\/\/(?:www\.)?modelscope\.(?:cn|com)\/models\/([^/?#\s]+\/[^/?#\s]+)/i,
|
||||||
|
canonical: (id) => `https://modelscope.cn/models/${id}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
platform: 'tensorart',
|
||||||
|
label: 'TensorArt',
|
||||||
|
groupPrefix: 'ta',
|
||||||
|
supportsEnrichment: false,
|
||||||
|
supportsDownload: false,
|
||||||
|
exampleUrl: 'https://tensor.art/models/827823520299086029',
|
||||||
|
placeholder: 'https://tensor.art/models/827823520299086029',
|
||||||
|
pattern: /^https?:\/\/(?:www\.)?(?:tensor\.art|tusi\.cn)\/models\/(\d+)/i,
|
||||||
|
canonical: (id) => `https://tensor.art/models/${id}`,
|
||||||
|
},
|
||||||
|
];
|
||||||
|
|
||||||
|
/** Return the source descriptor for a platform id, or null. */
|
||||||
|
export function getModelSource(platform) {
|
||||||
|
if (!platform || typeof platform !== 'string') return null;
|
||||||
|
const needle = platform.trim().toLowerCase();
|
||||||
|
return MODEL_SOURCES.find((source) => source.platform === needle) || null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse any supported model URL.
|
||||||
|
* @returns {{platform: string, label: string, groupPrefix: string,
|
||||||
|
* supportsEnrichment: boolean, supportsDownload: boolean,
|
||||||
|
* sourceId: string, url: string}|null}
|
||||||
|
*/
|
||||||
|
export function parseModelSourceUrl(url) {
|
||||||
|
if (!url || typeof url !== 'string') return null;
|
||||||
|
const candidate = url.trim();
|
||||||
|
if (!candidate) return null;
|
||||||
|
for (const source of MODEL_SOURCES) {
|
||||||
|
const match = candidate.match(source.pattern);
|
||||||
|
if (match) {
|
||||||
|
return {
|
||||||
|
...source,
|
||||||
|
sourceId: match[1],
|
||||||
|
url: source.canonical(match[1]),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return null;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Return the stored source URL of a model (new field, then legacy). */
|
||||||
|
export function getModelSourceUrl(model) {
|
||||||
|
if (!model) return '';
|
||||||
|
const value = model.source_url || model.hf_url || '';
|
||||||
|
return typeof value === 'string' ? value.trim() : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Return the stored source platform of a model. */
|
||||||
|
export function getModelSourcePlatform(model) {
|
||||||
|
if (!model) return '';
|
||||||
|
const value = model.source_platform || '';
|
||||||
|
return typeof value === 'string' ? value.trim().toLowerCase() : '';
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Resolve the full source descriptor for a model, tolerating models that
|
||||||
|
* predate the `source_*` fields.
|
||||||
|
*/
|
||||||
|
export function getModelSourceInfo(model) {
|
||||||
|
if (!model) return null;
|
||||||
|
const url = getModelSourceUrl(model);
|
||||||
|
const declared = getModelSource(getModelSourcePlatform(model));
|
||||||
|
const parsed = parseModelSourceUrl(url);
|
||||||
|
|
||||||
|
if (declared) {
|
||||||
|
return {
|
||||||
|
...declared,
|
||||||
|
sourceId: parsed ? parsed.sourceId : '',
|
||||||
|
url: parsed ? parsed.url : url,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
return parsed;
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Version-group key for a model, matching the backend's `_extract_group_key`.
|
||||||
|
* Returns `''` when the model has no external source.
|
||||||
|
*/
|
||||||
|
export function getModelSourceGroupKey(model) {
|
||||||
|
const info = getModelSourceInfo(model);
|
||||||
|
if (!info || !info.sourceId) return '';
|
||||||
|
return `${info.groupPrefix}:${info.sourceId}`;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Whether AI metadata enrichment can run for this model's source. */
|
||||||
|
export function canEnrichModelSource(model) {
|
||||||
|
const info = getModelSourceInfo(model);
|
||||||
|
return Boolean(info && info.supportsEnrichment);
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Parse a version-group key such as `hf:user/repo`, `ms:user/repo`, or
|
||||||
|
* `ta:827823520299086029` back into its source descriptor.
|
||||||
|
*
|
||||||
|
* These keys are NOT CivitAI model ids, so callers must not send them to the
|
||||||
|
* CivitAI API.
|
||||||
|
*
|
||||||
|
* @returns {{platform: string, label: string, sourceId: string}|null}
|
||||||
|
*/
|
||||||
|
export function parseModelSourceGroupKey(groupKey) {
|
||||||
|
if (!groupKey || typeof groupKey !== 'string') return null;
|
||||||
|
const separator = groupKey.indexOf(':');
|
||||||
|
if (separator <= 0) return null;
|
||||||
|
|
||||||
|
const prefix = groupKey.slice(0, separator);
|
||||||
|
const source = MODEL_SOURCES.find((candidate) => candidate.groupPrefix === prefix);
|
||||||
|
if (!source) return null;
|
||||||
|
|
||||||
|
return {
|
||||||
|
platform: source.platform,
|
||||||
|
label: source.label,
|
||||||
|
sourceId: groupKey.slice(separator + 1),
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Localised "View on X" title for the source globe icon. */
|
||||||
|
export function getModelSourceViewTitle(info) {
|
||||||
|
if (!info) return '';
|
||||||
|
if (info.platform === 'huggingface') {
|
||||||
|
return translate('modelCard.actions.viewOnHuggingFace', {}, 'View on Hugging Face');
|
||||||
|
}
|
||||||
|
return translate(
|
||||||
|
'modelCard.actions.viewOnSource',
|
||||||
|
{ source: info.label },
|
||||||
|
`View on ${info.label}`
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Open a model page on its external site in a new tab. */
|
||||||
|
export function openModelSource(url) {
|
||||||
|
if (!url) return;
|
||||||
|
window.open(url, '_blank', 'noopener,noreferrer');
|
||||||
|
}
|
||||||
@@ -21,7 +21,7 @@
|
|||||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="context-menu-item" data-action="link-hf">
|
<div class="context-menu-item" data-action="link-hf">
|
||||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -21,7 +21,7 @@
|
|||||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="context-menu-item" data-action="link-hf">
|
<div class="context-menu-item" data-action="link-hf">
|
||||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -1,24 +1,29 @@
|
|||||||
<!-- Link to HuggingFace Modal -->
|
<!-- Link to Model Source Modal -->
|
||||||
<div id="linkHfModal" class="modal">
|
<div id="linkHfModal" class="modal">
|
||||||
<div class="modal-content">
|
<div class="modal-content">
|
||||||
<button class="close" onclick="modalManager.closeModal('linkHfModal')">×</button>
|
<button class="close" onclick="modalManager.closeModal('linkHfModal')">×</button>
|
||||||
<h2>{{ t('modals.linkHuggingFace.title') }}</h2>
|
<h2>{{ t('modals.linkModelSource.title') }}</h2>
|
||||||
<div class="warning-box">
|
<div class="warning-box">
|
||||||
<i class="fas fa-info-circle"></i>
|
<i class="fas fa-info-circle"></i>
|
||||||
<p>{{ t('modals.linkHuggingFace.infoText') }}</p>
|
<p>{{ t('modals.linkModelSource.infoText') }}</p>
|
||||||
</div>
|
</div>
|
||||||
<div class="input-group">
|
<div class="input-group">
|
||||||
<label for="hfModelUrl">{{ t('modals.linkHuggingFace.urlLabel') }}</label>
|
<label for="hfModelUrl">{{ t('modals.linkModelSource.urlLabel') }}</label>
|
||||||
<input type="text" id="hfModelUrl" placeholder="{{ t('modals.linkHuggingFace.urlPlaceholder') }}" />
|
<input type="text" id="hfModelUrl" placeholder="{{ t('modals.linkModelSource.urlPlaceholder') }}" />
|
||||||
<div class="input-error" id="hfModelUrlError"></div>
|
<div class="input-error" id="hfModelUrlError"></div>
|
||||||
<div class="input-help">
|
<div class="input-help">
|
||||||
{{ t('modals.linkHuggingFace.helpText') }}<br>
|
{{ t('modals.linkModelSource.helpText') }}
|
||||||
<strong>https://huggingface.co/user/repo</strong>
|
<div id="hfSupportedSources">
|
||||||
|
<strong>https://huggingface.co/user/repo</strong><br>
|
||||||
|
<strong>https://modelscope.cn/models/user/repo</strong><br>
|
||||||
|
<strong>https://tensor.art/models/827823520299086029</strong>
|
||||||
|
</div>
|
||||||
|
{{ t('modals.linkModelSource.enrichNote') }}
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="modal-actions">
|
<div class="modal-actions">
|
||||||
<button class="cancel-btn" onclick="modalManager.closeModal('linkHfModal')">{{ t('common.actions.cancel') }}</button>
|
<button class="cancel-btn" onclick="modalManager.closeModal('linkHfModal')">{{ t('common.actions.cancel') }}</button>
|
||||||
<button class="confirm-btn" id="confirmLinkHfBtn">{{ t('modals.linkHuggingFace.confirmAction') }}</button>
|
<button class="confirm-btn" id="confirmLinkHfBtn">{{ t('modals.linkModelSource.confirmAction') }}</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -21,7 +21,7 @@
|
|||||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="context-menu-item" data-action="link-hf">
|
<div class="context-menu-item" data-action="link-hf">
|
||||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -80,7 +80,7 @@
|
|||||||
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
<i class="fas fa-external-link-alt"></i> <span>{{ t('loras.contextMenu.linkCivitai') }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="context-menu-item" data-action="link-hf">
|
<div class="context-menu-item" data-action="link-hf">
|
||||||
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkHuggingFace') }}</span>
|
<i class="fas fa-robot"></i> <span>{{ t('loras.contextMenu.linkModelSource') }}</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -191,4 +191,60 @@ describe('ModelCard source globe (#1094)', () => {
|
|||||||
expect(openHuggingFace).toHaveBeenCalledWith('https://huggingface.co/user/repo');
|
expect(openHuggingFace).toHaveBeenCalledWith('https://huggingface.co/user/repo');
|
||||||
expect(openCivitai).not.toHaveBeenCalled();
|
expect(openCivitai).not.toHaveBeenCalled();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('points the globe at ModelScope for a ModelScope-linked model', () => {
|
||||||
|
const card = mountCard(
|
||||||
|
createModelCard,
|
||||||
|
makeModel({
|
||||||
|
from_civitai: false,
|
||||||
|
civitai: {},
|
||||||
|
source_platform: 'modelscope',
|
||||||
|
source_url: 'https://modelscope.cn/models/user/repo',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(card.dataset.has_civitai).toBe('false');
|
||||||
|
expect(card.dataset.source_platform).toBe('modelscope');
|
||||||
|
expect(card.dataset.hf_url).toBe('');
|
||||||
|
expect(card.querySelector('.fa-globe').getAttribute('title')).toBe('View on ModelScope');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('opens the ModelScope page when the globe is clicked', () => {
|
||||||
|
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||||
|
const card = mountCard(
|
||||||
|
createModelCard,
|
||||||
|
makeModel({
|
||||||
|
from_civitai: false,
|
||||||
|
civitai: {},
|
||||||
|
source_platform: 'modelscope',
|
||||||
|
source_url: 'https://modelscope.cn/models/user/repo',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
setupModelCardEventDelegation('loras');
|
||||||
|
|
||||||
|
card.querySelector('.fa-globe').dispatchEvent(new MouseEvent('click', { bubbles: true }));
|
||||||
|
|
||||||
|
expect(openSpy).toHaveBeenCalledWith(
|
||||||
|
'https://modelscope.cn/models/user/repo',
|
||||||
|
'_blank',
|
||||||
|
'noopener,noreferrer'
|
||||||
|
);
|
||||||
|
expect(openCivitai).not.toHaveBeenCalled();
|
||||||
|
expect(openHuggingFace).not.toHaveBeenCalled();
|
||||||
|
openSpy.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('points the globe at TensorArt for a TensorArt-linked model', () => {
|
||||||
|
const card = mountCard(
|
||||||
|
createModelCard,
|
||||||
|
makeModel({
|
||||||
|
from_civitai: false,
|
||||||
|
civitai: {},
|
||||||
|
source_platform: 'tensorart',
|
||||||
|
source_url: 'https://tensor.art/models/827823520299086029',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
expect(card.querySelector('.fa-globe').getAttribute('title')).toBe('View on TensorArt');
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import { describe, expect, it } from 'vitest';
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
|
|
||||||
import { ModelContextMenuMixin } from '../../../static/js/components/ContextMenu/ModelContextMenuMixin.js';
|
import { ModelContextMenuMixin } from '../../../static/js/components/ContextMenu/ModelContextMenuMixin.js';
|
||||||
|
|
||||||
@@ -14,3 +14,115 @@ describe('ModelContextMenuMixin.getModelTypePrefix', () => {
|
|||||||
expect(ModelContextMenuMixin.getModelTypePrefix.call({})).toBe('loras');
|
expect(ModelContextMenuMixin.getModelTypePrefix.call({})).toBe('loras');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('ModelContextMenuMixin.updateEnrichMenuItem', () => {
|
||||||
|
function setupMenu() {
|
||||||
|
document.body.innerHTML = '<div id="menu"><div data-action="enrich-hf-llm"></div></div>';
|
||||||
|
return { menu: document.getElementById('menu') };
|
||||||
|
}
|
||||||
|
|
||||||
|
function cardWith(dataset) {
|
||||||
|
return { dataset };
|
||||||
|
}
|
||||||
|
|
||||||
|
it('enables enrichment for Hugging Face links', () => {
|
||||||
|
const context = setupMenu();
|
||||||
|
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||||
|
context,
|
||||||
|
cardWith({ hf_url: 'https://huggingface.co/user/repo' })
|
||||||
|
);
|
||||||
|
|
||||||
|
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||||
|
expect(item.classList.contains('disabled')).toBe(false);
|
||||||
|
expect(item.title).toBe('');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('enables enrichment for ModelScope links', () => {
|
||||||
|
const context = setupMenu();
|
||||||
|
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||||
|
context,
|
||||||
|
cardWith({
|
||||||
|
source_platform: 'modelscope',
|
||||||
|
source_url: 'https://modelscope.cn/models/user/repo',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||||
|
expect(item.classList.contains('disabled')).toBe(false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('disables enrichment for TensorArt and explains why', () => {
|
||||||
|
const context = setupMenu();
|
||||||
|
ModelContextMenuMixin.updateEnrichMenuItem.call(
|
||||||
|
context,
|
||||||
|
cardWith({
|
||||||
|
source_platform: 'tensorart',
|
||||||
|
source_url: 'https://tensor.art/models/827823520299086029',
|
||||||
|
})
|
||||||
|
);
|
||||||
|
|
||||||
|
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||||
|
expect(item.classList.contains('disabled')).toBe(true);
|
||||||
|
expect(item.title).toContain('TensorArt');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('disables enrichment when no source is linked', () => {
|
||||||
|
const context = setupMenu();
|
||||||
|
ModelContextMenuMixin.updateEnrichMenuItem.call(context, cardWith({}));
|
||||||
|
|
||||||
|
const item = context.menu.querySelector('[data-action="enrich-hf-llm"]');
|
||||||
|
expect(item.classList.contains('disabled')).toBe(true);
|
||||||
|
expect(item.title).toContain('Link this model to a model source');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('ModelContextMenuMixin._renderSupportedSources', () => {
|
||||||
|
const originalFetch = global.fetch;
|
||||||
|
|
||||||
|
beforeEach(() => {
|
||||||
|
document.body.innerHTML = '<div id="hfSupportedSources">static fallback</div>';
|
||||||
|
});
|
||||||
|
|
||||||
|
afterEach(() => {
|
||||||
|
global.fetch = originalFetch;
|
||||||
|
});
|
||||||
|
|
||||||
|
it('renders the server-provided example URLs', async () => {
|
||||||
|
global.fetch = vi.fn().mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: async () => [
|
||||||
|
{ platform: 'huggingface', example_url: 'https://huggingface.co/user/repo' },
|
||||||
|
{ platform: 'modelscope', example_url: 'https://modelscope.cn/models/user/repo' },
|
||||||
|
{ platform: 'tensorart', example_url: 'https://tensor.art/models/123' },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||||
|
|
||||||
|
const html = document.getElementById('hfSupportedSources').innerHTML;
|
||||||
|
expect(html).toContain('https://huggingface.co/user/repo');
|
||||||
|
expect(html).toContain('https://modelscope.cn/models/user/repo');
|
||||||
|
expect(html).toContain('https://tensor.art/models/123');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the static fallback when the request fails', async () => {
|
||||||
|
global.fetch = vi.fn().mockRejectedValue(new Error('offline'));
|
||||||
|
|
||||||
|
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||||
|
|
||||||
|
expect(document.getElementById('hfSupportedSources').innerHTML).toBe('static fallback');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('escapes markup from the server payload', async () => {
|
||||||
|
global.fetch = vi.fn().mockResolvedValue({
|
||||||
|
ok: true,
|
||||||
|
json: async () => [{ example_url: '<img src=x onerror=alert(1)>' }],
|
||||||
|
});
|
||||||
|
|
||||||
|
await ModelContextMenuMixin._renderSupportedSources.call({});
|
||||||
|
|
||||||
|
const html = document.getElementById('hfSupportedSources').innerHTML;
|
||||||
|
expect(html).not.toContain('<img');
|
||||||
|
expect(html).toContain('<img');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|||||||
@@ -0,0 +1,177 @@
|
|||||||
|
import { describe, it, expect, vi } from 'vitest';
|
||||||
|
|
||||||
|
const { I18N_MODULE } = vi.hoisted(() => ({
|
||||||
|
I18N_MODULE: new URL('../../../static/js/utils/i18nHelpers.js', import.meta.url).pathname,
|
||||||
|
}));
|
||||||
|
|
||||||
|
vi.mock(I18N_MODULE, () => ({
|
||||||
|
translate: vi.fn((key, params, fallback) => (typeof fallback === 'string' ? fallback : key)),
|
||||||
|
}));
|
||||||
|
|
||||||
|
const {
|
||||||
|
MODEL_SOURCES,
|
||||||
|
parseModelSourceUrl,
|
||||||
|
getModelSource,
|
||||||
|
getModelSourceInfo,
|
||||||
|
getModelSourceUrl,
|
||||||
|
getModelSourceGroupKey,
|
||||||
|
canEnrichModelSource,
|
||||||
|
getModelSourceViewTitle,
|
||||||
|
parseModelSourceGroupKey,
|
||||||
|
openModelSource,
|
||||||
|
} = await import('../../../static/js/utils/modelSourceHelpers.js');
|
||||||
|
|
||||||
|
describe('modelSourceHelpers', () => {
|
||||||
|
it('exposes one descriptor per supported platform', () => {
|
||||||
|
expect(MODEL_SOURCES.map((s) => s.platform)).toEqual([
|
||||||
|
'huggingface',
|
||||||
|
'modelscope',
|
||||||
|
'tensorart',
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('parseModelSourceUrl', () => {
|
||||||
|
it('recognises Hugging Face URLs', () => {
|
||||||
|
const info = parseModelSourceUrl('https://huggingface.co/user/repo');
|
||||||
|
expect(info.platform).toBe('huggingface');
|
||||||
|
expect(info.sourceId).toBe('user/repo');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('recognises ModelScope URLs with view sub-paths', () => {
|
||||||
|
const info = parseModelSourceUrl('https://modelscope.cn/models/user/repo/summary');
|
||||||
|
expect(info.platform).toBe('modelscope');
|
||||||
|
expect(info.sourceId).toBe('user/repo');
|
||||||
|
expect(info.url).toBe('https://modelscope.cn/models/user/repo');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('recognises TensorArt URLs and keeps only the numeric id', () => {
|
||||||
|
const info = parseModelSourceUrl(
|
||||||
|
'https://tensor.art/models/827823520299086029/Vivid-Impressions-Storybook-Sstyle-V1.0'
|
||||||
|
);
|
||||||
|
expect(info.platform).toBe('tensorart');
|
||||||
|
expect(info.sourceId).toBe('827823520299086029');
|
||||||
|
expect(info.url).toBe('https://tensor.art/models/827823520299086029');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects unsupported URLs', () => {
|
||||||
|
expect(parseModelSourceUrl('https://example.com/x')).toBeNull();
|
||||||
|
expect(parseModelSourceUrl('')).toBeNull();
|
||||||
|
expect(parseModelSourceUrl(null)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('getModelSourceInfo', () => {
|
||||||
|
it('falls back to the legacy hf_url field', () => {
|
||||||
|
const info = getModelSourceInfo({ hf_url: 'https://huggingface.co/user/repo' });
|
||||||
|
expect(info.platform).toBe('huggingface');
|
||||||
|
expect(info.sourceId).toBe('user/repo');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('prefers the canonical source fields', () => {
|
||||||
|
const info = getModelSourceInfo({
|
||||||
|
source_platform: 'modelscope',
|
||||||
|
source_url: 'https://modelscope.cn/models/user/repo',
|
||||||
|
hf_url: 'https://huggingface.co/old/repo',
|
||||||
|
});
|
||||||
|
expect(info.platform).toBe('modelscope');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns null when there is no source', () => {
|
||||||
|
expect(getModelSourceInfo({})).toBeNull();
|
||||||
|
expect(getModelSourceInfo({ hf_url: '' })).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('getModelSourceUrl', () => {
|
||||||
|
it('reads source_url then hf_url', () => {
|
||||||
|
expect(getModelSourceUrl({ source_url: 'https://a.example/1' })).toBe('https://a.example/1');
|
||||||
|
expect(getModelSourceUrl({ hf_url: 'https://huggingface.co/u/r' })).toBe(
|
||||||
|
'https://huggingface.co/u/r'
|
||||||
|
);
|
||||||
|
expect(getModelSourceUrl({})).toBe('');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('getModelSourceGroupKey', () => {
|
||||||
|
it('matches the backend group-key shapes', () => {
|
||||||
|
expect(getModelSourceGroupKey({ hf_url: 'https://huggingface.co/u/r' })).toBe('hf:u/r');
|
||||||
|
expect(
|
||||||
|
getModelSourceGroupKey({ source_url: 'https://modelscope.cn/models/u/r' })
|
||||||
|
).toBe('ms:u/r');
|
||||||
|
expect(getModelSourceGroupKey({ source_url: 'https://tensor.art/models/123' })).toBe(
|
||||||
|
'ta:123'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('returns an empty string without a source', () => {
|
||||||
|
expect(getModelSourceGroupKey({})).toBe('');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('canEnrichModelSource', () => {
|
||||||
|
it('allows Hugging Face and ModelScope', () => {
|
||||||
|
expect(canEnrichModelSource({ hf_url: 'https://huggingface.co/u/r' })).toBe(true);
|
||||||
|
expect(
|
||||||
|
canEnrichModelSource({ source_url: 'https://modelscope.cn/models/u/r' })
|
||||||
|
).toBe(true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('disallows TensorArt and unlinked models', () => {
|
||||||
|
expect(canEnrichModelSource({ source_url: 'https://tensor.art/models/123' })).toBe(false);
|
||||||
|
expect(canEnrichModelSource({})).toBe(false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('getModelSourceViewTitle', () => {
|
||||||
|
it('uses the branded label for non-HF sources', () => {
|
||||||
|
expect(getModelSourceViewTitle(getModelSource('modelscope'))).toBe('View on ModelScope');
|
||||||
|
expect(getModelSourceViewTitle(getModelSource('tensorart'))).toBe('View on TensorArt');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('keeps the historical Hugging Face title', () => {
|
||||||
|
expect(getModelSourceViewTitle(getModelSource('huggingface'))).toBe(
|
||||||
|
'View on Hugging Face'
|
||||||
|
);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('parseModelSourceGroupKey', () => {
|
||||||
|
it('parses every external group-key prefix', () => {
|
||||||
|
expect(parseModelSourceGroupKey('hf:user/repo')).toEqual({
|
||||||
|
platform: 'huggingface',
|
||||||
|
label: 'Hugging Face',
|
||||||
|
sourceId: 'user/repo',
|
||||||
|
});
|
||||||
|
expect(parseModelSourceGroupKey('ms:user/repo').platform).toBe('modelscope');
|
||||||
|
expect(parseModelSourceGroupKey('ta:123').platform).toBe('tensorart');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('rejects numeric CivitAI model ids and unknown prefixes', () => {
|
||||||
|
expect(parseModelSourceGroupKey(222)).toBeNull();
|
||||||
|
expect(parseModelSourceGroupKey('222')).toBeNull();
|
||||||
|
expect(parseModelSourceGroupKey('unknown:1')).toBeNull();
|
||||||
|
expect(parseModelSourceGroupKey('')).toBeNull();
|
||||||
|
expect(parseModelSourceGroupKey(null)).toBeNull();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe('openModelSource', () => {
|
||||||
|
it('opens the URL in a new tab', () => {
|
||||||
|
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||||
|
openModelSource('https://modelscope.cn/models/u/r');
|
||||||
|
expect(openSpy).toHaveBeenCalledWith(
|
||||||
|
'https://modelscope.cn/models/u/r',
|
||||||
|
'_blank',
|
||||||
|
'noopener,noreferrer'
|
||||||
|
);
|
||||||
|
openSpy.mockRestore();
|
||||||
|
});
|
||||||
|
|
||||||
|
it('does nothing without a URL', () => {
|
||||||
|
const openSpy = vi.spyOn(window, 'open').mockImplementation(() => {});
|
||||||
|
openModelSource('');
|
||||||
|
expect(openSpy).not.toHaveBeenCalled();
|
||||||
|
openSpy.mockRestore();
|
||||||
|
});
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -154,3 +154,155 @@ async def test_set_hf_url_rejects_non_repo_url(tmp_path, hf_env):
|
|||||||
payload = _json_payload(response)
|
payload = _json_payload(response)
|
||||||
assert payload["success"] is False
|
assert payload["success"] is False
|
||||||
hf_env["cache_write"].assert_not_awaited()
|
hf_env["cache_write"].assert_not_awaited()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Multi-source linking (ModelScope / TensorArt)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_plain_model(model_path, sha: str = "d" * 64) -> None:
|
||||||
|
await _write_model(
|
||||||
|
model_path,
|
||||||
|
{
|
||||||
|
"file_name": "model",
|
||||||
|
"model_name": "model",
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"size": 32,
|
||||||
|
"modified": 1.0,
|
||||||
|
"sha256": sha,
|
||||||
|
"base_model": "Unknown",
|
||||||
|
"preview_url": "",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_hf_url_accepts_modelscope_and_stores_source_fields(tmp_path, hf_env):
|
||||||
|
model_path = tmp_path / "ms_model.safetensors"
|
||||||
|
await _write_plain_model(model_path)
|
||||||
|
|
||||||
|
response = await HfHandler().set_hf_url(
|
||||||
|
FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": "https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status == 200
|
||||||
|
payload = _json_payload(response)
|
||||||
|
assert payload["source_platform"] == "modelscope"
|
||||||
|
assert payload["source_url"] == "https://modelscope.cn/models/jj3550945163/Krea-2-LORA"
|
||||||
|
|
||||||
|
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||||
|
assert saved["source_platform"] == "modelscope"
|
||||||
|
assert saved["source_url"] == "https://modelscope.cn/models/jj3550945163/Krea-2-LORA"
|
||||||
|
# No stale Hugging Face alias for a ModelScope model.
|
||||||
|
assert saved.get("hf_url", "") == ""
|
||||||
|
|
||||||
|
cached_metadata = hf_env["cache_write"].await_args.args[1]
|
||||||
|
assert cached_metadata["source_platform"] == "modelscope"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_hf_url_accepts_tensorart_url(tmp_path, hf_env):
|
||||||
|
model_path = tmp_path / "ta_model.safetensors"
|
||||||
|
await _write_plain_model(model_path, sha="e" * 64)
|
||||||
|
|
||||||
|
response = await HfHandler().set_hf_url(
|
||||||
|
FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": (
|
||||||
|
"https://tensor.art/models/827823520299086029/"
|
||||||
|
"Vivid-Impressions-Storybook-Sstyle-V1.0"
|
||||||
|
),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status == 200
|
||||||
|
payload = _json_payload(response)
|
||||||
|
assert payload["source_platform"] == "tensorart"
|
||||||
|
# The canonical page URL is stored, without the slug.
|
||||||
|
assert payload["source_url"] == "https://tensor.art/models/827823520299086029"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_hf_url_canonicalises_modelscope_subpage(tmp_path, hf_env):
|
||||||
|
model_path = tmp_path / "ms_sub.safetensors"
|
||||||
|
await _write_plain_model(model_path, sha="f" * 64)
|
||||||
|
|
||||||
|
response = await HfHandler().set_hf_url(
|
||||||
|
FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo/summary",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status == 200
|
||||||
|
assert _json_payload(response)["source_url"] == "https://modelscope.cn/models/user/repo"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_hf_url_is_idempotent_for_modelscope(tmp_path, hf_env):
|
||||||
|
model_path = tmp_path / "ms_twice.safetensors"
|
||||||
|
await _write_plain_model(model_path, sha="1" * 64)
|
||||||
|
|
||||||
|
request = FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
await HfHandler().set_hf_url(request)
|
||||||
|
await HfHandler().set_hf_url(request)
|
||||||
|
|
||||||
|
# The second call short-circuits without rewriting the cache entry.
|
||||||
|
assert hf_env["cache_write"].await_count == 1
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_set_hf_url_switching_source_clears_hf_alias(tmp_path, hf_env):
|
||||||
|
model_path = tmp_path / "switch.safetensors"
|
||||||
|
await _write_plain_model(model_path, sha="2" * 64)
|
||||||
|
|
||||||
|
await HfHandler().set_hf_url(
|
||||||
|
FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": "https://huggingface.co/user/repo",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await HfHandler().set_hf_url(
|
||||||
|
FakeRequest(
|
||||||
|
json_data={
|
||||||
|
"file_path": str(model_path),
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
saved = json.loads(open(_sidecar_path(model_path), encoding="utf-8").read())
|
||||||
|
assert saved["source_platform"] == "modelscope"
|
||||||
|
assert saved.get("hf_url", "") == ""
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_get_model_sources_lists_capabilities():
|
||||||
|
response = await HfHandler().get_model_sources(FakeRequest())
|
||||||
|
sources = _json_payload(response)
|
||||||
|
|
||||||
|
by_platform = {s["platform"]: s for s in sources}
|
||||||
|
assert set(by_platform) == {"huggingface", "modelscope", "tensorart"}
|
||||||
|
assert by_platform["huggingface"]["supports_enrichment"] is True
|
||||||
|
assert by_platform["modelscope"]["supports_enrichment"] is True
|
||||||
|
# TensorArt is link-only: no accessible model card for the backend.
|
||||||
|
assert by_platform["tensorart"]["supports_enrichment"] is False
|
||||||
|
assert by_platform["modelscope"]["supports_download"] is False
|
||||||
|
assert all(s["example_url"] for s in sources)
|
||||||
|
|||||||
@@ -0,0 +1,182 @@
|
|||||||
|
"""Tests for source-aware AI enrichment orchestration.
|
||||||
|
|
||||||
|
Covers the fast-fail gate (:meth:`AgentService._enrichment_skip_reason`) and
|
||||||
|
the prompt-context builder for non-Hugging Face model sources.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from unittest import mock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from py.services.agent.agent_service import AgentService
|
||||||
|
|
||||||
|
|
||||||
|
class TestEnrichmentSkipReason:
|
||||||
|
def test_skips_when_no_source_linked(self):
|
||||||
|
reason = AgentService._enrichment_skip_reason({})
|
||||||
|
assert "source_url" in reason
|
||||||
|
|
||||||
|
def test_allows_huggingface(self):
|
||||||
|
assert (
|
||||||
|
AgentService._enrichment_skip_reason(
|
||||||
|
{"hf_url": "https://huggingface.co/user/repo"}
|
||||||
|
)
|
||||||
|
== ""
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_allows_modelscope(self):
|
||||||
|
assert (
|
||||||
|
AgentService._enrichment_skip_reason(
|
||||||
|
{
|
||||||
|
"source_platform": "modelscope",
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
== ""
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_skips_tensorart_with_reason(self):
|
||||||
|
reason = AgentService._enrichment_skip_reason(
|
||||||
|
{
|
||||||
|
"source_platform": "tensorart",
|
||||||
|
"source_url": "https://tensor.art/models/827823520299086029",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
assert "TensorArt" in reason
|
||||||
|
assert "not available" in reason
|
||||||
|
|
||||||
|
def test_skips_unknown_platform(self):
|
||||||
|
reason = AgentService._enrichment_skip_reason(
|
||||||
|
{"source_platform": "somewhere", "source_url": "https://somewhere.example/m/1"}
|
||||||
|
)
|
||||||
|
assert "somewhere" in reason
|
||||||
|
|
||||||
|
|
||||||
|
class TestBuildPromptContext:
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_modelscope_card_populates_source_variables(self):
|
||||||
|
service = AgentService()
|
||||||
|
readme = "---\nbase_model: krea/Krea-2-Turbo\n---\n# krea\n"
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"py.services.model_sources.modelscope.ModelScopeSource.fetch_model_card",
|
||||||
|
new=mock.AsyncMock(return_value=readme),
|
||||||
|
) as mock_fetch,
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.list_base_models",
|
||||||
|
new=mock.AsyncMock(return_value=["Krea 2 Turbo"]),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.identify_model_type",
|
||||||
|
new=mock.AsyncMock(return_value="lora"),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||||
|
return_value={"lora": "style, subject"},
|
||||||
|
),
|
||||||
|
):
|
||||||
|
context = await service._build_prompt_context(
|
||||||
|
skill_name="enrich_hf_metadata",
|
||||||
|
model_path="/models/loras/krea.safetensors",
|
||||||
|
metadata={
|
||||||
|
"source_platform": "modelscope",
|
||||||
|
"source_url": "https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||||
|
"file_name": "krea",
|
||||||
|
},
|
||||||
|
registry=mock.Mock(),
|
||||||
|
llm=mock.Mock(),
|
||||||
|
)
|
||||||
|
|
||||||
|
mock_fetch.assert_awaited_once_with("jj3550945163/Krea-2-LORA")
|
||||||
|
assert context["source_platform"] == "modelscope"
|
||||||
|
assert context["source_id"] == "jj3550945163/Krea-2-LORA"
|
||||||
|
assert context["source_label"] == "ModelScope"
|
||||||
|
assert (
|
||||||
|
context["asset_base_url"]
|
||||||
|
== "https://modelscope.cn/models/jj3550945163/Krea-2-LORA/resolve/master"
|
||||||
|
)
|
||||||
|
assert readme in context["readme_content_full"]
|
||||||
|
# Hugging Face aliases stay empty for a non-HF source.
|
||||||
|
assert context["hf_url"] == ""
|
||||||
|
assert context["repo"] == "jj3550945163/Krea-2-LORA"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_huggingface_keeps_legacy_aliases(self):
|
||||||
|
service = AgentService()
|
||||||
|
readme = "# card\n"
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"py.services.model_sources.huggingface.HuggingFaceSource.fetch_model_card",
|
||||||
|
new=mock.AsyncMock(return_value=readme),
|
||||||
|
) as mock_fetch,
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.list_base_models",
|
||||||
|
new=mock.AsyncMock(return_value=[]),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.identify_model_type",
|
||||||
|
new=mock.AsyncMock(return_value="lora"),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||||
|
return_value={},
|
||||||
|
),
|
||||||
|
):
|
||||||
|
context = await service._build_prompt_context(
|
||||||
|
skill_name="enrich_hf_metadata",
|
||||||
|
model_path="/models/loras/thing.safetensors",
|
||||||
|
metadata={"hf_url": "https://huggingface.co/user/repo"},
|
||||||
|
registry=mock.Mock(),
|
||||||
|
llm=mock.Mock(),
|
||||||
|
)
|
||||||
|
|
||||||
|
mock_fetch.assert_awaited_once_with("user/repo")
|
||||||
|
assert context["source_platform"] == "huggingface"
|
||||||
|
assert context["hf_url"] == "https://huggingface.co/user/repo"
|
||||||
|
assert context["repo"] == "user/repo"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tensorart_never_fetches_a_card(self):
|
||||||
|
service = AgentService()
|
||||||
|
|
||||||
|
with (
|
||||||
|
mock.patch(
|
||||||
|
"py.services.model_sources.huggingface.HuggingFaceSource.fetch_model_card",
|
||||||
|
new=mock.AsyncMock(),
|
||||||
|
) as hf_fetch,
|
||||||
|
mock.patch(
|
||||||
|
"py.services.model_sources.modelscope.ModelScopeSource.fetch_model_card",
|
||||||
|
new=mock.AsyncMock(),
|
||||||
|
) as ms_fetch,
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.list_base_models",
|
||||||
|
new=mock.AsyncMock(return_value=[]),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.metadata_ops.identify_model_type",
|
||||||
|
new=mock.AsyncMock(return_value="lora"),
|
||||||
|
),
|
||||||
|
mock.patch(
|
||||||
|
"py.services.settings_manager.SettingsManager.get_priority_tag_config",
|
||||||
|
return_value={},
|
||||||
|
),
|
||||||
|
):
|
||||||
|
context = await service._build_prompt_context(
|
||||||
|
skill_name="enrich_hf_metadata",
|
||||||
|
model_path="/models/loras/thing.safetensors",
|
||||||
|
metadata={
|
||||||
|
"source_platform": "tensorart",
|
||||||
|
"source_url": "https://tensor.art/models/827823520299086029",
|
||||||
|
},
|
||||||
|
registry=mock.Mock(),
|
||||||
|
llm=mock.Mock(),
|
||||||
|
)
|
||||||
|
|
||||||
|
hf_fetch.assert_not_awaited()
|
||||||
|
ms_fetch.assert_not_awaited()
|
||||||
|
assert context["readme_content"] == ""
|
||||||
|
assert context["source_platform"] == "tensorart"
|
||||||
@@ -964,6 +964,8 @@ def _make_cache_entry(**overrides) -> Dict[str, Any]:
|
|||||||
"civitai": {"id": 111, "modelId": 222, "name": "v1"},
|
"civitai": {"id": 111, "modelId": 222, "name": "v1"},
|
||||||
"civitai_deleted": False,
|
"civitai_deleted": False,
|
||||||
"skip_metadata_refresh": False,
|
"skip_metadata_refresh": False,
|
||||||
|
"source_platform": "",
|
||||||
|
"source_url": "",
|
||||||
"hf_url": "",
|
"hf_url": "",
|
||||||
"license_flags": 113,
|
"license_flags": 113,
|
||||||
"hash_status": "completed",
|
"hash_status": "completed",
|
||||||
|
|||||||
@@ -0,0 +1,324 @@
|
|||||||
|
"""Tests for the external model-source provider registry.
|
||||||
|
|
||||||
|
Covers URL recognition for Hugging Face / ModelScope / TensorArt, the
|
||||||
|
legacy ``hf_url`` → ``source_url`` normalisation, version-group keys, and
|
||||||
|
each provider's model-card fetching and capability flags.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from py.services.model_sources import (
|
||||||
|
HuggingFaceSource,
|
||||||
|
ModelScopeSource,
|
||||||
|
TensorArtSource,
|
||||||
|
detect_source,
|
||||||
|
get_source,
|
||||||
|
get_source_platform,
|
||||||
|
has_external_source,
|
||||||
|
list_sources,
|
||||||
|
normalize_metadata_source,
|
||||||
|
resolve_source_ref,
|
||||||
|
source_group_key,
|
||||||
|
source_label,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# URL recognition
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestDetectSource:
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
("url", "platform", "source_id"),
|
||||||
|
[
|
||||||
|
("https://huggingface.co/user/repo", "huggingface", "user/repo"),
|
||||||
|
("https://www.huggingface.co/user/repo", "huggingface", "user/repo"),
|
||||||
|
(
|
||||||
|
"https://huggingface.co/user/repo/resolve/main/model.safetensors",
|
||||||
|
"huggingface",
|
||||||
|
"user/repo",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"https://modelscope.cn/models/jj3550945163/Krea-2-LORA",
|
||||||
|
"modelscope",
|
||||||
|
"jj3550945163/Krea-2-LORA",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"https://www.modelscope.cn/models/jj3550945163/Krea-2-LORA/summary",
|
||||||
|
"modelscope",
|
||||||
|
"jj3550945163/Krea-2-LORA",
|
||||||
|
),
|
||||||
|
(
|
||||||
|
"https://tensor.art/models/827823520299086029/Vivid-Impressions-Storybook-Sstyle-V1.0",
|
||||||
|
"tensorart",
|
||||||
|
"827823520299086029",
|
||||||
|
),
|
||||||
|
("https://tusi.cn/models/827823520299086029", "tensorart", "827823520299086029"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_recognises_supported_urls(self, url, platform, source_id):
|
||||||
|
ref = detect_source(url)
|
||||||
|
assert ref is not None
|
||||||
|
assert ref.platform == platform
|
||||||
|
assert ref.source_id == source_id
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"url",
|
||||||
|
[
|
||||||
|
"",
|
||||||
|
None,
|
||||||
|
"not-a-url",
|
||||||
|
"https://example.com/x",
|
||||||
|
"https://civitai.com/models/123",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_ignores_unsupported_urls(self, url):
|
||||||
|
assert detect_source(url) is None
|
||||||
|
|
||||||
|
def test_canonical_url_is_stable(self):
|
||||||
|
assert detect_source("https://huggingface.co/u/r").url == "https://huggingface.co/u/r"
|
||||||
|
assert (
|
||||||
|
detect_source("https://modelscope.cn/models/u/r/summary").url
|
||||||
|
== "https://modelscope.cn/models/u/r"
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
detect_source("https://tensor.art/models/123/some-slug").url
|
||||||
|
== "https://tensor.art/models/123"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class TestStrictParsing:
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"url",
|
||||||
|
[
|
||||||
|
"https://huggingface.co/user/repo",
|
||||||
|
"https://huggingface.co/user/repo/",
|
||||||
|
"https://modelscope.cn/models/user/repo",
|
||||||
|
"https://modelscope.cn/models/user/repo/summary",
|
||||||
|
"https://tensor.art/models/827823520299086029",
|
||||||
|
"https://tensor.art/models/827823520299086029/Vivid-Impressions",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_accepts_user_facing_urls(self, url):
|
||||||
|
assert detect_source(url, strict=True) is not None
|
||||||
|
|
||||||
|
@pytest.mark.parametrize(
|
||||||
|
"url",
|
||||||
|
[
|
||||||
|
"https://huggingface.co/user/repo/resolve/main/model.safetensors",
|
||||||
|
"https://example.com/x",
|
||||||
|
"https://tensor.art/models/not-a-number",
|
||||||
|
],
|
||||||
|
)
|
||||||
|
def test_rejects_non_page_urls(self, url):
|
||||||
|
assert detect_source(url, strict=True) is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Capabilities
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestCapabilities:
|
||||||
|
def test_huggingface_supports_everything(self):
|
||||||
|
source = get_source("huggingface")
|
||||||
|
assert source.supports_enrichment is True
|
||||||
|
assert source.supports_download is True
|
||||||
|
|
||||||
|
def test_modelscope_supports_enrichment_but_not_download(self):
|
||||||
|
source = get_source("modelscope")
|
||||||
|
assert source.supports_enrichment is True
|
||||||
|
assert source.supports_download is False
|
||||||
|
|
||||||
|
def test_tensorart_is_link_only(self):
|
||||||
|
source = get_source("tensorart")
|
||||||
|
assert source.supports_enrichment is False
|
||||||
|
assert source.supports_download is False
|
||||||
|
|
||||||
|
def test_registry_lists_every_source(self):
|
||||||
|
platforms = {s.platform for s in list_sources()}
|
||||||
|
assert platforms == {"huggingface", "modelscope", "tensorart"}
|
||||||
|
|
||||||
|
def test_labels_are_brand_names(self):
|
||||||
|
assert source_label("huggingface") == "Hugging Face"
|
||||||
|
assert source_label("modelscope") == "ModelScope"
|
||||||
|
assert source_label("tensorart") == "TensorArt"
|
||||||
|
assert source_label("unknown", "fallback") == "fallback"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Metadata normalisation
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestNormalizeMetadataSource:
|
||||||
|
def test_derives_source_fields_from_legacy_hf_url(self):
|
||||||
|
metadata = {"hf_url": "https://huggingface.co/user/repo"}
|
||||||
|
normalize_metadata_source(metadata)
|
||||||
|
assert metadata["source_platform"] == "huggingface"
|
||||||
|
assert metadata["source_url"] == "https://huggingface.co/user/repo"
|
||||||
|
assert metadata["hf_url"] == "https://huggingface.co/user/repo"
|
||||||
|
|
||||||
|
def test_canonicalises_modelscope_url_and_clears_hf_alias(self):
|
||||||
|
metadata = {
|
||||||
|
"source_platform": "modelscope",
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo/summary",
|
||||||
|
"hf_url": "https://huggingface.co/old/repo",
|
||||||
|
}
|
||||||
|
normalize_metadata_source(metadata)
|
||||||
|
assert metadata["source_platform"] == "modelscope"
|
||||||
|
assert metadata["source_url"] == "https://modelscope.cn/models/user/repo"
|
||||||
|
# A stale HF alias must not make a ModelScope model look like HF.
|
||||||
|
assert metadata["hf_url"] == ""
|
||||||
|
|
||||||
|
def test_preserves_unknown_url_for_unknown_platform(self):
|
||||||
|
metadata = {"source_url": "https://example.com/model/1", "source_platform": "other"}
|
||||||
|
normalize_metadata_source(metadata)
|
||||||
|
assert metadata["source_url"] == "https://example.com/model/1"
|
||||||
|
assert metadata["source_platform"] == "other"
|
||||||
|
|
||||||
|
def test_empty_metadata_gets_default_fields(self):
|
||||||
|
metadata: dict = {}
|
||||||
|
normalize_metadata_source(metadata)
|
||||||
|
assert metadata["source_platform"] == ""
|
||||||
|
assert metadata["source_url"] == ""
|
||||||
|
|
||||||
|
def test_infers_platform_from_url_when_missing(self):
|
||||||
|
metadata = {"source_url": "https://modelscope.cn/models/user/repo"}
|
||||||
|
normalize_metadata_source(metadata)
|
||||||
|
assert metadata["source_platform"] == "modelscope"
|
||||||
|
|
||||||
|
|
||||||
|
class TestResolveSourceRef:
|
||||||
|
def test_resolves_from_canonical_fields(self):
|
||||||
|
ref = resolve_source_ref(
|
||||||
|
{"source_platform": "modelscope", "source_url": "https://modelscope.cn/models/u/r"}
|
||||||
|
)
|
||||||
|
assert ref is not None
|
||||||
|
assert ref.platform == "modelscope"
|
||||||
|
assert ref.source_id == "u/r"
|
||||||
|
|
||||||
|
def test_resolves_from_legacy_hf_url(self):
|
||||||
|
ref = resolve_source_ref({"hf_url": "https://huggingface.co/u/r"})
|
||||||
|
assert ref is not None
|
||||||
|
assert ref.platform == "huggingface"
|
||||||
|
|
||||||
|
def test_returns_none_without_any_source(self):
|
||||||
|
assert resolve_source_ref({}) is None
|
||||||
|
assert resolve_source_ref({"hf_url": ""}) is None
|
||||||
|
|
||||||
|
|
||||||
|
class TestHelpers:
|
||||||
|
def test_has_external_source_accepts_both_field_shapes(self):
|
||||||
|
assert has_external_source({"hf_url": "https://huggingface.co/u/r"}) is True
|
||||||
|
assert has_external_source({"source_url": "https://modelscope.cn/models/u/r"}) is True
|
||||||
|
assert has_external_source({"source_url": ""}) is False
|
||||||
|
assert has_external_source({}) is False
|
||||||
|
|
||||||
|
def test_get_source_platform_infers_from_url(self):
|
||||||
|
assert get_source_platform({"hf_url": "https://huggingface.co/u/r"}) == "huggingface"
|
||||||
|
assert get_source_platform({"source_platform": "tensorart"}) == "tensorart"
|
||||||
|
assert get_source_platform({}) == ""
|
||||||
|
|
||||||
|
def test_group_keys_match_legacy_hf_shape(self):
|
||||||
|
assert source_group_key({"hf_url": "https://huggingface.co/u/r"}) == "hf:u/r"
|
||||||
|
assert (
|
||||||
|
source_group_key({"source_url": "https://modelscope.cn/models/u/r"}) == "ms:u/r"
|
||||||
|
)
|
||||||
|
assert (
|
||||||
|
source_group_key({"source_url": "https://tensor.art/models/123"}) == "ta:123"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_group_key_is_none_without_source(self):
|
||||||
|
assert source_group_key({}) is None
|
||||||
|
assert source_group_key({"hf_url": "https://example.com/x"}) is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Model card fetching
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class TestFetchModelCard:
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_huggingface_tries_main_then_master(self, monkeypatch):
|
||||||
|
calls: list[str] = []
|
||||||
|
|
||||||
|
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||||
|
calls.append(url)
|
||||||
|
if url.endswith("/master/README.md"):
|
||||||
|
return "# card"
|
||||||
|
return ""
|
||||||
|
|
||||||
|
monkeypatch.setattr("py.services.model_sources.huggingface.fetch_text", fake_fetch_text)
|
||||||
|
|
||||||
|
card = await HuggingFaceSource().fetch_model_card("user/repo")
|
||||||
|
|
||||||
|
assert card == "# card"
|
||||||
|
assert calls == [
|
||||||
|
"https://huggingface.co/user/repo/raw/main/README.md",
|
||||||
|
"https://huggingface.co/user/repo/raw/master/README.md",
|
||||||
|
]
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_modelscope_prefers_resolve_url(self, monkeypatch):
|
||||||
|
calls: list[str] = []
|
||||||
|
|
||||||
|
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||||
|
calls.append(url)
|
||||||
|
return "---\nbase_model: krea/Krea-2-Turbo\n---\n# krea"
|
||||||
|
|
||||||
|
monkeypatch.setattr("py.services.model_sources.modelscope.fetch_text", fake_fetch_text)
|
||||||
|
|
||||||
|
card = await ModelScopeSource().fetch_model_card("u/r")
|
||||||
|
|
||||||
|
assert card.startswith("---")
|
||||||
|
assert calls == ["https://modelscope.cn/models/u/r/resolve/master/README.md"]
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_modelscope_falls_back_to_repo_api(self, monkeypatch):
|
||||||
|
calls: list[str] = []
|
||||||
|
|
||||||
|
async def fake_fetch_text(url: str, **_kwargs) -> str:
|
||||||
|
calls.append(url)
|
||||||
|
if "/api/v1/models/" in url:
|
||||||
|
return "# from api"
|
||||||
|
return ""
|
||||||
|
|
||||||
|
monkeypatch.setattr("py.services.model_sources.modelscope.fetch_text", fake_fetch_text)
|
||||||
|
|
||||||
|
card = await ModelScopeSource().fetch_model_card("u/r")
|
||||||
|
|
||||||
|
assert card == "# from api"
|
||||||
|
assert "resolve/master/README.md" in calls[0]
|
||||||
|
assert (
|
||||||
|
"https://modelscope.cn/api/v1/models/u/r/repo?Revision=master&FilePath=README.md"
|
||||||
|
in calls
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_tensorart_never_fetches(self):
|
||||||
|
# TensorArt enrichment is disabled: the provider must not issue any
|
||||||
|
# HTTP request, so it deliberately does not import `fetch_text`.
|
||||||
|
import importlib
|
||||||
|
|
||||||
|
module = importlib.import_module("py.services.model_sources.tensorart")
|
||||||
|
assert not hasattr(module, "fetch_text")
|
||||||
|
assert await TensorArtSource().fetch_model_card("123") == ""
|
||||||
|
|
||||||
|
|
||||||
|
class TestAssetBaseUrl:
|
||||||
|
def test_huggingface_uses_main_revision(self):
|
||||||
|
assert (
|
||||||
|
HuggingFaceSource().asset_base_url("u/r")
|
||||||
|
== "https://huggingface.co/u/r/resolve/main"
|
||||||
|
)
|
||||||
|
|
||||||
|
def test_modelscope_uses_master_revision(self):
|
||||||
|
assert (
|
||||||
|
ModelScopeSource().asset_base_url("u/r")
|
||||||
|
== "https://modelscope.cn/models/u/r/resolve/master"
|
||||||
|
)
|
||||||
@@ -292,6 +292,61 @@ Content
|
|||||||
)
|
)
|
||||||
assert images[0]["meta"]["prompt"] == "a cat"
|
assert images[0]["meta"]["prompt"] == "a cat"
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_gallery_images_use_modelscope_asset_base_url(self, processor):
|
||||||
|
"""A ModelScope-linked model resolves relative images against ModelScope."""
|
||||||
|
readme = """---
|
||||||
|
widget:
|
||||||
|
- text: "a cat"
|
||||||
|
output:
|
||||||
|
url: images/cat.png
|
||||||
|
---
|
||||||
|
Content
|
||||||
|
"""
|
||||||
|
with (
|
||||||
|
mock.patch("py.metadata_ops.apply_metadata_updates") as mock_apply,
|
||||||
|
mock.patch("py.metadata_ops.download_preview", return_value=None),
|
||||||
|
mock.patch("py.metadata_ops.refresh_cache"),
|
||||||
|
):
|
||||||
|
await processor.process(
|
||||||
|
skill_name="enrich_hf_metadata",
|
||||||
|
model_path="/p.safetensors",
|
||||||
|
llm_output=self.MIN_LLM_OUTPUT,
|
||||||
|
metadata={
|
||||||
|
"from_civitai": False,
|
||||||
|
"source_platform": "modelscope",
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo",
|
||||||
|
},
|
||||||
|
readme_content=readme,
|
||||||
|
)
|
||||||
|
applied = mock_apply.call_args[0][1]
|
||||||
|
images = applied.get("civitai", {}).get("images", [])
|
||||||
|
assert len(images) == 1
|
||||||
|
assert images[0]["url"] == (
|
||||||
|
"https://modelscope.cn/models/user/repo/resolve/master/images/cat.png"
|
||||||
|
)
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_base_model_overwrites_existing_modelscope_model(self, processor):
|
||||||
|
"""ModelScope is an external source, so the LLM may overwrite base_model."""
|
||||||
|
llm = {**self.MIN_LLM_OUTPUT, "base_model": "Flux.1 D"}
|
||||||
|
with (
|
||||||
|
mock.patch("py.metadata_ops.apply_metadata_updates") as mock_apply,
|
||||||
|
mock.patch("py.metadata_ops.download_preview", return_value=False),
|
||||||
|
mock.patch("py.metadata_ops.refresh_cache"),
|
||||||
|
):
|
||||||
|
await processor.process(
|
||||||
|
skill_name="enrich_hf_metadata",
|
||||||
|
model_path="/p.safetensors",
|
||||||
|
llm_output=llm,
|
||||||
|
metadata={
|
||||||
|
"base_model": "SD 1.5",
|
||||||
|
"source_platform": "modelscope",
|
||||||
|
"source_url": "https://modelscope.cn/models/user/repo",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
assert mock_apply.call_args[0][1]["base_model"] == "Flux.1 D"
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_gallery_images_skipped_without_hf_url(self, processor):
|
async def test_gallery_images_skipped_without_hf_url(self, processor):
|
||||||
"""Gallery images NOT extracted when the model has no HF source."""
|
"""Gallery images NOT extracted when the model has no HF source."""
|
||||||
|
|||||||
Reference in New Issue
Block a user