Compare commits

..

14 Commits

Author SHA1 Message Date
Will Miao
3c83e78d9f feat(ui): auto-newline after pasting URL in download and batch-import textareas
Extract auto-newline-on-paste logic into shared setupAutoNewlineOnPaste() utility in uiHelpers.js.
Apply it to both the Download modal (modelUrl) and Batch Import modal (batchUrlInput)
textarea, so users can paste multiple URLs in succession without manually pressing Enter.
2026-07-02 10:53:33 +08:00
Will Miao
d7291f73c9 fix(download): recognize civitai.red and civitai.green URLs in batch download (#1003) 2026-07-02 10:28:03 +08:00
Will Miao
fe90f7f9b1 feat(ui): add searchable base model dropdown with filename inference in model modal
Replace native <select> with a searchable dropdown that:
- Filters options as the user types
- Shows filename-inferred suggestions at the top in a "Suggested" section
- Supports keyboard navigation (ArrowUp/Down/Enter/Escape)
- Allows typing custom values not in the list
- Removes dead .base-model-selector CSS

Adds 3 new i18n keys (baseModelSearchPlaceholder, baseModelSuggested,
baseModelNoMatch) with translations for all 9 locales.
2026-07-01 14:31:08 +08:00
Will Miao
8b344ea39f feat(ui): add View on Hugging Face button, plumb hf_url through full cache pipeline 2026-07-01 08:38:16 +08:00
Will Miao
8348a0cef8 fix(download): harden HF download path validation, fix WebSocket leak, add URL detection tests (#965, #977)
Security hardening:
- Validate repo format with strict regex (reject .. traversal)
- Validate filename rejects path separators and ..
- Validate relative_path rejects absolute paths and ..
- Verify model_root is within configured scanner roots using
  realpath + os.sep guard to prevent prefix-match bypass
- Add realpath-based escape detection for final dest_path

Bug fixes:
- Fix WebSocket leak in _downloadHfSingle: wrap ws.close() in
  try/finally so it closes even if downloadHfModel() throws
- Same fix for batch HF download per-file WebSocket loop

Frontend hardening:
- Tighten HF repo regex: require huggingface.co for full URLs,
  reject bare .. patterns
- Add 12 unit tests for detectUrlType() covering HF resolve,
  HF repo, CivitAI, CivArchive, direct HTTP, edge cases
2026-07-01 05:51:58 +08:00
Will Miao
7cf785b72f fix(ui): unify HF file selection UI, remove cloud icon, add select-all, cleanup dead code (#965, #977)
- Unify single-URL and multi-URL HF repo flows to use the same batch
  preview interface (remove separate repoFileStep)
- Remove unnecessary cloud icon from HF batch preview items
- Use formatFileSize() instead of hardcoded MB text
- Change default selection to unchecked (no preselected files)
- Add select all / deselect all checkbox with dynamic Next button
- Clean up dead CSS, HTML template, and JS methods from removed
  repoFileStep
- Add selectAll i18n key with translations for all 10 locales
- Fix batch progress bar name fallback for HF items
2026-06-30 23:28:35 +08:00
Will Miao
e8913f4481 feat(ui): dynamically populate base model dropdown from CivitAI API, add Krea 2 constants (#1001) 2026-06-30 22:41:17 +08:00
Will Miao
f9c3d8dc97 fix(metadata): demote CivArchive hash lookup failure from ERROR to DEBUG
A model not being found on CivArchive by hash is a routine case (the
model simply isn't published there), not an error. The callers already
log the outcome at WARNING (bulk_metadata_refresh) or DEBUG
(metadata_sync_service) with full context, making this ERROR-level log
both misleading and redundant.
2026-06-30 19:42:30 +08:00
Will Miao
09ca91fc0e feat(download): add Hugging Face model download to standalone UI wizard (#965, #977)
Integrate HF model downloading into the existing CivitAI-style wizard flow:
- URL type detection (civitai / hf-resolve / hf-repo / direct-http)
- Repo file explorer with checkbox-based file selection
- Batch/queue download with per-file WebSocket progress
- Aria2 backend support (respects download_backend setting)
- Scanner cache integration via create_default_metadata + add_model_to_cache
- i18n updates for all 10 locales
2026-06-30 19:36:12 +08:00
Will Miao
16f5222efd fix(cache): prevent corrupted cache rows from breaking model listings (#730)
Cache corruption (NULL model_name/file_name from legacy DB rows or partial
writes) caused format_response to raise KeyError/AttributeError, failing the
entire /loras/list request and showing no models in the UI.

Fix across three layers:
- format_response (lora/checkpoint/embedding): replace direct dict[] access
  with .get() fallbacks; return None for entries missing file_path
- handlers: filter None entries from list/excluded/fetch/duplicate/conflict
  endpoints instead of letting them crash or appear as null in responses
- model_scanner: always use validate_batch repaired copies (previously
  discarded when no invalid entries, leaving None values in raw_data)
- persistent_model_cache: add or-empty-string guards on read and write for
  nullable TEXT columns (model_name, file_name, folder, base_model, etc.)
2026-06-30 09:02:42 +08:00
Will Miao
28e7c04b37 fix(settings): migrate all settings subdirectories on portable mode switch 2026-06-29 21:40:37 +08:00
Will Miao
28f99c46d3 fix(update): preserve user data dirs during Git-based update via git clean -e excludes
git clean -fd in _perform_git_update deleted untracked, non-ignored
directories (wildcards, stats, backups, civitai, caches, logs) during
portable-mode updates, since released tags do not list them in .gitignore.
Add -e excludes for all user-managed paths to both nightly and stable
update branches. Add regression tests for both paths.
2026-06-29 21:10:38 +08:00
Will Miao
205194f4e6 chore: add stats, wildcards, backups, and logs dirs to .gitignore 2026-06-29 19:46:04 +08:00
willmiao
402d8b07cf docs: auto-update supporters list in README 2026-06-28 14:17:19 +00:00
45 changed files with 2378 additions and 250 deletions

4
.gitignore vendored
View File

@@ -7,6 +7,10 @@ py/run_test.py
.vscode/
cache/
civitai/
stats/
wildcards/
backups/
logs/
node_modules/
coverage/
.coverage

File diff suppressed because one or more lines are too long

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "Aus Favoriten entfernen",
"viewOnCivitai": "Auf Civitai anzeigen",
"notAvailableFromCivitai": "Nicht auf Civitai verfügbar",
"viewOnHuggingFace": "Auf Hugging Face ansehen",
"sendToWorkflow": "An ComfyUI senden (Klick: Anhängen, Shift+Klick: Ersetzen)",
"copyLoRASyntax": "LoRA-Syntax kopieren",
"checkpointNameCopied": "Checkpoint-Name kopiert",
@@ -1134,7 +1135,10 @@
"titleWithType": "{type} von URL herunterladen",
"civitaiUrl": "Civitai URL:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Geben Sie eine CivitAI- oder CivArchive-URL pro Zeile ein. Unterstützt mehrere URLs für den Batch-Download.",
"urlHint": "Geben Sie eine CivitAI-, CivArchive- oder Hugging Face-URL pro Zeile ein. Unterstützt mehrere URLs für den Batch-Download.",
"selectHfFiles": "Datei(en) zum Herunterladen aus diesem Repository auswählen:",
"selectAll": "Alle auswählen",
"fetchingRepoFiles": "Repository-Dateien werden abgerufen...",
"locationPreview": "Download-Speicherort Vorschau",
"useDefaultPath": "Standardpfad verwenden",
"useDefaultPathTooltip": "Wenn aktiviert, werden Dateien automatisch mit konfigurierten Pfadvorlagen organisiert",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Ungültiges Civitai URL-Format",
"noVersions": "Keine Versionen für dieses Modell verfügbar"
"noVersions": "Keine Versionen für dieses Modell verfügbar",
"mixedSources": "CivitAI- und Hugging Face-URLs können nicht in derselben Charge gemischt werden.",
"noModelFiles": "In diesem Repository wurden keine Modelldateien gefunden."
},
"status": {
"preparing": "Download wird vorbereitet...",
@@ -1314,6 +1320,8 @@
"editVersionName": "Versionsname bearbeiten",
"viewOnCivitai": "Auf Civitai anzeigen",
"viewOnCivitaiText": "Auf Civitai anzeigen",
"viewOnHuggingFace": "Auf Hugging Face ansehen",
"viewOnHuggingFaceText": "Auf Hugging Face ansehen",
"viewCreatorProfile": "Ersteller-Profil anzeigen",
"openFileLocation": "Dateispeicherort öffnen",
"sendToWorkflow": "An ComfyUI senden",
@@ -1339,7 +1347,10 @@
"additionalNotes": "Zusätzliche Notizen",
"notesHint": "Enter zum Speichern, Shift+Enter für neue Zeile",
"addNotesPlaceholder": "Fügen Sie hier Ihre Notizen hinzu...",
"aboutThisVersion": "Über diese Version"
"aboutThisVersion": "Über diese Version",
"baseModelSearchPlaceholder": "Basismodell suchen…",
"baseModelSuggested": "Vorschlag",
"baseModelNoMatch": "Keine passenden Basismodelle"
},
"notes": {
"saved": "Notizen erfolgreich gespeichert",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "Remove from favorites",
"viewOnCivitai": "View on Civitai",
"notAvailableFromCivitai": "Not available from Civitai",
"viewOnHuggingFace": "View on Hugging Face",
"sendToWorkflow": "Send to ComfyUI (Click: Append, Shift+Click: Replace)",
"copyLoRASyntax": "Copy LoRA Syntax",
"checkpointNameCopied": "Checkpoint name copied",
@@ -1134,7 +1135,10 @@
"titleWithType": "Download {type} from URL",
"civitaiUrl": "Civitai URL(s):",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Enter one CivitAI or CivArchive URL per line. Supports multiple URLs for batch download.",
"urlHint": "Enter one CivitAI, CivArchive, or Hugging Face URL per line. Supports multiple URLs for batch download.",
"selectHfFiles": "Select file(s) to download from this repository:",
"selectAll": "Select All",
"fetchingRepoFiles": "Fetching repository files...",
"locationPreview": "Download Location Preview",
"useDefaultPath": "Use Default Path",
"useDefaultPathTooltip": "When enabled, files are automatically organized using configured path templates",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Invalid Civitai URL format",
"noVersions": "No versions available for this model"
"noVersions": "No versions available for this model",
"mixedSources": "Cannot mix CivitAI and Hugging Face URLs in the same batch.",
"noModelFiles": "No model files found in this repository."
},
"status": {
"preparing": "Preparing download...",
@@ -1314,6 +1320,8 @@
"editVersionName": "Edit version name",
"viewOnCivitai": "View on Civitai",
"viewOnCivitaiText": "View on Civitai",
"viewOnHuggingFace": "View on Hugging Face",
"viewOnHuggingFaceText": "View on Hugging Face",
"viewCreatorProfile": "View Creator Profile",
"openFileLocation": "Open File Location",
"sendToWorkflow": "Send to ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "Additional Notes",
"notesHint": "Press Enter to save, Shift+Enter for new line",
"addNotesPlaceholder": "Add your notes here...",
"aboutThisVersion": "About this version"
"aboutThisVersion": "About this version",
"baseModelSearchPlaceholder": "Search base model…",
"baseModelSuggested": "Suggested",
"baseModelNoMatch": "No matching base models"
},
"notes": {
"saved": "Notes saved successfully",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "Eliminar de favoritos",
"viewOnCivitai": "Ver en Civitai",
"notAvailableFromCivitai": "No disponible en Civitai",
"viewOnHuggingFace": "Ver en Hugging Face",
"sendToWorkflow": "Enviar a ComfyUI (Clic: Añadir, Shift+Clic: Reemplazar)",
"copyLoRASyntax": "Copiar sintaxis de LoRA",
"checkpointNameCopied": "Nombre del checkpoint copiado",
@@ -1134,7 +1135,10 @@
"titleWithType": "Descargar {type} desde URL",
"civitaiUrl": "URL de Civitai:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Ingrese una URL de CivitAI o CivArchive por línea. Admite múltiples URLs para descarga por lotes.",
"urlHint": "Ingrese una URL de CivitAI, CivArchive o Hugging Face por línea. Admite múltiples URLs para descarga por lotes.",
"selectHfFiles": "Seleccione el/los archivo(s) para descargar de este repositorio:",
"selectAll": "Seleccionar todo",
"fetchingRepoFiles": "Obteniendo archivos del repositorio...",
"locationPreview": "Vista previa de ubicación de descarga",
"useDefaultPath": "Usar ruta predeterminada",
"useDefaultPathTooltip": "Cuando está habilitado, los archivos se organizan automáticamente usando plantillas de rutas configuradas",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Formato de URL de Civitai inválido",
"noVersions": "No hay versiones disponibles para este modelo"
"noVersions": "No hay versiones disponibles para este modelo",
"mixedSources": "No se pueden mezclar URL de CivitAI y Hugging Face en el mismo lote.",
"noModelFiles": "No se encontraron archivos de modelo en este repositorio."
},
"status": {
"preparing": "Preparando descarga...",
@@ -1314,6 +1320,8 @@
"editVersionName": "Editar nombre de versión",
"viewOnCivitai": "Ver en Civitai",
"viewOnCivitaiText": "Ver en Civitai",
"viewOnHuggingFace": "Ver en Hugging Face",
"viewOnHuggingFaceText": "Ver en Hugging Face",
"viewCreatorProfile": "Ver perfil del creador",
"openFileLocation": "Abrir ubicación del archivo",
"sendToWorkflow": "Enviar a ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "Notas adicionales",
"notesHint": "Presiona Enter para guardar, Shift+Enter para nueva línea",
"addNotesPlaceholder": "Añade tus notas aquí...",
"aboutThisVersion": "Acerca de esta versión"
"aboutThisVersion": "Acerca de esta versión",
"baseModelSearchPlaceholder": "Buscar modelo base…",
"baseModelSuggested": "Sugerido",
"baseModelNoMatch": "No hay modelos base que coincidan"
},
"notes": {
"saved": "Notas guardadas exitosamente",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "Retirer des favoris",
"viewOnCivitai": "Voir sur Civitai",
"notAvailableFromCivitai": "Non disponible sur Civitai",
"viewOnHuggingFace": "Voir sur Hugging Face",
"sendToWorkflow": "Envoyer vers ComfyUI (Clic: Ajouter, Maj+Clic: Remplacer)",
"copyLoRASyntax": "Copier la syntaxe LoRA",
"checkpointNameCopied": "Nom du checkpoint copié",
@@ -1134,7 +1135,10 @@
"titleWithType": "Télécharger {type} depuis une URL",
"civitaiUrl": "URL Civitai :",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Entrez une URL CivitAI ou CivArchive par ligne. Prend en charge plusieurs URLs pour le téléchargement par lot.",
"urlHint": "Entrez une URL CivitAI, CivArchive ou Hugging Face par ligne. Prend en charge plusieurs URL pour le téléchargement par lot.",
"selectHfFiles": "Sélectionnez le(s) fichier(s) à télécharger depuis ce dépôt :",
"selectAll": "Tout sélectionner",
"fetchingRepoFiles": "Récupération des fichiers du dépôt...",
"locationPreview": "Aperçu de l'emplacement de téléchargement",
"useDefaultPath": "Utiliser le chemin par défaut",
"useDefaultPathTooltip": "Lorsque activé, les fichiers sont automatiquement organisés selon les modèles de chemin configurés",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Format d'URL Civitai invalide",
"noVersions": "Aucune version disponible pour ce modèle"
"noVersions": "Aucune version disponible pour ce modèle",
"mixedSources": "Impossible de mélanger les URL CivitAI et Hugging Face dans le même lot.",
"noModelFiles": "Aucun fichier de modèle trouvé dans ce dépôt."
},
"status": {
"preparing": "Préparation du téléchargement...",
@@ -1314,6 +1320,8 @@
"editVersionName": "Modifier le nom de la version",
"viewOnCivitai": "Voir sur Civitai",
"viewOnCivitaiText": "Voir sur Civitai",
"viewOnHuggingFace": "Voir sur Hugging Face",
"viewOnHuggingFaceText": "Voir sur Hugging Face",
"viewCreatorProfile": "Voir le profil du créateur",
"openFileLocation": "Ouvrir l'emplacement du fichier",
"sendToWorkflow": "Envoyer vers ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "Notes supplémentaires",
"notesHint": "Appuyez sur Entrée pour sauvegarder, Maj+Entrée pour nouvelle ligne",
"addNotesPlaceholder": "Ajoutez vos notes ici...",
"aboutThisVersion": "À propos de cette version"
"aboutThisVersion": "À propos de cette version",
"baseModelSearchPlaceholder": "Rechercher un modèle de base…",
"baseModelSuggested": "Suggéré",
"baseModelNoMatch": "Aucun modèle de base correspondant"
},
"notes": {
"saved": "Notes sauvegardées avec succès",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "הסר מהמועדפים",
"viewOnCivitai": "הצג ב-Civitai",
"notAvailableFromCivitai": "לא זמין מ-Civitai",
"viewOnHuggingFace": "צפייה ב-Hugging Face",
"sendToWorkflow": "שלח ל-ComfyUI (לחיצה: הוסף, Shift+לחיצה: החלף)",
"copyLoRASyntax": "העתק תחביר LoRA",
"checkpointNameCopied": "שם Checkpoint הועתק",
@@ -1134,7 +1135,10 @@
"titleWithType": "הורד {type} מכתובת URL",
"civitaiUrl": "כתובת URL של Civitai:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "יש להזין כתובת URL אחת של CivitAI או CivArchive בכל שורה. תומך במספר כתובות URL להורדה בבת אחת.",
"urlHint": "יש להזין כתובת URL אחת של CivitAI, CivArchive או Hugging Face בכל שורה. תומך במספר כתובות URL להורדה בקבוצה.",
"selectHfFiles": "בחר קבצים להורדה ממאגר זה:",
"selectAll": "בחר הכל",
"fetchingRepoFiles": "מביא קבצים מהמאגר...",
"locationPreview": "תצוגה מקדימה של מיקום ההורדה",
"useDefaultPath": "השתמש בנתיב ברירת מחדל",
"useDefaultPathTooltip": "כאשר מופעל, קבצים מאורגנים אוטומטית באמצעות תבניות נתיב מוגדרות",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "פורמט URL של Civitai לא חוקי",
"noVersions": "אין גרסאות זמינות למודל זה"
"noVersions": "אין גרסאות זמינות למודל זה",
"mixedSources": "לא ניתן לערבב כתובות URL של CivitAI ו-Hugging Face באותה קבוצה.",
"noModelFiles": "לא נמצאו קבצי מודל במאגר זה."
},
"status": {
"preparing": "מכין הורדה...",
@@ -1314,6 +1320,8 @@
"editVersionName": "ערוך שם גרסה",
"viewOnCivitai": "הצג ב-Civitai",
"viewOnCivitaiText": "הצג ב-Civitai",
"viewOnHuggingFace": "צפייה ב-Hugging Face",
"viewOnHuggingFaceText": "צפייה ב-Hugging Face",
"viewCreatorProfile": "הצג פרופיל יוצר",
"openFileLocation": "פתח מיקום קובץ",
"sendToWorkflow": "שלח ל-ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "הערות נוספות",
"notesHint": "לחץ Enter לשמירה, Shift+Enter לשורה חדשה",
"addNotesPlaceholder": "הוסף את ההערות שלך כאן...",
"aboutThisVersion": "אודות גרסה זו"
"aboutThisVersion": "אודות גרסה זו",
"baseModelSearchPlaceholder": "חפש מודל בסיס…",
"baseModelSuggested": "מוצע",
"baseModelNoMatch": "אין מודלי בסיס תואמים"
},
"notes": {
"saved": "הערות נשמרו בהצלחה",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "お気に入りから削除",
"viewOnCivitai": "Civitaiで表示",
"notAvailableFromCivitai": "Civitaiでは利用できません",
"viewOnHuggingFace": "Hugging Face で見る",
"sendToWorkflow": "ComfyUIに送信クリック追加、Shift+クリック:置換)",
"copyLoRASyntax": "LoRA構文をコピー",
"checkpointNameCopied": "checkpointの名前をコピーしました",
@@ -1134,7 +1135,10 @@
"titleWithType": "URLから{type}をダウンロード",
"civitaiUrl": "Civitai URL",
"placeholder": "https://civitai.com/models/...",
"urlHint": "1行に1つのCivitAIまたはCivArchive URLを入力してください。複数のURLを一括ダウンロードできます。",
"urlHint": "1行に1つのCivitAICivArchive、またはHugging Face URLを入力してください。複数のURLを一括ダウンロードできます。",
"selectHfFiles": "このリポジトリからダウンロードするファイルを選択してください:",
"selectAll": "すべて選択",
"fetchingRepoFiles": "リポジトリのファイルを取得中...",
"locationPreview": "ダウンロード場所プレビュー",
"useDefaultPath": "デフォルトパスを使用",
"useDefaultPathTooltip": "有効にすると、設定されたパステンプレートを使用してファイルが自動的に整理されます",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "無効なCivitai URL形式",
"noVersions": "このモデルの利用可能なバージョンがありません"
"noVersions": "このモデルの利用可能なバージョンがありません",
"mixedSources": "同じバッチ内でCivitAIとHugging FaceのURLを混在させることはできません。",
"noModelFiles": "このリポジトリにモデルファイルが見つかりませんでした。"
},
"status": {
"preparing": "ダウンロードを準備中...",
@@ -1314,6 +1320,8 @@
"editVersionName": "バージョン名を編集",
"viewOnCivitai": "Civitaiで表示",
"viewOnCivitaiText": "Civitaiで表示",
"viewOnHuggingFace": "Hugging Face で見る",
"viewOnHuggingFaceText": "Hugging Face で見る",
"viewCreatorProfile": "作成者プロフィールを表示",
"openFileLocation": "ファイルの場所を開く",
"sendToWorkflow": "ComfyUI に送信",
@@ -1339,7 +1347,10 @@
"additionalNotes": "追加メモ",
"notesHint": "Enterで保存、Shift+Enterで改行",
"addNotesPlaceholder": "メモをここに追加...",
"aboutThisVersion": "このバージョンについて"
"aboutThisVersion": "このバージョンについて",
"baseModelSearchPlaceholder": "ベースモデルを検索…",
"baseModelSuggested": "おすすめ",
"baseModelNoMatch": "該当するベースモデルがありません"
},
"notes": {
"saved": "メモが正常に保存されました",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "즐겨찾기에서 제거",
"viewOnCivitai": "Civitai에서 보기",
"notAvailableFromCivitai": "Civitai에서 사용할 수 없음",
"viewOnHuggingFace": "Hugging Face에서 보기",
"sendToWorkflow": "ComfyUI로 전송 (클릭: 추가, Shift+클릭: 교체)",
"copyLoRASyntax": "LoRA 문법 복사",
"checkpointNameCopied": "Checkpoint 이름 복사됨",
@@ -1134,7 +1135,10 @@
"titleWithType": "URL에서 {type} 다운로드",
"civitaiUrl": "Civitai URL:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "한 줄에 하나의 CivitAI 또는 CivArchive URL을 입력하세요. 여러 URL을 일괄 다운로드할 수 있습니다.",
"urlHint": "한 줄에 하나의 CivitAI, CivArchive 또는 Hugging Face URL을 입력하세요. 여러 URL을 일괄 다운로드할 수 있습니다.",
"selectHfFiles": "이 저장소에서 다운로드할 파일을 선택하세요:",
"selectAll": "모두 선택",
"fetchingRepoFiles": "저장소 파일을 가져오는 중...",
"locationPreview": "다운로드 위치 미리보기",
"useDefaultPath": "기본 경로 사용",
"useDefaultPathTooltip": "활성화하면 구성된 경로 템플릿을 사용하여 파일이 자동으로 정리됩니다",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "잘못된 Civitai URL 형식",
"noVersions": "이 모델에 사용 가능한 버전이 없습니다"
"noVersions": "이 모델에 사용 가능한 버전이 없습니다",
"mixedSources": "동일한 배치에서 CivitAI와 Hugging Face URL을 혼합할 수 없습니다.",
"noModelFiles": "이 저장소에서 모델 파일을 찾을 수 없습니다."
},
"status": {
"preparing": "다운로드 준비 중...",
@@ -1314,6 +1320,8 @@
"editVersionName": "버전명 편집",
"viewOnCivitai": "Civitai에서 보기",
"viewOnCivitaiText": "Civitai에서 보기",
"viewOnHuggingFace": "Hugging Face에서 보기",
"viewOnHuggingFaceText": "Hugging Face에서 보기",
"viewCreatorProfile": "제작자 프로필 보기",
"openFileLocation": "파일 위치 열기",
"sendToWorkflow": "ComfyUI로 보내기",
@@ -1339,7 +1347,10 @@
"additionalNotes": "추가 메모",
"notesHint": "Enter로 저장, Shift+Enter로 줄바꿈",
"addNotesPlaceholder": "메모를 여기에 추가하세요...",
"aboutThisVersion": "이 버전에 대해"
"aboutThisVersion": "이 버전에 대해",
"baseModelSearchPlaceholder": "베이스 모델 검색…",
"baseModelSuggested": "추천",
"baseModelNoMatch": "일치하는 베이스 모델 없음"
},
"notes": {
"saved": "메모가 성공적으로 저장됨",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "Удалить из избранного",
"viewOnCivitai": "Посмотреть на Civitai",
"notAvailableFromCivitai": "Недоступно на Civitai",
"viewOnHuggingFace": "Открыть Hugging Face",
"sendToWorkflow": "Отправить в ComfyUI (Клик: Добавить, Shift+Клик: Заменить)",
"copyLoRASyntax": "Копировать синтаксис LoRA",
"checkpointNameCopied": "Имя checkpoint скопировано",
@@ -1134,7 +1135,10 @@
"titleWithType": "Скачать {type} по URL",
"civitaiUrl": "Civitai URL:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "Введите один URL CivitAI или CivArchive в каждой строке. Поддерживается пакетная загрузка нескольких URL.",
"urlHint": "Введите один URL CivitAI, CivArchive или Hugging Face в каждой строке. Поддерживает несколько URL для пакетной загрузки.",
"selectHfFiles": "Выберите файл(ы) для загрузки из этого репозитория:",
"selectAll": "Выбрать все",
"fetchingRepoFiles": "Получение файлов репозитория...",
"locationPreview": "Предпросмотр места загрузки",
"useDefaultPath": "Использовать путь по умолчанию",
"useDefaultPathTooltip": "При включении файлы автоматически организуются с использованием настроенных шаблонов путей",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Неверный формат URL Civitai",
"noVersions": "Нет доступных версий для этой модели"
"noVersions": "Нет доступных версий для этой модели",
"mixedSources": "Нельзя смешивать URL-адреса CivitAI и Hugging Face в одном пакете.",
"noModelFiles": "В этом репозитории не найдено файлов моделей."
},
"status": {
"preparing": "Подготовка загрузки...",
@@ -1314,6 +1320,8 @@
"editVersionName": "Редактировать название версии",
"viewOnCivitai": "Посмотреть на Civitai",
"viewOnCivitaiText": "Посмотреть на Civitai",
"viewOnHuggingFace": "Открыть Hugging Face",
"viewOnHuggingFaceText": "Открыть Hugging Face",
"viewCreatorProfile": "Посмотреть профиль создателя",
"openFileLocation": "Открыть расположение файла",
"sendToWorkflow": "Отправить в ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "Дополнительные заметки",
"notesHint": "Нажмите Enter для сохранения, Shift+Enter для новой строки",
"addNotesPlaceholder": "Добавьте ваши заметки здесь...",
"aboutThisVersion": "Об этой версии"
"aboutThisVersion": "Об этой версии",
"baseModelSearchPlaceholder": "Поиск базовой модели…",
"baseModelSuggested": "Предполагаемые",
"baseModelNoMatch": "Нет подходящих базовых моделей"
},
"notes": {
"saved": "Заметки успешно сохранены",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "从收藏移除",
"viewOnCivitai": "在 Civitai 查看",
"notAvailableFromCivitai": "Civitai 上不可用",
"viewOnHuggingFace": "在 Hugging Face 查看",
"sendToWorkflow": "发送到 ComfyUI点击追加Shift+点击:替换)",
"copyLoRASyntax": "复制 LoRA 语法",
"checkpointNameCopied": "检查点名称已复制",
@@ -1134,7 +1135,10 @@
"titleWithType": "从 URL 下载 {type}",
"civitaiUrl": "Civitai URL:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "每行输入一个 CivitAICivArchive URL。支持批量下载多个 URL。",
"urlHint": "每行输入一个 CivitAICivArchive 或 Hugging Face URL。支持批量下载多个 URL。",
"selectHfFiles": "选择从此仓库下载的文件:",
"selectAll": "全选",
"fetchingRepoFiles": "正在获取仓库文件...",
"locationPreview": "下载位置预览",
"useDefaultPath": "使用默认路径",
"useDefaultPathTooltip": "启用后,文件将自动按配置的路径模板进行整理",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "无效的 Civitai URL 格式",
"noVersions": "此模型没有可用版本"
"noVersions": "此模型没有可用版本",
"mixedSources": "无法在同一批次中混合使用 CivitAI 和 Hugging Face URL。",
"noModelFiles": "在此仓库中未找到模型文件。"
},
"status": {
"preparing": "正在准备下载...",
@@ -1314,6 +1320,8 @@
"editVersionName": "编辑版本名称",
"viewOnCivitai": "在 Civitai 查看",
"viewOnCivitaiText": "在 Civitai 查看",
"viewOnHuggingFace": "在 Hugging Face 查看",
"viewOnHuggingFaceText": "在 Hugging Face 查看",
"viewCreatorProfile": "查看创作者主页",
"openFileLocation": "打开文件位置",
"sendToWorkflow": "发送到 ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "附加备注",
"notesHint": "回车保存Shift+回车换行",
"addNotesPlaceholder": "在此添加你的备注...",
"aboutThisVersion": "关于此版本"
"aboutThisVersion": "关于此版本",
"baseModelSearchPlaceholder": "搜索基础模型…",
"baseModelSuggested": "推荐",
"baseModelNoMatch": "没有匹配的基础模型"
},
"notes": {
"saved": "备注保存成功",

View File

@@ -105,6 +105,7 @@
"removeFromFavorites": "移除收藏",
"viewOnCivitai": "在 Civitai 查看",
"notAvailableFromCivitai": "Civitai 不提供",
"viewOnHuggingFace": "在 Hugging Face 查看",
"sendToWorkflow": "傳送到 ComfyUI點擊附加Shift+點擊:取代)",
"copyLoRASyntax": "複製 LoRA 語法",
"checkpointNameCopied": "Checkpoint 名稱已複製",
@@ -1134,7 +1135,10 @@
"titleWithType": "從網址下載 {type}",
"civitaiUrl": "Civitai 網址:",
"placeholder": "https://civitai.com/models/...",
"urlHint": "每行輸入一個 CivitAICivArchive URL。支援批量下載多個 URL。",
"urlHint": "每行輸入一個 CivitAICivArchive 或 Hugging Face URL。支援批量下載多個 URL。",
"selectHfFiles": "選擇從此倉庫下載的檔案:",
"selectAll": "全選",
"fetchingRepoFiles": "正在獲取倉庫檔案...",
"locationPreview": "下載位置預覽",
"useDefaultPath": "使用預設路徑",
"useDefaultPathTooltip": "啟用後,檔案將依照設定的路徑範本自動整理",
@@ -1163,7 +1167,9 @@
},
"errors": {
"invalidUrl": "Civitai 網址格式無效",
"noVersions": "此模型無可用版本"
"noVersions": "此模型無可用版本",
"mixedSources": "無法在同一批次中混合使用 CivitAI 和 Hugging Face URL。",
"noModelFiles": "在此倉庫中未找到模型檔案。"
},
"status": {
"preparing": "準備下載中...",
@@ -1314,6 +1320,8 @@
"editVersionName": "編輯版本名稱",
"viewOnCivitai": "在 Civitai 查看",
"viewOnCivitaiText": "在 Civitai 查看",
"viewOnHuggingFace": "在 Hugging Face 查看",
"viewOnHuggingFaceText": "在 Hugging Face 查看",
"viewCreatorProfile": "查看創作者個人檔案",
"openFileLocation": "開啟檔案位置",
"sendToWorkflow": "傳送到 ComfyUI",
@@ -1339,7 +1347,10 @@
"additionalNotes": "附加備註",
"notesHint": "按 Enter 儲存Shift+Enter 換行",
"addNotesPlaceholder": "在此新增備註...",
"aboutThisVersion": "關於此版本"
"aboutThisVersion": "關於此版本",
"baseModelSearchPlaceholder": "搜尋基礎模型…",
"baseModelSuggested": "推薦",
"baseModelNoMatch": "沒有符合的基礎模型"
},
"notes": {
"saved": "備註已儲存",

View File

@@ -0,0 +1,409 @@
"""Handlers for Hugging Face model listing and download.
Minimal MVP implementation — uses direct HTTP to the HF API for file
listing and the project's existing aiohttp-based Downloader for
downloading. No huggingface_hub dependency required.
"""
from __future__ import annotations
import json
import logging
import os
import re
from typing import Any
import aiohttp
from aiohttp import web
from ...config import config
from ...services.downloader import (
DownloadProgress,
get_downloader,
)
from ...services.aria2_downloader import Aria2Downloader
from ...services.settings_manager import get_settings_manager
from ...services.service_registry import ServiceRegistry
from ...services.websocket_manager import ws_manager
from ...utils.constants import MODEL_FILE_EXTENSIONS
from ...utils.metadata_manager import MetadataManager
from ...utils.models import LoraMetadata, CheckpointMetadata, EmbeddingMetadata
logger = logging.getLogger(__name__)
_DEFAULT_MODEL_CLASS = LoraMetadata
_DEFAULT_SCANNER_GETTER = "get_lora_scanner"
# Shared aiohttp session for HF API calls (created on first use)
_hf_api_session: aiohttp.ClientSession | None = None
async def _get_hf_api_session() -> aiohttp.ClientSession:
"""Get or create the shared aiohttp session for HF API calls."""
global _hf_api_session # needed because we reassign the module-level name
if _hf_api_session is None or _hf_api_session.closed:
_hf_api_session = aiohttp.ClientSession(
headers={"User-Agent": "ComfyUI-LoRA-Manager/1.0"},
timeout=aiohttp.ClientTimeout(total=30),
)
return _hf_api_session
def _infer_model_type(model_root: str) -> tuple[Any, str]:
"""Determine model class and scanner by matching ``model_root`` against the
configured root paths for each model type (from ``Config``).
The ``model_root`` value comes from the frontend's model-root dropdown,
which is populated from the current page's scanner roots. By checking
which scanner's root list it belongs to, we avoid fragile heuristics
like substring-matching path names.
"""
norm = os.path.normpath(model_root).replace(os.sep, "/")
# LoRA roots
for p in (config.loras_roots or []) + (config.extra_loras_roots or []):
if os.path.normpath(p).replace(os.sep, "/") == norm:
return LoraMetadata, "get_lora_scanner"
# Checkpoint / UNet roots
for p in (
(config.checkpoints_roots or [])
+ (config.extra_checkpoints_roots or [])
+ (config.unet_roots or [])
+ (config.extra_unet_roots or [])
):
if os.path.normpath(p).replace(os.sep, "/") == norm:
return CheckpointMetadata, "get_checkpoint_scanner"
# Embedding roots
for p in (config.embeddings_roots or []) + (config.extra_embeddings_roots or []):
if os.path.normpath(p).replace(os.sep, "/") == norm:
return EmbeddingMetadata, "get_embedding_scanner"
# Fallback — should not happen in normal use
logger.warning(
"Could not determine model type for root '%s'; defaulting to LoRA",
model_root,
)
return _DEFAULT_MODEL_CLASS, _DEFAULT_SCANNER_GETTER
async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
"""Create a proper .metadata.json and add the model to the scanner cache.
Uses ``MetadataManager.create_default_metadata()`` which computes the
SHA256 hash, extracts safetensors header metadata (base_model), and
produces a fully-populated ``LoraMetadata`` (or ``CheckpointMetadata`` /
``EmbeddingMetadata``) object. We then overlay HF-specific fields and
register the model in the in-memory scanner cache so it appears
immediately without a full filesystem walk.
"""
try:
hf_url = f"https://huggingface.co/{repo}"
model_class, scanner_getter_name = _infer_model_type(model_root)
# 1. Create proper metadata (computes SHA256, reads safetensors headers)
metadata = await MetadataManager.create_default_metadata(
dest_path, model_class=model_class
)
if metadata is None:
logger.warning("create_default_metadata returned None for %s", dest_path)
return
# 2. Overlay HF-specific fields
metadata._unknown_fields["hf_url"] = hf_url
metadata.from_civitai = False # HF models are not from CivitAI
# 3. Save metadata atomically
await MetadataManager.save_metadata(dest_path, metadata)
logger.info("Saved HF metadata (with hf_url) for %s", dest_path)
# 4. Determine relative folder path for cache
# model_root is an absolute path; dest_path is under it
folder = ""
if os.path.isabs(model_root) and dest_path.startswith(model_root):
rel = os.path.relpath(os.path.dirname(dest_path), model_root)
folder = rel.replace(os.sep, "/") if rel != "." else ""
# 5. Add to scanner cache (same as CivitAI's _execute_download does)
scanner_getter = getattr(ServiceRegistry, scanner_getter_name, None)
if scanner_getter is not None:
scanner = await scanner_getter()
if scanner is not None:
metadata_dict = metadata.to_dict()
metadata_dict["hf_url"] = hf_url
await scanner.add_model_to_cache(metadata_dict, folder)
logger.info("Added %s to scanner cache (folder=%s)", dest_path, folder)
except Exception as exc:
logger.warning("Failed to save HF metadata for %s: %s", dest_path, exc)
class HfHandler:
"""Handle Hugging Face model browsing and download."""
async def get_hf_repo_files(self, request: web.Request) -> web.Response:
"""List model-weight files from a HF repo with real file sizes.
Uses the HF tree API endpoint which returns accurate file sizes
(including LFS-tracked files), unlike the model info endpoint.
"""
repo = request.query.get("repo", "").strip()
if not repo or "/" not in repo:
return web.json_response(
{"error": "Missing or invalid 'repo' parameter (expected user/repo)"},
status=400,
)
url = f"https://huggingface.co/api/models/{repo}/tree/main"
try:
session = await _get_hf_api_session()
async with session.get(url) as resp:
if resp.status == 404:
return web.json_response(
{"error": f"Repo '{repo}' not found"}, status=404
)
if resp.status != 200:
text = await resp.text()
return web.json_response(
{"error": f"HF API error {resp.status}: {text[:200]}"},
status=resp.status,
)
tree: list[dict[str, Any]] = await resp.json()
except Exception as exc:
logger.error("Failed to fetch HF repo files: %s", exc)
return web.json_response({"error": str(exc)}, status=502)
files: list[dict[str, Any]] = []
for entry in tree:
path: str = entry.get("path", "")
ext = os.path.splitext(path)[1].lower()
if ext not in MODEL_FILE_EXTENSIONS:
continue
size = entry.get("size", 0) or 0
if size == 0 and "lfs" in entry:
size = entry["lfs"].get("size", 0) or 0
files.append({
"filename": path,
"size": size,
})
files.sort(key=lambda f: f["size"], reverse=True)
return web.json_response(files)
async def download_hf_model(self, request: web.Request) -> web.Response:
"""Download a single file from Hugging Face into the model directory.
POST JSON body::
{
"repo": "dx8152/Flux2-Klein-9B-Consistency",
"filename": "Flux2-Klein-9B-consistency-V2.safetensors",
"revision": "main",
"model_root": "loras",
"relative_path": "",
"use_default_paths": false,
"download_id": "optional-batch-id"
}
If ``download_id`` is provided, real-time progress (bytes, speed,
percentage) is broadcast via the WebSocket progress system, matching
the CivitAI download experience.
Respects the ``download_backend`` setting (``aria2`` or ``default``).
"""
try:
payload: dict[str, Any] = await request.json()
except json.JSONDecodeError:
return web.json_response({"error": "Invalid JSON"}, status=400)
repo = (payload.get("repo") or "").strip()
filename = (payload.get("filename") or "").strip()
revision = (payload.get("revision") or "main").strip()
model_root = (payload.get("model_root") or "").strip()
relative_path = (payload.get("relative_path") or "").strip()
use_default_paths = bool(payload.get("use_default_paths", False))
download_id: str | None = payload.get("download_id")
logger.info(
"download_hf_model: repo=%s file=%s root=%s download_id=%s",
repo, filename, model_root, download_id,
)
if not repo or not filename:
return web.json_response(
{"error": "Missing required fields: 'repo' and 'filename'"}, status=400
)
# Validate repo format — must be user/repo_name
if repo.count("/") != 1 or not re.match(r"^[a-zA-Z0-9_.-]+/[a-zA-Z0-9_.-]+$", repo):
return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
author, repo_name = repo.split("/", 1)
if ".." in (author, repo_name) or "." in (author, repo_name):
return web.json_response({"error": f"Invalid repo format: {repo}"}, status=400)
# Validate filename — must not contain path separators or ..
if "/" in filename or "\\" in filename or ".." in filename:
return web.json_response({"error": "Invalid filename"}, status=400)
# Validate relative_path — must not be absolute or escape base directory
if relative_path:
if os.path.isabs(relative_path):
return web.json_response({"error": "relative_path must not be absolute"}, status=400)
if ".." in relative_path.split("/") or "\\" in relative_path:
return web.json_response({"error": "Invalid relative_path"}, status=400)
# Validate model_root — must not contain path traversal
if not os.path.isabs(model_root):
# For relative model_root, check it doesn't escape
resolved_model_root = os.path.realpath(
os.path.join(os.getcwd(), "models", model_root)
)
else:
resolved_model_root = os.path.realpath(model_root)
# Verify model_root is within a configured scanner root
allowed_roots = set()
for root_list in (
config.loras_roots or [],
config.extra_loras_roots or [],
config.checkpoints_roots or [],
config.extra_checkpoints_roots or [],
config.unet_roots or [],
config.extra_unet_roots or [],
config.embeddings_roots or [],
config.extra_embeddings_roots or [],
):
for r in root_list:
allowed_roots.add(os.path.realpath(r))
if not any(resolved_model_root == root or resolved_model_root.startswith(root + os.sep) for root in allowed_roots):
logger.warning("Invalid model_root rejected: %s", model_root)
return web.json_response({"error": f"Invalid model_root: {model_root}"}, status=400)
base_dir = resolved_model_root
if use_default_paths:
target_dir = os.path.join(base_dir, "huggingface", author, repo_name)
elif relative_path:
target_dir = os.path.join(base_dir, relative_path)
else:
target_dir = base_dir
os.makedirs(target_dir, exist_ok=True)
dest_path = os.path.join(target_dir, filename)
# Resolve symlinks and check for path traversal escape
real_dest = os.path.realpath(dest_path)
real_base = os.path.realpath(target_dir)
if not real_dest.startswith(real_base + os.sep):
logger.warning("Path traversal blocked: %s -> %s", dest_path, real_dest)
return web.json_response({"error": "Path traversal detected"}, status=400)
# Check if already exists (simple skip)
if os.path.exists(dest_path) and os.path.getsize(dest_path) > 0:
logger.info("download_hf_model: file already exists, skipping — %s", dest_path)
return web.json_response({
"success": True,
"message": f"File already exists: {dest_path}",
"path": dest_path,
})
# Build HF resolve URL
resolve_url = (
f"https://huggingface.co/{repo}/resolve/{revision}/{filename}"
)
# Set up progress callback if download_id is provided
progress_callback = None
if download_id:
async def _progress_callback(
progress: float | DownloadProgress,
snapshot: DownloadProgress | None = None,
) -> None:
percent = 0.0
metrics = snapshot if isinstance(snapshot, DownloadProgress) else None
if isinstance(progress, DownloadProgress):
percent = progress.percent_complete
metrics = progress
elif isinstance(snapshot, DownloadProgress):
percent = snapshot.percent_complete
else:
percent = float(progress)
broadcast: dict[str, Any] = {
"status": "progress",
"progress": round(percent),
}
if metrics:
broadcast["bytes_downloaded"] = metrics.bytes_downloaded
broadcast["total_bytes"] = metrics.total_bytes
broadcast["bytes_per_second"] = metrics.bytes_per_second
await ws_manager.broadcast_download_progress(download_id, broadcast)
progress_callback = _progress_callback
# Respect download backend setting (aria2 vs default)
download_backend = (
get_settings_manager().get("download_backend", "default")
)
if download_backend == "aria2":
aria2 = await Aria2Downloader.get_instance()
aid = download_id or f"hf_{repo}_{filename}"
try:
hf_success, hf_result = await aria2.download_file(
url=resolve_url,
save_path=dest_path,
download_id=aid,
progress_callback=progress_callback,
)
if hf_success:
await _save_hf_metadata(dest_path, repo, model_root)
return web.json_response({
"success": True,
"message": f"Downloaded to {dest_path}",
"path": dest_path,
})
else:
return web.json_response(
{"success": False, "error": hf_result or "aria2 download failed"},
status=500,
)
except Exception as exc:
logger.error("HF download (aria2) failed: %s", exc)
return web.json_response(
{"success": False, "error": str(exc)}, status=500
)
# Default: use built-in aiohttp Downloader
downloader = await get_downloader()
try:
success, result = await downloader.download_file(
url=resolve_url,
save_path=dest_path,
use_auth=False,
allow_resume=True,
progress_callback=progress_callback,
)
if success:
await _save_hf_metadata(dest_path, repo, model_root)
return web.json_response({
"success": True,
"message": f"Downloaded to {result}",
"path": result,
})
else:
return web.json_response(
{"success": False, "error": result or "Download failed"},
status=500,
)
except Exception as exc:
logger.error("HF download failed: %s", exc)
return web.json_response(
{"success": False, "error": str(exc)}, status=500
)

View File

@@ -48,6 +48,7 @@ from ...utils.constants import (
SUPPORTED_MEDIA_EXTENSIONS,
VALID_LORA_TYPES,
)
from .hf_handlers import HfHandler
from ...utils.civitai_utils import rewrite_preview_url
from ...utils.example_images_paths import (
find_non_compliant_items_in_example_images_root,
@@ -3315,6 +3316,7 @@ class MiscHandlerSet:
doctor: DoctorHandler,
example_workflows: ExampleWorkflowsHandler,
base_model: BaseModelHandlerSet,
hf_handler: HfHandler | None = None,
) -> None:
self.health = health
self.settings = settings
@@ -3333,6 +3335,7 @@ class MiscHandlerSet:
self.doctor = doctor
self.example_workflows = example_workflows
self.base_model = base_model
self.hf_handler = hf_handler
def to_route_mapping(
self,
@@ -3378,6 +3381,9 @@ class MiscHandlerSet:
"get_supporters": self.supporters.get_supporters,
"get_example_workflows": self.example_workflows.get_example_workflows,
"get_example_workflow": self.example_workflows.get_example_workflow,
# Hugging Face handlers
"get_hf_repo_files": self.hf_handler.get_hf_repo_files,
"download_hf_model": self.hf_handler.download_hf_model,
# Base model handlers
"get_base_models": self.base_model.get_base_models,
"refresh_base_models": self.base_model.refresh_base_models,

View File

@@ -203,11 +203,17 @@ class ModelListingHandler:
result = await self._service.get_paginated_data(**params)
format_start = time.perf_counter()
formatted_raw = [
await self._service.format_response(entry)
for entry in result["items"]
]
# Filter out None entries returned for corrupted cache rows (issue #730).
# Note: "total" intentionally remains the pre-filter count to reflect
# the true number of models in the cache; corrupted entries are rare
# and adjusting total would cause pagination drift on every page.
formatted_items = [item for item in formatted_raw if item is not None]
formatted_result = {
"items": [
await self._service.format_response(item)
for item in result["items"]
],
"items": formatted_items,
"total": result["total"],
"page": result["page"],
"page_size": result["page_size"],
@@ -238,11 +244,15 @@ class ModelListingHandler:
result = await self._service.get_excluded_paginated_data(**params)
format_start = time.perf_counter()
formatted_raw = [
await self._service.format_response(entry)
for entry in result["items"]
]
# Filter out None entries returned for corrupted cache rows (issue #730).
# "total" stays at the pre-filter count; see get_models for rationale.
formatted_items = [item for item in formatted_raw if item is not None]
formatted_result = {
"items": [
await self._service.format_response(item)
for item in result["items"]
],
"items": formatted_items,
"total": result["total"],
"page": result["page"],
"page_size": result["page_size"],
@@ -533,8 +543,13 @@ class ModelManagementHandler:
if not success:
return web.json_response({"success": False, "error": error})
formatted_metadata = await self._service.format_response(model_data)
return web.json_response({"success": True, "metadata": formatted_metadata})
formatted = await self._service.format_response(model_data)
if formatted is None:
return web.json_response(
{"success": False, "error": "Model entry is corrupted (missing file_path)"},
status=500,
)
return web.json_response({"success": True, "metadata": formatted})
except Exception as exc:
if is_expected_offline_error(str(exc)):
return web.json_response(
@@ -1091,10 +1106,12 @@ class ModelQueryHandler:
# Sort: originals first, copies last
sorted_models = self._sort_duplicate_group(filtered)
# Format response
# Format response, filtering out corrupted entries (issue #730)
group = {"hash": sha256, "models": []}
for model in sorted_models:
group["models"].append(await self._service.format_response(model))
formatted = await self._service.format_response(model)
if formatted is not None:
group["models"].append(formatted)
# Only include groups with 2+ models after filtering
if len(group["models"]) > 1:
@@ -1211,9 +1228,9 @@ class ModelQueryHandler:
(m for m in cache.raw_data if m["file_path"] == path), None
)
if model:
group["models"].append(
await self._service.format_response(model)
)
formatted = await self._service.format_response(model)
if formatted is not None:
group["models"].append(formatted)
hash_val = self._service.scanner.get_hash_by_filename(filename)
if hash_val:
main_path = self._service.get_path_by_hash(hash_val)
@@ -1223,9 +1240,9 @@ class ModelQueryHandler:
None,
)
if main_model:
group["models"].insert(
0, await self._service.format_response(main_model)
)
formatted = await self._service.format_response(main_model)
if formatted is not None:
group["models"].insert(0, formatted)
if group["models"]:
result.append(group)
return web.json_response(

View File

@@ -94,6 +94,13 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition(
"GET", "/api/lm/delete-model-version", "delete_model_version"
),
# Hugging Face model endpoints
RouteDefinition(
"GET", "/api/lm/hf-repo-files", "get_hf_repo_files"
),
RouteDefinition(
"POST", "/api/lm/download-hf-model", "download_hf_model"
),
)

View File

@@ -39,6 +39,7 @@ from .handlers.misc_handlers import (
build_service_registry_adapter,
)
from .handlers.base_model_handlers import BaseModelHandlerSet
from .handlers.hf_handlers import HfHandler
from .misc_route_registrar import MiscRouteRegistrar
logger = logging.getLogger(__name__)
@@ -136,6 +137,7 @@ class MiscRoutes:
doctor = DoctorHandler(settings_service=self._settings)
example_workflows = ExampleWorkflowsHandler()
base_model = BaseModelHandlerSet()
hf_handler = HfHandler()
return self._handler_set_factory(
health=health,
@@ -155,6 +157,7 @@ class MiscRoutes:
doctor=doctor,
example_workflows=example_workflows,
base_model=base_model,
hf_handler=hf_handler,
)

View File

@@ -16,6 +16,27 @@ logger = logging.getLogger(__name__)
NETWORK_EXCEPTIONS = (ClientError, OSError, asyncio.TimeoutError)
# User-managed directories that live inside the plugin folder (portable
# mode) and must survive a Git-based update. ``git clean -fd`` would
# otherwise delete them because they are untracked and, in released tags,
# not listed in ``.gitignore``. ``-e`` excludes a path from cleaning
# regardless of whether it is ignored.
_PRESERVE_DIRS = ('settings.json', 'civitai', 'wildcards', 'backups', 'stats', 'logs', 'cache', 'model_cache')
def _clean_excludes() -> List[str]:
"""Build the ``-e`` arguments for ``git clean`` from :data:`_PRESERVE_DIRS`."""
excludes: List[str] = []
for name in _PRESERVE_DIRS:
excludes.append('-e')
excludes.append(name)
# For directories, also exclude nested matches explicitly
# (``-e dir`` alone matches the dir entry; ``-e dir/**`` guards
# contents under all git versions as defense-in-depth).
excludes.append('-e')
excludes.append(f'{name}/**')
return excludes
class UpdateRoutes:
"""Routes for handling plugin update checks"""
@@ -365,6 +386,8 @@ class UpdateRoutes:
)
return False, ""
clean_excludes = _clean_excludes()
try:
# Open the Git repository
repo = git.Repo(plugin_root)
@@ -376,8 +399,9 @@ class UpdateRoutes:
if nightly:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Clean untracked files, but preserve user-managed directories
# (wildcards, backups, stats, civitai, caches, settings.json).
repo.git.clean('-fd', *clean_excludes)
# Switch to main branch and pull latest
main_branch = 'main'
@@ -394,8 +418,9 @@ class UpdateRoutes:
else:
# Reset to discard any local changes
repo.git.reset('--hard')
# Clean untracked files
repo.git.clean('-fd')
# Clean untracked files, but preserve user-managed directories
# (wildcards, backups, stats, civitai, caches, settings.json).
repo.git.clean('-fd', *clean_excludes)
# Get latest release tag
tags = sorted(repo.tags, key=lambda t: t.commit.committed_datetime, reverse=True)

View File

@@ -791,8 +791,12 @@ class BaseModelService(ABC):
}
@abstractmethod
async def format_response(self, model_data: Dict) -> Dict:
"""Format model data for API response - must be implemented by subclasses"""
async def format_response(self, model_data: Dict) -> Optional[Dict]:
"""Format model data for API response - must be implemented by subclasses.
Subclasses should return None for corrupted entries so the handler
layer can filter them out. See issue #730.
"""
pass
# Common service methods that delegate to scanner

View File

@@ -1,6 +1,6 @@
import os
import logging
from typing import Dict
from typing import Dict, Optional
from .base_model_service import BaseModelService
from .auto_tag_service import extract_auto_tags
@@ -21,20 +21,37 @@ class CheckpointService(BaseModelService):
"""
super().__init__("checkpoint", scanner, CheckpointMetadata, update_service=update_service)
async def format_response(self, checkpoint_data: Dict) -> Dict:
"""Format Checkpoint data for API response"""
async def format_response(self, checkpoint_data: Dict) -> Optional[Dict]:
"""Format Checkpoint data for API response.
Returns None when the entry is missing critical fields (corrupted cache
row), so the handler layer can filter it out. See issue #730.
"""
# Guard against corrupted cache entries missing critical fields
file_path = checkpoint_data.get("file_path")
if not file_path or not isinstance(file_path, str):
logger.warning(
"Skipping corrupted checkpoint entry (missing file_path): %s",
checkpoint_data.get("file_name", "<unknown>"),
)
return None
# Get sub_type from cache entry (new canonical field)
sub_type = checkpoint_data.get("sub_type", "checkpoint")
file_name = checkpoint_data.get("file_name") or ""
model_name = checkpoint_data.get("model_name") or file_name
folder = checkpoint_data.get("folder") or ""
return {
"model_name": checkpoint_data["model_name"],
"file_name": checkpoint_data["file_name"],
"model_name": model_name,
"file_name": file_name,
"preview_url": config.get_preview_static_url(checkpoint_data.get("preview_url", "")),
"preview_nsfw_level": checkpoint_data.get("preview_nsfw_level", 0),
"base_model": checkpoint_data.get("base_model", ""),
"folder": checkpoint_data["folder"],
"folder": folder,
"sha256": checkpoint_data.get("sha256", ""),
"file_path": checkpoint_data["file_path"].replace(os.sep, "/"),
"file_path": file_path.replace(os.sep, "/"),
"file_size": checkpoint_data.get("size", 0),
"modified": checkpoint_data.get("modified", ""),
"tags": checkpoint_data.get("tags", []),
@@ -49,6 +66,7 @@ class CheckpointService(BaseModelService):
"civitai": self.filter_civitai_data(checkpoint_data.get("civitai", {}), minimal=True),
"auto_tags": checkpoint_data.get("auto_tags") or extract_auto_tags(checkpoint_data),
"version_count": checkpoint_data.get("version_count"),
"hf_url": checkpoint_data.get("hf_url", ""),
}
def find_duplicate_hashes(self) -> Dict:

View File

@@ -327,7 +327,7 @@ class CivArchiveClient:
if resolved:
return resolved, None
logger.error("Error fetching version of CivArchive model by hash %s", model_hash[:10])
logger.debug("Error fetching version of CivArchive model by hash %s", model_hash[:10])
return None, "No version data found"
except RateLimitError:

View File

@@ -196,6 +196,7 @@ class CivitaiBaseModelService:
"ernie": "ERNI",
"ernie turbo": "ETRB",
"nucleus": "NUCL",
"krea 2": "KR2",
"svd": "SVD",
"ltxv": "LTXV",
"ltxv2": "LTV2",
@@ -424,6 +425,7 @@ class CivitaiBaseModelService:
"Ernie",
"Ernie Turbo",
"Nucleus",
"Krea 2",
],
}

View File

@@ -1,6 +1,6 @@
import os
import logging
from typing import Dict
from typing import Dict, Optional
from .base_model_service import BaseModelService
from .auto_tag_service import extract_auto_tags
@@ -21,20 +21,37 @@ class EmbeddingService(BaseModelService):
"""
super().__init__("embedding", scanner, EmbeddingMetadata, update_service=update_service)
async def format_response(self, embedding_data: Dict) -> Dict:
"""Format Embedding data for API response"""
async def format_response(self, embedding_data: Dict) -> Optional[Dict]:
"""Format Embedding data for API response.
Returns None when the entry is missing critical fields (corrupted cache
row), so the handler layer can filter it out. See issue #730.
"""
# Guard against corrupted cache entries missing critical fields
file_path = embedding_data.get("file_path")
if not file_path or not isinstance(file_path, str):
logger.warning(
"Skipping corrupted embedding entry (missing file_path): %s",
embedding_data.get("file_name", "<unknown>"),
)
return None
# Get sub_type from cache entry (new canonical field)
sub_type = embedding_data.get("sub_type", "embedding")
file_name = embedding_data.get("file_name") or ""
model_name = embedding_data.get("model_name") or file_name
folder = embedding_data.get("folder") or ""
return {
"model_name": embedding_data["model_name"],
"file_name": embedding_data["file_name"],
"model_name": model_name,
"file_name": file_name,
"preview_url": config.get_preview_static_url(embedding_data.get("preview_url", "")),
"preview_nsfw_level": embedding_data.get("preview_nsfw_level", 0),
"base_model": embedding_data.get("base_model", ""),
"folder": embedding_data["folder"],
"folder": folder,
"sha256": embedding_data.get("sha256", ""),
"file_path": embedding_data["file_path"].replace(os.sep, "/"),
"file_path": file_path.replace(os.sep, "/"),
"file_size": embedding_data.get("size", 0),
"modified": embedding_data.get("modified", ""),
"tags": embedding_data.get("tags", []),
@@ -49,6 +66,7 @@ class EmbeddingService(BaseModelService):
"civitai": self.filter_civitai_data(embedding_data.get("civitai", {}), minimal=True),
"auto_tags": embedding_data.get("auto_tags") or extract_auto_tags(embedding_data),
"version_count": embedding_data.get("version_count"),
"hf_url": embedding_data.get("hf_url", ""),
}
def find_duplicate_hashes(self) -> Dict:

View File

@@ -24,23 +24,41 @@ class LoraService(BaseModelService):
"""
super().__init__("lora", scanner, LoraMetadata, update_service=update_service)
async def format_response(self, lora_data: Dict) -> Dict:
"""Format LoRA data for API response"""
async def format_response(self, lora_data: Dict) -> Optional[Dict]:
"""Format LoRA data for API response.
Returns None when the entry is missing critical fields (corrupted cache
row), so the handler layer can filter it out instead of crashing the
whole listing request. See issue #730.
"""
# Guard against corrupted cache entries missing critical fields
file_path = lora_data.get("file_path")
if not file_path or not isinstance(file_path, str):
logger.warning(
"Skipping corrupted LoRA entry (missing file_path): %s",
lora_data.get("file_name", "<unknown>"),
)
return None
# Resolve sub_type using priority: sub_type > model_type > civitai.model.type > default
# Normalize to lowercase for consistent API responses
sub_type = resolve_sub_type(lora_data).lower()
file_name = lora_data.get("file_name") or ""
model_name = lora_data.get("model_name") or file_name
folder = lora_data.get("folder") or ""
return {
"model_name": lora_data["model_name"],
"file_name": lora_data["file_name"],
"model_name": model_name,
"file_name": file_name,
"preview_url": config.get_preview_static_url(
lora_data.get("preview_url", "")
),
"preview_nsfw_level": lora_data.get("preview_nsfw_level", 0),
"base_model": lora_data.get("base_model", ""),
"folder": lora_data["folder"],
"folder": folder,
"sha256": lora_data.get("sha256", ""),
"file_path": lora_data["file_path"].replace(os.sep, "/"),
"file_path": file_path.replace(os.sep, "/"),
"file_size": lora_data.get("size", 0),
"modified": lora_data.get("modified", ""),
"tags": lora_data.get("tags", []),
@@ -60,6 +78,7 @@ class LoraService(BaseModelService):
),
"auto_tags": lora_data.get("auto_tags") or extract_auto_tags(lora_data),
"version_count": lora_data.get("version_count"),
"hf_url": lora_data.get("hf_url", ""),
}
async def _apply_specific_filters(self, data: List[Dict], **kwargs) -> List[Dict]:

View File

@@ -248,6 +248,7 @@ class ModelScanner:
'civitai': civitai_slim,
'civitai_deleted': bool(get_value('civitai_deleted', False)),
'skip_metadata_refresh': bool(get_value('skip_metadata_refresh', False)),
'hf_url': get_value('hf_url', '') or '',
}
license_source: Dict[str, Any] = {}
@@ -476,11 +477,20 @@ class ModelScanner:
for tag in adjusted_item.get('tags') or []:
tags_count[tag] = tags_count.get(tag, 0) + 1
# Validate cache entries and check health
# Validate cache entries and check health.
# Always use the validated/repaired entries — even when there are no
# invalid entries, auto_repair may have filled in missing optional
# fields (model_name, file_name, folder) with safe defaults on a copied
# working_entry. Without this unconditional replacement the repaired
# copies are discarded and None values propagate to format_response.
# See issue #730.
valid_entries, invalid_entries = CacheEntryValidator.validate_batch(
adjusted_raw_data, auto_repair=True
)
# Always use the validated entries (repaired copies)
adjusted_raw_data = valid_entries
if invalid_entries:
monitor = CacheHealthMonitor()
report = monitor.check_health(adjusted_raw_data, auto_repair=True)

View File

@@ -57,6 +57,7 @@ class PersistentModelCache:
"db_checked",
"last_checked_at",
"hash_status",
"hf_url",
)
_MODEL_UPDATE_COLUMNS: Tuple[str, ...] = _MODEL_COLUMNS[2:]
_instances: Dict[str, "PersistentModelCache"] = {}
@@ -165,8 +166,8 @@ class PersistentModelCache:
item = {
"file_path": file_path,
"file_name": row["file_name"],
"model_name": row["model_name"],
"file_name": row["file_name"] or "",
"model_name": row["model_name"] or "",
"folder": row["folder"] or "",
"size": row["size"] or 0,
"modified": row["modified"] or 0.0,
@@ -188,6 +189,7 @@ class PersistentModelCache:
"skip_metadata_refresh": bool(row["skip_metadata_refresh"]),
"license_flags": int(license_value),
"hash_status": row["hash_status"] or "completed",
"hf_url": row["hf_url"] or "",
}
raw_data.append(item)
@@ -452,6 +454,7 @@ class PersistentModelCache:
db_checked INTEGER,
last_checked_at REAL,
hash_status TEXT,
hf_url TEXT DEFAULT '',
PRIMARY KEY (model_type, file_path)
);
@@ -500,6 +503,7 @@ class PersistentModelCache:
# Persisting without explicit flags should assume CivitAI's documented defaults (0b111001 == 57).
"license_flags": f"INTEGER DEFAULT {DEFAULT_LICENSE_FLAGS}",
"hash_status": "TEXT DEFAULT 'completed'",
"hf_url": "TEXT DEFAULT ''",
}
for column, definition in required_columns.items():
@@ -548,19 +552,19 @@ class PersistentModelCache:
return (
model_type,
item.get("file_path"),
item.get("file_name"),
item.get("model_name"),
item.get("folder"),
item.get("file_name") or "",
item.get("model_name") or "",
item.get("folder") or "",
int(item.get("size") or 0),
float(item.get("modified") or 0.0),
(item.get("sha256") or "").lower() or None,
item.get("base_model"),
item.get("preview_url"),
item.get("base_model") or "",
item.get("preview_url") or "",
int(item.get("preview_nsfw_level") or 0),
1 if item.get("from_civitai", True) else 0,
1 if item.get("favorite") else 0,
item.get("notes"),
item.get("usage_tips"),
item.get("notes") or "",
item.get("usage_tips") or "",
metadata_source,
civitai.get("id"),
civitai.get("modelId"),
@@ -575,6 +579,7 @@ class PersistentModelCache:
1 if item.get("db_checked") else 0,
float(item.get("last_checked_at") or 0.0),
item.get("hash_status", "completed"),
item.get("hf_url") or "",
)
def _insert_model_sql(self) -> str:

View File

@@ -1568,7 +1568,7 @@ class SettingsManager:
previous_dir = os.path.dirname(previous_path) or target_dir
if os.path.abspath(previous_path) != os.path.abspath(target_path):
self._copy_model_cache_directory(previous_dir, target_dir)
self._migrate_settings_directory_content(previous_dir, target_dir)
logger.info("Switching settings file to: %s", target_path)
self._pending_portable_switch = {"other_path": other_path}
@@ -1603,46 +1603,52 @@ class SettingsManager:
finally:
self._pending_portable_switch = None
def _copy_model_cache_directory(self, source_dir: str, target_dir: str) -> None:
"""Copy model_cache artifacts when switching storage locations."""
def _migrate_settings_directory_content(
self, source_dir: str, target_dir: str
) -> None:
"""Migrate settings directory subdirectories when switching storage locations.
Copies the canonical subdirectories (cache, backups, logs, stats, wildcards)
from the old settings directory to the new one. Legacy cache artifacts
(model_cache, recipe_cache, etc.) are migrated lazily by
``resolve_cache_path_with_migration`` on first access.
Args:
source_dir: The previous settings directory path.
target_dir: The new settings directory path.
"""
if not source_dir or not target_dir:
return
source_cache_dir = os.path.join(source_dir, "model_cache")
target_cache_dir = os.path.join(target_dir, "model_cache")
if os.path.isdir(source_cache_dir) and os.path.abspath(
source_cache_dir
) != os.path.abspath(target_cache_dir):
try:
shutil.copytree(
source_cache_dir,
target_cache_dir,
dirs_exist_ok=True,
ignore=shutil.ignore_patterns("*.sqlite-shm", "*.sqlite-wal"),
)
except Exception as exc:
logger.warning(
"Failed to copy model_cache directory from %s to %s: %s",
source_cache_dir,
target_cache_dir,
exc,
)
def _copy_dir(name: str) -> None:
source = os.path.join(source_dir, name)
target = os.path.join(target_dir, name)
if os.path.isdir(source) and os.path.abspath(source) != os.path.abspath(
target
):
try:
shutil.copytree(
source,
target,
dirs_exist_ok=True,
ignore=shutil.ignore_patterns("*.sqlite-shm", "*.sqlite-wal"),
)
except Exception as exc:
logger.warning(
"Failed to copy directory %s from %s to %s: %s",
name,
source,
target,
exc,
)
source_cache_file = os.path.join(source_dir, "model_cache.sqlite")
target_cache_file = os.path.join(target_dir, "model_cache.sqlite")
if os.path.isfile(source_cache_file) and os.path.abspath(
source_cache_file
) != os.path.abspath(target_cache_file):
try:
shutil.copy2(source_cache_file, target_cache_file)
except Exception as exc:
logger.warning(
"Failed to copy model_cache.sqlite from %s to %s: %s",
source_cache_file,
target_cache_file,
exc,
)
# Managed subdirectories under settings_dir
_copy_dir("cache")
_copy_dir("backups")
_copy_dir("logs")
_copy_dir("stats")
_copy_dir("wildcards")
def _get_user_config_directory(self) -> str:
"""Return the user configuration directory, falling back to ~/.config."""

View File

@@ -47,6 +47,20 @@ SUPPORTED_MEDIA_EXTENSIONS = {
"videos": [".mp4", ".webm"],
}
# Model weight file extensions recognised by scanners.
# This is the union of all scanner extensions (lora, checkpoint, embedding).
MODEL_FILE_EXTENSIONS = {
".safetensors",
".ckpt",
".pt",
".pt2",
".bin",
".pth",
".pkl",
".sft",
".gguf",
}
# Valid sub-types for each scanner type
VALID_LORA_SUB_TYPES = ["lora", "locon", "dora"]
VALID_CHECKPOINT_SUB_TYPES = ["checkpoint", "diffusion_model"]
@@ -215,5 +229,6 @@ SUPPORTED_DOWNLOAD_SKIP_BASE_MODELS = frozenset(
"Ernie",
"Ernie Turbo",
"Nucleus",
"Krea 2",
]
)

View File

@@ -444,16 +444,161 @@
flex: 1;
}
.base-model-selector {
width: 100%;
padding: 3px 5px;
/* ── Base Model Search Dropdown ─────────────────────────────────────────── */
.base-model-search-wrapper {
position: relative;
flex: 1;
min-width: 0;
z-index: 100;
}
.base-model-search-input-wrapper {
display: flex;
align-items: center;
background: var(--bg-color);
border: 1px solid var(--lora-accent);
border-radius: var(--border-radius-xs);
padding: 0 6px;
gap: 4px;
}
.base-model-search-input-wrapper .search-icon {
color: var(--text-color);
opacity: 0.45;
font-size: 12px;
flex-shrink: 0;
pointer-events: none;
/* Reset global .search-icon rules from search-filter.css */
position: static;
right: auto;
top: auto;
transform: none;
}
.base-model-search-input {
flex: 1;
background: transparent;
border: none;
outline: none;
color: var(--text-color);
font-size: 0.9em;
outline: none;
margin-right: var(--space-1);
padding: 3px 0;
width: 100%;
min-width: 0;
}
.base-model-search-input::placeholder {
color: var(--text-color);
opacity: 0.35;
}
.base-model-dropdown {
position: absolute;
top: 100%;
left: -1px;
right: -1px;
max-height: 270px;
overflow-y: auto;
background: var(--bg-color);
border: 1px solid var(--lora-border);
border-top: none;
border-radius: 0 0 var(--border-radius-xs) var(--border-radius-xs);
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.22);
z-index: 101;
}
[data-theme="dark"] .base-model-dropdown {
box-shadow: 0 8px 28px rgba(0, 0, 0, 0.5);
}
/* Dropdown scrollbar styling */
.base-model-dropdown::-webkit-scrollbar {
width: 6px;
}
.base-model-dropdown::-webkit-scrollbar-thumb {
background: var(--lora-border);
border-radius: 3px;
}
.base-model-dropdown::-webkit-scrollbar-track {
background: transparent;
}
/* Section */
.base-model-dropdown-section {
border-bottom: 1px solid var(--lora-border);
}
.base-model-dropdown-section:last-child {
border-bottom: none;
}
/* Section header */
.base-model-dropdown-header {
padding: 5px 10px;
font-size: 0.72em;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.08em;
color: var(--text-color);
opacity: 0.5;
background: var(--surface-subtle);
position: sticky;
top: 0;
z-index: 1;
}
.base-model-dropdown-header.suggested-header {
color: var(--lora-accent);
opacity: 1;
background: oklch(from var(--lora-accent) l c h / 0.08);
}
.base-model-dropdown-header.suggested-header i {
margin-right: 4px;
font-size: 0.85em;
}
/* Dropdown items */
.base-model-dropdown-item {
padding: 5px 12px;
cursor: pointer;
font-size: 0.9em;
color: var(--text-color);
transition: background 0.1s;
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
.base-model-dropdown-item:hover {
background: oklch(from var(--lora-accent) l c h / 0.1);
}
.base-model-dropdown-item.active {
background: oklch(from var(--lora-accent) l c h / 0.16);
}
.base-model-dropdown-item.selected {
font-weight: 600;
}
.base-model-dropdown-item.selected::after {
content: '✓';
float: right;
color: var(--lora-accent);
margin-left: 8px;
}
/* Empty state */
.base-model-dropdown-empty {
padding: 18px 12px;
text-align: center;
color: var(--text-color);
opacity: 0.4;
font-size: 0.88em;
}
.size-wrapper {

View File

@@ -821,4 +821,66 @@
[data-theme="dark"] .batch-preview-item {
background: var(--lora-surface);
}
}
.hf-badge {
display: inline-block;
padding: 1px 6px;
border-radius: 8px;
background: oklch(0.55 0.12 250 / 0.15);
color: oklch(0.7 0.12 250);
font-size: 0.75em;
font-weight: 600;
margin-left: 4px;
}
/* Checkbox inside HF batch preview items */
.batch-preview-checkbox {
width: 18px;
height: 18px;
cursor: pointer;
accent-color: var(--lora-accent);
flex-shrink: 0;
padding: 0;
border: none;
margin: 0;
}
/* Select All toolbar in batch preview */
.batch-preview-select-all {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 12px;
border-bottom: 1px solid var(--border-color);
background: var(--lora-surface);
cursor: pointer;
position: sticky;
top: 0;
z-index: 1;
}
.batch-preview-select-all input[type="checkbox"] {
width: 18px;
height: 18px;
cursor: pointer;
accent-color: var(--lora-accent);
flex-shrink: 0;
padding: 0;
border: none;
margin: 0;
}
.batch-preview-select-all label {
cursor: pointer;
font-size: 0.9em;
color: var(--text-color);
font-weight: 500;
margin: 0;
user-select: none;
}
[data-theme="dark"] .batch-preview-select-all {
background: var(--lora-surface);
}

View File

@@ -190,6 +190,12 @@ export const DOWNLOAD_ENDPOINTS = {
exampleImages: '/api/lm/force-download-example-images' // New endpoint for downloading example images
};
// Hugging Face API endpoints
export const HF_ENDPOINTS = {
repoFiles: '/api/lm/hf-repo-files',
download: '/api/lm/download-hf-model',
};
// WebSocket endpoints
export const WS_ENDPOINTS = {
fetchProgress: '/ws/fetch-progress'

View File

@@ -7,6 +7,7 @@ import {
getCurrentModelType,
isValidModelType,
DOWNLOAD_ENDPOINTS,
HF_ENDPOINTS,
WS_ENDPOINTS
} from './apiConfig.js';
import { resetAndReload } from './modelApiFactory.js';
@@ -1243,6 +1244,48 @@ export class BaseModelApiClient {
}
}
async fetchHfRepoFiles(repo, revision = 'main') {
try {
const params = new URLSearchParams({ repo, revision });
const response = await fetch(`${HF_ENDPOINTS.repoFiles}?${params}`);
if (!response.ok) {
const err = await response.json().catch(() => ({}));
throw new Error(err.error || 'Failed to fetch HF repo files');
}
return await response.json();
} catch (error) {
console.error('Error fetching HF repo files:', error);
throw error;
}
}
async downloadHfModel({ repo, filename, revision, modelRoot, relativePath, useDefaultPaths, download_id }) {
try {
const response = await fetch(HF_ENDPOINTS.download, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
repo,
filename,
revision: revision || 'main',
model_root: modelRoot,
relative_path: relativePath || '',
use_default_paths: useDefaultPaths || false,
...(download_id ? { download_id } : {}),
})
});
if (!response.ok) {
throw new Error(await response.text());
}
return await response.json();
} catch (error) {
console.error('Error downloading HF model:', error);
throw error;
}
}
_buildQueryParams(baseParams, pageState) {
const params = new URLSearchParams(baseParams);
const isExcludedView = pageState.viewMode === 'excluded';

View File

@@ -1,4 +1,4 @@
import { showToast, openCivitai, copyToClipboard, copyLoraSyntax, sendLoraToWorkflow, sendEmbeddingToWorkflow, openExampleImagesFolder, buildLoraSyntax, sendModelPathToWorkflow } from '../../utils/uiHelpers.js';
import { showToast, openCivitai, openHuggingFace, copyToClipboard, copyLoraSyntax, sendLoraToWorkflow, sendEmbeddingToWorkflow, openExampleImagesFolder, buildLoraSyntax, sendModelPathToWorkflow } from '../../utils/uiHelpers.js';
import { state, getCurrentPageState } from '../../state/index.js';
import { showModelModal } from './ModelModal.js';
import { toggleShowcase } from './showcase/ShowcaseView.js';
@@ -66,6 +66,8 @@ function handleModelCardEvent_internal(event, modelType) {
event.stopPropagation();
if (card.dataset.from_civitai === 'true') {
openCivitai(card.dataset.filepath);
} else if (card.dataset.hf_url) {
openHuggingFace(card.dataset.hf_url);
}
return true; // Stop propagation
}
@@ -313,6 +315,7 @@ async function showModelModalFromCard(card, modelType) {
modified: card.dataset.modified,
file_size: parseInt(card.dataset.file_size || '0'),
from_civitai: card.dataset.from_civitai === 'true',
hf_url: card.dataset.hf_url || '',
base_model: card.dataset.base_model,
notes: card.dataset.notes || '',
favorite: card.dataset.favorite === 'true',
@@ -401,6 +404,7 @@ function showExampleAccessModal(card, modelType) {
modified: card.dataset.modified,
file_size: card.dataset.file_size,
from_civitai: card.dataset.from_civitai === 'true',
hf_url: card.dataset.hf_url || '',
base_model: card.dataset.base_model,
notes: card.dataset.notes,
favorite: card.dataset.favorite === 'true',
@@ -467,6 +471,7 @@ export function createModelCard(model, modelType) {
card.dataset.base_model = model.base_model || 'Unknown';
card.dataset.favorite = model.favorite ? 'true' : 'false';
card.dataset.exclude = model.exclude ? 'true' : 'false';
card.dataset.hf_url = model.hf_url || '';
const hasUpdateAvailable = Boolean(model.update_available);
card.dataset.update_available = hasUpdateAvailable ? 'true' : 'false';
card.dataset.skip_metadata_refresh = model.skip_metadata_refresh ? 'true' : 'false';
@@ -578,7 +583,10 @@ export function createModelCard(model, modelType) {
translate('modelCard.actions.addToFavorites', {}, 'Add to favorites');
const globeTitle = model.from_civitai ?
translate('modelCard.actions.viewOnCivitai', {}, 'View on Civitai') :
translate('modelCard.actions.notAvailableFromCivitai', {}, 'Not available from Civitai');
model.hf_url ?
translate('modelCard.actions.viewOnHuggingFace', {}, 'View on Hugging Face') :
translate('modelCard.actions.notAvailableFromCivitai', {}, 'Not available from Civitai');
const globeEnabled = model.from_civitai || !!model.hf_url;
let sendTitle;
let copyTitle;
if (modelType === MODEL_TYPES.LORA) {
@@ -603,7 +611,7 @@ export function createModelCard(model, modelType) {
</i>
<i class="fas fa-globe"
title="${globeTitle}"
${!model.from_civitai ? 'style="opacity: 0.5; cursor: not-allowed"' : ''}>
${!globeEnabled ? 'style="opacity: 0.5; cursor: not-allowed"' : ''}>
</i>
<i class="fas fa-paper-plane"
title="${sendTitle}">

View File

@@ -3,9 +3,75 @@
* Handles model metadata editing functionality - General version
*/
import { BASE_MODEL_CATEGORIES } from '../../utils/constants.js';
import { BASE_MODEL_CATEGORIES, getMergedBaseModels } from '../../utils/constants.js';
import { showToast } from '../../utils/uiHelpers.js';
import { getModelApiClient } from '../../api/modelApiFactory.js';
import { translate } from '../../utils/i18nHelpers.js';
// ── Filename-based base model inference ──────────────────────────────────────
// Rules are ordered by specificity — first match wins for dedup.
// Each rule checks the filename (lowercased) for a regex pattern and suggests
// the associated base model values.
const BASE_MODEL_FILENAME_RULES = [
{ pattern: /flux\.?\s*2\s*klein/i, models: ['Flux.2 Klein 9B', 'Flux.2 Klein 9B-base', 'Flux.2 Klein 4B', 'Flux.2 Klein 4B-base'] },
{ pattern: /flux\.?\s*2/i, models: ['Flux.2 D', 'Flux.2 Klein 9B', 'Flux.2 Klein 4B'] },
{ pattern: /flux\.?\s*1\s*(dev|d)\b/i, models: ['Flux.1 D'] },
{ pattern: /flux\.?\s*1\s*(schnell|s)\b/i, models: ['Flux.1 S'] },
{ pattern: /flux/i, models: ['Flux.1 D', 'Flux.1 S', 'Flux.2 D'] },
{ pattern: /sdxl/i, models: ['SDXL 1.0', 'SDXL Lightning', 'SDXL Hyper'] },
{ pattern: /sd\s*1[._-\s]?5/i, models: ['SD 1.5'] },
{ pattern: /sd\s*1[._-\s]?4/i, models: ['SD 1.4'] },
{ pattern: /sd\s*1/i, models: ['SD 1.5', 'SD 1.4', 'SD 1.5 LCM', 'SD 1.5 Hyper'] },
{ pattern: /sd\s*3[._-\s]?5/i, models: ['SD 3.5', 'SD 3.5 Medium', 'SD 3.5 Large', 'SD 3.5 Large Turbo'] },
{ pattern: /sd\s*3/i, models: ['SD 3', 'SD 3.5'] },
{ pattern: /wan\s*\.?\s*video/i, models: ['Wan Video', 'Wan Video 1.3B t2v', 'Wan Video 14B t2v', 'Wan Video 14B i2v 480p', 'Wan Video 14B i2v 720p'] },
{ pattern: /hunyuan\s*\.?\s*video/i, models: ['Hunyuan Video'] },
{ pattern: /ltxv/i, models: ['LTXV', 'LTXV2', 'LTXV 2.3'] },
{ pattern: /cogvideo/i, models: ['CogVideoX'] },
{ pattern: /pony/i, models: ['Pony', 'Pony V7'] },
{ pattern: /illustrious/i, models: ['Illustrious'] },
{ pattern: /noobai/i, models: ['NoobAI'] },
{ pattern: /pixart/i, models: ['PixArt a', 'PixArt E'] },
{ pattern: /aura\s*\.?\s*flow/i, models: ['AuraFlow'] },
{ pattern: /kolors/i, models: ['Kolors'] },
{ pattern: /hunyuan\s*1/i, models: ['Hunyuan 1'] },
{ pattern: /lumina/i, models: ['Lumina'] },
{ pattern: /hidream/i, models: ['HiDream'] },
{ pattern: /qwen/i, models: ['Qwen'] },
{ pattern: /chroma/i, models: ['Chroma'] },
{ pattern: /anima/i, models: ['Anima'] },
{ pattern: /sd\s*2[._-\s]?[01]/i, models: ['SD 2.0', 'SD 2.1'] },
{ pattern: /mochi/i, models: ['Mochi'] },
{ pattern: /svd/i, models: ['SVD'] },
{ pattern: /zimage/i, models: ['ZImageTurbo', 'ZImageBase'] },
{ pattern: /nucleus/i, models: ['Nucleus'] },
{ pattern: /krea/i, models: ['Flux.1 Krea', 'Krea 2'] },
{ pattern: /ernie/i, models: ['Ernie', 'Ernie Turbo'] },
];
/**
* Infer likely base model(s) from a filename + model name string.
* Returns a deduplicated array in match-priority order.
* @param {string} filename
* @returns {string[]}
*/
function inferBaseModelsFromFilename(filename) {
if (!filename || typeof filename !== 'string') return [];
const seen = new Set();
const results = [];
for (const rule of BASE_MODEL_FILENAME_RULES) {
if (rule.pattern.test(filename)) {
for (const model of rule.models) {
if (!seen.has(model)) {
seen.add(model);
results.push(model);
}
}
}
}
return results;
}
/**
* Resolve the active file path for the currently open model modal.
@@ -226,7 +292,9 @@ export function setupModelNameEditing(filePath) {
}
/**
* Set up base model editing functionality
* Set up base model editing functionality with searchable dropdown
* Shows filename-inferred suggestions at the top, supports keyboard navigation,
* and allows typing custom values.
* @param {string} filePath - File path
*/
export function setupBaseModelEditing(filePath) {
@@ -257,98 +325,251 @@ export function setupBaseModelEditing(filePath) {
// Store the original value to check for changes later
const originalValue = baseModelContent.textContent.trim();
// Create dropdown selector to replace the base model content
const currentValue = originalValue;
const dropdown = document.createElement('select');
dropdown.className = 'base-model-selector';
// ── Build the full option list ────────────────────────────────────────
const allModels = []; // { value, label, category }
const categorizedModels = new Set();
// Flag to track if a change was made
let valueChanged = false;
// Add options from BASE_MODEL_CATEGORIES constants
const baseModelCategories = BASE_MODEL_CATEGORIES;
// Create option groups for better organization
Object.entries(baseModelCategories).forEach(([category, models]) => {
const group = document.createElement('optgroup');
group.label = category;
Object.entries(BASE_MODEL_CATEGORIES).forEach(([category, models]) => {
models.forEach(model => {
const option = document.createElement('option');
option.value = model;
option.textContent = model;
option.selected = model === currentValue;
group.appendChild(option);
allModels.push({ value: model, label: model, category });
categorizedModels.add(model);
});
});
const mergedModels = getMergedBaseModels();
const uncategorizedModels = mergedModels.filter(model => !categorizedModels.has(model));
if (uncategorizedModels.length > 0) {
uncategorizedModels.forEach(model => {
allModels.push({ value: model, label: model, category: 'Other (API)' });
});
}
// ── Filename-based inference ──────────────────────────────────────────
const fileName = (document.querySelector('.file-name-content')?.textContent || '') + ' ' +
(document.querySelector('.model-name-content')?.textContent || '');
const inferredModels = inferBaseModelsFromFilename(fileName);
const inferredSet = new Set(inferredModels);
// ── Build search widget DOM ───────────────────────────────────────────
const wrapper = document.createElement('div');
wrapper.className = 'base-model-search-wrapper';
// Search input row
const inputWrapper = document.createElement('div');
inputWrapper.className = 'base-model-search-input-wrapper';
const searchIcon = document.createElement('i');
searchIcon.className = 'fas fa-search search-icon';
searchIcon.setAttribute('aria-hidden', 'true');
inputWrapper.appendChild(searchIcon);
const searchInput = document.createElement('input');
searchInput.type = 'text';
searchInput.className = 'base-model-search-input';
searchInput.placeholder = translate('modals.model.metadata.baseModelSearchPlaceholder', {}, 'Search base model…');
searchInput.autocomplete = 'off';
searchInput.spellcheck = false;
inputWrapper.appendChild(searchInput);
wrapper.appendChild(inputWrapper);
// Dropdown list
const dropdown = document.createElement('div');
dropdown.className = 'base-model-dropdown';
wrapper.appendChild(dropdown);
// ── Render ────────────────────────────────────────────────────────────
function renderDropdown(filterText) {
const lowerFilter = (filterText || '').toLowerCase().trim();
dropdown.innerHTML = '';
let hasVisibleItems = false;
const fragment = document.createDocumentFragment();
// 1. Suggested section (filename-inferred, filtered by search)
let suggestedToShow = inferredModels;
if (lowerFilter) {
suggestedToShow = inferredModels.filter(m =>
m.toLowerCase().includes(lowerFilter)
);
}
if (suggestedToShow.length > 0) {
const section = document.createElement('div');
section.className = 'base-model-dropdown-section';
const header = document.createElement('div');
header.className = 'base-model-dropdown-header suggested-header';
header.innerHTML = '<i class="fas fa-star" aria-hidden="true"></i> ' +
translate('modals.model.metadata.baseModelSuggested', {}, 'Suggested');
section.appendChild(header);
suggestedToShow.forEach(model => {
const item = document.createElement('div');
item.className = 'base-model-dropdown-item';
if (model === originalValue) item.classList.add('selected');
item.dataset.value = model;
item.textContent = model;
section.appendChild(item);
hasVisibleItems = true;
});
fragment.appendChild(section);
}
// 2. Categorized options (deduplicated against suggestions)
const categoryMap = {};
allModels.forEach(m => {
if (inferredSet.has(m.value)) return; // already shown in Suggested
if (lowerFilter && !m.label.toLowerCase().includes(lowerFilter)) return;
if (!categoryMap[m.category]) categoryMap[m.category] = [];
categoryMap[m.category].push(m);
});
dropdown.appendChild(group);
Object.entries(categoryMap).forEach(([category, items]) => {
if (items.length === 0) return;
const section = document.createElement('div');
section.className = 'base-model-dropdown-section';
const header = document.createElement('div');
header.className = 'base-model-dropdown-header';
header.textContent = category;
section.appendChild(header);
items.forEach(m => {
const item = document.createElement('div');
item.className = 'base-model-dropdown-item';
if (m.value === originalValue) item.classList.add('selected');
item.dataset.value = m.value;
item.textContent = m.label;
section.appendChild(item);
hasVisibleItems = true;
});
fragment.appendChild(section);
});
// 3. Empty state
if (!hasVisibleItems) {
const empty = document.createElement('div');
empty.className = 'base-model-dropdown-empty';
empty.textContent = translate('modals.model.metadata.baseModelNoMatch', {}, 'No matching base models');
fragment.appendChild(empty);
}
dropdown.appendChild(fragment);
// Scroll the selected item into view
const selected = dropdown.querySelector('.base-model-dropdown-item.selected');
if (selected) {
selected.scrollIntoView({ block: 'nearest' });
}
}
// Initial render — show everything
renderDropdown('');
// ── Events ────────────────────────────────────────────────────────────
let filterTimeout;
searchInput.addEventListener('input', () => {
clearTimeout(filterTimeout);
filterTimeout = setTimeout(() => renderDropdown(searchInput.value), 50);
});
// Replace content with dropdown
// Click to select
dropdown.addEventListener('click', (e) => {
const item = e.target.closest('.base-model-dropdown-item');
if (!item) return;
baseModelContent.textContent = item.dataset.value;
cleanup();
const finalValue = baseModelContent.textContent.trim();
if (finalValue !== originalValue) {
saveBaseModel(
getActiveModalFilePath(baseModelContent.dataset.filePath),
originalValue
);
}
});
// Replace content with search widget
baseModelContent.style.display = 'none';
baseModelDisplay.insertBefore(dropdown, editBtn);
// Hide edit button during editing
editBtn.style.display = 'none';
baseModelDisplay.insertBefore(wrapper, editBtn);
searchInput.focus();
// Focus the dropdown
dropdown.focus();
// Handle dropdown change
dropdown.addEventListener('change', function() {
const selectedModel = this.value;
baseModelContent.textContent = selectedModel;
// Mark that a change was made if the value differs from original
if (selectedModel !== originalValue) {
valueChanged = true;
} else {
valueChanged = false;
// ── Cleanup ───────────────────────────────────────────────────────────
function cleanup() {
if (wrapper.parentNode === baseModelDisplay) {
baseModelDisplay.removeChild(wrapper);
}
});
// Function to save changes and exit edit mode
const saveAndExit = function() {
// Check if dropdown still exists and remove it
if (dropdown && dropdown.parentNode === baseModelDisplay) {
baseModelDisplay.removeChild(dropdown);
}
// Show the content and edit button
baseModelContent.style.display = '';
editBtn.style.display = '';
// Remove editing class
baseModelDisplay.classList.remove('editing');
// Only save if the value has actually changed
if (valueChanged || baseModelContent.textContent.trim() !== originalValue) {
const resolvedPath = getActiveModalFilePath(baseModelContent.dataset.filePath);
saveBaseModel(resolvedPath, originalValue);
}
// Remove this event listener
document.removeEventListener('click', outsideClickHandler);
};
}
// Handle outside clicks to save and exit
// Outside click save typed/custom value if any
const outsideClickHandler = function(e) {
// If click is outside the dropdown and base model display
if (!baseModelDisplay.contains(e.target)) {
saveAndExit();
if (wrapper.contains(e.target)) return;
// If user typed a custom value (not just empty), apply it
const typedValue = searchInput.value.trim();
if (typedValue) {
baseModelContent.textContent = typedValue;
}
cleanup();
const finalValue = baseModelContent.textContent.trim();
if (finalValue !== originalValue) {
saveBaseModel(
getActiveModalFilePath(baseModelContent.dataset.filePath),
originalValue
);
}
};
// Add delayed event listener for outside clicks
// Defer listener to avoid the opening click itself
setTimeout(() => {
document.addEventListener('click', outsideClickHandler);
}, 0);
// Also handle dropdown blur event
dropdown.addEventListener('blur', function(e) {
// Only save if the related target is not the edit button or inside the baseModelDisplay
if (!baseModelDisplay.contains(e.relatedTarget)) {
saveAndExit();
// Keyboard navigation
searchInput.addEventListener('keydown', function onKeydown(e) {
const items = Array.from(dropdown.querySelectorAll('.base-model-dropdown-item'));
const activeIdx = items.findIndex(el => el.classList.contains('active'));
if (e.key === 'ArrowDown') {
e.preventDefault();
items.forEach(el => el.classList.remove('active'));
const next = Math.min(activeIdx + 1, items.length - 1);
if (items[next]) {
items[next].classList.add('active');
items[next].scrollIntoView({ block: 'nearest' });
}
} else if (e.key === 'ArrowUp') {
e.preventDefault();
items.forEach(el => el.classList.remove('active'));
const prev = Math.max(activeIdx - 1, 0);
if (items[prev]) {
items[prev].classList.add('active');
items[prev].scrollIntoView({ block: 'nearest' });
}
} else if (e.key === 'Enter') {
e.preventDefault();
const activeItem = items.find(el => el.classList.contains('active'));
if (activeItem) {
activeItem.click();
} else if (searchInput.value.trim()) {
// Custom value typed
baseModelContent.textContent = searchInput.value.trim();
cleanup();
const finalValue = baseModelContent.textContent.trim();
if (finalValue !== originalValue) {
saveBaseModel(
getActiveModalFilePath(baseModelContent.dataset.filePath),
originalValue
);
}
}
} else if (e.key === 'Escape') {
e.preventDefault();
baseModelContent.textContent = originalValue;
cleanup();
}
});
});

View File

@@ -360,6 +360,11 @@ export async function showModelModal(model, modelType) {
const viewOnCivitaiAction = modelWithFullData.from_civitai ? `
<div class="civitai-view" title="${translate('modals.model.actions.viewOnCivitai', {}, 'View on Civitai')}" data-action="view-civitai" data-filepath="${escapedFilePathAttr}">
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnCivitaiText', {}, 'View on Civitai')}
</div>`.trim() : '';
const escapedHfUrl = modelWithFullData.hf_url ? escapeAttribute(modelWithFullData.hf_url) : '';
const viewOnHuggingFaceAction = escapedHfUrl ? `
<div class="civitai-view" title="${translate('modals.model.actions.viewOnHuggingFace', {}, 'View on Hugging Face')}" data-action="view-huggingface" data-hf-url="${escapedHfUrl}">
<i class="fas fa-globe"></i> ${translate('modals.model.actions.viewOnHuggingFaceText', {}, 'View on Hugging Face')}
</div>`.trim() : '';
const creatorInfoAction = modelWithFullData.civitai?.creator ? `
<div class="creator-info" data-username="${modelWithFullData.civitai.creator.username}" data-action="view-creator" title="${translate('modals.model.actions.viewCreatorProfile', {}, 'View Creator Profile')}">
@@ -377,6 +382,9 @@ export async function showModelModal(model, modelType) {
if (viewOnCivitaiAction) {
creatorActionItems.push(indentMarkup(viewOnCivitaiAction, 24));
}
if (viewOnHuggingFaceAction) {
creatorActionItems.push(indentMarkup(viewOnHuggingFaceAction, 24));
}
if (creatorInfoAction) {
creatorActionItems.push(indentMarkup(creatorInfoAction, 24));
}
@@ -869,6 +877,11 @@ function setupEventHandlers(filePath, modelType) {
case 'view-civitai':
openCivitai(target.dataset.filepath);
break;
case 'view-huggingface':
if (target.dataset.hfUrl) {
window.open(target.dataset.hfUrl, '_blank', 'noopener,noreferrer');
}
break;
case 'view-creator':
const username = target.dataset.username;
if (username) {

View File

@@ -1,5 +1,5 @@
import { modalManager } from './ModalManager.js';
import { showToast } from '../utils/uiHelpers.js';
import { showToast, setupAutoNewlineOnPaste } from '../utils/uiHelpers.js';
import { translate } from '../utils/i18nHelpers.js';
import { WS_ENDPOINTS } from '../api/apiConfig.js';
import { getStorageItem, setStorageItem } from '../utils/storageHelpers.js';
@@ -43,6 +43,9 @@ export class BatchImportManager {
setStorageItem('batch_import_skip_no_metadata', e.target.checked);
});
}
// Auto-append newline after pasting a URL in the batch URL input
setupAutoNewlineOnPaste('batchUrlInput');
}
/**

View File

@@ -1,5 +1,5 @@
import { modalManager } from './ModalManager.js';
import { showToast } from '../utils/uiHelpers.js';
import { showToast, setupAutoNewlineOnPaste } from '../utils/uiHelpers.js';
import { state } from '../state/index.js';
import { LoadingManager } from './LoadingManager.js';
import { getModelApiClient, resetAndReload } from '../api/modelApiFactory.js';
@@ -7,6 +7,7 @@ import { getStorageItem, setStorageItem } from '../utils/storageHelpers.js';
import { FolderTreeManager } from '../components/FolderTreeManager.js';
import { translate } from '../utils/i18nHelpers.js';
import { extractCivitaiModelUrlParts } from '../utils/civitaiUtils.js';
import { formatFileSize } from '../utils/formatters.js';
export class DownloadManager {
constructor() {
@@ -27,6 +28,10 @@ export class DownloadManager {
this.isBatchMode = false;
this.editingBatchIndex = -1;
// HF download state
this.hfRepoId = null;
this.hfSelectedFiles = [];
this.loadingManager = new LoadingManager();
this.folderTreeManager = new FolderTreeManager();
this.folderClickHandler = null;
@@ -44,6 +49,8 @@ export class DownloadManager {
this.handleToggleDefaultPath = this.toggleDefaultPath.bind(this);
this.handleBackToUrlFromBatch = this.backToUrlFromBatch.bind(this);
this.handleNextFromBatch = this.nextFromBatch.bind(this);
}
showDownloadModal() {
@@ -99,6 +106,9 @@ export class DownloadManager {
// Default path toggle handler
document.getElementById('useDefaultPath').addEventListener('change', this.handleToggleDefaultPath);
// Auto-append newline after pasting a URL so users can paste multiple URLs in succession
setupAutoNewlineOnPaste('modelUrl');
}
updateModalLabels() {
@@ -160,6 +170,10 @@ export class DownloadManager {
// Reset default path toggle
this.loadDefaultPathSetting();
// Reset HF state
this.hfRepoId = null;
this.hfSelectedFiles = [];
}
async retrieveVersionsForModel(modelId, source = null) {
@@ -180,6 +194,29 @@ export class DownloadManager {
return;
}
// Detect URL types — all URLs must share the same source type
const urlTypes = urls.map(u => DownloadManager.detectUrlType(u));
const isHf = urlTypes.every(t => t && (t.type === 'hf-resolve' || t.type === 'hf-repo'));
const isCivitai = urlTypes.every(t => t && t.type === 'civitai');
if (!isHf && !isCivitai) {
const allValid = urlTypes.every(t => t !== null);
if (!allValid) {
errorElement.textContent = translate('modals.download.errors.invalidUrl');
return;
}
// Mixed sources not supported in one batch
if (urls.length > 1) {
errorElement.textContent = translate('modals.download.errors.mixedSources');
return;
}
}
if (isHf) {
return this._validateAndFetchHf(urls, errorElement);
}
// --- Original CivitAI flow below ---
if (urls.length === 1) {
this.isBatchMode = false;
try {
@@ -271,6 +308,112 @@ export class DownloadManager {
this.showBatchPreviewStep();
}
// ---- Hugging Face download flow ----
async _validateAndFetchHf(urls, errorElement) {
if (urls.length === 1) {
const info = DownloadManager.detectUrlType(urls[0]);
// Direct file resolve URL → skip file selection, go to location
if (info.type === 'hf-resolve') {
this.isBatchMode = false;
this.hfRepoId = info.repo;
this.hfSelectedFiles = [info.filename];
this.source = 'huggingface';
this.proceedToLocation();
return;
}
// Repo URL → fetch file list and convert to batch items
try {
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingRepoFiles'));
const files = await this.apiClient.fetchHfRepoFiles(info.repo);
if (!files || files.length === 0) {
throw new Error(translate('modals.download.errors.noModelFiles'));
}
this.isBatchMode = true;
this.batchModels = [];
this.source = 'huggingface';
for (const file of files) {
this.batchModels.push({
url: urls[0],
source: 'huggingface',
repo: info.repo,
filename: file.filename,
revision: 'main',
displayName: file.filename,
fileSizeBytes: file.size,
selectedVersion: true,
versions: [],
checked: false,
error: null,
});
}
this.showBatchPreviewStep();
} catch (err) {
errorElement.textContent = err.message;
} finally {
this.loadingManager.hide();
}
return;
}
// Multiple HF URLs → batch mode: flatten all files from all repos
this.isBatchMode = true;
this.batchModels = [];
this.source = 'huggingface';
this.loadingManager.showSimpleLoading(translate('modals.download.fetchingRepoFiles'));
for (const url of urls) {
const info = DownloadManager.detectUrlType(url);
if (!info) {
this.batchModels.push({ url, error: 'Invalid URL', versions: [], selectedVersion: null });
continue;
}
if (info.type === 'hf-resolve') {
this.batchModels.push({
url,
source: 'huggingface',
repo: info.repo,
filename: info.filename,
revision: info.revision || 'main',
displayName: info.filename,
selectedVersion: true,
versions: [],
checked: false,
error: null,
});
} else if (info.type === 'hf-repo') {
try {
const files = await this.apiClient.fetchHfRepoFiles(info.repo);
if (!files || files.length === 0) {
this.batchModels.push({ url, error: 'No model files found', versions: [], selectedVersion: null });
continue;
}
// Flatten: create one batch item per file, all checked by default
for (const file of files) {
this.batchModels.push({
url,
source: 'huggingface',
repo: info.repo,
filename: file.filename,
revision: 'main',
displayName: file.filename,
fileSizeBytes: file.size,
selectedVersion: true,
versions: [],
checked: false,
error: null,
});
}
} catch (err) {
this.batchModels.push({ url, error: err.message, versions: [], selectedVersion: null });
}
}
}
this.loadingManager.hide();
this.showBatchPreviewStep();
}
async fetchVersionsForCurrentModel() {
const errorElement = document.getElementById('urlError');
if (errorElement) {
@@ -311,6 +454,60 @@ export class DownloadManager {
return { modelId: null, modelVersionId: null, source: null };
}
/**
* Detect the source type of a download URL.
* @param {string} url
* @returns {{ type: string, repo?: string, filename?: string, revision?: string } | null}
* type: 'civitai' | 'civarchive' | 'hf-resolve' | 'hf-repo' | 'direct-http'
*/
static detectUrlType(url) {
const trimmed = url.trim();
if (!trimmed) return null;
// CivitAI — matches civitai.com, civitai.red, civitai.green, etc.
if (/civitai\.(?:com|red|green)\/models\//i.test(trimmed) || /civitaiarchive|civarchive/i.test(trimmed)) {
// Will be parsed by existing CivitAI logic
return { type: 'civitai' };
}
// Hugging Face resolve URL → direct file
const hfResolveMatch = trimmed.match(/huggingface\.co\/([^/\s]+\/[^/\s]+)\/resolve\/([^/\s]+)\/(.+)/i);
if (hfResolveMatch) {
return {
type: 'hf-resolve',
repo: hfResolveMatch[1],
revision: hfResolveMatch[2],
filename: hfResolveMatch[3],
};
}
// Hugging Face repo URL (huggingface.co/user/repo or bare user/repo path)
// Require huggingface.co prefix for full URLs; bare user/repo only without ://
const hfRepoMatch = trimmed.match(
trimmed.includes('://')
? /^https?:\/\/huggingface\.co\/([a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+)(?:\/?$|$)/
: /^([a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+)$/
);
if (hfRepoMatch) {
// Reject path-traversal patterns like "../.." or "user/.."
const parts = hfRepoMatch[1].split('/');
if (parts.some(p => p === '.' || p === '..')) {
return null;
}
return {
type: 'hf-repo',
repo: hfRepoMatch[1],
};
}
// Direct HTTP(S) URL (non-HF)
if (/^https?:\/\//i.test(trimmed)) {
return { type: 'direct-http' };
}
return null;
}
extractModelId(url) {
const result = DownloadManager.parseModelUrl(url);
this.modelVersionId = result.modelVersionId;
@@ -559,8 +756,8 @@ export class DownloadManager {
return;
}
// In single-URL mode, validate version selection
if (!this.isBatchMode) {
// In single-URL mode, validate version selection (skip for HF)
if (!this.isBatchMode && this.source !== 'huggingface') {
if (!this.currentVersion) {
showToast('toast.loras.pleaseSelectVersion', {}, 'error');
return;
@@ -784,6 +981,77 @@ export class DownloadManager {
}
}
async _downloadHfSingle({ modelRoot, targetFolder, useDefaultPaths }) {
modalManager.closeModal('downloadModal');
this.loadingManager.restoreProgressBar();
const totalFiles = this.hfSelectedFiles.length;
const updateProgress = this.loadingManager.showDownloadProgress(totalFiles);
try {
let completedDownloads = 0;
for (let i = 0; i < totalFiles; i++) {
const filename = this.hfSelectedFiles[i];
updateProgress(0, completedDownloads, filename);
this.loadingManager.setStatus(`Downloading ${filename}...`);
const downloadId = Date.now().toString() + '_' + i;
const wsProtocol = window.location.protocol === 'https:' ? 'wss://' : 'ws://';
const ws = new WebSocket(`${wsProtocol}${window.location.host}/ws/download-progress?id=${downloadId}`);
try {
await new Promise((resolve, reject) => {
ws.onopen = resolve;
ws.onerror = reject;
});
// Capture completed count at WS creation time so progress
// updates arriving after completedDownloads increments still
// show the correct "N / total" position.
const snapshotCompleted = completedDownloads;
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.status === 'progress') {
const metrics = {
bytesDownloaded: data.bytes_downloaded,
totalBytes: data.total_bytes,
bytesPerSecond: data.bytes_per_second,
};
updateProgress(data.progress, snapshotCompleted, filename, metrics);
}
};
const response = await this.apiClient.downloadHfModel({
repo: this.hfRepoId,
filename,
revision: 'main',
modelRoot,
relativePath: targetFolder,
useDefaultPaths,
download_id: downloadId,
});
if (response?.success) {
completedDownloads++;
updateProgress(100, completedDownloads, filename);
}
} finally {
ws.close();
}
}
showToast('toast.loras.downloadCompleted', {}, 'success');
// Reload page data — model is already in scanner cache via backend
await resetAndReload(true);
return true;
} catch (error) {
console.error('Failed to download HF model:', error);
showToast('toast.downloads.downloadError', { message: error?.message }, 'error');
return false;
} finally {
this.loadingManager.hide();
}
}
updatePathSelectionUI() {
const manualSelection = document.getElementById('manualPathSelection');
@@ -812,13 +1080,19 @@ export class DownloadManager {
document.querySelectorAll('.download-step').forEach(step => step.style.display = 'none');
document.getElementById('batchPreviewStep').style.display = 'block';
const validCount = this.batchModels.filter(m => !m.error && m.selectedVersion).length;
const validCount = this.batchModels.filter(m => {
if (m.error) return false;
if (m.source === 'huggingface') return m.checked !== false;
return m.selectedVersion;
}).length;
document.getElementById('downloadModalTitle').textContent =
translate('modals.download.titleWithType', { type: this.apiClient.apiConfig.config.displayName }) +
` (${validCount})`;
const list = document.getElementById('batchPreviewList');
list.innerHTML = this.batchModels.map((item, index) => {
const hasHfItems = this.batchModels.some(m => m.source === 'huggingface' && !m.error);
let itemsHtml = this.batchModels.map((item, index) => {
if (item.error) {
return `
<div class="batch-preview-item batch-preview-error" data-index="${index}">
@@ -837,6 +1111,30 @@ export class DownloadManager {
}
const ver = item.selectedVersion;
// HF batch item rendering with checkbox
if (item.source === 'huggingface') {
const hfSize = item.fileSizeBytes
? formatFileSize(item.fileSizeBytes)
: '?';
return `
<div class="batch-preview-item" data-index="${index}">
<input type="checkbox" class="batch-preview-checkbox"
data-index="${index}" ${item.checked !== false ? 'checked' : ''} />
<div class="batch-preview-info">
<div class="batch-preview-name">${item.displayName || item.filename || `HF #${index}`} <span class="hf-badge">HF</span></div>
<div class="batch-preview-meta">
<span>${hfSize}</span>
<span>${item.repo || ''}</span>
</div>
</div>
<button class="batch-preview-remove" data-index="${index}" title="${translate('common.actions.remove', {}, 'Remove')}">
<i class="fas fa-times"></i>
</button>
</div>
`;
}
const firstImage = ver?.images?.find(img => !img.url.endsWith('.mp4'));
const thumbnailUrl = firstImage ? firstImage.url : '/loras_static/images/no-preview.png';
const fileSize = ver?.modelSizeKB
@@ -866,6 +1164,21 @@ export class DownloadManager {
`;
}).join('');
// Prepend select-all toolbar if there are HF items with checkboxes
if (hasHfItems) {
const allChecked = this.batchModels
.filter(m => m.source === 'huggingface' && !m.error)
.every(m => m.checked !== false);
itemsHtml = `
<div class="batch-preview-select-all">
<input type="checkbox" id="batchSelectAll" ${allChecked ? 'checked' : ''} />
<label for="batchSelectAll">${translate('modals.download.selectAll', {}, 'Select All')}</label>
</div>
` + itemsHtml;
}
list.innerHTML = itemsHtml;
list.onclick = (e) => {
const removeBtn = e.target.closest('.batch-preview-remove');
if (removeBtn) {
@@ -881,6 +1194,59 @@ export class DownloadManager {
}
};
// Checkbox handler for HF batch items
const checkboxes = list.querySelectorAll('.batch-preview-checkbox');
checkboxes.forEach(cb => {
cb.addEventListener('change', (e) => {
const idx = parseInt(e.target.dataset.index);
if (this.batchModels[idx]) {
this.batchModels[idx].checked = e.target.checked;
}
// Update valid count in title and Next button
const checkedCount = this.batchModels.filter(
m => !m.error && m.checked !== false
).length;
document.getElementById('downloadModalTitle').textContent =
translate('modals.download.titleWithType', { type: this.apiClient.apiConfig.config.displayName }) +
` (${checkedCount})`;
const nextBtn = document.getElementById('nextFromBatchBtn');
nextBtn.disabled = checkedCount === 0;
nextBtn.classList.toggle('disabled', checkedCount === 0);
// Update select-all checkbox state
const selectAll = document.getElementById('batchSelectAll');
if (selectAll) {
const hfItems = this.batchModels.filter(m => m.source === 'huggingface' && !m.error);
selectAll.checked = hfItems.length > 0 && hfItems.every(m => m.checked !== false);
}
});
});
// Select-all handler
const selectAll = document.getElementById('batchSelectAll');
if (selectAll) {
selectAll.addEventListener('change', (e) => {
const checked = e.target.checked;
const hfCheckboxes = list.querySelectorAll('.batch-preview-checkbox');
hfCheckboxes.forEach(cb => {
cb.checked = checked;
const idx = parseInt(cb.dataset.index);
if (this.batchModels[idx]) {
this.batchModels[idx].checked = checked;
}
});
// Update valid count in title and Next button
const checkedCount = this.batchModels.filter(
m => !m.error && m.checked !== false
).length;
document.getElementById('downloadModalTitle').textContent =
translate('modals.download.titleWithType', { type: this.apiClient.apiConfig.config.displayName }) +
` (${checkedCount})`;
const nextBtn = document.getElementById('nextFromBatchBtn');
nextBtn.disabled = checkedCount === 0;
nextBtn.classList.toggle('disabled', checkedCount === 0);
});
}
const nextBtn = document.getElementById('nextFromBatchBtn');
nextBtn.disabled = validCount === 0;
nextBtn.classList.toggle('disabled', validCount === 0);
@@ -903,7 +1269,12 @@ export class DownloadManager {
}
nextFromBatch() {
const validModels = this.batchModels.filter(m => !m.error && m.selectedVersion);
// For HF items, respect the checked flag; for CivitAI items, use selectedVersion
const validModels = this.batchModels.filter(m => {
if (m.error) return false;
if (m.source === 'huggingface') return m.checked !== false;
return m.selectedVersion;
});
if (validModels.length === 0) return;
this.proceedToLocation();
}
@@ -953,6 +1324,15 @@ export class DownloadManager {
targetFolder = this.folderTreeManager.getSelectedPath();
}
if (!this.isBatchMode) {
// Single-item download
if (this.source === 'huggingface') {
return this._downloadHfSingle({
modelRoot,
targetFolder,
useDefaultPaths,
});
}
const fileParams = this.selectedFile ? {
type: this.selectedFile.type || 'Model',
format: this.selectedFile.metadata?.format || 'SafeTensor',
@@ -974,7 +1354,13 @@ export class DownloadManager {
}
// Batch download mode
const downloadItems = this.batchModels.filter(m => !m.error && m.selectedVersion && !m.selectedVersion.existsLocally);
const downloadItems = this.batchModels.filter(m => {
if (m.error) return false;
if (!m.selectedVersion) return false;
// HF items have selectedVersion as a boolean marker + checked flag
if (m.source === 'huggingface') return m.checked !== false;
return !m.selectedVersion.existsLocally;
});
if (downloadItems.length === 0) {
showToast('toast.loras.downloadCompleted', {}, 'info');
modalManager.closeModal('downloadModal');
@@ -999,7 +1385,7 @@ export class DownloadManager {
if (data.status === 'progress' && data.download_id?.startsWith(batchDownloadId)) {
const current = downloadItems[completedDownloads + failedDownloads];
const name = current?.selectedVersion?.name || `#${completedDownloads + failedDownloads + 1}`;
const name = current?.selectedVersion?.name || current?.displayName || current?.filename || `#${completedDownloads + failedDownloads + 1}`;
const metrics = {
bytesDownloaded: data.bytes_downloaded,
totalBytes: data.total_bytes,
@@ -1016,22 +1402,59 @@ export class DownloadManager {
for (let i = 0; i < downloadItems.length; i++) {
const item = downloadItems[i];
const ver = item.selectedVersion;
const name = ver?.name || `Model #${item.modelId}`;
const name = item.displayName || item.filename || (item.selectedVersion?.name || `Model #${item.modelId}`);
const isHf = item.source === 'huggingface';
updateProgress(0, completedDownloads, name);
loadingManager.setStatus(`${i + 1}/${downloadItems.length}: ${name}`);
try {
const response = await this.apiClient.downloadModel(
item.modelId,
ver.id,
modelRoot,
targetFolder,
useDefaultPaths,
batchDownloadId,
item.source
);
let response;
if (isHf) {
// Per-file WebSocket for real-time progress
const downloadId = Date.now().toString() + '_hf_' + i;
const wsHf = new WebSocket(`${wsProtocol}${window.location.host}/ws/download-progress?id=${downloadId}`);
try {
await new Promise((resolve, reject) => {
wsHf.onopen = resolve;
wsHf.onerror = reject;
});
const snapshotCompleted = completedDownloads;
wsHf.onmessage = (event) => {
const data = JSON.parse(event.data);
if (data.status === 'progress') {
const metrics = {
bytesDownloaded: data.bytes_downloaded,
totalBytes: data.total_bytes,
bytesPerSecond: data.bytes_per_second,
};
updateProgress(data.progress, snapshotCompleted, name, metrics);
}
};
response = await this.apiClient.downloadHfModel({
repo: item.repo,
filename: item.filename,
revision: item.revision || 'main',
modelRoot,
relativePath: targetFolder,
useDefaultPaths,
download_id: downloadId,
});
} finally {
wsHf.close();
}
} else {
response = await this.apiClient.downloadModel(
item.modelId,
item.selectedVersion.id,
modelRoot,
targetFolder,
useDefaultPaths,
batchDownloadId,
item.source
);
}
if (!response.success) {
failedDownloads++;

View File

@@ -70,6 +70,7 @@ export const BASE_MODELS = {
ERNIE_TURBO: "Ernie Turbo",
NUCLEUS: "Nucleus",
PONY_V7: "Pony V7",
KREA_2: "Krea 2",
// Default
UNKNOWN: "Other"
};
@@ -197,6 +198,7 @@ export const BASE_MODEL_ABBREVIATIONS = {
[BASE_MODELS.ERNIE]: 'ERNI',
[BASE_MODELS.ERNIE_TURBO]: 'ETRB',
[BASE_MODELS.NUCLEUS]: 'NUCL',
[BASE_MODELS.KREA_2]: 'KR2',
// Default
[BASE_MODELS.UNKNOWN]: 'OTH'
@@ -401,6 +403,7 @@ export const BASE_MODEL_CATEGORIES = {
BASE_MODELS.PIXART_A, BASE_MODELS.PIXART_E, BASE_MODELS.HUNYUAN_1,
BASE_MODELS.LUMINA, BASE_MODELS.KOLORS, BASE_MODELS.NOOBAI, BASE_MODELS.ANIMA,
BASE_MODELS.ERNIE, BASE_MODELS.ERNIE_TURBO, BASE_MODELS.NUCLEUS,
BASE_MODELS.KREA_2,
BASE_MODELS.UNKNOWN
]
};

View File

@@ -319,6 +319,15 @@ export function openCivitai(filePath) {
openCivitaiByMetadata(civitaiId, versionId, modelName);
}
/**
* Open a Hugging Face model page in a new tab
* @param {string} hfUrl - The Hugging Face URL
*/
export function openHuggingFace(hfUrl) {
if (!hfUrl) return;
window.open(hfUrl, '_blank', 'noopener,noreferrer');
}
/**
* Dynamically positions the search options panel and filter panel
* based on the current layout and folder tags container height
@@ -1473,3 +1482,40 @@ export async function openExampleImagesFolder(modelHash) {
return false;
}
}
/**
* Set up a paste handler on a textarea that automatically appends a newline
* after pasted content that looks like a URL (http/https). This lets users
* paste multiple URLs one after another without manually pressing Enter.
* @param {string} textareaId - The id of the textarea element
*/
export function setupAutoNewlineOnPaste(textareaId) {
const el = document.getElementById(textareaId);
if (!el || el.tagName !== 'TEXTAREA') return;
el.addEventListener('paste', (e) => {
const pastedText = (e.clipboardData || window.clipboardData).getData('text');
// Only apply to text that starts with http:// or https://
if (/^https?:\/\//.test(pastedText) && !pastedText.endsWith('\n')) {
e.preventDefault();
const start = el.selectionStart;
const end = el.selectionEnd;
const text = el.value;
const before = text.substring(0, start);
const after = text.substring(end);
// Append newline after the pasted URL
const modifiedText = pastedText + '\n';
el.value = before + modifiedText + after;
// Move cursor to just after the inserted text
const newCursorPos = start + modifiedText.length;
el.selectionStart = el.selectionEnd = newCursorPos;
// Trigger input event so any listeners stay in sync
el.dispatchEvent(new Event('input', { bubbles: true }));
}
// Non-URL text or text already ending with \n — let default paste happen
});
}

View File

@@ -14,7 +14,7 @@
<div class="error-message" id="urlError"></div>
<div class="input-hint">
<i class="fas fa-info-circle"></i>
<span>{{ t('modals.download.urlHint') }}</span>
<span id="urlHint">{{ t('modals.download.urlHint') }}</span>
</div>
</div>
<div class="modal-actions">

View File

@@ -0,0 +1,117 @@
import { describe, it, expect } from 'vitest';
import { DownloadManager } from '../../../static/js/managers/DownloadManager.js';
describe('DownloadManager.detectUrlType — HF URL detection', () => {
it('detects HF resolve URL with file', () => {
const result = DownloadManager.detectUrlType(
'https://huggingface.co/dx8152/Flux2-Klein-9B-Consistency/resolve/main/Flux2-Klein-9B-consistency-V2.safetensors'
);
expect(result).toEqual({
type: 'hf-resolve',
repo: 'dx8152/Flux2-Klein-9B-Consistency',
revision: 'main',
filename: 'Flux2-Klein-9B-consistency-V2.safetensors',
});
});
it('detects HF resolve URL with subdirectory file', () => {
const result = DownloadManager.detectUrlType(
'https://huggingface.co/user/repo/resolve/main/subdir/model.safetensors'
);
expect(result).toEqual({
type: 'hf-resolve',
repo: 'user/repo',
revision: 'main',
filename: 'subdir/model.safetensors',
});
});
it('detects HF repo URL (full URL)', () => {
const result = DownloadManager.detectUrlType(
'https://huggingface.co/dx8152/Flux2-Klein-9B-Consistency'
);
expect(result).toEqual({
type: 'hf-repo',
repo: 'dx8152/Flux2-Klein-9B-Consistency',
});
});
it('detects HF repo URL (bare user/repo)', () => {
const result = DownloadManager.detectUrlType('dx8152/Flux2-Klein-9B-Consistency');
expect(result).toEqual({
type: 'hf-repo',
repo: 'dx8152/Flux2-Klein-9B-Consistency',
});
});
it('detects HF repo URL with trailing slash', () => {
const result = DownloadManager.detectUrlType(
'https://huggingface.co/user/repo/'
);
expect(result).toEqual({
type: 'hf-repo',
repo: 'user/repo',
});
});
it('detects CivitAI URL', () => {
const result = DownloadManager.detectUrlType(
'https://civitai.com/models/123/some-model'
);
expect(result).toEqual({ type: 'civitai' });
});
it('detects CivitAI URL on civitai.red domain', () => {
const result = DownloadManager.detectUrlType(
'https://civitai.red/models/12345/my-model'
);
expect(result).toEqual({ type: 'civitai' });
});
it('detects CivitAI URL on civitai.green domain', () => {
const result = DownloadManager.detectUrlType(
'https://civitai.green/models/67890/another-model'
);
expect(result).toEqual({ type: 'civitai' });
});
it('detects CivArchive URL', () => {
const result = DownloadManager.detectUrlType(
'https://civarchive.com/models/456'
);
expect(result).toEqual({ type: 'civitai' });
});
it('detects direct HTTP URL', () => {
const result = DownloadManager.detectUrlType(
'https://example.com/file.zip'
);
expect(result).toEqual({ type: 'direct-http' });
});
it('returns null for invalid input', () => {
expect(DownloadManager.detectUrlType('')).toBeNull();
expect(DownloadManager.detectUrlType(' ')).toBeNull();
});
it('returns null for unrecognized path', () => {
expect(DownloadManager.detectUrlType('justrandomtext')).toBeNull();
});
it('prefers HF resolve over repo when both match', () => {
const result = DownloadManager.detectUrlType(
'https://huggingface.co/user/repo/resolve/main/file.safetensors'
);
expect(result?.type).toBe('hf-resolve');
});
it('prefers CivitAI over HF when both match', () => {
// CivitAI check comes first in detectUrlType
// This URL should be detected as CivitAI, not HF
const result = DownloadManager.detectUrlType(
'https://civitai.com/models/123?huggingface.co/test/repo'
);
expect(result?.type).toBe('civitai');
});
});

View File

@@ -201,6 +201,45 @@ def test_list_models_returns_formatted_items(mock_service, mock_scanner):
asyncio.run(scenario())
def test_list_models_filters_out_corrupted_entries(mock_service, mock_scanner):
"""Corrupted cache entries (format_response returns None) must not appear
in the response items nor cause a 500. See issue #730.
"""
mock_service.paginated_items = [
{"file_path": "/tmp/good.safetensors", "name": "Good"},
{"file_path": None, "name": "Corrupted"}, # triggers None from format_response
{"file_path": "/tmp/also_good.safetensors", "name": "AlsoGood"},
]
# Override format_response to return None for corrupted entries
original_format = mock_service.format_response
async def conditional_format(item):
if item.get("file_path") is None:
return None
return await original_format(item)
mock_service.format_response = conditional_format
async def scenario():
client = await create_test_client(mock_service)
try:
response = await client.get("/api/lm/test-models/list")
payload = await response.json()
assert response.status == 200
# Only the 2 non-corrupted entries should appear
assert len(payload["items"]) == 2
assert payload["items"][0]["name"] == "Good"
assert payload["items"][1]["name"] == "AlsoGood"
# None should never appear in the items list
assert None not in payload["items"]
finally:
await client.close()
asyncio.run(scenario())
def test_model_types_endpoint_returns_counts(mock_service, mock_scanner):
mock_service.model_types = [
{"type": "LoRa", "count": 3},

View File

@@ -59,3 +59,180 @@ async def test_get_nightly_version_network_error_logs_warning(monkeypatch, caplo
assert changelog == []
assert "Unable to reach GitHub for nightly version" in caplog.text
assert "Traceback" not in caplog.text
def test_clean_excludes_covers_user_data_dirs():
"""git clean must receive -e excludes for every user-managed dir."""
excludes = update_routes._clean_excludes()
assert "-e" in excludes # at least one exclude flag present
for name in update_routes._PRESERVE_DIRS:
assert name in excludes
assert f"{name}/**" in excludes
@pytest.mark.asyncio
async def test_perform_git_update_preserves_user_dirs(monkeypatch, tmp_path):
"""``git clean`` must be called with -e excludes for user data dirs.
Regression test for portable-mode updates wiping wildcards/, stats/,
backups/, etc. because ``git clean -fd`` removed untracked, non-ignored
directories.
"""
calls = []
class FakeGit:
def reset(self, *args, **kwargs):
calls.append(("reset", args))
def clean(self, *args, **kwargs):
calls.append(("clean", args))
def checkout(self, *args, **kwargs):
calls.append(("checkout", args))
class FakeRemote:
def fetch(self):
calls.append(("fetch", ()))
def pull(self, *args, **kwargs):
calls.append(("pull", args))
class FakeRemotes:
origin = FakeRemote()
class FakeCommit:
hexsha = "abcdef123456"
class FakeHeads:
def __getitem__(self, name):
class Head:
def checkout(self_inner):
calls.append(("head-checkout", (name,)))
return Head()
class FakeBranches:
names = ["main"]
def __iter__(self):
class B:
name = "main"
return iter([B()])
class FakeRepo:
def __init__(self, path):
calls.append(("repo", (path,)))
git = FakeGit()
remotes = FakeRemotes()
head = type("H", (), {"commit": FakeCommit()})()
branches = FakeBranches()
heads = FakeHeads()
def create_head(self, name, ref):
calls.append(("create_head", (name, ref)))
class FakeGitModule:
class Repo:
def __new__(cls, path):
return FakeRepo(path)
class exc:
class GitError(Exception):
pass
import builtins
real_import = builtins.__import__
def fake_import(name, *args, **kwargs):
if name == "git":
return FakeGitModule
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", fake_import)
success, version = await update_routes.UpdateRoutes._perform_git_update(
str(tmp_path), nightly=True
)
assert success is True
clean_calls = [c for c in calls if c[0] == "clean"]
assert len(clean_calls) == 1
clean_args = clean_calls[0][1]
# Every preserved dir must be excluded via -e
for name in update_routes._PRESERVE_DIRS:
assert name in clean_args, f"{name} missing from git clean excludes"
assert f"{name}/**" in clean_args, f"{name}/** missing from git clean excludes"
# Ensure there's an -e before each name occurrence
idx = clean_args.index(name)
assert clean_args[idx - 1] == "-e"
@pytest.mark.asyncio
async def test_perform_git_update_stable_preserves_user_dirs(monkeypatch, tmp_path):
"""Stable (tag) update path must also pass -e excludes to git clean."""
calls = []
class FakeGit:
def reset(self, *args, **kwargs):
calls.append(("reset", args))
def clean(self, *args, **kwargs):
calls.append(("clean", args))
def checkout(self, *args, **kwargs):
calls.append(("checkout", args))
class FakeRemote:
def fetch(self):
calls.append(("fetch", ()))
class FakeRemotes:
origin = FakeRemote()
class FakeCommit:
committed_datetime = "2026-01-01"
class FakeTag:
name = "v9.9.9"
commit = FakeCommit()
class FakeRepo:
def __init__(self, path):
calls.append(("repo", (path,)))
git = FakeGit()
remotes = FakeRemotes()
tags = [FakeTag()]
class FakeGitModule:
class Repo:
def __new__(cls, path):
return FakeRepo(path)
class exc:
class GitError(Exception):
pass
import builtins
real_import = builtins.__import__
def fake_import(name, *args, **kwargs):
if name == "git":
return FakeGitModule
return real_import(name, *args, **kwargs)
monkeypatch.setattr(builtins, "__import__", fake_import)
success, version = await update_routes.UpdateRoutes._perform_git_update(
str(tmp_path), nightly=False
)
assert success is True
assert version == "v9.9.9"
clean_calls = [c for c in calls if c[0] == "clean"]
assert len(clean_calls) == 1
clean_args = clean_calls[0][1]
for name in update_routes._PRESERVE_DIRS:
assert name in clean_args, f"{name} missing from git clean excludes (stable)"

View File

@@ -199,8 +199,107 @@ class TestEmbeddingServiceFormatResponse:
"from_civitai": True,
"civitai": {},
}
result = await embedding_service.format_response(embedding_data)
assert result["sub_type"] == "embedding"
assert "model_type" not in result # Removed in refactoring
class TestFormatResponseCorruptedEntries:
"""Test format_response handles corrupted cache entries gracefully (issue #730).
When cache rows have None/missing critical fields (e.g. from a partially
written or legacy DB), format_response must NOT raise KeyError/AttributeError.
Instead it returns None so the handler layer can filter the bad entry out
instead of failing the entire listing request.
"""
@pytest.fixture
def mock_scanner(self):
scanner = MagicMock()
scanner._hash_index = MagicMock()
return scanner
@pytest.fixture
def lora_service(self, mock_scanner):
return LoraService(mock_scanner)
@pytest.fixture
def checkpoint_service(self, mock_scanner):
return CheckpointService(mock_scanner)
@pytest.fixture
def embedding_service(self, mock_scanner):
return EmbeddingService(mock_scanner)
@pytest.mark.asyncio
async def test_lora_returns_none_on_missing_file_path(self, lora_service):
"""format_response returns None when file_path is missing (corrupted row)."""
lora_data = {
"model_name": "Test LoRA",
"file_name": "test_lora",
"file_path": None, # corrupted: missing file_path
"folder": "",
"sha256": "abc123",
"tags": [],
"from_civitai": True,
"civitai": {},
}
result = await lora_service.format_response(lora_data)
assert result is None
@pytest.mark.asyncio
async def test_lora_handles_none_model_name_gracefully(self, lora_service):
"""format_response should not crash when model_name is None (legacy DB row)."""
lora_data = {
"model_name": None, # NULL from old DB row
"file_name": "test_lora",
"file_path": "/models/test_lora.safetensors",
"folder": "",
"sha256": "abc123",
"tags": [],
"from_civitai": True,
"civitai": {},
}
result = await lora_service.format_response(lora_data)
# Should not raise; model_name falls back to file_name
assert result is not None
assert result["model_name"] == "test_lora"
@pytest.mark.asyncio
async def test_checkpoint_returns_none_on_missing_file_path(self, checkpoint_service):
"""format_response returns None when file_path is missing (corrupted row)."""
checkpoint_data = {
"model_name": "Test",
"file_name": "test",
"file_path": "", # empty string == corrupted
"folder": "",
"sha256": "abc",
"tags": [],
"from_civitai": True,
"civitai": {},
"sub_type": "checkpoint",
}
result = await checkpoint_service.format_response(checkpoint_data)
assert result is None
@pytest.mark.asyncio
async def test_embedding_handles_none_fields_gracefully(self, embedding_service):
"""format_response should not crash when optional fields are None."""
embedding_data = {
"model_name": None,
"file_name": None,
"file_path": "/models/test.pt",
"folder": None,
"sha256": "abc",
"tags": [],
"from_civitai": True,
"civitai": {},
"sub_type": "embedding",
}
result = await embedding_service.format_response(embedding_data)
assert result is not None
assert result["file_path"] == "/models/test.pt"
# model_name falls back to file_name which falls back to ""
assert result["model_name"] == ""

View File

@@ -200,52 +200,97 @@ def _setup_storage_paths(tmp_path, monkeypatch):
return project_root, user_dir, user_settings_path
def _populate_cache(root_dir, marker_name, db_text):
cache_dir = root_dir / "model_cache"
cache_dir.mkdir(exist_ok=True)
marker_file = cache_dir / marker_name
marker_file.write_text(marker_name, encoding="utf-8")
(root_dir / "model_cache.sqlite").write_text(db_text, encoding="utf-8")
def _populate_settings_dir(root_dir):
"""Create test data for all managed subdirectories under a settings directory."""
(root_dir / "cache" / "symlink").mkdir(parents=True, exist_ok=True)
(root_dir / "cache" / "symlink" / "symlink_map.json").write_text(
'{"migrated": true}', encoding="utf-8"
)
(root_dir / "backups").mkdir(parents=True, exist_ok=True)
(root_dir / "backups" / "backup_test.zip").write_text(
"backup", encoding="utf-8"
)
(root_dir / "logs").mkdir(parents=True, exist_ok=True)
(root_dir / "logs" / "session.log").write_text("log", encoding="utf-8")
(root_dir / "stats").mkdir(parents=True, exist_ok=True)
(root_dir / "stats" / "stats.json").write_text(
'{"stats": true}', encoding="utf-8"
)
(root_dir / "wildcards").mkdir(parents=True, exist_ok=True)
(root_dir / "wildcards" / "test.txt").write_text("wildcard", encoding="utf-8")
def test_switch_to_portable_mode_copies_cache(tmp_path, monkeypatch):
def test_switch_to_portable_mode_copies_subdirectories(tmp_path, monkeypatch):
project_root, user_dir, user_settings = _setup_storage_paths(tmp_path, monkeypatch)
_populate_cache(user_dir, "user_marker.txt", "user_db")
_populate_settings_dir(user_dir)
manager = SettingsManager()
manager.set("use_portable_settings", True)
assert manager.settings_file == str(project_root / "settings.json")
marker_copy = project_root / "model_cache" / "user_marker.txt"
assert marker_copy.read_text(encoding="utf-8") == "user_marker.txt"
assert (project_root / "model_cache.sqlite").read_text(
# Managed subdirectories should all be migrated
assert (
project_root / "cache" / "symlink" / "symlink_map.json"
).read_text(encoding="utf-8") == '{"migrated": true}'
assert (
project_root / "backups" / "backup_test.zip"
).read_text(encoding="utf-8") == "backup"
assert (project_root / "logs" / "session.log").read_text(
encoding="utf-8"
) == "user_db"
) == "log"
assert (project_root / "stats" / "stats.json").read_text(
encoding="utf-8"
) == '{"stats": true}'
assert (project_root / "wildcards" / "test.txt").read_text(
encoding="utf-8"
) == "wildcard"
assert user_settings.exists()
def test_switching_back_to_user_config_moves_cache(tmp_path, monkeypatch):
def test_switching_back_to_user_config_moves_subdirectories(tmp_path, monkeypatch):
project_root, user_dir, user_settings = _setup_storage_paths(tmp_path, monkeypatch)
_populate_cache(user_dir, "user_marker.txt", "user_db")
_populate_settings_dir(user_dir)
manager = SettingsManager()
manager.set("use_portable_settings", True)
project_cache_dir = project_root / "model_cache"
project_cache_dir.mkdir(exist_ok=True)
(project_cache_dir / "project_marker.txt").write_text(
"project_marker", encoding="utf-8"
# Populate project-root managed subdirectories
(project_root / "cache" / "model").mkdir(parents=True, exist_ok=True)
(project_root / "cache" / "model" / "default.sqlite").write_text(
"project_db", encoding="utf-8"
)
(project_root / "backups" / "project_backup.zip").write_text(
"project_backup", encoding="utf-8"
)
(project_root / "logs" / "project.log").write_text(
"project_log", encoding="utf-8"
)
(project_root / "stats" / "project_stats.json").write_text(
'{"project": true}', encoding="utf-8"
)
(project_root / "wildcards" / "project.txt").write_text(
"project_wildcard", encoding="utf-8"
)
(project_root / "model_cache.sqlite").write_text("project_db", encoding="utf-8")
manager.set("use_portable_settings", False)
assert manager.settings_file == str(user_settings)
assert (user_dir / "model_cache" / "project_marker.txt").read_text(
assert (user_dir / "cache" / "model" / "default.sqlite").read_text(
encoding="utf-8"
) == "project_marker"
assert (user_dir / "model_cache.sqlite").read_text(encoding="utf-8") == "project_db"
) == "project_db"
assert (user_dir / "backups" / "project_backup.zip").read_text(
encoding="utf-8"
) == "project_backup"
assert (user_dir / "logs" / "project.log").read_text(
encoding="utf-8"
) == "project_log"
assert (user_dir / "stats" / "project_stats.json").read_text(
encoding="utf-8"
) == '{"project": true}'
assert (user_dir / "wildcards" / "project.txt").read_text(
encoding="utf-8"
) == "project_wildcard"
def test_download_path_template_parses_json_string(manager):