alphard/ComfyUI-Lora-Manager
1
0
Fork 0
mirror of https://github.com/willmiao/ComfyUI-Lora-Manager.git synced 2026-09-21 03:01:27 -03:00
Code Issues Packages Projects Releases Wiki Activity

feat(links): support ModelScope and TensorArt as model sources

Browse Source 
A model file could only ever be linked to huggingface.co: `set_hf_url`
validated the URL with a huggingface-only regex, the agent fetched the card
from a hardcoded HF URL, and the readme processor built every relative image
path off `https://huggingface.co/{repo}/resolve/main`. ModelScope publishes the
same model-card convention (README.md + YAML frontmatter, often carrying
`base_model:` and `trigger_words:`) behind a public, key-less API, so the
enrichment pipeline could already serve it - it was the plumbing that was
HF-shaped, not the idea.

Make the external source a first-class, provider-driven concept:

- New `py/services/model_sources/` registry. A `ModelSource` owns URL
  recognition (lenient for stored values, strict for user input), the
  canonical page URL, model-card fetching, the asset base URL and the
  capability flags. `HuggingFaceSource` is the previous logic relocated;
  `ModelScopeSource` reads `/models/{o}/{n}/resolve/{master|main}/README.md`
  and falls back to `/api/v1/models/{o}/{n}/repo`. `TensorArtSource` is
  link-only on purpose: tensor.art answers plain HTTP clients with a
  Cloudflare challenge and its internal API (ap-east-1.tensorart.cloud /
  cn.tensorart.net) rejects every /v1/model/* route with "invalid
  authorization header", so it declares supports_enrichment=False rather than
  failing silently later.
- Metadata gains `source_platform` + `source_url`; `hf_url` stays as a
  read/write alias, written only for Hugging Face, so existing sidecars,
  cached rows and third-party consumers keep working. Normalisation runs at
  the scanner, the persistent cache (both directions, plus two new columns
  behind an ALTER migration) and the linking handler - which is what stops a
  user who switches sources from leaving a stale `hf_url` on a ModelScope
  model.
- The agent pipeline keys off the provider instead of `hf_url`: the fast-fail
  gate now explains *why* a model is skipped (no source / unknown source /
  source without a reachable card), the prompt context exposes
  source_url/source_id/source_label/asset_base_url while still filling the
  legacy hf_url/repo aliases, and the four README image extractors take a
  base_url (defaulting to HF) so relative paths resolve against the right
  site. Version grouping generalises to hf: / ms: / ta: keys.
- `POST /api/lm/set-hf-url` keeps its path and its legacy payload keys but
  accepts `source_url`, validates against every provider and returns the
  platform. `GET /api/lm/model-sources` lets the UI render the supported-site
  list from the server.
- Frontend: a `modelSourceHelpers` mirror of the registry drives the link
  dialog, the card/modal globe (branded "View on ModelScope/TensorArt"), the
  version-group key and the enrichment gate; the versions tab no longer sends
  ms:/ta: keys to the CivitAI API.

TensorArt stays in the list because provenance is worth keeping even when the
card is unreadable - the dialog says so plainly ("Sites that don't expose one
(currently TensorArt) can only be linked") and the context menu disables
enrichment with a matching tooltip, instead of the user getting
"Unsupported URL".

Verified against the real ModelScope API: jj3550945163/Krea-2-LORA returns a
1882-byte card whose frontmatter carries base_model/tags/trigger_words, and
relative images resolve to .../resolve/master/....

Tests: backend 2815 passed; frontend 1130 JS + 91 Vue passed; pytest
tests/i18n and a Jinja compile pass over templates/. The nine locales carry
[TODO: Translate] for the new strings, completed in the next commit.
This commit is contained in:
Will Miao
2026-09-14 07:24:08 +08:00
parent 84146b62fd
commit 5ab0e88abc
51 changed files with 2518 additions and 279 deletions
Download Patch File Download Diff File Expand all files Collapse all files
py/routes/handlers/hf_handlers.py
+97 -21
View File
@@ -22,6 +22,11 @@ from ...services.downloader import (
get_downloader,
)
from ...services.aria2_downloader import Aria2Downloader
from ...services.model_sources import (
detect_source,
list_sources,
normalize_metadata_source,
)
from ...services.settings_manager import get_settings_manager
from ...services.service_registry import ServiceRegistry
from ...services.websocket_manager import ws_manager
@@ -120,6 +125,8 @@ async def _save_hf_metadata(dest_path: str, repo: str, model_root: str) -> None:
# 2. Overlay HF-specific fields
metadata._unknown_fields["hf_url"] = hf_url
metadata._unknown_fields["source_url"] = hf_url
metadata._unknown_fields["source_platform"] = "huggingface"
metadata.from_civitai = False # HF models are not from CivitAI
# 3. Save metadata atomically
@@ -189,27 +196,72 @@ async def _add_to_scanner_cache(dest_path: str, metadata: dict[str, Any]) -> Non
class HfHandler:
"""Handle Hugging Face model browsing and download."""
async def get_model_sources(self, request: web.Request) -> web.Response:
"""List the external model sites the UI can link a model to.
Used by the "Link Model" dialog to validate URLs client-side and to
explain which sites support AI metadata enrichment.
"""
return web.json_response([
{
"platform": source.platform,
"label": source.label,
"supports_enrichment": source.supports_enrichment,
"supports_download": source.supports_download,
"example_url": source.canonical_url(
"user/repo" if source.platform != "tensorart" else "827823520299086029"
),
}
for source in list_sources()
])
async def set_hf_url(self, request: web.Request) -> web.Response:
"""Link a model file to its page on an external model site.
Accepts ``source_url`` (preferred) or the legacy ``hf_url`` /
``url`` payload key. Hugging Face, ModelScope, and TensorArt URLs
are recognised; the platform is stored alongside the canonical URL.
TensorArt models can be linked and browsed, but not AI-enriched.
"""
try:
payload: dict[str, Any] = await request.json()
except json.JSONDecodeError:
return web.json_response({"success": False, "error": "Invalid JSON"}, status=400)
file_path = (payload.get("file_path") or "").strip()
hf_url = (payload.get("hf_url") or "").strip()
raw_url = (
payload.get("source_url")
or payload.get("hf_url")
or payload.get("url")
or ""
)
source_url = raw_url.strip() if isinstance(raw_url, str) else ""
if not file_path or not hf_url:
return web.json_response(
{"success": False, "error": "Missing required fields: 'file_path' and 'hf_url'"},
status=400,
)
m = re.match(r"^https?://huggingface\.co/([^/]+/[^/]+)/?$", hf_url)
if not m:
if not file_path or not source_url:
return web.json_response(
{
"success": False,
"error": "Invalid HuggingFace URL. Expected format: https://huggingface.co/user/repo",
"error": "Missing required fields: 'file_path' and 'source_url'",
},
status=400,
)
ref = detect_source(source_url, strict=True)
if ref is None:
return web.json_response(
{
"success": False,
"error": (
"Unsupported model URL. Supported formats: "
+ ", ".join(
f"{s.label} ({s.canonical_url('user/repo')})"
if s.platform != "tensorart"
else f"{s.label} (https://tensor.art/models/<id>)"
for s in list_sources()
)
),
},
status=400,
)
@@ -225,37 +277,61 @@ class HfHandler:
return web.json_response(
{
"success": False,
"error": "File is not within any configured model directory. Cannot link to HuggingFace.",
"error": "File is not within any configured model directory. Cannot link to a model source.",
},
status=400,
)
try:
existing = await MetadataManager.load_metadata_payload(file_path)
if existing.get("hf_url") == hf_url:
already_linked = (
(existing.get("source_url") or "").strip() == ref.url
and (existing.get("source_platform") or "").strip().lower()
== ref.platform
) or (
not existing.get("source_url")
and ref.platform == "huggingface"
and (existing.get("hf_url") or "").strip() == ref.url
)
if already_linked:
return web.json_response({
"success": True,
"message": "hf_url already set",
"hf_url": hf_url,
"message": "source_url already set",
"source_url": ref.url,
"source_platform": ref.platform,
"hf_url": ref.url if ref.platform == "huggingface" else "",
})
existing["hf_url"] = hf_url
existing["source_url"] = ref.url
existing["source_platform"] = ref.platform
if ref.platform == "huggingface":
existing["hf_url"] = ref.url
else:
existing.pop("hf_url", None)
normalize_metadata_source(existing)
# NOTE: deliberately do NOT touch `from_civitai` here. It records
# where the metadata came from, and the UI must show the CivitAI
# link whenever CivitAI data is present — linking HuggingFace must
# not hide it (#1094). HF provenance is tracked via `hf_url`.
# link whenever CivitAI data is present — linking an external
# source must not hide it (#1094). Source provenance is tracked
# via `source_platform` / `source_url`.
await MetadataManager.save_metadata(file_path, existing)
await _add_to_scanner_cache(file_path, existing)
logger.info("Set hf_url=%s for %s", hf_url, file_path)
logger.info(
"Linked %s to %s source (%s)", file_path, ref.platform, ref.url
)
return web.json_response({
"success": True,
"message": f"hf_url set to {hf_url}",
"hf_url": hf_url,
"message": f"Linked to {ref.url}",
"source_url": ref.url,
"source_platform": ref.platform,
"hf_url": existing.get("hf_url", ""),
})
except Exception as exc:
logger.error("Failed to set hf_url for %s: %s", file_path, exc)
logger.error("Failed to link %s to a model source: %s", file_path, exc)
return web.json_response(
{"success": False, "error": str(exc)},
status=500,
py/routes/handlers/misc_handlers.py
+1
View File
@@ -4079,6 +4079,7 @@ class MiscHandlerSet:
"get_hf_repo_files": self.hf_handler.get_hf_repo_files,
"download_hf_model": self.hf_handler.download_hf_model,
"set_hf_url": self.hf_handler.set_hf_url,
"get_model_sources": self.hf_handler.get_model_sources,
# Agent skill handlers
"get_agent_skills": self.agent_handler.get_agent_skills,
"execute_agent_skill": self.agent_handler.execute_agent_skill,
py/routes/misc_route_registrar.py
+4
View File
@@ -113,6 +113,10 @@ MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition(
"POST", "/api/lm/set-hf-url", "set_hf_url"
),
# Supported external model sites (Hugging Face / ModelScope / TensorArt)
RouteDefinition(
"GET", "/api/lm/model-sources", "get_model_sources"
),
# Agent skill endpoints
RouteDefinition(
"GET", "/api/lm/agent/skills", "get_agent_skills"