feat(metadata): add OpenModelDB metadata provider and model source for upscalers

Add OpenModelDB (openmodeldb.info) as a metadata and download source for
the existing upscaler model type.

Metadata:
- New OpenModelDBClient: fetches the site's bulk JSON dumps, caches them
  on disk (24h TTL + ETag revalidation), and builds a local sha256 index
- New OpenModelDBModelMetadataProvider adapts catalogue entries to the
  CivitAI-shaped version dict contract; registered in the fallback chain
  behind the enable_openmodeldb_api setting (default on), gated to the
  upscaler sub-type so other model types never trigger the dump download
- Persisted provenance uses metadata_source "openmodeldb" plus a nested
  openmodeldb block (page URL, architecture, scale, license)

Images: paired-image LR/SR URLs are ephemeral imgdiff.net sessions, so
displayable images come from the site-hosted auto-generated thumbnails
(model-level cover leads images[], per-image thumbs for the rest); the
original comparison URL is kept in meta.comparisonUrl.

Downloads:
- New OpenModelDBSource (flat model ids, omdb: group prefix) with
  resource filename derivation that recovers names hidden mid-path
  (mediafire) or synthesizes {id}.{type} for folder links
- HTML-gateway mirrors (mediafire/mega/drive) are rejected with a clear
  manual-download hint instead of silently saving an HTML page as .pth
- ModelSource base gains is_valid_source_id / default_subdir_parts /
  resolve_download_url hooks so flat-id sources need no platform branches

UI: "View on OpenModelDB" link in the model modal (downloaded and
hash-enriched models), settings toggle next to the CivArchive one.
This commit is contained in:
Will Miao
2026-10-03 21:13:07 +08:00
parent 515469054c
commit 35f1ced41a
39 changed files with 2787 additions and 37 deletions
+38
View File
@@ -47,6 +47,7 @@ GROUP_PREFIXES: dict[str, str] = {
"modelscope": "ms",
"modelscope-ai": "msai",
"tensorart": "ta",
"openmodeldb": "omdb",
}
@@ -288,6 +289,11 @@ class ModelSource:
#: Sub-directory the "use default paths" template places downloads in.
default_subdir: str = ""
#: Source id used to build the example URL shown in UI copy and error
#: messages. ``owner/name`` suits repository sites; sites with a
#: different identity shape override it with a real example.
example_source_id: str = "user/repo"
#: Lenient pattern used to recognise URLs already stored in metadata.
#: Captures the site-specific source id in group ``id``.
url_pattern: re.Pattern[str] | None = None
@@ -340,6 +346,26 @@ class ModelSource:
raise NotImplementedError
def is_valid_source_id(self, source_id: str) -> bool:
"""Return ``True`` when *source_id* is a safe id on this site.
Defaults to the ``owner/name`` repository rule; sites whose ids are
not repositories (OpenModelDB's flat model ids) override it.
"""
return is_valid_source_id(source_id)
def default_subdir_parts(self, source_id: str) -> tuple[str, ...]:
"""Path segments appended to the model root by "use default paths".
Defaults to ``<default_subdir>/<owner>/<repo>`` so downloads from
repository sites stay namespaced by author. Sites without an
owner/repo split override it.
"""
owner, repo_name = source_id.split("/", 1)
return (self.default_subdir, owner, repo_name)
def asset_base_url(self, source_id: str, revision: str = "") -> str:
"""Base URL used to resolve repository-relative asset paths."""
@@ -432,6 +458,18 @@ class ModelSource:
f"{self.label or self.platform} does not support downloads", status=400
)
async def resolve_download_url(
self, source_id: str, filename: str, revision: str = ""
) -> str:
"""Resolve the download URL for one file, allowing async lookups.
Defaults to the synchronous :meth:`file_download_url`; sites whose
download URL is not derivable from the id alone (OpenModelDB stores
the URL inside its catalogue entry) override this to look it up.
"""
return self.file_download_url(source_id, filename, revision)
def resolve_revision(self, revision: str = "") -> str:
"""Return *revision*, falling back to this site's default branch."""