The post-processor stored the LLM's confidence as `_llm_confidence`, but
that value could never be read back: `BaseModelMetadata.from_dict()`
deliberately excludes underscore-prefixed keys from `_unknown_fields` and
`to_dict()` strips private fields, so it was erased by the next metadata
write and was invisible to `read_metadata()`. The enrichment evaluation
harness reads this field to score runs, so confidence was always scored
as blank.
Store it as `llm_confidence`, which round-trips as an ordinary unknown
field — the same mechanism `llm_enriched_at` already relies on. Nothing
else consumed the old name, and the harness still accepts it so sidecars
written by earlier versions keep evaluating.
Covered by a metadata load/save round-trip regression test plus
assertions that the post-processor writes the persisted key and no longer
writes the private one.
`_build_prompt_context()` was only reached when the LLM was configured,
so a user with no provider got nothing at all from a linked model source
— no preview, no example images, no author summary, no tags — even
though all of that is deterministic data from a public API.
Split the model-card fetch into `_load_source_card()`, which runs for
every source-backed enrichment, and have the post-processor apply its
result whether or not the LLM runs. The prompt is then built from the
already-fetched card rather than re-fetching it.
Invoking "Enrich Metadata with AI" still always calls the provider; a
model source supplying a description, images and tags is not treated as
a reason to skip it, since the LLM's summary and notes are richer and an
action that silently does not call out to the provider would be
unpredictable. The site data acts as a fallback for the gaps the LLM
leaves.
Add `base_model_resolver.resolve_base_model()` to map the site's own
names (`krea/Krea-2-Turbo`, `KREA_2_TURBO`) onto the canonical
vocabulary, used only when the LLM returns no base model. It is strictly
conservative — exact normalised matching plus a bounded set of variant
suffixes, and it only ever returns a name that is already in the
vocabulary — so an uncertain hint defers to the LLM instead of writing a
plausible-looking wrong value.
ModelScope's model card is not just README.md: the author's summary
(Description), the site-curated tags (OfficialTags), the internal
architecture enums (VisionFoundation/SubVisionFoundation) and — per
published version — the model filenames with that file's example images
(coverImages) and trigger words all live in the model-detail API.
AIGC repositories there frequently ship an auto-generated boilerplate
README and put the only useful text in Description, so reading just the
README yielded almost nothing.
Add `ModelSource.fetch_model_card_context()` returning a new
`ModelCardContext`, implemented by ModelScopeSource against the public
(no API key) detail endpoint. Example images are matched to the model's
basename through each version's `stats.fileList`, so every checkpoint in
a collection repository gets its own images rather than a sibling's.
Consume the context in the post-processor:
* example images seed `civitai.images` and, being per-file, take priority
in the preview fallback chain
* the author summary becomes a paragraph in `modelDescription` and fills
`civitai.description` when the LLM returns no short description
* site-curated tags are always merged in, which also fixes the official
`character-enhancement` being dropped by the prompt's no-hyphen rule
* per-file trigger words are used before the repo-wide YAML
`instance_prompt`
* an explicitly stated strength range is recovered by regex so
`usage_tips` is populated even without an LLM
The prompt gains a Site-Provided Metadata section so the LLM can prefer
the site's first-hand data over its own guesses.
A model file could only ever be linked to huggingface.co: `set_hf_url`
validated the URL with a huggingface-only regex, the agent fetched the card
from a hardcoded HF URL, and the readme processor built every relative image
path off `https://huggingface.co/{repo}/resolve/main`. ModelScope publishes the
same model-card convention (README.md + YAML frontmatter, often carrying
`base_model:` and `trigger_words:`) behind a public, key-less API, so the
enrichment pipeline could already serve it - it was the plumbing that was
HF-shaped, not the idea.
Make the external source a first-class, provider-driven concept:
- New `py/services/model_sources/` registry. A `ModelSource` owns URL
recognition (lenient for stored values, strict for user input), the
canonical page URL, model-card fetching, the asset base URL and the
capability flags. `HuggingFaceSource` is the previous logic relocated;
`ModelScopeSource` reads `/models/{o}/{n}/resolve/{master|main}/README.md`
and falls back to `/api/v1/models/{o}/{n}/repo`. `TensorArtSource` is
link-only on purpose: tensor.art answers plain HTTP clients with a
Cloudflare challenge and its internal API (ap-east-1.tensorart.cloud /
cn.tensorart.net) rejects every /v1/model/* route with "invalid
authorization header", so it declares supports_enrichment=False rather than
failing silently later.
- Metadata gains `source_platform` + `source_url`; `hf_url` stays as a
read/write alias, written only for Hugging Face, so existing sidecars,
cached rows and third-party consumers keep working. Normalisation runs at
the scanner, the persistent cache (both directions, plus two new columns
behind an ALTER migration) and the linking handler - which is what stops a
user who switches sources from leaving a stale `hf_url` on a ModelScope
model.
- The agent pipeline keys off the provider instead of `hf_url`: the fast-fail
gate now explains *why* a model is skipped (no source / unknown source /
source without a reachable card), the prompt context exposes
source_url/source_id/source_label/asset_base_url while still filling the
legacy hf_url/repo aliases, and the four README image extractors take a
base_url (defaulting to HF) so relative paths resolve against the right
site. Version grouping generalises to hf: / ms: / ta: keys.
- `POST /api/lm/set-hf-url` keeps its path and its legacy payload keys but
accepts `source_url`, validates against every provider and returns the
platform. `GET /api/lm/model-sources` lets the UI render the supported-site
list from the server.
- Frontend: a `modelSourceHelpers` mirror of the registry drives the link
dialog, the card/modal globe (branded "View on ModelScope/TensorArt"), the
version-group key and the enrichment gate; the versions tab no longer sends
ms:/ta: keys to the CivitAI API.
TensorArt stays in the list because provenance is worth keeping even when the
card is unreadable - the dialog says so plainly ("Sites that don't expose one
(currently TensorArt) can only be linked") and the context menu disables
enrichment with a matching tooltip, instead of the user getting
"Unsupported URL".
Verified against the real ModelScope API: jj3550945163/Krea-2-LORA returns a
1882-byte card whose frontmatter carries base_model/tags/trigger_words, and
relative images resolve to .../resolve/master/....
Tests: backend 2815 passed; frontend 1130 JS + 91 Vue passed; pytest
tests/i18n and a Jinja compile pass over templates/. The nine locales carry
[TODO: Translate] for the new strings, completed in the next commit.
A model could have CivitAI metadata and a HuggingFace link at the same time,
but only one of the two "View on ..." entries ever rendered, because both the
model modal and the card globe asked the `from_civitai` provenance flag which
source to show. `set_hf_url` wrote `false` and a CivitAI refresh wrote `true`,
so whichever ran last erased the other: linking HF hid "View on CivitAI" even
though the civitai payload was still in the sidecar, and (on the card) a later
refresh pointed the single globe icon back at CivitAI, hiding the HF entry.
Decide the links from the data itself instead:
- `set_hf_url` no longer touches `from_civitai`; it records where the metadata
came from, and HF provenance is already tracked by `hf_url`.
- Add `hasCivitaiSource(civitai)` in the shared card/modal utils and gate the
modal's CivitAI link, the card globe (title, enabled state, click target,
new `data-has_civitai`) and the context-menu `civitai` action on actual
CivitAI data (`modelId` / `model_id` / `id`). A dual-source model now shows
both links, and a CivitAI-only model with no `hf_url` stays as before.
- Agent HF enrichment (`PostProcessor.is_hf_model`) keyed off
`not from_civitai`, which stopped being a synonym for "has an HF source" once
both sources can coexist (and already broke after a CivitAI refresh flipped
the flag back to true). Key it off `hf_url` directly; the post-processor
tests move to that discriminator and gain a dual-source case.
Regression tests: the set-hf-url handler preserves civitai + `from_civitai`
and no longer forces the flag false, the modal renders both links (including
with `from_civitai: false`), and the card globe targets/opens the right source
and is disabled when neither is available.
Backend: 2749 passed. Frontend: 1098 JS + 91 Vue tests passed.
The field dates back to a development-stage bug in the enrich-metadata
(agent) pipeline, which briefly wrote trigger words at the top level of
model metadata instead of the established civitai.trainedWords location.
The write path was fixed before the feature merged to main (PR #1013)
and never shipped in any release, so no writer has existed since.
Remove the leftover pieces:
- BaseModelMetadata.trainedWords field (py/utils/models.py); sidecars
from that dev window now pass the key through _unknown_fields instead
- HF download handler's strip-empty-trainedWords special case, reverting
to saving the metadata object directly (py/routes/handlers/hf_handlers.py)
- trainedWords in the LLM enrichment context (agent_service.py)
- matching fallbacks/fixtures in the enrich_hf_validation harness and
post-processor test
Trigger words continue to live in civitai.trainedWords for all model
sources, which is what the UI, agent post-processor, and metadata sync
all read and write.
The bare call inside _build_prompt_context
would raise NameError because class methods don't close over class-level
scope. Use instead to trigger attribute lookup.
Update enrich_hf_metadata prompt.md clue locations for better LLM accuracy.
Update baseline report to v2 (mean 69.0, 46 models, +2.2pp vs baseline 71.1%).
Consolidate README snapshots into baselines/readmes/.
- Rename py/agent_cli/ -> py/metadata_ops/ (module was never agent-related)
- Rename tests/agent_cli/ -> tests/metadata_ops/
- Remove 9 low-value/debug INFO log points across agent_handlers.py,
agent_service.py, llm_service.py, and metadata_ops/__init__.py
- Keep LLM raw response at DEBUG level for diagnostics
- Consolidate per-model progress + LLM result into single concise
log line with basename instead of full path
- Update package/class/method docstrings to clarify this is a
pipeline infrastructure, not a true agent loop
Three-part fix for enrich_hf_metadata failing to extract correct preview_url
from HuggingFace collection repos where models share flat heading levels:
1. _strip_standalone_images() now converts <img> tags to markdown image
syntax  instead of stripping the URL entirely, so the LLM
can still extract preview URLs.
2. _extract_section() uses a line-count-based forward window (stopping at
<a id> anchors) for non-heading matches, instead of stopping at the
very next heading. This prevents same-level sub-headings (# Download,
# Trigger, # Sample prompt within a single model section) from
truncating the window before sample images are included.
3. Post-processor preview fallback now filters gallery images to the
model-specific README section before falling back to the repo-wide
first image.
- agent_service._format_base_models: output bullet list instead of
JSON array for cleaner LLM parsing
- prompt.md mapping section: replace 14-row HF→CivitAI table with
compact rule set covering 14 mapping paths including new entries
for HiDream-ai, OnomaAIResearch/Illustrious, ideogram-ai/ideogram,
Tongyi-MAI/Z-Image-Turbo, and Wan-AI/Wan2.*
- base_model extraction instruction: add guidance to infer from
model filename, YAML tags, and README body text when YAML
frontmatter has no explicit base_model:
- Rename md_to_html.py → readme_processor.py (file no longer just HTML conversion)
- _extract_section: include YAML frontmatter, use heading-level-aware forward
walk (sub-headings under # are included), increase walk limit past 30 lines
- _is_heading: exclude </hN> closing tags from boundary detection
- _heading_level: new helper for heading-level-aware section matching
- css: yield 0 for heading like closing tags, was unexpectedly caught by _is_heading
- extract_gallery_images: fix YAML block scalar (text: >-) prompt extraction;
use endswith instead of == to detect the block marker
- _strip_widget_section: add to clean_readme_for_llm (widget text is handled
by post-processor, not needed in LLM prompt)
- _strip_standalone_images: keep markdown image URLs intact for LLM preview
extraction (was stripping to alt text only)
- list_base_models: switch from scanner-cache aggregation to
CivitaiBaseModelService.get_base_models() - always returns full list
- Ollama: add num_ctx=32768 to payload options so thinking models have room
to both reason and produce output
- Add tests/agent_cli/test_readme_processor.py: 59 tests covering extraction,
cleaning, section matching, heading detection
- Update existing tests for behavioral changes
- PostProcessor returns updates dict from enrich_hf_metadata
- AgentService includes updated_data per model in WebSocket progress events
- Convert preview_url to HTTP URL via config.get_preview_static_url()
- LoraContextMenu: showEnhancedProgress + updateSingleItem per model
- BulkContextMenu: same pattern, remove window.location.reload()
- Guard empty updated_data and clean up callbacks on HTTP error
- Add clean_readme_for_llm() to strip noise from README before LLM injection
- Keep widget section text (valuable tag signal) and unmarked code blocks (trigger words)
- Preserve standalone image alt text instead of removing entirely
- Switch Ollama to native /api/chat with think:false to fix empty content on thinking models
- Extract Sample Gallery table images and deduplicate with widget images
- Only strip code blocks with explicit language tags (bash)
- Add notes and usage_tips fields to SKILL.md output format and post-processor
- Clean up dead code, fix regex edge cases, remove double type annotation
Widget entries with unquoted multi-line YAML scalars (e.g. "text: two samurais...\n continuation") were not parsed, leaving gallery image prompts empty. Add a third branch for plain scalar format alongside the existing quoted and >- folded block handlers.
- Add extract_gallery_images() to parse YAML widget entries from README
frontmatter, convert relative image URLs to absolute HF URLs, and
build civitai.images-compatible entries with prompt metadata
- LLM now extracts recommended_width/recommended_height from README
(e.g. "Best Dimensions"), used as gallery image dimensions
- extract_gallery_images() accepts default_width/height parameters,
falling back to 512x512 when LLM provides no recommendation
- Frontend ShowcaseView.js: defensive NaN guard for 0 width/height
- post_processor: consistently merge civitai updates across triggers,
description, and gallery blocks with distinct variable names
- SKILL.md: add recommended_width/recommended_height to output schema
- 62 tests pass, including gallery extraction and dimension tests
- Add identify_model_type() helper to determine lora/checkpoint/embedding
- Pass priority_tags from user settings to LLM prompt for tag relevance
- SKILL.md: instruct LLM to exclude technical/generic HF tags, cross-reference
against priority_tags; forbid ['None'] placeholder for trigger words
- post_processor: fix preview_url not updated after download (now writes local
.webp path to metadata); write trigger words to civitai.trainedWords instead
of top-level; sanitize ['None']/'null'/'n/a' placeholder values to []
- download_preview() now returns str | None (local path) instead of bool
- Update tests for new return type and nested civitai.trainedWords structure
Merge skill.yaml (metadata) and prompt.md (prompt template) into a
single SKILL.md file with YAML frontmatter, matching the agent-skill
convention used by opencode and Claude Code.
- Add frontmatter parser (_parse_skill_file) to SkillRegistry
- Remove skill.yaml, prompt.md, empty skills/__init__.py
- Remove obsolete load_handler method
- Update tests for new format and cleaned-up fields
Introduce an agent skill framework for LLM-driven metadata enrichment:
- AgentCLI (py/agent_cli/): in-process wrappers around internal services
using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
(replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales
Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
Will Miao