mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-09-20 18:51:26 -03:00
fix(agent): persist the LLM confidence through metadata writes
The post-processor stored the LLM's confidence as `_llm_confidence`, but that value could never be read back: `BaseModelMetadata.from_dict()` deliberately excludes underscore-prefixed keys from `_unknown_fields` and `to_dict()` strips private fields, so it was erased by the next metadata write and was invisible to `read_metadata()`. The enrichment evaluation harness reads this field to score runs, so confidence was always scored as blank. Store it as `llm_confidence`, which round-trips as an ordinary unknown field — the same mechanism `llm_enriched_at` already relies on. Nothing else consumed the old name, and the harness still accepts it so sidecars written by earlier versions keep evaluating. Covered by a metadata load/save round-trip regression test plus assertions that the post-processor writes the persisted key and no longer writes the private one.
This commit is contained in:
@@ -273,10 +273,13 @@ class PostProcessor:
|
||||
updates["metadata_source"] = "agent:enrich_hf_metadata"
|
||||
updates["llm_enriched_at"] = datetime.now(timezone.utc).isoformat()
|
||||
|
||||
# Store LLM confidence in metadata so it's accessible for evaluation
|
||||
# LLM confidence, stored for the enrichment evaluation harness. The key
|
||||
# must NOT start with an underscore: `BaseModelMetadata.from_dict()`
|
||||
# deliberately drops underscore-prefixed keys so they never round-trip,
|
||||
# which silently erased this field on the next metadata write.
|
||||
raw_confidence = (llm_output.get("confidence") or "").strip()
|
||||
if raw_confidence:
|
||||
updates["_llm_confidence"] = raw_confidence
|
||||
updates["llm_confidence"] = raw_confidence
|
||||
|
||||
# Fallback: use the trigger words the site records for this exact file,
|
||||
# then the README's YAML `instance_prompt`, when the LLM returned none.
|
||||
|
||||
Reference in New Issue
Block a user