refactor(llm): use catalog-based max_tokens, remove JSON retry, reduce Ollama num_ctx

- Parse limit.output from model catalog alongside model IDs
  for per-model max output token limits
- Use catalog lookup in chat_completion_json() to set max_tokens;
  fall back to 4096 for unknown models (e.g. local Ollama)
- Remove the JSON retry (response_format → plain text fallback);
  keep _try_salvage_json as last-resort for truncated responses
- Reduce Ollama num_ctx from 32768 to 8192 (sufficient for
  metadata enrichment, saves VRAM)
- Fix stale test comment referencing removed retry
This commit is contained in:
Will Miao
2026-07-06 09:13:42 +08:00
parent 07fa454f72
commit 5983eaa1ce
2 changed files with 84 additions and 72 deletions

View File

@@ -219,7 +219,7 @@ class TestLLMServiceChatCompletionJson:
@pytest.mark.asyncio
async def test_chat_completion_json_raises_on_non_json(self, llm_service):
# First attempt: non-JSON; second attempt (retry): also non-JSON
# Non-JSON content raises LLMResponseError (salvage also fails)
mock_response = MockResponse(
200,
json_data={