Commit Graph
8 Commits
Author SHA1 Message Date
Will Miao bccd494a56 feat(recipes): explain empty LoRA lists with collapsible "Why no LoRAs?" panel
Record import provenance on every recipe: a new import_info block
(channel, machine-readable no-LoRA reason, diagnostic details) built at
import time across all channels (batch import, single URL, local file,
upload, widget save, re-imports) and persisted in the recipe JSON plus
the SQLite persistent cache (new import_info_json column with ALTER
TABLE migration).

The recipe modal renders the empty LoRA list with a collapsed details
panel showing the import method, the reason (CivitAI API returned no
LoRA resource data, API meta missing, no embedded metadata, ComfyUI
workflow metadata, video, unparsable format), and recorded diagnostics.
Legacy recipes without import_info fall back to heuristics labeled as
inferred. Genuine no-LoRA generations show no panel.

CivitAI images are always classified by API meta shape: the onsite
generator writes A1111-style EXIF without LoRA references, so parsed
EXIF cannot prove "no LoRAs used".

Adds recipes.resources.noLoras* i18n keys (all 10 locales) plus
frontend vitest and backend pytest coverage.
2026-08-30 16:28:41 +08:00
Will Miao df34efafbc feat(recipes): skip rate-limited batch-import items and register download 429s (#1085)
Phase 2 of docs/plans/issue-1085-rate-limit-design.md:

- Batch import: items that fail due to vendor rate limiting are now
  SKIPPED with a "re-run the import later" hint instead of FAILED, so a
  transient 429 no longer pollutes failure accounting; the progress
  broadcast carries a rate_limited flag.
- Batch import UI: show a one-time "rate limited — slowing down" toast
  and swap the running status text while rate_limited; i18n keys synced
  to all locales.
- Downloader: download_file / download_to_memory / get_response_headers
  register 429 cooldowns with the RateLimitCoordinator, so subsequent
  API calls queue behind a download-triggered rate-limit window.
2026-08-27 10:08:32 +08:00
Will Miao ee233548e5 fix(recipes): enforce batch-import concurrency bound and harden ingest errors (#1085)
Address the rate-limit flood and secondary errors seen during large
recipe ingestion (example-images directory import):

- batch import: share one adaptive-concurrency semaphore across the whole
  batch (previously each item got a fresh semaphore, so the min/max
  concurrency bounds never applied and every item ran concurrently);
  synchronize the shared semaphore capacity after each completed item.
- comfy parser: guard ckpt_name against list/None values so re.search no
  longer raises TypeError and fails the whole image import.
- civarchive client: normalize empty-string failure payloads to
  "Request failed" and treat a missing payload as an error, fixing the
  "'NoneType' object has no attribute 'get'" crash.
- civarchive client: log connectivity-guard offline-cooldown
  short-circuits at DEBUG instead of one ERROR per request.
2026-08-27 07:58:48 +08:00
Will Miao d188cec306 fix(recipes): restore batch import modal on reopen and log recipe ingest progress (#1084) 2026-08-26 22:32:20 +08:00
Will Miao 283730cf38 fix(import): discover LoRA + checkpoint from modelVersionIds when API meta is null
When CivitAI image API returns meta=null and modelVersionIds at root
level, the import flow now:

- Injects modelVersionIds + browsingLevel into a minimal metadata dict
  so the parser can discover LoRAs and checkpoints (both import-from-url
  and analyze-image paths)
- Adds checkpoint dedup + fallback in the parser's modelVersionIds
  handler to avoid duplicate API calls
- Runs EXIF extraction unconditionally in analyze-image path, then
  merges with API metadata (fixes gen params loss)
- Propagates preview_nsfw_level through all three import paths:
  import-from-url, analyze-image (UI Import), and batch-import,
  plus the frontend save flow
2026-06-27 17:05:38 +08:00
Will Miao 86118d0654 fix(recipes): persist source_path in SQLite cache and eliminate source_url redundancy
- Add source_path column to PersistentRecipeCache SQLite schema with
  migration for existing databases (ALTER TABLE ADD COLUMN)
- Backfill source_path from recipe JSON files on first startup after
  migration to avoid requiring manual cache rebuild
- Remove all source_url recipe field references (import_remote_recipe,
  import_from_url, check_image_exists, enrichment, batch_import)
  and consolidate on source_path as the single source of truth
- Add civitai.green to supported Civitai page hosts
- Register check-image-exists and import-from-url recipe endpoints
2026-05-12 20:39:09 +08:00
Will Miao ee466113d5 feat: implement batch import recipe functionality (frontend + backend fixes)
Backend fixes:
- Add missing API route for /api/lm/recipes/batch-import/progress (GET)
- Add missing API route for /api/lm/recipes/batch-import/directory (POST)
- Add missing API route for /api/lm/recipes/browse-directory (POST)
- Register WebSocket endpoint for batch import progress
- Fix skip_no_metadata default value (True -> False) to allow no-LoRA imports
- Add items array to BatchImportProgress.to_dict() for detailed results

Frontend implementation:
- Create BatchImportManager.js with complete batch import workflow
- Add directory browser UI for selecting folders
- Add batch import modal with URL list and directory input modes
- Implement real-time progress tracking (WebSocket + HTTP polling)
- Add results summary with success/failed/skipped statistics
- Add expandable details view showing individual item status
- Auto-refresh recipe list after import completion

UI improvements:
- Add spinner animation for importing status
- Simplify results summary UI to match progress stats styling
- Fix current item text alignment
- Fix dark theme styling for directory browser button
- Fix batch import button styling consistency

Translations:
- Add batch import related i18n keys to all locale files
- Run sync_translation_keys.py to sync all translations

Fixes:
- Batch import now allows images without LoRAs (matches single import behavior)
- Progress endpoint now returns complete items array with status details
- Results view correctly displays skipped items with error messages
2026-03-16 09:41:58 +08:00
Will Miao f86651652c feat(batch-import): implement backend batch import service with adaptive concurrency
- Add BatchImportService with concurrent execution using asyncio.gather
- Implement AdaptiveConcurrencyController with dynamic adjustment
- Add input validation for URLs and local paths
- Support duplicate detection via skip_duplicates parameter
- Add WebSocket progress broadcasting for real-time updates
- Create comprehensive unit tests for batch import functionality
- Update API handlers and route registrations
- Add i18n translation keys for batch import UI
2026-03-16 09:41:58 +08:00