Commit Graph
67 Commits
Author SHA1 Message Date
Will Miao e42d649df0 refactor(update)!: make obtainability a property of updates, not a surface
The owner could not tell from the UI what "Buzz Price Tracking" enabled, what the
"Price alert threshold" number meant, or what "Price Alerts" was alerting about.
That was not a copy problem: the implementation exposed our mechanism (a page
scrape) and our SQL predicates as the user's concepts. Two concrete defects came
from the same root:

- the alert population included versions the user already owns (neither the event
  generator nor the query filtered on is_in_library; in the owner's library 28 of
  52 gated versions were already downloaded, so most "alerts" were about files
  already on disk, which cannot become cheaper *for them*);
- a threshold-filtered state list lived in a notification surface, so an empty
  panel had three indistinguishable causes and read as a broken feature.

The information model is now the version plus ownership: cost is shown only where
a decision exists. Owned -> nothing. Not owned and free -> nothing. Not owned and
gated -> the price when it is known, `Paid` without a number when it is not, and
early access keeps its countdown because "free on <date>" decides between waiting
and paying. The numeric threshold has no place in that model: every decision is
categorical (wait / pay / skip), so the setting, the comparison and the whole
alert-state machine are gone.

- both alert-state columns are removed from the schema rather than left dead; a
  database created by an unreleased build has them dropped on open (native
  ALTER TABLE ... DROP COLUMN, guarded), which is a no-op for everyone else
- gate events are emitted only for versions the user does not have, and the
  price-drop event goes with the threshold it belonged to
- both alert endpoints, PriceAlertsHandler and the service-registry adapter field
  it needed are removed: events already reach the UI through the refresh response
- the bell tab, panel, CSS, both entry points, the unread watermark and their
  locales are removed; the setting keeps only the enable flag and the refresh
  interval and is framed as plumbing
- "Price unavailable" is replaced by `Paid`: the gate is certain from the public
  API, only the number is best-effort, and that is our plumbing, not the user's
  problem

Verified against a copy of the owner's real database: 52 gated versions ->
28 owned (now silent) + 24 the feature is actually about; the drop migration ran
and both removed endpoints 404.
2026-10-05 08:54:08 +08:00
Will Miao 7ed19c185c feat(update): add a Buzz price alerts panel to the notification bell
P5a of docs/plans/paid-model-price-tracking.md: one surface that answers "what
got cheaper / became free", without a permanent button (the grid filter was
dropped by owner decision, so the panel carries the actions itself).

- price_alert_since records when an alert started, so the panel can say
  "dropped 3 d ago" and count what is new since the user last looked; it is set
  on the first sight of an already-cheap version, preserved while the alert
  stands, and cleared when the price rises back above the threshold
- get_price_alerts() compares the threshold at read time (editing it takes
  effect immediately, no refresh needed) and returns both kinds in one list;
  model_type=None covers every type, which the shared update DB makes a single
  query. "became free" needs no price data, so it is reported even while price
  tracking is off
- GET /api/lm/price-alerts, registered once in MiscRoutes rather than per model
  type, decorating rows best-effort with the local model name and file path from
  the scanner indexes (a cold cache just omits them)
- a third tab in the notification bell: segments for under-threshold and
  became-free, the three states (tracking off / nothing matching / stale), and
  per-row actions (CivitAI always, Open when the model is local)
- two non-permanent entry points share one helper: the controls-bar updates
  dropdown and the global context menu, whose label carries the unread count
- unread state stays client-side (localStorage watermark); the count is fetched
  once on init and only when price tracking is enabled
- the per-type frontend client method is removed as dead code; the per-type
  backend route stays for the companion extension
2026-10-04 15:08:48 +08:00
Will Miao f5e983eaaa feat: optional centralized storage for sidecar metadata and previews (#1045)
Add an opt-in 'centralized' sidecar storage mode alongside the default
'alongside' layout. In centralized mode, .metadata.json sidecars and
preview assets live under a configurable root (sidecar_storage_path,
default <settings_dir>/sidecars), mirroring the library-relative
directory structure: <root>/<library>/<root_basename>/<rel_dir>/.

Backend:
- settings: sidecar_storage_mode / sidecar_storage_path with validation;
  changing either refreshes the preview allowlist
- config: centralized root added to preview-serving allowlist
- lifecycle: delete / move / rename / folder-rename / folder-delete and
  undoable-delete staging all operate on the mirror tree in centralized
  mode (model files themselves never move); EXDEV-tolerant cross-
  filesystem moves
- scanners: pending-hash filesystem scan walks the mirror tree in
  centralized mode; preview discovery reads from the sidecar dir;
  .civitai.info stays co-located in both modes
- migration: SidecarMigrationUseCase moves sidecars+previews between
  layouts both directions (keep-newer conflict resolution, preview_url
  rewriting, WebSocket progress), exposed as POST+GET
  /api/lm/sidecars/migrate with a mode guard (force=true for the
  settings-first flow)

Frontend:
- settings modal: sidecar storage section (mode select + path input with
  browse/validation), mode-change confirmation offering immediate
  migration (force=true), and a 'Migrate Sidecars Now' action
- i18n keys synced to all locales ([TODO: Translate] placeholders)

Docs: metadata-json-schema.md gains a storage-location section;
AGENTS.md records the sidecar_paths helper convention.
2026-09-26 10:00:58 +08:00
Will Miao 38d4c59b4c feat(download): support ModelScope repositories in the URL downloader
ModelScope became a linkable source, but downloading from it was impossible:
the URL picker only recognised huggingface.co, the file listing hit a
huggingface-only endpoint, the resolve URL was hardcoded, and the default
path template always wrote into a `huggingface/` directory.

Move the download knowledge into the providers so the handlers stay generic:

- `ModelSource` gains `list_files()`, `file_download_url()`,
  `default_revision` and `default_subdir`. `HuggingFaceSource` keeps the Hub
  tree API (`/api/models/{id}/tree/{rev}`, LFS-aware sizes, `main`).
  `ModelScopeSource` uses `/api/v1/models/{id}/repo/files?Revision=master`
  — which reports real byte sizes for LFS files, so no HEAD probe is needed,
  and which only accepts `master` (an HF-imported repo still 404s on `main`)
  — and downloads through `/models/{id}/resolve/{rev}/{path}`. That URL
  redirects to a CDN target carrying a time-limited `auth_key`, so it is
  rebuilt on every request and never cached, which is also what keeps
  resumable Range requests working.
- `hf_handlers.py`/`HfHandler` become `model_source_handlers.py`/
  `ModelSourceHandler` with `list_model_source_files` and
  `download_model_source`. New routes `/api/lm/model-source-files` and
  `/api/lm/download-model-source`; the old `/api/lm/hf-repo-files` and
  `/api/lm/download-hf-model` paths stay as aliases, and a payload without
  `platform` still means Hugging Face, so existing callers are unaffected.
- A downloaded sidecar now records `source_platform` + `source_url` (with the
  `hf_url` alias only for Hugging Face) instead of always writing `hf_url`,
  and `use_default_paths` files ModelScope downloads under
  `modelscope/<owner>/<repo>`. The now-unused shared HF aiohttp session and
  its shutdown hook are gone; providers open short-lived sessions.
- Frontend: `detectUrlType` returns the platform-neutral
  `model-source-repo` / `model-source-file` plus an explicit `platform`, the
  DownloadManager's `hf*` state and methods are renamed to `source*`, every
  `source === 'huggingface'` check becomes `isExternalModelSource()`, and
  batch groups are keyed by `platform:repo` so the same `owner/name` on two
  sites renders as two groups. A bare `owner/name` still means Hugging Face.
- `is_valid_source_id()` centralises repo-id validation (exactly
  `owner/name`, no traversal, no leading dot). This also fixes the old HF
  download check that rejected any dot in the name, i.e. legitimate repos
  such as `black-forest-labs/FLUX.1-dev`.

Verified against the live APIs: the example repo lists 8 weight files with
correct sizes, and a ranged GET of the built resolve URL returns 206 after
following the redirect to the CDN. Backend 2853 passed; frontend 1143 JS +
91 Vue passed. The nine locales carry the refreshed download copy in the
next commit.
2026-09-14 07:42:51 +08:00
Will Miao e0052cd237 fix(download): align location-step root selection with backend diffusion routing
The download modal's location step decided between checkpoint and unet
roots using only the CivitAI file-type signal, while the backend also
falls back to DIFFUSION_MODEL_BASE_MODELS. Models like Anima (file type
"Model") were offered checkpoint roots in the UI even though
use_default_paths would route them to the unet root.

- Extract the two-tier decision into py/services/download_routing.py and
  reuse it in DownloadManager._execute_download
- Add POST /api/lm/download/routing so the UI asks the backend for the
  routing decision; fall back to the local file-type check on failure
- ModelVersionsTab: search both checkpoint and unet roots when resolving
  an existing version's download path
2026-09-11 12:41:03 +08:00
Will Miao 8e724538bd fix(types): resolve pre-existing basedpyright errors in py/ and standalone.py
Fix ~950 basedpyright errors across the backend:
- Convert ineffective # type: ignore comments to # pyright: ignore[rule]
- Add missing generic type arguments (Dict[str, Any], list[Any], ...)
- Annotate dynamic dict literals and runtime-initialized attributes
- Widen CivitAI provider tuple signatures in recipe parsers
- Remove dead LoraRoutes handlers calling nonexistent LoraService methods
- Suppress unavoidable ServiceRegistry import cycles (basedpyright counts
  function-local imports as cycle edges)
2026-08-08 20:12:52 +08:00
Will Miao cf898da193 feat(agent): add LLM-powered metadata enrichment system with AgentCLI and PostProcessor
Introduce an agent skill framework for LLM-driven metadata enrichment:

- AgentCLI (py/agent_cli/): in-process wrappers around internal services
  using standard relative imports, eliminating the need for sys.path hacks
- LLMService: centralized BYOK (bring-your-own-key) LLM client supporting
  OpenAI, Ollama, and custom OpenAI-compatible endpoints
- PostProcessor: deterministic engine that applies LLM output via AgentCLI
  (replaces old handler.py + _BASE_MODEL_ALIASES approach)
- SkillRegistry: filesystem-based skill discovery (skill.yaml + prompt.md)
- AgentService: orchestrates skill execution with WebSocket progress
- Frontend AgentManager: WebSocket listeners, skill execution, config UI
- Context menu entries (single + bulk) for "Enrich Metadata (Agent)"
- Settings UI for AI Provider configuration (BYOK)
- Full i18n support across 9 locales

Bug fixes found during review:
- aiohttp.web.json_response: status_code= -> status=
- settings_modal cancelEditApiKey: wrong argument position
- AgentManager.isLlmConfigured: allow Ollama without API key
- PostProcessor._merge_tags: lowercase all tags to match TagUpdateService
2026-07-02 21:27:01 +08:00
Will Miao 09ca91fc0e feat(download): add Hugging Face model download to standalone UI wizard (#965, #977)
Integrate HF model downloading into the existing CivitAI-style wizard flow:
- URL type detection (civitai / hf-resolve / hf-repo / direct-http)
- Repo file explorer with checkbox-based file selection
- Batch/queue download with per-file WebSocket progress
- Aria2 backend support (respects download_backend setting)
- Scanner cache integration via create_default_metadata + add_model_to_cache
- i18n updates for all 10 locales
2026-06-30 19:36:12 +08:00
Will Miao 62247bdd87 feat(prompt): expand wildcards at runtime (#895) 2026-04-15 20:42:27 +08:00
Will Miao 1817142a7b feat(doctor): add system diagnostics feature 2026-04-11 16:03:38 +08:00
Will Miao 72f8e0d1be fix(backup): add user-state backup UI and storage 2026-04-10 20:49:30 +08:00
Will Miao 00f5c1e887 feat: Dynamic base model fetching from Civitai API (#854)
Implement automatic fetching of base models from Civitai API to keep
data up-to-date without manual updates.

Backend:
- Add CivitaiBaseModelService with 7-day TTL caching
- Add /api/lm/base-models endpoints for fetching and refreshing
- Merge hardcoded and remote models for backward compatibility
- Smart abbreviation generation for unknown models

Frontend:
- Add civitaiBaseModelApi client for API communication
- Dynamic base model loading on app initialization
- Update SettingsManager to use merged model lists
- Add support for 8 new models: Anima, CogVideoX, LTXV 2.3, Mochi,
  Pony V7, Wan Video 2.5 T2V/I2V

API Endpoints:
- GET /api/lm/base-models - Get merged models
- POST /api/lm/base-models/refresh - Force refresh
- GET /api/lm/base-models/categories - Get categories
- GET /api/lm/base-models/cache-status - Check cache status

Closes #854
2026-03-29 00:18:15 +08:00
Will Miao cda271890a feat(workflow-template): add new tab template workflow with auto-zoom
- Add GET /api/lm/example-workflows endpoint to list available templates
- Add GET /api/lm/example-workflows/{filename} to retrieve specific workflow
- Add 'New Tab Template Workflow' setting in LoRA Manager settings
- Automatically apply 80% zoom level when loading template workflows
- Override workflow's saved view settings to prevent visual zoom flicker

The feature allows users to select a template workflow from example_workflows/
directory to load when creating new workflow tabs, with a hardcoded 0.8 zoom
level for better initial view experience.
2026-03-08 21:03:14 +08:00
Will Miao 78b55d10ba refactor: move supporters loading to separate API endpoint
- Add SupportersHandler in misc_handlers.py to serve /api/lm/supporters
- Register new endpoint in misc_route_registrar.py
- Remove supporters from page load template context in model_handlers.py
- Create supportersService.js for frontend data fetching
- Update Header.js to fetch supporters when support modal opens
- Modify support_modal.html to use client-side rendering

This change improves page load performance by loading supporters data
on-demand instead of during initial page render.
2026-02-28 20:14:20 +08:00
Will Miao d5a2bd1e24 feat: add custom words autocomplete support for Prompt node
Adds custom words autocomplete functionality similar to comfyui-custom-scripts,
with the following features:

Backend (Python):
- Create CustomWordsService for CSV parsing and priority-based search
- Add API endpoints: GET/POST /api/lm/custom-words and
  GET /api/lm/custom-words/search
- Share storage with pysssss plugin (checks for their user/autocomplete.txt first)
- Fallback to Lora Manager's user directory for storage

Frontend (JavaScript/Vue):
- Add 'custom_words' and 'prompt' model types to autocomplete system
- Prompt node now supports dual-mode autocomplete:
  * Type 'emb:' prefix → search embeddings
  * Type normally → search custom words (no prefix required)
- Add AUTOCOMPLETE_TEXT_PROMPT widget type
- Update Vue component and composable types

Key Features:
- CSV format: word[,priority] compatible with danbooru-tags.txt
- Priority-based sorting: 20% top priority + prefix + include matches
- Preview tooltip for embeddings (not for custom words)
- Dynamic endpoint switching based on prefix detection

Breaking Changes:
- Prompt (LoraManager) node widget type changed from
  AUTOCOMPLETE_TEXT_EMBEDDINGS to AUTOCOMPLETE_TEXT_PROMPT
- Removed standalone web/comfyui/prompt.js (integrated into main widgets)

Fixes comfy_dir path calculation by prioritizing folder_paths.base_path
from ComfyUI when available, with fallback to computed path.
2026-01-25 12:24:32 +08:00
Will Miao 102defe29c feat: add open settings location endpoint
- Add `open_settings_location` method to `FileSystemHandler` to open OS file explorer at settings file location
- Register new POST route `/api/lm/settings/open-location` for settings file access
- Inject `SettingsManager` dependency into `FileSystemHandler` constructor
- Add cross-platform support for Windows, macOS, and Linux file explorers
- Include error handling for missing settings files and system exceptions
2025-12-31 16:09:23 +08:00
pixelpaws c53fa8692b refactor(settings): lazily initialize manager 2025-10-08 08:56:57 +08:00
pixelpaws 40e3c6134c refactor(routes): modularize misc route handling 2025-10-03 22:19:09 +08:00
Will Miao 375b5a49f3 fix(config): update standalone mode environment variable usage 2025-10-02 09:40:24 +08:00
pixelpaws 8aa76ce5c1 feat(settings): centralize frontend settings on backend 2025-09-23 20:28:32 +08:00
Will Miao fec85bcc08 refactor: unify standalone mode check using environment variable 2025-09-21 22:45:11 +08:00
Will Miao d30fbeb286 feat(example_images): add dedicated folder check and update settings handling for example images path, see #431 2025-09-18 19:22:29 +08:00
Will Miao bdc86ddf15 Refactor API endpoints to use '/api/lm/' prefix
- Updated all relevant routes in `stats_routes.py` and `update_routes.py` to include the new '/api/lm/' prefix for consistency.
- Modified API endpoint configurations in `apiConfig.js` to reflect the new structure, ensuring all CRUD and bulk operations are correctly routed.
- Adjusted fetch calls in various components and managers to utilize the updated API paths, including recipe, model, and example image operations.
- Ensured all instances of the old API paths were replaced with the new '/api/lm/' prefix across the codebase for uniformity and to prevent broken links.
2025-09-18 14:50:40 +08:00
Will Miao ded17c1479 feat(routes): add model versions status endpoint and enhance metadata retrieval 2025-09-17 22:06:59 +08:00
Will Miao 4bb8981e78 refactor(routes): update API endpoints for settings to use '/api/lm/settings', see #435 2025-09-15 16:22:59 +08:00
Will Miao 2f7e44a76f refactor(settings): Update synchronization logic 2025-09-15 10:30:06 +08:00
Will Miao 9366d3d2d0 feat: add API endpoint for fetching application settings and update frontend settings management 2025-09-14 22:57:17 +08:00
Will Miao e713bd1ca2 feat: add app-level proxy settings with UI integration and session management, fixes #382 2025-09-12 11:22:45 +08:00
Will Miao 8b4e3128ff feat: add functionality to open file location from model modal and update translations, fixes #405 2025-09-11 15:54:32 +08:00
Will Miao c66cbc800b refactor: remove clear cache functionality and associated modal from settings manager 2025-09-11 15:21:06 +08:00
Will Miao ba1ac58721 feat(metadata): trigger metadata provider update when enabling metadata archive database 2025-09-10 16:18:04 +08:00
Will Miao a4fbeb6295 feat(metadata): update metadata archive management and remove provider priority settings 2025-09-10 15:55:29 +08:00
Will Miao 1ea468cfc4 feat(metadata): enhance metadata archive management with download progress and status updates 2025-09-09 15:24:28 +08:00
Will Miao 821827a375 feat(metadata): implement metadata archive management and update settings for metadata providers 2025-09-08 22:41:17 +08:00
Will Miao ee84571bdb refactor: Simplify handling of base model path mappings and download path templates by removing unnecessary JSON.stringify calls 2025-08-19 20:20:30 +08:00
Will Miao e4195f874d feat: implement download path templates configuration with support for multiple model types and custom templates 2025-08-13 17:42:40 +08:00
Will Miao 96517cbdef fix: update model_id and model_version_id handling across various services for improved flexibility 2025-08-11 15:31:49 +08:00
Will Miao 4081b7f022 feat: implement settings synchronization with backend and migrate legacy settings 2025-07-29 20:29:19 +08:00
Will Miao 3cf9121a8c refactor: enhance scanner handling and add embedding support in download manager and misc routes 2025-07-25 23:59:27 +08:00
Will Miao a2b81ea099 refactor: Implement base model routes and services for LoRA and Checkpoint
- Added BaseModelRoutes class to handle common routes and logic for model types.
- Created CheckpointRoutes class inheriting from BaseModelRoutes for checkpoint-specific routes.
- Implemented CheckpointService class for handling checkpoint-related data and operations.
- Developed LoraService class for managing LoRA-specific functionalities.
- Introduced ModelServiceFactory to manage service and route registrations for different model types.
- Established methods for fetching, filtering, and formatting model data across services.
- Integrated CivitAI metadata handling within model routes and services.
- Added pagination and filtering capabilities for model data retrieval.
2025-07-23 14:39:02 +08:00
Will Miao 124002a472 feat: Add JSON parsing for base_model_path_mappings and refactor path handling in DownloadManager 2025-07-21 07:37:34 +08:00
Will Miao c692713ffb refactor: Simplify model version existence checks and enhance version retrieval methods in scanners 2025-07-09 10:26:03 +08:00
Will Miao 8f10765254 feat: Add health check route to MiscRoutes for server status monitoring 2025-07-06 21:40:47 +08:00
Will Miao 2d4f6ae7ce feat: Add route to check if a model exists in the library 2025-07-02 14:45:19 +08:00
Will Miao 64dd2ed141 feat: enhance node registration and management with support for multiple nodes and improved UI elements. Fixes #220 2025-06-26 23:00:55 +08:00
Will Miao eb57e04e95 feat: implement thread-safe node registry and registration endpoints for Lora nodes 2025-06-26 18:31:14 +08:00
Will Miao 4d6ea0236b Add centralized example images setting and update related UI components 2025-06-08 17:38:46 +08:00
Will Miao e872a06f22 Refactor MiscRoutes and move example images related api to ExampleImagesRoutes 2025-06-08 14:40:30 +08:00
Will Miao 647bda2160 Add API endpoint and frontend integration for fetching example image files 2025-06-07 22:31:57 +08:00
Will Miao c96550cc68 Enhance migration and download processes: add backend path update and prevent duplicate completion toasts 2025-06-07 11:29:53 +08:00