diff --git a/docs/i18n-translation-guidelines.md b/docs/i18n-translation-guidelines.md index eca53c6f..3d64dc28 100644 --- a/docs/i18n-translation-guidelines.md +++ b/docs/i18n-translation-guidelines.md @@ -139,6 +139,17 @@ Locales: `en`, `zh-CN`, `zh-TW`, `ja`, `ko`, `fr`, `de`, `es`, `ru`, `he` (RTL). > `.showcaseLayoutOptions.gallery` / `.vertical`, `modals.model.showcase.layoutGallery` / > `.layoutList`). All 9 locales are translated (terminology in §2, "Showcase layout > feature"), so the "no remaining placeholders" claim holds again. +> **Status (2026-10, Buzz download prices):** the paid/early-access obtainability feature added +> 13 keys — the two `globalContextMenu.checkModelUpdates.gateEvents.*` counts, five +> `modals.model.versions.badges.*` (sale, Blue Buzz, "free now" and its tooltip, and the early +> access end-date tooltip) and the six `settings.priceTracking.*` strings — plus the section +> header. All 13 are now translated in all 9 locales. **Buzz** and **Blue Buzz** stay as-is +> everywhere (CivitAI currency names, R3). Two source fixes came with the pass: the unused +> `settings.priceTracking.label` key was removed (no template renders it; the toggle uses +> `enabled`/`enabledHelp`) and `settings.sections.priceTracking` was reworded to +> "Buzz Download Prices" so the header matches what the feature does (prices are displayed; +> nothing is tracked for alerts). Register follows each file's existing norm: 你 (zh-CN), +> 您 (zh-TW), Sie (de), tú (es), вы (ru). No remaining `[TODO: Translate]` placeholders. --- diff --git a/docs/plans/paid-model-price-tracking.md b/docs/plans/paid-model-price-tracking.md new file mode 100644 index 00000000..79ead9d6 --- /dev/null +++ b/docs/plans/paid-model-price-tracking.md @@ -0,0 +1,841 @@ +# Plan: Buzz Price Tracking and Threshold Alerts for Paid / Early Access Versions + +**Origin:** FR "Price tracker for models/loras" (Geekier, Discord) — track the buzz price of paid +model versions and flag when it drops below a threshold or becomes free. +**Related:** [#1060 — Some "Early Access" models are not identified correctly](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1060) +(closed) established `is_paid`, the Paid badge and `hide_paid_updates`; this FR is the next step +after gate *state* — gate *price*. +**Status:** v3 — **P0–P3 implemented** (§10 records what shipped and the deviations); **P5 (alerts +panel) planned** in §11, not implemented. The grid-level "price alert only" filter was dropped by +owner decision, so §11 defines the panel as the only new browsing surface. +Feasibility was verified against this repo and upstream CivitAI `main` (`6d29ed1368`), including live +probes against `civitai.com` / `civitai.red`. +**Scope:** every model type that goes through `ModelUpdateService` (lora / checkpoint / embedding / +other). Out of scope: purchasing, downloading gated content, and any change that depends on a future +CivitAI API addition. + +--- + +## 1. Problem statement + +Creators increasingly gate model versions behind buzz. LoRA Manager already knows **whether** a +version is gated (`is_paid`, `is_early_access`, `paid_access`) but not **how much it costs**, so it +cannot answer the two questions the FR asks: + +1. "Tell me when this model's price drops below X buzz." +2. "Tell me when it becomes free." + +Additionally, the current code only compares version ids when deciding `has_update`, so a version +that a user already tracks can go from paid to free (or gain a gate) without producing any signal at +all. + +## 2. Verified current state + +### 2.1 LoRA Manager (all references confirmed in this checkout) + +| Fact | Reference | +| --- | --- | +| Gate fields persisted per version: `is_paid`, `is_early_access`, `paid_access` (raw DTO JSON), `early_access_ends_at` | `py/services/model_update_service.py:65-84` | +| Gate data comes from the bulk `GET /api/v1/models?ids=…` response (`availability`, `paidAccess`, `earlyAccessEndsAt`) | `py/services/civitai_client.py:329-380`, parsed at `py/services/model_update_service.py:1756-1793` | +| **No numeric price anywhere in `py/`** | `grep -rn "price" py/` → only license/`allowCommercialUse` hits | +| Update refresh is user-triggered, not scheduled; TTL 24 h | route `py/routes/handlers/model_handlers.py:2899`; `ModelUpdateService.__init__` default `ttl_seconds=24*60*60` at `py/services/model_update_service.py:338` | +| `has_update` compares version ids + gate filters only; no state-transition detection | `py/services/model_update_service.py:117-160` | +| `{permanent:false, endsAt:null}` is deliberately treated as **not** a gate on the download path | `py/services/download_manager.py:2000-2003` | +| Version payload sent to the frontend already carries `isPaid` / `paidAccess` | `py/routes/handlers/model_handlers.py:3444-3455` | +| UI: Paid / Early Access badges, `hide_paid_updates` filter | `static/js/components/shared/ModelVersionsTab.js:169-196, 474-545`; `static/js/state/index.js:58-59`; `templates/components/modals/settings/library.html:191-192` | + +### 2.2 Upstream CivitAI — what is and is not public + +| Fact | Evidence | +| --- | --- | +| Public v1 API intentionally strips pricing: `paidAccess` is reduced to `{permanent, endsAt}`. Source comment: *"Omits terms (pricing belongs to the purchase flow)"* | `~/code/civitai/src/server/services/paid-access.service.ts:640-652`; verified live: `/api/v1/models` and `/api/v1/model-versions/{id}` return exactly that shape | +| The internal tRPC route is **not** usable anonymously. `isAcceptableOrigin` rejects non-site origins with `401 "Please use the public API instead"` | `~/code/civitai/src/server/trpc.ts:120-128`; `acceptableOrigin` = `!isProd \|\| isBearerAuth \|\| isAllowedOriginRequest(req)` (`createContext.ts:43`), and the origin check is header-based (`src/server/utils/origin-helpers.ts:27-32`). **Live probe warning:** an anonymous tRPC GET can return `200` when Cloudflare serves a cached copy (`cf-cache-status: HIT`); uncached it is `401`. Do not build on this path. | +| **The price *is* publicly reachable inside the public model page.** The SSR payload embeds the site's own `model.getById` result, including full `terms` | `https://civitai.com/models/` and `https://civitai.red/models/` → ``; `json.loads` it. +* Walk `props.pageProps.trpcState.json.queries[*]`; pick the entry whose `queryKey[0] == ["model","getById"]` + (the query order is not stable — 7 queries were observed, `getById` first only incidentally). +* From `state.data.modelVersions[*]`, per version emit: + * `price_buzz` = `paidAccess.sale.buyerTerms.download.price` if `sale` present, else + `paidAccess.terms.download.price` + * `list_price_buzz` = `paidAccess.terms.download.price` + * `generation_price_buzz` = `terms.generation.price` (undefined for `{free:true}` / bundled) + * `accepts_blue_buzz`, `price_sale_ends_at` + * no `paidAccess` → the version is ungated: emit an explicit "no price" marker so the caller can + clear stale prices and raise a *became free* event. +* Return `None` (never raise) for: missing script tag, JSON error, shape mismatch, empty + `modelVersions`, or a Cloudflare/challenge page. + +### 4.3 Refresh flow and request budget + +Hook: `_refresh_single_model` (`py/services/model_update_service.py:1099-1215`), inside the existing +"lock released during network I/O" window, after `fetched_versions` is built: + +``` +prices_needed = feature_enabled and (any gated version or any version with a stored price) +if prices_needed: + prices = await provider.get_model_prices(model_id) # never raises; None on failure + if prices is not None: + fetched_versions = [apply_price(v, prices.get(v.version_id)) for v in fetched_versions] + # price_checked_at advances only for versions actually present in the payload +``` + +* Candidate selection uses the gate data already in hand from the bulk API, so no extra call is made + for the ~98% of models with no gate (except models that previously had a price and must confirm + "now free"). +* TTL: `price_check_ttl_hours` (default 24 h), independent of the update TTL — a price refresh can be + skipped while the version list is fresh, and vice versa. Force refresh (`force_refresh=True`) also + forces prices. +* Failure behaviour: keep the previous price, do **not** advance `price_checked_at` (so the next + refresh retries), log at debug/info. A `RateLimitError` from the shared gate must remain a skip, + not a hard failure of the update check. After N consecutive parse failures within one refresh, + stop attempting price fetches for the remainder of that run. +* Offline/cooldown: reuse the existing `ConnectivityGuard` path in `downloader.make_request` — no new + connectivity handling. + +Client/provider seam: + +* `ModelMetadataProvider.get_model_prices(model_id)` — default returns `None` (not abstract, so + CivArchive / SQLite providers are untouched). +* `CivitaiModelMetadataProvider` → `CivitaiClient.get_model_prices(model_id)`. +* Add the pass-through to the composite provider wrapper + (`py/services/model_metadata_provider.py:855-955`) so the rate-limit helper applies. +* `CivitaiClient.get_model_prices`: build the URL with the existing helper + `build_civitai_model_page_url(model_id, host=self._settings.get("civitai_host"))` + (`py/utils/civitai_utils.py:44-66`; `civitai.red` is already a supported page host, + `:10`), fetch with `downloader.make_request("GET", url, use_auth=False, custom_headers={"Accept": "text/html"})` + (returns `str` because the body is not JSON), then call the parser. Cap the body size before + parsing. + +### 4.4 Alert semantics + +* **Threshold hit**: `price_buzz is not None and price_buzz <= threshold_buzz`. `threshold_buzz = 0` + means "free only, plus any gate removal". +* **Became free**: a version with a stored gate/price now has no `paidAccess` (from the bulk API + and/or the page payload). +* **Newly gated**: a version that had none now has `paidAccess`. +* Edge detection uses `price_alert_state` plus the previous `paid_access` JSON, compared in + `_build_record_from_remote` where `existing_map` is already built + (`py/services/model_update_service.py:1636-1665`). Do not put this in `has_update`: it is a + different question, and `has_update` is filtered by the hide-* settings. +* `ModelUpdateRecord` gains a **non-persisted** field (e.g. `events: list | None = None`) carrying + `{version_id, kind: "price_drop"|"became_free"|"new_gate", price_buzz, previous_price_buzz}`. + `_upsert_record` ignores it; `_get_record` leaves it `None`. + +### 4.5 Settings and routes + +New settings (defaults in `DEFAULT_SETTINGS`, `py/services/settings_manager.py:68`; +`settings.json.example` stays minimal per repo policy; frontend defaults in +`static/js/state/index.js:58`): + +| Key | Default | Meaning | +| --- | --- | --- | +| `price_tracking_enabled` | `False` | master switch; off means no model-page fetches at all | +| `price_alert_threshold_buzz` | `0` | alert when effective price ≤ this; 0 = free only | +| `price_check_ttl_hours` | `24` | price freshness window | + +Routes (all under the existing model route registrars, `POST` + `GET` because the companion +extension is GET-only per `AGENTS.md`): + +* `GET /api/lm/models/price-alerts` — aggregate over the update DB: + `{modelId, modelType, versionId, versionName, priceBuzz, listPriceBuzz, thresholdBuzz, isFree}`. +* Prices themselves need no new read route: they ride along in the existing version payload + (`py/routes/handlers/model_handlers.py:3438-3455` gains `priceBuzz`, `listPriceBuzz`, + `generationPriceBuzz`, `acceptsBlueBuzz`, `priceCheckedAt`, `priceAlert`). +* The existing update-refresh response gains `events` (from §4.4) so the frontend can toast. + +### 4.6 UI + +* `static/js/components/shared/ModelVersionsTab.js` + * Extend the existing Paid / Early Access badge tooltips with the price when known + ("Paid · 5,000 Buzz") — helpers at `:169-196`, badges at `:474-545`. + * New badges: price chip (`paid`/`info` styling) and "Free now" for a became-free transition. + * Filter toggle reusing the `hide_paid_updates` pattern (`:349-400`, plus the settings modal and + `SettingsManager.js:1157-1163`) — e.g. "Show price drops only". +* Optional, cheap win: include the price in the paid-download error text + (`py/services/download_manager.py:1987-2031`, `py/services/use_cases/download_model_use_case.py:35`). +* Strings: add to `locales/en.json`, then `python scripts/sync_translation_keys.py`, then **stop** — + do not translate the other locales (see `docs/i18n-translation-guidelines.md` §7). + +## 5. Phased tasks + +### P0 — settle gate semantics (prerequisite, small) + +- [ ] Decide the meaning of `{permanent:false, endsAt:null}`. The repo currently treats it as *no + gate* (`py/services/download_manager.py:2000-2003`), while upstream `isPaidAccessActive` + (`endsAt == null || endsAt > now`) treats it as *active*. Live sample: version 3372655 + currently returns exactly this DTO from `/api/v1/models`. +- [ ] Introduce one shared helper (e.g. `_has_active_gate(paid_access, early_access_ends_at)` in + `py/services/model_update_service.py`) and use it from both the update service and the download + gate, so alerts and download blocking cannot disagree. +- [ ] Unit test for the chosen rule. +- **Why first:** every alert kind inherits the gate-state decision; getting it wrong produces false + "free" alerts. + +### P1 — gate-state change detection (no new data source) + +- [ ] Emit `new_gate` / `became_free` events from `_build_record_from_remote`; add the non-persisted + `events` field; surface in the refresh response. +- [ ] Keep `has_update` semantics unchanged (regression risk to `hide_paid_updates`, + `hide_early_access_updates`). +- [ ] UI: "Free now" badge + one batched toast per refresh; extend paid/EA tooltips. +- [ ] Tests in `tests/services/test_model_update_service.py` for each transition and for the + no-change case. +- **Acceptance:** a version whose gate lapses is visibly marked free after a refresh, with no new + network calls; existing hide-* filters behave identically. + +### P2 — price capture (opt-in) + +- [ ] Schema + all eight column touchpoints from §4.1, with a migration test. +- [ ] `parse_model_page_prices` + fixture-based tests (`tests/services/fixtures/…`, one trimmed page + with a timed gate, one with a permanent gate, one malformed/missing payload). +- [ ] `CivitaiClient.get_model_prices` + provider methods + composite pass-through; assert redirect + following and that a non-JSON body is returned as text. +- [ ] Wire into `_refresh_single_model` behind `price_tracking_enabled`, with the failure matrix from + §4.3. +- [ ] Tests: feature off ⇒ zero extra calls; fetch failure ⇒ previous price retained and + `price_checked_at` unchanged; gated-only candidate selection. +- **Acceptance:** with tracking enabled, a refresh on a gated model stores `price_buzz` for all of + its versions from one request; with tracking disabled or offline, refresh behaves exactly as today. + +### P3 — threshold alerts and surface + +- [ ] Settings + defaults (§4.5) and the settings-modal toggles/inputs. +- [ ] Threshold evaluation + `price_alert_state` edges → `price_drop` events. +- [ ] `GET /api/lm/models/price-alerts` aggregate endpoint. +- [ ] Version payload gains the price fields; UI price chip, "Free now" badge, "price drops only" + filter; optional price in the paid-download error. +- [ ] Locale keys + `scripts/sync_translation_keys.py`. +- [ ] Tests: threshold boundary (`<=` vs `<`), Blue Buzz labelling, sale-adjusted vs list price, + alerts for a version already in the library. +- **Acceptance:** a user can set "alert me under 500 buzz", refresh, and see the models that + qualify; nothing alerts when the price rises above the threshold again (state resets). + +### P4 — docs and upstream + +- [ ] Short feature note in the repo docs; keep this plan's data-source rationale in a code comment + (public page read, no auth or origin spoofing, no internal API). +- [ ] Draft the upstream request: add a price field to the public `paidAccess` DTO. Point out the + asymmetry — writes already go through the official v1 endpoint + (`~/code/civitai/src/pages/api/v1/model-versions/early-access.ts`, sharing + `updateModelVersionPaidAccessSchema`) while reads withhold it deliberately. If it lands, the + page scraper becomes an optional fallback rather than the only path. + +## 6. Test plan summary + +* `pytest tests/services/test_model_update_service.py` — migrations, extraction, transitions, + threshold logic, fetch-failure matrix, feature-off path. +* New parser tests with HTML fixtures (no network in tests). +* `npm run test:js` for badge/helper changes in `ModelVersionsTab`. +* Manual UI verification by the user (per repo policy — no browser automation for layout). +* No live CivitAI calls in the test suite. + +## 7. Risks and mitigations + +| Risk | Mitigation | +| --- | --- | +| Page payload format changes (Next.js is migrating away from `__NEXT_DATA__` to the flight payload) | Parser is a single pure function behind a provider method; fixture test fails loudly at build time; parse failure is non-fatal and keeps the last known price | +| Cloudflare / UA sensitivity (verified: `civitai.com`, `civitai.red` and even no-UA all returned the payload for the sampled pages) | Reuse `downloader` (proxy, rate-limit coordinator, connectivity guard); detect challenge/absence and back off; never retry in a tight loop | +| Traffic/size (~200-270 KB per model) | Gated models only (~2%), one request per model, `price_check_ttl_hours`, force-refresh only on user action | +| False "free" alerts from gate-semantics ambiguity | P0 helper + explicit tests before any alert ships | +| Misleading price (list vs sale-adjusted; Blue Buzz is the same number but non-withdrawable) | Store both list and effective price; tooltip states the unit and Blue Buzz acceptance | +| Alerts are pull-only (no scheduler, no push channel today) | State it in the UI copy; treat background push as a separate follow-up, not a silent gap | + +## 8. Out of scope / follow-ups + +* Purchasing or downloading gated content. +* Generation-fee tracking beyond displaying `generation_price_buzz` (the public `mini/{id}` `fees` + field could feed a richer view later). +* A background/notification channel; today alerts appear when the user refreshes or opens the panel. +* Waiting on CivitAI to expose prices; P1–P3 stand alone. + +## 9. Open questions + +1. `{permanent:false, endsAt:null}` — gate or not? (P0; affects download blocking too.) +2. Should the threshold apply to versions already in the library (watching for a future re-buy) or + only to versions the user does not own? +3. Is a per-model threshold override needed in v1, or is one global threshold enough? + +--- + +## 10. What shipped (P0–P3) + +### 10.1 Answers to the open questions + +1. **`{permanent:false, endsAt:null}` is a gate.** Decided by evidence, not preference: the public v1 + API returns `null` for lapsed gates (`toPublicPaidAccessDto` filters on `isPaidAccessActive`), so a + non-null DTO is always an *active* gate; on the live model 1802980 that version reports + `canDownload: false` while its lapsed-tombstone siblings report `true`. The previous rule dropped + it, which is exactly the #1060 class of bug. +2. The global threshold applies to **every** gated version the update service tracks, in library or + not — the user may be watching a version they intend to buy later. +3. One global threshold in v1; a per-model override is still open. + +### 10.2 Files and seams + +| Area | Where | +| --- | --- | +| Shared gate semantics (`normalize_paid_access`, `is_gate_active`, `is_early_access_deadline_active`) | `py/utils/paid_access.py` (new) | +| Model page price parser (`parse_model_page_prices`) | `py/utils/civitai_page_prices.py` (new) | +| Price fetch | `CivitaiClient.get_model_prices` (`py/services/civitai_client.py`) | +| Provider seam | `ModelMetadataProvider.get_model_prices` default None; CivitAI impl; fallback + rate-limit wrappers (`py/services/model_metadata_provider.py`) | +| Columns, migration, transition + threshold logic, alerts query | `py/services/model_update_service.py` | +| Version payload, refresh `events`, `GET /api/lm/{type}/updates/price-alerts` | `py/routes/handlers/model_handlers.py`, `py/routes/model_route_registrar.py` | +| Badges, price chip, gate/price toast | `static/js/components/shared/ModelVersionsTab.js`, `static/js/utils/updateCheckHelpers.js`, `static/css/components/lora-modal/versions.css` | +| Settings | defaults in `py/services/settings_manager.py`; UI in `templates/components/modals/settings/library.html` + `static/js/managers/SettingsManager.js`; client defaults in `static/js/state/index.js` | +| API client | `getPriceAlerts` in `static/js/api/baseModelApi.js` + `priceAlerts` endpoint in `static/js/api/apiConfig.js` | + +### 10.3 Deviations from the plan + +* **`gate_lapsed_at` column added** (8th column): a transient event would leave a "became free" + marker invisible minutes later, so the lapse timestamp is persisted and the UI shows a `Free Now` + badge with it. +* **Alerts are edge-triggered, and first sight is silent.** When a price first appears under the + threshold, `price_alert_state` is stored and the version shows up in `price-alerts` immediately, + but no toast fires — otherwise switching the feature on would announce every cheap version in the + library at once. The toast fires when a price *crosses* down through the threshold. +* **No dedicated alerts panel.** The surfaces are: price chip + `Free Now` badge in the versions tab, + one combined toast per update check, and the `price-alerts` endpoint (GET, so the companion + extension can use it). A browse surface over the endpoint is a follow-up. +* **Prices are only fetched for models with a gate** (~2% of a typical library), one page request per + model, on `price_check_ttl_hours`, and immediately when the gate DTO changes. +* **The paid-download warning message does not include the price**: that path reads the model's + CivitAI metadata, which has no price, and looking one up would add a request to a failure path. + Left as a follow-up. + +### 10.4 Test coverage added + +* `tests/utils/test_paid_access.py` — gate shapes, activity, timestamps. +* `tests/utils/test_civitai_page_prices.py` + `tests/utils/fixtures/civitai_model_page_paid.html` — + parser, including sale-adjusted vs list price and lapsed tombstones. +* `tests/services/test_model_update_service.py` — migration from a pre-feature DB, column round-trip, + transitions, price capture on/off, failure retention, TTL, threshold edges, alerts query. +* `tests/services/test_civitai_client.py` — page fetch happy path, unusable payloads, rate limits. +* `tests/routes/test_model_update_handler.py` — alerts endpoint, gate events for non-updating records. + +--- + +## 11. P5 — Price alerts panel + +**Goal:** a single surface that answers "what got cheaper / became free, and can I act on it". + +**Status:** **P5a implemented** (see §11.9); P5b (inline threshold editing, *Refresh prices*, type +chips, ignore action) and P5c (polish) are not. + +**Naming:** called P4a–c in earlier discussion; renumbered to **P5** because §5 already uses P4 for +"docs and upstream". + +**Owner decision:** the grid-level "price alert only" filter is **out of scope**. The panel is the +only new browsing surface; the existing per-version badges and the update-check toast stay as they +are. Consequence: the panel carries the "act on it" affordances itself (§11.2), and discoverability +rests on the bell badge plus the controls-dropdown entry (§11.1 D8). + +### 11.1 Locked decisions + +| # | Decision | Why / evidence | +| --- | --- | --- | +| D1 | Host it as a **third tab in the existing notification bell modal**, driven by `UpdateService` | `templates/components/modals.html` is included by `templates/base.html:75`, so the bell exists on every page; `UpdateService` already has `toggleUpdateModal()`, `switchNotificationTab()`, tab badges, arrow-key tab navigation and a list/empty-state pattern (`renderRecentBanners`) | +| D2 | One **global** endpoint `GET /api/lm/price-alerts`, registered **once** in `MiscRoutes` | The update DB is one file per library shared by all model types (`cache/model_update/.sqlite`), and `ServiceRegistry.get_model_update_service()` returns one shared instance (`py/services/service_registry.py:153`). `get_price_alerts` only filters on `s.model_type`, so `model_type=None` is all types in one query. Adding it to `COMMON_ROUTE_DEFINITIONS` would bind the same path once per model type (4×) | +| D3 | **Compare the threshold at read time** (`WHERE v.price_buzz <= ?`); keep `price_alert_state` for the toast edge only | As shipped, panel membership only changes after a refresh, so editing the threshold in the panel would look broken | +| D4 | New column `price_alert_since REAL` | `price_alert_state` is a boolean; "dropped 3 days ago" and an unread count both need the moment the state flipped, which the existing edge detection already knows | +| D5 | Unread state is **client-side** (`localStorage` watermark compared against `price_alert_since`) | No per-user read-state table and no sync logic; the worst case is a conservative badge on a second browser | +| D6 | Row actions: **CivitAI** always; **Open** only when the payload resolved a local file path | `showModelModal(model, modelType)` needs a local metadata object and re-fetches by `file_path` (`static/js/components/shared/ModelModal.js:343-361`); a gated version the user does not own has no local file | +| D7 | **Refresh prices** must bypass the price TTL | Prices stay fresh for `price_check_ttl_hours` (24 h by default), so without a force flag the button would appear to do nothing | +| D8 | **Two non-permanent entry points**, both opening the bell on the Price alerts tab: the **updates dropdown** in the controls bar, and the **global context menu** (right-click on empty space) | The controls dropdown already hosts `checkUpdatesMenuItem` (`templates/components/controls.html:114-126`) and costs no layout space. The global menu is the established home for library-wide occasional actions and already holds the sibling `check-model-updates` (`templates/components/context_menu.html:185`); it has only 7 items and existing hide/separator machinery (`GlobalContextMenu.showMenu` / `_updateSeparatorVisibility`). The **model-card menu is deliberately left alone** — it already has 20 items, and P5b's *Refresh prices* covers the per-model case | +| D9 | Bell badge shows the **unread alert count**; when there are none it keeps today's behaviour (dot for app updates). The same cached count is appended to the global context menu item (`Price alerts (3)`) | Otherwise a price drop is invisible until the user opens the bell. The count is fetched once on init **only when `price_tracking_enabled`**, so users who never enable the feature pay nothing | +| D10 | The global context menu item is **always visible** on model pages (hidden on recipes, like its siblings) | The panel's disabled state is the explanation plus a deep link into Settings; hiding the entry would make the feature undiscoverable for exactly the users who have not enabled it yet | + +### 11.2 Information architecture + +``` +┌ Price alerts [Refresh prices] ┐ +│ Threshold: 500 Buzz (click to edit) │ +│ [ Under threshold ] [ Became free ] Type: All · LoRA · … │ +├──────────────────────────────────────────────────────────────────────┤ +│ ▸ Glorious Art · Checkpoint · 2 versions │ +│ Alpha · 250 Buzz (was 500) · Blue Buzz OK │ +│ EA until Oct 10 · Not in library · dropped 3 days ago │ +│ [CivitAI] [Open] [Ignore] │ +│ ▸ Eira Kishida · LoRA · 1 version │ +│ 125 Buzz · In library · dropped today [CivitAI] [Open] │ +└──────────────────────────────────────────────────────────────────────┘ +``` + +* Segments cover both halves of the FR: `Under threshold` (price at or below the threshold) and + `Became free` (versions with `gate_lapsed_at`, no price). +* Grouped by model (a model often has several versions at the same price), sorted cheapest first + then most recently crossed; the type chips filter client-side over the loaded list. +* Row: effective price with the list price struck through when on sale, Blue Buzz note, EA end date, + in-library marker, and "dropped X ago" from `price_alert_since`. +* Actions: open on CivitAI, open the local model modal (only when a file path was resolved), ignore + this version (reuses `setVersionUpdateIgnore`). + +### 11.3 Data contract + +``` +GET /api/lm/price-alerts?limit=200 +{ + "success": true, + "enabled": true, // price_tracking_enabled + "thresholdBuzz": 500, + "newestCheckedAt": 1791039694.5, // staleness copy ("prices last checked X ago") + "alerts": [ + { + "modelId": 2981320, "modelType": "checkpoint", + "modelName": "Glorious Art", // best-effort, from the scanner cache + "versionId": 3379626, "versionName": "Alpha", + "kind": "below_threshold", // or "became_free" + "priceBuzz": 250, "listPriceBuzz": 500, + "acceptsBlueBuzz": true, "priceSaleEndsAt": null, + "priceAlertSince": 1791000000.0, + "gateLapsedAt": null, + "earlyAccessEndsAt": null, "isPaid": true, "isEarlyAccess": false, + "isInLibrary": false, + "filePath": null, // set only when resolvable + "civitaiUrl": "https://civitai.com/models/2981320?modelVersionId=3379626" + } + ] +} +``` + +One list with a `kind` discriminator (not two lists) so "All" needs no second request. `civitaiUrl` +is built server-side with the existing `build_civitai_model_page_url` so `civitai_host` stays the +single source of truth. + +### 11.4 States (all three must be designed, not just the happy path) + +1. **Tracking off** — explain that CivitAI publishes no price in its public API and that the feature + reads the model page, plus a button that opens Settings → Library. +2. **On, nothing matching** — "nothing under N Buzz right now", with the threshold editable inline. +3. **Stale / offline / rate-limited** — keep the last known list, add a "prices last checked X ago" + line and a retry; never blank the panel. + +### 11.5 Tasks + +**P5a — panel usable end to end** + +1. `py/services/model_update_service.py` + * `get_price_alerts(model_type=None, *, threshold_buzz, limit=200)` — optional type filter, live + threshold comparison, `kind`, `price_alert_since`; keep the shipped per-type call site working. + * `price_alert_since` column through all eight enumerations (§4.1), set on the `0 → 1` edge and + cleared on `1 → 0` inside `_build_record_from_remote`. +2. `py/routes/misc_route_registrar.py` + `py/routes/handlers/misc_handlers.py` — register + `GET /api/lm/price-alerts` **once**, using `ServiceRegistry.get_model_update_service()` and the + settings service; resolve `modelName` / `filePath` best-effort per model type from the scanner + caches (`version_index[version_id]` → `file_path`, `file_name`; item `model_name` with a + `file_name` fallback), and omit them when resolution fails. +3. `templates/components/modals/update_modal.html` — third tab + panel skeleton (segments, threshold + row, list container, empty/disabled blocks), reusing `data-notification-tab` / + `data-notification-panel`. +4. `static/js/managers/UpdateService.js` — `renderPriceAlerts()`, panel fetch on open, tab badge + count, threshold display, row rendering and actions, `localStorage` watermark; extend + `updateTabBadges()` and `switchNotificationTab()`. +5. `static/js/api/apiConfig.js` + a small fetch helper — the endpoint is global, so it does not + belong in the per-model-type `endpoints` map. +6. `templates/components/controls.html` + `static/js/components/controls/PageControls.js` — the + controls-dropdown item (D8). +7. `templates/components/context_menu.html` + `static/js/components/ContextMenu/GlobalContextMenu.js` + — the global-context-menu item (D8): one template entry, the recipes-page hide list in + `showMenu()`, one `case` in `handleMenuAction`, and the optional `(N)` count from the cached + alert count. Both entries call one shared `openPriceAlertsPanel()` helper. +8. `locales/en.json` + `python scripts/sync_translation_keys.py`. + +*Acceptance:* with tracking on and one refresh done, the bell shows a count, the tab lists the +versions with prices, CivitAI opens the right page, Open appears only for in-library models and opens +the modal, all three states render, and it works from every page type. + +**P5b — actions and polish** + +* Inline threshold editing (read-time comparison makes it instant) and **Refresh prices** with a + `force_price_refresh` flag on the existing refresh endpoint (or a dedicated + `POST /api/lm/price-alerts/refresh`). +* Type chips and a "only versions I do not own" toggle (this is open question 2 from §9). +* Ignore-this-version action. + +**P5c — optional** + +* Thumbnails from the scanner cache, "unread only", focus-management pass on the new tab, and a + deep link from the panel into the settings section. + +### 11.6 Tests + +* `tests/services/test_model_update_service.py` — `model_type=None` spans types; live threshold + parameter; `price_alert_since` set/cleared on the edge. +* `tests/routes/` — the global route is registered exactly once and returns the documented shape; + `filePath` resolution is best-effort (mock scanner; omit on failure). +* `tests/frontend/` — vitest for `renderPriceAlerts` covering rows, both segments, the three states + and the badge count with a mocked fetch. +* Manual eyeball for layout, per the repo's UI verification policy. + +### 11.7 Risks + +| Risk | Mitigation | +| --- | --- | +| The bell modal is nominally about app updates; a third tab changes its character | Clear labelling, and only count/badge when the feature is on; the app-update dot behaviour is unchanged when there are no alerts | +| Resolving local file paths pulls scanner caches into an app-wide route | Best-effort with `try/except`, one `get_cached_data()` per type (in-memory), omit the field and hide the Open button when it fails | +| `price_alert_since` is another migration | Same eight-touchpoint discipline as §4.1, covered by the existing migration test | +| A global route in `COMMON_ROUTE_DEFINITIONS` would bind 4× | Register it in `MiscRoutes` instead (D2) | +| Menu bloat / entry sprawl | Two entries maximum (D8), both calling one helper; the 20-item model-card menu is explicitly untouched | +| Panel data goes stale between refreshes | Show `newestCheckedAt`; the refresh action (P5b) makes it explicit rather than silent | + +### 11.8 Open questions for the owner + +1. **Badge policy** — count unread alerts on the bell (recommended, D9), or leave the bell alone and + put the number only on the tab? +2. **Unread at all** — the `localStorage` watermark (recommended, D5), or simply "current matches" + with no read state? +3. **Threshold editing inside the panel** — convenient, but it silently changes a global setting from + a surface the user opened just to look. +4. **"Open in library"** — worth the scanner-cache coupling (D6), or should the row offer only + "CivitAI" plus a copy-link action? +5. **Thumbnails** — skip for weight (recommended) or show them? +6. **Entry visibility when tracking is off** — always show the global-context-menu item + (recommended, D10: the panel's disabled state educates and deep-links to Settings), or hide it + until the feature is enabled for a cleaner menu? +7. **Count in the menu label** — `Price alerts (3)` using the cached count (recommended, D9), or a + plain label with the number only on the bell/tab? + +### 11.9 What shipped (P5a) + +Answers to §11.8: badge counts unread alerts (D9), unread uses the `localStorage` watermark (D5), +threshold editing is **display-only in P5a** (editing is P5b), "Open in library" ships with the +best-effort path resolution (D6), no thumbnails, the context-menu item is always visible (D10), and +the menu label carries the count (D9). + +| Area | Where | +| --- | --- | +| `price_alert_since` column through all eight enumerations, set on the `0 → 1` edge, preserved while the alert stands, cleared when the price rises | `py/services/model_update_service.py` | +| `get_price_alerts(model_type=None, *, threshold_buzz, limit)` — read-time threshold, `kind`, one list across types; `newest_price_checked_at()` | `py/services/model_update_service.py` | +| Global endpoint `GET /api/lm/price-alerts`, registered once | `py/routes/misc_route_registrar.py`, `py/routes/misc_routes.py`, `PriceAlertsHandler` in `py/routes/handlers/misc_handlers.py` | +| Third bell tab, panel skeleton, segments, three states | `templates/components/modals/update_modal.html`, `static/css/components/modal/update-modal.css` | +| Loader, renderer, unread watermark, badge, `openPriceAlertsPanel()` | `static/js/managers/UpdateService.js` | +| Entry points (controls dropdown + global context menu, count in the label) | `templates/components/controls.html`, `static/js/components/controls/PageControls.js`, `templates/components/context_menu.html`, `static/js/components/ContextMenu/GlobalContextMenu.js` | + +Deviations and decisions made while implementing: + +* **`became_free` is reported even while price tracking is off.** It needs no price data, so the + panel shows the "tracking is off" explanation *and* whatever became free instead of hiding the + half of the feature that already works. +* **`price_alert_since` is also set on first sight** of an already-cheap version. The toast stays + silent (edge-triggered, §10.3), but the count is non-zero, which is what invites the user into the + panel after enabling the feature. +* **The per-type frontend client method was removed** (`baseModelApi.getPriceAlerts` and the + `priceAlerts` entry in `apiConfig.js`): with the global endpoint it was dead code. The per-type + **backend** route stays for the companion extension and a possible future grid filter. +* **`openPriceAlertsTab()` exists because `toggleUpdateModal()` closes an open bell** — an entry + point calling it unconditionally would dismiss the modal instead of switching tabs. +* **Local context uses the existing indexes** (`cache.model_id_index` for `model_name`, + `cache.version_index` for `file_path`/`file_name`), so it is O(1) per row; every failure just + omits the fields and the row loses its "Open" button. + +Verification: `pytest` 3652 passed / 7 skipped, `npm run test:js` 1444 passed, plus a sandboxed +standalone server run that seeded the update DB and confirmed the payload shape, the `kind` +split, the `civitaiUrl`, and that changing `price_alert_threshold_buzz` through `POST /api/lm/settings` +changes panel membership immediately with no refresh. + +### 11.10 Known limitation: mature (NSFW) model prices, and the host fallback + +End-to-end verification against the live site found that the page fetch was broken for a whole class +of users: **the hosts are not interchangeable**, and the user's `civitai_host` preference was +silently fatal. + +Measured with the app's own HTTP stack (aiohttp) and with httpx, browser User-Agent in both cases: + +| Target | civitai.com | civitai.green | civitai.red | +| --- | --- | --- | --- | +| Anonymously visible model page | 200, prices parse | 200, prices parse (same bytes as .com) | **403 Cloudflare challenge** | +| Mature (NSFW) model page | 404 | 404 | **403 Cloudflare challenge** | + +* `civitai.red` refuses non-browser clients outright — User-Agent, `Accept-Language`, `Sec-Fetch-*` + and switching HTTP library all make no difference. (An earlier manual check with `curl` passed by + luck of TLS fingerprint, which is why this was missed: it was a false positive.) +* A real user on `civitai_host=civitai.red` therefore captured **zero** prices + (`price_checked_at = 0` in their update DB) even with tracking enabled. +* Mature models are hidden from anonymous visitors on `.com`/`.green` and only served by `.red`, so + **no host can currently read their price.** The public API cannot help: it trims `paidAccess` to + `{permanent, endsAt}` by design, and the public `mini/{id}` endpoint exposes only per-generation + `fees`. + +What P5a now does about it: + +* **Host fallback** (`CivitaiClient.get_model_prices`): the configured host is tried first, then the + others (`civitai_page_host_candidates`), first parseable payload wins. The host that worked is + remembered, and a host that refuses outright (403) is parked for 15 minutes — a host-wide failure + must not cost three requests per mature model in the library. A 404 is model-specific and does + **not** park the host. +* **Honest "unavailable" state**: `price_check_attempted_at` separates "we tried and could not read + a price" from "we never looked". Gated versions in that state show a muted `Price unavailable` + badge in the versions tab, and the panel reports `unavailableCount`. +* **Diagnostics**: a host refusal is warned once per host per TTL, and a model with no price source + logs the per-host reasons (404 vs challenge) instead of failing silently at debug level. + +Recorded options for mature models, deliberately **not** implemented: + +1. **Internal tRPC with the user's API key** (`modelVersion.getById` on `.red`): the route is a + `publicProcedure` with `requiredScope: ModelsRead`, and `isBearerAuth` satisfies + `acceptableOrigin`, so the user's own key would work and `.red`'s `/api` paths are not challenged. + Rejected as the default because the endpoint is undocumented and its own 401 message says to use + the public API. If ever wanted, it belongs behind an off-by-default setting. +2. **Extension-assisted fetch**: `lm-civitai-extension` runs inside the user's browser, so it has + both the Cloudflare clearance and the login session needed to read mature pages. This is the only + route that would work without an undocumented API, but it is a cross-component design of its own. + +**Strengthened upstream ask:** the public API should expose the price. The argument is no longer +"convenience" — the page route is demonstrably unreliable (one host challenges non-browser clients, +the other two hide mature models from anonymous visitors), so a supported field is the only way for +any third-party tool to show prices for the models where creators monetize most. + +### 11.11 Price capture is independent of the metadata TTL + +Found in a real instance: after enabling price tracking, a normal "Check updates" captured exactly +**one** price out of 718 models. The price capture only ran when the *version list* was re-fetched +(`refresh_succeeded`), so it inherited the metadata TTL: with 24 h metadata and 24 h price TTLs, only +the 6 models whose metadata happened to be stale were ever priced. + +The cached record already carries the gate, so the price pass now runs off whichever version list is +available — freshly fetched or stored — and applies the result without touching `last_checked_at`, so +a price-only pass cannot silently extend the metadata TTL. Two related semantics: + +* A **failed** attempt (`price_check_attempted_at`) satisfies the price TTL, so a mature model whose + page no host will serve is not retried on every single update check. +* An explicitly **forced** check re-prices within the TTL (`_should_fetch_prices(..., force=True)`). + +Verified by copying a real instance's update DB into a sandbox and running a non-forced check: +`bulk metadata fetches: 0` (version lists came entirely from cache) while priced versions went +**1 → 20** and the panel listed 19 alerts. + +### 11.12 The empty state has to explain a 0 Buzz threshold + +The default threshold is 0 ("only tell me when a version becomes free"), and the settings copy says +so. The panel did not: a real instance with **52 priced paid versions** and an untouched threshold +showed "Nothing is under your price threshold right now" plus a small "Alert threshold: 0 Buzz" in +the corner, which reads as "the feature is broken". + +The empty state is now threshold-aware, using a new `pricedCount` in the payload: when the threshold +is 0 and prices are known, it says how many paid versions have a price and where to set a threshold. +The read-time threshold comparison means changing it takes effect immediately — no re-check needed +(measured on a copy of that instance's DB: 0 Buzz -> 0 alerts, 100 -> 44, 500 -> 48, 5000 -> 51). + +## 12. Redesign (locked): obtainability of versions you do not own + +### 12.1 Why the P5a model was wrong + +The owner could not tell, from the UI alone, what "Buzz Price Tracking" enables, what the +"Price alert threshold" number means, or what "Price Alerts" is alerting about. That is not a copy +problem: the implementation exposed **our mechanism** (a page scrape) and **our SQL predicates** +(`price_buzz <= ?`, `gate_lapsed_at IS NOT NULL`) as the user's concepts. Two concrete defects came +out of the same root: + +* **The alert population included versions the user already owns.** Neither the event generator nor + the panel query filtered on `is_in_library` (the payload even carried `isInLibrary`). In the + owner's library **28 of 52 gated versions were already downloaded** — more than half of the + "alerts" were about things already on disk. A version you own cannot become cheaper *for you*. +* **A filtered list in a notification surface.** "Price Alerts" was a state list whose contents were + entirely decided by a number stored in Settings, so an empty panel had three indistinguishable + causes (tracking off / threshold 0 / genuinely nothing), and it read as a broken feature. + +Two measurements shaped the correction: + +* **Permanent prices dominate** (87 permanent vs 3 timed gates in a 600-model sample), so "wait for a + price drop" is a low-frequency proposition, while "wait for early access to end" is high-frequency + and deterministic. +* **Prices cluster at the floor** (44 of the owner's 52 priced versions are 100 Buzz), so + "price <= N" carries little information; the real distinction is *free / soon free / paid*. + +Also corrected during the discussion: the **generation fee** is the cost of generating *on the +CivitAI site*. A locally downloaded model generated in ComfyUI has no such fee, so for this tool it +is not a price axis at all (42 of the owner's 52 gated versions carry one — displaying it would +mislead on 80% of paid rows). It is not displayed today and must stay that way. + +### 12.2 The corrected information model + +The unit is the **version**, and the boundary is **ownership**: information exists only where a +decision exists. + +| State of a version **not in the library** | The user's decision | What is shown | +| --- | --- | --- | +| Downloadable for free | download or not | nothing special (the download action already says it) | +| Early access, free on D, buyable for N | **wait until D / pay N now** | `N Buzz - free on D` | +| Early access, free on D, not buyable | wait | `free on D` | +| Permanently paid, price known (N) | buy or skip | `N Buzz` | +| Permanently paid, price unknown | buy or skip, price unknown | `Paid` (no number) | +| **In the library** | none | **nothing** | + +The numeric threshold has no place in this table: every decision is categorical (wait / pay / skip), +not numeric. + +### 12.3 Locked decisions + +* **D1 - No independent surface.** Obtainability is an attribute of the existing update surfaces: + the model card's update badge, the grid's "models with updates" filter, the post-check toast, and + the model modal's version list. No new bell tab. +* **D2 - No numeric threshold anywhere.** The setting, the comparison, and the tests around it go. +* **D3 - Only versions that are not in the library** produce events or badges (`is_in_library = 0`), + enforced in both the event generator and any query. +* **D4 - Owned versions show no download price** (and carry no alert state). +* **D5 - The generation fee is never displayed.** +* **D6 - Models outside the library are out of scope.** Their decision moment is the download/browse + flow; that is a recorded follow-up, not a background feed (the update service only knows local + models, and there is no follow mechanism to build on). +* **D7 - The plumbing stays**: page fetch, host fallback, attempt markers, both TTLs, storage, + parser, `gate_lapsed_at`, and the price columns. + +### 12.4 What happens to what shipped in P5a + +| Piece | Fate | +| --- | --- | +| `GET /api/lm/price-alerts`, `GET /api/lm/{prefix}/updates/price-alerts`, `PriceAlertsHandler` | **removed** - no surface consumes them; events reach the UI through the refresh response | +| `get_price_alerts`, `newest_price_checked_at`, `count_priced_versions`, `count_unavailable_prices` | **removed** (threshold comparison and the panel's counters) | +| `price_alert_threshold_buzz` (settings, `_price_alert_threshold_buzz`) | **removed** | +| `price_alert_since` | **stop writing**; the column stays (SQLite drops need a table rebuild) and is marked deprecated | +| `price_alert_state` | **kept** - it is the one-shot edge for the "became free" toast | +| Bell tab, panel markup, `.price-alerts*` CSS, `UpdateService` panel code, watermark, the two entry points (controls dropdown, global context menu), their locales | **removed** | +| Version badges | **reworked** to the 12.2 table, ownership-scoped | +| "Price unavailable" badge | **replaced** by `Paid` (the gate is known from the public API; only the number is best-effort, and that is our plumbing, not the user's problem) | +| `priceAttemptedAt` in the version payload | **removed** from the payload (kept internally for the price-TTL retry policy) | +| The "became free" toast | **kept**, reworded, ownership-scoped | + +### 12.5 Tasks + +**P6a - the core correction (no new surface).** +Backend: ownership-scope the event generator; remove the threshold, the alert query, the counters, +the two routes and the handler; stop writing `price_alert_since`. Frontend: rework the version +badges to the 12.2 table; delete the panel, tab, entry points, watermark and their locales; reword +the toast. Tests: replace the panel/alert suites with a badge-state matrix and ownership-scoping +tests; assert the removed routes are gone. + +**P6b - obtainability in the update surfaces.** +Per-model obtainability summary in the model list payload (next to `hasUpdate`) so the card badge can +say `Update - 500 Buzz` / `Update - free on Oct 15`; toast copy that counts the free ones +("3 models have updates, 1 is now free"). + +**P6c - deferred (recorded, not planned).** A price-drop event (rare by measurement); obtainability +in the download/browse flow for models outside the library (D6). + +### 12.6 Verification + +Unit tests for the state matrix and the ownership filter; a sandbox run against a **copy** of the +owner's real update DB, where the population must shrink from 52 gated versions to the **24 that are +not in the library**, with the owned ones showing nothing. + +### 12.7 What shipped (P6a) + +* The two alert-state columns are **gone from the schema** (not merely unused): the dataclass, + `_SCHEMA`, the additive migration table, the primary-key rebuild's column list and defaults, the + bulk `SELECT`, the row mapping and the `INSERT`. A database created by an unreleased build still + has them, so `_apply_migrations` drops them with a native `ALTER TABLE ... DROP COLUMN` (SQLite + 3.35+, guarded) and logs it; for everyone else it is a no-op. +* `price_alert_threshold_buzz`, `_price_alert_threshold_buzz`, `_evaluate_price_alert`, the + threshold-crossing event, `get_price_alerts`, `newest_price_checked_at`, + `count_priced_versions`, `count_unavailable_prices`, both alert routes, `PriceAlertsHandler`, its + handler-set slot, its route mapping entry and the service-registry adapter field it needed: all + removed. +* Gate events (and only gate events) are emitted, and only for versions the user does not have - + `isInLibrary` is now the filter rather than a field in the payload. +* Version badges follow the 12.2 table: cost information only for versions not in the library, the + price replaces the redundant `Paid` badge when it is known, `Paid` appears without a number when + it is not, and `Price unavailable` is gone (the gate is certain; only the number is best-effort). +* The bell tab, panel markup, panel CSS, the controls-dropdown and global-context-menu entries, the + unread watermark and their locales are removed; the toast keeps its gate transitions and drops the + price-drop line. The setting is now framed as plumbing ("Show download prices for paid versions") + and keeps only the enable flag and the refresh interval. + +Verified: `pytest` 3653 passed / 7 skipped, `npm run test:js` 1444 passed. Against a copy of the +owner's real database the population is 52 gated versions -> **28 owned (now silent) + 24 that the +feature is actually about**; the drop migration logged twice and left no `price_alert_*` column, and +both removed endpoints return 404. + +### 12.8 Correction to P6b: the model card cannot carry a price + +P6b proposed annotating the model card's update badge with `Update - 500 Buzz` / +`Update - free on Oct 15`. The owner objected that the badge represents a **model** already in the +library while the update is a set of versions, each with its own price and its own early access end +date. The objection is correct, and the data is blunt about it: + +* Live model `958009` has **37 gated versions across 6 price points** (1111, 2220, 4440, 8880, + 88800, 888888); model `153568` has two gated versions at 600 and 4000. +* In the owner's own library, model `647926` has two early access versions with **two different end + dates**. + +So a single price or date on the card would be fabricated, not summarised. It looked plausible only +because that library is uniform (44 of its 52 priced versions sit at the 100 Buzz floor) - an +artefact of floor pricing, not a guarantee. + +What is well defined at model level, by construction: counts ("3 updates"), the earliest upcoming +free date (a minimum over the set), a minimum price ("from 100 Buzz", well defined but misleading +when the cheap version is an old one), and uniformity statements only when they are true. "The price +is X" is not among them. + +Revised scope: the card badge stays a plain update indicator, and **prices stay in the version list**, +where every row has exactly one subject. The only model-level fact worth considering is the deadline +- "1 of 3 updates becomes free on Oct 15" - because a deadline can change what the user does *before* +opening the model; it is rare in practice (4 unexpired early access versions across 3 models in that +library), so it is optional polish rather than part of the correction. diff --git a/locales/de.json b/locales/de.json index c39795ef..465bf26f 100644 --- a/locales/de.json +++ b/locales/de.json @@ -201,7 +201,11 @@ "loading": "Prüfe auf {type}-Updates...", "success": "{count} Update(s) für {type} gefunden", "none": "Alle {type} sind auf dem neuesten Stand", - "error": "Fehler beim Prüfen auf {type}-Updates: {message}" + "error": "Fehler beim Prüfen auf {type}-Updates: {message}", + "gateEvents": { + "becameFree": "{count} Version(en), die Sie nicht besitzen, sind jetzt kostenlos", + "newGate": "{count} Version(en) sind jetzt kostenpflichtig" + } }, "cleanupExampleImages": { "label": "Beispielbild-Ordner bereinigen", @@ -387,6 +391,7 @@ "downloadPathTemplates": "Download-Pfad-Vorlagen", "priorityTags": "Prioritäts-Tags", "versionScope": "Update-Markierungen", + "priceTracking": "Buzz-Downloadpreise", "exampleImages": "Beispielbilder", "autoOrganize": "Auto-Organisierung", "metadata": "Metadaten", @@ -759,6 +764,13 @@ "label": "Bezahlte Updates ausblenden", "help": "Wenn aktiviert, zeigen Modelle mit nur bezahlten Updates kein 'Update verfügbar'-Badge an" }, + "priceTracking": { + "help": "Die öffentliche API von CivitAI veröffentlicht keine Downloadpreise, daher werden sie von der Modellseite gelesen - eine zusätzliche Anfrage pro bezahltem Modell, aktualisiert im unten stehenden Intervall. Preise werden nur bei Versionen angezeigt, die Sie noch nicht haben.", + "enabled": "Downloadpreise für bezahlte Versionen anzeigen", + "enabledHelp": "Liest den Downloadpreis von der Modellseite, damit bezahlte Versionen und Versionen mit frühem Zugriff ihren Preis und das Ende des frühen Zugriffs zeigen", + "refreshHours": "Aktualisierungsintervall für Preise (Stunden)", + "refreshHoursHelp": "Wie lange ein gespeicherter Preis gültig bleibt, bevor er erneut gelesen wird" + }, "licenseIcons": { "useNewStyle": "Aktualisierte Lizenzsymbole verwenden", "useNewStyleHelp": "Lizenzberechtigungen mit farbigen Indikatoren (neuer Stil) oder nur Einschränkungssymbolen (klassischer Stil) anzeigen. Orientiert sich am aktuellen CivitAI-Design." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "Für diese Version ist derzeit CivitAI Early Access erforderlich", "paid": "Bezahlt", "paidTooltip": "Diese Version erfordert eine Zahlung zum Herunterladen", + "priceSaleTooltip": "Im Angebot, normalerweise {list} Buzz", + "priceBlueBuzzTooltip": "Kann auch mit Blue Buzz bezahlt werden", + "freeNow": "Jetzt kostenlos", + "freeNowTooltip": "Diese Version erfordert keine Zahlung mehr", "ignored": "Ignoriert", "ignoredTooltip": "Für diese Version sind Update-Benachrichtigungen deaktiviert", "onSiteOnly": "Nur On-Site", - "onSiteOnlyTooltip": "Diese Version ist nur für die On-Site-Generierung auf CivitAI verfügbar" + "onSiteOnlyTooltip": "Diese Version ist nur für die On-Site-Generierung auf CivitAI verfügbar", + "earlyAccessTooltipUntil": "Früher Zugriff bis {date}, danach kostenlos zum Herunterladen" }, "actions": { "download": "Herunterladen", diff --git a/locales/en.json b/locales/en.json index 12c7fe64..c29ade8e 100644 --- a/locales/en.json +++ b/locales/en.json @@ -201,7 +201,11 @@ "loading": "Checking for {type} updates...", "success": "Found {count} update(s) for {type}s", "none": "All {type}s are up to date", - "error": "Failed to check for {type} updates: {message}" + "error": "Failed to check for {type} updates: {message}", + "gateEvents": { + "becameFree": "{count} version(s) you do not have became free", + "newGate": "{count} version(s) are now paid" + } }, "cleanupExampleImages": { "label": "Clean up example image folders", @@ -387,6 +391,7 @@ "downloadPathTemplates": "Download Path Templates", "priorityTags": "Priority Tags", "versionScope": "Version Scope", + "priceTracking": "Buzz Download Prices", "exampleImages": "Example Images", "autoOrganize": "Auto-organize", "metadata": "Metadata", @@ -759,6 +764,13 @@ "label": "Hide Paid Updates", "help": "When enabled, models with only paid updates will not show 'Update available' badge" }, + "priceTracking": { + "help": "CivitAI does not publish download prices in its public API, so they are read from the model page - one extra request per paid model, refreshed on the interval below. Prices are shown on versions you do not have yet.", + "enabled": "Show download prices for paid versions", + "enabledHelp": "Reads the download price from the model page so paid and early access versions show what they cost, and when early access ends", + "refreshHours": "Price refresh interval (hours)", + "refreshHoursHelp": "How long a stored price stays fresh before it is read again" + }, "licenseIcons": { "useNewStyle": "Use updated license icons", "useNewStyleHelp": "Display license permissions with colored indicators (new style) or restriction-only icons (classic style). Mirroring the current CivitAI design." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "This version currently requires CivitAI early access", "paid": "Paid", "paidTooltip": "This version requires payment to download", + "priceSaleTooltip": "On sale, normally {list} Buzz", + "priceBlueBuzzTooltip": "Can also be paid with Blue Buzz", + "freeNow": "Free Now", + "freeNowTooltip": "This version no longer requires payment", "ignored": "Ignored", "ignoredTooltip": "Update notifications are disabled for this version", "onSiteOnly": "On-Site Only", - "onSiteOnlyTooltip": "This version is only available for on-site generation on CivitAI" + "onSiteOnlyTooltip": "This version is only available for on-site generation on CivitAI", + "earlyAccessTooltipUntil": "Early access until {date}, free to download after that" }, "actions": { "download": "Download", diff --git a/locales/es.json b/locales/es.json index 5b98470a..21285964 100644 --- a/locales/es.json +++ b/locales/es.json @@ -201,7 +201,11 @@ "loading": "Buscando actualizaciones de {type}...", "success": "Se encontraron {count} actualización(es) para {type}", "none": "Todos los {type} están actualizados", - "error": "Error al buscar actualizaciones de {type}: {message}" + "error": "Error al buscar actualizaciones de {type}: {message}", + "gateEvents": { + "becameFree": "{count} versión(es) que no tienes se han vuelto gratuitas", + "newGate": "{count} versión(es) ahora son de pago" + } }, "cleanupExampleImages": { "label": "Limpiar carpetas de imágenes de ejemplo", @@ -387,6 +391,7 @@ "downloadPathTemplates": "Plantillas de rutas de descarga", "priorityTags": "Etiquetas prioritarias", "versionScope": "Indicadores de actualización", + "priceTracking": "Precios de descarga en Buzz", "exampleImages": "Imágenes de ejemplo", "autoOrganize": "Organización automática", "metadata": "Metadatos", @@ -759,6 +764,13 @@ "label": "Ocultar actualizaciones de pago", "help": "Cuando está activado, los modelos que solo tienen actualizaciones de pago no mostrarán la insignia de 'Actualización disponible'" }, + "priceTracking": { + "help": "La API pública de CivitAI no publica los precios de descarga, así que se leen de la página del modelo: una petición adicional por modelo de pago, actualizada según el intervalo de abajo. Los precios se muestran solo en las versiones que aún no tienes.", + "enabled": "Mostrar los precios de descarga de las versiones de pago", + "enabledHelp": "Lee el precio de descarga de la página del modelo para mostrar cuánto cuestan las versiones de pago y de acceso temprano, y cuándo termina el acceso temprano", + "refreshHours": "Intervalo de actualización de precios (horas)", + "refreshHoursHelp": "Cuánto tiempo sigue vigente un precio guardado antes de volver a leerse" + }, "licenseIcons": { "useNewStyle": "Usar iconos de licencia actualizados", "useNewStyleHelp": "Mostrar permisos de licencia con indicadores de color (nuevo estilo) o solo iconos de restricción (estilo clásico). Refleja el diseño actual de CivitAI." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "Esta versión requiere actualmente acceso temprano de CivitAI", "paid": "De pago", "paidTooltip": "Esta versión requiere pago para descargarse", + "priceSaleTooltip": "En oferta, normalmente {list} Buzz", + "priceBlueBuzzTooltip": "También se puede pagar con Blue Buzz", + "freeNow": "Ahora gratis", + "freeNowTooltip": "Esta versión ya no requiere pago", "ignored": "Ignorada", "ignoredTooltip": "Las notificaciones de actualización están desactivadas para esta versión", "onSiteOnly": "Solo en Sitio", - "onSiteOnlyTooltip": "Esta versión solo está disponible para generación en el sitio de CivitAI" + "onSiteOnlyTooltip": "Esta versión solo está disponible para generación en el sitio de CivitAI", + "earlyAccessTooltipUntil": "Acceso temprano hasta el {date}, descarga gratuita después" }, "actions": { "download": "Descargar", diff --git a/locales/fr.json b/locales/fr.json index f45a4b30..366e35d8 100644 --- a/locales/fr.json +++ b/locales/fr.json @@ -201,7 +201,11 @@ "loading": "Recherche de mises à jour pour {type}...", "success": "{count} mise(s) à jour trouvée(s) pour {type}", "none": "Tous les {type} sont à jour", - "error": "Échec de la vérification des mises à jour pour {type} : {message}" + "error": "Échec de la vérification des mises à jour pour {type} : {message}", + "gateEvents": { + "becameFree": "{count} version(s) que vous ne possédez pas sont devenues gratuites", + "newGate": "{count} version(s) sont désormais payantes" + } }, "cleanupExampleImages": { "label": "Supprimer les dossiers d'exemples orphelins", @@ -387,6 +391,7 @@ "downloadPathTemplates": "Modèles de chemin de téléchargement", "priorityTags": "Tags prioritaires", "versionScope": "Indicateurs de mise à jour", + "priceTracking": "Prix de téléchargement en Buzz", "exampleImages": "Images d'exemple", "autoOrganize": "Organisation automatique", "metadata": "Métadonnées", @@ -759,6 +764,13 @@ "label": "Masquer les mises à jour payantes", "help": "Lorsque cette option est activée, les modèles n'ayant que des mises à jour payantes n'affichent pas le badge « Mise à jour disponible »" }, + "priceTracking": { + "help": "L'API publique de CivitAI ne publie pas les prix de téléchargement : ils sont donc lus sur la page du modèle - une requête supplémentaire par modèle payant, actualisée selon l'intervalle ci-dessous. Les prix sont affichés sur les versions que vous ne possédez pas encore.", + "enabled": "Afficher les prix de téléchargement des versions payantes", + "enabledHelp": "Lit le prix de téléchargement sur la page du modèle pour indiquer le coût des versions payantes et en accès anticipé, ainsi que la fin de l'accès anticipé", + "refreshHours": "Intervalle d'actualisation des prix (heures)", + "refreshHoursHelp": "Durée pendant laquelle un prix enregistré reste valide avant d'être relu" + }, "licenseIcons": { "useNewStyle": "Utiliser les icônes de licence mises à jour", "useNewStyleHelp": "Afficher les permissions de licence avec des indicateurs colorés (nouveau style) ou des icônes de restriction uniquement (style classique). Reprend le design actuel de CivitAI." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "Cette version nécessite actuellement l'accès anticipé CivitAI", "paid": "Payant", "paidTooltip": "Cette version nécessite un paiement pour être téléchargée", + "priceSaleTooltip": "En promotion, normalement {list} Buzz", + "priceBlueBuzzTooltip": "Peut aussi être payé avec des Blue Buzz", + "freeNow": "Gratuit maintenant", + "freeNowTooltip": "Cette version ne nécessite plus de paiement", "ignored": "Ignorée", "ignoredTooltip": "Les notifications de mise à jour sont désactivées pour cette version", "onSiteOnly": "Uniquement sur Site", - "onSiteOnlyTooltip": "Cette version n'est disponible que pour la génération sur le site CivitAI" + "onSiteOnlyTooltip": "Cette version n'est disponible que pour la génération sur le site CivitAI", + "earlyAccessTooltipUntil": "Accès anticipé jusqu'au {date}, téléchargement gratuit ensuite" }, "actions": { "download": "Télécharger", diff --git a/locales/he.json b/locales/he.json index 3bac6305..8103085f 100644 --- a/locales/he.json +++ b/locales/he.json @@ -201,7 +201,11 @@ "loading": "בודק עדכונים עבור {type}...", "success": "נמצאו {count} עדכונים עבור {type}", "none": "כל ה-{type} מעודכנים", - "error": "נכשל בבדיקת העדכונים עבור {type}: {message}" + "error": "נכשל בבדיקת העדכונים עבור {type}: {message}", + "gateEvents": { + "becameFree": "{count} גרסאות שאין לך הפכו לחינמיות", + "newGate": "{count} גרסאות דורשות כעת תשלום" + } }, "cleanupExampleImages": { "label": "נקה תיקיות תמונות דוגמה", @@ -387,6 +391,7 @@ "downloadPathTemplates": "תבניות נתיב הורדה", "priorityTags": "תגיות עדיפות", "versionScope": "תגי עדכון", + "priceTracking": "מחירי הורדה ב-Buzz", "exampleImages": "תמונות דוגמה", "autoOrganize": "ארגון אוטומטי", "metadata": "מטא-נתונים", @@ -759,6 +764,13 @@ "label": "הסתר עדכונים בתשלום", "help": "כשאפשרות זו מופעלת, מודלים עם עדכונים בתשלום בלבד לא יציגו את תגית 'עדכון זמין'" }, + "priceTracking": { + "help": "ה-API הציבורי של CivitAI אינו מפרסם מחירי הורדה, ולכן הם נקראים מדף המודל - בקשה נוספת אחת לכל מודל בתשלום, ברענון לפי המרווח שלמטה. המחירים מוצגים רק בגרסאות שאין לך עדיין.", + "enabled": "הצג מחירי הורדה של גרסאות בתשלום", + "enabledHelp": "קורא את מחיר ההורדה מדף המודל כדי להציג כמה עולות גרסאות בתשלום וגרסאות בגישה מוקדמת, ומתי מסתיימת הגישה המוקדמת", + "refreshHours": "מרווח רענון המחירים (שעות)", + "refreshHoursHelp": "כמה זמן מחיר שמור נחשב מעודכן לפני קריאה חוזרת" + }, "licenseIcons": { "useNewStyle": "השתמש בסמלי רישיון מעודכנים", "useNewStyleHelp": "הצג הרשאות רישיון עם מחוונים צבעוניים (סגנון חדש) או סמלי הגבלה בלבד (סגנון קלאסי). משקף את העיצוב העדכני של CivitAI." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "גרסה זו דורשת כרגע גישת Early Access של CivitAI", "paid": "בתשלום", "paidTooltip": "גרסה זו דורשת תשלום כדי להוריד", + "priceSaleTooltip": "במבצע, בדרך כלל {list} Buzz", + "priceBlueBuzzTooltip": "ניתן לשלם גם ב-Blue Buzz", + "freeNow": "חינם כעת", + "freeNowTooltip": "גרסה זו אינה דורשת עוד תשלום", "ignored": "התעלם", "ignoredTooltip": "התראות העדכון מושבתות עבור גרסה זו", "onSiteOnly": "רק באתר", - "onSiteOnlyTooltip": "גרסה זו זמינה רק ליצירה באתר CivitAI" + "onSiteOnlyTooltip": "גרסה זו זמינה רק ליצירה באתר CivitAI", + "earlyAccessTooltipUntil": "גישה מוקדמת עד {date}, לאחר מכן ההורדה חינם" }, "actions": { "download": "הורדה", diff --git a/locales/ja.json b/locales/ja.json index fcfe18d6..722caea0 100644 --- a/locales/ja.json +++ b/locales/ja.json @@ -201,7 +201,11 @@ "loading": "{type} のアップデートを確認中...", "success": "{type} のアップデートが {count} 件見つかりました", "none": "すべての {type} は最新です", - "error": "{type} のアップデート確認に失敗しました: {message}" + "error": "{type} のアップデート確認に失敗しました: {message}", + "gateEvents": { + "becameFree": "未ダウンロードの {count} バージョンが無料になりました", + "newGate": "{count} バージョンが有料になりました" + } }, "cleanupExampleImages": { "label": "例画像フォルダをクリーンアップ", @@ -387,6 +391,7 @@ "downloadPathTemplates": "ダウンロードパステンプレート", "priorityTags": "優先タグ", "versionScope": "アップデートフラグ", + "priceTracking": "Buzz ダウンロード価格", "exampleImages": "例画像", "autoOrganize": "自動整理", "metadata": "メタデータ", @@ -759,6 +764,13 @@ "label": "有料更新を非表示", "help": "有効にすると、有料の更新のみがあるモデルには「更新あり」バッジが表示されません" }, + "priceTracking": { + "help": "CivitAI の公開 API はダウンロード価格を提供していないため、モデルページから読み取ります。有料モデルごとに 1 回の追加リクエストが発生し、以下の間隔で更新されます。価格は未ダウンロードのバージョンにのみ表示されます。", + "enabled": "有料バージョンのダウンロード価格を表示", + "enabledHelp": "モデルページからダウンロード価格を読み取り、有料バージョンと早期アクセスバージョンの費用と早期アクセスの終了時期を表示します", + "refreshHours": "価格の更新間隔（時間）", + "refreshHoursHelp": "保存された価格が再読み取りまで有効な期間" + }, "licenseIcons": { "useNewStyle": "更新されたライセンスアイコンを使用", "useNewStyleHelp": "カラーインジケーター付きでライセンス許可を表示（新スタイル）するか、制限のみのアイコンを表示（クラシックスタイル）します。現在のCivitAIデザインを反映しています。" @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "このバージョンは現在 CivitAI の早期アクセスが必要です", "paid": "有料", "paidTooltip": "このバージョンのダウンロードには支払いが必要です", + "priceSaleTooltip": "セール中、通常 {list} Buzz", + "priceBlueBuzzTooltip": "Blue Buzz でも支払えます", + "freeNow": "無料化", + "freeNowTooltip": "このバージョンは支払いが不要になりました", "ignored": "無視中", "ignoredTooltip": "このバージョンの更新通知は無効です", "onSiteOnly": "サイト内のみ", - "onSiteOnlyTooltip": "このバージョンはCivitAIサイト内でのみ利用可能で、ダウンロードはできません" + "onSiteOnlyTooltip": "このバージョンはCivitAIサイト内でのみ利用可能で、ダウンロードはできません", + "earlyAccessTooltipUntil": "{date} まで早期アクセス、以降は無料でダウンロードできます" }, "actions": { "download": "ダウンロード", diff --git a/locales/ko.json b/locales/ko.json index d583214c..d8549300 100644 --- a/locales/ko.json +++ b/locales/ko.json @@ -201,7 +201,11 @@ "loading": "{type} 업데이트를 확인 중...", "success": "{type} 업데이트 {count}개를 찾았습니다", "none": "모든 {type}가 최신 상태입니다", - "error": "{type} 업데이트 확인 실패: {message}" + "error": "{type} 업데이트 확인 실패: {message}", + "gateEvents": { + "becameFree": "보유하지 않은 버전 {count}개가 무료가 되었습니다", + "newGate": "버전 {count}개가 유료로 전환되었습니다" + } }, "cleanupExampleImages": { "label": "예시 이미지 폴더 정리", @@ -387,6 +391,7 @@ "downloadPathTemplates": "다운로드 경로 템플릿", "priorityTags": "우선순위 태그", "versionScope": "업데이트 표시", + "priceTracking": "Buzz 다운로드 가격", "exampleImages": "예시 이미지", "autoOrganize": "자동 정리", "metadata": "메타데이터", @@ -759,6 +764,13 @@ "label": "유료 업데이트 숨기기", "help": "활성화하면 유료 업데이트만 있는 모델에 '업데이트 가능' 배지가 표시되지 않습니다" }, + "priceTracking": { + "help": "CivitAI 공개 API는 다운로드 가격을 제공하지 않으므로 모델 페이지에서 읽어옵니다. 유료 모델마다 요청이 한 번씩 추가되며, 아래 간격에 따라 갱신됩니다. 가격은 아직 보유하지 않은 버전에만 표시됩니다.", + "enabled": "유료 버전의 다운로드 가격 표시", + "enabledHelp": "모델 페이지에서 다운로드 가격을 읽어 유료 및 얼리 액세스 버전의 비용과 얼리 액세스 종료 시점을 표시합니다", + "refreshHours": "가격 갱신 간격(시간)", + "refreshHoursHelp": "저장된 가격을 다시 읽기 전까지 유지하는 기간" + }, "licenseIcons": { "useNewStyle": "업데이트된 라이선스 아이콘 사용", "useNewStyleHelp": "색상 표시기가 있는 라이선스 권한(새 스타일) 또는 제한 전용 아이콘(클래식 스타일)을 표시합니다. 현재 CivitAI 디자인을 반영합니다." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "이 버전은 현재 CivitAI 얼리 액세스가 필요합니다", "paid": "유료", "paidTooltip": "이 버전은 다운로드하려면 결제가 필요합니다", + "priceSaleTooltip": "할인 중, 원래 {list} Buzz", + "priceBlueBuzzTooltip": "Blue Buzz로도 결제할 수 있습니다", + "freeNow": "지금 무료", + "freeNowTooltip": "이 버전은 더 이상 결제가 필요하지 않습니다", "ignored": "무시됨", "ignoredTooltip": "이 버전은 업데이트 알림이 비활성화되어 있습니다", "onSiteOnly": "사이트 내 전용", - "onSiteOnlyTooltip": "이 버전은 CivitAI 사이트 내에서만 사용 가능하며 다운로드할 수 없습니다" + "onSiteOnlyTooltip": "이 버전은 CivitAI 사이트 내에서만 사용 가능하며 다운로드할 수 없습니다", + "earlyAccessTooltipUntil": "{date}까지 얼리 액세스, 이후 무료로 다운로드할 수 있습니다" }, "actions": { "download": "다운로드", diff --git a/locales/ru.json b/locales/ru.json index 61548626..7de2cb90 100644 --- a/locales/ru.json +++ b/locales/ru.json @@ -201,7 +201,11 @@ "loading": "Проверка обновлений для {type}...", "success": "Найдено {count} обновлений для {type}", "none": "Все {type} актуальны", - "error": "Не удалось проверить обновления для {type}: {message}" + "error": "Не удалось проверить обновления для {type}: {message}", + "gateEvents": { + "becameFree": "{count} версий, которых у вас нет, стали бесплатными", + "newGate": "{count} версий стали платными" + } }, "cleanupExampleImages": { "label": "Очистить папки с примерами изображений", @@ -387,6 +391,7 @@ "downloadPathTemplates": "Шаблоны путей загрузки", "priorityTags": "Приоритетные теги", "versionScope": "Метки обновлений", + "priceTracking": "Цены за скачивание в Buzz", "exampleImages": "Примеры изображений", "autoOrganize": "Автоорганизация", "metadata": "Метаданные", @@ -759,6 +764,13 @@ "label": "Скрывать платные обновления", "help": "Если включено, у моделей, для которых доступны только платные обновления, не будет отображаться значок «Доступно обновление»" }, + "priceTracking": { + "help": "Публичный API CivitAI не публикует цены за скачивание, поэтому они читаются со страницы модели — один дополнительный запрос на каждую платную модель, обновление с интервалом ниже. Цены показываются только для версий, которых у вас ещё нет.", + "enabled": "Показывать цены за скачивание платных версий", + "enabledHelp": "Читает цену за скачивание со страницы модели, чтобы платные версии и версии с ранним доступом показывали стоимость и дату окончания раннего доступа", + "refreshHours": "Интервал обновления цен (часы)", + "refreshHoursHelp": "Сколько хранится цена, прежде чем её прочитают снова" + }, "licenseIcons": { "useNewStyle": "Использовать обновлённые значки лицензии", "useNewStyleHelp": "Отображать разрешения лицензии с цветными индикаторами (новый стиль) или только значки ограничений (классический стиль). Соответствует текущему дизайну CivitAI." @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "Для этой версии сейчас требуется ранний доступ CivitAI", "paid": "Платная", "paidTooltip": "Скачивание этой версии платное", + "priceSaleTooltip": "Со скидкой, обычно {list} Buzz", + "priceBlueBuzzTooltip": "Можно также оплатить Blue Buzz", + "freeNow": "Теперь бесплатно", + "freeNowTooltip": "Эта версия больше не требует оплаты", "ignored": "Игнорируется", "ignoredTooltip": "Уведомления об обновлениях для этой версии отключены", "onSiteOnly": "Только на Сайте", - "onSiteOnlyTooltip": "Эта версия доступна только для генерации на сайте CivitAI" + "onSiteOnlyTooltip": "Эта версия доступна только для генерации на сайте CivitAI", + "earlyAccessTooltipUntil": "Ранний доступ до {date}, после этого скачивание бесплатно" }, "actions": { "download": "Скачать", diff --git a/locales/zh-CN.json b/locales/zh-CN.json index 26970753..ce63542a 100644 --- a/locales/zh-CN.json +++ b/locales/zh-CN.json @@ -201,7 +201,11 @@ "loading": "正在检查 {type} 更新...", "success": "找到 {count} 条 {type} 更新", "none": "所有 {type} 均已是最新版本", - "error": "检查 {type} 更新失败：{message}" + "error": "检查 {type} 更新失败：{message}", + "gateEvents": { + "becameFree": "{count} 个你未拥有的版本已免费", + "newGate": "{count} 个版本已变为付费" + } }, "cleanupExampleImages": { "label": "清理示例图片文件夹", @@ -387,6 +391,7 @@ "downloadPathTemplates": "下载路径模板", "priorityTags": "优先标签", "versionScope": "版本范围", + "priceTracking": "Buzz 下载价格", "exampleImages": "示例图片", "autoOrganize": "自动整理", "metadata": "元数据", @@ -759,6 +764,13 @@ "label": "隐藏付费更新", "help": "启用后，仅有付费更新的模型将不显示“有可用更新”徽标" }, + "priceTracking": { + "help": "CivitAI 的公开 API 不提供下载价格，因此价格从模型页面读取 —— 每个付费模型多一次请求，按下面的间隔刷新。价格只显示在你尚未拥有的版本上。", + "enabled": "显示付费版本的下载价格", + "enabledHelp": "从模型页面读取下载价格，让付费与抢先体验版本显示所需费用，以及抢先体验的结束时间", + "refreshHours": "价格刷新间隔（小时）", + "refreshHoursHelp": "已保存的价格在重新读取前保持有效的时长" + }, "licenseIcons": { "useNewStyle": "使用新版许可协议图标", "useNewStyleHelp": "以彩色指示器显示许可权限（新样式），或仅显示限制图标（经典样式）。与当前 CivitAI 设计保持一致。" @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "此版本当前需要 CivitAI 抢先体验权限", "paid": "付费", "paidTooltip": "此版本需要付费后才能下载", + "priceSaleTooltip": "促销中，原价 {list} Buzz", + "priceBlueBuzzTooltip": "也可以使用 Blue Buzz 支付", + "freeNow": "现已免费", + "freeNowTooltip": "此版本已不再需要付费", "ignored": "已忽略", "ignoredTooltip": "此版本已关闭更新通知", "onSiteOnly": "仅站内生成", - "onSiteOnlyTooltip": "此版本仅在 CivitAI 站内可用，无法下载" + "onSiteOnlyTooltip": "此版本仅在 CivitAI 站内可用，无法下载", + "earlyAccessTooltipUntil": "抢先体验至 {date}，之后可免费下载" }, "actions": { "download": "下载", diff --git a/locales/zh-TW.json b/locales/zh-TW.json index d37d2c58..9695e40b 100644 --- a/locales/zh-TW.json +++ b/locales/zh-TW.json @@ -201,7 +201,11 @@ "loading": "正在檢查 {type} 更新...", "success": "找到 {count} 個 {type} 更新", "none": "所有 {type} 都是最新版本", - "error": "檢查 {type} 更新失敗：{message}" + "error": "檢查 {type} 更新失敗：{message}", + "gateEvents": { + "becameFree": "{count} 個您未擁有的版本已免費", + "newGate": "{count} 個版本已變為付費" + } }, "cleanupExampleImages": { "label": "清理範例圖片資料夾", @@ -387,6 +391,7 @@ "downloadPathTemplates": "下載路徑範本", "priorityTags": "優先標籤", "versionScope": "版本範圍", + "priceTracking": "Buzz 下載價格", "exampleImages": "範例圖片", "autoOrganize": "自動整理", "metadata": "中繼資料", @@ -759,6 +764,13 @@ "label": "隱藏付費更新", "help": "啟用後，只有付費更新的模型將不會顯示「有可用更新」徽章" }, + "priceTracking": { + "help": "CivitAI 的公開 API 不提供下載價格，因此價格從模型頁面讀取 —— 每個付費模型多一次請求，並依下方間隔重新整理。價格只顯示在您尚未擁有的版本上。", + "enabled": "顯示付費版本的下載價格", + "enabledHelp": "從模型頁面讀取下載價格，讓付費與搶先體驗版本顯示所需費用，以及搶先體驗的結束時間", + "refreshHours": "價格重新整理間隔（小時）", + "refreshHoursHelp": "已儲存的價格在重新讀取前保持有效的時長" + }, "licenseIcons": { "useNewStyle": "使用新版許可協議圖標", "useNewStyleHelp": "以彩色指示器顯示許可權限（新樣式），或僅顯示限制圖標（經典樣式）。與當前 CivitAI 設計保持一致。" @@ -2016,10 +2028,15 @@ "earlyAccessTooltip": "此版本目前需要 CivitAI 搶先體驗權限", "paid": "付費", "paidTooltip": "此版本需要付費才能下載", + "priceSaleTooltip": "特價中，原價 {list} Buzz", + "priceBlueBuzzTooltip": "也可以使用 Blue Buzz 支付", + "freeNow": "現已免費", + "freeNowTooltip": "此版本已不再需要付費", "ignored": "已忽略", "ignoredTooltip": "此版本已關閉更新通知", "onSiteOnly": "僅站內生成", - "onSiteOnlyTooltip": "此版本僅在 CivitAI 站內可用，無法下載" + "onSiteOnlyTooltip": "此版本僅在 CivitAI 站內可用，無法下載", + "earlyAccessTooltipUntil": "搶先體驗至 {date}，之後可免費下載" }, "actions": { "download": "下載", diff --git a/py/routes/handlers/misc_handlers.py b/py/routes/handlers/misc_handlers.py index 0acf06ad..459a6221 100644 --- a/py/routes/handlers/misc_handlers.py +++ b/py/routes/handlers/misc_handlers.py @@ -48,6 +48,7 @@ from ...services.cache_health_monitor import CacheHealthMonitor, CacheHealthStat from ...services.use_cases.sidecar_migration_use_case import SidecarMigrationUseCase from ...services.websocket_progress_callback import WebSocketBroadcastCallback from ...utils.models import BaseModelMetadata +from ...utils.civitai_utils import build_civitai_model_page_url from ...utils.constants import ( CIVITAI_USER_MODEL_TYPES, DEFAULT_NODE_COLOR, diff --git a/py/routes/handlers/model_handlers.py b/py/routes/handlers/model_handlers.py index d2fe867b..f7c74c58 100644 --- a/py/routes/handlers/model_handlers.py +++ b/py/routes/handlers/model_handlers.py @@ -50,6 +50,7 @@ from ...services.errors import RateLimitError, ResourceNotFoundError from ...utils.civitai_utils import resolve_license_payload from ...utils.file_utils import calculate_sha256 from ...utils.metadata_manager import MetadataManager +from ...utils.paid_access import is_early_access_deadline_active from ...utils.sidecar_paths import get_metadata_path from ...utils.url_utils import relative_root_prefix @@ -2941,6 +2942,20 @@ class ModelUpdateHandler: same_base_scope = self._uses_same_base_update_scope() + # Gate/price transitions are reported for every refreshed model, not only + # for the ones that qualify as updates: "this version became free" matters + # for a version the user already has, which never shows up as an update. + events = [] + for record in records.values(): + for event in getattr(record, "events", None) or []: + events.append( + { + "modelId": record.model_id, + "modelType": record.model_type, + **event, + } + ) + serialized_records = [] for record in records.values(): has_update_fn = getattr(record, "has_update", None) @@ -2962,6 +2977,7 @@ class ModelUpdateHandler: { "success": True, "records": serialized_records, + "events": events, } ) @@ -3391,6 +3407,7 @@ class ModelUpdateHandler: hide_early_access=hide_early_access, hide_paid=hide_paid, ), + "events": list(getattr(record, "events", None) or []), "versions": [ self._serialize_version(version, context.get(version.version_id)) for version in record.versions @@ -3414,16 +3431,11 @@ class ModelUpdateHandler: if getattr(version, "is_paid", False) and not version.early_access_ends_at: is_early_access = False elif version.early_access_ends_at: - try: - from datetime import datetime, timezone - - ea_date = datetime.fromisoformat( - version.early_access_ends_at.replace("Z", "+00:00") - ) - is_early_access = ea_date > datetime.now(timezone.utc) - except (ValueError, AttributeError): - # If date parsing fails, treat as active EA (conservative) - is_early_access = True + # Shared with the update service and the download gate so the badge, + # the update filter and the download warning cannot disagree. + is_early_access = is_early_access_deadline_active( + version.early_access_ends_at + ) elif getattr(version, "is_early_access", False): # Fallback to basic EA flag from bulk API is_early_access = True @@ -3450,6 +3462,15 @@ class ModelUpdateHandler: "usageControl": version.usage_control, "isPaid": bool(getattr(version, "is_paid", False)), "paidAccess": paid_access_payload, + # Set when a version that used to be gated became free, so the UI can + # keep showing "Free" long after the transition. + "gateLapsedAt": getattr(version, "gate_lapsed_at", None), + "priceBuzz": getattr(version, "price_buzz", None), + "listPriceBuzz": getattr(version, "list_price_buzz", None), + "generationPriceBuzz": getattr(version, "generation_price_buzz", None), + "acceptsBlueBuzz": bool(getattr(version, "accepts_blue_buzz", False)), + "priceSaleEndsAt": getattr(version, "price_sale_ends_at", None), + "priceCheckedAt": getattr(version, "price_checked_at", None), "filePath": context.get("file_path"), "fileName": context.get("file_name"), # Weight-file variant count (None when unknown); lets the UI hide diff --git a/py/routes/misc_route_registrar.py b/py/routes/misc_route_registrar.py index ef6c9e9d..73e5311d 100644 --- a/py/routes/misc_route_registrar.py +++ b/py/routes/misc_route_registrar.py @@ -22,6 +22,8 @@ class RouteDefinition: MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = ( RouteDefinition("GET", "/api/lm/settings", "get_settings"), RouteDefinition("POST", "/api/lm/settings", "update_settings"), + # App-wide and registered once: the alerts panel spans every model type, and + # the update DB is shared, so there is nothing per-type about it. RouteDefinition("GET", "/api/lm/llm/models", "get_llm_models"), RouteDefinition("GET", "/api/lm/llm/provider-models", "get_provider_models"), RouteDefinition("GET", "/api/lm/doctor/diagnostics", "get_doctor_diagnostics"), diff --git a/py/services/civitai_client.py b/py/services/civitai_client.py index b668bfd9..ed8a3c30 100644 --- a/py/services/civitai_client.py +++ b/py/services/civitai_client.py @@ -20,7 +20,12 @@ from .model_metadata_provider import ( ) from .downloader import get_downloader from .errors import RateLimitError, ResourceNotFoundError -from ..utils.civitai_utils import resolve_license_payload +from ..utils.civitai_utils import ( + build_civitai_model_page_url, + civitai_page_host_candidates, + resolve_license_payload, +) +from ..utils.civitai_page_prices import parse_model_page_prices from ..utils.constants import MODEL_WEIGHT_FILE_TYPES, is_empty_placeholder_hash logger = logging.getLogger(__name__) @@ -31,6 +36,24 @@ logger = logging.getLogger(__name__) _CREATOR_COUNT_CACHE_TTL_SECONDS = 600 _creator_model_count_cache: Dict[str, Tuple[float, Optional[int]]] = {} +# How long a page host stays on the skip list after refusing a request outright +# (Cloudflare's challenge, surfaced as 403 "Access forbidden"). Long enough to +# cover a whole update refresh, short enough to recover within a session. +_PAGE_HOST_BLOCK_TTL = 15 * 60 + + +def _is_host_level_refusal(message: str) -> bool: + """Whether a failed request means "this host refuses us" rather than "this + model is unavailable". + + ``downloader.make_request`` collapses statuses into prose, and 403 ("Access + forbidden") is the one a Cloudflare challenge produces. A 404 ("Resource not + found") is model-specific — mature pages are hidden from anonymous visitors — + so it must not put the whole host on the skip list. + """ + + return "forbidden" in message.lower() + class CivitaiClient: _instance = None @@ -66,6 +89,10 @@ class CivitaiClient: str, Tuple[Optional[Dict[str, Any]], Optional[str]] ] = OrderedDict() self._MAX_CACHE_ENTRIES = 500 + # Model-page host bookkeeping: which host last worked, and which ones are + # currently refusing us (see get_model_prices). + self._page_host_preference: Optional[str] = None + self._page_host_blocked: Dict[str, float] = {} def _build_image_info_url(self, image_id: str) -> str: return f"{self.base_url}/images?imageId={image_id}&nsfw=X&withMeta=true" @@ -383,6 +410,138 @@ class CivitaiClient: logger.error(f"Error fetching model versions in bulk: {exc}") return None + async def get_model_prices( + self, model_id: int + ) -> Optional[Dict[int, Dict[str, Any]]]: + """Fetch per-version buzz prices for one model from its public page. + + CivitAI's public REST API deliberately omits prices, but the model page + embeds the site's own ``model.getById`` result (including + ``paidAccess.terms``) in its server-rendered payload. One request covers + every version of the model. Returns ``{version_id: price fields}``, an + empty dict when the page loads but lists no gated version, or None when + the page could not be read or understood — callers keep any stored price. + + Several hosts are tried in order, because the hosts are not equivalent: + + * ``civitai.red`` serves mature model pages that ``civitai.com`` hides from + anonymous visitors, but it is behind a Cloudflare challenge that refuses + non-browser clients outright (403 for any User-Agent). + * ``civitai.com`` / ``civitai.green`` answer normally for anonymously + visible models, and 404 for the mature ones. + + So the user's ``civitai_host`` preference is a starting point, not the only + option. Mature models whose page cannot be read from any host stay + priceless — see the known limitation in + ``docs/plans/paid-model-price-tracking.md``. + + This is a public anonymous page fetch: no API key and no internal + endpoint, so a failure here must never fail the update check itself. + """ + + try: + normalized_id = int(model_id) + except (TypeError, ValueError): + return None + + candidates = self._page_host_candidates() + failures: List[str] = [] + + for host in candidates: + url = build_civitai_model_page_url(normalized_id, host=host) + if not url: + continue + + try: + success, result = await self._make_request( + "GET", + url, + use_auth=False, + custom_headers={"Accept": "text/html"}, + ) + except RateLimitError: + # The shared rate-limit gate already recorded it; skip this model. + raise + except Exception as exc: # pragma: no cover - defensive + failures.append(f"{host}: {exc}") + continue + + if not success or not isinstance(result, str): + message = result if isinstance(result, str) else type(result).__name__ + failures.append(f"{host}: {message}") + if isinstance(result, str) and _is_host_level_refusal(result): + # A refusal like Cloudflare's "Access forbidden" applies to the + # host, not to this model, so stop paying for it for a while. + self._block_page_host(host) + continue + + prices = parse_model_page_prices(result) + if prices is None: + failures.append(f"{host}: no usable price payload") + continue + + self._remember_page_host(host) + return prices + + logger.warning( + "No price source for model %s; tried %s. Mature models are only served " + "by civitai.red, which challenges non-browser clients.", + model_id, + "; ".join(failures) or "no candidate hosts", + ) + return None + + def _page_host_candidates(self) -> List[str]: + """Ordered hosts to try: the last one that worked, then the preference.""" + + preferred = self._page_host() + ordered = list(civitai_page_host_candidates(preferred)) + if self._page_host_preference and self._page_host_preference in ordered: + ordered.remove(self._page_host_preference) + ordered.insert(0, self._page_host_preference) + + now = time.time() + usable = [ + host + for host in ordered + if now - self._page_host_blocked.get(host, 0.0) >= _PAGE_HOST_BLOCK_TTL + ] + # Never return an empty list: a blocked host is still better than no attempt + # once the preference and the memo disagree. + return usable or ordered + + def _remember_page_host(self, host: str) -> None: + if self._page_host_preference != host: + logger.info("CivitAI model pages are being read from %s", host) + self._page_host_preference = host + self._page_host_blocked.pop(host, None) + + def _block_page_host(self, host: str) -> None: + if host in self._page_host_blocked: + return + # Log once per host per TTL: the failure is host-wide, so repeating it for + # every mature model in the library would be pure noise. + logger.warning( + "CivitAI model pages on %s refused the request (likely a Cloudflare " + "challenge); skipping that host for %d minutes", + host, + _PAGE_HOST_BLOCK_TTL // 60, + ) + self._page_host_blocked[host] = time.time() + if self._page_host_preference == host: + self._page_host_preference = None + + def _page_host(self) -> Optional[str]: + """Resolve the page host from the ``civitai_host`` setting.""" + + try: + from .settings_manager import get_settings_manager + + settings = get_settings_manager() + return settings.get("civitai_host") if settings else None + except Exception: + return None + async def get_model_version( self, model_id: int | None = None, version_id: int | None = None ) -> Optional[Dict[str, Any]]: diff --git a/py/services/download_manager.py b/py/services/download_manager.py index 570f19c9..29dcca9b 100644 --- a/py/services/download_manager.py +++ b/py/services/download_manager.py @@ -33,6 +33,13 @@ from ..utils.constants import ( VALID_OTHER_CIVITAI_TYPES, ) from ..utils.civitai_utils import normalize_civitai_download_url, rewrite_preview_url +from ..utils.paid_access import ( + is_early_access_deadline_active, + is_gate_active, + is_permanent_paid, + normalize_paid_access, + parse_civitai_timestamp, +) from ..utils.file_utils import calculate_sha256, calculate_autov3 from ..utils.preview_selection import resolve_mature_threshold, select_preview_media from ..utils.utils import calculate_filename_for_model, sanitize_folder_name @@ -1985,40 +1992,32 @@ class DownloadManager: os.makedirs(save_dir, exist_ok=True) # Check if this is a paid or early access model - paid_access = version_info.get("paidAccess") - if isinstance(paid_access, str): - # Some providers (e.g. CivArchive fallback) carry the DTO as JSON text - try: - parsed = json.loads(paid_access) - paid_access = parsed if isinstance(parsed, dict) else None - except (TypeError, ValueError): - paid_access = None - if not isinstance(paid_access, dict): - paid_access = None - # An empty DTO ({"permanent": false, "endsAt": null}) is not a gate - if paid_access and not paid_access.get("permanent") and not paid_access.get("endsAt"): - paid_access = None - if version_info.get("earlyAccessEndsAt") or paid_access: - permanent_paid = bool(paid_access.get("permanent")) if paid_access else False + # CivitAI reports a non-null paidAccess only for an ACTIVE gate, so + # {"permanent": false, "endsAt": null} (a timed gate whose end is not + # recorded yet) still counts as gated here. + paid_access = normalize_paid_access(version_info.get("paidAccess")) + legacy_ea_ends_at = version_info.get("earlyAccessEndsAt") + gate_active = is_gate_active(paid_access) or is_early_access_deadline_active( + legacy_ea_ends_at + ) + if gate_active: + permanent_paid = is_permanent_paid(paid_access) if permanent_paid: early_access_msg = ( "This model requires payment. Please ensure you have " "purchased access and are logged in to Civitai." ) else: - early_access_date = version_info.get("earlyAccessEndsAt") + early_access_date = legacy_ea_ends_at if not early_access_date and paid_access: early_access_date = paid_access.get("endsAt") if not early_access_date: early_access_date = "" # Convert to a readable date if possible try: - from datetime import datetime - - date_obj = datetime.fromisoformat( - early_access_date.replace("Z", "+00:00") - ) - formatted_date = date_obj.strftime("%Y-%m-%d") + formatted_date = parse_civitai_timestamp( + early_access_date + ).strftime("%Y-%m-%d") early_access_msg = ( f"This model requires payment (until {formatted_date}). " ) diff --git a/py/services/model_metadata_provider.py b/py/services/model_metadata_provider.py index 9676c6a3..ef4d736d 100644 --- a/py/services/model_metadata_provider.py +++ b/py/services/model_metadata_provider.py @@ -183,6 +183,17 @@ class ModelMetadataProvider(ABC): """ return None + async def get_model_prices( + self, model_id: int + ) -> Optional[Dict[int, Dict[str, Any]]]: + """Fetch per-version buzz prices for a model, when the provider has them. + + CivitAI publishes prices only inside its public model page payload; + providers that cannot read it (CivArchive, SQLite, OpenModelDB) keep the + default of None, which callers treat as "no price information". + """ + return None + class CivitaiModelMetadataProvider(ModelMetadataProvider): """Provider that uses Civitai API for metadata""" @@ -222,6 +233,11 @@ class CivitaiModelMetadataProvider(ModelMetadataProvider): ) -> Optional[Dict[str, Any]]: return await self.client.get_version_file_mini(version_id, file_id) + async def get_model_prices( + self, model_id: int + ) -> Optional[Dict[int, Dict[str, Any]]]: + return await self.client.get_model_prices(model_id) + class CivArchiveModelMetadataProvider(ModelMetadataProvider): """Provider that uses CivArchive API for metadata""" @@ -783,6 +799,34 @@ class FallbackMetadataProvider(ModelMetadataProvider): def _iter_providers(self): return zip(self.providers, self._provider_labels) + async def get_model_prices( + self, model_id: int + ) -> Optional[Dict[int, Dict[str, Any]]]: + rate_limited = False + for provider, label in self._iter_providers(): + if rate_limited and label not in _LOCAL_PROVIDER_LABELS: + continue + try: + result = await self._call_with_rate_limit( + label, + provider.get_model_prices, + model_id, + ) + if result: + return result + except RateLimitError as exc: + rate_limited = True + logger.warning( + "Provider %s is rate-limited (retry_after=%.0fs); not failing over to other network providers", + label, + exc.retry_after or 0, + ) + continue + except Exception as e: + logger.debug("Provider %s failed for get_model_prices: %s", label, e) + continue + return None + def excluding(self, labels: "frozenset[str] | set[str]") -> "FallbackMetadataProvider": """Return a copy of this chain without the providers named in *labels*. @@ -907,6 +951,15 @@ class RateLimitRetryingProvider(ModelMetadataProvider): file_id, ) + async def get_model_prices( + self, model_id: int + ) -> Optional[Dict[int, Dict[str, Any]]]: + return await self._rate_limit_helper.run( + self._label, + self._provider.get_model_prices, + model_id, + ) + class ModelMetadataProviderManager: """Manager for selecting and using model metadata providers""" diff --git a/py/services/model_update_service.py b/py/services/model_update_service.py index 00147828..d90ed9ab 100644 --- a/py/services/model_update_service.py +++ b/py/services/model_update_service.py @@ -11,7 +11,7 @@ import logging import os import sqlite3 import time -from dataclasses import dataclass, replace +from dataclasses import dataclass, field, replace from datetime import datetime, timezone from typing import Any, Dict, Iterable, Iterator, List, Mapping, Optional, Sequence @@ -20,10 +20,25 @@ from .settings_manager import get_settings_manager from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration from ..utils.constants import MODEL_WEIGHT_FILE_TYPES from ..utils.civitai_utils import rewrite_preview_url +from ..utils.paid_access import ( + is_early_access_deadline_active, + normalize_paid_access as _normalize_paid_access_payload, +) from ..utils.preview_selection import resolve_mature_threshold, select_preview_media logger = logging.getLogger(__name__) +# Version fields a price payload may set (see py/utils/civitai_page_prices.py). +_PRICE_FIELD_NAMES = frozenset( + { + "price_buzz", + "list_price_buzz", + "generation_price_buzz", + "accepts_blue_buzz", + "price_sale_ends_at", + } +) + def _normalize_int(value) -> Optional[int]: """Safely convert a value to an integer.""" @@ -36,6 +51,16 @@ def _normalize_int(value) -> Optional[int]: return None +def _format_utc_timestamp(value: float) -> str: + """Format a POSIX timestamp the way CivitAI serializes its timestamps.""" + + return ( + datetime.fromtimestamp(value, tz=timezone.utc) + .isoformat() + .replace("+00:00", "Z") + ) + + def _normalize_string(value) -> Optional[str]: """Return a stripped string or None if the value is empty.""" @@ -82,6 +107,24 @@ class ModelVersionRecord: # e.g. records persisted before this field existed or locally-synthesized # entries). Mirrors the frontend isModelWeightFile() filter. file_count: Optional[int] = None + # When a version that used to carry a paid gate stopped carrying one. Kept so + # the UI can mark a version "free" long after the transition happened; cleared + # again if the version is gated once more. + gate_lapsed_at: Optional[str] = None + # Download price in Buzz for the gated version, and friends. All None until a + # price refresh succeeds (see the price tracking feature); `price_buzz` is the + # effective price a buyer pays now (sale-adjusted) and `list_price_buzz` the + # undiscounted stored price. + price_buzz: Optional[int] = None + list_price_buzz: Optional[int] = None + generation_price_buzz: Optional[int] = None + accepts_blue_buzz: bool = False + price_sale_ends_at: Optional[str] = None + price_checked_at: Optional[float] = None + # When a price fetch was last *attempted* (success or failure). Distinguishes + # "never tried" from "tried and no price is readable", which is what lets the + # UI say "price unavailable" for mature models instead of showing nothing. + price_check_attempted_at: Optional[float] = None @dataclass @@ -93,6 +136,12 @@ class ModelUpdateRecord: versions: List[ModelVersionRecord] last_checked_at: Optional[float] should_ignore_model: bool + # Gate-state and price transitions observed during the refresh that produced + # this record. Derived, never persisted: a record read back from SQLite has an + # empty list. Each entry is {"versionId", "kind", "versionName", "isInLibrary"} + # with kind in {"new_gate", "became_free"}. Only versions the user does not + # have produce events. + events: List[Dict[str, Any]] = field(default_factory=list) @property def largest_version_id(self) -> Optional[int]: @@ -172,16 +221,10 @@ class ModelUpdateRecord: if version.is_paid and not version.early_access_ends_at: return False - # Phase 2: Precise check with exact end time + # Phase 2: Precise check with exact end time (None when the gate is timed + # but its window end has not been recorded yet -> treated as active below) if version.early_access_ends_at: - try: - ea_date = datetime.fromisoformat( - version.early_access_ends_at.replace("Z", "+00:00") - ) - return ea_date > datetime.now(timezone.utc) - except (ValueError, AttributeError): - # If date parsing fails, treat as active EA (conservative) - return True + return is_early_access_deadline_active(version.early_access_ends_at) # Phase 1: Basic EA flag from bulk API return version.is_early_access @@ -324,6 +367,14 @@ class ModelUpdateService: paid_access TEXT, is_paid INTEGER NOT NULL DEFAULT 0, file_count INTEGER, + gate_lapsed_at TEXT, + price_buzz INTEGER, + list_price_buzz INTEGER, + generation_price_buzz INTEGER, + accepts_blue_buzz INTEGER NOT NULL DEFAULT 0, + price_sale_ends_at TEXT, + price_checked_at REAL, + price_check_attempted_at REAL, PRIMARY KEY (model_id, version_id), FOREIGN KEY(model_id) REFERENCES model_update_status(model_id) ON DELETE CASCADE ); @@ -575,12 +626,59 @@ class ModelUpdateService: "ALTER TABLE model_update_versions " "ADD COLUMN file_count INTEGER" ), + "gate_lapsed_at": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN gate_lapsed_at TEXT" + ), + "price_buzz": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN price_buzz INTEGER" + ), + "list_price_buzz": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN list_price_buzz INTEGER" + ), + "generation_price_buzz": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN generation_price_buzz INTEGER" + ), + "accepts_blue_buzz": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN accepts_blue_buzz INTEGER NOT NULL DEFAULT 0" + ), + "price_sale_ends_at": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN price_sale_ends_at TEXT" + ), + "price_checked_at": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN price_checked_at REAL" + ), + "price_check_attempted_at": ( + "ALTER TABLE model_update_versions " + "ADD COLUMN price_check_attempted_at REAL" + ), } for column, statement in migrations.items(): if column not in version_columns: conn.execute(statement) + # Columns this feature carried while it was still in development. They are + # gone from the schema, so only a database created by an unreleased build + # can still have them; for everyone else this is a no-op. `DROP COLUMN` + # needs SQLite 3.35+, hence the guard. + for obsolete in ("price_alert_state", "price_alert_since"): + if obsolete not in version_columns: + continue + try: + conn.execute( + f"ALTER TABLE model_update_versions DROP COLUMN {obsolete}" + ) + logger.info("Dropped obsolete column %s", obsolete) + except sqlite3.OperationalError as exc: # pragma: no cover - old SQLite + logger.debug("Could not drop obsolete column %s: %s", obsolete, exc) + # Refresh column metadata after applying additive migrations. version_columns = self._get_table_columns(conn, "model_update_versions") @@ -679,6 +777,14 @@ class ModelUpdateService: paid_access TEXT, is_paid INTEGER NOT NULL DEFAULT 0, file_count INTEGER, + gate_lapsed_at TEXT, + price_buzz INTEGER, + list_price_buzz INTEGER, + generation_price_buzz INTEGER, + accepts_blue_buzz INTEGER NOT NULL DEFAULT 0, + price_sale_ends_at TEXT, + price_checked_at REAL, + price_check_attempted_at REAL, PRIMARY KEY (model_id, version_id), FOREIGN KEY(model_id) REFERENCES model_update_status(model_id) ON DELETE CASCADE ) @@ -701,6 +807,14 @@ class ModelUpdateService: "paid_access", "is_paid", "file_count", + "gate_lapsed_at", + "price_buzz", + "list_price_buzz", + "generation_price_buzz", + "accepts_blue_buzz", + "price_sale_ends_at", + "price_checked_at", + "price_check_attempted_at", ] defaults = { "sort_index": "0", @@ -716,6 +830,14 @@ class ModelUpdateService: "paid_access": "NULL", "is_paid": "0", "file_count": "NULL", + "gate_lapsed_at": "NULL", + "price_buzz": "NULL", + "list_price_buzz": "NULL", + "generation_price_buzz": "NULL", + "accepts_blue_buzz": "0", + "price_sale_ends_at": "NULL", + "price_checked_at": "NULL", + "price_check_attempted_at": "NULL", } select_parts = [] @@ -1170,6 +1292,37 @@ class ModelUpdateService: elif fallback_attempted and fallback_error_message is None: fallback_error_message = "no response" + # Optional price capture. Runs after the version list is known so only the + # models that actually carry a gate pay for a second (page) request, and + # stays outside the lock along with the other network I/O. + # + # Deliberately independent of the metadata TTL: the cached record already + # carries the gate, so enabling price tracking (or letting the *price* TTL + # lapse) must not wait for the version list to go stale. Otherwise a user + # who turns the feature on prices only the handful of models that happened + # to need a metadata refresh that round. + priced_versions: Optional[List[ModelVersionRecord]] = None + price_candidates: Optional[Sequence[ModelVersionRecord]] = None + if refresh_succeeded and isinstance(fetched_versions, list) and fetched_versions: + price_candidates = fetched_versions + elif existing is not None and existing.versions: + price_candidates = existing.versions + + if ( + price_candidates + and metadata_provider is not None + and self._should_fetch_prices( + price_candidates, existing, force=force_refresh + ) + ): + price_candidates = await self._apply_model_prices( + metadata_provider, model_id, price_candidates + ) + if refresh_succeeded and isinstance(fetched_versions, list): + fetched_versions = list(price_candidates) + else: + priced_versions = list(price_candidates) + if fallback_attempted: if refresh_succeeded and isinstance(fetched_versions, list): logger.info( @@ -1234,6 +1387,11 @@ class ModelUpdateService: local_base_models=local_base_models, ) else: + if priced_versions is not None and existing is not None: + # Metadata came from the cache, but the prices did not: keep + # them without touching last_checked_at, so the metadata TTL is + # not silently extended by a price-only pass. + existing = replace(existing, versions=priced_versions) record = self._merge_with_local_versions( existing, normalized_local, @@ -1633,10 +1791,55 @@ class ModelUpdateService: existing_map = {version.version_id: version for version in existing.versions} if existing else {} versions: List[ModelVersionRecord] = [] + events: List[Dict[str, Any]] = [] seen_ids: set[int] = set() + lapse_timestamp = _format_utc_timestamp(timestamp) for index, remote_version in enumerate(remote_versions): version_id = remote_version.version_id seen_ids.add(version_id) + existing_version = existing_map.get(version_id) + is_gated = self._has_structural_gate(remote_version) + was_gated = ( + self._has_structural_gate(existing_version) + if existing_version is not None + else None + ) + should_ignore = ignore_map.get(version_id, remote_version.should_ignore) + + # `gate_lapsed_at` survives refreshes so a version that became free keeps + # its "free since" marker, and is cleared if the gate comes back. + if is_gated: + gate_lapsed_at = None + elif was_gated: + gate_lapsed_at = lapse_timestamp + elif existing_version is not None: + gate_lapsed_at = existing_version.gate_lapsed_at + else: + gate_lapsed_at = None + + price_fields = self._price_fields_for( + remote_version, existing_version, is_gated=is_gated + ) + + # Only the edge fires, and only for a version the user does not have: + # a version already on disk cannot become cheaper *for them*, and a + # version that stays gated must not re-announce itself every refresh. + in_library = version_id in effective_local_set + if ( + existing_version is not None + and not should_ignore + and not in_library + and was_gated != is_gated + ): + events.append( + { + "versionId": version_id, + "kind": "new_gate" if is_gated else "became_free", + "versionName": remote_version.name, + "isInLibrary": False, + } + ) + versions.append( ModelVersionRecord( version_id=version_id, @@ -1646,7 +1849,7 @@ class ModelUpdateService: size_bytes=remote_version.size_bytes, preview_url=remote_version.preview_url or preview_map.get(version_id), is_in_library=version_id in effective_local_set, - should_ignore=ignore_map.get(version_id, remote_version.should_ignore), + should_ignore=should_ignore, sort_index=sort_map.get(version_id, index), early_access_ends_at=remote_version.early_access_ends_at, is_early_access=remote_version.is_early_access, @@ -1658,6 +1861,8 @@ class ModelUpdateService: if remote_version.file_count is not None else file_count_map.get(version_id) ), + gate_lapsed_at=gate_lapsed_at, + **price_fields, ) ) @@ -1696,8 +1901,199 @@ class ModelUpdateService: versions=self._sorted_versions(versions), last_checked_at=timestamp, should_ignore_model=existing.should_ignore_model if existing else False, + events=events, ) + @staticmethod + def _has_structural_gate(version: Optional[ModelVersionRecord]) -> bool: + """True when a version carries a paid gate, ignoring the clock. + + Transition detection uses this rather than a time-based check so a gate + that lapsed since the previous refresh still reads as "was gated" — the + public API reports no ``paidAccess`` at all once a gate expires, so the + time-based helper alone would silently see "free before, free now". + """ + + if version is None: + return False + return bool(version.paid_access) or version.is_paid or version.is_early_access + + @staticmethod + def _price_fields_for( + remote_version: ModelVersionRecord, + existing_version: Optional[ModelVersionRecord], + *, + is_gated: bool, + ) -> Dict[str, Any]: + """Resolve the price columns for a refreshed version. + + A refresh only carries prices when a price fetch actually ran (recognizable + by ``price_checked_at``), so otherwise the previously stored price is kept — + but only while the version is still gated: once it is free the stored + numbers are stale and must not keep firing alerts. + """ + + if not is_gated: + return { + "price_buzz": None, + "list_price_buzz": None, + "generation_price_buzz": None, + "accepts_blue_buzz": False, + "price_sale_ends_at": None, + "price_checked_at": None, + "price_check_attempted_at": None, + } + + if remote_version.price_checked_at is not None: + source = remote_version + else: + source = existing_version or remote_version + + return { + "price_buzz": source.price_buzz, + "list_price_buzz": source.list_price_buzz, + "generation_price_buzz": source.generation_price_buzz, + "accepts_blue_buzz": source.accepts_blue_buzz, + "price_sale_ends_at": source.price_sale_ends_at, + "price_checked_at": source.price_checked_at, + "price_check_attempted_at": source.price_check_attempted_at, + } + + def _price_tracking_enabled(self) -> bool: + """Whether the optional (page-fetching) price capture is switched on.""" + + try: + return bool(self._settings.get("price_tracking_enabled", False)) + except Exception: + return False + + def _price_check_ttl_seconds(self) -> float: + try: + hours = float(self._settings.get("price_check_ttl_hours", 24)) + except (TypeError, ValueError): + hours = 24.0 + if hours <= 0: + hours = 24.0 + return hours * 3600.0 + + def _should_fetch_prices( + self, + remote_versions: Sequence[ModelVersionRecord], + existing: Optional[ModelUpdateRecord], + *, + force: bool = False, + ) -> bool: + """Whether this model needs a price fetch this round. + + Only gated versions are considered, so an unmodified free library pays + nothing. A stored price is refreshed once its TTL lapses, and immediately + when the gate itself changed (a new end date or sale window is a reason to + believe the price moved). + + A *failed* attempt counts as an attempt: without that, a mature model whose + page no host will serve would be retried on every single update check. + ``force`` (the user asked explicitly) overrides the TTL. + """ + + if not self._price_tracking_enabled(): + return False + if force: + return any( + self._has_structural_gate(version) for version in remote_versions + ) + + existing_map = ( + {version.version_id: version for version in existing.versions} + if existing + else {} + ) + now = time.time() + ttl = self._price_check_ttl_seconds() + + for remote_version in remote_versions: + if not self._has_structural_gate(remote_version): + continue + stored = existing_map.get(remote_version.version_id) + if stored is None: + return True + anchor = stored.price_checked_at or stored.price_check_attempted_at + if anchor is None: + return True + if (now - anchor) >= ttl: + return True + if (stored.paid_access or None) != (remote_version.paid_access or None): + return True + + return False + + async def _apply_model_prices( + self, + metadata_provider, + model_id: int, + versions: Sequence[ModelVersionRecord], + ) -> List[ModelVersionRecord]: + """Attach freshly fetched prices to the versions that have them. + + Never raises for a provider problem: price tracking is a convenience, and + an unreadable page (or a provider that has no prices at all) must leave + the update check exactly as it was. + + A failed attempt is still recorded (``price_check_attempted_at``) so the UI + can distinguish "we could not read a price" from "we never looked" — that + is the honest state for mature models, whose pages are served only by + civitai.red, which refuses non-browser clients. + """ + + getter = getattr(metadata_provider, "get_model_prices", None) + if not callable(getter): + return list(versions) + + attempted_at = time.time() + try: + prices = await getter(model_id) + except RateLimitError: + raise + except Exception as exc: # pragma: no cover - defensive + logger.debug("Price fetch failed for model %s: %s", model_id, exc) + prices = None + + if not isinstance(prices, Mapping): + prices = {} + + enriched: List[ModelVersionRecord] = [] + for version in versions: + if not self._has_structural_gate(version): + enriched.append(version) + continue + + fields = prices.get(version.version_id) + recognized = ( + { + key: value + for key, value in fields.items() + if key in _PRICE_FIELD_NAMES + } + if isinstance(fields, Mapping) + else {} + ) + if not recognized: + # Keep the stored price (there may be none) but remember the try, so + # the UI can say "unavailable" instead of showing nothing at all. + enriched.append( + replace(version, price_check_attempted_at=attempted_at) + ) + continue + + enriched.append( + replace( + version, + price_checked_at=attempted_at, + price_check_attempted_at=attempted_at, + **recognized, + ) + ) + return enriched + def _sorted_versions(self, versions: Sequence[ModelVersionRecord]) -> List[ModelVersionRecord]: ordered = sorted(versions, key=lambda version: (version.sort_index, version.version_id)) normalized: List[ModelVersionRecord] = [] @@ -1762,6 +2158,9 @@ class ModelUpdateService: # CivitAI's paidAccess DTO ({"permanent": bool, "endsAt": ISO|null}) # gates versions behind a paid tier while availability stays "Public". + # A non-null DTO from the public API is always an ACTIVE gate: lapsed + # (tombstone) gates come back as null. That includes the timed gate whose + # end is not recorded yet, {"permanent": false, "endsAt": null}. paid_access = self._normalize_paid_access(entry.get("paidAccess")) paid_access_json = json.dumps(paid_access) if paid_access else None is_paid = bool(paid_access.get("permanent")) if paid_access else False @@ -1769,7 +2168,7 @@ class ModelUpdateService: early_access_ends_at = _normalize_string(paid_access.get("endsAt")) # Only timed gates are early access; permanent paid versions are not # (consumers filter them via is_paid), so the stored flag stays accurate. - if not is_early_access and paid_access and paid_access.get("endsAt"): + if not is_early_access and paid_access and not paid_access.get("permanent"): is_early_access = True return ModelVersionRecord( @@ -1797,24 +2196,12 @@ class ModelUpdateService: Accepts a dict, None, or a JSON string (as carried by the by-hash enrichment path) and returns ``{"permanent": bool, "endsAt": str|None}`` or None when the input carries no paid-access signal. + + Delegates to :mod:`py.utils.paid_access` so the update service and the + download gate cannot disagree about what counts as a gate. """ - if value is None: - return None - if isinstance(value, str): - try: - parsed = json.loads(value) - except (TypeError, ValueError): - return None - if not isinstance(parsed, dict): - return None - value = parsed - if not isinstance(value, Mapping): - return None - permanent = bool(value.get("permanent")) - ends_at = _normalize_string(value.get("endsAt")) - if not permanent and ends_at is None: - return None - return {"permanent": permanent, "endsAt": ends_at} + + return _normalize_paid_access_payload(value) @staticmethod def _extract_file_count(files) -> Optional[int]: @@ -1947,7 +2334,10 @@ class ModelUpdateService: f""" SELECT model_id, version_id, sort_index, name, base_model, released_at, size_bytes, preview_url, is_in_library, should_ignore, early_access_ends_at, - is_early_access, usage_control, paid_access, is_paid, file_count + is_early_access, usage_control, paid_access, is_paid, file_count, + gate_lapsed_at, price_buzz, list_price_buzz, generation_price_buzz, + accepts_blue_buzz, price_sale_ends_at, price_checked_at, + price_check_attempted_at FROM model_update_versions WHERE model_id IN ({placeholders}) ORDER BY model_id ASC, sort_index ASC, version_id ASC @@ -1979,6 +2369,22 @@ class ModelUpdateService: paid_access=row["paid_access"], is_paid=bool(row["is_paid"]), file_count=_normalize_int(row["file_count"]), + gate_lapsed_at=row["gate_lapsed_at"], + price_buzz=_normalize_int(row["price_buzz"]), + list_price_buzz=_normalize_int(row["list_price_buzz"]), + generation_price_buzz=_normalize_int(row["generation_price_buzz"]), + accepts_blue_buzz=bool(row["accepts_blue_buzz"]), + price_sale_ends_at=row["price_sale_ends_at"], + price_checked_at=( + float(row["price_checked_at"]) + if row["price_checked_at"] is not None + else None + ), + price_check_attempted_at=( + float(row["price_check_attempted_at"]) + if row["price_check_attempted_at"] is not None + else None + ), ) ) @@ -2041,8 +2447,11 @@ class ModelUpdateService: INSERT INTO model_update_versions ( version_id, model_id, sort_index, name, base_model, released_at, size_bytes, preview_url, is_in_library, should_ignore, early_access_ends_at, - is_early_access, usage_control, paid_access, is_paid, file_count - ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) + is_early_access, usage_control, paid_access, is_paid, file_count, + gate_lapsed_at, price_buzz, list_price_buzz, generation_price_buzz, + accepts_blue_buzz, price_sale_ends_at, price_checked_at, + price_check_attempted_at + ) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) """, ( version.version_id, @@ -2061,6 +2470,14 @@ class ModelUpdateService: paid_access_value, 1 if version.is_paid else 0, version.file_count, + version.gate_lapsed_at, + version.price_buzz, + version.list_price_buzz, + version.generation_price_buzz, + 1 if version.accepts_blue_buzz else 0, + version.price_sale_ends_at, + version.price_checked_at, + version.price_check_attempted_at, ), ) conn.commit() diff --git a/py/services/settings_manager.py b/py/services/settings_manager.py index 24216116..e06910af 100644 --- a/py/services/settings_manager.py +++ b/py/services/settings_manager.py @@ -130,6 +130,12 @@ DEFAULT_SETTINGS: Dict[str, Any] = { "model_card_footer_action": "replace_preview", "show_version_on_card": True, "version_grouping": "same_base", + # Buzz price tracking for paid/early-access versions. Opt-in because reading a + # price costs one extra (public) model-page request per gated model. + # Plumbing switch: reading a price costs one extra request per paid model, so + # it stays opt-in. Prices decorate the version list; nothing alerts on a number. + "price_tracking_enabled": False, + "price_check_ttl_hours": 24, "auto_organize_exclusions": [], "metadata_refresh_skip_paths": [], "skip_previously_downloaded_model_versions": False, diff --git a/py/utils/civitai_page_prices.py b/py/utils/civitai_page_prices.py new file mode 100644 index 00000000..190b9cfd --- /dev/null +++ b/py/utils/civitai_page_prices.py @@ -0,0 +1,257 @@ +"""Extract buzz prices from a public CivitAI model page. + +CivitAI's public REST API deliberately omits prices: ``paidAccess`` is trimmed to +``{permanent, endsAt}`` because "pricing belongs to the purchase flow" (see the +upstream ``toPublicPaidAccessDto``). The model *page*, however, ships the site's +own ``model.getById`` result inside its server-rendered Next.js payload, and that +payload carries the full ``paidAccess.terms`` — download price, generation price, +sale and the Blue Buzz flag. + +Reading it is a plain anonymous page fetch: no API key, no internal endpoint, no +forged ``Origin``. It is still page data rather than a contract, so nothing in +this module may raise: an unrecognized shape degrades to ``None`` and the caller +keeps whatever price it already stored. + +Shape this understands (verified against civitai.com and civitai.red):: + + +""" + +from __future__ import annotations + +import json +import logging +import re +from typing import Any, Dict, Iterator, Mapping, Optional + +from .paid_access import is_gate_active, normalize_paid_access + +logger = logging.getLogger(__name__) + +__all__ = ["parse_model_page_prices", "MAX_PAGE_BYTES"] + +# Bound the payload before parsing: a model page is a few hundred KB, so anything +# far larger is not a page we want to hold in memory. +MAX_PAGE_BYTES = 8 * 1024 * 1024 + +_NEXT_DATA_RE = re.compile( + r']*id="__NEXT_DATA__"[^>]*>(.*?)', + re.DOTALL, +) + +# The procedure that carries per-version pricing on a model page. +_MODEL_QUERY_PATH = ("model", "getById") + + +def parse_model_page_prices(html: Any) -> Optional[Dict[int, Dict[str, Any]]]: + """Return ``{version_id: price fields}`` for a model page, or None. + + Only versions with an *active* gate appear: the page payload also carries + lapsed gates as tombstones (a version with ``endsAt`` in the past is freely + downloadable), and those must not be reported as priced. + + Returns None when the page carries no usable payload at all (missing script + tag, malformed JSON, unexpected structure, challenge page). + """ + + if not isinstance(html, str) or not html: + return None + if len(html) > MAX_PAGE_BYTES: + logger.debug("CivitAI model page too large to parse (%d bytes)", len(html)) + return None + + match = _NEXT_DATA_RE.search(html) + if match is None: + return None + + try: + payload = json.loads(match.group(1)) + except (TypeError, ValueError): + return None + + versions = _find_model_versions(payload) + if versions is None: + return None + + prices: Dict[int, Dict[str, Any]] = {} + for entry in versions: + if not isinstance(entry, Mapping): + continue + version_id = _coerce_int(entry.get("id")) + if version_id is None: + continue + price_fields = _price_fields(entry.get("paidAccess")) + if price_fields is not None: + prices[version_id] = price_fields + + return prices + + +def _find_model_versions(payload: Any) -> Optional[list]: + """Locate ``modelVersions`` inside the dehydrated tRPC state. + + The query order is not stable (several procedures are dehydrated per page), so + the query is selected by key rather than position. + """ + + if not isinstance(payload, Mapping): + return None + + for state in _iter_trpc_states(payload): + queries = state.get("queries") + if not isinstance(queries, list): + continue + for query in queries: + if not isinstance(query, Mapping): + continue + if not _matches_model_query(query.get("queryKey")): + continue + data = (query.get("state") or {}).get("data") + if isinstance(data, Mapping): + versions = data.get("modelVersions") + if isinstance(versions, list): + return versions + return None + + +def _iter_trpc_states(payload: Mapping[str, Any]) -> Iterator[Mapping[str, Any]]: + """Yield the dehydrated tRPC state objects found under ``props.pageProps``.""" + + props = payload.get("props") + if not isinstance(props, Mapping): + return + page_props = props.get("pageProps") + if not isinstance(page_props, Mapping): + return + + candidates = [page_props.get("trpcState")] + # Older/other Next.js builds keep the state one level deeper. + dehydrated = page_props.get("dehydratedState") + if dehydrated is not None: + candidates.append(dehydrated) + + for candidate in candidates: + if not isinstance(candidate, Mapping): + continue + for key in ("json", "superjson", "dehydratedState"): + nested = candidate.get(key) + if isinstance(nested, Mapping): + yield nested + yield candidate + + +def _matches_model_query(query_key: Any) -> bool: + if not isinstance(query_key, (list, tuple)) or not query_key: + return False + first = query_key[0] + if not isinstance(first, (list, tuple)) or len(first) < 2: + return False + return str(first[0]) == _MODEL_QUERY_PATH[0] and str(first[1]) == _MODEL_QUERY_PATH[1] + + +def _price_fields(raw_paid_access: Any) -> Optional[Dict[str, Any]]: + """Build the price fields for one version, or None when it is not gated.""" + + dto = _coerce_dto(raw_paid_access) + if dto is None: + return None + + info = normalize_paid_access(dto) + if not info or not is_gate_active(info): + return None + + terms = dto.get("terms") + if not isinstance(terms, Mapping): + terms = {} + + list_price = _download_price(terms) + effective_price = list_price + + sale = dto.get("sale") + sale_ends_at = None + if isinstance(sale, Mapping): + sale_ends_at = _coerce_str(sale.get("endsAt")) + buyer_terms = sale.get("buyerTerms") + if isinstance(buyer_terms, Mapping): + sale_price = _download_price(buyer_terms) + if sale_price is not None: + effective_price = sale_price + + return { + "price_buzz": effective_price, + "list_price_buzz": list_price, + "generation_price_buzz": _generation_price(terms), + "accepts_blue_buzz": bool(terms.get("acceptsBlueBuzz")), + "price_sale_ends_at": sale_ends_at, + } + + +def _coerce_dto(value: Any) -> Optional[Mapping[str, Any]]: + if isinstance(value, str): + try: + value = json.loads(value) + except (TypeError, ValueError): + return None + return value if isinstance(value, Mapping) else None + + +def _download_price(terms: Mapping[str, Any]) -> Optional[int]: + download = terms.get("download") + if not isinstance(download, Mapping): + return None + return _coerce_price(download.get("price")) + + +def _generation_price(terms: Mapping[str, Any]) -> Optional[int]: + generation = terms.get("generation") + if not isinstance(generation, Mapping): + return None + # `{free: true}` has no price of its own, and a paid tier may omit `price` to + # fall back to the download price — the download price is already captured + # separately, so only an explicit number is reported here. + return _coerce_price(generation.get("price")) + + +def _coerce_price(value: Any) -> Optional[int]: + if isinstance(value, bool): + return None + if isinstance(value, (int, float)): + price = int(value) + return price if price >= 0 else None + if isinstance(value, str): + try: + price = int(float(value.strip())) + except (TypeError, ValueError): + return None + return price if price >= 0 else None + return None + + +def _coerce_int(value: Any) -> Optional[int]: + if isinstance(value, bool): + return None + if isinstance(value, int): + return value + if isinstance(value, str): + try: + return int(value.strip()) + except (TypeError, ValueError): + return None + return None + + +def _coerce_str(value: Any) -> Optional[str]: + if isinstance(value, str) and value.strip(): + return value.strip() + return None diff --git a/py/utils/civitai_utils.py b/py/utils/civitai_utils.py index 9466d0dd..ceacdc77 100644 --- a/py/utils/civitai_utils.py +++ b/py/utils/civitai_utils.py @@ -41,6 +41,21 @@ def normalize_civitai_page_host(hostname: str | None) -> str: return DEFAULT_CIVITAI_PAGE_HOST +def civitai_page_host_candidates(hostname: str | None = None) -> tuple[str, ...]: + """Ordered page hosts to try, the preferred one first. + + The hosts are not interchangeable: ``civitai.red`` serves mature model pages + that ``civitai.com`` hides from anonymous visitors, but it also sits behind a + Cloudflare challenge that rejects non-browser HTTP clients outright. Trying the + others when the preferred host refuses a request is what keeps a user's + ``civitai_host`` preference from disabling page reads entirely. + """ + + preferred = normalize_civitai_page_host(hostname) + rest = sorted(_SUPPORTED_CIVITAI_PAGE_HOSTS - {preferred}) + return (preferred, *rest) + + def build_civitai_model_page_url( model_id: str | int | None, version_id: str | int | None = None, diff --git a/py/utils/paid_access.py b/py/utils/paid_access.py new file mode 100644 index 00000000..d2d9f15a --- /dev/null +++ b/py/utils/paid_access.py @@ -0,0 +1,158 @@ +"""Shared interpretation of CivitAI's ``paidAccess`` DTO. + +One rule for the whole backend: the update service (badges, update filters, price +tracking) and the download gate both ask these helpers, so a version can never be +"paid" for one consumer and "free" for another. + +Why the rules look like this +---------------------------- +CivitAI's public v1 API only returns a non-null DTO for an *active* gate: a lapsed +gate stays in the database as a tombstone but is filtered out server-side +(``toPublicPaidAccessDto`` in civitai's ``server/services/paid-access.service.ts``). +A timed gate whose end time has not been recorded yet is reported as +``{"permanent": false, "endsAt": null}`` and is still enforced — such a version +reports ``canDownload: false`` on the model page — so it must NOT be discarded. +Dropping it (the previous behaviour) hid real gates, which is the class of bug +reported in issue #1060. +""" + +from __future__ import annotations + +import json +from datetime import datetime, timezone +from typing import Any, Mapping, Optional + +__all__ = [ + "normalize_paid_access", + "is_permanent_paid", + "is_gate_active", + "is_early_access_deadline_active", + "parse_civitai_timestamp", +] + +# A DTO that carries neither key is not a gate signal at all (defensive: CivitAI +# never emits a bare ``{}``, but other metadata sources might). +_PAID_ACCESS_KEYS = ("permanent", "endsAt") + + +def parse_civitai_timestamp(value: Any) -> Optional[datetime]: + """Parse a CivitAI ISO-8601 timestamp into an aware UTC datetime. + + Returns None for anything that is not a parsable string. Naive timestamps are + assumed to be UTC, matching CivitAI's serialization. + """ + + if not isinstance(value, str): + return None + text = value.strip() + if not text: + return None + if text.endswith("Z"): + text = f"{text[:-1]}+00:00" + try: + parsed = datetime.fromisoformat(text) + except ValueError: + return None + if parsed.tzinfo is None: + parsed = parsed.replace(tzinfo=timezone.utc) + return parsed + + +def normalize_paid_access(value: Any) -> Optional[dict]: + """Normalize a ``paidAccess`` value into ``{"permanent": bool, "endsAt": str|None}``. + + Accepts a mapping, a JSON string (the by-hash enrichment path carries the DTO as + text), or None. Returns None when the value carries no gate signal. + + Note that ``{"permanent": False, "endsAt": None}`` is a *gate*: CivitAI reports + it for a timed gate whose window end is not set yet, and enforces it. + """ + + if value is None: + return None + + if isinstance(value, str): + try: + parsed = json.loads(value) + except (TypeError, ValueError): + return None + if not isinstance(parsed, dict): + return None + value = parsed + + if not isinstance(value, Mapping): + return None + + if not any(key in value for key in _PAID_ACCESS_KEYS): + return None + + ends_at = value.get("endsAt") + normalized_ends_at = ends_at.strip() if isinstance(ends_at, str) and ends_at.strip() else None + return { + "permanent": bool(value.get("permanent")), + "endsAt": normalized_ends_at, + } + + +def is_permanent_paid(paid_access: Optional[Mapping[str, Any]]) -> bool: + """True when the gate never expires (a permanent paid version).""" + + return bool(paid_access and paid_access.get("permanent")) + + +def is_gate_active( + paid_access: Optional[Mapping[str, Any]], + *, + now: Optional[datetime] = None, +) -> bool: + """True when a normalized ``paidAccess`` gate is currently in force. + + Active means permanent, or a timed gate whose end is either still in the future + or not recorded yet (CivitAI enforces the latter too). An unparsable end time is + treated as active rather than free: the download would fail anyway, so the + conservative reading matches what the user will experience. + """ + + if not paid_access: + return False + if paid_access.get("permanent"): + return True + + ends_at = paid_access.get("endsAt") + if not ends_at: + # Timed gate with no recorded end — CivitAI still gates the download. + return True + + parsed = parse_civitai_timestamp(ends_at) + if parsed is None: + return True + + reference = now or datetime.now(timezone.utc) + if reference.tzinfo is None: + reference = reference.replace(tzinfo=timezone.utc) + return parsed > reference + + +def is_early_access_deadline_active( + ends_at: Any, + *, + now: Optional[datetime] = None, +) -> bool: + """True when a legacy ``earlyAccessEndsAt`` deadline is still in the future. + + Kept separate from :func:`is_gate_active` because the legacy field is a bare + timestamp rather than a DTO. A present-but-unparsable value is treated as active, + matching the previous conservative behaviour. + """ + + if not ends_at: + return False + + parsed = parse_civitai_timestamp(ends_at) + if parsed is None: + return True + + reference = now or datetime.now(timezone.utc) + if reference.tzinfo is None: + reference = reference.replace(tzinfo=timezone.utc) + return parsed > reference diff --git a/static/css/components/lora-modal/versions.css b/static/css/components/lora-modal/versions.css index 362bc689..bbec690d 100644 --- a/static/css/components/lora-modal/versions.css +++ b/static/css/components/lora-modal/versions.css @@ -460,6 +460,20 @@ border-color: color-mix(in oklch, #845EF7 45%, transparent); } +/* Price alert badge - green tone (#37B24D) for a price under the user's threshold */ +.version-badge-price-alert { + background: color-mix(in oklch, #37B24D 25%, transparent); + color: #2B8A3E; + border-color: color-mix(in oklch, #37B24D 55%, transparent); + font-variant-numeric: tabular-nums; +} + +[data-theme="dark"] .version-badge-price-alert { + background: color-mix(in oklch, #37B24D 20%, transparent); + color: #69DB7C; + border-color: color-mix(in oklch, #37B24D 45%, transparent); +} + .version-meta-ea { color: #E67700; font-weight: 600; diff --git a/static/js/api/apiConfig.js b/static/js/api/apiConfig.js index 9b277769..8d0b3dc4 100644 --- a/static/js/api/apiConfig.js +++ b/static/js/api/apiConfig.js @@ -99,6 +99,9 @@ export function getApiEndpoints(modelType) { ignoreModelUpdate: `/api/lm/${modelType}/updates/ignore`, ignoreVersionUpdate: `/api/lm/${modelType}/updates/ignore-version`, + // Price alerts are app-wide (all model types in one query), so they are + // not part of this per-type endpoint map; see UpdateService.js. + // Preview management replacePreview: `/api/lm/${modelType}/replace-preview`, setPreviewFromUrl: `/api/lm/${modelType}/set-preview-from-url`, diff --git a/static/js/components/shared/ModelVersionsTab.js b/static/js/components/shared/ModelVersionsTab.js index f14602b7..2d7f3664 100644 --- a/static/js/components/shared/ModelVersionsTab.js +++ b/static/js/components/shared/ModelVersionsTab.js @@ -187,6 +187,53 @@ function isPaidPermanent(version) { return version && version.isPaid === true; } +function isGated(version) { + return isEarlyAccessActive(version) || isPaidPermanent(version); +} + +/** + * Format a Buzz price for display, or '' when the price is unknown (price + * tracking is opt-in, so most versions have no price at all). + */ +function formatBuzzPrice(value) { + if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) { + return ''; + } + return `${value.toLocaleString()} Buzz`; +} + +/** + * Tooltip for a known price: notes a sale against the list price and the Blue + * Buzz option, since both change what the number means to a buyer. + */ +function buildPriceTooltip(version, baseTooltip) { + const effective = formatBuzzPrice(version?.priceBuzz); + const segments = [baseTooltip]; + if (!effective) { + return baseTooltip; + } + const list = version?.listPriceBuzz; + if (typeof list === 'number' && Number.isFinite(list) && list > version.priceBuzz) { + segments.push( + translate( + 'modals.model.versions.badges.priceSaleTooltip', + { list: list.toLocaleString() }, + `On sale, normally ${list.toLocaleString()} Buzz` + ) + ); + } + if (version?.acceptsBlueBuzz) { + segments.push( + translate( + 'modals.model.versions.badges.priceBlueBuzzTooltip', + {}, + 'Can also be paid with Blue Buzz' + ) + ); + } + return segments.join(' · '); +} + function isDownloadAllowed(version) { if (!version.usageControl) { return true; @@ -522,22 +569,64 @@ function renderRow(version, options) { })); } - if (isEarlyAccess) { + // Obtainability, only where there is a decision to make: a version already in + // the library cannot become cheaper *for this user*, so it gets no cost badge. + // The download price is the only price axis that matters here — the generation + // fee is what CivitAI charges to generate on its own site, which a local model + // never incurs. + const priceLabel = formatBuzzPrice(version.priceBuzz); + if (!version.isInLibrary && isEarlyAccess) { + // Early access is the one state with a real countdown: "free on " + // decides between waiting and paying. + const eaTime = formatEarlyAccessTime(version.earlyAccessEndsAt); badges.push(buildBadge(earlyAccessBadgeLabel, 'early-access', { - title: translate( - 'modals.model.versions.badges.earlyAccessTooltip', - {}, - 'This version currently requires Civitai early access' + title: buildPriceTooltip( + version, + eaTime + ? translate( + 'modals.model.versions.badges.earlyAccessTooltipUntil', + { date: eaTime }, + `Early access until ${eaTime}` + ) + : translate( + 'modals.model.versions.badges.earlyAccessTooltip', + {}, + 'This version currently requires Civitai early access' + ) ), })); } - if (isPaidPermanent(version)) { - badges.push(buildBadge(paidBadgeLabel, 'paid', { + if (!version.isInLibrary && isGated(version)) { + if (priceLabel) { + // The number already says "paid", so a separate Paid badge would be + // noise next to it. + badges.push(buildBadge(priceLabel, isEarlyAccess ? 'early-access' : 'paid', { + title: buildPriceTooltip(version, paidBadgeLabel), + })); + } else if (!isEarlyAccess) { + // Paid, but no readable price: the gate is certain (it comes from the + // public API), only the number is best-effort. Saying nothing here + // would read as "free", which is the one thing this version is not. + badges.push(buildBadge(paidBadgeLabel, 'paid', { + title: translate( + 'modals.model.versions.badges.paidTooltip', + {}, + 'This version requires payment to download' + ), + })); + } + } + + // A version that used to be gated and no longer is. `gateLapsedAt` is + // persisted, so the marker survives long after the refresh that saw it. + if (!version.isInLibrary && !isGated(version) && version.gateLapsedAt) { + const freeBadgeLabel = translate('modals.model.versions.badges.freeNow', {}, 'Free Now'); + badges.push(buildBadge(freeBadgeLabel, 'success', { title: translate( - 'modals.model.versions.badges.paidTooltip', - {}, - 'This version requires payment to download' + 'modals.model.versions.badges.freeNowTooltip', + { date: formatDateLabel(version.gateLapsedAt) || '' }, + 'This version no longer requires payment' ), })); } diff --git a/static/js/managers/SettingsManager.js b/static/js/managers/SettingsManager.js index a34c801e..a0c9baa1 100644 --- a/static/js/managers/SettingsManager.js +++ b/static/js/managers/SettingsManager.js @@ -1169,6 +1169,17 @@ export class SettingsManager { hidePaidUpdatesCheckbox.checked = state.global.settings.hide_paid_updates || false; } + // Set buzz price tracking settings + const priceTrackingEnabledCheckbox = document.getElementById('priceTrackingEnabled'); + if (priceTrackingEnabledCheckbox) { + priceTrackingEnabledCheckbox.checked = state.global.settings.price_tracking_enabled || false; + } + + const priceCheckTtlInput = document.getElementById('priceCheckTtlHours'); + if (priceCheckTtlInput) { + priceCheckTtlInput.value = state.global.settings.price_check_ttl_hours ?? 24; + } + const skipPreviouslyDownloadedModelVersionsCheckbox = document.getElementById('skipPreviouslyDownloadedModelVersions'); if (skipPreviouslyDownloadedModelVersionsCheckbox) { skipPreviouslyDownloadedModelVersionsCheckbox.checked = diff --git a/static/js/state/index.js b/static/js/state/index.js index ff80aa07..d7e4f593 100644 --- a/static/js/state/index.js +++ b/static/js/state/index.js @@ -58,6 +58,8 @@ const DEFAULT_SETTINGS_BASE = Object.freeze({ version_grouping: 'same_base', hide_early_access_updates: false, hide_paid_updates: false, + price_tracking_enabled: false, + price_check_ttl_hours: 24, auto_organize_exclusions: [], metadata_refresh_skip_paths: [], skip_previously_downloaded_model_versions: false, diff --git a/static/js/utils/updateCheckHelpers.js b/static/js/utils/updateCheckHelpers.js index 5ecb7175..56ad6090 100644 --- a/static/js/utils/updateCheckHelpers.js +++ b/static/js/utils/updateCheckHelpers.js @@ -8,6 +8,54 @@ import { modalManager } from '../managers/ModalManager.js'; const CHECK_UPDATES_CONFIRMATION_KEY = 'ack_check_updates_for_all_models'; +/** + * Count the gate/price transitions reported by an update refresh. + * @param {Array} events - `events` array from the refresh response. + * @returns {{becameFree: number, newGate: number, priceDrop: number, total: number}} + */ +export function summarizeGateEvents(events) { + const list = Array.isArray(events) ? events : []; + return { + becameFree: list.filter((event) => event?.kind === 'became_free').length, + newGate: list.filter((event) => event?.kind === 'new_gate').length, + total: list.length + }; +} + +/** + * Toast the gate/price transitions of an update refresh, if any. Kept to a single + * toast so a large refresh cannot bury the user in notifications. + * @param {Array} events + */ +export function showGateEventsToast(events) { + const { becameFree, newGate } = summarizeGateEvents(events); + if (!becameFree && !newGate) { + return; + } + + const parts = []; + if (becameFree) { + parts.push( + translate( + 'globalContextMenu.checkModelUpdates.gateEvents.becameFree', + { count: becameFree }, + `${becameFree} version(s) became free` + ) + ); + } + if (newGate) { + parts.push( + translate( + 'globalContextMenu.checkModelUpdates.gateEvents.newGate', + { count: newGate }, + `${newGate} version(s) are now paid` + ) + ); + } + + showToast(parts.join(' · '), {}, becameFree ? 'success' : 'info'); +} + /** * Perform a model update check using the shared backend endpoint. * @param {Object} [options] @@ -85,6 +133,8 @@ export async function performModelUpdateCheck({ onStart, onComplete } = {}) { showToast('globalContextMenu.checkModelUpdates.none', { type: displayName }, 'info'); } + showGateEventsToast(payload.events); + await resetAndReload(false); } catch (err) { if (err?.name === 'AbortError') { @@ -180,6 +230,8 @@ export async function performFolderUpdateCheck(folderPath, { onComplete } = {}) showToast('sidebar.folderUpdateCheck.none', { type: displayName }, 'info'); } + showGateEventsToast(payload.events); + await resetAndReload(false); } catch (err) { if (err?.name === 'AbortError') { diff --git a/templates/components/modals/settings/library.html b/templates/components/modals/settings/library.html index 3a342d6b..f3314c77 100644 --- a/templates/components/modals/settings/library.html +++ b/templates/components/modals/settings/library.html @@ -192,6 +192,35 @@ {{ sm.setting_toggle('hidePaidUpdates', 'hide_paid_updates', 'settings.hidePaidUpdates.label', 'settings.hidePaidUpdates.help') }} + +
+ {{ sm.subsection_header('settings.sections.priceTracking') }} +
{{ t('settings.priceTracking.help') }}
+ {{ sm.setting_toggle('priceTrackingEnabled', 'price_tracking_enabled', 'settings.priceTracking.enabled', 'settings.priceTracking.enabledHelp') }} +
+
+
+ +
+
+
+ +
+
+
+
+
+
{{ sm.subsection_header('settings.sections.exampleImages') }} diff --git a/tests/frontend/components/modelVersionsTab.downloadButton.test.js b/tests/frontend/components/modelVersionsTab.downloadButton.test.js index 5510ebac..2c89f6f3 100644 --- a/tests/frontend/components/modelVersionsTab.downloadButton.test.js +++ b/tests/frontend/components/modelVersionsTab.downloadButton.test.js @@ -283,3 +283,116 @@ describe('ModelVersionsTab download button visibility', () => { expect(openFileSelectionForVersion).not.toHaveBeenCalled(); }); }); + +describe('ModelVersionsTab obtainability badges', () => { + let getModelApiClient; + let fetchModelUpdateVersions; + + beforeEach(async () => { + vi.resetModules(); + document.body.innerHTML = ` +
+
+
+
+
+ `; + ({ getModelApiClient } = await import(API_FACTORY_MODULE)); + fetchModelUpdateVersions = vi.fn(); + getModelApiClient.mockReturnValue({ + fetchModelUpdateVersions, + fetchModelRoots: vi.fn(), + setModelUpdateIgnore: vi.fn(), + setVersionUpdateIgnore: vi.fn(), + deleteModel: vi.fn(), + }); + }); + + afterEach(() => { + document.body.innerHTML = ''; + }); + + function rowFor(versionId) { + return document.querySelector(`.model-version-row[data-version-id="${versionId}"]`); + } + + function renderOne(version) { + fetchModelUpdateVersions.mockResolvedValue(buildRecord([ + { + versionId: 1, + name: 'v1', + isInLibrary: false, + shouldIgnore: false, + ...version, + }, + ])); + return renderVersions(); + } + + const PAID_GATE = { permanent: true, endsAt: null }; + + it('shows the download price for a paid version the user does not have', async () => { + await renderOne({ isPaid: true, paidAccess: PAID_GATE, priceBuzz: 5000 }); + + expect(rowFor(1).textContent).toContain('5,000 Buzz'); + }); + + it('says Paid without a number when the price could not be read', async () => { + // The gate is certain (it comes from the public API); only the number is + // best-effort, and that is our plumbing rather than the user's problem. + await renderOne({ isPaid: true, paidAccess: PAID_GATE, priceBuzz: null }); + + expect(rowFor(1).textContent).toContain('Paid'); + expect(rowFor(1).textContent).not.toContain('Buzz'); + }); + + it('shows the early access countdown alongside the price', async () => { + await renderOne({ + isEarlyAccess: true, + earlyAccessEndsAt: new Date(Date.now() + 3 * 86400000).toISOString(), + paidAccess: { permanent: false, endsAt: '2999-01-01T00:00:00.000Z' }, + priceBuzz: 500, + }); + + const text = rowFor(1).textContent; + expect(text).toContain('Early Access'); + expect(text).toContain('500 Buzz'); + }); + + it('shows nothing about cost for a version already in the library', async () => { + // A version on disk cannot become cheaper *for this user*. + await renderOne({ + isInLibrary: true, + isPaid: true, + paidAccess: PAID_GATE, + priceBuzz: 5000, + gateLapsedAt: '2026-09-28T00:00:00.000Z', + }); + + const text = rowFor(1).textContent; + expect(text).not.toContain('Buzz'); + expect(text).not.toContain('Paid'); + expect(text).not.toContain('Free Now'); + expect(text).toContain('In Library'); + }); + + it('marks a version that became free and is not in the library', async () => { + await renderOne({ + isInLibrary: false, + isPaid: false, + priceBuzz: null, + gateLapsedAt: '2026-09-28T00:00:00.000Z', + }); + + expect(rowFor(1).textContent).toContain('Free Now'); + }); + + it('shows nothing at all for a plain free version', async () => { + await renderOne({ isInLibrary: false, isPaid: false, priceBuzz: null }); + + const text = rowFor(1).textContent; + expect(text).not.toContain('Buzz'); + expect(text).not.toContain('Paid'); + expect(text).not.toContain('Free Now'); + }); +}); diff --git a/tests/routes/test_misc_routes.py b/tests/routes/test_misc_routes.py index 8453ed44..2c0a0974 100644 --- a/tests/routes/test_misc_routes.py +++ b/tests/routes/test_misc_routes.py @@ -6,7 +6,7 @@ import subprocess import zipfile from types import SimpleNamespace from typing import Any -from unittest.mock import patch, MagicMock +from unittest.mock import AsyncMock, patch, MagicMock import pytest from aiohttp import web @@ -16,6 +16,7 @@ from py.routes.handlers import misc_handlers from py.routes.handlers.misc_handlers import ( BackupHandler, DoctorHandler, + MiscHandlerSet, FileSystemHandler, HealthCheckHandler, LoraCodeHandler, @@ -2865,3 +2866,66 @@ async def test_sidecar_migration_handler_relocate_root_requires_old_root(): assert response.status == 400 assert "old_root" in payload["error"] assert use_case.calls == [] + + +# --- Global price alerts panel endpoint ------------------------------------- + + +class _AnyHandler: + def __getattr__(self, _name): + return lambda request: None + + +def _stub_misc_handler_set(**overrides) -> MiscHandlerSet: + names = ( + "health", + "settings", + "usage_stats", + "lora_code", + "trained_words", + "model_examples", + "node_registry", + "model_library", + "metadata_archive", + "backup", + "filesystem", + "custom_words", + "wildcards", + "supporters", + "doctor", + "example_workflows", + "base_model", + "model_source_handler", + "agent_handler", + "download_routing", + "sidecar_migration", + ) + handlers = {name: _AnyHandler() for name in names} + handlers.update(overrides) + return MiscHandlerSet(**handlers) + + +def test_every_misc_route_definition_resolves_to_a_handler(): + """A route added to the table without a mapping entry 500s only on a live + server, so assert the whole table resolves here.""" + + mapping = _stub_misc_handler_set().to_route_mapping() + + assert [ + definition.handler_name + for definition in MISC_ROUTE_DEFINITIONS + if definition.handler_name not in mapping + ] == [] + + +def test_price_alert_endpoints_are_gone(): + """The redesign removed the standalone alerts surface: obtainability is an + attribute of the update surfaces, so no route may serve an alert list.""" + + leftovers = [ + definition + for definition in MISC_ROUTE_DEFINITIONS + if "price-alert" in definition.path or "price_alert" in definition.handler_name + ] + + assert leftovers == [] diff --git a/tests/routes/test_model_update_handler.py b/tests/routes/test_model_update_handler.py index 5aea007e..77f17479 100644 --- a/tests/routes/test_model_update_handler.py +++ b/tests/routes/test_model_update_handler.py @@ -10,9 +10,11 @@ import pytest from py.config import config from py.routes.handlers.model_handlers import ( ModelCivitaiHandler, + ModelHandlerSet, ModelManagementHandler, ModelUpdateHandler, ) +from py.routes.model_route_registrar import COMMON_ROUTE_DEFINITIONS from py.services.service_registry import ServiceRegistry from py.utils.metadata_manager import MetadataManager from py.services.model_update_service import ModelUpdateRecord, ModelVersionRecord @@ -324,6 +326,72 @@ async def test_refresh_model_updates_filters_records_without_updates(): assert call["target_model_ids"] is None + + +@pytest.mark.asyncio +async def test_refresh_model_updates_reports_gate_events_for_all_records(): + """Gate transitions are reported even for records that do not qualify as + updates (a version already in the library that became free).""" + + cache = SimpleNamespace(version_index={}) + service = DummyService(cache) + record = ModelUpdateRecord( + model_type="lora", + model_id=1, + versions=[ + ModelVersionRecord( + version_id=11, + name="v11", + base_model=None, + released_at=None, + size_bytes=None, + preview_url=None, + is_in_library=True, + should_ignore=False, + ) + ], + last_checked_at=None, + should_ignore_model=False, + events=[{"versionId": 11, "kind": "became_free", "versionName": "v11", "isInLibrary": True}], + ) + update_service = DummyUpdateService({1: record}) + metadata_selector = AsyncMock(return_value=SimpleNamespace()) + + handler = ModelUpdateHandler( + service=service, + update_service=update_service, + metadata_provider_selector=metadata_selector, + settings_service=SimpleNamespace(get=lambda *_: False), + logger=logging.getLogger(__name__), + ) + + class DummyRequest: + can_read_body = True + query = {} + + async def json(self): + return {} + + response = await handler.refresh_model_updates( + DummyRequest() # pyright: ignore[reportArgumentType] + ) + payload = json.loads(response.text) + + # The record itself does not qualify as an update... + assert payload["records"] == [] + # ...but its transition is still surfaced. + assert payload["events"] == [ + { + "modelId": 1, + "modelType": "lora", + "versionId": 11, + "kind": "became_free", + "versionName": "v11", + "isInLibrary": True, + } + ] + + @pytest.mark.asyncio async def test_refresh_model_updates_same_base_scope_excludes_cross_base_updates(): """Issue #1083: with version_grouping=same_base (the default), a newer @@ -1077,3 +1145,50 @@ async def test_relink_civitai_surfaces_provider_unavailable_without_500(): payload = json.loads(response.text) assert payload["success"] is False assert "CivitArchive" in payload["error"] + + +def test_every_common_route_definition_resolves_to_a_handler(): + """Guard the declarative route table against a missing mapping entry. + + Adding a RouteDefinition without registering it in + ``ModelHandlerSet.to_route_mapping`` fails at request time with a bare + KeyError from the handler lookup (and only on a live server), so assert the + whole table resolves here instead. + """ + + class AnyHandler: + def __getattr__(self, _name): + return lambda request: None + + handler_set = ModelHandlerSet( + page_view=AnyHandler(), + listing=AnyHandler(), + management=AnyHandler(), + query=AnyHandler(), + download=AnyHandler(), + civitai=AnyHandler(), + move=AnyHandler(), + auto_organize=AnyHandler(), + filename_template=AnyHandler(), + updates=AnyHandler(), + ) + + mapping = handler_set.to_route_mapping() + missing = [ + definition.handler_name + for definition in COMMON_ROUTE_DEFINITIONS + if definition.handler_name not in mapping + ] + + assert missing == [] + + +def test_price_alert_endpoints_are_not_registered(): + """The redesign dropped the standalone alerts surface: obtainability rides on + the update surfaces, so no per-type alert route may remain.""" + + assert [ + definition.path_template + for definition in COMMON_ROUTE_DEFINITIONS + if "price-alert" in definition.path_template + ] == [] diff --git a/tests/services/test_civitai_client.py b/tests/services/test_civitai_client.py index 84abe602..a023ee23 100644 --- a/tests/services/test_civitai_client.py +++ b/tests/services/test_civitai_client.py @@ -1,4 +1,5 @@ import copy +import pathlib from unittest.mock import AsyncMock import pytest @@ -859,3 +860,166 @@ async def test_get_version_file_mini_propagates_rate_limit(downloader): with pytest.raises(RateLimitError): await client.get_version_file_mini(1, 2) + + +async def test_get_model_prices_parses_public_page(downloader): + """Prices come from the page payload, read anonymously (no API key).""" + client = await CivitaiClient.get_instance() + page_html = ( + '' + ) + + async def fake_make_request(method, url, use_auth=True, **kwargs): + assert method == "GET" + assert "/models/7" in url + assert use_auth is False + assert kwargs.get("custom_headers", {}).get("Accept") == "text/html" + return True, page_html + + downloader.make_request = fake_make_request + + result = await client.get_model_prices(7) + + assert result is not None + assert result[42]["price_buzz"] == 5000 + + +async def test_get_model_prices_returns_none_on_unusable_page(downloader): + client = await CivitaiClient.get_instance() + + async def fake_make_request(method, url, use_auth=True, **kwargs): + return True, "challenge" + + downloader.make_request = fake_make_request + + assert await client.get_model_prices(7) is None + + +async def test_get_model_prices_rejects_json_body(downloader): + """A JSON response is not the page; it must not be parsed as one.""" + client = await CivitaiClient.get_instance() + + async def fake_make_request(method, url, use_auth=True, **kwargs): + return True, {"error": "nope"} + + downloader.make_request = fake_make_request + + assert await client.get_model_prices(7) is None + + +async def test_get_model_prices_propagates_rate_limit(downloader): + client = await CivitaiClient.get_instance() + + async def fake_make_request(method, url, use_auth=True, **kwargs): + return False, RateLimitError("limited", retry_after=1.0) + + downloader.make_request = fake_make_request + + with pytest.raises(RateLimitError): + await client.get_model_prices(7) + + +# --- Model page host fallback ------------------------------------------------- + +_PAGE_FIXTURE = ( + pathlib.Path(__file__).resolve().parents[1] + / "utils" + / "fixtures" + / "civitai_model_page_paid.html" +) + + +def _page_html() -> str: + return _PAGE_FIXTURE.read_text(encoding="utf-8") + + +async def test_get_model_prices_falls_back_when_the_preferred_host_refuses( + downloader, monkeypatch +): + """civitai.red refuses non-browser clients (Cloudflare); the price must still + be readable from a host that answers. This is the user-visible bug: with + civitai_host=civitai.red every fetch used to fail.""" + + client = await CivitaiClient.get_instance() + monkeypatch.setattr(client, "_page_host", lambda: "civitai.red") + seen = [] + + async def fake_make_request(method, url, use_auth=True, **kwargs): + seen.append(url) + if "civitai.red" in url: + return False, "Access forbidden" + return True, _page_html() + + downloader.make_request = fake_make_request + + result = await client.get_model_prices(4242) + + assert result is not None + assert result[1001]["price_buzz"] == 5000 + assert seen[0].startswith("https://civitai.red/") + assert any("civitai.com" in url for url in seen) + # The working host is remembered and the refusing one is parked. + assert client._page_host_preference == "civitai.com" + assert "civitai.red" in client._page_host_blocked + + +async def test_get_model_prices_reuses_the_working_host(downloader, monkeypatch): + client = await CivitaiClient.get_instance() + monkeypatch.setattr(client, "_page_host", lambda: "civitai.red") + calls = [] + + async def fake_make_request(method, url, use_auth=True, **kwargs): + calls.append(url) + if "civitai.red" in url: + return False, "Access forbidden" + return True, _page_html() + + downloader.make_request = fake_make_request + + await client.get_model_prices(4242) + calls.clear() + await client.get_model_prices(4242) + + # Second time around the parked host is not retried. + assert calls and all("civitai.red" not in url for url in calls) + + +async def test_get_model_prices_returns_none_when_no_host_can_serve(downloader, monkeypatch): + """Mature models: 404 on civitai.com/green, challenge on civitai.red.""" + + client = await CivitaiClient.get_instance() + monkeypatch.setattr(client, "_page_host", lambda: "civitai.com") + + async def fake_make_request(method, url, use_auth=True, **kwargs): + if "civitai.red" in url: + return False, "Access forbidden" + return False, "Resource not found" + + downloader.make_request = fake_make_request + + assert await client.get_model_prices(2981320) is None + + +async def test_get_model_prices_does_not_park_a_host_on_404(downloader, monkeypatch): + """A 404 is model-specific (mature content hidden anonymously), not a reason + to stop using the host for other models.""" + + client = await CivitaiClient.get_instance() + monkeypatch.setattr(client, "_page_host", lambda: "civitai.com") + + async def fake_make_request(method, url, use_auth=True, **kwargs): + if "civitai.com" in url: + return False, "Resource not found" + return False, "Access forbidden" + + downloader.make_request = fake_make_request + + await client.get_model_prices(1) + + assert "civitai.com" not in client._page_host_blocked + assert "civitai.red" in client._page_host_blocked diff --git a/tests/services/test_model_update_service.py b/tests/services/test_model_update_service.py index 61d48bb2..83b60ce7 100644 --- a/tests/services/test_model_update_service.py +++ b/tests/services/test_model_update_service.py @@ -1,5 +1,6 @@ import logging import sqlite3 +from dataclasses import replace from types import SimpleNamespace import pytest @@ -826,6 +827,32 @@ def test_extract_single_version_paid_access_permanent(tmp_path): assert version.paid_access is not None +def test_extract_single_version_paid_access_pending_end(tmp_path): + """A timed gate whose window end is not recorded yet + ({"permanent": false, "endsAt": null}) is still an active gate, so it is + early access with no known end date rather than a free version.""" + db_path = tmp_path / "updates.sqlite" + service = ModelUpdateService(str(db_path)) + + entry = { + "id": 42, + "name": "v1 paid", + "availability": "Public", + "paidAccess": {"permanent": False, "endsAt": None}, + "files": [], + "images": [], + } + + version = service._extract_single_version(entry, index=0) + + assert version is not None + assert version.is_early_access is True + assert version.is_paid is False + assert version.early_access_ends_at is None + assert version.paid_access == '{"permanent": false, "endsAt": null}' + assert ModelUpdateRecord._is_early_access_active(version) is True + + def test_normalize_paid_access_accepts_json_string(): """The by-hash enrichment path may hand paidAccess to _normalize_paid_access as a JSON string; both the permanent and timed shapes must normalize.""" @@ -841,14 +868,21 @@ def test_normalize_paid_access_accepts_json_string(): ) assert timed == {"permanent": False, "endsAt": "2026-08-22T18:30:00.000Z"} - empty = ModelUpdateService._normalize_paid_access( + # A timed gate whose window end is not recorded yet. CivitAI only returns a + # non-null DTO for an ACTIVE gate (tombstones come back as null) and enforces + # this shape too - the model page reports canDownload: false for it - so it + # must be kept. Dropping it was the #1060 class of bug. + pending_end = ModelUpdateService._normalize_paid_access( '{"permanent": false, "endsAt": null}' ) - assert empty is None + assert pending_end == {"permanent": False, "endsAt": None} malformed = ModelUpdateService._normalize_paid_access("{not json") assert malformed is None + # No gate keys at all is not a gate signal. + assert ModelUpdateService._normalize_paid_access("{}") is None + def test_has_update_for_base_hide_paid(): """hide_paid also suppresses permanent paid versions in the same-base @@ -1059,3 +1093,811 @@ def test_build_record_from_remote_preserves_file_count(tmp_path): timestamp=2.0, ) assert record.versions[0].file_count == 1 + + +# --- Gate-state transitions and price persistence --------------------------- + + +def _remote_gated(version_id, *, price_checked_at=None, price_buzz=None): + return ModelVersionRecord( + version_id=version_id, + name=f"v{version_id}", + base_model=None, + released_at=None, + size_bytes=None, + preview_url=None, + is_in_library=False, + should_ignore=False, + paid_access='{"permanent": false, "endsAt": "2026-10-10T13:10:17.404Z"}', + is_early_access=True, + early_access_ends_at="2026-10-10T13:10:17.404Z", + price_checked_at=price_checked_at, + price_buzz=price_buzz, + ) + + +def _remote_free(version_id): + return ModelVersionRecord( + version_id=version_id, + name=f"v{version_id}", + base_model=None, + released_at=None, + size_bytes=None, + preview_url=None, + is_in_library=False, + should_ignore=False, + ) + + +def test_build_record_emits_became_free_event(tmp_path): + """A version whose gate lapsed produces a became_free event, keeps a lapse + timestamp, and drops the now-meaningless stored price.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record( + replace( + _remote_gated(7, price_checked_at=100.0, price_buzz=500), + ) + ) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_free(7)], + existing=existing, + timestamp=1000.0, + ) + + version = record.versions[0] + assert version.gate_lapsed_at is not None + assert version.price_buzz is None + assert version.price_checked_at is None + assert [event["kind"] for event in record.events] == ["became_free"] + assert record.events[0]["versionId"] == 7 + assert record.events[0]["isInLibrary"] is False + + +def test_build_record_emits_new_gate_event(tmp_path): + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record(_remote_free(7)) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_gated(7)], + existing=existing, + timestamp=1000.0, + ) + + assert [event["kind"] for event in record.events] == ["new_gate"] + assert record.versions[0].gate_lapsed_at is None + + +def test_build_record_no_events_without_previous_snapshot(tmp_path): + """First sight of a model must not report every existing gate as a new one.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_gated(7), _remote_free(8)], + existing=None, + timestamp=1000.0, + ) + + assert record.events == [] + + +def test_build_record_keeps_lapse_marker_and_skips_ignored(tmp_path): + """An already-free version keeps its original lapse marker and an ignored + version reports nothing at all.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + lapsed = replace(_remote_free(7), gate_lapsed_at="2026-09-01T00:00:00.000Z") + existing = make_record(lapsed, replace(_remote_gated(8), should_ignore=True)) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_free(7), _remote_free(8)], + existing=existing, + timestamp=1000.0, + ) + + by_id = {version.version_id: version for version in record.versions} + assert by_id[7].gate_lapsed_at == "2026-09-01T00:00:00.000Z" + assert record.events == [] + + +def test_build_record_preserves_price_when_fetch_skipped(tmp_path): + """A refresh that did not run a price fetch (no price_checked_at) must keep the + stored price instead of wiping it.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record( + replace(_remote_gated(7, price_checked_at=100.0, price_buzz=500)) + ) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_gated(7)], + existing=existing, + timestamp=1000.0, + ) + + assert record.versions[0].price_buzz == 500 + assert record.versions[0].price_checked_at == 100.0 + assert record.events == [] + + +def test_build_record_applies_fresh_price(tmp_path): + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record( + replace(_remote_gated(7, price_checked_at=100.0, price_buzz=500)) + ) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[], + remote_versions=[_remote_gated(7, price_checked_at=200.0, price_buzz=250)], + existing=existing, + timestamp=1000.0, + ) + + assert record.versions[0].price_buzz == 250 + assert record.versions[0].price_checked_at == 200.0 + + +def _legacy_version_table_sql() -> str: + """The model_update_versions schema before gate-lapse/price columns.""" + + return """ + CREATE TABLE model_update_versions ( + model_id INTEGER NOT NULL, + version_id INTEGER NOT NULL, + sort_index INTEGER NOT NULL DEFAULT 0, + name TEXT, + base_model TEXT, + released_at TEXT, + size_bytes INTEGER, + preview_url TEXT, + is_in_library INTEGER NOT NULL DEFAULT 0, + should_ignore INTEGER NOT NULL DEFAULT 0, + early_access_ends_at TEXT, + is_early_access INTEGER NOT NULL DEFAULT 0, + usage_control TEXT, + paid_access TEXT, + is_paid INTEGER NOT NULL DEFAULT 0, + file_count INTEGER, + PRIMARY KEY (model_id, version_id) + ) + """ + + +def test_migration_adds_price_columns_to_legacy_db(tmp_path): + """Opening a database written before this feature must add the columns and keep + the existing rows (the migration path real users hit).""" + + db_path = tmp_path / "updates.sqlite" + conn = sqlite3.connect(db_path) + conn.execute( + "CREATE TABLE model_update_status (" + "model_id INTEGER PRIMARY KEY, model_type TEXT NOT NULL, " + "last_checked_at REAL, should_ignore_model INTEGER NOT NULL DEFAULT 0)" + ) + conn.execute(_legacy_version_table_sql()) + conn.execute( + "INSERT INTO model_update_status (model_id, model_type, last_checked_at) " + "VALUES (999, 'lora', 1.0)" + ) + conn.execute( + "INSERT INTO model_update_versions (" + "model_id, version_id, is_in_library, is_early_access, paid_access, is_paid" + ") VALUES (999, 7, 1, 1, '{\"permanent\": false, \"endsAt\": null}', 0)" + ) + conn.commit() + conn.close() + + service = ModelUpdateService(str(db_path)) + columns = service._get_table_columns( + service._connect(), "model_update_versions" + ) + for column in ( + "gate_lapsed_at", + "price_buzz", + "list_price_buzz", + "generation_price_buzz", + "accepts_blue_buzz", + "price_sale_ends_at", + "price_checked_at", + "price_check_attempted_at", + ): + assert column in columns + # The alert-state columns were removed together with the threshold they served. + assert "price_alert_state" not in columns + assert "price_alert_since" not in columns + + record = service._get_record("lora", 999) + assert record is not None + assert [version.version_id for version in record.versions] == [7] + version = record.versions[0] + assert version.paid_access == '{"permanent": false, "endsAt": null}' + assert version.price_buzz is None + assert version.price_checked_at is None + assert version.gate_lapsed_at is None + assert version.accepts_blue_buzz is False + + +def test_price_fields_round_trip_through_sqlite(tmp_path): + """Price and lapse columns survive an upsert/read cycle.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + version = replace( + _remote_gated(7, price_checked_at=123.5, price_buzz=500), + list_price_buzz=600, + generation_price_buzz=100, + accepts_blue_buzz=True, + price_sale_ends_at="2026-10-01T00:00:00.000Z", + is_in_library=True, + gate_lapsed_at=None, + ) + service._upsert_record(make_record(version)) + + stored = service._get_record("lora", 999) + assert stored is not None + loaded = stored.versions[0] + assert loaded.price_buzz == 500 + assert loaded.list_price_buzz == 600 + assert loaded.generation_price_buzz == 100 + assert loaded.accepts_blue_buzz is True + assert loaded.price_sale_ends_at == "2026-10-01T00:00:00.000Z" + assert loaded.price_checked_at == 123.5 + assert loaded.price_check_attempted_at is None + + +# --- Optional price capture -------------------------------------------------- + + +class FakeSettings: + """Minimal stand-in for SettingsManager (only `.get` is used by these paths).""" + + def __init__(self, values=None): + self._values = dict(values or {}) + + def get(self, key, default=None): + return self._values.get(key, default) + + def set(self, key, value): + self._values[key] = value + + +class PriceProvider(DummyProvider): + """DummyProvider that can also serve prices.""" + + def __init__(self, response, *, prices=None, error=None): + super().__init__(response) + self.prices = prices + self.error = error + self.price_calls = 0 + + async def get_model_prices(self, model_id): + self.price_calls += 1 + if self.error is not None: + raise self.error + return self.prices + + +GATED_RESPONSE = { + "modelVersions": [ + { + "id": 12, + "baseModel": "Pony", + "availability": "Public", + "paidAccess": {"permanent": False, "endsAt": "2999-01-01T00:00:00.000Z"}, + "files": [], + "images": [], + } + ] +} +FREE_RESPONSE = { + "modelVersions": [ + {"id": 12, "baseModel": "Pony", "availability": "Public", "files": [], "images": []} + ] +} +LOCAL_RAW_DATA = [{"civitai": {"modelId": 1, "id": 11}, "base_model": "Pony"}] + +PRICE_PAYLOAD = { + 12: { + "price_buzz": 250, + "list_price_buzz": 500, + "generation_price_buzz": 100, + "accepts_blue_buzz": True, + "price_sale_ends_at": "2999-01-02T00:00:00.000Z", + } +} + + +def _price_service(tmp_path, **settings): + return ModelUpdateService( + str(tmp_path / "updates.sqlite"), + ttl_seconds=0, + settings_manager=FakeSettings(settings), + ) + + +@pytest.mark.asyncio +async def test_price_capture_off_by_default(tmp_path): + service = _price_service(tmp_path) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + record = await service.get_record("lora", 1) + + assert provider.price_calls == 0 + assert record.versions[0].price_checked_at is None + assert record.versions[0].price_buzz is None + + +@pytest.mark.asyncio +async def test_price_capture_stores_prices_for_gated_versions(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + record = await service.get_record("lora", 1) + + assert provider.price_calls == 1 + version = next(v for v in record.versions if v.version_id == 12) + assert version.price_buzz == 250 + assert version.list_price_buzz == 500 + assert version.generation_price_buzz == 100 + assert version.accepts_blue_buzz is True + assert version.price_sale_ends_at == "2999-01-02T00:00:00.000Z" + assert version.price_checked_at is not None + + +@pytest.mark.asyncio +async def test_price_capture_skips_ungated_models(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(FREE_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + + assert provider.price_calls == 0 + + +@pytest.mark.asyncio +async def test_price_capture_failure_keeps_stored_price(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + + await service.refresh_for_model_type( + "lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + ) + first = await service.get_record("lora", 1) + stored_checked_at = next( + v for v in first.versions if v.version_id == 12 + ).price_checked_at + assert stored_checked_at is not None + + # The page is unreadable this time (a changed gate still triggers a fetch): + # the price must survive untouched rather than being blanked. + failing = PriceProvider( + { + "modelVersions": [ + { + "id": 12, + "baseModel": "Pony", + "availability": "Public", + "paidAccess": { + "permanent": False, + "endsAt": "2999-06-01T00:00:00.000Z", + }, + "files": [], + "images": [], + } + ] + }, + prices=None, + ) + await service.refresh_for_model_type("lora", scanner, failing) + + record = await service.get_record("lora", 1) + version = next(v for v in record.versions if v.version_id == 12) + assert failing.price_calls == 1 + assert version.price_buzz == 250 + assert version.price_checked_at == stored_checked_at + + +@pytest.mark.asyncio +async def test_price_capture_respects_ttl(tmp_path): + """A second refresh with an unchanged gate and a fresh price must not refetch.""" + + service = _price_service( + tmp_path, price_tracking_enabled=True, price_check_ttl_hours=24 + ) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + await service.refresh_for_model_type("lora", scanner, provider) + + assert provider.price_calls == 1 + + +@pytest.mark.asyncio +async def test_price_capture_refetches_when_gate_changes(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + + changed_gate = { + "modelVersions": [ + { + "id": 12, + "baseModel": "Pony", + "availability": "Public", + "paidAccess": { + "permanent": True, + "endsAt": None, + }, + "files": [], + "images": [], + } + ] + } + provider.response = changed_gate + await service.refresh_for_model_type("lora", scanner, provider) + + assert provider.price_calls == 2 + + +@pytest.mark.asyncio +async def test_price_capture_survives_provider_without_support(tmp_path): + """A provider that cannot serve prices must not break the refresh.""" + + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = DummyProvider(GATED_RESPONSE) + + await service.refresh_for_model_type("lora", scanner, provider) + record = await service.get_record("lora", 1) + + assert record is not None + version = next(v for v in record.versions if v.version_id == 12) + assert version.price_buzz is None + + +@pytest.mark.asyncio +async def test_price_capture_drops_price_when_version_becomes_free(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + + await service.refresh_for_model_type( + "lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + ) + + await service.refresh_for_model_type("lora", scanner, PriceProvider(FREE_RESPONSE)) + record = await service.get_record("lora", 1) + version = next(v for v in record.versions if v.version_id == 12) + + assert version.price_buzz is None + assert version.price_checked_at is None + assert version.gate_lapsed_at is not None + + +# --- Price alerts ------------------------------------------------------------ + + +def _gated_response(ends_at: str) -> dict: + """Gated response with a specific EA end. + + Varying the end date between refreshes is what forces a re-fetch inside a + single test (a price is otherwise considered fresh for the whole TTL). + """ + + return { + "modelVersions": [ + { + "id": 12, + "baseModel": "Pony", + "availability": "Public", + "paidAccess": {"permanent": False, "endsAt": ends_at}, + "files": [], + "images": [], + } + ] + } + + +def _prices(price_buzz: int) -> dict: + return {12: {"price_buzz": price_buzz, "list_price_buzz": price_buzz}} + + + + + + + + + + + + +# --- Price alerts panel semantics (read-time threshold, across types) --------- + + +def _gated_response_for(version_id: int, ends_at: str) -> dict: + return { + "modelVersions": [ + { + "id": version_id, + "baseModel": "Pony", + "availability": "Public", + "paidAccess": {"permanent": False, "endsAt": ends_at}, + "files": [], + "images": [], + } + ] + } + + +def _free_response_for(version_id: int) -> dict: + return { + "modelVersions": [ + { + "id": version_id, + "baseModel": "Pony", + "availability": "Public", + "files": [], + "images": [], + } + ] + } + + + + + + + + + + + + +@pytest.mark.asyncio +async def test_failed_price_attempt_is_recorded_as_unavailable(tmp_path): + """A gated version we tried to price and could not must be distinguishable + from one we never looked at — that is the honest "unavailable" state for + mature models, whose pages no host will serve anonymously.""" + + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + failing = PriceProvider(GATED_RESPONSE, prices=None) + + await service.refresh_for_model_type("lora", scanner, failing) + record = await service.get_record("lora", 1) + version = next(v for v in record.versions if v.version_id == 12) + + assert failing.price_calls == 1 + assert version.price_buzz is None + assert version.price_checked_at is None + assert version.price_check_attempted_at is not None + + +@pytest.mark.asyncio +async def test_successful_price_attempt_sets_both_markers(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + + await service.refresh_for_model_type( + "lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + ) + record = await service.get_record("lora", 1) + version = next(v for v in record.versions if v.version_id == 12) + + assert version.price_buzz == 250 + assert version.price_checked_at is not None + assert version.price_check_attempted_at is not None + + +@pytest.mark.asyncio +async def test_no_price_attempt_is_recorded_while_tracking_is_off(tmp_path): + service = _price_service(tmp_path) # tracking off + scanner = DummyScanner(LOCAL_RAW_DATA) + + await service.refresh_for_model_type("lora", scanner, DummyProvider(GATED_RESPONSE)) + record = await service.get_record("lora", 1) + + assert record.versions[0].price_check_attempted_at is None + + +@pytest.mark.asyncio +async def test_unavailable_marker_clears_when_the_version_becomes_free(tmp_path): + service = _price_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + + await service.refresh_for_model_type( + "lora", scanner, PriceProvider(GATED_RESPONSE, prices=None) + ) + + await service.refresh_for_model_type("lora", scanner, DummyProvider(FREE_RESPONSE)) + record = await service.get_record("lora", 1) + + assert record.versions[0].price_check_attempted_at is None + + +def _long_ttl_service(tmp_path, **settings): + """A service whose metadata TTL does not lapse during the test.""" + + return ModelUpdateService( + str(tmp_path / "updates.sqlite"), + ttl_seconds=86400, + settings_manager=FakeSettings(settings), + ) + + +@pytest.mark.asyncio +async def test_prices_are_fetched_even_when_the_version_list_is_fresh(tmp_path): + """Enabling price tracking must not wait for the metadata TTL. + + The cached record already carries the gate, so a fresh version list is no + reason to skip the price: otherwise turning the feature on prices only the + handful of models that happened to need a metadata refresh that round. + """ + + service = _long_ttl_service(tmp_path, price_tracking_enabled=False) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + assert provider.price_calls == 0 + metadata_calls = provider.calls + checked_at_before = (await service.get_record("lora", 1)).last_checked_at + + service._settings.set("price_tracking_enabled", True) + await service.refresh_for_model_type("lora", scanner, provider) + + record = await service.get_record("lora", 1) + version = next(v for v in record.versions if v.version_id == 12) + + # The version list came from the cache this round ... + assert provider.calls == metadata_calls + # ... and the price was captured anyway. + assert provider.price_calls == 1 + assert version.price_buzz == 250 + assert version.price_checked_at is not None + assert version.price_check_attempted_at is not None + # A price-only pass must not extend the metadata TTL. + assert record.last_checked_at == checked_at_before + + +@pytest.mark.asyncio +async def test_failed_price_attempt_is_not_retried_within_the_ttl(tmp_path): + """A mature model whose page no host will serve must not cost two requests + on every single update check.""" + + service = _long_ttl_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=None) + + await service.refresh_for_model_type("lora", scanner, provider) + assert provider.price_calls == 1 + + await service.refresh_for_model_type("lora", scanner, provider) + + assert provider.price_calls == 1 + + + + +@pytest.mark.asyncio +async def test_forced_refresh_reprices_within_the_ttl(tmp_path): + service = _long_ttl_service(tmp_path, price_tracking_enabled=True) + scanner = DummyScanner(LOCAL_RAW_DATA) + provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD) + + await service.refresh_for_model_type("lora", scanner, provider) + assert provider.price_calls == 1 + + await service.refresh_for_model_type( + "lora", scanner, provider, force_refresh=True + ) + + assert provider.price_calls == 2 + + +# --- Ownership scoping and the removed threshold ----------------------------- + + +def test_gate_event_is_suppressed_for_a_version_the_user_owns(tmp_path): + """A version already on disk cannot become cheaper *for this user*, so it must + not produce an event. More than half of a real library's gated versions are + owned, so this is the common case, not an edge case.""" + + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record(replace(_remote_gated(7), is_in_library=True)) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[7], + remote_versions=[_remote_free(7)], + existing=existing, + timestamp=1000.0, + ) + + assert record.events == [] + # The lapse itself is still recorded - it is the *notification* that is + # pointless for a version the user already has. + assert record.versions[0].gate_lapsed_at is not None + + +def test_new_gate_event_is_suppressed_for_a_version_the_user_owns(tmp_path): + service = ModelUpdateService(str(tmp_path / "updates.sqlite")) + existing = make_record(replace(_remote_free(7), is_in_library=True)) + + record = service._build_record_from_remote( + model_type="lora", + model_id=999, + local_versions=[7], + remote_versions=[_remote_gated(7)], + existing=existing, + timestamp=1000.0, + ) + + assert record.events == [] + + +def test_threshold_setting_is_gone_from_the_defaults(): + """The redesign removed the numeric threshold: every obtainability decision is + categorical (wait / pay / skip), so no number is stored or compared.""" + + from py.services.settings_manager import DEFAULT_SETTINGS + + assert "price_alert_threshold_buzz" not in DEFAULT_SETTINGS + assert "price_tracking_enabled" in DEFAULT_SETTINGS + assert "price_check_ttl_hours" in DEFAULT_SETTINGS + + +def test_obsolete_alert_columns_are_dropped_from_a_development_database(tmp_path): + """A database created while the feature was unreleased still carries the two + alert-state columns; opening it must drop them rather than leave dead schema.""" + + db_path = tmp_path / "updates.sqlite" + service = ModelUpdateService(str(db_path)) + with service._connect() as conn: + conn.execute( + "ALTER TABLE model_update_versions " + "ADD COLUMN price_alert_state INTEGER NOT NULL DEFAULT 0" + ) + conn.execute( + "ALTER TABLE model_update_versions ADD COLUMN price_alert_since REAL" + ) + conn.commit() + assert "price_alert_state" in service._get_table_columns( + conn, "model_update_versions" + ) + + # A fresh service instance runs the migration on open. + reopened = ModelUpdateService(str(db_path)) + columns = reopened._get_table_columns( + reopened._connect(), "model_update_versions" + ) + + assert "price_alert_state" not in columns + assert "price_alert_since" not in columns diff --git a/tests/utils/fixtures/civitai_model_page_paid.html b/tests/utils/fixtures/civitai_model_page_paid.html new file mode 100644 index 00000000..0640855f --- /dev/null +++ b/tests/utils/fixtures/civitai_model_page_paid.html @@ -0,0 +1,108 @@ + + + + + Fixture model + + + + + diff --git a/tests/utils/test_civitai_page_prices.py b/tests/utils/test_civitai_page_prices.py new file mode 100644 index 00000000..7c1542a5 --- /dev/null +++ b/tests/utils/test_civitai_page_prices.py @@ -0,0 +1,91 @@ +"""Tests for the CivitAI model page price parser. + +The fixture mirrors a real page payload; the cases that matter are the ones that +decide whether a user gets a price at all, and whether a lapsed gate is mistaken +for a live one. +""" + +from pathlib import Path + +import pytest + +from py.utils.civitai_page_prices import MAX_PAGE_BYTES, parse_model_page_prices + +FIXTURE = Path(__file__).parent / "fixtures" / "civitai_model_page_paid.html" + + +@pytest.fixture(scope="module") +def fixture_html() -> str: + return FIXTURE.read_text(encoding="utf-8") + + +def test_parses_prices_for_gated_versions(fixture_html): + prices = parse_model_page_prices(fixture_html) + + assert prices is not None + # Lapsed gates and never-gated versions must not appear. + assert set(prices) == {1001, 1002, 1003} + + +def test_permanent_gate_price(fixture_html): + prices = parse_model_page_prices(fixture_html) + permanent = prices[1001] + + assert permanent["price_buzz"] == 5000 + assert permanent["list_price_buzz"] == 5000 + assert permanent["generation_price_buzz"] == 100 + assert permanent["accepts_blue_buzz"] is False + assert permanent["price_sale_ends_at"] is None + + +def test_timed_gate_uses_sale_price_as_effective(fixture_html): + prices = parse_model_page_prices(fixture_html) + timed = prices[1002] + + # The stored/list price stays visible for a strikethrough, the effective price + # is what a buyer pays now. + assert timed["price_buzz"] == 100 + assert timed["list_price_buzz"] == 125 + assert timed["accepts_blue_buzz"] is True + assert timed["price_sale_ends_at"] == "2999-09-01T00:00:00.000Z" + # Generation is bundled with the download tier here (`trialLimit` only). + assert timed["generation_price_buzz"] is None + + +def test_timed_gate_without_recorded_end_is_still_priced(fixture_html): + prices = parse_model_page_prices(fixture_html) + + assert prices[1003]["price_buzz"] == 300 + assert prices[1003]["generation_price_buzz"] is None + + +@pytest.mark.parametrize( + "html", + [ + None, + "", + "no payload here", + '', + # Valid JSON, but not the shape we need. + '', + # Model query present, but no versions. + '', + ], +) +def test_unusable_payloads_return_none(html): + assert parse_model_page_prices(html) is None + + +def test_oversized_page_is_skipped(): + oversized = "x" * (MAX_PAGE_BYTES + 1) + assert parse_model_page_prices(oversized) is None + + +def test_unrelated_first_query_does_not_win(fixture_html): + """Query order varies per page; selection is by procedure name.""" + + prices = parse_model_page_prices(fixture_html) + assert prices is not None + assert 1001 in prices diff --git a/tests/utils/test_paid_access.py b/tests/utils/test_paid_access.py new file mode 100644 index 00000000..3d63da0e --- /dev/null +++ b/tests/utils/test_paid_access.py @@ -0,0 +1,112 @@ +"""Tests for the shared CivitAI paidAccess interpretation. + +These rules decide both the Paid/Early Access badges and whether a download is +warned about, so the interesting cases are the boundary shapes CivitAI actually +emits. +""" + +from datetime import datetime, timezone + +import pytest + +from py.utils.paid_access import ( + is_early_access_deadline_active, + is_gate_active, + is_permanent_paid, + normalize_paid_access, + parse_civitai_timestamp, +) + +NOW = datetime(2026, 9, 30, 12, 0, 0, tzinfo=timezone.utc) + + +@pytest.mark.parametrize( + "value,expected", + [ + (None, None), + ("", None), + ("{not json", None), + ("[1, 2]", None), + ({}, None), + ({"unknown": 1}, None), + # A non-null DTO from CivitAI's public API is always an active gate, so a + # timed gate with no recorded end must survive normalization. + ({"permanent": False, "endsAt": None}, {"permanent": False, "endsAt": None}), + ({"permanent": True, "endsAt": None}, {"permanent": True, "endsAt": None}), + ( + {"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"}, + {"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"}, + ), + ], +) +def test_normalize_paid_access_shapes(value, expected): + assert normalize_paid_access(value) == expected + + +def test_normalize_paid_access_accepts_json_text(): + """The by-hash enrichment path can carry the DTO as a JSON string.""" + + assert normalize_paid_access('{"permanent": true, "endsAt": null}') == { + "permanent": True, + "endsAt": None, + } + + +def test_normalize_paid_access_blank_ends_at_is_none(): + assert normalize_paid_access({"permanent": False, "endsAt": " "}) == { + "permanent": False, + "endsAt": None, + } + + +def test_is_permanent_paid(): + assert is_permanent_paid({"permanent": True, "endsAt": None}) is True + assert is_permanent_paid({"permanent": False, "endsAt": None}) is False + assert is_permanent_paid(None) is False + + +@pytest.mark.parametrize( + "paid_access,expected", + [ + (None, False), + ({"permanent": False, "endsAt": None}, True), + ({"permanent": True, "endsAt": None}, True), + # A permanent gate with a stale endsAt stays active. + ({"permanent": True, "endsAt": "2020-01-01T00:00:00.000Z"}, True), + ({"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"}, True), + ({"permanent": False, "endsAt": "2026-08-27T16:56:57.438Z"}, False), + ({"permanent": False, "endsAt": "not-a-date"}, True), + ], +) +def test_is_gate_active(paid_access, expected): + assert is_gate_active(paid_access, now=NOW) is expected + + +@pytest.mark.parametrize( + "ends_at,expected", + [ + (None, False), + ("", False), + ("2026-10-10T13:10:17.404Z", True), + ("2026-08-27T16:56:57.438Z", False), + ("garbage", True), + ], +) +def test_is_early_access_deadline_active(ends_at, expected): + assert is_early_access_deadline_active(ends_at, now=NOW) is expected + + +@pytest.mark.parametrize( + "value,expected", + [ + (None, None), + (123, None), + ("", None), + ("garbage", None), + ("2026-10-10T13:10:17.404Z", datetime(2026, 10, 10, 13, 10, 17, 404000, tzinfo=timezone.utc)), + # Naive timestamps are assumed UTC. + ("2026-10-10T13:10:17", datetime(2026, 10, 10, 13, 10, 17, tzinfo=timezone.utc)), + ], +) +def test_parse_civitai_timestamp(value, expected): + assert parse_civitai_timestamp(value) == expected