Merge pull request #1137 from willmiao/feat/paid-model-price-tracking

feat(updates): show Buzz download prices and surface obtainability of available versions
This commit is contained in:
pixelpaws
2026-10-05 12:32:42 +08:00
committed by GitHub
38 changed files with 4017 additions and 98 deletions
+11
View File
@@ -139,6 +139,17 @@ Locales: `en`, `zh-CN`, `zh-TW`, `ja`, `ko`, `fr`, `de`, `es`, `ru`, `he` (RTL).
> `.showcaseLayoutOptions.gallery` / `.vertical`, `modals.model.showcase.layoutGallery` /
> `.layoutList`). All 9 locales are translated (terminology in §2, "Showcase layout
> feature"), so the "no remaining placeholders" claim holds again.
> **Status (2026-10, Buzz download prices):** the paid/early-access obtainability feature added
> 13 keys — the two `globalContextMenu.checkModelUpdates.gateEvents.*` counts, five
> `modals.model.versions.badges.*` (sale, Blue Buzz, "free now" and its tooltip, and the early
> access end-date tooltip) and the six `settings.priceTracking.*` strings — plus the section
> header. All 13 are now translated in all 9 locales. **Buzz** and **Blue Buzz** stay as-is
> everywhere (CivitAI currency names, R3). Two source fixes came with the pass: the unused
> `settings.priceTracking.label` key was removed (no template renders it; the toggle uses
> `enabled`/`enabledHelp`) and `settings.sections.priceTracking` was reworded to
> "Buzz Download Prices" so the header matches what the feature does (prices are displayed;
> nothing is tracked for alerts). Register follows each file's existing norm: 你 (zh-CN),
> 您 (zh-TW), Sie (de), tú (es), вы (ru). No remaining `[TODO: Translate]` placeholders.
---
+841
View File
@@ -0,0 +1,841 @@
# Plan: Buzz Price Tracking and Threshold Alerts for Paid / Early Access Versions
**Origin:** FR "Price tracker for models/loras" (Geekier, Discord) — track the buzz price of paid
model versions and flag when it drops below a threshold or becomes free.
**Related:** [#1060 — Some "Early Access" models are not identified correctly](https://github.com/willmiao/ComfyUI-Lora-Manager/issues/1060)
(closed) established `is_paid`, the Paid badge and `hide_paid_updates`; this FR is the next step
after gate *state* — gate *price*.
**Status:** v3 — **P0–P3 implemented** (§10 records what shipped and the deviations); **P5 (alerts
panel) planned** in §11, not implemented. The grid-level "price alert only" filter was dropped by
owner decision, so §11 defines the panel as the only new browsing surface.
Feasibility was verified against this repo and upstream CivitAI `main` (`6d29ed1368`), including live
probes against `civitai.com` / `civitai.red`.
**Scope:** every model type that goes through `ModelUpdateService` (lora / checkpoint / embedding /
other). Out of scope: purchasing, downloading gated content, and any change that depends on a future
CivitAI API addition.
---
## 1. Problem statement
Creators increasingly gate model versions behind buzz. LoRA Manager already knows **whether** a
version is gated (`is_paid`, `is_early_access`, `paid_access`) but not **how much it costs**, so it
cannot answer the two questions the FR asks:
1. "Tell me when this model's price drops below X buzz."
2. "Tell me when it becomes free."
Additionally, the current code only compares version ids when deciding `has_update`, so a version
that a user already tracks can go from paid to free (or gain a gate) without producing any signal at
all.
## 2. Verified current state
### 2.1 LoRA Manager (all references confirmed in this checkout)
| Fact | Reference |
| --- | --- |
| Gate fields persisted per version: `is_paid`, `is_early_access`, `paid_access` (raw DTO JSON), `early_access_ends_at` | `py/services/model_update_service.py:65-84` |
| Gate data comes from the bulk `GET /api/v1/models?ids=…` response (`availability`, `paidAccess`, `earlyAccessEndsAt`) | `py/services/civitai_client.py:329-380`, parsed at `py/services/model_update_service.py:1756-1793` |
| **No numeric price anywhere in `py/`** | `grep -rn "price" py/` → only license/`allowCommercialUse` hits |
| Update refresh is user-triggered, not scheduled; TTL 24 h | route `py/routes/handlers/model_handlers.py:2899`; `ModelUpdateService.__init__` default `ttl_seconds=24*60*60` at `py/services/model_update_service.py:338` |
| `has_update` compares version ids + gate filters only; no state-transition detection | `py/services/model_update_service.py:117-160` |
| `{permanent:false, endsAt:null}` is deliberately treated as **not** a gate on the download path | `py/services/download_manager.py:2000-2003` |
| Version payload sent to the frontend already carries `isPaid` / `paidAccess` | `py/routes/handlers/model_handlers.py:3444-3455` |
| UI: Paid / Early Access badges, `hide_paid_updates` filter | `static/js/components/shared/ModelVersionsTab.js:169-196, 474-545`; `static/js/state/index.js:58-59`; `templates/components/modals/settings/library.html:191-192` |
### 2.2 Upstream CivitAI — what is and is not public
| Fact | Evidence |
| --- | --- |
| Public v1 API intentionally strips pricing: `paidAccess` is reduced to `{permanent, endsAt}`. Source comment: *"Omits terms (pricing belongs to the purchase flow)"* | `~/code/civitai/src/server/services/paid-access.service.ts:640-652`; verified live: `/api/v1/models` and `/api/v1/model-versions/{id}` return exactly that shape |
| The internal tRPC route is **not** usable anonymously. `isAcceptableOrigin` rejects non-site origins with `401 "Please use the public API instead"` | `~/code/civitai/src/server/trpc.ts:120-128`; `acceptableOrigin` = `!isProd \|\| isBearerAuth \|\| isAllowedOriginRequest(req)` (`createContext.ts:43`), and the origin check is header-based (`src/server/utils/origin-helpers.ts:27-32`). **Live probe warning:** an anonymous tRPC GET can return `200` when Cloudflare serves a cached copy (`cf-cache-status: HIT`); uncached it is `401`. Do not build on this path. |
| **The price *is* publicly reachable inside the public model page.** The SSR payload embeds the site's own `model.getById` result, including full `terms` | `https://civitai.com/models/<id>` and `https://civitai.red/models/<id>` → `<script id="__NEXT_DATA__" type="application/json">` → `props.pageProps.trpcState.json.queries[*]` where `queryKey[0] == ["model","getById"]` → `state.data.modelVersions[*].paidAccess` |
| Observed payloads (anonymous, no cookies) | version 3379626 → `{terms:{download:{price:5000},generation:{price:100,trialLimit:5}}, endsAt:null, sale:null}`; version 3380114 → `download.price 125` `acceptsBlueBuzz:true` `endsAt:2026-10-10`; another → `200` |
| Per-generation licensing fees are already public and official | `GET /api/v1/model-versions/mini/{id}` (`MixedAuthEndpoint`, anonymous 200) exposes `fees` / `freeTrialLimit` — `~/code/civitai/src/pages/api/v1/model-versions/mini/[id].ts:455` |
| Webhooks cannot carry price changes | `updated-model` is driven by `Model.lastVersionAt` (fires only on new versions) and its select has no `paidAccess`: `~/code/civitai/src/server/webhooks/model.webooks.ts:88-96`, `~/code/civitai/src/server/selectors/model.selector.ts:15-36` |
| Prevalence (sampled newest 600 models / 4634 versions via public API cursor): **90 gated (1.9%), 87 permanent vs 3 timed** | live scan, 2026-xx; dominated by permanent gates, so "EA expiry" alone covers ~3% of gated versions |
### 2.3 Consequences for the design
* Threshold alerts on download price require **one new data source**: the model page payload. It is
public, but it is page data — so it must be isolated, opt-in, cheap, and fail-open.
* Request budget is small because only gated models need it: ~2% of models, one extra ~200 KB
request per gated model per price TTL.
* "Becomes free" and "gained a gate" need no new data source at all and should ship first.
* One page fetch returns **all** versions of a model, so prices cost one request per model, not per
version.
## 3. Data source decision
**Chosen: `__NEXT_DATA__` of the public model page**, fetched with the existing HTTP stack.
Rejected alternatives:
* *Internal tRPC* — origin-gated; passing it means forging `Origin` or relying on bearer auth against
an undocumented endpoint whose own error message says to use the public API. Not worth the
stability/compliance risk for a convenience feature.
* *Official v1 API as-is* — no price field, by design. Track it as an upstream request (§8), not as a
dependency.
* *`mini/{id}` `fees`* — useful extra signal for per-generation fees, but it is not the download
price and does not cover the gate.
## 4. Design
### 4.1 Schema (all additive, in the existing update DB)
`model_update_versions` gains:
```sql
ALTER TABLE model_update_versions ADD COLUMN price_buzz INTEGER; -- effective (sale-adjusted) download price
ALTER TABLE model_update_versions ADD COLUMN list_price_buzz INTEGER; -- undiscounted download price
ALTER TABLE model_update_versions ADD COLUMN generation_price_buzz INTEGER;
ALTER TABLE model_update_versions ADD COLUMN accepts_blue_buzz INTEGER NOT NULL DEFAULT 0;
ALTER TABLE model_update_versions ADD COLUMN price_sale_ends_at TEXT;
ALTER TABLE model_update_versions ADD COLUMN price_checked_at REAL; -- last *successful* price fetch
ALTER TABLE model_update_versions ADD COLUMN price_alert_state INTEGER NOT NULL DEFAULT 0; -- last computed alert hit, for edge detection
```
Adding a column to this table requires touching **all eight** enumerations (missing one silently
drops data or breaks reads):
1. dataclass `ModelVersionRecord` — `py/services/model_update_service.py:65-84`
2. `_SCHEMA` `CREATE TABLE` — `:300-330`
3. `_apply_migrations` column dict — `:554-580`
4. `_migrate_model_update_versions_primary_key` `target_columns` + `defaults` — `:680-745` (**easy to
miss; omitting it loses the new columns whenever the PK migration runs**)
5. `_build_record_from_remote` — `:1630-1665`
6. `_extract_single_version` — `:1740-1795`
7. `_get_records_bulk` SELECT + row→record mapping — `:1945-1990`
8. `_upsert_record` INSERT — `:2028-2066`
`price_checked_at` lives on the version row (not the model status row) so a stale price on one
version does not force a re-fetch for its siblings; the fetch itself is per model.
### 4.2 Parser contract
New pure function, unit-testable without network:
```
parse_model_page_prices(html: str) -> dict[int, dict] | None
```
* Locate `<script id="__NEXT_DATA__" type="application/json">…</script>`; `json.loads` it.
* Walk `props.pageProps.trpcState.json.queries[*]`; pick the entry whose `queryKey[0] == ["model","getById"]`
(the query order is not stable — 7 queries were observed, `getById` first only incidentally).
* From `state.data.modelVersions[*]`, per version emit:
* `price_buzz` = `paidAccess.sale.buyerTerms.download.price` if `sale` present, else
`paidAccess.terms.download.price`
* `list_price_buzz` = `paidAccess.terms.download.price`
* `generation_price_buzz` = `terms.generation.price` (undefined for `{free:true}` / bundled)
* `accepts_blue_buzz`, `price_sale_ends_at`
* no `paidAccess` → the version is ungated: emit an explicit "no price" marker so the caller can
clear stale prices and raise a *became free* event.
* Return `None` (never raise) for: missing script tag, JSON error, shape mismatch, empty
`modelVersions`, or a Cloudflare/challenge page.
### 4.3 Refresh flow and request budget
Hook: `_refresh_single_model` (`py/services/model_update_service.py:1099-1215`), inside the existing
"lock released during network I/O" window, after `fetched_versions` is built:
```
prices_needed = feature_enabled and (any gated version or any version with a stored price)
if prices_needed:
prices = await provider.get_model_prices(model_id) # never raises; None on failure
if prices is not None:
fetched_versions = [apply_price(v, prices.get(v.version_id)) for v in fetched_versions]
# price_checked_at advances only for versions actually present in the payload
```
* Candidate selection uses the gate data already in hand from the bulk API, so no extra call is made
for the ~98% of models with no gate (except models that previously had a price and must confirm
"now free").
* TTL: `price_check_ttl_hours` (default 24 h), independent of the update TTL — a price refresh can be
skipped while the version list is fresh, and vice versa. Force refresh (`force_refresh=True`) also
forces prices.
* Failure behaviour: keep the previous price, do **not** advance `price_checked_at` (so the next
refresh retries), log at debug/info. A `RateLimitError` from the shared gate must remain a skip,
not a hard failure of the update check. After N consecutive parse failures within one refresh,
stop attempting price fetches for the remainder of that run.
* Offline/cooldown: reuse the existing `ConnectivityGuard` path in `downloader.make_request` — no new
connectivity handling.
Client/provider seam:
* `ModelMetadataProvider.get_model_prices(model_id)` — default returns `None` (not abstract, so
CivArchive / SQLite providers are untouched).
* `CivitaiModelMetadataProvider` → `CivitaiClient.get_model_prices(model_id)`.
* Add the pass-through to the composite provider wrapper
(`py/services/model_metadata_provider.py:855-955`) so the rate-limit helper applies.
* `CivitaiClient.get_model_prices`: build the URL with the existing helper
`build_civitai_model_page_url(model_id, host=self._settings.get("civitai_host"))`
(`py/utils/civitai_utils.py:44-66`; `civitai.red` is already a supported page host,
`:10`), fetch with `downloader.make_request("GET", url, use_auth=False, custom_headers={"Accept": "text/html"})`
(returns `str` because the body is not JSON), then call the parser. Cap the body size before
parsing.
### 4.4 Alert semantics
* **Threshold hit**: `price_buzz is not None and price_buzz <= threshold_buzz`. `threshold_buzz = 0`
means "free only, plus any gate removal".
* **Became free**: a version with a stored gate/price now has no `paidAccess` (from the bulk API
and/or the page payload).
* **Newly gated**: a version that had none now has `paidAccess`.
* Edge detection uses `price_alert_state` plus the previous `paid_access` JSON, compared in
`_build_record_from_remote` where `existing_map` is already built
(`py/services/model_update_service.py:1636-1665`). Do not put this in `has_update`: it is a
different question, and `has_update` is filtered by the hide-* settings.
* `ModelUpdateRecord` gains a **non-persisted** field (e.g. `events: list | None = None`) carrying
`{version_id, kind: "price_drop"|"became_free"|"new_gate", price_buzz, previous_price_buzz}`.
`_upsert_record` ignores it; `_get_record` leaves it `None`.
### 4.5 Settings and routes
New settings (defaults in `DEFAULT_SETTINGS`, `py/services/settings_manager.py:68`;
`settings.json.example` stays minimal per repo policy; frontend defaults in
`static/js/state/index.js:58`):
| Key | Default | Meaning |
| --- | --- | --- |
| `price_tracking_enabled` | `False` | master switch; off means no model-page fetches at all |
| `price_alert_threshold_buzz` | `0` | alert when effective price ≤ this; 0 = free only |
| `price_check_ttl_hours` | `24` | price freshness window |
Routes (all under the existing model route registrars, `POST` + `GET` because the companion
extension is GET-only per `AGENTS.md`):
* `GET /api/lm/models/price-alerts` — aggregate over the update DB:
`{modelId, modelType, versionId, versionName, priceBuzz, listPriceBuzz, thresholdBuzz, isFree}`.
* Prices themselves need no new read route: they ride along in the existing version payload
(`py/routes/handlers/model_handlers.py:3438-3455` gains `priceBuzz`, `listPriceBuzz`,
`generationPriceBuzz`, `acceptsBlueBuzz`, `priceCheckedAt`, `priceAlert`).
* The existing update-refresh response gains `events` (from §4.4) so the frontend can toast.
### 4.6 UI
* `static/js/components/shared/ModelVersionsTab.js`
* Extend the existing Paid / Early Access badge tooltips with the price when known
("Paid · 5,000 Buzz") — helpers at `:169-196`, badges at `:474-545`.
* New badges: price chip (`paid`/`info` styling) and "Free now" for a became-free transition.
* Filter toggle reusing the `hide_paid_updates` pattern (`:349-400`, plus the settings modal and
`SettingsManager.js:1157-1163`) — e.g. "Show price drops only".
* Optional, cheap win: include the price in the paid-download error text
(`py/services/download_manager.py:1987-2031`, `py/services/use_cases/download_model_use_case.py:35`).
* Strings: add to `locales/en.json`, then `python scripts/sync_translation_keys.py`, then **stop** —
do not translate the other locales (see `docs/i18n-translation-guidelines.md` §7).
## 5. Phased tasks
### P0 — settle gate semantics (prerequisite, small)
- [ ] Decide the meaning of `{permanent:false, endsAt:null}`. The repo currently treats it as *no
gate* (`py/services/download_manager.py:2000-2003`), while upstream `isPaidAccessActive`
(`endsAt == null || endsAt > now`) treats it as *active*. Live sample: version 3372655
currently returns exactly this DTO from `/api/v1/models`.
- [ ] Introduce one shared helper (e.g. `_has_active_gate(paid_access, early_access_ends_at)` in
`py/services/model_update_service.py`) and use it from both the update service and the download
gate, so alerts and download blocking cannot disagree.
- [ ] Unit test for the chosen rule.
- **Why first:** every alert kind inherits the gate-state decision; getting it wrong produces false
"free" alerts.
### P1 — gate-state change detection (no new data source)
- [ ] Emit `new_gate` / `became_free` events from `_build_record_from_remote`; add the non-persisted
`events` field; surface in the refresh response.
- [ ] Keep `has_update` semantics unchanged (regression risk to `hide_paid_updates`,
`hide_early_access_updates`).
- [ ] UI: "Free now" badge + one batched toast per refresh; extend paid/EA tooltips.
- [ ] Tests in `tests/services/test_model_update_service.py` for each transition and for the
no-change case.
- **Acceptance:** a version whose gate lapses is visibly marked free after a refresh, with no new
network calls; existing hide-* filters behave identically.
### P2 — price capture (opt-in)
- [ ] Schema + all eight column touchpoints from §4.1, with a migration test.
- [ ] `parse_model_page_prices` + fixture-based tests (`tests/services/fixtures/…`, one trimmed page
with a timed gate, one with a permanent gate, one malformed/missing payload).
- [ ] `CivitaiClient.get_model_prices` + provider methods + composite pass-through; assert redirect
following and that a non-JSON body is returned as text.
- [ ] Wire into `_refresh_single_model` behind `price_tracking_enabled`, with the failure matrix from
§4.3.
- [ ] Tests: feature off ⇒ zero extra calls; fetch failure ⇒ previous price retained and
`price_checked_at` unchanged; gated-only candidate selection.
- **Acceptance:** with tracking enabled, a refresh on a gated model stores `price_buzz` for all of
its versions from one request; with tracking disabled or offline, refresh behaves exactly as today.
### P3 — threshold alerts and surface
- [ ] Settings + defaults (§4.5) and the settings-modal toggles/inputs.
- [ ] Threshold evaluation + `price_alert_state` edges → `price_drop` events.
- [ ] `GET /api/lm/models/price-alerts` aggregate endpoint.
- [ ] Version payload gains the price fields; UI price chip, "Free now" badge, "price drops only"
filter; optional price in the paid-download error.
- [ ] Locale keys + `scripts/sync_translation_keys.py`.
- [ ] Tests: threshold boundary (`<=` vs `<`), Blue Buzz labelling, sale-adjusted vs list price,
alerts for a version already in the library.
- **Acceptance:** a user can set "alert me under 500 buzz", refresh, and see the models that
qualify; nothing alerts when the price rises above the threshold again (state resets).
### P4 — docs and upstream
- [ ] Short feature note in the repo docs; keep this plan's data-source rationale in a code comment
(public page read, no auth or origin spoofing, no internal API).
- [ ] Draft the upstream request: add a price field to the public `paidAccess` DTO. Point out the
asymmetry — writes already go through the official v1 endpoint
(`~/code/civitai/src/pages/api/v1/model-versions/early-access.ts`, sharing
`updateModelVersionPaidAccessSchema`) while reads withhold it deliberately. If it lands, the
page scraper becomes an optional fallback rather than the only path.
## 6. Test plan summary
* `pytest tests/services/test_model_update_service.py` — migrations, extraction, transitions,
threshold logic, fetch-failure matrix, feature-off path.
* New parser tests with HTML fixtures (no network in tests).
* `npm run test:js` for badge/helper changes in `ModelVersionsTab`.
* Manual UI verification by the user (per repo policy — no browser automation for layout).
* No live CivitAI calls in the test suite.
## 7. Risks and mitigations
| Risk | Mitigation |
| --- | --- |
| Page payload format changes (Next.js is migrating away from `__NEXT_DATA__` to the flight payload) | Parser is a single pure function behind a provider method; fixture test fails loudly at build time; parse failure is non-fatal and keeps the last known price |
| Cloudflare / UA sensitivity (verified: `civitai.com`, `civitai.red` and even no-UA all returned the payload for the sampled pages) | Reuse `downloader` (proxy, rate-limit coordinator, connectivity guard); detect challenge/absence and back off; never retry in a tight loop |
| Traffic/size (~200-270 KB per model) | Gated models only (~2%), one request per model, `price_check_ttl_hours`, force-refresh only on user action |
| False "free" alerts from gate-semantics ambiguity | P0 helper + explicit tests before any alert ships |
| Misleading price (list vs sale-adjusted; Blue Buzz is the same number but non-withdrawable) | Store both list and effective price; tooltip states the unit and Blue Buzz acceptance |
| Alerts are pull-only (no scheduler, no push channel today) | State it in the UI copy; treat background push as a separate follow-up, not a silent gap |
## 8. Out of scope / follow-ups
* Purchasing or downloading gated content.
* Generation-fee tracking beyond displaying `generation_price_buzz` (the public `mini/{id}` `fees`
field could feed a richer view later).
* A background/notification channel; today alerts appear when the user refreshes or opens the panel.
* Waiting on CivitAI to expose prices; P1–P3 stand alone.
## 9. Open questions
1. `{permanent:false, endsAt:null}` — gate or not? (P0; affects download blocking too.)
2. Should the threshold apply to versions already in the library (watching for a future re-buy) or
only to versions the user does not own?
3. Is a per-model threshold override needed in v1, or is one global threshold enough?
---
## 10. What shipped (P0–P3)
### 10.1 Answers to the open questions
1. **`{permanent:false, endsAt:null}` is a gate.** Decided by evidence, not preference: the public v1
API returns `null` for lapsed gates (`toPublicPaidAccessDto` filters on `isPaidAccessActive`), so a
non-null DTO is always an *active* gate; on the live model 1802980 that version reports
`canDownload: false` while its lapsed-tombstone siblings report `true`. The previous rule dropped
it, which is exactly the #1060 class of bug.
2. The global threshold applies to **every** gated version the update service tracks, in library or
not — the user may be watching a version they intend to buy later.
3. One global threshold in v1; a per-model override is still open.
### 10.2 Files and seams
| Area | Where |
| --- | --- |
| Shared gate semantics (`normalize_paid_access`, `is_gate_active`, `is_early_access_deadline_active`) | `py/utils/paid_access.py` (new) |
| Model page price parser (`parse_model_page_prices`) | `py/utils/civitai_page_prices.py` (new) |
| Price fetch | `CivitaiClient.get_model_prices` (`py/services/civitai_client.py`) |
| Provider seam | `ModelMetadataProvider.get_model_prices` default None; CivitAI impl; fallback + rate-limit wrappers (`py/services/model_metadata_provider.py`) |
| Columns, migration, transition + threshold logic, alerts query | `py/services/model_update_service.py` |
| Version payload, refresh `events`, `GET /api/lm/{type}/updates/price-alerts` | `py/routes/handlers/model_handlers.py`, `py/routes/model_route_registrar.py` |
| Badges, price chip, gate/price toast | `static/js/components/shared/ModelVersionsTab.js`, `static/js/utils/updateCheckHelpers.js`, `static/css/components/lora-modal/versions.css` |
| Settings | defaults in `py/services/settings_manager.py`; UI in `templates/components/modals/settings/library.html` + `static/js/managers/SettingsManager.js`; client defaults in `static/js/state/index.js` |
| API client | `getPriceAlerts` in `static/js/api/baseModelApi.js` + `priceAlerts` endpoint in `static/js/api/apiConfig.js` |
### 10.3 Deviations from the plan
* **`gate_lapsed_at` column added** (8th column): a transient event would leave a "became free"
marker invisible minutes later, so the lapse timestamp is persisted and the UI shows a `Free Now`
badge with it.
* **Alerts are edge-triggered, and first sight is silent.** When a price first appears under the
threshold, `price_alert_state` is stored and the version shows up in `price-alerts` immediately,
but no toast fires — otherwise switching the feature on would announce every cheap version in the
library at once. The toast fires when a price *crosses* down through the threshold.
* **No dedicated alerts panel.** The surfaces are: price chip + `Free Now` badge in the versions tab,
one combined toast per update check, and the `price-alerts` endpoint (GET, so the companion
extension can use it). A browse surface over the endpoint is a follow-up.
* **Prices are only fetched for models with a gate** (~2% of a typical library), one page request per
model, on `price_check_ttl_hours`, and immediately when the gate DTO changes.
* **The paid-download warning message does not include the price**: that path reads the model's
CivitAI metadata, which has no price, and looking one up would add a request to a failure path.
Left as a follow-up.
### 10.4 Test coverage added
* `tests/utils/test_paid_access.py` — gate shapes, activity, timestamps.
* `tests/utils/test_civitai_page_prices.py` + `tests/utils/fixtures/civitai_model_page_paid.html` —
parser, including sale-adjusted vs list price and lapsed tombstones.
* `tests/services/test_model_update_service.py` — migration from a pre-feature DB, column round-trip,
transitions, price capture on/off, failure retention, TTL, threshold edges, alerts query.
* `tests/services/test_civitai_client.py` — page fetch happy path, unusable payloads, rate limits.
* `tests/routes/test_model_update_handler.py` — alerts endpoint, gate events for non-updating records.
---
## 11. P5 — Price alerts panel
**Goal:** a single surface that answers "what got cheaper / became free, and can I act on it".
**Status:** **P5a implemented** (see §11.9); P5b (inline threshold editing, *Refresh prices*, type
chips, ignore action) and P5c (polish) are not.
**Naming:** called P4a–c in earlier discussion; renumbered to **P5** because §5 already uses P4 for
"docs and upstream".
**Owner decision:** the grid-level "price alert only" filter is **out of scope**. The panel is the
only new browsing surface; the existing per-version badges and the update-check toast stay as they
are. Consequence: the panel carries the "act on it" affordances itself (§11.2), and discoverability
rests on the bell badge plus the controls-dropdown entry (§11.1 D8).
### 11.1 Locked decisions
| # | Decision | Why / evidence |
| --- | --- | --- |
| D1 | Host it as a **third tab in the existing notification bell modal**, driven by `UpdateService` | `templates/components/modals.html` is included by `templates/base.html:75`, so the bell exists on every page; `UpdateService` already has `toggleUpdateModal()`, `switchNotificationTab()`, tab badges, arrow-key tab navigation and a list/empty-state pattern (`renderRecentBanners`) |
| D2 | One **global** endpoint `GET /api/lm/price-alerts`, registered **once** in `MiscRoutes` | The update DB is one file per library shared by all model types (`cache/model_update/<library>.sqlite`), and `ServiceRegistry.get_model_update_service()` returns one shared instance (`py/services/service_registry.py:153`). `get_price_alerts` only filters on `s.model_type`, so `model_type=None` is all types in one query. Adding it to `COMMON_ROUTE_DEFINITIONS` would bind the same path once per model type (4×) |
| D3 | **Compare the threshold at read time** (`WHERE v.price_buzz <= ?`); keep `price_alert_state` for the toast edge only | As shipped, panel membership only changes after a refresh, so editing the threshold in the panel would look broken |
| D4 | New column `price_alert_since REAL` | `price_alert_state` is a boolean; "dropped 3 days ago" and an unread count both need the moment the state flipped, which the existing edge detection already knows |
| D5 | Unread state is **client-side** (`localStorage` watermark compared against `price_alert_since`) | No per-user read-state table and no sync logic; the worst case is a conservative badge on a second browser |
| D6 | Row actions: **CivitAI** always; **Open** only when the payload resolved a local file path | `showModelModal(model, modelType)` needs a local metadata object and re-fetches by `file_path` (`static/js/components/shared/ModelModal.js:343-361`); a gated version the user does not own has no local file |
| D7 | **Refresh prices** must bypass the price TTL | Prices stay fresh for `price_check_ttl_hours` (24 h by default), so without a force flag the button would appear to do nothing |
| D8 | **Two non-permanent entry points**, both opening the bell on the Price alerts tab: the **updates dropdown** in the controls bar, and the **global context menu** (right-click on empty space) | The controls dropdown already hosts `checkUpdatesMenuItem` (`templates/components/controls.html:114-126`) and costs no layout space. The global menu is the established home for library-wide occasional actions and already holds the sibling `check-model-updates` (`templates/components/context_menu.html:185`); it has only 7 items and existing hide/separator machinery (`GlobalContextMenu.showMenu` / `_updateSeparatorVisibility`). The **model-card menu is deliberately left alone** — it already has 20 items, and P5b's *Refresh prices* covers the per-model case |
| D9 | Bell badge shows the **unread alert count**; when there are none it keeps today's behaviour (dot for app updates). The same cached count is appended to the global context menu item (`Price alerts (3)`) | Otherwise a price drop is invisible until the user opens the bell. The count is fetched once on init **only when `price_tracking_enabled`**, so users who never enable the feature pay nothing |
| D10 | The global context menu item is **always visible** on model pages (hidden on recipes, like its siblings) | The panel's disabled state is the explanation plus a deep link into Settings; hiding the entry would make the feature undiscoverable for exactly the users who have not enabled it yet |
### 11.2 Information architecture
```
┌ Price alerts [Refresh prices] ┐
│ Threshold: 500 Buzz (click to edit) │
│ [ Under threshold ] [ Became free ] Type: All · LoRA · … │
├──────────────────────────────────────────────────────────────────────┤
│ ▸ Glorious Art · Checkpoint · 2 versions │
│ Alpha · 250 Buzz (was 500) · Blue Buzz OK │
│ EA until Oct 10 · Not in library · dropped 3 days ago │
│ [CivitAI] [Open] [Ignore] │
│ ▸ Eira Kishida · LoRA · 1 version │
│ 125 Buzz · In library · dropped today [CivitAI] [Open] │
└──────────────────────────────────────────────────────────────────────┘
```
* Segments cover both halves of the FR: `Under threshold` (price at or below the threshold) and
`Became free` (versions with `gate_lapsed_at`, no price).
* Grouped by model (a model often has several versions at the same price), sorted cheapest first
then most recently crossed; the type chips filter client-side over the loaded list.
* Row: effective price with the list price struck through when on sale, Blue Buzz note, EA end date,
in-library marker, and "dropped X ago" from `price_alert_since`.
* Actions: open on CivitAI, open the local model modal (only when a file path was resolved), ignore
this version (reuses `setVersionUpdateIgnore`).
### 11.3 Data contract
```
GET /api/lm/price-alerts?limit=200
{
"success": true,
"enabled": true, // price_tracking_enabled
"thresholdBuzz": 500,
"newestCheckedAt": 1791039694.5, // staleness copy ("prices last checked X ago")
"alerts": [
{
"modelId": 2981320, "modelType": "checkpoint",
"modelName": "Glorious Art", // best-effort, from the scanner cache
"versionId": 3379626, "versionName": "Alpha",
"kind": "below_threshold", // or "became_free"
"priceBuzz": 250, "listPriceBuzz": 500,
"acceptsBlueBuzz": true, "priceSaleEndsAt": null,
"priceAlertSince": 1791000000.0,
"gateLapsedAt": null,
"earlyAccessEndsAt": null, "isPaid": true, "isEarlyAccess": false,
"isInLibrary": false,
"filePath": null, // set only when resolvable
"civitaiUrl": "https://civitai.com/models/2981320?modelVersionId=3379626"
}
]
}
```
One list with a `kind` discriminator (not two lists) so "All" needs no second request. `civitaiUrl`
is built server-side with the existing `build_civitai_model_page_url` so `civitai_host` stays the
single source of truth.
### 11.4 States (all three must be designed, not just the happy path)
1. **Tracking off** — explain that CivitAI publishes no price in its public API and that the feature
reads the model page, plus a button that opens Settings → Library.
2. **On, nothing matching** — "nothing under N Buzz right now", with the threshold editable inline.
3. **Stale / offline / rate-limited** — keep the last known list, add a "prices last checked X ago"
line and a retry; never blank the panel.
### 11.5 Tasks
**P5a — panel usable end to end**
1. `py/services/model_update_service.py`
* `get_price_alerts(model_type=None, *, threshold_buzz, limit=200)` — optional type filter, live
threshold comparison, `kind`, `price_alert_since`; keep the shipped per-type call site working.
* `price_alert_since` column through all eight enumerations (§4.1), set on the `0 → 1` edge and
cleared on `1 → 0` inside `_build_record_from_remote`.
2. `py/routes/misc_route_registrar.py` + `py/routes/handlers/misc_handlers.py` — register
`GET /api/lm/price-alerts` **once**, using `ServiceRegistry.get_model_update_service()` and the
settings service; resolve `modelName` / `filePath` best-effort per model type from the scanner
caches (`version_index[version_id]` → `file_path`, `file_name`; item `model_name` with a
`file_name` fallback), and omit them when resolution fails.
3. `templates/components/modals/update_modal.html` — third tab + panel skeleton (segments, threshold
row, list container, empty/disabled blocks), reusing `data-notification-tab` /
`data-notification-panel`.
4. `static/js/managers/UpdateService.js` — `renderPriceAlerts()`, panel fetch on open, tab badge
count, threshold display, row rendering and actions, `localStorage` watermark; extend
`updateTabBadges()` and `switchNotificationTab()`.
5. `static/js/api/apiConfig.js` + a small fetch helper — the endpoint is global, so it does not
belong in the per-model-type `endpoints` map.
6. `templates/components/controls.html` + `static/js/components/controls/PageControls.js` — the
controls-dropdown item (D8).
7. `templates/components/context_menu.html` + `static/js/components/ContextMenu/GlobalContextMenu.js`
— the global-context-menu item (D8): one template entry, the recipes-page hide list in
`showMenu()`, one `case` in `handleMenuAction`, and the optional `(N)` count from the cached
alert count. Both entries call one shared `openPriceAlertsPanel()` helper.
8. `locales/en.json` + `python scripts/sync_translation_keys.py`.
*Acceptance:* with tracking on and one refresh done, the bell shows a count, the tab lists the
versions with prices, CivitAI opens the right page, Open appears only for in-library models and opens
the modal, all three states render, and it works from every page type.
**P5b — actions and polish**
* Inline threshold editing (read-time comparison makes it instant) and **Refresh prices** with a
`force_price_refresh` flag on the existing refresh endpoint (or a dedicated
`POST /api/lm/price-alerts/refresh`).
* Type chips and a "only versions I do not own" toggle (this is open question 2 from §9).
* Ignore-this-version action.
**P5c — optional**
* Thumbnails from the scanner cache, "unread only", focus-management pass on the new tab, and a
deep link from the panel into the settings section.
### 11.6 Tests
* `tests/services/test_model_update_service.py` — `model_type=None` spans types; live threshold
parameter; `price_alert_since` set/cleared on the edge.
* `tests/routes/` — the global route is registered exactly once and returns the documented shape;
`filePath` resolution is best-effort (mock scanner; omit on failure).
* `tests/frontend/` — vitest for `renderPriceAlerts` covering rows, both segments, the three states
and the badge count with a mocked fetch.
* Manual eyeball for layout, per the repo's UI verification policy.
### 11.7 Risks
| Risk | Mitigation |
| --- | --- |
| The bell modal is nominally about app updates; a third tab changes its character | Clear labelling, and only count/badge when the feature is on; the app-update dot behaviour is unchanged when there are no alerts |
| Resolving local file paths pulls scanner caches into an app-wide route | Best-effort with `try/except`, one `get_cached_data()` per type (in-memory), omit the field and hide the Open button when it fails |
| `price_alert_since` is another migration | Same eight-touchpoint discipline as §4.1, covered by the existing migration test |
| A global route in `COMMON_ROUTE_DEFINITIONS` would bind 4× | Register it in `MiscRoutes` instead (D2) |
| Menu bloat / entry sprawl | Two entries maximum (D8), both calling one helper; the 20-item model-card menu is explicitly untouched |
| Panel data goes stale between refreshes | Show `newestCheckedAt`; the refresh action (P5b) makes it explicit rather than silent |
### 11.8 Open questions for the owner
1. **Badge policy** — count unread alerts on the bell (recommended, D9), or leave the bell alone and
put the number only on the tab?
2. **Unread at all** — the `localStorage` watermark (recommended, D5), or simply "current matches"
with no read state?
3. **Threshold editing inside the panel** — convenient, but it silently changes a global setting from
a surface the user opened just to look.
4. **"Open in library"** — worth the scanner-cache coupling (D6), or should the row offer only
"CivitAI" plus a copy-link action?
5. **Thumbnails** — skip for weight (recommended) or show them?
6. **Entry visibility when tracking is off** — always show the global-context-menu item
(recommended, D10: the panel's disabled state educates and deep-links to Settings), or hide it
until the feature is enabled for a cleaner menu?
7. **Count in the menu label** — `Price alerts (3)` using the cached count (recommended, D9), or a
plain label with the number only on the bell/tab?
### 11.9 What shipped (P5a)
Answers to §11.8: badge counts unread alerts (D9), unread uses the `localStorage` watermark (D5),
threshold editing is **display-only in P5a** (editing is P5b), "Open in library" ships with the
best-effort path resolution (D6), no thumbnails, the context-menu item is always visible (D10), and
the menu label carries the count (D9).
| Area | Where |
| --- | --- |
| `price_alert_since` column through all eight enumerations, set on the `0 → 1` edge, preserved while the alert stands, cleared when the price rises | `py/services/model_update_service.py` |
| `get_price_alerts(model_type=None, *, threshold_buzz, limit)` — read-time threshold, `kind`, one list across types; `newest_price_checked_at()` | `py/services/model_update_service.py` |
| Global endpoint `GET /api/lm/price-alerts`, registered once | `py/routes/misc_route_registrar.py`, `py/routes/misc_routes.py`, `PriceAlertsHandler` in `py/routes/handlers/misc_handlers.py` |
| Third bell tab, panel skeleton, segments, three states | `templates/components/modals/update_modal.html`, `static/css/components/modal/update-modal.css` |
| Loader, renderer, unread watermark, badge, `openPriceAlertsPanel()` | `static/js/managers/UpdateService.js` |
| Entry points (controls dropdown + global context menu, count in the label) | `templates/components/controls.html`, `static/js/components/controls/PageControls.js`, `templates/components/context_menu.html`, `static/js/components/ContextMenu/GlobalContextMenu.js` |
Deviations and decisions made while implementing:
* **`became_free` is reported even while price tracking is off.** It needs no price data, so the
panel shows the "tracking is off" explanation *and* whatever became free instead of hiding the
half of the feature that already works.
* **`price_alert_since` is also set on first sight** of an already-cheap version. The toast stays
silent (edge-triggered, §10.3), but the count is non-zero, which is what invites the user into the
panel after enabling the feature.
* **The per-type frontend client method was removed** (`baseModelApi.getPriceAlerts` and the
`priceAlerts` entry in `apiConfig.js`): with the global endpoint it was dead code. The per-type
**backend** route stays for the companion extension and a possible future grid filter.
* **`openPriceAlertsTab()` exists because `toggleUpdateModal()` closes an open bell** — an entry
point calling it unconditionally would dismiss the modal instead of switching tabs.
* **Local context uses the existing indexes** (`cache.model_id_index` for `model_name`,
`cache.version_index` for `file_path`/`file_name`), so it is O(1) per row; every failure just
omits the fields and the row loses its "Open" button.
Verification: `pytest` 3652 passed / 7 skipped, `npm run test:js` 1444 passed, plus a sandboxed
standalone server run that seeded the update DB and confirmed the payload shape, the `kind`
split, the `civitaiUrl`, and that changing `price_alert_threshold_buzz` through `POST /api/lm/settings`
changes panel membership immediately with no refresh.
### 11.10 Known limitation: mature (NSFW) model prices, and the host fallback
End-to-end verification against the live site found that the page fetch was broken for a whole class
of users: **the hosts are not interchangeable**, and the user's `civitai_host` preference was
silently fatal.
Measured with the app's own HTTP stack (aiohttp) and with httpx, browser User-Agent in both cases:
| Target | civitai.com | civitai.green | civitai.red |
| --- | --- | --- | --- |
| Anonymously visible model page | 200, prices parse | 200, prices parse (same bytes as .com) | **403 Cloudflare challenge** |
| Mature (NSFW) model page | 404 | 404 | **403 Cloudflare challenge** |
* `civitai.red` refuses non-browser clients outright — User-Agent, `Accept-Language`, `Sec-Fetch-*`
and switching HTTP library all make no difference. (An earlier manual check with `curl` passed by
luck of TLS fingerprint, which is why this was missed: it was a false positive.)
* A real user on `civitai_host=civitai.red` therefore captured **zero** prices
(`price_checked_at = 0` in their update DB) even with tracking enabled.
* Mature models are hidden from anonymous visitors on `.com`/`.green` and only served by `.red`, so
**no host can currently read their price.** The public API cannot help: it trims `paidAccess` to
`{permanent, endsAt}` by design, and the public `mini/{id}` endpoint exposes only per-generation
`fees`.
What P5a now does about it:
* **Host fallback** (`CivitaiClient.get_model_prices`): the configured host is tried first, then the
others (`civitai_page_host_candidates`), first parseable payload wins. The host that worked is
remembered, and a host that refuses outright (403) is parked for 15 minutes — a host-wide failure
must not cost three requests per mature model in the library. A 404 is model-specific and does
**not** park the host.
* **Honest "unavailable" state**: `price_check_attempted_at` separates "we tried and could not read
a price" from "we never looked". Gated versions in that state show a muted `Price unavailable`
badge in the versions tab, and the panel reports `unavailableCount`.
* **Diagnostics**: a host refusal is warned once per host per TTL, and a model with no price source
logs the per-host reasons (404 vs challenge) instead of failing silently at debug level.
Recorded options for mature models, deliberately **not** implemented:
1. **Internal tRPC with the user's API key** (`modelVersion.getById` on `.red`): the route is a
`publicProcedure` with `requiredScope: ModelsRead`, and `isBearerAuth` satisfies
`acceptableOrigin`, so the user's own key would work and `.red`'s `/api` paths are not challenged.
Rejected as the default because the endpoint is undocumented and its own 401 message says to use
the public API. If ever wanted, it belongs behind an off-by-default setting.
2. **Extension-assisted fetch**: `lm-civitai-extension` runs inside the user's browser, so it has
both the Cloudflare clearance and the login session needed to read mature pages. This is the only
route that would work without an undocumented API, but it is a cross-component design of its own.
**Strengthened upstream ask:** the public API should expose the price. The argument is no longer
"convenience" — the page route is demonstrably unreliable (one host challenges non-browser clients,
the other two hide mature models from anonymous visitors), so a supported field is the only way for
any third-party tool to show prices for the models where creators monetize most.
### 11.11 Price capture is independent of the metadata TTL
Found in a real instance: after enabling price tracking, a normal "Check updates" captured exactly
**one** price out of 718 models. The price capture only ran when the *version list* was re-fetched
(`refresh_succeeded`), so it inherited the metadata TTL: with 24 h metadata and 24 h price TTLs, only
the 6 models whose metadata happened to be stale were ever priced.
The cached record already carries the gate, so the price pass now runs off whichever version list is
available — freshly fetched or stored — and applies the result without touching `last_checked_at`, so
a price-only pass cannot silently extend the metadata TTL. Two related semantics:
* A **failed** attempt (`price_check_attempted_at`) satisfies the price TTL, so a mature model whose
page no host will serve is not retried on every single update check.
* An explicitly **forced** check re-prices within the TTL (`_should_fetch_prices(..., force=True)`).
Verified by copying a real instance's update DB into a sandbox and running a non-forced check:
`bulk metadata fetches: 0` (version lists came entirely from cache) while priced versions went
**1 → 20** and the panel listed 19 alerts.
### 11.12 The empty state has to explain a 0 Buzz threshold
The default threshold is 0 ("only tell me when a version becomes free"), and the settings copy says
so. The panel did not: a real instance with **52 priced paid versions** and an untouched threshold
showed "Nothing is under your price threshold right now" plus a small "Alert threshold: 0 Buzz" in
the corner, which reads as "the feature is broken".
The empty state is now threshold-aware, using a new `pricedCount` in the payload: when the threshold
is 0 and prices are known, it says how many paid versions have a price and where to set a threshold.
The read-time threshold comparison means changing it takes effect immediately — no re-check needed
(measured on a copy of that instance's DB: 0 Buzz -> 0 alerts, 100 -> 44, 500 -> 48, 5000 -> 51).
## 12. Redesign (locked): obtainability of versions you do not own
### 12.1 Why the P5a model was wrong
The owner could not tell, from the UI alone, what "Buzz Price Tracking" enables, what the
"Price alert threshold" number means, or what "Price Alerts" is alerting about. That is not a copy
problem: the implementation exposed **our mechanism** (a page scrape) and **our SQL predicates**
(`price_buzz <= ?`, `gate_lapsed_at IS NOT NULL`) as the user's concepts. Two concrete defects came
out of the same root:
* **The alert population included versions the user already owns.** Neither the event generator nor
the panel query filtered on `is_in_library` (the payload even carried `isInLibrary`). In the
owner's library **28 of 52 gated versions were already downloaded** — more than half of the
"alerts" were about things already on disk. A version you own cannot become cheaper *for you*.
* **A filtered list in a notification surface.** "Price Alerts" was a state list whose contents were
entirely decided by a number stored in Settings, so an empty panel had three indistinguishable
causes (tracking off / threshold 0 / genuinely nothing), and it read as a broken feature.
Two measurements shaped the correction:
* **Permanent prices dominate** (87 permanent vs 3 timed gates in a 600-model sample), so "wait for a
price drop" is a low-frequency proposition, while "wait for early access to end" is high-frequency
and deterministic.
* **Prices cluster at the floor** (44 of the owner's 52 priced versions are 100 Buzz), so
"price <= N" carries little information; the real distinction is *free / soon free / paid*.
Also corrected during the discussion: the **generation fee** is the cost of generating *on the
CivitAI site*. A locally downloaded model generated in ComfyUI has no such fee, so for this tool it
is not a price axis at all (42 of the owner's 52 gated versions carry one — displaying it would
mislead on 80% of paid rows). It is not displayed today and must stay that way.
### 12.2 The corrected information model
The unit is the **version**, and the boundary is **ownership**: information exists only where a
decision exists.
| State of a version **not in the library** | The user's decision | What is shown |
| --- | --- | --- |
| Downloadable for free | download or not | nothing special (the download action already says it) |
| Early access, free on D, buyable for N | **wait until D / pay N now** | `N Buzz - free on D` |
| Early access, free on D, not buyable | wait | `free on D` |
| Permanently paid, price known (N) | buy or skip | `N Buzz` |
| Permanently paid, price unknown | buy or skip, price unknown | `Paid` (no number) |
| **In the library** | none | **nothing** |
The numeric threshold has no place in this table: every decision is categorical (wait / pay / skip),
not numeric.
### 12.3 Locked decisions
* **D1 - No independent surface.** Obtainability is an attribute of the existing update surfaces:
the model card's update badge, the grid's "models with updates" filter, the post-check toast, and
the model modal's version list. No new bell tab.
* **D2 - No numeric threshold anywhere.** The setting, the comparison, and the tests around it go.
* **D3 - Only versions that are not in the library** produce events or badges (`is_in_library = 0`),
enforced in both the event generator and any query.
* **D4 - Owned versions show no download price** (and carry no alert state).
* **D5 - The generation fee is never displayed.**
* **D6 - Models outside the library are out of scope.** Their decision moment is the download/browse
flow; that is a recorded follow-up, not a background feed (the update service only knows local
models, and there is no follow mechanism to build on).
* **D7 - The plumbing stays**: page fetch, host fallback, attempt markers, both TTLs, storage,
parser, `gate_lapsed_at`, and the price columns.
### 12.4 What happens to what shipped in P5a
| Piece | Fate |
| --- | --- |
| `GET /api/lm/price-alerts`, `GET /api/lm/{prefix}/updates/price-alerts`, `PriceAlertsHandler` | **removed** - no surface consumes them; events reach the UI through the refresh response |
| `get_price_alerts`, `newest_price_checked_at`, `count_priced_versions`, `count_unavailable_prices` | **removed** (threshold comparison and the panel's counters) |
| `price_alert_threshold_buzz` (settings, `_price_alert_threshold_buzz`) | **removed** |
| `price_alert_since` | **stop writing**; the column stays (SQLite drops need a table rebuild) and is marked deprecated |
| `price_alert_state` | **kept** - it is the one-shot edge for the "became free" toast |
| Bell tab, panel markup, `.price-alerts*` CSS, `UpdateService` panel code, watermark, the two entry points (controls dropdown, global context menu), their locales | **removed** |
| Version badges | **reworked** to the 12.2 table, ownership-scoped |
| "Price unavailable" badge | **replaced** by `Paid` (the gate is known from the public API; only the number is best-effort, and that is our plumbing, not the user's problem) |
| `priceAttemptedAt` in the version payload | **removed** from the payload (kept internally for the price-TTL retry policy) |
| The "became free" toast | **kept**, reworded, ownership-scoped |
### 12.5 Tasks
**P6a - the core correction (no new surface).**
Backend: ownership-scope the event generator; remove the threshold, the alert query, the counters,
the two routes and the handler; stop writing `price_alert_since`. Frontend: rework the version
badges to the 12.2 table; delete the panel, tab, entry points, watermark and their locales; reword
the toast. Tests: replace the panel/alert suites with a badge-state matrix and ownership-scoping
tests; assert the removed routes are gone.
**P6b - obtainability in the update surfaces.**
Per-model obtainability summary in the model list payload (next to `hasUpdate`) so the card badge can
say `Update - 500 Buzz` / `Update - free on Oct 15`; toast copy that counts the free ones
("3 models have updates, 1 is now free").
**P6c - deferred (recorded, not planned).** A price-drop event (rare by measurement); obtainability
in the download/browse flow for models outside the library (D6).
### 12.6 Verification
Unit tests for the state matrix and the ownership filter; a sandbox run against a **copy** of the
owner's real update DB, where the population must shrink from 52 gated versions to the **24 that are
not in the library**, with the owned ones showing nothing.
### 12.7 What shipped (P6a)
* The two alert-state columns are **gone from the schema** (not merely unused): the dataclass,
`_SCHEMA`, the additive migration table, the primary-key rebuild's column list and defaults, the
bulk `SELECT`, the row mapping and the `INSERT`. A database created by an unreleased build still
has them, so `_apply_migrations` drops them with a native `ALTER TABLE ... DROP COLUMN` (SQLite
3.35+, guarded) and logs it; for everyone else it is a no-op.
* `price_alert_threshold_buzz`, `_price_alert_threshold_buzz`, `_evaluate_price_alert`, the
threshold-crossing event, `get_price_alerts`, `newest_price_checked_at`,
`count_priced_versions`, `count_unavailable_prices`, both alert routes, `PriceAlertsHandler`, its
handler-set slot, its route mapping entry and the service-registry adapter field it needed: all
removed.
* Gate events (and only gate events) are emitted, and only for versions the user does not have -
`isInLibrary` is now the filter rather than a field in the payload.
* Version badges follow the 12.2 table: cost information only for versions not in the library, the
price replaces the redundant `Paid` badge when it is known, `Paid` appears without a number when
it is not, and `Price unavailable` is gone (the gate is certain; only the number is best-effort).
* The bell tab, panel markup, panel CSS, the controls-dropdown and global-context-menu entries, the
unread watermark and their locales are removed; the toast keeps its gate transitions and drops the
price-drop line. The setting is now framed as plumbing ("Show download prices for paid versions")
and keeps only the enable flag and the refresh interval.
Verified: `pytest` 3653 passed / 7 skipped, `npm run test:js` 1444 passed. Against a copy of the
owner's real database the population is 52 gated versions -> **28 owned (now silent) + 24 that the
feature is actually about**; the drop migration logged twice and left no `price_alert_*` column, and
both removed endpoints return 404.
### 12.8 Correction to P6b: the model card cannot carry a price
P6b proposed annotating the model card's update badge with `Update - 500 Buzz` /
`Update - free on Oct 15`. The owner objected that the badge represents a **model** already in the
library while the update is a set of versions, each with its own price and its own early access end
date. The objection is correct, and the data is blunt about it:
* Live model `958009` has **37 gated versions across 6 price points** (1111, 2220, 4440, 8880,
88800, 888888); model `153568` has two gated versions at 600 and 4000.
* In the owner's own library, model `647926` has two early access versions with **two different end
dates**.
So a single price or date on the card would be fabricated, not summarised. It looked plausible only
because that library is uniform (44 of its 52 priced versions sit at the 100 Buzz floor) - an
artefact of floor pricing, not a guarantee.
What is well defined at model level, by construction: counts ("3 updates"), the earliest upcoming
free date (a minimum over the set), a minimum price ("from 100 Buzz", well defined but misleading
when the cheap version is an old one), and uniformity statements only when they are true. "The price
is X" is not among them.
Revised scope: the card badge stays a plain update indicator, and **prices stay in the version list**,
where every row has exactly one subject. The only model-level fact worth considering is the deadline
- "1 of 3 updates becomes free on Oct 15" - because a deadline can change what the user does *before*
opening the model; it is rare in practice (4 unexpired early access versions across 3 models in that
library), so it is optional polish rather than part of the correction.
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "Prüfe auf {type}-Updates...",
"success": "{count} Update(s) für {type} gefunden",
"none": "Alle {type} sind auf dem neuesten Stand",
"error": "Fehler beim Prüfen auf {type}-Updates: {message}"
"error": "Fehler beim Prüfen auf {type}-Updates: {message}",
"gateEvents": {
"becameFree": "{count} Version(en), die Sie nicht besitzen, sind jetzt kostenlos",
"newGate": "{count} Version(en) sind jetzt kostenpflichtig"
}
},
"cleanupExampleImages": {
"label": "Beispielbild-Ordner bereinigen",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "Download-Pfad-Vorlagen",
"priorityTags": "Prioritäts-Tags",
"versionScope": "Update-Markierungen",
"priceTracking": "Buzz-Downloadpreise",
"exampleImages": "Beispielbilder",
"autoOrganize": "Auto-Organisierung",
"metadata": "Metadaten",
@@ -759,6 +764,13 @@
"label": "Bezahlte Updates ausblenden",
"help": "Wenn aktiviert, zeigen Modelle mit nur bezahlten Updates kein 'Update verfügbar'-Badge an"
},
"priceTracking": {
"help": "Die öffentliche API von CivitAI veröffentlicht keine Downloadpreise, daher werden sie von der Modellseite gelesen - eine zusätzliche Anfrage pro bezahltem Modell, aktualisiert im unten stehenden Intervall. Preise werden nur bei Versionen angezeigt, die Sie noch nicht haben.",
"enabled": "Downloadpreise für bezahlte Versionen anzeigen",
"enabledHelp": "Liest den Downloadpreis von der Modellseite, damit bezahlte Versionen und Versionen mit frühem Zugriff ihren Preis und das Ende des frühen Zugriffs zeigen",
"refreshHours": "Aktualisierungsintervall für Preise (Stunden)",
"refreshHoursHelp": "Wie lange ein gespeicherter Preis gültig bleibt, bevor er erneut gelesen wird"
},
"licenseIcons": {
"useNewStyle": "Aktualisierte Lizenzsymbole verwenden",
"useNewStyleHelp": "Lizenzberechtigungen mit farbigen Indikatoren (neuer Stil) oder nur Einschränkungssymbolen (klassischer Stil) anzeigen. Orientiert sich am aktuellen CivitAI-Design."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "Für diese Version ist derzeit CivitAI Early Access erforderlich",
"paid": "Bezahlt",
"paidTooltip": "Diese Version erfordert eine Zahlung zum Herunterladen",
"priceSaleTooltip": "Im Angebot, normalerweise {list} Buzz",
"priceBlueBuzzTooltip": "Kann auch mit Blue Buzz bezahlt werden",
"freeNow": "Jetzt kostenlos",
"freeNowTooltip": "Diese Version erfordert keine Zahlung mehr",
"ignored": "Ignoriert",
"ignoredTooltip": "Für diese Version sind Update-Benachrichtigungen deaktiviert",
"onSiteOnly": "Nur On-Site",
"onSiteOnlyTooltip": "Diese Version ist nur für die On-Site-Generierung auf CivitAI verfügbar"
"onSiteOnlyTooltip": "Diese Version ist nur für die On-Site-Generierung auf CivitAI verfügbar",
"earlyAccessTooltipUntil": "Früher Zugriff bis {date}, danach kostenlos zum Herunterladen"
},
"actions": {
"download": "Herunterladen",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "Checking for {type} updates...",
"success": "Found {count} update(s) for {type}s",
"none": "All {type}s are up to date",
"error": "Failed to check for {type} updates: {message}"
"error": "Failed to check for {type} updates: {message}",
"gateEvents": {
"becameFree": "{count} version(s) you do not have became free",
"newGate": "{count} version(s) are now paid"
}
},
"cleanupExampleImages": {
"label": "Clean up example image folders",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "Download Path Templates",
"priorityTags": "Priority Tags",
"versionScope": "Version Scope",
"priceTracking": "Buzz Download Prices",
"exampleImages": "Example Images",
"autoOrganize": "Auto-organize",
"metadata": "Metadata",
@@ -759,6 +764,13 @@
"label": "Hide Paid Updates",
"help": "When enabled, models with only paid updates will not show 'Update available' badge"
},
"priceTracking": {
"help": "CivitAI does not publish download prices in its public API, so they are read from the model page - one extra request per paid model, refreshed on the interval below. Prices are shown on versions you do not have yet.",
"enabled": "Show download prices for paid versions",
"enabledHelp": "Reads the download price from the model page so paid and early access versions show what they cost, and when early access ends",
"refreshHours": "Price refresh interval (hours)",
"refreshHoursHelp": "How long a stored price stays fresh before it is read again"
},
"licenseIcons": {
"useNewStyle": "Use updated license icons",
"useNewStyleHelp": "Display license permissions with colored indicators (new style) or restriction-only icons (classic style). Mirroring the current CivitAI design."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "This version currently requires CivitAI early access",
"paid": "Paid",
"paidTooltip": "This version requires payment to download",
"priceSaleTooltip": "On sale, normally {list} Buzz",
"priceBlueBuzzTooltip": "Can also be paid with Blue Buzz",
"freeNow": "Free Now",
"freeNowTooltip": "This version no longer requires payment",
"ignored": "Ignored",
"ignoredTooltip": "Update notifications are disabled for this version",
"onSiteOnly": "On-Site Only",
"onSiteOnlyTooltip": "This version is only available for on-site generation on CivitAI"
"onSiteOnlyTooltip": "This version is only available for on-site generation on CivitAI",
"earlyAccessTooltipUntil": "Early access until {date}, free to download after that"
},
"actions": {
"download": "Download",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "Buscando actualizaciones de {type}...",
"success": "Se encontraron {count} actualización(es) para {type}",
"none": "Todos los {type} están actualizados",
"error": "Error al buscar actualizaciones de {type}: {message}"
"error": "Error al buscar actualizaciones de {type}: {message}",
"gateEvents": {
"becameFree": "{count} versión(es) que no tienes se han vuelto gratuitas",
"newGate": "{count} versión(es) ahora son de pago"
}
},
"cleanupExampleImages": {
"label": "Limpiar carpetas de imágenes de ejemplo",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "Plantillas de rutas de descarga",
"priorityTags": "Etiquetas prioritarias",
"versionScope": "Indicadores de actualización",
"priceTracking": "Precios de descarga en Buzz",
"exampleImages": "Imágenes de ejemplo",
"autoOrganize": "Organización automática",
"metadata": "Metadatos",
@@ -759,6 +764,13 @@
"label": "Ocultar actualizaciones de pago",
"help": "Cuando está activado, los modelos que solo tienen actualizaciones de pago no mostrarán la insignia de 'Actualización disponible'"
},
"priceTracking": {
"help": "La API pública de CivitAI no publica los precios de descarga, así que se leen de la página del modelo: una petición adicional por modelo de pago, actualizada según el intervalo de abajo. Los precios se muestran solo en las versiones que aún no tienes.",
"enabled": "Mostrar los precios de descarga de las versiones de pago",
"enabledHelp": "Lee el precio de descarga de la página del modelo para mostrar cuánto cuestan las versiones de pago y de acceso temprano, y cuándo termina el acceso temprano",
"refreshHours": "Intervalo de actualización de precios (horas)",
"refreshHoursHelp": "Cuánto tiempo sigue vigente un precio guardado antes de volver a leerse"
},
"licenseIcons": {
"useNewStyle": "Usar iconos de licencia actualizados",
"useNewStyleHelp": "Mostrar permisos de licencia con indicadores de color (nuevo estilo) o solo iconos de restricción (estilo clásico). Refleja el diseño actual de CivitAI."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "Esta versión requiere actualmente acceso temprano de CivitAI",
"paid": "De pago",
"paidTooltip": "Esta versión requiere pago para descargarse",
"priceSaleTooltip": "En oferta, normalmente {list} Buzz",
"priceBlueBuzzTooltip": "También se puede pagar con Blue Buzz",
"freeNow": "Ahora gratis",
"freeNowTooltip": "Esta versión ya no requiere pago",
"ignored": "Ignorada",
"ignoredTooltip": "Las notificaciones de actualización están desactivadas para esta versión",
"onSiteOnly": "Solo en Sitio",
"onSiteOnlyTooltip": "Esta versión solo está disponible para generación en el sitio de CivitAI"
"onSiteOnlyTooltip": "Esta versión solo está disponible para generación en el sitio de CivitAI",
"earlyAccessTooltipUntil": "Acceso temprano hasta el {date}, descarga gratuita después"
},
"actions": {
"download": "Descargar",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "Recherche de mises à jour pour {type}...",
"success": "{count} mise(s) à jour trouvée(s) pour {type}",
"none": "Tous les {type} sont à jour",
"error": "Échec de la vérification des mises à jour pour {type} : {message}"
"error": "Échec de la vérification des mises à jour pour {type} : {message}",
"gateEvents": {
"becameFree": "{count} version(s) que vous ne possédez pas sont devenues gratuites",
"newGate": "{count} version(s) sont désormais payantes"
}
},
"cleanupExampleImages": {
"label": "Supprimer les dossiers d'exemples orphelins",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "Modèles de chemin de téléchargement",
"priorityTags": "Tags prioritaires",
"versionScope": "Indicateurs de mise à jour",
"priceTracking": "Prix de téléchargement en Buzz",
"exampleImages": "Images d'exemple",
"autoOrganize": "Organisation automatique",
"metadata": "Métadonnées",
@@ -759,6 +764,13 @@
"label": "Masquer les mises à jour payantes",
"help": "Lorsque cette option est activée, les modèles n'ayant que des mises à jour payantes n'affichent pas le badge « Mise à jour disponible »"
},
"priceTracking": {
"help": "L'API publique de CivitAI ne publie pas les prix de téléchargement : ils sont donc lus sur la page du modèle - une requête supplémentaire par modèle payant, actualisée selon l'intervalle ci-dessous. Les prix sont affichés sur les versions que vous ne possédez pas encore.",
"enabled": "Afficher les prix de téléchargement des versions payantes",
"enabledHelp": "Lit le prix de téléchargement sur la page du modèle pour indiquer le coût des versions payantes et en accès anticipé, ainsi que la fin de l'accès anticipé",
"refreshHours": "Intervalle d'actualisation des prix (heures)",
"refreshHoursHelp": "Durée pendant laquelle un prix enregistré reste valide avant d'être relu"
},
"licenseIcons": {
"useNewStyle": "Utiliser les icônes de licence mises à jour",
"useNewStyleHelp": "Afficher les permissions de licence avec des indicateurs colorés (nouveau style) ou des icônes de restriction uniquement (style classique). Reprend le design actuel de CivitAI."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "Cette version nécessite actuellement l'accès anticipé CivitAI",
"paid": "Payant",
"paidTooltip": "Cette version nécessite un paiement pour être téléchargée",
"priceSaleTooltip": "En promotion, normalement {list} Buzz",
"priceBlueBuzzTooltip": "Peut aussi être payé avec des Blue Buzz",
"freeNow": "Gratuit maintenant",
"freeNowTooltip": "Cette version ne nécessite plus de paiement",
"ignored": "Ignorée",
"ignoredTooltip": "Les notifications de mise à jour sont désactivées pour cette version",
"onSiteOnly": "Uniquement sur Site",
"onSiteOnlyTooltip": "Cette version n'est disponible que pour la génération sur le site CivitAI"
"onSiteOnlyTooltip": "Cette version n'est disponible que pour la génération sur le site CivitAI",
"earlyAccessTooltipUntil": "Accès anticipé jusqu'au {date}, téléchargement gratuit ensuite"
},
"actions": {
"download": "Télécharger",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "בודק עדכונים עבור {type}...",
"success": "נמצאו {count} עדכונים עבור {type}",
"none": "כל ה-{type} מעודכנים",
"error": "נכשל בבדיקת העדכונים עבור {type}: {message}"
"error": "נכשל בבדיקת העדכונים עבור {type}: {message}",
"gateEvents": {
"becameFree": "{count} גרסאות שאין לך הפכו לחינמיות",
"newGate": "{count} גרסאות דורשות כעת תשלום"
}
},
"cleanupExampleImages": {
"label": "נקה תיקיות תמונות דוגמה",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "תבניות נתיב הורדה",
"priorityTags": "תגיות עדיפות",
"versionScope": "תגי עדכון",
"priceTracking": "מחירי הורדה ב-Buzz",
"exampleImages": "תמונות דוגמה",
"autoOrganize": "ארגון אוטומטי",
"metadata": "מטא-נתונים",
@@ -759,6 +764,13 @@
"label": "הסתר עדכונים בתשלום",
"help": "כשאפשרות זו מופעלת, מודלים עם עדכונים בתשלום בלבד לא יציגו את תגית 'עדכון זמין'"
},
"priceTracking": {
"help": "ה-API הציבורי של CivitAI אינו מפרסם מחירי הורדה, ולכן הם נקראים מדף המודל - בקשה נוספת אחת לכל מודל בתשלום, ברענון לפי המרווח שלמטה. המחירים מוצגים רק בגרסאות שאין לך עדיין.",
"enabled": "הצג מחירי הורדה של גרסאות בתשלום",
"enabledHelp": "קורא את מחיר ההורדה מדף המודל כדי להציג כמה עולות גרסאות בתשלום וגרסאות בגישה מוקדמת, ומתי מסתיימת הגישה המוקדמת",
"refreshHours": "מרווח רענון המחירים (שעות)",
"refreshHoursHelp": "כמה זמן מחיר שמור נחשב מעודכן לפני קריאה חוזרת"
},
"licenseIcons": {
"useNewStyle": "השתמש בסמלי רישיון מעודכנים",
"useNewStyleHelp": "הצג הרשאות רישיון עם מחוונים צבעוניים (סגנון חדש) או סמלי הגבלה בלבד (סגנון קלאסי). משקף את העיצוב העדכני של CivitAI."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "גרסה זו דורשת כרגע גישת Early Access של CivitAI",
"paid": "בתשלום",
"paidTooltip": "גרסה זו דורשת תשלום כדי להוריד",
"priceSaleTooltip": "במבצע, בדרך כלל {list} Buzz",
"priceBlueBuzzTooltip": "ניתן לשלם גם ב-Blue Buzz",
"freeNow": "חינם כעת",
"freeNowTooltip": "גרסה זו אינה דורשת עוד תשלום",
"ignored": "התעלם",
"ignoredTooltip": "התראות העדכון מושבתות עבור גרסה זו",
"onSiteOnly": "רק באתר",
"onSiteOnlyTooltip": "גרסה זו זמינה רק ליצירה באתר CivitAI"
"onSiteOnlyTooltip": "גרסה זו זמינה רק ליצירה באתר CivitAI",
"earlyAccessTooltipUntil": "גישה מוקדמת עד {date}, לאחר מכן ההורדה חינם"
},
"actions": {
"download": "הורדה",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "{type} のアップデートを確認中...",
"success": "{type} のアップデートが {count} 件見つかりました",
"none": "すべての {type} は最新です",
"error": "{type} のアップデート確認に失敗しました: {message}"
"error": "{type} のアップデート確認に失敗しました: {message}",
"gateEvents": {
"becameFree": "未ダウンロードの {count} バージョンが無料になりました",
"newGate": "{count} バージョンが有料になりました"
}
},
"cleanupExampleImages": {
"label": "例画像フォルダをクリーンアップ",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "ダウンロードパステンプレート",
"priorityTags": "優先タグ",
"versionScope": "アップデートフラグ",
"priceTracking": "Buzz ダウンロード価格",
"exampleImages": "例画像",
"autoOrganize": "自動整理",
"metadata": "メタデータ",
@@ -759,6 +764,13 @@
"label": "有料更新を非表示",
"help": "有効にすると、有料の更新のみがあるモデルには「更新あり」バッジが表示されません"
},
"priceTracking": {
"help": "CivitAI の公開 API はダウンロード価格を提供していないため、モデルページから読み取ります。有料モデルごとに 1 回の追加リクエストが発生し、以下の間隔で更新されます。価格は未ダウンロードのバージョンにのみ表示されます。",
"enabled": "有料バージョンのダウンロード価格を表示",
"enabledHelp": "モデルページからダウンロード価格を読み取り、有料バージョンと早期アクセスバージョンの費用と早期アクセスの終了時期を表示します",
"refreshHours": "価格の更新間隔(時間)",
"refreshHoursHelp": "保存された価格が再読み取りまで有効な期間"
},
"licenseIcons": {
"useNewStyle": "更新されたライセンスアイコンを使用",
"useNewStyleHelp": "カラーインジケーター付きでライセンス許可を表示(新スタイル)するか、制限のみのアイコンを表示(クラシックスタイル)します。現在のCivitAIデザインを反映しています。"
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "このバージョンは現在 CivitAI の早期アクセスが必要です",
"paid": "有料",
"paidTooltip": "このバージョンのダウンロードには支払いが必要です",
"priceSaleTooltip": "セール中、通常 {list} Buzz",
"priceBlueBuzzTooltip": "Blue Buzz でも支払えます",
"freeNow": "無料化",
"freeNowTooltip": "このバージョンは支払いが不要になりました",
"ignored": "無視中",
"ignoredTooltip": "このバージョンの更新通知は無効です",
"onSiteOnly": "サイト内のみ",
"onSiteOnlyTooltip": "このバージョンはCivitAIサイト内でのみ利用可能で、ダウンロードはできません"
"onSiteOnlyTooltip": "このバージョンはCivitAIサイト内でのみ利用可能で、ダウンロードはできません",
"earlyAccessTooltipUntil": "{date} まで早期アクセス、以降は無料でダウンロードできます"
},
"actions": {
"download": "ダウンロード",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "{type} 업데이트를 확인 중...",
"success": "{type} 업데이트 {count}개를 찾았습니다",
"none": "모든 {type}가 최신 상태입니다",
"error": "{type} 업데이트 확인 실패: {message}"
"error": "{type} 업데이트 확인 실패: {message}",
"gateEvents": {
"becameFree": "보유하지 않은 버전 {count}개가 무료가 되었습니다",
"newGate": "버전 {count}개가 유료로 전환되었습니다"
}
},
"cleanupExampleImages": {
"label": "예시 이미지 폴더 정리",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "다운로드 경로 템플릿",
"priorityTags": "우선순위 태그",
"versionScope": "업데이트 표시",
"priceTracking": "Buzz 다운로드 가격",
"exampleImages": "예시 이미지",
"autoOrganize": "자동 정리",
"metadata": "메타데이터",
@@ -759,6 +764,13 @@
"label": "유료 업데이트 숨기기",
"help": "활성화하면 유료 업데이트만 있는 모델에 '업데이트 가능' 배지가 표시되지 않습니다"
},
"priceTracking": {
"help": "CivitAI 공개 API는 다운로드 가격을 제공하지 않으므로 모델 페이지에서 읽어옵니다. 유료 모델마다 요청이 한 번씩 추가되며, 아래 간격에 따라 갱신됩니다. 가격은 아직 보유하지 않은 버전에만 표시됩니다.",
"enabled": "유료 버전의 다운로드 가격 표시",
"enabledHelp": "모델 페이지에서 다운로드 가격을 읽어 유료 및 얼리 액세스 버전의 비용과 얼리 액세스 종료 시점을 표시합니다",
"refreshHours": "가격 갱신 간격(시간)",
"refreshHoursHelp": "저장된 가격을 다시 읽기 전까지 유지하는 기간"
},
"licenseIcons": {
"useNewStyle": "업데이트된 라이선스 아이콘 사용",
"useNewStyleHelp": "색상 표시기가 있는 라이선스 권한(새 스타일) 또는 제한 전용 아이콘(클래식 스타일)을 표시합니다. 현재 CivitAI 디자인을 반영합니다."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "이 버전은 현재 CivitAI 얼리 액세스가 필요합니다",
"paid": "유료",
"paidTooltip": "이 버전은 다운로드하려면 결제가 필요합니다",
"priceSaleTooltip": "할인 중, 원래 {list} Buzz",
"priceBlueBuzzTooltip": "Blue Buzz로도 결제할 수 있습니다",
"freeNow": "지금 무료",
"freeNowTooltip": "이 버전은 더 이상 결제가 필요하지 않습니다",
"ignored": "무시됨",
"ignoredTooltip": "이 버전은 업데이트 알림이 비활성화되어 있습니다",
"onSiteOnly": "사이트 내 전용",
"onSiteOnlyTooltip": "이 버전은 CivitAI 사이트 내에서만 사용 가능하며 다운로드할 수 없습니다"
"onSiteOnlyTooltip": "이 버전은 CivitAI 사이트 내에서만 사용 가능하며 다운로드할 수 없습니다",
"earlyAccessTooltipUntil": "{date}까지 얼리 액세스, 이후 무료로 다운로드할 수 있습니다"
},
"actions": {
"download": "다운로드",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "Проверка обновлений для {type}...",
"success": "Найдено {count} обновлений для {type}",
"none": "Все {type} актуальны",
"error": "Не удалось проверить обновления для {type}: {message}"
"error": "Не удалось проверить обновления для {type}: {message}",
"gateEvents": {
"becameFree": "{count} версий, которых у вас нет, стали бесплатными",
"newGate": "{count} версий стали платными"
}
},
"cleanupExampleImages": {
"label": "Очистить папки с примерами изображений",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "Шаблоны путей загрузки",
"priorityTags": "Приоритетные теги",
"versionScope": "Метки обновлений",
"priceTracking": "Цены за скачивание в Buzz",
"exampleImages": "Примеры изображений",
"autoOrganize": "Автоорганизация",
"metadata": "Метаданные",
@@ -759,6 +764,13 @@
"label": "Скрывать платные обновления",
"help": "Если включено, у моделей, для которых доступны только платные обновления, не будет отображаться значок «Доступно обновление»"
},
"priceTracking": {
"help": "Публичный API CivitAI не публикует цены за скачивание, поэтому они читаются со страницы модели — один дополнительный запрос на каждую платную модель, обновление с интервалом ниже. Цены показываются только для версий, которых у вас ещё нет.",
"enabled": "Показывать цены за скачивание платных версий",
"enabledHelp": "Читает цену за скачивание со страницы модели, чтобы платные версии и версии с ранним доступом показывали стоимость и дату окончания раннего доступа",
"refreshHours": "Интервал обновления цен (часы)",
"refreshHoursHelp": "Сколько хранится цена, прежде чем её прочитают снова"
},
"licenseIcons": {
"useNewStyle": "Использовать обновлённые значки лицензии",
"useNewStyleHelp": "Отображать разрешения лицензии с цветными индикаторами (новый стиль) или только значки ограничений (классический стиль). Соответствует текущему дизайну CivitAI."
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "Для этой версии сейчас требуется ранний доступ CivitAI",
"paid": "Платная",
"paidTooltip": "Скачивание этой версии платное",
"priceSaleTooltip": "Со скидкой, обычно {list} Buzz",
"priceBlueBuzzTooltip": "Можно также оплатить Blue Buzz",
"freeNow": "Теперь бесплатно",
"freeNowTooltip": "Эта версия больше не требует оплаты",
"ignored": "Игнорируется",
"ignoredTooltip": "Уведомления об обновлениях для этой версии отключены",
"onSiteOnly": "Только на Сайте",
"onSiteOnlyTooltip": "Эта версия доступна только для генерации на сайте CivitAI"
"onSiteOnlyTooltip": "Эта версия доступна только для генерации на сайте CivitAI",
"earlyAccessTooltipUntil": "Ранний доступ до {date}, после этого скачивание бесплатно"
},
"actions": {
"download": "Скачать",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "正在检查 {type} 更新...",
"success": "找到 {count} 条 {type} 更新",
"none": "所有 {type} 均已是最新版本",
"error": "检查 {type} 更新失败:{message}"
"error": "检查 {type} 更新失败:{message}",
"gateEvents": {
"becameFree": "{count} 个你未拥有的版本已免费",
"newGate": "{count} 个版本已变为付费"
}
},
"cleanupExampleImages": {
"label": "清理示例图片文件夹",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "下载路径模板",
"priorityTags": "优先标签",
"versionScope": "版本范围",
"priceTracking": "Buzz 下载价格",
"exampleImages": "示例图片",
"autoOrganize": "自动整理",
"metadata": "元数据",
@@ -759,6 +764,13 @@
"label": "隐藏付费更新",
"help": "启用后,仅有付费更新的模型将不显示“有可用更新”徽标"
},
"priceTracking": {
"help": "CivitAI 的公开 API 不提供下载价格,因此价格从模型页面读取 —— 每个付费模型多一次请求,按下面的间隔刷新。价格只显示在你尚未拥有的版本上。",
"enabled": "显示付费版本的下载价格",
"enabledHelp": "从模型页面读取下载价格,让付费与抢先体验版本显示所需费用,以及抢先体验的结束时间",
"refreshHours": "价格刷新间隔(小时)",
"refreshHoursHelp": "已保存的价格在重新读取前保持有效的时长"
},
"licenseIcons": {
"useNewStyle": "使用新版许可协议图标",
"useNewStyleHelp": "以彩色指示器显示许可权限(新样式),或仅显示限制图标(经典样式)。与当前 CivitAI 设计保持一致。"
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "此版本当前需要 CivitAI 抢先体验权限",
"paid": "付费",
"paidTooltip": "此版本需要付费后才能下载",
"priceSaleTooltip": "促销中,原价 {list} Buzz",
"priceBlueBuzzTooltip": "也可以使用 Blue Buzz 支付",
"freeNow": "现已免费",
"freeNowTooltip": "此版本已不再需要付费",
"ignored": "已忽略",
"ignoredTooltip": "此版本已关闭更新通知",
"onSiteOnly": "仅站内生成",
"onSiteOnlyTooltip": "此版本仅在 CivitAI 站内可用,无法下载"
"onSiteOnlyTooltip": "此版本仅在 CivitAI 站内可用,无法下载",
"earlyAccessTooltipUntil": "抢先体验至 {date},之后可免费下载"
},
"actions": {
"download": "下载",
+19 -2
View File
@@ -201,7 +201,11 @@
"loading": "正在檢查 {type} 更新...",
"success": "找到 {count} 個 {type} 更新",
"none": "所有 {type} 都是最新版本",
"error": "檢查 {type} 更新失敗:{message}"
"error": "檢查 {type} 更新失敗:{message}",
"gateEvents": {
"becameFree": "{count} 個您未擁有的版本已免費",
"newGate": "{count} 個版本已變為付費"
}
},
"cleanupExampleImages": {
"label": "清理範例圖片資料夾",
@@ -387,6 +391,7 @@
"downloadPathTemplates": "下載路徑範本",
"priorityTags": "優先標籤",
"versionScope": "版本範圍",
"priceTracking": "Buzz 下載價格",
"exampleImages": "範例圖片",
"autoOrganize": "自動整理",
"metadata": "中繼資料",
@@ -759,6 +764,13 @@
"label": "隱藏付費更新",
"help": "啟用後,只有付費更新的模型將不會顯示「有可用更新」徽章"
},
"priceTracking": {
"help": "CivitAI 的公開 API 不提供下載價格,因此價格從模型頁面讀取 —— 每個付費模型多一次請求,並依下方間隔重新整理。價格只顯示在您尚未擁有的版本上。",
"enabled": "顯示付費版本的下載價格",
"enabledHelp": "從模型頁面讀取下載價格,讓付費與搶先體驗版本顯示所需費用,以及搶先體驗的結束時間",
"refreshHours": "價格重新整理間隔(小時)",
"refreshHoursHelp": "已儲存的價格在重新讀取前保持有效的時長"
},
"licenseIcons": {
"useNewStyle": "使用新版許可協議圖標",
"useNewStyleHelp": "以彩色指示器顯示許可權限(新樣式),或僅顯示限制圖標(經典樣式)。與當前 CivitAI 設計保持一致。"
@@ -2016,10 +2028,15 @@
"earlyAccessTooltip": "此版本目前需要 CivitAI 搶先體驗權限",
"paid": "付費",
"paidTooltip": "此版本需要付費才能下載",
"priceSaleTooltip": "特價中,原價 {list} Buzz",
"priceBlueBuzzTooltip": "也可以使用 Blue Buzz 支付",
"freeNow": "現已免費",
"freeNowTooltip": "此版本已不再需要付費",
"ignored": "已忽略",
"ignoredTooltip": "此版本已關閉更新通知",
"onSiteOnly": "僅站內生成",
"onSiteOnlyTooltip": "此版本僅在 CivitAI 站內可用,無法下載"
"onSiteOnlyTooltip": "此版本僅在 CivitAI 站內可用,無法下載",
"earlyAccessTooltipUntil": "搶先體驗至 {date},之後可免費下載"
},
"actions": {
"download": "下載",
+1
View File
@@ -48,6 +48,7 @@ from ...services.cache_health_monitor import CacheHealthMonitor, CacheHealthStat
from ...services.use_cases.sidecar_migration_use_case import SidecarMigrationUseCase
from ...services.websocket_progress_callback import WebSocketBroadcastCallback
from ...utils.models import BaseModelMetadata
from ...utils.civitai_utils import build_civitai_model_page_url
from ...utils.constants import (
CIVITAI_USER_MODEL_TYPES,
DEFAULT_NODE_COLOR,
+31 -10
View File
@@ -50,6 +50,7 @@ from ...services.errors import RateLimitError, ResourceNotFoundError
from ...utils.civitai_utils import resolve_license_payload
from ...utils.file_utils import calculate_sha256
from ...utils.metadata_manager import MetadataManager
from ...utils.paid_access import is_early_access_deadline_active
from ...utils.sidecar_paths import get_metadata_path
from ...utils.url_utils import relative_root_prefix
@@ -2941,6 +2942,20 @@ class ModelUpdateHandler:
same_base_scope = self._uses_same_base_update_scope()
# Gate/price transitions are reported for every refreshed model, not only
# for the ones that qualify as updates: "this version became free" matters
# for a version the user already has, which never shows up as an update.
events = []
for record in records.values():
for event in getattr(record, "events", None) or []:
events.append(
{
"modelId": record.model_id,
"modelType": record.model_type,
**event,
}
)
serialized_records = []
for record in records.values():
has_update_fn = getattr(record, "has_update", None)
@@ -2962,6 +2977,7 @@ class ModelUpdateHandler:
{
"success": True,
"records": serialized_records,
"events": events,
}
)
@@ -3391,6 +3407,7 @@ class ModelUpdateHandler:
hide_early_access=hide_early_access,
hide_paid=hide_paid,
),
"events": list(getattr(record, "events", None) or []),
"versions": [
self._serialize_version(version, context.get(version.version_id))
for version in record.versions
@@ -3414,16 +3431,11 @@ class ModelUpdateHandler:
if getattr(version, "is_paid", False) and not version.early_access_ends_at:
is_early_access = False
elif version.early_access_ends_at:
try:
from datetime import datetime, timezone
ea_date = datetime.fromisoformat(
version.early_access_ends_at.replace("Z", "+00:00")
)
is_early_access = ea_date > datetime.now(timezone.utc)
except (ValueError, AttributeError):
# If date parsing fails, treat as active EA (conservative)
is_early_access = True
# Shared with the update service and the download gate so the badge,
# the update filter and the download warning cannot disagree.
is_early_access = is_early_access_deadline_active(
version.early_access_ends_at
)
elif getattr(version, "is_early_access", False):
# Fallback to basic EA flag from bulk API
is_early_access = True
@@ -3450,6 +3462,15 @@ class ModelUpdateHandler:
"usageControl": version.usage_control,
"isPaid": bool(getattr(version, "is_paid", False)),
"paidAccess": paid_access_payload,
# Set when a version that used to be gated became free, so the UI can
# keep showing "Free" long after the transition.
"gateLapsedAt": getattr(version, "gate_lapsed_at", None),
"priceBuzz": getattr(version, "price_buzz", None),
"listPriceBuzz": getattr(version, "list_price_buzz", None),
"generationPriceBuzz": getattr(version, "generation_price_buzz", None),
"acceptsBlueBuzz": bool(getattr(version, "accepts_blue_buzz", False)),
"priceSaleEndsAt": getattr(version, "price_sale_ends_at", None),
"priceCheckedAt": getattr(version, "price_checked_at", None),
"filePath": context.get("file_path"),
"fileName": context.get("file_name"),
# Weight-file variant count (None when unknown); lets the UI hide
+2
View File
@@ -22,6 +22,8 @@ class RouteDefinition:
MISC_ROUTE_DEFINITIONS: tuple[RouteDefinition, ...] = (
RouteDefinition("GET", "/api/lm/settings", "get_settings"),
RouteDefinition("POST", "/api/lm/settings", "update_settings"),
# App-wide and registered once: the alerts panel spans every model type, and
# the update DB is shared, so there is nothing per-type about it.
RouteDefinition("GET", "/api/lm/llm/models", "get_llm_models"),
RouteDefinition("GET", "/api/lm/llm/provider-models", "get_provider_models"),
RouteDefinition("GET", "/api/lm/doctor/diagnostics", "get_doctor_diagnostics"),
+160 -1
View File
@@ -20,7 +20,12 @@ from .model_metadata_provider import (
)
from .downloader import get_downloader
from .errors import RateLimitError, ResourceNotFoundError
from ..utils.civitai_utils import resolve_license_payload
from ..utils.civitai_utils import (
build_civitai_model_page_url,
civitai_page_host_candidates,
resolve_license_payload,
)
from ..utils.civitai_page_prices import parse_model_page_prices
from ..utils.constants import MODEL_WEIGHT_FILE_TYPES, is_empty_placeholder_hash
logger = logging.getLogger(__name__)
@@ -31,6 +36,24 @@ logger = logging.getLogger(__name__)
_CREATOR_COUNT_CACHE_TTL_SECONDS = 600
_creator_model_count_cache: Dict[str, Tuple[float, Optional[int]]] = {}
# How long a page host stays on the skip list after refusing a request outright
# (Cloudflare's challenge, surfaced as 403 "Access forbidden"). Long enough to
# cover a whole update refresh, short enough to recover within a session.
_PAGE_HOST_BLOCK_TTL = 15 * 60
def _is_host_level_refusal(message: str) -> bool:
"""Whether a failed request means "this host refuses us" rather than "this
model is unavailable".
``downloader.make_request`` collapses statuses into prose, and 403 ("Access
forbidden") is the one a Cloudflare challenge produces. A 404 ("Resource not
found") is model-specific — mature pages are hidden from anonymous visitors —
so it must not put the whole host on the skip list.
"""
return "forbidden" in message.lower()
class CivitaiClient:
_instance = None
@@ -66,6 +89,10 @@ class CivitaiClient:
str, Tuple[Optional[Dict[str, Any]], Optional[str]]
] = OrderedDict()
self._MAX_CACHE_ENTRIES = 500
# Model-page host bookkeeping: which host last worked, and which ones are
# currently refusing us (see get_model_prices).
self._page_host_preference: Optional[str] = None
self._page_host_blocked: Dict[str, float] = {}
def _build_image_info_url(self, image_id: str) -> str:
return f"{self.base_url}/images?imageId={image_id}&nsfw=X&withMeta=true"
@@ -383,6 +410,138 @@ class CivitaiClient:
logger.error(f"Error fetching model versions in bulk: {exc}")
return None
async def get_model_prices(
self, model_id: int
) -> Optional[Dict[int, Dict[str, Any]]]:
"""Fetch per-version buzz prices for one model from its public page.
CivitAI's public REST API deliberately omits prices, but the model page
embeds the site's own ``model.getById`` result (including
``paidAccess.terms``) in its server-rendered payload. One request covers
every version of the model. Returns ``{version_id: price fields}``, an
empty dict when the page loads but lists no gated version, or None when
the page could not be read or understood — callers keep any stored price.
Several hosts are tried in order, because the hosts are not equivalent:
* ``civitai.red`` serves mature model pages that ``civitai.com`` hides from
anonymous visitors, but it is behind a Cloudflare challenge that refuses
non-browser clients outright (403 for any User-Agent).
* ``civitai.com`` / ``civitai.green`` answer normally for anonymously
visible models, and 404 for the mature ones.
So the user's ``civitai_host`` preference is a starting point, not the only
option. Mature models whose page cannot be read from any host stay
priceless — see the known limitation in
``docs/plans/paid-model-price-tracking.md``.
This is a public anonymous page fetch: no API key and no internal
endpoint, so a failure here must never fail the update check itself.
"""
try:
normalized_id = int(model_id)
except (TypeError, ValueError):
return None
candidates = self._page_host_candidates()
failures: List[str] = []
for host in candidates:
url = build_civitai_model_page_url(normalized_id, host=host)
if not url:
continue
try:
success, result = await self._make_request(
"GET",
url,
use_auth=False,
custom_headers={"Accept": "text/html"},
)
except RateLimitError:
# The shared rate-limit gate already recorded it; skip this model.
raise
except Exception as exc: # pragma: no cover - defensive
failures.append(f"{host}: {exc}")
continue
if not success or not isinstance(result, str):
message = result if isinstance(result, str) else type(result).__name__
failures.append(f"{host}: {message}")
if isinstance(result, str) and _is_host_level_refusal(result):
# A refusal like Cloudflare's "Access forbidden" applies to the
# host, not to this model, so stop paying for it for a while.
self._block_page_host(host)
continue
prices = parse_model_page_prices(result)
if prices is None:
failures.append(f"{host}: no usable price payload")
continue
self._remember_page_host(host)
return prices
logger.warning(
"No price source for model %s; tried %s. Mature models are only served "
"by civitai.red, which challenges non-browser clients.",
model_id,
"; ".join(failures) or "no candidate hosts",
)
return None
def _page_host_candidates(self) -> List[str]:
"""Ordered hosts to try: the last one that worked, then the preference."""
preferred = self._page_host()
ordered = list(civitai_page_host_candidates(preferred))
if self._page_host_preference and self._page_host_preference in ordered:
ordered.remove(self._page_host_preference)
ordered.insert(0, self._page_host_preference)
now = time.time()
usable = [
host
for host in ordered
if now - self._page_host_blocked.get(host, 0.0) >= _PAGE_HOST_BLOCK_TTL
]
# Never return an empty list: a blocked host is still better than no attempt
# once the preference and the memo disagree.
return usable or ordered
def _remember_page_host(self, host: str) -> None:
if self._page_host_preference != host:
logger.info("CivitAI model pages are being read from %s", host)
self._page_host_preference = host
self._page_host_blocked.pop(host, None)
def _block_page_host(self, host: str) -> None:
if host in self._page_host_blocked:
return
# Log once per host per TTL: the failure is host-wide, so repeating it for
# every mature model in the library would be pure noise.
logger.warning(
"CivitAI model pages on %s refused the request (likely a Cloudflare "
"challenge); skipping that host for %d minutes",
host,
_PAGE_HOST_BLOCK_TTL // 60,
)
self._page_host_blocked[host] = time.time()
if self._page_host_preference == host:
self._page_host_preference = None
def _page_host(self) -> Optional[str]:
"""Resolve the page host from the ``civitai_host`` setting."""
try:
from .settings_manager import get_settings_manager
settings = get_settings_manager()
return settings.get("civitai_host") if settings else None
except Exception:
return None
async def get_model_version(
self, model_id: int | None = None, version_id: int | None = None
) -> Optional[Dict[str, Any]]:
+21 -22
View File
@@ -33,6 +33,13 @@ from ..utils.constants import (
VALID_OTHER_CIVITAI_TYPES,
)
from ..utils.civitai_utils import normalize_civitai_download_url, rewrite_preview_url
from ..utils.paid_access import (
is_early_access_deadline_active,
is_gate_active,
is_permanent_paid,
normalize_paid_access,
parse_civitai_timestamp,
)
from ..utils.file_utils import calculate_sha256, calculate_autov3
from ..utils.preview_selection import resolve_mature_threshold, select_preview_media
from ..utils.utils import calculate_filename_for_model, sanitize_folder_name
@@ -1985,40 +1992,32 @@ class DownloadManager:
os.makedirs(save_dir, exist_ok=True)
# Check if this is a paid or early access model
paid_access = version_info.get("paidAccess")
if isinstance(paid_access, str):
# Some providers (e.g. CivArchive fallback) carry the DTO as JSON text
try:
parsed = json.loads(paid_access)
paid_access = parsed if isinstance(parsed, dict) else None
except (TypeError, ValueError):
paid_access = None
if not isinstance(paid_access, dict):
paid_access = None
# An empty DTO ({"permanent": false, "endsAt": null}) is not a gate
if paid_access and not paid_access.get("permanent") and not paid_access.get("endsAt"):
paid_access = None
if version_info.get("earlyAccessEndsAt") or paid_access:
permanent_paid = bool(paid_access.get("permanent")) if paid_access else False
# CivitAI reports a non-null paidAccess only for an ACTIVE gate, so
# {"permanent": false, "endsAt": null} (a timed gate whose end is not
# recorded yet) still counts as gated here.
paid_access = normalize_paid_access(version_info.get("paidAccess"))
legacy_ea_ends_at = version_info.get("earlyAccessEndsAt")
gate_active = is_gate_active(paid_access) or is_early_access_deadline_active(
legacy_ea_ends_at
)
if gate_active:
permanent_paid = is_permanent_paid(paid_access)
if permanent_paid:
early_access_msg = (
"This model requires payment. Please ensure you have "
"purchased access and are logged in to Civitai."
)
else:
early_access_date = version_info.get("earlyAccessEndsAt")
early_access_date = legacy_ea_ends_at
if not early_access_date and paid_access:
early_access_date = paid_access.get("endsAt")
if not early_access_date:
early_access_date = ""
# Convert to a readable date if possible
try:
from datetime import datetime
date_obj = datetime.fromisoformat(
early_access_date.replace("Z", "+00:00")
)
formatted_date = date_obj.strftime("%Y-%m-%d")
formatted_date = parse_civitai_timestamp(
early_access_date
).strftime("%Y-%m-%d")
early_access_msg = (
f"This model requires payment (until {formatted_date}). "
)
+53
View File
@@ -183,6 +183,17 @@ class ModelMetadataProvider(ABC):
"""
return None
async def get_model_prices(
self, model_id: int
) -> Optional[Dict[int, Dict[str, Any]]]:
"""Fetch per-version buzz prices for a model, when the provider has them.
CivitAI publishes prices only inside its public model page payload;
providers that cannot read it (CivArchive, SQLite, OpenModelDB) keep the
default of None, which callers treat as "no price information".
"""
return None
class CivitaiModelMetadataProvider(ModelMetadataProvider):
"""Provider that uses Civitai API for metadata"""
@@ -222,6 +233,11 @@ class CivitaiModelMetadataProvider(ModelMetadataProvider):
) -> Optional[Dict[str, Any]]:
return await self.client.get_version_file_mini(version_id, file_id)
async def get_model_prices(
self, model_id: int
) -> Optional[Dict[int, Dict[str, Any]]]:
return await self.client.get_model_prices(model_id)
class CivArchiveModelMetadataProvider(ModelMetadataProvider):
"""Provider that uses CivArchive API for metadata"""
@@ -783,6 +799,34 @@ class FallbackMetadataProvider(ModelMetadataProvider):
def _iter_providers(self):
return zip(self.providers, self._provider_labels)
async def get_model_prices(
self, model_id: int
) -> Optional[Dict[int, Dict[str, Any]]]:
rate_limited = False
for provider, label in self._iter_providers():
if rate_limited and label not in _LOCAL_PROVIDER_LABELS:
continue
try:
result = await self._call_with_rate_limit(
label,
provider.get_model_prices,
model_id,
)
if result:
return result
except RateLimitError as exc:
rate_limited = True
logger.warning(
"Provider %s is rate-limited (retry_after=%.0fs); not failing over to other network providers",
label,
exc.retry_after or 0,
)
continue
except Exception as e:
logger.debug("Provider %s failed for get_model_prices: %s", label, e)
continue
return None
def excluding(self, labels: "frozenset[str] | set[str]") -> "FallbackMetadataProvider":
"""Return a copy of this chain without the providers named in *labels*.
@@ -907,6 +951,15 @@ class RateLimitRetryingProvider(ModelMetadataProvider):
file_id,
)
async def get_model_prices(
self, model_id: int
) -> Optional[Dict[int, Dict[str, Any]]]:
return await self._rate_limit_helper.run(
self._label,
self._provider.get_model_prices,
model_id,
)
class ModelMetadataProviderManager:
"""Manager for selecting and using model metadata providers"""
+449 -32
View File
@@ -11,7 +11,7 @@ import logging
import os
import sqlite3
import time
from dataclasses import dataclass, replace
from dataclasses import dataclass, field, replace
from datetime import datetime, timezone
from typing import Any, Dict, Iterable, Iterator, List, Mapping, Optional, Sequence
@@ -20,10 +20,25 @@ from .settings_manager import get_settings_manager
from ..utils.cache_paths import CacheType, resolve_cache_path_with_migration
from ..utils.constants import MODEL_WEIGHT_FILE_TYPES
from ..utils.civitai_utils import rewrite_preview_url
from ..utils.paid_access import (
is_early_access_deadline_active,
normalize_paid_access as _normalize_paid_access_payload,
)
from ..utils.preview_selection import resolve_mature_threshold, select_preview_media
logger = logging.getLogger(__name__)
# Version fields a price payload may set (see py/utils/civitai_page_prices.py).
_PRICE_FIELD_NAMES = frozenset(
{
"price_buzz",
"list_price_buzz",
"generation_price_buzz",
"accepts_blue_buzz",
"price_sale_ends_at",
}
)
def _normalize_int(value) -> Optional[int]:
"""Safely convert a value to an integer."""
@@ -36,6 +51,16 @@ def _normalize_int(value) -> Optional[int]:
return None
def _format_utc_timestamp(value: float) -> str:
"""Format a POSIX timestamp the way CivitAI serializes its timestamps."""
return (
datetime.fromtimestamp(value, tz=timezone.utc)
.isoformat()
.replace("+00:00", "Z")
)
def _normalize_string(value) -> Optional[str]:
"""Return a stripped string or None if the value is empty."""
@@ -82,6 +107,24 @@ class ModelVersionRecord:
# e.g. records persisted before this field existed or locally-synthesized
# entries). Mirrors the frontend isModelWeightFile() filter.
file_count: Optional[int] = None
# When a version that used to carry a paid gate stopped carrying one. Kept so
# the UI can mark a version "free" long after the transition happened; cleared
# again if the version is gated once more.
gate_lapsed_at: Optional[str] = None
# Download price in Buzz for the gated version, and friends. All None until a
# price refresh succeeds (see the price tracking feature); `price_buzz` is the
# effective price a buyer pays now (sale-adjusted) and `list_price_buzz` the
# undiscounted stored price.
price_buzz: Optional[int] = None
list_price_buzz: Optional[int] = None
generation_price_buzz: Optional[int] = None
accepts_blue_buzz: bool = False
price_sale_ends_at: Optional[str] = None
price_checked_at: Optional[float] = None
# When a price fetch was last *attempted* (success or failure). Distinguishes
# "never tried" from "tried and no price is readable", which is what lets the
# UI say "price unavailable" for mature models instead of showing nothing.
price_check_attempted_at: Optional[float] = None
@dataclass
@@ -93,6 +136,12 @@ class ModelUpdateRecord:
versions: List[ModelVersionRecord]
last_checked_at: Optional[float]
should_ignore_model: bool
# Gate-state and price transitions observed during the refresh that produced
# this record. Derived, never persisted: a record read back from SQLite has an
# empty list. Each entry is {"versionId", "kind", "versionName", "isInLibrary"}
# with kind in {"new_gate", "became_free"}. Only versions the user does not
# have produce events.
events: List[Dict[str, Any]] = field(default_factory=list)
@property
def largest_version_id(self) -> Optional[int]:
@@ -172,16 +221,10 @@ class ModelUpdateRecord:
if version.is_paid and not version.early_access_ends_at:
return False
# Phase 2: Precise check with exact end time
# Phase 2: Precise check with exact end time (None when the gate is timed
# but its window end has not been recorded yet -> treated as active below)
if version.early_access_ends_at:
try:
ea_date = datetime.fromisoformat(
version.early_access_ends_at.replace("Z", "+00:00")
)
return ea_date > datetime.now(timezone.utc)
except (ValueError, AttributeError):
# If date parsing fails, treat as active EA (conservative)
return True
return is_early_access_deadline_active(version.early_access_ends_at)
# Phase 1: Basic EA flag from bulk API
return version.is_early_access
@@ -324,6 +367,14 @@ class ModelUpdateService:
paid_access TEXT,
is_paid INTEGER NOT NULL DEFAULT 0,
file_count INTEGER,
gate_lapsed_at TEXT,
price_buzz INTEGER,
list_price_buzz INTEGER,
generation_price_buzz INTEGER,
accepts_blue_buzz INTEGER NOT NULL DEFAULT 0,
price_sale_ends_at TEXT,
price_checked_at REAL,
price_check_attempted_at REAL,
PRIMARY KEY (model_id, version_id),
FOREIGN KEY(model_id) REFERENCES model_update_status(model_id) ON DELETE CASCADE
);
@@ -575,12 +626,59 @@ class ModelUpdateService:
"ALTER TABLE model_update_versions "
"ADD COLUMN file_count INTEGER"
),
"gate_lapsed_at": (
"ALTER TABLE model_update_versions "
"ADD COLUMN gate_lapsed_at TEXT"
),
"price_buzz": (
"ALTER TABLE model_update_versions "
"ADD COLUMN price_buzz INTEGER"
),
"list_price_buzz": (
"ALTER TABLE model_update_versions "
"ADD COLUMN list_price_buzz INTEGER"
),
"generation_price_buzz": (
"ALTER TABLE model_update_versions "
"ADD COLUMN generation_price_buzz INTEGER"
),
"accepts_blue_buzz": (
"ALTER TABLE model_update_versions "
"ADD COLUMN accepts_blue_buzz INTEGER NOT NULL DEFAULT 0"
),
"price_sale_ends_at": (
"ALTER TABLE model_update_versions "
"ADD COLUMN price_sale_ends_at TEXT"
),
"price_checked_at": (
"ALTER TABLE model_update_versions "
"ADD COLUMN price_checked_at REAL"
),
"price_check_attempted_at": (
"ALTER TABLE model_update_versions "
"ADD COLUMN price_check_attempted_at REAL"
),
}
for column, statement in migrations.items():
if column not in version_columns:
conn.execute(statement)
# Columns this feature carried while it was still in development. They are
# gone from the schema, so only a database created by an unreleased build
# can still have them; for everyone else this is a no-op. `DROP COLUMN`
# needs SQLite 3.35+, hence the guard.
for obsolete in ("price_alert_state", "price_alert_since"):
if obsolete not in version_columns:
continue
try:
conn.execute(
f"ALTER TABLE model_update_versions DROP COLUMN {obsolete}"
)
logger.info("Dropped obsolete column %s", obsolete)
except sqlite3.OperationalError as exc: # pragma: no cover - old SQLite
logger.debug("Could not drop obsolete column %s: %s", obsolete, exc)
# Refresh column metadata after applying additive migrations.
version_columns = self._get_table_columns(conn, "model_update_versions")
@@ -679,6 +777,14 @@ class ModelUpdateService:
paid_access TEXT,
is_paid INTEGER NOT NULL DEFAULT 0,
file_count INTEGER,
gate_lapsed_at TEXT,
price_buzz INTEGER,
list_price_buzz INTEGER,
generation_price_buzz INTEGER,
accepts_blue_buzz INTEGER NOT NULL DEFAULT 0,
price_sale_ends_at TEXT,
price_checked_at REAL,
price_check_attempted_at REAL,
PRIMARY KEY (model_id, version_id),
FOREIGN KEY(model_id) REFERENCES model_update_status(model_id) ON DELETE CASCADE
)
@@ -701,6 +807,14 @@ class ModelUpdateService:
"paid_access",
"is_paid",
"file_count",
"gate_lapsed_at",
"price_buzz",
"list_price_buzz",
"generation_price_buzz",
"accepts_blue_buzz",
"price_sale_ends_at",
"price_checked_at",
"price_check_attempted_at",
]
defaults = {
"sort_index": "0",
@@ -716,6 +830,14 @@ class ModelUpdateService:
"paid_access": "NULL",
"is_paid": "0",
"file_count": "NULL",
"gate_lapsed_at": "NULL",
"price_buzz": "NULL",
"list_price_buzz": "NULL",
"generation_price_buzz": "NULL",
"accepts_blue_buzz": "0",
"price_sale_ends_at": "NULL",
"price_checked_at": "NULL",
"price_check_attempted_at": "NULL",
}
select_parts = []
@@ -1170,6 +1292,37 @@ class ModelUpdateService:
elif fallback_attempted and fallback_error_message is None:
fallback_error_message = "no response"
# Optional price capture. Runs after the version list is known so only the
# models that actually carry a gate pay for a second (page) request, and
# stays outside the lock along with the other network I/O.
#
# Deliberately independent of the metadata TTL: the cached record already
# carries the gate, so enabling price tracking (or letting the *price* TTL
# lapse) must not wait for the version list to go stale. Otherwise a user
# who turns the feature on prices only the handful of models that happened
# to need a metadata refresh that round.
priced_versions: Optional[List[ModelVersionRecord]] = None
price_candidates: Optional[Sequence[ModelVersionRecord]] = None
if refresh_succeeded and isinstance(fetched_versions, list) and fetched_versions:
price_candidates = fetched_versions
elif existing is not None and existing.versions:
price_candidates = existing.versions
if (
price_candidates
and metadata_provider is not None
and self._should_fetch_prices(
price_candidates, existing, force=force_refresh
)
):
price_candidates = await self._apply_model_prices(
metadata_provider, model_id, price_candidates
)
if refresh_succeeded and isinstance(fetched_versions, list):
fetched_versions = list(price_candidates)
else:
priced_versions = list(price_candidates)
if fallback_attempted:
if refresh_succeeded and isinstance(fetched_versions, list):
logger.info(
@@ -1234,6 +1387,11 @@ class ModelUpdateService:
local_base_models=local_base_models,
)
else:
if priced_versions is not None and existing is not None:
# Metadata came from the cache, but the prices did not: keep
# them without touching last_checked_at, so the metadata TTL is
# not silently extended by a price-only pass.
existing = replace(existing, versions=priced_versions)
record = self._merge_with_local_versions(
existing,
normalized_local,
@@ -1633,10 +1791,55 @@ class ModelUpdateService:
existing_map = {version.version_id: version for version in existing.versions} if existing else {}
versions: List[ModelVersionRecord] = []
events: List[Dict[str, Any]] = []
seen_ids: set[int] = set()
lapse_timestamp = _format_utc_timestamp(timestamp)
for index, remote_version in enumerate(remote_versions):
version_id = remote_version.version_id
seen_ids.add(version_id)
existing_version = existing_map.get(version_id)
is_gated = self._has_structural_gate(remote_version)
was_gated = (
self._has_structural_gate(existing_version)
if existing_version is not None
else None
)
should_ignore = ignore_map.get(version_id, remote_version.should_ignore)
# `gate_lapsed_at` survives refreshes so a version that became free keeps
# its "free since" marker, and is cleared if the gate comes back.
if is_gated:
gate_lapsed_at = None
elif was_gated:
gate_lapsed_at = lapse_timestamp
elif existing_version is not None:
gate_lapsed_at = existing_version.gate_lapsed_at
else:
gate_lapsed_at = None
price_fields = self._price_fields_for(
remote_version, existing_version, is_gated=is_gated
)
# Only the edge fires, and only for a version the user does not have:
# a version already on disk cannot become cheaper *for them*, and a
# version that stays gated must not re-announce itself every refresh.
in_library = version_id in effective_local_set
if (
existing_version is not None
and not should_ignore
and not in_library
and was_gated != is_gated
):
events.append(
{
"versionId": version_id,
"kind": "new_gate" if is_gated else "became_free",
"versionName": remote_version.name,
"isInLibrary": False,
}
)
versions.append(
ModelVersionRecord(
version_id=version_id,
@@ -1646,7 +1849,7 @@ class ModelUpdateService:
size_bytes=remote_version.size_bytes,
preview_url=remote_version.preview_url or preview_map.get(version_id),
is_in_library=version_id in effective_local_set,
should_ignore=ignore_map.get(version_id, remote_version.should_ignore),
should_ignore=should_ignore,
sort_index=sort_map.get(version_id, index),
early_access_ends_at=remote_version.early_access_ends_at,
is_early_access=remote_version.is_early_access,
@@ -1658,6 +1861,8 @@ class ModelUpdateService:
if remote_version.file_count is not None
else file_count_map.get(version_id)
),
gate_lapsed_at=gate_lapsed_at,
**price_fields,
)
)
@@ -1696,8 +1901,199 @@ class ModelUpdateService:
versions=self._sorted_versions(versions),
last_checked_at=timestamp,
should_ignore_model=existing.should_ignore_model if existing else False,
events=events,
)
@staticmethod
def _has_structural_gate(version: Optional[ModelVersionRecord]) -> bool:
"""True when a version carries a paid gate, ignoring the clock.
Transition detection uses this rather than a time-based check so a gate
that lapsed since the previous refresh still reads as "was gated" — the
public API reports no ``paidAccess`` at all once a gate expires, so the
time-based helper alone would silently see "free before, free now".
"""
if version is None:
return False
return bool(version.paid_access) or version.is_paid or version.is_early_access
@staticmethod
def _price_fields_for(
remote_version: ModelVersionRecord,
existing_version: Optional[ModelVersionRecord],
*,
is_gated: bool,
) -> Dict[str, Any]:
"""Resolve the price columns for a refreshed version.
A refresh only carries prices when a price fetch actually ran (recognizable
by ``price_checked_at``), so otherwise the previously stored price is kept —
but only while the version is still gated: once it is free the stored
numbers are stale and must not keep firing alerts.
"""
if not is_gated:
return {
"price_buzz": None,
"list_price_buzz": None,
"generation_price_buzz": None,
"accepts_blue_buzz": False,
"price_sale_ends_at": None,
"price_checked_at": None,
"price_check_attempted_at": None,
}
if remote_version.price_checked_at is not None:
source = remote_version
else:
source = existing_version or remote_version
return {
"price_buzz": source.price_buzz,
"list_price_buzz": source.list_price_buzz,
"generation_price_buzz": source.generation_price_buzz,
"accepts_blue_buzz": source.accepts_blue_buzz,
"price_sale_ends_at": source.price_sale_ends_at,
"price_checked_at": source.price_checked_at,
"price_check_attempted_at": source.price_check_attempted_at,
}
def _price_tracking_enabled(self) -> bool:
"""Whether the optional (page-fetching) price capture is switched on."""
try:
return bool(self._settings.get("price_tracking_enabled", False))
except Exception:
return False
def _price_check_ttl_seconds(self) -> float:
try:
hours = float(self._settings.get("price_check_ttl_hours", 24))
except (TypeError, ValueError):
hours = 24.0
if hours <= 0:
hours = 24.0
return hours * 3600.0
def _should_fetch_prices(
self,
remote_versions: Sequence[ModelVersionRecord],
existing: Optional[ModelUpdateRecord],
*,
force: bool = False,
) -> bool:
"""Whether this model needs a price fetch this round.
Only gated versions are considered, so an unmodified free library pays
nothing. A stored price is refreshed once its TTL lapses, and immediately
when the gate itself changed (a new end date or sale window is a reason to
believe the price moved).
A *failed* attempt counts as an attempt: without that, a mature model whose
page no host will serve would be retried on every single update check.
``force`` (the user asked explicitly) overrides the TTL.
"""
if not self._price_tracking_enabled():
return False
if force:
return any(
self._has_structural_gate(version) for version in remote_versions
)
existing_map = (
{version.version_id: version for version in existing.versions}
if existing
else {}
)
now = time.time()
ttl = self._price_check_ttl_seconds()
for remote_version in remote_versions:
if not self._has_structural_gate(remote_version):
continue
stored = existing_map.get(remote_version.version_id)
if stored is None:
return True
anchor = stored.price_checked_at or stored.price_check_attempted_at
if anchor is None:
return True
if (now - anchor) >= ttl:
return True
if (stored.paid_access or None) != (remote_version.paid_access or None):
return True
return False
async def _apply_model_prices(
self,
metadata_provider,
model_id: int,
versions: Sequence[ModelVersionRecord],
) -> List[ModelVersionRecord]:
"""Attach freshly fetched prices to the versions that have them.
Never raises for a provider problem: price tracking is a convenience, and
an unreadable page (or a provider that has no prices at all) must leave
the update check exactly as it was.
A failed attempt is still recorded (``price_check_attempted_at``) so the UI
can distinguish "we could not read a price" from "we never looked" — that
is the honest state for mature models, whose pages are served only by
civitai.red, which refuses non-browser clients.
"""
getter = getattr(metadata_provider, "get_model_prices", None)
if not callable(getter):
return list(versions)
attempted_at = time.time()
try:
prices = await getter(model_id)
except RateLimitError:
raise
except Exception as exc: # pragma: no cover - defensive
logger.debug("Price fetch failed for model %s: %s", model_id, exc)
prices = None
if not isinstance(prices, Mapping):
prices = {}
enriched: List[ModelVersionRecord] = []
for version in versions:
if not self._has_structural_gate(version):
enriched.append(version)
continue
fields = prices.get(version.version_id)
recognized = (
{
key: value
for key, value in fields.items()
if key in _PRICE_FIELD_NAMES
}
if isinstance(fields, Mapping)
else {}
)
if not recognized:
# Keep the stored price (there may be none) but remember the try, so
# the UI can say "unavailable" instead of showing nothing at all.
enriched.append(
replace(version, price_check_attempted_at=attempted_at)
)
continue
enriched.append(
replace(
version,
price_checked_at=attempted_at,
price_check_attempted_at=attempted_at,
**recognized,
)
)
return enriched
def _sorted_versions(self, versions: Sequence[ModelVersionRecord]) -> List[ModelVersionRecord]:
ordered = sorted(versions, key=lambda version: (version.sort_index, version.version_id))
normalized: List[ModelVersionRecord] = []
@@ -1762,6 +2158,9 @@ class ModelUpdateService:
# CivitAI's paidAccess DTO ({"permanent": bool, "endsAt": ISO|null})
# gates versions behind a paid tier while availability stays "Public".
# A non-null DTO from the public API is always an ACTIVE gate: lapsed
# (tombstone) gates come back as null. That includes the timed gate whose
# end is not recorded yet, {"permanent": false, "endsAt": null}.
paid_access = self._normalize_paid_access(entry.get("paidAccess"))
paid_access_json = json.dumps(paid_access) if paid_access else None
is_paid = bool(paid_access.get("permanent")) if paid_access else False
@@ -1769,7 +2168,7 @@ class ModelUpdateService:
early_access_ends_at = _normalize_string(paid_access.get("endsAt"))
# Only timed gates are early access; permanent paid versions are not
# (consumers filter them via is_paid), so the stored flag stays accurate.
if not is_early_access and paid_access and paid_access.get("endsAt"):
if not is_early_access and paid_access and not paid_access.get("permanent"):
is_early_access = True
return ModelVersionRecord(
@@ -1797,24 +2196,12 @@ class ModelUpdateService:
Accepts a dict, None, or a JSON string (as carried by the by-hash
enrichment path) and returns ``{"permanent": bool, "endsAt": str|None}``
or None when the input carries no paid-access signal.
Delegates to :mod:`py.utils.paid_access` so the update service and the
download gate cannot disagree about what counts as a gate.
"""
if value is None:
return None
if isinstance(value, str):
try:
parsed = json.loads(value)
except (TypeError, ValueError):
return None
if not isinstance(parsed, dict):
return None
value = parsed
if not isinstance(value, Mapping):
return None
permanent = bool(value.get("permanent"))
ends_at = _normalize_string(value.get("endsAt"))
if not permanent and ends_at is None:
return None
return {"permanent": permanent, "endsAt": ends_at}
return _normalize_paid_access_payload(value)
@staticmethod
def _extract_file_count(files) -> Optional[int]:
@@ -1947,7 +2334,10 @@ class ModelUpdateService:
f"""
SELECT model_id, version_id, sort_index, name, base_model, released_at,
size_bytes, preview_url, is_in_library, should_ignore, early_access_ends_at,
is_early_access, usage_control, paid_access, is_paid, file_count
is_early_access, usage_control, paid_access, is_paid, file_count,
gate_lapsed_at, price_buzz, list_price_buzz, generation_price_buzz,
accepts_blue_buzz, price_sale_ends_at, price_checked_at,
price_check_attempted_at
FROM model_update_versions
WHERE model_id IN ({placeholders})
ORDER BY model_id ASC, sort_index ASC, version_id ASC
@@ -1979,6 +2369,22 @@ class ModelUpdateService:
paid_access=row["paid_access"],
is_paid=bool(row["is_paid"]),
file_count=_normalize_int(row["file_count"]),
gate_lapsed_at=row["gate_lapsed_at"],
price_buzz=_normalize_int(row["price_buzz"]),
list_price_buzz=_normalize_int(row["list_price_buzz"]),
generation_price_buzz=_normalize_int(row["generation_price_buzz"]),
accepts_blue_buzz=bool(row["accepts_blue_buzz"]),
price_sale_ends_at=row["price_sale_ends_at"],
price_checked_at=(
float(row["price_checked_at"])
if row["price_checked_at"] is not None
else None
),
price_check_attempted_at=(
float(row["price_check_attempted_at"])
if row["price_check_attempted_at"] is not None
else None
),
)
)
@@ -2041,8 +2447,11 @@ class ModelUpdateService:
INSERT INTO model_update_versions (
version_id, model_id, sort_index, name, base_model, released_at,
size_bytes, preview_url, is_in_library, should_ignore, early_access_ends_at,
is_early_access, usage_control, paid_access, is_paid, file_count
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
is_early_access, usage_control, paid_access, is_paid, file_count,
gate_lapsed_at, price_buzz, list_price_buzz, generation_price_buzz,
accepts_blue_buzz, price_sale_ends_at, price_checked_at,
price_check_attempted_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(
version.version_id,
@@ -2061,6 +2470,14 @@ class ModelUpdateService:
paid_access_value,
1 if version.is_paid else 0,
version.file_count,
version.gate_lapsed_at,
version.price_buzz,
version.list_price_buzz,
version.generation_price_buzz,
1 if version.accepts_blue_buzz else 0,
version.price_sale_ends_at,
version.price_checked_at,
version.price_check_attempted_at,
),
)
conn.commit()
+6
View File
@@ -130,6 +130,12 @@ DEFAULT_SETTINGS: Dict[str, Any] = {
"model_card_footer_action": "replace_preview",
"show_version_on_card": True,
"version_grouping": "same_base",
# Buzz price tracking for paid/early-access versions. Opt-in because reading a
# price costs one extra (public) model-page request per gated model.
# Plumbing switch: reading a price costs one extra request per paid model, so
# it stays opt-in. Prices decorate the version list; nothing alerts on a number.
"price_tracking_enabled": False,
"price_check_ttl_hours": 24,
"auto_organize_exclusions": [],
"metadata_refresh_skip_paths": [],
"skip_previously_downloaded_model_versions": False,
+257
View File
@@ -0,0 +1,257 @@
"""Extract buzz prices from a public CivitAI model page.
CivitAI's public REST API deliberately omits prices: ``paidAccess`` is trimmed to
``{permanent, endsAt}`` because "pricing belongs to the purchase flow" (see the
upstream ``toPublicPaidAccessDto``). The model *page*, however, ships the site's
own ``model.getById`` result inside its server-rendered Next.js payload, and that
payload carries the full ``paidAccess.terms`` — download price, generation price,
sale and the Blue Buzz flag.
Reading it is a plain anonymous page fetch: no API key, no internal endpoint, no
forged ``Origin``. It is still page data rather than a contract, so nothing in
this module may raise: an unrecognized shape degrades to ``None`` and the caller
keeps whatever price it already stored.
Shape this understands (verified against civitai.com and civitai.red)::
<script id="__NEXT_DATA__" type="application/json">
{"props": {"pageProps": {"trpcState": {"json": {"queries": [
{"queryKey": [["model", "getById"], {...}],
"state": {"data": {"modelVersions": [
{"id": 3379626, "paidAccess": {
"endsAt": null, "timeframeDays": null,
"terms": {"download": {"price": 5000},
"generation": {"price": 100, "trialLimit": 5}},
"sale": null}}
]}}}
]}}}}}
</script>
"""
from __future__ import annotations
import json
import logging
import re
from typing import Any, Dict, Iterator, Mapping, Optional
from .paid_access import is_gate_active, normalize_paid_access
logger = logging.getLogger(__name__)
__all__ = ["parse_model_page_prices", "MAX_PAGE_BYTES"]
# Bound the payload before parsing: a model page is a few hundred KB, so anything
# far larger is not a page we want to hold in memory.
MAX_PAGE_BYTES = 8 * 1024 * 1024
_NEXT_DATA_RE = re.compile(
r'<script[^>]*id="__NEXT_DATA__"[^>]*>(.*?)</script>',
re.DOTALL,
)
# The procedure that carries per-version pricing on a model page.
_MODEL_QUERY_PATH = ("model", "getById")
def parse_model_page_prices(html: Any) -> Optional[Dict[int, Dict[str, Any]]]:
"""Return ``{version_id: price fields}`` for a model page, or None.
Only versions with an *active* gate appear: the page payload also carries
lapsed gates as tombstones (a version with ``endsAt`` in the past is freely
downloadable), and those must not be reported as priced.
Returns None when the page carries no usable payload at all (missing script
tag, malformed JSON, unexpected structure, challenge page).
"""
if not isinstance(html, str) or not html:
return None
if len(html) > MAX_PAGE_BYTES:
logger.debug("CivitAI model page too large to parse (%d bytes)", len(html))
return None
match = _NEXT_DATA_RE.search(html)
if match is None:
return None
try:
payload = json.loads(match.group(1))
except (TypeError, ValueError):
return None
versions = _find_model_versions(payload)
if versions is None:
return None
prices: Dict[int, Dict[str, Any]] = {}
for entry in versions:
if not isinstance(entry, Mapping):
continue
version_id = _coerce_int(entry.get("id"))
if version_id is None:
continue
price_fields = _price_fields(entry.get("paidAccess"))
if price_fields is not None:
prices[version_id] = price_fields
return prices
def _find_model_versions(payload: Any) -> Optional[list]:
"""Locate ``modelVersions`` inside the dehydrated tRPC state.
The query order is not stable (several procedures are dehydrated per page), so
the query is selected by key rather than position.
"""
if not isinstance(payload, Mapping):
return None
for state in _iter_trpc_states(payload):
queries = state.get("queries")
if not isinstance(queries, list):
continue
for query in queries:
if not isinstance(query, Mapping):
continue
if not _matches_model_query(query.get("queryKey")):
continue
data = (query.get("state") or {}).get("data")
if isinstance(data, Mapping):
versions = data.get("modelVersions")
if isinstance(versions, list):
return versions
return None
def _iter_trpc_states(payload: Mapping[str, Any]) -> Iterator[Mapping[str, Any]]:
"""Yield the dehydrated tRPC state objects found under ``props.pageProps``."""
props = payload.get("props")
if not isinstance(props, Mapping):
return
page_props = props.get("pageProps")
if not isinstance(page_props, Mapping):
return
candidates = [page_props.get("trpcState")]
# Older/other Next.js builds keep the state one level deeper.
dehydrated = page_props.get("dehydratedState")
if dehydrated is not None:
candidates.append(dehydrated)
for candidate in candidates:
if not isinstance(candidate, Mapping):
continue
for key in ("json", "superjson", "dehydratedState"):
nested = candidate.get(key)
if isinstance(nested, Mapping):
yield nested
yield candidate
def _matches_model_query(query_key: Any) -> bool:
if not isinstance(query_key, (list, tuple)) or not query_key:
return False
first = query_key[0]
if not isinstance(first, (list, tuple)) or len(first) < 2:
return False
return str(first[0]) == _MODEL_QUERY_PATH[0] and str(first[1]) == _MODEL_QUERY_PATH[1]
def _price_fields(raw_paid_access: Any) -> Optional[Dict[str, Any]]:
"""Build the price fields for one version, or None when it is not gated."""
dto = _coerce_dto(raw_paid_access)
if dto is None:
return None
info = normalize_paid_access(dto)
if not info or not is_gate_active(info):
return None
terms = dto.get("terms")
if not isinstance(terms, Mapping):
terms = {}
list_price = _download_price(terms)
effective_price = list_price
sale = dto.get("sale")
sale_ends_at = None
if isinstance(sale, Mapping):
sale_ends_at = _coerce_str(sale.get("endsAt"))
buyer_terms = sale.get("buyerTerms")
if isinstance(buyer_terms, Mapping):
sale_price = _download_price(buyer_terms)
if sale_price is not None:
effective_price = sale_price
return {
"price_buzz": effective_price,
"list_price_buzz": list_price,
"generation_price_buzz": _generation_price(terms),
"accepts_blue_buzz": bool(terms.get("acceptsBlueBuzz")),
"price_sale_ends_at": sale_ends_at,
}
def _coerce_dto(value: Any) -> Optional[Mapping[str, Any]]:
if isinstance(value, str):
try:
value = json.loads(value)
except (TypeError, ValueError):
return None
return value if isinstance(value, Mapping) else None
def _download_price(terms: Mapping[str, Any]) -> Optional[int]:
download = terms.get("download")
if not isinstance(download, Mapping):
return None
return _coerce_price(download.get("price"))
def _generation_price(terms: Mapping[str, Any]) -> Optional[int]:
generation = terms.get("generation")
if not isinstance(generation, Mapping):
return None
# `{free: true}` has no price of its own, and a paid tier may omit `price` to
# fall back to the download price — the download price is already captured
# separately, so only an explicit number is reported here.
return _coerce_price(generation.get("price"))
def _coerce_price(value: Any) -> Optional[int]:
if isinstance(value, bool):
return None
if isinstance(value, (int, float)):
price = int(value)
return price if price >= 0 else None
if isinstance(value, str):
try:
price = int(float(value.strip()))
except (TypeError, ValueError):
return None
return price if price >= 0 else None
return None
def _coerce_int(value: Any) -> Optional[int]:
if isinstance(value, bool):
return None
if isinstance(value, int):
return value
if isinstance(value, str):
try:
return int(value.strip())
except (TypeError, ValueError):
return None
return None
def _coerce_str(value: Any) -> Optional[str]:
if isinstance(value, str) and value.strip():
return value.strip()
return None
+15
View File
@@ -41,6 +41,21 @@ def normalize_civitai_page_host(hostname: str | None) -> str:
return DEFAULT_CIVITAI_PAGE_HOST
def civitai_page_host_candidates(hostname: str | None = None) -> tuple[str, ...]:
"""Ordered page hosts to try, the preferred one first.
The hosts are not interchangeable: ``civitai.red`` serves mature model pages
that ``civitai.com`` hides from anonymous visitors, but it also sits behind a
Cloudflare challenge that rejects non-browser HTTP clients outright. Trying the
others when the preferred host refuses a request is what keeps a user's
``civitai_host`` preference from disabling page reads entirely.
"""
preferred = normalize_civitai_page_host(hostname)
rest = sorted(_SUPPORTED_CIVITAI_PAGE_HOSTS - {preferred})
return (preferred, *rest)
def build_civitai_model_page_url(
model_id: str | int | None,
version_id: str | int | None = None,
+158
View File
@@ -0,0 +1,158 @@
"""Shared interpretation of CivitAI's ``paidAccess`` DTO.
One rule for the whole backend: the update service (badges, update filters, price
tracking) and the download gate both ask these helpers, so a version can never be
"paid" for one consumer and "free" for another.
Why the rules look like this
----------------------------
CivitAI's public v1 API only returns a non-null DTO for an *active* gate: a lapsed
gate stays in the database as a tombstone but is filtered out server-side
(``toPublicPaidAccessDto`` in civitai's ``server/services/paid-access.service.ts``).
A timed gate whose end time has not been recorded yet is reported as
``{"permanent": false, "endsAt": null}`` and is still enforced — such a version
reports ``canDownload: false`` on the model page — so it must NOT be discarded.
Dropping it (the previous behaviour) hid real gates, which is the class of bug
reported in issue #1060.
"""
from __future__ import annotations
import json
from datetime import datetime, timezone
from typing import Any, Mapping, Optional
__all__ = [
"normalize_paid_access",
"is_permanent_paid",
"is_gate_active",
"is_early_access_deadline_active",
"parse_civitai_timestamp",
]
# A DTO that carries neither key is not a gate signal at all (defensive: CivitAI
# never emits a bare ``{}``, but other metadata sources might).
_PAID_ACCESS_KEYS = ("permanent", "endsAt")
def parse_civitai_timestamp(value: Any) -> Optional[datetime]:
"""Parse a CivitAI ISO-8601 timestamp into an aware UTC datetime.
Returns None for anything that is not a parsable string. Naive timestamps are
assumed to be UTC, matching CivitAI's serialization.
"""
if not isinstance(value, str):
return None
text = value.strip()
if not text:
return None
if text.endswith("Z"):
text = f"{text[:-1]}+00:00"
try:
parsed = datetime.fromisoformat(text)
except ValueError:
return None
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=timezone.utc)
return parsed
def normalize_paid_access(value: Any) -> Optional[dict]:
"""Normalize a ``paidAccess`` value into ``{"permanent": bool, "endsAt": str|None}``.
Accepts a mapping, a JSON string (the by-hash enrichment path carries the DTO as
text), or None. Returns None when the value carries no gate signal.
Note that ``{"permanent": False, "endsAt": None}`` is a *gate*: CivitAI reports
it for a timed gate whose window end is not set yet, and enforces it.
"""
if value is None:
return None
if isinstance(value, str):
try:
parsed = json.loads(value)
except (TypeError, ValueError):
return None
if not isinstance(parsed, dict):
return None
value = parsed
if not isinstance(value, Mapping):
return None
if not any(key in value for key in _PAID_ACCESS_KEYS):
return None
ends_at = value.get("endsAt")
normalized_ends_at = ends_at.strip() if isinstance(ends_at, str) and ends_at.strip() else None
return {
"permanent": bool(value.get("permanent")),
"endsAt": normalized_ends_at,
}
def is_permanent_paid(paid_access: Optional[Mapping[str, Any]]) -> bool:
"""True when the gate never expires (a permanent paid version)."""
return bool(paid_access and paid_access.get("permanent"))
def is_gate_active(
paid_access: Optional[Mapping[str, Any]],
*,
now: Optional[datetime] = None,
) -> bool:
"""True when a normalized ``paidAccess`` gate is currently in force.
Active means permanent, or a timed gate whose end is either still in the future
or not recorded yet (CivitAI enforces the latter too). An unparsable end time is
treated as active rather than free: the download would fail anyway, so the
conservative reading matches what the user will experience.
"""
if not paid_access:
return False
if paid_access.get("permanent"):
return True
ends_at = paid_access.get("endsAt")
if not ends_at:
# Timed gate with no recorded end — CivitAI still gates the download.
return True
parsed = parse_civitai_timestamp(ends_at)
if parsed is None:
return True
reference = now or datetime.now(timezone.utc)
if reference.tzinfo is None:
reference = reference.replace(tzinfo=timezone.utc)
return parsed > reference
def is_early_access_deadline_active(
ends_at: Any,
*,
now: Optional[datetime] = None,
) -> bool:
"""True when a legacy ``earlyAccessEndsAt`` deadline is still in the future.
Kept separate from :func:`is_gate_active` because the legacy field is a bare
timestamp rather than a DTO. A present-but-unparsable value is treated as active,
matching the previous conservative behaviour.
"""
if not ends_at:
return False
parsed = parse_civitai_timestamp(ends_at)
if parsed is None:
return True
reference = now or datetime.now(timezone.utc)
if reference.tzinfo is None:
reference = reference.replace(tzinfo=timezone.utc)
return parsed > reference
@@ -460,6 +460,20 @@
border-color: color-mix(in oklch, #845EF7 45%, transparent);
}
/* Price alert badge - green tone (#37B24D) for a price under the user's threshold */
.version-badge-price-alert {
background: color-mix(in oklch, #37B24D 25%, transparent);
color: #2B8A3E;
border-color: color-mix(in oklch, #37B24D 55%, transparent);
font-variant-numeric: tabular-nums;
}
[data-theme="dark"] .version-badge-price-alert {
background: color-mix(in oklch, #37B24D 20%, transparent);
color: #69DB7C;
border-color: color-mix(in oklch, #37B24D 45%, transparent);
}
.version-meta-ea {
color: #E67700;
font-weight: 600;
+3
View File
@@ -99,6 +99,9 @@ export function getApiEndpoints(modelType) {
ignoreModelUpdate: `/api/lm/${modelType}/updates/ignore`,
ignoreVersionUpdate: `/api/lm/${modelType}/updates/ignore-version`,
// Price alerts are app-wide (all model types in one query), so they are
// not part of this per-type endpoint map; see UpdateService.js.
// Preview management
replacePreview: `/api/lm/${modelType}/replace-preview`,
setPreviewFromUrl: `/api/lm/${modelType}/set-preview-from-url`,
+99 -10
View File
@@ -187,6 +187,53 @@ function isPaidPermanent(version) {
return version && version.isPaid === true;
}
function isGated(version) {
return isEarlyAccessActive(version) || isPaidPermanent(version);
}
/**
* Format a Buzz price for display, or '' when the price is unknown (price
* tracking is opt-in, so most versions have no price at all).
*/
function formatBuzzPrice(value) {
if (typeof value !== 'number' || !Number.isFinite(value) || value < 0) {
return '';
}
return `${value.toLocaleString()} Buzz`;
}
/**
* Tooltip for a known price: notes a sale against the list price and the Blue
* Buzz option, since both change what the number means to a buyer.
*/
function buildPriceTooltip(version, baseTooltip) {
const effective = formatBuzzPrice(version?.priceBuzz);
const segments = [baseTooltip];
if (!effective) {
return baseTooltip;
}
const list = version?.listPriceBuzz;
if (typeof list === 'number' && Number.isFinite(list) && list > version.priceBuzz) {
segments.push(
translate(
'modals.model.versions.badges.priceSaleTooltip',
{ list: list.toLocaleString() },
`On sale, normally ${list.toLocaleString()} Buzz`
)
);
}
if (version?.acceptsBlueBuzz) {
segments.push(
translate(
'modals.model.versions.badges.priceBlueBuzzTooltip',
{},
'Can also be paid with Blue Buzz'
)
);
}
return segments.join(' · ');
}
function isDownloadAllowed(version) {
if (!version.usageControl) {
return true;
@@ -522,22 +569,64 @@ function renderRow(version, options) {
}));
}
if (isEarlyAccess) {
// Obtainability, only where there is a decision to make: a version already in
// the library cannot become cheaper *for this user*, so it gets no cost badge.
// The download price is the only price axis that matters here — the generation
// fee is what CivitAI charges to generate on its own site, which a local model
// never incurs.
const priceLabel = formatBuzzPrice(version.priceBuzz);
if (!version.isInLibrary && isEarlyAccess) {
// Early access is the one state with a real countdown: "free on <date>"
// decides between waiting and paying.
const eaTime = formatEarlyAccessTime(version.earlyAccessEndsAt);
badges.push(buildBadge(earlyAccessBadgeLabel, 'early-access', {
title: translate(
'modals.model.versions.badges.earlyAccessTooltip',
{},
'This version currently requires Civitai early access'
title: buildPriceTooltip(
version,
eaTime
? translate(
'modals.model.versions.badges.earlyAccessTooltipUntil',
{ date: eaTime },
`Early access until ${eaTime}`
)
: translate(
'modals.model.versions.badges.earlyAccessTooltip',
{},
'This version currently requires Civitai early access'
)
),
}));
}
if (isPaidPermanent(version)) {
badges.push(buildBadge(paidBadgeLabel, 'paid', {
if (!version.isInLibrary && isGated(version)) {
if (priceLabel) {
// The number already says "paid", so a separate Paid badge would be
// noise next to it.
badges.push(buildBadge(priceLabel, isEarlyAccess ? 'early-access' : 'paid', {
title: buildPriceTooltip(version, paidBadgeLabel),
}));
} else if (!isEarlyAccess) {
// Paid, but no readable price: the gate is certain (it comes from the
// public API), only the number is best-effort. Saying nothing here
// would read as "free", which is the one thing this version is not.
badges.push(buildBadge(paidBadgeLabel, 'paid', {
title: translate(
'modals.model.versions.badges.paidTooltip',
{},
'This version requires payment to download'
),
}));
}
}
// A version that used to be gated and no longer is. `gateLapsedAt` is
// persisted, so the marker survives long after the refresh that saw it.
if (!version.isInLibrary && !isGated(version) && version.gateLapsedAt) {
const freeBadgeLabel = translate('modals.model.versions.badges.freeNow', {}, 'Free Now');
badges.push(buildBadge(freeBadgeLabel, 'success', {
title: translate(
'modals.model.versions.badges.paidTooltip',
{},
'This version requires payment to download'
'modals.model.versions.badges.freeNowTooltip',
{ date: formatDateLabel(version.gateLapsedAt) || '' },
'This version no longer requires payment'
),
}));
}
+11
View File
@@ -1169,6 +1169,17 @@ export class SettingsManager {
hidePaidUpdatesCheckbox.checked = state.global.settings.hide_paid_updates || false;
}
// Set buzz price tracking settings
const priceTrackingEnabledCheckbox = document.getElementById('priceTrackingEnabled');
if (priceTrackingEnabledCheckbox) {
priceTrackingEnabledCheckbox.checked = state.global.settings.price_tracking_enabled || false;
}
const priceCheckTtlInput = document.getElementById('priceCheckTtlHours');
if (priceCheckTtlInput) {
priceCheckTtlInput.value = state.global.settings.price_check_ttl_hours ?? 24;
}
const skipPreviouslyDownloadedModelVersionsCheckbox = document.getElementById('skipPreviouslyDownloadedModelVersions');
if (skipPreviouslyDownloadedModelVersionsCheckbox) {
skipPreviouslyDownloadedModelVersionsCheckbox.checked =
+2
View File
@@ -58,6 +58,8 @@ const DEFAULT_SETTINGS_BASE = Object.freeze({
version_grouping: 'same_base',
hide_early_access_updates: false,
hide_paid_updates: false,
price_tracking_enabled: false,
price_check_ttl_hours: 24,
auto_organize_exclusions: [],
metadata_refresh_skip_paths: [],
skip_previously_downloaded_model_versions: false,
+52
View File
@@ -8,6 +8,54 @@ import { modalManager } from '../managers/ModalManager.js';
const CHECK_UPDATES_CONFIRMATION_KEY = 'ack_check_updates_for_all_models';
/**
* Count the gate/price transitions reported by an update refresh.
* @param {Array} events - `events` array from the refresh response.
* @returns {{becameFree: number, newGate: number, priceDrop: number, total: number}}
*/
export function summarizeGateEvents(events) {
const list = Array.isArray(events) ? events : [];
return {
becameFree: list.filter((event) => event?.kind === 'became_free').length,
newGate: list.filter((event) => event?.kind === 'new_gate').length,
total: list.length
};
}
/**
* Toast the gate/price transitions of an update refresh, if any. Kept to a single
* toast so a large refresh cannot bury the user in notifications.
* @param {Array} events
*/
export function showGateEventsToast(events) {
const { becameFree, newGate } = summarizeGateEvents(events);
if (!becameFree && !newGate) {
return;
}
const parts = [];
if (becameFree) {
parts.push(
translate(
'globalContextMenu.checkModelUpdates.gateEvents.becameFree',
{ count: becameFree },
`${becameFree} version(s) became free`
)
);
}
if (newGate) {
parts.push(
translate(
'globalContextMenu.checkModelUpdates.gateEvents.newGate',
{ count: newGate },
`${newGate} version(s) are now paid`
)
);
}
showToast(parts.join(' · '), {}, becameFree ? 'success' : 'info');
}
/**
* Perform a model update check using the shared backend endpoint.
* @param {Object} [options]
@@ -85,6 +133,8 @@ export async function performModelUpdateCheck({ onStart, onComplete } = {}) {
showToast('globalContextMenu.checkModelUpdates.none', { type: displayName }, 'info');
}
showGateEventsToast(payload.events);
await resetAndReload(false);
} catch (err) {
if (err?.name === 'AbortError') {
@@ -180,6 +230,8 @@ export async function performFolderUpdateCheck(folderPath, { onComplete } = {})
showToast('sidebar.folderUpdateCheck.none', { type: displayName }, 'info');
}
showGateEventsToast(payload.events);
await resetAndReload(false);
} catch (err) {
if (err?.name === 'AbortError') {
@@ -192,6 +192,35 @@
{{ sm.setting_toggle('hidePaidUpdates', 'hide_paid_updates', 'settings.hidePaidUpdates.label', 'settings.hidePaidUpdates.help') }}
</div>
<!-- Buzz Price Tracking -->
<div class="settings-subsection">
{{ sm.subsection_header('settings.sections.priceTracking') }}
<div class="settings-help-text subtle">{{ t('settings.priceTracking.help') }}</div>
{{ sm.setting_toggle('priceTrackingEnabled', 'price_tracking_enabled', 'settings.priceTracking.enabled', 'settings.priceTracking.enabledHelp') }}
<div class="setting-item">
<div class="setting-row">
<div class="setting-info">
<label for="priceCheckTtlHours">
{{ t('settings.priceTracking.refreshHours') }}
<i class="fas fa-info-circle info-icon" data-tooltip="{{ t('settings.priceTracking.refreshHoursHelp') }}"></i>
</label>
</div>
<div class="setting-control">
<div class="text-input-wrapper">
<input
type="number"
id="priceCheckTtlHours"
min="1"
step="1"
onblur="settingsManager.saveInputSetting('priceCheckTtlHours', 'price_check_ttl_hours')"
onkeydown="if(event.key === 'Enter') { this.blur(); }"
/>
</div>
</div>
</div>
</div>
</div>
<!-- Example Images -->
<div class="settings-subsection">
{{ sm.subsection_header('settings.sections.exampleImages') }}
@@ -283,3 +283,116 @@ describe('ModelVersionsTab download button visibility', () => {
expect(openFileSelectionForVersion).not.toHaveBeenCalled();
});
});
describe('ModelVersionsTab obtainability badges', () => {
let getModelApiClient;
let fetchModelUpdateVersions;
beforeEach(async () => {
vi.resetModules();
document.body.innerHTML = `
<div id="model-versions-modal">
<div id="versions-tab">
<div class="model-versions-tab"></div>
</div>
</div>
`;
({ getModelApiClient } = await import(API_FACTORY_MODULE));
fetchModelUpdateVersions = vi.fn();
getModelApiClient.mockReturnValue({
fetchModelUpdateVersions,
fetchModelRoots: vi.fn(),
setModelUpdateIgnore: vi.fn(),
setVersionUpdateIgnore: vi.fn(),
deleteModel: vi.fn(),
});
});
afterEach(() => {
document.body.innerHTML = '';
});
function rowFor(versionId) {
return document.querySelector(`.model-version-row[data-version-id="${versionId}"]`);
}
function renderOne(version) {
fetchModelUpdateVersions.mockResolvedValue(buildRecord([
{
versionId: 1,
name: 'v1',
isInLibrary: false,
shouldIgnore: false,
...version,
},
]));
return renderVersions();
}
const PAID_GATE = { permanent: true, endsAt: null };
it('shows the download price for a paid version the user does not have', async () => {
await renderOne({ isPaid: true, paidAccess: PAID_GATE, priceBuzz: 5000 });
expect(rowFor(1).textContent).toContain('5,000 Buzz');
});
it('says Paid without a number when the price could not be read', async () => {
// The gate is certain (it comes from the public API); only the number is
// best-effort, and that is our plumbing rather than the user's problem.
await renderOne({ isPaid: true, paidAccess: PAID_GATE, priceBuzz: null });
expect(rowFor(1).textContent).toContain('Paid');
expect(rowFor(1).textContent).not.toContain('Buzz');
});
it('shows the early access countdown alongside the price', async () => {
await renderOne({
isEarlyAccess: true,
earlyAccessEndsAt: new Date(Date.now() + 3 * 86400000).toISOString(),
paidAccess: { permanent: false, endsAt: '2999-01-01T00:00:00.000Z' },
priceBuzz: 500,
});
const text = rowFor(1).textContent;
expect(text).toContain('Early Access');
expect(text).toContain('500 Buzz');
});
it('shows nothing about cost for a version already in the library', async () => {
// A version on disk cannot become cheaper *for this user*.
await renderOne({
isInLibrary: true,
isPaid: true,
paidAccess: PAID_GATE,
priceBuzz: 5000,
gateLapsedAt: '2026-09-28T00:00:00.000Z',
});
const text = rowFor(1).textContent;
expect(text).not.toContain('Buzz');
expect(text).not.toContain('Paid');
expect(text).not.toContain('Free Now');
expect(text).toContain('In Library');
});
it('marks a version that became free and is not in the library', async () => {
await renderOne({
isInLibrary: false,
isPaid: false,
priceBuzz: null,
gateLapsedAt: '2026-09-28T00:00:00.000Z',
});
expect(rowFor(1).textContent).toContain('Free Now');
});
it('shows nothing at all for a plain free version', async () => {
await renderOne({ isInLibrary: false, isPaid: false, priceBuzz: null });
const text = rowFor(1).textContent;
expect(text).not.toContain('Buzz');
expect(text).not.toContain('Paid');
expect(text).not.toContain('Free Now');
});
});
+65 -1
View File
@@ -6,7 +6,7 @@ import subprocess
import zipfile
from types import SimpleNamespace
from typing import Any
from unittest.mock import patch, MagicMock
from unittest.mock import AsyncMock, patch, MagicMock
import pytest
from aiohttp import web
@@ -16,6 +16,7 @@ from py.routes.handlers import misc_handlers
from py.routes.handlers.misc_handlers import (
BackupHandler,
DoctorHandler,
MiscHandlerSet,
FileSystemHandler,
HealthCheckHandler,
LoraCodeHandler,
@@ -2865,3 +2866,66 @@ async def test_sidecar_migration_handler_relocate_root_requires_old_root():
assert response.status == 400
assert "old_root" in payload["error"]
assert use_case.calls == []
# --- Global price alerts panel endpoint -------------------------------------
class _AnyHandler:
def __getattr__(self, _name):
return lambda request: None
def _stub_misc_handler_set(**overrides) -> MiscHandlerSet:
names = (
"health",
"settings",
"usage_stats",
"lora_code",
"trained_words",
"model_examples",
"node_registry",
"model_library",
"metadata_archive",
"backup",
"filesystem",
"custom_words",
"wildcards",
"supporters",
"doctor",
"example_workflows",
"base_model",
"model_source_handler",
"agent_handler",
"download_routing",
"sidecar_migration",
)
handlers = {name: _AnyHandler() for name in names}
handlers.update(overrides)
return MiscHandlerSet(**handlers)
def test_every_misc_route_definition_resolves_to_a_handler():
"""A route added to the table without a mapping entry 500s only on a live
server, so assert the whole table resolves here."""
mapping = _stub_misc_handler_set().to_route_mapping()
assert [
definition.handler_name
for definition in MISC_ROUTE_DEFINITIONS
if definition.handler_name not in mapping
] == []
def test_price_alert_endpoints_are_gone():
"""The redesign removed the standalone alerts surface: obtainability is an
attribute of the update surfaces, so no route may serve an alert list."""
leftovers = [
definition
for definition in MISC_ROUTE_DEFINITIONS
if "price-alert" in definition.path or "price_alert" in definition.handler_name
]
assert leftovers == []
+115
View File
@@ -10,9 +10,11 @@ import pytest
from py.config import config
from py.routes.handlers.model_handlers import (
ModelCivitaiHandler,
ModelHandlerSet,
ModelManagementHandler,
ModelUpdateHandler,
)
from py.routes.model_route_registrar import COMMON_ROUTE_DEFINITIONS
from py.services.service_registry import ServiceRegistry
from py.utils.metadata_manager import MetadataManager
from py.services.model_update_service import ModelUpdateRecord, ModelVersionRecord
@@ -324,6 +326,72 @@ async def test_refresh_model_updates_filters_records_without_updates():
assert call["target_model_ids"] is None
@pytest.mark.asyncio
async def test_refresh_model_updates_reports_gate_events_for_all_records():
"""Gate transitions are reported even for records that do not qualify as
updates (a version already in the library that became free)."""
cache = SimpleNamespace(version_index={})
service = DummyService(cache)
record = ModelUpdateRecord(
model_type="lora",
model_id=1,
versions=[
ModelVersionRecord(
version_id=11,
name="v11",
base_model=None,
released_at=None,
size_bytes=None,
preview_url=None,
is_in_library=True,
should_ignore=False,
)
],
last_checked_at=None,
should_ignore_model=False,
events=[{"versionId": 11, "kind": "became_free", "versionName": "v11", "isInLibrary": True}],
)
update_service = DummyUpdateService({1: record})
metadata_selector = AsyncMock(return_value=SimpleNamespace())
handler = ModelUpdateHandler(
service=service,
update_service=update_service,
metadata_provider_selector=metadata_selector,
settings_service=SimpleNamespace(get=lambda *_: False),
logger=logging.getLogger(__name__),
)
class DummyRequest:
can_read_body = True
query = {}
async def json(self):
return {}
response = await handler.refresh_model_updates(
DummyRequest() # pyright: ignore[reportArgumentType]
)
payload = json.loads(response.text)
# The record itself does not qualify as an update...
assert payload["records"] == []
# ...but its transition is still surfaced.
assert payload["events"] == [
{
"modelId": 1,
"modelType": "lora",
"versionId": 11,
"kind": "became_free",
"versionName": "v11",
"isInLibrary": True,
}
]
@pytest.mark.asyncio
async def test_refresh_model_updates_same_base_scope_excludes_cross_base_updates():
"""Issue #1083: with version_grouping=same_base (the default), a newer
@@ -1077,3 +1145,50 @@ async def test_relink_civitai_surfaces_provider_unavailable_without_500():
payload = json.loads(response.text)
assert payload["success"] is False
assert "CivitArchive" in payload["error"]
def test_every_common_route_definition_resolves_to_a_handler():
"""Guard the declarative route table against a missing mapping entry.
Adding a RouteDefinition without registering it in
``ModelHandlerSet.to_route_mapping`` fails at request time with a bare
KeyError from the handler lookup (and only on a live server), so assert the
whole table resolves here instead.
"""
class AnyHandler:
def __getattr__(self, _name):
return lambda request: None
handler_set = ModelHandlerSet(
page_view=AnyHandler(),
listing=AnyHandler(),
management=AnyHandler(),
query=AnyHandler(),
download=AnyHandler(),
civitai=AnyHandler(),
move=AnyHandler(),
auto_organize=AnyHandler(),
filename_template=AnyHandler(),
updates=AnyHandler(),
)
mapping = handler_set.to_route_mapping()
missing = [
definition.handler_name
for definition in COMMON_ROUTE_DEFINITIONS
if definition.handler_name not in mapping
]
assert missing == []
def test_price_alert_endpoints_are_not_registered():
"""The redesign dropped the standalone alerts surface: obtainability rides on
the update surfaces, so no per-type alert route may remain."""
assert [
definition.path_template
for definition in COMMON_ROUTE_DEFINITIONS
if "price-alert" in definition.path_template
] == []
+164
View File
@@ -1,4 +1,5 @@
import copy
import pathlib
from unittest.mock import AsyncMock
import pytest
@@ -859,3 +860,166 @@ async def test_get_version_file_mini_propagates_rate_limit(downloader):
with pytest.raises(RateLimitError):
await client.get_version_file_mini(1, 2)
async def test_get_model_prices_parses_public_page(downloader):
"""Prices come from the page payload, read anonymously (no API key)."""
client = await CivitaiClient.get_instance()
page_html = (
'<script id="__NEXT_DATA__" type="application/json">'
'{"props":{"pageProps":{"trpcState":{"json":{"queries":['
'{"queryKey":[["model","getById"],{"input":{"id":7}}],"state":{"data":'
'{"modelVersions":[{"id":42,"paidAccess":{"endsAt":null,'
'"timeframeDays":null,"terms":{"download":{"price":5000}},'
'"sale":null}}]}}}]}}}}}</script>'
)
async def fake_make_request(method, url, use_auth=True, **kwargs):
assert method == "GET"
assert "/models/7" in url
assert use_auth is False
assert kwargs.get("custom_headers", {}).get("Accept") == "text/html"
return True, page_html
downloader.make_request = fake_make_request
result = await client.get_model_prices(7)
assert result is not None
assert result[42]["price_buzz"] == 5000
async def test_get_model_prices_returns_none_on_unusable_page(downloader):
client = await CivitaiClient.get_instance()
async def fake_make_request(method, url, use_auth=True, **kwargs):
return True, "<html><body>challenge</body></html>"
downloader.make_request = fake_make_request
assert await client.get_model_prices(7) is None
async def test_get_model_prices_rejects_json_body(downloader):
"""A JSON response is not the page; it must not be parsed as one."""
client = await CivitaiClient.get_instance()
async def fake_make_request(method, url, use_auth=True, **kwargs):
return True, {"error": "nope"}
downloader.make_request = fake_make_request
assert await client.get_model_prices(7) is None
async def test_get_model_prices_propagates_rate_limit(downloader):
client = await CivitaiClient.get_instance()
async def fake_make_request(method, url, use_auth=True, **kwargs):
return False, RateLimitError("limited", retry_after=1.0)
downloader.make_request = fake_make_request
with pytest.raises(RateLimitError):
await client.get_model_prices(7)
# --- Model page host fallback -------------------------------------------------
_PAGE_FIXTURE = (
pathlib.Path(__file__).resolve().parents[1]
/ "utils"
/ "fixtures"
/ "civitai_model_page_paid.html"
)
def _page_html() -> str:
return _PAGE_FIXTURE.read_text(encoding="utf-8")
async def test_get_model_prices_falls_back_when_the_preferred_host_refuses(
downloader, monkeypatch
):
"""civitai.red refuses non-browser clients (Cloudflare); the price must still
be readable from a host that answers. This is the user-visible bug: with
civitai_host=civitai.red every fetch used to fail."""
client = await CivitaiClient.get_instance()
monkeypatch.setattr(client, "_page_host", lambda: "civitai.red")
seen = []
async def fake_make_request(method, url, use_auth=True, **kwargs):
seen.append(url)
if "civitai.red" in url:
return False, "Access forbidden"
return True, _page_html()
downloader.make_request = fake_make_request
result = await client.get_model_prices(4242)
assert result is not None
assert result[1001]["price_buzz"] == 5000
assert seen[0].startswith("https://civitai.red/")
assert any("civitai.com" in url for url in seen)
# The working host is remembered and the refusing one is parked.
assert client._page_host_preference == "civitai.com"
assert "civitai.red" in client._page_host_blocked
async def test_get_model_prices_reuses_the_working_host(downloader, monkeypatch):
client = await CivitaiClient.get_instance()
monkeypatch.setattr(client, "_page_host", lambda: "civitai.red")
calls = []
async def fake_make_request(method, url, use_auth=True, **kwargs):
calls.append(url)
if "civitai.red" in url:
return False, "Access forbidden"
return True, _page_html()
downloader.make_request = fake_make_request
await client.get_model_prices(4242)
calls.clear()
await client.get_model_prices(4242)
# Second time around the parked host is not retried.
assert calls and all("civitai.red" not in url for url in calls)
async def test_get_model_prices_returns_none_when_no_host_can_serve(downloader, monkeypatch):
"""Mature models: 404 on civitai.com/green, challenge on civitai.red."""
client = await CivitaiClient.get_instance()
monkeypatch.setattr(client, "_page_host", lambda: "civitai.com")
async def fake_make_request(method, url, use_auth=True, **kwargs):
if "civitai.red" in url:
return False, "Access forbidden"
return False, "Resource not found"
downloader.make_request = fake_make_request
assert await client.get_model_prices(2981320) is None
async def test_get_model_prices_does_not_park_a_host_on_404(downloader, monkeypatch):
"""A 404 is model-specific (mature content hidden anonymously), not a reason
to stop using the host for other models."""
client = await CivitaiClient.get_instance()
monkeypatch.setattr(client, "_page_host", lambda: "civitai.com")
async def fake_make_request(method, url, use_auth=True, **kwargs):
if "civitai.com" in url:
return False, "Resource not found"
return False, "Access forbidden"
downloader.make_request = fake_make_request
await client.get_model_prices(1)
assert "civitai.com" not in client._page_host_blocked
assert "civitai.red" in client._page_host_blocked
+844 -2
View File
@@ -1,5 +1,6 @@
import logging
import sqlite3
from dataclasses import replace
from types import SimpleNamespace
import pytest
@@ -826,6 +827,32 @@ def test_extract_single_version_paid_access_permanent(tmp_path):
assert version.paid_access is not None
def test_extract_single_version_paid_access_pending_end(tmp_path):
"""A timed gate whose window end is not recorded yet
({"permanent": false, "endsAt": null}) is still an active gate, so it is
early access with no known end date rather than a free version."""
db_path = tmp_path / "updates.sqlite"
service = ModelUpdateService(str(db_path))
entry = {
"id": 42,
"name": "v1 paid",
"availability": "Public",
"paidAccess": {"permanent": False, "endsAt": None},
"files": [],
"images": [],
}
version = service._extract_single_version(entry, index=0)
assert version is not None
assert version.is_early_access is True
assert version.is_paid is False
assert version.early_access_ends_at is None
assert version.paid_access == '{"permanent": false, "endsAt": null}'
assert ModelUpdateRecord._is_early_access_active(version) is True
def test_normalize_paid_access_accepts_json_string():
"""The by-hash enrichment path may hand paidAccess to _normalize_paid_access
as a JSON string; both the permanent and timed shapes must normalize."""
@@ -841,14 +868,21 @@ def test_normalize_paid_access_accepts_json_string():
)
assert timed == {"permanent": False, "endsAt": "2026-08-22T18:30:00.000Z"}
empty = ModelUpdateService._normalize_paid_access(
# A timed gate whose window end is not recorded yet. CivitAI only returns a
# non-null DTO for an ACTIVE gate (tombstones come back as null) and enforces
# this shape too - the model page reports canDownload: false for it - so it
# must be kept. Dropping it was the #1060 class of bug.
pending_end = ModelUpdateService._normalize_paid_access(
'{"permanent": false, "endsAt": null}'
)
assert empty is None
assert pending_end == {"permanent": False, "endsAt": None}
malformed = ModelUpdateService._normalize_paid_access("{not json")
assert malformed is None
# No gate keys at all is not a gate signal.
assert ModelUpdateService._normalize_paid_access("{}") is None
def test_has_update_for_base_hide_paid():
"""hide_paid also suppresses permanent paid versions in the same-base
@@ -1059,3 +1093,811 @@ def test_build_record_from_remote_preserves_file_count(tmp_path):
timestamp=2.0,
)
assert record.versions[0].file_count == 1
# --- Gate-state transitions and price persistence ---------------------------
def _remote_gated(version_id, *, price_checked_at=None, price_buzz=None):
return ModelVersionRecord(
version_id=version_id,
name=f"v{version_id}",
base_model=None,
released_at=None,
size_bytes=None,
preview_url=None,
is_in_library=False,
should_ignore=False,
paid_access='{"permanent": false, "endsAt": "2026-10-10T13:10:17.404Z"}',
is_early_access=True,
early_access_ends_at="2026-10-10T13:10:17.404Z",
price_checked_at=price_checked_at,
price_buzz=price_buzz,
)
def _remote_free(version_id):
return ModelVersionRecord(
version_id=version_id,
name=f"v{version_id}",
base_model=None,
released_at=None,
size_bytes=None,
preview_url=None,
is_in_library=False,
should_ignore=False,
)
def test_build_record_emits_became_free_event(tmp_path):
"""A version whose gate lapsed produces a became_free event, keeps a lapse
timestamp, and drops the now-meaningless stored price."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(
replace(
_remote_gated(7, price_checked_at=100.0, price_buzz=500),
)
)
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_free(7)],
existing=existing,
timestamp=1000.0,
)
version = record.versions[0]
assert version.gate_lapsed_at is not None
assert version.price_buzz is None
assert version.price_checked_at is None
assert [event["kind"] for event in record.events] == ["became_free"]
assert record.events[0]["versionId"] == 7
assert record.events[0]["isInLibrary"] is False
def test_build_record_emits_new_gate_event(tmp_path):
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(_remote_free(7))
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_gated(7)],
existing=existing,
timestamp=1000.0,
)
assert [event["kind"] for event in record.events] == ["new_gate"]
assert record.versions[0].gate_lapsed_at is None
def test_build_record_no_events_without_previous_snapshot(tmp_path):
"""First sight of a model must not report every existing gate as a new one."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_gated(7), _remote_free(8)],
existing=None,
timestamp=1000.0,
)
assert record.events == []
def test_build_record_keeps_lapse_marker_and_skips_ignored(tmp_path):
"""An already-free version keeps its original lapse marker and an ignored
version reports nothing at all."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
lapsed = replace(_remote_free(7), gate_lapsed_at="2026-09-01T00:00:00.000Z")
existing = make_record(lapsed, replace(_remote_gated(8), should_ignore=True))
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_free(7), _remote_free(8)],
existing=existing,
timestamp=1000.0,
)
by_id = {version.version_id: version for version in record.versions}
assert by_id[7].gate_lapsed_at == "2026-09-01T00:00:00.000Z"
assert record.events == []
def test_build_record_preserves_price_when_fetch_skipped(tmp_path):
"""A refresh that did not run a price fetch (no price_checked_at) must keep the
stored price instead of wiping it."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(
replace(_remote_gated(7, price_checked_at=100.0, price_buzz=500))
)
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_gated(7)],
existing=existing,
timestamp=1000.0,
)
assert record.versions[0].price_buzz == 500
assert record.versions[0].price_checked_at == 100.0
assert record.events == []
def test_build_record_applies_fresh_price(tmp_path):
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(
replace(_remote_gated(7, price_checked_at=100.0, price_buzz=500))
)
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[],
remote_versions=[_remote_gated(7, price_checked_at=200.0, price_buzz=250)],
existing=existing,
timestamp=1000.0,
)
assert record.versions[0].price_buzz == 250
assert record.versions[0].price_checked_at == 200.0
def _legacy_version_table_sql() -> str:
"""The model_update_versions schema before gate-lapse/price columns."""
return """
CREATE TABLE model_update_versions (
model_id INTEGER NOT NULL,
version_id INTEGER NOT NULL,
sort_index INTEGER NOT NULL DEFAULT 0,
name TEXT,
base_model TEXT,
released_at TEXT,
size_bytes INTEGER,
preview_url TEXT,
is_in_library INTEGER NOT NULL DEFAULT 0,
should_ignore INTEGER NOT NULL DEFAULT 0,
early_access_ends_at TEXT,
is_early_access INTEGER NOT NULL DEFAULT 0,
usage_control TEXT,
paid_access TEXT,
is_paid INTEGER NOT NULL DEFAULT 0,
file_count INTEGER,
PRIMARY KEY (model_id, version_id)
)
"""
def test_migration_adds_price_columns_to_legacy_db(tmp_path):
"""Opening a database written before this feature must add the columns and keep
the existing rows (the migration path real users hit)."""
db_path = tmp_path / "updates.sqlite"
conn = sqlite3.connect(db_path)
conn.execute(
"CREATE TABLE model_update_status ("
"model_id INTEGER PRIMARY KEY, model_type TEXT NOT NULL, "
"last_checked_at REAL, should_ignore_model INTEGER NOT NULL DEFAULT 0)"
)
conn.execute(_legacy_version_table_sql())
conn.execute(
"INSERT INTO model_update_status (model_id, model_type, last_checked_at) "
"VALUES (999, 'lora', 1.0)"
)
conn.execute(
"INSERT INTO model_update_versions ("
"model_id, version_id, is_in_library, is_early_access, paid_access, is_paid"
") VALUES (999, 7, 1, 1, '{\"permanent\": false, \"endsAt\": null}', 0)"
)
conn.commit()
conn.close()
service = ModelUpdateService(str(db_path))
columns = service._get_table_columns(
service._connect(), "model_update_versions"
)
for column in (
"gate_lapsed_at",
"price_buzz",
"list_price_buzz",
"generation_price_buzz",
"accepts_blue_buzz",
"price_sale_ends_at",
"price_checked_at",
"price_check_attempted_at",
):
assert column in columns
# The alert-state columns were removed together with the threshold they served.
assert "price_alert_state" not in columns
assert "price_alert_since" not in columns
record = service._get_record("lora", 999)
assert record is not None
assert [version.version_id for version in record.versions] == [7]
version = record.versions[0]
assert version.paid_access == '{"permanent": false, "endsAt": null}'
assert version.price_buzz is None
assert version.price_checked_at is None
assert version.gate_lapsed_at is None
assert version.accepts_blue_buzz is False
def test_price_fields_round_trip_through_sqlite(tmp_path):
"""Price and lapse columns survive an upsert/read cycle."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
version = replace(
_remote_gated(7, price_checked_at=123.5, price_buzz=500),
list_price_buzz=600,
generation_price_buzz=100,
accepts_blue_buzz=True,
price_sale_ends_at="2026-10-01T00:00:00.000Z",
is_in_library=True,
gate_lapsed_at=None,
)
service._upsert_record(make_record(version))
stored = service._get_record("lora", 999)
assert stored is not None
loaded = stored.versions[0]
assert loaded.price_buzz == 500
assert loaded.list_price_buzz == 600
assert loaded.generation_price_buzz == 100
assert loaded.accepts_blue_buzz is True
assert loaded.price_sale_ends_at == "2026-10-01T00:00:00.000Z"
assert loaded.price_checked_at == 123.5
assert loaded.price_check_attempted_at is None
# --- Optional price capture --------------------------------------------------
class FakeSettings:
"""Minimal stand-in for SettingsManager (only `.get` is used by these paths)."""
def __init__(self, values=None):
self._values = dict(values or {})
def get(self, key, default=None):
return self._values.get(key, default)
def set(self, key, value):
self._values[key] = value
class PriceProvider(DummyProvider):
"""DummyProvider that can also serve prices."""
def __init__(self, response, *, prices=None, error=None):
super().__init__(response)
self.prices = prices
self.error = error
self.price_calls = 0
async def get_model_prices(self, model_id):
self.price_calls += 1
if self.error is not None:
raise self.error
return self.prices
GATED_RESPONSE = {
"modelVersions": [
{
"id": 12,
"baseModel": "Pony",
"availability": "Public",
"paidAccess": {"permanent": False, "endsAt": "2999-01-01T00:00:00.000Z"},
"files": [],
"images": [],
}
]
}
FREE_RESPONSE = {
"modelVersions": [
{"id": 12, "baseModel": "Pony", "availability": "Public", "files": [], "images": []}
]
}
LOCAL_RAW_DATA = [{"civitai": {"modelId": 1, "id": 11}, "base_model": "Pony"}]
PRICE_PAYLOAD = {
12: {
"price_buzz": 250,
"list_price_buzz": 500,
"generation_price_buzz": 100,
"accepts_blue_buzz": True,
"price_sale_ends_at": "2999-01-02T00:00:00.000Z",
}
}
def _price_service(tmp_path, **settings):
return ModelUpdateService(
str(tmp_path / "updates.sqlite"),
ttl_seconds=0,
settings_manager=FakeSettings(settings),
)
@pytest.mark.asyncio
async def test_price_capture_off_by_default(tmp_path):
service = _price_service(tmp_path)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
record = await service.get_record("lora", 1)
assert provider.price_calls == 0
assert record.versions[0].price_checked_at is None
assert record.versions[0].price_buzz is None
@pytest.mark.asyncio
async def test_price_capture_stores_prices_for_gated_versions(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
record = await service.get_record("lora", 1)
assert provider.price_calls == 1
version = next(v for v in record.versions if v.version_id == 12)
assert version.price_buzz == 250
assert version.list_price_buzz == 500
assert version.generation_price_buzz == 100
assert version.accepts_blue_buzz is True
assert version.price_sale_ends_at == "2999-01-02T00:00:00.000Z"
assert version.price_checked_at is not None
@pytest.mark.asyncio
async def test_price_capture_skips_ungated_models(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(FREE_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 0
@pytest.mark.asyncio
async def test_price_capture_failure_keeps_stored_price(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
await service.refresh_for_model_type(
"lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
)
first = await service.get_record("lora", 1)
stored_checked_at = next(
v for v in first.versions if v.version_id == 12
).price_checked_at
assert stored_checked_at is not None
# The page is unreadable this time (a changed gate still triggers a fetch):
# the price must survive untouched rather than being blanked.
failing = PriceProvider(
{
"modelVersions": [
{
"id": 12,
"baseModel": "Pony",
"availability": "Public",
"paidAccess": {
"permanent": False,
"endsAt": "2999-06-01T00:00:00.000Z",
},
"files": [],
"images": [],
}
]
},
prices=None,
)
await service.refresh_for_model_type("lora", scanner, failing)
record = await service.get_record("lora", 1)
version = next(v for v in record.versions if v.version_id == 12)
assert failing.price_calls == 1
assert version.price_buzz == 250
assert version.price_checked_at == stored_checked_at
@pytest.mark.asyncio
async def test_price_capture_respects_ttl(tmp_path):
"""A second refresh with an unchanged gate and a fresh price must not refetch."""
service = _price_service(
tmp_path, price_tracking_enabled=True, price_check_ttl_hours=24
)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 1
@pytest.mark.asyncio
async def test_price_capture_refetches_when_gate_changes(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
changed_gate = {
"modelVersions": [
{
"id": 12,
"baseModel": "Pony",
"availability": "Public",
"paidAccess": {
"permanent": True,
"endsAt": None,
},
"files": [],
"images": [],
}
]
}
provider.response = changed_gate
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 2
@pytest.mark.asyncio
async def test_price_capture_survives_provider_without_support(tmp_path):
"""A provider that cannot serve prices must not break the refresh."""
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = DummyProvider(GATED_RESPONSE)
await service.refresh_for_model_type("lora", scanner, provider)
record = await service.get_record("lora", 1)
assert record is not None
version = next(v for v in record.versions if v.version_id == 12)
assert version.price_buzz is None
@pytest.mark.asyncio
async def test_price_capture_drops_price_when_version_becomes_free(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
await service.refresh_for_model_type(
"lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
)
await service.refresh_for_model_type("lora", scanner, PriceProvider(FREE_RESPONSE))
record = await service.get_record("lora", 1)
version = next(v for v in record.versions if v.version_id == 12)
assert version.price_buzz is None
assert version.price_checked_at is None
assert version.gate_lapsed_at is not None
# --- Price alerts ------------------------------------------------------------
def _gated_response(ends_at: str) -> dict:
"""Gated response with a specific EA end.
Varying the end date between refreshes is what forces a re-fetch inside a
single test (a price is otherwise considered fresh for the whole TTL).
"""
return {
"modelVersions": [
{
"id": 12,
"baseModel": "Pony",
"availability": "Public",
"paidAccess": {"permanent": False, "endsAt": ends_at},
"files": [],
"images": [],
}
]
}
def _prices(price_buzz: int) -> dict:
return {12: {"price_buzz": price_buzz, "list_price_buzz": price_buzz}}
# --- Price alerts panel semantics (read-time threshold, across types) ---------
def _gated_response_for(version_id: int, ends_at: str) -> dict:
return {
"modelVersions": [
{
"id": version_id,
"baseModel": "Pony",
"availability": "Public",
"paidAccess": {"permanent": False, "endsAt": ends_at},
"files": [],
"images": [],
}
]
}
def _free_response_for(version_id: int) -> dict:
return {
"modelVersions": [
{
"id": version_id,
"baseModel": "Pony",
"availability": "Public",
"files": [],
"images": [],
}
]
}
@pytest.mark.asyncio
async def test_failed_price_attempt_is_recorded_as_unavailable(tmp_path):
"""A gated version we tried to price and could not must be distinguishable
from one we never looked at — that is the honest "unavailable" state for
mature models, whose pages no host will serve anonymously."""
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
failing = PriceProvider(GATED_RESPONSE, prices=None)
await service.refresh_for_model_type("lora", scanner, failing)
record = await service.get_record("lora", 1)
version = next(v for v in record.versions if v.version_id == 12)
assert failing.price_calls == 1
assert version.price_buzz is None
assert version.price_checked_at is None
assert version.price_check_attempted_at is not None
@pytest.mark.asyncio
async def test_successful_price_attempt_sets_both_markers(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
await service.refresh_for_model_type(
"lora", scanner, PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
)
record = await service.get_record("lora", 1)
version = next(v for v in record.versions if v.version_id == 12)
assert version.price_buzz == 250
assert version.price_checked_at is not None
assert version.price_check_attempted_at is not None
@pytest.mark.asyncio
async def test_no_price_attempt_is_recorded_while_tracking_is_off(tmp_path):
service = _price_service(tmp_path) # tracking off
scanner = DummyScanner(LOCAL_RAW_DATA)
await service.refresh_for_model_type("lora", scanner, DummyProvider(GATED_RESPONSE))
record = await service.get_record("lora", 1)
assert record.versions[0].price_check_attempted_at is None
@pytest.mark.asyncio
async def test_unavailable_marker_clears_when_the_version_becomes_free(tmp_path):
service = _price_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
await service.refresh_for_model_type(
"lora", scanner, PriceProvider(GATED_RESPONSE, prices=None)
)
await service.refresh_for_model_type("lora", scanner, DummyProvider(FREE_RESPONSE))
record = await service.get_record("lora", 1)
assert record.versions[0].price_check_attempted_at is None
def _long_ttl_service(tmp_path, **settings):
"""A service whose metadata TTL does not lapse during the test."""
return ModelUpdateService(
str(tmp_path / "updates.sqlite"),
ttl_seconds=86400,
settings_manager=FakeSettings(settings),
)
@pytest.mark.asyncio
async def test_prices_are_fetched_even_when_the_version_list_is_fresh(tmp_path):
"""Enabling price tracking must not wait for the metadata TTL.
The cached record already carries the gate, so a fresh version list is no
reason to skip the price: otherwise turning the feature on prices only the
handful of models that happened to need a metadata refresh that round.
"""
service = _long_ttl_service(tmp_path, price_tracking_enabled=False)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 0
metadata_calls = provider.calls
checked_at_before = (await service.get_record("lora", 1)).last_checked_at
service._settings.set("price_tracking_enabled", True)
await service.refresh_for_model_type("lora", scanner, provider)
record = await service.get_record("lora", 1)
version = next(v for v in record.versions if v.version_id == 12)
# The version list came from the cache this round ...
assert provider.calls == metadata_calls
# ... and the price was captured anyway.
assert provider.price_calls == 1
assert version.price_buzz == 250
assert version.price_checked_at is not None
assert version.price_check_attempted_at is not None
# A price-only pass must not extend the metadata TTL.
assert record.last_checked_at == checked_at_before
@pytest.mark.asyncio
async def test_failed_price_attempt_is_not_retried_within_the_ttl(tmp_path):
"""A mature model whose page no host will serve must not cost two requests
on every single update check."""
service = _long_ttl_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=None)
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 1
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 1
@pytest.mark.asyncio
async def test_forced_refresh_reprices_within_the_ttl(tmp_path):
service = _long_ttl_service(tmp_path, price_tracking_enabled=True)
scanner = DummyScanner(LOCAL_RAW_DATA)
provider = PriceProvider(GATED_RESPONSE, prices=PRICE_PAYLOAD)
await service.refresh_for_model_type("lora", scanner, provider)
assert provider.price_calls == 1
await service.refresh_for_model_type(
"lora", scanner, provider, force_refresh=True
)
assert provider.price_calls == 2
# --- Ownership scoping and the removed threshold -----------------------------
def test_gate_event_is_suppressed_for_a_version_the_user_owns(tmp_path):
"""A version already on disk cannot become cheaper *for this user*, so it must
not produce an event. More than half of a real library's gated versions are
owned, so this is the common case, not an edge case."""
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(replace(_remote_gated(7), is_in_library=True))
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[7],
remote_versions=[_remote_free(7)],
existing=existing,
timestamp=1000.0,
)
assert record.events == []
# The lapse itself is still recorded - it is the *notification* that is
# pointless for a version the user already has.
assert record.versions[0].gate_lapsed_at is not None
def test_new_gate_event_is_suppressed_for_a_version_the_user_owns(tmp_path):
service = ModelUpdateService(str(tmp_path / "updates.sqlite"))
existing = make_record(replace(_remote_free(7), is_in_library=True))
record = service._build_record_from_remote(
model_type="lora",
model_id=999,
local_versions=[7],
remote_versions=[_remote_gated(7)],
existing=existing,
timestamp=1000.0,
)
assert record.events == []
def test_threshold_setting_is_gone_from_the_defaults():
"""The redesign removed the numeric threshold: every obtainability decision is
categorical (wait / pay / skip), so no number is stored or compared."""
from py.services.settings_manager import DEFAULT_SETTINGS
assert "price_alert_threshold_buzz" not in DEFAULT_SETTINGS
assert "price_tracking_enabled" in DEFAULT_SETTINGS
assert "price_check_ttl_hours" in DEFAULT_SETTINGS
def test_obsolete_alert_columns_are_dropped_from_a_development_database(tmp_path):
"""A database created while the feature was unreleased still carries the two
alert-state columns; opening it must drop them rather than leave dead schema."""
db_path = tmp_path / "updates.sqlite"
service = ModelUpdateService(str(db_path))
with service._connect() as conn:
conn.execute(
"ALTER TABLE model_update_versions "
"ADD COLUMN price_alert_state INTEGER NOT NULL DEFAULT 0"
)
conn.execute(
"ALTER TABLE model_update_versions ADD COLUMN price_alert_since REAL"
)
conn.commit()
assert "price_alert_state" in service._get_table_columns(
conn, "model_update_versions"
)
# A fresh service instance runs the migration on open.
reopened = ModelUpdateService(str(db_path))
columns = reopened._get_table_columns(
reopened._connect(), "model_update_versions"
)
assert "price_alert_state" not in columns
assert "price_alert_since" not in columns
@@ -0,0 +1,108 @@
<!DOCTYPE html>
<!--
Trimmed stand-in for a CivitAI model page, mirroring the shape verified against
civitai.com / civitai.red. Values are synthetic; the structure (unrelated query
first, then the model.getById query, tombstones included) is what matters.
-->
<html>
<head>
<title>Fixture model</title>
</head>
<body>
<script id="__NEXT_DATA__" type="application/json">
{
"props": {
"pageProps": {
"trpcState": {
"json": {
"queries": [
{
"queryKey": [["common", "getEntityAccess"], { "entityId": 1001 }],
"state": { "data": [] }
},
{
"queryKey": [
["model", "getById"],
{ "input": { "id": 4242, "browsingLevel": 1 }, "type": "query" }
],
"state": {
"data": {
"id": 4242,
"name": "Fixture model",
"modelVersions": [
{
"id": 1001,
"name": "permanent paid",
"canDownload": false,
"paidAccess": {
"endsAt": null,
"timeframeDays": null,
"terms": {
"download": { "price": 5000 },
"generation": { "price": 100, "trialLimit": 5 }
},
"sale": null
}
},
{
"id": 1002,
"name": "timed gate on sale",
"canDownload": false,
"paidAccess": {
"endsAt": "2999-10-10T13:10:17.404Z",
"timeframeDays": 7,
"terms": {
"download": { "price": 125 },
"generation": { "trialLimit": 10 },
"acceptsBlueBuzz": true
},
"sale": {
"listTerms": { "download": { "price": 125 } },
"buyerTerms": { "download": { "price": 100 } },
"endsAt": "2999-09-01T00:00:00.000Z",
"discountType": "percent",
"discountAmount": 20
}
}
},
{
"id": 1003,
"name": "timed gate with no recorded end",
"canDownload": false,
"paidAccess": {
"endsAt": null,
"timeframeDays": 7,
"terms": { "download": { "price": 300 } },
"sale": null
}
},
{
"id": 1004,
"name": "lapsed gate (tombstone)",
"canDownload": true,
"paidAccess": {
"endsAt": "2000-08-27T16:56:57.438Z",
"timeframeDays": 3,
"terms": { "download": { "price": 200 } },
"sale": null
}
},
{
"id": 1005,
"name": "never gated",
"canDownload": true,
"paidAccess": null
}
]
}
}
}
]
}
}
}
}
}
</script>
</body>
</html>
+91
View File
@@ -0,0 +1,91 @@
"""Tests for the CivitAI model page price parser.
The fixture mirrors a real page payload; the cases that matter are the ones that
decide whether a user gets a price at all, and whether a lapsed gate is mistaken
for a live one.
"""
from pathlib import Path
import pytest
from py.utils.civitai_page_prices import MAX_PAGE_BYTES, parse_model_page_prices
FIXTURE = Path(__file__).parent / "fixtures" / "civitai_model_page_paid.html"
@pytest.fixture(scope="module")
def fixture_html() -> str:
return FIXTURE.read_text(encoding="utf-8")
def test_parses_prices_for_gated_versions(fixture_html):
prices = parse_model_page_prices(fixture_html)
assert prices is not None
# Lapsed gates and never-gated versions must not appear.
assert set(prices) == {1001, 1002, 1003}
def test_permanent_gate_price(fixture_html):
prices = parse_model_page_prices(fixture_html)
permanent = prices[1001]
assert permanent["price_buzz"] == 5000
assert permanent["list_price_buzz"] == 5000
assert permanent["generation_price_buzz"] == 100
assert permanent["accepts_blue_buzz"] is False
assert permanent["price_sale_ends_at"] is None
def test_timed_gate_uses_sale_price_as_effective(fixture_html):
prices = parse_model_page_prices(fixture_html)
timed = prices[1002]
# The stored/list price stays visible for a strikethrough, the effective price
# is what a buyer pays now.
assert timed["price_buzz"] == 100
assert timed["list_price_buzz"] == 125
assert timed["accepts_blue_buzz"] is True
assert timed["price_sale_ends_at"] == "2999-09-01T00:00:00.000Z"
# Generation is bundled with the download tier here (`trialLimit` only).
assert timed["generation_price_buzz"] is None
def test_timed_gate_without_recorded_end_is_still_priced(fixture_html):
prices = parse_model_page_prices(fixture_html)
assert prices[1003]["price_buzz"] == 300
assert prices[1003]["generation_price_buzz"] is None
@pytest.mark.parametrize(
"html",
[
None,
"",
"<html><body>no payload here</body></html>",
'<script id="__NEXT_DATA__" type="application/json">{not json</script>',
# Valid JSON, but not the shape we need.
'<script id="__NEXT_DATA__" type="application/json">{"props":{}}</script>',
# Model query present, but no versions.
'<script id="__NEXT_DATA__" type="application/json">'
'{"props":{"pageProps":{"trpcState":{"json":{"queries":['
'{"queryKey":[["model","getById"]],"state":{"data":{}}}]}}}}}</script>',
],
)
def test_unusable_payloads_return_none(html):
assert parse_model_page_prices(html) is None
def test_oversized_page_is_skipped():
oversized = "x" * (MAX_PAGE_BYTES + 1)
assert parse_model_page_prices(oversized) is None
def test_unrelated_first_query_does_not_win(fixture_html):
"""Query order varies per page; selection is by procedure name."""
prices = parse_model_page_prices(fixture_html)
assert prices is not None
assert 1001 in prices
+112
View File
@@ -0,0 +1,112 @@
"""Tests for the shared CivitAI paidAccess interpretation.
These rules decide both the Paid/Early Access badges and whether a download is
warned about, so the interesting cases are the boundary shapes CivitAI actually
emits.
"""
from datetime import datetime, timezone
import pytest
from py.utils.paid_access import (
is_early_access_deadline_active,
is_gate_active,
is_permanent_paid,
normalize_paid_access,
parse_civitai_timestamp,
)
NOW = datetime(2026, 9, 30, 12, 0, 0, tzinfo=timezone.utc)
@pytest.mark.parametrize(
"value,expected",
[
(None, None),
("", None),
("{not json", None),
("[1, 2]", None),
({}, None),
({"unknown": 1}, None),
# A non-null DTO from CivitAI's public API is always an active gate, so a
# timed gate with no recorded end must survive normalization.
({"permanent": False, "endsAt": None}, {"permanent": False, "endsAt": None}),
({"permanent": True, "endsAt": None}, {"permanent": True, "endsAt": None}),
(
{"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"},
{"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"},
),
],
)
def test_normalize_paid_access_shapes(value, expected):
assert normalize_paid_access(value) == expected
def test_normalize_paid_access_accepts_json_text():
"""The by-hash enrichment path can carry the DTO as a JSON string."""
assert normalize_paid_access('{"permanent": true, "endsAt": null}') == {
"permanent": True,
"endsAt": None,
}
def test_normalize_paid_access_blank_ends_at_is_none():
assert normalize_paid_access({"permanent": False, "endsAt": " "}) == {
"permanent": False,
"endsAt": None,
}
def test_is_permanent_paid():
assert is_permanent_paid({"permanent": True, "endsAt": None}) is True
assert is_permanent_paid({"permanent": False, "endsAt": None}) is False
assert is_permanent_paid(None) is False
@pytest.mark.parametrize(
"paid_access,expected",
[
(None, False),
({"permanent": False, "endsAt": None}, True),
({"permanent": True, "endsAt": None}, True),
# A permanent gate with a stale endsAt stays active.
({"permanent": True, "endsAt": "2020-01-01T00:00:00.000Z"}, True),
({"permanent": False, "endsAt": "2026-10-10T13:10:17.404Z"}, True),
({"permanent": False, "endsAt": "2026-08-27T16:56:57.438Z"}, False),
({"permanent": False, "endsAt": "not-a-date"}, True),
],
)
def test_is_gate_active(paid_access, expected):
assert is_gate_active(paid_access, now=NOW) is expected
@pytest.mark.parametrize(
"ends_at,expected",
[
(None, False),
("", False),
("2026-10-10T13:10:17.404Z", True),
("2026-08-27T16:56:57.438Z", False),
("garbage", True),
],
)
def test_is_early_access_deadline_active(ends_at, expected):
assert is_early_access_deadline_active(ends_at, now=NOW) is expected
@pytest.mark.parametrize(
"value,expected",
[
(None, None),
(123, None),
("", None),
("garbage", None),
("2026-10-10T13:10:17.404Z", datetime(2026, 10, 10, 13, 10, 17, 404000, tzinfo=timezone.utc)),
# Naive timestamps are assumed UTC.
("2026-10-10T13:10:17", datetime(2026, 10, 10, 13, 10, 17, tzinfo=timezone.utc)),
],
)
def test_parse_civitai_timestamp(value, expected):
assert parse_civitai_timestamp(value) == expected