mirror of
https://github.com/willmiao/ComfyUI-Lora-Manager.git
synced 2026-09-20 18:51:26 -03:00
d572292142
A ModelScope or Hugging Face download landed as a bare filename, hash and
source link; the model card stayed empty until the user ran "Enrich
Metadata with AI" by hand. But everything that makes a CivitAI download
useful — the display name, the description, the tags, the trigger words,
the example images, the preview — is already published by those sites'
public APIs, so asking for it at download time is deterministic work, not
model work.
Add `py/services/model_sources/hydration.py`, called by
`_save_source_metadata()` once the sidecar exists and the file is in the
scanner cache. It fetches the model card plus the site's card extras and
hands them to the same `PostProcessor` the AI skill uses, with an empty
`llm_output`, so the two paths cannot drift apart. What lands:
* `model_name` from the site's own display name (ModelScope's `Name`), so
the card stops showing the local filename — written only while the value
still equals the file stem, since once a user renames a model that
choice is theirs to keep
* `civitai.name` from the matched version's label (`showName`), which the
card renders as the version chip
* `civitai.description` / `modelDescription` from the author summary plus
the README as HTML
* `civitai.images` / `preview_url` from the per-file example images
* `civitai.trainedWords` from the per-file trigger words
* `base_model`, `tags` and `usage_tips` as before
Provenance stays honest: the pass records
`metadata_source = "source:<platform>"` rather than the skill's
`agent:enrich_hf_metadata`, and — because no provider ran — it no longer
stamps `llm_enriched_at`; that stamp is now conditional on the LLM
actually answering, which is what the field means. The five hand-rolled
`civitai` dict merges in the post-processor collapse into one
`_merge_civitai()` helper.
Two guards keep it safe. Only a model whose stored
`source_platform`/`source_url` match the repository being downloaded is
updated, so a local file that merely shares a name never receives another
model's card; and a file already on disk is topped up too, which
back-fills models downloaded before this existed. READMEs and detail
payloads describe the repository rather than the file, so a short-lived
process-wide `ModelSourceCache` (300 s, 32 entries) keeps a batch over one
repository to two HTTP requests. Every failure is logged and swallowed:
hydration can never fail a download.
Fix the hash policy while here. `_save_source_metadata()` went straight to
`MetadataManager.create_default_metadata()`, bypassing the per-type
factory on the owning scanner, so a checkpoint paid a full SHA256 inside
the download request — `CheckpointScanner`/`OtherScanner` deliberately
record `hash_status="pending"` with an empty `sha256` for their multi-GB
files. Metadata is now created through `scanner._create_default_metadata()`.
Hydration copes with the empty hash: `_matching_versions()` falls back to
the repository basename, which is exactly what the download just wrote.
Report both post-transfer stages, which advance no byte counter and so
read as a stall: the bar sat at 100% showing `0 B/s` for the seconds spent
hashing and fetching. `_report_phase()` broadcasts
`{"status": "metadata", "stage": "indexing" | "source", "platform": ...}`,
and `LoadingManager` names the stage in the status line (keeping the batch
position), retitles the item line, replaces the dead speed figure and runs
a sheen over the bar. `stage`/`platform` are machine-readable; the wording
is localised in the frontend.
Finally, `modelscope.ai` is its own catalogue rather than an alias of
`modelscope.cn` — `referall13/EM1` exists only on `.ai` and
`jj3550945163/Krea-2-LORA` only on `.cn` — so its URLs were rejected with
"Invalid model URL format". Register it as `ModelScopeIntlSource`
(`platform="modelscope-ai"`, `msai:` group prefix, its own default
download directory) and derive every URL either deployment builds from a
per-class `base_url`. `modelscope.com` stays an alias of `.cn`, which is
what it redirects to. The frontend source table, the link dialog hints and
the docs mirror the split.
Verified against the live APIs: both reported `.ai` repositories list
their files, read their READMEs and yield name / version / base model /
trigger words / example images. Backend 3092 passed; frontend 1259 JS +
91 Vue passed. The nine locales carry the new progress copy in the next
commit.
442 lines
16 KiB
JavaScript
442 lines
16 KiB
JavaScript
import { translate } from '../utils/i18nHelpers.js';
|
|
import { formatFileSize } from '../utils/formatters.js';
|
|
import { getModelSource } from '../utils/modelSourceHelpers.js';
|
|
|
|
// Loading management
|
|
export class LoadingManager {
|
|
constructor() {
|
|
if (LoadingManager.instance) {
|
|
return LoadingManager.instance;
|
|
}
|
|
|
|
// Delay DOM creation until first use to ensure i18n is ready
|
|
this._initialized = false;
|
|
this.overlay = null;
|
|
this.loadingContent = null;
|
|
this.progressBar = null;
|
|
this.statusText = null;
|
|
this.cancelButton = null;
|
|
this.onCancelCallback = null;
|
|
this.detailsContainer = null;
|
|
|
|
LoadingManager.instance = this;
|
|
}
|
|
|
|
_ensureInitialized() {
|
|
if (this._initialized) return;
|
|
|
|
this.overlay = document.getElementById('loading-overlay');
|
|
|
|
if (!this.overlay) {
|
|
this.overlay = document.createElement('div');
|
|
this.overlay.id = 'loading-overlay';
|
|
this.overlay.style.display = 'none';
|
|
document.body.appendChild(this.overlay);
|
|
}
|
|
|
|
this.loadingContent = this.overlay.querySelector('.loading-content');
|
|
if (!this.loadingContent) {
|
|
this.loadingContent = document.createElement('div');
|
|
this.loadingContent.className = 'loading-content';
|
|
this.overlay.appendChild(this.loadingContent);
|
|
}
|
|
|
|
this.progressBar = this.loadingContent.querySelector('.progress-bar');
|
|
if (!this.progressBar) {
|
|
this.progressBar = document.createElement('div');
|
|
this.progressBar.className = 'progress-bar';
|
|
this.progressBar.setAttribute('role', 'progressbar');
|
|
this.progressBar.setAttribute('aria-valuemin', '0');
|
|
this.progressBar.setAttribute('aria-valuemax', '100');
|
|
this.progressBar.setAttribute('aria-valuenow', '0');
|
|
this.progressBar.style.width = '0%';
|
|
this.loadingContent.appendChild(this.progressBar);
|
|
}
|
|
|
|
this.statusText = this.loadingContent.querySelector('.loading-status');
|
|
if (!this.statusText) {
|
|
this.statusText = document.createElement('div');
|
|
this.statusText.className = 'loading-status';
|
|
this.loadingContent.appendChild(this.statusText);
|
|
}
|
|
|
|
this.cancelButton = this.loadingContent.querySelector('.loading-cancel');
|
|
if (!this.cancelButton) {
|
|
this.cancelButton = document.createElement('button');
|
|
this.cancelButton.className = 'loading-cancel secondary-btn';
|
|
this.cancelButton.style.display = 'none';
|
|
this.cancelButton.style.margin = 'var(--space-2) auto 0';
|
|
this.cancelButton.textContent = translate('common.actions.cancel', {}, 'Cancel');
|
|
this.loadingContent.appendChild(this.cancelButton);
|
|
}
|
|
|
|
this.cancelButton.onclick = () => {
|
|
if (this.onCancelCallback) {
|
|
this.onCancelCallback();
|
|
this.cancelButton.disabled = true;
|
|
this.cancelButton.textContent = translate('common.status.cancelling', {}, 'Cancelling...');
|
|
}
|
|
};
|
|
|
|
this._initialized = true;
|
|
}
|
|
|
|
show(message = 'Loading...', progress = 0) {
|
|
this._ensureInitialized();
|
|
this.overlay.style.display = 'flex';
|
|
this.setProgress(progress);
|
|
this.setStatus(message);
|
|
|
|
// Remove any existing details container
|
|
this.removeDetailsContainer();
|
|
}
|
|
|
|
hide() {
|
|
if (!this._initialized) return;
|
|
this.overlay.style.display = 'none';
|
|
this.reset();
|
|
this.removeDetailsContainer();
|
|
}
|
|
|
|
setProgress(percent) {
|
|
if (!this._initialized) return;
|
|
this.progressBar.style.width = `${percent}%`;
|
|
this.progressBar.setAttribute('aria-valuenow', percent);
|
|
}
|
|
|
|
setStatus(message) {
|
|
if (!this._initialized) return;
|
|
this.statusText.textContent = message;
|
|
}
|
|
|
|
reset() {
|
|
if (!this._initialized) return;
|
|
this.setProgress(0);
|
|
this.setStatus('');
|
|
this.removeDetailsContainer();
|
|
this.hideCancelButton();
|
|
this.progressBar.style.display = 'block';
|
|
}
|
|
|
|
showCancelButton(onCancel) {
|
|
this._ensureInitialized();
|
|
if (this.cancelButton) {
|
|
this.onCancelCallback = onCancel;
|
|
this.cancelButton.style.display = 'flex';
|
|
this.cancelButton.disabled = false;
|
|
this.cancelButton.textContent = translate('common.actions.cancel', {}, 'Cancel');
|
|
}
|
|
}
|
|
|
|
hideCancelButton() {
|
|
if (!this._initialized) return;
|
|
if (this.cancelButton) {
|
|
this.cancelButton.style.display = 'none';
|
|
this.onCancelCallback = null;
|
|
}
|
|
}
|
|
|
|
// Create a details container for enhanced progress display
|
|
createDetailsContainer() {
|
|
this._ensureInitialized();
|
|
// Remove existing container if any
|
|
this.removeDetailsContainer();
|
|
|
|
// Create new container
|
|
this.detailsContainer = document.createElement('div');
|
|
this.detailsContainer.className = 'progress-details-container';
|
|
|
|
// Insert after the main progress bar
|
|
if (this.loadingContent) {
|
|
this.loadingContent.appendChild(this.detailsContainer);
|
|
}
|
|
|
|
return this.detailsContainer;
|
|
}
|
|
|
|
// Remove details container
|
|
removeDetailsContainer() {
|
|
if (this.detailsContainer) {
|
|
this.detailsContainer.remove();
|
|
this.detailsContainer = null;
|
|
}
|
|
}
|
|
|
|
// Show enhanced progress for downloads
|
|
showDownloadProgress(totalItems = 1) {
|
|
this.show(translate('modals.download.status.preparing', {}, 'Preparing download...'), 0);
|
|
this.progressBar.style.display = 'none';
|
|
|
|
// Create details container
|
|
const detailsContainer = this.createDetailsContainer();
|
|
|
|
// Create current item progress
|
|
const currentItemContainer = document.createElement('div');
|
|
currentItemContainer.className = 'current-item-progress';
|
|
|
|
const currentItemLabel = document.createElement('div');
|
|
currentItemLabel.className = 'current-item-label';
|
|
currentItemLabel.textContent = translate('modals.download.progress.currentFile', {}, 'Current file:');
|
|
|
|
const currentItemBar = document.createElement('div');
|
|
currentItemBar.className = 'current-item-bar-container';
|
|
|
|
const currentItemProgress = document.createElement('div');
|
|
currentItemProgress.className = 'current-item-bar';
|
|
currentItemProgress.style.width = '0%';
|
|
|
|
const currentItemPercent = document.createElement('span');
|
|
currentItemPercent.className = 'current-item-percent';
|
|
currentItemPercent.textContent = '0%';
|
|
|
|
currentItemBar.appendChild(currentItemProgress);
|
|
currentItemContainer.appendChild(currentItemLabel);
|
|
currentItemContainer.appendChild(currentItemBar);
|
|
currentItemContainer.appendChild(currentItemPercent);
|
|
|
|
// Create overall progress elements if multiple items
|
|
let overallLabel = null;
|
|
if (totalItems > 1) {
|
|
overallLabel = document.createElement('div');
|
|
overallLabel.className = 'overall-progress-label';
|
|
overallLabel.textContent = `Overall progress (0/${totalItems} complete):`;
|
|
detailsContainer.appendChild(overallLabel);
|
|
}
|
|
|
|
// Add current item progress to container
|
|
detailsContainer.appendChild(currentItemContainer);
|
|
|
|
// Create transfer stats container
|
|
const transferStats = document.createElement('div');
|
|
transferStats.className = 'download-transfer-stats';
|
|
|
|
const bytesDetail = document.createElement('div');
|
|
bytesDetail.className = 'download-transfer-bytes';
|
|
bytesDetail.textContent = translate(
|
|
'modals.download.progress.transferredUnknown',
|
|
{},
|
|
'Transferred: --'
|
|
);
|
|
|
|
const speedDetail = document.createElement('div');
|
|
speedDetail.className = 'download-transfer-speed';
|
|
speedDetail.textContent = translate(
|
|
'modals.download.progress.speed',
|
|
{ speed: '--' },
|
|
'Speed: --'
|
|
);
|
|
|
|
transferStats.appendChild(bytesDetail);
|
|
transferStats.appendChild(speedDetail);
|
|
detailsContainer.appendChild(transferStats);
|
|
|
|
const formatMetricSize = (value) => {
|
|
if (value === undefined || value === null || isNaN(value)) {
|
|
return '--';
|
|
}
|
|
if (value < 1) {
|
|
return '0 B';
|
|
}
|
|
return formatFileSize(value);
|
|
};
|
|
|
|
const updateTransferStats = (metrics = {}) => {
|
|
const { bytesDownloaded, totalBytes, bytesPerSecond } = metrics;
|
|
|
|
if (bytesDetail) {
|
|
const formattedDownloaded = formatMetricSize(bytesDownloaded);
|
|
const formattedTotal = formatMetricSize(totalBytes);
|
|
|
|
if (formattedDownloaded === '--' && formattedTotal === '--') {
|
|
bytesDetail.textContent = translate(
|
|
'modals.download.progress.transferredUnknown',
|
|
{},
|
|
'Transferred: --'
|
|
);
|
|
} else if (formattedTotal === '--') {
|
|
bytesDetail.textContent = translate(
|
|
'modals.download.progress.transferredSimple',
|
|
{ downloaded: formattedDownloaded },
|
|
`Transferred: ${formattedDownloaded}`
|
|
);
|
|
} else {
|
|
bytesDetail.textContent = translate(
|
|
'modals.download.progress.transferred',
|
|
{ downloaded: formattedDownloaded, total: formattedTotal },
|
|
`Transferred: ${formattedDownloaded} / ${formattedTotal}`
|
|
);
|
|
}
|
|
}
|
|
|
|
if (speedDetail) {
|
|
const formattedSpeed = formatMetricSize(bytesPerSecond);
|
|
const displaySpeed = formattedSpeed === '--' ? '--' : `${formattedSpeed}/s`;
|
|
speedDetail.textContent = translate(
|
|
'modals.download.progress.speed',
|
|
{ speed: displaySpeed },
|
|
`Speed: ${displaySpeed}`
|
|
);
|
|
}
|
|
};
|
|
|
|
/**
|
|
* Describe a post-transfer stage in the status line.
|
|
*
|
|
* The byte counter stops as soon as the last byte lands, but the
|
|
* backend still hashes the file and reads the model site's API. Naming
|
|
* that work is what stops the bar looking frozen at 100%.
|
|
*/
|
|
const describeMetadataStage = (stage, platform) => {
|
|
if (stage === 'indexing') {
|
|
return translate(
|
|
'modals.download.progress.indexingFile',
|
|
{},
|
|
'Reading model file...'
|
|
);
|
|
}
|
|
const label = getModelSource(platform)?.label || platform || '';
|
|
return label
|
|
? translate(
|
|
'modals.download.progress.fetchingSourceMetadata',
|
|
{ source: label },
|
|
`Fetching metadata from ${label}...`
|
|
)
|
|
: translate(
|
|
'modals.download.progress.fetchingMetadata',
|
|
{},
|
|
'Fetching metadata...'
|
|
);
|
|
};
|
|
|
|
// Initialize transfer stats with empty data
|
|
updateTransferStats();
|
|
|
|
if (this.cancelButton) {
|
|
this.loadingContent.appendChild(this.cancelButton);
|
|
}
|
|
|
|
/**
|
|
* Update the progress UI.
|
|
*
|
|
* @param {number} currentProgress Percentage of the current item.
|
|
* @param {number} [currentIndex] Items finished so far.
|
|
* @param {string} [currentName] File being processed.
|
|
* @param {object} [metrics] Byte counters; only meaningful while
|
|
* transferring.
|
|
* @param {object} [phase] `{ phase: 'metadata', stage, platform }` once
|
|
* the transfer has finished, so the UI can show what is still running
|
|
* instead of a 0 B/s speed.
|
|
*/
|
|
return (
|
|
currentProgress,
|
|
currentIndex = 0,
|
|
currentName = '',
|
|
metrics = {},
|
|
phase = null
|
|
) => {
|
|
const isMetadata = phase?.phase === 'metadata';
|
|
|
|
// Update current item progress
|
|
currentItemProgress.style.width = `${currentProgress}%`;
|
|
currentItemPercent.textContent = `${Math.floor(currentProgress)}%`;
|
|
currentItemProgress.classList.toggle('is-indeterminate', isMetadata);
|
|
|
|
// Update current item label if name provided
|
|
if (currentName) {
|
|
currentItemLabel.textContent = isMetadata
|
|
? translate(
|
|
'modals.download.progress.metadata',
|
|
{ name: currentName },
|
|
`Metadata: ${currentName}`
|
|
)
|
|
: translate(
|
|
'modals.download.progress.downloading',
|
|
{ name: currentName },
|
|
`Downloading: ${currentName}`
|
|
);
|
|
}
|
|
|
|
// No bytes are moving any more, so report the stage instead of a
|
|
// rate that has dropped to zero.
|
|
if (isMetadata) {
|
|
updateTransferStats({ bytesDownloaded: metrics.bytesDownloaded, totalBytes: metrics.totalBytes });
|
|
const stageText = describeMetadataStage(phase.stage, phase.platform);
|
|
speedDetail.textContent = stageText;
|
|
// Keep the batch position visible; the status line is the one
|
|
// place a caller also writes to.
|
|
this.setStatus(
|
|
totalItems > 1
|
|
? `${Math.min(currentIndex + 1, totalItems)}/${totalItems}: ${stageText}`
|
|
: stageText
|
|
);
|
|
} else {
|
|
updateTransferStats(metrics);
|
|
}
|
|
|
|
// Update overall label if multiple items
|
|
if (totalItems > 1 && overallLabel) {
|
|
overallLabel.textContent = `Overall progress (${currentIndex}/${totalItems} complete):`;
|
|
|
|
// Calculate and update overall progress
|
|
const overallProgress = Math.floor((currentIndex + currentProgress / 100) / totalItems * 100);
|
|
this.setProgress(overallProgress);
|
|
} else {
|
|
// Single item, just update main progress
|
|
this.setProgress(currentProgress);
|
|
}
|
|
};
|
|
}
|
|
|
|
async showWithProgress(callback, options = {}) {
|
|
const { initialMessage = 'Processing...', completionMessage = 'Complete' } = options;
|
|
|
|
try {
|
|
this.show(initialMessage);
|
|
await callback(this);
|
|
this.setProgress(100);
|
|
this.setStatus(completionMessage);
|
|
await new Promise(resolve => setTimeout(resolve, 500));
|
|
} finally {
|
|
this.hide();
|
|
}
|
|
}
|
|
|
|
// Enhanced progress display without callback pattern
|
|
showEnhancedProgress(message = 'Processing...') {
|
|
this.show(message, 0);
|
|
|
|
// Return update functions
|
|
return {
|
|
updateProgress: (percent, currentItem = '', statusMessage = '') => {
|
|
this.setProgress(percent);
|
|
if (statusMessage) {
|
|
this.setStatus(statusMessage);
|
|
}
|
|
},
|
|
|
|
showCancelButton: (onCancel) => {
|
|
this.showCancelButton(onCancel);
|
|
},
|
|
|
|
complete: async (completionMessage = 'Complete') => {
|
|
this.setProgress(100);
|
|
this.setStatus(completionMessage);
|
|
await new Promise(resolve => setTimeout(resolve, 500));
|
|
this.hide();
|
|
}
|
|
};
|
|
}
|
|
|
|
showSimpleLoading(message = 'Loading...') {
|
|
this._ensureInitialized();
|
|
this.overlay.style.display = 'flex';
|
|
this.progressBar.style.display = 'none';
|
|
this.setStatus(message);
|
|
}
|
|
|
|
restoreProgressBar() {
|
|
if (!this._initialized) return;
|
|
this.progressBar.style.display = 'block';
|
|
}
|
|
}
|