diff --git a/.agents/skills/lora-manager-e2e/SKILL.md b/.agents/skills/lora-manager-e2e/SKILL.md deleted file mode 100644 index 1e264e76..00000000 --- a/.agents/skills/lora-manager-e2e/SKILL.md +++ /dev/null @@ -1,146 +0,0 @@ ---- -name: lora-manager-e2e -description: "End-to-end testing and validation for LoRa Manager features. Use ONLY for sandboxed E2E validation of LoRa Manager standalone mode: start the standalone server on a free port with --settings-path, drive the web UI (http://127.0.0.1:{PORT}/loras) via Chrome DevTools MCP, and verify frontend-to-backend integration. NOT for UI behavior checks that unit tests (Vitest/jsdom) can cover. Trigger keywords: E2E, standalone, Chrome DevTools MCP, lora-manager-e2e, sandbox." ---- - -# LoRa Manager E2E Testing - -End-to-end testing of LoRa Manager standalone mode using Chrome DevTools MCP. - -## When to Use — and When NOT To - -E2E runs are slow and token-heavy. Reach for them only when the question genuinely -spans server + browser (routing, scan persistence, websocket updates, EXIF writes). - -- **Default to unit/component tests first**: `npm run test:js` (Vitest/jsdom) covers - DOM rendering, modal behavior, event handling and API-client calls deterministically - in seconds. Backend logic goes through `pytest`. A UI-behavior question answered by - jsdom MUST NOT be escalated to E2E. -- **Use E2E only when** the behavior cannot be observed without a live server and a - real browser, e.g. template rendering through the aiohttp server, scanner → SQLite - persistence → API → DOM round-trips, or real EXIF/image writes. -- If you start an E2E and realize a unit test would answer the question, stop and - switch. - -**Browser driver is fixed: Chrome DevTools MCP.** Do not substitute kimi-webbridge — -it operates on the user's real browser (real tabs, real sessions, synthetic -`isTrusted=false` events), which breaks the isolation this skill requires and lacks -the console/network inspection E2E debugging relies on. kimi-webbridge is for -interactive browsing with the user's real login sessions, not for sandboxed E2E. - -## Conventions - -- **`{PORT}`**: default candidate `8188`, but it is **commonly occupied by a live - ComfyUI** — always check first (`ss -tlnp | grep ':{PORT}'`) and use a free port - (e.g. `8199`). Substitute the chosen port everywhere below. Never kill a process - you did not start for this E2E. -- **``**: the repository/worktree root; run all commands from there. -- **``**: a throwaway dir, e.g. `/tmp/opencode/-e2e`. - -## SANDBOX (MANDATORY) - -> Every E2E run MUST target a throwaway sandbox, never real user data. - -1. **Explicit settings directory**: always launch with `--settings-path /settings`. - This pins ALL runtime data (`settings.json`, `cache/`, `backups/`, `logs/`, `stats/`, - `wildcards/`) under the sandbox. **Never** create `/settings.json` — the repo - folder is usually the real ComfyUI plugin folder and a portable settings file there is - read by the real instance. -2. **Sandboxed library paths**: point `folder_paths` / `recipes_path` / - `example_images_path` at disposable dirs under `` — never the real library, - real recipe dir, or real settings: - - ```json - { - "folder_paths": { - "loras": ["/models/loras"], - "checkpoints": ["/models/checkpoints"], - "unet": ["/models/checkpoints"], - "diffusers": [] - }, - "recipes_path": "/recipes", - "example_images_path": "/example_images" - } - ``` - -3. **Real-data protection proof**: before starting and after finishing, snapshot the real - config and recipe library and confirm they are byte-identical; also confirm - `` gained no `settings.json` or `cache/`: - - ```bash - sha256sum ~/.config/ComfyUI-LoRA-Manager/settings.json > /settings.before.sha256 - ls ~/models/recipes/*.recipe.json 2>/dev/null | wc -l > /recipes-count.before.txt - # AFTER the run: record again and diff. Any change = the run leaked into real data. - ``` - -## Quick Start - -```bash -cd -# 1. Sandbox -mkdir -p /settings /models/{loras,checkpoints} /{recipes,example_images} -# write /settings/settings.json per the SANDBOX example -# 2. Port -ss -tlnp | grep ':{PORT}' || echo "port {PORT} is free" -# 3. Server — MUST be fully detached (a plain background & dies with the shell); -# the helper enforces this and manages its own pidfile -python .agents/skills/lora-manager-e2e/scripts/start_server.py \ - --port {PORT} --settings-path /settings --wait --timeout 30 --detach -ss -tlnp | grep ':{PORT}' # verify listening BEFORE proceeding -# 4. Chrome with remote debugging, then connect Chrome DevTools MCP (verify via list_pages) -google-chrome --remote-debugging-port=9222 --user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:{PORT}/loras -``` - -Then drive the UI with the MCP tools (`take_snapshot`, `click`, `fill`, `fill_form`, -`evaluate_script`, `wait_for`, `list_network_requests`, `list_console_messages`) — -see [references/mcp-cheatsheet.md](references/mcp-cheatsheet.md) for patterns. - -Server restart after config/fixture changes: - -```bash -python .agents/skills/lora-manager-e2e/scripts/start_server.py \ - --port {PORT} --settings-path /settings --restart --wait --detach -# then reload the browser page (ignoreCache=True) -``` - -`--restart` only kills the E2E server the script itself started (via its pidfile) and -aborts instead of killing unrelated processes on the port. - -## Abort Rule - -A sandboxed E2E should finish in well under 30 minutes. If any phase exceeds ~2x its -expected duration (server readiness > 60 s, MCP connect > 2 min, a single scenario > -10 min), or any single tool call fails 3+ times in a row, **STOP** — do not retry -blindly. Report `BLOCKED` with the phase, last observed state (server PID, -`ss -tlnp` output, page snapshot, last API response) and suspected cause. A clean -BLOCKED report beats an hour of retries. - -## Troubleshooting - -- **"browser is already running" / `list_pages` fails**: a stale Chrome holds the - profile dir. Find it (`ps -ef | grep -i '[c]hrome.*user-data-dir'`), confirm it is a - leftover QA Chrome (not the live ComfyUI, not your current MCP browser), kill only - that PID, then retry `list_pages`. -- **MCP refuses to write screenshots into the worktree**: save to `/tmp` via - `take_screenshot(filePath="/tmp/...")` and copy into the evidence dir from the shell. - -## Cleanup - -1. Stop the standalone server: `kill ` (only the PID you started), then - confirm `ss -tlnp | grep ':{PORT}'` is empty. -2. Close browser pages (keep at least one open). -3. `rm -rf `; verify `` gained no `settings.json` or `cache/`. -4. Re-run the real-data protection check from the SANDBOX section and record the result. - -## References & Scripts - -- [references/mcp-cheatsheet.md](references/mcp-cheatsheet.md) — Chrome DevTools MCP - command patterns (navigation, waiting, snapshots, forms, network, console, performance). -- [references/test-scenarios.md](references/test-scenarios.md) — detailed test scenarios - (list display, metadata editing, recipes, settings, import/export). -- [references/recipe-rematch-fixtures.md](references/recipe-rematch-fixtures.md) — - fixture format, fresh-state reset and known gaps for recipe rematch/repair E2E runs. -- `scripts/start_server.py` — start/restart the standalone server - (`--port --settings-path --restart --wait --timeout --detach`); refuses to touch - unrelated processes on the port. -- `scripts/wait_for_server.py` — poll readiness (`--port --timeout`). diff --git a/.agents/skills/lora-manager-e2e/references/mcp-cheatsheet.md b/.agents/skills/lora-manager-e2e/references/mcp-cheatsheet.md deleted file mode 100644 index 2f92035d..00000000 --- a/.agents/skills/lora-manager-e2e/references/mcp-cheatsheet.md +++ /dev/null @@ -1,360 +0,0 @@ -# Chrome DevTools MCP Cheatsheet for LoRa Manager - -Quick reference for common MCP commands used in LoRa Manager E2E testing. - -> **Port convention**: `{PORT}` is the port chosen for the E2E run (default candidate `8188`, but only if actually free — see the SKILL.md Port Selection section; use e.g. `8199` when `8188` is occupied by a live ComfyUI). Always run against the **sandboxed** standalone server, never a live instance. - -## Navigation - -```python -# Navigate to LoRA list page -navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras") - -# Reload page with cache clear -navigate_page(type="reload", ignoreCache=True) - -# Go back/forward -navigate_page(type="back") -navigate_page(type="forward") -``` - -## Waiting - -```python -# Wait for text to appear -wait_for(text="LoRAs", timeout=10000) - -# Wait for specific element (via evaluate_script) -evaluate_script(function=""" -() => { - return new Promise((resolve) => { - const check = () => { - if (document.querySelector('.lora-card')) { - resolve(true); - } else { - setTimeout(check, 100); - } - }; - check(); - }); -} -""") -``` - -## Taking Snapshots - -```python -# Full page snapshot -snapshot = take_snapshot() - -# Verbose snapshot (more details) -snapshot = take_snapshot(verbose=True) - -# Save to file -take_snapshot(filePath="test-snapshots/page-load.json") -``` - -## Element Interaction - -```python -# Click element -click(uid="element-uid-from-snapshot") - -# Double click -click(uid="element-uid", dblClick=True) - -# Fill input -fill(uid="search-input", value="test query") - -# Fill multiple inputs -fill_form(elements=[ - {"uid": "input-1", "value": "value 1"}, - {"uid": "input-2", "value": "value 2"}, -]) - -# Hover -hover(uid="lora-card-1") - -# Upload file -upload_file(uid="file-input", filePath="/path/to/file.safetensors") -``` - -## Keyboard Input - -```python -# Press key -press_key(key="Enter") -press_key(key="Escape") -press_key(key="Tab") - -# Keyboard shortcuts -press_key(key="Control+A") # Select all -press_key(key="Control+F") # Find -``` - -## JavaScript Evaluation - -```python -# Simple evaluation -result = evaluate_script(function="() => document.title") - -# Async evaluation -result = evaluate_script(function=""" -async () => { - const response = await fetch('/loras/api/list'); - return await response.json(); -} -""") - -# Check element existence -exists = evaluate_script(function=""" -() => document.querySelector('.lora-card') !== null -""") - -# Get element count -count = evaluate_script(function=""" -() => document.querySelectorAll('.lora-card').length -""") -``` - -## Network Monitoring - -```python -# List all network requests -requests = list_network_requests() - -# Filter by resource type -xhr_requests = list_network_requests(resourceTypes=["xhr", "fetch"]) - -# Get specific request details -details = get_network_request(reqid=123) - -# Include preserved requests from previous navigations -all_requests = list_network_requests(includePreservedRequests=True) -``` - -## Console Monitoring - -```python -# List all console messages -messages = list_console_messages() - -# Filter by type -errors = list_console_messages(types=["error", "warn"]) - -# Include preserved messages -all_messages = list_console_messages(includePreservedMessages=True) - -# Get specific message -details = get_console_message(msgid=1) -``` - -## Performance Testing - -```python -# Start trace with page reload -performance_start_trace(reload=True, autoStop=False) - -# Start trace without reload -performance_start_trace(reload=False, autoStop=True, filePath="trace.json.gz") - -# Stop trace -results = performance_stop_trace() - -# Stop and save -performance_stop_trace(filePath="trace-results.json.gz") - -# Analyze specific insight -insight = performance_analyze_insight( - insightSetId="results.insightSets[0].id", - insightName="LCPBreakdown" -) -``` - -## Page Management - -```python -# List open pages -pages = list_pages() - -# Select a page -select_page(pageId=0, bringToFront=True) - -# Create new page -new_page(url="http://127.0.0.1:{PORT}/loras") - -# Close page (keep at least one open!) -close_page(pageId=1) - -# Resize page -resize_page(width=1920, height=1080) -``` - -## Screenshots - -```python -# Full page screenshot -take_screenshot(fullPage=True) - -# Viewport screenshot -take_screenshot() - -# Element screenshot -take_screenshot(uid="lora-card-1") - -# Save to file -take_screenshot(filePath="screenshots/page.png", format="png") - -# JPEG with quality -take_screenshot(filePath="screenshots/page.jpg", format="jpeg", quality=90) -``` - -## Dialog Handling - -```python -# Accept dialog -handle_dialog(action="accept") - -# Accept with text input -handle_dialog(action="accept", promptText="user input") - -# Dismiss dialog -handle_dialog(action="dismiss") -``` - -## Device Emulation - -```python -# Mobile viewport -emulate(viewport={"width": 375, "height": 667, "isMobile": True, "hasTouch": True}) - -# Tablet viewport -emulate(viewport={"width": 768, "height": 1024, "isMobile": True, "hasTouch": True}) - -# Desktop viewport -emulate(viewport={"width": 1920, "height": 1080}) - -# Network throttling -emulate(networkConditions="Slow 3G") -emulate(networkConditions="Fast 4G") - -# CPU throttling -emulate(cpuThrottlingRate=4) # 4x slowdown - -# Geolocation -emulate(geolocation={"latitude": 37.7749, "longitude": -122.4194}) - -# User agent -emulate(userAgent="Mozilla/5.0 (Custom)") - -# Reset emulation -emulate(viewport=None, networkConditions="No emulation", userAgent=None) -``` - -## Drag and Drop - -```python -# Drag element to another -drag(from_uid="draggable-item", to_uid="drop-zone") -``` - -## Common LoRa Manager Test Patterns - -### Verify LoRA Cards Loaded - -```python -navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras") -wait_for(text="LoRAs", timeout=10000) - -# Check if cards loaded -result = evaluate_script(function=""" -() => { - const cards = document.querySelectorAll('.lora-card'); - return { - count: cards.length, - hasData: cards.length > 0 - }; -} -""") -``` - -### Search and Verify Results - -```python -fill(uid="search-input", value="character") -press_key(key="Enter") -wait_for(timeout=2000) # Wait for debounce - -# Check results -result = evaluate_script(function=""" -() => { - const cards = document.querySelectorAll('.lora-card'); - const names = Array.from(cards).map(c => c.dataset.name || c.textContent); - return { count: cards.length, names }; -} -""") -``` - -### Check API Response - -```python -# Trigger API call -evaluate_script(function=""" -() => window.loraApiCallPromise = fetch('/loras/api/list').then(r => r.json()) -""") - -# Wait and get result -import time -time.sleep(1) - -result = evaluate_script(function=""" -async () => await window.loraApiCallPromise -""") -``` - -### Monitor Console for Errors - -```python -# Before test: clear console (navigate reloads) -navigate_page(type="reload") - -# ... perform actions ... - -# Check for errors -errors = list_console_messages(types=["error"]) -assert len(errors) == 0, f"Console errors: {errors}" -``` - -## Troubleshooting - -### Stale profile lock ("browser is already running" / `list_pages` fails) - -A Chrome profile held by a stale Chrome from a prior MCP session makes `list_pages` -fail with "browser is already running". Fix: - -1. Find the stale Chrome that owns the profile dir (e.g. `~/.config/chrome-dev-profile`): - ```bash - ps -ef | grep -i '[c]hrome.*user-data-dir' - ``` -2. Confirm it is a QA Chrome from a completed task (NOT the live ComfyUI server, NOT - your current MCP instance). -3. Kill ONLY that stale Chrome (`kill `), then retry `list_pages`. - -### Screenshot-write restrictions - -The MCP may refuse to write into paths outside its configured workspace roots -(e.g. `.omo/evidence/screenshots/` under a worktree that canonicalizes to an unmapped -path). Save the screenshot to `/tmp` via the MCP, then copy it into the evidence dir: - -```bash -# MCP: take_screenshot(filePath="/tmp/-e2e/recipe-b-after.png", format="png") -# Shell: -mkdir -p /.omo/evidence/screenshots -cp /tmp/-e2e/recipe-b-after.png /.omo/evidence/screenshots/ -``` - -### Time budgets & abort rule - -See SKILL.md "Time Budgets & Abort Guidance": if a phase exceeds ~2x its budget or a -tool call retries 3+ times in a row, STOP and report BLOCKED with the last observed -state (server PID + `ss -tlnp`, page snapshot, last API response). Do not loop. diff --git a/.agents/skills/lora-manager-e2e/references/recipe-rematch-fixtures.md b/.agents/skills/lora-manager-e2e/references/recipe-rematch-fixtures.md deleted file mode 100644 index 2e95e39c..00000000 --- a/.agents/skills/lora-manager-e2e/references/recipe-rematch-fixtures.md +++ /dev/null @@ -1,72 +0,0 @@ -# Recipe Rematch/Repair E2E — Fixtures, Fresh State, Known Gaps - -Specialized guidance for recipe rematch/repair E2E runs, extracted from the SKILL.md -main flow. Read the SKILL.md SANDBOX section first — everything here assumes a -sandboxed run. - -## Fixture Rules (validated by the task-8 E2E) - -Seed the **sandboxed** `recipes_path` with hand-written fixture recipes: - -1. **Filename constraint**: each file MUST be named `f"{id}.recipe.json"` **and** the - in-JSON `id` field MUST equal the filename. Discovery accepts any `*.recipe.json`, - but persistence resolves the path via `get_recipe_json_path` and - `_save_recipe_persistently` returns `False` on a mismatch → the fixture would be - counted as an error. - - `recipe-a.recipe.json` → in-JSON `"id": "recipe-a"` -2. **File format**: mirror an existing recipe JSON — top-level `id`, `file_path`, - `title`, `loras`, `fingerprint`, `gen_params`; lora entries per the persistence - conventions (`hash`, `file_name`, `modelVersionId`, `isDeleted`, ...). -3. **Companion image**: each recipe needs an image (e.g. a `.webp` generated with PIL) - referenced by `file_path`, used for EXIF verification - (`ExifUtils.append_recipe_metadata` writes a `"Recipe metadata: ..."` marker; a - freshly generated `.webp` with no marker is the clean "untouched" control). -4. **autov3 three-state contract**: for L3 (autov3-only, renamed-file) fixtures the - local model's `.metadata.json` sidecar MUST have the `autov3` key **ABSENT** (the - "unchecked" state), NOT `""` — `""` is the TERMINAL "checked but unavailable" state - that L3 deliberately skips. The scanner computes + persists `autov3` from the file - header during the normal library scan (`model_scanner.py` `_process_model_file`), so - the live L3 match resolves through the local autov3/hash cache; the - computed-autov3 branch for unchecked items is covered by the unit suite. -5. **Fixture design for a rematch run** (mirrors the task-8 E2E): - - `recipe-a`: lora entry `isDeleted=True`, `hash` = 12-char autov3 computed from the - local model (`calculate_autov3`, `py/utils/file_utils.py`), whose local model file - was RENAMED after the recipe was written so `file_name` differs (proves L3 match - without filename). - - `recipe-b`: parser-convention checkpoint entry (uses `id`, no `modelVersionId`) - matching a local checkpoint via L2 — the local checkpoint's `.metadata.json` MUST - carry civitai version data with that `id` so `version_index` contains it (L2 - cannot match otherwise). - - `recipe-c`: healthy recipe (no deleted entries) → must remain untouched. - -The scanner computes and persists model hashes during the library scan, so the sandbox -model dirs just need the model files + `.metadata.json` sidecars. With -`--settings-path`, all derived data lands under the sandbox settings dir (`cache/`, -`backups/`, `logs/`, `stats/`, `wildcards/`), and NO `cache/` appears in the repo root. - -## Fresh State Between Entry-Point Runs - -Each entry point (global / per-recipe / selection-bulk) must start from the same -deleted state. Between runs (keep a pristine copy in `/recipes-before/`): - -```bash -# 1. Reset fixtures to the before-state snapshot -cp /recipes-before/*.recipe.json /recipes/ -# 2. Clear the recipe/FTS caches (with --settings-path these live under the sandbox -# settings dir, NOT /cache) -rm -f /settings/cache/recipe/*.sqlite -rm -rf /settings/cache/fts/* -# 3. Restart the server (fresh process, fresh scan) -python .agents/skills/lora-manager-e2e/scripts/start_server.py \ - --port {PORT} --settings-path /settings --restart --wait --timeout 30 --detach -# 4. Re-verify the server is listening + reload the browser page -``` - -## Cancellation Testing (KNOWN GAP) - -Testing the rematch-cancel path E2E requires a run long enough to cancel mid-flight. A -tiny 3-recipe fixture set completes in **seconds** — too fast to reliably cancel. The -cancel path is currently **unit-covered only** (`rematch_all_recipes` cancellation -tests); do not block an E2E run on cancel-path verification. If you must attempt it, -you would need an artificially large/deferred fixture set to create a cancellable -window — treat this as a research task, not part of the standard E2E. diff --git a/.agents/skills/lora-manager-e2e/references/test-scenarios.md b/.agents/skills/lora-manager-e2e/references/test-scenarios.md deleted file mode 100644 index 1ddcc6f0..00000000 --- a/.agents/skills/lora-manager-e2e/references/test-scenarios.md +++ /dev/null @@ -1,280 +0,0 @@ -# LoRa Manager E2E Test Scenarios - -This document provides detailed test scenarios for end-to-end validation of LoRa Manager features. - -> **Run preconditions (from SKILL.md)**: every run uses the **sandboxed** standalone -> server on a free port `{PORT}` (default candidate `8188`, only if actually free — pick -> e.g. `8199` when `8188` is occupied by a live ComfyUI). Fixtures live in the sandboxed -> `recipes_path` as `f"{id}.recipe.json"` files with matching in-JSON `id`; the real user -> config and real library are never touched (record protection proof before/after). -> Abort if a phase exceeds ~2x its budget or a tool call retries 3+ times (SKILL.md -> "Time Budgets & Abort Guidance"). - -## Table of Contents - -1. [LoRA List Page](#lora-list-page) -2. [Model Details](#model-details) -3. [Recipes](#recipes) -4. [Settings](#settings) -5. [Import/Export](#importexport) - ---- - -## LoRA List Page - -### Scenario: Page Load and Display - -**Objective**: Verify the LoRA list page loads correctly and displays models. - -**Steps**: -1. Navigate to `http://127.0.0.1:{PORT}/loras` -2. Wait for page title "LoRAs" to appear -3. Take snapshot to verify: - - Header with "LoRAs" title is visible - - Search/filter controls are present - - Grid/list view toggle exists - - LoRA cards are displayed (if models exist) - - Pagination controls (if applicable) - -**Expected Result**: Page loads without errors, UI elements are present. - -### Scenario: Search Functionality - -**Objective**: Verify search filters LoRA models correctly. - -**Steps**: -1. Ensure at least one LoRA exists with known name (e.g., "test-character") -2. Navigate to LoRA list page -3. Enter search term in search box: "test" -4. Press Enter or click search button -5. Wait for results to update - -**Expected Result**: Only LoRAs matching search term are displayed. - -**Verification Script**: -```python -# After search, verify filtered results -evaluate_script(function=""" -() => { - const cards = document.querySelectorAll('.lora-card'); - const names = Array.from(cards).map(c => c.dataset.name); - return { count: cards.length, names }; -} -""") -``` - -### Scenario: Filter by Tags - -**Objective**: Verify tag filtering works correctly. - -**Steps**: -1. Navigate to LoRA list page -2. Click on a tag (e.g., "character", "style") -3. Wait for filtered results - -**Expected Result**: Only LoRAs with selected tag are displayed. - -### Scenario: View Mode Toggle - -**Objective**: Verify grid/list view toggle works. - -**Steps**: -1. Navigate to LoRA list page -2. Click list view button -3. Verify list layout -4. Click grid view button -5. Verify grid layout - -**Expected Result**: View mode changes correctly, layout updates. - ---- - -## Model Details - -### Scenario: Open Model Details - -**Objective**: Verify clicking a LoRA opens its details. - -**Steps**: -1. Navigate to LoRA list page -2. Click on a LoRA card -3. Wait for details panel/modal to open - -**Expected Result**: Details panel shows: -- Model name -- Preview image -- Metadata (trigger words, tags, etc.) -- Action buttons (edit, delete, etc.) - -### Scenario: Edit Model Metadata - -**Objective**: Verify metadata editing works end-to-end. - -**Steps**: -1. Open a LoRA's details -2. Click "Edit" button -3. Modify trigger words field -4. Add/remove tags -5. Save changes -6. Refresh page -7. Reopen the same LoRA - -**Expected Result**: Changes persist after refresh. - -### Scenario: Delete Model - -**Objective**: Verify model deletion works. - -**Steps**: -1. Open a LoRA's details -2. Click "Delete" button -3. Confirm deletion in dialog -4. Wait for removal - -**Expected Result**: Model removed from list, success message shown. - ---- - -## Recipes - -### Scenario: Recipe List Display - -**Objective**: Verify recipes page loads and displays recipes. - -**Steps**: -1. Navigate to `http://127.0.0.1:{PORT}/recipes` -2. Wait for "Recipes" title -3. Take snapshot - -**Expected Result**: Recipe list displayed with cards/items. - -### Scenario: Create New Recipe - -**Objective**: Verify recipe creation workflow. - -**Steps**: -1. Navigate to recipes page -2. Click "New Recipe" button -3. Fill recipe form: - - Name: "Test Recipe" - - Description: "E2E test recipe" - - Add LoRA models -4. Save recipe -5. Verify recipe appears in list - -**Expected Result**: New recipe created and displayed. - -### Scenario: Apply Recipe - -**Objective**: Verify applying a recipe to ComfyUI. - -**Steps**: -1. Open a recipe -2. Click "Apply" or "Load in ComfyUI" -3. Verify action completes - -**Expected Result**: Recipe applied successfully. - ---- - -## Settings - -### Scenario: Settings Page Load - -**Objective**: Verify settings page displays correctly. - -**Steps**: -1. Navigate to `http://127.0.0.1:{PORT}/settings` -2. Wait for "Settings" title -3. Take snapshot - -**Expected Result**: Settings form with various options displayed. - -### Scenario: Change Setting and Restart - -**Objective**: Verify settings persist after restart. - -**Steps**: -1. Navigate to settings page -2. Change a setting (e.g., default view mode) -3. Save settings -4. Restart server: `python scripts/start_server.py --port {PORT} --restart --wait --timeout 30 --detach` -5. Refresh browser page -6. Navigate to settings - -**Expected Result**: Changed setting value persists. - ---- - -## Import/Export - -### Scenario: Export Models List - -**Objective**: Verify export functionality. - -**Steps**: -1. Navigate to LoRA list -2. Click "Export" button -3. Select format (JSON/CSV) -4. Download file - -**Expected Result**: File downloaded with correct data. - -### Scenario: Import Models - -**Objective**: Verify import functionality. - -**Steps**: -1. Prepare import file -2. Navigate to import page -3. Upload file -4. Verify import results - -**Expected Result**: Models imported successfully, confirmation shown. - ---- - -## API Integration Tests - -### Scenario: Verify API Endpoints - -**Objective**: Verify backend API responds correctly. - -**Test via browser console**: -```javascript -// List LoRAs -fetch('/loras/api/list').then(r => r.json()).then(console.log) - -// Get LoRA details -fetch('/loras/api/detail/').then(r => r.json()).then(console.log) - -// Search LoRAs -fetch('/loras/api/search?q=test').then(r => r.json()).then(console.log) -``` - -**Expected Result**: APIs return valid JSON with expected structure. - ---- - -## Console Error Monitoring - -During all tests, monitor browser console for errors: - -```python -# Check for JavaScript errors -messages = list_console_messages(types=["error"]) -assert len(messages) == 0, f"Console errors found: {messages}" -``` - -## Network Request Verification - -Verify key API calls are made: - -```python -# List XHR requests -requests = list_network_requests(resourceTypes=["xhr", "fetch"]) - -# Look for specific endpoints -lora_list_requests = [r for r in requests if "/api/list" in r.get("url", "")] -assert len(lora_list_requests) > 0, "LoRA list API not called" -``` diff --git a/.agents/skills/lora-manager-e2e/scripts/example_e2e_test.py b/.agents/skills/lora-manager-e2e/scripts/example_e2e_test.py deleted file mode 100755 index d3519bbe..00000000 --- a/.agents/skills/lora-manager-e2e/scripts/example_e2e_test.py +++ /dev/null @@ -1,215 +0,0 @@ -#!/usr/bin/env python3 -""" -Example E2E test demonstrating LoRa Manager testing workflow. - -This script shows how to: -1. Start the standalone server -2. Use Chrome DevTools MCP to interact with the UI -3. Verify functionality end-to-end - -Note: This is a template. Actual execution requires Chrome DevTools MCP. - -Port: pick a FREE port for the run — 8188 is commonly occupied by a live -ComfyUI (see the skill's Port Selection section). Set PORT below to e.g. 8199 -when 8188 is taken. Always run against a SANDBOXED standalone server. -""" - -import subprocess -import sys - -# Choose the E2E port. 8188 is only the default candidate; use 8199 (or any -# free port checked with `ss -tlnp`) when 8188 is occupied by a live ComfyUI. -PORT = "8188" - - -def run_test(): - """Run example E2E test flow.""" - - print("=" * 60) - print("LoRa Manager E2E Test Example") - print("=" * 60) - - # Step 1: Start server (detached so it survives the shell) - print("\n[1/5] Starting LoRa Manager standalone server...") - result = subprocess.run( - [sys.executable, "start_server.py", "--port", PORT, "--wait", "--timeout", "30", "--detach"], - capture_output=True, - text=True, - ) - if result.returncode != 0: - print(f"Failed to start server: {result.stderr}") - return 1 - print("Server ready!") - - # Step 2: Open Chrome (manual step - show command) - print("\n[2/5] Open Chrome with debug mode:") - print( - f"google-chrome --remote-debugging-port=9222 " - f"--user-data-dir=/tmp/chrome-lora-manager http://127.0.0.1:{PORT}/loras" - ) - print("(In actual test, this would be automated via MCP)") - - # Step 3: Navigate and verify page load - print("\n[3/5] Page Load Verification:") - print( - f""" - MCP Commands to execute: - 1. navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras") - 2. wait_for(text="LoRAs", timeout=10000) - 3. snapshot = take_snapshot() - """ - ) - - # Step 4: Test search functionality - print("\n[4/5] Search Functionality Test:") - print( - """ - MCP Commands to execute: - 1. fill(uid="search-input", value="test") - 2. press_key(key="Enter") - 3. wait_for(text="Results", timeout=5000) - 4. result = evaluate_script(function=` - () => { - const cards = document.querySelectorAll('.lora-card'); - return { count: cards.length }; - } - `) - """ - ) - - # Step 5: Verify API - print("\n[5/5] API Verification:") - print( - """ - MCP Commands to execute: - 1. api_result = evaluate_script(function=` - async () => { - const response = await fetch('/loras/api/list'); - const data = await response.json(); - return { count: data.length, status: response.status }; - } - `) - 2. Verify api_result['status'] == 200 - """ - ) - - print("\n" + "=" * 60) - print("Test flow completed!") - print("=" * 60) - - return 0 - - -def example_restart_flow(): - """Example: Testing configuration change that requires restart.""" - - print("\n" + "=" * 60) - print("Example: Server Restart Flow") - print("=" * 60) - - print( - f""" - Scenario: Change setting and verify after restart - - Steps: - 1. Navigate to settings page - - navigate_page(type="url", url="http://127.0.0.1:{PORT}/settings") - - 2. Change a setting (e.g., theme) - - fill(uid="theme-select", value="dark") - - click(uid="save-settings-button") - - 3. Restart server - - subprocess.run([python, "start_server.py", "--port", "{PORT}", "--restart", "--wait", "--detach"]) - - 4. Refresh browser - - navigate_page(type="reload", ignoreCache=True) - - wait_for(text="LoRAs", timeout=15000) - - 5. Verify setting persisted - - navigate_page(type="url", url="http://127.0.0.1:{PORT}/settings") - - theme = evaluate_script(function="() => document.querySelector('#theme-select').value") - - assert theme == "dark" - """ - ) - - -def example_modal_interaction(): - """Example: Testing modal dialog interaction.""" - - print("\n" + "=" * 60) - print("Example: Modal Dialog Interaction") - print("=" * 60) - - print( - """ - Scenario: Add new LoRA via modal - - Steps: - 1. Open modal - - click(uid="add-lora-button") - - wait_for(text="Add LoRA", timeout=3000) - - 2. Fill form - - fill_form(elements=[ - {"uid": "lora-name", "value": "Test Character"}, - {"uid": "lora-path", "value": "/models/test.safetensors"}, - ]) - - 3. Submit - - click(uid="modal-submit-button") - - 4. Verify success - - wait_for(text="Successfully added", timeout=5000) - - snapshot = take_snapshot() - """ - ) - - -def example_network_monitoring(): - """Example: Network request monitoring.""" - - print("\n" + "=" * 60) - print("Example: Network Request Monitoring") - print("=" * 60) - - print( - f""" - Scenario: Verify API calls during user interaction - - Steps: - 1. Clear network log (implicit on navigation) - - navigate_page(type="url", url="http://127.0.0.1:{PORT}/loras") - - 2. Perform action that triggers API call - - fill(uid="search-input", value="character") - - press_key(key="Enter") - - 3. List network requests - - requests = list_network_requests(resourceTypes=["xhr", "fetch"]) - - 4. Find search API call - - search_requests = [r for r in requests if "/api/search" in r.get("url", "")] - - assert len(search_requests) > 0, "Search API was not called" - - 5. Get request details - - if search_requests: - details = get_network_request(reqid=search_requests[0]["reqid"]) - - Verify request method, response status, etc. - """ - ) - - -if __name__ == "__main__": - print("LoRa Manager E2E Test Examples\n") - print("This script demonstrates E2E testing patterns.\n") - print("Note: Actual execution requires Chrome DevTools MCP connection.\n") - - run_test() - example_restart_flow() - example_modal_interaction() - example_network_monitoring() - - print("\n" + "=" * 60) - print("All examples shown!") - print("=" * 60) diff --git a/AGENTS.md b/AGENTS.md index 5db5eae7..e9b3a604 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -215,6 +215,26 @@ The system runs in two modes: - Vanilla JS tests: `tests/frontend/**/*.test.js` with jsdom; setup in `tests/frontend/setup.js` - Vue widget tests: `vue-widgets/tests/**/*.test.ts` with jsdom + `@vue/test-utils` +### UI Verification (manual default) + +UI/layout changes are verified by the user by eye — do NOT spin up a sandbox, +standalone server, or browser automation to "prove" a visual fix. Ask the user to +look instead. The full browser E2E ceremony (server + Chrome DevTools MCP + +screenshots) is slow, token-heavy, and fragile; reserve it for genuine +server+browser integration bugs, and only when the user explicitly agrees. + +If a cross-layer issue ever needs a live server, the sandboxed helpers live in +`scripts/e2e/` (`start_server.py`, `wait_for_server.py`). Non-negotiable rules: + +- Always launch with `--settings-path /settings` and sandboxed + `folder_paths` under `/tmp` — the repo folder is the real plugin folder and a + `settings.json` there is read by the live instance. Never touch real config or + real model libraries. +- Never kill a process you did not start; `start_server.py` tracks its own PIDs + via pidfile and refuses to touch unrelated processes on the port. +- Abort after ~30 minutes or 3 consecutive tool failures; report `BLOCKED` with + observed state instead of retrying blindly. Clean up sandbox and server after. + ## Key Integration Points - **Settings:** Stored in the user config directory (via `platformdirs`) or portable mode (`"use_portable_settings": true`) diff --git a/.agents/skills/lora-manager-e2e/scripts/start_server.py b/scripts/e2e/start_server.py similarity index 100% rename from .agents/skills/lora-manager-e2e/scripts/start_server.py rename to scripts/e2e/start_server.py diff --git a/.agents/skills/lora-manager-e2e/scripts/wait_for_server.py b/scripts/e2e/wait_for_server.py similarity index 100% rename from .agents/skills/lora-manager-e2e/scripts/wait_for_server.py rename to scripts/e2e/wait_for_server.py