A regular Refresh walked every configured root sequentially, so a full
25 TB drive delayed the roots behind it, and the dialog sat on "Checking
for changes..." at 0 % for the whole walk with no way to tell it was
working. On a cold external drive the walk itself dominates the cost, so
the fix is to overlap the drives and to show what the walk is doing.
Backend (py/services/model_scanner.py):
* Extract the per-root walk into the synchronous _walk_root_for_reconcile()
worker and merge its results on the event loop afterwards, in configured
root order: which business path wins a file reachable through several
roots must not depend on the order the workers happened to finish in.
* Group roots by device (_root_device_key: drive letter on Windows, st_dev
on POSIX) and run one worker per device. Roots sharing a device stay
sequential, so directory claims and the overlap dedup (#871, #1041) keep
their configured-order semantics; different devices run in parallel.
* Track walk progress per root (_ReconcileWalkTracker), weighted by each
root's cached entry count because the real file count is only known once
the walk ends. The bar splits into walk (0-50 %) and new-file (50-99 %)
phases so it never jumps backwards, and the walk broadcasts files seen,
active roots and the ETA counters.
* Replace the Windows case-insensitive fallback -- a scan of every cached
path per miss, i.e. O(files x cached) -- with a lazily built lower-cased
index (_CachedPathLookups, also now guarding the realpath alias index for
worker threads), and lift the os.name == "nt" gate into the module-level
_CASE_INSENSITIVE_PATHS so the branch is testable off Windows.
* Excluded-model membership is a set lookup instead of a list scan.
Frontend:
* render the walk phase as "Checking for changes... <roots> (N files)" with
the ETA, and reset the ETA tracker when the stage changes: the per-file
rate of counting files says nothing about processing them.
* add common.scanProgress.walkFiles; the other locales keep the sanctioned
[TODO: Translate] placeholder until the translation pass.
Verified: no-change reconcile over 10k files/500 dirs 101 ms and 10k/5000
dirs 230 ms (was 93/229 ms, within noise); a two-device sandbox walk runs
both roots concurrently, names them in the progress messages and finishes
with added=15, removed=0; 3665 passed, 7 skipped; frontend 1456 passed,
vue widgets 96 passed.
The Windows-only case-insensitive match in ModelScanner._reconcile_cache
is the only pass left unverified by the recent realpath cleanup: realpath
may already cover case differences on Windows, and if the branch is ever
reachable it is O(files x cache entries). Records the reachability
question, the verification steps for a Windows run, and the two possible
fixes.
Will Miao