Pre-M3 Performance

Item 1: Web Worker Streaming Execution

Design

New file: src/workers/worker-manager.ts

Singleton class owning Worker lifecycle and orchestrating execution.

• Instantiate Worker: new Worker(new URL('./stats.worker.ts', import.meta.url), { type: 'module' })
• Called lazily on first use (any public method calls init() if not yet initialized)

• Posts LOAD_DATASET { csv } to Worker
• Worker parses CSV via PapaParse, stores dataset, responds with DATASET_LOADED { dataset }
• Resolves with the parsed Dataset for main-thread store state (column names, row counts, UI display)

• Runs topologicalSort() on main thread (reuse exported function from executor.ts)
• Builds edge lookup (same logic as current executePipeline)
• For each node in topological order:
  1. Collect input data from already-completed nodes (tracked locally in a Map<string, unknown>)
  2. Post EXECUTE_NODE { node, inputData } to Worker
  3. Await NODE_COMPLETE or NODE_ERROR response
  4. On complete: store result locally, call onNodeComplete(nodeId, result)
  5. On error: store error, call onNodeError(nodeId, error), mark dependents as upstream-error (skip them)
• Resolves when all nodes are processed

• Kill the Worker

Data Flow

CSV loading (currently freezes UI):

Pipeline execution (currently synchronous):

Changes to src/ui/store/data.ts

loadCSVFile() delegates CSV parsing to the Worker. No more parseCSV() call on main thread.

Changes to src/ui/store/pipeline.ts

runPipeline(dataset) becomes async:

1. Mark all nodes as running, set executing: true (unchanged)
2. Call workerManager.executePipeline() with streaming callbacks:
   • onComplete(nodeId, result) → set() to update that node to complete
   • onError(nodeId, error) → set() to update that node to error
3. When promise resolves, set executing: false

Changes to src/core/pipeline/executor.ts

• Export topologicalSort() (currently module-private) for WorkerManager to reuse
• Synchronous executePipeline() stays as-is — used by Vitest tests and as Worker fallback

No changes to Worker files

The existing protocol and Worker implementation already support exactly this flow. LOAD_DATASET accepts CSV string, parses it, responds with Dataset. EXECUTE_NODE accepts a node + inputs, executes, responds with result.

Testing

• Unit test worker-manager.ts: Mock Worker via a minimal MessagePort shim. Verify topological execution order, streaming callbacks, error propagation to dependents, fallback to sync on Worker failure.
• Integration test: Existing pipeline/integration.test.ts tests continue to pass (they use the synchronous path).
• E2E: Existing Playwright tests cover the full upload → execute → results flow and should pass without changes.

Item 2: Spec Curve Hover Performance

Design

Remove hoveredCellCoordinate from renderPlot dependencies

Change dependency array from [data, hoveredCellCoordinate] to [data]. SVGs rebuild only when underlying data changes.

Tag SVG elements with data-coordinate

After each Plot.plot() call, post-process the returned SVG:

1. Use the existing sortedColumns map (sort index → coordinate)
2. Query mark elements:
   • Estimate panel: circle (dots), line (CI rules)
   • Indicator panel: rect (cells)
3. Bucket elements by x-position (cx or x attribute), sort by x, assign data-coordinate from sortedColumns[i].coordinate. Positions are deterministic because Plot uses the same linear scale with a shared xDomain.

Hover effect via direct DOM mutation

const containerRef = useRef<HTMLDivElement>(null);

useEffect(() => {
  const container = containerRef.current;
  if (!container) return;
  const elements = container.querySelectorAll('[data-coordinate]');
  for (const el of elements) {
    const coord = el.getAttribute('data-coordinate');
    const match = hoveredCellCoordinate === null
      || coord === hoveredCellCoordinate;
    (el as HTMLElement).style.opacity = match ? '' : '0.25';
  }
}, [hoveredCellCoordinate]);

Remove opacity from Plot mark data

Lines 99 and 106 currently compute per-mark opacity based on hoveredCellCoordinate. Remove these — all marks render at full opacity. Hover highlighting is handled entirely by DOM mutation.

No changes to chart-container.tsx

ChartContainer stays unchanged. It re-renders when renderPlot changes, which now only happens on data changes.

After fix:

Zero SVG rebuilds. Direct DOM mutation on <200 elements is effectively free.

Testing

• E2E: Existing spec curve tests assert on rendered content and edges — should pass without changes.
• Manual: Hover over spec curve columns, confirm smooth highlighting without jank.

Execution Order

These two items are independent and can be implemented in parallel. Recommended sequence: Worker first (higher impact), then spec curve hover (polish).

Files Changed