Stu Mason · Folkestone, Kent
One entryMerged PR· crunch.stumason.dev· PHP

PR #39 merged: perf(pack): parallelise the pipeline + live job progress

additions
+466
deletions
-76
files changed
14

What

Makes /pack sing. The pipeline was fully sequential — ~250 one-at-a-time ffmpeg frame extractions, then ~250 one-at-a-time OCR calls, then two ASR calls — and the OCR sidecar couldn't have served parallel requests anyway: its async def endpoints ran blocking pytesseract on the event loop, serialising everything behind one uvicorn worker.

Sidecar (the enabler)

  • /ocr + /ocr/words become sync defs → FastAPI threadpool, with the anyio pool capped via OCR_MAX_PARALLEL (default 4) so excess requests queue cheaply instead of forking more tesseracts.
  • Paddle path serialised behind a lock (not thread-safe, 1GB pipeline); tesseract needs no lock (subprocess per call).
  • OMP_THREAD_LIMIT=1 pins each tesseract to one core, so parallelism ≈ cores used, and compose gives the service a cpus: "4" ceiling — same shared-box reasoning as the asr service.

Pipeline

  • OcrClient::linesMany() — frames fan out in bounded Http::pool waves, keyed by t_ms; a failed frame is omitted, never fatal (unchanged semantics).
  • FrameExtractor — ffmpeg runs in bounded Process::pool waves.
  • Wave widths via crunch.pack.{ocr,extract}_concurrency (CRUNCH_PACK_*_CONCURRENCY, default 4), sized to the sidecar's cap.
  • Mic/sysaudio transcribe paths deduped into one trackWords() helper (only difference was the sync-offset mapping).

Job progress

inference_jobs gains a nullable progress column; the worker reports {stage, done?, total?} at stage boundaries and per OCR wave (unpack → extract_frames → ocr n/N → transcribe → analyze → assemble), surfaced in GET /jobs/{id}. A minutes-long pack no longer sits at a bare "processing"; on failure the last progress shows where it died.

Effect

OCR + extraction leg ≈ ¼ of its old wall-time at the default width. Transcription is now the long pole (possible follow-up: overlap ASR with the OCR leg — needs shared-handler async HTTP, deliberately out of scope here).

Tests

  • PackPipelineTest (new): parallel linesMany keyed results + failed/unreadable frame omission + wave progress; parallel extractor result mapping + per-timestamp seeks.
  • PackTest: progress surfaced in the poll envelope.
  • Full suite 142/142, phpstan clean, pint clean. py_compile on the sidecar.

Note: the claude-review CI check is known-always-red and non-required.

🤖 Generated with Claude Code