Stu Mason · Folkestone, Kent
One entryMerged PR· crunch.stumason.dev· PHP

PR #37 merged: feat: sysaudio transcript + on-camera presence track (roll pack 0.0.15)

additions
+307
deletions
-18
files changed
11

Adds the two 0.0.15 pack tracks @StuMason signed off on (both V1, best-effort, opt-in by payload — a mic-only pack's crunch.json is unchanged).

sys_transcript (system audio)

The sysaudio track (computer output — a video/call the user played) is transcribed via the ASR sidecar and placed on the shared clock (PackManifest::sysAudioTMsForSeconds). Surfaced as its own sys_transcript {text, words} block, kept distinct from the mic (presenter voice).

camera (on-camera presence)

Sample the camera at a coarse cadence (FrameSampler::cadence), score each frame with in-process zero-shot CLIP (cheap — no Florence-2 sidecar) for on-camera presence, collapse contiguous present frames into camera {t_start, t_end} spans, and emit an on_camera moment (source: camera) at each span onset. Presence only — no emotion/expression model (that'd be a later slice + new model).

Config in config/crunch.php (crunch.camera): cadence, frame cap, present/absent labels, threshold.

Infra

  • /pack upload cap 100MB → 300MB (PHP post_max_size/upload_max_filesize + Laravel validation) so full packs including camera.mp4 (~142MB) upload.
  • FrameExtractor generalised (extractFrom) to pull frames from any track; screen path unchanged.

Notes / decisions

  • 0.0.15: roll will start writing this version; crunch stays version-tolerant (no gate).
  • Both tracks degrade gracefully — missing/unreadable sysaudio or camera, or CLIP/ffmpeg failures, just omit the track.

Validation

  • Real 0.0.15 take: camera frames extract from camera.mp4, sysaudio clock round-trips (5s → 5013ms → 5s), cadence widens to the 180-frame cap.
  • 6 new unit tests (camera spans, onset moments, sys_transcript, cadence, clock round-trip); full suite 137 green; Pint + PHPStan clean.
  • End-to-end on the same pack with camera to follow after deploy.

🤖 Generated with Claude Code