I am constructing an internet app the place customers add a video, the app crops the primary 60 seconds client-side, extracts audio for server-side AI processing, then muxes the returned audio again into the cropped video. This should work on cell together with iOS.
The issue
iPhones report HEVC (H.265) by default. Each client-side method I’ve tried fails on iOS Safari for big HEVC recordsdata:
WebCodecs:
VideoDecoder.isConfigSupported({ codec: "hev1..." }) → false
ffmpeg.wasm: fetchFile() copies complete file to JS heap → OOM (tab crash)
captureStream: video.captureStream() not supported on iOS Safari
canvas: should decode HEVC frames → ~8MB × 5 body buffer → OOM
File.slice() works for byte-level cropping with no decode, however I nonetheless have to mux a brand new audio monitor in afterward — which brings the identical reminiscence constraints again.
What works
– iOS + H.264: WebCodecs pipeline works effective (decode → re-encode at 720p through VideoEncoder → mp4box.js mux)
– Android Chrome: WebCodecs or ffmpeg.wasm each work
What I’ve thought of
1. Extract audio-only through mp4box.js + AudioDecoder (AAC is all the time supported even on iOS HEVC) — solves the crop step, not the mux step
2. Server-side mux — works however provides latency and bandwidth
3. mp4box.js container-level audio monitor substitute — nonetheless requires loading moov + mdat into reminiscence
Query
Is there any iOS Safari 16.4+ net API that permits changing the audio monitor in an MP4/MOV container with out decoding video frames? Or any memory-safe method to mux new audio into a big HEVC file client-side?
Or is server-side mux the one sensible possibility right here?
