Windows Media Player doesn't reproduce audio from file that was created with mp4box library

Question

I am using the mp4box library to create a media file that includes video and audio tracks. I have set up my audio encoder output callback using the following code:

if (encodingAudioTrack === null) {
    let encodingAudioTrackOptions = {
        timescale: this.timescale, 
        samplerate: this.options.audioConfig?.sampleRate, 
        channel_count: this.options.audioConfig?.numberOfChannels,
        samplesize: 16,
        hdlr: 'soun',
        name: 'SoundHandler',
        type: 'mp4a',
        brands: ['mp42', 'mp41', 'isom'],
    }
    encodingAudioTrack = this.file.addTrack(encodingAudioTrackOptions);
}

if (this.previousEncodedAudioChunk) {
    let ab = new ArrayBuffer(this.previousEncodedAudioChunk.byteLength);
    this.previousEncodedAudioChunk.copyTo(ab);
    const sampleDuration = chunk.timestamp - this.previousEncodedAudioChunk.timestamp;
    this.file.addSample(encodingAudioTrack, ab, {
        dts: this.audioSampleTimestamp,
        cts: this.audioSampleTimestamp,
        duration: sampleDuration,
    }); 
    this.audioChunkCount++;
    this.audioSampleTimestamp = this.audioSampleTimestamp + (sampleDuration ?? 0);
}

This is the config for AudioEncoder from WebCodec API:

audioEncoderConfig = {
    codec: 'mp4a.40.2',
    sampleRate: combinedStream.getAudioTracks()[0].getSettings().sampleRate ?? 0,
    numberOfChannels: combinedStream.getAudioTracks()[0].getSettings().channelCount ?? 0,
    bitrate: 128_000,
}

However, when I save the created file and launch it with Windows 10 Media Player or Movies & TV, the audio doesn't play. MPC_HC Player, on the other hand, opens the file correctly. I noticed that when I check the info with MediaInfo tool, the codec identifier is mp4a instead of mp4a.40.2 that was set.

Am I missing some details or did I set something wrong? I appreciate any help.

score 0 · Answer 1 · answered Jun 30 '23 at 14:09

To get a better grip of your problem, it would be good if you could provide details of how file.addTrack and file.addSample works. Are you using the MediaStreamTrack API or is it a muxer? The reason I am asking, is because the problem might not be in the code you showed, but instead in the way you call .addSample function. Which arguments this function expect?

Regardless of the above, I noticed in your code that you copy the chunk data from the orignial EncodedAudioChunk to the new buffer but you never created a new instance of the EncodedAudioChunk with this new buffer, instead, you fed it straight to the addSample function, along with other arguments. Take a look at this example code below (where a new chunk is cretaed with differente cts) to see what I mean:

audioEncoder = new window.AudioEncoder({
      output: (encodedChunk) => {
        //copy the data of the encoded chunk to buffer
        const buffer = new ArrayBuffer(encodedChunk.byteLength);
        encodedChunk.copyTo(buffer);
        //create new encoded audio chunk, already opus, with correct CTS
        // let correctedCTS = encodedChunk.timestamp - 1000000 * Number.parseFloat(lowerSlider.value) / 20 + videos[currentVideoIndex].start * 1000000;
        // console.log(`Corrected Audio CTS = ${correctedCTS}`);
        let correctedCTSEncodedAudioChunk = new EncodedAudioChunk({
          type: encodedChunk.type,
          timestamp: correctedAudioCTS,
          duration: encodedChunk.duration,
          data: buffer
        })
        correctedAudioCTS += encodedChunk.duration;

        muxer1.addAudioChunk(correctedCTSEncodedAudioChunk);
        console.log(`Muxed Audio w/ CTS=${correctedAudioCTS}`);
      },

score 0 · Accepted Answer · answered Jul 12 '23 at 05:34

When adding audio track, you need to provide description property (esds box)

mp4File.addTrack({
  ...otherAudioTrackOpts,
  description: createESDSBox(meta.decoderConfig?.description)
})

Full code

Unfortunately, there is a bug in mp4box.js creating esds box, refer to https://github.com/gpac/mp4box.js/issues/341.

Missing data causes the size to be calculated incorrectly, which ultimately prevents the file from being parsed.

If you want a quick fix, you also need to modify package.json. https://github.com/hughfenghen/WebAV/blob/3936c814333e19497b0dd8f40d6adebb3e334693/packages/av-cliper/package.json#L33

"dependencies": {
  "mp4box": "hughfenghen/mp4box.js"
}

This issue contains more detail: https://github.com/hughfenghen/WebAV/issues/10

Windows Media Player doesn't reproduce audio from file that was created with mp4box library

2 Answers2