Monodroid(Xamarin) how to play multiple mp3 in sync

Question

My project needs to have an mp3 player where multiple mp3 files play together completely in sync. I've tried MediaPlayer but the problem is that when in my loop I start two mp3 files they are slightly off sync. Of course they are created and prepared before calling Play(). These are not just sounds, but 3-4 minute music files that need to be seamlessly looped as well.

At the moment I am struggling with AudioTrack, because the files are AndroidAssets and when I create a stream out of it the ByteReader gives an out of memory error...

So is there a better way of doing this mp3 music sync playing?

Thanks Greg

score 2 · Answer 1 · edited May 23 '17 at 12:31

I have been strugling with the same problem for a few days now. My aproach was to have mutiple threads streaming/decoding/playing a mp3 file using MediaExtractor/MediaCodec/AudioTrack. I got it working in C# but observed a lot of GC activity during playback. Below you can find my code for a single track. You can read more about the problems I encountered and how I am going to solve it here: How to stream data from MediaCodec to AudioTrack with Xamarin for Android.

I have also found out that on low end devices the tracks would not play in sync (delays of 10s to 100s ms). I think the problem is that when I do audioTrack.Play(), AudioTrack does not have enough data in its buffer to start the playback immediately and depending on the input file format it take a different numer of mp3 frames to fill it up, hence the tracks start with different delays. The solution to this I am experimenting with is to postpone audioTrack.Play() until I know the buffer has enough bytes (AudioTrack.GetMinBufferSize(...)) for the playback to start immediately and only than call audioTrack.Play().

var fd = this.Resources.OpenRawResourceFd(Resource.Raw.PianoInsideMics);

var extractor = new MediaExtractor();
extractor.SetDataSource(fd.FileDescriptor, fd.StartOffset, fd.Length);
extractor.SelectTrack(0);

var trackFormat = extractor.GetTrackFormat(0);

var decoder = MediaCodec.CreateDecoderByType(trackFormat.GetString(MediaFormat.KeyMime));
decoder.Configure(trackFormat, null, null, MediaCodecConfigFlags.None);

var thread = new Thread(() =>
{
    decoder.Start();
    var decoderInputBuffers = decoder.GetInputBuffers();
    var decoderOutputBuffers = decoder.GetOutputBuffers();

    var inputIndex = decoder.DequeueInputBuffer(-1);
    var inputBuffer = decoderInputBuffers[inputIndex];
    var bufferInfo = new MediaCodec.BufferInfo();
    byte[] audioBuffer = null;
    AudioTrack audioTrack = null;

    var read = extractor.ReadSampleData(inputBuffer, 0);
    while (read > 0)
    {
        decoder.QueueInputBuffer(inputIndex, 0, read, extractor.SampleTime,
            extractor.SampleFlags == MediaExtractorSampleFlags.Sync ? MediaCodecBufferFlags.SyncFrame : MediaCodecBufferFlags.None);

        extractor.Advance();

        var outputIndex = decoder.DequeueOutputBuffer(bufferInfo, -1);
        if (outputIndex == (int) MediaCodecInfoState.OutputFormatChanged)
        {
            trackFormat = decoder.OutputFormat;
        }
        else if (outputIndex >= 0)
        {
            if (bufferInfo.Size > 0)
            {
                var outputBuffer = decoderOutputBuffers[outputIndex];
                if (audioBuffer == null || audioBuffer.Length < bufferInfo.Size)
                {
                    audioBuffer = new byte[bufferInfo.Size];
                    Debug.WriteLine("Allocated new audiobuffer: {0}", audioBuffer.Length);
                }

                outputBuffer.Rewind();
                outputBuffer.Get(audioBuffer, 0, bufferInfo.Size);
                decoder.ReleaseOutputBuffer(outputIndex, false);

                if (audioTrack == null)
                {
                    var sampleRateInHz = trackFormat.GetInteger(MediaFormat.KeySampleRate);
                    var channelCount = trackFormat.GetInteger(MediaFormat.KeyChannelCount);
                    var channelConfig = channelCount == 1 ? ChannelOut.Mono : ChannelOut.Stereo;

                    audioTrack = new AudioTrack(
                        Stream.Music,
                        sampleRateInHz,
                        channelConfig,
                        Encoding.Pcm16bit,
                        AudioTrack.GetMinBufferSize(sampleRateInHz, channelConfig, Encoding.Pcm16bit)*2,
                        AudioTrackMode.Stream);

                    audioTrack.Play();
                }

                audioTrack.Write(audioBuffer, 0, bufferInfo.Size);
            }
        }

        inputIndex = decoder.DequeueInputBuffer(-1);
        inputBuffer = decoderInputBuffers[inputIndex];

        read = extractor.ReadSampleData(inputBuffer, 0);
    }
});

Monodroid(Xamarin) how to play multiple mp3 in sync

1 Answers1