1

I'm trying to play audio from a stream of bytes in Flutter. I've used the Just Audio package's example on reading from a stream of bytes, and I'm still getting an error. The error is:

Playback error
E/ExoPlayerImplInternal(19173):   com.google.android.exoplayer2.ExoPlaybackException: Source error
E/ExoPlayerImplInternal(19173):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:632)
E/ExoPlayerImplInternal(19173):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:602)
E/ExoPlayerImplInternal(19173):       at android.os.Handler.dispatchMessage(Handler.java:98)
E/ExoPlayerImplInternal(19173):       at android.os.Looper.loop(Looper.java:154)
E/ExoPlayerImplInternal(19173):       at android.os.HandlerThread.run(HandlerThread.java:61)
E/ExoPlayerImplInternal(19173):   Caused by: com.google.android.exoplayer2.source.UnrecognizedInputFormatException: None of the available extractors (Mp3Extractor, FlvExtractor, FlacExtractor, WavExtractor, FragmentedMp4Extractor, Mp4Extractor, AmrExtractor, PsExtractor, OggExtractor, TsExtractor, MatroskaExtractor, AdtsExtractor, Ac3Extractor, Ac4Extractor, AviExtractor, JpegExtractor) could read the stream.
E/ExoPlayerImplInternal(19173):       at com.google.android.exoplayer2.source.BundledExtractorsAdapter.init(BundledExtractorsAdapter.java:92)
E/ExoPlayerImplInternal(19173):       at com.google.android.exoplayer2.source.ProgressiveMediaPeriod$ExtractingLoadable.load(ProgressiveMediaPeriod.java:1017)
E/ExoPlayerImplInternal(19173):       at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:412)
E/ExoPlayerImplInternal(19173):       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
E/ExoPlayerImplInternal(19173):       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
E/ExoPlayerImplInternal(19173):       at java.lang.Thread.run(Thread.java:762)
E/AudioPlayer(19173): TYPE_SOURCE: None of the available extractors (Mp3Extractor, FlvExtractor, FlacExtractor, WavExtractor, FragmentedMp4Extractor, Mp4Extractor, AmrExtractor, PsExtractor, OggExtractor, TsExtractor, MatroskaExtractor, AdtsExtractor, Ac3Extractor, Ac4Extractor, AviExtractor, JpegExtractor) could read the stream.

I can see that something is wrong with the input format. However, I'm just not sure what it is. Here is the code that is supposed to perform the playback:

void initAudio() async {
    final server = await ServerSocket.bind(InternetAddress.anyIPv4, port);
    final player = AudioPlayer();
    final session = await AudioSession.instance;
    await session.configure(const AudioSessionConfiguration.speech());
    await session.setActive(true);
    server.listen((socket) {
      // event is a socket
      socket.listen((Uint8List data) async {
        print(data);
        player.setAudioSource(MyCustomSource(data), preload: false);
        player.play();
      });
    });
  }

MyCustomSource came from the Just Audio pub.dev listing, and it looks like this:

// Feed your own stream of bytes into the player
class MyCustomSource extends StreamAudioSource {
  final List<int> bytes;
  MyCustomSource(this.bytes);

  @override
  Future<StreamAudioResponse> request([int? start, int? end]) async {
    start ??= 0;
    end ??= bytes.length;
    return StreamAudioResponse(
      sourceLength: bytes.length,
      contentLength: end - start,
      offset: start,
      stream: Stream.value(bytes.sublist(start, end)),
      contentType: 'audio/mpeg',
    );
  }
}

I'm trying to send a Stream<List<int>> over a socket to stream audio over the internet. Here's the audio sender side:

void recordAudio() async {
    print("recording and sampling");
    final socket = await Socket.connect(a5atHome, port);
    Stream<Uint8List>? stream = await MicStream.microphone(sampleRate: 44100);
    socket.addStream(stream!);

    StreamSubscription<List<int>> listener = stream.listen((samples) async {
      print(samples);
    });
  }

I'm not sure if something is wrong with the sender side, either. I'm new to using JustAudio and I'm not sure where the error is coming from. Any help would be greatly appreciated.

Edit: By the way, I'm using the mic_stream pub.dev package to get microphone data in the form of a Stream

Zachary Haslam
  • 103
  • 1
  • 12
  • 1
    I will guess you are using a package called mic_stream which wasn't mentioned in your question - that package says in the first sentence of the README that it outputs in the PCM format which is basically raw, and unencoded. Yet your custom audio source declares that it is outputting audio/mpeg encoded data which is a mismatch. Also, Android and iOS only support a limited set of "encoded" audio formats (such as audio/mpeg) so you need to encode your raw audio data before a player can decode and play it. The simplest encoded format to go for is WAV. – Ryan Heise Dec 13 '22 at 02:26
  • By the way, I'm writing as a comment because I wasn't sure what MicStream in fact was, as there was no information in your question about it. But if the above comment puts you on the right track, consider writing your own answer to your question and marking it as correct. – Ryan Heise Dec 13 '22 at 02:27
  • @RyanHeise how exactly would I go about doing that? how do I encode the raw PCM format to WAV before playing it? I've tried searching for solutions and haven't been able to find any that works. I tried using another package called `Soundpool` to play sound from a Uint8List but I keep getting the same format errors. – Zachary Haslam Dec 14 '22 at 16:44

1 Answers1

2

Eventually figured it out. Thanks to Ryan Heise for pointing me in the right direction.

The MicStream package records audio data in the raw PCM format, so I needed to find a package that could play audio in this format too. From this answer, I found someone was looking to do the exact same thing - basically playing audio from a List<int> of audio data.

The package called flutter_sound was able to do this for me, per that other answer.

The sending side:

final socket = await Socket.connect(a5atHome, port);

Stream<Uint8List>? stream = await MicStream.microphone(
        sampleRate: 44100, audioFormat: AudioFormat.ENCODING_PCM_16BIT);
await socket.addStream(stream!);

and the playing side, which needed the fixing:

FlutterSoundPlayer player = FlutterSoundPlayer();

final server = await ServerSocket.bind(InternetAddress.anyIPv4, port);

final session = await AudioSession.instance;
await session.configure(const AudioSessionConfiguration.speech());
await session.setActive(true);
await player.openPlayer(enableVoiceProcessing: true);
server.listen((socket) async {
   await player.startPlayerFromStream(
        codec: Codec.pcm16, numChannels: 1, sampleRate: 44100);
   socket.listen((Uint8List data) async {
     player.foodSink!.add(FoodData(data));
   });
 });

MicStream's ENCODING_PCM_16BIT is the same as flutter_sound's Codec.pcm16 format. The format matches, and sound is played!

Zachary Haslam
  • 103
  • 1
  • 12