C# LibVLCSharp switching media stream causes HTTP exception

Question

We are using libvlcsharp to play a live mp3 network stream in our xamarin.ios app using the following code snippet

public bool Play(Uri uri)
{
    Media newMedia = new(this.LibVLC, uri);
    Media? oldMedia = this.VLCMediaPlayer.Media;
    bool success = this.VLCMediaPlayer.Play(newMedia);
    oldMedia?.Dispose();
    return success;
}

where VLCMediaPlayer is an instance of VLC's MediaPlayer class. The code above works and streaming works flawlessly.

However:

at some point we need to switch streams (i.e. the original live mp3 stream continues / doesn't technically end and we need to stop streaming it and switch to a different one).

As far as the documentation goes it seems like simply calling Play() on a different URL should do the trick. The problem is, it doesn't. The original stream stops for a couple of milliseconds and then just continues.

Our code looks something like this:

// use our custom Play() wrapper
Play(new Uri("https://whatever.com/some-live-stream.mp3"));

// ... do other stuff

// at some time later switch to a different stream using the same Play() wrapper
Play(new Uri("https://whatever.com/file-stream.mp3"));

The problem:

VLC doesn't start playback of the file-stream.mp3 but instead hangs for a couple of milliseconds to then continue playing some-live-stream.mp3. Why is this and how do we fix it?

Update (seems to be an integration bug):

running libVLC with debug output reveals this:

[00000177e9c21830] main input debug: Creating an input for 'file-stream.mp3'
[00000177e9c21830]playing uri
 main input debug: using timeshift granularity of 50 MiB
[00000177e9c21830] main input debug: using timeshift path: C:\Users\EXPUNGED\AppData\Local\Temp
[00000177e9c21830] main input debug: `http://127.0.0.1:5050/file-stream.mp3' gives access `http' demux `any' path `127.0.0.1:5050/file-stream.mp3'
[00000177e9b29350] main input source debug: creating demux: access='http' demux='any' location='127.0.0.1:5050/file-stream.mp3' file='\\127.0.0.1:5050\file-stream.mp3'
[00000177e92e1d60] main demux debug: looking for access_demux module matching "http": 15 candidates
[00000177e92e1d60] main demux debug: no access_demux modules matched
[00000177e904e6d0] main stream debug: creating access: http://127.0.0.1:5050/file-stream.mp3
[00000177e904e6d0] main stream debug:  (path: \\127.0.0.1:5050\file-stream.mp3)
[00000177e904e6d0] main stream debug: looking for access module matching "http": 27 candidates
[00000177e904e6d0] http stream debug: resolving 127.0.0.1 ...
[00000177e904e6d0] http stream debug: outgoing request:
GET /file-stream.mp3 HTTP/1.1
Host: 127.0.0.1:5050
Accept: */*
Accept-Language: en_US
User-Agent: VLC/3.0.16 LibVLC/3.0.16
Range: bytes=0-


[00000177e904e6d0] http stream debug: connection failed 
[00000177e904e6d0] access stream error: HTTP connection failure
[00000177e904e6d0] http stream debug: querying proxy for http://127.0.0.1:5050/file-stream.mp3
[00000177e904e6d0] http stream debug: no proxy
[00000177e904e6d0] http stream debug: http: server='127.0.0.1' port=5050 file='/file-stream.mp3'
[00000177e904e6d0] main stream debug: net: connecting to 127.0.0.1 port 5050
[00000177e904e6d0] http stream error: cannot connect to 127.0.0.1:5050
[00000177e904e6d0] main stream debug: no access modules matched

(above snippet starts at when the second Play() is called.)

What immediately catches attention is the HTTP connection failure.

However let me expand on our minimal reproducible example. In production we don't only need switch streams once but multiple times. We discontinue the first some-live-stream.mp3 stream (which runs live 24/7 server side). We then need to switch to file-stream.mp3 which is a ~10 second long mp3 file. After the file is played we shall continue playing the first stream (some-live-stream.mp3).

We therefore wrote a custom Enqueue() method in addition to our Play() method originally included in this question.

The whole relevant code of our internal VLC wrapper class therefore actually looks like this:

public sealed class MediaService
{
    private readonly LibVLC _libVLC;
    private readonly ConcurrentQueue<Uri> _playlist = new();

    public MediaPlayer VLCPlayer { get; }

    internal MediaService(LibVLC libVLC)
    {
        _libVLC = libVLC;
        VLCPlayer = new MediaPlayer(_libVLC);
        VLCPlayer.EndReached += VLCPlayer_EndReached;
    }
    
    public bool Play(Uri uri)
    {
        Media newMedia = new(_libVLC, uri);
        Media? oldMedia = VLCPlayer.Media;
        bool success = VLCPlayer.Play(newMedia);
        oldMedia?.Dispose();
        CurrentUrl = uri.AbsoluteUri;
        return success;
    }

    public bool IsStartingOrPlaying() =>
        VLCPlayer.State is VLCState.Buffering
        or VLCState.Opening
        or VLCState.Playing;
        
    public void Enqueue(Uri uri)
    {
        if (IsStartingOrPlaying())
        {
            _playlist.Enqueue(uri);
        }
        else
        {
            Play(uri);
        }
    }

    private void VLCPlayer_EndReached(object sender, EventArgs e)
    {
        if (_playlist.TryDequeue(out Uri? result))
        {
            // don't deadlock on VLC callback
            Task.Run(() => Play(result));
        }
    }
}

Now to the minimal reproducible example:

the code below fails (i.e. causes the HTTP connection failure)

using our.vlc.wrapper;

CoreLoader.Initialize(true);
MediaService mediaService = MediaServiceFactory.GetSharedInstance();

// start streaming the live stream
Task.Run(() => mediaService.Play("http://127.0.0.1:5050/some-live-stream.mp3"))
    // do other stuff...
    .ContinueWith((_) => Thread.Sleep(10000)) 
    // now interrupt the original stream with the mp3 file
    .ContinueWith((_) => mediaService.Play("http://127.0.0.1:5050/file-stream.mp3"))
    // and continue the live stream after mp3 file stream ends
    .ContinueWith((_) => mediaService.Enqueue("http://127.0.0.1:5050/some-live-stream.mp3"));

Console.ReadLine();

The HTTP exception when trying to stream http://127.0.0.1:5050/file-stream.mp3 then obviously causes the "lag" described in the original question and after that the original stream continues as expected.

HOWEVER this code snippet works:

using our.vlc.wrapper;

CoreLoader.Initialize(true);
MediaService mediaService = MediaServiceFactory.GetSharedInstance();

// start streaming the live stream
Task.Run(() => mediaService.Play("http://127.0.0.1:5050/some-live-stream.mp3"))
    // do other stuff...
    .ContinueWith((_) => Thread.Sleep(10000)) 
    // now interrupt the original stream with the mp3 file
    .ContinueWith((_) => mediaService.Play("http://127.0.0.1:5050/file-stream.mp3"))
    // another sleep seems to cause no connection failure
    .ContinueWith((_) => Thread.Sleep(10000))
    // call PLAY() instead of ENQUEUE()
    .ContinueWith((_) => mediaService.Play("http://127.0.0.1:5050/some-live-stream.mp3"));

Console.ReadLine();

So actually our Enqueue() method seems to be the culprit. However I don't see why it's implementation would be an issue and cause a seemingly random HTTP connection failure when the /file-stream.mp3 endpoint is well tested and even works in the second example.

Using some sort of queueing is essential for our project, so we can't just fall back to Thread.Sleep() calls as we did in our minimal reproducible examples. How do we fix our Enqueue() method and why isn't it working as intended?

Update 2

Inspecting traffic during the "HTTP connection failure" using Wireshark shows this:

We can see the FIN, FIN-ACK, ACK of the terminating TCP connection to the first /some-live-stream.mp3 get intermingled with the SYN, SYN-ACK, ACK of the HTTP request to /file-stream.mp3 so it's a bit hard to read but it's clear that the actual HTTP GET never gets send so the endpoint at /file-stream.mp3 is never invoked. It's easier to understand with the following in mind: Port 5050 is the server providing the streams. Port 20118 is VLC with the first some-live-stream connection. Port 20224 is VLC with the second connection to file-stream.mp3 which is failing and at the bottom you can see a connection from port 20225 initializing which is the continuation of some-live-stream.

For some reason VLC immediately terminates the newly established TCP connection for the /file-stream.mp3 request as soon as it's established (look for the requests going from port 20124->5050). So VLC actively terminates the connection :C.

And then in the last few lines the original /some-live-stream.mp3 connection is re-established.

So why does VLC fail to even send the HTTP GET for the /file-stream.mp3 request?

Answer 1

Don't forget that Play() is not a synchronous method as you might expect. It is a method that posts a stop message to a background thread, and only then starts to play the media.

When you're executing your IsStartingOrPlaying() method right after, chances are that the state is not the one that you might have expected, thus calling the second Play()

Answer 2

After @cube45 pointed us in the right direction that the problem at hand was a simple threading issue all we had to do is to introduce basic thread-safety in our Enqueue(), Stop() and Play() methods as can be seen below.

public sealed class MediaService
{
    private const int NOT_PLAYING = 0x0;
    private const int PLAYING = 0x1;

    private readonly LibVLC _libVLC;
    private readonly ConcurrentQueue<Uri> _playlist = new();

    private volatile int _playbackStatus = NOT_PLAYING;

    public MediaPlayer VLCPlayer { get; }

    internal MediaService(LibVLC libVLC)
    {
        _libVLC = libVLC;
        VLCPlayer = new MediaPlayer(_libVLC);
        VLCPlayer.EndReached += VLCPlayer_EndReached;
    }
    
    public void Enqueue(Uri uri)
    {
        int status = _playbackStatus;

        if (status is PLAYING)
        {
            _playlist.Enqueue(uri);
            if (Interlocked.CompareExchange(ref _playbackStatus, NOT_PLAYING, PLAYING) is NOT_PLAYING)
            {
                // vlc finished while we were enqueuing...
                DequeueAndPlay();
            }
        }
        else
        {
            Play(uri);
        }
    }

    public bool Play(Uri uri)
    {
        _ = Interlocked.Exchange(ref _playbackStatus, PLAYING);
        Media newMedia = new(_libVLC, uri);
        Media? oldMedia = VLCPlayer.Media;
        bool success = VLCPlayer.Play(newMedia);
        oldMedia?.Dispose();
        CurrentUrl = uri.AbsoluteUri;
        return success;
    }

    public void Stop()
    {
        if (Interlocked.CompareExchange(ref _playbackStatus, NOT_PLAYING, PLAYING) is PLAYING)
        {
            VLCPlayer.Stop();
            if (VLCPlayer.Media is not null)
            {
                Media oldMedia = VLCPlayer.Media;
                VLCPlayer.Media = null;
                oldMedia.Dispose();
            }
        }
    }

    private void VLCPlayer_EndReached(object sender, EventArgs e) => DequeueAndPlay();

    private void DequeueAndPlay()
    {
        if (_playlist.TryDequeue(out Uri? result))
        {
            Task.Run(() => Play(result));
        }
        else
        {
            _ = Interlocked.Exchange(ref _playbackStatus, NOT_PLAYING);
        }
    }
}

while the code above is by no means the "perfect" solution and does not provide thread safety for multiple threads calling Enqueue(), Stop() or Play(), it does however alleviate the exact issue we ran into and it should point anyone encountering a similar problem in the right direction :)