Handling video streaming correctly on the example of existing real-world solutions

Question

I am experimenting/building a website with streaming functionality, where the user could upload the video (.mp4) and view the video on the page (video contents are returned partially/incrementally). I am using the Azure Blob Storage and the dotnet technology stack.

html:

<video src="https://localhost:5001/Files/DownloadFileStream?fileName=VID_20210719_110859.mp4" controls="true" />

controller:

    [HttpGet]
    public async Task<IActionResult> DownloadFileStream([FromQuery] string fileName)
    {
        var range = Request.Headers["range"].ToString();
        Console.WriteLine(range);

        var container = new BlobContainerClient("UseDevelopmentStorage=true", "sample-container");
        await container.CreateIfNotExistsAsync();

        BlobClient blobClient = container.GetBlobClient(fileName);

        var response = await blobClient.GetPropertiesAsync();
        string contentType = response.Value.ContentType;
        long contentLength = response.Value.ContentLength;

        Azure.Response<BlobDownloadStreamingResult> result = await blobClient.DownloadStreamingAsync();

        return new FileStreamResult(result.Value.Content, contentType)
        {
            EnableRangeProcessing = true,
        };
        
        // following approach supports video Seek functionality:
        /*
        using var memory = new MemoryStream();
        await blobClient.DownloadToAsync(memory);

        return new FileContentResult(memory.ToArray(), contentType)
        {
            EnableRangeProcessing = true,
        };*/
    }

What I have noticed is, when I stream the contents - in Chrome's Inspect mode we can see the media file being downloaded. In fact, if you use the seek functionality - multiple request records will appear:

My current problem is, after the file got fully downloaded/played until the end of stream - if you start playing it again - the browser will start to download the contents again, instead of using already cached/downloaded content.

So I went to investigate how the huge market leaders like YouTube and Instagram handle the video streaming, but what I noticed that upon video playback - not a single "media" type of content/request appears.

In fact, from the inspector's perspective - it looks like nothing is being downloaded at all.. So this raises some questions.

1. How do market leaders like YouTube and Instagram stream videos to clients, without exposing to inspector any media traffic? Is this behavior replicable by .Net 5?
2. Why does my application keep re-downloading the same file over and over again and how to prevent it from happening?
3. Why does the seek functionality not work when DownloadStreamingAsync() method is used and how to fix it? I use this approach mainly because the application has smaller memory footprint in this case. We don't have to download the whole stream into memory before returning contents to the client.

While I have listed 3 questions here - my main concern is the first question, so any answers on that topic are very welcomed. :)

Answer 1

Okay, after a bit of studying and goodling around - I have found a good article/s on the topic:

How video streaming works on the web: An introduction

https://medium.com/canal-tech/how-video-streaming-works-on-the-web-an-introduction-7919739f7e1

The short story is - instead of providing the source file directly to the <video> html tag - you could instead write a javascript code that would inject the MediaSource into the tag. You could then use the technology like Fetch API or XMLHttpRequest to write a custom logic to that would inject the video stream into the buffer. In case of XMLHttpRequest type of technology it seems like you will see the xhr types of requests and in case of Fetch API - fetch types of requests in inspector.. The topic is much more complex and complicated and I don't have extensive knowledge on it. Partly because I am not a JS developer :) but also because you have to have an understanding how codecs/encoders/video data frames works etc..

Some more details on above mentioned techologies:

Fetch API vs XMLHttpRequest

Another notable topic you could look into - are streaming formats/standards. You could, for example use the adaptive streaming technology to adjust the video quality depending of bandwidth capacity. Two of the streaming standards are HLS (older) and DASH (newer?).

Also, since we are at it - I suggest looking into advanced video player libraries:

• video.js: https://github.com/videojs/video.js
• hls.js: https://github.com/video-dev/hls.js/
• dash.js: https://github.com/Dash-Industry-Forum/dash.js/

If you look them up on github - you will get some insight into this whole topic.

Edit 11.08.2021 In regards to my own question about DownloadStreamingAsync method usage and to answer the question of @Nate - you can't use this method. Download means - get me the whole contents.. What you are looking for instead is OpenReadAsync method which gets you the stream.

Here is the working example (which I will not be using myself but somebody might find it helpful):

    [HttpGet]
    [Route("Stream4/{fileName}")]
    public async Task<IActionResult> Get3(
        [FromRoute] string fileName,
        CancellationToken ct)
    {
        var container = new BlobContainerClient("UseDevelopmentStorage=true", "sample-container");
        await container.CreateIfNotExistsAsync();

        BlobClient blobClient = container.GetBlobClient(fileName);
        var response = await blobClient.GetPropertiesAsync();
        string contentType = response.Value.ContentType;
        long contentLength = response.Value.ContentLength;

        long kBytesToReadAtOnce = 300;
        long bytesToReadAtOnce = kBytesToReadAtOnce * 1024;
        var result = await blobClient.OpenReadAsync(0, bufferSize: (int)bytesToReadAtOnce, cancellationToken: ct);
        
        return new FileStreamResult(result, contentType)
        {
            // this is important
            EnableRangeProcessing = true,
        };
    }

Answer 2

Here's something I got to work. Ultimately it takes the blob and uses the range headers in the request to only download part of the file.

Since there's no magic PhysicalFile-esc helper that automatically supports the EnableRangeProcessing for the Azure Blob, we do it all manually.

The browser looks for the 206: Partial Content header to be returned, in conjunction with the Accept-Ranges: bytes, Content-Length, and Content-Range response headers so it knows whether there is more video to download.

[HttpGet, Route("video")]
public async Task<IActionResult> VideoAsync()
{
    var rangeHeaders = Request.GetTypedHeaders().Range;

    var blobServiceClient = new BlobServiceClient(connectionString);
    var containerClient = blobServiceClient.GetBlobContainerClient("data");
    var blobClient = containerClient.GetBlobClient(fileName);
    if (blobClient.Exists())
    {
        var getPropertiesResponse = await blobClient.GetPropertiesAsync();
        string contentType = getPropertiesResponse.Value.ContentType;
        long contentLength = getPropertiesResponse.Value.ContentLength;

        Azure.HttpRange? range = null;
        if (rangeHeaders?.Ranges?.Count > 0)
        {
            var rangeHeader = rangeHeaders.Ranges.First();

            // 1. If the unit is not 'bytes'.
            // 2. If there are multiple ranges in header value.
            // 3. If start or end position is greater than file length.
            if (rangeHeaders.Unit != "bytes" || rangeHeaders.Ranges.Count > 1)
            {
                Response.StatusCode = (int)HttpStatusCode.RequestedRangeNotSatisfiable;
                Response.Headers.Add("Content-Range", $"bytes {range.Value.Offset}-{contentLength - 1}/{contentLength}");
                Response.Headers.Add("Content-Type", "video/mp4");

                return null;
            }

            var offset = rangeHeader.From ?? 0;
            var length = rangeHeader.From.HasValue && rangeHeader.To.HasValue
                ? rangeHeader.To - rangeHeader.From
                : null;

            range = new Azure.HttpRange(offset, length);
        }

        // Append appropriate response headers based on status
        Response.Headers.Add("Accept-Ranges", "bytes");
        Response.Headers.Add("Content-Length", $"{contentLength}");
        if (range.HasValue)
        {
            Response.Headers.Add("Content-Range", $"bytes {range.Value.Offset}-{range.Value.Length ?? (contentLength - 1)}/{contentLength}");
        }

        var streamingFileResponse = await blobClient.DownloadStreamingAsync(range ?? default);
        var videoContent = streamingFileResponse.Value.Content;
        var videoMeta = streamingFileResponse.Value.Details;

        var fileResponse = new FileStreamResult(videoContent, "video/mp4");
        Response.StatusCode = (int)HttpStatusCode.PartialContent;
        return fileResponse;
    }

    throw new BadHttpRequestException("Bad request");
}