UPDATE
Please upgrading the Azure SignalR SDK to the latest version 1.12.2
( Change the version from the project file
<PackageReference Include="Microsoft.Azure.SignalR" Version="1.21.2" />)
It provides better stability and provide more logs if something is wrong.
When uses client stream to download large files (say over 200MB), currently your client has high possibility hit the service SlowClient
throttling rule
(buffering 16 data frames with max 1MB size per client) and the client will be closed with SlowClient reason by the service.
For now we should use the "download a file via a URL" solution if the size is larger than 100MB.
It's very interesting and I also test in my local by using this repo. And find that blazor server side uses SignalR and has its data transfer (buffer) limitations.

You can adjust the code below according to your actual needs.
builder.Services.AddSignalR(options =>
{
options.EnableDetailedErrors = true;
// According your needs
options.MaximumReceiveMessageSize = 102400000; // 100 MB
});
...
// According your needs
app.MapHub<MainHub>("/mainhub", options => {
options.ApplicationMaxBufferSize = 102400000; // 100 MB
options.TransportMaxBufferSize = 102400000; // 100 MB
});
Related Link
Buffer management