File download from Google Drive to byte array failing - C# HttpClient

Question

I'm new to HttpClient. The following code to download a binary file returns a byte array of the wrong length and containing the wrong data:

public static async Task<byte []> ReadWebFileBinaryAsync ( HttpClient httpClient, string webUrl )
{
    byte [] bytes;

    try
    {
        HttpResponseMessage response = await httpClient.GetAsync ( webUrl );
        response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue( "application/octet-stream" );
        using ( MemoryStream memoryStream = new MemoryStream () )
        {
            using ( Stream stream = await response.Content.ReadAsStreamAsync () )
            {
                stream.CopyTo ( memoryStream );
            }
            memoryStream.Seek ( 0, SeekOrigin.Begin );
            bytes = memoryStream.ToArray ();
        }

        return bytes;
    }
    catch ( Exception ex )
    {
        // ...
    }
}

httpClient is initialised app-wide as follows:

httpClientHandler = new HttpClientHandler
{
    AllowAutoRedirect = true,           // Allegedly the default, but to make sure
    MaxAutomaticRedirections = 20
};

httpClient = new HttpClient ( httpClientHandler );
httpClient.Timeout = TimeSpan.FromSeconds ( 60 );

// Accept all languages
httpClient.DefaultRequestHeaders.AcceptLanguage.Clear();
httpClient.DefaultRequestHeaders.AcceptLanguage.Add ( new System.Net.Http.Headers.StringWithQualityHeaderValue ( "*" ) );

// Accept binary data
httpClient.DefaultRequestHeaders.Accept.Add ( new System.Net.Http.Headers.MediaTypeWithQualityHeaderValue ( "application/octet-stream" ) );

The web file is 688kb and has a public url of the form

https://drive.google.com/file/d/xxxxx/view?usp=share_link

Anyone who has the link can open it in a web browser.

The returned array contains only 75K bytes, and as mentioned the data is completely wrong (comparing with the web file in a hex editor). Another file of size just 14KB likewise returns an array of 75K bytes! The exact number of bytes varies slightly each time the app is run, but between 75,400 and 75,600.

This is in .NET 7.0.1, Xamarin.Forms.

I don't know if my code is wrong or there is something funny about downloading from Google Drive. Maybe the Google API must be used? A way to transform the url maybe? The files I care about will be less than 1MB. Any help appreciated!

I created a GitHub website and the file is correctly downloaded from there. So the problem is related to Google Drive. I've begun a search and it appears there may be a way to modify the Google Drive url to allow download, but so far I haven't worked out what it is. — BillF, Jun 11 '23 at 14:23
I'd guess that 75k response actually contains an error page or redirect script. Write it to a file and examine the results. If you turn on the dev console and paste the url into your browser, it is probably doing a redirect. — Jason, Jun 11 '23 at 15:25
also, this code `response.Content.Headers.ContentType = ...` doesn't do anything useful. You have already received the response from the server, changing it's `ContentType` is pointless — Jason, Jun 11 '23 at 15:26
Thanks, @Jason. Your posts are always super helpful. Yes, with a Google Drive file url, it turns out that Google first throws up a page with a download button, causing the symptoms I saw. Google provides a page to convert such urls into a 'direct' download link. For example, this transforms drive.google.com/file/d/xxxxx/… into https://drive.google.com/uc?export=download&id=xxxxx - will see if this works — BillF, Jun 12 '23 at 02:36

score 0 · Accepted Answer · answered Jun 12 '23 at 08:53

For Google Drive links, Google interposes a page with a Download button, causing the code above to fail.

The problem is solved by first calling the following method, which works for files up to around 25Mb in size. Larger files would probably require a Google API solution.

private string CheckForGoogleDriveUrl ( string url )
// If given url is a Google Drive url, returns an equivalent direct url, avoiding initial page with download button
// Transforms https://drive.google.com/file/d/xxxxx/view?usp=share_link to https://drive.google.com/uc?export=download&id=xxxxx
// Note: Probably does not work for files that Google considers too large to virus check, around 25MB, as Google will issue a prompt
{
    if ( url.Contains ( "https://drive.google.com/file" ) )
    {
        string [] s = url.Split ( '/' );
        url = s [5];
        s = url.Split ( '/' );
        url = "https://drive.google.com/uc?export=download&id=" + s [0];
    }

    return url;
}

For full solution, see https://stackoverflow.com/questions/76496242/how-to-directly-download-files-including-from-google-drive-onedrive-and-dropbo — BillF, Jun 20 '23 at 02:25

File download from Google Drive to byte array failing - C# HttpClient

1 Answers1