VB.net Uploading Large Files to Google Drive Failing

Question

Okay...first post...hopefully ill get it right! (If I miss something, or do something wrong, do let me know and ill try my best to fix it! :) )

The Scenario:

We have a CCTV server which records footage to a folder. The footage files are broken up into 1gb files. (Averaging 45gb a day)

After 1 month, these files are to be moved up to a Google Drive folder and removed from the local HDD on the server. (One way sync with deletion after successful upload...we have unlimited google drive storage) [As for the size of the uploads, uploading 45gb overnight really isn't an issue for us. We have a symmetrical 1gbps dedicated internet line]

This is the way management have requested it be done, so sadly I have to try to make this work...

Where I'm at!

So...I knocked together a console app in VB.net which works! (Code will be attached below...). This will be ran on the server at midnight every night to I have it set to upload to the right folder, and using a set of test files (which happen to be text files in a folder on my desktop), they all upload and delete correctly! Wahoo, all is looking promising!

As ever...there was always going to be something that didn't quite work right.

I replaced the test text files with some of the CCTV footage files (They're .xsf files) and well...it didn't work, in a nutshell.

I put it back to .txt files, and it works again. However, as soon as the text files gain any size (IE...create a text file and make it 50mb or so), it refuses to work again.

I'm using the Google Drive API v3, and OAuth2 for the authentication.

My initial thought was because I wasn't using async uploads. I was literally trying to do:

Dim UploadRequest As FilesResource.CreateMediaUpload = Service.Files.Create(CurrentFile, Stream, CurrentFile.MimeType)
Dim Response = UploadRequest.Upload()

Which never seemed to work.

I then read into things and found the Google Drive API Allows for resumable uploads, which can pick up mid-upload. I thought...maybe that might do it, since the files are quite large and maybe there's an issue here.

And this is where I stand...I cant seem to get the Async upload working...

I have replaced the upload() with uploadASync() as below

Dim Response = UploadRequest.UploadAsync()

And no errors are thrown. Surely it cant be that simple to make it work?

Nope.

After each of the UploadAsync()'s, I ran a short loop which returns Response.Status.ToString, just to show me the status of the async upload and try to see what was going on... All that I get is "Waiting for activation"...

I'm not sure where to go from here.

Ill throw the code relevant to uploading below so you can see whats going on.. There are, however, other bits which run before, during, and after...such as an internet test (to check that the connection is live before attempting any sort of upload), debug lines (where its writing to a file) etc... I would also like to apologize if any of my code seems like bad-practice. I try to work with what I feel is right, and haven't had all that much experience working with devs/etc to learn about good practices. But on the grounds that the basic core functionality is working, in that it does actually upload my test text files, I guess I'm doing something right. Constructive feedback and helpful tips are welcomed, please be gentle though!

Main Sub

Sub Main()
    Try
        createDirectory()
        UploadFiles()
    Catch ex As Exception
        *logs any errors here*
    End Try
End Sub

The CreateDirectory() works well, and returns the folderID of the folder it created. No issue here.

Drive service (Working well)

Private Sub CreateService()

    Dim ClientId = *obviously removed*
    Dim ClientSecret = *Also removed*
    Dim MyUserCredential As UserCredential = GoogleWebAuthorizationBroker.AuthorizeAsync(New ClientSecrets() With {.ClientId = ClientId, .ClientSecret = ClientSecret}, {DriveService.Scope.Drive}, "user", CancellationToken.None).Result
    Try
        Service = New DriveService(New BaseClientService.Initializer() With {.HttpClientInitializer = MyUserCredential, .ApplicationName = "Google Drive CCTV Uploader"})

    Catch Ex As Exception
        *logging any errors to file*
    End Try
End Sub

Code to actually upload the file (Anndddd this is where it falls apart)

Private Sub UploadFiles()
    For Each footageFile As IO.FileInfo In directory.GetFiles
        If Service.ApplicationName <> "Google Drive CCTV Uploader" Then CreateService()
        UploadSuccess = False
        If footageFile.Extension.Equals(".xsf") AndAlso (Now - footageFile.CreationTime).Days >= 7 Then

            CurrentFile.Name = blah *other metadata added here*

            Dim ByteArray As Byte() = System.IO.File.ReadAllBytes(*Path to file*)
            Dim Stream As New System.IO.MemoryStream(ByteArray)
            Dim UploadRequest As FilesResource.CreateMediaUpload = Service.Files.Create(CurrentFile, Stream, CurrentFile.MimeType)
            Try
                Dim Response = UploadRequest.UploadAsync()
                If Response.Status.ToString = "Failed" Then
                    Throw New System.Exception(Response.Exception.ToString)
                Else
                    UploadSuccess = True
                End If
            Catch ex As Exception
                *log error*
            End Try
            If UploadSuccess = True Then
                footageFile.Delete()
            End If
        End If
    Next
    Console.WriteLine("Backup Complete!")
    System.Threading.Thread.Sleep(6000)
    Exit
End Sub

And that's it! Any thoughts would be appreciated. Thanks in advance :)

EDIT 1

So, as Adrian commented, I made a change to the layout of the code such that is now ran as below...

Using StreamBob = New System.IO.FileStream(FilePath, System.IO.FileMode.Open, System.IO.FileAccess.Read)
    Dim request As FilesResource.CreateMediaUpload = Service.Files.Create(CurrentFile, StreamBob, CurrentFile.MimeType)
    request.Upload()
End Using

However, this still did not fix it sadly. The Request.upload() never actually uploads any files (back to using very small txt files). No exceptions are thrown either, it just continues past the request.upload()...

Wondering what is going wrong I, again, modified the code slightly so I had

dim response = request.Upload()
console.writeline(reponse.status.tostring)

Which returned a pretty interesting result...500 error...stack trace below

response.status.tostring returned "Failed"
Google.Apis.Requests.RequestError
 [500]
 No individual errors

at Google.Apis.Upload.ResumableUpload`1.<HandleResponse>d__95.MoveNext() in C:\Apiary\v1.20\google-api-dotnet-client\Src\Support\GoogleApis\Apis\[Media]\Upload\ResumableUpload.cs:line 676
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload`1.<SendNextChunkAsync>d__94.MoveNext() in C:\Apiary\v1.20\google-api-dotnet-client\Src\Support\GoogleApis\Apis\[Media]\Upload\ResumableUpload.cs:line 654
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Google.Apis.Upload.ResumableUpload`1.<UploadCoreAsync>d__91.MoveNext() in C:\Apiary\v1.20\google-api-dotnet-client\Src\Support\GoogleApis\Apis\[Media]\Upload\ResumableUpload.cs:line 581

I also tried request.uploadAsync() again, however this just gave me the "Waiting for activation" again....