503 error with multipart upload GoogleDrive Api v2

Question

I'm attempting to work with the Google Drive API in Delphi XE2, and thus far, I have just about everything working. One thing I'm struggling with is the multipart upload. When I attempt to upload a file, I get a 503 error. Normally, that should indicate a problem with the server. However, when I send an upload request with the same body and headers to the same URL using fiddler rather than my API, the file is uploaded successfully. This tells me there has to be a problem with my code. This particular function is a mess, but here it is.

function TGoogleDriveApi.MultipartUpload(aStream : TStream; aTitle : string = 'Untitled';
  aDescription : string = ''; mimeType : string = 'application/octet-stream';
  indexableText : IGoogleDriveIndexableText = nil;
  lastViewedByMeDate : string = ''; modifiedDate : string = '';
  parents : IGoogleDriveParentList = nil) : IGoogleDriveFile;
var
  json, url, body, boundry, contentType : string;
  ss : TStringStream;
  ms : TMemoryStream;
  lHttpHelper : IHttpHelper;
  streamBytes : TBytes;
begin
  boundry := 'a_boundry';
  body := '--'+boundry+sLineBreak;
  body := body + 'Content-Type: application/json; charset=UTF-8';
  lHttpHelper := THttpHelper.Create;
  if(Token.IsExpired) then
    Token.Refresh(TokenUrl, OAuth.ClientId, OAuth.ClientSecret);
  url := 'https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart&access_token='+lHttpHelper.UrlEncode(Token.access_token);
  json := '{';
  json := json + '"title": "'+aTitle+'"';
  if(aDescription <> '') then
    json := json + ', "description": "'+aDescription+'"';
  if(lastViewedByMeDate <> '') then
    json := json + ', "lastViewedByMeDate": "'+lastViewedByMeDate+'"';
  json := json + ', "mimeType": "'+mimeType+'"';
  if(modifiedDate <> '') then
    json := json + ', "modifiedDate": "'+modifiedDate+'"';
  json := json + '}';
  body := body + sLineBreak + sLineBreak + json + sLineBreak + sLineBreak + '--'+boundry;
  body := body + sLineBreak + 'Content-Type: '+mimeType + sLineBreak  + sLineBreak;
  body := body + 'some test text from the api' + sLineBreak + sLineBreak + '--'+boundry+'--';

  ss := TStringStream.Create;
  ss.WriteString(body);
  ss.Position := 0;
  ms := TMemoryStream.Create;
  ms.Write(ss,ss.Size);
  SetLength(streamBytes,aStream.Size);
  aStream.Read(streamBytes,aStream.Size);
  ms.Write(streamBytes[0],aStream.Size);
  ss.Clear;
  ss.Position := 0;
  ss.WriteString(sLineBreak + sLineBreak + '--'+boundry+'--');
  ss.Position := 0;
  ms.Write(ss,ss.Size);

  contentType := 'multipart/related; boundary="'+boundry+'"';
  json := lHttpHelper.PostResponse(url,contentType,ms);
  FreeAndNil(ss);
  FreeAndNil(ms);
  Result := nil;
end;

The line that causes problems is the lHttpHelper.PostResponse call. The code for that is shown here:

function THttpHelper.PostResponse(url, contentType : string; aStream : TStream) : string;
var
  lHTTP: TIdHTTP;
  lStream : TStringStream;
  handler : TIdSSLIOHandlerSocketOpenSSL;
begin
  lStream := TStringStream.Create;
  lHTTP := TIdHTTP.Create(nil);
  handler := TidSSLIOHandlerSocketOpenSSL.Create(nil);
  handler.SSLOptions.Method := sslvSSLv3;
  lHTTP.IOHandler := handler;
  try
    lHTTP.Request.ContentType := contentType;
    lHTTP.Post(url,aStream,lStream);
    lStream.Position := 0;
    Result := lStream.ReadString(lStream.Size);
      //except
  finally
    lStream.Free;
    lHTTP.Free;
    handler.Free;

    lStream := nil;
    lHTTP := nil;
    handler := nil;
  end;
end;

I'm currently calling the MultipartUpload function from my test, shown here

procedure TestIGoogleDriveApi.TestMultipartUpload;
var
  ReturnValue : IGoogleDriveFile;
  fs : TFileStream;
begin
  fs := TFileStream.Create('testupload.jpg',fmOpenRead);
  ReturnValue := FIGoogleDriveApi.MultipartUpload(fs,'Test Multipart Image Upload from API.txt','test file','image/jpeg');
  FreeAndNil(fs);
  if(ReturnValue = nil) then
    fail('ReturnValue cannot be nil');
end;

Any ideas what might be causing the problems? I'm not even sure what to suspect at this point.

If Fiddler works and your program doesn't, then use Wireshark to see what's really different between the two. — Rob Kennedy, Sep 26 '12 at 20:38
@RobKennedy It's over https, and I'm not sure where to find the RSA key to decrypt the https traffic with wireshark. — nick, Sep 26 '12 at 22:15
Apart from 503, what else is in the response? The response body normally has some sort of clue. — pinoyyid, Sep 27 '12 at 03:50
The documentation (https://developers.google.com/drive/manage-uploads) says you need to include the content-type and content-length in the request header, but I did not see you include the content-length in the request header in your function THttpHelper.PostResponse — Hendra, Sep 27 '12 at 04:58
@Hendra, I thought the content-length was auto generated by indy? — nick, Sep 27 '12 at 07:02
@pinoyyid there's nothing else in the response. The body is empty. — nick, Sep 27 '12 at 19:09
@Hendra I sent a request using fiddler without a Content-Length and I received a 504 error rather than the 503 I'm experiencing with my program. I also tried manually setting the Content-Length header in my code and I still got the same 503 error — nick, Sep 27 '12 at 19:13
@nick, maybe you need to implement Exponential backoff as mentioned in the bottom part of the (https://developers.google.com/drive/manage-uploads) — Hendra, Sep 28 '12 at 05:39

score 1 · Accepted Answer · answered Sep 28 '12 at 07:48

1

You'll need to somehow follow @RobKennedy's advice. You need to see what's happening on the wire to debug this. Try replacing the https drive url with http and then trace with Wireshark.

@Hendra is also correct, that generally your app needs to implement exponential backoff and retry for 500 errors, although I suspect that isn't your specific problem (yet :-))

answered Sep 28 '12 at 07:48

pinoyyid

21,499
14
64
115

I was able to debug it by writing a simple listener in .NET that wrote the headers and body to text files. They were very different. I was able to get everything working in the end though. – nick Sep 28 '12 at 23:06

score 1 · Answer 2 · answered Sep 28 '12 at 23:11

After changing two things in my code, and doing some substantial cleanup, I was able to get the upload working. The main changes I made were writing everything directly to the TMemoryStream rather than to a TStringStream then to a TMemoryStream and using AnsiStrings rather than strings. Here is my cleaned up code.

function TGoogleDriveApi.MultipartUpload(aStream: TStream; aFile: TGoogleDriveFileUp) : IGoogleDriveFile;
var
  json, url, boundary, body : AnsiString;
  lHttpHelper : IHttpHelper;
  ms : TMemoryStream;
  ss : TStringStream;
  writer : IRtSerializeWriter;
  reader : IRtSerializeReader;
begin
  if(Token.IsExpired) then
    Token.Refresh(TokenUrl, OAuth.ClientId, OAuth.ClientSecret);
  lHttpHelper := THttpHelper.Create;
  ss := TStringStream.Create;
  writer := TRtJsonSerializeWriter.Create;
  writer.SaveToStream(ss,aFile);
  ss.Position := 0;
  json := ss.ReadString(ss.Size);
  FreeAndNil(ss);
  url := 'https://www.googleapis.com/upload/drive/v2/files?uploadType=multipart&access_token='+lHttpHelper.UrlEncode(Token.access_token);
  boundary := 'a_bondary';
  body := '--'+boundary+sLineBreak;
  body := body + 'Content-Type: application/json; charset=UTF-8' + sLineBreak + sLineBreak;
  body := body + json + sLineBreak + sLineBreak +'--'+boundary;
  body := body + sLineBreak + sLineBreak;
  ms := TMemoryStream.Create;
  ms.Write(body[1],Length(body));
  ms.CopyFrom(aStream, aStream.Size);
  body := sLineBreak + sLineBreak + '--' + boundary + '--';
  ms.Write(body[1],Length(body));
  ms.Position := 0;
  json := lHttpHelper.PostResponse(url,'multipart/related; boundary="'+boundary+'"',ms);
  ss := TStringStream.Create;
  ss.WriteString(json);
  ss.Position := 0;
  reader := TRtJsonSerializeReader.Create;
  Result := TGoogleDriveFileDown.Create;
  reader.LoadFromStream(ss,Result as TGoogleDriveFileDown);
  FreeAndNil(ms);
  FreeAndNil(ss);
end;

503 error with multipart upload GoogleDrive Api v2

2 Answers2