Invalid Upload Request

Question

I'm getting the following response, when trying to upload data to BigQuery using the Google API Client libraries for Go.

{
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "badRequest",
    "message": "Invalid Upload Request"
   }
  ],
  "code": 400,
  "message": "Invalid Upload Request"
 }
}

My job descriptor looks like this:

j := &bigquery.Job{
    Configuration: &bigquery.JobConfiguration{
        Load: &bigquery.JobConfigurationLoad{
            DestinationTable: &bigquery.TableReference{
                projectId,
                "xyz",
                name + "_" + yyyymmdd,
            },
            SkipLeadingRows: 1,
            FieldDelimiter:  "|",
            MaxBadRecords:   3,
            Schema: &bigquery.TableSchema{
                []*bigquery.TableFieldSchema{
                    {Name: "f1", Type: "STRING"},
                    {Name: "f2", Type: "STRING"},
                    {Name: "f3", Type: "STRING"},
                    {Name: "f4", Type: "STRING"},
                    {Name: "f5", Type: "STRING"},
                    {Name: "f6", Type: "STRING"},
                    {Name: "f7", Type: "STRING"},
                    {Name: "f8", Type: "STRING"},
                    {Name: "f9", Type: "STRING"},
                    {Name: "f10", Type: "STRING"},
                    {Name: "f11", Type: "STRING"},
                    {Name: "f12", Type: "STRING"},
                    {Name: "f13", Type: "STRING"},
                    {Name: "f14", Type: "STRING"},
                    {Name: "f15", Type: "STRING"},
                    {Name: "f16", Type: "STRING"},
                    {Name: "f17", Type: "STRING"},
                },
            },
        },
    },
}

FWIW, other calls using this library work fine, so I have ruled out oauth problems, etc.

EDIT: I captured the request before it went out. Please note that this is before the http.Client is able to add the oauth headers. The actual request is large, so I have shortened it and removed the identifiers.

POST /upload/bigquery/v2/projects/.../jobs HTTP/1.1
Host: www.googleapis.com
User-Agent: google-api-go-client/0.5
Content-Length: 56369074
Content-Type: multipart/related; boundary=2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: application/json

{"configuration":{"load":{"destinationTable":{"projectId":"...","datasetId":"...","tableId":"..."},"skipLeadingRows":1,"fieldDelimiter":"|","maxBadRecords":3,"schema":{"fields":[{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"},{"name":"...","type":"STRING"}]}}}}

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf
Content-Type: text/plain; charset=utf-8

H0|H1|H2|H3|H4|H5|H6|H7|H8|H9|H10|H11|H12|H13|H14|H15|H16
f0_0|f0_1|f0_2|f0_3|f0_4|f0_5|f0_6|f0_7|f0_8|f0_9|f0_10|f0_11|f0_12|f0_13|f0_14|f0_15|f0_16
f1_0|f1_1|f1_2|f1_3|f1_4|f1_5|f1_6|f1_7|f1_8|f1_9|f1_10|f1_11|f1_12|f1_13|f1_14|f1_15|f1_16
# Three lines
# of comments
# in the trailer

--2220cacc03485c8143026fe3f0c40dcb1aaec8c8c2426e69adf620fc12cf--

laslowh · Accepted Answer · 2012-05-14T15:34:57.403

1

It turns out that this was caused by a bug in the library that was not sending the required "uploadType" query parameter. See these diffs for a patch that fixes the bug.

However this exposed another bug in the library that I am currently working with the author to fix. I will update this answer when that fix is in.

EDIT: As of this patch, the bug referenced above is fixed.

edited May 14 '12 at 15:34

answered May 10 '12 at 16:20

laslowh

8,482
5
34
45

The links don't seem to work any more, can you please let us know which query parameter, and which value it was needed? If you could provide us with an updated link it would be even better. – Gus Jul 23 '12 at 18:02
Links seem to work for me. The first points to a patch that adds the line "params.Set("uploadType", "multipart")" at about line 1490. The second makes some changes to ConditionallyIncludeMedia (you'll need to click on the "diff" link to see these) – laslowh Jul 23 '12 at 18:23

score 0 · Answer 2 · answered May 09 '12 at 21:20

0

Let me start with the caveat that I'm not a Go expert. That said, it looks like the problem is with the data that you're uploading. It looks like you're doing a media upload request, since you're not providing a source Google Storage path in the job configuration, and the error you sent comes from the media uploader.

However, you don't show the call to JobInsertCall.Media -- how are you setting the Media field? Can you verify that it is valid data, or try with dummy data?

answered May 09 '12 at 21:20

Jordan Tigani

26,089
4
60
63

I do have a call to JobInsertCall.Media in the code. I even hacked in a call to http.Request.Write(), right before the library makes the call, and can see that the MIME multipart payload at least looks OK. – laslowh May 09 '12 at 21:47
Can you share the posted data being sent via the http request (with the authorization headers redacted)? – Jordan Tigani May 10 '12 at 01:06
Please see edit above for capture of the request. If you can tell me how to get it to you, I can give you the original request. – laslowh May 10 '12 at 13:34
Jordan, thanks for looking at this. In the end it was a bug in the library. Please see my answer for details. – laslowh May 10 '12 at 16:21

Invalid Upload Request

2 Answers2