0

I'm bulk-uploading documents to a CouchDB with a System.Net.WebClient like this:

webClient.UploadString("couchdb.host.com:5984/db/_bulk_docs", "POST", json_data);

This usually works fine, but sometimes I get a WebException with a 'Bad Request' Status (400). Unfortunately, that's the only information I get, so I don't really know what causes the exception.

I checked the logfiles under /var/log/couchdb/couchdb.log, but I did not get any further info there:

[notice] 2019-09-10T15:26:09.173949Z couchdb@host.com <0.5318.568> f50ab02d22 host.com:5984 x.x.x.x undefined POST /db/_bulk_docs 400 ok 75

So my questions are:

  • Is there more information hidden somewhere in the WebException object and if so, where do I find it?

  • is there a way to get more informations on the server side?

My suspicion is that the documents I am trying to upload, are too big. How can I check that?

Update:

I dumped the faulting request into a file with:

        catch (WebException ex)
        {
            File.WriteAllText("C:\\temp\\couchdb_bad_request.json", json_data);
        }

and tried it again with:

curl -X POST http:/couchdb.host.com:5984/db/_bulk_docs -H "Content-Type: application/json" -d @couchdb_bad_request.json

This time I got a 201. So what's the difference between a curl and a WebClient request?

My json data look like this:

{
  "docs": [
    {
      "Type": "Image",
      "_attachments": {
        "123_original.jpg": {
          "content_type": "image/jpeg",
          "data": "<base64-encoded image data>"
        }
      },
      "ID": "123",
      "_id": "123",
      "FileName": "some_file.jpg",
      "AttachmentMetaData": {
        "original": {
          "Width": 864,
          "Height": 576,
          "MD5Checksum": "a20ecae8e3a7df8cc8558a71efc2b573"
        }
      }
    },
    {...}
  ]
}
Ben
  • 4,486
  • 6
  • 33
  • 48
  • What's your JSON data? Can you try to send it through curl or another tool? – Alexis Côté Sep 11 '19 at 03:01
  • @AlexisCôté I updated my question. – Ben Sep 11 '19 at 15:30
  • What's the content-type of your request? – Alexis Côté Sep 11 '19 at 19:19
  • application/json like in the curl request. Some requests are fine, others return a 400. – Ben Sep 12 '19 at 10:06
  • Maybe an encoding issue? What's the response when you have a 400 status code? That might be helpful to get the error : https://stackoverflow.com/questions/15235308/using-a-webclient-and-c-how-do-i-get-returned-data-even-when-the-response-is – Alexis Côté Sep 12 '19 at 19:12
  • Thanks, that was it! I saw the Response object and looked at all properties to get the info, but I oversaw the GetResponseStream() method, which gave me the reason: invalid UTF-8 encoding. I found out that the default encoding for the WebClient isn't UTF-8, which sometimes caused a problem. So if you write that into an answer, I can accept it! – Ben Sep 16 '19 at 12:24

2 Answers2

1

With the informations of your problem, the request data must be wrong at some one point. Since some of the request are ok and some are not, this is probably due to an encoding issue.

You might want to look into the response from CouchDB which normally mention the encoding issue.

To read the response:

catch ( WebException exception )
{
   string responseText;

   using(var reader = new StreamReader(exception.Response.GetResponseStream()))
   {
     responseText = reader.ReadToEnd();
   }
}
Alexis Côté
  • 3,670
  • 2
  • 14
  • 30
0

On node, you can read the exception response with await error.json()

try {
  await request();
} catch(error) {
  console.log(await error.json());
}
Carter
  • 1,184
  • 11
  • 5