CrateDB cannot connect in HttpEndpoint to remote secure server

Question

I've been testing crateDB 3.3.4 for a few weeks now, and I've always been connecting through localhost (127.0.0.1) in http. I've been bulk importing data with HttpEndpoint on my localhost successfully.

I'm now testing on a cloud cluster in https. I manage to open Chrome and logon on the server, but I cannot manage to remote in C# via HttpEndpoint.

HttpWebRequest httpWebRequest = (HttpWebRequest)WebRequest.Create("https://remoteServerUrl:4200/_sql");
      httpWebRequest.Method = "POST";
      httpWebRequest.Credentials = new NetworkCredential("username", "password");
      httpWebRequest.Timeout = 600000;
      httpWebRequest.ContentType = "application/json";

      using(var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream()))
      {
        streamWriter.Write(request);
        streamWriter.Flush();
      }

      HttpWebResponse response = (HttpWebResponse)httpWebRequest.GetResponse();
      success = response.StatusCode == HttpStatusCode.OK;

When I try the following code, I always get "The underlying connection was closed: An unexpected error occurred on a send." and in the exception, I see "Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host."

Any Idea what I'm doing wrong here? I was doing the same thing before, on my localhost, but without specifying credentials, and it was working great.

score 1 · Answer 1 · answered Jul 04 '19 at 19:42

It's not a permanent solution but for test purposes it works. I worked around the problem by adding the Authorization Header directly in my httpwebRequest.

httpWebRequest.Headers.Add(HttpRequestHeader.Authorization, "SomeBasicauthenticationToken");

To find the authentication token to pass, simply put the username and password in a basic authentication token generator like this one : https://www.blitter.se/utils/basic-authentication-header-generator/

But if anyone has a permanent solution, I would still like to see what it is.

what auth methods do you have set up in crate? Im also assuming you setup ssl as per their recommendations? — metase, Jul 05 '19 at 10:13

score 1 · Accepted Answer · answered Jul 05 '19 at 10:21

Assuming from your code sample you are using password authentication. Permanent solution would indeed be using autorization header.

As described here https://crate.io/docs/crate/reference/en/latest/admin/auth/methods.html#password-authentication-method

CrateDB cannot connect in HttpEndpoint to remote secure server

2 Answers2