0

Could you, please, let me know if it is possible to connect to a password protected KDB instance with capability?

I seem to be able to either connect with capability to a KDB instance without password protection or to connect without capability to a password protected KDB instance, but I can not do both.

Here is what I do:

  1. Create file users.txt with the following content user:202cb962ac59075b964b07152d234b70 (the hash corresponds to password 123: echo -n '123' | md5sum)
  2. Start KDB server q -p 11111 -u users.txt
  3. Attempt to connect to the server from another KDB instance:
q)h: hopen `$(":localhost:11111:user:123"); / works
q)h: hopen `$(":localhost:11111:user:123\\6");
'access
  [0]  h: hopen `$(":localhost:11111:user:123\\6");
          ^

On the other hand, if the server is started without password protection (q -p 11111), then connecting to it with capability works fine:

q)h: hopen `$(":localhost:11111:user:123\\6"); / works
q)h: hopen `$(":localhost:11111::\\6");        / works

How can I do both, i.e. have a password protected KDB server and be able to connect to it with a capability?

Thank you very much for your help!

S.V
  • 2,149
  • 2
  • 18
  • 41
  • Hi, three questions: 1. why do you need to specify capability? 2. Do you plan to do this for a non-kdb application/interface (C++, java?)? 3. Are you sure that your second example is actually working? i.e. does changing the "\\6" actually give you different capabilities? I've a feeling it's just treating "\\6" as part of the password and simply ignoring it when there's no access protection. This handshake occurs at a lower level than `hopen` so a straight kdb-to-kdb `hopen` I believe does this handshake automatically under the covers. In other words you don't specify it when using `hopen` – terrylynch May 31 '22 at 21:50
  • I need to specify the capability to be able to download a large data table in one command to save it into a historical DB, which simplifies the code significantly. The alternative is to download the data in small chunks with subsequent concatenation. It is certainly important for at least older versions of KDB: attempting to download from them data chunks larger than 2GB in size results in `'limit` error. No, I am not certain that in my second example I get additional capabilities. If I do it incorrectly, what is the correct syntax? – S.V May 31 '22 at 22:11
  • I access data in KDB from code written in Q/C++/python. The example above is needed for code written in Q. – S.V May 31 '22 at 22:16
  • What version of kdb are you running? > 2gb over IPC is only supported in >=3.4 https://code.kx.com/q/releases/ChangesIn3.4/ – Matt Moore May 31 '22 at 22:52
  • Some data lives in KDB served by version 3.1 and to access this data, I have to download it in chunks. But some data lives in version 4.1 and to access this data I open connections with capabilities: it works great in C++. – S.V May 31 '22 at 23:01
  • Have you tried connecting to the -u enabled process with capability from your C++ application? As Terry has said, I don't think you can specify capability between 2 kdb processes as this will happen automatically behind the scenes. 123\6 is being interpreted as your password giving the access error – Matt Moore Jun 01 '22 at 08:47
  • 1
    @MattMoore I have not tried connecting to a -u enabled KDB process with capability from a C++ application, but I did try connecting from my C++ applications to a KDB without having capability for 1TB message-size enabled and I hit the 2GB message limit. However, @terrylynch gave me an idea to test if I need to set the capability 6 for `Q <=> Q` communication at all, and tests seem to indicate that it is enabled by default in KDB 4.1 since I was able to receive 7GB message with no problems without explicitly asking for any capability in the `hopen` call. – S.V Jun 01 '22 at 12:56
  • 1
    Haha, got a reply from KX support: _"For q<->q connections you don’t need to pass the capability byte."_ – S.V Jun 01 '22 at 13:17

0 Answers0