kdb/q: Query multiple handles with hopen

Question

I would like to be able to query several handles at once, where the tables have the same format like:

handles: 8000,8001,8003 tables: foo

Want to do something like:

x:hopen `8000`8001`8003
x select from foo col1,col2

So i get rows from each foo table on each handle.

Is there a way to achieve this?

Thank you

score 7 · Accepted Answer · answered Jul 26 '17 at 14:28

7

Use 'each' to hopen each handle

  q)h:hopen each 8000 8001 8002
  q)h
  476 480 484i

Use apply each-left to send the same query to each server

  q)r:h@\:"select col1,col2 from foo"
  q)r
  +`col1`col2!(1 2;2 3)
  +`col1`col2!(1 2;2 3)
  +`col1`col2!(1 2;2 3)

Then you'll have to raze the result:

 q)show res:raze r
col1 col2
--------- 
 1    2
 2    3
 1    2
 2    3
 1    2
 2    3

answered Jul 26 '17 at 14:28

jomahony

Thanks will try it asap – Alim Hasanov Jul 26 '17 at 14:56
2

Don't forget to `hclose each h`. – Alexander Belopolsky Jul 26 '17 at 21:17
I have a follow up question. How do I do the same from a remote machine: h:hopen each `:server: 8000 8001 8003 and then r: h@\:"select from tbl col1 col2 col3" – Alim Hasanov Jul 27 '17 at 11:37
..And also what if I have handles on server1, server2, and etc what would be the syntax? – Alim Hasanov Jul 27 '17 at 11:42

score 1 · Answer 2 · answered Jul 26 '17 at 21:28

1

If you are not planning to reuse the handles, you can do

q)raze`::8000`::8001`::8003@\:"select from foo col1,col2"

answered Jul 26 '17 at 21:28

and can use peach so that requests are sent immediately – effbiae Jul 27 '17 at 08:53
I will probably be reusing them but Thank you Alexander. – Alim Hasanov Jul 27 '17 at 12:26
Jack can you please elaborate on the use of peach? – Alim Hasanov Jul 27 '17 at 12:27
@AlimHasanov in both answers so far, the result has to come back on port 8000 before the query is sent on port 8001. That's synchronous communication (aka "get"). It could be done as 3 concurrent "gets" in 3 slaves using [peach](http://code.kx.com/q/ref/peach/). As well as peach, async ("set") communication can help. – effbiae Jul 27 '17 at 13:01

effbiae · Answer 3 · 2017-07-27T14:20:39.527

Same as other answers, but more complicated using set (neg h) rather than get (h)

The cookbook/load-balancing helps in this example, too.

q)h:hopen each 8000 8001 8002
q)h
476 480 484i
q)r:(0#0i)!()               /dictionary of handles and results

Set the callback for the response from the servers

q).z.ps:{@[`r;.z.w;:;x]}

Send a "set" query to each handle

q)(neg h)@\:({(neg .z.w)value"select col1,col2 from foo"};`)

Wait until all messages have a response

q)h .\:()

Finally, putting the result together

q)raze r h

The only advantage is concurrency.

As @AlexanderBelopolsky pointed out, don't forget

q)hclose each h

3 Answers3