1

I am trying to create an order book data structure where a top level dictionary holds 3 basic order types, each of those types has a bid and ask side and each of the sides has a list of tables, one for each ticker. For example, if I want to retrieve all the ask orders of type1 for Google stock, I'd call book[`orderType1][`ask][`GOOG]. I implemented that using the following: 

bookTemplate: ([]orderID:`int$();date:"d"$();time:`time$();sym:`$();side:`$();
orderType:`$();price:`float$();quantity:`int$());
bookDict:(1#`)!enlist`orderID xkey bookTemplate;
book: `orderType1`orderType2`orderType3 ! (3# enlist(`ask`bid!(2# enlist bookDict)));

Data retrieval using book[`orderType1][`ask][`ticker] seems to be working fine. The problem appears when I try to add new order to a specific order book e.g:

testorder:`orderID`date`time`sym`side`orderType`price`quantity!(111111111;.z.D;.z.T;
`GOOG;`ask;`orderType1;100.0f;123);
book[`orderType1][`ask][`GOOG],:testorder;

Executing the last query gives 'assign error. What's the reason? How to solve it?

Thomas Smyth - Treliant
  • 4,993
  • 6
  • 25
  • 36
dhk
  • 13
  • 2

1 Answers1

3

A couple of issues here. First one being that while you can lookup into dictionaries using a series of in-line repeated keys, i.e.

q)book[`orderType1][`ask][`GOOG]
orderID| date time sym side orderType price quantity
-------| -------------------------------------------

you can't assign values like this (can only assign at one level deep). The better approach is to use dot-indexing (and dot-amend to reassign values). However, the problem is that the value of your book dictionary is getting flattened to a table due to the list of dictionaries being uniform. So this fails:

q)book . `orderType1`ask`GOOG
'rank

You can see how it got flattened by inspecting the terminal

q)book
          | ask
----------| -----------------------------------------------------------------
orderType1| (,`)!,(+(,`orderID)!,`int$())!+`date`time`sym`side`orderType`pric
orderType2| (,`)!,(+(,`orderID)!,`int$())!+`date`time`sym`side`orderType`pric
orderType3| (,`)!,(+(,`orderID)!,`int$())!+`date`time`sym`side`orderType`pric

To prevent this flattening you can force the value to be a mixed list by adding a generic null

q)book: ``orderType1`orderType2`orderType3 !(::),(3# enlist(`ask`bid!(2# enlist bookDict)));

Then it looks like this:

q)book
          | ::
orderType1| `ask`bid!+(,`)!,((+(,`orderID)!,`int$())!+`date`time`sym`side`ord
orderType2| `ask`bid!+(,`)!,((+(,`orderID)!,`int$())!+`date`time`sym`side`ord
orderType3| `ask`bid!+(,`)!,((+(,`orderID)!,`int$())!+`date`time`sym`side`ord

Dot-indexing now works:

q)book . `orderType1`ask`GOOG
orderID| date time sym side orderType price quantity
-------| -------------------------------------------

which means that dot-amend will now work too

q).[`book;`orderType1`ask`GOOG;,;testorder]
`book
q)book
          | ::
orderType1| `ask`bid!+``GOOG!(((+(,`orderID)!,`int$())!+`date`time`sym`side`o
orderType2| `ask`bid!+(,`)!,((+(,`orderID)!,`int$())!+`date`time`sym`side`ord
orderType3| `ask`bid!+(,`)!,((+(,`orderID)!,`int$())!+`date`time`sym`side`ord

Finally, I would recommend reading this FD whitepaper on how to best store book data: http://www.firstderivatives.com/downloads/q_for_Gods_Nov_2012.pdf

terrylynch
  • 11,844
  • 13
  • 21
  • Many thanks. Can you please explain why append (comma) should not be used on a keyed table? It seems that **.[\`book;\`orderType1\`ask\`GOOG;,;testorder]** works fine (i.e. appending new entries if order ID is not in the table and updating the existing entry otherwise). It is also much faster (~36s to upsert 100000 orders and ~7s to append 100000 orders). They are doing that in the FD paper you linked as well (e.g. updBySymSide2 where the underlying table is keyed on price). – dhk Apr 02 '16 at 06:25
  • You're right, my bad. I thought it had failed in my testing. I'll edit my answer thanks – terrylynch Apr 04 '16 at 13:19