1

I have a (big) CSV file with some data. And I have a importer from code.kx.com using .Q.fsn 

colnames:`Symbol`Date`Time`Sequence`Exchange`Type`Level`Condition`Price`Size`BuyerID`SellerID
.Q.fsn[{`:newCreatedFile upsert flip colnames!("SDTISSISFISS";",") 0:x};`:C:/myDir/data.csv 5000000]

This code creates a file named newCreatedFile with the data from data.csv, assuming a big data file, processed the data in chunks of 5000000 bytes.

Question:

I want to create 2 separate files from this data, and let's say the basis for the distinction is the values in the "Condition" column. For each row, if the value in Condition column is x,y,or z put in file A.csv, else B.csv.

Here is a pseudocode for the if statement:

$[Condition in `x`y`z; Afunction ; Bfunction]
Afunction:{`:newA upsert flip ...};
Bfunction:{`:newB upsert flip ...};

How would I have to set up the if statement? I currently have:

$[datatable.Condition = `SomeCondition; fileA;fileB]

But I get a type error. How do I match each value of a certain column?

Should it check while importing the original data file or after the data file has been created into a table in kdb?

chrisaycock
  • 36,470
  • 14
  • 88
  • 125
user2498110
  • 105
  • 1
  • 1
  • 9

1 Answers1

0

You haven't parsed the column yet in your example when you check the condition, this needs to be done first. You could use something like the following:

foo:{t:flip colnames!("SDTISSISFISS";",") 0:x;
     `:newA upsert select from t where Condition = `SomeCondition;
     `:newB upsert select from t where not Condition = `SomeCondition}

.Q.fsn[foo;`:C:/myDir/data.csv;5000000]
user1895961
  • 1,176
  • 9
  • 22
  • Let me just see if i am understanding your code correctly, you made a function named foo and the first line assigns the variable t to a table imported from file x. second and third line creating newA or newB based on Conditions, how did you do this without using "$" or if operators? – user2498110 Jun 18 '13 at 19:23
  • It's done in the select statement. So t is the kdb form of the chunk of the table. Then we select the rows from t where Condition = `SomeCondition and upsert into newA. And we then select the rows from t where the condition doesn't hold and upsert them into newB. – user1895961 Jun 18 '13 at 19:48
  • Oh i see. Yea there was no way for me to do this using $ or if-else because the condition has to evaluate to a boolean and there is no way for me to match that value type with the Condition values which are symbols.. – user2498110 Jun 18 '13 at 19:58
  • `Condition in \`x\`y\`z` evaluates to a boolean. eg `1b = \`x in \`x\`y` – skeevey Jun 19 '13 at 00:42