How can I use Deedle to get every row from some key and below?

Question

I want to return every value up to and including some key. Whilst I could generate every such key and chuck them all into the Get, I suspect this will inefficiently search for the value of every key.

Inspired by this answer, I have come up with the following

let getAllUpTo key (frame:Frame<'key,'col>) : Frame<'key, 'col> =
    let endRng = frame.RowIndex.Locate key
    let startRng = frame.RowIndex.KeyRange |> fst |> frame.RowIndex.Locate
    let fixedRange = RangeRestriction.Fixed (startRng, endRng)
    frame.GetAddressRange fixedRange

Is there a built in method for doing this efficiently?

score 2 · Accepted Answer · answered Nov 15 '20 at 20:42

2

If you want to access a sub-range of a data frame with a specified starting/ending key, you can do this using the df.Rows.[ ... ] indexer. Say we have some data indexed by (sorted) dates:

let s1 = series [
  let rnd = Random()
  for d in 0 .. 365 ->
    DateTime(2020, 1, 1).AddDays(float d) => rnd.Next() ]

let df = frame [ "S1" => s1 ]

To get a part of the data frame starting/ending on a specific date, you can use:

// Get all rows from 1 June (inclusive)  
df.Rows.[DateTime(2020, 6, 1) ..]

// Get all rows until 1 June (inclusive)  
df.Rows.[.. DateTime(2020, 6, 1)]

The API you are using is essentially what this does under the cover - but you are using a very low-level operations that you do not typically need to use in user code.

answered Nov 15 '20 at 20:42

Tomas Petricek

240,744
19
378
553

Thanks. For the avoidance of doubt, using the `..` does not result in a full generation of every date and then an iteration of all those keys, but is just a shorthand for what I did in my function. Is that correct? – Chechy Levas Nov 15 '20 at 20:50
Yes, this does not generate full list of keys. It eventually calls `GetSubRange` on the series representing the rows of the data frame - it's somewhat different than what you're doing, but should be reasonably well optimized: https://github.com/fslaborg/Deedle/blob/master/src/Deedle/Series.fs#L187 – Tomas Petricek Nov 16 '20 at 13:57

How can I use Deedle to get every row from some key and below?

1 Answers1