Use TableContinuationToken as bookmark

Question

This query taken from here is quite straight forward:

TableContinuationToken token = null;
List<Footwear> shoes = new List<Footwear>();

do
{
  TableQuerySegment<Footwear> queryResult = query.ExecuteSegmented(token);
  token = queryResult.ContinuationToken;
  shoes.AddRange(queryResult.Results);
} while (token != null);

Once null is returned the while loop finishes. Is it possible to store the last TableContinuationToken and then check after a while if there is more Footwear entered after the last TableContinuationToken. To create 'order' I use this approach.

PS:

I hope the following provides a bit more context. I currently store instances of this class:

public class SomeClass : TableEntity
{
    public long Ticks { get; set; }
    public SomeClass(){}

    public SomeClass(string partitionKey)
    {
        PartitionKey = partitionKey;
        Ticks = DateTime.UtcNow.Ticks - DateTime.MinValue.Ticks;
        // rowkey + partition = guid/pk
        // used to order events at the other end - very important  
        RowKey = (DateTime.UtcNow.Ticks - DateTime.MinValue.Ticks).ToString();
    }
}

in Azure table storage. To create a rowkey like this allows me to sort entities like so:

var query =
(from s in _table.CreateQuery<SomeClass>()
where
s.PartitionKey == _partitionKey &&
string.Compare(s.RowKey, rowKeyToUse, StringComparison.Ordinal) > 0 
select s).AsTableQuery();

from time to time I want to check if there are new entities. I do not think I can use the rowkey for this (see also here). So currently I store the latest Ticks of the last entity in the process that consumes the table storage. I can use this query to pull for new entities:

 var query =
  (from s in _table.CreateQuery<SomeClass>()
   where
   storedEvent.PartitionKey == _partitionKey &&
   storedEvent.Ticks > _ticks && // only above last threshold
   string.Compare(s.RowKey, rowKeyToUse, StringComparison.Ordinal) > 0 
   select s).AsTableQuery();

So in a nutshell - this is what I want to achieve:

(1) Page over all entities until token = null. (2) 'Pull' the table storage. If there are new entities loop until token = null. (3) Repeat (2)

I am after the simplest and most robust approach. Currently I am using the Ticks approach described above. However, this feels wrong as the same information is already stored in the rowkey as string. I hope this makes sense.

The easy answer is just 'yes'... so what have you tried that didn't work and what happened? — Emily Gerner, Feb 25 '16 at 17:42
How? In the above the token will be null once the first looping finishes. I want to check after a while that rows where inserted after the token/checkpoint. — cs0815, Feb 25 '16 at 18:40

score 1 · Answer 1 · answered Feb 25 '16 at 21:43

1

If you store the partition key and row key of the final entity returned, you can execute a TableQuery in the future to check this. The filter should be partition key = to that entity's partition key (like you already have) and row key > that entity's row key. Then you can use the exact same code you already have.

answered Feb 25 '16 at 21:43

Emily Gerner

2,427
16
16

Thanks. What happens if the 'primary key' (partition + row key) is not ordered? In sql I can just say where pk > 5 if pk is identity(1, 1) and 5 was the pk of the last row retrieved. At the moment I just use a colum that contains the 'ticks' (time) as long, which means i can simply do where ticks > lastticks – cs0815 Feb 25 '16 at 21:51
Not sure I quite understand once again. I read the two things you linked to but I think I'm missing what portions of them you're using since they're both larger documents. My impression was your row keys were when the entity was created and the partition key what you were already querying on (there isn't code for the query, so I don't know). I think it would help us answer better if you updated your question with what you were doing specifically instead of the links. – Emily Gerner Feb 25 '16 at 22:51
Apologies. I have added more context to my question - see PS. – cs0815 Feb 26 '16 at 08:49

Use TableContinuationToken as bookmark

1 Answers1