1

As per QLDB documentation, QLDB provides views(projections) like User, Committed views to query data in it. Are these views are provided from indexed storage? If so, Current and History sections will be part of Indexed storage. Then, why there is a need to store history in indexed storage(a costlier storage), as reading history should have been made available from journal storage(which has all the data stored in an accurate and sequenced entry way)?

Shivaganesh
  • 123
  • 8

1 Answers1

1

The QLDB Journal contains all of your transactions. It is sequenced by block address, so the first transaction is written to block-1, then the next to block-2, etc.

To find the history of a document, one would need to look through every single transaction and determine if a document with that id was modified by block-N. QLDB Journals will contain millions of transactions, so it is undesirable to scan many millions of blocks - for both performance and cost reasons.

Indexing is exactly the way to solve scanning, so that's why we do it. There are some obvious alternatives, such as:

  1. Instead of keeping the document in indexed storage, keep only the block address
  2. Keep only the last N revisions of a document in indexed storage
  3. Keep only a year of history in indexed storage

If you have these or other requirements, we'd love to hear about them. The current history retention policy is based on what customers asked for during the QLDB preview. Other policies would be a cost vs performance trade-off.

Marc
  • 928
  • 5
  • 8