0

I have a table that has about 3 millions row in 1 partition key. I need to load all those data and save it as file.

Actually it is data recorded for 1 day sensor input. And what I want to build is a playback service to replay the sensor event at least in the past 3 months. So I am thinking it will be really big volume of data.

I am new in NoSql database, any approach how to achieve these goal?

panoet
  • 3,608
  • 1
  • 16
  • 27
  • what do you mean playback service - loading the data from files? – Alex Ott Feb 14 '20 at 09:25
  • I want to load from database, store it in file, and then process it. Why do I need to store? Because I want to reuse it later, if needed. – panoet Feb 14 '20 at 13:40
  • "3 millions row in 1 partition key" - You will want to find some way to break that up. Cassandra can't handle that many rows in a single partition. – Aaron Feb 14 '20 at 14:58
  • Are you sure about that? – panoet Feb 14 '20 at 18:05
  • Can you post the table DDL? That would help, but @Aaron is right that is a very large amount of data in one partition. The partition size would easily exceed the recommended limit. – apesa Feb 19 '20 at 22:32

0 Answers0