0

I have a 2 node cluster in my current ignite setup. All caches are in partitioned mode with 0 backups. I need to add backups to all the caches and add more nodes to the same cluster.

From what I've tried, it looks like an existing cache cannot be modified to have backups. Is this possible? Can an existing cache with 0 backups, be modified to have 1 or more backups? If the modification on an existing cache is not possible, what is the solution for this? I created a new cache with tried migrating the data from the old cache to the new cache. But the data in the old cache is very large (Around 150 million rows). I used a client (Java thin client) to read an entire cache and process each row, then used ignite streamer to write data to the new cache. Since the data is huge, the Ignite cluster is crashing and only around 3 million rows are getting processed and migrated to the new cache.

What is the solution for adding backups to an existing cache with 0 backups? What is the optimal way to migrate 150 million records from one cache to another?

  • I had emailed the Apache Ignite user support with the same query, this is the reply I got: 1. Backups cant be changed for a cache, you need to drop and recreate the cache with your desired backup copies. 2. Data Migration, Ignite offers various APIs - SQL, Java KV and Java JDBC etc. 3. Even use Java thin clients to read data from one cache and write to another cache. 4. DataStreamers offers best performance when you are ingesting huge data, but thin client does not have this API exposed if you are earlier version of Ignite prior to Ignite 2.9 – Ch. Vishal Ratnam Jan 24 '22 at 12:34
  • What is the error? Something like out of memory exception? – Alexandr Shapkin Jan 26 '22 at 22:17
  • Yes, the ignite server logs show "Critical system error detected" and "blocked system critical thread detected". The 2 ignite nodes crash after transferring almost 60million records. – Ch. Vishal Ratnam Jan 27 '22 at 02:45

0 Answers0