1

We are trying to do a POC to change the way we are storing content in the geode region. We operate on the sketches (sizes can vary from 1GB to 30GB) and currently breaking them into parcels and storing the parcels in the region. We then read these parcels, merge them to create a complete sketch for our processing. We are seeing some inconsistencies in the data due to the cache eviction and trying to come up with an approach of storing the complete object in the region instead of storing the parts.

I was looking at Geode documentation but did not seem to find the size limitation for any entry in the region, but wanted to reach a broader group in case anyone has done anything similar or have some insights into it.

Thanks for your response in advance.

Best Regards, Amit

AmitkAgrawal
  • 33
  • 6

1 Answers1

2

According to what I've been investigating, the maximum object size is set as 1GB, you can have a look at GEODE-478 and commit 1e3f89ddcd for further details. It's worth mentioning, as a side note, that objects that big might cause problems with GC, so you might want to stay away from that. Cheers.

Dharman
  • 30,962
  • 25
  • 85
  • 135
Juan Ramos
  • 1,421
  • 1
  • 8
  • 13
  • Thanks for the prompt and informative inputs. Looks like we might have difficulty going this route then. I was certainly suspicious about the GC performance with larger size objects. We might have to think of some other way of making sure that we are able to maintain the consistency of the objects even with Geode cache eviction. – AmitkAgrawal Aug 07 '20 at 22:46