1

Is there a way to store Protobufs in a Gemfire cache by using some sort of pluggable serialization instead of the Gemfire provided PDX serialization? My environment uses Gemfire 8.2.5 and Spring Data Gemfire 1.9.6

I'm looking at building a WAN replicated cache used for OLTP that will host around 100mm POJOs with query patterns being mainly key based with the occasional OQL thrown in.

I looked at the this post from 2009 (https://communities.vmware.com/thread/410418) and I even implemented it using the code(tuned it to suit v3 Protobufs with the required 0 enum type) posted on that thread but am not sure if it's actually reaping the benefits of Protobuf serialization especially on the OQL part (OQL queries on a single cache node loaded with 5mm 3-attribute Protobufs take ~ 3-4 secs)

Any pointers especially from a Spring Data Gemfire perspective would be greatly appreciated.

Thanks

Pete
  • 449
  • 1
  • 4
  • 7
  • +1 for interesting question... IMHO a PDX wrapper is akin to a Protobuf wrapper but I hadn't thought of costs / benefits of combining these. Hmm... – rupweb Sep 04 '17 at 13:21
  • thanks for the +1, for me its about using ProtoBuf POJOs as is/in lieu of PDX so that I dont need to go through another serialization transformation layer and still get the benefits of using a leaner serialization mechanism. – Pete Sep 04 '17 at 17:13
  • for performance point you mentioned, did you have index on the field you are querying? – Xiawei Zhang Oct 04 '17 at 08:20
  • yes we have a mixture of both primary key and functional indices on the fields used in the OQL predicate – Pete Oct 05 '17 at 13:44
  • PDX has the advantage that the GemFire query implementation knows how to deserialize only the fields it needs to evaluate the query. With the custom protobuf implementation it will have to fully deserialize every entry to satisfy the query (unless it is using an index). Depending on how important the OQL use case is for you, that may tip the balance in the direction of PDX and not protobuf serialization. For most use cases I have seen, PDX , though not the fastest possible, is fast enough and comes with other benefits like OQL and handling new fields well. – Randy May Jul 16 '18 at 14:25

0 Answers0