0

I am trying to develop a cache application which will load itself on server start and client application will be able to read the cache data through REST service calls.

Hence, I have to initialize the GemFire cache at the deployment of the application. This would load the data (derived the data from the RDBMS) which is in the form of Map into the Cache.

I have seen a CacheLoader loading one entry at a time into a GemFire Region, but can Region.putAll(map) load all data at a time or are there some other methods?

Please help.

John Blum
  • 7,381
  • 1
  • 20
  • 30
Swagatan
  • 1
  • 2

2 Answers2

0

I believe that the <initializer> element would be a perfect match for your use case here. It's used to Launch an Application after Initializing the Cache, and it can certainly be used to populate the regions upon cache initialization.

There are other options as well, like writing a custom Function and execute it as soon as your startup script returns, but I think that the <initializer> element would be the appropriate here.

Hope this helps. Cheers.

Juan Ramos
  • 1,421
  • 1
  • 8
  • 13
  • Thank you for your response. I am looking to put up data in the form of a map into the cache during this time as well. How can I do that ? – Swagatan Jan 10 '18 at 14:25
  • In the initializer you get all your Data from your background data source in the form of a map and then say myRegion.putAll(map). That’s how you do it – Wes Williams Jan 10 '18 at 15:09
  • As the previous comment states, the easiest is to use `Region.putAll()`. From the [javadocs](http://gemfire-92-javadocs.docs.pivotal.io/): Copies all of the entries from the specified map to this region. The effect of this call is equivalent to that of calling put(Object, Object) on this region once for each entry in the specified map. – Juan Ramos Jan 10 '18 at 16:11
  • Thanks. This helps. I am using Gemfire 8.2.7 version. Can anybody help me with the Spring Data Gemfire and other Maven dependencies associated with it ? I had been using 8.2.4 version, but with the server upgrade, I am getting Missing Artifact issue in my pom. For Gemfire 8.2.4, I am using the following dependencies spring-data-gemfire : 1.5.1.RELEASE spring-data-common-core : 1.0.0.RELEASE – Swagatan Jan 12 '18 at 08:28
  • Hey Swagatan. You might want to have a look at [Spring-Data-GemFire-to-GemFire-Version-Compatibility](https://github.com/spring-projects/spring-data-gemfire/wiki/Spring-Data-GemFire-to-GemFire-Version-Compatibility). If you are using Maven, adding the correct spring-data-gemfire should be sufficient as the transitive dependencies will be automatically added. Cheers. – Juan Ramos Jan 12 '18 at 15:15
  • Thanks much for the help. I am through. – Swagatan Jan 31 '18 at 17:24
0

I would be very careful about putting all the data from your underlying data source, e.g. from an RDBMS, in memory, in a java.util.Map, since it would be very easy to run out of memory (hence an OutOfMemoryError) pretty quickly depending on the size of the result set.

None-the-less, if you want an example of this see here; configuration is here.

Essentially, I am using a Spring BeanPostProcessor, the RegionPutAllBeanPostProcessor, to put a Map of data into a "target" Region.

For example, I have a Region (i.e. "RegionOne") and I can use the RegionPutAllBeanPostProcessor to target this Region and put data into the Region from the Map.

Obviously, you have many different options when it comes to triggering this Region load/"warming": a GemFire Initializer, a Spring BeanPostProcessor (docs here) or even a Spring ApplicationListener listening for ApplicationContextEvents, such as an on ContextRefreshedEvent (docs here).

However, while the Map in this test is hard-coded in XML, you could envision populating this Map from any data source, including a java.sql.ResultSet derived from a SQL query executed against the RDBMS.

So, perhaps a better approach/solution, that would not eat up as much memory, would be to use a BBP "injected" with Spring's JdbcTemplate or a JPA EntityManager, or even better yet, use Spring Data JPA, and load the data from your framework of choice to put data directly into the Region. After all, if the Region.putAll(:Map) is essentially just iterating the Map.Entries of the incoming Map and calling Region.put(key, value) individually for each Map.Entry (this, this and this) , then clearly it is not buying you that much and certainly does not justify putting all the data in-memory before putting it into the Region.

For instance, most ResultSets are implemented with a DB cursor that allows you to fetch a certain number of rows at once, but not all the possible rows. Clearly, your SQL query can even be more selective about which rows are returned based on interests/pertinence, think of loading only a subset of the most important data, or some other criteria specifiable in the query predicate. Then simply just put the data into the Region when iterating the ResultSet.

Food for thought.

-John

John Blum
  • 7,381
  • 1
  • 20
  • 30
  • Thanks a lot John. The approach now I am thinking is to change in a way that only on Cache miss event, I shall be putting data into the Cache. – Swagatan Jan 22 '18 at 16:03