I'm using Caffeine's AsyncLoadingCache to cache lookup data from a database.
Now I'd like to integrate this into Spark Streaming. My idea is to let each worker maintain such a cache with a given refresh rate. But how to do that?
Is it better to define it as a broadcast variable and at each timeout recreate it and spread among the workers?
What is the best practice here?
