1

So, I'm trying to implement flink's mapState using remote functions. In the mapState I'll be storing around 1000 values mapped with one key. Can someone point me in direction to implement it for remote function. I've found the article with embedded functions https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/dev/datastream/fault-tolerance/state/.

Also, when I access it from remote function will my entire state be transferred in each request? If yes, is there any way this could be implemented to avoid this huge data transfer each time I request?

ouflak
  • 2,458
  • 10
  • 44
  • 49
Singh3y
  • 336
  • 1
  • 7
  • Are you talking about using MapState with Flink's Stateful Functions API? Or in other words, something like statefun's PersistedTables, but extended for use with remote functions? If so, that isn't supported. – David Anderson Feb 24 '22 at 15:47
  • yeah! I was talking about that. Is there any other way of storing a map in state apart from this when it comes to remote function. Because, I'll be storing around 1000 events in that and when a trigger event occurs i would access that entire 1000 objects map and clear it from state. – Singh3y Feb 24 '22 at 15:50
  • Because, there's around 1000 objects I don't want them to receive in each state call and modify it. Basically I'll be doing around 1000 put and when trigger comes i'll read the entire state and clean it. Some optimized way of 1000 puts and just one get. – Singh3y Feb 24 '22 at 15:51
  • Remote functions only support single key/value pairs. If you want the value to be a collection, that's for you to deal with. And as you point out, it could be expensive. – David Anderson Feb 24 '22 at 15:52

0 Answers0