0

I have a Riak installation with many nodes. It stores entries, with relatively big blobs on a value side. In some cases, clients will need only a subset of this data, so there is no need to transfer them over the network.

Is there any elegant solution to pre-process this data on the server side, before actually sending them to client.

The only idea I have is to install a small "agent" on every node, that will interact with client in it's own protocol and acts as a proxy, that will reduce data based on query.

But such solution will work only, if I can know (based on key) on which node particular entry is stored. Is there a way to do it?

aav
  • 2,514
  • 1
  • 19
  • 27

1 Answers1

1

You may be able to do that with MapReduce, if you specify a single bucket/key as input. The map function would run local to where the data is stored, and send its result to the reduce function on whichever node received the request from the client. Since you are providing a specific key, there shouldn't be any coverage folding which is what causes the heavy load the docs warn about.

This would mean that your requests would effectively be using r=1, so if there is ever an outage you would get some false not found results.

Joe
  • 25,000
  • 3
  • 22
  • 44