MapReduce: map objects from another bucket against a list of keys (without link walking)?

Question

Edit: Looks like the concept is known as "forwarding and collecting". Going to see how this works in the Riak-Java client.

Right now performing a simple MapReduce in Riak that returns a list of keys of User objects. To keep things simple, I want to simply map the values of a User to the keys and return a list of users. Here's what I've got so far in Scala (using Java client) and FYI the key is composite and looks like clientId-userId:

val map = new JSSourceFunction("""
      function(riakObject){ 
            var rolek = riakObject.key;   
            return [rolek];
          }
    """)
        val reduce = new JSSourceFunction("""
      function(value){ 
            var returnValue=[], splitarr=[];  
            for(i=0;i<value.length;i++){ 
              splitarr=String(value[i]).split("-"); 
              returnValue = returnValue.concat([splitarr[1]]);  
            } 
            return returnValue; 
          }
    """)
    DB.client.mapReduce("rolesOfClientAdmins")
    .addKeyFilter(new TokenizeFilter("-", 1))
    .addKeyFilter(new MatchFilter(clientId))
    .addMapPhase(map)
    .addReducePhase(reduce)
    .execute().getResult(classOf[String])

Do I add another Map phase after the Reduce phase? Do I need to somehow change buckets? Thanks for the help!

You wouldn't add a map after the reduce - you run a reduce after you've gathered all your data with one or more map phases — mafrosis, Mar 18 '13 at 22:36
Ah understood. And from Google research, it looks like I also need to return the bucket during the map phase, like `return [[value.bucket, value.key]]` — crockpotveggies, Mar 18 '13 at 22:41
If you want to add another map phase, then you need to return the bucket & key together in that way. I'm not exactly sure what your question is in order to write a full answer :) — mafrosis, Mar 18 '13 at 22:56
I can see where I wasn't clear. Basically, the code above is already returning a list of keys (userIDs) from `rolesOfClientAdmins` bucket. Instead, I want to return full user objects themselves stored in `users` bucket. Similar to a SQL join, but just not sure how it works in the Java client. — crockpotveggies, Mar 18 '13 at 23:03

mafrosis · Accepted Answer · 2013-03-19T00:58:43.180

2

We start with the rolesOfClientAdmins bucket as the input and split the key to get the User object's ID. Then pass the bucket name "users" and the user ID into the next map phase.

val mapRolesBucket = new JSSourceFunction("""
      function(value){
            var splitarr=value.key.split('-'); 
            return [[ 'users', splitarr[1] ]];
          }
    """)
val mapUsersBucket = new JSSourceFunction("""
      function(value){
            var obj = Riak.mapValuesJson(value)[0];
            return [ obj ];
          }
    """)
    DB.client.mapReduce("rolesOfClientAdmins")
    .addKeyFilter(new TokenizeFilter("-", 1))
    .addKeyFilter(new MatchFilter(clientId))
    .addMapPhase(mapRolesBucket)
    .addMapPhase(mapUsersBucket)
    .execute().getResult(classOf[String])

edited Mar 19 '13 at 00:58

answered Mar 18 '13 at 23:19

mafrosis

2,720
1
25
34

Seems simple, except does this reference the `users` bucket to pull the values? Do I need to add a link somewhere? Thanks for your help on this. – crockpotveggies Mar 18 '13 at 23:23
Np. I'm not actually familiar with the client you're using - but in this code the bucket is called `rolesOfClientAdmins`. If your bucket is called `users`, change the code there. I'll update the answer if it works for you. – mafrosis Mar 18 '13 at 23:27
Ah I see. Basically since a user can have multiple admin relationships, we're working with 2 different buckets. So we're getting the list of keys from `rolesOfClientAdmins` but the user objects themselves would come from `users` bucket. Sorry if I didn't make that clear. – crockpotveggies Mar 18 '13 at 23:30
Aha! I'm with you now. We'll get this answer sorted, and then it might be worth updating your question a bit ;) – mafrosis Mar 18 '13 at 23:32
Appreciated. And yes, I can assure the question will be updated LOL. – crockpotveggies Mar 18 '13 at 23:36
PhaseError: `ReferenceError: i is not defined`. Was the `mapRoles` meant to be inside a loop? – crockpotveggies Mar 19 '13 at 00:22
Ah sorry about that- I couldn't test that bit and didn't check it carefully enough. The code is splitting your composite key of `clientId-userId`. Answer has been updated. – mafrosis Mar 19 '13 at 00:25
WOW this is still really simple. Accepting this, and I'll say it again: thank you! – crockpotveggies Mar 19 '13 at 00:38
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/26411/discussion-between-mafro-and-delongey) – mafrosis Mar 19 '13 at 00:39

MapReduce: map objects from another bucket against a list of keys (without link walking)?

1 Answers1