Cannot use Nested VariableOperators.mapItemsOf in Spring Data MongoDb

Question

I'm forced to use the aggregation framework and the project operation of Spring Data MongoDb.

What I'd like to do is creating an array of object as a result of a project operation.

Considering this intermediate aggregation result:

{
  "processes": [
    {
      "id": "101a",
      "assignees": [
        {
          "id": "201a",
          "username": "carl93"
        },
        {
          "id": "202a",
          "username": "susan"
        }
      ]
    },
    {
      "id": "101b",
      "assignees": [
        {
          "id": "201a",
          "username": "carl93"
        },
        {
          "id": "202a",
          "username": "susan"
        }
      ]
    }
  ]
}

I'm trying to get for each process, all the assignee usernames and ids. Hence, what I want to obtain is something like this:

[
  {
    "results": [
      {
        "id": "201a",
        "value": "carl93",
        "parentObjectId": "101a"
      },
      {
        "id": "202a",
        "value": "susan",
        "parentObjectId": "101a"
      },
      {
        "id": "201a",
        "value": "carl93",
        "parentObjectId": "101b"
      },
      {
        "id": "202a",
        "value": "susan",
        "parentObjectId": "101b"
      }
    ]
  }
]

To reach this goal I'm using 2 nested VariableOperators.mapItemsOf obtaining:

org.springframework.data.mapping.MappingException: Cannot convert [Document{{id= 201a, value= carl93, parentObjectId= 101a}}, Document{{id= 202a, value = susan, parentObjectId= 101a}}]
of type class java.util.ArrayList into an instance of class java.lang.Object! 
Implement a custom Converter<class java.util.ArrayList, class java.lang.Object> and register it with the CustomConversions.

Here's the code that I'm currently using:

new ProjectionOperation().and(
    VariableOperators.mapItemsOf("processes")
      .as("pr")
      .andApply(
           VariableOperators.mapItemsOf("$pr.ownership.assignees")
               .as("ass")
               .andApply(aggregationOperationContext -> {
           Document document = new Document();

           document.append("id", "$$ass.id");
           document.append("value", "$$ass.username");
           document.append("parentObjectId", "$$pr.id");

           return document;
          })
    )
).as("results");

The code produces this:

[ 
  [
    {
      "id": "201a",
      "value": "carl93",
      "parentObjectId": "101a"
    },
    {
      "id": "202a",
      "value": "susan",
      "parentObjectId": "101a"
    }
  ], 
  [
    {
      "id": "201a",
      "value": "carl93",
      "parentObjectId": "101b"
    },
    {
      "id": "202a",
      "value": "susan",
      "parentObjectId": "101b"
    }
  ]
]

As you can see there are 2 nested arrays, [[],[]]. This is the reason why the exception is thrown. Nevertheless what I want to obtain is just one array, adding all the objects in it (possibly without duplicates or null values). I've tried the addToSet operator and other aggregtion operators, without any success.

I don't see why you get nulls and 4 elements when you're sample document has only two elements. Are you sure you have the right document in the question ? — s7vr, Sep 25 '18 at 15:35
Good point, I've written this question starting from a more complicated example. The null values are not compliant to the initial example. Hence, just forget about the null values. What is messing things ups is the array of arrays structure that comes from the 2 nested VariableOperators.mapItemsOf() — desoss, Sep 25 '18 at 15:39
did you intend to include `results` field for each array element in the `processes` ? That is the only difference I see between output. — s7vr, Sep 25 '18 at 15:49
I've updated the question, with the proper goal I want to reach and removing the null values that weren't compliant to the initial example. — desoss, Sep 25 '18 at 15:58

s7vr · Answer 1 · 2018-09-25T16:44:35.977

3

Use $reduce with $concatArrays to join the arrays.

 new ProjectionOperation().and(
    ArrayOperators.arrayOf("processes")
      .reduce(ArrayOperators.ConcatArrays.arrayOf("$$value").concat(
           VariableOperators.mapItemsOf("$$this.ownership.assignees")
               .as("ass")
               .andApply(aggregationOperationContext -> {
           Document document = new Document();
           document.append("id", "$$ass.id");
           document.append("value", "$$ass.username");
           document.append("parentObjectId", "$$this.id");
           return document;
          })
    )).startingWith(Arrays.asList())
).as("results");

edited Sep 25 '18 at 16:44

answered Sep 25 '18 at 16:15

s7vr

73,656
11
106
127

Nice approach! But I obtain `java.lang.IllegalArgumentException: Invalid reference '$$value'!` – desoss Sep 25 '18 at 16:27
I have a misplaced braces. Try now and it works in my version 2.0.4 spring mongo jar. – s7vr Sep 25 '18 at 16:45
I had noticed the misplaced brace. I obtain `java.lang.IllegalArgumentException: Invalid reference '$$value'!`. What is the purpose of `$$value`? Moreover, I've tried to place the content of `concat` inside `arrayOf` with bad results :( – desoss Sep 25 '18 at 16:52
1

`$$value` keeps accumulated values as we concat array elements inside `$reduce` operation. `$$this` refers to the current element in iteration. More [here](https://docs.mongodb.com/manual/reference/operator/aggregation/reduce/). What is your spring mongo version ? Do you have the right imports ? – s7vr Sep 25 '18 at 16:53
spring-data-mongodb 2.0.8.RELEASE – desoss Sep 25 '18 at 16:58
Can you show me the code you use to invoke aggregation query with mongotemplate ? It worked for me when I use `mongoTemplate.aggregate(newAggregation(project), colname, Document.class)` – s7vr Sep 25 '18 at 17:04
mongoTemplate.aggregate(aggregation, COLLECTION,ConfigurationResult.class); Do you want the mongoTemplate configuration? (it's pretty standard) The aggregation has 9 stages, if you want I can send it to you. – desoss Sep 25 '18 at 17:09
That's not needed. COLLECTION - Is this an input pojo or collection name ? It looks like spring is trying to validate your field name instead of passing it directly with your set up. That explains why it would work with Document.class. – s7vr Sep 25 '18 at 17:11
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/180749/discussion-between-desoss-and-veeram). – desoss Sep 25 '18 at 17:15
no, I got problems both with $$value and with null values t – desoss Oct 11 '18 at 16:19
what exactly is the problem ? Can you explain with an example may be? – s7vr Oct 11 '18 at 16:56

Cannot use Nested VariableOperators.mapItemsOf in Spring Data MongoDb

1 Answers1

Linked