1

I have a database representing the accesslog of my webserver with columns such as ip-address, method, requested path, etc. I want to create an endpoint which returns a json similar to this:

{
    "addresses": [
        {
            "address": "1.2.3.4",
            "requests": 10
        },
        {
            "address": "5.6.7.8",
            "requests": 5
        }
    ],
    "methods": [
        {
            "method": "GET",
            "requests": 14
        },
        {
            "method": "POST",
            "requests": 1
        }
    ],
    "paths": [
        {
            "path": "/index",
            "requests": 15
        }
    ]
}

select address, count(address) as requests
from logs
group by address
order by requests desc
select method, count(method) as requests
from logs
group by method
order by requests desc
select path, count(path) as requests
from logs
group by path
order by requests desc

So this endpoint should execute 3 queries to get these 3 resultsets.

This is my first reactive/mutiny project so ive come as far as the tutorials go:

    @Inject
    io.vertx.mutiny.pgclient.PgPool client;

    @GET
    @Path("/getOne")
    public Multi<Map<String, Integer>> getOne() {

        String select = "select address, count(address) as requests" 
                     + " from logs group by address order by requests";

        Uni<RowSet<Row>> set1 = client.query(select).execute();

        return set1.onItem().transformToMulti(set -> Multi.createFrom().iterable(set)).onItem()
                .transform(row -> Map.of(row.getString("address"), row.getInteger("requests")));
    }

This already returns the array of ips and their requests.

[
  {
    "1.2.3.4": 10
  },
  {
    "5.6.7.8": 5
  }
]

I am now struggling to combine 2 resultsets into one response object:


    @Inject
    io.vertx.mutiny.pgclient.PgPool client;

    @GET
    @Path("/getCombined")
    public Uni<Tuple2<Map<String, Integer>, Map<String, Integer>>> getCombined() {

        String selectIps = "select address, count(address) as requests"
                + " from logs group by address order by requests";

        String selectMethods = "select method, count(method) as requests"
                + " from logs group by method order by requests";

        Uni<RowSet<Row>> set1 = client.query(selectIps).execute();
        Uni<RowSet<Row>> set2 = client.query(selectMethods).execute();

        Multi<Map<String, Integer>> multi1 = set1.onItem().transformToMulti(set -> Multi.createFrom().iterable(set))
                .onItem().transform(row -> Map.of(row.getString("address"), row.getInteger("requests")));

        Multi<Map<String, Integer>> multi2 = set2.onItem().transformToMulti(set -> Multi.createFrom().iterable(set))
                .onItem().transform(row -> Map.of(row.getString("method"), row.getInteger("requests")));

        return Uni.combine().all().unis(multi1.toUni(), multi2.toUni()).asTuple();
    }

this seems to go in the right direction but i end up having my arrays converted to objects:

{
  "item1": {
    "1.2.3.4": 10
  },
  "item2": {
    "GET": 7
  }
}

How can i combine multiple resultsets into one, to return one response object like the one at the top?

Jonathan
  • 13
  • 3

1 Answers1

0

Instead of returning it as a Tuple, you could return it as a Map:


        Multi<Map<String, Integer>> multi1 = ... // Same as in the quesion
        Multi<Map<String, Integer>> multi2 = ... // Same as in the quesion

        Uni<Map<String, Map<String, Integer>>> uni1 = multi1
                .toUni()
                .map( map -> Map.of( "addresses", map ) );

        Uni<Map<String, Map<String, Integer>>> uni2 = multi2
                .toUni()
                .map( map -> Map.of( "methods", map ) );

        return Uni.combine().all().unis( uni1, uni2 )
                .combinedWith( objects -> {
                    final Map<String, Object> resultMap = new HashMap<>();
                    ((List<Map<String, Object>>) objects).forEach( resultMap::putAll );
                    return resultMap;
                } ); 

But the result will look like this (different than the example in the question):

{
  "addresses": [
    { "1.2.3.4": 10},
    { "5.6.7.3": 15},
  ],
  "methods": [
    { "GET": 7 }
  ]
}

I don't think I would convert everything to a Multi and then convert it back to a Uni. This will return the JSON you mention in the question:

    public Uni<Map<String, Object>> getCombined() {
        ...
        Uni<RowSet<Row>> set1 = client.query(selectIps).execute();
        Uni<RowSet<Row>> set2 = client.query(selectMethods).execute();

        Uni<Map<String, Object>> addressesUni = convertRowSet("addresses", "address", set1);
        Uni<Map<String, Object>> methodsUni = convertRowSet("methods", "method", set2);

        return Uni.combine().all().unis( addressesUni, methodsUni )
                .combinedWith( objects -> {
                    final Map<String, Object> resultMap = new HashMap<>();
                    ((List<Map<String, Object>>) objects).forEach( resultMap::putAll );
                    return resultMap;
                } );
    }

    private static Uni<Map<String, Object>> convertRowSet(String plural, String singular, Uni<RowSet<Row>> rowSetUni) {
        return rowSetUni.map( rowSet -> {
            if ( rowSet.size() > 0 ) {
                List<Map<String, Object>> resultList = new ArrayList<>();
                rowSet.forEach( row -> {
                    resultList.add( Map.of(
                            singular, row.getString( singular ),
                            "requests", row.getInteger( "requests" )
                    ) );
                } );
                return Map.of( plural, resultList );
            }
            return Collections.emptyMap();
        } );
    }

Assuming that all results from the queries look the same.

Note that this approach works because you are using the Vert.x client and not the Hibernate Reactive session. Because the session cannot be used for parallel operations, you would need to make sure that each query is executed using a different session or that the two queries are executed in a sequence.

Davide D'Alto
  • 7,421
  • 2
  • 16
  • 30
  • Hi Davide, thank you for your detailed answer, i got it working! I modified your code to pass a list of unis to the Uni.combine().all().unis() function. This works like a charm. I tried to apply your suggested uni approach using hibernate panache which cause HR000061: Session is currently connecting to database. when i use multis with hibernate instead, it works as expected. Is there a reason why you prefer unis before multis? Are there any significant differences performance wise? – Jonathan Oct 22 '22 at 01:03
  • The Hibernate Reactive session is not made for the execution of queries in parallel. This means that you cannot use `.combine().all().uni(...)` with unis sharing the same session. The reason you see the exception HR000061 is that as soon as one of uni ends, it will close the session even if another uni is still using it. If you want to make it work, you need to make sure that the queries run in the right order or that you use a new session for every uni. I've mentioned this in a similar issue: https://github.com/quarkusio/quarkus/issues/22433#issuecomment-1284041669 – Davide D'Alto Oct 22 '22 at 14:49
  • With my approach, I'm converting a RowSet into a Map with a loop. By converting the Uni to a Multi, you are iterating over the same list, emitting eache element into a stream of events so that you can collect them into a map again. Maybe there is not much difference in practice, but I find my approach easier to understand and refactor. Anyway, it's just a personal preference, I don't know if it's better in term of performance. – Davide D'Alto Oct 22 '22 at 15:00