0

I'm using Quarkus + Munity + Reactive Driver for MariaDB. I have a question to understand how reactive driver is working.

I want to process 20_000_000 entries stored in my database. For that, I want to stream entries efficiently to apply a non-blocking transformation.

select COUNT(*) from entries -> 20_000_000

How the MariaDB reactive driver is working?

It opens a TCP connection and fetch all entries -> a nightmare and blocking processing :(

It opens a connection and it requests a limited package of entries, and it streams them (like a paging) -> not very efficient

It opens a connection and wait to receive an event to fetch one by one entry -> reactive way. ...

Please could you confirm or explain how it's working (documentation, schema, ...)?

Many thanks for your help

Malys
  • 61
  • 1
  • 5
  • Selecting data from a table is not a blocking process in MariaDB. The table is still available for INSERT, UPDATE and SELECT from other tables. Can you describe what your actual problem is? – Louis Jan 20 '22 at 16:12
  • @Louis, I don't want to fetch a big amount of data, but I want to process every entry of my table. I will check followed proposal of tsegismont, It could be interesting – Malys Jan 21 '22 at 09:31

1 Answers1

1

The Reactive Client for MySQL will fetch all rows if you execute a query or prepared query.

For large amounts of rows, it's better to use the streaming API:

Multi<Row> rows = connection.prepare("SELECT * FROM users WHERE age > ?")
            .onItem().transformToMulti(ps -> ps.createStream(50, Tuple.of(18)).toMulti());

For the amount of rows you're dealing with, it might be interesting to consider using a stored procedure working with the data where it is instead of sending it over the network. Your mileage may vary.

tsegismont
  • 8,591
  • 1
  • 17
  • 27
  • Stored procedure with local transformation is not possible because it depends on an external component not usable from stored procedure. – Malys Jan 21 '22 at 09:36