How to properly batch insert using Quarkus reactive hibernate with panache

Question

How to properly batch insert using Quarkus reactive hibernate with panache?

I haven't found any proper documentations, here is my failed attempt:

    @ReactiveTransactional
    public Uni<List<KlineEntity>> persistAll(List<KlineEntity> klines) {
        return getSession().onItem()
                           .transformToUni(session -> {
                               val batch = session.setBatchSize(BATCH_SIZE);

                               return Uni.createFrom().item(klines)
                                         .onItem()
                                         .call(entities -> batch.persistAll(entities))
                                         .call(this::flush)
                                         .invoke(session::clear);
                           });
    }

which results in:

javax.persistence.PersistenceException: org.hibernate.HibernateException: java.util.concurrent.CompletionException: org.hibernate.MappingException: Unknown entity: java.util.ArrayList

    at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:154)
    at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:181)
    at org.hibernate.reactive.session.impl.ReactiveExceptionConverter.convert(ReactiveExceptionConverter.java:31)
    at org.hibernate.reactive.session.impl.ReactiveSessionImpl.lambda$firePersist$18(ReactiveSessionImpl.java:685)
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
    at java.base/java.util.concurrent.CompletableFuture.uniHandleStage(CompletableFuture.java:950)
    at java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:2340)
    at java.base/java.util.concurrent.CompletableFuture.handle(CompletableFuture.java:144)
    at org.hibernate.reactive.session.impl.ReactiveSessionImpl.firePersist(ReactiveSessionImpl.java:678)
    at org.hibernate.reactive.session.impl.ReactiveSessionImpl.reactivePersist(ReactiveSessionImpl.java:662)
    at org.hibernate.reactive.util.impl.CompletionStages.applyToAll(CompletionStages.java:418)
    at org.hibernate.reactive.mutiny.impl.MutinySessionImpl.lambda$persistAll$10(MutinySessionImpl.java:145)
    at io.smallrye.context.impl.wrappers.SlowContextualSupplier.get(SlowContextualSupplier.java:21)
    at io.smallrye.mutiny.operators.uni.builders.UniCreateFromCompletionStage.subscribe(UniCreateFromCompletionStage.java:24)
    at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
    at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.lambda$subscribe$0(UniRunSubscribeOn.java:27)
    at org.hibernate.reactive.context.impl.VertxContext.execute(VertxContext.java:90)
    at io.smallrye.mutiny.operators.uni.UniRunSubscribeOn.subscribe(UniRunSubscribeOn.java:25)
    at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
    at io.smallrye.mutiny.operators.uni.UniOnItemTransform.subscribe(UniOnItemTransform.java:22)
    at io.smallrye.mutiny.operators.AbstractUni.subscribe(AbstractUni.java:36)
    at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.performInnerSubscription(UniOnItemTransformToUni.java:81)
    at io.smallrye.mutiny.operators.uni.UniOnItemTransformToUni$UniOnItemTransformToUniProcessor.onItem(UniOnItemTransformToUni.java:57)
    at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.forward(UniCreateFromKnownItem.java:38)
    at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem$KnownItemSubscription.access$100(UniCreateFromKnownItem.java:26)
    at io.smallrye.mutiny.operators.uni.builders.UniCreateFromKnownItem.subscribe(UniCreateFromKnownItem.java:23)
...

Note: I haven't set anything specific in application.yml as I would have done with hibernate JDBC

Here is a piece of my entity whether it might help:

@Builder
@AllArgsConstructor
@NoArgsConstructor
@Getter
@Setter
@ToString
@Entity
@Table(name = "kline", schema = "binance", catalog = "trading", indexes = {
    @Index(name = "idx_kline_pair_close_time_unq", columnList = "pair_id, closeTime", unique = true)
})
public class KlineEntity {
    @Setter(NONE)
    @Id
    @GeneratedValue(strategy = IDENTITY)
    @Column(name = "id", nullable = false, updatable = false)
    private Long id;

    @OneToOne(cascade = {CascadeType.ALL}, orphanRemoval = true)
    @JoinColumn(name = "pair_id")
    @ToString.Exclude
    private CurrencyPairEntity pair;
}

The error goes away if I use `batch.persistAll(entities.toArray())` but it does not look like batching is used. — Frédéric Thomas, Jun 11 '22 at 11:46
HIbernate disable batching when IDENTITY generation is used: https://stackoverflow.com/questions/27697810/why-does-hibernate-disable-insert-batching-when-using-an-identity-identifier-gen — Davide D'Alto, Jun 11 '22 at 20:38

Davide D'Alto · Answer 1 · 2022-06-11T20:55:52.063

2

persistAll accepts arrays, not lists:

@ReactiveTransactional
public Uni<List<KlineEntity>> persistAll(List<KlineEntity> klines) {
    return getSession().chain(session -> session
        .setBatchSize(BATCH_SIZE)
        .persistAll(klines.toArray(new KlineEntity[klines.size()]))
    )
    .map(v -> klines);
}

Because it's transactional, you don't need to flush or clear the session.

Also, Hibernate disables batching for IDENTITY type identifiers.

edited Jun 11 '22 at 20:55

answered Jun 11 '22 at 11:08

Davide D'Alto

7,421
2
16
30

Thanks a lot, it helped to understand few concepts, I had to completely change the way my ID was managed, that's why it took me time to test it. Bth, do you know whether @ReactiveTransactional is interchangeable with Panache.currentTransaction() or sessionFactory.withTransaction() ? – Frédéric Thomas Jun 16 '22 at 10:33
I haven't checked Panache code in a while but it's basically a layer on top of HIbernate Reactive. `@ReactiveTransactional` wraps the result of the method using `Panache.withTransaction`. `Panache.withTransaction` at some point calls `sf.withTransaction` or `session.withTransaction`. Anyway, they should all work the same. – Davide D'Alto Jun 16 '22 at 10:39
Thanks @davide-dalto, For sessionFactory.withTransaction, AFAIK, it requires a Context from a test class, would you have an example on how to make the call? – Frédéric Thomas Jun 17 '22 at 12:16
What are you trying to test? – Davide D'Alto Jun 17 '22 at 13:30
I don't know if you are still interested but I took one of the quickstarts with Hibernate Reactive and Panache and added a test class: https://github.com/DavideD/quarkus-quickstarts/commit/c817ddd9311be86bb636ee0091dace98cbbc4a39 It seems to work fine. Did you mean something like this? – Davide D'Alto Jun 18 '22 at 17:29
Hi Davide, actually what I'm trying to test is described better [here](https://stackoverflow.com/questions/72648062/how-to-chain-2-uni-in-unit-test-using-panache-withtransaction-without-getti), also your test passes because despite a transaction is used, it is not rollback at the end of the test and there are no chained Unis, [this one](https://gist.github.com/doublefx/e882196066da13adca1083064d6c1e93) fails with TimeOut, ran in context as [this one](https://gist.github.com/doublefx/4da6c6da202553a9d50befd276b8c111) – Frédéric Thomas Jun 19 '22 at 06:44

How to properly batch insert using Quarkus reactive hibernate with panache

1 Answers1

Linked