3

I am pretty new in Hibernate and I have the following problem: I have to perform a massive insert operation (I need to insert a lot of the model object and each of these object contains a byte[] field mapping a blob field on the DB).

I am using Spring Boot (so I need to configure it on Spring Boot) and Spring Data JPA.

So I have the following situation:

1) RoomMedia my model object:

@Entity
@Table(name = "room_media")
public class RoomMedia {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    @Column(name = "id")
    private Long id;

    //@Column(name = "id_room")
    //private Long idRoom;

    @ManyToOne
    @JoinColumn(name = "id_room", nullable = false)     // da rinominare anche sul DB in room_fk
    private Room room;

    @Lob
    @Column(name = "media")
    private byte[] media;

    private String description;

    private Date time_stamp;

    public RoomMedia() {
    }

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    ........................................................................
    ........................................................................
    GETTER AND SETTER METHODS
    ........................................................................
    ........................................................................
}

Then into another class I have this code that persist a list of RoomMedia objects:

roomMediaDAO.save(mediaListToBeInserted);

where mediaListToBeInserted is a List.

The problm is that doing in this way it is very slow because this statment generate insertion queries like these:

Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
Hibernate: insert into room_media (description, media, id_room, time_stamp) values (?, ?, ?, ?)
............................................................................
............................................................................
............................................................................

So the records are inserted one by one.

I need to insert these records massively, genereting a code like this:

INSERT INTO MyTable ( Column1, Column2 ) VALUES
( Value1, Value2 ), 
( Value1, Value2 ),
( Value1, Value2 ),
.................................
.................................
.................................
( Value1, Value2 )

Reading this documentation: https://docs.jboss.org/hibernate/orm/3.3/reference/en/html/batch.html I know that I can set Hibernate to perform Batch Insert that should do something like this.

But I have some doubts:

1) Reading the previous documentation link it says that:

Hibernate disables insert batching at the JDBC level transparently if you use an identity identifier generator.

and I have that my primary key of my model object is annoted with:

@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
@Column(name = "id")
private Long id;

So what it means? That I can't do massive insert of a list of these objects? It seem me strange that there is no way to do it.

In the previous article it explain:

You can also do this kind of work in a process where interaction with the second-level cache is completely disabled:

hibernate.cache.use_second_level_cache false

However, this is not absolutely necessary, since we can explicitly set the CacheMode to disable interaction with the second-level cache.

So it seem to me that I can do it in some way but I don't know what is this second-level cache and why it have to be disabled to perform batch insert of object having an identity identifier generator setted.

2) In my Spring Boot project I have this application.properties file containing the following settings related to the database connection and Hibernate configuration. So I think that this should be the point where to add the set to handle the batch massive insert. But what have I to set?

#No auth  protected
endpoints.shutdown.sensitive=true
#Enable shutdown endpoint
endpoints.shutdown.enabled=true

spring.datasource.url = jdbc:mysql://XX.YY.ZZZ.WW:3306/BeTriviusTest?useUnicode=true&characterEncoding=UTF-8
spring.datasource.username = MyUserName
spring.datasource.password = MyPswd

spring.datasource.driverClassName=com.mysql.jdbc.Driver

# Keep the connection alive if idle for a long time (needed in production)
spring.datasource.testWhileIdle = true
spring.datasource.validationQuery = SELECT 1

# Show or not log for each sql query
spring.jpa.show-sql = true
spring.jpa.database = mysql
# Hibernate ddl auto (create, create-drop, update)
spring.jpa.hibernate.ddl-auto = validate

# Naming strategy
spring.jpa.hibernate.naming-strategy = org.hibernate.cfg.ImprovedNamingStrategy

# Use spring.jpa.properties.* for Hibernate native properties (the prefix is
# stripped before adding them to the entity manager)

# The SQL dialect makes Hibernate generate better SQL for the chosen database
spring.jpa.properties.hibernate.dialect = org.hibernate.spatial.dialect.mysql.MySQLSpatial5InnoDBDialect
spring.jpa.database-platform = org.hibernate.spatial.dialect.mysql.MySQLSpatial5InnoDBDialect
AndreaNobili
  • 40,955
  • 107
  • 324
  • 596
  • You have read the documentation, set the properties and test. It is important to set a reasonable batch size AND to order INSERT and UPDATEs (else it still might not work). I also suggest to read a bit more recent documentation instead of the (quite old) 3.3 version. – M. Deinum Jan 02 '17 at 10:14
  • Ok, now I will search more recent documentation but my doubt still remain about the "identity identifier generator" – AndreaNobili Jan 02 '17 at 10:16
  • 2
    See http://stackoverflow.com/questions/27697810/hibernate-disabled-insert-batching-when-using-an-identity-identifier-generator for a deeper explanation on that. – M. Deinum Jan 02 '17 at 10:22
  • Voted to close, because this are mutliple questions, some already answered. – Jens Schauder Jan 02 '17 at 15:49
  • 1
    You CANNOT do bulk insert with IDENTITY ... since the persistence provider needs to know the id assigned __in the datastore__ for each record inserted, and using bulk insert it cannot know. So you cannot use it with that. – Neil Stockton Jan 02 '17 at 19:04

0 Answers0