HSQLDB: How to create a table with size larger than heap space?

Question

I want to create a simple table (id INT PRIMARY KEY, value INT) and insert 16M rows.

If I use MySQL, that may take 128M+ data (MyISAM fixed format).

Now I'm using HSQLDB and only give 128M heap space (i.e. run with -Xmx128m).

I use the file mode jdbc:hsqldb:file:... and here is my code:

import java.sql.Connection;
import java.sql.PreparedStatement;
import java.util.Collections;

import org.hsqldb.jdbc.JDBCDriver;

public class HSQLDBTest {
    private static final int BULK_SIZE = 16384;
    private static final int BATCHES = 1024;

    public static void main(String[] args) throws Exception {
        String sql = "INSERT INTO test (value) VALUES " + String.join(", ", Collections.nCopies(BULK_SIZE, "(?)"));
        try (Connection conn = new JDBCDriver().connect("jdbc:hsqldb:file:test", null)) {
            conn.createStatement().execute("CREATE TABLE test (id INT NOT NULL PRIMARY KEY IDENTITY, value INT DEFAULT 0 NOT NULL)");
            PreparedStatement ps = conn.prepareStatement(sql);
            for (int i = 0; i < BATCHES; i ++) {
                for (int j = 0; j < BULK_SIZE; j ++) {
                    ps.setInt(j + 1, j * j);
                }
                ps.executeUpdate();
                System.out.println(i);
            }
        }
    }
}

The insertion stops at about 0.7M rows (i = 40) and throws java.sql.SQLException: java.lang.OutOfMemoryError: GC overhead limit exceeded

Is there any other mode or property that can make HSQLDB create table with size larger than heap space?

score 2 · Accepted Answer · answered Oct 01 '19 at 13:58

Use a cached table. If you don't specify table type memory is assumed.

create cached table test (...);

Quote from the manual

CACHED tables are created with the CREATE CACHED TABLE command. Only part of their data or indexes is held in memory, allowing large tables that would otherwise take up to several hundred megabytes of memory.

HSQLDB: How to create a table with size larger than heap space?

1 Answers1