3

I am developing an Android app that needs to fetch data from a huge MS Access database of 120MB size.

I have written code to establish connectivity and execute a simple query on the database. I run the same java code on my laptop and my Android device.Here's the code: p

ackage practiceDB;

import java.io.IOException;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.ResultSet;


import java.util.Scanner;

import net.ucanaccess.converters.TypesMap.AccessType;
import net.ucanaccess.ext.FunctionType;
import net.ucanaccess.jdbc.UcanaccessConnection;
import net.ucanaccess.jdbc.UcanaccessDriver;

public class Example {
    private Connection ucaConn;
    public Example() {
        try {
            this.ucaConn = getUcanaccessConnection("VehicleDatabase2.mdb");
        } catch (SQLException e) {
            e.printStackTrace();
        } catch(IOException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws ClassNotFoundException, SQLException {
        System.out.println("Please enter an int");
        new Scanner(System.in).nextInt();

        try {
            Example example = new Example();

            example.executeQuery();
        } catch (Exception ex) {
            System.out.println("An exception : " + ex.getMessage());
        }
    }

    private void executeQuery() throws SQLException {
        Statement st = null;
        try {
            System.out.println("Please enter an int");
            new Scanner(System.in).nextInt();
            st = this.ucaConn.createStatement();
            System.out.println("Please enter an int");
            new Scanner(System.in).nextInt();
            ResultSet rs = st.executeQuery("Select * from PersonData where EngNo = '1544256'");
            System.out.println(" result:");
            dump (rs, "executeQuery");
        } catch(Exception ex) {
            System.out.println("Sarah exception: " + ex.getMessage());
        } finally {
            if ( st != null ) {
                st.close();
            }
        }
    }

    private Connection getUcanaccessConnection(String pathNewDB) throws SQLException, IOException {
        String url  = UcanaccessDriver.URL_PREFIX + "VehicleDatabase2.mdb;newDatabaseVersion=V2003";

        return DriverManager.getConnection(url);
    }

    private void dump(ResultSet rs, String exName) 
            throws SQLException {
        System.out.println("-------------------------------------------------");
        System.out.println();

        System.out.println();
        int jk = 0;
        while (rs.next()) {

            System.out.print("| ");
            int j=rs.getMetaData().getColumnCount();
            for (int i = 1; i <=j ; ++i) {
                Object o = rs.getObject(i);
                System.out.print(o + " | ");
            }
        System.out.println();
        System.out.println();
        }
    }
}

When it runs on my laptop, the connection takes only about a minute to establish. But when it runs on my Android device, the connection takes more than 10 minutes, and takes up all the heap space, and when the device runs out of memory, the app crashes

What should i do??

Note:
i made some slight changes in this code to run it on android, like adding toasts instead of System.out.println for debugging, i removed the static main function for android, used Environment.getAbsolutePath() to locate the database, etc. Also, the code that I am running on Android, I first used a 9MB database to check if it works. The code fetches the data as expected from the 9MB database without any issues. The connection takes around 10 seconds to establish in Android in case of the 9MB database (in desktop, it takes less than a second to establish connection with 9MB database)

sga4
  • 350
  • 4
  • 15
  • "I run the same java code on my laptop and my Android device" -- this code will not run on an Android device, insofar as an Android app does not have `stdin` (used by your `Scanner`) and does not use `main()` as an entry point. If you want help with your Android code, it would be useful if you posted your Android code. – CommonsWare May 13 '15 at 13:08
  • Sorry for not mentioning it in my question, but yes i did make some slight changes in this code to run it on android, like adding toasts instead of System.out.println for debugging, i removed the static main function for android, etc. – sga4 May 13 '15 at 14:48
  • And I added Scanner statements in the java code to figure out which statement was consuming all that time. Actually, that's how I figured out that the connection was taking time to establish, and not the query. In my code for android, I used buttons and onClick callbacks instead of Scanner. I made two buttons, one of which only establishes the connection, and the second button executes the query only – sga4 May 13 '15 at 14:49
  • Also, the code that I am running on Android, I am sure that code works. I first used a 9MB database to check. The code fetches the data as expected from the 9MB database without any issues. The connection takes around 10 seconds to establish in Android in case of the 9MB database. – sga4 May 13 '15 at 14:54

1 Answers1

2

Yes, I know, it should work on a medium sized db. With a huge one...

Firstly, notice that the time you're measuring, it's the time of the very first connection to the database in the VM life, the followings(if needed) will be instantaneous.

Never tried something like that on Android because your experiment is challenging, yet, should it fit your requirements, you may try:

-use MirrorFolder (or keepMirror) connection parameter(see the ucanaccess web site for more details about it). In this case the very first connection to the db will be very slow, all the followings(even if the vm ends) will be instantaneous. But the access database should be updated only with ucanaccess and on your android

or, alternatively

-use a filter database(configure it on windows) that links the real database within the subset of external linked tables which are closely needed for your app(the memory usage might be dropped down). In this case, you'll have to use the remap connection parameter, because you're on a linux based SO.

See another suggestion related to jackcess(the underlying I/O library) here and use the latest ucanaccess release.

jamadei
  • 1,700
  • 9
  • 8
  • I have question. Could indexing solve the problem? It makes fetching data faster. I think it'll make it worse in my case, because indexing takes up some space, which means the database will become larger in size, so it'll take more time to establish connection – sga4 May 13 '15 at 19:31
  • Yes, I agree. Why don't you try with the Skipindexes=true connection parameter(so that just the indexes need for referential integrity are created). Memory occupation may be dropped down. A question for you: are there many ole data in your db? – jamadei May 14 '15 at 08:56
  • Yes, the data is never deleted, it's only added. So records dating back to 1994 are also in it. And i can't do anything about it, because I do not own the database. – sga4 May 14 '15 at 10:09
  • Maybe you misunderstood, I said OLE data(BLOB) not old data. – jamadei May 14 '15 at 10:36
  • oh yes i misunderstood. No, there is no BLOB data. It's just text. The number of records is 300,000+, 23 fields per record – sga4 May 14 '15 at 11:30
  • So, what if you just add Skipindexes=true (since UCanAccess 2.0.9.4) connection parameter? May you try on windows and android? – jamadei May 14 '15 at 12:35
  • won't help. Indexes take only 5MB which is negligible compared to the size of the databse – sga4 May 14 '15 at 13:29