4

Looking for a recommendation. I have a command line utility that now needs to process a large amount of data coming from a web service in a form of a CSV.

I need to run multiple tests on the data to look for anomalies and patterns. I'd like the utility to be something that someone can download and not have to install or configure.

Is there a recommendation for a NoSQL or SQL database that I can just spin up in memory, load the CSV into a table and then run my queries against that data?

Can Redis be used without installation? Something else?

Community
  • 1
  • 1
mikebz
  • 3,277
  • 8
  • 37
  • 50
  • what exactly do you need out of it? just a blob kvstore by primary key? json processing? query language? – radai Nov 29 '15 at 15:39
  • How large does the data is going to be? – The Coder Nov 29 '15 at 15:43
  • Possible duplicate of [Simple and reliable in memory database for fast java integration tests with support for JPA](http://stackoverflow.com/questions/3444265/simple-and-reliable-in-memory-database-for-fast-java-integration-tests-with-supp) – teobais Nov 29 '15 at 15:51

2 Answers2

7

How about H2 Database? It's pure Java, in-memory and can be embedded into your application, see Connecting to an Embedded (Local) Database

example:

import org.h2.jdbcx.JdbcDataSource;

JdbcDataSource ds = new JdbcDataSource();
ds.setURL("jdbc:h2:˜/test.db");
ds.setUser("sa");
ds.setPassword("sa");
Connection conn = ds.getConnection();

(adapted from http://www.h2database.com/javadoc/org/h2/jdbcx/JdbcDataSource.html)

If you want an in-memory database, change URL to something like jdbc:h2:mem:test.db or similar. Look for "in-memory Databases" in documentation.

julianwki
  • 392
  • 2
  • 9
2

I have not yet used it, but have a look at JasDB

From the website.

Quick Install guide

JasDB is incredibly easy to get started with it is up and running under a minute simply download and run it, or simply include it in your project Instructions

  • Install JasDB by unzipping the download
  • Start the database using start.bat or start.sh
  • Open http://localhost:7050

From https://github.com/oberasoftware/jasdb/wiki/Using-In-memory-indexes-and-storage

JasDB can run with full in memory based indexes and record storage. The following configuration needs to be used in that case:

<Storage>
    <RecordWriter provider="inmemory"/>
    <!--<RecordWriter provider="transactional"/>-->
</Storage>
serv-inc
  • 35,772
  • 9
  • 166
  • 188