How to use the Embedded PostgreSQL Server Java component as a separate service?

Question

I am trying to create a comprehensive integration test suite for a RESTful (Services) Java-based application that runs in Tomcat(7.x) and depends on an Postgresql (9.x) instance. Further, I would like to be able to run this suite as a self-contained process, exclusively from maven 3.x, if possible, by using the maven failsafe plugin. That way, the tests can be run across the 3 major platforms (Mac OSX, Linux & Windows).

From what I have learned, I believe that the key to making this happen is doing something like these steps (in this order):

1. Start an "embedded" version of the postgresql DB (and schema set up) somehow in the pre-integration-test phase of maven lifecycle, have that postgres-db-process running in the background
2. Start my Java container which loads my Services App: I am using the Jetty 9.1.5 plugin
3. Run my (JUnit-based) tests from Failsafe plugin during the integration-test phase
4. Shut down my Jetty container in maven's post-integration-test phase
5. Finally, have some mechanism shutdown the previously started (background) postgres-db-process in the post-integration-test phase of the lifecycle (kill/clean-up this process)

In my current implementation completes steps 1 - 3 successfully. In step 2, it uses of the exec-maven-plugin, which calls a Java class that uses the postgresql-embedded Java component. An Excerpt from the POM.xml:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <version>1.2.1</version>
  <executions>
    <execution>
          <id>Run Postgres DB start/schema setup</id>
          <phase>pre-integration-test</phase>
            <goals>
              <goal>java</goal>
            </goals>
      </execution>
    </executions>
    <configuration>
      <mainClass>com.some.package.test.utils.DbSetup</mainClass>
      <arguments>
        <argument>setup</argument>
      </arguments>
    </configuration>
</plugin>

And here is an excerpt of the DBSetup class that uses postgresql-embedded to start a postgresql instance:

try {
        DownloadConfigBuilder downloadConfigBuilder = new DownloadConfigBuilder();
        downloadConfigBuilder.defaultsForCommand(Command.Postgres);
        downloadConfigBuilder.proxyFactory(new HttpProxyFactory(PROXY_ADDRESS, DEFAULT_PROXY_PORT));
        IRuntimeConfig runtimeConfig = new RuntimeConfigBuilder()
                .defaults(Command.Postgres)
                .artifactStore(new ArtifactStoreBuilder()
                        .defaults(Command.Postgres)
                        .download(downloadConfigBuilder)).build();  

        PostgresStarter<PostgresExecutable, PostgresProcess> runtime = PostgresStarter.getInstance(runtimeConfig);        
        final PostgresConfig config = new PostgresConfig(Version.V9_2_4, new AbstractPostgresConfig.Net(
                    "localhost", 5432
            ), new AbstractPostgresConfig.Storage(dbName), new AbstractPostgresConfig.Timeout(),
                    new AbstractPostgresConfig.Credentials(username, password));        
        config.getAdditionalInitDbParams().addAll(Arrays.asList(
                "-E", "UTF-8",
                "--locale=en_US.UTF-8",
                "--lc-collate=en_US.UTF-8",
                "--lc-ctype=en_US.UTF-8"
            ));     

        exec = runtime.prepare(config);
        process = exec.start();
        System.out.println("embedded Postgres started");
        Thread.sleep(1200L);

    } catch (IOException e) {
        System.out.println("Something Went Wrong initializing embedded Postgres: " + e);
        e.printStackTrace();
    } 
    catch (InterruptedException e) {
        System.out.println("Something Went Wrong Pausing this thread " + e);
        e.printStackTrace();
    }

Please note that I am not calling process.stop(); within the method that starts the Postgres instance. So, I currently have no way to shutdown the DB process. Once I have exited out of this class DbSetup, all reference to that existing process will be lost. And the Postgres process never shuts down. In fact, it seems like the jetty container will not shut down either and the entire maven job is hung-up, so I have to manually kill it (excerpts from my maven console output):

[INFO] --- exec-maven-plugin:1.2.1:java (Run Postgres DB start/schema setup) @ BLAHBLAH ---
***START of Postgres DB Setup Process ***
Extract /Users/myUserName/.embedpostgresql/postgresql-9.2.4-1-osx-binaries.zip START
...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Extract /Users/myUserName/.embedpostgresql/postgresql-9.2.4-1-osx-binaries.zip DONE
INFO:20161021 12:58:00: de.flapdoodle.embed.process.runtime.Executable de.flapdoodle.embed.process.runtime.Executable start AbstractPostgresConfig{storage=Storage{dbDir=/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-66cfc41f-0e16-439f-a24b-6e5b6dbc683d/db-content-3bc4b9cc-dd21-43a7-9058-285767f5c53d, dbName='BLAH', isTmpDir=true}, network=Net{host='localhost', port=5432}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{BLAH, BLAH}, args=[], additionalInitDbParams=[-E, UTF-8, --locale=en_US.UTF-8, --lc-collate=en_US.UTF-8, --lc-ctype=en_US.UTF-8]}
INFO:20161021 12:58:01: de.flapdoodle.embed.process.runtime.Executable de.flapdoodle.embed.process.runtime.Executable start AbstractPostgresConfig{storage=Storage{dbDir=/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-66cfc41f-0e16-439f-a24b-6e5b6dbc683d/db-content-3bc4b9cc-dd21-43a7-9058-285767f5c53d, dbName='BLAH', isTmpDir=true}, network=Net{host='localhost', port=5432}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{BLAH, BLAH}, args=[BLAH], additionalInitDbParams=[]}
INFO:20161021 12:58:04: de.flapdoodle.embed.process.runtime.Executable de.flapdoodle.embed.process.runtime.Executable start AbstractPostgresConfig{storage=Storage{dbDir=/var/folders/8g/69wh31fn7nx3q81phwfdpld00000gn/T/postgresql-embed-66cfc41f-0e16-439f-a24b-6e5b6dbc683d/db-content-3bc4b9cc-dd21-43a7-9058-285767f5c53d, dbName='BLAH', isTmpDir=true}, network=Net{host='localhost', port=5432}, timeout=Timeout{startupTimeout=15000}, credentials=Credentials{BLAH, BLAH}, args=[], additionalInitDbParams=[-E, UTF-8, --locale=en_US.UTF-8, --lc-collate=en_US.UTF-8, --lc-ctype=en_US.UTF-8]}
embedded Postgres started
***END of Postgres DB Setup Process ***

...

[INFO] --- jetty-maven-plugin:9.1.2.v20140210:run-war (start-jetty) @ BLAH ---
[INFO] Configuring Jetty for project: BlahProject
[INFO] Context path = /
[INFO] Tmp directory = some/path/to/my/webapp/target/tmp
[INFO] Web defaults = org/eclipse/jetty/webapp/webdefault.xml
[INFO] Web overrides =  none
[INFO] jetty-9.1.2.v20140210
[INFO] Scanned 1 container path jars, 133 WEB-INF/lib jars, 1 WEB-INF/classes dirs in 1887ms for context o.e.j.m.p.JettyWebAppContext@444942b0{/,file:/some/path/to/my/webapp/,STARTING}{/some/path/to/my/Application-2.3.33+46be96b464dc5b57b2e2e04ce31718a01360e5fb.war}
[INFO] Initializing Spring root WebApplicationContext
INFO:20161021 12:58:27: org.springframework.web.context.ContextLoader org.springframework.web.context.ContextLoader Root WebApplicationContext: initialization started
INFO:20161021 12:58:27: org.springframework.web.context.support.XmlWebApplicationContext org.springframework.context.support.AbstractApplicationContext Refreshing Root WebApplicationContext: startup date [Fri Oct 21 12:58:27 EDT 2016]; root of context hierarchy
INFO:20161021 12:58:27: org.springframework.beans.factory.xml.XmlBeanDefinitionReader org.springframework.beans.factory.xml.XmlBeanDefinitionReader Loading XML bean definitions from class path resource [spring/app-config.xml]
INFO:20161021 12:58:27: org.springframework.beans.factory.xml.XmlBeanDefinitionReader org.springframework.beans.factory.xml.XmlBeanDefinitionReader Loading XML bean definitions from class path resource [spring/db-config.xml]
INFO:20161021 12:58:28: org.springframework.beans.factory.support.DefaultListableBeanFactory org.springframework.beans.factory.support.DefaultListableBeanFactory Overriding bean definition for bean 'endpointLTERepository': replacing [Root bean: class [org.springframework.data.jpa.repository.support.JpaRepositoryFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null] with [Root bean: class [org.springframework.data.jpa.repository.support.JpaRepositoryFactoryBean]; scope=; abstract=false; lazyInit=false; autowireMode=0; dependencyCheck=0; autowireCandidate=true; primary=false; factoryBeanName=null; factoryMethodName=null; initMethodName=null; destroyMethodName=null]

[INFO] Started ServerConnector@3a8f9130{HTTP/1.1}{0.0.0.0:8080}
[INFO] Started Jetty Server

/End of Maven console output

I recognize that the postgresql-embedded component is designed to start-up and shut-down a Postgres DB instance all within the same Unit test. I am trying to find a way to use it that it goes further than the originally intended use case. Essentially, I would like a postgresql-embedded service of some kind that can be launched from a maven plugin and can subsequently then be used to shut down that postgres process.

Any suggestions on how to create/build such a service &/or plugin?

Answer 1

The core problem here, is to be able to share some state between two different goals of a plugin: a start goal that would launch a process, and then a stop goal that would kill it. A good way to do that is to make use of the ContextEnabled interface that all mojos implement. It provides a getPluginContext() method that returns a (raw) map, in which you can store objects to be shared among mojos.

With this approach, you can store something you created in the start goal of a plugin, and get it back in the stop goal. Here's a minimalistic example to show this in action, where a simple String value is shared between mojos.

Set up a Maven plugin project. This basically comes down to having a project with the following POM, which is the standard POM for a Maven plugin, using Java 8 and annotations for configuration:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>sample.plugin</groupId>
  <artifactId>test-maven-plugin</artifactId>
  <version>1.0.0</version>
  <packaging>maven-plugin</packaging>
  <properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <maven.compiler.source>1.8</maven.compiler.source>
    <maven.compiler.target>1.8</maven.compiler.target>
  </properties>
  <build>
    <plugins>
      <plugin>
        <artifactId>maven-plugin-plugin</artifactId>
        <version>3.5</version>
      </plugin>
    </plugins>
  </build>
  <dependencies>
    <dependency>
      <groupId>org.apache.maven</groupId>
      <artifactId>maven-plugin-api</artifactId>
      <version>3.3.9</version>
    </dependency>

    <!-- dependencies to annotations -->
    <dependency>
      <groupId>org.apache.maven.plugin-tools</groupId>
      <artifactId>maven-plugin-annotations</artifactId>
      <version>3.4</version>
      <scope>provided</scope>
    </dependency>
  </dependencies>
</project>

Note the packaging of type maven-plugin which declares to Maven that this is a plugin project. In this new project, consider the following StartMojo:

@Mojo(name = "start", defaultPhase = LifecyclePhase.PRE_INTEGRATION_TEST)
public class StartMojo extends AbstractMojo {

    @SuppressWarnings("unchecked")
    @Override
    public void execute() throws MojoExecutionException {
        getPluginContext().put("myService", new MyService("foo"));
    }

}

This is declaring a new start mojo which is bound by default to the pre-integration-test phase. It retrieves the plugin context and puts a new object in it. In the above, it is a simple custom POJO called MyService which takes a value in its constructor. This object is mapped to a key of "myService", which serves as look-up.

Then, we can have:

@Mojo(name = "stop", defaultPhase = LifecyclePhase.POST_INTEGRATION_TEST)
public class StopMojo extends AbstractMojo {

    @Override
    public void execute() throws MojoExecutionException {
        MyService service = (MyService) getPluginContext().get("myService");
        getLog().info(service.getValue());
    }

}

This is declaring a new stop mojo which is bound by default to the post-integration-test phase. It retrieves the plugin context, extracts the object under the key "myService", and finally get its value and logs it.

After packaging and installing this Maven plugin (with mvn clean install) into your local repo, you can use it in a sample project with

<plugin>
  <groupId>sample.plugin</groupId>
  <artifactId>test-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>sample</id>
      <goals>
        <goal>start</goal>
        <goal>stop</goal>
      </goals>
    </execution>
  </executions>
</plugin>

If you run mvn clean verify on that sample project, you'll end up having "foo" printed in your logs, in the post-integration-test phase. This shows that the value was correctly set-up by the start mojo, and then correctly retrieved by the stop mojo.

Of course, you can store complex objects in this map, not just a String (for which there could be more simple solutions). Notably, it could be a host for your process instance that you want to stop. You can get rid of the exec-maven-plugin, create a new Maven plugin containing the code you already have to set up the embedded database in a start goal, store the process instance in the plugin context in this goal, and finally stop this process later in another stop mojo by retrieving it from the plugin context.

Answer 2

Follow-up: Based on @Tunaki suggestion, I did indeed create my own maven plugin that uses a fork of the postgresql-embedded to run an in-process download/installation/initialization of a 9.2 postgreSQL DB.

Had to work out a few kinks, but this approach did succeed for me.

Here is the small maven plugin that can be used as a maven wrapper around postgresql-embedded.