0

Considering an example for a shop

I have an XSD for a shop, and a number of tables having data relating to inventory, records etc. I have a DB-file specifying where the data related to the XSD lies in the tables. I need to refer this file and XSD to create XML records for multiple shops

My current solution is to generate JPA entities using HyperJAXB through the XSD and read the data to generate the XML's but I need to make code changes every time the DB-file and XSD have changes.

Is it possible to accommodate these changes at runtime whilst using JPA because the DB structure is complex. How can I minimize the effort required to accommodate changes if not doing at runtime.

SKaul
  • 349
  • 2
  • 7
  • 22
  • Would you be able to give some details about what your trying to accomplish with those xml files? Usually a database is the backend for a restful webservice or an application. It almost looks like you want to continuously create an xml representation of your current database state. What are these xml files used for? – Matthew Fontana May 12 '16 at 12:10
  • The xml is for an external legacy system we need to feed information to, it is not ideal but I cannot create a webservice for the same. – SKaul May 12 '16 at 12:21
  • ok thanks for the update – Matthew Fontana May 12 '16 at 12:40

1 Answers1

0

Ok based off of your current use case I really think using a product like SSIS which specializes in ETL processes could do this the most efficiently. I am fairly certain there is even a way to avoid translation by having SSIS create the xml.

However, if you want to keep doing the process in java I would recommend moving away from JPA due to an object representing the database. This means you will always have coding changes when the database schema gets updated. I would take a step back and leverage more raw SQL over a JdbcTemplate. You can get a fairly generic process by leveraging a few SQL commands:

SHOW tables

then you can get tall results for each table

SELECT * FROM table

This will return a result set which could be turned into XML for each table with a method like...

public static Document toDocument(ResultSet rs) throws ParserConfigurationException, SQLException  {
   DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
   DocumentBuilder builder = factory.newDocumentBuilder();
   Document doc = builder.newDocument();

   Element results = doc.createElement("Results");
   doc.appendChild(results);

   ResultSetMetaData rsmd = rs.getMetaData();
   int colCount = rsmd.getColumnCount();

   while (rs.next()) {
      Element row = doc.createElement("Row");
      results.appendChild(row);

      for (int i = 1; i <= colCount; i++)   {
         String columnName = rsmd.getColumnName(i);
         Object value = rs.getObject(i);

         Element node = doc.createElement(columnName);
         node.appendChild(doc.createTextNode(value.toString()));
         row.appendChild(node);
      }
  }
   return doc;
}

This should let you print every table as XML, if you need it to be hierarchy based there is obviously a bit more work involved, but this would enable you to create a generic process that can export all database tables as xml.

Update #1:

Based off some conversations with my co-workers the best practices we follow to helping to manage database migrations still has a lot of manual work, but here are some of the mitigation steps we take from our practices.

We use schema versioning with either liquibase or flyway to manage schema changes in our database across multiple environments. This can guarantee that regardless of environment the schema will be correct.

We also generate the schema from hibernate and use schema comparison tools (these tend to be database specific) to validate that the hibernate model will be identical to the database model. This operation is done by our Jenkins build server typically to ensure every build is compatible with the desired database schema version.

Hibernate configuration is also set to hibernate.hbm2ddl.auto=validate to ensure compatibility with the database tables on start up.

Our local application uses the Junit framework and creates an instance of hibernate running on derby for database style unit / integration testing. These tests ensure that previous compatibility should not be broken.

When a database is shared between multiple java applications the JPA layer is occasionally shared across all of these applications and multiple groups will be responsible for that particular code base. This practice helps keep all applications in sync with database changes and leveraging the same access patterns. Typically a host of generic named queries are also created to avoid application teams needing to create custom queries.

Unfortunately we have not come across a silver bullet that could automatically evaluate a database schema change and update the application. Currently the process of driving database changes from application JPA changes isn't accepted due to generation process of hibernate not being ideal for every database. As well as a lot of the risks with the hibernate auto update the schema feature which isn't a production grade feature.

Well that turned out to be quite... wordy, but I hope I gave you some insight into your problem.

Matthew Fontana
  • 3,790
  • 2
  • 30
  • 50
  • The data operations we need to perform are highly governed by business logic and we need to make a lot of customizations before making the XML, also this could be used by a third party which is why I am more into using JPA rather than go for ETL tool. – SKaul May 12 '16 at 13:17
  • Ok gotcha starting to see what is happening here. So there are business rules and potentially a transformation layer before the xml conversion. Ok let me check with a few of my co-workers and see if we do anything special here. My hunch is we just mitigate the risks with our CI process and a ton of unit / integration tests on top of derby. But I will get back to you. – Matthew Fontana May 12 '16 at 13:22
  • It was a bit wordier than intended, but I hope some of the process we use can help you out. – Matthew Fontana May 12 '16 at 15:05