FHIR Migration and Backward Compatibility

Question

As system implementors, we face a dilemma when we migrate from one version of FHIR to the next. We started off using FHIR 0.0.81 and then moved to SVN revision 2833 on Sept. 10, 2014, to incorporate a bug fix. As suggested, we downloaded the Java code from SVN trunk and followed the directions on the FHIR Build Process page.

FHIR 0.0.82 Incompatibilities

Now that FHIR 0.0.82 is available, we want to upgrade to a released version. After downloading 0.0.82, however, we noticed that several resources such as Appointment that were in trunk rev2833 are not in release 0.0.82. This leads to our first questions:

1. What does trunk contain if it does not contain the latest code destined for the next release?

2. Should anyone ever use what's in trunk?

3. Is there a release branch from which 0.0.82 was created?

Trunk Incompatibilities

Since our code has dependencies on resources introduced on trunk but not included in 0.0.82, we have to continue to checkout FHIR directly from SVN. On Oct. 21, 2014, we downloaded SVN revision 3218 Java code. When we integrated that code into our system, we discovered numerous compatibility issues. Here are some of them:

1. Various Enum values changed from lowercase to uppercase, including Patient.AdministrativeGender and HumanName.NameUser. Though conforming to the Java naming convention is a good idea, changing fundamental data types breaks compilation.

2. Method names have changed, also resulting in compilation errors. We also discovered that simultaneous name changes occurred. For example, in the HumanName class the old setTextSimple(String) is now setText(String), and the old setText(StringType) is now setTextElement(StringType). Both the name and parameter type of setText() has changed, making migration error prone because one has to decide at each use whether to change the method or its parameter.

3. The ResourceReference resource type has changed its class name. In the FHIR model package alone, 859 occurences of ResourceReference in 61 file were affected. This does not include changes that rippled through other FHIR packages, or changes that will ripple through our application code and our database schemas.

4. We notice several new resources in the rev3218 trunk code, including NewBundle. Previously, we had suggested that bundles should be resources, so it's great to see this change. Since trunk is not backward compatible with the 0.0.8x releases, however, I'm not sure if we will have to support the both the old and new way of parsing and composing JSON and XML bundles.

To put a finer point on things, it's important to recognize that some of the above FHIR changes not only affect compilation, but could easily introduce subtle bugs at runtime. In addition, the FHIR changes could require database schema changes and data migration in some applications. For example, our application saves JSON resource streams in a database. Something as simple as changing an enum value from "male" to "MALE" requires migration utilities that update existing database content.

Going Forward

We are investing heavily in FHIR; we want it to succeed and to be adopted widely as a standard. In order for that to occur, issues of backward compatibility and version migration need to be addressed. In that vain, any light that can be shed on the following question will move us all forward:

1. What is the purpose of the 0.0.8x line of code? Who are its target users?

2. What is the purpose of the code in trunk? Who are its target users?

3. Will users of 0.0.8x ever be expected to migrate to the trunk codebase?

   1. If so, what migration strategy will used to address the many incompatibilties between the codebases?

4. What is the deprecation policy for code in each codebase?

5. What level of backward compatibility can be expected from revision to revision in the code in trunk?

6. Is there a FHIR roadmap that system developers can use to plan there own development cycles?

Thanks, Rich C

Answer 1

My apologies for not documenting the way versioning affects the Java reference implementation more. I'll do so. I will assume that you are familiar with the versioning policy here: http://hl7-fhir.github.io/history.html

There are two versions of FHIR extant at the moment. The first is DSTU 1. This is a fork in SVN ("dstu1"), and is only changed for significant bug reports. The reference implementation there is maintained and backwards compatible. The second version is the trunk version, in which we are preparing for the second DSTU release. The svn is highly unstable at the moment - changing constantly, and we are sometimes reversing changes several times, as we consider various options in committee. Further, there are several large breaking changes between DSTU1 and trunk, and more are coming. So you should not expect that switching between DSTU1 and trunk will be painless. Nor should implementers be using trunk unless they're really bleeding edge (and tightly connected, e.g. the implementers skype channel). When trunk is stable, and we think it's worth releasing an implementers beta, we update the versions and version history, and make a release here: http://hl7.org/implement/standards/FHIR-Develop/ and release a maven package for that version.

In the trunk, since there are many changes being made, we also changed the constants to uppercase, and flipped the way that get/set properties were generated. Agree that this has a price, but there was already a price to pay for switching from DSTU1 to trunk. And when I do a beta release (soon, actually), I'll update the Java reference implementation number and so forth. Note that the Java constants went to uppercase, but the wire format constants did not change, so stored json streams are fine (though they are broken by many other changes in the specification)

Given the scope of the changes between DSTU 1 and trunk (there is no list of these yet, I will have to prepare that when I update the beta), you should expect extensive rework for the transition. Presently, I maintain a single source that implements a server for both (in Pascal, http://github.com/grahamegrieve/fhirserver) but I suspect that this approach is about to get broken by the change the NewBundle represents.

So, specific answers:

1. What is the purpose of the 0.0.8x line of code? Who are its target users?

Supporting users of the existing DSTU1 specification

2. What is the purpose of the code in trunk? Who are its target users?

preparing to be DSTU 2. It should start to be more stable in a few weeks time - once we started making backwards incompatible changes, we are trying to get as many of them done as possible now

3. Will users of 0.0.8x ever be expected to migrate to the trunk codebase?

yes, when DSTU 2 is released, or at least, when we start having connectathons on the trunk version preparing for DSTU2 (first one is planned for January)

4. If so, what migration strategy will used to address the many incompatibilities between the codebases?

There's going to be a lot of re-writing code. We may release xml transforms for migrating resources from DSTU1 to DSTU2 when it is finalised, but that may not even be possible

4a. What is the deprecation policy for code in each codebase?

DSTU 1 is extremely conservative. trunk will settle, though we will never guarantee stability. The beta releases will be point in time releases of these.

5. What level of backward compatibility can be expected from revision to revision in the code in trunk?

None, really, at the moment.

6. Is there a FHIR roadmap that system developers can use to plan their own development cycles?

Well, in addition to the version policy referenced above, there's this: http://www.healthintersections.com.au/?p=2234 (which was for you, no?)

Answer 2

As a supplement to Grahame's response: On the Documentation tab of the spec, there's only one bolded link - Read Prior to Use. That page that tries to make clear that the DSTU release promises neither forward nor backward compatibility. It can't - the whole purpose of DSTU is to gather implementation feedback about what sort of substantive changes are needed to make the standard ready to be locked in stone when we go normative. If we promised forward and backward compatibility in DSTU, then we'd be stuck with whatever decisions we'd made during the initial draft, whether they turned out to be good ones or not.