Handling null values in protobuffers

Question

I am working on something which fetches data from database and constructs protobuff message. Given the possibility that null values can be fetched from the database for certain fields , I will get Null-pointer exception while trying to construct the protobuff message. Getting to know that null is not supported in protobuffs from the thread http://code.google.com/p/protobuf/issues/detail?id=57, I am wondering whether the only other way to handle NPE getting thrown is to insert manual checks into the java file corresponding to the proto like below!

message ProtoPerson{
    optional string firstName = 1;
    optional string lastName = 2;
    optional string address1 = 3;
}

ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
if (p.getFirstName() != null) builder.setFirstName(p.getFirstName());
if (p.getLastName() != null) builder.setLastName(p.getLastName());
if (p.getAddress1() != null) builder.setAddress1(p.getAddress1());
...

So can someone please clarify whether there is any other possible efficient way to handle the null values during protobuff construction??

Answer 1

Disclaimer: Answer from a Googler using protobufs on a daily basis. I'm by no means representing Google in any way.

1. Name your proto Person instead of PersonProto or ProtoPerson. Compiled protobufs are just class definitions specified by the language you are using, with some improvements. Adding "Proto" is extra verbosity.
2. Use YourMessage.hasYourField() instead of YourMessage.getYourField() != null. Default value for protobuf string is an empty string, which does NOT equal to null. Whereas, no matter whether your field is unset or cleared or empty string, .hasYourField() always returns false. See default values for common protobuf field types.
3. You've probably known, but I wanna say explicitly: Don't programmatically set a protobuf field to null. Even for outside of protobuf, null causes all sorts of problems. Use .clearYourField() instead.
4. Person.Builder class does NOT have a .newBuilder() method. Person class does. Understand the Builder Pattern like this: You create a new builder only if you do not have it yet.

A rewrite of your protobuf:

message Person {
  optional string first_name = 1;
  optional string last_name = 2;
  optional string address_1 = 3;
}

A rewrite of your logic:

Person thatPerson = Person.newBuilder()
    .setFirstName("Aaa")
    .setLastName("Bbb")
    .setAddress1("Ccc")
    .build();

Person.Builder thisPersonBuilder = Person.newBuilder()

if (thatPerson.hasFirstName()) {
  thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}

if (thatPerson.hasLastName()) {
  thisPersonBuilder.setLastName(thatPerson.getLastName());
}

if (thatPerson.hasAddress1()) {
  thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}

Person thisPerson = thisPersonBuilder.build();

And if thatPerson is a person object that you created that has attribute values that could be an empty string, empty spaces or null, then I'd recommend using Guava's Strings library:

import static com.google.common.base.Strings.nullToEmpty;

Person.Builder thisPersonBuilder = Person.newBuilder()

if (!nullToEmpty(thatPerson.getFirstName()).trim().isEmpty()) {
  thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}

if (!nullToEmpty(thatPerson.hasLastName()).trim().isEmpty()) {
  thisPersonBuilder.setLastName(thatPerson.getLastName());
}

if (!nullToEmpty(thatPerson.hasAddress1()).trim().isEmpty()) {
  thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}

Person thisPerson = thisPersonBuilder.build();

Answer 2

Proto 3

wrappers.proto supports nullable values:

• string(StringValue),
• int(Int32Value),
• bool(BoolValue)
• and etc

Example

syntax = "proto3";
import "google/protobuf/wrappers.proto";

message ProtoPerson {
    google.protobuf.StringValue firstName = 1;
    google.protobuf.StringValue lastName = 2;
    google.protobuf.StringValue address1 = 3;
    google.protobuf.Int32Value age = 4;
}

Answer 3

There's no easy solution to this. I'd recommend just dealing with the null checks. But if you really want to get rid of them, here are a couple ideas:

• You could write a code generator plugin which adds setOrClearFoo() methods to each Java class. The Java code generator provides insertion points for this (see the end of that page).
• You could use Java reflection to iterate over the get*() methods of p, call each one, check for null, and then call the set*() method of builder if non-null. This will have the added advantage that you won't have to update your copy code every time you add a new field, but it will be much slower than writing code that copies each field explicitly.