How to make Protobuf 3 fields mandatory?

Question

I'm writing my first API endpoint in GoLang using GRPC/proto-buffers. I'm rather new to GoLang. Below is the file I'm writing for my test case(s)

package my_package

import (
    "context"
    "testing"

    "github.com/stretchr/testify/require"

    "google.golang.org/protobuf/types/known/structpb"
    "github.com/MyTeam/myproject/cmd/eventstream/setup"
    v1handler "github.com/MyTeam/myproject/internal/handlers/myproject/v1"
    v1interface "github.com/MyTeam/myproject/proto/.gen/go/myteam/myproject/v1"
)

func TestEndpoint(t *testing.T) {
    conf := &setup.Config{}

    // Initialize our API handlers
    myhandler := v1handler.New(&v1handler.Config{})

    t.Run("Success", func(t *testing.T) {
        res, err := myhandler.Endpoint(context.Background(), &v1interface.EndpointRequest{
            A: "S",
            B: &structpb.Struct{
                Fields: map[string]*structpb.Value{
                    "T": &structpb.Value{
                        Kind: &structpb.Value_StringValue{
                            StringValue: "U",
                        },
                    },
                    "V": &structpb.Value{
                        Kind: &structpb.Value_StringValue{
                            StringValue: "W",
                        },
                    },
                },
            },
            C: &timestamppb.Timestamp{Seconds: 1590179525, Nanos: 0},
        })
        require.Nil(t, err)

        // Assert we got what we want.
        require.Equal(t, "Ok", res.Text)
    })


}

This is how the EndpointRequest object is defined in the v1.go file included above:

// An v1 interface Endpoint Request object.
message EndpointRequest {

  // a is something.
  string a = 1 [(validate.rules).string.min_len = 1];

  // b can be a complex object.
  google.protobuf.Struct b = 2;

  // c is a timestamp.
  google.protobuf.Timestamp c = 3;

}

The test-case above seems to work fine.

I put validation rule in place that effectively makes argument a mandatory because it requires that a is a string of at least one. So if you omit a, the endpoint returns a 400.

But now I want to ensure that the endpoint returns 400 if c or b are omitted. How can I do that? In Protobufs 3, they got rid of the required keyword. So how can I check if a non-string argument was passed in and react accordingly?

Answer 1

Required fields were removed in proto3. Here is github issue where you can read detailed explanation why that was done. Here is excerpt:

We dropped required fields in proto3 because required fields are generally considered harmful and violating protobuf's compatibility semantics. The whole idea of using protobuf is that it allows you to add/remove fields from your protocol definition while still being fully forward/backward compatible with newer/older binaries. Required fields break this though. You can never safely add a required field to a .proto definition, nor can you safely remove an existing required field because both of these actions break wire compatibility

IMO, that was questionable decision and obviously i'm not alone, who's thinking that. Final decision should have been left to developer.

Answer 2

The short version: you can't.

required was removed mostly because it made changes backwards incompatible. Attempting to re-implement it using validation options is not quite as drastic (changes are easier), but will run into shortcomings as you can see.

Instead, keep the validation out of the proto definition and move it into the application itself. Anytime you receive a message, you should be checking its contents anyway (this was also true when required was a thing). It is a rare case that the simple validation provided by options or required is sufficient.