0

Following the Quick Start gRPC Go guide on the official gRPC website it has a step which asks the user to recompile the updated .proto file using this command:

$ protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative helloworld/helloworld.proto

I'm a little confused as to how the newly compiled protobuf files are consumed by the "human written" Go code.

In the example "human written" Go code they reference the protobuf code using the following import:

pb "google.golang.org/grpc/examples/helloworld/helloworld"

protoc does not update this package but instead updates the helloworld/helloworld.proto in the same directory that the command is run in. How does the protoc command ensure that the "human written" Go code consumes the newly compiled protobuf code?

PJConnol
  • 119
  • 1
  • 9

1 Answers1

1

I find this one of the more confusing aspects of Protobufs (and gRPC).

I think the issue being solved is that Protobufs need to:

  1. Permit namespacing to scope services|messages to e.g. DNS domains
  2. Support a myriad programming languages (which implement namespacing differently).

protochas options that permit (re)mapping of e.g. a protobuf's package v1/api to a language-specific namespace using protobuf options, e.g. go_package). See Go Generated Code for Golang.

This is slightly more complex when using Go Modules but, in summary, what you're experiencing is probably some (unexpected) combination of the above where, the sample code assumes one module name and protoc is building on the assumption of a different one.

TL;DR update the code's module reference to reflect the correctly generated pb path. If the generate code is in the wrong place, you can simply move it to the correct subdirectory (path) but it's better to update your protoc command to generate the files to the correct directory.

Example

something.proto:

syntax = "proto3";

package v1alpha1;

option go_package = "github.com/me/my-protos;v1alpha1";

NOTE go_package aliases the proto package v1alpha1 to what I want to reference as github.com/me/my-protos in Golang.

Then I generate with:

MODULE="github.com/me/my-protos"
protoc \
--proto_path=. \
--go_out=./api/v1alpha1 \
--go_opt=module=${MODULE} \
--go-grpc_out=./api/v1alpha1 \
--go-grpc_opt=module=${MODULE} \
./something.proto

NOTE This example generate gRPC code too. It avoids (!) protoc creating a path github.com/me/my-protos for the generated code because I'm generating the source in that repo. I just want the relative path ./api/v1alpha where the files will be created.

Which yields:

my-protos
├── something.proto
├── api
│   └── v1alpha1
│       ├── something_grpc.pb.go
│       └── something.pb.go
├── go.mod
├── go.sum
└── README.md

And I can import with:

import (
    pb "github.com/me/my-protos/api/v1alpha1"
)

NOTE From a different repo, I can now access my protos repo github.com/me/my-project/api/v1alpha1 combining the repro and the generated location to give my desired path.

DazWilkin
  • 32,823
  • 5
  • 47
  • 88
  • This is great, thank you very much, I'll try to test this out as soon as possible. Your example is a lot clearer than the tutorial here https://grpc.io/docs/languages/go/quickstart , would it be correct to ask the GRPC developers to update the documentation to use something similar to your example instead? – PJConnol Jun 19 '21 at 23:49
  • I'm pleased to hear that you find it useful, thank you. The tutorials aim to provide a straightforward overview without overwhelming the reader. I think this complexity should be reflected somewhere on the site. It's always "correct" to propose updates to the docs if you want them. Perhaps submit a PR? – DazWilkin Jun 20 '21 at 15:13