Unable to use google/apis/annotations

Question

I try to use REST over gRPC using google api annotations. Unfortunately, I'm facing a protoc issue telling me annotations.proto is not exists or had errors. I tried several fixes in vain.

I even tried to reinstall a complete stack in case I did something wrong. I will detail you as much as possible the full lines and files I have set from my fresh install.

From a fresh go install VM I type these shell lines :

$ mkdir sources/golang

$ echo 'export GOPATH=$HOME/sources/golang' >> $HOME/.zshrc

$ source ~/.zshrc

$ cd sources/golang

$ mkdir src

$ cd src

$ export PATH=$PATH:$GOPATH/bin

$ go get -u google.golang.org/grpc

$ go get -u github.com/golang/protobuf/protoc-gen-go

$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway

$ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-swagger

$ mkdir -p test/proto/test

$ cd test/proto/test

$ vim test.proto

In My test.proto file, I wrote very basic lines :

syntax = "proto3";

package main;

import "google/api/annotations.proto";

service Tester {
    rpc Test (Request) returns (Reply) {
        option (google.api.http) = { get: "/v1/test" };
    }
}

message Request {
    string Message = 1;
}

message Reply {
    string Message = 1;
}

Then

$ cd $GOPATH/src

$ vim main.go

In my main.go, very basic too :

package main

import (
        tc "test/proto/test"
        "context"
        "fmt"
        "log"
        "net"

        grpc "google.golang.org/grpc"
        codes "google.golang.org/grpc/codes"
)

func main() {
        err := StartServer("tcp", "127.0.0.1:50051")
        if err != nil {
                fmt.Printf("Error!! %s", err)
        }
}

type Server struct {
        tc.UnimplementedTesterServer
}

func (s *Server) Test(ctx context.Context, in *tc.Request) (*tc.Reply, error) {
        return &tc.Reply{Message: ""}, nil
}

func StartServer(protocol string, port string) error {
        lis, err := net.Listen(protocol, port)
        if err != nil {
                fmt.Printf("failed to listen: %v", err)
        }
        s := grpc.NewServer()
        tc.RegisterTesterServer(s, &Server{})

        error := s.Serve(lis)

        if error != nil {
                fmt.Printf("failed to serve: %v", error)
                return error
        }

        return nil
}

Finally, I try to compile my proto files :

$ protoc --proto_path=.:$GOPATH/src --go_out=plugins=grpc:. proto/*/*.proto

And I systematically have the following error :

proto/test/test.proto:5:1: Import "github.com/grpc-ecosystem/grpc-gateway/third_party/googleapis/google/api/annotations.proto" was not found or had errors.

When I see files get by go get ..., they are placed under $GOPATH/pkg/mod/

For example the googleapis' annotations.proto file is under : $GOPATH/pkg/mod/github.com/grpc-ecosystem/grpc-gateway@v1.12.1/third_party/googleapis/google/api/annotations.proto

Maybe is it the cause?

Answer 1

Mixing up your protobuf commands and your go commands is making this more complicated than it needs to be. Just focus on the protobuf to handle the error.

You are importing "google/api/annotations.proto"

Your proto_path is --proto_path=.:$GOPATH/src

So that means that when you execute protoc you should have the file located at ./google/api/annotations.proto or $GOPATH/src/google/api/annotations.proto

Answer 2

I faced the same problem and I was trying to automate a solution that would work on any Linux with as little dependencies as possible.

The official docs for grpc_gateway states:

You will need to provide the required third party protobuf files to the protoc compiler. They are included in this repo under the third_party/googleapis folder, and we recommend copying them into your protoc generation file structure. If you've structured your proto files according to something like the Buf style guide, you could copy the files into a top-level ./google folder.

In other words: download it somehow and make it work.

Also, since this quote is also a quote in the original source, it is not very visible which made me skip it the first 2 times I read it looking for a solution, which is bad.

So, as I argued on the comments of @SeanF answer, using go get is not a maintainable option since it saves the project in a folder whose name contains the latest version, which would cause headaches to maintain when the version changes.

So the best option is actually to clone the grpc-gateway project:

git clone https://github.com/grpc-ecosystem/grpc-gateway
protoc -I grpc-gateway/ \
    -I grpc-gateway/third_party/googleapis \
    --go_out=plugins=grpc:. proto/*/*.proto

I also wrote a gist with my solution which depends only on docker and bash and, thus, should be stable:

https://gist.github.com/VinGarcia/43dfa71b412c16b9365c224d3760af5e

There is a problem with it, where I am using go.mod but discarding it after every execution, which is not good. But it works well enough for now and it might be helpful to some people.

Regards