12

I have to write a RPC service for some Python apps. I have an IDL file called solver.proto and generated some code from it using the grpcio-tools module. Here is the resulting project structure:

solver-service
  ├── main.py
  └── protos/
       ├── __init__.py       
       ├── __pycache__
       │     └── ... 
       ├── generate_pb.py
       ├── solver.proto
       ├── solver_pb2.py
       └── solver_pb2_grpc.py

I have my PYTHONPATH set to /abs/path/to/solver-service and I try to import my auto-generated gRPC classes from this directory but hit the following:

solver-service$ python3
Python 3.5.2 (v3.5.2:4def2a2901a5, Jun 26 2016, 10:47:25)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import protos.solver_pb2
>>> import protos.solver_pb2_grpc
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/abs/path/to/solver-service/protos/solver_pb2_grpc.py", line 4, in <module>
    import solver_pb2 as solver__pb2
ImportError: No module named 'solver_pb2'

So for whatever reason I can import protos.solver_pb2 successfully but the interpreter complains about importing protos.solver_pb2_grpc due to some relative path issue. Calling all Python devs doing gRPC work out there -- how do I get this working?

nmurthy
  • 1,337
  • 1
  • 12
  • 24
  • Sometimes looking at `sys.path` hard helps. Isn't there anything funny in it? – 9000 Sep 01 '17 at 16:52
  • My `sys.path` has the project path (i.e PYTHONPATH) and all the libs in my venv. – nmurthy Sep 01 '17 at 18:28
  • 2
    The `--proto_path` (aka `-I`) flags passed to `protoc` could be relevant here. Please add your `protoc` command-line to the question. (In particular it looks like the code might have been generated with `--proto_path=solver-service/protos` but based on the way you're trying to import it, it should have been `--proto_path=solver-service`.) – Kenton Varda Sep 03 '17 at 01:26
  • 1
    Here's what I used to generate protobuf classes in python: https://gist.github.com/natemurthy/f1139c02269faca98b6eae60ea033a2f – nmurthy Sep 04 '17 at 20:44
  • `solver_pb2_grpc` currently expects to find `solver_pb2` at path top-level, but `solver_pb2` is in `protos`, not path top-level. I think the clearest solution is to move `generate_pb` to top-level and change its `.` contents to `protos/` (in three places). This should have the effect of changing the `import solver_pb2 as solver__pb2` line in `solver_pb2_grpc` to `from protos import solver_pb2 as solver__pb2` (or maybe `import protos.solver_pb2 as solver__pb2`?) and make the generated files usable. Try it out? – Nathaniel Manista At Google Sep 05 '17 at 17:25

2 Answers2

0

I know this is old, but usually you have to add from . before the import in the _grpc.py file. solver_pb2_grpc imports solver_pb2, but for some reason the PYTHONPATH doesn't check the directory where the protos are exported in python.

Eli Selkin
  • 29
  • 3
0

I had a similar issue couple of years back with a proto that imported another proto. Long story short, I spent about a month on this, tried all the public python proto generation libs I could find on Github but still couldn't solve the issue. The final solution was to fix the relative path of the import inline using sed as part of the build script. Trust me, anyone seeing this issue in the future, it's not worth spending time fixing it, just figure out the correct path and write a script to replace the imports in the generated files.

farbo
  • 1
  • 1