How do I build WHL packages after changing the code?

Question

I added some log codes in grpc/third_party/protobuf/src/google/protobuf code, to help locate the problem, i install grpc python from source, but The change did not take effect. Please give me some advice, thanks.

Supported Python Versions

Python 3.7.9

What operating system (Linux, Windows,...) and version?

Ubuntu 18.04.5 LTS

What did you do?

I added some log codes in grpc/third_party/protobuf/src/google/protobuf code， then i compiled protobuf , compiled grpc and install grpc python from source

mkdir -p "third_party/protobuf/cmake/build"
cd  "third_party/protobuf/cmake/build"
cmake -Dprotobuf_BUILD_TESTS=OFF -DCMAKE_BUILD_TYPE=Release ..
make -j14 install
popd
make -j14

i modified setup.py, add this code:

CORE_C_FILES = filter(lambda x: 'third_party/protobuf' not in x, CORE_C_FILES)
PROTOBUF_INCLUDE = (os.path.join('/usr', 'include', 'protobuf'),)
EXTENSION_INCLUDE_DIRECTORIES = ((PYTHON_STEM,) + CORE_INCLUDE + ABSL_INCLUDE +
                                 ADDRESS_SORTING_INCLUDE + CARES_INCLUDE +
                                 RE2_INCLUDE + PROTOBUF_INCLUDE + SSL_INCLUDE + UPB_INCLUDE +
                                 UPB_GRPC_GENERATED_INCLUDE +
                                 UPBDEFS_GRPC_GENERATED_INCLUDE +
                                 XXHASH_INCLUDE + ZLIB_INCLUDE)

then i run:

pip install -rrequirements.txt
GRPC_PYTHON_BUILD_WITH_CYTHON=1 pip install .

I'm confused, why aren't the changes to Protobuf working? Maybe there is something wrong with my installation method, please give me some suggestions,thanks.

Answer 1

Answered in the grpc-io thread:

The grpcio package doesn't actually depend on protobuf. Only generated code (i.e. _pb2.py and _pb2_grpc.py) have this dependency. Instead, you'll want to build the protobuf wheel from the protobuf repository and use that alongside an off-the-shelf grpcio wheel.

Also, to answer the obvious follow-up question: "Why do we have a vendored protobuf directory in our repo at all then?". The grpcio-tools wheel does build in the libprotobuf native code to generate the aforementioned _pb2.py and _pb2_grpc.py files.