Python protofub: how to pass response message from one grpc call to another

Question

I'm new to grpc/protobuf so please excuse any terminology errors in my question.

I need to take a response from one gRPC request and feed it into the next request. I can't figure out how to populate the "spec" line.

Proto file1:

message UpdateClusterRequest {
    string service_name = 3;

    ClusterTemplate spec = 4;
    string config_revision = 5;
    string deploy_strategy = 6;

}

Proto file2:

message ClusterTemplate {
    message AppSettings {
        string version = 1;
        repeated InstanceType instance_layout = 2;
        repeated ClientIDTemplate client_ids = 3;
    }

    AppSettings app = 1;
}

So in my code, the template_response captures the output from the get_template_revisions gRPC API call. I then need to pass the contents to request.spec to the next gRPC API request, which is what I need help with.

template_response=get_template_revisions(client_stub,payload_project_id,metadata_okta_token_and_env)grpc_logger.debug(template_response.revisions[0].template.app)

request=app_pb2.UpdateClusterRequest()
request.spec = ???

response=client_stub.get_grpc_app_stub(grpc_stub_method).UpdateCluster(request=request,metadata=metadata_okta_token_and_env)

This is a heavily nested message mapping and I have tried many permutations without success below and not limited to:

request.spec.extend([template_response.revisions[0].template.app])

request.spec = template_response.revisions[0].template

request.spec.MergeFromString(template_response.revisions[0].template.app)

I've read all the python protobuf documentation and I just can't get it.

You don't include the (proto message) type of `template_response` in your question but, from the code (`template_response.revisions[0].template.app`), it suggests that you consider the type of `app` in the response to be similar (!) to the type to `AppSettings`, correct? Similar types (multiple definitions that have the same structure) aren't considered equivalent (instances of the same message type) and, because `AppSettings` is defined as a nested type (within `ClusterTemplate`) it **cannot** be equivalent the `template_response`'s `app` as this must be defined elsewhere. — DazWilkin, Nov 17 '22 at 04:05
I went cross eyed lol, I clearly dont have a full understanding of how all this works. The proto files were built and supplied by another team (who offer little to no support). What I know is that the output from "template_response.revisions[0].template.app" is that data that I need to somehow pass to request.spec. Ill keep rereading your reply in hopes I'll realize something. ty. — New2Python, Nov 17 '22 at 18:09
Not sure if this helps but this is the data content that needs to be passed into request.spec: app { version: "v1.0.3" instance_layout { name: "Master" count: 1 } instance_layout { name: "Slave" count: 2 } client_ids { platform: PLATFORM1 client_id: "Y24_PL1_APP_SERVER" } client_ids { platform: PLATFORM2 client_id: "Y24_PL2_APP_SERVER" } } } — New2Python, Nov 17 '22 at 18:17
Understood. Rereading this morning (less wine :-)) `request.spec` is a `ClusterTemplate`. Is any part of `template_response` a `ClusterTemplate` too? And, if so, is it **exactly** the same class (i.e `type(...)` same)? — DazWilkin, Nov 17 '22 at 18:41
Let me add an example as an answer and see whether that helps — DazWilkin, Nov 17 '22 at 19:06

score 0 · Accepted Answer · edited Jan 30 '23 at 23:47

It's unclear from your question because the (message) type of template_response isn't explicit but hinted (template_response.revisions[0].template.app).

So...if the Proto were:

foo.proto:

syntax = "proto3";


message UpdateClusterRequest {
    string service_name = 3;

    ClusterTemplate spec = 4;
    string config_revision = 5;
    string deploy_strategy = 6;

}

message ClusterTemplate {
    message AppSettings {
        string version = 1;
        // repeated InstanceType instance_layout = 2;
        // repeated ClientIDTemplate client_ids = 3;
    }

    AppSettings app = 1;
}

// Assume TemplateResponse.Revision's template is a ClusterTemplate
message TemplateResponse {
    message Revision {
        ClusterTemplate template = 1;
    }

    repeated Revision revisions = 1;
}

Note: I've commented out InstanceType and ClientIDTemplate because they're also undefined but not necessary for the explanation.

And:

python3 \
-m grpc_tools.protoc \
--proto_path=${PWD} \
--python_out=${PWD} \
${PWD}/foo.proto

Then:

from google.protobuf.json_format import ParseDict

import foo_pb2

d = {
    "revisions":[
        {
            "template": {
                "app": {
                    "version": "1",
                }
            }
        },
        {
            "template": {
                "app": {
                    "version": "2",
                }
            }
        }

    ]
}
template_response = foo_pb2.TemplateResponse()

# Create a TemplateResponse from the dictionary
ParseDict(d,template_response)

# Its type is <class 'foo_pb2.ClusterTemplate'>
print(type(template_response.revisions[0].template))


# Create `UpdateClusterResponse`
update_cluster_request = foo_pb2.UpdateClusterRequest()

# Scalar assignments
update_cluster_request.service_name = "xxx"
update_cluster_request.config_revision = "xxx"
update_cluster_request.deploy_strategy = "xxx"

# Uses `google.protobuf.message.CopyFrom`
# Can't assign messages
update_cluster_request.spec.CopyFrom(template_response.revisions[0].template)


print(update_cluster_request)

Python is a little gnarly around protocol buffers. In other language implementations, you'd be able to assign the message more "idiomatically" but, in Python, it's not possible to assign messages (among other quirks).

So, assuming that template_response.revisions[*].template is exactly the same type as UpdateClusterRequest's spec type, then you can use CopyFrom to achieve this.

Thank you so much for spending your time on this, much appreciated. CopyFrom is indeed what this working for me. request.spec.CopyFrom(template_response.revisions[0].template). — New2Python, Nov 18 '22 at 17:35

Python protofub: how to pass response message from one grpc call to another

1 Answers1