Python how to build dictionary from nested protobuf definitions

Question

This is my first time working with grpc/protobuf tech. So far, so good I have coded grpc methods for simple (flat?) protobuf definitions. However I have run into nested protobuf declarations and have no idea how to formulate the dictionary payload. Unfortunately the team the created these files is offering little to no coding support.

Snippet of proto file:

message UpdateClusterRequest {
    myapp.infrastructure.common.Context context = 1;
    myapp.infrastructure.common.MyAppEnvironment myapp_environment = 2;
    string service_name = 3;

    ClusterTemplate spec = 4;
    string config_revision = 5;

}

message ClusterTemplate {
    message MyAppSettings {
        string version = 1;
        repeated InstanceType instance_layout = 2;
        repeated ClientIDTemplate client_ids = 3;

        bool dedicated_mock_service = 4;
        int64 mock_service_replicas = 5;
    }

    message DbSettings {
        string version = 1;
        repeated DBHostTemplate db_hosts = 2;
        bool analytics_enabled = 3;
    }

My code snippet is below. The issue I have is I don't know how to code the message Cluster Template part into the playload_cluster_detail payload:

def update_cluster(client_stub, console_project_id, metadata_okta_token_and_env):

    try:

        payload_cluster_detail = {"context": {"project_id": console_project_id},
                                  "myapp_environment": common_pb2.MYAPP_ENVIRONMENT_TEST,
                                  "service_name": "web-2024-app",
                                  "config_revision": "1824866",
                                  "spec": {
                                    "MyAppSettings": {
                                        "version": "1824866-unix64-clang-debug",
                                        "branch": "web_mainline",
                                    }
                                  }
                                  }
        request = myapp_pb2.UpdateClusterRequest(**payload_cluster_detail)
        response = client_stub.get_grpc_myapp_stub(grpc_stub_method).UpdateCluster(request=request, metadata=metadata_okta_token_and_env)

When I run my code I get the following error:

ValueError: Protocol message ClusterTemplate has no "MyAppSettings" field.

`MyAppSettings` is a message (type). It defines a sub-message (type) for `ClusterTemplate` but the proto is missing a field within `ClusterTemplate` that references `MyAppSettings` (type). Something like `MyAppSettings my_app_settings = 99;` — DazWilkin, Nov 05 '22 at 01:46
If you were to have a definition per above, the dict key would be `"my_app_settings": { "version": "blah" }` but note that `MyAppSettings` has no `branch` field so this is invalid. — DazWilkin, Nov 05 '22 at 01:52
thanks @DazWilkin I get how to do that based on 1 level nesting as you pasted, however, focusing specifically on 'version' How can I write that as a dictionary? And sorry please ignore the branch I deleted some lines to make this post a lot smaller. — New2Python, Nov 07 '22 at 18:15

score 0 · Accepted Answer · answered Nov 07 '22 at 18:19

I munged your proto file to get it to work for me (as I don't have all your dependencies):

x.proto:

syntax = "proto3";

// (ADDED:dazwilkin) to compile
message Context {
    string project_id = 1;
}

message UpdateClusterRequest {
    Context context = 1;
    // myapp.infrastructure.common.MyAppEnvironment myapp_environment = 2;
    string service_name = 3;

    ClusterTemplate spec = 4;
    string config_revision = 5;

}

message ClusterTemplate {
    message MyAppSettings {
        string version = 1;
        // repeated InstanceType instance_layout = 2;
        // repeated ClientIDTemplate client_ids = 3;

        bool dedicated_mock_service = 4;
        int64 mock_service_replicas = 5;
    }

    message DbSettings {
        string version = 1;
        // repeated DBHostTemplate db_hosts = 2;
        bool analytics_enabled = 3;
    }

    MyAppSettings my_app_settings = 1;
    DbSettings db_settings = 2;
}

NOTE I added MyAppSettings my_app_settings = 1;

Then:

python \
-m grpc_tools.protoc \
--proto_path=${PWD} \
--python_out=${PWD} \
${PWD}/x.proto

Then I can:

import x_pb2


console_project_id = "x"

payload_cluster_detail = {
    "context": {
        "project_id": console_project_id
        },
    "service_name": "web-2024-app",
    "config_revision": "1824866",
    "spec": {
        "my_app_settings": {
            "version": "1824866-unix64-clang-debug",
        },
        "db_settings": {
            "version": "",
            "analytics_enabled": True,
        }

    }
}

x = x_pb2.UpdateClusterRequest(**payload_cluster_detail)
print(x)

Thank you! I was about to reply with another "but how?". After staring back and forth I was, I discovered a line identical to what you added. I failed to notice this as our message definition is fairly long and I missed it. Anywho thank you!! — New2Python, Nov 07 '22 at 22:03
It's one reason why I prefer flat (rather than nested) messages. — DazWilkin, Nov 07 '22 at 23:07

Python how to build dictionary from nested protobuf definitions

1 Answers1