2

I am using Client Library.

for entry in logging_client.list_entries(order_by=DESCENDING, filter_=FILTER, page_size=1):

    timestamp = entry.timestamp.isoformat()
    print('****{}'.format(entry.payload_pb))
    print (entry.payload_pb)
    print('* {}: {}: {}'.format(timestamp, entry.logger.name, entry.insert_id))

But payload_pb, has just two attributes: value and type_url. And otherwise I can fetch just a few values like timestamp, logName, insertId.

I want to fetch the values inside 'protopayload'. Please suggest a way!

Swati Sneha
  • 355
  • 2
  • 17

4 Answers4

3

Actually you can make the API return a dict (JSON) payload by setting the environment variable GOOGLE_CLOUD_DISABLE_GRPC to a non-empty string, e.g. GOOGLE_CLOUD_DISABLE_GRPC=true.

This will populate payload instead of payload_pb.

yan-hic
  • 1,374
  • 11
  • 26
1

Despite the hints provided, I could not make it work with the API.

Instead you could export the logs to a sink like BigQuery and query using SQL from there. See here how-to https://cloud.google.com/logging/docs/export/configure_export_v2

Since there is no code out there - sample or example - on how to read the specific protobuf payload from GCP services, it may not be ready for prime time or not meant to be used that way. AAMOF the only documentation is - as of today - only for writing & reading custom logs.

Here the issue I had raised if you want to follow up on: https://github.com/GoogleCloudPlatform/google-cloud-python/issues/5313

yan-hic
  • 1,374
  • 11
  • 26
0

The value of the proto payload is a serialized protocol buffer message. You'll need to deserialize it to read the fields within the message.

https://developers.google.com/protocol-buffers/docs/pythontutorial

The proto payload is an Any message, and the type of the value is indicated by type_url

https://developers.google.com/protocol-buffers/docs/proto3#any

Once you have the proto message, you can convert it to JSON with json_format.MessageToJson.

It would also be possible to handle the Any message directly with MessageToJson as long as the descriptor of the proto message is known. This can be done by updating the proto symbol database via `RegisterMessage'.

Kirk Kelsey
  • 4,259
  • 1
  • 23
  • 26
  • OP asked specifically for GCP messages as `json_format.MessageToJson(entry.payload_pb)` fails on `TypeError: Can not find message descriptor by type_url: type.googleapis.com/google.cloud.audit.AuditLog.` – yan-hic May 09 '18 at 13:00
  • Sounds like you're trying to convert the `Any` directly to JSON, rather than a message deserialized from `Any.value`. I added a note about registering the descriptor that might help. – Kirk Kelsey May 11 '18 at 20:35
0

I couldn't find any way for client library. However, I used Python API,

loglist = bq.entries().list(body=request_body).execute()

Here, bq is the response from discovery.build and loglist provides the response in JSON format, with combinations of dicts and lists. We have to dig in them, to get the exact key value pair.

I hope this just helps somebody who needs to get job done somehow, like me!

Swati Sneha
  • 355
  • 2
  • 17