2

Cloud Trace and Cloud Logging integrate quite nicely in most cases, described in https://cloud.google.com/trace/docs/trace-log-integration

Unfortunately, this doesn't seem to include the HTTP request logs generated by a Load Balancer when request logging is enabled.

The LB logs show the traces icon, and are correctly associated with an overall trace in the Cloud Trace system, but the context menu 'show trace details' is greyed out for those log items.

A similar problem arose with my application level logging/tracing, and was solved by setting the traceSampled attribute on the LogEntry, but this can't work for LB logs, since I'm not in control of their generation.

In this instance I'm tracing 100% of requests since the service is M2M and fairly low volume, but in the general case it makes sense that the LB can't know if something is actually generating traces without being told.

I can't find any good references in the docs, but in theory a response header indicating it was sampled could be observed by the LB and cause it to issue the appropriate log.

Any ideas if such a feature exists, in this form or any other?

(Last-ditch workaround might be to use Logs Router to feed LB logs into a pubsub queue (and exclude them from normal logging sinks), and resubmit them to the normal sink(s) with fields appropriately set by some Cloud Function or other pubsub consumer, but that seems like a lot of work and complexity for this purpose)

Shabble
  • 542
  • 3
  • 10

1 Answers1

3

There is currently a Feature Request created for this, you can follow the status in the following link 1.

As a workaround, you could implement target proxies along with your Load Balancer, according to the documentation for a Global external HTTP(S) load balancer:

The proxies set HTTP request/response headers as follows:

  • Via: 1.1 google (requests and responses)
  • X-Forwarded-Proto: [http | https] (requests only)
  • X-Cloud-Trace-Context: <trace-id>/<span-id>;<trace-options> (requests only) Contains parameters for Cloud Trace.
  • X-Forwarded-For: [<supplied-value>,]<client-ip>,<load-balancer-ip> (see X-Forwarded-For header) (requests only)

You can find the complete documentation about external HTTP(S) load balancers and target proxies here 2.

And finally, take a look at the documentation on how to use and configure target proxies here 3.

Gabriel Robledo Ahumada
  • 1,542
  • 2
  • 14
  • 2
    Thanks. I'm not sure I follow about how target-proxies would work to solve/workaround this issue? I currently have an External Global HTTPS LB, terminating in a Target Proxy, which then uses a URL Map to direct all traffic to a 'Serverless NEG' backend. The problem isn't that Cloud Run doesn't receive the x-trace header set by teh LB (it does), it's that the LB logs don't appear to set traceSampled, allowing the trace<->logs linkage in Logs Explorer. – Shabble Dec 20 '21 at 16:23