How to retrieve bazel remote cache metrics on the client

Question

Context

We want to get client metrics so that we can track how long each target takes to build, how big the target outputs are and if they're remote cache hits.

Normally, we can retrieve the size of each output, if it's a remote cache hit and how long it took to compile/download using the following two flags:

• --execution_log_json_file: Gives us output size, remoteCacheHit, number of files downloaded

• --profile: Gives us details on when an action started and how long it took (but it's not very detailed)

Our use case

We can't use the execution log because our build is much too big (several GBs in size).

We can't use the profile file because it doesn't have information on remote cache hits or size of the files.

Ideally we would want a combination of both.

Question

Is there a way to retrieve all of this information in one location? Would the Build event protocol expose these metrics? There's this issue open that might be related.

Answer 1

The build event protocol is the way to go. We're using BuildBuddy to get those metrics. We also modified our fork of BuildBuddy to save/retrieve custom metrics.