0

I am looking into gcloud log shell command line, I started with a classic sample:

gcloud beta logging write --payload-type=struct my-test-log "{\"message\": \"My second entry\", \"weather\": \"aaaaa\"}"

It works fine so I checked the throughputwith the following code its works veru slaw (about 2 records a sec) is this the best way to do so?

Here is my sample code

tail -F -q -n0 /root/logs/general/*.log | while read line
do
    echo $line
    b=`date`
    gcloud beta logging write --payload-type=struct my-test-log "{\"message\": \"My second entryi $b\", \"weather\": \"aaaaa\"}"
done
Avi Zloof
  • 2,923
  • 4
  • 22
  • 28

1 Answers1

2

If you assume each command execution takes around 150ms at best, you can only write a handful of entries every second. You can try using the API directly to send the entries in batches. Unfortunately, the command line can currently only write one entry at a time. We will look into adding the capability to write multiple entries at a time.

If you want to stream large number of messages fast, you may want to look into Pub/Sub.

Vilas
  • 1,405
  • 12
  • 15
  • So if I understand correctly, with the current 500 ms response when using stack driver the quta limit is not relevant because i will not have an capabilities to get even near that. Its an issue – Avi Zloof Apr 27 '16 at 09:26
  • 500ms response is not a stackdriver limitation, it depends on your internet connection. In general, when you make a HTTP request over the internet at best it will take around 100ms depending on where you are and where the request is being served from. That is why you would want to batch or stream to get more throughput. – Vilas Apr 27 '16 at 14:51