I'm aware of the usage limits when using Google's free, publicly available Javascript Maps API. Currently 25,000 daily Maps loads are your limit.
It seems though that also limits on the delivering servers are somehow in place.
I'm running a website on https://zugspitze360.com that is heavily using the API to display photo spheres (panorama photos) and it seems that for certain users or times the loading of imagery is resulting in 403 HTTP responses on Google servers.
Can't find any errors in the Google Developer Console though. And also can't find any information on usage limits in terms of imagery loads, etc