I implemented curl code to fetch to live inventory from the vendor web service when any user opens the product detail page.
But search engine bots are accessing the page and this results in curl call to web service 1000 times every second.
How to rectify this issue. I searched over the google but did not find accurate answer yet.
At least you should detect User-Agent and IP address when page called. If received arguments matched as bot's data, just don't call to web service and use permanent static data (only for bots).
In addition, you can use tool for human check (captcha etc.): if checking failed, don't call web service.
Also, you can use caching and call web service only for cache update (e.g., once every hour): if data updated less than an hour ago, page should use cached data.
