How to make async http request in php across datacenters

Question

We are working on Kafka based event collection pipeline which will accept incoming event through HTTP GET / POST requests. It should be able to receive event by user browser as well as web server which are running on Nginx + PHP-FPM. By the very nature of requirement, this pipeline has to be elastic and hence we'll deploying in AWS Singapore, while our client serving web servers lie in physical datacenter in Mumbai, India. We don't want to introduce any latency due to this hence PHP app should just fire a request and forget about it, to begin with even ~5% requests getting lost is OK. Roundtrip between Mumbai and Singapore is around ~70 milisecond. What options can we use :-

• Curl or fsockopen requests with timeout of ~100 ms -- not acceptable
• Using libraries like Guzzle or curl-easy -- which internally uses curl only, so will again introduce latency
• Write the request locally to a file and use a bus like Flume or Kafka to transfer -- essentially we'll be replicating the even gathering pipeline, increased hardware cost.
• Using worker queue like gearman -- Not an experts on thisso don't know the pros and cons.
• Any thing under the sun, provided its free and low maintenance.

Please suggest.

Answer 1

I would use a jobqueue system, even though it introduces another layer of complexity. It is one of the simplest ways of asynchronously doing things in PHP and guaranteed to take almost no synchronous processor time.

You could use Gearman or ZeroMQ. They are both very well supported in PHP.