Make bulk http calls

Question

Hi below is my problem scenario

I have a Java Servlet which can handle http requests to 4 different servers. So for one user in my database I have to make 20 different API calls to each of the server i.e. 80 http calls.

Now i have 1000 users which makes it 80000 calls one time to retrieve data and store in my database.

The problem part is that I have to call this after a gap of every 10 mins to get the new data. So i have a scheduler to do so. So finally its 80000 calls every 10 mins from my servlet and handling the response. I need to optimize the situation and improve performance somehow with threads Async Calls or both or some other way.

Have anyone faced a similar situation? Please help me optimize it.

yes ,I am using db pooling to write the data to database once i get response from server. — hybrid, Mar 12 '15 at 06:14

score 1 · Answer 1 · answered Mar 12 '15 at 06:10

1

Have you looked at using Apache HttpComponents? They have a "Concurrent asynchronous HTTP exchanges" example on the examples page that sounds like a starting point to what you're doing.

answered Mar 12 '15 at 06:10

Jeff Evans

1,257
1
15
31

score 1 · Answer 2 · answered Sep 13 '15 at 09:20

1

You can create a thread pool of fixed size and connect to the server.This is better than a single thread , single connection.

answered Sep 13 '15 at 09:20

harshali mahajan

19
4

Make bulk http calls

2 Answers2