0

I have a problem my java web service. How can i manage big data (more than 10 GB). I have to send to consumer this data. Please give me some advise.

ILHOMJON
  • 9
  • 2
  • Where is the data stored? Or is it generated on-the-fly? – Thilo Dec 18 '14 at 03:45
  • How about storing such data in one common downloadable location, and providing the link of file to user. Now it's users responsiblity to download appropriately. E.g Make data Serializable and write it to some file, let client to download this file and de-serialize it. – Gaurav Gupta Dec 18 '14 at 03:48

2 Answers2

1

It depends on many things :

  1. Response time required
  2. Is data computation heavy or are you generating data
  3. Where is data stored

Most probably your data will be stored somewhere and I assume you have map-reduced it. You can provide this data to your clients using pagination.

prady00
  • 721
  • 1
  • 6
  • 17
0

-If it is really not a necessity to serve all the data at one go,I would suggest creating filter based API's .

-Even batch processing this data serving let's say 50 records at a time with pagination at the front end will help you with the response time.

-You can use compression techniques like gzip compression on the webserver

abhati
  • 309
  • 1
  • 6