2

I currently have a program that takes in a query from a user via REST (Spring)and runs it across the database and returns the results via REST. The issue I'm running into is that if the user queries for a large set of data, at a certain point the server runs into a out of memory error.

Is there a way to stream the results using REST Spring to avoid the out of memory error? I've been researching and it seems like HTTP Chunked Encoding might be an option.

hyperstack
  • 29
  • 1
  • 4
  • See [Spring Web MVC](https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc) documentation on [Async Requests](https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-async), specifically [**HTTP Streaming**](https://docs.spring.io/spring/docs/current/spring-framework-reference/web.html#mvc-ann-async-http-streaming). – Andreas Jun 20 '18 at 21:59
  • [Spring Web Reactive](https://docs.spring.io/spring-framework/docs/5.0.0.M1/spring-framework-reference/html/web-reactive.html) also comes to mind. – RikH Jun 20 '18 at 23:19

2 Answers2

1

Maybe SSE (https://en.wikipedia.org/wiki/Server-sent_events) can help you. SSE is a web technology where a browser receives updates from a server via HTTP connection. Examples:

Mariano Z Lopez
  • 36
  • 2
0

I would recommend looking at the new Spring repository streamAll functionality. I found the article linked below tremendously helpful (though quite old).

I am using this approach myself to stream pretty large datasets and I have no memory issues. The secret sauce is to limit the fetch size as described in the article below.

https://knes1.github.io/blog/2015/2015-10-19-streaming-mysql-results-using-java8-streams-and-spring-data.html

Simbosan
  • 244
  • 2
  • 11