0

I am facing issue while reading large JSON data (>100MB) using webClient in one shot (Chunks/pagination is not supported from Client end). Also I tried using restTemplate but no luck.

WebClient code:

public List<Phone> fetchPhoneDetails(PhoneRequest request) {
        try {
            PhoneResponseDTO response = webClientBuiler.build()
                    .post()
                    .uri(url)
                    .contentType(MediaType.APPLICATION_JSON).accept(MediaType.APPLICATION_JSON)
                    .body(BodyInserters.fromObject(request))
                    .retrieve().bodyToMono(PhoneResponseDTO.class).block();

            if(null != response.getResponse()) {
                return objectMapper.convertValue(response.getResponse(), new TypeReference<List<Phone>>(){ });
            }
        } catch (HttpClientErrorException e) {
            throw e;
        }
        return new ArrayList<>();
    }

Using RestTemplate:

@Override
    public List<Phone> fetchPhoneDetails(Phone request) {
        try {
            HttpEntity<Phone> requestEntity = new HttpEntity<>(request, getHttpHeaders());
            PhoneResponseDTO response = new RestTemplate().postForObject(url, requestEntity, PhoneResponseDTO.class);
            if(null != response && null != response.getResponse()) {
                return objectMapper.convertValue(response.getResponse(), new TypeReference<List<Phone>>(){ });
            }
        } catch (HttpClientErrorException e) {
            throw e;
        }
        return new ArrayList<>();
    }

Error:

com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input in VALUE_STRING
 at [Source: (PushbackInputStream); line: 1, column: 84980163]
    at com.fasterxml.jackson.core.base.ParserMinimalBase._reportInvalidEOF(ParserMinimalBase.java:618) ~[jackson-core-2.9.9.jar:2.9.9]

Any help would be highly appreciated.

Thej
  • 191
  • 1
  • 6
  • 23
  • 1
    You can try two things, try to increase the Content-Length:in header, It will work. or you can use the GSON streaming check the below link about it https://sites.google.com/site/gson/streaming – Arun Prasat Feb 26 '20 at 04:44
  • Paging is always available. You just need to request them multiple times with a page parameter. Split data on server side, return total pages and do the loops on client side. – Kimi Chiu Feb 26 '20 at 06:19
  • @KimiChiu True, but we do are not allowed to modify anything from client side. So Paging cannot be done in my case. – Thej Feb 26 '20 at 07:18

0 Answers0