How to hit ElasticSearch using Apache HttpClient

Question

I have SearchRequest object with all the Elasticsearch(ES) query data set. I cannot use RestHighLevel client for my usecase because it requires endpoint need to be passed at the time of instantiation. I gets ES endpoint dynamically based on some condition. One way is to always create new RestHighLevel client which will be inefficient approach. Other way is to create static CloseableHttpClient on service start and make HttpPost request with dynamic endpoint. I wanted to take later approach but don't know how to convert SearchRequest object into json query string.

Any code reference/snippet would be very helpful

private final CloseableHttpClient client;

    public GenericElasticSearchResponse search(@Nonnull final SearchRequest searchRequest,
                                                   @Nonnull final RoutingConfig route) {
    final URIBuilder builder = new URIBuilder()
                            .setScheme(route.getScheme())
                            .setHost(route.getESEndpoint())
                            .setPort(Optional.ofNullable(route.getPort())
                                    .orElse(80))
                            .setPath("/sessions*/_search");
    
                    final URI uri = builder.build();
    
                    final ContentType contentType = ContentType.create("application/json", "UTF-8");
                   
    
                    final HttpPost httpPost = new HttpPost(uri);
                    httpPost.setEntity(entity);
    
                    final CloseableHttpResponse response = client.execute(httpPost);
    
                    final String responseEntity;
                    try (final Reader reader = new InputStreamReader(response.getEntity().getContent(), Charsets.UTF_8)) {
                        responseEntity = CharStreams.toString(reader);
                    }
    
                    
                    final SearchResponse searchResponse = objectMapper.readValue(responseEntity, SearchResponse.class);
                    
                    return new ElasticSearchResponse(searchResponse);
    }

Can you please add what dynamic URl you want to create or what is your condition for creating URL? If you want to pass diffrent query and parameter based on condition then i will suggest to use DSL query in HighLevel client. — Sagar Patel, Jan 11 '22 at 10:51

score 0 · Answer 1 · answered Jan 12 '22 at 04:32

I found searchRequest.source().toString() was actually returning json form of SearchRequest. Following is complete code snippet for hitting ES via Apache client

 final EndpointConfig endpoint = route.getEndpoint();
                final URIBuilder builder = new URIBuilder()
                        .setScheme(endpoint.getScheme())
                        .setHost(endpoint.getHost())
                        .setPort(Optional.ofNullable(endpoint.getPort())
                                .orElse(HTTPS_PORT))
                        .setPath(Optional.ofNullable(endpoint.getQueryPath())
                                .orElse(StringUtils.EMPTY));
    
                final URI uri = builder.build();
                final ContentType contentType = ContentType.create("application/json", "UTF-8");
                final String queryString = searchRequest.source().toString();
                final StringEntity entity = new StringEntity(queryString, contentType);
                
    
                final HttpPost httpPost = new HttpPost(uri);
                httpPost.setEntity(entity);
    
                final CloseableHttpResponse response = sendRequest(httpPost);
    
                final String responseEntity;
                try (final Reader reader = new InputStreamReader(response.getEntity().getContent(), Charsets.UTF_8)) {
                    responseEntity = CharStreams.toString(reader);
                }
                log.info("ElasticSearchClient response: Code: {}, Entity {}", response.getCode(), responseEntity);
                SearchResponse searchResponse = null;
                if (Objects.nonNull(responseEntity)) {
                    searchResponse = parseResponse(responseEntity, searchRequest, response.getCode());
                    log.info("ElasticSearchClient searchResponse- {} ", searchResponse);
                }
    
                return new ElasticSearchResponse(searchResponse);
    
            } catch (final URISyntaxException e) {
               
                throw new IllegalStateException(
                        String.format("Invalid URI. host: %s", route.getEndpoint()), e);
            } catch (final IOException e) {
                
                throw new IllegalStateException("ElasticSearch Request failed.", e);
            }





private SearchResponse parseResponse(@Nonnull final String responseEntity,
                                         @Nonnull final SearchRequest searchRequest,
                                         final int responseCode) {
        if (responseCode >= 400 || responseCode < 200) {
            log.info("ES error response - {} ", responseEntity);
            final ESErrorResponse response = GSON.fromJson(responseEntity, ESErrorResponse.class);
            throw new IllegalStateException();
        }

        SearchResponse searchResponse = null;
        final NamedXContentRegistry registry = new NamedXContentRegistry(getDefaultNamedXContents());
        final XContentParser parser;
        try {
            parser = JsonXContent.jsonXContent.createParser(registry,
                    DeprecationHandler.THROW_UNSUPPORTED_OPERATION, responseEntity);
            searchResponse = SearchResponse.fromXContent(parser);
        } catch (IOException e) {
            
            throw new IllegalStateException("Error while parsing response ", e);
        }
        return searchResponse;
    }

    public static List<NamedXContentRegistry.Entry> getDefaultNamedXContents() {
        final Map<String, ContextParser<Object, ? extends Aggregation>> map = new HashMap<>();
        map.put(TopHitsAggregationBuilder.NAME, (p, c) -> ParsedTopHits.fromXContent(p, (String) c));
        map.put(StringTerms.NAME, (p, c) -> ParsedStringTerms.fromXContent(p, (String) c));
        return map.entrySet().stream()
                .map(entry -> new NamedXContentRegistry.Entry(Aggregation.class, new ParseField(entry.getKey()), entry.getValue()))
                .collect(Collectors.toList());
    }

    
    private CloseableHttpResponse sendRequest(final HttpPost httpPost) throws IOException {
        return client.execute(httpPost);
    }

How to hit ElasticSearch using Apache HttpClient

1 Answers1