Indexing In ElasticSearch For Auditing

Question

There is a microservice-based architecture wherein each service has a different type of entity. For example:

Service-1:

{
    "entity_type": "SKU",
    "sku": "123",
    "ext_sku": "201",
    "store": "1",
    "product": "abc",
    "timestamp": 1564484862000
}

Service-2:

{
    "entity_type": "PRODUCT",
    "product": "abc",
    "parent": "xyz",
    "description": "curd",
    "unit_of_measure": "gm",
    "quantity": "200",
    "timestamp": 1564484863000
}

Service-3:

{
  "entity_type": "PRICE",
  "meta": {
    "store": "1",
    "sku": "123"
  },
  "price": "200",
  "currency": "INR",
  "timestamp": 1564484962000
}

Service-4:

{
  "entity_type": "INVENTORY",
  "meta": {
    "store": "1",
    "sku": "123"
  },
  "in_stock": true,
  "inventory": 10,
  "timestamp": 1564484864000
}

I want to write an Audit Service backed by elasticsearch, which will ingest all these entities and it will index based on entity_type, store, sku, timestamp.

Will elasticsearch be a good choice here? Also, how will the indexing work? So, for example, if I search for store=1, it should return all the different entities that have store as 1. Secondly, will I be able to get all the entities between 2 timestamps?

Will ES and Kibana (to visualize) be good choices here?

score 0 · Answer 1 · answered Jul 30 '19 at 23:54

Yes. Your use case is pretty much exactly what is described in the docs under filter context:

In filter context, a query clause answers the question “Does this document match this query clause?” The answer is a simple Yes or No — no scores are calculated. Filter context is mostly used for filtering structured data, e.g.

Does this timestamp fall into the range 2015 to 2016?

Is the status field set to published?

Indexing In ElasticSearch For Auditing

1 Answers1