1

Is there a way to denormalize data once its already loaded in elasticsearch/kibana? Here is a specific example of what I'm trying to do:

If you have records pertaining to sales of cars like so: "Car Sale", "Camaro", "$20,000" "Car Sale", "Malibu", "$15,000" "Car Sale", "TrailBlazer", "$23,000" "Car Sale", "Fusion", "$18,000"

And you have separate records with further information about these cars: "Camaro", "Chevrolet" "Malibu", "Chevrolet" "TrailBlazer", "Chevrolet" "Fusion", "Ford"

The chart I want to generate is the car prices aggregated based on make, so what are the total prices for Chevy vs. Fords? One option would be to denormalize the data before I load it into elasticsearch but I feel like that's going to force me to know what my data is up front and not adapt to changes very well.

So I can generate a bar chart based on the car sale records I have and get prices by specific models of cars but at that point, I don't have access to those other records that give me that car make aggregation since they exist on separate records. I feel like I want to "start a new query" in the filter boxes of the kibana chart but it doesn't look like you can do that? Any other ideas?

Bill Niebruegge
  • 11
  • 1

1 Answers1

1

Elasticsearch/Kibana really don't support joins in a relational sense.

You really do need to know what you planned usage looks like before you model you data, but you can still do things nimbly.

You just have to be willing to reload your data from source and denormalize it at that point. Reloading records into elasticsearch can be done at the rate of thousands per second, so unless you have sale records for the entire world, it's not going to take you more than a few hours to re-index your data.

Alcanzar
  • 16,985
  • 6
  • 42
  • 59