0

We're a new Adtech company and I was planning to design a database where I'll pull all the data to a single table and then make new tables with a materialized views for others to generate multiple reports.

Say we have Inventory, impression, views for multiple reasons.

enter image description here

Our main table looks like this, to recreate this

CREATE TABLE report.empty_summing (times DateTime64,inventory_id String,city Nullable(String), country Nullable(String),inventory Int32 default 0, impression Int32 default 0, views Int32 default 0) ENGINE=SummingMergeTree() primary key inventory_id;

When a request comes from google ADX to our Adengine , it has a unique id which is "inventory_id" and other parameters like country, city..... other string type parameters are inserted.

When 3 types of data are inserted it looks like this.

enter image description here

You can see that Every row have their values inserted but I want to

Our inventory request insert looks like this.

INSERT INTO report.empty_summing (times,inventory_id,country,city,inventory,impression,views) VALUES (now(),'7120426e6abd0b04ec8c777460a78bdf4b9de0','Bangladesh','Dhaka',1,0,0);

Our impression insert looks like this.

INSERT INTO report.empty_summing (times,inventory_id,impression) VALUES (now(),'7120426e6abd0b04ec8c777460a78bdf4b9de0',1);

Our view insert looks like this.

INSERT INTO report.empty_summing (times,inventory_id,views) VALUES (now(),'7120426e6abd0b04ec8c777460a78bdf4b9de0',1);

You can see that "inventory_id" is the same for all these rows. is there any DB engine or any technique I can use where data will be merged and look like this?

enter image description here

Help is much appreciated. thanks in advance!

Aniruddha Chakraborty
  • 1,849
  • 1
  • 20
  • 32
  • That's an aggregation query, supported by every database I've ever come across. – Nick.Mc Aug 24 '21 at 10:50
  • okay but using group by query, the results are not what i expected @Nick.McDermaid – Aniruddha Chakraborty Aug 24 '21 at 11:02
  • 2
    I suggest you edit your question and explain further, including the query you used. Also please use text rather than pictures – Nick.Mc Aug 24 '21 at 11:03
  • 3
    It's worth thinking about data size - if you're an adtech company, presumably you're handling millions of requests / day, so this table could very easily grow to billions or trillions of records. While I agree with @Nick.McDermaid that this looks like a trivial query on every RDBMS, "trillions of records" may present a problem. I recommend Kleppmann's book for perspective on this. – Neville Kuyt Aug 24 '21 at 11:22

1 Answers1