0

What are the standard ways in which data from a data warehouse can be used to query an external API and then write back to the table in the warehouse?

With a normal database + backend server, it can talk directly to the DB, read from it, query an API, and write back to the row to update a column. But with a data warehouse, a normal backend server won't have direct access to it.

So what are the usual techniques that people use for warehouses? Lambda functions that run on demand? Schedulers like Airflow?

rasen58
  • 4,672
  • 8
  • 39
  • 74
  • If you have a process that works with a “normal” database why do you think this won’t work with a data warehouse database - especially considering that the same DBMS can run multiple types of database? – NickW Feb 01 '22 at 08:20
  • @NickW because normally the warehouse is meant for reading by BI analytics tools and your application level backend wouldn't have direct access to it – rasen58 Feb 01 '22 at 15:56
  • But whether you writeback to a DW, or not, is a business decision not a technology decision. If you've decided that you have a requirement to writeback to your DW then any process that works with a DBMS being used as a "normal" database will also work with the same DBMS being used as a data warehouse database – NickW Feb 01 '22 at 17:10
  • Sure, but given this business decision, what do other companies who have made similar decisions do in this case? Just looking to understand standard practices. – rasen58 Feb 01 '22 at 22:29
  • There is no standard practice. There 100s/1000s of software solutions/components that can call APIs and write data to a database - you’ve described one in your question – NickW Feb 01 '22 at 22:46

0 Answers0