So, my question is: i have a bronze - silver injestion, and later this data will be consumed by gold notebooks, i wan't to check if all tables that notebooks uses is updated. What i thinked, use dlt expectations and in my golds, i will create expectations to check if the used silvers timestamps are dated, but dlt expectations only support expectations in the current table column scope...
Another ways that i think to solve:
Check with code inside each notebook (bad to keep changing, always need to warn to put a check block)
run before a notebook, a test one in the pipeline for each gold notebook, which checks and generates an error if any table is missing (it will clog the test nb)
check with code which notebooks can run and from a loop go to understand the available notebooks (do not modify anything inside the codes, the pipeline is scalable, however, visibility is lost)
block overwrite and update of delta tables that should not be updated, generating an error (kind of bad because the append still works)
ideal: silver generates an error, gold does not update (it does not work in our scenario)
