Is it possible to run subpart of the whole dependency tree under aws data pipeline. As to what i could understand there is no way to do that.
Either one has to rerun the entire pipeline or just some single sql activity. Which becomes difficult to do as the number of sql activity starts increasing. ..
I'm not sure what your use case is, but I think this trick may help you.
You will need to decompose your graph in set of trees without intersections. All edges those connect nodes from different tree should be replaced with joint-nodes. Fore example if you graph is:
A1 -> A2 -> B1 -> B2
Here are two trees A and B. This graph should be transformed into graph bellow:
A1 -> A2 -> J1 -> B1 -> B2
J1 is joint-node.
All nodes in trees should have property failureAndRerunMode set to cascade, all joint-nodes should have this property set to none.
This allow you to localize cascade failures down to the size of subtrees.
