I would need some help with the pipeline of the Rhino ETL framework.
Required is a kind of operation chaining which,
- Allows branching,
- Operates sequential (row by row),
- Returns the original input row to the pipeline which entered the branch.
A - B / | \ A - A - C - A \ | / A - D[]
1. Register / Partial.Register
Requirement b.) is covered by usual Register(...) and Partial.Register(...) initialisation, but by design this return the converted row to the pipeline
Register(new ExampleProduce<ClassA>());
Register(Partial
.Register(new ExampleConvertTo<ClassB>())
.Register(new ExampleOperateWith<ClassB>()));
// wrong row: B
Register(Partial
.Register(new ExampleConvertTo<ClassC>())
.Register(new ExampleOperateWith<ClassC>()));
// wrong row: C
Register(Partial
.Register(new ExampleConvertTo<IList<ClassD>>())
.Register(new ExampleOperateWith<IList<ClassD>>()));
// completely other row context, multiple rows of type D
Register(new ExampleOperateWith<ClassA>());
2. Branching Operation
Requirement a.) is covered by the use of the BranchingOperation, but every Branch processes all rows before the pipeline switches to another and never returns to the original pipeline.
Register(new ExampleProduce<ClassA>());
// runs every branch for all rows before switching to the next one
Register(new BranchingOperation()
.Add(Partial.Register(new ExampleConvertTo<ClassB>())
.Register(new ExampleOperateWith<ClassB>()))
.Add(Partial.Register(new ExampleConvertTo<ClassC>())
.Register(new ExampleOperateWith<ClassC>()))
.Add(Partial.Register(new ExampleConvertTo<IList<ClassD>>())
.Register(new ExampleOperateWith<IList<ClassD>>())));
// never reaches
Register(new Deploy<ClassA>());
3. Caching rows
Dirty (and not working in my scenario) possibility to resolve that issue: Saving the current row in a column before the "branch" and restoring it after. This would depend, that every operation in the branch does not change the column in which the original row is stored, at least in the last branch of the example this is impossible.
Register(new ExampleProduce<ClassA>());
Register(new StoreToRow<ClassA>());
Register(Partial
.Register(new ExampleConvertTo<ClassB>())
.Register(new ExampleOperateWith<ClassB>()));
Register(new RestoreFromRow<ClassA>());
Register(new StoreToRow<ClassA>());
Register(Partial
.Register(new ExampleConvertTo<ClassC>())
.Register(new ExampleOperateWith<ClassC>()));
Register(new RestoreFromRow<ClassA>());
Register(new StoreToRow<ClassA>());
Register(Partial
.Register(new ExampleConvertTo<IList<ClassD>>())
.Register(new ExampleOperateWith<IList<ClassD>>()));
Register(new RestoreFromRow<ClassA>());
// completely other "row context", multiple rows of type D
Register(new ExampleOperateWith<ClassA>());
Is there any implementation of the BranchingOperation (like SequentialBranchingOperation...?) that meet these requirements? Or any other idea how to resolve that issue?