Pipeline class storing different templates with constraints

Question

I have a class template DataProcessor, which looks like this:

struct DataProcessorBase
{
    typedef std::shared_ptr<DataProcessorBase> Ptr;
}; // struct DataProcessorBase

template <class _Input, class _Output>
struct DataProcessor : DataProcessorBase
{
    typedef _Input Input;
    typedef _Output Output;

    virtual Output process(const Input * input) = 0;
}; // struct DataProcessor

I am looking to create a Pipeline class, which concatenates multiple DataProcessor instances together. This means that the Output of processor 1 must match the input of processor 2, and so forth. Something like the following:

template <class _Input, class _Output>
class Pipeline : DataProcessor<_Input, _Output>
{
public:
    Output process(const Input * input);
private:    
    std::vector<DataProcessorBase::Ptr> _processors;
}; // class Pipeline

template <class _Input, class _Output>
_Output Pipeline<_Input, _Output>::process(const _Input * input)
{
    // this is where I start guessing...
    auto rawPtr = dynamic_cast<DataProcessor<_Input, TYPEOFFIRSTPROCESSORSOUTPUT>*>(_processors[0]);
    assert(rawPtr);
    for (size_t i = 0; i < _processors.size(); ++i)
    {
        ...
    }
}

I can tell that this way of implementing Pipeline::process is not the right way. Can someone point me in the right direction?

Yakk - Adam Nevraumont · Accepted Answer · 2015-08-26T23:37:44.817

Decouple the in and out calls.

Data coming in and data coming out should happen at different steps. Then each consumer of data can know what it needs to demand, and do the casting for you (possibly throwing or error flagging if things go wrong).

struct processor {
  virtual ~processor () {};
  virtual bool can_read_from( processor const& ) const = 0;
  virtual void read_from( processor& ) = 0;
  virtual bool ready_to_sink() const = 0;
  virtual bool ready_to_source() const = 0;
};
template<class T>
struct sink {
  virtual void operator()( T&& t ) = 0;
  virtual ~sink() {}
};
template<class T>
struct source {
  virtual T operator()() = 0;
  virtual ~source() {}
};
template<class In, class Out, class F>
struct step: processor, sink<In>, source<Out> {
  F f;
  step( F&& fin ):f(std::move(fin)) {}

  step(step&&)=default;
  step(step const&)=default;
  step& operator=(step&&)=default;
  step& operator=(step const&)=default;
  step()=default;

  std::experimental::optional<Out> data;
  virtual void operator()( In&& t ) final override {
    data = f(std::move(t));
  }
  virtual bool ready_to_sink() const {
    return !data;
  }
  virtual Out operator()() final override {
    auto tmp = std::move(data);
    data = {};
    return std::move(*tmp);
  }
  virtual bool ready_to_source() const final override {
    return static_cast<bool>(data);
  }
  virtual bool can_read_from( processor const& o ) final override {
    return dynamic_cast<source<In> const*>(&o);
  }
  virtual void read_from( processor &o ) final override {
    (*this)( dynamic_cast<source<In>&>(o)() );
  }
};
template<class In, class Out>
struct pipe {
  std::shared_ptr<processor> first_step;
  std::vector< std::shared_ptr<processor> > steps;
  pipe(std::shared_ptr<processor> first, std::vector<std::shared_ptr<processor>> rest):
    first_step(first), steps(std::move(rest))
  {}
  Out operator()( In&& in ) {
    (*dynamic_cast<sink<In>*>(steps.first_step.get()))( std::move(in) );
    auto last = first_step;
    for (auto step:steps) {
      step->read_from( *last );
      last = step;
    }
    return (*dynamic_cast<source<Out>*>(last.get())();
  }
};
template<class In, class Out>
struct pipeline:step<In, Out, pipe<In,Out>> {
  pipeline( std::shared_pointer<processor> first, std::vector<std::shared_ptr<processor>> steps ):
    step<In, Out, pipe<In,Out>>({ first, std::move(steps) })
  {}
};

@KevinBrown Honestly, the first line is the answer. The rest is a sketch or illustration. The OPs big problem was that the in type, and the out type, are tied in one template and one call, while each element of the chain only knows half of those steps. — Yakk - Adam Nevraumont, Aug 26 '15 at 19:37
I stand in awe at your coding prowess. Thank you Yakk, this is exactly what I am looking for. — Moos Hueting, Aug 26 '15 at 20:27
Quick comment - std::experimental::optional data should be std::experimental::optional, and the operator()(T&& t) below should be opeator()(In&& t), no? — Moos Hueting, Aug 26 '15 at 21:01
@MoosHueting seems reasonable. I did a quick audit and replaced stuff that seemed wrong, especially around `T`s in templates that lack `T` types. Also note that while the above has type-checking mechanisms, I didn't use them in the sample code: failure is always an option. Probably you'd want to check the types of the piped data for compatibility when you populate it. Also, having the first processor be distinct was just because I was too lazy to write a first-excluding range-for adapter (or the equivalent manual loop). — Yakk - Adam Nevraumont, Aug 27 '15 at 12:58

Pipeline class storing different templates with constraints

1 Answers1