0

I'm looking to implement something similar to the deferred IEnumerable concept in C++, but without template implementations. (My question is very similar to this other question, except for the deal-breaking templates.)

In our code, we have many functions that receive a spec to query and will store the results in a passed std::vector&. Often these functions call lower level versions which do their own filtering and combining with other sets. Sometimes they will create local vectors (with a different temp allocator) to call the lower-level functions, then push_back to the caller's vector after filtering that. These are not generic functions, in either word or spirit.

I'm looking to eliminate the copies and allocs, and even the need for a results container. Plus while I'm doing this, I'd like to give it a short-circuit ability.

There's an obvious need for deferred operations here. In C#, this would be easy. Everything yields into an IEnumerable, and use the Where() etc. operators, all deferred.

I have an idea how to do this in C++, passing in callbacks rather than containers. We just need the "push_back" abstracted, plus a returned bool for "stop here". The callbacks would be chained very similarly to IEnumerable operators, though in operation it would be more of a push-pull, and maybe feel a bit like IObservable.

And on top of all this, I have to avoid templatizing these function implementations. I don't want to refactor a lot of code, and I don't want to run into unsupported compiler surprises on the 15 or so different platforms we're currently compiling to. So no promises, no (I think). And no C++11.

This seems like a solved problem, but I've poked around on Google and maybe haven't found the right way to ask it. Has this problem been solved already?

Community
  • 1
  • 1
scobi
  • 14,252
  • 13
  • 80
  • 114
  • 4
    Why are you allergic to templates? – Rapptz Aug 01 '14 at 08:42
  • 1
    Have you tried looking for a c++ linq library? The basic ideas behind linq are also implementable in c++ so theres a few around as the standard library method doesn't handle it to well becasue some of what linq does requires virtual functions, perticuarly the wrappers, due to typechecking. – user1937198 Aug 01 '14 at 08:49
  • This cannot work without templates and cannot work without C++11. I'm not sure what you were expecting since IEnumerable is, by definition, generic. – Puppy Aug 01 '14 at 08:51
  • @Puppy you can get away with out c++11 if you use virtual functions. – user1937198 Aug 01 '14 at 08:55
  • Also .net has shown that a wrapper class and some casting means you can interface the templates with objects that dont use generics, by having every IEnumerable inherit IEnumerable. – user1937198 Aug 01 '14 at 08:58
  • 1
    [Why? Why? Why? Why? Why?](http://en.wikipedia.org/wiki/5_Whys). Why no templates? Why no C++11? Why don't you want to refactor code? Why not [cpplinq](http://cpplinq.codeplex.com/)/[rxcpp](https://github.com/Reactive-Extensions/RxCpp)? Why C++ at all then? – Dmitry Ledentsov Aug 01 '14 at 09:02
  • I would love to use C++ 11, but we run on platforms that don't have supporting compilers yet. I would love to use templates, but (as I said) I don't want to refactor a lot of code. All of this stuff is currently in cpp files. (And why don't I want to refactor code? Um, because of the time involved? Potential for added bugs? Extra build time? Let's just pretend I know what my constraints are here and go with my no-big-refactor limitation.) – scobi Aug 05 '14 at 06:45
  • Re-reading my question, I think I wasn't quite clear. We are using templates - the functions are receiving std::vectors. I want to replace that with SomethingElse where T is concrete, not left parameterized (i.e. same as the std::vector being received). But in searching for deferred operation help elsewhere on SO and Google, I am only finding solutions where the receiving function is *itself* parameterized, which is no-go for us. – scobi Aug 05 '14 at 06:47

1 Answers1

2

I re-read your question and I actually think this can be servicable, but it's definitely not ideal without C++11.

You can consider boost::function<boost::optional<T>()> (yay Boost) as a range. The semantics of this range are super simple- you invoke it, and when it's finished, it gives back an empty optional instead of a value.

For output operations like push_back, you can consider a boost::function<void(T)> that will accept the value and then do whatever you need with it. However generally, these aren't that useful, as most functions can be written as returning a modified range, then the caller can simply use that range directly rather than requiring to take an output range.

Not having lambdas here is a total bitch, but you can look into e.g. Boost.Phoenix/Lambda to help.

Puppy
  • 144,682
  • 38
  • 256
  • 465
  • .net doesnt acually let you use push_back on plain ienumerables for exactly that reason. You use .concate exstention method which does what you describe of returning the modified range. – user1937198 Aug 01 '14 at 09:02
  • Yes, I know that. I'm not sure why you bothered commenting it. – Puppy Aug 01 '14 at 09:06
  • Sorry for the late reply but I interpreted you answer as suggesting that operations like .push_back are possible in linq. – user1937198 Aug 02 '14 at 14:21
  • This is a good starting point for me, thanks! Though I decided to go with the "impossibly fast delegates" (http://www.codeproject.com/Articles/11015/The-Impossibly-Fast-C-Delegates?fid=199666) to avoid allocations and bringing in Boost. – scobi Aug 08 '14 at 13:49
  • @ScottBilas: You know that boost::function uses small buffer optimization, right? And that "impossibly fast delegates" cannot, in fact, represent the vast majority of C++ function objects? – Puppy Aug 08 '14 at 13:52
  • Didn't realize it had that optimization, that's nice. Though bringing Boost into our codebase is still not an option. Perhaps just the function module..might try that. But what is the "vast majority" of function objects that the other method cannot represent? I have run into no problems so far. – scobi Aug 08 '14 at 14:01
  • @ScottBilas: Impossibly fast delegates can only reference (potentially member) functions. It cannot capture anything by value, for example, so the vast utility of lambdas or bound function objects is out. This renders it nearly useless. The real utility to functional APIs, a'la LINQ, requires binding useful state. – Puppy Aug 08 '14 at 14:04
  • @Puppy we can't use lambdas anyway, as C++11 is not an option. I'm currently using a local class inside the function to handle the upvalues, which is working well enough, though verbose. – scobi Aug 08 '14 at 14:08